mirror of
https://github.com/GSA/notifications-api.git
synced 2026-09-11 02:23:32 -04:00
Stop killing reporting processes after each task
Previously we think this setting was necessary to avoid a memory leak [1], but it's unclear if this is still an issue: - We've advanced two major versions of Celery. - Some of the tasks are now quicker and leaner. Restarting worker sub-processes after each task is a big problem for performance, as we move towards parallelising our reporting. This is something of a test to see if we can manage without this setting. Note that we need to unset the variable manually: cf unset-env notify-delivery-worker-reporting CELERYD_MAX_TASKS_PER_CHILD In the worst case we can always re-run any failed tasks. To check the worker is still behaving as expected, we can: - Monitor CPU / memory graphs for it. - Check `cf events` for unexpected restarts / crashes. - Compare numbers of task completion logs to previous days. - Check the number of new billing / status rows looks right. [1]: https://github.com/alphagov/notifications-api/commit/ad419f7592524292f8da8447814cc0bbc37f240e
This commit is contained in:
@@ -56,7 +56,6 @@
|
||||
'notify-delivery-worker-periodic': {},
|
||||
'notify-delivery-worker-reporting': {
|
||||
'additional_env_vars': {
|
||||
'CELERYD_MAX_TASKS_PER_CHILD': 1,
|
||||
'CELERYD_PREFETCH_MULTIPLIER': 1,
|
||||
'SQLALCHEMY_STATEMENT_TIMEOUT': 7200
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user