mirror of
https://github.com/GSA/notifications-api.git
synced 2025-12-10 23:32:27 -05:00
5d88a1dbf4
What this setting does is best described in https://medium.com/@taylorhughes/three-quick-tips-from-two-years-with-celery-c05ff9d7f9eb#d7ec This should be useful for the reporting app because tasks run by this app are long running (many seconds). Ideally this code change will mean that we are quicker to process the overnight reporting tasks, so they all finish earlier in the morning (although are not individually quicker). This is only being set on the reporting celery app because this change trying to do the minimum possible to improve the reliability and speed of our overnight reporting tasks. It may very well be useful to set this flag on all our apps, but this should be done with some more consideration as some of them will deal with much faster tasks (sub 0.5s) and so it may be still be appropriate or may not. Proper investigation would be needed. Note, the celery docs on this are also worth a read: https://docs.celeryproject.org/en/3.1/userguide/optimizing.html#optimizing-prefetch-limit. However, the language can confuse this with setting with the prefetch limit. The distinction is that prefetch grabs items off the queue, whereas the -Ofair behaviour is to do with when items have already been prefetched and then whether the master celery process straight away gives them to the child (worker) processes or not. Note, this behaviour is default for celery version 4 and above but we are still on version 3.1.26 so we have to enable it ourselves.