darkhelm/notifications-api
1
0
Fork 0
mirror of https://github.com/GSA/notifications-api.git synced 2025-12-10 23:32:27 -05:00
Code Issues Packages Projects Releases Wiki Activity

Change distribution of queues among notify delivery workers

Browse Source 
This is so that retry-tasks queue, which can have quite a lot of
load, has its own worker, and other queues are paired with queues
that flow similarly:
- letter-tasks with create-letters-pdf-tasks
- job-tasks with database-tasks
This commit is contained in:
Pea Tyczynska
2019-04-30 11:29:28 +01:00
parent 6de67411df
commit 6163ca8b45
2 changed files with 11 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
manifest.yml.j2
View File

@@ -8,12 +8,13 @@
'notify-api-db-migration': {'NOTIFY_APP_NAME': 'api', 'instances': 0},
'notify-delivery-celery-beat': {'memory': '128M'},
'notify-delivery-worker-database': {},
'notify-delivery-worker-database-and-job-tasks': {},
'notify-delivery-worker-research': {'local_statsd': ['preview', 'staging']},
'notify-delivery-worker-sender': {'disk_quota': '2G', 'memory': '3G', 'local_statsd': ['preview', 'staging']},
'notify-delivery-worker-periodic': {},
'notify-delivery-worker-priority': {},
'notify-delivery-worker': {},
'notify-delivery-worker-letters-tasks': {},
'notify-delivery-worker-retry-tasks': {},
'notify-delivery-worker-internal': {},
'notify-delivery-worker-receipts': {},
'notify-delivery-worker-service-callbacks': {'disk_quota': '2G'},

12
scripts/paas_app_wrapper.sh
View File

@@ -4,13 +4,17 @@ case $NOTIFY_APP_NAME in
unset GUNICORN_CMD_ARGS
scripts/run_app_paas.sh gunicorn -c /home/vcap/app/gunicorn_config.py application
;;
delivery-worker)
delivery-worker-retry-tasks)
scripts/run_app_paas.sh celery -A run_celery.notify_celery worker --loglevel=INFO --concurrency=11 \
-Q job-tasks,retry-tasks,create-letters-pdf-tasks,letter-tasks 2> /dev/null
-Q retry-tasks 2> /dev/null
;;
delivery-worker-database)
delivery-worker-letters-tasks)
scripts/run_app_paas.sh celery -A run_celery.notify_celery worker --loglevel=INFO --concurrency=11 \
-Q database-tasks 2> /dev/null
-Q create-letters-pdf-tasks,letter-tasks 2> /dev/null
;;
delivery-worker-database-and-job-tasks)
scripts/run_app_paas.sh celery -A run_celery.notify_celery worker --loglevel=INFO --concurrency=11 \
-Q database-tasks,job-tasks 2> /dev/null
;;
delivery-worker-research)
scripts/run_app_paas.sh celery -A run_celery.notify_celery worker --loglevel=INFO --concurrency=5 \