Reduce the pressure on the db for API post email requests.

Instead of saving the email notification to the db add it to a queue to save later.
This is an attempt to alleviate pressure on the db from the api requests.
This initial PR is to trial it see if we see improvement in the api performance an a reduction in queue pool errors. If we are happy with this we could remove the hard coding of the service id.

In a nutshell:
 - If POST /v2/notification/email is from our high volume service (hard coded for now) then create a notification to send to a queue to persist the notification to the db.
 - create a save_api_email task to persist the notification
 - return the notification
 - New worker app to process the save_api_email tasks.
This commit is contained in:
Rebecca Law
2020-03-25 07:59:05 +00:00
parent 544537791d
commit a13bcc6697
8 changed files with 182 additions and 12 deletions

View File

@@ -49,6 +49,10 @@ case $NOTIFY_APP_NAME in
exec scripts/run_app_paas.sh celery -A run_celery.notify_celery worker --loglevel=INFO --concurrency=11 \
-Q service-callbacks 2> /dev/null
;;
delivery-worker-save-api-notifications)
exec scripts/run_app_paas.sh celery -A run_celery.notify_celery worker --loglevel=INFO --concurrency=11 \
-Q save-api-email 2> /dev/null
;;
delivery-celery-beat)
exec scripts/run_app_paas.sh celery -A run_celery.notify_celery beat --loglevel=INFO
;;