sakisv 92086e2090 Reduce concurrency on high volume workers ...

We noticed that having high concurrency led to significant memory usage.

The hypothesis is that because of long polling, there are many
connections being held open which seems to impact the memory usage.

Initially the high concurrency was put in place as a way to get around
the lack of long polling: We were spawning multiple processes and each
one was doing many requests to SQS to check for and receive new tasks.

Now with long polling enabled and reduced concurrency, the workers are
much more efficient at their job (the tasks are being picked up so fast
that the queues are practically empty) and much lighter on resource
requirements. (This last bit will allow us to reduce the memory
requirement for heavy workers like the sender and reduce our costs)

The concurrency number was chosen semi-arbitrarily: Usually this is set
to the number of CPUs available to the system. Because we're running on
PaaS and that number is both abstracted and may be claimed for by other
processes, we went for a conservative one to also reduce the competion
for CPU among the processes of the same worker instance.

2021-11-04 11:38:05 +02:00

..

check_if_new_migration.py

run migrations if app is down

2020-06-26 15:28:28 +01:00

common_functions.sh

Create Docker build image, build project with Docker

2016-08-22 17:25:41 +01:00

delete_sqs_queues.py

use flake8 instead of pycodestyle

2017-11-28 14:28:01 +00:00

fix_migrations.py

add warning to fix_migrations script for when the filename is different

2017-12-07 10:41:10 +00:00

paas_app_wrapper.sh

Reduce concurrency on high volume workers

2021-11-04 11:38:05 +02:00

run_app_paas.sh

Terminate instance if it's running out of disk space

2020-12-23 19:40:04 +02:00

run_multi_worker_app_paas.sh

Add disk space check for sender worker

2021-01-04 14:01:19 +02:00