David McDonald 2584946823 Close DB connection whilst making HTTP to SMS providers ...

At the moment, when we are processing and sending an SMS we open
a DB connection at the start of the celery task and then close it
at the end of the celery task. Nice and simple.

However, during that celery task we make an HTTP call out to our
SMS providers. If our SMS providers have problems or response times
start to slow then it means we have an open DB connection sat waiting
for our SMS providers to respond which could take seconds. If our
SMS providers grind to a halt, this would cause all of the
celery tasks to hold on to their connections and we would run out
of DB connections and Notify would fall over.

We think we can solve this by closing the DB session which releases
the DB connection back to the pool.

Note, we've seen this happen in staging during load testing if our
SMS provider stub has fallen over. We've never seen it in production
and it may be less unlikely to happen as we are balancing traffic
across two providers and they generally have very good uptime.

One downside to be aware of is there could be a slight increase in
time spent to send an SMS as we will now spend a bit of extra time
closing the DB session and then reopening it again after the HTTP
request is done.

Note, there is no reason this approach couldn't be copied for our
email provider too if it appears successful.

2021-12-21 17:45:53 +00:00

..

app

Close DB connection whilst making HTTP to SMS providers

2021-12-21 17:45:53 +00:00

__init__.py

DRY-up creating auth headers for requests

2021-08-04 16:06:23 +01:00

conftest.py

Add a type table for broadcast providers

2021-05-06 15:30:04 +01:00

test_all_queues_used.py

…