Before we had a long back off, now we have more, but shorter backoffs.
- PREVIOUS
When we had an error talking to a provider we retried quickly and if we still got errors we backed off more and more. Maximum attempts was 5, max delay 4hours. This was to allow us time to ship a build if that was required.
- NOW
Backing off 48 times of 5 minutes each. This gives us the same total backoff, but many more tries in that period.
- WHY
Having the long back off meant messages could be delayed 4 hours. This was happening more and more, as PaaS deploys can place things into the "inflight" state in SQS. The inflight state MUST have an expiry time LONGER than the maximum retry back off. This meant that messages would be delayed 4 hours, even when there was no app error.
By doing this we can reduce this delay to 5 minutes. Whilst still giving us time to fix issues.
- This is to be used throughout the app.
- Allows us to manage queue names centrally
- All queues renamed to allow us to change the retry processing/visibility timeout
Added a validation method that always fails for scheduled notifications.
Comment out config for scheduled task.
The schedule notifications will be turned on once we can invite services to use it.
Waiting for the service permission story, must commit this in order to keep things from going stale.
Marshmallow validates and deserialises - BUT, when it deserialises,
it explicitly sets `sms_sender=None`, even when you haven't passed
sms_sender in. This is problematic, because we wanted to take advantage
of sqlalchemy's default value to set sms_sender to `GOVUK` when the
actual DB commit happens.
Instead, still use marshmallow for validating, but manually carry out
the json deserialisation in the model class.
This fixes a bug that only manifested when the database was upgraded,
but the code hadn't updated. 🎉
- TEST and DEV builds set up their own queue lists for reading
- PROD/STAGE/PREVIEW use the separate worker process with the -Q flag
This enables us to rename queues in due course
