- uses the reference field on the notifications table to store a 16char random string used to cross reference DVLA letters back to the notification
- used as letter barcode does not have space for a UUID notification id
Depends on https://github.com/alphagov/notifications-utils/pull/149
Renamed the numeric_id to notification_reference in utils and changed validation rules to match this
Note also the persist_notification method set "reference" to be "client_reference" which is confusing and they are different things, so fixed this too.
s3upload function has been moved to notifications-utils.
https://github.com/alphagov/notifications-utils/pull/138 must be merge first.
Remove scratch code that I was using to test celery.send_task method
Revert the update to default_retry_delay.
Increase the countdown, wait 1 minute before executing the task.
Some of the changes in my previous commit are included here, that branch will be merged before this one, so it will make sense.
Update version of notification_utils
Update the version of notifications-utils
- previously this was unbounded, so it got all jobs older then 7 days. In excess of 75,000 🔥
- this meant that the job took (a) a long time and (b) a lot memory and (c) doing the same thing every day
These changes mean that the job has a 2 day eligible window for jobs, minimising the number of eligible jobs in a run, whilst still retaining some leeway in event if it failing one night.
In principle the job runs early morning on a given day. The previous 7 days are left along, and then the previous 2 days worth of files are deleted:
so:
runs on
31st
30,29,28,27,26,25,24 are ignored
23,22 jobs here have files deleted
21 and earlier are ignored.
Whatever a user has entered for their service’s contact block should
appear in the right place in the file we give to DVLA.
The work to output in the right fields in the DVLA file has already been
done. We just weren’t passing it through. This commit passes it through.
Rather than generating each digit of the number, generate the whole
random number in one go. Should be faster, and is a bit easier to read.
Don’t need to worry about it not being zero-padded because the
`Template` class handles this here:
6ddd2ff352/notifications_utils/template.py (L410)
Using `+` to concatenate strings isn’t very memory efficient. Not sure
if there are real-world implications for how it’s being used here, but
can’t hurt to use `.join` instead.
Rewriting it as a generator also lets us remove some unneeded variable
assignment.
when we made the change to async persist notifications, we forgot to
pass through api_key_id and key_type. in send_sms/email, for legacy
reasons, they default to None/KEY_TYPE_NORMAL, so regardless of what
your api key was set up as, we would send real messages!
TODO: Once the PaaS transition is complete and the task changes are
reverted, remove the api_key_id and key_type params from the send_*
tasks entirely, as those are only called from the csv job flow, and
don't need them
This is being done for the PaaS migration to allow us to keep traffic coming in whilst we migrate the database.
uses the same tasks as the CSV uploaded notifications. Simple changes to not persist the notification, and call into a different task.
This will transform each notification in a job to a row in a file.
The file is then uploaded to S3.
The files will later be aggregated by the notifications-ftp app to send to dvla.
The method to upload the file to S3 should be pulled into notifications-utils package.
It is the same method used in notifications-admin.
Update the PermissionsDao.get_permissions_by_user_id to only return permissions for active services,
this will make the admin app return a 403 if someone (otherthan platform admin) tries to look at an inactive service.
Removed the active flag in sample_service the dao_create_service overiddes this attribute.
Previously research mode created a task to fake the callback from the providers. This meant there is an extra task for each notification than would be generated in a live situation.
This PR changes that, so that a research mode/test key notification calls the callback API rather than make a task to do that .
This ensures that the flow for research mode more closely mimics that of live, and removes a task from the process so we can more accurately test throughput,
