adapted to recording inserts and updates.
This removes need to manually create history tables.
Our code still remains in control of when history records are
created.
This PR removes the need for the email_safe function. The api does not create the email_from field for the service.
Tests were updated to reflect this change.
- SES sends a reference to allow us to identify the notification
- use this to update status
If source of email is one of our internal emails (invites or validations) - don't try and update a notification.
- rate limiting is a hard number per day
- not limited in terms of rate of request
- limit is a single number held against the service
- every notification counts against the limit, regardless of type
- return a 429 if limit exceeded.
Currently when the Celery task processes a CSV it will call the API with the
values for all the non-recipient columns in the `personalisation` field. This
means that those API calls would fail, even though the CSV has been processed
‘successfully’.
This was not being caught by the tests, so this commit adds extra tests to check
what data the task is passing to the API call.
It then updates utils to version 2.0.1 which brings in this fix:
https://github.com/alphagov/notifications-utils/pull/10
- to capture the counts of things that we do
- initial commit captures when we create an email or sms
DOES NOT know about ultimate success only that we asked our partners to ship the notification
Requires some updates when we retry sending in event of error.
This was added to utils in 5914da74f1
This means that:
- we are doing the exact same validation in the API and admin app
- we are actually validating phone numbers for the correct format (hence all the
changes to the tests)
This is checked on 3rd party API calls, but jobs (CSV files) were able expected to only allow valid files.
Change in tack means we want to have restricted notification failures reported in the UI.
This commit replaces placeholders in a template with the user’s data, using
the Template class from utils
(https://github.com/alphagov/notifications-utils/tree/master/utils#template)
It also extracts the personalisation data from the CSV, taking account of the
different column headings that SMS and email CSVs will have.
At the point of creating the task to send an individual messages, validation of
the placeholders matching the template is assumed to have been done either:
- in processing the CSV in the admin app
- in the endpoint for the API call
No exceptions should be raised at this point.
This commit allows the send notification endpoint to accept an extra parameter,
`personalisation`, the contents of which will be used (later) to replace the
placeholders in the template.
It does validation in the following places:
- at the schema level, to validate the type and (optional) presence of
personalisation
- at the endpoint, to check whether the personalisation provided matches exactly
the placeholders in the template
It does not do validation when processing CSV files, as these are assumed to
already have been validated by the admin app.
It explicitly does not persist either the names of the placeholders (these
should always be derived from the template contents unless it really becomes a
performance concern) or the values of the placeholders (because they might be
personal data).
- brings boto S3 into new AWS folder
- CSV processing utils method
Rejigs the jobs rest endpoint - removes some now unused endpoints,
Calls to the task with the job, job processing in task, delegating SMS calls to the sms task
- uses the new subject/email from fields present on the templates / service tables
- brings the send email api call into line with the sms one.
- same fields (to/template_id)
- same rules regarding restricted services
- wired in as a task into celery
Requires
- new celery queue
- new env property (NOTIFY_EMAIL_DOMAIN)
