- new client for statsd, follows conventions used elsewhere for configuration
- client wraps underlying library so we can use a config property to send/not send statsd
Added statsd metrics for:
- count of API successful calls SMS/Email
- count of successful task execution for SMS/Email
- count of errors from Client libraries
- timing of API calls to third party clients
- timing of how long messages live on the SQS queue
This version of the client removed the request method, path and body from the encode and decode methods.
The biggest changes here is to the unit tests.
The change is backward compatible so an older client that sends a jwt with the extra claims will pass authentication.
Once all the clients have been updated to not include the extra claims some updates to exclude them from the method signatures will happen as well.
Implements
https://github.com/alphagov/notifications-utils/pull/16
Once
https://github.com/alphagov/notifications-admin/pull/376
is merged it will no longer be possible for a user to upload a CSV file
containing recipients that they’re not allowed to send to.
So this commit also removes any restricted service checks in the task,
because any public phone numbers/email addresses no longer have any way
of reach this point if the service is restricted.
Currently when the Celery task processes a CSV it will call the API with the
values for all the non-recipient columns in the `personalisation` field. This
means that those API calls would fail, even though the CSV has been processed
‘successfully’.
This was not being caught by the tests, so this commit adds extra tests to check
what data the task is passing to the API call.
It then updates utils to version 2.0.1 which brings in this fix:
https://github.com/alphagov/notifications-utils/pull/10
Templates are created in the admin app and persisted in the API.
They are consumed:
- in the admin app, by requesting them from the API
- in the API, by loading them from the database
There are two potential places where unescaped HTML could be sent to a user:
- when the admin app is previewing a template (it has to render the template as
markup in order to show the placeholders)
- in the body of an email
For all consumers to have confidence that the templates are safe, it makes sense
to santitise them at the point of creation (and modification). This also avoids
any performance issues that could come from doing it at the point of requesting
a template.
In the future they could be created by a direct API call, bypassing the admin
app. Therefore it makes sense for the API to sanitise them.
The commit sanitises templates using a Mozilla’s Bleach library[1]. It is
configured to get the text content of the template, minus any HTML tags. It is
not using a regex because[2].
1. https://github.com/mozilla/bleach
2. http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454
This commit replaces placeholders in a template with the user’s data, using
the Template class from utils
(https://github.com/alphagov/notifications-utils/tree/master/utils#template)
It also extracts the personalisation data from the CSV, taking account of the
different column headings that SMS and email CSVs will have.
At the point of creating the task to send an individual messages, validation of
the placeholders matching the template is assumed to have been done either:
- in processing the CSV in the admin app
- in the endpoint for the API call
No exceptions should be raised at this point.
