Templates are created in the admin app and persisted in the API.
They are consumed:
- in the admin app, by requesting them from the API
- in the API, by loading them from the database
There are two potential places where unescaped HTML could be sent to a user:
- when the admin app is previewing a template (it has to render the template as
markup in order to show the placeholders)
- in the body of an email
For all consumers to have confidence that the templates are safe, it makes sense
to santitise them at the point of creation (and modification). This also avoids
any performance issues that could come from doing it at the point of requesting
a template.
In the future they could be created by a direct API call, bypassing the admin
app. Therefore it makes sense for the API to sanitise them.
The commit sanitises templates using an XML parser. It extracts the text content
of the template, minus any HTML tags, because of this famous StackOverflow
answer:
http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454
This is checked on 3rd party API calls, but jobs (CSV files) were able expected to only allow valid files.
Change in tack means we want to have restricted notification failures reported in the UI.
When building the template it was looking for a placeholder called
((phone number)). This caused it to fail because the template it had did not
match the personalisation it was being given.
`Template` has an optional parameter for specifying personalisation values that
should be ignored. The recipient of a message is an example of such a value.
This commit passes that extra parameter, which fixes that bug.
This commit replaces placeholders in a template with the user’s data, using
the Template class from utils
(https://github.com/alphagov/notifications-utils/tree/master/utils#template)
It also extracts the personalisation data from the CSV, taking account of the
different column headings that SMS and email CSVs will have.
At the point of creating the task to send an individual messages, validation of
the placeholders matching the template is assumed to have been done either:
- in processing the CSV in the admin app
- in the endpoint for the API call
No exceptions should be raised at this point.
This commit allows the send notification endpoint to accept an extra parameter,
`personalisation`, the contents of which will be used (later) to replace the
placeholders in the template.
It does validation in the following places:
- at the schema level, to validate the type and (optional) presence of
personalisation
- at the endpoint, to check whether the personalisation provided matches exactly
the placeholders in the template
It does not do validation when processing CSV files, as these are assumed to
already have been validated by the admin app.
It explicitly does not persist either the names of the placeholders (these
should always be derived from the template contents unless it really becomes a
performance concern) or the values of the placeholders (because they might be
personal data).
Refactored service/rest.py so that all methods are returning a properly formatted error message so that the error message can deal with the response.
Refactoed errors.py to properly format the error message.
