Currently templates are ordered by the newest created first. This made
sense when, after creating a new template, you were landed on the page
that listed all the templates. In other words, you needed to see
confirmation of the thing that you’ve just done.
Now (since https://github.com/alphagov/notifications-admin/pull/1195)
you get landed on the page for just that template. So you always see
the template you’ve just created, no matter how the list of templates is
ordered. So we are free to change the order of the templates.
Ordering by time created is not great, because it gives users no control
over which templates appear first. For example, our research reported
this from one team:
> One frustration they have is that when you add a new template it
> automatically goes to the top of the list. To get round this, whenever
> they add a new template they delete all of the existing ones and start
> again because they want to keep their templates in numerical order.
> They'd like to be able to control the order of templates in the list.
We _could_ give people some sort of drag-and-drop template ordering
thing. But this feels like overkill. I think that alphabetical order is
better because:
- it’s easily discoverable – anyone who wants to know how a list is
ordered can quickly tell just by looking at it
- it’s universal – everyone knows how alphabetical ordering works
- it’s familiar – this is how people documents on their computer are
ordered; there’s no new UI to learn
- it’s what users are doing already – from the same service as above:
> They number their templates 1,2a, 2b, 3a etc
So this commit changes the ordering from newest created first to
alphabetical.
Previous changes to template order and navigation:
- https://github.com/alphagov/notifications-admin/pull/1163
- https://github.com/alphagov/notifications-admin/pull/1195
- https://github.com/alphagov/notifications-admin/pull/1330
Implementation notes
---
I refactored some of the tests here. I still don’t think they’re great
tests, but they’re a little more Pythonic now at least.
I also added a sort by template type, so that the order is deterministic
when you have, for example, an email template and a text message
template with the same name. If you have two text message templates with
the same name you’re on your own.
> If a user makes an API request with additional personalisation fields,
> we should simply discard any fields that the template doesn't have.
>
> This gives a couple of related advantages:
>
> - modifying template parameters no longer requires downtime for
> clients - as they can pass in extra new parameters before a template
> change, or continue passing in old unused parameters after removing
> them from a template
>
> - services can pass in large user objects, for example, and then play
> around with templates adding and removing fields at will
>
> we should make sure we still return an error if a user doesn't pass in
> a required parameter.
– https://www.pivotaltracker.com/story/show/140774195
Right now we strip HTML from templates at the point of saving them. This
also converts stuff like ampersands to their entity form (eg &) and
this is what we save in the database.
This is a bad idea when you’re sending a text message or a letter, in
which an HTML entity makes no sense. But we still need to encode HTML in
the body of HTML emails.
The right place to do this is when rendering the templates. The code to
do this is now in utils. So this commit:
- pull in this new utils code
- removes the old
- adds some integration tests to make sure that everything is working
as expected (more thorough unit tests are happening in utils)
Before:
```json
{'data': {'template': '…'}}
```
There’s no need to wrap the response in key because there will only
ever be one valid key for the template preview endpoint.
Flatter is better:
```json
{
'content': '…',
'subject': '…',
'template_type': '…',
…
}
```
The response will be different if there’s an error, but you should be
checking the status code first anyway.
This commit:
- changes the template preview endpoint to return the above format
- adds a test to make sure that the original `/service/…/template/…`
endpoint still returns JSON in the same format (with a `data` key)
There’s a need for users of the API to be able to take advantage of
Notify’s template rendering.
For example, there’s a service that’s building a case management system.
Their users are sending emails on a case-by-case basis. Before they
send an email, it’s ressuring to double check that the right data is
being inserted, that the right template is being used, etc.
This commit:
- adds a separate endpoint for previewing a template, with
personalisation taken from the get parameters of the request
- beefs up the tests around getting a template
Not part of this pull request:
- making this enpoint publicly accessible
When you add a new template, it’s probably the one that you want to do
subsequent stuff with. But it’s also helpful to see the template in
context (with its siblings) to understand that there are multiple
templates. So we don’t want to do what we do in
https://github.com/alphagov/notifications-admin/pull/648
for adding a new template.
But we _can_ make your brand-new template appear first by always
ordering by when the template was created.
This also removes the confusion caused by having `updated_at` affecting
order, and causing the templates to move around all the time.
It is also discovered that columns that have a default value and use the version mixin must set the value when creating the db object before the insert otherwise the history table will be missing the default value.
Updated the templates_history.created_by_id with a value where missing, using the current version of the template for this value.
Update templates_history.archived to false. This is okay as we do not yet have a way to set this value to true.
Removed the versions attribute from the TemplateSchema, there is not a need for this column.
Update notifications/sms|email endpoint to send the template version to the queue.
Update the process_job celery talk to send the template version to the queue.
When the send_sms|send_email task runs it will get the template by id and version.
Created a data migration script to add the template_vesion column for jobs and notifications.
The existing jobs and notifications are given the template_version of the current template.
There is a chance this is the wrong template version, but deemed okay since the application is not live.
Create unit test for the dao_get_template_versions method.
Rename /template/<id>/version to /template/<id>/versions which returns all versions for that template id and service id.
This version of the client removed the request method, path and body from the encode and decode methods.
The biggest changes here is to the unit tests.
This PR removes the need for the email_safe function. The api does not create the email_from field for the service.
Tests were updated to reflect this change.
- to capture the counts of things that we do
- initial commit captures when we create an email or sms
DOES NOT know about ultimate success only that we asked our partners to ship the notification
Requires some updates when we retry sending in event of error.
Templates are created in the admin app and persisted in the API.
They are consumed:
- in the admin app, by requesting them from the API
- in the API, by loading them from the database
There are two potential places where unescaped HTML could be sent to a user:
- when the admin app is previewing a template (it has to render the template as
markup in order to show the placeholders)
- in the body of an email
For all consumers to have confidence that the templates are safe, it makes sense
to santitise them at the point of creation (and modification). This also avoids
any performance issues that could come from doing it at the point of requesting
a template.
In the future they could be created by a direct API call, bypassing the admin
app. Therefore it makes sense for the API to sanitise them.
The commit sanitises templates using a Mozilla’s Bleach library[1]. It is
configured to get the text content of the template, minus any HTML tags. It is
not using a regex because[2].
1. https://github.com/mozilla/bleach
2. http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-contained-tags/1732454#1732454
