I don’t think it adds anything to tell you that a key’s never been used.
The value of the ‘key was used 3 minutes ago’ message is in stopping you
accidentally revoking something you shouldn’t have.
This makes errors on all pages have a `<h1>` element, which is important
for accessibility. It means a bit of rewriting the messages, but I think
they’re better for it.
If sending SMS is disabled for a service, it should not be possible to
add or modify SMS templates. If a user tries to do this, they should see
a different page with a link to go back. The same thing should happen
with email templates.
`test_utils.py` is actually a file for testing some utils, not
some utils for testing.
Test helper functions and fixtures are better placed in conftest,
and also means that we can use them in conftest.
We’ve made a few changes to the tour recently, without changing the
help text on the left hand side of the screen. So the stuff you see on
the right side of the screen doesn’t quite sync up any more.
This commit adds an extra, introductory page that just shows the
template and a next button, which better matches the ‘every message
starts with a template’ help text.
Works similarly to the delete template flow, because it’s a destructive,
one-way action.
Not on the edit template page, because it’s not something you want to be
considering every time you’re editing a template. And we saw that people
couldn’t find the delete button when it was on this page.
Adds a bit more CSS for the `dangerous` banner type, because the content
here is quite complicated. Breaking it into a list helps, but the
spacing didn’t look right, so needed some tweaking.
Can ship independently of the code that shows the redaction, but needs
the API first.
There are lots of services that only send emails, or only send text
messages. For these services, being able to filter the list of templates
but type is pointless – it won’t cut the list down at all.
This commit adds some logic to only show the navigation if the service
has some variety of template types.
When users are trying to find a template there’s a fair chance that they
know whether or not it’s an email/text message/(letter) that they’re
looking for.
Making them scroll past a whole bunch of templates of a different type
means it will take them longer to find the template they are looking
for.
We already have search on the templates page, but this is only good for
where they can remember the name of the template. This will be
sometimes but not always.
This commit adds some navigation to filter down the list of templates to
only show one type at a time. By default it will show all templates. It
adapts the pattern we use for filtering notifications by
sending/failed/delivered, but without the counts of how many things are
in each bucket (I don’t think there’s any value in knowing you have X
text message templates; on this page you only really care
about the one template you’re looking for).
_Note: required re-arranging the functions in `templates.py`. The route
for `/template/:uuid` needs to come before the route for
`template/:string` otherwise Flask tries to interpret a template’s ID
as its type.
It’s a confusing proposition to have two features which are almost
identical. Even differentiating between them in the template menu would
be tricky.
I think the better thing to do is rename the whole feature to ‘send
one-off message’.
Then if someone wants to use there own phone number or email address,
give them a quick shortcut to doing that, once they’re in the flow.
In the background this reuses the ‘send yourself a test’ code, but
the user is never aware that they’re going through a different route
to send an individual message. So the proposition stays nice and clean.
The CSV upload route has always quietly ignored excess personalisation.
We changed the API to do the same here:
https://github.com/alphagov/notifications-api/pull/853
This means that removing a placeholder from a template is never a
breaking change, because the data that you were providing to populate it
is now just ignored.
So we don’t need to show the interstitial page in this case.
This is a hangover from when we had separate email/SMS template pages.
If the view doesn’t use a parameter Flask adds it to the querystring. So
we were getting URLs that ended in `?template_type=sms`, to no effect.
This is another problem with sanitising HTML, this with with it getting
encoded where it shouldn’t be. The result was, when editing a template,
the API getting sent an encoded rather than raw version of the subject
(for letters and emails).
The reason this happened is because BeautifulSoup behaves in an
unexpected way.
When accessing the `value` attribute of an `input` BeautifulSoup returns
an unencoded version of the contents. In other words it returns what the
user would see in the page, not what is in the raw HTML of the page.
This meant that we were trying too hard to see an `&` instead of a
`&` in our tests[1]. So things were actually working fine before adding
the call to `escape_html`[2], but from the output of the tests it didn’t
look like HTML was getting escaped.
So this commit fixes the bug by removing the call to `escape_html` and
adding a test that looks at the raw HTML, to complement the existing
test which looks at just the `value` attribute.
1. Relevant test added here: https://github.com/alphagov/notifications-admin/pull/1178/files#diff-f2eb304b93cc383727c0ab7fc8fbd464R289
2. Call added here: https://github.com/alphagov/notifications-admin/pull/1178/files#diff-f0af582449ebf426f27f37e38f310057R252
The breaking change page wasn’t properly accounting for the fact that
letter recipients span multiple columns – it was assuming they’d only
take up one column like they do for email and SMS.
This commit fixes:
- the number of column headers (A, B, C, …) to be correct
- the count of columns (you will need X columns in your file) to be
correct
It then parameterises the test to look at a case where a recipient is
in one column (email) and multiple columns (letter).
Not necessary to have it on its own page – it’s one line of stuff. And
definitely not as frequent use as the ‘Upload recipients’ or ‘Send
yourself a test’ links.
Letter templates have (or will have) multiple different editable
regions. I think that the most intuitive way for this to work is to have
- an edit link for each of these areas
- positioned next to the thing to be edited
Again, this isn’t fully hooked up, but since no-one is using letters
live yet this is a good way of getting research feedback and pointing
towards where we want the feature to go.
Uses percentages for the positioning so that the alignment is maintained
on mobile.
The links to the right of the template take up valuable horizontal
space. This means that the preview of email and letter templates isn’t
as big as it could be. By making the letter preview bigger it removes
the need to click through to the PDF to see a preview.
Reuses the navigation style used on the API integration page, because I
think it damages consistency to create another new button style.
The breaking change page temporarily holds the changes in hidden inputs
on the page. The messages content it gets from the `.content` property
on the subject. This is raw and not transformed in any way, so fine.
For the subject it gets the value from the `.subject` attribute on the
template. For email templates, this will be transformed to highlight
placeholders with `<span class='placeholder'>…`. This means that when
the change is confirmed, it’s this encoded version that gets sent to the
API. Which is bad, because we then save `<span class='placeholder'>` in
the database.
This commit changes the page to look at the `._subject` attribute
instead, which is the internal, untransformed version of the subject.
For some reason we were rebuilding `new_template` as a dictionary,
without the `placeholders` attribute. This meant that we were never
actually counting the placeholders, just counting the length of `None`
and adding 1 to it.
So this commit fixes that, beefs up the tests, and makes sure that
everything is pluralised properly.
Right now we have separate pages for email and text message templates.
In the future we will also have a separate page for letter templates.
This commit changes Notify to only have one page for all templates.
What is the problem?
---
The left-hand navigation is getting quite crowded, at 8 items for a
service that can send letters. Research suggests that the number of
objects an average human can hold in working memory is 7 ± 2 [1]. So
we’re at the limit of how many items the navigation should have.
In the future we will need to search/sort/filter templates by attributes
other than type, for example:
- show me the ‘confirmation’ templates
- show me the most recently used templates
- show me all templates containing the placeholder `((ref_no))`
These are hypothetical for now, but these needs (or others) may become
real in the future. At this point pre-filtering the list of templates
by type would restrict what searches a user could do. So by making this
change now we’re in a better position to iterate the design in the
future.
What’s the change?
---
This commit replaces the ‘Email templates’, ‘Text message templates’ and
‘Letter templates’ pages with one page called ‘Templates’.
This new templates page shows all the templates for the service, sorted
by most recently created first (as before).
To add a new template there is a new page with a form asking you what
kind of template you want to create. This is necessary because in the
past we knew what kind of template you wanted to create based on the
kind you were looking at.
What’s the impact of this change on new users?
---
This change alters the onboarding process slightly. We still want to
take people through the empty templates page from the call-to-action on
the dashboard because it helps them understand that to send a message
using Notify you need a template. But because we don’t have separate
pages for emails/text messages we will have to send users through the
extra step of choosing what kind of template to create. This is a bit
clunkier on first use but:
- it still gets the point across
- it takes them through the actual flow they will be using to create new
templates in the future (ie they’re learning how to use Notify, not
just being taken through a special onboarding route)
I’m not too worried about this change in terms of the experience for new
users. Furthermore, by making it now we get to validate whether it’s
causing any problems in the lab research booked for next week.
What’s the impact of this change on current services?
---
Looking at the top 15 services by number of templates[2], most are using
either text messages or emails. So this change would not have a
significant impact on these services because the page will not get any
longer. In other words we wouldn’t be making it worse for them.
Those services who do use both are not using as many templates. The
worst-case scenario is SSCS, who have 16 templates, evenly split between
email and text messages. So they would go from having 8 templates per
page to 16, which is still less than half the number that HMPO or
Digital Marketplace are managing.
References
---
1. https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two
2. Template usage by service
Service name | Template count | Template types
---------------------------------------|----------------|---------------
Her Majesty's Passport Office | 40 | sms
Digital Marketplace | 40 | email
GovWifi-Staging | 19 | sms
GovWifi | 18 | sms
Digital Apprenticeship Service | 16 | email
SSCS | 16 | both
Crown Commercial Service MI Collection | 15 | email
Help with Prison Visits | 12 | both
Digital Future | 12 | email
Export Licensing Service | 11 | email
Civil Money Claims | 9 | both
DVLA Drivers Medical Service | 9 | sms
GOV.UK Notify | 8 | both
Manage your benefit overpayments | 8 | both
Tax Renewals | 8 | both
bump utils to 13.8.0
we still save the content as the user intended, and they'll still see
that content in the text field if they go to edit the template, but
the SMS previews will appear as they will on a user's phone
Use `it`/`they` depending on how many different characters you've used
Also don't wrap the message with quotes, as it looks confusing and
potentialy implies that you can't use apostrophes
mostly making sure that the correct user is set up. some minor changes,
such as giving the platform_admin service permissions (so that we can
test that platform admins can send letters)
We have a bunch of different styles of handling when function
definitions span multiple lines, which they almost always do with tests.
Here’s why an argument per line, single indent is best:
- cleaner diffs when you change the name of a method (one line change
instead of multiple lines)
- works better on narrow screens, eg Github’s diff view, or with two
terminals side by side on a laptop screen
- works with any editor’s indenting shortcuts, no need for an IDE
Also, trailing comma in the list of arguments is good because adding a
new argument to a method becomes a one line, not two line diff.
The url was not being properly formed, missing the tempalte_id, meaning the back button on the page did not work.
This fixes that, includes a check of the url for the back button.
We don’t want sanitized content going into the database, because
sometimes we need the content unsanitised. The path from admin to the
API is where the template goes on it’s way to the database. So let’s
make sure we’re not sanitizing it at this stage.
We can no longer trust that the content of templates stored in the
database is safe.
Utils now has code to sanitise the content of templates.
This commit:
- updates utils to bring this code in
- modifies some integration tests to make sure everything is working
(there are more extensive unit tests in utils)
There is a check that the template can not be created as priority if the user is not a platform admin.
There is a check that the template can not change the `priority` unless they are a platform admin.
Right now we can show what a letter template looks like as a PDF or PNG.
This commit completes the work so this is also possible when:
- showing a template with the placeholders replaced
- showing any version of a template
Also removes dependency on `Exception().message`, which was deprecated
in Python 2.6. See
97f82d565f
for full details.
The breaking change page was taking the rendered template and saving
that if the user confirmed the change. This meant that templates could
be saved with `<span class="placeholder">…</span>` in their subject line
for example.
This commit fixes it so that it uses whatever data the user submitted,
not the rendered version of this.
The PDF preview is all good, but it’s hard, finickeity and feels dirty
to embed a PDF in a web page. It’s a more natural thing to embed an
image in a web page.
So this commit adds another endpoint to return an image of a letter
template. It generates this image from the PDF preview, so the stack
looks like:
1. `template.png` (generated in admin)
2. `template.pdf` (generated in admin)
3. HTML preview (generated by a `Renderer` in utils)
4. `Template` instance
5. serialised template from API
6. Template stored in database
The library used to convert the PDF to an image is Wand[1], which binds
to ImageMagick underneath. So in order to get this working locally on a
Mac you will probably need to do:
`brew install imagemagick ghostscript cairo pango`.
To get it working on Ubuntu/EC2 is an exercise left to the reader…
1. http://docs.wand-py.org/en/0.4.4/
Previewing a letter is different to previewing an email or text message
because:
- a letter has a layout
- the layout is fixed, ie it doesn’t depend on the user’s device
- the ‘send yourself a test’ feature won’t be as useful because it has
a lead time, so the feedback loop will be much longer
For these reasons a HTML-only preview of the letter won’t be enough (we
don’t think). A PDF is more appropriate because:
- it can replicate the layout of the letter exactly
- it is a print format, so the user could even print themselves a copy
locally to get a feel for how it will look
This commit makes use of Flask WeasyPrint [1] to take a HTML
representation of the letter, convert it to a PDF and serve it back to
the user.
The actual work to generate the HTML and specify the layout is done in
utils, same as we do for rendering other messages.
1. https://pythonhosted.org/Flask-WeasyPrint/
Since you can’t really send or edit a deleted template we should show
a message telling you that the template has been deleted.
This is important because deleted templates still show up in the
template statistics.
