Calculating the number of pages in a letter is quite slow. And the send
yourself a test pages need to load _fast_. Since filling in placeholders
is very unlikely to change the number of pages in the resultant letter,
it’s pretty safe to cache that count, and makes the subsequent pages
load a lot faster.
The send yourself a test feature is useful for two things:
- constructing an email/text message/letter without uploading a CSV file
- seeing what the thing your going to send will look like (either by
getting it in your inbox or downloading the PDF)
- learning the concept of placeholders, ie understanding they’re thing
that gets populated with _stuff_
The problem we’re seeing is that the current UI breaks when a template
has a lot of placeholders. This is especially apparent with letter
templates, which have a minimum of 7 placeholders by virtue of the
address.
The idea behind having the form fields side-by-side was to help people
understand the relationship between their spreadsheet columns and the
placeholders. But this means that the page was doing a lot of work,
trying to teach:
- replacement of placeholders
- link between placeholders and spreadsheet columns
The latter is better explained by the example spreadsheet shown on the
upload page. So it can safely be removed from the send yourself a test
page – in other words the fields don’t need to be shown side by side.
Showing them one-at-a-time works well because:
- it’s really obvious, even on first use, what the page is asking you to
do
- as your step through each placeholder, you see the message build up
with the data you’ve entered – you’re learning how replacement of
placeholders works by repetition
This also means adding a matching endpoint for viewing each step of
making the test letter as a PDF/PNG because we can’t reuse the view of
the template without any placeholders filled any more.
It makes the error message quite noisy.
We’re going to move the table right underneath the error message, so
you’ll be able to see the column names right there.
Send yourself a test is:
- a good way of explaining how placeholders work
- a useful tool for checking your work before you send a big batch
It’s not a good way of learning about the relationship between columns
in a spreadsheet and placeholders. The ‘example spreadsheet’ thing is
good at making that connection. The table on this page isn’t, because
it doesn’t _feel_ like you’re making a spreadsheet with the send
yourself a test feature (even though that’s what you’re doing in the
background). This will be even more the case when we stop putting the
input boxes horizontally on one page.
By removing the table from this page it makes the page simpler, which
allows people to focus on the important thing – what’s happening to
their message.
If a service can send internationally, our CSV validation should not
catch valid international phone numbers. This means calling through
to code added to utils in:
- [ ] https://github.com/alphagov/notifications-utils/pull/156
This has been removed from utils (so things will break if utils is
upgraded without this change isn’t made). I think it’s friendlier to
present the phone number as the user entered it anyway – because this is
what they think a ‘correct’ phone number representation looks like
anyway.
We’ve seen in letters usability testing that people get stuck in a
“no-man’s land” when trying to go back from the _Send yourself a test_
page.
This was broken for two reasons:
- we hadn’t considered that a letter template without placeholder still
requires you to fill in
- we’ve changed subsequently made the _view template_ page the place
where you do your actions, rather than the (old) page with all the
templates shown
So this commit fixes it so that the back link always take you back to
the page you were previously on, and adds some more test cases so we
have all the scenarios accounted for.
Not necessary to have it on its own page – it’s one line of stuff. And
definitely not as frequent use as the ‘Upload recipients’ or ‘Send
yourself a test’ links.
When a team has lots of templates the choose template page gets very
long. It gets hard to find the template that you are looking for.
Our initial reckon was that teams would not be giving their templates
very useful names, and therefore a preview would be helpful. What we
have found is that:
- teams actually do give their templates useful names, and refer to
these template names elsewhere
- the previews are less useful for emails and text messages, because
they have so much content (which for emails also makes it harder to
`ctrl` + `f` the template name)
The other problem we found was that this page presented the user with
a _lot_ of options. For each template there were 4 actions, plus the
click-to-preview action for letters, plus the ‘see previous version’
action for templates that had been edited multiple times. It was a very
busy page.
And the final problem (that we recently introduced) was that there was
no way, other than the visual cues, to know whether a template was a
letter, email, or text message.
So this commit strips back the choose template page to be very focused
on finding the right template, by only showing the template name and
type. The user can then click through to a page that shows just a single
template, and perform actions relevant to that template from that page.
We’ve had a couple of instances where teams have sent the wrong template
to a …number of users.
Sometimes templates can be very similar and only have slight variations
to tailor them to a specific subset of users. So identifying the right
template by sight can be difficult.
We know that teams do give their templates meaningful names, and use
these names in other tools (spreadsheets etc) to refer to the templates.
So putting the name of the template on the page where you’re about to
send all the messages seems like it’s gives people an easier way of
double checking that they’re doing the right thing.
I umm’d and ahh’d over the wording a bit, and think ‘Preview of…’ reads
the best. It looks a bit weird because most template names are Title
Case. I think it’s better than some ambiguous punctuation (eg ‘Preview:
Template name’ or ‘Template name – preview’).
Some examples of real template names:
- Preview of Example text message templates
- Preview of Online LPA payment application reminder
- Preview of Create user account
- Preview of Split journey - Unknown credentials
- Preview of Public user: application without supporting documents
- Preview of Renewal Survey – February
- Preview of CEX New adult
- Preview of Applications are closing tomorrow
- Preview of Your application result - if successful
Right now we have separate pages for email and text message templates.
In the future we will also have a separate page for letter templates.
This commit changes Notify to only have one page for all templates.
What is the problem?
---
The left-hand navigation is getting quite crowded, at 8 items for a
service that can send letters. Research suggests that the number of
objects an average human can hold in working memory is 7 ± 2 [1]. So
we’re at the limit of how many items the navigation should have.
In the future we will need to search/sort/filter templates by attributes
other than type, for example:
- show me the ‘confirmation’ templates
- show me the most recently used templates
- show me all templates containing the placeholder `((ref_no))`
These are hypothetical for now, but these needs (or others) may become
real in the future. At this point pre-filtering the list of templates
by type would restrict what searches a user could do. So by making this
change now we’re in a better position to iterate the design in the
future.
What’s the change?
---
This commit replaces the ‘Email templates’, ‘Text message templates’ and
‘Letter templates’ pages with one page called ‘Templates’.
This new templates page shows all the templates for the service, sorted
by most recently created first (as before).
To add a new template there is a new page with a form asking you what
kind of template you want to create. This is necessary because in the
past we knew what kind of template you wanted to create based on the
kind you were looking at.
What’s the impact of this change on new users?
---
This change alters the onboarding process slightly. We still want to
take people through the empty templates page from the call-to-action on
the dashboard because it helps them understand that to send a message
using Notify you need a template. But because we don’t have separate
pages for emails/text messages we will have to send users through the
extra step of choosing what kind of template to create. This is a bit
clunkier on first use but:
- it still gets the point across
- it takes them through the actual flow they will be using to create new
templates in the future (ie they’re learning how to use Notify, not
just being taken through a special onboarding route)
I’m not too worried about this change in terms of the experience for new
users. Furthermore, by making it now we get to validate whether it’s
causing any problems in the lab research booked for next week.
What’s the impact of this change on current services?
---
Looking at the top 15 services by number of templates[2], most are using
either text messages or emails. So this change would not have a
significant impact on these services because the page will not get any
longer. In other words we wouldn’t be making it worse for them.
Those services who do use both are not using as many templates. The
worst-case scenario is SSCS, who have 16 templates, evenly split between
email and text messages. So they would go from having 8 templates per
page to 16, which is still less than half the number that HMPO or
Digital Marketplace are managing.
References
---
1. https://en.wikipedia.org/wiki/The_Magical_Number_Seven,_Plus_or_Minus_Two
2. Template usage by service
Service name | Template count | Template types
---------------------------------------|----------------|---------------
Her Majesty's Passport Office | 40 | sms
Digital Marketplace | 40 | email
GovWifi-Staging | 19 | sms
GovWifi | 18 | sms
Digital Apprenticeship Service | 16 | email
SSCS | 16 | both
Crown Commercial Service MI Collection | 15 | email
Help with Prison Visits | 12 | both
Digital Future | 12 | email
Export Licensing Service | 11 | email
Civil Money Claims | 9 | both
DVLA Drivers Medical Service | 9 | sms
GOV.UK Notify | 8 | both
Manage your benefit overpayments | 8 | both
Tax Renewals | 8 | both
> When the CSV is missing the header row, we get an error and the user
> will see "Sorry, we are experiencing technical difficulties..."
>
> We should return a better error message for the user.
– https://www.pivotaltracker.com/story/show/140668615
This was caused by an attempt to access the `first_recipient` variable
before it was assigned. It would only be assigned when there was at
least one row in the file.
Fixing this means doing two things:
- defaulting `first_recipient` to be `None` before looking in the file
- adding an error message for when we can’t extract any rows out of the
file (which is more nuanced than the file just being completely empty)
(There’s a nasty `sort` in the Jinja template because when there are no
rows in the file the order of the required column headers is not
deterministic.)
mostly making sure that the correct user is set up. some minor changes,
such as giving the platform_admin service permissions (so that we can
test that platform admins can send letters)
When your CSV file is missing the recipient column (eg ‘phone number’
or ‘email address’) we give you a helpful error message telling you that
this is the case.
When we changed the recipient column to be columns, plural, we didn’t
update the code that generated the error message. So you would get
errors that looked this like this:
> Your file needs to have a column called ‘’
This commit fixes the error message.
The accessibility audit raised the issue that screen reader users could
miss the table of data on the preview page, because it’s after the
submit/back buttons.
> The back button is before the table of error messages - a screen
> reader user might read the initial error summary then get to the back
> button and not realise the error detail are later in the sequence.
> The send and back buttons are before the table of field values - a
> screen reader user might read the template details ror summary then
> get to the buttons and not realise the field details are later in the
> sequence.
This commit add a skip link to navigate the users directly to the table,
which:
- allows them to skip past a lot of content which they might already
have read
- makes them aware that the table exists
It’s added:
– as the first thing after the `<h1>` when there are no errors with the
file
- as the last thing in the error summary when there are errors with the
file
The link is hidden from those interacting with the site visually.
We have a bunch of different styles of handling when function
definitions span multiple lines, which they almost always do with tests.
Here’s why an argument per line, single indent is best:
- cleaner diffs when you change the name of a method (one line change
instead of multiple lines)
- works better on narrow screens, eg Github’s diff view, or with two
terminals side by side on a laptop screen
- works with any editor’s indenting shortcuts, no need for an IDE
Also, trailing comma in the list of arguments is good because adding a
new argument to a method becomes a one line, not two line diff.
Problem: it wasn’t saying ‘phone number’ or ‘email address’
Why: we renamed `Recipients.recipient_column_header` to
`Recipients.recipient_column_headers`, and made it return a list, not
a string.
The fix: take the first item of the list, and use that to decide whether
it’s phone numbers or email addresses that you’re not allowed to send
to.
N.B. This won’t work for letters, but we don’t know how trial mode is
going to work for letters anyway.
(previously it would have sent them to the choose template page)
if the user has added new templates or deleted the example one,
they're clearly competent enough to use the app so don't worry
(we wouldn't know what URL the tour starts on since the UUID of
the example template is random)
We can no longer trust that the content of templates stored in the
database is safe.
Utils now has code to sanitise the content of templates.
This commit:
- updates utils to bring this code in
- modifies some integration tests to make sure everything is working
(there are more extensive unit tests in utils)
previously we were issuing a flask redirect (302) from the function,
which we then attempted to unpack as a dict further down the line.
raise a werkzeug.routing.RequestRedirect (301 MOVED PERMANENTLY)
instead. note: only use this pattern when the URL they attempted to
access will *NEVER* be valid, as 301s are cached by browsers.
Right now we can show what a letter template looks like as a PDF or PNG.
This commit completes the work so this is also possible when:
- showing a template with the placeholders replaced
- showing any version of a template
Also removes dependency on `Exception().message`, which was deprecated
in Python 2.6. See
97f82d565f
for full details.
The breaking change page was taking the rendered template and saving
that if the user confirmed the change. This meant that templates could
be saved with `<span class="placeholder">…</span>` in their subject line
for example.
This commit fixes it so that it uses whatever data the user submitted,
not the rendered version of this.
Who knows what would happen if a job with a letter template actually
got into the database. `403`ing the page is a quick and dirty hack to
stop this from happening.
werkzeug's internal workings keep files under 500kb in memory, and files
greater than 500kb as a TemporaryFile
(https://github.com/pallets/werkzeug/blob/0.11-maintenance/werkzeug/formparser.py#L38)
when we encounter a CSV or TSV, we call normalise_newlines, which invokes
`.read()`, however when we were passing straight into pyexcel, we called
`file.getvalue()` - this exists on BytesIO (small files) but not on
TemporaryFile objects (large files) - we were seeing 500 errors
Was using the wrong attribute name on the instance of `RecipientCSV`
(needed to match 19df1eb4ef/notifications_utils/recipients.py (L156) )
The ‘can’t show file contents’ message wasn’t showing up, now it is.
We want to limit the number of rows someone can have in a job, because
it gets too slow to process the file otherwise.
This should be the first error that a user sees, because we can’t work
out if there are other errors until they’ve got the file down to a
processable size.
This also means adding a message to say that the file can’t be displayed
if it doesn’t contain any processed rows.
***
https://www.pivotaltracker.com/story/show/129830161
For two reasons:
- it’s extra stuff in tour that users dont yet need to know about
- test messages are hidden from the dashboard, so you’d have no
visibility of when they were sending once you’d scheduled them
Users need to pick a time in the next 24hrs, or send a file immediately.
Rationale for this is a bit lost in time-before-holiday, but generally:
‘Now’ and ‘later’ as the inital choices makes it really clear what
this feature is about conceptually.
The choice of times is absolute, eg ‘1pm’ not ‘in 3 hours’
