We look for `original_file_name` in the metadata now. Initially we were
still checking the query string too, but now that the change to add the
filename to the metadata has been deployed for a while there shouldn't
be any cases of the filename still being in a query string.
Since the `original_file_name` is not being added to the metadata in
`.check_messages` (it has happened earlier in the process) a few tests
are no longer needed.
When sending from an uploaded CSV `.send_messages` now puts the filename
in the metadata. It previously used the query string to pass the
filename to `.check_messages`, where it can be lost.
The `.send_from_contact_list` function redirected to `.check_messages`
with `original_file_name` in the query string. Contact lists already
have `original_file_name` as part of their metadata, so we can stop
sending it in the query string and use the metadata instead.
We don't need these anymore as all users will use the `one-off/step`
routes.
This has mostly involved tidying up the tests which are still a little
disorganised and not as good as I'd like but it's a step in the right
direction.
More refactoring is still possible to the routes, it may come in a later
PR if I have time.
There is no real reason to have to support both 'one-off' steps and also
'test' steps when sending a one off notification. It's a lot of complex
code, just to now set the one of the placeholders in the session.
We make our code much simpler but no longer using the 'test' routes but
instead adding a new endpoint to set the notification recipient when
sending to yourself before continuing on with the rest of the 'one-off'
flow.
After this is deployed for a day then we can completely remove the
'test' routes and this will help remove a lot of code complexity.
We no longer need the `start_tour` page as this has been replaced with
the new `begin_tour` page.
We also no longer need to handle the `help` argument in the
`send_test_step` or `send_one_off_step` as these no longer are
responsible for the tour and don't need to show the help text.
Worth pointing out, the new tour joins into the send one off flow. When
doing a GET `check_tour_notification`, and submitting the form shown on
this page you are POSTed to `send_notification` with `help=3`. Also for
general sending of one off notifications, the POST to
`send_notification` is done with `help=0` which is a bit of a hack to
make sure that we don't show a back link on the `view_notification` page
for when someone gets there having just sent a one off notification.
This use of `help=0` may be a candidate for a refactor in the future as
it feels like a bit of a hacky way of doing things and is therefore not
as clear to developers what is going on.
Also removes the help argument from the csv routes used here. There is
no reason that we need to ever show help for CSVs and this is leftover
code from when we used to do the tour that way.
If a service doesn’t have permission to send international letters but
someone tries to upload a letter with a valid international address we
just tell them that the last line must be a UK postcode.
This is a bit opaque and:
- suggests that we’re not recognising at all that it’s not a UK letter
- doesn’t explain why it must be a UK postcode
This commit adds a new, error message which tells users why their letter
can’t be sent. And hopefully will give them a better idea of how to
resolve the problem, if they really do need to be able to send
international letters.
At the moment we don’t check whether a one-off letter is international
until the user’s clicked send. It’s more accurate to show that the
letter will be sent internationally as soon as we know the address.
Depends on:
- [ ] https://github.com/alphagov/notifications-utils/pull/786
error pages
The skiplink in those pages is in the error summary
at the top. In this case, it seems redundant
because:
- it jumps to a section of the page just below it
- the section it would jump to is the only error
on the page
Because of this, and because The Digital
Accessibility Centre (DAC) reported that the
skiplinks in these pages may not be needed, we
decided to remove it.
This commit also adds a prefix to the table caption to
indicate that it only contains the errors from the
CSV, not the rest of its contents.
Changes those fields (and sometimes also regular text input fields)
in the following forms:
- LoginForm
- RegisterUserForm
- ChangeEmailForm
- FeedbackOrProblem
- AcceptAgreementForm
- ChangeNameForm (only name field here, but used in the same template
field as ChangeEmailForm here: app/templates/views/user-profile/change.html)
Also includes changes to templates that use this form
and associated tests.
This now adds validation for invalid characters on the
LetterAddressForm for one off letters. It also adds a validation failed
message for uploaded letters, precompiled letters sent through the API,
and CSV rows with errors.
At the moment the page is the same as for text message templates,
except:
- different H1
- no guidance about personalisation, links, etc (until we decide how
these should work)
For now you won’t be able to really create a broadcast template, because
the API doesn’t support it (the API will respond with a 400). But that’s
OK because no real services have the broadcast permission yet.
This required a bit of refactoring of how we check which template types
a service can use, because there were some hard-coded assumptions about
emails and text messages.
Why we did this originally[1]:
> Calculating the number of pages in a letter is quite slow. And the
> send yourself a test pages need to load _fast_. Since filling in
> placeholders is very unlikely to change the number of pages in the
> resultant letter, it’s pretty safe to cache that count, and makes the
> subsequent pages load a lot faster.
However things have changed since then:
- this journey is used for sending real letters, not just test ones
- we’re doing enough letters that even an unlikely discrepancy will (and
does) happen
- we cache the generation of the PDF now[2], so at least it’s not
generating the PDF twice, once for the preview and once for the page
count
- it’s no longer necessary to step through each address placeholder to
populate a one-off letter, so a little bit slower isn’t so bad
1. e7896f283a
2. c9c6271aa0/app/preview.py (L140)
For services with permission, they can now put international addresses
into their spreadsheets without getting a postcode error.
This also means they can start using address line 7 instead of postcode,
since it doesn’t make sense to put a country in a field called
‘postcode’. But this will be undocumented to start with, because we’re
not giving any real users the permission.
It does now mean that the number of possible placeholders (7 + postcode)
is greater than the number of allowed placeholders (7), so we have to
account for that in the one-off address flow where we’re populating the
placeholders automatically. We’re sticking with 6 + postcode here for
backwards compatibility.
Context:
- postal addresses can be made from any of the 7 address lines now, and
the postcode can be in any one of the 7
- we can put errors across a whole row now, not just on individual cells
This commit put errors to do with the postal address as a whole across
the whole row now, rather than tying them to any one cell.
Our rules about address columns are relaxing, so that none of them are
mandatory any more. Instead you just need any 3 of the 7 to make a valid
address.
This commit updates our error messaging to reflect that.
This involves three changes which broke our code.
To validate email addresses, the optional dependency `email-validator`
must be installed<sup>1</sup>. But since we don’t use WTForms’ email
validation, we shouldn’t need to subclass it – it can just be its own
self contained thing. Then we don’t need to add the extra dependency.
When rendering textareas, and extra `\r\n` is inserted at the beginning
<sup>2</sup>. Browsers will strip this when displaying the textbox and
submitting the form, but some of our tests need updating to account for
this.
The error message for when you don’t choose an option from some radio
buttons has now changed. Rather than just accepting WTForms’ new
message, this commit makes the error messages like the examples from
the Design System<sup>3</sup>. By default it will say ‘Select an
option’, but by passing in an extra parameter (`thing`) it can be
customised to be more specific, for example ‘Select a type of
organisation’.
***
1. https://github.com/wtforms/wtforms/pull/429
2. https://github.com/wtforms/wtforms/issues/238
3. https://design-system.service.gov.uk/components/radios/#error-messages
Errors with messages being too long or empty aren’t specific to a single
cell of the uploaded spreadsheet, they’re the results of combining all
the cells with the template.
Previously we could only show errors against a specific cell. This
commit makes it possible to add a super-row which spans all the cells,
into which we can put errors.
The index (header) column then spans both these rows, to show that they
are both associated with the same row of input.
Depends on:
- [x] https://github.com/alphagov/notifications-utils/pull/719
Tests that just assert some content `in` the whole page are tricky to
debug, and make it harder to be sure that said content is showing up in
the right place, with the right markup and styling.
If you’re on a page with a normal placeholder and the previous
placeholder is one that’s also in the address block then going back to
the previous page will send you immediately forward to the current page
again.
This commit makes the back link a bit smarter by skipping over pages
where it can see that they relate to a placeholder from the address
block.
If it gets all the way to the start of the list of placeholders without
finding any non-address ones then it will default to generating a link
that will redirect the user to filling in the address block again.
We don’t really want you modifying lines of the address after you’ve
entered it. Especially when it might not be obvious that modifying the
address line placeholder will modify the address you’re sending the
letter to.
If you have an placeholder from the address block elsewhere in your
letter then you currently get redirected to the address block page
instead of being offered to fill that placeholder in. This commit
tightens up the check to only do this when the placeholder is in the
first 7 placeholders, which is where we store the address placeholders.
As part of making the API call we extra the recipient from the first
line of the address. This code was assuming that the recipient would
always have the key `address line 1`, but we’re no longer guaranteeing
that it will be capitalised and spaced exactly like that.
We’re doing this everywhere else now, so this completes the story.
It uses the same regex as elsewhere and the error messaging is
consistent (but not uniform) with the other places.
Since we’re doing normalisation and line-count-checking of addresses in
multiple places it makes sense for that code to be shared. Which is
what happened here:
https://github.com/alphagov/notifications-utils/pull/713
This commit refactors the admin code to make use of the new utils code.
Note about placeholders:
- they now go into the session as `address_line_1` instead of `address
line 1` because this is the format the API uses, so should be
considered canonical
- they are now fetched from the session in a way that isn’t sensitive
to case or underscores (using the `Columns` class)
- the API doesn’t care about case or underscores vs spaces in
placeholder names because it’s checking an instance of `Template` to
see if all the required placeholders are present (see
401c8e41d6/app/notifications/process_notifications.py (L40))
if someone starts a new one-off flow they'll get taken to the address
page. However, if someone hits the back button, they'll cycle backwards
through placeholders and will end up on the individual line pages. Lets
redirect them to the correct place.
We'll additionally need to reconstruct the address block from the
various session variables that may or may not be populated
rather than in multiple placeholders - this is the first step towards
making postcodes non-required, which is the first step towards
international letters.
they still populate address_line_# and postcode fields under the hood -
to keep validation working the same, the last line always goes into
`postcode`.
the form normalises whitespace, removes extra new lines, and enforces
that you have between three and seven lines.
if the letter repeats address placeholders further down (eg "Dear
((address_line_1))"), then it'll fill those in as well. It'll still
prompt you to fill them in, but they'll be pre-filled.
When we first built the contact list feature this endpoint didn’t exist.
Now it does we can remove the slightly cludgy looping-through-all-the-lists
code.
24px with 19px is what we use on the uploads page. On notifications page
we use 19px with 16px.
There’s some loose idea that the bigger size is for items that contain
other items.
This also increases the line height for recipients of PDF letters to
make things line up.
Since contact lists only store the email address or phone number, this
won’t work. So we should warn people, and highlight that the
personalisation in the template is the problem, since this is what they
will need to change.
You can’t send an email message template to a list of phone numbers. So
we shouldn’t show you the lists of phone numbers when you’ve chosen an
email template.
Platform admin users now have a link to upload a contact list. This
commit gives them access to the page where they can use this.
Once this commit is merged we have a way of testing the whole flow end
to end, without giving it to all users yet.
If we add another link under it will look weird having two under and
one off to the side. It looked sort of balanced when it was just one
off to the side and one under.
Having all the links under the box makes it less visually noisy.
You’ll be able to use a contact list by first choosing a template, then
choosing the list you want to use.
At the moment this shows all lists, not just the ones that are
compatible with your template.
In order to use a contact list we’re going to put you in the normal
upload a spreadsheet journey. Except without having to upload again.
So this commit adds an endpoint that takes the file from the contact
list bucket, puts it in the same place it would have gone if you’d
uploaded it, then sends you on your way.
There was a bug that displayed the error message with 2 red error boxes around the error message.
Also need another else to handle a new error message from the API, the old one can be removed after the API is deployed. But this can go first. I tested this branch with API master and the API branch with the change. I tested one off SMS and a CSV upload.
