We look for `original_file_name` in the metadata now. Initially we were
still checking the query string too, but now that the change to add the
filename to the metadata has been deployed for a while there shouldn't
be any cases of the filename still being in a query string.
Since the `original_file_name` is not being added to the metadata in
`.check_messages` (it has happened earlier in the process) a few tests
are no longer needed.
When sending from an uploaded CSV `.send_messages` now puts the filename
in the metadata. It previously used the query string to pass the
filename to `.check_messages`, where it can be lost.
The `.send_from_contact_list` function redirected to `.check_messages`
with `original_file_name` in the query string. Contact lists already
have `original_file_name` as part of their metadata, so we can stop
sending it in the query string and use the metadata instead.
We don't need these anymore as all users will use the `one-off/step`
routes.
This has mostly involved tidying up the tests which are still a little
disorganised and not as good as I'd like but it's a step in the right
direction.
More refactoring is still possible to the routes, it may come in a later
PR if I have time.
There is no real reason to have to support both 'one-off' steps and also
'test' steps when sending a one off notification. It's a lot of complex
code, just to now set the one of the placeholders in the session.
We make our code much simpler but no longer using the 'test' routes but
instead adding a new endpoint to set the notification recipient when
sending to yourself before continuing on with the rest of the 'one-off'
flow.
After this is deployed for a day then we can completely remove the
'test' routes and this will help remove a lot of code complexity.
We no longer need the `start_tour` page as this has been replaced with
the new `begin_tour` page.
We also no longer need to handle the `help` argument in the
`send_test_step` or `send_one_off_step` as these no longer are
responsible for the tour and don't need to show the help text.
Worth pointing out, the new tour joins into the send one off flow. When
doing a GET `check_tour_notification`, and submitting the form shown on
this page you are POSTed to `send_notification` with `help=3`. Also for
general sending of one off notifications, the POST to
`send_notification` is done with `help=0` which is a bit of a hack to
make sure that we don't show a back link on the `view_notification` page
for when someone gets there having just sent a one off notification.
This use of `help=0` may be a candidate for a refactor in the future as
it feels like a bit of a hacky way of doing things and is therefore not
as clear to developers what is going on.
Also removes the help argument from the csv routes used here. There is
no reason that we need to ever show help for CSVs and this is leftover
code from when we used to do the tour that way.
When sending a letter we check how many pages it has and this number
then determines how many PNG images we ask template preview for. When
calculating the page count, we were getting the page count for the
template as it comes from the database (so without any placeholders
filled in). But filling in placeholders in a letter may cause the number
of pages to change, which was the cause of the 'Letter does not have a
page x' errors we were seeing from template-preview.
Now, when we calculate the letter page count during sending, we take the
placeholders that have already been filled in into account.
At the moment the page is the same as for text message templates,
except:
- different H1
- no guidance about personalisation, links, etc (until we decide how
these should work)
For now you won’t be able to really create a broadcast template, because
the API doesn’t support it (the API will respond with a 400). But that’s
OK because no real services have the broadcast permission yet.
This required a bit of refactoring of how we check which template types
a service can use, because there were some hard-coded assumptions about
emails and text messages.
Why we did this originally[1]:
> Calculating the number of pages in a letter is quite slow. And the
> send yourself a test pages need to load _fast_. Since filling in
> placeholders is very unlikely to change the number of pages in the
> resultant letter, it’s pretty safe to cache that count, and makes the
> subsequent pages load a lot faster.
However things have changed since then:
- this journey is used for sending real letters, not just test ones
- we’re doing enough letters that even an unlikely discrepancy will (and
does) happen
- we cache the generation of the PDF now[2], so at least it’s not
generating the PDF twice, once for the preview and once for the page
count
- it’s no longer necessary to step through each address placeholder to
populate a one-off letter, so a little bit slower isn’t so bad
1. e7896f283a
2. c9c6271aa0/app/preview.py (L140)
For services with permission, they can now put international addresses
into their spreadsheets without getting a postcode error.
This also means they can start using address line 7 instead of postcode,
since it doesn’t make sense to put a country in a field called
‘postcode’. But this will be undocumented to start with, because we’re
not giving any real users the permission.
It does now mean that the number of possible placeholders (7 + postcode)
is greater than the number of allowed placeholders (7), so we have to
account for that in the one-off address flow where we’re populating the
placeholders automatically. We’re sticking with 6 + postcode here for
backwards compatibility.
Context:
- postal addresses can be made from any of the 7 address lines now, and
the postcode can be in any one of the 7
- we can put errors across a whole row now, not just on individual cells
This commit put errors to do with the postal address as a whole across
the whole row now, rather than tying them to any one cell.
Our rules about address columns are relaxing, so that none of them are
mandatory any more. Instead you just need any 3 of the 7 to make a valid
address.
This commit updates our error messaging to reflect that.
If you’re on a page with a normal placeholder and the previous
placeholder is one that’s also in the address block then going back to
the previous page will send you immediately forward to the current page
again.
This commit makes the back link a bit smarter by skipping over pages
where it can see that they relate to a placeholder from the address
block.
If it gets all the way to the start of the list of placeholders without
finding any non-address ones then it will default to generating a link
that will redirect the user to filling in the address block again.
We don’t really want you modifying lines of the address after you’ve
entered it. Especially when it might not be obvious that modifying the
address line placeholder will modify the address you’re sending the
letter to.
Optional address placeholders aren’t a thing for one-off letters any
more, so we can tidy up the code a bit by removing the parts of the flow
that are accounting for them.
If you have an placeholder from the address block elsewhere in your
letter then you currently get redirected to the address block page
instead of being offered to fill that placeholder in. This commit
tightens up the check to only do this when the placeholder is in the
first 7 placeholders, which is where we store the address placeholders.
As part of making the API call we extra the recipient from the first
line of the address. This code was assuming that the recipient would
always have the key `address line 1`, but we’re no longer guaranteeing
that it will be capitalised and spaced exactly like that.
Since we’re doing normalisation and line-count-checking of addresses in
multiple places it makes sense for that code to be shared. Which is
what happened here:
https://github.com/alphagov/notifications-utils/pull/713
This commit refactors the admin code to make use of the new utils code.
Note about placeholders:
- they now go into the session as `address_line_1` instead of `address
line 1` because this is the format the API uses, so should be
considered canonical
- they are now fetched from the session in a way that isn’t sensitive
to case or underscores (using the `Columns` class)
- the API doesn’t care about case or underscores vs spaces in
placeholder names because it’s checking an instance of `Template` to
see if all the required placeholders are present (see
401c8e41d6/app/notifications/process_notifications.py (L40))
if someone starts a new one-off flow they'll get taken to the address
page. However, if someone hits the back button, they'll cycle backwards
through placeholders and will end up on the individual line pages. Lets
redirect them to the correct place.
We'll additionally need to reconstruct the address block from the
various session variables that may or may not be populated
rather than in multiple placeholders - this is the first step towards
making postcodes non-required, which is the first step towards
international letters.
they still populate address_line_# and postcode fields under the hood -
to keep validation working the same, the last line always goes into
`postcode`.
the form normalises whitespace, removes extra new lines, and enforces
that you have between three and seven lines.
if the letter repeats address placeholders further down (eg "Dear
((address_line_1))"), then it'll fill those in as well. It'll still
prompt you to fill them in, but they'll be pre-filled.
On the uploads page this should be newest first, same as the scheduled
and immediate jobs on this page.
On the _choose_ page this should be alphabetical, same as choosing a
template.
Since contact lists only store the email address or phone number, this
won’t work. So we should warn people, and highlight that the
personalisation in the template is the problem, since this is what they
will need to change.
You can’t send an email message template to a list of phone numbers. So
we shouldn’t show you the lists of phone numbers when you’ve chosen an
email template.
You’ll be able to use a contact list by first choosing a template, then
choosing the list you want to use.
At the moment this shows all lists, not just the ones that are
compatible with your template.
In order to use a contact list we’re going to put you in the normal
upload a spreadsheet journey. Except without having to upload again.
So this commit adds an endpoint that takes the file from the contact
list bucket, puts it in the same place it would have gone if you’d
uploaded it, then sends you on your way.
We don’t want to muddy them up with the normal CSV uploads.
I’ve tried to reuse the existing S3 code where possible because it’s
well tested.
Buckets have already been created.
There was a bug that displayed the error message with 2 red error boxes around the error message.
Also need another else to handle a new error message from the API, the old one can be removed after the API is deployed. But this can go first. I tested this branch with API master and the API branch with the change. I tested one off SMS and a CSV upload.
make sure everything is using the `nl2br` formatter that properly wraps
it in markdown to keep everything sanitised nicely. Also write a couple
of tests
We mostly rely on the API returning a 404 to generate 404s for trying
to get things with non-UUID IDs. This is fine, except our tests often
mock these API calls. So it could look like everything is working fine,
except the thing your passing in might never be a valid UUID, and thus
would 404 in a non-test environment.
So this commit:
1. uses the `uuid` URL converter everywhere there’s something that looks
like an ID in a URL parameter
2. adds a test which automates checking for 1.
Because it means you often have to cast to string in your application
code just to get your tests passing.
The method being monkey patched is originally defined here: b81aa0f18c/src/werkzeug/routing.py (L1272)
Rather than force us to write the decorators in a specific order let’s
just have one decorator call the other. This should make fewer lines of
code, and fewer annoying test failures. It also means that the same way
of raising a `401` (through the `current_app` method) is used
everywhere.
