We don’t really want you modifying lines of the address after you’ve
entered it. Especially when it might not be obvious that modifying the
address line placeholder will modify the address you’re sending the
letter to.
Optional address placeholders aren’t a thing for one-off letters any
more, so we can tidy up the code a bit by removing the parts of the flow
that are accounting for them.
If you have an placeholder from the address block elsewhere in your
letter then you currently get redirected to the address block page
instead of being offered to fill that placeholder in. This commit
tightens up the check to only do this when the placeholder is in the
first 7 placeholders, which is where we store the address placeholders.
As part of making the API call we extra the recipient from the first
line of the address. This code was assuming that the recipient would
always have the key `address line 1`, but we’re no longer guaranteeing
that it will be capitalised and spaced exactly like that.
Since we’re doing normalisation and line-count-checking of addresses in
multiple places it makes sense for that code to be shared. Which is
what happened here:
https://github.com/alphagov/notifications-utils/pull/713
This commit refactors the admin code to make use of the new utils code.
Note about placeholders:
- they now go into the session as `address_line_1` instead of `address
line 1` because this is the format the API uses, so should be
considered canonical
- they are now fetched from the session in a way that isn’t sensitive
to case or underscores (using the `Columns` class)
- the API doesn’t care about case or underscores vs spaces in
placeholder names because it’s checking an instance of `Template` to
see if all the required placeholders are present (see
401c8e41d6/app/notifications/process_notifications.py (L40))
if someone starts a new one-off flow they'll get taken to the address
page. However, if someone hits the back button, they'll cycle backwards
through placeholders and will end up on the individual line pages. Lets
redirect them to the correct place.
We'll additionally need to reconstruct the address block from the
various session variables that may or may not be populated
rather than in multiple placeholders - this is the first step towards
making postcodes non-required, which is the first step towards
international letters.
they still populate address_line_# and postcode fields under the hood -
to keep validation working the same, the last line always goes into
`postcode`.
the form normalises whitespace, removes extra new lines, and enforces
that you have between three and seven lines.
if the letter repeats address placeholders further down (eg "Dear
((address_line_1))"), then it'll fill those in as well. It'll still
prompt you to fill them in, but they'll be pre-filled.
On the uploads page this should be newest first, same as the scheduled
and immediate jobs on this page.
On the _choose_ page this should be alphabetical, same as choosing a
template.
Since contact lists only store the email address or phone number, this
won’t work. So we should warn people, and highlight that the
personalisation in the template is the problem, since this is what they
will need to change.
You can’t send an email message template to a list of phone numbers. So
we shouldn’t show you the lists of phone numbers when you’ve chosen an
email template.
You’ll be able to use a contact list by first choosing a template, then
choosing the list you want to use.
At the moment this shows all lists, not just the ones that are
compatible with your template.
In order to use a contact list we’re going to put you in the normal
upload a spreadsheet journey. Except without having to upload again.
So this commit adds an endpoint that takes the file from the contact
list bucket, puts it in the same place it would have gone if you’d
uploaded it, then sends you on your way.
We don’t want to muddy them up with the normal CSV uploads.
I’ve tried to reuse the existing S3 code where possible because it’s
well tested.
Buckets have already been created.
There was a bug that displayed the error message with 2 red error boxes around the error message.
Also need another else to handle a new error message from the API, the old one can be removed after the API is deployed. But this can go first. I tested this branch with API master and the API branch with the change. I tested one off SMS and a CSV upload.
make sure everything is using the `nl2br` formatter that properly wraps
it in markdown to keep everything sanitised nicely. Also write a couple
of tests
We mostly rely on the API returning a 404 to generate 404s for trying
to get things with non-UUID IDs. This is fine, except our tests often
mock these API calls. So it could look like everything is working fine,
except the thing your passing in might never be a valid UUID, and thus
would 404 in a non-test environment.
So this commit:
1. uses the `uuid` URL converter everywhere there’s something that looks
like an ID in a URL parameter
2. adds a test which automates checking for 1.
Because it means you often have to cast to string in your application
code just to get your tests passing.
The method being monkey patched is originally defined here: b81aa0f18c/src/werkzeug/routing.py (L1272)
Rather than force us to write the decorators in a specific order let’s
just have one decorator call the other. This should make fewer lines of
code, and fewer annoying test failures. It also means that the same way
of raising a `401` (through the `current_app` method) is used
everywhere.
At the moment we mostly have `user_has_permissions` execute first. It
shouldn’t matter, but it feels right for us to check that a user is
logged in before we check their permissions to a service. Otherwise a
malicious user could (maybe) check if a service ID belongs to a real
service, and go on to do something malicious with that information.
This commit adds some extra test code to enforce that the order is
always the same.
N.B. decorators in Python execute from closest to furthest (from the
line on which the function is defined).
This fixes the bug where if you have three placeholders:
> ((one)) ((two)) ((three))
The first one you are asked to fill in is `((three))` (ie
`template.placeholders[-1]`).
This reintroduces a bug where if you use the ‘Use my phone number’ link
you skip straight to filling in `((two))` and can never proceed because
you’re never given the chance to fill in `((one))`. This commit also
fixes that bug.
The data flow of other bits of our application looks like this:
```
API (returns JSON)
⬇
API client (returns a built in type, usually `dict`)
⬇
Model (returns an instance, eg of type `Service`)
⬇
View (returns HTML)
```
The user API client was architected weirdly, in that it returned a model
directly, like this:
```
API (returns JSON)
⬇
API client (returns a model, of type `User`, `InvitedUser`, etc)
⬇
View (returns HTML)
```
This mixing of different layers of the application is bad because it
makes it hard to write model code that doesn’t have circular
dependencies. As our application gets more complicated we will be
relying more on models to manage this complexity, so we should make it
easy, not hard to write them.
It also means that most of our mocking was of the User model, not just
the underlying JSON. So it would have been easy to introduce subtle bugs
to the user model, because it wasn’t being comprehensively tested. A lot
of the changed lines of code in this commit mean changing the tests to
mock only the JSON, which means that the model layer gets implicitly
tested.
For those reasons this commit changes the user API client to return
JSON, not an instance of `User` or other models.
first way round and then collected placeholders again. Now the flow
collects all placeholders in one round.
Also fix the back link for step-1 for test flow so it goes back
to choosing recipient number
Also move operators around following flake8's advice :)
Doing a lookup with `step_index - 1` means that on step `0` we were
looking up `placeholders[-1]`, ie we were making people fill in the last
placeholder first.
Fixing this reintroduces the bug fixed by this pull request:
https://github.com/alphagov/notifications-admin/pull/2551
So this commit also re-fixes that bug but in a different way.
Instead of using the API client directly views are now calling one
of two Service model methods:
`get_template` is used for view actions, where the user should see
the template page even if they don't have access to the template
folder (since all templates are still inked from the dashboard or
the sent notifications pages).
`get_template_with_user_permission_or_403` will check if the user
has access to the template's folder first and return 403 otherwise.
This method is used for any endpoints that result in an action: editing
template attributes, deleting templates or sending messages.
Upgraded pyexcel-io from 0.5.14 to 0.5.16. This change causes Werkzeug
to be upgraded from 0.14.1 to 0.15.1 which requires some changes:
* ProxyFix now needs to be imported from a different location
* The status code of RequestRedirect has changed from 301 to 308. Since
status code 308 is not currently supported on Internet Explorer with
Windows 7 and 8.1, this subclasses RequestRedirect to keep the status
code of 301.
changelog: https://werkzeug.palletsprojects.com/en/0.15.x/changes/#version-0-15-0
Separated s3_client.py into 3 files - for logos, CSV files and the MOU.
This helps to keep things clearer now that we need to add lots more logo
functions for letters.
We have some teams who haver a series of files they have to send each
day. It’s easy to get muddled up and accidentally send the same file
again, if you think you haven’t already sent it.
This commit blocks you from sending the same combination of template
version and filename more than once on the same day[1].
This won’t affect teams who re-use the same template to give (for
example) updates on an incident for business continuity. These teams
edit the template between each send, thereby updating the version
number of the template.
1. This is based on how the `limit_days` argument to the API works - you
can dig into the code here: 2bd4f74ad0/app/dao/jobs_dao.py (L50)
