Google’s documentation says:
> robots.txt is not a mechanism for keeping a web page out of Google. To
> keep a web page out of Google, you should use noindex directives
We’ve implemented a noindex directive now, so we don’t need to serve
robots.txt any more.
Reimplements https://github.com/alphagov/notifications-aws/pull/796
Since deploying alphagov/notifications-utils#736 I’ve been looking at
how members of the public are ending up on our support page. The vast
majority are landing on https://www.notifications.service.gov.uk/features/email
Previously we thought that they were clicking the ‘contact us’ link in
the page, which deep linked into the support journey, so we removed
these deep links in alphagov/notifications-admin#3451
But the tickets are still coming in, so I think that people are still
landing on this page, then going directly to ‘support’ in the top
navigation. So the next measure we have available is to try to stop
people from landing on this page in the first place. All the examples
I’ve looked at show people coming from Google to this page. By putting
the page’s URL in our robots.txt it should stop Google (and other search
engines) listing it in search results.
Google’s documentation says:
> robots.txt is not a mechanism for keeping a web page out of Google. To
> keep a web page out of Google, you should use noindex directives
A noindex directive means adding the following meta tag to pages that
shouldn’t be indexed:
```html
<meta name="robots" content="noindex" />
```
It’s also possible to set the directive as a HTTP header, but this seems
trickier to achieve on a per-view basis in Flask.
I’ve implemented this as a decorator so it can quickly be added to any
other pages that we decide shouldn’t appear in search results.
International letters don’t have a choice of postage. Under the hood
they are either `europe` or `rest-of-world`.
So, for letters that we detect are international, this commit:
- removes the radios buttons that give users the choice of postage
- passes through either `europe` or `rest-of-world` to the API,
depending on what address we find in the letter
This will cause the API to 500 until it can accept `europe` or
`rest-of-world` as postage types, but this is probably OK because it’s
only our services that have international letters switched on at the
moment.
Because we no longer need the form to get the `file_id`, we can get the
metadata before building the form.
This will, in subsequent commits, let us build the form differently
based on the recipient metadata.
Also removed some variables that were assigned to then only used once
and reformatted arguments for readability.
In the future we need to get the metadata from the file in order to work
out what form validation rules should apply (postage is only required
for UK letters).
To start doing this we need all instances of the app accepting `post`
requests with the `file_id` in the URL, as well as in the form data (for
backwards compatibility).
The `_add_invited_user_to_service` function was calling the
`user_api_client` directly to add a user to a service. It now calls the
`add_to_service` method on the User model instead so that there is only
one place in the code that calls the `user_api_client`.
This is for consistency with how we do it for filenames in the previous
commit and moves the decoding into the `LetterMetadata` class for
abstracting this behaviour.
Small refactor of the LetterMetadata class needed to handle None case as
recipient can be None.
Up till now, when adding new organisation domain, if it was already
in use, we didn't handle the 400 we got back from API. This PR
adds handling for that error.
Why we did this originally[1]:
> Calculating the number of pages in a letter is quite slow. And the
> send yourself a test pages need to load _fast_. Since filling in
> placeholders is very unlikely to change the number of pages in the
> resultant letter, it’s pretty safe to cache that count, and makes the
> subsequent pages load a lot faster.
However things have changed since then:
- this journey is used for sending real letters, not just test ones
- we’re doing enough letters that even an unlikely discrepancy will (and
does) happen
- we cache the generation of the PDF now[2], so at least it’s not
generating the PDF twice, once for the preview and once for the page
count
- it’s no longer necessary to step through each address placeholder to
populate a one-off letter, so a little bit slower isn’t so bad
1. e7896f283a
2. c9c6271aa0/app/preview.py (L140)
Single checkboxes are distinct because:
- they don't need to be wrapped in a `<fieldset>`
- they are a subclass of BooleanField so their
data is either True or False
Nested checkboxes with a single top-level node
will only have one item in their `items` list.
This is because the other choices are children of
that list item.
This means we need to check the `choices`
attribute, which lists all the checkboxes, to see
if they should be marked as a group (by being
wrapped in a `<fieldset>`) or not.
Includes:
1. changes to make NestedFieldMixin work
with new fields and CSS for nested checkboxes
2. adds custom version of GOVUK checkboxes
component to allow us to:
- add classes to elements currently inaccessible
- wrap the checkboxes in a list
- add child checkboxes to each checkbox (making
tree structures possible through recursion
Change 2. should be pushed upstream to the GOVUK
Design System as a proposal for changes to the
GOVUK Checkboxes component.
Allows checkboxes to be collapsed so they take up
less space in the page. The collapsed state
includes a live summary tracking which of them are
selected.
Includes changes to the JS for collapsible
checkboxes to make it work with the GOVUK
Checkboxes component HTML.
govukCheckboxesField subclasses
SelectMultipleField and overwrites how it renders
HTML to let us use the GOVUK Checkboxes component
while retaining all the functionality of WTForms
fields.
Based on work on github.com/richardjpope/recourse:
https://github.com/richardjpope/recourse/blob/master/recourse/forms.py#L6
If you’ve come to look at a notification via the uploaded letters page
then the ‘< back’ link should take you back there, not to the usual
activity page.
Some teams have started uploading quite a lot of letters (in the
hundreds per week). They’re also uploading CSVs of emails. This means
the uploads page ends up quite jumbled.
This is because:
- there’s just a lot of items to scan through
- conceptually it’s a bit odd to have batches of things displayed
alongside individual things on the same page
So instead we’re going to start grouping together uploaded letters. This
will be by the date on which we ‘start’ printing them, or in other
words the time at which they can no longer be cancelled.
This feels like a natural grouping, and it matches what we know about
people’s mental models of ‘batches’ and ‘runs’ when talking about
printing.
This grouping will be done in the API, so all this commit need to do is:
- be ready to display this new type of pseudo-job
- link to the page that displays all the uploaded letters for a given
print day
Because we won’t be showing uploaded letters individually on the uploads
page any more we need a way of listing them. This should be by printing
day, to match how we’re grouping them on the uploads page.
This code reuses the notifications.html template, but flips the
precedence of the filename and recipient because I reckon when you’re
looking at uploads you’re thinking filename-first.
From a question on cross-government Slack:
> re the Usage tab - currently it shows 3 financial years - last year,
> this year and next year. is it possible to replace the "next year" tab
> with something more useful? its always going to be blank! I was
> thinking it would be good to have 2 financial years ago, 1 financial
> year ago and this financial year.
This seems like a reasonable idea, and is something we’ve talked about
before. The original intention[1] was that seeing your (unchanged) free
allowance for next year would be useful, but that doesn’t really seem to
be a user need.
***
1. See https://github.com/alphagov/notifications-admin/pull/1094
> so that you can check what your SMS allowance is going to be before
> you actually get into it
The search form makes a post request, so that phone numbers and email
addresses don’t show up in logs or browser history.
At most the API will return 50 results, with some pagination links. We
can’t easily give you links to click in the admin app, because links can
only perform get requests.
Because the value of seeing more than 50 results feels quite low (users
will probably make their search more specific before scrolling through
all 50) let’s just show a message saying only the first 50 results are
displayed.
Some email clients will pre-fetch links in emails to check whether
they’re safe. This has the unfortunate side effect of claiming the token
that’s in the link.
Long term, we don’t want to let the link be used multiple times, because
this reduces how secure it is (eg someone with access to your browser
history could re-use the link even if you’d signed out).
Instead, this commit adds an extra page which is served when the user
clicks the link from the email. This page includes a form which submits
to the actual URL that uses the token, thereby not claiming the token as
soon as the page is loaded.
For convenience, this page also includes some Javascript which clicks
the link on the user’s behalf. If the user has Javascript turned off
they will see the link and can click it themselves. This is going on the
assumption that whatever the email clients are doing when prefetching
the link doesn’t involve running any Javascript.
This Javascript is inlined so that:
- it is run as fast as possible
- it’s more resilient – even if our assets domain is unreachable or the
connection is interrupted, it will still run
We’re going to add an interstitial page that redirects to this new URL.
But we don’t want that redirect to 404 while the change is deploying,
because some boxes will have the new URL and some won’t. So let’s deploy
the new URL to all the boxes first, then the redirect page can safely
take over the new one.
The new URL is going to be `post` not `get` because that feels more
HTTP-y, so we need to make sure that’s part of this change too.
If a service has permission to send international letters then the admin
app should tell template preview, so that template preview knows what
rules to apply when it’s validating the address of the letter.
We don’t need to wait for template preview to start looking at this
query string argument – it will just ignore it for now.
For services with permission, they can now put international addresses
into their spreadsheets without getting a postcode error.
This also means they can start using address line 7 instead of postcode,
since it doesn’t make sense to put a country in a field called
‘postcode’. But this will be undocumented to start with, because we’re
not giving any real users the permission.
It does now mean that the number of possible placeholders (7 + postcode)
is greater than the number of allowed placeholders (7), so we have to
account for that in the one-off address flow where we’re populating the
placeholders automatically. We’re sticking with 6 + postcode here for
backwards compatibility.
