Google’s documentation says:
> robots.txt is not a mechanism for keeping a web page out of Google. To
> keep a web page out of Google, you should use noindex directives
We’ve implemented a noindex directive now, so we don’t need to serve
robots.txt any more.
Reimplements https://github.com/alphagov/notifications-aws/pull/796
Since deploying alphagov/notifications-utils#736 I’ve been looking at
how members of the public are ending up on our support page. The vast
majority are landing on https://www.notifications.service.gov.uk/features/email
Previously we thought that they were clicking the ‘contact us’ link in
the page, which deep linked into the support journey, so we removed
these deep links in alphagov/notifications-admin#3451
But the tickets are still coming in, so I think that people are still
landing on this page, then going directly to ‘support’ in the top
navigation. So the next measure we have available is to try to stop
people from landing on this page in the first place. All the examples
I’ve looked at show people coming from Google to this page. By putting
the page’s URL in our robots.txt it should stop Google (and other search
engines) listing it in search results.
Google’s documentation says:
> robots.txt is not a mechanism for keeping a web page out of Google. To
> keep a web page out of Google, you should use noindex directives
A noindex directive means adding the following meta tag to pages that
shouldn’t be indexed:
```html
<meta name="robots" content="noindex" />
```
It’s also possible to set the directive as a HTTP header, but this seems
trickier to achieve on a per-view basis in Flask.
I’ve implemented this as a decorator so it can quickly be added to any
other pages that we decide shouldn’t appear in search results.
International letters don’t have a choice of postage. Under the hood
they are either `europe` or `rest-of-world`.
So, for letters that we detect are international, this commit:
- removes the radios buttons that give users the choice of postage
- passes through either `europe` or `rest-of-world` to the API,
depending on what address we find in the letter
This will cause the API to 500 until it can accept `europe` or
`rest-of-world` as postage types, but this is probably OK because it’s
only our services that have international letters switched on at the
moment.
Because we no longer need the form to get the `file_id`, we can get the
metadata before building the form.
This will, in subsequent commits, let us build the form differently
based on the recipient metadata.
Also removed some variables that were assigned to then only used once
and reformatted arguments for readability.
The endpoint works fine with it in the URL now instead, so we need stop
posting it. We can’t stop expecting it yet, because some old instances
will still be posting to the endpoint without the ID in the url.
In the future we need to get the metadata from the file in order to work
out what form validation rules should apply (postage is only required
for UK letters).
To start doing this we need all instances of the app accepting `post`
requests with the `file_id` in the URL, as well as in the form data (for
backwards compatibility).
API gives an error if it tries to add a user to a service and that user is
already a member of the service. This situation shouldn't occur - admin checks
if an invited user is a member of a service before calling API, but we
have seen this error occurring when there are two requests processing at
the same time.
This change catches the errors from API if a user is already a member of
a service and redirects the user to the service dashboard so that they
don't see an error page.
By deep linking to the support form we skip the question that asks if
someone is a member of the public.
This seems like a helpful thing when we’re directing people to where
they can ask for document download to be switched on. But what I think
is happening (backed up from one example I can see in Kibana):
1. Someone searches for something like ‘email GOV.UK’
2. They land on https://www.notifications.service.gov.uk/features/email
3. They scan the page and think, hmm not sure what this is, I’m not
seeing what I’m looking for here…
4. …but aha, there’s a link that says ‘contact us’
5. They click it and land on
https://www.notifications.service.gov.uk/support/ask-question-give-feedback
which has a nice big box to type in
This commit removes these deep links from non-logged-in pages, so that
everyone has to go through the ‘are you a member of the public’ question
before they get to the big typey box.
The `_add_invited_user_to_service` function was calling the
`user_api_client` directly to add a user to a service. It now calls the
`add_to_service` method on the User model instead so that there is only
one place in the code that calls the `user_api_client`.
This is for consistency with how we do it for filenames in the previous
commit and moves the decoding into the `LetterMetadata` class for
abstracting this behaviour.
Small refactor of the LetterMetadata class needed to handle None case as
recipient can be None.
S3 can only handle ascii characters, therefore for filename which could
include non ascii characters, for example a filename with the character
'£' in it, we must encode these using urllib before saving it as s3
metadata. We then also make sure that it comes back decoded when
presenting it to the user.
These args are not inputs to the function under test, neither as way of
named arguments or as GET query parameters. I assume this has been
leftover from a previous refactor of behaviour.
S3 metadata only supports ascii characters. Whenever we save data to it
we need to make sure we encode it to save it and then decode it to
display it again to users. This abstraction will act as the place for
that decoding to happen so the rest of the code in our views doesn't
need to care about the encoding abstraction.
