This repeats the pattern we already have for previewing a letter,
where we assume the error is because the notification has already
been sent and redirect the user to see it.
I've improved the original pattern a bit:
- I've DRYed-up the low-level boto code and moved the error handler
there so it can be reused.
- I've introduced a custom exception, which the calling code can
choose to log.
- I've introduced the moto library, which we use elsewhere, to make
it easier to test S3 code.
I've used an error level log when sending a notification - now that
we have a more descriptive log, we can verify the assumption is true
and then make an informed decision to downgrade the log.
In future we may want to merge this handler with the similar code
in utils [1], but we'll need to be careful as the utils handler is
superficial - it doesn't check the reason for the error.
[1]: bce0f4e596/notifications_utils/s3.py (L52)
This continues the work from Template Preview [1], so that we have
a complete store of original PDFs to use for testing changes to it.
Previously we did store some originals, but these were only invalid
PDFs that had failed sanitisation; for valid PDFs, the "transient"
bucket only contains the sanitised versions, which the API deletes
/ moves when the notification is sent [2].
Since the notification is only created at a later stage [3], there's
no easy way to get the final name of the PDF we send to DVLA. Instead,
we use the "upload_id", which eventually becomes the notification ID
[4]. This should be enough to trace the file for specific debugging.
Note that we only want to store original PDFs if they're valid (and
virus free!), since there's no point testing changes with bad data.
[1]: https://github.com/alphagov/notifications-template-preview/pull/545
[2]: c44ec57c17/app/service/send_notification.py (L212)
[3]: 7930a53a58/app/main/views/uploads.py (L362)
[4]: 7930a53a58/app/main/views/uploads.py (L373)
This is for consistency with how we do it for filenames in the previous
commit and moves the decoding into the `LetterMetadata` class for
abstracting this behaviour.
Small refactor of the LetterMetadata class needed to handle None case as
recipient can be None.
S3 can only handle ascii characters, therefore for filename which could
include non ascii characters, for example a filename with the character
'£' in it, we must encode these using urllib before saving it as s3
metadata. We then also make sure that it comes back decoded when
presenting it to the user.
S3 metadata only supports ascii characters. Whenever we save data to it
we need to make sure we encode it to save it and then decode it to
display it again to users. This abstraction will act as the place for
that decoding to happen so the rest of the code in our views doesn't
need to care about the encoding abstraction.
Now persisting the address to the "to" field of the Notification, after the notification has been validated.
If the letter is pending validation, then "Checking..." will appear as the identifier for the letter.
If the letter has passed validation, then the first line of the address (now persisted in the "to" field) will be displayed, with the client reference underneath.
If the letter has failed validation the "Provided as PDF" will show be displayed, which is now the initial value of the "to" field.
The recipient of the letter now displays at the bottom of the page when
previewing a valid letter. The template preview `/precompiled/sanitise`
endpoint returns the address, but we format it to display on a single
line with commas between each line. We also need to convert the
recipient address to ASCII so that it can be stored as S3 metadata.
We had been storing whether or not a file was valid in the S3 metadata,
but using the query string of the URL to store the original filename
and the page count. This meant that if you tried to view the preview
letter page without the query string you would see a `500`. It was
possible for this to happen if you were signed out of Notify while on
the preview page - you would be redirected back to the preview page but
without the query string, causing an error.
This sanitises uploaded letters and stores the sanitised result in S3
with if it passes validation or the original PDF in S3 if validation
fails. A metadata value of 'status' is set to either 'valid' or
'invalid'.
