also create a PDFNotReadyError class, separate to BadRequestError, to
imply to the end user that this is something they should handle
separately to all the other errors
it felt very awkward when the body of a pdf might be empty, might have
things in it, and whether it is empty or not can change even when the
status is the same (a created template notification might have a pdf,
but might not, we don't know).
So move it to its own endpoint, so we can hand craft some 400 errors
that appropriately explain what's going on.
We want a way of getting the hidden precompiled template from admin,
so this adds an endpoint which gets the template or creates it if it doesn't
exist. The function to get or create the hidden template already existed
but has been moved to the template DAO now that it is used in more
places.
previously, we didn't create templated letters, and just marked them as
delivered straight away. However, we may need to return PDFs for these
letters, so we should create them the same as live letters. Then update
the functions so that they know where to look for these letters.
this is only applicable when getting a single notification by id. it's
also ignored if the notification isn't a letter.
Otherwise, it overwrites the 'body' field in the response.
If the notification's pdf is available, it returns that, base64
encoded. If the pdf is not available, it returns an empty string.
The pdf won't be available if the notification's status is:
* pending-virus-scan
* virus-scan-failed
* validation-failed
* technical-failure
The pdf will be retrieved from the correct s3 bucket based on its type
antivirus is sometimes tough to get running locally - now in dev
antivirus is skipped unless `ANTIVIRUS_ENABLED=1` is set on the command
line. on all other environments it is always enabled.
Renamed the 'datetime' format to 'datetime_within_next_day'. This format
is used to validate the date and time of scheduled notifications, not to
check the format of a datetime.
We need to set template_with_content placeholder values with an
updated personalisation dictionary after processing the document
uploads in order to avoid returning base64 file data instead of
the document URLs in the response contents.
Catches the requests exception for document-download-api calls, logs
a warning and returns a matching response code and message.
Connection errors to document download result in 503 response to the
user.
Adds support for a new personalisation value type: file upload.
File uploads are represented as a dictionary with a "file" key and
a base64-encoded file data as the key's value:
```
personalisation={
'field1': {'file': '<base64-encoded file contents>'}
}
```
Post notification endpoint checks the request personalisation data
looking for the file uploads in personalisation data. If any are
found and the service has permissions to upload documents the files
are sent to document download API and personalisation values are
replaced with the URLs returned in the document download response.
A fake document URL is returned for simulated notifications, no
documents are stored in Document Download.
Multiple files can be uploaded for one notification by providing
a file upload in more than one personalisation field.
- Separated the logic of precompiled and template letters.
- Remove the check for research mode, research mode is not relevant to api calls. The test key is used for testing.
Refactor upload_pdf_letter to accept a precompile boolean to save a query to template.
Previously "Result not found" would be returned when the id is not a valid uuid, which does not make sense.
Now the message says "notification_id is not a valid UUID", this should be a clearer message for the client service.
- precompiled PDFs sent by test key uploaded to scan bucket
- set status to VIRUS-SCAN-FAILED for pdfs failing virus scan rather than PERMANENT-FAILURE
- Make call to AV app for precompiled letters sent via a test key, and set notification status to PENDING-VIRUS-SCAN
and the client reference. It could confuse the client consumer so best
to remove it from the response altogether.
* Respond with only the notification_id and client_reference
* Updated the test to check for the response without the template
The template in precompiled is only used internally and cannot be
accessed from the UI. It could confuse the client consumer so best to
remove it from the response altogether.
* Remove the template from the dict before returning it and therefore
ensuring it still stays similar to letter
* Updated the test to check for the response without the template
We were already counting the billable units in PDFs that we generate.
We are now also counting the number of billable units in pre-compiled letters
and saving the result.
Pre-compiled letter endpoint uploads PDF contents to S3 directly
instead of creating a letter task to generate PDF using template
preview.
This moves some of the utility functions used by existing letter
celery tasks to app.letters.utils, so that they can be reused by
the API endpoint.
This PR is a proposal to reduce the average messages we see for a single notification from about 7 messages to 2.
Messaging would change to something like this:
February 2nd 2018, 15:39:05.885 Full delivery response from Firetext for notification: 8eda51d5-cd82-4569-bfc9-d5570cdf2126
{'status': ['0'], 'reference': ['8eda51d5-cd82-4569-bfc9-d5570cdf2126'], 'time': ['2018-02-02 15:39:01'], 'code': ['000']}
February 2nd 2018, 15:39:05.885 Firetext callback return status of 0 for reference: 8eda51d5-cd82-4569-bfc9-d5570cdf2126
February 2nd 2018, 15:38:57.727 SMS 8eda51d5-cd82-4569-bfc9-d5570cdf2126 sent to provider firetext at 2018-02-02 15:38:56.716814
February 2nd 2018, 15:38:56.727 Starting sending SMS 8eda51d5-cd82-4569-bfc9-d5570cdf2126 to provider at 2018-02-02 15:38:56.408181
February 2nd 2018, 15:38:56.727 Firetext request for 8eda51d5-cd82-4569-bfc9-d5570cdf2126 finished in 0.30376038211397827
February 2nd 2018, 15:38:49.449 sms 8eda51d5-cd82-4569-bfc9-d5570cdf2126 created at 2018-02-02 15:38:48.439113
February 2nd 2018, 15:38:49.449 sms 8eda51d5-cd82-4569-bfc9-d5570cdf2126 sent to the priority-tasks queue for delivery
To somthing like this:
February 2nd 2018, 15:39:05.885 Firetext callback return status of 0 for reference: 8eda51d5-cd82-4569-bfc9-d5570cdf2126
February 2nd 2018, 15:38:49.449 sms 8eda51d5-cd82-4569-bfc9-d5570cdf2126 created at 2018-02-02 15:38:48.439113
- task not really necessary as the status is already set to 'sending' before the task is called if it is not sending i.e. in reseach mode or using a test key
- calls create fake response file to allow functional tests to run and trigger update of status to delivered only on preview and development so that FT response files don't pollute staging and live buckets
needed for monitoring the performance of the v2 endpoints. They were put
in as a temporary measure whilst sustained performance testing was
taking place.
- Disable Redis as there is a current connection limit of 256 which
could slow down the request if they are all used
- Added statd to methods in the post to help spot any bottlenecks
