The JobStatistics table is going to be deleted. There are currently
3 tasks which use the JobStatistics model via the Statistics DAO, so we
need to make sure that these tasks aren't being used before they are
deleted in a separate PR.
This commit deletes:
* The `create_initial_notification_statistic_tasks` function which gets
used to call the `record_initial_job_statistics` task.
* The `create_outcome_notification_statistic_tasks` function which gets
used to call the `record_outcome_job_statistics` task.
* And the scheduling of the `timeout-job-statistics` scheduled task.
correct type for the call as there are some that require json and some
binary. The additional checks ensure that that json decode either fails
or succeeds in the correct case.
In the update_letter_notifications_statuses task we now check whether
each row in the response file that we receive from the DVLA has the
value of 'Sorted' or 'Unsorted' in the postcode validation field. We
then calculate the number of Sorted and Unsorted rows for each day and
save each day as a row in the daily_sorted_letter table.
The data in daily_sorted_letter table should be in local time, so we
convert the datetime before saving.
We don’t have any way of playing back the totals we send to performance
platform.
This commit copies the command used to backfill the processing time and
adapts it to backfill the totals instead. Under the hood it uses the
same code that we use in the scheduled tasks to update performance
platform on a daily basis. I had to modify this code to take a `day`
argument because it was hardcoded to only work for ‘yesterday’.
* Rather than an abort 404 returned a 500 and InvalidRequest so that the
error is more easily handled on the admin console. If the file is
missing but expected to be there is actually an internal error for admin
* Refactored the code to remove duplicate code in calls to template
preview by creating a new private method which is called with specific
parameters
* Added is_precompiled_letter method to letter/utils.py
* Added tests for letter/utils.py
* Added tests for the rest endpoint
* Moved the Precompiled name to a central location
* Added hidden field to the test method to create a template
This will continue to update the notification history for letter notifications.
We currently have an issue where the responses to letters from the provider is taking a long time.
This is due to the manual nature of their process.
Updating the status of the letter will still work if the notification has been purged.
Also turned back on the purge letter notification scheduled task.
* Added missing items from template which are required
* Returned the file as a JSON string with the file as a base64 encoded
string
* Updated tests to match teh desired format
Is will allow the admin application to call into the api instead of
making the call itself. This will allow the api to make decision for
precompiled pdf without having to update the admin app.
- Added new endpoint
- Added tests for the endpoint
We were already counting the billable units in PDFs that we generate.
We are now also counting the number of billable units in pre-compiled letters
and saving the result.
The vast majority of messages that are being sent one-off are
time-sensitive. A typical example is a caseworker on the phone who sends
a message at the end of the call. They normally wait until the message
has been delivered, so all the time they’re waiting is time when they
can’t be helping someone else.
What we don’t want to happen is for the messages they’re sending to get
stuck behind a big lump of GOV.UK Subscription emails or passport
reminder texts. I think the best way to do this is shift them onto the
priority queue.
We’re currently seeing queue sizes of up to 5,000 on the ‘normal’
queues; I don’t think there’s any risk of this change making the
priority queue more heavily-laden than this. Especially since the
traffic patterns of users sending one-off messages won’t be spiky.
- was getting KeyError: 'Error' test failures due to the side_effect not generating enough information to be used in the ClientError class, this PR adds missing information.
Postcodes are required for created letters, but not for precompiled, this fix allows postcodes to be None in the model.
As postcodes are still required for created letter they should be caught by validation schemas in the POST handler
Filtering out hidden templates requires all existing templates to
have `hidden` flag set, which can only be done by a migration after
the code that sets the flag to `False` by default for new templates
has been released.
This removes the filtering logic until the migration has been released.
Pre-compiled letter endpoint uploads PDF contents to S3 directly
instead of creating a letter task to generate PDF using template
preview.
This moves some of the utility functions used by existing letter
celery tasks to app.letters.utils, so that they can be reused by
the API endpoint.
The Notify team needs to investigate when a notification is marked as failed.
We will process the whole file and mark the notifications with the appropriate status, if any are failed an exception is raised.
The exception will trigger a cloud watch error for the team to investigate.
