Letters should always have a reference, because that’s what DVLA use to
tell us when they’ve sent a letter.
If a letter has a reference of `None` then DVLA say they’ve sent a
letter with a reference of `'None'`. This means we can never reconcile
the letter, which means it stays in `created`, which means it never
gets billed.
We don’t think this has affected any real letters yet, just ones that
we’ve sent as tests.
By default, SQLAlchemy will start a transaction with an
existing connection without checking that the connection
is still valid.
Enabling "pre-ping" makes the ORM send a `SELECT 1` when
acquiring a connection, which should help avoid some errors
caused by connections breaking during a DB failover.
The added statement has a constant overhead for all transactions,
so we should only keep it enabled when we're planning to switch
or upgrade the database server.
https://docs.sqlalchemy.org/en/latest/core/pooling.html#disconnect-handling-pessimistic
Also test deleting jobs with flexible data retention
Also update tests for default data retention following logic
change: dao_get_jobs_older_than_data_retention now counts
today at the start of the day, not at a time when function runs
and updated tests reflect that
Services reaching rate limits are triggering our alerts and make it
hard to find actual exceptions in the logs.
As far as the API is concerned this is not an exceptional state,
so we shouldn't log it as such.
Bumped notifications-utils to 3.7.0. Version 3.7.0 includes the
`convert_utc_to_bst` and `convert_bst_to_utc` functions and the
`LETTER_PROCESSING_DEADLINE` constant, so these have been removed from
this repo and anywhere using these has now been updated to get these
from `notifications-utils`.
Also bumped pytest by a patch version to bring in a bug fix.
When we first built letters you could only send them via a CSV upload, initially we needed a way to send those files to dvla per job.
We since stopped using this page. So let's delete it!
It was initialy called "rename" which does not comply with
RESTful CRUD (create, update, read, delete) naming practice.
We remove the 'rename' operation in favour of template folder
resource update endpoint as it allows us to extend it with other
attributes.
The `save_email` and `save_sms` jobs were updated previously to take an
optional `sender_id` and to use this if it was available. This commit
now gets the `sender_id` from the S3 metadata if it exists and passes it
through the the tasks which save the job notifications. This means SMS
and emails sent through jobs can use a specified `sender_id` instead of
the default.
Started passing `sender_id` to the `save_email`, `save_sms` and
`process_job` tasks, with a default value of `None`.
If `sender_id` is provided, the `save_email` and `save_sms` tasks will
use it to determine the reply-to email address or the SMS sender for the
notifications in the job. The `process_job` task will start using the
value in another commit.
When a template is archived, it should no longer belong to any folder.
If we don’t do this it will make it very hard to delete folders later
(because folders can only be deleted if they have no templates or folders
inside them).
We originally tried to check if the link between a template and folder
should be removed with
`if template.archived and template.folder:`
instead of using `if template.archived:`. However, this caused issues
because checking `template.folder` flushes the session. Since the
session is no longer dirty, the versioning decorator doesn't work as
expected and doesn't create a new row in `TemplateHistory`.
new endpoints:
/services/<service_id>/move-to-folder
/services/<service_id>/move-to-folder/<target_template_folder_id>
* takes in a dict containing lists of `templates` and `folders` uuids.
* sets parent of templates and folders to the folder specified in the
URL. Or None, if there was no id specified.
* if any template or folder has a differen service id, then the whole
update fails
* if any folder is an ancestor of the target folder, then the whole
update fails (as that would cause a cyclical folder structure).
* the whole function is wrapped in a single `transactional` decorator,
so in case of error nothing will be saved.
This was done so when notification is timed out from sending/pending
to temporary_failure, this change has to always be caught
in the ft_notification_status
Timeout sending notifications updates up to 4 days of notifications
older than 72 hours to correct failure status. It needs to run before
we update ft_notifications_status table. Otherwise the changes
don't get picked up. Notifications deletion tasks have to run
after those jobs in case our users set short data retention
policy.
If the parent_folder_id then check if the folder exists and is for the same service. If it is add the folder to the template model object, the relationship will be persisted when the template is saved. If the folder does not exist or is for a different service, then return a ResultNotFound error.
When creating the Tempalte from_json, the folder is passed in. Since some validation should done, as in the folder exists and is for the same service, the folder is passed through to the Tempalte.from_json method.
When the template is persisted so is the relationship to folders.
TODO: If the folder is invalid a specific message should be returned.
Updated jsonschema to Draft7, this allowed a conditional validation on subject, if template_type == 'email' or 'letter' then subject is required.
This version is backward compatible with Draft4.
When creating TempalteRedacted, I've built the dict depending on if the created_by or created_by_id exists.
