Added the migration for the `ft_notification_status` table which will be
used to look up the counts for notifications by various different
fields.
The `job_id` column is non-nullable because we want to include it in the
composite primary key. For notifications which weren't part of a job,
the `job_id` will be set to `0000-00000-0000-00000` when populating the
data.
Changed the name of the `is_active` columns to `archived` in these
tables:
* `service_email_reply_to`
* `service_sms_senders`
* `service_letter_contacts`
These columns were not being used yet, but because we describe Templates as
'archived', this keeps the names consistent.
The default value of these columns now needs to be False, not True.
Added a new boolean column, `is_active` to these tables
* service_email_reply_to
* service_sms_senders
* service_letter_contacts
This has a database default of True in order to backfill the data, but
this default will be replaced with a model default later.
As it turns out the SNS topic is trigger more that once when a file is placed in S3, this is caused by a bug in the s3ftp software used to mount the S3 bucket to the FTP server.
S3ftp performs the create file operation more than once. This is resulting in duplicate counts of DailySortedLetter (the counts of how many letters marked as sorted or unsorted, which affect how much the provider will charge Notify for the letter).
This PR adds a new column to DailySortedLetter called file_name. A new unique constraint on billing_day + file_name is added. Each time we write a row to the table the counts are over written rather than aggregated.
I am aware that this PR is not backwards compatiable. However, since the code is typically triggered once a day around 13:00 then it is very unlikely an exception will occur during the deploy. Also a complete migration of the data based on all our response files on S3 will be performed soon, meaning the data will be corrected. Also if an exception does occur it is after the updates to notification status has already occurred.
We need to deal with this, it's ok when updating a notification status from delivered to delivered. But the DailySortedLetter counts are being doubled.
Adding the file_name to the table as a unique key to get around this issue. It will mean we have multiple rows for each billing_day, but that's ok we can aggregate that.
This will also give us a way to see which file created which count.
Added the following new notification statuses:
* pending_virus_check
* virus_scan_failed
If we decide to remove these statuses in future, we will need to replace
them with a different status in the notifications and
Notification_history tables where they are referenced, so
pending-virus-check will be replaced with sending, and virus-scan-failed
will be replaced with permanent-failure.
The response files we receive from the DVLA when we send letters
contain a row for each letter and a field with a value of 'Unsorted' or
'Sorted'. This table will be used to store the total number of
'Unsorted' and 'Sorted' letter notifications per day.
Since we send all one off messages as priority
[now](https://github.com/alphagov/notifications-api/pull/1722), we don’t
need to explicitly mark the ‘Example text message template’ template as
being priority.
This commit retroactively removes the priority flag from these
templates, so that they can’t be reused to send other messages as
priority. I don’t think this is a real problem now, but worth cleaning
this up.
This query will hit approximately 827 templates.
Since the application code gets released after the migration setting
a new Template field as non-nullable prevents new templates from being
created by the existing application instances.
This splits the migration to set fields as nullable first and then
update existing records once the application code has been released.
0168 is modified not to run UPDATE query or set non-nullable flag in
staging and production. 0169 is added to rollback preview.
Creates the column as nullable, sets the value to false for all
existing templates and template versions and then applies a
not-nullable constraint.
All future Templates are created with `False` as the default set
in SQLAlchemy.
we have problems where govuk service is not migrated to sms_senders on new datbases.
update this old script so we don't affect live systems, but when people rebuild their
database from scratch they get the sms sender for govuk notify.
Will specifically affect the email auth template, which was seeded
without a matching row in the `template_redacted` table.
Effectively re-runs c24edcf388
> add historical redaction data
>
> every current template gets a row in the template_redacted table -
> this inserts one for any template that doesn't already have a row,
> with redact set to false, the user set to NOTIFY_USER since it was
> just a script, and the updated_at set to the time the script is run
The default for the rate_limit column in the services and
services_history model is now set in the model, so we can remove the
default from the database.
Pivotal story: https://www.pivotaltracker.com/story/show/153992529
The API rate limit will be removed from the config and added to services
so that it is possible to change the rate_limit for individual services
in rare cases.
Pivotal story: https://www.pivotaltracker.com/story/show/153992529
