This is text value of the sender_id, depending on the channel this will be a SMS sender, email reply to address or a letter contact block.
This is the first PR in a series to refactor how we send the "reply_to" the provider, eventually we can eliminate the notification_to_sms_sender and notification_to_email_to tables.
Removes "NOT VALID" mark from the notifications -> template_history
foreign key constraint by validating existing records.
From postgres docs:
> This form validates a foreign key or check constraint that was
> previously created as NOT VALID, by scanning the table to ensure there
> are no rows for which the constraint is not satisfied. Nothing happens
> if the constraint is already marked valid.
> Validation can be a long process on larger tables. The value of
> separating validation from initial creation is that you can defer
> validation to less busy times, or can be used to give additional time
> to correct pre-existing errors while preventing new errors. Note also
> that validation on its own does not prevent normal write commands
> against the table while it runs.
> Validation acquires only a SHARE UPDATE EXCLUSIVE lock on the table
> being altered. If the constraint is a foreign key then a ROW SHARE
> lock is also required on the table referenced by the constraint.
We have now:
- defaulted new services to start with this column set to `true`
- migrated all preexisting services[1] to have either `true` or `false` set
for this column
There is no way for a service to switch back from `true`/`false` to
`null`.
This means that we can safely enforce a non-nullable constraint on this
column now.
1. There is a little gotcha: the GOV.UK Notify service still relies on
the `sms_sender` column. It doesn’t have a row in the
`service_sms_senders` table. This means the previous migration
never changed this service’s value for `prefix_sms` from `null`. So
this commit also changes its value to `False`, so that the rest of
the migration, of the whole column, is possible.
Removes notifications.template_id foreign key and replaces it with
a composite foreign key constraint to TemplateHistory using the
existing notification columns for template ID and template version.
Foreign key constraint is created as NOT VALID to avoid locking the
notifications table while postgres verifies that existing records
don't break the constraint. From postgres docs:
> If the constraint is marked NOT VALID, the potentially-lengthy initial
> check to verify that all rows in the table satisfy the constraint is
> skipped. The constraint will still be enforced against subsequent
> inserts or updates (that is, they'll fail unless there is a matching
> row in the referenced table, in the case of foreign keys; and they'll
> fail unless the new row matches the specified check constraints). But
> the database will not assume that the constraint holds for all rows
> in the table, until it is validated by using the VALIDATE CONSTRAINT
> option.
VALIDATE CONSTRAINT will be issued as a separate migration in a
follow-up PR.
- Modified the services_dao to return an int instead of a datetime to
make usage easier and removed the BST function on year as it is not
relevant for year
- Improved tests do there is less logic by ordering the result so there
is less reliance on the template id
- Renamed variable in stats_template_usage_by_month_dao.py to make it
consistent with the method
Currently the 'See templates used by month' report is timing out for
some services due to the query time of the data from notification_history
This is a stats table which will hold the data and will be updated by a
scheduled celery task once a day. This data will be combined with the
'live' data from notifications tables (which will be considerably less)
to form the data of the new report.
Refactor tests/db/create_service() to behave more like the real world.
Created new create_service_with_inbound_number and create_service_with_defined_sms_sender() test/db methods.
We want services to have control over this setting, rather than deriving
it from the value of their sender. This commit does that derivation one
last time, and stores it in the column, where it can be changed as and
when needed.
In future changes, services will be able to control whether their text
messages will be prefixed with the name of their service.
This commit:
- adds a column to store the value of that setting
- makes the service model take notice of it, if it were to have a value
set
It doesn’t:
- provide a way of setting the value of this column
Currently the column can have three values:
- `None` – ignore it (this is what all current services will start as)
and continue to determine whether to prefix messages by looking at the
sender
- `True` – always the service name to the start of text messages
- `False` – never add the service name to the start of text messages
In the future we’ll migrate all services to be either `True` or `False`,
the `None` will go away and all services will have direct control over
the setting.
we can back-fill data, however, between alembic running and the api
updating to the new code, it'll still try and create new users without
adding the auth_type, because that won't be referenced in the model yet.
Instead of using a python-level default, a postgres-level
server_default will make postgres set every row to sms_auth if it's not
defined in the application.
> So that we can default services to their appropriate text allowance,
> we need to find out what sector they're in. So let's start collecting
> that from teams as they create new services.
In order to work out what a team’s allowance should be, we need to know
what part of government they’re from. We’re going to do this logic in
the admin app and then `POST` the allowance to the API.
So all we need to do with the information that the users give us is
store it against the service, so we have a record. Doesn’t need any
logic doing as a result of it, doesn’t need foreign keying to the
organisations table, etc.
