it's not acceptable for a constantly failing provider to take 50 minutes
to drain (5x reducing priority by 10). But similarly, we need _some_
delay, or a handful of concurrent failures will completely turn off a
provider, rendering the whole excercise kinda pointless. Setting the
delay before it tries to reduce priority again to one minute is nice
because it means that if one request times out and returns 502, then any
other requests that are in flight at that time will time out before the
one minute is up and not switch, but any requests made after the switch
that take sixty seconds to time out will affect it.
retrive the sms providers from the DB, and decrease the chosen
provider's priority by 10, while increasing the other by 10.
add a check in to ensure we never decrease below 0 or increase above 100
- this is per provider, we don't check that the two add up to 100 or
anything. If the values are outside of this range (eg: set via the UI)
then they'll probably* fix themselves at some point - we've added tests
to document these cases.
Use with_for_update to ensure that the method can only run once at a
time - other invocations of the function will be held on that line until
the currently running one ends and commits the transaction. This doesn't
affect anyone doing things from the UI.
Utils 33.0.0 adds alt text to email branding - the HTMLEmailTemplate now
initializes slightly differently as a result (with both `branding_name`
and `branding_text`).
- Remove some redundant code for research mode.
- The international parameter in update_notification_to_sending is not needed.
- Update unit tests and removed duplicates
If we try to send an SMS to the provider and the provider throws an exception
(because they return a 503 status code) the notification should retry. But if
we get the callback from the provider before the notification has been retried, the
notification will have no billable units or provider set.
To avoid this, we now set billable_units and provider even if there has been
an exception from our provider.
‘GOV.UK’ doesn’t make sense as a type of brand. It only made sense as
a type of branding that a service had.
Since we’ve:
- deprecated the service branding column
- made sure it’s not used as a value in the email branding table
we can remove this value from the table of possible brand types.
https://www.pivotaltracker.com/story/show/159986276
We are now setting the type of branding on the branding itself, not on
the service.
This commit switches over from looking in the old place (on the service)
to looking in the new place (on the branding).
The JobStatistics table is going to be deleted. There are currently
3 tasks which use the JobStatistics model via the Statistics DAO, so we
need to make sure that these tasks aren't being used before they are
deleted in a separate PR.
This commit deletes:
* The `create_initial_notification_statistic_tasks` function which gets
used to call the `record_initial_job_statistics` task.
* The `create_outcome_notification_statistic_tasks` function which gets
used to call the `record_outcome_job_statistics` task.
* And the scheduling of the `timeout-job-statistics` scheduled task.
notable things that have been kept until migration is complete:
* passing in `organisation` to update_service will update email branding
* both `/email-branding` and `/organisation` hit the same code
* service endpoints still return organisation as well as email branding
This PR is a proposal to reduce the average messages we see for a single notification from about 7 messages to 2.
Messaging would change to something like this:
February 2nd 2018, 15:39:05.885 Full delivery response from Firetext for notification: 8eda51d5-cd82-4569-bfc9-d5570cdf2126
{'status': ['0'], 'reference': ['8eda51d5-cd82-4569-bfc9-d5570cdf2126'], 'time': ['2018-02-02 15:39:01'], 'code': ['000']}
February 2nd 2018, 15:39:05.885 Firetext callback return status of 0 for reference: 8eda51d5-cd82-4569-bfc9-d5570cdf2126
February 2nd 2018, 15:38:57.727 SMS 8eda51d5-cd82-4569-bfc9-d5570cdf2126 sent to provider firetext at 2018-02-02 15:38:56.716814
February 2nd 2018, 15:38:56.727 Starting sending SMS 8eda51d5-cd82-4569-bfc9-d5570cdf2126 to provider at 2018-02-02 15:38:56.408181
February 2nd 2018, 15:38:56.727 Firetext request for 8eda51d5-cd82-4569-bfc9-d5570cdf2126 finished in 0.30376038211397827
February 2nd 2018, 15:38:49.449 sms 8eda51d5-cd82-4569-bfc9-d5570cdf2126 created at 2018-02-02 15:38:48.439113
February 2nd 2018, 15:38:49.449 sms 8eda51d5-cd82-4569-bfc9-d5570cdf2126 sent to the priority-tasks queue for delivery
To somthing like this:
February 2nd 2018, 15:39:05.885 Firetext callback return status of 0 for reference: 8eda51d5-cd82-4569-bfc9-d5570cdf2126
February 2nd 2018, 15:38:49.449 sms 8eda51d5-cd82-4569-bfc9-d5570cdf2126 created at 2018-02-02 15:38:48.439113
* unused variables
* variables in loops overshadowing imports
* excepts with no defined exc type (tried to avoid `except Exception` too)
* history mapper is still too complex
* default variables should never be mutable
this involved:
* moving that task to callback_tasks to prevent circular imports
* updating the dummy research mode callbacks (with actual SNS messages from the
ses simulator emails)
* refactoring tests
In future changes, services will be able to control whether their text
messages will be prefixed with the name of their service.
This commit:
- adds a column to store the value of that setting
- makes the service model take notice of it, if it were to have a value
set
It doesn’t:
- provide a way of setting the value of this column
Currently the column can have three values:
- `None` – ignore it (this is what all current services will start as)
and continue to determine whether to prefix messages by looking at the
sender
- `True` – always the service name to the start of text messages
- `False` – never add the service name to the start of text messages
In the future we’ll migrate all services to be either `True` or `False`,
the `None` will go away and all services will have direct control over
the setting.
`service.sms_sender` has been deprecated; we should be looking at which
of the service’s SMS senders is default to work out if the message
has been sent from GOVUK or not (and if it has, then prefix the message
with the service name).
The arguments to `SMSMessageTemplate` are _super_ badly named – `sender`
isn’t really used as a string, it’s a boolean that effectively means
‘is this a custom sender (`True`) or the platform default (`False`)’. We
should rename it once this bug is fixed.
we allow some invalid to addresses - for example, phone numbers with
spaces or brackets - in the database. This is so that users can match
up their data in a format that they expect (since they passed it in).
When we send SMS, we strip this formatting just before sending - but we
weren't with email. This commit changes that and adds some tests.
It also adds formatting for reply_to addresses. We should never expect
invalid reply_to email addresses in our data, but just in case, lets
validate them here.
Also, bump requirements.txt to capture some more email validation
Code was not expecting logo to be `None`, thereby causing the task to
throw an exception, and retry until eventually putting the email in
technical error (for services with org branding but no logo).
Added an extra name, 'org_banner', for branding types into branding_type table
Added org banner into user model in database
Added checks for new branding type to ensure that the correct data is passed into the dict
Tested new checks in html email options
If a user was to have an organisation selected in the email settings within the platform admin section, they would be sending emails that contained both the organisation's branding and GOV.UK's.
Fix adds a check to ensure that the branding dictionary does not contain organisation details if the the service branding settings is set to 'gov'
