Commit Graph

8150 Commits

Author SHA1 Message Date
Rebecca Law
21edf7bfdd Persist the processing time statistics to the database.
The performance platform is going away soon. The only stat that we do not have in our database is the processing time. Let me clarify the only statistic we don't have in our database that we can query efficiently is the processing time. Any queries on notification_history are too inefficient to use on a web page.
Processing time = the total number of normal/team emails and text messages plus the number of messages that have gone from created to sending within 10 seconds per whole day. We can then easily calculate the percentage of messages that were marked as sending under 10 seconds.
2021-02-26 07:49:49 +00:00
David McDonald
82e5a1804b Merge pull request #3155 from alphagov/migrate-broadcast-settings
Backfill services_broadcast_settings table
2021-02-25 12:16:36 +00:00
Rebecca Law
1b22de3830 Merge pull request #3161 from alphagov/check-use_numeric_sender
Check international rules to decide on a sender
2021-02-25 09:36:29 +00:00
Rebecca Law
97d1bfaee8 Rename method for clarity
Added unit test for new method.
2021-02-25 08:10:52 +00:00
Rebecca Law
11bd906338 Update utils to get the new function 2021-02-24 16:56:33 +00:00
Rebecca Law
5986a65005 Check international number for alpha: NO if true then use number to send
SMS.

This is not a catch all for international SMS, the rules are quite
complex and still not completely understood. We are talking with our
provider who maybe able to sort this out for us. But in the meantime,
this should solve for the case that we understand.
2021-02-24 15:12:03 +00:00
Pea Tyczynska
516a982592 Merge pull request #3159 from alphagov/add-date-to-dvla-email-personalisation
Add date to personalisation for DVLA email
2021-02-24 14:54:06 +00:00
Pea Tyczynska
f3e0cfc727 Pull DVLA address from credentials on staging
So that we can test this flow on staging.
2021-02-24 11:34:29 +00:00
Pea Tyczynska
8e3ef5ff05 Add DVLA_EMAIL_ADDRESS to manifest so it gets picked up from
credentials.
2021-02-24 10:32:20 +00:00
Pea Tyczynska
4fc3af9811 Add date to personalisation for DVLA email
Personalisation was missing date attribute. The email still got sent
tonight, just it didn't have a value for date placeholder.
2021-02-24 10:22:22 +00:00
David McDonald
3ea86bfb48 Remove hardcoded default to use test channel
There is no need for a default now as every broadcast service has set on
it which broadcast channel to use.
2021-02-23 17:15:07 +00:00
David McDonald
9b21e6b04f Use sample_broadcast_service fixture
Now that every service has a row in the service_broadcast_settings
table, we want all our tests to use the `sample_broadcast_service`
fixture as this ensures it has a row in that table and is correctly
representitive of what a real broadcast service looks like.
2021-02-23 17:15:06 +00:00
David McDonald
ce2f550387 Backfill services_broadcast_settings table
At the moment, we currently have broadcast services that have the
'broadcast' service permission in the DB but don't have a corresponding
row in the service_broadcast_settings table. It's important to give them
a row in this table because this will control which broadcast channel
their messages will go out on. All broadcast services will be expected
to have this row so we can then remove hardcoded defaults from our code
that currently set what channel a message should go out on.

Whilst, when this gets deployed, we will have released
https://github.com/alphagov/notifications-admin/pull/3794, which means
that when a service setting is changed via that form, they will get the
corresponding row in the service_broadcast_settings form, we don't want
to ask every developer to go and fill in this form for every service on
their local dev environment and also do the same to every service we
have in preview, staging and production. Therefore a migration is the
best way to backfill the data.

Note, I made the decision that a service that is in trial mode will be
given the 'severe' channel. This is because this is what most trial mode
services should have. Only the MNOs would ever really have a trial mode
service on the test channel. And given they are in trial mode, it is not
too risky to give them the 'severe' channel.

For services that are live though, although there are no services in
production that have the broadcast permission and are live, it is not
worth taking that risk and accidently setting that service to send an
alert out on the 'severe' channel. We play it say by choosing 'test'.
2021-02-23 17:15:05 +00:00
Pea Tyczynska
8e5f956009 Merge pull request #3151 from alphagov/daily-volumes-dvla
Send daily email with volumes of letters and sheets to DVLA
2021-02-23 15:24:31 +00:00
Pea Tyczynska
b05ddd4978 Don't donwgrade if the environment is production
Because we are nervous about deleting data on prod, since it can
block the db.
2021-02-23 15:13:53 +00:00
Pea Tyczynska
5c22c926b0 Stub DVLA email for all envs except prod
In prod we will get it from Credentials. In other envs, we don't
really want to send real email.
2021-02-23 15:13:52 +00:00
Pea Tyczynska
e0c73ac342 Send daily email with letter and sheet volumes to DVLA 2021-02-23 15:13:19 +00:00
Pea Tyczynska
6dab63130d Make import order alphabetical 2021-02-23 15:13:19 +00:00
Pea Tyczynska
c8ffebcce8 Query to get letter and sheet volumes
So we can send daily email with these volumes to DVLA.
2021-02-23 15:13:18 +00:00
Pea Tyczynska
88233543c3 Add email template for daily letter volumes 2021-02-23 15:13:18 +00:00
David McDonald
d7426c0c42 Merge pull request #3158 from alphagov/mnos-prod
Turn on o2 and three mnos in prod
2021-02-23 14:26:58 +00:00
David McDonald
c03ad82227 Turn on o2 and three mnos in prod
Supporting infrastructure is ready for these two mnos
2021-02-23 13:56:47 +00:00
Ben Thorner
71bf89bf0c Merge pull request #3150 from alphagov/refresh-dev-tasks
Simplify README and consolidate scripts
2021-02-22 13:20:49 +00:00
Ben Thorner
f2c6a49ab6 Update install instructions for Postgres
This apps works with v11, but not with v13. Adding '|| true' in the
Makefile means the 'bootstrap' rule can be run multiple times, even
if the DB already exists.
2021-02-22 12:41:35 +00:00
Rebecca Law
965a2ff0cb Merge pull request #3145 from alphagov/add-caching-send-to-provider
Add caching and remove extra call to database
2021-02-22 08:53:20 +00:00
Leo Hemsted
079e1cd034 Merge pull request #3156 from alphagov/cronitor
make sure status is always set by finally block
2021-02-19 10:04:55 +00:00
Leo Hemsted
6a859982d0 make sure status is always set by finally block
confusingly, some errs are not subclasses of Exception (things like sys.exit()). Lets make sure the status is always set so cronitor knows what to ping.
2021-02-18 18:27:41 +00:00
Leo Hemsted
b2cf8cc5de Merge pull request #3152 from alphagov/retry-bug-fix
Retry bug fix
2021-02-18 13:42:11 +00:00
Rebecca Law
b464894325 update to check for instance of SerialisedService 2021-02-18 12:54:22 +00:00
Leo Hemsted
0088bcd98b only retry if the broadcast message task is in sending
previously we would retry if the task was queued up for retry but the
status is in "received-ack" or "received-err". We don't expect that a
task will be retried after getting this status, but if there are
duplicate tasks that could happen. Lets plan for the worst by saying
"only process a retry if the task is currently in sending".

this way, if a duplicate task is on retry and the first task goes
through succesfully, the duplicate task will give up.
2021-02-18 12:03:36 +00:00
Leo Hemsted
90e82aff3e properly log the lambda response correctly
boto returns a `StreamingBody`[1] response rather than a json struct.
We're currently just logging things like "Error calling lambda
o2-1-proxy with function error <botocore.response.StreamingBody object
at 0x7f74cd6e02e8>" which is obviously less than ideal. Also make the
tests properly reflect this - annoyingly it appears like we can't use
moto to reliably test this interface as the moto `mock_lambda` decorator
needs you to be running inside a docker container??

[1] https://botocore.amazonaws.com/v1/documentation/api/latest/reference/response.html#botocore.response.StreamingBody
2021-02-18 11:51:38 +00:00
Ben Thorner
126d810d5a Merge pull request #3147 from alphagov/remove-redundant-task
Remove redundant (renamed) letters task
2021-02-18 10:49:46 +00:00
Katie Smith
ac11de89d3 Merge pull request #3144 from alphagov/🚮-an-email
Stop emailing Notify when an MOU is signed
2021-02-18 09:20:36 +00:00
Ben Thorner
5ff7c2013e Swap FLASK_DEBUG for FLASK_ENV
This achieves the same thing and gets rid of the warning about being
in a production environment when the app starts up.
2021-02-18 09:11:16 +00:00
Katie Smith
c77e73b26f Fix failing test
This was passing locally, but failing on Concourse due to a different
order of TemplateHistory items being returned. This changes the
test so that it can't randomly fail based on the order of template
history items returned.
2021-02-18 09:07:21 +00:00
Katie Smith
c59e0091ee Stop emailing Notify when an MOU is signed
We've decided we don't get any value from these emails any more, so this
stops us (Notify support) receiving them. We still let teams know an MOU
has been signed.
2021-02-18 09:07:19 +00:00
Ben Thorner
af95ad68ea Move bootstrap tasks into the Makefile
This is more consistent with how we run all other tasks. Note that
the virtual env setup is not generally applicable, and developers
of this repo should follow the guidance in the README.
2021-02-18 09:01:32 +00:00
Ben Thorner
dc6fb1d1f2 Remove unused single test script 2021-02-18 09:01:31 +00:00
Ben Thorner
5ad821428b Make instructions to run app/tests more concise 2021-02-18 09:01:30 +00:00
Ben Thorner
273fd339ac Clarify setup instructions for environment.sh 2021-02-18 09:01:29 +00:00
Ben Thorner
de6ed47239 Revise section about AWS setup
This is covered in Wiki in more detail.
2021-02-18 09:01:28 +00:00
Ben Thorner
89dc7218fe Remove section about virtualenv
This is covered generically in the Wiki [1].

[1]: https://github.com/alphagov/notifications-manuals/wiki/Getting-Started#development-environment
2021-02-18 09:01:27 +00:00
Ben Thorner
ba4d399982 Switch to 'make' for running app processes
These are simple enough that they don't need their own scripts.
2021-02-18 09:01:26 +00:00
Ben Thorner
4c0aea50ae Remove redundant Docker tasks
Nothing and no one uses these.
2021-02-18 09:01:25 +00:00
Rebecca Law
933bad857a Merge pull request #3146 from alphagov/use-number-for-international-text
Send text messages from a number for international
2021-02-17 13:31:42 +00:00
Rebecca Law
77b76ea0a4 Rename variable, it's a better name now. 2021-02-17 13:15:29 +00:00
Ben Thorner
474b93f183 Remove redundant (renamed) letters task
This was renamed in [1], and enough time has elapsed that instances
of the task should all have finished processing.

[1]: 5d6f2da155
2021-02-17 12:57:50 +00:00
Rebecca Law
88036befcf Merge branch 'master' into add-caching-send-to-provider 2021-02-17 12:40:18 +00:00
Rebecca Law
e77534fb17 Send text message that are to an international number from a number rather than "Notify"
Update `send_user_2fa_code` to send from number when recipient is international
Update `update_user_attribute` to send from number when recipient is international
2021-02-17 12:14:47 +00:00
Chris Hill-Scott
8e8601338e Merge pull request #3136 from alphagov/validate-template-length-broadcast-api
Validate content length on broadcast API
2021-02-17 11:34:29 +00:00