Commit Graph

7863 Commits

Author SHA1 Message Date
David McDonald
a3d966056a Merge pull request #3110 from alphagov/test-channel
Set the default broadcast channel to test
2021-02-01 10:18:35 +00:00
Katie Smith
a31cdb44a6 Merge pull request #3113 from alphagov/bump-utils-43.8.1
Bump utils to 43.8.1
2021-02-01 09:06:39 +00:00
Pea Tyczynska
df4ba22912 Merge pull request #3114 from alphagov/preview-wont-stub
Don't stub broadcasts on preview
2021-01-29 16:00:11 +00:00
Katie Smith
fc9ecaba1d Bump utils to 43.8.1
This brings in the change to stop TV numbers from being treated as
international numbers.
2021-01-29 15:53:35 +00:00
Pea Tyczynska
552e543bc2 Don't stub broadcasts on preview
So that MNOs can use training mode accounts to test end-to-end
broadcast sending. This will enable them to approve their own
broadcasts.
2021-01-29 15:49:50 +00:00
David McDonald
86ea89cf76 Merge pull request #3098 from alphagov/downgrade-to-warning
Downgrade SMS provider request exceptions to warnings
2021-01-29 11:52:10 +00:00
Katie Smith
7b60aeb14d Merge pull request #3109 from alphagov/letter-rates
Add February 2021 letter rates
2021-01-29 10:28:16 +00:00
Katie Smith
d7387869c4 Add February 2021 letter rates
All rates are changing, so we add an end date for the current rates and
insert new rates for every post_class, sheet count and crown status.
2021-01-28 14:35:21 +00:00
Chris Hill-Scott
837e464081 Merge pull request #3060 from alphagov/add-broadcast-api-endpoint
Add public API endpoint to create broadcast messages
2021-01-28 12:59:41 +00:00
Pea Tyczynska
51c0ece130 Merge pull request #3108 from alphagov/stub-training-broadcasts
Stub training broadcasts
2021-01-28 11:58:47 +00:00
Katie Smith
4ed79dae48 Merge pull request #3105 from alphagov/rename-bt-ee
Rename bt-ee-proxy to ee-proxy
2021-01-27 17:05:26 +00:00
David McDonald
f9b1d3d573 Set the default broadcast channel to test
This used to be hardcoded in the CBC proxy but now we will hardcode it
in the cbc_proxy_client.

In a future PR we can start choosing which channel a broadcast will go
to based on the channel configured for that broadcast service.
2021-01-27 15:27:11 +00:00
Pea Tyczynska
d4cc250510 Don't create broadcast provider messages for stubbed broadcasts 2021-01-27 10:20:44 +00:00
Pea Tyczynska
26d6b4a958 Mark broadcast message as stubbed when sent from training account 2021-01-27 10:20:43 +00:00
Pea Tyczynska
a93a35de8d Add 'stubbed' column to broadcast_message table
This is a boolean column. It will be set to True for broadcasts
created from training broadcast accounts.

This will help us debug, for example by excluding all the stubbed
broadcasts when we have some trouble with real broadcasts.
2021-01-27 10:20:43 +00:00
Chris Hill-Scott
ca6c46c4dd Add logging for succesful broadcast message creation 2021-01-26 16:24:45 +00:00
Chris Hill-Scott
0398ac57f1 Use correct HTTP status code for bad content type
415 is the status code for ‘Unsupported media type’

https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/415
2021-01-26 16:24:45 +00:00
Chris Hill-Scott
b85fcafd46 Don’t allow broadcasts to be created from JSON
Until we know we’re going to have real users for this, let’s not expose
it.
2021-01-26 16:24:45 +00:00
Chris Hill-Scott
c9d55039eb Simplify polygons before storing them
We’re going to let people pass in fairly complex polygons, but:
- we don’t want to store massive polygons
- we don’t want to pass the CBCs massive polygons

So this commit adds a step to simplify the polygons before storing them.

We think it’s best for us to do this because:
- writing code to do polygon simplification is non-trivial, and we don’t
  want to make all potential integrators do it
- the simplification we’ve developed is domain-specific to emergency
  alerting, so should throw away less information than

There’s a bit more detail about how we simplify polygons in
https://github.com/alphagov/notifications-admin/pull/3590/files
2021-01-26 16:24:45 +00:00
Chris Hill-Scott
26871eeacc Validate CAP against the spec
This gives us some extra confidence that there aren’t any problems with
the data we’re getting from the other service. It doesn’t address any
specific problems we’ve seen, rather it seems like a sensible precaution
to take.
2021-01-26 16:24:45 +00:00
Chris Hill-Scott
38f07db23e Accept CAP XML
This commit makes the existing endpoint also accept CAP XML, should the
appropriate `Content-Type` header be set.

It uses the translation code we added in a previous commit to convert
the CAP to a dict. We can then validate that dict against with the JSON
schema to ensure it’s something we can work with.
2021-01-26 16:24:44 +00:00
Chris Hill-Scott
f98aca05e9 Add translators from CAP to dict
Other systems we’re working with won’t easily be adapted to emit JSON
instead of CAP, so it’s less work for us to do that conversion.

This commit adds to code to parse the XML and turn it into a dict that
we can work with, including converting the polygon string into native
Python lists.
2021-01-26 16:24:44 +00:00
Chris Hill-Scott
7530408a21 Validate broadcast against schema
This commit adds a JSONSchema which can validate the fields in an API
call to create a broadcast. It takes the CAP XML schema as a starting
point.
2021-01-26 16:24:44 +00:00
Chris Hill-Scott
61c9e50ed9 Add public API endpoint to create emergency alerts
We know there is at least one system which wants to integrate with
Notify to send out emergency alerts, rather than creating them manually.

This commit adds an endpoint to the public API to let them do that.

To start with we’ll just let the system create them in a single call,
meaning they still have to be approved manually. This reduces the risk
of an attacker being able to broadcast an alert via the API, should the
other system be compromised.

We’ve worked with the owners of the other system to define which fields
we should care about initially.
2021-01-26 16:24:44 +00:00
Pea Tyczynska
dfbd31cef8 Merge pull request #3106 from alphagov/billing-fields-for-service
Add billing details fields to Service model and db table
2021-01-26 15:14:05 +00:00
Katie Smith
2681752f15 Rename bt-ee-proxy to ee-proxy
We want to rename the `bt-ee-1-proxy` lambda function to `ee-1-proxy`.
This change will need to be deployed at the same time that we change
the name of the lambda function in the Terraform.
2021-01-26 14:36:20 +00:00
Richard Baker
50369b4aa9 Merge pull request #3107 from alphagov/o2-proxy
Add proxy client for o2 cell broadcasting
2021-01-26 11:36:19 +00:00
Richard Baker
6256cdf792 Add proxy client for o2 cell croadcasting
o2 use One-2-many CBC so we can use the O2M/CAP client.

Once differences between CBCs have been worked out we can consolidate O2M clients to reduce duplication.

Signed-off-by: Richard Baker <richard.baker@digital.cabinet-office.gov.uk>
2021-01-26 11:11:44 +00:00
Pea Tyczynska
b3abdfb401 Rename billing contact email and name fields to plural
So:

'billing_contact_email_address' becomes 'billing_contact_email_addresses'
AND
'billing_contact_name' becomes 'billing_contact_names'

This is to signify that each of those fields can contain numerous
items
2021-01-25 17:53:27 +00:00
Pea Tyczynska
ffac16a2a0 Add new billing details to test_get_service_by_id 2021-01-25 17:42:18 +00:00
David McDonald
770f81a1c7 Merge pull request #3104 from alphagov/remove-admin-hack
Remove ability for platform admins to approve own broadcast
2021-01-25 13:54:43 +00:00
David McDonald
f07471b42a Update app/broadcast_message/rest.py
Co-authored-by: Chris Hill-Scott <me@quis.cc>
2021-01-25 11:09:49 +00:00
David McDonald
c61bd9976f Remove ability for platform admins to approve own broadcast
This has been added in for speed of development but now we are getting
close to integrating with production systems, we will be turning off
these helpful hacks to reduce the risk of someone sending a real
broadcast to citizens.

Note, platforms are still able to approve broadcasts when their service
is in training mode.
2021-01-22 16:56:05 +00:00
Pea Tyczynska
e703d1a172 Add billing details fields to Service model and db table
The fields are:
Purchase order number - string field
Billing contact name - text field to acommodate possible multiple
contacts
Billing contact email address - text field to acommodate possible
multiple contacts
Billing reference - string field

All these fields are nullable. Notify platform admins will be
able to check and edit those values in Service Settings
section in Notify interface.

This will help make billing tasks and reports simpler.

Similar fields will also be added to Organisation model and
db table.
2021-01-20 18:00:43 +00:00
David McDonald
024ce30cc8 Merge pull request #3100 from alphagov/exclude-order
Tidy up the excludes for schemas
2021-01-20 12:42:39 +00:00
Chris Hill-Scott
5c8b5e0488 Merge pull request #3095 from alphagov/allow-admin-to-create-broadcasts-without-templates
Let the admin app create broadcasts without templates
2021-01-19 16:57:33 +00:00
Chris Hill-Scott
94aea8a820 Add test for when no content or template provided
This is the missing invalid permutation of fields for creating a
broadcast.
2021-01-19 15:28:08 +00:00
Pea Tyczynska
882da84182 Merge pull request #3096 from alphagov/add-notes-to-service
Add notes column to services table
2021-01-19 14:45:42 +00:00
David McDonald
c20fb8abce Remove duplicate keys 2021-01-19 14:01:51 +00:00
David McDonald
c1a77eefa1 Sort exclude list
This will make it easier to review and compare changes and also will
help identify duplicates.
2021-01-19 14:00:03 +00:00
Chris Hill-Scott
a496ef0a97 Merge pull request #2986 from alphagov/cache-on-tasks
Use cache to improve performance of CSV processing
2021-01-19 11:15:18 +00:00
David McDonald
9bd7607016 Merge pull request #3097 from alphagov/proxy-client-fix
Fix incorrect log line
2021-01-19 10:20:18 +00:00
David McDonald
ac6837cde5 Downgrade exception to warning for provider API call
When we send an HTTP request to our SMS providers, there is a
chance we get a 5xx status code back from them. Currently we log this as
two different exception level logs.

If a provider has a funny few minutes, we could end up with
hundreds of exceptions thrown and pagerduty waking someone up in the
middle of the night. These problems tend to pretty quickly fix
themselves as we balance traffic from one SMS to the other SMS provider
within 5 minutes.

By downgrading both exceptions to warning in the case of a
`SmsClientResponseException`, we will reduce the change of waking us up
in the middle of the night for no reason.

If the error is not a `SmsClientResponseException`, then we will still
log at the exception level as before as this is more unexpected and we
may want to be alerted sooner.

What we still want to happen though is that let's say both SMS providers
went down at the same time for 1 hour. We don't want our tasks to just
sit there, retrying every 5 minutes for the whole time without us being
aware (so we can at least raise a statuspage update). Luckily we will
still be alerted because our smoke tests will fail after 10 minutes and
raise a p1:
https://github.com/alphagov/notifications-functional-tests/blob/master/tests/functional/staging_and_prod/notify_api/test_notify_api_sms.py#L21
2021-01-18 17:00:21 +00:00
David McDonald
1cfd19fb6a Reorder tests
So all tests related to sms sending is done first and then all email
sending task tests happen after
2021-01-18 16:46:12 +00:00
Pea Tyczynska
22f2eb7bfe Add notes column to services table 2021-01-18 10:36:51 +00:00
Chris Hill-Scott
4eb4ea1772 Use cache for tasks that save notifications
These tasks need to repeatedly get the same template and service from
the database. We should be able to improve their performance by getting
the template and service from the cache instead, like we do in the REST
endpoint code.
2021-01-18 10:25:24 +00:00
David McDonald
b9ec70acc2 Fix incorrect log line
Should have been `lambda_name` not `self.lambda_name`.
2021-01-15 16:48:04 +00:00
David McDonald
9c01d8018d Merge pull request #3093 from alphagov/broadcast-tasks-onto-worker
Broadcast tasks onto worker
2021-01-15 15:51:35 +00:00
Chris Hill-Scott
e161f6e4a1 Require reference if template not provided
In the admin app we need something to use in show in lieu of template
name when a template isn’t used. Let’s store this in the reference field
for now.
2021-01-15 14:57:36 +00:00
Chris Hill-Scott
0510311d63 Don’t require template when content is provided
So that the admin app can create broadcasts without a template it needs
to be allowed to create broadcasts from content instead.
2021-01-15 14:57:36 +00:00