Commit Graph

37 Commits

Author SHA1 Message Date
Chris Hill-Scott
cc207ac11f Raise error if multiple broadcasts found for reference
Because the `<reference>` field of a `cancel` message can contain an
arbitrary number of items it’s possible for it to reference more than
one current alert.

In this case it is ambiguous which alert should be cancelled, so we
should raise a custom error.

This will help people know that they have to manually go into Notify and
figure out which alert(s) to cancel there.
2022-02-17 15:23:13 +00:00
Chris Hill-Scott
d73131bbec Allow cancel of alert via API with no description
The XML for an alerts requires a `<description>` field. The XML for
a `<cancel>` may have a `<description>` field populated (although we
ignore the contents) but it may also be empty.

This commit updates the schema to leave the all the validation to the
view layer, which can decide when or when not to validate the content of
the `<description>` field.
2022-02-16 15:31:50 +00:00
Chris Hill-Scott
bbc444699a Return 400 if references missing from cancel broadcast
If someone tries to cancel a broadcast but the references don’t match
and existing broadcast we correctly return a 404.

If they don’t provide any references then we get an exception. This
commit catches the missing references and returns a 400. I think this
is more appropriate because it’s malformed request, rather than a
well-formed request that doesn’t match our data. It also lets us write a
more specific and helpful error message.
2022-02-14 12:34:09 +00:00
Pea Tyczynska
3dc1907321 Audit api key id when cancelling broadcast via api 2022-02-11 12:01:56 +00:00
Pea Tyczynska
a780933893 Revert "Audit api key id when cancelling broadcast via api" 2022-02-09 11:01:39 +00:00
Pea Tyczynska
82f08f230c Save api key id when cancelling broadcast by API call
This is so that we can audit who cancelled the broadcast if
there are any issues.
2022-01-26 17:26:58 +00:00
Pea Tyczynska
52dbdb7518 Move validate_and_update_broadcast_message_status to a utils file
This is because that function is used both when broadcast status
is updated via API and via admin, so it's a shared resource.

Also move and update tests for updating broadcast message status
so things are tested at source and repetition is avoided.
2022-01-20 18:14:41 +00:00
Pea Tyczynska
c9afb2f038 Remove unnecessary error handling
The context here should be enough for the users, custom error
message is not needed.
2022-01-20 18:14:40 +00:00
Pea Tyczynska
c2a389e81a Move updating user validation out of validate_and_update_broadcast_message_status
As only 1 of 2 functions calling it needs that check, it's better
to perform it inside that 1 function.
2022-01-20 18:14:39 +00:00
Pea Tyczynska
a4c20e8ba6 Return 404 if reference from cancel message does not match
If the reference from cancel CAP XML we received via API does not
match with any existing broadcast, return 404.

Do the same if service id doesn't match.

Also refactor code to cancel broadcast out into separate function

It should be a separate function that is only called by create_broadcast
function. This will prevent create_broadcast from becoming too
big and complex and doing too many things.
2022-01-19 15:42:27 +00:00
Pea Tyczynska
3b4a9d8942 Cancel broadcast via API
When a service sends us an XML CAP broadcast message with Cancel
status, and that broadcast is in broadcasting state, we cancel it.
2022-01-19 15:42:26 +00:00
Pea Tyczynska
940126abfb Reject unapproved broadcast upon cancel API request
When a service sends us a cancel broadcast XML via API, if that
broadcast was not approved yet, reject it.
2022-01-19 15:41:38 +00:00
Chris Hill-Scott
c0742fe83d Pass polygons through if they’re small already
If a polygon is smaller than the largest polygon in our dataset of
simplified polygons then we’re only throwing away useful detail by
simplifying it.

We should still simplify larger polygons as a fallback, to avoid sending
anything to the CBC that we’re not sure it will like.

The thresholds here are low: we can raise them as we test and experiment
more.

Here’s some data about the Flood Warning Service polygons

Percentile | 80% | 90%   | 95%    | 98%     | 99%     | 99.9%
-----------|-----|-------|--------|---------|---------|---------
Point count| 226 | 401.9 | 640.45 | 1015.38 | 1389.07 | 3008.609

Percentile    | 80% | 90%   | 95%    | 98%     | 99%     | 99.9%
--------------|-----|-------|--------|---------|---------|---------
Polygon count |2----|3------|5-------|8--------|10-------|40.469
2021-11-18 15:48:45 +00:00
Chris Hill-Scott
4feb3fdc10 Bump utils
This new version of utils implements the transformation of our polygons
to a Cartesian plane. In other words, it converts them from being
defined in spherical degrees to metres.

For the API this means our simplification will be slightly more
accurate.
2021-11-18 15:43:39 +00:00
Chris Hill-Scott
54bcf618da Store the event field from CAP XML broadcasts
We don’t store everything that comes in the CAP XML when someone creates
a broadcast via the API.

One thing we do store is `<identifier>` (in a column called `reference`)
which is a unique (to the external system) identifier for the broadcast.
We show this in the front end instead of the template name, because
broadcasts created from the API don’t use templates.

However this ID isn’t very friendly – the Environment Agency just supply
a UUID.

The Environment Agency also populate the `<event>` field with some human
readable text, for example:
> 013 Issue Severe Flood Warning EA

(013 is an area code which will be meaningful to the Flood Warning
Service team)

We should show this in the UI instead of the reference. The first step
towards this is storing it in the database and returning it in the REST
endpoints.

Later we can have the admin app prefer `cap_event` over `reference`,
where `cap_event` is present.

We can’t backfill this data because we don’t keep a copy of the original
XML.

Seems like `<event>` is a mandatory property of `<info>`, so we don’t
need to worry about the field being missing (`<info>` is optional in
CAP but we require it because it contains stuff like the areas which
we need in order to send out the broadcast`).

***

https://www.pivotaltracker.com/story/show/176927060
2021-10-26 11:12:27 +01:00
Ben Thorner
d8a0967ec0 Add validation to prevent blank area names
Now that these are used for display on gov.uk/alerts we need to
make sure the data is being set properly. We've already found an
example where it wasn't [1]. We validate external broadcasts in
two stages: with the official CAP XML schema [2] and then again
with our own, more specific schema for the converted JSON. Since
this validation is a custom requirement I've made it part of the
JSON schema. Note that jsonschema recommends avoiding metachars
like "\w" since they're not supported by all implementations [3].

I've tested the new validation manually and it works as expected
by disallowing e.g. "  " but still alowing "foo" and "foo bar".

[1]: https://www.notifications.service.gov.uk/services/120107d0-d99a-4c42-8b70-f37d2f28879b/rejected-alerts/d6e0c70e-60f6-4422-8589-2a2d159c63f2
[2]: 81a25ff1ef/app/xml_schemas/CAP-v1.2.xsd
[3]: http://json-schema.org/understanding-json-schema/reference/regular_expressions.html
2021-09-17 13:33:52 +01:00
Ben Thorner
35430e9a9f Refactor custom validation into own function
This sets a pattern for adding another in the next commits.
2021-09-15 11:02:50 +01:00
Ben Thorner
ec1171f85c Switch "areas" field to "areas_2" format
The Admin app is now temporarily using the "areas_2" field while
we migrate "areas" to the new format [1].

[1]: https://github.com/alphagov/notifications-admin/pull/4004
2021-08-27 14:22:11 +01:00
Ben Thorner
023a06d5fb Start dual running with "areas" and "names"
For the public API we actually receive a "name" instead of an ID,
which we also want to start sending from the Admin app.

Unlike IDs, which aren't really used anywhere, we want the names
to display the alerts on gov.uk/alerts.
2021-08-26 15:34:25 +01:00
Chris Hill-Scott
f3e6d92046 Revert "Update utils to bring in coordinate transformation" 2021-08-20 16:05:39 +01:00
Chris Hill-Scott
e92be8b034 Pass polygons through if they’re small already
If a polygon is smaller than the largest polygon in our dataset of
simplified polygons then we’re only throwing away useful detail by
simplifying it.

We should still simplify larger polygons as a fallback, to avoid sending
anything to the CBC that we’re not sure it will like.

The thresholds here are low: we can raise them as we test and experiment
more.

Here’s some data about the Flood Warning Service polygons

Percentile | 80% | 90%   | 95%    | 98%     | 99%     | 99.9%
-----------|-----|-------|--------|---------|---------|---------
Point count| 226 | 401.9 | 640.45 | 1015.38 | 1389.07 | 3008.609

Percentile    | 80% | 90%   | 95%    | 98%     | 99%     | 99.9%
--------------|-----|-------|--------|---------|---------|---------
Polygon count |2----|3------|5-------|8--------|10-------|40.469
2021-08-19 11:08:53 +01:00
Chris Hill-Scott
a6135fb8ab Bump utils
This new version of utils implements the transformation of our polygons
to a Cartesian plane. In other words, it converts them from being
defined in spherical degrees to metres.

For the API this means our simplification will be slightly more
accurate.
2021-08-19 11:08:18 +01:00
Ben Thorner
a91fde2fda Run auto-correct on app/ and tests/ 2021-03-12 11:45:45 +00:00
Chris Hill-Scott
8e8601338e Merge pull request #3136 from alphagov/validate-template-length-broadcast-api
Validate content length on broadcast API
2021-02-17 11:34:29 +00:00
Chris Hill-Scott
0bb671df45 Validate content length on broadcast API
The maximum content count of a broadcast varies depending on its
encoding, so we can’t simply validate it against a schema. This commit
moves to using the validation from `notifications-utils`, and raising a
custom error response.
2021-02-16 09:30:40 +00:00
Katie Smith
6b8ebb3421 Fix linting errors 2021-02-16 09:03:38 +00:00
Chris Hill-Scott
e8a79f5413 Don’t accept cancel or update via broadcast API
We don’t support these methods at the moment. Instead we were just
ignoring the `msgType` field, so issuing one of these commands would
cause a new alert to be broadcast 🙃

We might want to support `Cancel` in the future, but for now let’s
reject anything that isn’t `Alert` (CAP terminology for the initial
broadcast).
2021-02-15 09:32:33 +00:00
Pea Tyczynska
3037bf5fff Set broadcast message to stubbed when posting broadcast via API 2021-02-09 10:41:36 +00:00
Chris Hill-Scott
dec16a98f6 Handle XML files that have a declaration
`lxml` wants its input in bytes:

> XML is explicitly defined as a stream of bytes. It's not Unicode text.
> […] rule number one: do not decode your XML data yourself.

– https://lxml.de/FAQ.html#why-can-t-lxml-parse-my-xml-from-unicode-strings

It will accept strings unless, unless the document contains a
declaration[1] with an `encoding` attribute. Then it will refuse to
parse the document and raises a `ValueError`[2].

We can get fix this by passing `lxml` the bytes from the request, rather
than the decoded text.

1. > XML documents may begin with an XML declaration that describes some
   > information about themselves. An example is
   > `<?xml version="1.0" encoding="UTF-8"?>`.
   – https://en.wikipedia.org/wiki/XML#XML_declaration
2. See an example of this exception being raised in production here:
   https://kibana.logit.io/s/9423a789-282c-4113-908d-0be3b1bc9d1d/app/kibana#/doc/logstash-*/logstash-2021.02.05/syslog?id=AXdzfZVz5ZSa5DKpJiYd&_g=()
2021-02-08 08:51:14 +00:00
Chris Hill-Scott
ca6c46c4dd Add logging for succesful broadcast message creation 2021-01-26 16:24:45 +00:00
Chris Hill-Scott
0398ac57f1 Use correct HTTP status code for bad content type
415 is the status code for ‘Unsupported media type’

https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/415
2021-01-26 16:24:45 +00:00
Chris Hill-Scott
b85fcafd46 Don’t allow broadcasts to be created from JSON
Until we know we’re going to have real users for this, let’s not expose
it.
2021-01-26 16:24:45 +00:00
Chris Hill-Scott
c9d55039eb Simplify polygons before storing them
We’re going to let people pass in fairly complex polygons, but:
- we don’t want to store massive polygons
- we don’t want to pass the CBCs massive polygons

So this commit adds a step to simplify the polygons before storing them.

We think it’s best for us to do this because:
- writing code to do polygon simplification is non-trivial, and we don’t
  want to make all potential integrators do it
- the simplification we’ve developed is domain-specific to emergency
  alerting, so should throw away less information than

There’s a bit more detail about how we simplify polygons in
https://github.com/alphagov/notifications-admin/pull/3590/files
2021-01-26 16:24:45 +00:00
Chris Hill-Scott
26871eeacc Validate CAP against the spec
This gives us some extra confidence that there aren’t any problems with
the data we’re getting from the other service. It doesn’t address any
specific problems we’ve seen, rather it seems like a sensible precaution
to take.
2021-01-26 16:24:45 +00:00
Chris Hill-Scott
38f07db23e Accept CAP XML
This commit makes the existing endpoint also accept CAP XML, should the
appropriate `Content-Type` header be set.

It uses the translation code we added in a previous commit to convert
the CAP to a dict. We can then validate that dict against with the JSON
schema to ensure it’s something we can work with.
2021-01-26 16:24:44 +00:00
Chris Hill-Scott
7530408a21 Validate broadcast against schema
This commit adds a JSONSchema which can validate the fields in an API
call to create a broadcast. It takes the CAP XML schema as a starting
point.
2021-01-26 16:24:44 +00:00
Chris Hill-Scott
61c9e50ed9 Add public API endpoint to create emergency alerts
We know there is at least one system which wants to integrate with
Notify to send out emergency alerts, rather than creating them manually.

This commit adds an endpoint to the public API to let them do that.

To start with we’ll just let the system create them in a single call,
meaning they still have to be approved manually. This reduces the risk
of an attacker being able to broadcast an alert via the API, should the
other system be compromised.

We’ve worked with the owners of the other system to define which fields
we should care about initially.
2021-01-26 16:24:44 +00:00