Commit Graph

26 Commits

Author SHA1 Message Date
Pea Tyczynska
a4c20e8ba6 Return 404 if reference from cancel message does not match
If the reference from cancel CAP XML we received via API does not
match with any existing broadcast, return 404.

Do the same if service id doesn't match.

Also refactor code to cancel broadcast out into separate function

It should be a separate function that is only called by create_broadcast
function. This will prevent create_broadcast from becoming too
big and complex and doing too many things.
2022-01-19 15:42:27 +00:00
Pea Tyczynska
3b4a9d8942 Cancel broadcast via API
When a service sends us an XML CAP broadcast message with Cancel
status, and that broadcast is in broadcasting state, we cancel it.
2022-01-19 15:42:26 +00:00
Pea Tyczynska
940126abfb Reject unapproved broadcast upon cancel API request
When a service sends us a cancel broadcast XML via API, if that
broadcast was not approved yet, reject it.
2022-01-19 15:41:38 +00:00
Chris Hill-Scott
c0742fe83d Pass polygons through if they’re small already
If a polygon is smaller than the largest polygon in our dataset of
simplified polygons then we’re only throwing away useful detail by
simplifying it.

We should still simplify larger polygons as a fallback, to avoid sending
anything to the CBC that we’re not sure it will like.

The thresholds here are low: we can raise them as we test and experiment
more.

Here’s some data about the Flood Warning Service polygons

Percentile | 80% | 90%   | 95%    | 98%     | 99%     | 99.9%
-----------|-----|-------|--------|---------|---------|---------
Point count| 226 | 401.9 | 640.45 | 1015.38 | 1389.07 | 3008.609

Percentile    | 80% | 90%   | 95%    | 98%     | 99%     | 99.9%
--------------|-----|-------|--------|---------|---------|---------
Polygon count |2----|3------|5-------|8--------|10-------|40.469
2021-11-18 15:48:45 +00:00
Chris Hill-Scott
4feb3fdc10 Bump utils
This new version of utils implements the transformation of our polygons
to a Cartesian plane. In other words, it converts them from being
defined in spherical degrees to metres.

For the API this means our simplification will be slightly more
accurate.
2021-11-18 15:43:39 +00:00
Chris Hill-Scott
54bcf618da Store the event field from CAP XML broadcasts
We don’t store everything that comes in the CAP XML when someone creates
a broadcast via the API.

One thing we do store is `<identifier>` (in a column called `reference`)
which is a unique (to the external system) identifier for the broadcast.
We show this in the front end instead of the template name, because
broadcasts created from the API don’t use templates.

However this ID isn’t very friendly – the Environment Agency just supply
a UUID.

The Environment Agency also populate the `<event>` field with some human
readable text, for example:
> 013 Issue Severe Flood Warning EA

(013 is an area code which will be meaningful to the Flood Warning
Service team)

We should show this in the UI instead of the reference. The first step
towards this is storing it in the database and returning it in the REST
endpoints.

Later we can have the admin app prefer `cap_event` over `reference`,
where `cap_event` is present.

We can’t backfill this data because we don’t keep a copy of the original
XML.

Seems like `<event>` is a mandatory property of `<info>`, so we don’t
need to worry about the field being missing (`<info>` is optional in
CAP but we require it because it contains stuff like the areas which
we need in order to send out the broadcast`).

***

https://www.pivotaltracker.com/story/show/176927060
2021-10-26 11:12:27 +01:00
Ben Thorner
35430e9a9f Refactor custom validation into own function
This sets a pattern for adding another in the next commits.
2021-09-15 11:02:50 +01:00
Ben Thorner
ec1171f85c Switch "areas" field to "areas_2" format
The Admin app is now temporarily using the "areas_2" field while
we migrate "areas" to the new format [1].

[1]: https://github.com/alphagov/notifications-admin/pull/4004
2021-08-27 14:22:11 +01:00
Ben Thorner
023a06d5fb Start dual running with "areas" and "names"
For the public API we actually receive a "name" instead of an ID,
which we also want to start sending from the Admin app.

Unlike IDs, which aren't really used anywhere, we want the names
to display the alerts on gov.uk/alerts.
2021-08-26 15:34:25 +01:00
Chris Hill-Scott
f3e6d92046 Revert "Update utils to bring in coordinate transformation" 2021-08-20 16:05:39 +01:00
Chris Hill-Scott
e92be8b034 Pass polygons through if they’re small already
If a polygon is smaller than the largest polygon in our dataset of
simplified polygons then we’re only throwing away useful detail by
simplifying it.

We should still simplify larger polygons as a fallback, to avoid sending
anything to the CBC that we’re not sure it will like.

The thresholds here are low: we can raise them as we test and experiment
more.

Here’s some data about the Flood Warning Service polygons

Percentile | 80% | 90%   | 95%    | 98%     | 99%     | 99.9%
-----------|-----|-------|--------|---------|---------|---------
Point count| 226 | 401.9 | 640.45 | 1015.38 | 1389.07 | 3008.609

Percentile    | 80% | 90%   | 95%    | 98%     | 99%     | 99.9%
--------------|-----|-------|--------|---------|---------|---------
Polygon count |2----|3------|5-------|8--------|10-------|40.469
2021-08-19 11:08:53 +01:00
Chris Hill-Scott
a6135fb8ab Bump utils
This new version of utils implements the transformation of our polygons
to a Cartesian plane. In other words, it converts them from being
defined in spherical degrees to metres.

For the API this means our simplification will be slightly more
accurate.
2021-08-19 11:08:18 +01:00
Ben Thorner
a91fde2fda Run auto-correct on app/ and tests/ 2021-03-12 11:45:45 +00:00
Chris Hill-Scott
8e8601338e Merge pull request #3136 from alphagov/validate-template-length-broadcast-api
Validate content length on broadcast API
2021-02-17 11:34:29 +00:00
Chris Hill-Scott
0bb671df45 Validate content length on broadcast API
The maximum content count of a broadcast varies depending on its
encoding, so we can’t simply validate it against a schema. This commit
moves to using the validation from `notifications-utils`, and raising a
custom error response.
2021-02-16 09:30:40 +00:00
Katie Smith
6b8ebb3421 Fix linting errors 2021-02-16 09:03:38 +00:00
Pea Tyczynska
3037bf5fff Set broadcast message to stubbed when posting broadcast via API 2021-02-09 10:41:36 +00:00
Chris Hill-Scott
dec16a98f6 Handle XML files that have a declaration
`lxml` wants its input in bytes:

> XML is explicitly defined as a stream of bytes. It's not Unicode text.
> […] rule number one: do not decode your XML data yourself.

– https://lxml.de/FAQ.html#why-can-t-lxml-parse-my-xml-from-unicode-strings

It will accept strings unless, unless the document contains a
declaration[1] with an `encoding` attribute. Then it will refuse to
parse the document and raises a `ValueError`[2].

We can get fix this by passing `lxml` the bytes from the request, rather
than the decoded text.

1. > XML documents may begin with an XML declaration that describes some
   > information about themselves. An example is
   > `<?xml version="1.0" encoding="UTF-8"?>`.
   – https://en.wikipedia.org/wiki/XML#XML_declaration
2. See an example of this exception being raised in production here:
   https://kibana.logit.io/s/9423a789-282c-4113-908d-0be3b1bc9d1d/app/kibana#/doc/logstash-*/logstash-2021.02.05/syslog?id=AXdzfZVz5ZSa5DKpJiYd&_g=()
2021-02-08 08:51:14 +00:00
Chris Hill-Scott
ca6c46c4dd Add logging for succesful broadcast message creation 2021-01-26 16:24:45 +00:00
Chris Hill-Scott
0398ac57f1 Use correct HTTP status code for bad content type
415 is the status code for ‘Unsupported media type’

https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/415
2021-01-26 16:24:45 +00:00
Chris Hill-Scott
b85fcafd46 Don’t allow broadcasts to be created from JSON
Until we know we’re going to have real users for this, let’s not expose
it.
2021-01-26 16:24:45 +00:00
Chris Hill-Scott
c9d55039eb Simplify polygons before storing them
We’re going to let people pass in fairly complex polygons, but:
- we don’t want to store massive polygons
- we don’t want to pass the CBCs massive polygons

So this commit adds a step to simplify the polygons before storing them.

We think it’s best for us to do this because:
- writing code to do polygon simplification is non-trivial, and we don’t
  want to make all potential integrators do it
- the simplification we’ve developed is domain-specific to emergency
  alerting, so should throw away less information than

There’s a bit more detail about how we simplify polygons in
https://github.com/alphagov/notifications-admin/pull/3590/files
2021-01-26 16:24:45 +00:00
Chris Hill-Scott
26871eeacc Validate CAP against the spec
This gives us some extra confidence that there aren’t any problems with
the data we’re getting from the other service. It doesn’t address any
specific problems we’ve seen, rather it seems like a sensible precaution
to take.
2021-01-26 16:24:45 +00:00
Chris Hill-Scott
38f07db23e Accept CAP XML
This commit makes the existing endpoint also accept CAP XML, should the
appropriate `Content-Type` header be set.

It uses the translation code we added in a previous commit to convert
the CAP to a dict. We can then validate that dict against with the JSON
schema to ensure it’s something we can work with.
2021-01-26 16:24:44 +00:00
Chris Hill-Scott
7530408a21 Validate broadcast against schema
This commit adds a JSONSchema which can validate the fields in an API
call to create a broadcast. It takes the CAP XML schema as a starting
point.
2021-01-26 16:24:44 +00:00
Chris Hill-Scott
61c9e50ed9 Add public API endpoint to create emergency alerts
We know there is at least one system which wants to integrate with
Notify to send out emergency alerts, rather than creating them manually.

This commit adds an endpoint to the public API to let them do that.

To start with we’ll just let the system create them in a single call,
meaning they still have to be approved manually. This reduces the risk
of an attacker being able to broadcast an alert via the API, should the
other system be compromised.

We’ve worked with the owners of the other system to define which fields
we should care about initially.
2021-01-26 16:24:44 +00:00