Commit Graph

10164 Commits

Author SHA1 Message Date
Toby Lorne
01ffc815e7 broadcast-areas: do not use group table
do not need it

Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
17af32978f broadcast-areas: use cached_property not a hack
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
8ecbfe5996 broadcast-areas: package works with groups
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
58915ac968 broadcast-areas: plot script works with groups
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
7f4dd0acb5 broadcast-areas: create broadcast areas in groups
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
65bef7bdc3 broadcast-areas: only parse geojson once
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
35a881d950 broadcast-areas: only load geojson when needed
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
0ae8ef650f broadcast-areas: apply buffer to areas
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
22cf63a440 broadcast-areas: simple feature plots show points
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
e21d424f97 broadcast-areas: (dev) add visualisation tool
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
6d5593dc32 broadcast-areas: include simple feature
simple feature is a feature where there are no islands and all polygons
are capped to 125 points

Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
7074655b84 broadcast-areas: BroadcastAreaLibraries uses sql
sqlite

Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
9a9bf3c270 broadcast-areas: refactor to use repo.py
share functionality between repo used by create script and by package

Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
15d4d3c0a1 broadcast-areas: initial repository class
repository to represent the sqlite broadcast areas db

Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Toby Lorne
3f54af13b5 broadcast-areas: add a script to build sqlite db
import the raw datasets into a vendored sqlite3 db which can be queried
and versioned with code

Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:18 +01:00
Leo Hemsted
b3ed21252d ensure you can get broadcast_areas from a list as well as from *args 2020-08-10 12:50:17 +01:00
Chris Hill-Scott
078f1dd8d3 Add broadcast area model, loading from GeoJSON
This commit adds a new model class which can be used by any app to
interact with a broadcast area. A broadcast area is one or more polygons
representing geographical areas.

It also adds some models that make browsing collections of these areas
more straightforward. So the hierarchy looks like:

> **BroadcastAreaLibraries*
> Contains multiple libraries of broadcast area
> > **BroadcastAreaLibrary**
> > A collection of geographic areas, all of the same type, for example
> > counties or electoral wards
> > **BroadcastArea**
> > Contains one or more shapes that make up an area, for example
> > England
> > > **BroadcastArea.polygons[n]**
> > > A single shape, for example the Isle of Wight or Lindisfarne
> > > > **BroadcastArea.polygons[n][o]**
> > > > A single coordinate along a polygons

The classes support iteration, so all the areas in a library can be
looped over, for example if `countries` is an instance of
`BroadcastAreaLibrary` you can do:
```python
for country in countries:
    print(country.name)
```

The `BroadcastAreaLibraries` class also provides some useful methods for
quickly getting the polygons for an area or areas, for example to
render them on a map. So if `libraries` is an instance of
`BroadcastAreaLibraries` you can do:
```python
libraries.get_polygons_for_areas_long_lat('england', 'wales')
```

This will give polygons for the Welsh mainland, the Isle of Wight,
Anglesey, etc.

The models load data from GeoJSON files, which is an open standard for
serialising geographic data. I’ve added a few example files taken from
http://geoportal.statistics.gov.uk to show how it works.
2020-08-10 12:50:17 +01:00
Chris Hill-Scott
3573ce1437 Merge pull request #3550 from alphagov/fix-html-escaping-email-subject-preview
Fix HTML being escaped in preview of email subject
2020-08-10 08:59:44 +01:00
Chris Hill-Scott
0e99bede68 Merge pull request #3552 from alphagov/cant-self-approve
Don’t let users self-approve broadcasts
2020-08-06 14:04:07 +01:00
Chris Hill-Scott
479406c02d Don’t let users self-approve broadcasts
At the moment they will get a ‘technical difficulties’ error if they
try.

We probably want to do something around letting people self-approve
broadcasts in trial mode, but for now just telling them they can’t is a
better experience than ‘technical difficulties’ (and will probably be
close to what they should see on a live service as well).
2020-08-05 16:01:21 +01:00
Tom Byers
cd398ab55c Merge pull request #3543 from alphagov/remove-code-supporting-old-permissions-form_fields
Revert support for old user permissions params
2020-08-05 15:51:28 +01:00
David McDonald
8b44919551 Merge pull request #3548 from alphagov/pyup-scheduled-update-2020-08-03
Scheduled weekly dependency update for week 31
2020-08-05 10:30:19 +01:00
Chris Hill-Scott
2cd6db6357 Merge pull request #3551 from alphagov/permssions-permissions
Fix typo on invite page
2020-08-05 09:21:40 +01:00
Chris Hill-Scott
cc9fecfaa0 Fix typo on invite page
There’s an _i_ missing from _Permssions_
2020-08-04 18:17:38 +01:00
Chris Hill-Scott
b31fb7de66 Merge pull request #3546 from alphagov/redis-cache-broadcast-message
Cache broadcast messages in Redis
2020-08-04 16:17:21 +01:00
Chris Hill-Scott
4d65b94c77 Fix HTML being escaped in preview of email subject
`EmailPreviewTemplate.subject` returns a string of HTML, with any
user-submitted HTML already escaped:
b5a61bfb7b/notifications_utils/template.py (L672)

What won’t be escaped is the HTML needed to redact the placeholders. We
generate this HTML so we know its safe, and doesn’t need to be escaped.
However when we pass it to Jinja, Jinja doesn’t know this, so will try
to escape it. This means users will see the raw HTML.

We can get around this by using Flask’s `Markup` class to tell Jinja
that the string is already sanitised and doesn’t need escaping again.

Text message templates don’t have this problem because they already
return `Markup`: b5a61bfb7b/notifications_utils/template.py (L288)

Letter templates don’t suffer from this problem (because they don’t
support redaction) but without making the same change they would still
double-escape ampersands, greater-than symbols, and so on.
2020-08-04 15:16:18 +01:00
Tom Byers
a74501f1d8 Merge pull request #3535 from alphagov/try-introducing-govuk-checkboxes-again
Try introducing govuk checkboxes again
2020-08-04 14:39:54 +01:00
David McDonald
b872db3032 freeze requirements 2020-08-04 12:13:35 +01:00
Chris Hill-Scott
63095f7a6c Merge pull request #3536 from alphagov/guest-list-endpoint
Use new guest list API URLs
2020-08-04 11:40:42 +01:00
Chris Hill-Scott
16563cb8e8 Merge pull request #3549 from alphagov/go-to-broadcast-once-created
Go to broadcast, not dashboard after submitting for approval
2020-08-04 11:40:34 +01:00
Chris Hill-Scott
3faf92bfef Go to broadcast, not dashboard after submitting
Once you’ve created a broadcast you’re taken back to the dashboard. This
feels too passive, and you might miss that the broadcast still needs
approval.

We should be much more explicit that you now need to find someone to
approve your broadcast. Taking someone directly to the page for a
broadcast lets us give more information about the status of the
broadcast and what the next steps should be.
2020-08-03 15:46:23 +01:00
pyup-bot
7eefa1f2ad Update gds-metrics from 0.2.2 to 0.2.4 2020-08-03 14:53:16 +01:00
pyup-bot
9ff4adda0f Update notifications-python-client from 5.6.0 to 5.7.0 2020-08-03 14:53:15 +01:00
pyup-bot
6da970fdca Update pyexcel from 0.6.2 to 0.6.3 2020-08-03 14:53:15 +01:00
Chris Hill-Scott
47a5c2abba Merge pull request #3540 from alphagov/search-whole-postal-address
Tell users that they can search whole postal address
2020-08-03 14:47:58 +01:00
Chris Hill-Scott
4bf4680031 Merge pull request #3547 from alphagov/broadcast-tour
Add a tour for users new to broadcast services
2020-08-03 14:21:03 +01:00
Chris Hill-Scott
19b42e3331 Add a tour for users new to broadcast services
This is an initial, prototype-quality attempt at introducing some kind
of tour for users new to broadcasting. A lot of the users we’re speaking
to don’t have a good concept of what broadcasting means, which is
causing usability problems down the line.

We did a similar thing in the early days of Notify to explain the
concept of message templates and personalisation.
2020-08-03 14:13:48 +01:00
Leo Hemsted
7e257f32ad Merge pull request #3545 from alphagov/notify-tags
add err msgs for letters with notify tags on pages 2+
2020-08-03 13:26:13 +01:00
Chris Hill-Scott
058553b6ee Cache broadcast messages in Redis
This should make the pages slightly quicker to load, because Redis will
return the JSON string faster than the API.

The only change that can happen to a broadcast which doesn’t go through
the admin app is a broadcast ending at its scheduled time. So this could
result in a cached broadcast having a status of `broadcasting` when it
had in fact finished. We already account for this here though:
b2b58ec044/app/models/broadcast_message.py (L89-L94)
2020-08-03 11:38:10 +01:00
Leo Hemsted
867ef30d18 add err msgs for letters with notify tags on pages 2+ 2020-07-31 18:31:26 +01:00
Tom Byers
468a43bd68 Revert support for old user permissions params
The fields used for user permissions on
permissions forms were changed as part of the work
converting the checkboxes to GOVUK Frontend.

This removes code added to protect against a
situation where the server-side app was running
this updated code but clients were POSTing from
pages that were not, and so sending the old HTTP
params.
2020-07-31 15:40:12 +01:00
Tom Byers
75bac87a4d Make permissions forms handle old/new params 2020-07-31 15:15:37 +01:00
Tom Byers
e3c434bb8f Change permissions tests to support old API
User permissions were handled by a group of
BooleanFields but introducing the new checkboxes
changed this to just one field that stores its
data in a list.

It was mentioned in a comment that there could be
a situation, when the instances roll, where clients
are using the old fields but POSTing to a server
running the new code.

https://github.com/alphagov/notifications-admin/pull/3535#discussion_r460872903

This introduces tests for that situation.
2020-07-31 15:15:26 +01:00
Tom Byers
f322229614 Change tests to check permissions_field name
A comment on this group of changes mentioned the
'name' attribute of checkboxes for the
permissions_field should be checked as well as the
'value' attribute:

https://github.com/alphagov/notifications-admin/pull/3535#discussion_r460869797

This adds checks to support that point.
2020-07-31 15:15:16 +01:00
Katie Smith
b2b58ec044 Merge pull request #3541 from alphagov/bump-eventlet
Bump eventlet
2020-07-31 10:50:51 +01:00
Katie Smith
a1b9644290 Bump eventlet
dnspython had been changed from 1.16.0 to 2.0.0 in a previous commit,
but this was not compatible with eventlet 0.25.2. This bumps eventlet to
a later version, which has the effect of downgrading dnspython again.
2020-07-31 10:38:44 +01:00
Tom Byers
e266b11ee2 Fix tests for pages using govukCheckboxField
These fields used to use govukCheckboxesField and
so stored their data in a list. They were since
migrated to govukCheckboxField, which extends
BooleanField and so keeps its data as a boolean
value.
2020-07-31 10:20:44 +01:00
Katie Smith
2307cb6815 Merge pull request #3539 from alphagov/more-letter-validation
Check for invalid characters in letter addresses
2020-07-31 09:10:44 +01:00
Katie Smith
ba5f7d7c36 Check for invalid chars in letter addresses
This now adds validation for invalid characters on the
LetterAddressForm for one off letters. It also adds a validation failed
message for uploaded letters, precompiled letters sent through the API,
and CSV rows with errors.
2020-07-31 08:51:07 +01:00
Katie Smith
0304edd7a2 Bump utils to 40.6.0
This brings in validation for invalid characters in address lines.
2020-07-31 08:51:07 +01:00