Commit Graph

7023 Commits

Author SHA1 Message Date
Chris Hill-Scott
dfa2ae0ced Order areas by name in examples
When you click through to the page for a library you see the available
areas in alphabetical order. The examples given for each library should
match this.
2020-08-12 16:34:17 +01:00
Chris Hill-Scott
53fd5b8869 Exclude grouped areas from examples
The given examples should match the choices offered when you visit the
next page. The choices offered on the next page are either the areas
(when a library is not grouped) or the groups (when a library is
grouped).

This commit makes the examples match the choices by excluding sub-areas,
ie those that have a grouping ID.
2020-08-12 16:34:17 +01:00
Chris Hill-Scott
781847d32c Merge pull request #3557 from alphagov/view-only-broadcast-page
Let users without `send_messages` view broadcasts
2020-08-12 15:16:39 +01:00
Chris Hill-Scott
8d74e9c08d Store names and IDs of areas in memory
Now that the data needed to create a `BroadcastArea` is pretty
lightweight because it doesn’t include the GeoJSON we can go back to
putting it in memory when we start up the app, to make the pages load
really fast.

Rough estimate for the size of this dataset:
> 10,000 areas
> Average length of area name = 20 characters
> Average length of area id = 20 characters
> Size of one area in bytes = 20 + 20 = 40
> Size of dataset = 40 * 10,000 = 400,000 bytes = 400kb
2020-08-12 10:45:44 +01:00
Chris Hill-Scott
6c21a1732c Refactor to assign directly
There’s no need for an intermediate variable here, and it’s confusing
having two properties containing the same data.
2020-08-12 10:45:14 +01:00
Chris Hill-Scott
f96c43f5bc Store GeoJSON in a separate table
I think that even with good indexes, querying the area names from one
table is always going to be slow because there’s so much GeoJSON to scan
past.

This commit splits the data into two tables, one for the names and
grouping IDs and one for the blobs of GeoJSON. So for most pages the app
will never even be looking at the table where the GeoJSON is held.

I don’t know if this is a proper, normalised way of structuring the
data, but it does go brrr.
2020-08-12 10:43:01 +01:00
Chris Hill-Scott
e267e3d9f1 Refactor to use .query
All the other methods of this class call through to query, we can make
this code clearer by making `get_areas` do the same.
2020-08-12 10:42:28 +01:00
Chris Hill-Scott
adad27dadb Lazy load feature from the SQLite database
Rather than querying all the features whenever we look up area(s) let’s
only get them when we need them.

The features are really big blobs of data to pass around, so there’s a
significant performance gain to be had from doing this.
2020-08-12 10:40:27 +01:00
Chris Hill-Scott
3f9e49603d Merge pull request #3544 from alphagov/choose-electoral-ward
Let users select specific electoral wards in a local authority to broadcast to
2020-08-12 08:48:13 +01:00
Chris Hill-Scott
78c88530b5 Let users without send_messages view broadcasts
At the moment viewing a broadcast is limited to those users who have
the `send_messages` permission.

This doesn’t match how we describe the permissions on the team members
page

This commit makes it so that any team member can see a broadcast that’s
in any state other than `draft`.
2020-08-12 08:19:49 +01:00
Chris Hill-Scott
8570901731 Let users select electoral wards of local authorities
If a library has groups, we should show a link instead of selecting the
group directly.

Then we can give the user the choice of selecting the whole of that
group, or specific areas within the group.

For now the only libraries we have with groups are local authorities,
which group electoral wards.
2020-08-11 17:38:15 +01:00
David McDonald
2aaca11f8b Turn redis back on 2020-08-11 14:26:12 +01:00
David McDonald
32cc168c23 Merge pull request #3556 from alphagov/turn-redis-off
Turn off redis in all environments for rotation
2020-08-11 14:25:30 +01:00
Chris Hill-Scott
0ce64bcfde Merge pull request #3553 from alphagov/move-broadcast-areas-into-admin
Move broadcast areas into admin
2020-08-11 13:25:21 +01:00
Toby Lorne
b0ff2d41c5 broadcast-areas: examples are deterministic
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-11 12:04:02 +01:00
Chris Hill-Scott
cf999b5d8e Add inline comment explaining presence of field 2020-08-11 11:26:31 +01:00
Chris Hill-Scott
834b0fc3d5 Put hidden email field on register from invite page
Password managers will try to guess what they should save as a username
by looking at the fields on the page where you set up your password.

When registering from an invite the email address (what we use as a
username) is predefined, and only shown on the page as text, not an
input.

This commit also adds a hidden input field for password managers to pick
up.

Adapted from: https://github.com/UKGovernmentBEIS/beis-opss-psd/blob/master/app/views/users/complete_registration.html.erb#L29-L36
2020-08-10 15:59:27 +01:00
Toby Lorne
7a1b80e533 broadcast-areas: sort imports
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:22 +01:00
Toby Lorne
698e9816ec models: use broadcast areas from admin package
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:22 +01:00
Toby Lorne
74b83ffa8b broadcast-areas: vendor areas sqlite3 db
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:22 +01:00
Toby Lorne
488a5440cd broadcast-areas: move broadcast areas into app
Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
2020-08-10 12:50:20 +01:00
David McDonald
30ed483b7a Turn off redis in all environments for rotation
This is a very short term turn off for while we rotate creds. It will
then be followed immediately by a PR to turn it back on.
2020-08-10 10:25:08 +01:00
Chris Hill-Scott
3573ce1437 Merge pull request #3550 from alphagov/fix-html-escaping-email-subject-preview
Fix HTML being escaped in preview of email subject
2020-08-10 08:59:44 +01:00
Chris Hill-Scott
0e99bede68 Merge pull request #3552 from alphagov/cant-self-approve
Don’t let users self-approve broadcasts
2020-08-06 14:04:07 +01:00
Chris Hill-Scott
479406c02d Don’t let users self-approve broadcasts
At the moment they will get a ‘technical difficulties’ error if they
try.

We probably want to do something around letting people self-approve
broadcasts in trial mode, but for now just telling them they can’t is a
better experience than ‘technical difficulties’ (and will probably be
close to what they should see on a live service as well).
2020-08-05 16:01:21 +01:00
Tom Byers
cd398ab55c Merge pull request #3543 from alphagov/remove-code-supporting-old-permissions-form_fields
Revert support for old user permissions params
2020-08-05 15:51:28 +01:00
Chris Hill-Scott
cc9fecfaa0 Fix typo on invite page
There’s an _i_ missing from _Permssions_
2020-08-04 18:17:38 +01:00
Chris Hill-Scott
b31fb7de66 Merge pull request #3546 from alphagov/redis-cache-broadcast-message
Cache broadcast messages in Redis
2020-08-04 16:17:21 +01:00
Chris Hill-Scott
4d65b94c77 Fix HTML being escaped in preview of email subject
`EmailPreviewTemplate.subject` returns a string of HTML, with any
user-submitted HTML already escaped:
b5a61bfb7b/notifications_utils/template.py (L672)

What won’t be escaped is the HTML needed to redact the placeholders. We
generate this HTML so we know its safe, and doesn’t need to be escaped.
However when we pass it to Jinja, Jinja doesn’t know this, so will try
to escape it. This means users will see the raw HTML.

We can get around this by using Flask’s `Markup` class to tell Jinja
that the string is already sanitised and doesn’t need escaping again.

Text message templates don’t have this problem because they already
return `Markup`: b5a61bfb7b/notifications_utils/template.py (L288)

Letter templates don’t suffer from this problem (because they don’t
support redaction) but without making the same change they would still
double-escape ampersands, greater-than symbols, and so on.
2020-08-04 15:16:18 +01:00
Tom Byers
a74501f1d8 Merge pull request #3535 from alphagov/try-introducing-govuk-checkboxes-again
Try introducing govuk checkboxes again
2020-08-04 14:39:54 +01:00
Chris Hill-Scott
63095f7a6c Merge pull request #3536 from alphagov/guest-list-endpoint
Use new guest list API URLs
2020-08-04 11:40:42 +01:00
Chris Hill-Scott
3faf92bfef Go to broadcast, not dashboard after submitting
Once you’ve created a broadcast you’re taken back to the dashboard. This
feels too passive, and you might miss that the broadcast still needs
approval.

We should be much more explicit that you now need to find someone to
approve your broadcast. Taking someone directly to the page for a
broadcast lets us give more information about the status of the
broadcast and what the next steps should be.
2020-08-03 15:46:23 +01:00
Chris Hill-Scott
47a5c2abba Merge pull request #3540 from alphagov/search-whole-postal-address
Tell users that they can search whole postal address
2020-08-03 14:47:58 +01:00
Chris Hill-Scott
4bf4680031 Merge pull request #3547 from alphagov/broadcast-tour
Add a tour for users new to broadcast services
2020-08-03 14:21:03 +01:00
Chris Hill-Scott
19b42e3331 Add a tour for users new to broadcast services
This is an initial, prototype-quality attempt at introducing some kind
of tour for users new to broadcasting. A lot of the users we’re speaking
to don’t have a good concept of what broadcasting means, which is
causing usability problems down the line.

We did a similar thing in the early days of Notify to explain the
concept of message templates and personalisation.
2020-08-03 14:13:48 +01:00
Chris Hill-Scott
058553b6ee Cache broadcast messages in Redis
This should make the pages slightly quicker to load, because Redis will
return the JSON string faster than the API.

The only change that can happen to a broadcast which doesn’t go through
the admin app is a broadcast ending at its scheduled time. So this could
result in a cached broadcast having a status of `broadcasting` when it
had in fact finished. We already account for this here though:
b2b58ec044/app/models/broadcast_message.py (L89-L94)
2020-08-03 11:38:10 +01:00
Leo Hemsted
867ef30d18 add err msgs for letters with notify tags on pages 2+ 2020-07-31 18:31:26 +01:00
Tom Byers
468a43bd68 Revert support for old user permissions params
The fields used for user permissions on
permissions forms were changed as part of the work
converting the checkboxes to GOVUK Frontend.

This removes code added to protect against a
situation where the server-side app was running
this updated code but clients were POSTing from
pages that were not, and so sending the old HTTP
params.
2020-07-31 15:40:12 +01:00
Tom Byers
75bac87a4d Make permissions forms handle old/new params 2020-07-31 15:15:37 +01:00
Katie Smith
ba5f7d7c36 Check for invalid chars in letter addresses
This now adds validation for invalid characters on the
LetterAddressForm for one off letters. It also adds a validation failed
message for uploaded letters, precompiled letters sent through the API,
and CSV rows with errors.
2020-07-31 08:51:07 +01:00
Chris Hill-Scott
3fbf966719 Display full address on the notifications page
This is what we do on the uploads page now. It makes it more obvious
why your search term has returned a certain result if you can see most
of the address, not just the first line.
2020-07-30 16:40:25 +01:00
Chris Hill-Scott
2789b6a596 Tell users that they can search whole postal address
We’re now normalising and storing the whole address in the
`normalised_to` field. Previously we were only storing the first line
of the address.

Enough time should now have passed that the field will have been
populated for all letters in the database.

Thus we can now tell users that it’s not just the first line they can
search by.
2020-07-30 16:20:19 +01:00
Chris Hill-Scott
52c9cf3e76 Merge pull request #3538 from alphagov/start-time-hint
Make start time explicit when previewing a broadcast
2020-07-29 09:15:21 +01:00
Chris Hill-Scott
053ed96974 Make start time explicit when previewing a broadcast
We recently introduced a form control that lets user choose when a
broadcast ends.

Based on the most recent research participant, we think:
- there is a specific misunderstanding of what this control does
- there is a general low level of understanding of what a ‘broadcast’
  means

People will try to understand what a ‘broadcast’ is by using mental
models they have for other kinds of messaging, for example text
messages.

Other kinds of messaging are one-to-one, i.e. they go from a sender to a
recipient. They are not ongoing in any way.

Emails and texts are sent at a time (and for all practicable purposes
are received at that same time). So, when we present the user with
a form that controls time, they might well assume it controls the time
when the message will be sent.

This is a feature we offer for sending messages using a spreadsheet, and
that’s where we’ve borrowed this pattern from.

We reinforce this assumption with the labelling of the form control. By
front-loading it with the word ‘When’ we are playing to the users
confirmation bias, i.e. they are interpreting the meaning of the control
in a way that confirms their prior beliefs about how messaging works.

So this commit does two things:
- re-labels the form to front-load the word ‘End’ not ‘When’
- adds text to the page explaining when the broadcast will start, so
  there’s a chance of overriding that confirmation bias

If we can get users to go through this before sending a broadcast for
real, it could help them learn what a broadcast is, and how it differs
from sending text messages.
2020-07-27 17:33:34 +01:00
Katie Smith
716977fe75 Include template values when calculating letter page count
When sending a letter we check how many pages it has and this number
then determines how many PNG images we ask template preview for. When
calculating the page count, we were getting the page count for the
template as it comes from the database (so without any placeholders
filled in). But filling in placeholders in a letter may cause the number
of pages to change, which was the cause of the 'Letter does not have a
page x' errors we were seeing from template-preview.

Now, when we calculate the letter page count during sending, we take the
placeholders that have already been filled in into account.
2020-07-27 17:22:14 +01:00
Chris Hill-Scott
e0cb487093 Use new guest list API URLs
The API now[1] accepts requests on `…/guest-list` as well as
`…/whitelist`. This commit starts using the former, which means:
- the use of ‘whitelist’ is fully gone from the admin app
- the API can stop using it, at least in URLs

1. As of https://github.com/alphagov/notifications-api/pull/2928
2020-07-27 13:24:37 +01:00
Tom Byers
0d6ef2977d Merge pull request #3465 from alphagov/add-scroll-to-reveal-method
Add scroll to reveal element method
2020-07-24 14:21:15 +01:00
Tom Byers
ca9b8a8ca3 Add analytics error tracking to checkbox fields
The existing macros added data attributes to any
error message displayed which communicated the
error to Google Analytics (if the user had given
consent).

This re-implements that functionality.
2020-07-24 11:34:02 +01:00
Tom Byers
01f84d5443 Convert checkboxes for broadcast areas
Includes removal of MultiCheckboxField due to it
no longer being used elsewhere in this file.
2020-07-24 11:34:02 +01:00
Tom Byers
ee03753187 Make template-list checkbox label text match link
The checkboxes need an accessible name that
identifies the folder/template and this needs to
include their full path to avoid duplication.

There's a lot of debate about how to write out
breadcrumb/path syntax so this just puts all the
words together under the assumption that the
folder naming will describe the path (and to
introduce as little extra semantics as possible
to start with).
2020-07-24 11:34:02 +01:00