This now adds validation for invalid characters on the
LetterAddressForm for one off letters. It also adds a validation failed
message for uploaded letters, precompiled letters sent through the API,
and CSV rows with errors.
We recently introduced a form control that lets user choose when a
broadcast ends.
Based on the most recent research participant, we think:
- there is a specific misunderstanding of what this control does
- there is a general low level of understanding of what a ‘broadcast’
means
People will try to understand what a ‘broadcast’ is by using mental
models they have for other kinds of messaging, for example text
messages.
Other kinds of messaging are one-to-one, i.e. they go from a sender to a
recipient. They are not ongoing in any way.
Emails and texts are sent at a time (and for all practicable purposes
are received at that same time). So, when we present the user with
a form that controls time, they might well assume it controls the time
when the message will be sent.
This is a feature we offer for sending messages using a spreadsheet, and
that’s where we’ve borrowed this pattern from.
We reinforce this assumption with the labelling of the form control. By
front-loading it with the word ‘When’ we are playing to the users
confirmation bias, i.e. they are interpreting the meaning of the control
in a way that confirms their prior beliefs about how messaging works.
So this commit does two things:
- re-labels the form to front-load the word ‘End’ not ‘When’
- adds text to the page explaining when the broadcast will start, so
there’s a chance of overriding that confirmation bias
If we can get users to go through this before sending a broadcast for
real, it could help them learn what a broadcast is, and how it differs
from sending text messages.
Different emergencies will need broadcasts to last for a variable amount
of time. We give users some control over this by letting them stop a
broadcast early. But we should also let them set a maximum broadcast
time, for:
- when the duration of the danger is known
- when the broadcast has been live long enough to alert everyone who
needs to know about it
This code re-uses the pattern for scheduling jobs, which has some
constraints that are probably OK for now:
- end time is limited to an hour
- longest duration is 3 whole days (eg if you start broadcasting Friday
you have the choice of Saturday, Sunday and all of Monday, up to
midnight)
For services with the broadcast permission this hides:
- the ‘View dashboard’ permission (and defaults it to _checked_) because
all users of broadcast services will need to see the dashboard
- the ‘Manage API keys’ permission (and defaults it to _not checked_)
because we don’t offer an API integration for broadcast services yet
– if we do we won’t want existing users to automatically get the
permission
It relabels:
- the ‘Send’ permission to ‘Prepare and approve’ to match the current,
slightly clunky language on the templates page
- the ‘Manage settings’ label to not refer to ‘usage’ because broadcast
services won’t incur cost
At the moment the page is the same as for text message templates,
except:
- different H1
- no guidance about personalisation, links, etc (until we decide how
these should work)
For now you won’t be able to really create a broadcast template, because
the API doesn’t support it (the API will respond with a 400). But that’s
OK because no real services have the broadcast permission yet.
This required a bit of refactoring of how we check which template types
a service can use, because there were some hard-coded assumptions about
emails and text messages.
‘Commonly used passwords’ is more specific, and avoids the terminology
‘blacklist’ which the National Cyber Security Centre explain to be
problematic:
> It's fairly common to say whitelisting and blacklisting to describe desirable and undesirable things in cyber security. For instance, when talking about which applications you will allow or deny on your corporate network; or deciding which bad passwords you want your users not to be able to use.
>
> However, there's an issue with the terminology. It only makes sense if you equate white with 'good, permitted, safe' and black with 'bad, dangerous, forbidden'. There are some obvious problems with this. So in the name of helping to stamp out racism in cyber security, we will
> avoid this casually pejorative wording on our website in the future. No, it's not the biggest issue in the world - but to borrow a slogan from elsewhere: every little helps.
– https://www.ncsc.gov.uk/blog-post/terminology-its-not-black-and-white
International letters don’t have a choice of postage. Under the hood
they are either `europe` or `rest-of-world`.
So, for letters that we detect are international, this commit:
- removes the radios buttons that give users the choice of postage
- passes through either `europe` or `rest-of-world` to the API,
depending on what address we find in the letter
This will cause the API to 500 until it can accept `europe` or
`rest-of-world` as postage types, but this is probably OK because it’s
only our services that have international letters switched on at the
moment.
Single checkboxes are distinct because:
- they don't need to be wrapped in a `<fieldset>`
- they are a subclass of BooleanField so their
data is either True or False
Nested checkboxes with a single top-level node
will only have one item in their `items` list.
This is because the other choices are children of
that list item.
This means we need to check the `choices`
attribute, which lists all the checkboxes, to see
if they should be marked as a group (by being
wrapped in a `<fieldset>`) or not.
Includes:
1. changes to make NestedFieldMixin work
with new fields and CSS for nested checkboxes
2. adds custom version of GOVUK checkboxes
component to allow us to:
- add classes to elements currently inaccessible
- wrap the checkboxes in a list
- add child checkboxes to each checkbox (making
tree structures possible through recursion
Change 2. should be pushed upstream to the GOVUK
Design System as a proposal for changes to the
GOVUK Checkboxes component.
Allows checkboxes to be collapsed so they take up
less space in the page. The collapsed state
includes a live summary tracking which of them are
selected.
Includes changes to the JS for collapsible
checkboxes to make it work with the GOVUK
Checkboxes component HTML.
govukCheckboxesField subclasses
SelectMultipleField and overwrites how it renders
HTML to let us use the GOVUK Checkboxes component
while retaining all the functionality of WTForms
fields.
Based on work on github.com/richardjpope/recourse:
https://github.com/richardjpope/recourse/blob/master/recourse/forms.py#L6
For services with permission, they can now put international addresses
into their spreadsheets without getting a postcode error.
This also means they can start using address line 7 instead of postcode,
since it doesn’t make sense to put a country in a field called
‘postcode’. But this will be undocumented to start with, because we’re
not giving any real users the permission.
It does now mean that the number of possible placeholders (7 + postcode)
is greater than the number of allowed placeholders (7), so we have to
account for that in the one-off address flow where we’re populating the
placeholders automatically. We’re sticking with 6 + postcode here for
backwards compatibility.
This involves three changes which broke our code.
To validate email addresses, the optional dependency `email-validator`
must be installed<sup>1</sup>. But since we don’t use WTForms’ email
validation, we shouldn’t need to subclass it – it can just be its own
self contained thing. Then we don’t need to add the extra dependency.
When rendering textareas, and extra `\r\n` is inserted at the beginning
<sup>2</sup>. Browsers will strip this when displaying the textbox and
submitting the form, but some of our tests need updating to account for
this.
The error message for when you don’t choose an option from some radio
buttons has now changed. Rather than just accepting WTForms’ new
message, this commit makes the error messages like the examples from
the Design System<sup>3</sup>. By default it will say ‘Select an
option’, but by passing in an extra parameter (`thing`) it can be
customised to be more specific, for example ‘Select a type of
organisation’.
***
1. https://github.com/wtforms/wtforms/pull/429
2. https://github.com/wtforms/wtforms/issues/238
3. https://design-system.service.gov.uk/components/radios/#error-messages
I noticed when using the dication software that saying ‘one two three
four five’ got dictated as `123 45`. This tripped the validation,
because the space character isn’t a digit.
So this commit normalises out spaces (and other spacing characters like
dashes and underscores) before validating the code and sending it to the
API.
I can also imagine that some people might like to space out the code to
make it easier to transcribe (like you might do with a credit card
number).
Optional address placeholders aren’t a thing for one-off letters any
more, so we can tidy up the code a bit by removing the parts of the flow
that are accounting for them.
We’re doing this everywhere else now, so this completes the story.
It uses the same regex as elsewhere and the error messaging is
consistent (but not uniform) with the other places.
Since we’re doing normalisation and line-count-checking of addresses in
multiple places it makes sense for that code to be shared. Which is
what happened here:
https://github.com/alphagov/notifications-utils/pull/713
This commit refactors the admin code to make use of the new utils code.
Note about placeholders:
- they now go into the session as `address_line_1` instead of `address
line 1` because this is the format the API uses, so should be
considered canonical
- they are now fetched from the session in a way that isn’t sensitive
to case or underscores (using the `Columns` class)
- the API doesn’t care about case or underscores vs spaces in
placeholder names because it’s checking an instance of `Template` to
see if all the required placeholders are present (see
401c8e41d6/app/notifications/process_notifications.py (L40))
rather than in multiple placeholders - this is the first step towards
making postcodes non-required, which is the first step towards
international letters.
they still populate address_line_# and postcode fields under the hood -
to keep validation working the same, the last line always goes into
`postcode`.
the form normalises whitespace, removes extra new lines, and enforces
that you have between three and seven lines.
if the letter repeats address placeholders further down (eg "Dear
((address_line_1))"), then it'll fill those in as well. It'll still
prompt you to fill them in, but they'll be pre-filled.
We can’t give advice to members of the public, but increasingly we’re
seeing them try to use our support form to ask.
It would be better for them if we can direct them straight to somewhere
more useful, before they have the chance to raise a support ticket.
This commit replaces the report a problem/ask a question triaging for
users who aren’t signed in. It’s not possible for non-signed-in users to
raise an priority 1 ticket, so we never need to triage the tickets in
this way.
Instead we can triage people based on whether they work in the public
sector or not. If they do then we send them on to the feedback form. If
not then they go to a new page which contains some useful links. We’ve
chosen these links based on some analysis of the support tickets we’ve
received recently[1]
1. https://docs.google.com/spreadsheets/d/1uBQn-ZnCYfz6ltFaUKZpytgvBF0-MeshCLZ1cD74R0c/edit?usp=sharing
We are seeing little benefit of allowing users to not put in their email
address. This will mean that you must provide it for feedback, not just
problems with the site.
There could maybe be some more refactoring of the support templates as
this is now very similar to the report a problem page but this is a
quick fix so haven't gone too in depth.
Their priority should always add up to 100%. Currently we have to ensure
this by hand. Adding this form means there’s no way to not set their
combined priorities to 100%. And it’s a bit more of an intuitive UI than
two textboxes on separate pages.
Flake8 Bugbear checks for some extra things that aren’t code style
errors, but are likely to introduce bugs or unexpected behaviour. A
good example is having mutable default function arguments, which get
shared between every call to the function and therefore mutating a value
in one place can unexpectedly cause it to change in another.
This commit enables all the extra warnings provided by Flake8 Bugbear,
except for the line length one (because we already lint for that
separately).
It disables:
- _B003: Assigning to os.environ_ because I don’t really understand this
- _B306: BaseException.message is removed in Python 3_ because I think
our exceptions have a custom structure that means the `.message`
attribute is still present
We now use the pattern of showing a box at the top of the page with the
error. The error message has a heading and can have additional details.
Error messages and the invalid pages get stored in the S3 metadata.
