We want to be able to set the free allowance for a service to 0, but the
form was not allowing this - it gave an error message of `Cannot be
empty`. This can be fixed by changing the WTForms validator from
`DataRequired` (which coerces 0 to falsey) to the `InputRequired`
validator.
Having to submit the form for each choice separately slowed us down
during an incident where Redis was unavailable and came back with
stale data, which we had to clear manually.
Note: we don't want to use the "flush" feature in case there are other
keys in Redis, which may not be safe to remove.
Changes:
53.0.0
---
* `notifications_utils.columns.Columns` has moved to
`notifications_utils.insensitive_dict.InsensitiveDict`
* `notifications_utils.columns.Rows` has moved to
`notifications_utils.recipients.Rows`
* `notifications_utils.columns.Cell` has moved to
`notifications_utils.recipients.Cell`
52.0.0
---
* Deprecate the following unused `redis_client` functions:
- `redis_client.increment_hash_value`
- `redis_client.decrement_hash_value`
- `redis_client.get_all_from_hash`
- `redis_client.set_hash_and_expire`
- `redis_client.expire`
51.3.1
---
* Bump govuk-bank-holidays to cache holidays for next year.
51.3.0
---
* Log exception and stacktrace when Celery tasks fail.
For the "Something else" branding form we want the form label to be the
title. This brings in the textarea component from GOV.UK Frontend in
order to do this since that contains code to set a the textarea label as
the page heading in an accessible way.
The rest of the textarea fields have not been switched to use the new
component yet.
We were showing the form to request email branding with a button which
submits your choice immediately. Now, we only submit the form
immediately if "Something else" is the only branding option available to
you. If you select any other radio button (or select "Something else"
when it's not the only option) we take you to another page which either
contains more information or a textbox to fill in the details for the
branding you want.
There is currently some duplication between the new pages and their
tests, but these will be changed in future versions of the work so will
start to differ more.
To render text in an SVG consistently the system rendering the SVG must
have the fonts specified by the SVG installed.
If the fonts are not installed then the renderer will fall back to a
system font and the text will look different. This is especially bad
news for branding where the right font is an integral part of any brand.
To fix this, the text should instead be converted to `<path>` elements.
This process is sometimes called ‘outlining’.
A few of our logos had this problem, and I’ve fixed most of them by
hand. Adding this validation will stop the problem, coming up again.
We use the `ChangeEmailForm` if you want to change your own email
address or someone else's email address. This has various validators
which get run. We check if the email address is valid (by using a
function from utils) and if the email address is already in use
(by calling API).
If the email address is not valid, we should not call API to see if it's
already in use because this will cause an exception in API leading to a
`500` in admin. We now only call API if there were no other errors with
the email address.
(The `test_should_redirect_after_name_change` test didn't need the
`mock_email_is_not_already_in_use` fixture, so this has been removed.)
We don't currently have any radio fields or check boxes where it's
possible to get more than one validation error. However, since we
never want to show more than one error at a time for a field, this
changes the error messages for the relevant widgets to only show the
first error if there ever were multiple.
If there were multiple errors, this widget was joining the messages
together and displaying all error messages. If a text input field does
have more than one validation error, we only want to show one.
This is a quick additional check to protect the user:
- From getting a CloudFront 502 error if the file takes too
long to upload. I was surprised to find it takes about 1 minute
to upload a 70Mb file to S3.*
- From getting a CloudFront 502 error when we follow the redirect
and run through the slow processing code in utils that builds a
RecipientCSV [1].
For context, a CSV with 100K rows and a few columns is around 5Mb,
so a 10Mb limit should be enough. Analysis over the past week shows
that the vast majority of CSV uploads are actually < 2.5Mb.
I haven't added any tests for this because:
- The check isn't critical, as the worst case scenario is the user
gets a worse error than this in-app one.
- There's no easy way to mock the validation, and I didn't want to
have a test that depends on a 10Mb+ file.
*We're using "key.put" to upload the file, when we could be doing
a multipart upload [2]. However, I tried this myself with a chunk
size of 1000 bytes and found it only led to a marginal improvement.
[1]: https://github.com/alphagov/notifications-utils/pull/930
[2]: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-uploading-files.html
WTForms versions less than 3.0.0 have a security vulnerability where
arbitrary HTML can be inserted into the label of a form, allowing the
possibility of a cross-site scripting attack.
I don’t know if there’s anywhere we put user-generated content into form
labels but it’s possible we are vulnerable somewhere.
This require moving some imports because as of
https://github.com/wtforms/wtforms/pull/614/files
there is no longer a separate module for HTML 5 fields, they are now
considered core fields.
As of https://github.com/wtforms/wtforms/issues/445/files custom
implementations of `pre_validate` or `post_validate` must raise
`ValidationError` to trigger a validation message, where we were raising
`ValueError` this was no longer being caught.
As of https://github.com/wtforms/wtforms/pull/355/files `StringField`
returns `None` for empty data, not `''` but our `validate_email_address`
function only accepts strings.
We don't use this term consistently and it's not defined anywhere.
Since most of the Admin app deals with user-facing permssions, it's
OK to just use the term "permissions". Where both types of permission
are present in the same file, we can more clearly distinguish them
as "UI permissions" and "DB permissions".
These are more than a list of permissions: each item includes the
label to use when displaying it as an option on a form. Switching
to a name that reflects how the attributes are used will help to
avoid confusion when we rename some of the other attributes in the
same file in later commits.
This is so we can allow the sender name 'UC' for DWP.
Note, this is specifically only straight single quotes and not curly
quotes or double quotes. Curly quotes are not supported in the GSM
character set (https://en.wikipedia.org/wiki/GSM_03.38). There is
currently no defined user ask to support double quotes in sms sender
names.
I have tested this by sending a message through both Firetext and MMG to
make sure they both support the single quote character in SMS sender
names.
DWP also have had no particular issues using the SMS sender name with
their existing system in the past either.
At the moment we say that you either ‘add’ an alert or ‘send’ it.
This is confusing because:
- an alert isn’t received on people’s phones until it’s approved, so
this is really when it is ‘sent’ conceptually
- an alert can be rejected before anyone receives it, so the UI can say
an alert that no-one ever received was sent
This commit re-labels things so that the the first part of the process
is ‘creating’ the alert.
This makes all the permissions nice and distinct from each other. Adding
templates and adding alerts feel conceptually quite different things
(what are you adding the alert to?).
It’s exactly the same code as `BaseInviteUserForm` so there’s really no
need to duplicate it (and means that changes we make to
`BaseInviteUserForm` in the future will get inherited).
The reason the email address is considered invalid is because it is the
address of the person doing the inviting.
This commit renames the variable to be more specific and avoid confusion
with the email address of the person being invited.
We don’t want a single person to have two accounts on an emergency
alerts service because it would let them circumvent the two eyes
approval process.
We can go some way to mitigating against this by stopping people using
common methods that email providers use to alias email addresses. These
are:
- being case insensitive
- being insensitive to the position or number of dots in the local part
of an email address
- using ‘plus addressing’
We already prevent the first one, this commit adds normalisation which
strip out the second two before doing the comparision with the current
user’s email address.
This means that we can rewrite `validate_email_address` to do a
different comparison without having to also change `__init__`
I’ve moved the platform admin check into its own conditional to keep the
line length manageable.
Previously we applied this restriction to Platform Admins, on the
assumption that all of them use a security key to log in. Rather
than making that assumption, we can explicitly check their login
method, which also supports rolling out the feature to more users.
We hide the radio field in the HTML for platform admins, as we don't
want anyone to be able to change their auth type. However, when the form
is validated, the form has a field called login_authentication that it
expects a value for. It silently fails as it complains that when the
user POSTed they didn't select a value for that radio field, but the
error message is on the radio fields that don't get displayed to the
user so they'd never know.
Fixing this is actually pretty hard.
We use this form in two places, one where we have a user to edit, one
where we are creating an invite from scratch. So sometimes we don't know
about a user's auth type. In addition, radio buttons are mandatory by
design, but now sometimes we don't just want to make it optional but
explicitly ignore the value being passed in? To solve this, remove the
field entirely from the form if the user is a platform admin. This means
that if the code in manage_users.py tries to access the
login_authentication value from the form, it'll error, but I think
that's okay to leave for now given we concede that this isn't a perfect
final solution.
The tests didn't flag this previously as they tried to set from sms_auth
(the default for `platform_admin_user`) TO email_auth or sms_auth. Also,
the diagnosis of this bug was confounded further by the fact that
`mock_get_users_by_service` sets what is returned by the API - the
service model then takes the IDs out of that response and calls
`User.get_user_by_id` for the matching ID (as in, the code only uses
get_users_by_service to ensure the user belongs to that service). This
means that we accidentally set the form editing the current user, as
when we log in we set `get_user_by_id` to return the user of our choice
Was just in one of those meetings where it felt like writing this would
take less time than I’d already spent talking about its relative
priority…
---
In the admin app you can already set the broadcast channel as 'test', 'severe' or 'government'.
Aim:
- Add the 'operator' channel to the list of channels you can pick for the admin app broadcast services
Note:
- The API already supports this - https://github.com/alphagov/notifications-api/pull/3262
- The CBC proxy does not yet support the operator channel and this will need a separate card. That card has not yet been written because the interface has not been agreed between us and the MNOs yet.
- Will need to have the ability to select the operator channel for just a single MNO like we do for the other channels
- If we add this, we shouldn't actually start using it until the MNO in question gives us the go ahead.
---
https://www.pivotaltracker.com/story/show/178485177
This takes a similar approach as in the previous commit. Since the
"training channel" doesn't really exist, we need some extra code
to pre-select it if a service is already in training mode. As in
the previous commit, I've removed a few non-critical test cases
where we really don't need to test exhaustively.
Note that we also need some specific code to avoid pre-selecting an
option for non-broadcast services, which only used to work by fluke:
we would try to populate the field with (False, None, 'all'), which
isn't a valid combination, so nothing was selected.
Previously this field had to mimic the final hyphenated string of
the broadcast account type, even though it was only used to select
one of its components. The new, shorter choices make it easier to
simplify the test for the POST request.
I've also deleted a number of test cases for pre-selected radios.
This functionality isn't critical, so we don't need to exhaustively
test every single possible combination of values.
This allows us to start decoupling the form fields from the final,
hyphenated string, which we'll do in the next commits.
Note that I've also removed the conditional that changes the data
of the network field as part of validating it. We shouldn't change
data in validations, and having the new property directly above
makes it clear there's no need for this code.
Previously we had to cope with two forms of the hyphenated string
we use to represent a pending change in broadcast account type.
Using "all" to mean "all providers" matches the behaviour in the
API [1], and means we can remove some complexity.
"training-test-all" isn't ideal, since the provider is irrelevant
for a training mode service. However, this isn't much worse than
the previous "training-test", noting that the channel also has no
relevance. We'll iterate this in later commits.
[1]: 8e1a144f87/migrations/versions/0352_broadcast_provider_types.py (L14)
Previously we could only select a provider when using the test
channel, but this is also required for others channels when we
do tests on the production network with individual MNOs.
In order to reduce duplication and improve consistency, I've reused
the new broadcast_service_name_tag macro to show the setting.
Because we were redirecting in all cases the error message wasn’t being
shown.
This commit changes the endpoint to respond with content (including an
error message) if the `POST` is not successful.
We want people to be really sure before sending a live broadcast, not
just clicking through the green buttons.
This commit adds a checkbox which explains exactly the consequences of
what they’re about to do, tailored to the channel they’re on, and the
area chosen by the person creating the alert.
Only the test channel has the option to isolate messages to one network.
This commits makes the choices less confusing by only showing the
network choice to those who have selected the test channel.
We have been asked to support the government channel so that:
- it can be tested
- the option to use it is available for the most severe of emergencies,
where the public’s choice to opt-out is outweighed by the widespread
risk to life
It feels quite dangerous that it’s just one click to make an emergency
alerts service live.
This commit adds a confirmation step which explains the consequences of
what you’re about to do.
The current_service.allowed_broadcast_provider is now always "all" or
one of the four providers, which means we can simply the code by not
checking if it is None.
Until all the data is updated to always be "all", we have to handle the
case of provider_restriction being set to None or "all" (which mean the
same thing).
The code can be tidied up once the broadcast provider_restriction is never None.
We're replacing the value of None with the value of all. API has been
updated to accept both values
(1767535def)
so this change starts sending notifications-api the value of "all".
This is the first step in making the UI easier for setting the
options for a broadcast service. Here we remove the options for
"Training mode" test channels. When we create a broadcast message for a trail mode service it is marked as stubbed and does not create a broadcast event that is sent to a provider.
The label for the form and setting page have been updated to reflect the
change.
