This is a quick additional check to protect the user:
- From getting a CloudFront 502 error if the file takes too
long to upload. I was surprised to find it takes about 1 minute
to upload a 70Mb file to S3.*
- From getting a CloudFront 502 error when we follow the redirect
and run through the slow processing code in utils that builds a
RecipientCSV [1].
For context, a CSV with 100K rows and a few columns is around 5Mb,
so a 10Mb limit should be enough. Analysis over the past week shows
that the vast majority of CSV uploads are actually < 2.5Mb.
I haven't added any tests for this because:
- The check isn't critical, as the worst case scenario is the user
gets a worse error than this in-app one.
- There's no easy way to mock the validation, and I didn't want to
have a test that depends on a 10Mb+ file.
*We're using "key.put" to upload the file, when we could be doing
a multipart upload [2]. However, I tried this myself with a chunk
size of 1000 bytes and found it only led to a marginal improvement.
[1]: https://github.com/alphagov/notifications-utils/pull/930
[2]: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/s3-uploading-files.html
The original idea behind was to always ask users to re-enter their
password any time:
- we want them to be sure that they want to do what they’re about to do
- we want to be sure it’s really the user trying to do the thing (and
not someone malicious)
In reality we:
- removed this from the initial place it was added (a descendent of the
‘suspend service’ feature)
- only ever added it to the ‘rename service’ feature
So in reality it’s not a pattern we have persisted with. Arguably there
are several things you can now do in the admin app without re-entering
your password which are much more high consequence than changing the
service name.
Also, with browser autofill there’s a lot less chance that forcing
someone to re-enter a password really gives much defence against an
unatteneded laptop, for example.
I also wonder whether we might get people to give better service names
if we make the process of renaming the service less intimidating.
So this commit removes the need to re-enter your password when renaming
a service.
Note that re-naming an organisation still has the same check, but I
haven’t removed that too for the sake of keeping scope of the PR small.
WTForms versions less than 3.0.0 have a security vulnerability where
arbitrary HTML can be inserted into the label of a form, allowing the
possibility of a cross-site scripting attack.
I don’t know if there’s anywhere we put user-generated content into form
labels but it’s possible we are vulnerable somewhere.
This require moving some imports because as of
https://github.com/wtforms/wtforms/pull/614/files
there is no longer a separate module for HTML 5 fields, they are now
considered core fields.
As of https://github.com/wtforms/wtforms/issues/445/files custom
implementations of `pre_validate` or `post_validate` must raise
`ValidationError` to trigger a validation message, where we were raising
`ValueError` this was no longer being caught.
As of https://github.com/wtforms/wtforms/pull/355/files `StringField`
returns `None` for empty data, not `''` but our `validate_email_address`
function only accepts strings.
We think that the API is returning incorrect data for this column.
It’s going to take a while to figure out what’s going on with the
queries in the API, so this pull request temporarily removes the column
so we’ve not giving people incorrect data.
This means that the data in the report will match what’s on the page,
where the values are rounded to the nearest penny.
This uses the same string formatting to round the numbers which the
`big_number` component does, so it should round the numbers in the same
way.
This is so org level users can use this data easier for things
like determining spending per service.
We do not include sms fragments sent column and remove other sms columns
consistency.
Do not add sms fragments sent column for now until we agree on an
unambiguous name for it. The data in this column is sms billing units
multiplied by international sms weighing. My favourite for a clear
name would be 'text message credits used', but we need a naming
strategy for this.
This makes it clearer that this model collection isn’t the organisations
for a user or a service or some other entity, like most model
collections are.
It will also lets us make a separate Organisations model, without the
name conflicting.
https://www.pivotaltracker.com/story/show/180026726
There was a bug where if you enter an invalid email address in
to the edit reply to email address form and click save, the
form you get shown with your error message will always contain
the field to set as default the reply to and also delete. This should
not have been the case. If you make an error on the form when
changing a reply to that is already a default, then you should not
be given the chance to change it to not default, nor should you
be able to delete it.
This commit fixes that bug by making sure the additional form fields
are only shown if the reply to being changed is not the default.
Jinja is a better language for doing complex templating. And we can use
the global Jinja scope to automatically get access to things like
`current_user` and our formatters.
In https://github.com/alphagov/notifications-admin/pull/3663/files we
made specific routes for sending the ‘tour’ text message, rather than
sharing the ‘one-off’ routes in `send.py`.
This commit moves the final route in the tour journey into `tour.py` as
well, which is where I expected to find it when I was looking for it
just now.
There are no links to the `webauthn_begin_register` route - you are only
taken there if you are logged in and have clicked to register a key.
However, we have seen this route being crawled by bots making a GET
request which gives a `500` status code error because there isn't a
logged in current_user. For consistency, this also adds teh decorator to
the POST route.
The feedback endpoints use `ticket_type` to decide what to display and
whether or not a ticket should be escalated. We were using the
ticket_type as the value for the Zendesk ticket_type. However, the Zendesk
API accepts 4 values for its ticket_type and these are different from
the ticket_type values we use in our code.
This change converts the Notify ticket_type value to a valid Zendesk
ticket_type value when creating a Notify feedback ticket.
The new way of creating support tickets can be seen in
[notifications-utils](https://github.com/alphagov/notifications-utils/pull/899).
This changes tickets created when making a request to go live to use
the new way, while other tickets stay the same for now.
The go live tags have been removed. Some of these had become
unneccessary since you can't make the request to go live unless they are
true (e.g. `notify_go_live_email_reply_to`). Others will always get
added by a Zendesk macro when the ticket is replied to, so we don't need
to add them here.
The radio buttons to select the type of service - central, etc. -
are only shown if we can't infer the type based on the user's email
/ default organisation. However, the code to render the page in the
error case didn't accommodate this, nor did it show the version of
the page for adding a local government service.
This fixes the bug by DRYing-up the logic to render the pages. I've
not added a test for this for a couple of reasons:
- It's not a critical bug: no one has complained about it and it
doesn't block the user from adding service.
- It's unlikely to reoccur because the bug involved writing _more_
code than was necessary.
- It's not trivial to test this due to the 3 versions of the page
involved - these are tested for the happy path.
since we are hard-coding a generic error message on the front-end, we
have no need to do anything on the back end. This is also nice as it
standardises the two flows to behave more like each other (rather than
previously where one would `flash` an error message and the other would
return CBOR for the js to decode).
Note that the register flow returns 400 while the auth flow returns 403.
The js for both just checks `response.ok` so will handle both. The JS
completely discards any body returned if the status isn't 200 now.
lets us keep cabinet office financials safe in the credentials repo
the dict in the creds repo will either be an empty dict or a full dict,
so the env var on paas will always contain some parseable json. But
locally it might not, so if it's not set at all then default to the
string `null` so the json parsing doesn't throw a wobbly.
Accepting an invite means that you’ve just clicked a link in your email
inbox. This shows that you have access to your email.
We can make a record of this, thereby extending the time before we ask
you to revalidate your email address.
When someone uses a fresh password reset link they have proved that they
have access to their inbox.
At the moment, when revalidating a user’s email address we wait until
after they’ve put in the 2FA code before updating the timestamp which
records when they last validated their email address[1].
We can’t think of a good reason that we need the extra assurance of a
valid 2FA code to assert that the user has access to their email –
they’ve done that just by clicking the link. When the user clicks the
link we already update their failed login count before they 2fa. Think
it makes sense to handle `email_access_validated_at` then too.
As a bonus, the functional tests never go as far as getting a 2FA code
after a password reset[2], so the functional test user never gets its
timestamp updated. This causes the functional tests start failing after
90 days. By moving the update to this point we ensure that the
functional tests will keep passing indefinitely.
1. This code in the API (91542ad33e/app/dao/users_dao.py (L131))
which is called by this code in the admin app (9ba37249a4/app/utils/login.py (L26))
2. 5837eb01dc/tests/functional/preview_and_dev/test_email_auth.py (L43-L46)
While testing alerts on these channels the MNOs sometimes need to
restart their CBCs to make sure everything is failing over properly.
If the CBC does not come back up, for whatever reason, then we are left
in a state where the alert can’t be cancelled.
To minimise the impact to the public in this scenario we should keep the
expiry time at 4 hours for alerts sent on test channels. We recently
increased it back up to 24 hours for all channels, so this in effect is
reverting that change for channels that won’t be used in a real
emergency.
We don't use this term consistently and it's not defined anywhere.
Since most of the Admin app deals with user-facing permssions, it's
OK to just use the term "permissions". Where both types of permission
are present in the same file, we can more clearly distinguish them
as "UI permissions" and "DB permissions".
These are more than a list of permissions: each item includes the
label to use when displaying it as an option on a form. Switching
to a name that reflects how the attributes are used will help to
avoid confusion when we rename some of the other attributes in the
same file in later commits.
This is so we can allow the sender name 'UC' for DWP.
Note, this is specifically only straight single quotes and not curly
quotes or double quotes. Curly quotes are not supported in the GSM
character set (https://en.wikipedia.org/wiki/GSM_03.38). There is
currently no defined user ask to support double quotes in sms sender
names.
I have tested this by sending a message through both Firetext and MMG to
make sure they both support the single quote character in SMS sender
names.
DWP also have had no particular issues using the SMS sender name with
their existing system in the past either.
The `send_messages` permission has been deprecated for use with
broadcast services, so we can drop support for it in the code. We
were supporting both the old permissions and new permissions
(`create_broadcasts` and `approve_broadcasts`) while we switched people
over.
This removes `send_messages` from the `user_has_permissions` decorator
around the broadcast routes and from the page to view a broadcast and
broadcast dashboards. We can now git rid of a lot of the parameterization
that was temporarily added to the tests.
At the moment we say that you either ‘add’ an alert or ‘send’ it.
This is confusing because:
- an alert isn’t received on people’s phones until it’s approved, so
this is really when it is ‘sent’ conceptually
- an alert can be rejected before anyone receives it, so the UI can say
an alert that no-one ever received was sent
This commit re-labels things so that the the first part of the process
is ‘creating’ the alert.
This makes all the permissions nice and distinct from each other. Adding
templates and adding alerts feel conceptually quite different things
(what are you adding the alert to?).
