Commit Graph

3133 Commits

Author SHA1 Message Date
karlchillmaid
dcc38b5619 Change previous to past 2021-07-05 13:46:04 +01:00
Ben Thorner
1f33924ceb Send upload_id to Template Preview for logging
This means we can include the anonymous ID for the file in the log
we have about Type3 fonts [1]. Currently, we have no way of tracing
manually uploaded files with this potential defect.

[1]: https://github.com/alphagov/notifications-template-preview/pull/557
2021-07-01 12:09:47 +01:00
Leo Hemsted
71613dd942 remove old /two-factor endpoint and update test names
we redirect people to `/two-factor-sms` since #26ad20719
2021-06-29 16:19:24 +01:00
Leo Hemsted
126f9cf6be fix bug stopping editing of permissions of webauthn platform admins
We hide the radio field in the HTML for platform admins, as we don't
want anyone to be able to change their auth type. However, when the form
is validated, the form has a field called login_authentication that it
expects a value for. It silently fails as it complains that when the
user POSTed they didn't select a value for that radio field, but the
error message is on the radio fields that don't get displayed to the
user so they'd never know.

Fixing this is actually pretty hard.

We use this form in two places, one where we have a user to edit, one
where we are creating an invite from scratch. So sometimes we don't know
about a user's auth type. In addition, radio buttons are mandatory by
design, but now sometimes we don't just want to make it optional but
explicitly ignore the value being passed in? To solve this, remove the
field entirely from the form if the user is a platform admin. This means
that if the code in manage_users.py tries to access the
login_authentication value from the form, it'll error, but I think
that's okay to leave for now given we concede that this isn't a perfect
final solution.

The tests didn't flag this previously as they tried to set from sms_auth
(the default for `platform_admin_user`) TO email_auth or sms_auth. Also,
the diagnosis of this bug was confounded further by the fact that
`mock_get_users_by_service` sets what is returned by the API - the
service model then takes the IDs out of that response and calls
`User.get_user_by_id` for the matching ID (as in, the code only uses
get_users_by_service to ensure the user belongs to that service). This
means that we accidentally set the form editing the current user, as
when we log in we set `get_user_by_id` to return the user of our choice
2021-06-29 15:53:48 +01:00
Leo Hemsted
92b6885224 ensure webauthn page aborts if user isn't allowed 2021-06-29 15:53:48 +01:00
Rebecca Law
5534ecb5a4 Merge pull request #3939 from alphagov/check-daily-limit-for-csv-uploads
Check the daily limit get the daily notification_count from redis.
2021-06-29 14:41:22 +01:00
Rebecca Law
77c2aa9fd6 Stop passing in today_only for the get_service_statistics method.
We now only ever call it with False.
To remove it from the api call will require a change in the API so will do that at another time.
2021-06-29 07:33:40 +01:00
Ben Thorner
e7c1ce1869 Merge pull request #3938 from alphagov/save-original-pdf-uploads
Backup original precompiled uploads to S3
2021-06-24 12:43:40 +01:00
Ben Thorner
b3f48c1a84 Backup original precompiled uploads to S3
This continues the work from Template Preview [1], so that we have
a complete store of original PDFs to use for testing changes to it.

Previously we did store some originals, but these were only invalid
PDFs that had failed sanitisation; for valid PDFs, the "transient"
bucket only contains the sanitised versions, which the API deletes
/ moves when the notification is sent [2].

Since the notification is only created at a later stage [3], there's
no easy way to get the final name of the PDF we send to DVLA. Instead,
we use the "upload_id", which eventually becomes the notification ID
[4]. This should be enough to trace the file for specific debugging.

Note that we only want to store original PDFs if they're valid (and
virus free!), since there's no point testing changes with bad data.

[1]: https://github.com/alphagov/notifications-template-preview/pull/545
[2]: c44ec57c17/app/service/send_notification.py (L212)
[3]: 7930a53a58/app/main/views/uploads.py (L362)
[4]: 7930a53a58/app/main/views/uploads.py (L373)
2021-06-24 12:21:05 +01:00
Katie Smith
1862cd0223 Update data format for platform admin report
The format of dates returned from API for the "Monthly notification
statuses for live services" report has changed, so we no longer need to
format the dates here.

https://github.com/alphagov/notifications-api/pull/3279
2021-06-23 16:14:58 +01:00
Rebecca Law
44f02f2e30 To check the daily limit get the daily notification_count from redis.
The daily limit cache is set by the api when a notification is created. There is one cache key per service per day and it expires after 24 hours.
2021-06-23 15:56:05 +01:00
Pea Tyczynska
7930a53a58 Merge pull request #3935 from alphagov/more-detailed-request-go-live-ticket
Make a label more descriptive in request-to-go-live ticket
2021-06-18 10:01:50 +01:00
Pea Tyczynska
d1dc1aec6c Make a label more descriptive in request-to-go-live ticket
We have a label saying "other live services". This label means
other live services for a user making the request, but it could
also be interpreted as other live services for an organisation.

Hence, we are changing the label to "other live services for
that user" to avoid confusion
2021-06-17 17:53:28 +01:00
Ben Thorner
c17d438de8 DRY-up email revalidation check
Previously this was duplicated between the "two_factor" and the
"webauthn" views, and required more test setup. This DRYs up the
check and tests it once, using mocks to simplify the view tests.

As part of DRYing up the check into a util module, I've also moved
the "is_less_than_days_ago" function it uses.
2021-06-14 12:52:54 +01:00
Ben Thorner
bf2e6802bf Extract login utils out of two_factor view
This better reflects how the code is reused in other views and is
not specific to two factor actions. We have a pattern of testing
utility functionality for each view (as opposed to testing the util
+ the view calls the util), so I'm leaving the tests as-is.
2021-06-14 11:19:06 +01:00
Ben Thorner
3250f2b3ed Move redirect_to_signin helper to new util module
We'll expand this module in later commits.
2021-06-14 11:09:42 +01:00
Ben Thorner
edb0b8214d Move pagination utils into own module
This continues the pattern established in [1], just to chip away a
bit more at the random collection of stuff in utils/__init__.py.

[1]: https://github.com/alphagov/notifications-admin/pull/3923
2021-06-14 11:00:05 +01:00
Ben Thorner
44cf2b16b5 Merge pull request #3923 from alphagov/refactor-email-verify
Split out utils code into separate modules
2021-06-14 10:28:28 +01:00
Chris Hill-Scott
2d36ec8214 Add support for the operator channel
Was just in one of those meetings where it felt like writing this would
take less time than I’d already spent talking about its relative
priority…

---

In the admin app you can already set the broadcast channel as 'test', 'severe' or 'government'.

Aim:
- Add the 'operator' channel to the list of channels you can pick for the admin app broadcast services

Note:
- The API already supports this - https://github.com/alphagov/notifications-api/pull/3262
- The CBC proxy does not yet support the operator channel and this will need a separate card. That card has not yet been written because the interface has not been agreed between us and the MNOs yet.
- Will need to have the ability to select the operator channel for just a single MNO like we do for the other channels
- If we add this, we shouldn't actually start using it until the MNO in question gives us the go ahead.

---

https://www.pivotaltracker.com/story/show/178485177
2021-06-14 08:48:12 +01:00
Katie Smith
cc1a8254df Merge pull request #3928 from alphagov/billing-report
Add total_letters to the billing report
2021-06-11 16:43:29 +01:00
Katie Smith
266f1728e4 Merge pull request #3919 from alphagov/refactor-broadcast-settings
Refactor broadcast settings forms
2021-06-11 16:43:09 +01:00
Katie Smith
c1bfc280b7 Add total_letters to the billing report
This adds an extra column to the report that can be downloaded from
`platform-admin/reports/usage-for-all-services`.
2021-06-11 11:08:33 +01:00
Chris Hill-Scott
f6aa5bdfb8 Refactor User.webauthn_credentials into a ModelList
This saves a bit of repetition, and lets us attach other methods to the
collection, rather than having multiple methods on the user object
prefixed with the same name, or random functions floating about.
2021-06-09 15:21:41 +01:00
Ben Thorner
0326005aeb Extract template / csv utility code into modules
This follows a similar approach to the previous commits, noting that
one module depends on the other, so we have to extract both together.
2021-06-09 15:19:00 +01:00
Ben Thorner
2a4aa8b4e1 Extract letter utility code into own module
This provides more room for expansion, and reduces the amount of
arbitrary code in the __init__.py file for the new package.
2021-06-09 13:59:06 +01:00
Ben Thorner
7c27646d6a Extract user utility code into own module
This provides more room for expansion, and reduces the amount of
arbitrary code in the __init__.py file for the new package.
2021-06-09 13:19:05 +01:00
Chris Hill-Scott
f8f718dff8 Set user to sign in with newly-added key 2021-06-08 09:31:30 +01:00
Chris Hill-Scott
45645728c7 Refactor into model
It’s generally an antipattern for the view layer code to be calling the
API client directly.
2021-06-08 09:31:20 +01:00
Ben Thorner
9ada8ad11e Remove redundant code to populate account type
This field is now only used on the confirmation page of the settings
form, where we pre-populate it with the hyphenated string it expects.
2021-06-07 17:51:11 +01:00
Ben Thorner
5bfe5f86de Simplify channel selection using radio buttons
This takes a similar approach as in the previous commit. Since the
"training channel" doesn't really exist, we need some extra code
to pre-select it if a service is already in training mode. As in
the previous commit, I've removed a few non-critical test cases
where we really don't need to test exhaustively.

Note that we also need some specific code to avoid pre-selecting an
option for non-broadcast services, which only used to work by fluke:
we would try to populate the field with (False, None, 'all'), which
isn't a valid combination, so nothing was selected.
2021-06-07 17:51:10 +01:00
Ben Thorner
b38cdcad63 Simplify network choice to optional radio buttons
Previously this field had to mimic the final hyphenated string of
the broadcast account type, even though it was only used to select
one of its components. The new, shorter choices make it easier to
simplify the test for the POST request.

I've also deleted a number of test cases for pre-selected radios.
This functionality isn't critical, so we don't need to exhaustively
test every single possible combination of values.
2021-06-07 17:51:09 +01:00
Ben Thorner
ef8cab7fa4 Simplify network choice form to use boolean radio
This follows the same pattern as in other forms [1].

[1]: 1b459d6692/app/templates/views/organisations/add-gp-organisation.html (L20)
2021-06-07 17:51:09 +01:00
Ben Thorner
5ce76b8b33 Add property to construct account type string
This allows us to start decoupling the form fields from the final,
hyphenated string, which we'll do in the next commits.

Note that I've also removed the conditional that changes the data
of the network field as part of validating it. We shouldn't change
data in validations, and having the new property directly above
makes it clear there's no need for this code.
2021-06-07 17:51:08 +01:00
Ben Thorner
e848643361 Simplify provider selection with '-all' suffix
Previously we had to cope with two forms of the hyphenated string
we use to represent a pending change in broadcast account type.
Using "all" to mean "all providers" matches the behaviour in the
API [1], and means we can remove some complexity.

"training-test-all" isn't ideal, since the provider is irrelevant
for a training mode service. However, this isn't much worse than
the previous "training-test", noting that the channel also has no
relevance. We'll iterate this in later commits.

[1]: 8e1a144f87/migrations/versions/0352_broadcast_provider_types.py (L14)
2021-06-07 17:51:01 +01:00
Leo Hemsted
4ad93a0ea9 add logging when webauthn cred registration fails 2021-06-04 12:52:40 +01:00
Leo Hemsted
e9636119ef set user_verification to discouraged
this is in line with our settings during registration. user verification
involves the browser popping up a PIN prompt. Since the user has already
entered their password correctly to get to this stage, we don't need any
more proof of Something They Know, so there's no need for this.
2021-06-04 12:52:40 +01:00
Leo Hemsted
9fe8666733 add some docstrings for the webauthn endpoints 2021-06-04 12:52:40 +01:00
Leo Hemsted
26ad20719f send people to /two-factor-sms instead of /two-factor
both routes are already valid, however, the link from sign-in sends to
the old link. it fetches whichever URL is second in the route decorator
list when you call `url_for`. Swapping the order around keeps the routes
valid but starts pointing users to the new url.
2021-06-04 12:52:40 +01:00
Chris Hill-Scott
5ea82b0cdc Merge pull request #3911 from alphagov/fix-html-on-old-job-page
Fix HTML showing on old job page
2021-06-03 14:14:55 +01:00
Chris Hill-Scott
a149c6a853 Fix HTML showing on old job page
Using the `Markup` class tells Jinja that the content is safe to render
without any escaping.
2021-06-03 14:01:20 +01:00
Chris Hill-Scott
2a62d6dfb8 Add a success message when security key registered
This makes it clear that there’s nothing more the user needs to do,
until the next time they sign in.
2021-06-03 13:59:43 +01:00
David McDonald
0fcb7778ac Merge pull request #3893 from alphagov/allow-provider-all-channels
Allow setting provider for any channel
2021-06-03 09:36:43 +01:00
David McDonald
2d40208fec Merge pull request #3894 from alphagov/webauthn-login-python-tests
Webauthn login
2021-06-02 15:30:36 +01:00
Leo Hemsted
73a444b33a rename webauthn auth functions
_complete_webauthn_authentication -> _verify_webauthn_authentication

This function just does verification of the actual auth process -
checking the challenge is correct, the signature matches the public key
we have stored in our database, etc.

verify_webauthn_login -> _complete_webauthn_login_attempt

This function doesn't do any actual verification, we've already verified
the user is who they say they are (or not), it's about marking the
attempt, either unsuccessful (we bump the failed_login_count in the db)
or successful (we set the logged_in_at and current_session_id in the
db).

This change also informs changes to the names of methods on the user
model and in user_api_client.
2021-06-02 12:06:10 +01:00
Leo Hemsted
0ec92e8c2f DRY attested webauthn creds data 2021-06-02 12:06:10 +01:00
Leo Hemsted
e864100be7 make sure error message flashes work properly
flashes are consumed by the jinja template calling get_flashed_messages
in flash_messages.html.

When you call `abort(403)` the 403 error page is rendered, with the
flashed message on it. However, the webauthn endpoints just return that
page to the ajax `fetch`, which ignores the response and just reloads
the page.

Instead of calling abort, we can just return an empty response body and
the 403 error code, so that the flashed messages stay in the session and
will be rendered when the `GET /two-factor-webauthn` request happens
after the js reloads the page.
2021-06-02 12:06:09 +01:00
Leo Hemsted
a3870af87d allow password reset with webauthn login flow 2021-06-02 12:06:09 +01:00
Leo Hemsted
d05f127e41 return 200 to js instead of 302 when logging in
the js fetch function is really not designed to work with 302s. when it
receives a 302, it automatically follows it and fetches the next page.
This is awkward because I don't want js to do all this in ajax, I want
the browser to get the new URL so it can load the page.

A better approach is to view the admin endpoint as a more pure API: the
js sends a request for authentication to the admin app, and the admin
app responds with a 200 indicating success, and then a payload of
relevant data with that.

The relevant data in this case is "Which URL should I redirect to", it
might be the user's list of services page, or it might be a page telling
them that their email needs revalidating.
2021-06-02 11:51:12 +01:00
Leo Hemsted
c29f87f55d increment failed login count on unsuccesful webauthn login
this doesn't include timeouts or other errors on the browser side - the
main thing this catches is if the token doesn't belong to the user.
However I'm not entirely clear if that's something that will be caught
at this point, or if the browser would reject that key as it's not in
the credentials passed in to the begin_authentication process.
2021-06-02 11:51:11 +01:00
Leo Hemsted
92f78b14fe redirect on login; flash errors on failure
the js `fetch` function will follow redirects blindly and return you the
final 200 response. when there's an error, we don't want to go anywhere,
and we want to use the flask `flash` functionality to pop up an error
page (the likely reason for seeing this is using a yubikey that isn't
associated with your user). using `flash` and then
`window.location.reload()` handles this fine.

However, when the user does log in succesfully we need to properly log
them in - this includes:

* checking their account isn't over the max login count
* resetting failed login count to 0 if not
* setting a new session id in the database (so other browser windows are
  logged out)
* checking if they need to revalidate their email access (every 90 days)
* clearing old user out of the cache

This code all happens in the ajax function rather than being in a
separate redirect, so that you can't just navigate to the login flow. I
wasn't able to unit test that function due how it uses the session and
other flask globals, so moved the auth into its own function so it's
easy to stub out all that CBOR nonsense.

TODO: We still need to pass any `next` URLs through the chain from login
page all the way through the javascript AJAX calls and redirects to the
log_in_user function
2021-06-02 11:51:10 +01:00