Commit Graph

7954 Commits

Author SHA1 Message Date
Ben Thorner
b3f48c1a84 Backup original precompiled uploads to S3
This continues the work from Template Preview [1], so that we have
a complete store of original PDFs to use for testing changes to it.

Previously we did store some originals, but these were only invalid
PDFs that had failed sanitisation; for valid PDFs, the "transient"
bucket only contains the sanitised versions, which the API deletes
/ moves when the notification is sent [2].

Since the notification is only created at a later stage [3], there's
no easy way to get the final name of the PDF we send to DVLA. Instead,
we use the "upload_id", which eventually becomes the notification ID
[4]. This should be enough to trace the file for specific debugging.

Note that we only want to store original PDFs if they're valid (and
virus free!), since there's no point testing changes with bad data.

[1]: https://github.com/alphagov/notifications-template-preview/pull/545
[2]: c44ec57c17/app/service/send_notification.py (L212)
[3]: 7930a53a58/app/main/views/uploads.py (L362)
[4]: 7930a53a58/app/main/views/uploads.py (L373)
2021-06-24 12:21:05 +01:00
Pea Tyczynska
7930a53a58 Merge pull request #3935 from alphagov/more-detailed-request-go-live-ticket
Make a label more descriptive in request-to-go-live ticket
2021-06-18 10:01:50 +01:00
Pea Tyczynska
d1dc1aec6c Make a label more descriptive in request-to-go-live ticket
We have a label saying "other live services". This label means
other live services for a user making the request, but it could
also be interpreted as other live services for an organisation.

Hence, we are changing the label to "other live services for
that user" to avoid confusion
2021-06-17 17:53:28 +01:00
Ben Thorner
562245d9cd Merge pull request #3931 from alphagov/refactor-email-verify-webauthn
Refactor email revalidation check
2021-06-16 16:24:41 +01:00
karlchillmaid
3d158319ee add link state
Co-authored-by: David McDonald <david.mcdonald@digital.cabinet-office.gov.uk>
2021-06-16 13:59:06 +01:00
karlchillmaid
420c5f8365 Update content to make things clearer 2021-06-16 13:22:39 +01:00
karlchillmaid
cd04ca980d Add a line about retention for files sent by email 2021-06-16 11:28:14 +01:00
Ben Thorner
06789b8564 Show an error if user cannot login with WebAuthn
This follows the same approach as for registration [1]. None of
this code is tested, as:

- We don't have a way to test inline JS.
- The risk of this code not working is low.
- We might change the approach in future [2].

[1]: https://github.com/alphagov/notifications-admin/pull/3886
[2]: https://github.com/alphagov/notifications-admin/pull/3886#issuecomment-841128380
2021-06-15 14:04:53 +01:00
Rebecca Law
38edcae68a Merge pull request #3924 from alphagov/letter-permanent-failure
Permanent failure message for letters
2021-06-15 13:43:26 +01:00
Rebecca Law
6bf7c1e216 Merge branch 'letter-permanent-failure' of github.com:alphagov/notifications-admin into letter-permanent-failure 2021-06-15 13:09:57 +01:00
Rebecca Law
d7364eb21a Add permanent-failure to the format_notification_status formatter so the status appears on the activity page. 2021-06-15 13:08:07 +01:00
Ben Thorner
c17d438de8 DRY-up email revalidation check
Previously this was duplicated between the "two_factor" and the
"webauthn" views, and required more test setup. This DRYs up the
check and tests it once, using mocks to simplify the view tests.

As part of DRYing up the check into a util module, I've also moved
the "is_less_than_days_ago" function it uses.
2021-06-14 12:52:54 +01:00
Katie Smith
5eed20cbd2 Move the '- archived' label for orgs to be next to the org name 2021-06-14 12:40:10 +01:00
Ben Thorner
bf2e6802bf Extract login utils out of two_factor view
This better reflects how the code is reused in other views and is
not specific to two factor actions. We have a pattern of testing
utility functionality for each view (as opposed to testing the util
+ the view calls the util), so I'm leaving the tests as-is.
2021-06-14 11:19:06 +01:00
Ben Thorner
3250f2b3ed Move redirect_to_signin helper to new util module
We'll expand this module in later commits.
2021-06-14 11:09:42 +01:00
Ben Thorner
edb0b8214d Move pagination utils into own module
This continues the pattern established in [1], just to chip away a
bit more at the random collection of stuff in utils/__init__.py.

[1]: https://github.com/alphagov/notifications-admin/pull/3923
2021-06-14 11:00:05 +01:00
Ben Thorner
44cf2b16b5 Merge pull request #3923 from alphagov/refactor-email-verify
Split out utils code into separate modules
2021-06-14 10:28:28 +01:00
Chris Hill-Scott
2d36ec8214 Add support for the operator channel
Was just in one of those meetings where it felt like writing this would
take less time than I’d already spent talking about its relative
priority…

---

In the admin app you can already set the broadcast channel as 'test', 'severe' or 'government'.

Aim:
- Add the 'operator' channel to the list of channels you can pick for the admin app broadcast services

Note:
- The API already supports this - https://github.com/alphagov/notifications-api/pull/3262
- The CBC proxy does not yet support the operator channel and this will need a separate card. That card has not yet been written because the interface has not been agreed between us and the MNOs yet.
- Will need to have the ability to select the operator channel for just a single MNO like we do for the other channels
- If we add this, we shouldn't actually start using it until the MNO in question gives us the go ahead.

---

https://www.pivotaltracker.com/story/show/178485177
2021-06-14 08:48:12 +01:00
Katie Smith
cc1a8254df Merge pull request #3928 from alphagov/billing-report
Add total_letters to the billing report
2021-06-11 16:43:29 +01:00
Katie Smith
266f1728e4 Merge pull request #3919 from alphagov/refactor-broadcast-settings
Refactor broadcast settings forms
2021-06-11 16:43:09 +01:00
Katie Smith
c1bfc280b7 Add total_letters to the billing report
This adds an extra column to the report that can be downloaded from
`platform-admin/reports/usage-for-all-services`.
2021-06-11 11:08:33 +01:00
karlchillmaid
e7fdd768e6 Fix typo 2021-06-10 16:43:48 +01:00
Rebecca Law
aedf875110 Improve permenant failure message.
Update delivery status document page.
2021-06-10 16:34:00 +01:00
Ben Thorner
fba8d09875 Move broadcast model code into an explicit module
Previously this was hidden away in an anonymous __init__.py file.
I did think about splitting the models into individual files, like
we do with the top-level models for the app. Since the models are
only imported in one place - i.e. are all used together - it didn't
seem worth the hassle, so I've kept them in one file.
2021-06-10 15:05:38 +01:00
Chris Hill-Scott
e7713de4a5 Merge pull request #3920 from alphagov/refactor-webauthn-model
Refactor User.webauthn_credentials into a ModelList
2021-06-10 11:07:21 +01:00
Rebecca Law
498092f9ac When a letters has passed our validation but is not the postal provider
is unable to print the letter we need to mark the letter as failed.
If we mark the letter as a technical-failure, we say that we will fix
the issue, which is wrong because we can not fix the issue.
If we mark the letter as validation-failed, the letter is in wrong
bucket so the letter is not viewable/downloadable by the client.

This PR updates the message for a letter marked as permanent-failure to
better reflect what has actually happened.
2021-06-10 08:56:14 +01:00
Chris Hill-Scott
f6aa5bdfb8 Refactor User.webauthn_credentials into a ModelList
This saves a bit of repetition, and lets us attach other methods to the
collection, rather than having multiple methods on the user object
prefixed with the same name, or random functions floating about.
2021-06-09 15:21:41 +01:00
Ben Thorner
0326005aeb Extract template / csv utility code into modules
This follows a similar approach to the previous commits, noting that
one module depends on the other, so we have to extract both together.
2021-06-09 15:19:00 +01:00
Ben Thorner
2a4aa8b4e1 Extract letter utility code into own module
This provides more room for expansion, and reduces the amount of
arbitrary code in the __init__.py file for the new package.
2021-06-09 13:59:06 +01:00
Ben Thorner
7c27646d6a Extract user utility code into own module
This provides more room for expansion, and reduces the amount of
arbitrary code in the __init__.py file for the new package.
2021-06-09 13:19:05 +01:00
Ben Thorner
3f55c746aa Turn utils into a module
This provides more room for expansion, so we don't get another
massive file to scroll through. We do also have some top-level
files, such as "formatters.py", which we could consider moving
under utils/ in future.
2021-06-09 12:50:01 +01:00
Chris Hill-Scott
eca3454a39 Merge pull request #3914 from alphagov/prune-email-domains-list
Prune the email domains list
2021-06-09 10:25:27 +01:00
Chris Hill-Scott
f8f718dff8 Set user to sign in with newly-added key 2021-06-08 09:31:30 +01:00
Chris Hill-Scott
45645728c7 Refactor into model
It’s generally an antipattern for the view layer code to be calling the
API client directly.
2021-06-08 09:31:20 +01:00
Ben Thorner
9ada8ad11e Remove redundant code to populate account type
This field is now only used on the confirmation page of the settings
form, where we pre-populate it with the hyphenated string it expects.
2021-06-07 17:51:11 +01:00
Ben Thorner
5bfe5f86de Simplify channel selection using radio buttons
This takes a similar approach as in the previous commit. Since the
"training channel" doesn't really exist, we need some extra code
to pre-select it if a service is already in training mode. As in
the previous commit, I've removed a few non-critical test cases
where we really don't need to test exhaustively.

Note that we also need some specific code to avoid pre-selecting an
option for non-broadcast services, which only used to work by fluke:
we would try to populate the field with (False, None, 'all'), which
isn't a valid combination, so nothing was selected.
2021-06-07 17:51:10 +01:00
Ben Thorner
b38cdcad63 Simplify network choice to optional radio buttons
Previously this field had to mimic the final hyphenated string of
the broadcast account type, even though it was only used to select
one of its components. The new, shorter choices make it easier to
simplify the test for the POST request.

I've also deleted a number of test cases for pre-selected radios.
This functionality isn't critical, so we don't need to exhaustively
test every single possible combination of values.
2021-06-07 17:51:09 +01:00
Ben Thorner
ef8cab7fa4 Simplify network choice form to use boolean radio
This follows the same pattern as in other forms [1].

[1]: 1b459d6692/app/templates/views/organisations/add-gp-organisation.html (L20)
2021-06-07 17:51:09 +01:00
Ben Thorner
5ce76b8b33 Add property to construct account type string
This allows us to start decoupling the form fields from the final,
hyphenated string, which we'll do in the next commits.

Note that I've also removed the conditional that changes the data
of the network field as part of validating it. We shouldn't change
data in validations, and having the new property directly above
makes it clear there's no need for this code.
2021-06-07 17:51:08 +01:00
Ben Thorner
e848643361 Simplify provider selection with '-all' suffix
Previously we had to cope with two forms of the hyphenated string
we use to represent a pending change in broadcast account type.
Using "all" to mean "all providers" matches the behaviour in the
API [1], and means we can remove some complexity.

"training-test-all" isn't ideal, since the provider is irrelevant
for a training mode service. However, this isn't much worse than
the previous "training-test", noting that the channel also has no
relevance. We'll iterate this in later commits.

[1]: 8e1a144f87/migrations/versions/0352_broadcast_provider_types.py (L14)
2021-06-07 17:51:01 +01:00
Leo Hemsted
4ad93a0ea9 add logging when webauthn cred registration fails 2021-06-04 12:52:40 +01:00
Leo Hemsted
e9636119ef set user_verification to discouraged
this is in line with our settings during registration. user verification
involves the browser popping up a PIN prompt. Since the user has already
entered their password correctly to get to this stage, we don't need any
more proof of Something They Know, so there's no need for this.
2021-06-04 12:52:40 +01:00
Leo Hemsted
9fe8666733 add some docstrings for the webauthn endpoints 2021-06-04 12:52:40 +01:00
Leo Hemsted
0993792137 rename verify to complete in api endpoint
it was changed in this PR: https://github.com/alphagov/notifications-api/pull/3260
2021-06-04 12:52:40 +01:00
Leo Hemsted
26ad20719f send people to /two-factor-sms instead of /two-factor
both routes are already valid, however, the link from sign-in sends to
the old link. it fetches whichever URL is second in the route decorator
list when you call `url_for`. Swapping the order around keeps the routes
valid but starts pointing users to the new url.
2021-06-04 12:52:40 +01:00
Leo Hemsted
bb7343d846 pass nextUrl through yubikey flow
the next url comes from sign in via a query param, and needs to go to
the POST /webauthn/authenticate endpoint. That endpoint logs the user
in and returns the redirect to the browser, and will take the next from
the request query params to get there.

also moving the window mocks to beforeEach/afterEach ensures that
promise callbacks from previous tests aren't still associated in future
tests to ensure good test isolation.

unfortunately i couldn't get mocking location for a single js test to
work, but by changing the global config i was able to add some query
params that i can expect to be passed through. Don't love this at all
but not quite sure of a good way round this. I think we're not
practicing very good hygiene and best practices with our mocking and
it's really confounding me here.
2021-06-04 12:52:40 +01:00
Chris Hill-Scott
5c158891aa Prune the email domains list
We only need domains in here which either:
- don’t belong to a single organisation (eg gov.uk)

All other domains should be stored in the database.

This PR removes domains which are now in the database.

Before
---

```sql
select domain from domain where domain in ('gov.uk', 'mod.uk', 'mil.uk', 'd
 dc-mod.org', 'gov.scot', 'parliament.scot', 'parliament.uk', 'nhs.uk', 'nhs.net', 'nhs.scot', 'police.uk', 'scotent.c
 o.uk', 'assembly.wales', 'cjsm.net', 'gov.wales', 'ac.uk', 'sch.uk', 'onevoicewales.wales', 'mtvh.co.uk', 'wmca.org.u
 k', 'suttonmail.org');
 ```

+-----------------+
| domain          |
|-----------------+
| mtvh.co.uk      |
| wmca.org.uk     |
| gov.wales       |
| gov.scot        |
| parliament.uk   |
| assembly.wales  |
| mil.uk          |
| mod.uk          |
| ddc-mod.org     |
| parliament.scot |
| scotent.co.uk   |
+-----------------+

After
---

```sql
select domain from domain where domain in ('gov.uk', 'nhs.uk', 'nhs.ne
 t', 'nhs.scot', 'police.uk', 'cjsm.net', 'ac.uk', 'sch.uk', 'onevoicewales.wales', 'suttonmail.org') ;
```

+----------+
| domain   |
|----------|
+----------+
2021-06-04 11:45:48 +01:00
Chris Hill-Scott
5ea82b0cdc Merge pull request #3911 from alphagov/fix-html-on-old-job-page
Fix HTML showing on old job page
2021-06-03 14:14:55 +01:00
Chris Hill-Scott
64ac12d7c3 Merge pull request #3910 from alphagov/designerise-webauthn-2fa-page
Designerise the page for signing in with a security key
2021-06-03 14:14:47 +01:00
Chris Hill-Scott
a149c6a853 Fix HTML showing on old job page
Using the `Markup` class tells Jinja that the content is safe to render
without any escaping.
2021-06-03 14:01:20 +01:00