I prefer to avoid `assert_not_called`, because if I make a typo like
this, the tests will still pass:
```
app.invite_api_client.create_invite.asset_not_called()
```
It’s harder to have a false positive with the statement written this
way:
```
assert app.invite_api_client.create_invite.called is False
```
We had two identical callback form classes. One for delivery callbacks
and one for inbound sms callbacks. Since they did not differ I consolidated
them into one CallbackForm class that they both inherited from previously.
I also substituted field classes for this form with new fields
that cooperate with gov uk frontend.
ListEntry component uses FieldList field to group
textboxes. Textboxes can be text inputs, email fields
or international phone number fields. This converts
all field-lists to use:
- GovukTextInputField
- GovukEmailField
- InternationalPhoneNumber
Affects these forms:
- OrganisationDomainsForm
- GuestList
Also changes to related Javascript:
Update list-entry JS tests to match new HTML
Updates the HTML the JS operates on in the test
(a fixture representing the HTML in the page on
load) to match the new GOVUK Frontend we are
generating.
Make list-entry JS work with GOVUK Frontend HTML
The existing list-entry JS did a few things that
clashed with how the new HTML works:
- added a 'input-' prefix to the id attributes
of all text-inputs
- did not make its name and id attributes values
match
The new HTML has id and name attributes that
match so these changes remove the prefix for id
attributes and makes them match the name
attribute.
To understand these changes, it is useful to
know how the values for id and name attributes are
generated:
1. the id attribute for the component element is
stored
2. the 'list-entry-' prefix is removed and the
remainder is used to generate ids
For example, if the component's id is
'list-entry-domains', the id will be 'domains-1',
where the text-input is the first one.
This also adds some logic to the HoganJS template
to make the value attribute optional, so it is
only added if it has a non-null value. This
matches the behaviour of the text-input component
used in the new list-entry component.
Also change whitelist references to guestlist in tests
- we forgot to do it earlier, when we moved from calling this
feature whitelist to calling it guestlist.
This commit changes all the places where a user would see the term
‘whitelist’ in the content of page to say guestlist instead.
We’re removing the term ‘whitelist’ for two reasons. The first reason
is that we agree with the National Cyber Security Centre say:
> It's fairly common to say whitelisting and blacklisting to describe
> desirable and undesirable things in cyber security. For instance, when
> talking about which applications you will allow or deny on your
> corporate network; or deciding which bad passwords you want your users
> not to be able to use.
> However, there's an issue with the terminology. It only makes sense if
> you equate white with 'good, permitted, safe' and black with 'bad,
> dangerous, forbidden'. There are some obvious problems with this. So
> in the name of helping to stamp out racism in cyber security, we will
> avoid this casually pejorative wording on our website in the future.
> No, it's not the biggest issue in the world - but to borrow a slogan
> from elsewhere: every little helps.
– https://www.ncsc.gov.uk/blog-post/terminology-its-not-black-and-white
The second reason is that we’ve observed some users think that they have
to put recipients in the whitelist even when they’re already with in the
team. We think that the term ‘whitelist’ might be reinforcing this
mental model because of how ‘whitelists’ might work in other
applications.
We considered the following alternatives or concepts:
- Development
- Recipients
- Sandbox
- Extended team
- Smoke test recipients
- Allowed
- Nominated
- Bonus
- Additional
- Safe
- Team list
- Trusted contacts
- Designated people
- Guest list
- Team key list
We also considered not giving it a name, and explaining it as a nuance
of how the team key works. After mocking this up it felt more disjoined.
We think it’s still useful for the thing to have a name so that it’s
easy to refer to between the docs and the UI.
We like the term ‘guest list’ because:
- of how it sits with team members – members and guests in the abstract
- a guest list is a concept that a lot of people will be familiar with
– a list of people who can access a thing
- ‘guest’ is very different to ‘recipient’ – we want to mitigate any
confusion between this and the (emergency) contact lists
flask_login sets a bunch of variables in the session object. We only use
one of them, `user_id`. We set that to the user id from the database,
and refer to it all over the place.
However, in flask_login 0.5.0 they prefix this with an underscore to
prevent people accidentally overwriting it etc. So when a user logs in
we need to make sure that we set user_id manually so we can still use
it.
flask_login sets a bunch of variables on the `flask.session` object.
However, this session object isn't the one that gets passed in to the
request context by flask - that one can only be modified outside of
requests from within the session_transaction context manager (see [1]).
So, flask_login populates the normal session and then we need to copy
all of those values across.
We didn't need to do this previously because we already set the
`user_id` value on line 20 of tests/__init__.py, but now that
flask_login is looking for `_user_id` instead we need to do this
properly.
[1] https://flask.palletsprojects.com/en/1.1.x/testing/#accessing-and-modifying-sessions
flask_login sets a bunch of variables in the session object. We only use
one of them, `user_id`. We set that to the user id from the database,
and refer to it all over the place.
However, in flask_login 0.5.0 they prefix this with an underscore to
prevent people accidentally overwriting it etc. So when a user logs in
we need to make sure that we set user_id manually so we can still use
it.
flask_login sets a bunch of variables on the `flask.session` object.
However, this session object isn't the one that gets passed in to the
request context by flask - that one can only be modified outside of
requests from within the session_transaction context manager (see [1]).
So, flask_login populates the normal session and then we need to copy
all of those values across.
We didn't need to do this previously because we already set the
`user_id` value on line 20 of tests/__init__.py, but now that
flask_login is looking for `_user_id` instead we need to do this
properly.
[1] https://flask.palletsprojects.com/en/1.1.x/testing/#accessing-and-modifying-sessions
Flake8 Bugbear checks for some extra things that aren’t code style
errors, but are likely to introduce bugs or unexpected behaviour. A
good example is having mutable default function arguments, which get
shared between every call to the function and therefore mutating a value
in one place can unexpectedly cause it to change in another.
This commit enables all the extra warnings provided by Flake8 Bugbear,
except for the line length one (because we already lint for that
separately).
It disables:
- _B003: Assigning to os.environ_ because I don’t really understand this
- _B306: BaseException.message is removed in Python 3_ because I think
our exceptions have a custom structure that means the `.message`
attribute is still present
The data flow of other bits of our application looks like this:
```
API (returns JSON)
⬇
API client (returns a built in type, usually `dict`)
⬇
Model (returns an instance, eg of type `Service`)
⬇
View (returns HTML)
```
The user API client was architected weirdly, in that it returned a model
directly, like this:
```
API (returns JSON)
⬇
API client (returns a model, of type `User`, `InvitedUser`, etc)
⬇
View (returns HTML)
```
This mixing of different layers of the application is bad because it
makes it hard to write model code that doesn’t have circular
dependencies. As our application gets more complicated we will be
relying more on models to manage this complexity, so we should make it
easy, not hard to write them.
It also means that most of our mocking was of the User model, not just
the underlying JSON. So it would have been easy to introduce subtle bugs
to the user model, because it wasn’t being comprehensively tested. A lot
of the changed lines of code in this commit mean changing the tests to
mock only the JSON, which means that the model layer gets implicitly
tested.
For those reasons this commit changes the user API client to return
JSON, not an instance of `User` or other models.
Since we added template folders the templates page has had a ‘medium’
sized heading, where other pages have stuck with a ‘large’ size.
This commit rationalises the decision around which pages have which
heading size:
- ‘navigation’ pages (eg templates, team members, email reply to
addresses) have medium sized headings
- transactional pages (ie ones which have a green button) keep the
larger heading size
The Design System has standardised on back links being at the top of the
page, decorated with a small text-coloured arrow.
I think this makes more sense than having them at the bottom, because it
suggests, in some way, being able to go back before commiting to any of
the forms on the page. Whereas the things at the bottom of the page
should be performing actions on what’s in the page.
The reason for making this change now is that it de-clutters the area
around the green buttons. This was presenting a design challenge where
multiple levels of interaction were happening in the same form. Moving
these back links to the top of the page should mean that, in these
complicated forms, there’s one fewer thing to compete for the user’s
attention.
I’ve componentised this into a `page_header` macro so that the change is
easier to roll out and maintain.
It:
- saves repetetive boilerplate code
- does some extra checks (eg checking for a `200` response)
- makes the codebase less confusing to consistently do the same thing in
the same way
Currently the order of API keys seems to be non-deterministic:
d46caa184e/app/dao/api_key_dao.py (L32-L39)
Generally we sort things alphabetically, unless there’s a good reason to
do otherwise.
This commit sorts the API keys alphabetically.
It’s redundant to make two API calls here, one to get all keys and one
to get a single key. Since the API calls are sequential we can speed
things up by getting the one key from the list of all keys.
Once all our users have upgraded to the latest clients they won’t need
this. The latest clients only use the combined key and service ID.
Discuss: when can we safely remove it?
Pytest is deprecating the direct calling of fixtures. One fixture that
we call directly quite a lot is `fake_uuid`. Since it just returns the
value of `sample_uuid()` we can either call that instead (where we need
a fixed value) or generate a new UUID each time (where a fixed value is
not needed).
It’s useful to be able to see what the email or text message looks like,
especially if you’ve sent it with a test API key (so it isn’t in your
inbox or on your phone). We already have the page for this, so we just
need to link to it.
This is better than just keying into the JSON because it means you get
an exception straight away when looking up a key that doesn’t exist
(which via mocking you could ordinarily miss).
We’ve had a user who’s said:
> Seems configured callbacks cannot be removed once they’re set as the
> fields have a presence check. Is that intentional?
This means it’s not working as they expect. Rather than have to go and
change stuff in the database for them, let’s make it work as they’d
expect.
Only lets you clear the form if you remove both the token and the URL.
we were seeing isort produce different outputs locally and in docker -
this was due to it having different opinions about whether the tests
module (ie all our unit tests) is a first party (local) or third party
(pip installed) import. It's a first party import, so by defining this
in the setup.cfg isort settings, we can force it to be consistent
between environments.
Note: I don't know why it was different in the first place though
test_api_keys.py and test_api_integration.py were almost identical
files with only a few lines difference between them. By moving one
test we can now delete test_api_keys.py
Precompiled letters can now have two additional states:
* pending-virus-check
* virus-scan-failed
Both new states should show in the notifications dashboard, and
virus-scan-failed should appear as an error state, with a descriptive
message. You should not be able to preview a letter in one of the two
new states, so the preview link has been removed for precompiled letters
in these states.
Done using isort[1], with the following command:
```
isort -rc ./app ./tests
```
Adds linting to the `run_tests.sh` script to stop badly-sorted imports
getting re-introduced.
Chosen style is ‘Vertical Hanging Indent’ with trailing commas, because
I think it gives the cleanest diffs, eg:
```
from third_party import (
lib1,
lib2,
lib3,
lib4,
)
```
1. https://pypi.python.org/pypi/isort
Both `<button type='submit'>Submit<button>` and
`<input type='submit' value='Submit'>` can be used to submit a form.
We have historically[1] used `<input>` because it’s better-supported by
IE6 in that:
- the `submit` attribute is mandatory on `<button>`, not on `<input>`
- the `innerHTML` of a button will be submitted to the server, not the
value (as in other browsers)
Reasons to now use `<button>` instead:
- IE6/7 support is no longer a concern (especially with deprecation of
TLS 1.0 on the way)
- Because an `<input>` element can’t have children, the pseudo-element
hack[2] used to ensure the top edge of the button is clickable doesn’t
work. We’re seeing this bug[3] affect real users in research.
1. We inhereted our buttons from Digital Marketplace, here is me making
that change in their code: 8df7e2e79e (diff-b1420f7b7a25657d849edf90a70ef541)
2. 24e1906c0d (diff-ef0e4eb6f1e90b44b0c3fe39dce274a4R79)
3. https://github.com/alphagov/govuk_elements/issues/545
