Flake8 Bugbear checks for some extra things that aren’t code style
errors, but are likely to introduce bugs or unexpected behaviour. A
good example is having mutable default function arguments, which get
shared between every call to the function and therefore mutating a value
in one place can unexpectedly cause it to change in another.
This commit enables all the extra warnings provided by Flake8 Bugbear,
except for the line length one (because we already lint for that
separately).
It disables:
- _B003: Assigning to os.environ_ because I don’t really understand this
- _B306: BaseException.message is removed in Python 3_ because I think
our exceptions have a custom structure that means the `.message`
attribute is still present
Events should be sorted reverse-chronologically, no matter what order
they come back from the API in, or which field in the API response
they’ve been extracted from.
This commit introduces a slightly hacky way of putting usernames against
events, given that the API only returns user IDs.
It does so without:
- making changes to the API
- making a pages that could potentially fire off dozens of API calls (ie
one per user)
This comes with the limitation that it can only get names for those team
members who are still in the team. Otherwise it will say ‘Unknown’.
In the future the API should probably return the name and email address
for the user who initiated the event, and whether that user was acting
in a platform admin capacity.
At the moment we have two types of event, ‘service’ events and ‘API key’
events. They are munged together which is useful initially, but could
get noisy.
This commit adds filters (copied from the choose template page) that let
users narrow down the list to one of the two types of event. This might
help users get a clearer picture of what’s going on.
Scanning the page is difficult at the moment because it’s hard to tell
how far apart in time events are, and thereby determine which events
might be related.
Grouping the events by day quickly lets users narrow their focus to
a meaningful subset of the events.
We store our audit history in two ways:
1. A list of versions of a service
2. A list of events to do with API keys
In the future there could be auditing data which we want to display that
is stored in other formats (for example the event table).
This commit adds some objects which wrap around the different types of
auditing data, and expose a consistent interface to them. This
architecture will let us:
- write clean code in the presentation layer to display these events on
a page
- add more types of events in the future by subclassing the `Event` data
type, without having to rewrite anything in the presentation layer
Turns out our tests spent a lot of time recreating the app for each test
case, which is quite intense.
This commit makes the fixture sessions level, so the app is only created
once per test session, not once per test function.
This cuts down the time taken to run the test suite to about 50 seconds.
It also makes the tests more parallelizable. Before this change going
from 4 to 8 processes made the tests slower. Now it cuts them down from
about 50 seconds to about 35 seconds[1]. So this commit also lets Pytest
choose the best number of processes to run, since on my machine it
chooses 8, which is the fastest.
Overall this means the
1. With a 2.2GHz quad-core Intel Core i7 processor on a 2015 MacBook Pro
Users who work in local government can’t have GOV.UK branding on their
emails. And only those working for Companies House (for example) can
request the Companies House branding.
This commit adds:
- new choices of email branding, which offer the name of the branding,
rather than the style
- logic to filter this list to only the applicable options, based on
what we know about the user, service and organisation
This is a change from the previous approach which put the onus on users
to figure out the style of branding they wanted, when we might already
know that a lot of the options weren’t available to them, or would be
inconsistent with the branding of other services in their organisation.
of course it's logged in, it's a platform admin
also, reduce use of the `client` fixture in test_platform_admin
(replace it with platform_admin_client)
Query string ordering is non-deterministic. This can cause tests to fail
in a non helpful way when we’re comparing two URLs. The values in the
query string can match, but the string won’t because the order is
different. This commit adds some code to split up the URL and check that
each part of it matches, rather than checking the thing as a whole.
This makes it consistent that an option which contains more options has
a hint about how many options it contains.
Also adds a formatter to get us ready for 1,000 services 🎉
If you’ve come from a template to add a new letter sender then it’s
because you want those words on that template. This commit adds the
extra API call to make that happen.
Organisation team members will be ultimately interested in the detailed
usage of each service, but shouldn't necessarily have access to the
personal data of that services users.
So we should allow these organisation team members to navigate to live
services usage page from the organisation page. They may need to contact
the team so they should also be able to view the team members page.
So they'll then see just usage and team members pages.
If they are actually a team member of the service they're viewing, then
they'll see the full range of options as usual.
This commit implement the above by adding an extra flag to the
`user.has_permissions` decorator which allows certain pages to be marked
as viewable by an organisation user. The default (for all other existing
pages) is that organisation users don’t have permission.
Added a breadcrumb link to a service's organisation to the
withnav_template. This will only show if a service has an organisation
and the current user is also a member of that org, or the current user
is a platform admin user.
Also removed a couple of unused fixtures from the client_request
fixture.
so that platform admins (us) can view pages as regular users do easily.
Simply adds a flag in the session cookie that overrides the actual
platform admin flag on the user model if set. This way it's safe, since
this only downgrades existing functionality, so if someone managed to
alter it they could only get less permissions, not more.
You can change this value from the user profile page if either:
* you're a platform admin
* the flag is set (to any value) on the cookie.
This slightly weird check means that we don't check the underlying
`user._platform_admin` flag anywhere in the code, even when toggling
the suppression.
The API needs the id of the user, not the id of the invite.
The problem with the tests is that the update mock returned a different
user ID than the user it was being passed. So the tests didn’t catch
this.
the api always returns exactly:
```
id
name
email_address
auth_type
current_session_id
failed_login_count
logged_in_at
mobile_number
organisations
password_changed_at
permissions
platform_admin
services
state
```
it does this through `models.py::User.serialize` – there is an old
Marshmallow `user_schema` in `schemas.py` but this isn’t used for
dumping return data, only parsing the json in the create user rest
endpoint.
This means we can rely on these keys always being in the dictionary.
The data flow of other bits of our application looks like this:
```
API (returns JSON)
⬇
API client (returns a built in type, usually `dict`)
⬇
Model (returns an instance, eg of type `Service`)
⬇
View (returns HTML)
```
The user API client was architected weirdly, in that it returned a model
directly, like this:
```
API (returns JSON)
⬇
API client (returns a model, of type `User`, `InvitedUser`, etc)
⬇
View (returns HTML)
```
This mixing of different layers of the application is bad because it
makes it hard to write model code that doesn’t have circular
dependencies. As our application gets more complicated we will be
relying more on models to manage this complexity, so we should make it
easy, not hard to write them.
It also means that most of our mocking was of the User model, not just
the underlying JSON. So it would have been easy to introduce subtle bugs
to the user model, because it wasn’t being comprehensively tested. A lot
of the changed lines of code in this commit mean changing the tests to
mock only the JSON, which means that the model layer gets implicitly
tested.
For those reasons this commit changes the user API client to return
JSON, not an instance of `User` or other models.
At the moment we have to update a YAML file and deploy the change to get
a new domain whitelisted.
We already have a thing for adding new domains – the organisation stuff.
This commit extends the validation to look in the `domains` table on the
API if it can’t find anything in the YAML whitelist.
This has the advantage of:
- not having to deploy code to whitelist a new domain
- forcing us to create new organisations as they come along, so that
users’ services automatically get allocated to the organisation once
their domain is whitelisted
For accessibility reasons a page should have one (and only one) H1. This
commit fixes an instance where the H1 was duplicated as a result of the
work done to componentize our page headings.
It also adds an extra check to `client_request` so that we don’t
introduce pages with multiple or no H1s in the future.
Doing a lookup with `step_index - 1` means that on step `0` we were
looking up `placeholders[-1]`, ie we were making people fill in the last
placeholder first.
Fixing this reintroduces the bug fixed by this pull request:
https://github.com/alphagov/notifications-admin/pull/2551
So this commit also re-fixes that bug but in a different way.
Every time someone adds a new service we ask them what kind of
organisation they work for.
We can look this up based on the user’s email address now. So we should
only ask the question if:
- we don’t know about the organisation
- or we haven’t set what type of organisation it is (this shouldn’t be
possible on productions because we’ve populated the column for all
existing organisations and it’s impossible to add a new one without
setting it
At the moment you have to wait for whoever you’ve invited to accept the
invitation before you can go live. Since this check is mainly for the
benefit of the service, not us, we should trust that people’s intentions
are good when they invite someone.
So this commit also checks the invited users when counting how many team
members a service has.
We’re deprecating storing the domain as text on a branding in favour of
a database relationship between branding and organisation.
We need to do this now in order to remove the validation on these fields
(which depends on the data in `domains.yml`)
Settings looked at `domains.yml` when users were making go live requests
or email branding requests.
This will allow us to remove the `domains.yml` file, by using
information about organisations that is now stored in the database
instead.
Returns the data calculated by the API. Stored in Redis against a
hardcoded key so that no-one hammering the home page is directly hitting
the database.
Instead of using the API client directly views are now calling one
of two Service model methods:
`get_template` is used for view actions, where the user should see
the template page even if they don't have access to the template
folder (since all templates are still inked from the dashboard or
the sent notifications pages).
`get_template_with_user_permission_or_403` will check if the user
has access to the template's folder first and return 403 otherwise.
This method is used for any endpoints that result in an action: editing
template attributes, deleting templates or sending messages.
Added a folder permissions form to the page to invite users to services.
This only shows if the service has 'edit_folder_permissions' enabled,
and all folder checkboxes are checked by default. This change means that
InviteApiClient.create_invite now sends folder_permissions through to
notifications_api (so invites get created with folder permissions).
Started passing the folder_permissions through to notifications-api when
accepting an invite. This changes UserApiClient.add_user_to_service to
send folder_permissions to notifications_api so that new users get folder
permissions when they are added to the service.
We were already invitializing InvitedUser with folder_permissions
(defaulting to None), but this removes the default and adds
folder_permissions to the serialize method. Folder permissions should
now always be returned from api, either as an empty list or a list of
UUIDs.
Currently when you load the ‘edit user’ page (which has a URL like
`/service/<service_id>/users/<user_id>`) we check that:
- you belong to the service represented by `service_id`
- you have permission to edit users on this service
We don’t check that:
- the user represented by `user_id` belongs to this service
This means that if you could somehow determine another user’s `user_id`
(which I don’t think is possible if you don’t already have the manage
service permission for that service) then you could:
- edit their permissions on your service (weird, but wouldn’t have any
effect)
- change their email address (bad)
This commit adds checks to return a `404` any time you’re looking at a
service and trying to do stuff to a user who doesn’t belong to that
service.
We can’t add this check to the API easily because there are still times
that we want to get/modify users outside the context of a service (eg
platform admin pages, or users who have no services).
