It looks like, by default, Flask no longer makes full URLs, for example
`https://example.com/path`. Instead it does `/path`. This will still
work fine, and if anything is better because it reduces the number of
bytes of HTML we are sending.
It won’t mean that requests go over `http` instead of `https` without
the protocol because we set the appropriate HSTS header here:
0c57da7781/ansible/roles/paas-proxy/templates/admin.conf.j2 (L11)
This commit changes all our tests to reflect that URLs no longer have
the protocol and domain in them. `_external=True` is Flask’s way of
saying whether a URL should be generated with the domain and protocol
(`True`) or without it (`False`).
Again, I can’t find the changelog or diff where this was introuduced,
but if you’d like to go spelunking then here’s a starting point:
50374e3cfe/src/flask/helpers.py (L192)
make it consistent - always use a + to concatenate strings, always have
that + at the end of the line.
(also remove a rogue "when" from some test names)
Having to submit the form for each choice separately slowed us down
during an incident where Redis was unavailable and came back with
stale data, which we had to clear manually.
Note: we don't want to use the "flush" feature in case there are other
keys in Redis, which may not be safe to remove.
This will make it easier to add another test / feature to clear all
the cache keys. It's debatable which of "sum" and "max" is useful:
- "max" is a better (although still not accurate) indicator of the
number of "things" affected e.g. templates, services, etc.
- "sum" makes sense in places where "max" doesn't e.g. when we clear
the "organisations" group, which doesn't equate to individual orgs.
Using "sum() ... across" seems like a reasonable compromise and makes
it clear that we're iterating over different kinds of keys.
While the pluralisation is nice, I don't think it's worth the effort
to make it work for both "object(s)" and "format(s)".
This field caused some confusion and lots of unnecessary work
to our colleague because of unclear name.
The field was named sms_fragments, where in fact the value of
the field is: those sms fragments that go above free allowance
multiplied by the rate multiplier.
The new name was chosen through consultation with colleagues
who use billing report the most.
We have a `client_request` fixture which does a bunch of useful stuff
like:
- checking the status code of the response
- returning a `BeautifulSoup` object
Lots of our tests still use an older fixture called `client`. This is
not as good because it:
- returns a raw `Response` object
- doesn’t do the additional checks
- means our tests contain a lot of repetetive boilerplate like `page = BeautifulSoup(response.data.decode('utf-8'), 'html.parser')`
This commit converts all the tests which had a `client.get(…)` or
`client.post(…)` statement to use their equivalents on `client_request`
instead.
Subsequent commits will remove uses of `client` in other tests, but
doing it this way means the work can be broken up into more manageable
chunks.
We added a new argument to `client_request.get` and
`client_request.post` to specify that it should return a raw `Response`
object rather than an instance of `BeautifulSoup`.
This is useful because sometimes we need to look at stuff like the
response headers.
However it turns out we already have a separate method for this, so
rather than invent something new I think it’s better to stick with the
thing we already have.
We have a `client_request` fixture which does a bunch of useful stuff
like:
- checking the status code of the response
- returning a `BeautifulSoup` object
For most tests of a platform admin view we used `platform_admin_client`
instead. This is not as good because it returns a raw `Response` object
and doesn’t do the additional checks.
This commit converts all the tests using `platform_admin_client` to:
use new `client_request` and log in as `platform_admin_user` before
making any requests.
This is also nice because it makes any test easy to parametrize with
additional users, for example to test differences in behaviour dependant
on being platform admin or not.
We used to upload this to performance platform to show the list of
services and organisations.
There is no longer a performance platform to upload this file to.
An accessiblity audit done as part of Notify's
service assessment raised the following problem
with our big_number component.
When you turn CSS off, the sentence in the
component is split onto separate lines.
This was because the number part is wrapped in a
<div> which browsers were interpreting as being a
separate sentence to the label.
So "1 letter", where "letter" is the label, was
seen as:
"1"
"letter"
The accessibility expert consulted on this pointed
out that this would sound confusing for users of
screen readers when moving through the document
sentence by sentence.
These changes:
- make the <div>s into <span>s which are 'phrasing
content' and so are interpreted as part of the
same sentence
- change the CSS so the number will still sit
on top of its label text
The HTML5 spec has a section on how browsers
should arrange text into paragraphs that explains
what was happening in more detail:
https://www.w3.org/TR/html52/dom.html#paragraphs
It’s not clear where 103 is coming from unless you know that the code is
doing the slightly unintuitive thing of displaying the max of all return
values.
When we add a new property to the broadcast model, we need to delete any
cached broadcasts from Redis that are missing the new property. So this
adds an option to do this to the cache page in platform admin.
I’ve also tried to make it more obvious what the magic numbers in the
test fixture are doing.
I prefer to avoid `assert_not_called`, because if I make a typo like
this, the tests will still pass:
```
app.invite_api_client.create_invite.asset_not_called()
```
It’s harder to have a false positive with the statement written this
way:
```
assert app.invite_api_client.create_invite.called is False
```
Changes those fields in the following forms:
- DateFilterForm
- RequiredDateFilterForm
Includes changes to templates that use this form
and associated tests.
This should:
- make the page load faster because it has to render less HTML for each
service
- make the page easier to scan for services that are sending lots of
text messages or letters
We used to scan this page to look for services with high failure rates,
and the design of the page was gear towards this. Now we have alerting
for high failure rates, so the page can focus on volumes instead.
This commit also puts the service name above the statistics, so that
long service names don’t break the layout of the page.
This involves three changes which broke our code.
To validate email addresses, the optional dependency `email-validator`
must be installed<sup>1</sup>. But since we don’t use WTForms’ email
validation, we shouldn’t need to subclass it – it can just be its own
self contained thing. Then we don’t need to add the extra dependency.
When rendering textareas, and extra `\r\n` is inserted at the beginning
<sup>2</sup>. Browsers will strip this when displaying the textbox and
submitting the form, but some of our tests need updating to account for
this.
The error message for when you don’t choose an option from some radio
buttons has now changed. Rather than just accepting WTForms’ new
message, this commit makes the error messages like the examples from
the Design System<sup>3</sup>. By default it will say ‘Select an
option’, but by passing in an extra parameter (`thing`) it can be
customised to be more specific, for example ‘Select a type of
organisation’.
***
1. https://github.com/wtforms/wtforms/pull/429
2. https://github.com/wtforms/wtforms/issues/238
3. https://design-system.service.gov.uk/components/radios/#error-messages
A lot of pages in the admin app are now generated entirely from Redis,
without touching the API.
The one remaining API call that a lot of pages make, when the user is
platform admin or a member of an organisation, is to get the name of
the current service’s organisation.
This commit adds some code to start caching that as well, which should
speed up page load times for when we’re clicking around the admin app
(it’s typically 100ms just to get the organisation, and more than that
when the API is under load).
This means changing the service model to get the organisation from the
API by ID, not by service ID. Otherwise it would be very hard to clear
the cache if the name of the organisation ever changed.
We can’t cache the whole organisation because it has a
`count_of_live_services` field which can change at any time, without an
update being made.
This page is slow to load which means:
- it’s annoying for us
- it’s potentially causing load on the database
This commit does two things to reduce the amount we’re unnecessarily
looking at this page:
1. Avoid redirecting to it when signing in as a platform admin user
2. Don’t go directly to it when clicking ‘platform admin’ at the top,
but instead show a holding page (there’s a fair chance you’ve clicked
that link in order to go and manage some email branding or find a
user, not wait for stats to load)
We’re going to start using the returned letters summary to show some
info on the dashboard.
This means we will be accessing it more often than it changes. And we
know exactly when it changes because it’s us manually submitting the
references we get from DVLA.
This makes it a good candidate for being cached, and Redis is where we
cache stuff that we’d otherwise go to the API for.
We can use the ‘Uploads’ feature to check if letters are printable now.
This code works in a completely different way, so if we kept it we’d
have to maintain two different code paths, and make sure that they
didn’t diverge.
Also deletes the related HTML templates.
Show valdiation failed messages on letter notifications in red text,
not in the banner like we do on Uploads and Validation checker pages.
This is because it is a different step in the journey: the user
has already sent the notification and styling needs to be in line
with other places where user is checking the notification she already
has sent.
We now use the pattern of showing a box at the top of the page with the
error. The error message has a heading and can have additional details.
Error messages and the invalid pages get stored in the S3 metadata.
Added a form to upload a single letter. Currently this only uses the
form to validate that a file is submitted and that the file is a PDF. If
either of these validations fail, the form will display an error.
Otherwise, we redirect to a new preview page which just has the filename
as the heading for now.
of course it's logged in, it's a platform admin
also, reduce use of the `client` fixture in test_platform_admin
(replace it with platform_admin_client)
This report will be used by the engagement team. There is a form to give
a start and end date for the report, and the form is then downloaded
as a CSV file when the form is submitted.
We accidentally miss these sometimes. This code adds a test which
inspects the code to automatically check that any function which:
- handles a route
- accepts a service_id
For each function it checks that each of these routes have the
permissions decorator we’d expect.
Most of the introspection/AST code is adapted from here:
https://mvdwoord.github.io/exploration/2017/08/18/ast_explore.html
This should make the ‘All organisations’ page load a lil’ bit quicker.
Still worth caching the domains separately so the response is smaller
when we only care about domains. This is because the code that uses the
domains is part of the sign up flow, so it’s really important that it’s
snappy.
