The endpoint for setting permissions in api will now be used for both
user permissions and a user's folder permissions, so this changes the
format of the data we pass through.
when clients are defined in app/__init__.py, it increases the chance of
cyclical imports. By moving module level client singletons out to a
separate extensions file, we stop cyclical imports, but keep the same
code flow - the clients are still initialised in `create_app` in
`__init__.py`.
The redis client in particular is no longer separate - previously redis
was set up on the `NotifyAdminAPIClient` base class, but now there's one
singleton in `app.extensions`. This was done so that we can access redis
from outside of the existing clients.
This is a platform admin page which lists all letter brands and their
domain. Once the page to edit a letter brand has been created, each
brand will link to its edit page, but for now this is just a list of
brand names and domains.
This removes some code which is duplicative and obscure (ie it’s not
very clear why we do `"a" * 73` even though there is a Very Good Reason
for doing so).
We were getting all letter logos from a method in the email branding
client. Since we will be adding more client methods to deal with
letters, it makes things clearer to separate the email and letter
branding clients.
This commit adds logic to:
- take the list of selected folders and templates
- split it into two lists (of folders and templates)
- `POST` that data to the API, to effect the movement of said folders
and templates
I’ve tried to architect it in such a way that we can easily add more
template ‘operations’ in the future, as we add more forms to the choose
template page.
The add new templates page now has option to add template folders.
Tweaked wording of other options and h1 to clarify options since it's
not all about templates any more.
Added api client and stuff for it
This commit is the first step to disentangling the models from the API
clients. With the models in the same folder as the API clients it makes
it hard to import the API clients within the model without getting a
circular import.
After this commit the user API clients still has this problem, but at
least the service API client doesn’t.
`_get_current_service` is a function which gets called every time
`current_service` is referenced in a view method or Jinja template.
Because the service model was getting initialised inside this function
it was being reconstructed many times in one request. On the service
settings page, for example, it was getting initialised 43 times, adding
about 200ms to the response time.
This commit moves its initialisation to the point where we’re getting
the data from the API, which only happens once per request.
Pytest is deprecating the direct calling of fixtures. One fixture that
we call directly quite a lot is `fake_uuid`. Since it just returns the
value of `sample_uuid()` we can either call that instead (where we need
a fixed value) or generate a new UUID each time (where a fixed value is
not needed).
If we’re going to be referring to email branding as part of the service
creation journey then we should make sure it doesn’t slow things down
too much by adding an extra API call. Caching things in Redis is a way
of avoiding unneeded API calls.
Allows getting notification counts for a given number of days to
support services with custom data retention periods (admin
dashboard page should still display counts for the last 7 days,
while the notifications page displays all stored notifications).
Calling `.set()` with `True` stores the byte string `'True'` which
cannot subsequently be decoded from JSON (because boolean values in
JSON are lowercase, ie `true`).
At the moment the dashboard does two API calls to find out if a service
has:
1. Scheduled jobs
2. Normal jobs
API calls are slow because they are synchronous, go over the network and
touch the database. We can’t cache these API calls because:
- a scheduled job could become a normal job at any time
- the statistics on a normal job are constantly updating
However there are plenty of services which don’t have any jobs, and
probably never will. And finding out if a service has any jobs is
reliably cacheable (because as soon as a service creates its first job
it has some jobs).
So this commit:
- refactors the way we get scheduled/normal jobs into the job_api_client
to make the view a bit slimmer
- makes an additional, Redis-wrapped call to find out if any jobs exist
before trying to get the jobs
This should result in a speedup on the dashboard, and can be used in the
future if there’s anywhere else we want to show or hide something
depending on whether a service has created any jobs (I have some ideas).
In API, the endpoint for the new platform admin stats page has been
moved to a platform stats blueprint. This means we now need a platform
stats client.
* Added a new method to the ComplaintApiClient to get the total
complaints by date range from the API.
* Added a new method to the ServiceAPIClient to get the new platform
admin stats data from the API.
A new platform admin page Email complaints has been added to surface those complaints.
Eventually the complaints will be visible to the services so they can remove the email address from their mailing list.
Next thing to implement is "x email complaints" warning on the platform admin summary page.
we're not actually looking at the detailed service aspects - just
the stats. We're doing this in three places:
* dashboard
* notification activity page
* when checking jobs to see if we're over the daily limit
change these places to use a new api endpoint (service/id/statistics),
which hopefully be a little more performant, and will definitely be a
little more organised - moving away from generic endpoints with loads
of optional parameters.
We still need the detailed endpoints for the platform admin page tho.
Depends on https://github.com/alphagov/notifications-api/pull/1865
we were seeing isort produce different outputs locally and in docker -
this was due to it having different opinions about whether the tests
module (ie all our unit tests) is a first party (local) or third party
(pip installed) import. It's a first party import, so by defining this
in the setup.cfg isort settings, we can force it to be consistent
between environments.
Note: I don't know why it was different in the first place though
Redis is giving us a big performance boost (it’s roughly halved the
median request time on the admin app).
Once we’re confident that it’s working properly[1] we can eke out a bit
more performance from it by keeping the caches alive for longer. As
far as I can tell we’re still using Redis in a very low-volume way[2],
so increasing the number of things we’re storing shouldn’t start taxing
our Redis server at all. But reducing the number of times we have to
hit the API to refresh the cache _should_ result in some performance
increase.
---
1. ie we’re not seeing instances of stale caches not being invalidated
2. We have 2.5G of available space in Redis. Here is our current usage:
```
used_memory:7728960
used_memory_human:7.37M
used_memory_rss:7728960
used_memory_peak:16563776
used_memory_peak_human:15.79M
used_memory_lua:37888
```
This is easier to read than having to understand the arguments 1…n of
the cache decorator are ‘magic’, and gives us more flexibility about
how the cache keys are formatted, eg being able to add words in the
middle of them.
Also changes the key format for all templates to be
`service-{service_id}-templates` instead of `templates-{service_id}`
because then it’s clearer what the ID represents.
