The api returns letter details split by postage, so international
letters are returned with a postage of `europe` or `rest-of-world` not
`international` and these rows need to be added together when the rate
is the same before they are displayed on the usage page.
To do this, we need to replace the postage of `europe` and
`rest-of-world` with `international`. The data then needs to be sorted
by postage and rate before the letter units for rows which are
international and have the same rate are added together.
From a question on cross-government Slack:
> re the Usage tab - currently it shows 3 financial years - last year,
> this year and next year. is it possible to replace the "next year" tab
> with something more useful? its always going to be blank! I was
> thinking it would be good to have 2 financial years ago, 1 financial
> year ago and this financial year.
This seems like a reasonable idea, and is something we’ve talked about
before. The original intention[1] was that seeing your (unchanged) free
allowance for next year would be useful, but that doesn’t really seem to
be a user need.
***
1. See https://github.com/alphagov/notifications-admin/pull/1094
> so that you can check what your SMS allowance is going to be before
> you actually get into it
A lot of pages in the admin app are now generated entirely from Redis,
without touching the API.
The one remaining API call that a lot of pages make, when the user is
platform admin or a member of an organisation, is to get the name of
the current service’s organisation.
This commit adds some code to start caching that as well, which should
speed up page load times for when we’re clicking around the admin app
(it’s typically 100ms just to get the organisation, and more than that
when the API is under load).
This means changing the service model to get the organisation from the
API by ID, not by service ID. Otherwise it would be very hard to clear
the cache if the name of the organisation ever changed.
We can’t cache the whole organisation because it has a
`count_of_live_services` field which can change at any time, without an
update being made.
It was saying ‘16 hours ago’ instead of today. This is because, in
strftime:
- `%M` means minute, not month
- `%D` means short MM/DD/YY date, not day of the month
The test wasn’t catching this because the freeze time and mocked value
from the API were set to the same minute.
We’ve done this already for services with the upload letters permission.
And all services can upload letters now.
But we’re still returning it in the JSON response we use to AJAX-ify the
page.
Since the jobs response can query stats for up to 50 jobs at a time this
puts some load on the API/database. Hopefully this might drop that load
a bit.
Scheduled jobs push everything else on the dashboard down, which makes
them very prominent. This is exacerbated by people scheduling more jobs
simultaneously than we expected when we originally designed the feature.
We also want to remove all jobs from the dashboard, in favour of putting
them on the uploads page.
So this commit replaces them with one of our new dashboard banners (used
for received text messages in returned letters) which summarises:
- how many scheduled jobs you have
- when the first one is going out (i.e. how long you have to stop it, if
you need to)
Currently you have no way of getting to the returned letter page. This
commit adds a link to it from the dashboard, following the pattern of
the new received text messages banner.
This commit adds a new kind of banner to the dashboard for summarising
things you might need to action.
This way we’ll be able to have multiple instances of this banner on the
same page without it looking too intense.
I never really liked the big blue banner for inbound text messages
because it became the most prominent thing on the page. It was an
interim solution that let us ship the feature until we had something
better.
This test looks for how many times the string `50` appears in the text
of the `<main>` element of the page. The `<main>` element also contains
some times, for example 1:23pm.
This means that when the time reaches 1:50pm, 2:50pm, etc the number of
times the string `50` appears in the page changes. Which causes the test
assertions to fail.
This follows the pattern of what we’ve done with services, users and
events.
It gives us a way of neatly instantiating a model for each item in the
list we get back from the API and reduces the complexity of the view
layer code.
Now is a good time to do this because we’re going to be making a bunch
of changes to the jobs pages, and those changes will be easier to code
and understand with a sensible model behind them.
We were using user fixtures in a lot of parameterized tests, but this is
no longer allowed in Pytest 5. To avoid having to split up the parametrized
tests (which would make the test files a lot longer and slightly more
difficult to read) this commit creates functions which return various types
of user json so that we can use these as the test parameters instead.
This reverts 1b1839ad30, which removed
the usage from the dashboard because it was causing performance
problems:
> **The yearly usage section on the dashboard page takes too log as a
> result services with large yearly stats are timing out.**
>
> As a short term fix we have taken the yearly stats off the dashboard.
>
> There is a plan to create permanent statistic tables to warehouse the
> data.
The long term fix (the fact tables) is now in place, so it should be OK
to bring this back.
This is part of a wider piece of work to refresh the dashboard page now
that jobs are moving to their own page.
If you’ve only used one template then this section of the page isn’t
doing its job, which is to show a comparison of the different kinds of
message you’re showing.
I think our initial assumption was that everyone would be using multiple
templates, so it was good to show this part of the page during the
onboarding, to show users where the information was going to appear.
But we have lots of services who only send one template now, typically
where they’re populating the contents of the template themselves. In
which case this part of the page doesn’t offer them any value.
We hid letters originally because it wasn’t a mature feature. We rolled
it out by letting teams choose to use it (#1803)
and then automatically giving it to new teams (notifications-api/#1600).
This commit doesn’t change who has access to letters, but it does make
it more discoverable by revealing it in the UI. This is the same thing we do for emails/texts, where even if you switch them off they still show up on the dashboard and usage
page.
Even if your service doesn’t send letters now, it might have done
previously.
The original reason for hiding letters was because it wasn’t a mature
feature. But now that it is, we should make it discoverable even for
existing teams. So that means not conditionally hiding it.
This is the same thing we do for emails/texts, where even if you switch
them off they still show up on the dashboard and usage page.
For scheduled files we say ‘sending today at 5:00pm’ or ‘sending
tomorrow at 11:00am’. But once we’ve started processing these files we
say ‘Sent 27 September at 5:00pm’. This makes it sound like 27 September
is not today.
This commit makes the dates shown on the dashboard consistent, by saying
‘today’ and ‘yesterday’ instead of absolute dates.
of course it's logged in, it's a platform admin
also, reduce use of the `client` fixture in test_platform_admin
(replace it with platform_admin_client)
The uploads hub is just a page with text for now - there are no actions
available on the page. It is linked to from a new 'Uploads' menu item on
the left of the page which is only visible if your service has the
`letter` and `upload_letters` permissions and if the current user has
permissions to send messages.
Platform admin users can still see the organisation breadcrumbs for
trial mode services, but others uses can only see organisation
breadcrumbs for live services.
Added a breadcrumb link to a service's organisation to the
withnav_template. This will only show if a service has an organisation
and the current user is also a member of that org, or the current user
is a platform admin user.
Also removed a couple of unused fixtures from the client_request
fixture.
The data flow of other bits of our application looks like this:
```
API (returns JSON)
⬇
API client (returns a built in type, usually `dict`)
⬇
Model (returns an instance, eg of type `Service`)
⬇
View (returns HTML)
```
The user API client was architected weirdly, in that it returned a model
directly, like this:
```
API (returns JSON)
⬇
API client (returns a model, of type `User`, `InvitedUser`, etc)
⬇
View (returns HTML)
```
This mixing of different layers of the application is bad because it
makes it hard to write model code that doesn’t have circular
dependencies. As our application gets more complicated we will be
relying more on models to manage this complexity, so we should make it
easy, not hard to write them.
It also means that most of our mocking was of the User model, not just
the underlying JSON. So it would have been easy to introduce subtle bugs
to the user model, because it wasn’t being comprehensively tested. A lot
of the changed lines of code in this commit mean changing the tests to
mock only the JSON, which means that the model layer gets implicitly
tested.
For those reasons this commit changes the user API client to return
JSON, not an instance of `User` or other models.
It:
- saves repetetive boilerplate code
- does some extra checks (eg checking for a `200` response)
- makes the codebase less confusing to consistently do the same thing in
the same way
The new API response for template statistics returns separate
count for each status. We get rid of template stats for cancelled
notifications and group the rest of the statuses together.
At the moment the dashboard does two API calls to find out if a service
has:
1. Scheduled jobs
2. Normal jobs
API calls are slow because they are synchronous, go over the network and
touch the database. We can’t cache these API calls because:
- a scheduled job could become a normal job at any time
- the statistics on a normal job are constantly updating
However there are plenty of services which don’t have any jobs, and
probably never will. And finding out if a service has any jobs is
reliably cacheable (because as soon as a service creates its first job
it has some jobs).
So this commit:
- refactors the way we get scheduled/normal jobs into the job_api_client
to make the view a bit slimmer
- makes an additional, Redis-wrapped call to find out if any jobs exist
before trying to get the jobs
This should result in a speedup on the dashboard, and can be used in the
future if there’s anywhere else we want to show or hide something
depending on whether a service has created any jobs (I have some ideas).
