This reverts 1b1839ad30, which removed
the usage from the dashboard because it was causing performance
problems:
> **The yearly usage section on the dashboard page takes too log as a
> result services with large yearly stats are timing out.**
>
> As a short term fix we have taken the yearly stats off the dashboard.
>
> There is a plan to create permanent statistic tables to warehouse the
> data.
The long term fix (the fact tables) is now in place, so it should be OK
to bring this back.
This is part of a wider piece of work to refresh the dashboard page now
that jobs are moving to their own page.
If you’ve only used one template then this section of the page isn’t
doing its job, which is to show a comparison of the different kinds of
message you’re showing.
I think our initial assumption was that everyone would be using multiple
templates, so it was good to show this part of the page during the
onboarding, to show users where the information was going to appear.
But we have lots of services who only send one template now, typically
where they’re populating the contents of the template themselves. In
which case this part of the page doesn’t offer them any value.
We hid letters originally because it wasn’t a mature feature. We rolled
it out by letting teams choose to use it (#1803)
and then automatically giving it to new teams (notifications-api/#1600).
This commit doesn’t change who has access to letters, but it does make
it more discoverable by revealing it in the UI. This is the same thing we do for emails/texts, where even if you switch them off they still show up on the dashboard and usage
page.
Even if your service doesn’t send letters now, it might have done
previously.
The original reason for hiding letters was because it wasn’t a mature
feature. But now that it is, we should make it discoverable even for
existing teams. So that means not conditionally hiding it.
This is the same thing we do for emails/texts, where even if you switch
them off they still show up on the dashboard and usage page.
For scheduled files we say ‘sending today at 5:00pm’ or ‘sending
tomorrow at 11:00am’. But once we’ve started processing these files we
say ‘Sent 27 September at 5:00pm’. This makes it sound like 27 September
is not today.
This commit makes the dates shown on the dashboard consistent, by saying
‘today’ and ‘yesterday’ instead of absolute dates.
of course it's logged in, it's a platform admin
also, reduce use of the `client` fixture in test_platform_admin
(replace it with platform_admin_client)
The uploads hub is just a page with text for now - there are no actions
available on the page. It is linked to from a new 'Uploads' menu item on
the left of the page which is only visible if your service has the
`letter` and `upload_letters` permissions and if the current user has
permissions to send messages.
Platform admin users can still see the organisation breadcrumbs for
trial mode services, but others uses can only see organisation
breadcrumbs for live services.
Added a breadcrumb link to a service's organisation to the
withnav_template. This will only show if a service has an organisation
and the current user is also a member of that org, or the current user
is a platform admin user.
Also removed a couple of unused fixtures from the client_request
fixture.
The data flow of other bits of our application looks like this:
```
API (returns JSON)
⬇
API client (returns a built in type, usually `dict`)
⬇
Model (returns an instance, eg of type `Service`)
⬇
View (returns HTML)
```
The user API client was architected weirdly, in that it returned a model
directly, like this:
```
API (returns JSON)
⬇
API client (returns a model, of type `User`, `InvitedUser`, etc)
⬇
View (returns HTML)
```
This mixing of different layers of the application is bad because it
makes it hard to write model code that doesn’t have circular
dependencies. As our application gets more complicated we will be
relying more on models to manage this complexity, so we should make it
easy, not hard to write them.
It also means that most of our mocking was of the User model, not just
the underlying JSON. So it would have been easy to introduce subtle bugs
to the user model, because it wasn’t being comprehensively tested. A lot
of the changed lines of code in this commit mean changing the tests to
mock only the JSON, which means that the model layer gets implicitly
tested.
For those reasons this commit changes the user API client to return
JSON, not an instance of `User` or other models.
It:
- saves repetetive boilerplate code
- does some extra checks (eg checking for a `200` response)
- makes the codebase less confusing to consistently do the same thing in
the same way
The new API response for template statistics returns separate
count for each status. We get rid of template stats for cancelled
notifications and group the rest of the statuses together.
At the moment the dashboard does two API calls to find out if a service
has:
1. Scheduled jobs
2. Normal jobs
API calls are slow because they are synchronous, go over the network and
touch the database. We can’t cache these API calls because:
- a scheduled job could become a normal job at any time
- the statistics on a normal job are constantly updating
However there are plenty of services which don’t have any jobs, and
probably never will. And finding out if a service has any jobs is
reliably cacheable (because as soon as a service creates its first job
it has some jobs).
So this commit:
- refactors the way we get scheduled/normal jobs into the job_api_client
to make the view a bit slimmer
- makes an additional, Redis-wrapped call to find out if any jobs exist
before trying to get the jobs
This should result in a speedup on the dashboard, and can be used in the
future if there’s anywhere else we want to show or hide something
depending on whether a service has created any jobs (I have some ideas).
we're not actually looking at the detailed service aspects - just
the stats. We're doing this in three places:
* dashboard
* notification activity page
* when checking jobs to see if we're over the daily limit
change these places to use a new api endpoint (service/id/statistics),
which hopefully be a little more performant, and will definitely be a
little more organised - moving away from generic endpoints with loads
of optional parameters.
We still need the detailed endpoints for the platform admin page tho.
Depends on https://github.com/alphagov/notifications-api/pull/1865
we were seeing isort produce different outputs locally and in docker -
this was due to it having different opinions about whether the tests
module (ie all our unit tests) is a first party (local) or third party
(pip installed) import. It's a first party import, so by defining this
in the setup.cfg isort settings, we can force it to be consistent
between environments.
Note: I don't know why it was different in the first place though
Done using isort[1], with the following command:
```
isort -rc ./app ./tests
```
Adds linting to the `run_tests.sh` script to stop badly-sorted imports
getting re-introduced.
Chosen style is ‘Vertical Hanging Indent’ with trailing commas, because
I think it gives the cleanest diffs, eg:
```
from third_party import (
lib1,
lib2,
lib3,
lib4,
)
```
1. https://pypi.python.org/pypi/isort
It’s weird that `/services/<service_id>` returns `404`. The home page
for every service is the dashboard – should be possible to get there
from any other page just by stemming the URL.
Also makes it consistent with organisations, which will have
`/organisations/<org_id>`.
- Updated tests and added a new mock_get_monthly_template_usage
- Deleted get_monthly_template_statistics_for_service
- Added new test to test the redirection of the old endpoint
Numbers over a billion overflow the two column layout. Numbers over one
hundred thousand overflow the three column layout.
This commit makes the type size smaller in these cases, so that the
numbers still fit in the boxes.
At least one of our providers gives us messages with special characters
escaped, ie a newline comes through as `\n`, not a literal newline. We
shouldn’t be showing these backslashes to any of our users.
Python has built in codecs for dealing with encoding/decoding of
strings – see
https://docs.python.org/3/library/codecs.html#text-encodings
for details. Using these builtins is safer than trying to do anything
regex or parsing-based.
