By default our AJAX calls were 2 seconds. Then they were 5 seconds
because someone reckoned 2 seconds was putting too much load on the
system. Then we made them 10 seconds while we were having an incident.
Then we made them 20 seconds for the heaviest pages, but back to 5
seconds or 2 seconds for the rest of the pages.
This is not a good situation because:
- it slows all services down equally, no matter how much traffic they
have, or which features they have switched on
- it slows everything down by the same amount, no matter how much load
the platform is under
- the values are set based on our worst performance, until we manually
remember to switch them back
- we spend time during incidents deploying changes to slow down the
dashboard refresh time because it’s a nothing-to-lose change that
might relieve some symptoms, when we could be spending time digging
into the underlying cause
This pull request makes the Javascript smarter about how long it waits
until it makes another AJAX call. It bases the delay on how long the
server takes to respond (as a proxy for how much load the server is
under).
It’s based on the square root of the response time, so is more sensitive
to slow downs early on, and less sensitive to slow downs later on. This
helps us give a more pronounced difference in delay between an AJAX call
that is fast (for example the page for a single notification) and one
that is slow (for example a dashboard for a service with lots of
traffic).
*Some examples of what this would mean for various pages*
Page | Response time | Wait until next AJAX call
---|---|---
Check a reply to address | 130ms | 1,850ms
Brand new service dashboard | 229ms | 2,783ms
HM Passport Office dashboard | 634ms | 5,294ms
NHS Coronavirus Service dashboard | 779ms | 5,977ms
_Example of the kind of slowness we’ve seen during an incident_ | 6,000ms | 18,364ms
GOV.UK email dashboard | `HTTP 504` | 😬
Dashboard is the most intensive page we AJAX, and also the highest
traffic one. We’ve already slowed it from 2 to 5 seconds, this slows it
further to 20 seconds to reduce the load.
This leaves other pages (for example looking at a single job) at the
platform-level default of 5 seconds, because we think they cause less
load and the real-timelyness isn’t critical to people’s business
processes.
For looking at a single notification we know from research that someone
sending these one-at-a-time often waits to see if they’re delivered,
so let’s bring this back down to the previous value of 2 seconds.
We’ve done this already for services with the upload letters permission.
And all services can upload letters now.
But we’re still returning it in the JSON response we use to AJAX-ify the
page.
Since the jobs response can query stats for up to 50 jobs at a time this
puts some load on the API/database. Hopefully this might drop that load
a bit.
The default is 2 seconds and this will mean that we are halving traffic
for these ajax calls which can only be good for trying to limit queries
on the database.
I think the user impact on this will likely not be noticable.
Debatable whether we should up them all even further to 10 seconds but
this is definitely a quick although maybe small win.
This follows the pattern of what we’ve done with services, users and
events.
It gives us a way of neatly instantiating a model for each item in the
list we get back from the API and reduces the complexity of the view
layer code.
Now is a good time to do this because we’re going to be making a bunch
of changes to the jobs pages, and those changes will be easier to code
and understand with a sensible model behind them.
This reverts 1b1839ad30, which removed
the usage from the dashboard because it was causing performance
problems:
> **The yearly usage section on the dashboard page takes too log as a
> result services with large yearly stats are timing out.**
>
> As a short term fix we have taken the yearly stats off the dashboard.
>
> There is a plan to create permanent statistic tables to warehouse the
> data.
The long term fix (the fact tables) is now in place, so it should be OK
to bring this back.
This is part of a wider piece of work to refresh the dashboard page now
that jobs are moving to their own page.
If you’ve only used one template then this section of the page isn’t
doing its job, which is to show a comparison of the different kinds of
message you’re showing.
I think our initial assumption was that everyone would be using multiple
templates, so it was good to show this part of the page during the
onboarding, to show users where the information was going to appear.
But we have lots of services who only send one template now, typically
where they’re populating the contents of the template themselves. In
which case this part of the page doesn’t offer them any value.
With the addition of template folders we need to filter templates
based on a combination of type and parent folder ID.
This replaces the existing `templates_by_type` method with
`get_templates`, which supports both type and parent folder filters,
avoiding a need to create specific methods for each use case.
We still need the templates property to exist in some way in order
to cache it, but it needs to be clear that it's different from
`.get_templates`. One option was to make it "private" (i.e. `_templates`),
and always use `.get_templates` in the rest of the code, but this requires
adding "include all folders" to `.get_templates`, which doesn't have an
obvious interface since `parent_folder_id=None` already means "top-level
only".
This will probably come up again when we need to look into adding
templates from nested folders into the page for live search, but
for now renaming `Service.templates` to `.all_templates` makes it
clear what the property contains.
We do a lot of logic around choosing which templates to show. This logic
is all inside one view method.
It makes it cleaner to break this logic up into functions. But this
would mean passing around variables from one function to another.
Putting these methods onto a class (the service model) means that
there’s a place to store this data (rather than having to pass it around
a lot).
Making this code more manageable is important so that when we have
templates and folders it’s easy to encapsulate the logic around
combining the two.
We have teams who are using the dashboard every day, and being
confronted with this alarming yellow banner. There’s no action they need
to do since they’re only looking at the messages sent.
So this commit removes that banner from the dashboard. It also removes
the CSS and HTML for it from the app entirely because this is the last
remaining place we were using this style of banner.
we branch on any_ to either say "require ALL these permissions" or
"require ANY of these permissions". But we only ever call the decorator
with one permission, or with any_=True, so it's unnecessary
rather than allow admins to do everything specifically, we should
only block them from things we conciously don't want them to do.
This is "Don't let platform admins send letters from services they're
not in". Everything else the platform admins can do.
This is step one, adding a restrict_admin_usage flag, and setting that
for those restricted endpoints around creating api keys, uploading CSVs
and sending one-off messages.
Also, this commit separates the two use cases for permissions:
* user.has_permission for access control
* user.has_permission_for_service for user info - this is used for
showing checkboxes on the manage-users page for example
With this, we can remove the admin_override flag from the permission
decorator.
- Removed the code for the template_history endpoint and replaced with a
redirect to the new page so that anyone is forwarded on
- Updated the template to point to the new template_usage page
This commit adds two things:
a section on the dashboard to show how many inbound messages the
service has received in the last 7 days, and how recently an inbound
message has been received
---
Doesn’t show the contents of any messages, just like how the rest of the
dashboard is an aggregation, never individual messages.
a page to show all the inbound messages the service has received in
the last 7 days
---
This shows the first line of the message. Eventually this will link
through to a ‘conversation’ page, where a service can see all the
messages it’s received from a given phone number.
- ‘messages sent per month’ is a better description of what will appear
on the page than ‘activity breakdown’
- ‘templates used by month’ instead of ‘this year’ for consistency
I think ‘usage breakdown’ still works for the remaining link – it’s more
than a monthly breakdown, it also breaks down the spend.
Doesn’t need to say ‘by month’ in the `<h1>`s themselves, because you
can see from looking at the page that it’s broken down by month.
In pages specific to a service (e.g. dashboard and sub pages) the title
needs to distinguish which service it applies to. This is mainly to give
context to screen reader users who could be managing multiple services.
Implementing this uses template inheritance:
`page_title` includes `per_page_title` includes `service_page_title`
‘GOV.UK Notify’ is inserted into every page title.
Pages that set `service_page_title` get the service name inserted too.
The previous, weekly activity breakdown was what we reckoned might be
useful. But now that we have people using the platform it feels like
aggregating a service’s usage by month is:
- matches the timeframe users report on within their organisation
- is consistent with the usage page
And like the usage page this commit also limits the page to only show
one financial year’s worth of data at once (rather than data for all
time).
This commit also makes some changes to the jobs view code so that our
aggregation of failure states is consistent between the dashboard pages
and the jobs pages.
On the dashboard:
- adds a new ‘in the next 24 hours’ section to the dashboard which lists
upcoming jobs
- tweaks some spacing on the dashboard so that it doesn’t look like too
much of a mess
- don’t show scheduled jobs in the table of normal jobs
On the jobs page:
- don’t show scheduled jobs
The "you only have permission to view this service" banner sort of
makes sense if you don’t have _any_ permissions, but it doesn’t if you
have permission to create API keys. If you can create API keys you can
do a lot more than just view the service.
This banner was always being shown because the template was never
getting sent the service’s templates from the API.
This commit fixes this to only show the banner when a service has no
templates, and adds some tests to make sure it doesn’t happen again.
Since we’re removing the write email/write text message calls to action
from the tour, we should reintroduce them to the dashboard, for users
who are unsure what they should do next.
This commit extends the `ajax_block` component to take a `dict` of
partials, from which it can select the partial matching its `key`
argument and print its HTML to the page.
This means that the same markup is only rendered in one place, rather
than in two (individually in the JSON endpoint and as `include`s in the
parent template).
This is less repetitive than typing out the HTML with all its attributes
every time.
It also lets us wrap up the idea of ‘finished’ as a parameter, so the
AJAX code will only be initiated when it’s needed, eg if a job is still
processing.
Previously, the AJAX update for the dashboard was returning a big blob
of JSON with one key.
This commit splits it up to return:
- one key for each section of the page
- each containing a smaller chunk of HTML rendered from a partial
The jobs page was already working this way (pretty much) but just needed
a little tweaking to get it the same.
Previously the structure was
```
dashboard.html
|_ today.html
|_ some random html
|_ a few things split into partials
```
This commit simplifies the structure to just be:
```
dashboard.html
|_ partial
|_ partial
|_ …
```
The dashboard looked a bit table-y. This commit makes four main changes:
- show a bar chart (drawn in CSS) for template usage (only shown if
you’ve used more than one template recently)
- only break down template usage by template name, not template type
(because that’s happening with the big numbers)
- change the style of the ‘show more’ links under each section so that
they are all consistent, and a little less busy (one less keyline)
- remove the ‘recent templates‘ title so that the first two sections of
the page group under ‘in the last 7 days’
Takes the number of emails and SMS fragments sent from:
https://github.com/alphagov/notifications-api/pull/273
Using these numbers it’s possible to show:
- how much of your allowance is left
- or how much you have spent
For now the allowance and rates are hard coded.
Only for users that have manage service.
Currently, when we update a section of the page with AJAX we replace the
entire HTML of the section with the new HTML. This causes problems:
- if you’re trying to interact with that section of the page, eg by
inpecting it, clicking or hovering an element
- (probably) for screenreaders trying to navigate a page which is
changing more than is necessary
This commit replaces the call to `.html()` with a pretty clever library
called diffDOM[1]. DiffDOM works by taking a diff of the old element and
the new element, then doing a patch update, ie only modifying the parts
that have changed.
This is similar in concept to React’s virtual DOM, while still allowing
us to render all markup from one set of templates on the server-side.
1. https://github.com/fiduswriter/diffDOM
(except view activity, natch)
> No permissions – Remove trial mode banner
> No permissions – Add blue banner saying 'You only have permission to
view this service'
This commit adds a 3 screen tour, similar to those used on GOV.UK Verify
and Passports.
We guerilla tested this on Friday, and it really helped users to build a
mental model of how Notify works, so that when they’re playing around
with it they have a greater sense of what they’re aiming to do. This
makes concepts like templates and placeholders click more quickly.
https://www.pivotaltracker.com/story/show/116710119
Previously the ‘get started’ block on the dashboard pushed users into
starting with a text message. This led to users writing email content
into a tet message template because it didn’t match their expectations.
Also, the ‘send yourself a test’ link from the dashboard:
- wasn’t much used, because users responded to the similar call to
action on the choose template page instead
- was confusing if you had created an email template because it
presupposed that you’d created a text message template
So this commit changes the ‘get started’ block to be a choice between
creating an email template or a text message template. We reckon that
the language of ‘set up’ will help establish that templates are not
equivalent to individual messages. This language is now repeated on the
‘choose template’ page when you don’t have templates as well.
