By default our AJAX calls were 2 seconds. Then they were 5 seconds
because someone reckoned 2 seconds was putting too much load on the
system. Then we made them 10 seconds while we were having an incident.
Then we made them 20 seconds for the heaviest pages, but back to 5
seconds or 2 seconds for the rest of the pages.
This is not a good situation because:
- it slows all services down equally, no matter how much traffic they
have, or which features they have switched on
- it slows everything down by the same amount, no matter how much load
the platform is under
- the values are set based on our worst performance, until we manually
remember to switch them back
- we spend time during incidents deploying changes to slow down the
dashboard refresh time because it’s a nothing-to-lose change that
might relieve some symptoms, when we could be spending time digging
into the underlying cause
This pull request makes the Javascript smarter about how long it waits
until it makes another AJAX call. It bases the delay on how long the
server takes to respond (as a proxy for how much load the server is
under).
It’s based on the square root of the response time, so is more sensitive
to slow downs early on, and less sensitive to slow downs later on. This
helps us give a more pronounced difference in delay between an AJAX call
that is fast (for example the page for a single notification) and one
that is slow (for example a dashboard for a service with lots of
traffic).
*Some examples of what this would mean for various pages*
Page | Response time | Wait until next AJAX call
---|---|---
Check a reply to address | 130ms | 1,850ms
Brand new service dashboard | 229ms | 2,783ms
HM Passport Office dashboard | 634ms | 5,294ms
NHS Coronavirus Service dashboard | 779ms | 5,977ms
_Example of the kind of slowness we’ve seen during an incident_ | 6,000ms | 18,364ms
GOV.UK email dashboard | `HTTP 504` | 😬
notifications-admin
GOV.UK Notify admin application.
Features of this application
- Register and manage users
- Create and manage services
- Send batch emails and SMS by uploading a CSV
- Show history of notifications
First-time setup
Brew is a package manager for OSX. The following command installs brew:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Languages needed
brew install node
NPM is Node's package management tool. n is a tool for managing
different versions of Node. The following installs n and uses the long term support (LTS)
version of Node.
npm install -g n
n lts
npm rebuild node-sass
The app runs within a virtual environment. We use mkvirtualenv for easier working with venvs
pip install virtualenvwrapper
mkvirtualenv -p /usr/local/bin/python3 notifications-admin
Install dependencies and build the frontend assets:
workon notifications-admin
./scripts/bootstrap.sh
Rebuilding the frontend assets
If you want the front end assets to re-compile on changes, leave this running in a separate terminal from the app
npm run watch
Create a local environment.sh file containing the following:
echo "
export NOTIFY_ENVIRONMENT='development'
export FLASK_APP=application.py
export FLASK_DEBUG=1
export WERKZEUG_DEBUG_PIN=off
"> environment.sh
AWS credentials
Your aws credentials should be stored in a folder located at ~/.aws. Follow Amazon's instructions for storing them correctly
Running the application
workon notifications-admin
./scripts/run_app.sh
Then visit localhost:6012
Updating application dependencies
requirements.txt file is generated from the requirements-app.txt in order to pin
versions of all nested dependencies. If requirements-app.txt has been changed (or
we want to update the unpinned nested dependencies) requirements.txt should be
regenerated with
make freeze-requirements
requirements.txt should be committed alongside requirements-app.txt changes.
Working with static assets
When running locally static assets are served by Flask at http://localhost:6012/static/…
When running on preview, staging and production there’s a bit more to it:
