Sometimes we manually check that a URL parameter is in a required set.
Sometimes we don’t bother.
This commit adds a URL converter to do this so that:
- we don’t have to re-write the same code every time
- it’s easier to apply this check to other endpoints
This means endpoints that previously allowed a `template_type` or
`message_type` of `None` now 404. So I’ve had to add new routes for
with URLs that don’t include such parameters.
So this…:
```
/services/128b91b6-2996-4107-bb65-51b7c24a728d/notifications/sms.csv
/services/128b91b6-2996-4107-bb65-51b7c24a728d/notifications/None.csv
```
…becomes:
```
/services/128b91b6-2996-4107-bb65-51b7c24a728d/notifications/sms.csv
/services/128b91b6-2996-4107-bb65-51b7c24a728d/notifications.csv
```
This matches what we do for the HTML-responding equivalent (see
265931d217/app/main/views/jobs.py (L215-L216))
Because it means you often have to cast to string in your application
code just to get your tests passing.
The method being monkey patched is originally defined here: b81aa0f18c/src/werkzeug/routing.py (L1272)
Scanning the page is difficult at the moment because it’s hard to tell
how far apart in time events are, and thereby determine which events
might be related.
Grouping the events by day quickly lets users narrow their focus to
a meaningful subset of the events.
We store our audit history in two ways:
1. A list of versions of a service
2. A list of events to do with API keys
In the future there could be auditing data which we want to display that
is stored in other formats (for example the event table).
This commit adds some objects which wrap around the different types of
auditing data, and expose a consistent interface to them. This
architecture will let us:
- write clean code in the presentation layer to display these events on
a page
- add more types of events in the future by subclassing the `Event` data
type, without having to rewrite anything in the presentation layer
This makes it consistent that an option which contains more options has
a hint about how many options it contains.
Also adds a formatter to get us ready for 1,000 services 🎉
The data flow of other bits of our application looks like this:
```
API (returns JSON)
⬇
API client (returns a built in type, usually `dict`)
⬇
Model (returns an instance, eg of type `Service`)
⬇
View (returns HTML)
```
The user API client was architected weirdly, in that it returned a model
directly, like this:
```
API (returns JSON)
⬇
API client (returns a model, of type `User`, `InvitedUser`, etc)
⬇
View (returns HTML)
```
This mixing of different layers of the application is bad because it
makes it hard to write model code that doesn’t have circular
dependencies. As our application gets more complicated we will be
relying more on models to manage this complexity, so we should make it
easy, not hard to write them.
It also means that most of our mocking was of the User model, not just
the underlying JSON. So it would have been easy to introduce subtle bugs
to the user model, because it wasn’t being comprehensively tested. A lot
of the changed lines of code in this commit mean changing the tests to
mock only the JSON, which means that the model layer gets implicitly
tested.
For those reasons this commit changes the user API client to return
JSON, not an instance of `User` or other models.
Upgraded pyexcel-io from 0.5.14 to 0.5.16. This change causes Werkzeug
to be upgraded from 0.14.1 to 0.15.1 which requires some changes:
* ProxyFix now needs to be imported from a different location
* The status code of RequestRedirect has changed from 301 to 308. Since
status code 308 is not currently supported on Internet Explorer with
Windows 7 and 8.1, this subclasses RequestRedirect to keep the status
code of 301.
changelog: https://werkzeug.palletsprojects.com/en/0.15.x/changes/#version-0-15-0
when clients are defined in app/__init__.py, it increases the chance of
cyclical imports. By moving module level client singletons out to a
separate extensions file, we stop cyclical imports, but keep the same
code flow - the clients are still initialised in `create_app` in
`__init__.py`.
The redis client in particular is no longer separate - previously redis
was set up on the `NotifyAdminAPIClient` base class, but now there's one
singleton in `app.extensions`. This was done so that we can access redis
from outside of the existing clients.
We were getting all letter logos from a method in the email branding
client. Since we will be adding more client methods to deal with
letters, it makes things clearer to separate the email and letter
branding clients.
Changed the table for displaying all notifications to show letters which
have the status of 'validation-failed' as 'Validation failed' instead of
'Cancelled'.
The individual notification page for a letter which has failed
validation has not been changed since this already has a description
(letter has content outside the printable area).
In the long term, we don't want to show cancelled letters. But for now,
this changes cancelled letters to display in the same way that letters
with a status of permanent-failure, since we are currently giving
letters that we want to cancel the status of permanent failure.
The CDN URLs aren’t in included in the content security policy. So
browsers will refuse to load them.
This commit:
- adds each of the CDN URLs to the
- only prepend URLs in CSS files with `/static/` if we’re running
locally (because the CDN URLs are like `static.example.com` not
`example.com/static`)
`www.notifications.service.gov.uk` domain is:
- not gzipped
The PaaS proxy used to GZip and set headers for anything served from a
path starting with `/static/`:
76dd511a8a/ansible/roles/paas-proxy/templates/admin.conf.j2 (L53-L64)
Anything served from `static.notifications.service.gov.uk` is:
- GZipped
- and as a bonus, cached by Cloudfront where possible (meaning the
requests won’t ever hit our app)
This commit moves to serving static asset from `/static/` to
`static.notifications.service.gov.uk`, to get the above listed benefits.
***
We could do even better by setting long cache expiry headers on the static subdomain (currently they’re only set to cache for 60 seconds). But that’s out of scope for this commit.
Bumped the notifications-utils version. The `gmt_timezones` function in
this repo and the `utc_string_to_aware_gmt_datetime` in
notifications-utils are the same, so have updated the code to always use
the version in utils.
The add new templates page now has option to add template folders.
Tweaked wording of other options and h1 to clarify options since it's
not all about templates any more.
Added api client and stuff for it
This commit is the first step to disentangling the models from the API
clients. With the models in the same folder as the API clients it makes
it hard to import the API clients within the model without getting a
circular import.
After this commit the user API clients still has this problem, but at
least the service API client doesn’t.
`_get_current_service` is a function which gets called every time
`current_service` is referenced in a view method or Jinja template.
Because the service model was getting initialised inside this function
it was being reconstructed many times in one request. On the service
settings page, for example, it was getting initialised 43 times, adding
about 200ms to the response time.
This commit moves its initialisation to the point where we’re getting
the data from the API, which only happens once per request.
At the moment we are manually cancelling letters for people when they
ask us to. Once’s we’ve done this there is no indication that it’s
happened except for the date going red on the list of letters.
This commit adds some error messaging and styling to show when a letter
is cancelled.
Letting people cancel their own letters will be a future enhancement.
This duplicates how the task list pattern is coded in the GOV.UK
Prototype kit[1]. It adds ARIA attributes and the use of a
semantically-meaningful element (`<strong>`) to give more information to
screen reader users.
1. https://govuk-prototype-kit.herokuapp.com/docs/templates/task-list
For some reason we are getting pending back as a status from one of
our text message providers.
Because this is unrecognised by the admin app it shows up in the list of
messages as the raw status, styled like an error.
This commit maps it to sending instead, because the user doesn’t need to
know or care about the difference.
A platform admin form accepts a list of references (one per line)
received from DVLA and sends them to the API to update notification
statuses.
References we get from DVLA start with `NOTIFY00\d`, which isn't
part of the reference we store in the database, so we remove them
before sending the data to the API.
The new `returned-letter` status should be treated as `delivered`
for now until we decide a way to display returned letters to users.
Having the service floating about as JSON is a bit flakey. Could easily
introduce a mistake where you mistype the name of a key and silently
get `None`.
Also means doing awkward things like `if 'permission' in
current_service['permissions']`, whereas for users we can do the
much cleaner `user.has_permission()`.
So this commit:
- introduces a model
- adds a `.has_permission` method similar to the one we have for users
The main task that we think ‘caseworker’ users do is send one off
messages.
So this commit:
- makes sure users who don’t have the `view_activity` permission (ie
not ‘admin’ users) can still send messages
- adds navigation so that these users have a place to go from which to
start the process of sending a one off message
In API, the endpoint for the new platform admin stats page has been
moved to a platform stats blueprint. This means we now need a platform
stats client.
A new platform admin page Email complaints has been added to surface those complaints.
Eventually the complaints will be visible to the services so they can remove the email address from their mailing list.
Next thing to implement is "x email complaints" warning on the platform admin summary page.
