The previous, weekly activity breakdown was what we reckoned might be
useful. But now that we have people using the platform it feels like
aggregating a service’s usage by month is:
- matches the timeframe users report on within their organisation
- is consistent with the usage page
And like the usage page this commit also limits the page to only show
one financial year’s worth of data at once (rather than data for all
time).
This commit also makes some changes to the jobs view code so that our
aggregation of failure states is consistent between the dashboard pages
and the jobs pages.
Right now we tell people that the usage page is for the current
financial year. This is a lie – it’s for all time.
So this commit calls through to the API to get the stats for (by
default) the current financial year.
We already do this for the monthly breakdown, this just does the same
thing for the yearly totals.
It also adds navigation to show the data for other financial years:
- previous so you can go back and see your usage and verify that the
bill you’re about to pay is correct
- next so that you can check what your SMS allowance is going to be
before you actually get into it
The PNGs we were generating looked a bit fuzzy, and the text rendering
was a bit off.
This is because we weren’t rendering them at a high enough resolution.
I’m guessing the default was 72dpi. This commit increases the resolution
to 96dpi, which (locally) seems to be enough to give a good-looking
preview.
Right now we can show what a letter template looks like as a PDF or PNG.
This commit completes the work so this is also possible when:
- showing a template with the placeholders replaced
- showing any version of a template
Also removes dependency on `Exception().message`, which was deprecated
in Python 2.6. See
97f82d565f
for full details.
werkzeug's internal workings keep files under 500kb in memory, and files
greater than 500kb as a TemporaryFile
(https://github.com/pallets/werkzeug/blob/0.11-maintenance/werkzeug/formparser.py#L38)
when we encounter a CSV or TSV, we call normalise_newlines, which invokes
`.read()`, however when we were passing straight into pyexcel, we called
`file.getvalue()` - this exists on BytesIO (small files) but not on
TemporaryFile objects (large files) - we were seeing 500 errors
In research we’ve seen people mix up the service ID and API key because
they’re both 36 character UUIDs. We can’t get rid of the service ID
because it’s used to look up the API key.
Instead, we should change API key to be one long string, which contains
both the service ID, API key and (optionally) the name of the key. For
example:
```
casework_production-8b3aa916-ec82-434e-b0c5-d5d9b371d6a3-dcdc5083-2fee-4fba-8afd-51f3f4bcb7b0
```
We still need to keep the old, separate, key and service ID for a while
until people have updated their clients. But they’re now both on this
page, rather than on two separate pages, which should make for less
fussing anyway.
This shouldn’t be rolled out until the new clients are available.
- [ ] https://github.com/alphagov/notifications-python-client/pull/36
- [ ] https://github.com/alphagov/notifications-node-client/pull/10
- [ ] https://github.com/alphagov/notifications-ruby-client/pull/15
- [ ] https://github.com/alphagov/notifications-java-client/pull/38
- [ ] PHP????
The GOV.UK content style guide says:
> - 5:30pm (not 1730hrs)
> - midnight (not 00:00)
> - midday (not 12 noon, noon or 12pm)
This commit changes all times to be 12h not 24h, and adds a special case
for when a time is exactly 12:00am or 12:00pm.
This commit makes the CSV download use the same language for failure
reasons as the frontend.
It also adds a test around this stuff, which was patchily tested before.
This is re-fixing a bug which was re-introduced when adding the
`Spreadsheet` class in 1409ca36ca.
It was previously fixed in 19662d8329:
> Fix bug with large file uploads
>
> Depending on the size of the uploaded file, Flask will temporarily store
> it in different ways. This means that it comes back as a `TempFile` if
> the file is roughly <500k and as `BytesIO` if the file is larger.
>
> `TempFile` supports the `.getvalue()` method, but `BytesIO` does not.
> Both support the `.read()` method, so this commit changes to use that
> instead.
Our templates are a littered with `request.args.get('help', '0')`.
This commit refactors these into a single helper method, which can be
used by the view functions, then passed to the template.
This makes the templates cleaner, and should make it easier to refactor
`help` out of the query parameters entirely in the future.
parts of the initial setup/login stages were throwing 500s if user
not already in process (ie: user directly navigated to url):
* /resend-email-verification
* /text-not-received
* /send-new-code
* verify
This commit rearranges the CSV errors (again) to make them geared towards
teaching you how to match placeholders to the column headers.
So the order of errors now is:
1. No phone number/email column
2. Column headers don’t match placeholders
3. Missing or bad data
4. Trial mode
5. Daily limit reached
This depends on:
- [x] https://github.com/alphagov/notifications-utils/pull/39 for 1.
We require users to export their spreadsheets as CSV files before
uploading them. But this seems like the sort of thing a computer should
be able to do.
So this commit adds a wrapper class which:
- takes a the uploaded file
- returns it in a normalised format, or reads it using pyexcel[1]
- gives the data back in CSV format
This allows us to accept `.csv`, `.xlsx`, `.xls` (97 and 95), `.ods`,
`.xlsm` and `.tsv` files. We can upload the resultant CSV just like
normal, and process it for errors as before.
Testing
---
To test this I’ve added a selection of common spreadsheet files as test
data. They all contain the same data, so the tests look to see that the
resultant CSV output is the same for each.
UI changes
---
This commit doesn’t change the UI, apart from to give a different error
message if a user uploads a file type that we still don’t understand.
I intend to do this as a separate pull request, in order to fulfil
https://www.pivotaltracker.com/story/show/119371637
Each service on the list is linked to the dashboard page of the service.
The platform admin user can see/edit templates, see/invite users, see/edit service settings.
The platform admin user can not send messages, see/edit api keys and developer docs.
Makes uses of the additions to utils in https://github.com/alphagov/notifications-utils/pull/9
This commit strips out a lot of the complex stuff that the views and templates
in this app were doing. There is now a cleaner separation of concerns:
- utils returns the number and type of errors in the csv
- `get_errors_for_csv` helper in this app maps the number and type of errors
onto human-friendly error messages
- the view and template just doing the glueing-together of all the pieces
This is (hopefully) easier to understand, definitely makes the component
parts easier to test in isolation, and makes it easier to give more specific
error messages.
