This works locally for a long running request and a large number of messages. However I suspect that nginx may be timing out the request. I'd like to try this on staging.
When a platform admin user clicks on the button to add the
'upload_document' permission to a service, they should be taken to the
page to add a contact_link if the service does not already have one.
Added a page which lets users with the 'manage_service' permission change the
contact link for their service. There are no links to this page yet
since only services using document download will need to set a contact
link.
If a template has a placeholder like `((email address))` then the sample
spreadsheet and CSV file have the email column twice.
Trying to upload this spreadsheet will result in a ‘duplicate column’
error.
This commit fixes it so that the column will only appear once.
A new platform admin page Email complaints has been added to surface those complaints.
Eventually the complaints will be visible to the services so they can remove the email address from their mailing list.
Next thing to implement is "x email complaints" warning on the platform admin summary page.
I've left the old usage page there so we can compare the reports if we have questions. There are known issues with the old report, especially for email and letters.
At the moment branding is an undocumented feature. We get a bunch of
support tickets from teams asking its possible.
This commit:
- lets people know it’s possible, and what the options are
- is the first step towards making this process as self-service as
possible
In some cases we will be able to infer a user’s organisation from there
email address, and Google image search their logo. So the experience for
them is that they press a button and government just sorts it out for
you (also known as "the dream").
In other cases we will have to get back to people asking for a copy of
their logo, or to find out about their service, but this is what we have
to do at the moment anyway.
usage page used to make the assumption that the first row of the usage
stats would always be SMS. This now isn't always the case, so make sure
when working out the rate, it only looks at sms rows. Specifically, it
takes the rate from the first stats row. This makes a big assumption
that all the rows will have the same rate per financial year.
We want to set the metadata on a file when rendering the check page, but
not when rendering the preview of a letter. So let’s put the code that
does that only in the place where we’re rendering the check page. This
means not having to pass a variable around.
Refactoring only; no functional changes.
When you upload a CSV the check page takes the file name from the URL
and writes it to the S3 metadata for that file.
This also happens when you go to the .png version of this page.
The .png version of the page doesn’t have the filename in the URL. So it
re-writes the S3 metadata with an empty filename.
This means that all letter jobs sent recently have no file names. This
commit fixes this problem.
we're not actually looking at the detailed service aspects - just
the stats. We're doing this in three places:
* dashboard
* notification activity page
* when checking jobs to see if we're over the daily limit
change these places to use a new api endpoint (service/id/statistics),
which hopefully be a little more performant, and will definitely be a
little more organised - moving away from generic endpoints with loads
of optional parameters.
We still need the detailed endpoints for the platform admin page tho.
Depends on https://github.com/alphagov/notifications-api/pull/1865
Currently we have a bunch of users who aren’t signed in asking us for
the agreement.
This is bad because:
- it’s slower (for them) than just being able to download it
- it creates work for us
We can’t just offer the agreement to anyone, but we can offer to it to
anyone who’s signed in because we now let people self-select which
version to download when we can’t tell which one to give them.
If we don’t know whether people belong to a crown organisation we should
give them the option of self-selecting, because they might themselves
know.
This commit adds a new version of the ‘agreement’ page which gives
people exactly that choice. It doesn’t link to it yet.
Adds a platform admin button to the service settings to turn on/off
'upload_document' service permission. The permission allows uploading
documents to document-download-api through the post notification API
endpoint.
If a user clicks ‘back’ once they’ve sent a job we don’t want them to
land on the ‘check’ page again. This would suggest that they can send
the same job again (they can’t because that `job_id` is in the database
already). That said, it’s confusing to see that page; the natural thing
is to go jump back another step, to where you uploaded the file.
We’re going to stop storing job metadata in the session. So we can’t
rely on it for checking whether a file is valid. That safeguard is
happening in the API instead now (because it’s looking at the metadata
stored in S3).
For both SMS senders and email reply to addresses this commit adds:
- a delete link
- a confirmation loop
It doesn’t let users delete:
- default SMS senders or reply to addresses (they always have to have
one)
- inbound numbers
It assumes that the API will allow updating of an attribute named
`active` on the respective database rows. It could work in a different
way. We can’t do complete deletion though because these will still be
keyed to notifications.
S3 has a limit of 2kb for metadata:
> the user-defined metadata is limited to 2 KB in size. The size of
> user-defined metadata is measured by taking the sum of the number of
> bytes in the UTF-8 encoding of each key and value.
– https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html#object-metadata
This means we have a limit of 1870 bytes for the filename:
```python
encoded = 'notification_count50000template_id665d26e7-ceac-4cc5-82ed-63d773d21561validTrueoriginal_file_name'.encode('utf-8')
sys.getsizeof(b)
>>> 130
2000-130
>>> 1870
```
Or, in other words, ~918 characters:
```python
sys.getsizeof(('ü'*918).encode('utf-8'))
>>> 1869
```
We’re getting the template just to get back its `id`, which is the one
thing we do know in order to get it.
The call to get template is still happening inside `_check_messages`, so
we’ll still catch someone trying to look at this page for a template
that doesn’t exist.
By doing this we no longer have to store it in the session. This is the
last thing that’s currently in the session, so removing it means we can
drop session storage for file uploads entirely.
Storing things in the session is proving buggy – we still have one user
(that we know about) where the session data isn’t getting written, so
they’re blocked from uploading a file.
Since all the info we’re storing in the session is about the file, it
makes sense to store it with the file.
This commit only does the writing of the metadata, once we’re sure this
is working we can do subsequent work to read it back, and remove
reliance on the session.
