(previously it would have sent them to the choose template page)
if the user has added new templates or deleted the example one,
they're clearly competent enough to use the app so don't worry
(we wouldn't know what URL the tour starts on since the UUID of
the example template is random)
We can no longer trust that the content of templates stored in the
database is safe.
Utils now has code to sanitise the content of templates.
This commit:
- updates utils to bring this code in
- modifies some integration tests to make sure everything is working
(there are more extensive unit tests in utils)
previously we were issuing a flask redirect (302) from the function,
which we then attempted to unpack as a dict further down the line.
raise a werkzeug.routing.RequestRedirect (301 MOVED PERMANENTLY)
instead. note: only use this pattern when the URL they attempted to
access will *NEVER* be valid, as 301s are cached by browsers.
Right now we can show what a letter template looks like as a PDF or PNG.
This commit completes the work so this is also possible when:
- showing a template with the placeholders replaced
- showing any version of a template
Also removes dependency on `Exception().message`, which was deprecated
in Python 2.6. See
97f82d565f
for full details.
The breaking change page was taking the rendered template and saving
that if the user confirmed the change. This meant that templates could
be saved with `<span class="placeholder">…</span>` in their subject line
for example.
This commit fixes it so that it uses whatever data the user submitted,
not the rendered version of this.
Who knows what would happen if a job with a letter template actually
got into the database. `403`ing the page is a quick and dirty hack to
stop this from happening.
werkzeug's internal workings keep files under 500kb in memory, and files
greater than 500kb as a TemporaryFile
(https://github.com/pallets/werkzeug/blob/0.11-maintenance/werkzeug/formparser.py#L38)
when we encounter a CSV or TSV, we call normalise_newlines, which invokes
`.read()`, however when we were passing straight into pyexcel, we called
`file.getvalue()` - this exists on BytesIO (small files) but not on
TemporaryFile objects (large files) - we were seeing 500 errors
Was using the wrong attribute name on the instance of `RecipientCSV`
(needed to match 19df1eb4ef/notifications_utils/recipients.py (L156) )
The ‘can’t show file contents’ message wasn’t showing up, now it is.
We want to limit the number of rows someone can have in a job, because
it gets too slow to process the file otherwise.
This should be the first error that a user sees, because we can’t work
out if there are other errors until they’ve got the file down to a
processable size.
This also means adding a message to say that the file can’t be displayed
if it doesn’t contain any processed rows.
***
https://www.pivotaltracker.com/story/show/129830161
For two reasons:
- it’s extra stuff in tour that users dont yet need to know about
- test messages are hidden from the dashboard, so you’d have no
visibility of when they were sending once you’d scheduled them
Users need to pick a time in the next 24hrs, or send a file immediately.
Rationale for this is a bit lost in time-before-holiday, but generally:
‘Now’ and ‘later’ as the inital choices makes it really clear what
this feature is about conceptually.
The choice of times is absolute, eg ‘1pm’ not ‘in 3 hours’
We’ve seen people land on this page and expect the message to be on
their phone already.
‘Check and confirm’ sounds a lot like ‘check your phone’, which is
language that we use earlier on when we _have_ sent a message.
Hopefully ‘preview’ is a better indication that it’s not sent yet.
After the tour we should ground users by dropping them on the dashboard.
In the background, we delete the example text message template. This
means that users start from a clean slate when they go to add their own
templates
This also means some wording changes to the tour so it still makes
(some) sense:
- 1, 2 and 3 should refer to the current step, not describe the next
one
- the link should take you to the dashboard
- change from ‘Get started’ to ‘Try this example’ because we’re using
‘Get started on the dashboard’
Steps to reproduce:
- make a template with a placeholder
- click ‘send yourself a test’
- leave fields blank
- click ‘check’
- see error, click ‘back’
Expected: previous page
Actual: previous page with blue help sidebar
When the URL contains `help=0`, `request.args.get('help')` returns '0'.
Doing `if '0':` is the same as doing any `if <non empty string>:` which
returns `True`.
So we should only display the help when the help query parameter is:
- not missing
- AND a string that isn’t `'0'`
This commit rearranges the CSV errors (again) to make them geared towards
teaching you how to match placeholders to the column headers.
So the order of errors now is:
1. No phone number/email column
2. Column headers don’t match placeholders
3. Missing or bad data
4. Trial mode
5. Daily limit reached
This depends on:
- [x] https://github.com/alphagov/notifications-utils/pull/39 for 1.
This commit makes two main changes to what happens when a user is
in trial mode and they upload some email addresses belonging to
other people.
1. Add a specific banner error telling the user about trial mode
2. Make this error higher priority, eg it will show up before the
error about having too many recipients in your file
This means making some changes to the tests so that the example CSV
files include the user’s phone number, then making them invalid by
omitting data required by the templates.
Depends on: https://github.com/alphagov/notifications-utils/pull/34
Generating the UUID can be can be contained within this function,
thus any other part of the code using it doesn’t have to do the
ID-generating stuff itself.
There shouldn’t be a case where we see a `ValueError` on upload any
more. Our file handling should be robust enough to deal with whatever is
thrown at it.
This commit:
- adds test files with bad data (PNG files with their extensions changed to look
like spreadsheets)
- catches whatever exceptions are raised by trying to parse these files
- returns a helpful flash message to the user
Anything else should raise a `500`, eg if the file can’t be uploaded to S3.
We require users to export their spreadsheets as CSV files before
uploading them. But this seems like the sort of thing a computer should
be able to do.
So this commit adds a wrapper class which:
- takes a the uploaded file
- returns it in a normalised format, or reads it using pyexcel[1]
- gives the data back in CSV format
This allows us to accept `.csv`, `.xlsx`, `.xls` (97 and 95), `.ods`,
`.xlsm` and `.tsv` files. We can upload the resultant CSV just like
normal, and process it for errors as before.
Testing
---
To test this I’ve added a selection of common spreadsheet files as test
data. They all contain the same data, so the tests look to see that the
resultant CSV output is the same for each.
UI changes
---
This commit doesn’t change the UI, apart from to give a different error
message if a user uploads a file type that we still don’t understand.
I intend to do this as a separate pull request, in order to fulfil
https://www.pivotaltracker.com/story/show/119371637
This commit does two main things:
- adds textboxes to the send yourself a test page, so you can replace
((name)) with ‘Chris’, or whatever your name is
- rejigs the send page a bit to make it clearer what the CSV upload is
for and how to use it
The idea being that, since most users go into ‘send yourself a test’
first, it teaches them about how placeholders work by making them do the
replacing themselves.
https://www.pivotaltracker.com/story/show/117630691
There is a limit of 50 messages per day in trial mode. Right now, we
don’t tell you this, we just start failing your messages.
This commit adds an error message if you upload a CSV file that has too
many rows in it.
For users who:
- want to send messages from a template
- want to edit templates
For developers:
- who need to get the ID of a template
This commit mainly cleans up the choose template page so there are less
options, and the options that are there are less wordy.
This means:
- moving ‘send yourself a test’ onto the send messages page, and making
it button
- stripping a lot of stuff out of the ‘send from API’ page, so it’s more
obvious what the template ID is
We probably shouldn’t hide the contents of the CSV when people are just
testing the app, or if they’re starting off with small jobs.
A limit of 15 rows displayed was awkwardly on the cusp between just
testing and sending a small batch.
This commit increases the limit to 50; I reckon that over 50 recipients
no-one will be wanting to check them all individually.
The problem
---
1. Upload a CSV with problems
2. See the validation errors on the page
3. Try to re-upload the file
4. Get this error:
> Method Not Allowed
> The method is not allowed for the requested URL.
https://www.pivotaltracker.com/story/show/116882241
The cause
---
The POST route for the check page (where you see errors in your files)
was removed here:
f3fd5f6b15 (diff-1dd8b3bf53dfd9382c7721051f3d930dR280)
The fix
---
This commit adds:
- a simple route which re-calls the initial ‘I have uploaded a file’ route
- a test to make sure that both of these routes are POSTed to
