We can no longer trust that the content of templates stored in the
database is safe.
Utils now has code to sanitise the content of templates.
This commit:
- updates utils to bring this code in
- modifies some integration tests to make sure everything is working
(there are more extensive unit tests in utils)
Right now we can show what a letter template looks like as a PDF or PNG.
This commit completes the work so this is also possible when:
- showing a template with the placeholders replaced
- showing any version of a template
Also removes dependency on `Exception().message`, which was deprecated
in Python 2.6. See
97f82d565f
for full details.
html5lib version numbers look like 0.999, 0.9999, etc.
This was making people unhappy, so they’ve added version numbers like
1.0b1, 1.0b2, etc which map to the numbers with lots of nines.
The equivalent version for 0.9999999 (7 nines) is 1.0b8 😐
The PDF preview is all good, but it’s hard, finickeity and feels dirty
to embed a PDF in a web page. It’s a more natural thing to embed an
image in a web page.
So this commit adds another endpoint to return an image of a letter
template. It generates this image from the PDF preview, so the stack
looks like:
1. `template.png` (generated in admin)
2. `template.pdf` (generated in admin)
3. HTML preview (generated by a `Renderer` in utils)
4. `Template` instance
5. serialised template from API
6. Template stored in database
The library used to convert the PDF to an image is Wand[1], which binds
to ImageMagick underneath. So in order to get this working locally on a
Mac you will probably need to do:
`brew install imagemagick ghostscript cairo pango`.
To get it working on Ubuntu/EC2 is an exercise left to the reader…
1. http://docs.wand-py.org/en/0.4.4/
Implements https://github.com/alphagov/notifications-utils/pull/81
Handles addresses as multiple columns:
- in ‘Send yourself a test’
- in example CSV files
- in validating that a CSV file has recipients (eg at least an ‘address
line 1’ and ‘postcode’ column)
- when showing the contents of a CSV file
As few UI changes as possible, once we have the thing working end-to-end
we can think about how the UI might need to work differently.
We want to limit the number of rows someone can have in a job, because
it gets too slow to process the file otherwise.
This should be the first error that a user sees, because we can’t work
out if there are other errors until they’ve got the file down to a
processable size.
This also means adding a message to say that the file can’t be displayed
if it doesn’t contain any processed rows.
***
https://www.pivotaltracker.com/story/show/129830161
Uses more restrictive email regex to prevent people adding emails with leading quotes, semicolons, etc, that SES rejects and end up as a Technical Failure
When we say ‘delivery information is available for 7 days’ you have to
infer _when_ the seven days starts. When you come back to the page it
still says ‘available for 7 days’ even if you only have a day left to
download it. This is confusing.
This commit changes the text to be relative to now, eg ‘available for 7
days’, ‘available for 1 day’.
The date is counted to midnight on the seventh day, which is when the
data is actually deleted.
Implements: https://github.com/alphagov/notifications-utils/pull/51
Copies the same regex.
Adds some CSS to display conditional placeholders differently to
normal placeholders (vertical rather that curved right-hand edge).
Depends on:
- [ ] https://github.com/alphagov/notifications-utils/pull/40
In research we’ve noticed two problems with the appearance of
placeholders:
1. We are inconsistent about when we display the ((double brackets)).
Sometimes we show them, sometimes we don’t. This doesn’t help user’s
understanding about where the column names in their CSV file come
from, or how they can edit the template to fix any errors.
2. Because they look so different from normal `<textarea>` text, it’s
not immediately obvious that they can be edited just like normal
text. They look more like something that can be dragged/inserted.
So this commit:
1. Makes the brackets always-visible.
2. Makes the text colour of the placeholder `$text-colour`, and only
highlights the name of the ‘variable’, not the brackets themselves.
This commit rearranges the CSV errors (again) to make them geared towards
teaching you how to match placeholders to the column headers.
So the order of errors now is:
1. No phone number/email column
2. Column headers don’t match placeholders
3. Missing or bad data
4. Trial mode
5. Daily limit reached
This depends on:
- [x] https://github.com/alphagov/notifications-utils/pull/39 for 1.
When a user adds or removes placeholders in their template we should consider
this a ‘breaking change’ and warn them accordingly.
Implementing this mostly relies on using
https://github.com/alphagov/notifications-utils/pull/37
Temporarily storing the new template until the user confirms that they want to
make the changes in done using hidden fields. This is a bit hacky, but the
complexity of making sessions interact with WTForms was just too much to handle.
This commit also changes the example spreadsheet that we show on this page to
look more like a spreadsheet.
This commit makes two main changes to what happens when a user is
in trial mode and they upload some email addresses belonging to
other people.
1. Add a specific banner error telling the user about trial mode
2. Make this error higher priority, eg it will show up before the
error about having too many recipients in your file
This means making some changes to the tests so that the example CSV
files include the user’s phone number, then making them invalid by
omitting data required by the templates.
Depends on: https://github.com/alphagov/notifications-utils/pull/34
