werkzeug's internal workings keep files under 500kb in memory, and files
greater than 500kb as a TemporaryFile
(https://github.com/pallets/werkzeug/blob/0.11-maintenance/werkzeug/formparser.py#L38)
when we encounter a CSV or TSV, we call normalise_newlines, which invokes
`.read()`, however when we were passing straight into pyexcel, we called
`file.getvalue()` - this exists on BytesIO (small files) but not on
TemporaryFile objects (large files) - we were seeing 500 errors
In research we’ve seen people mix up the service ID and API key because
they’re both 36 character UUIDs. We can’t get rid of the service ID
because it’s used to look up the API key.
Instead, we should change API key to be one long string, which contains
both the service ID, API key and (optionally) the name of the key. For
example:
```
casework_production-8b3aa916-ec82-434e-b0c5-d5d9b371d6a3-dcdc5083-2fee-4fba-8afd-51f3f4bcb7b0
```
We still need to keep the old, separate, key and service ID for a while
until people have updated their clients. But they’re now both on this
page, rather than on two separate pages, which should make for less
fussing anyway.
This shouldn’t be rolled out until the new clients are available.
- [ ] https://github.com/alphagov/notifications-python-client/pull/36
- [ ] https://github.com/alphagov/notifications-node-client/pull/10
- [ ] https://github.com/alphagov/notifications-ruby-client/pull/15
- [ ] https://github.com/alphagov/notifications-java-client/pull/38
- [ ] PHP????
Was using the wrong attribute name on the instance of `RecipientCSV`
(needed to match 19df1eb4ef/notifications_utils/recipients.py (L156) )
The ‘can’t show file contents’ message wasn’t showing up, now it is.
We want to limit the number of rows someone can have in a job, because
it gets too slow to process the file otherwise.
This should be the first error that a user sees, because we can’t work
out if there are other errors until they’ve got the file down to a
processable size.
This also means adding a message to say that the file can’t be displayed
if it doesn’t contain any processed rows.
***
https://www.pivotaltracker.com/story/show/129830161
This PR changes the flow to change an email address.
Once the user enter their password, they are told "Check your email".
An email has been sent to them containing a link to notify which contains an encrypted token.
The encrypted token contains the user id and new email address. Once the link is clicked the user's email address is updated to the new email address.
They are redirected to the /user-profile page.
Also in this commit is an update from flask.ext.login to flask_login.
These are out of date now and potentially misleading.
Adds a new error page for HTTP `410 GONE`. Looks like the 404 page, but
returns the appropriate error code to be a good web citizen.
You can’t properly use a normal key when your service is in trial mode.
It’s theoretically useful to create a live key in preparation for going
live. This utitily outweighs the confusion it causes for people creating
their first keys in trial mode.
We should just remove the confusing option.
Basically:
- shows all the months from start of given financial year to now or end
of given financial year (whichever is earliest)
- shows a breakdown of free and paid text messages for each of these
months
Depends on:
- [x] https://github.com/alphagov/notifications-api/pull/699
The activity page was AJAXified in
https://github.com/alphagov/notifications-admin/pull/952 but this
didn’t take pagination into account.
So if you visited page 2, the contents of the page would be replaced by
Javascript with the contents of page 1. So you’d never be able to see
anything beyond page 1, expect for a brief fraction of a second when the
page loaded.
This commit makes sure the AJAX request uses whatever page parameter is
in the URL of the original page.
Now that we’ve removed simulated notifications from the dashboard and
activity pages they’re not visible anywhere in the app.
While they should’t be visible to non-technical users, developers have
a real need for Notify to confirm that their code is doing what they
expect. This is needed especially when they’re just getting started with
Notify.
There’s no way of seeing this info from the API either, because a key
can only get notifications created with a key of that type.
It doesn’t make sense to make this a ‘mode’ of the dashboard or activity
because the information about notifications that developers need is
also different. So this commit adds up to 50 of the most recent
notifications sent via the API to the page that developers use as their
‘home’ page.
This also lets us explain the 7 days thing to developers via the
empty slate state of this area of the page.
We want to show a log of notifications that have been sent from the API.
The admin app uses its own private `/service/…/notifications` endpoint
for listing activity. This commit allows us to pass through two
optional, additional parameters to tell the API to:
- include or not include notifications created from a job
- include or not include notifications created with a test API key
Services who are in alpha or building prototypes need a way of sending
to any email address or phone number without having to sign the MOU.
This commit adds a page where they can whitelist up to 5 email addresses
and 5 phone numbers.
It uses the ‘list entry’ UI pattern from the Digital Marketplace
frontend toolkit [1] [2] [3].
I had to do some modification:
- of the Javascript, to make it work with the GOV.UK Module pattern
- of the template to make it work with WTForms
- of the content security policy, because the list entry pattern uses
Hogan[1], which needs to use `eval()` (this should be fine if we’re
only allowing it for scripts that we serve)
- of our SASS lint config, to allow browser-targeting mixins to come
after normal rules (so that they can override them)
This commit also adds a new form class to validate and populate the two
whitelists. The validation is fairly rudimentary at the moment, and
doesn’t highlight which item in the list has the error, but it’s
probably good enough.
The list can only be updated all-at-once, this is how it’s possible to
remove items from the list without having to make multiple `POST`
requests.
1. 434ad30791/toolkit/templates/forms/list-entry.html
2. 434ad30791/toolkit/scss/forms/_list-entry.scss
3. 434ad30791/toolkit/javascripts/list-entry.js
4. http://twitter.github.io/hogan.js/
We’ve found in research that developers have no idea they’re in trial
mode until they hit an error. And even then they don’t really know what
trial mode means.
So this commit:
- adds a message to the API integration page about trial mode
- puts it in a really yellow banner to draw attention to it
- adds the same banner to the settings page
This commit adds a placeholder page which, for now, just has links to
the API keys page and links to the clients.
There’s more stuff to come on this page, but this commit just does the
reorganising so that it’s easier to review.
- it tried to send a verify code which no longer is applicable
- one stage of process removed and tests update properly
Flow is:
- Change email
- Confirm with password
- Done
There’s a chance that someone will run out of imagination and use
the name of the thing they’re signing up for as their password.
This wouldn’t be caught by the generic blacklist.
If a user chooses a very common password then an attacker could guess it
in relatively few attempts, circumventing the lockout.
CESG recommend blacklisting the most common passwords:
> …enforcing the requirement for complex character sets in passwords is
> not recommended. Instead, concentrate efforts on technical controls,
> especially:
>
> - defending against automated guessing attacks by either using account
> lockout, throttling, or protective monitoring
> - blacklisting the most common password choices
How I made this list:
- went to the OWASP repository of security lists:
https://github.com/danielmiessler/SecLists
- downloaded `10k_most_common.txt`, `twitter-banned.txt` and
`500-worst-passwords.txt`
- filtered out any under 8 characters:
```
sed -r '/^.{,7}$/d' passwords-twitter.txt > passwords-combined.txt
sed -r '/^.{,7}$/d' passwords-500.txt >> passwords-combined.txt
sed -r '/^.{,7}$/d' passwords.txt >> passwords-combined.txt
```
- filtered out any duplicates:
```
cat passwords-combined.txt | awk '!x[$0]++' > passwords-combined-deduped.txt
```
