Commit Graph

6920 Commits

Author SHA1 Message Date
Rebecca Law
291c6d6dc9 Add statsd annotations for the fact table queries. 2020-02-18 14:33:17 +00:00
David McDonald
ce253a990b Merge pull request #2701 from alphagov/jwt-nightmare
Catch uncaught JWT exceptions
2020-02-14 14:34:11 +00:00
David McDonald
f861da1843 Improve text for error messages 2020-02-14 14:15:41 +00:00
David McDonald
7a019df5a2 Catch previously uncaught jwt exceptions added in python client 5.5.0
This fixes the test in the previous commit and means we will catch other
unexpected jwt errors which are now raised as `TokenError`s and raise an
AuthError based on this.

This will stop us serving 5xx to users when we don't catch an exception.

Also runs make freeze-requirements
2020-02-14 14:14:13 +00:00
David McDonald
ba21d04080 Add test case for uncaught jwt exception 2020-02-14 14:10:12 +00:00
Leo Hemsted
c71de8d62a Merge pull request #2714 from alphagov/docs-in-the-db
Docs in the db
2020-02-13 14:55:39 +00:00
Leo Hemsted
ef8fe35fa0 Merge pull request #2707 from alphagov/pyup-scheduled-update-2020-02-05
Scheduled weekly dependency update for week 05
2020-02-13 14:13:10 +00:00
Leo Hemsted
a97948574a remove unused pytest flags 2020-02-13 12:52:12 +00:00
Leo Hemsted
9e84fb2a00 pin marshmallow to <v3
also, fix pyup comment syntax. descriptions in pyup comments need to
have an additional # prefix
2020-02-13 12:52:12 +00:00
pyup-bot
bbeb66398d pyup updates
Update cffi from 1.13.1 to 1.13.2
Update jsonschema from 3.1.1 to 3.2.0
Update marshmallow-sqlalchemy from 0.19.0 to 0.21.0
Update marshmallow from 2.20.2 to 3.4.0
Update sqlalchemy from 1.3.10 to 1.3.13
Update notifications-python-client from 5.4.1 to 5.5.1
2020-02-13 12:49:12 +00:00
Leo Hemsted
1694395b17 record document count when processing api notifications
if someone doesn't send any documents, set the value to None. If it's
not specified, it defaults to None anyway.
2020-02-13 12:43:06 +00:00
Leo Hemsted
743acf71e4 add nullable document_count field to Notifications
intention is for this to be null, 1, or many, based on how many
documents were linked to within the message. nullable column, so that it
doesn't require a lengthy access exclusive lock on the table when
creating.
2020-02-13 12:42:53 +00:00
David McDonald
fe19aef441 Merge pull request #2712 from alphagov/boto
use original `boto`
2020-02-13 11:51:10 +00:00
David McDonald
5a3d2e3162 Use the original version of boto
We aren't aware of any reason we need to use our fork of boto anymore.
We therefore swap to use `celery[sqs]` which brings in the original
version of boto and we can remove our use of the fork.

This has been tested by running the celery app and seeing it connect to
sqs and grab messages off a queue.
2020-02-12 15:30:02 +00:00
David McDonald
3dcac18849 Use correct exception for boto3
We use boto3 for our interaction with s3. Therefore if an expection is
thrown it will be thrown from the botocore library (which boto3 is built
on top of).

I have copied
app/aws/s3.py::file_exists for an example of this exception catching.
2020-02-12 15:28:46 +00:00
Rebecca Law
755c47c0c1 Merge pull request #2711 from alphagov/remove-unused-methods
Remove unused methods.
2020-02-11 16:58:05 +00:00
Rebecca Law
8445775be0 Remove unused methods.
A new endpoint to return the last date a template was used which means the old endpoint can be removed.
2020-02-07 15:50:54 +00:00
Pea M. Tyczynska
9cd433349c Merge pull request #2706 from alphagov/change-when-we-validate-email-access
Validate email access based on code_type and not auth_type
2020-02-06 14:31:08 +00:00
David McDonald
4d719e8f9d Merge pull request #2709 from alphagov/delete-old-task
Remove task that no longer runs
2020-02-06 13:11:12 +00:00
Rebecca Law
865e06e1a9 Merge pull request #2710 from alphagov/fix-last-used-bug
Fix None type error in last-used endpoint.
2020-02-06 11:51:21 +00:00
Rebecca Law
09f798ea14 Fix None type error in last-used endpoint. 2020-02-06 11:39:40 +00:00
David McDonald
a14d5f0225 Remove task that no longer runs
We no longer puts files in these s3 buckets (and have in fact deleted
the buckets) therefore this task is redundant and can be removed.
2020-02-06 10:57:43 +00:00
Rebecca Law
32082521ee Merge pull request #2708 from alphagov/change-last-used-query
Added a new endpoint to return the last used date for a template.
2020-02-06 10:02:26 +00:00
Rebecca Law
dec42b06cc Simplify the code in the query.
The date in the notifications table should always be the most recent date for the template.
Removed the template_type param for the query as well.
Simplified the tests.
2020-02-05 16:43:17 +00:00
Rebecca Law
3a32c35dd2 Added a new endpoint to return the last used date for a template.
The existing endpoint returned a whole notification for the last time the template was used. But this only takes into account data in the last week. This new methods allows us to be specific about when the template was last used if ever but looking into the ft_notification_status table as well.
2020-02-05 13:03:54 +00:00
Pea Tyczynska
79c456e60c Format email_access_validated_at when serializing
This is to bring it in line with other serialized dates in User
model, like logged_in_at and password_changed_at.

Also get rid of check if password_changed_at has value, as
it is a non-nullable column, so it needs to always have value.

Also set a default value for email_access_validated_at, to bring
it in line with other non-nullable columns.
2020-02-04 16:45:09 +00:00
Leo Hemsted
f4b137c658 Merge pull request #2666 from alphagov/gunicorn-bump
bump gunicorn to v20
2020-02-04 14:45:05 +00:00
Leo Hemsted
8d837eff0e bump gunicorn to v20
v20 brings in a host of changes, including a fix for
https://github.com/benoitc/gunicorn/issues/1847, which was stopping us
upgrading before
2020-02-04 14:21:56 +00:00
Pea Tyczynska
0132d76c16 Validate email access based on code_type anot auth_type
when verifying the code is correct. This way if user has sms_auth
and we send them verification code to validate their email access,
and they click the link in the email, their access will be validated
correctly.
2020-01-31 17:19:24 +00:00
Pea M. Tyczynska
0f6e8c330d Merge pull request #2705 from alphagov/populate-email-access-validated-at
Populate email_access_validated_at_column and make it non-nullable
2020-01-31 14:09:58 +00:00
Pea Tyczynska
558edff597 Populate email_access_validated_at_column and make it non-nullable 2020-01-31 10:59:54 +00:00
Pea M. Tyczynska
8f31d6c7b4 Merge pull request #2704 from alphagov/fix_migration
Simplify the first migration, we will do execute statements later
2020-01-30 17:41:58 +00:00
Pea Tyczynska
cea1f24aa0 Simplify the first migration, we will do execute statements later 2020-01-30 17:28:23 +00:00
Pea M. Tyczynska
bf855c98fe Merge pull request #2703 from alphagov/fix_migration
Fix typo where wrong column name was checked for being null
2020-01-30 16:49:10 +00:00
Pea Tyczynska
a2333c2009 Fix typo where wrong column name was checked for being null 2020-01-30 16:34:59 +00:00
Pea M. Tyczynska
1bbeee3441 Merge pull request #2702 from alphagov/fix_migration
Make sure email_access_validated_at is not null after being populated
2020-01-30 16:16:24 +00:00
Pea Tyczynska
20124d599c Make sure email_access_validated_at is not null after being populated 2020-01-30 16:02:34 +00:00
Pea M. Tyczynska
d34eafa1de Merge pull request #2700 from alphagov/re-validate-user-email
Add email_access_valdiated_at field to user table, populate it
2020-01-30 15:35:39 +00:00
Pea Tyczynska
0eed4c99a7 Add email_access_valdiated_at field to user table, populate it
and update it when users have to use their email to interact with
Notify service.

Initial population:
If user has email_auth, set last_validated_at to logged_in_at.
If user has sms_auth,  set it to created_at.

Then:
Update email_access_valdiated_at date when:
- user with email_auth logs in
- new user is created
- user resets password when logged out, meaning we send them an
email with a link they have to click to reset their password.
2020-01-30 14:51:54 +00:00
Katie Smith
6c4a681bc9 Merge pull request #2699 from alphagov/update-process-sanitised-letter-task
Update process sanitised letter task
2020-01-30 14:37:23 +00:00
Katie Smith
35e39bcfa8 Save recipient address in process_sanitised_letter task
If the letter passed sanitisation, the recipient address will be
returned from template preview, so we want to save this as the `to`
field of the notification.
2020-01-24 13:52:12 +00:00
Katie Smith
adf9906a96 Change process_sanitised_letter to take a single encrypted arg
Template preview will now send an encrypted dict containing all the args
to the `process_sanitised_letter` task, so this updates the task to
handle data in the new format.
2020-01-24 13:18:37 +00:00
Katie Smith
64c2061baa Use encryption module from utils
Now that the encryption module has been moved from this app to utils, we
can remove it from here (along with its tests) and import it from utils
instead. This also renames the `encryption.py` file to `hashing.py`,
since it no longer contains the encryption class.
2020-01-24 13:18:37 +00:00
Katie Smith
1703ae6031 Bump utils to version 36.5.0
This version of utils has the Encryption module included in it.
2020-01-24 13:18:27 +00:00
Leo Hemsted
8f2344e3b4 Merge pull request #2696 from alphagov/doc-dl-errors
handle document download errors properly
2020-01-20 14:20:43 +00:00
Leo Hemsted
a16f79896b refer to file rather than document
Karl Approved™
2020-01-20 13:44:52 +00:00
Leo Hemsted
99d008b383 handle document download errors properly
if doc download returns a 403, that's a screw-up on our side. it's not
helpful to a notify user for that to be passed on. the only thing they
should care about is if it's a 400, because they uploaded a filetype we
don't allow.

Everything else should return 500 internal server error.
2020-01-20 13:44:50 +00:00
David McDonald
65ce879825 Merge pull request #2698 from alphagov/alert-limit
Up threshold for sms to telephone numbers
2020-01-20 10:15:06 +00:00
David McDonald
3a0aece6a1 Up threshold for sms to telephone numbers
We were just ignoring the errors and our users were not fixing things.

Given that 500 texts cost approx £8 it's not the end of the world.

In the long run we may decide to just stop letting people try and send
messages to TV numbers but this is a quick fix to stop emails coming in
which we ignore.
2020-01-17 13:26:20 +00:00
Rebecca Law
879ba1d5f0 Merge pull request #2692 from alphagov/put-address-in-to-field-for-precompiled
Put address in to field for precompiled
2020-01-13 14:53:20 +00:00