Commit Graph

6987 Commits

Author SHA1 Message Date
Pea M. Tyczynska
bf855c98fe Merge pull request #2703 from alphagov/fix_migration
Fix typo where wrong column name was checked for being null
2020-01-30 16:49:10 +00:00
Pea Tyczynska
a2333c2009 Fix typo where wrong column name was checked for being null 2020-01-30 16:34:59 +00:00
Pea M. Tyczynska
1bbeee3441 Merge pull request #2702 from alphagov/fix_migration
Make sure email_access_validated_at is not null after being populated
2020-01-30 16:16:24 +00:00
Pea Tyczynska
20124d599c Make sure email_access_validated_at is not null after being populated 2020-01-30 16:02:34 +00:00
Pea M. Tyczynska
d34eafa1de Merge pull request #2700 from alphagov/re-validate-user-email
Add email_access_valdiated_at field to user table, populate it
2020-01-30 15:35:39 +00:00
Pea Tyczynska
0eed4c99a7 Add email_access_valdiated_at field to user table, populate it
and update it when users have to use their email to interact with
Notify service.

Initial population:
If user has email_auth, set last_validated_at to logged_in_at.
If user has sms_auth,  set it to created_at.

Then:
Update email_access_valdiated_at date when:
- user with email_auth logs in
- new user is created
- user resets password when logged out, meaning we send them an
email with a link they have to click to reset their password.
2020-01-30 14:51:54 +00:00
Katie Smith
6c4a681bc9 Merge pull request #2699 from alphagov/update-process-sanitised-letter-task
Update process sanitised letter task
2020-01-30 14:37:23 +00:00
Katie Smith
35e39bcfa8 Save recipient address in process_sanitised_letter task
If the letter passed sanitisation, the recipient address will be
returned from template preview, so we want to save this as the `to`
field of the notification.
2020-01-24 13:52:12 +00:00
Katie Smith
adf9906a96 Change process_sanitised_letter to take a single encrypted arg
Template preview will now send an encrypted dict containing all the args
to the `process_sanitised_letter` task, so this updates the task to
handle data in the new format.
2020-01-24 13:18:37 +00:00
Katie Smith
64c2061baa Use encryption module from utils
Now that the encryption module has been moved from this app to utils, we
can remove it from here (along with its tests) and import it from utils
instead. This also renames the `encryption.py` file to `hashing.py`,
since it no longer contains the encryption class.
2020-01-24 13:18:37 +00:00
Katie Smith
1703ae6031 Bump utils to version 36.5.0
This version of utils has the Encryption module included in it.
2020-01-24 13:18:27 +00:00
Leo Hemsted
8f2344e3b4 Merge pull request #2696 from alphagov/doc-dl-errors
handle document download errors properly
2020-01-20 14:20:43 +00:00
Leo Hemsted
a16f79896b refer to file rather than document
Karl Approved™
2020-01-20 13:44:52 +00:00
Leo Hemsted
99d008b383 handle document download errors properly
if doc download returns a 403, that's a screw-up on our side. it's not
helpful to a notify user for that to be passed on. the only thing they
should care about is if it's a 400, because they uploaded a filetype we
don't allow.

Everything else should return 500 internal server error.
2020-01-20 13:44:50 +00:00
David McDonald
65ce879825 Merge pull request #2698 from alphagov/alert-limit
Up threshold for sms to telephone numbers
2020-01-20 10:15:06 +00:00
David McDonald
3a0aece6a1 Up threshold for sms to telephone numbers
We were just ignoring the errors and our users were not fixing things.

Given that 500 texts cost approx £8 it's not the end of the world.

In the long run we may decide to just stop letting people try and send
messages to TV numbers but this is a quick fix to stop emails coming in
which we ignore.
2020-01-17 13:26:20 +00:00
Rebecca Law
879ba1d5f0 Merge pull request #2692 from alphagov/put-address-in-to-field-for-precompiled
Put address in to field for precompiled
2020-01-13 14:53:20 +00:00
Rebecca Law
722a4c9ebe Merge pull request #2697 from alphagov/send-alert-one-hour-sooner
Send the alert for `letters-still-sending` an hour earlier.
2020-01-13 13:49:55 +00:00
Rebecca Law
f4c0f70ba9 Send the alert for letters-still-sending an hour earlier.
These alerts are sent to our postal provider. And it usually arrives as they are getting ready to go home for the day or the weekend.
Which means they get missed/overlooked. They have agreed to get the alert an hour earlier, perhaps that will improved the response time.
2020-01-13 10:42:30 +00:00
David McDonald
9cd7c373b1 Merge pull request #2691 from alphagov/readme
Tidy up readme
2020-01-10 10:31:46 +00:00
Rebecca Law
bb2b514e12 Save recipient address in the "to" field of a notification
When a precompiled letter is sent via the admin app, we now pass in the address which can be set in the Notifications.to field.
Once a precompiled letters sent by the API has passed validation we can set the address in Notifications.to field.

The celery tasks to validate precompiled letters sent by the API will be done in another PR.
2020-01-07 14:35:48 +00:00
Rebecca Law
5ebd9a473c Add the recipient address in the "to" field for precompiled letters. 2020-01-07 14:35:48 +00:00
Rebecca Law
49eacced5a Merge pull request #2694 from alphagov/add-reference-to-notification-resp
Add client reference to notification reponse
2020-01-07 14:35:18 +00:00
Rebecca Law
cb87e66801 Add client_reference to the serialised notification returned for the csv reports. 2020-01-07 12:19:41 +00:00
Leo Hemsted
06298bd872 Merge pull request #2693 from alphagov/test-isolation-fix
Test isolation fix
2020-01-07 11:04:58 +00:00
David McDonald
ade08e23da Tidy up Readme 2020-01-07 10:26:07 +00:00
Leo Hemsted
eb72a0279f remove monkeypatch
it's interacting strangely with our os.environ stuff and not restoring
nicely, probably due to fixture order which is hard to visualise and
control.
2020-01-06 17:53:56 +00:00
Leo Hemsted
d244146638 fix weird test isolation errors with os.environ and boto3
test_config manipulates os.environ. os.environ is an `environ` object,
which acts like a dict but isn't in some subtle unknowable ways. The
`reload_config` fixture would create a dict copy of the env, and then
just call `os.environ = old_env` afterwards.

Boto3 would then complain that it couldn't load credentials (despite us
using the mock_s3 fixture and also not having creds in the environment
in the first place). Not entirely sure why this happens, but it does.
For some reason, it being a `dict` instead of an `environ` object causes
the mocking of boto3 to fail.

The solution is to not overwrite os.environ entirely, rather, use the
standard dictionary setitem syntax to update the values to their
previous values. Use `clear` to empty the environment too.
2020-01-06 16:21:34 +00:00
Leo Hemsted
c3b1766220 bump test requirements 2020-01-06 16:07:06 +00:00
Rebecca Law
b61262eb1c Merge pull request #2690 from alphagov/fix-job-counts
Fix job counts for older jobs
2020-01-02 09:52:30 +00:00
Rebecca Law
033bcb65d2 Update the dao_get_notification_outcomes_for_job to return the stats from either the notification table or the ft_notification_status table.
Currently if you visit the job page and the job is older than the data retention the totals on the page are all wrong because this query gets the counts from the notification table. With this change the data should always be correct. It also eliminates the need for looking at data retention. If the job is new and nothing has been created yet (i.e. the job hasn't started yet) then the page should show the correctly because the outcomes are empty (as expected), once the notifications for the jobs are created the numbers will start going up.
2019-12-30 16:17:00 +00:00
Rebecca Law
1d0247878f Merge pull request #2688 from alphagov/populate-returned-letters
Populate returned letters
2019-12-27 13:22:31 +00:00
Rebecca Law
84837b4eda Rename uploaded_letter to uploaded_letter_file_name 2019-12-27 12:39:24 +00:00
Rebecca Law
5e0ebe8e7a Fix conflicts with db migrations 2019-12-27 11:21:26 +00:00
Rebecca Law
63bdb0208b Create a new constant for datetime formats without timezone 2019-12-27 10:27:59 +00:00
Rebecca Law
e9baece3e7 Update the resultset to relect how users will consume the information.
- Do not show "hidden" or precompiled templates, users don't know about them.
- Remove the client reference if it is the file name of an uploaded file.
- Format the date for created_at
- Added a test for all the different types of letters.
 1) One off templated letter
 2) Letter created by a csv upload or job.
 3) Uploaded letter
 4) Templated letter sent by the API
 5) Precompiled letter sent by the API
2019-12-27 10:27:59 +00:00
Rebecca Law
cd29acc2f4 Add email address
Add uploaded_letter, the file name if the letter has been uploaded.
2019-12-27 10:27:59 +00:00
Rebecca Law
b853c4cdf1 Rename dao method to be more consistent.
Fix sort.
Add one to job_row_number, rows start at 0 which would confus the user.
2019-12-27 10:27:59 +00:00
Rebecca Law
aabaa4a971 Added joins to template, job and user for returned letter query.
Added unit tests
Comleted endpoint to get returned letter details
2019-12-27 10:27:59 +00:00
Rebecca Law
606b2b6c84 Change the insert to use updated_at as the reported_at date 2019-12-27 10:27:59 +00:00
Rebecca Law
5482c03bca [WIP] 2019-12-27 10:27:59 +00:00
Rebecca Law
22851c4ef0 Migration script to populate the returned letters. 2019-12-27 10:27:59 +00:00
David McDonald
9c8f5f8343 Merge pull request #2689 from alphagov/coveralls
Remove coveralls
2019-12-24 14:54:46 +00:00
David McDonald
6a8c7c1455 Merge pull request #2687 from alphagov/inbound-sms-history
Add and write into Inbound sms history
2019-12-24 10:13:32 +00:00
David McDonald
f948555ca8 Do nothing on db conflict
For notification and notification_history we do an upsert. Here, as the
inbound_sms table is never updated, only inserted to once (signified by
lack of updated_at field), an upsert would be unnecessary.

Therefore, if for some reason the delete statement failed as part of
moving data into the inbound_sms_history table, we can simply just
ignore any db conflicts raised by a rerun of
`delete_inbound_sms_older_than_retention`.
2019-12-24 09:39:06 +00:00
David McDonald
542f3b23aa Remove coveralls
We don't use it anymore
2019-12-24 08:37:19 +00:00
Pea Tyczynska
f8ff2d121f Changes following review:
- Check if right keys in new history rows
- Improve model and get rid of old revision version
- Add updated migration file
- Test data when inserting into inbound sms history
2019-12-20 16:17:27 +00:00
Pea Tyczynska
448cd1e94e Integrate inbound history insert into delete inbound sms function 2019-12-20 16:16:29 +00:00
Pea Tyczynska
a6b4675ae7 Populate inbound sms history when deleting inbound sms 2019-12-20 16:16:29 +00:00
Pea Tyczynska
de9d1a41eb Create InboundSmsHistory model and table 2019-12-20 16:16:29 +00:00