Commit Graph

6878 Commits

Author SHA1 Message Date
Katie Smith
64c2061baa Use encryption module from utils
Now that the encryption module has been moved from this app to utils, we
can remove it from here (along with its tests) and import it from utils
instead. This also renames the `encryption.py` file to `hashing.py`,
since it no longer contains the encryption class.
2020-01-24 13:18:37 +00:00
Katie Smith
1703ae6031 Bump utils to version 36.5.0
This version of utils has the Encryption module included in it.
2020-01-24 13:18:27 +00:00
Leo Hemsted
8f2344e3b4 Merge pull request #2696 from alphagov/doc-dl-errors
handle document download errors properly
2020-01-20 14:20:43 +00:00
Leo Hemsted
a16f79896b refer to file rather than document
Karl Approved™
2020-01-20 13:44:52 +00:00
Leo Hemsted
99d008b383 handle document download errors properly
if doc download returns a 403, that's a screw-up on our side. it's not
helpful to a notify user for that to be passed on. the only thing they
should care about is if it's a 400, because they uploaded a filetype we
don't allow.

Everything else should return 500 internal server error.
2020-01-20 13:44:50 +00:00
David McDonald
65ce879825 Merge pull request #2698 from alphagov/alert-limit
Up threshold for sms to telephone numbers
2020-01-20 10:15:06 +00:00
David McDonald
3a0aece6a1 Up threshold for sms to telephone numbers
We were just ignoring the errors and our users were not fixing things.

Given that 500 texts cost approx £8 it's not the end of the world.

In the long run we may decide to just stop letting people try and send
messages to TV numbers but this is a quick fix to stop emails coming in
which we ignore.
2020-01-17 13:26:20 +00:00
Rebecca Law
879ba1d5f0 Merge pull request #2692 from alphagov/put-address-in-to-field-for-precompiled
Put address in to field for precompiled
2020-01-13 14:53:20 +00:00
Rebecca Law
722a4c9ebe Merge pull request #2697 from alphagov/send-alert-one-hour-sooner
Send the alert for `letters-still-sending` an hour earlier.
2020-01-13 13:49:55 +00:00
Rebecca Law
f4c0f70ba9 Send the alert for letters-still-sending an hour earlier.
These alerts are sent to our postal provider. And it usually arrives as they are getting ready to go home for the day or the weekend.
Which means they get missed/overlooked. They have agreed to get the alert an hour earlier, perhaps that will improved the response time.
2020-01-13 10:42:30 +00:00
David McDonald
9cd7c373b1 Merge pull request #2691 from alphagov/readme
Tidy up readme
2020-01-10 10:31:46 +00:00
Rebecca Law
bb2b514e12 Save recipient address in the "to" field of a notification
When a precompiled letter is sent via the admin app, we now pass in the address which can be set in the Notifications.to field.
Once a precompiled letters sent by the API has passed validation we can set the address in Notifications.to field.

The celery tasks to validate precompiled letters sent by the API will be done in another PR.
2020-01-07 14:35:48 +00:00
Rebecca Law
5ebd9a473c Add the recipient address in the "to" field for precompiled letters. 2020-01-07 14:35:48 +00:00
Rebecca Law
49eacced5a Merge pull request #2694 from alphagov/add-reference-to-notification-resp
Add client reference to notification reponse
2020-01-07 14:35:18 +00:00
Rebecca Law
cb87e66801 Add client_reference to the serialised notification returned for the csv reports. 2020-01-07 12:19:41 +00:00
Leo Hemsted
06298bd872 Merge pull request #2693 from alphagov/test-isolation-fix
Test isolation fix
2020-01-07 11:04:58 +00:00
David McDonald
ade08e23da Tidy up Readme 2020-01-07 10:26:07 +00:00
Leo Hemsted
eb72a0279f remove monkeypatch
it's interacting strangely with our os.environ stuff and not restoring
nicely, probably due to fixture order which is hard to visualise and
control.
2020-01-06 17:53:56 +00:00
Leo Hemsted
d244146638 fix weird test isolation errors with os.environ and boto3
test_config manipulates os.environ. os.environ is an `environ` object,
which acts like a dict but isn't in some subtle unknowable ways. The
`reload_config` fixture would create a dict copy of the env, and then
just call `os.environ = old_env` afterwards.

Boto3 would then complain that it couldn't load credentials (despite us
using the mock_s3 fixture and also not having creds in the environment
in the first place). Not entirely sure why this happens, but it does.
For some reason, it being a `dict` instead of an `environ` object causes
the mocking of boto3 to fail.

The solution is to not overwrite os.environ entirely, rather, use the
standard dictionary setitem syntax to update the values to their
previous values. Use `clear` to empty the environment too.
2020-01-06 16:21:34 +00:00
Leo Hemsted
c3b1766220 bump test requirements 2020-01-06 16:07:06 +00:00
Rebecca Law
b61262eb1c Merge pull request #2690 from alphagov/fix-job-counts
Fix job counts for older jobs
2020-01-02 09:52:30 +00:00
Rebecca Law
033bcb65d2 Update the dao_get_notification_outcomes_for_job to return the stats from either the notification table or the ft_notification_status table.
Currently if you visit the job page and the job is older than the data retention the totals on the page are all wrong because this query gets the counts from the notification table. With this change the data should always be correct. It also eliminates the need for looking at data retention. If the job is new and nothing has been created yet (i.e. the job hasn't started yet) then the page should show the correctly because the outcomes are empty (as expected), once the notifications for the jobs are created the numbers will start going up.
2019-12-30 16:17:00 +00:00
Rebecca Law
1d0247878f Merge pull request #2688 from alphagov/populate-returned-letters
Populate returned letters
2019-12-27 13:22:31 +00:00
Rebecca Law
84837b4eda Rename uploaded_letter to uploaded_letter_file_name 2019-12-27 12:39:24 +00:00
Rebecca Law
5e0ebe8e7a Fix conflicts with db migrations 2019-12-27 11:21:26 +00:00
Rebecca Law
63bdb0208b Create a new constant for datetime formats without timezone 2019-12-27 10:27:59 +00:00
Rebecca Law
e9baece3e7 Update the resultset to relect how users will consume the information.
- Do not show "hidden" or precompiled templates, users don't know about them.
- Remove the client reference if it is the file name of an uploaded file.
- Format the date for created_at
- Added a test for all the different types of letters.
 1) One off templated letter
 2) Letter created by a csv upload or job.
 3) Uploaded letter
 4) Templated letter sent by the API
 5) Precompiled letter sent by the API
2019-12-27 10:27:59 +00:00
Rebecca Law
cd29acc2f4 Add email address
Add uploaded_letter, the file name if the letter has been uploaded.
2019-12-27 10:27:59 +00:00
Rebecca Law
b853c4cdf1 Rename dao method to be more consistent.
Fix sort.
Add one to job_row_number, rows start at 0 which would confus the user.
2019-12-27 10:27:59 +00:00
Rebecca Law
aabaa4a971 Added joins to template, job and user for returned letter query.
Added unit tests
Comleted endpoint to get returned letter details
2019-12-27 10:27:59 +00:00
Rebecca Law
606b2b6c84 Change the insert to use updated_at as the reported_at date 2019-12-27 10:27:59 +00:00
Rebecca Law
5482c03bca [WIP] 2019-12-27 10:27:59 +00:00
Rebecca Law
22851c4ef0 Migration script to populate the returned letters. 2019-12-27 10:27:59 +00:00
David McDonald
9c8f5f8343 Merge pull request #2689 from alphagov/coveralls
Remove coveralls
2019-12-24 14:54:46 +00:00
David McDonald
6a8c7c1455 Merge pull request #2687 from alphagov/inbound-sms-history
Add and write into Inbound sms history
2019-12-24 10:13:32 +00:00
David McDonald
f948555ca8 Do nothing on db conflict
For notification and notification_history we do an upsert. Here, as the
inbound_sms table is never updated, only inserted to once (signified by
lack of updated_at field), an upsert would be unnecessary.

Therefore, if for some reason the delete statement failed as part of
moving data into the inbound_sms_history table, we can simply just
ignore any db conflicts raised by a rerun of
`delete_inbound_sms_older_than_retention`.
2019-12-24 09:39:06 +00:00
David McDonald
542f3b23aa Remove coveralls
We don't use it anymore
2019-12-24 08:37:19 +00:00
Pea Tyczynska
f8ff2d121f Changes following review:
- Check if right keys in new history rows
- Improve model and get rid of old revision version
- Add updated migration file
- Test data when inserting into inbound sms history
2019-12-20 16:17:27 +00:00
Pea Tyczynska
448cd1e94e Integrate inbound history insert into delete inbound sms function 2019-12-20 16:16:29 +00:00
Pea Tyczynska
a6b4675ae7 Populate inbound sms history when deleting inbound sms 2019-12-20 16:16:29 +00:00
Pea Tyczynska
de9d1a41eb Create InboundSmsHistory model and table 2019-12-20 16:16:29 +00:00
Chris Hill-Scott
54fd83f21e Merge pull request #2685 from alphagov/strip-whitespace-sms
Remove extraneous spacing characters from text messages
2019-12-19 13:47:56 +00:00
David McDonald
20125cf81e Merge pull request #2686 from alphagov/integer
Set healthcheck timeout as integer
2019-12-19 11:04:14 +00:00
David McDonald
9de3a9ff43 Set healthcheck timeout as integer
Not allowed to be a non integer so have upped to 3 rather than going to
2 (fairly arbitrary choice).
2019-12-19 10:55:38 +00:00
David McDonald
f791b3b734 Merge pull request #2684 from alphagov/healthcheck
Up API health check timeout to 2.5 seconds
2019-12-19 10:22:01 +00:00
Chris Hill-Scott
462a621256 Remove extraneous spacing characters from text messages
Brings in:
- [x] alphagov/notifications-utils#680
2019-12-18 11:24:09 +00:00
David McDonald
5aaf109ae1 Up API health check timeout to 2.5 seconds
This was after we saw an instance of the API failing it's healthcheck
even though it was still healthy enough to serve requests to users.

This follows the change we've also made to template-preview and admin of
upping the health check timeout. Unlike those where we set it to be 10
seconds, we have been less allowing here and only chosen 2.5 seconds.
This was at suggestion of Toby from PaaS as the api should generally
have quicker response times and more annoyance might be created for
users if we let an instance stick around for 10 seconds where it was
unable to serve requests successfully.
2019-12-18 10:36:04 +00:00
Katie Smith
897ed66348 Merge pull request #2675 from alphagov/new-letter-tasks
Add new tasks for processing letters which have passed virus scan
2019-12-17 16:16:29 +00:00
David McDonald
13aebb48d6 Merge pull request #2683 from alphagov/failure-rates
Improve developer experience for zendesk ticket
2019-12-17 13:26:11 +00:00
David McDonald
61f1469a20 Improve developer experience for zendesk ticket
We don't need to log this as an exception. It's not an exception, it's
behaviour that is not ideal but is still expected so therefore I've
changed it to warn. Also it removes the email we get for the exception
which is not needed as we get the zendesk ticket instead.

I've also fixed the multiline string meaning the link to the runbook is
included in the zendesk ticket.
2019-12-17 11:47:23 +00:00