Commit Graph

4501 Commits

Author SHA1 Message Date
venusbb
d93b0d12d1 Minor change to raising deskpro ticket and errors when letter acknowledgement fails. 2018-01-23 09:51:43 +00:00
Venus Bailey
9629a69353 Merge pull request #1595 from alphagov/letter_ack_send_deskpro_prod_only
Processing ack files only send alerts in production environment.
2018-01-22 17:11:09 +00:00
venusbb
6c384342fd Merge branch 'letter_ack_send_deskpro_prod_only' of https://github.com/alphagov/notifications-api into letter_ack_send_deskpro_prod_only 2018-01-22 14:56:32 +00:00
venusbb
125b810050 Merge branch 'master' of https://github.com/alphagov/notifications-api into letter_ack_send_deskpro_prod_only 2018-01-22 14:54:06 +00:00
Venus Bailey
cc237b4273 Merge branch 'master' into letter_ack_send_deskpro_prod_only 2018-01-22 14:53:55 +00:00
venusbb
2edb0c0883 Grammar change to despro ticket
Grammar change to despro ticket
Strip sets of empty elements
Strip subfolder in returned zip file list
ar change to despro ticket
2018-01-22 14:51:06 +00:00
Chris Hill-Scott
930f28be0f Merge pull request #1593 from alphagov/stricter-phone
Be stricter about length of phone numbers
2018-01-22 13:52:32 +00:00
Chris Hill-Scott
c6e4aee80a Be stricter about length of phone numbers
Brings in:
- [x] https://github.com/alphagov/notifications-utils/pull/319
2018-01-22 13:37:47 +00:00
Chris Hill-Scott
83dd7ef156 Merge pull request #1581 from alphagov/dont-allow-whitelist-in-one-off
Don’t respect the whitelist for one off sending
2018-01-22 13:35:49 +00:00
venusbb
a87a17b996 Grammar change to despro ticket 2018-01-22 13:11:47 +00:00
venusbb
cf30e69e8c Processing ack files only send alerts in production environment.
Processing ack files only send alerts in production environment.
Deskpro alerts include bucket names for debugging purpose.
2018-01-22 12:44:03 +00:00
Chris Hill-Scott
01cf175cb2 Don’t respect the whitelist for one off sending
The whitelist was built to help developers and designers making
prototypes to do realistic usability testing of them, without having to
go through the whole go live process.

These users are sending messages using the API. The whitelist wasn’t
made available to users uploading spreadsheets. The users sending one
off messages are similar to those uploading spreadsheets, not those
using the API. Therefore they shouldn’t be able to use the whitelist to
expand the range of recipients they can send to.

Passing the argument through three methods doesn’t feel that great, but
can’t think of a better way without major refactoring…
2018-01-22 11:42:37 +00:00
Venus Bailey
2018660d12 Merge pull request #1589 from alphagov/letter-S3zipchange-deskpro
Fix a date time format problem - need to use AWS S3 format
2018-01-19 15:56:22 +00:00
Venus Bailey
5480dba44f Merge branch 'master' into letter-S3zipchange-deskpro 2018-01-19 14:40:05 +00:00
Rebecca Law
5029e4cb1f Merge pull request #1590 from alphagov/raise-400-for-validation-error
Raise 400 for a validation error
2018-01-19 13:56:00 +00:00
Rebecca Law
8a126bbfbb Merge branch 'master' into raise-400-for-validation-error 2018-01-19 13:41:24 +00:00
Rebecca Law
93ce1d2503 There were some exceptions in production today where a one off message was sending a message with an invalid email address.
The admin app validation did not catch this problem. But the API did.
This PR is a small fix to catch the erorr thrown by the notifications-utils/recipient validation methods and return a 400 status rather than a 500.
This only solves the issue of the user seeing "We are experiencing technical difficulties" rather than "invalid email address"

The bug can be replicated if you enter use quotes when entering the email address.

More work needs to be done so that the admin app does the same validation as the api so the user sees a nice form validtion error rather than a 400 after clicking send.
See: https://www.pivotaltracker.com/story/show/154472625
2018-01-19 12:23:07 +00:00
venusbb
4ffb84de36 Comparison of date needs to use AWS S3 format 2018-01-19 09:24:03 +00:00
Venus Bailey
db29874948 Merge pull request #1588 from alphagov/letter-S3zipchange-deskpro
Process letter acknowledge file with new S3 zip file meta file structure
2018-01-18 15:27:14 +00:00
venusbb
6a790b59aa Merge branch 'master' of https://github.com/alphagov/notifications-api into letter-S3zipchange-deskpro 2018-01-18 14:45:04 +00:00
venusbb
99dda99890 Use set rather than list to compare ack file and zip files difference 2018-01-18 14:44:23 +00:00
Chris Hill-Scott
ceb6e35924 Merge pull request #1582 from alphagov/backfill-redaction
Backfill redaction for pre-seeded templates
2018-01-18 13:11:28 +00:00
Chris Hill-Scott
d70fc958f6 Backfill redaction for pre-seeded templates
Will specifically affect the email auth template, which was seeded
without a matching row in the `template_redacted` table.

Effectively re-runs c24edcf388

> add historical redaction data
>
> every current template gets a row in the template_redacted table -
> this inserts one for any template that doesn't already have a row,
> with redact set to false, the user set to NOTIFY_USER since it was
> just a script, and the updated_at set to the time the script is run
2018-01-18 12:38:43 +00:00
Rebecca Law
cb66b7ae50 Merge pull request #1583 from alphagov/disable-letter-alerts-in-preview-staging
Disable Deskpro letters still sending alert in preview and staging
2018-01-18 12:37:38 +00:00
venusbb
357ec3a7d5 Call Deskpro ticket when there is an error 2018-01-18 11:06:07 +00:00
venusbb
8f5a5f8105 Parse acknowledgement files against .ZIP.TXT created by ftp app.
- Also convert the files info to upper() for comparison rather than lower
because original file names are in upper case. The unit tests contain examples of the returned lists.
2018-01-18 10:44:36 +00:00
Alexey Bezhan
70e3b07308 Disable Deskpro letters still sending alert in preview and staging
Since preview and staging environments don't have a full DVLA
integration they're likely to contain letter notifications in
a 'sending' state. To avoid spamming Deskpro we skip the check
unless we're in a production or test environment.
2018-01-17 17:14:20 +00:00
Alexey Bezhan
dec8a191a3 Merge pull request #1579 from alphagov/letters-still-sending-alert
Add a celery scheduled task to alert if letters are still sending
2018-01-17 16:12:28 +00:00
Alexey Bezhan
29777c3123 Add a celery scheduled task to alert if letters are still sending
We should receive a response file from DVLA by 4pm the next working
day (next Monday for letters created on Friday, Saturday or Sunday).

Response file triggers a task to update the letters status from
'sending' to either 'failed' or 'delivered', at which point there
should be no letter notifications in the 'sending' state for that day.

To catch any errors in the process (eg a missing response file from
DVLA) we add a scheduled task that checks letter notifications for
previous day (or Friday when run on Monday) and raises a Deskpro
ticket if it finds any in a 'sending' state. We're checking letter
notifications based on the `sent_at` date, which is set when the
letter PDF is sent to DVLA (so for letters created after 5:30pm it
will be the next day).

The task runs at 4:30pm, which should give the response file processing
task enough time to finish if the file was uploaded at 4pm.
2018-01-17 15:35:16 +00:00
Alexey Bezhan
5298f28f80 Add utils DeskproClient and configuration variables
Deskpro client is used to create tickets from celery alerting tasks
(eg alerts for missing ack or response files from DVLA).
2018-01-17 15:04:17 +00:00
Venus Bailey
6bdf59d80a Merge pull request #1578 from alphagov/raise-alert-when-no-ack-file
Fix a typo error on task argument
2018-01-17 15:01:03 +00:00
venusbb
7d84b4629a Using key query rather than try catch
Added a unit test for empty content
2018-01-17 14:45:34 +00:00
venusbb
9179802717 Fix a typo error on task argument
Modify unit test to be more robust
2018-01-17 12:21:56 +00:00
Venus Bailey
63bff9d9c3 Merge pull request #1567 from alphagov/raise-alert-when-no-ack-file
Raise alert as scheduled task to compare ack files with list of zip file on S3
2018-01-16 16:30:54 +00:00
venusbb
a90596fa1b Merge branch 'master' of https://github.com/alphagov/notifications-api into raise-alert-when-no-ack-file 2018-01-16 16:20:40 +00:00
venusbb
cd2e98c388 Change datetime to use utc 2018-01-16 16:06:08 +00:00
Leo Hemsted
3c3de0ca1f Merge pull request #1571 from alphagov/pyup-update-sqlalchemy-1.2.0-to-1.2.1
Update sqlalchemy to 1.2.1
2018-01-16 12:16:45 +00:00
Leo Hemsted
a1ebcc436e Merge branch 'master' into pyup-update-sqlalchemy-1.2.0-to-1.2.1 2018-01-16 12:00:27 +00:00
Leo Hemsted
3f574f2853 Merge pull request #1573 from alphagov/pyup-update-boto3-1.5.6-to-1.5.15
Update boto3 to 1.5.15
2018-01-16 12:00:06 +00:00
Leo Hemsted
bda70b99f7 Merge branch 'master' into pyup-update-boto3-1.5.6-to-1.5.15 2018-01-16 11:35:17 +00:00
Leo Hemsted
77cc0d1241 Merge pull request #1574 from alphagov/pyup-update-awscli-1.14.16-to-1.14.25
Update awscli to 1.14.25
2018-01-16 11:34:59 +00:00
Leo Hemsted
bcee0af4eb Merge branch 'master' into pyup-update-awscli-1.14.16-to-1.14.25 2018-01-16 11:17:51 +00:00
Leo Hemsted
0bf0a95b76 Merge pull request #1563 from alphagov/pyup-update-pytest-xdist-1.21.0-to-1.22.0
Update pytest-xdist to 1.22.0
2018-01-16 11:17:35 +00:00
Leo Hemsted
1369186797 Merge branch 'master' into pyup-update-pytest-xdist-1.21.0-to-1.22.0 2018-01-16 11:00:39 +00:00
Leo Hemsted
7533300967 Merge pull request #1572 from alphagov/only-collate-created
only send letters in created state to ftp app for zipping
2018-01-16 11:00:30 +00:00
venusbb
f273b23c25 Get ack files only from day before the ack file is received.
Take care of upper and lower case of file names and contents
Add a test for s3 get_list_of_files_by_suffix
2018-01-16 09:34:09 +00:00
pyup-bot
a2cf254a17 Update awscli from 1.14.16 to 1.14.25 2018-01-16 02:54:36 +00:00
pyup-bot
afcbfda566 Update boto3 from 1.5.6 to 1.5.15 2018-01-16 02:54:27 +00:00
Leo Hemsted
218fc5e14d only send letters in created state to ftp app for zipping
this means if we end up with some notifications sending and others not,
due to problems with the ftp connectivity for example, we don't re-send
those that worked.

As a reminder, letter pdf notifications start as created and stay that
way until we have sent the zip file to DVLA, at which point they are
updated to sending
 #
2018-01-15 17:00:00 +00:00
pyup-bot
0e6b18854b Update sqlalchemy from 1.2.0 to 1.2.1 2018-01-15 15:00:27 +00:00