Commit Graph

4782 Commits

Author SHA1 Message Date
venusbb
6a790b59aa Merge branch 'master' of https://github.com/alphagov/notifications-api into letter-S3zipchange-deskpro 2018-01-18 14:45:04 +00:00
venusbb
99dda99890 Use set rather than list to compare ack file and zip files difference 2018-01-18 14:44:23 +00:00
Chris Hill-Scott
ceb6e35924 Merge pull request #1582 from alphagov/backfill-redaction
Backfill redaction for pre-seeded templates
2018-01-18 13:11:28 +00:00
Chris Hill-Scott
d70fc958f6 Backfill redaction for pre-seeded templates
Will specifically affect the email auth template, which was seeded
without a matching row in the `template_redacted` table.

Effectively re-runs c24edcf388

> add historical redaction data
>
> every current template gets a row in the template_redacted table -
> this inserts one for any template that doesn't already have a row,
> with redact set to false, the user set to NOTIFY_USER since it was
> just a script, and the updated_at set to the time the script is run
2018-01-18 12:38:43 +00:00
Rebecca Law
cb66b7ae50 Merge pull request #1583 from alphagov/disable-letter-alerts-in-preview-staging
Disable Deskpro letters still sending alert in preview and staging
2018-01-18 12:37:38 +00:00
venusbb
357ec3a7d5 Call Deskpro ticket when there is an error 2018-01-18 11:06:07 +00:00
venusbb
8f5a5f8105 Parse acknowledgement files against .ZIP.TXT created by ftp app.
- Also convert the files info to upper() for comparison rather than lower
because original file names are in upper case. The unit tests contain examples of the returned lists.
2018-01-18 10:44:36 +00:00
Alexey Bezhan
70e3b07308 Disable Deskpro letters still sending alert in preview and staging
Since preview and staging environments don't have a full DVLA
integration they're likely to contain letter notifications in
a 'sending' state. To avoid spamming Deskpro we skip the check
unless we're in a production or test environment.
2018-01-17 17:14:20 +00:00
Alexey Bezhan
dec8a191a3 Merge pull request #1579 from alphagov/letters-still-sending-alert
Add a celery scheduled task to alert if letters are still sending
2018-01-17 16:12:28 +00:00
Alexey Bezhan
29777c3123 Add a celery scheduled task to alert if letters are still sending
We should receive a response file from DVLA by 4pm the next working
day (next Monday for letters created on Friday, Saturday or Sunday).

Response file triggers a task to update the letters status from
'sending' to either 'failed' or 'delivered', at which point there
should be no letter notifications in the 'sending' state for that day.

To catch any errors in the process (eg a missing response file from
DVLA) we add a scheduled task that checks letter notifications for
previous day (or Friday when run on Monday) and raises a Deskpro
ticket if it finds any in a 'sending' state. We're checking letter
notifications based on the `sent_at` date, which is set when the
letter PDF is sent to DVLA (so for letters created after 5:30pm it
will be the next day).

The task runs at 4:30pm, which should give the response file processing
task enough time to finish if the file was uploaded at 4pm.
2018-01-17 15:35:16 +00:00
Alexey Bezhan
5298f28f80 Add utils DeskproClient and configuration variables
Deskpro client is used to create tickets from celery alerting tasks
(eg alerts for missing ack or response files from DVLA).
2018-01-17 15:04:17 +00:00
Venus Bailey
6bdf59d80a Merge pull request #1578 from alphagov/raise-alert-when-no-ack-file
Fix a typo error on task argument
2018-01-17 15:01:03 +00:00
venusbb
7d84b4629a Using key query rather than try catch
Added a unit test for empty content
2018-01-17 14:45:34 +00:00
venusbb
9179802717 Fix a typo error on task argument
Modify unit test to be more robust
2018-01-17 12:21:56 +00:00
Venus Bailey
63bff9d9c3 Merge pull request #1567 from alphagov/raise-alert-when-no-ack-file
Raise alert as scheduled task to compare ack files with list of zip file on S3
2018-01-16 16:30:54 +00:00
venusbb
a90596fa1b Merge branch 'master' of https://github.com/alphagov/notifications-api into raise-alert-when-no-ack-file 2018-01-16 16:20:40 +00:00
venusbb
cd2e98c388 Change datetime to use utc 2018-01-16 16:06:08 +00:00
Leo Hemsted
3c3de0ca1f Merge pull request #1571 from alphagov/pyup-update-sqlalchemy-1.2.0-to-1.2.1
Update sqlalchemy to 1.2.1
2018-01-16 12:16:45 +00:00
Leo Hemsted
a1ebcc436e Merge branch 'master' into pyup-update-sqlalchemy-1.2.0-to-1.2.1 2018-01-16 12:00:27 +00:00
Leo Hemsted
3f574f2853 Merge pull request #1573 from alphagov/pyup-update-boto3-1.5.6-to-1.5.15
Update boto3 to 1.5.15
2018-01-16 12:00:06 +00:00
Leo Hemsted
bda70b99f7 Merge branch 'master' into pyup-update-boto3-1.5.6-to-1.5.15 2018-01-16 11:35:17 +00:00
Leo Hemsted
77cc0d1241 Merge pull request #1574 from alphagov/pyup-update-awscli-1.14.16-to-1.14.25
Update awscli to 1.14.25
2018-01-16 11:34:59 +00:00
Leo Hemsted
bcee0af4eb Merge branch 'master' into pyup-update-awscli-1.14.16-to-1.14.25 2018-01-16 11:17:51 +00:00
Leo Hemsted
0bf0a95b76 Merge pull request #1563 from alphagov/pyup-update-pytest-xdist-1.21.0-to-1.22.0
Update pytest-xdist to 1.22.0
2018-01-16 11:17:35 +00:00
Leo Hemsted
1369186797 Merge branch 'master' into pyup-update-pytest-xdist-1.21.0-to-1.22.0 2018-01-16 11:00:39 +00:00
Leo Hemsted
7533300967 Merge pull request #1572 from alphagov/only-collate-created
only send letters in created state to ftp app for zipping
2018-01-16 11:00:30 +00:00
venusbb
f273b23c25 Get ack files only from day before the ack file is received.
Take care of upper and lower case of file names and contents
Add a test for s3 get_list_of_files_by_suffix
2018-01-16 09:34:09 +00:00
pyup-bot
a2cf254a17 Update awscli from 1.14.16 to 1.14.25 2018-01-16 02:54:36 +00:00
pyup-bot
afcbfda566 Update boto3 from 1.5.6 to 1.5.15 2018-01-16 02:54:27 +00:00
Leo Hemsted
218fc5e14d only send letters in created state to ftp app for zipping
this means if we end up with some notifications sending and others not,
due to problems with the ftp connectivity for example, we don't re-send
those that worked.

As a reminder, letter pdf notifications start as created and stay that
way until we have sent the zip file to DVLA, at which point they are
updated to sending
 #
2018-01-15 17:00:00 +00:00
pyup-bot
0e6b18854b Update sqlalchemy from 1.2.0 to 1.2.1 2018-01-15 15:00:27 +00:00
venusbb
6a13450bfc remove s3 testscript from this repo
This file does not belong to this repo, although it is useful in initially testing the s3 boto3 scipts. Will consider to move it somewhere else.
2018-01-15 12:30:54 +00:00
Katie Smith
2f2fc712df Merge pull request #1562 from alphagov/fix-phone-validator-bug
Add validation to check that sms recipient is not None
2018-01-15 11:00:33 +00:00
Katie Smith
7b4abd076b Add validation to check that sms recipient is not None
Previously, if the SMS recipient was None there would be a 500 error
with no message displayed to the user. We now check if the recipient is
None and raise a BadRequestError if this is the case.
2018-01-15 09:20:46 +00:00
venusbb
96b3c6c2ed Add tests for checking not updating database when ack file is received.
Moved S3 test scripts
Modified scheduled_task test
2018-01-12 19:24:36 +00:00
venusbb
a016b3f8d6 Added a new sns callback for testing old dvla response file type.
Merge branch 'raise-alert-when-no-ack-file' of https://github.com/alphagov/notifications-api into raise-alert-when-no-ack-file
2018-01-12 16:11:06 +00:00
venusbb
27dd15ec9f Merge branch 'master' of https://github.com/alphagov/notifications-api into raise-alert-when-no-ack-file 2018-01-12 15:46:48 +00:00
venusbb
24b785e7e0 Added process for dvla acknowledgement file
Daily schedule task to check ack file against zip file lists
if we haven't receive ack for a zip file, raise a 500 exception
2018-01-12 15:44:00 +00:00
venusbb
e1d150a882 Added process for dvla acknowledgement file
Daily schedule task to check ack file against zip file lists
if we haven't receive ack for a zip file, raise a 500 exception
2018-01-12 15:10:42 +00:00
pyup-bot
a997fe94a8 Update pytest-xdist from 1.21.0 to 1.22.0 2018-01-11 20:27:25 +00:00
Rebecca Law
9c4e43bfac Some pseudo code and notes of how to implement a check for the letter acknowledgement file. 2018-01-11 16:37:39 +00:00
Chris Waszczuk
f206c5d032 Merge pull request #1561 from gov-cjwaszczuk/master
bumped utils version
2018-01-11 16:13:15 +00:00
chrisw
ccdbcf8499 bumped utils version 2018-01-11 16:03:07 +00:00
Katie Smith
79a7355da7 Merge pull request #1555 from alphagov/drop-rate-limit-database-default
Drop rate limit database default
2018-01-11 15:15:58 +00:00
Katie Smith
f6ef0ebbd7 Remove database default on rate_limit column
The default for the rate_limit column in the services and
services_history model is now set in the model, so we can remove the
default from the database.

Pivotal story: https://www.pivotaltracker.com/story/show/153992529
2018-01-11 10:37:26 +00:00
Katie Smith
e8fb9ee01e Merge pull request #1554 from alphagov/use-service-rate-limit-column
Get rate limit from service.rate_limit column (not config)
2018-01-11 10:36:43 +00:00
Katie Smith
b07db16cd1 Get rate limit from service.rate_limit column (not config)
PR #1550 added the rate_limit column to the Service table.

This PR removes the rate limits from the config and uses rate_limit from
the Service model instead. Rate limits are still separated into 'team',
'normal' and 'test', but these values are the same for a service.

Pivotal story https://www.pivotaltracker.com/story/show/153992529
2018-01-11 10:28:11 +00:00
Rebecca Law
f45469c98e Merge pull request #1556 from alphagov/template_default_address
Fix the template history when updating the reply_to for a template.
2018-01-10 13:45:35 +00:00
Rebecca Law
03deb804b0 -Fix typo
- return template from dao not the history
2018-01-10 13:32:54 +00:00
Rebecca Law
e59d6d470e Fix the problem with updating the reply_to or service_letter_contact_id for templates.
The history was not being updated properly, we think this is because the declaritive attribute is not being set propery by the property.
When reply_to: None it will update the service_letter_contact_id, but not the service_letter_contact, we think when the history_meta is build the history class and checking if the value is updated it depends which attribute it is checking first.

In order to fix this issue, there is a new dao method to update the reply_to on the Template and insert a new Template history.
2018-01-10 12:40:14 +00:00