Commit Graph

181 Commits

Author SHA1 Message Date
Rebecca Law
316b5d0e64 Move scheduled tasks into their own module.
There is no change to the functionality - only moving the code around.
2016-06-20 13:33:53 +01:00
Martyn Inglis
838bf1d7d4 Merge branch 'master' into split-sms-and-retry
Conflicts:
	app/celery/tasks.py
	tests/app/celery/test_tasks.py
2016-06-17 16:39:03 +01:00
Rebecca Law
fa6ae864d6 Remove the invitation email and reset password email tasks. 2016-06-17 13:49:33 +01:00
Rebecca Law
0200d3ab11 Remove the extra task created to handle the change to the send_email signature 2016-06-15 15:28:53 +01:00
Martyn Inglis
0a933021d6 Merge branch 'master' into split-sms-and-retry 2016-06-14 12:55:16 +01:00
Rebecca Law
b31c4e801e Remove old email_registration_verification task 2016-06-14 11:43:13 +01:00
Martyn Inglis
f143aade71 Added new tests/code to ensure failure stats recorded 2016-06-13 16:40:46 +01:00
Rebecca Law
a6ce109c62 Rename first send_email to send_email_v1.
Fix the tests
2016-06-13 16:16:56 +01:00
Rebecca Law
cd2627e5ed Create a new task called send_email_v2 so that I can get rid of the from_address in the signature.
This is done to make sure we do not lose any messages in the queue during deployment.
2016-06-13 15:31:45 +01:00
Rebecca Law
5fc14940f3 Let the send_email task set the from address 2016-06-13 14:58:00 +01:00
Rebecca Law
c9f1eb65a7 Build the from address in the task instead of the rest call. 2016-06-13 14:57:59 +01:00
Martyn Inglis
bec882ba41 Merge branch 'master' into split-sms-and-retry 2016-06-13 11:39:13 +01:00
Rebecca Law
50ec2eb7fc Remove task send_sms_code.
We use a notify service to send the sms code via using a template.
2016-06-09 11:54:34 +01:00
Nicholas Staples
8a1f4de217 Task added to update 'sending' notifications after 72 hours, set task to temporary-failure. 2016-06-08 15:25:57 +01:00
Martyn Inglis
92e4c0872b Pulled out the tests that send SMSs into provider_tasks file/tests
- updated tests to validate creation of next task, not sending direct to provider
2016-06-07 12:53:31 +01:00
Martyn Inglis
642aa2890e Starting to implement tests across the new tasks file 2016-06-07 11:09:54 +01:00
Martyn Inglis
12503d6291 First spike to split up send-sms task
- 2 separate tasks - DB and Provider
- DB to persist notification
- Provider to contact provider
- Each piece has separate retries
- Provider retries have configured back-off
2016-06-03 14:54:46 +01:00
Martyn Inglis
754ccbe9af Removed update reference from updating the provider stats
- single focus method
- allows not to pollute DAO with research mode
2016-06-02 09:52:47 +01:00
Martyn Inglis
099c17192d Merged provider stats only on success branch
- ensures that don't raise stats in research mode
2016-06-02 09:30:01 +01:00
Martyn Inglis
001149c263 Merge branch 'master' into research-mode-implementation 2016-06-02 09:28:35 +01:00
Martyn Inglis
6ca2e588a9 Re-aligned the dao update that moved by mistake 2016-06-02 08:59:30 +01:00
Martyn Inglis
97e29b9039 Merge branch 'master' into research-mode-implementation 2016-06-01 17:06:38 +01:00
Martyn Inglis
290f416485 Various tidy ups and changes
- use new queue
- remove some new lines in AWS response data
- migration script to populate new fields
2016-06-01 16:57:57 +01:00
Nicholas Staples
276ca15919 Provider stats only updated if the provider successfully sends the message. 2016-06-01 12:43:26 +01:00
Martyn Inglis
909fac3c05 Added research mode tasks
- if a service is in research mode the don't send the notifications via the providers (MMG/SES/etc)
- instead set up a task to mimic those services callbacks - this completes the loop, and show stats, delivery receipts and so on.
- Use the "to" field to choose the response, allows users to create successful and errored notifications

temp fail sms, uses  "07833333333"
perm fail sms, uses = "07822222222"
success = "07811111111" (or anything else)

success email = "delivered@simulator.notify"
perm fail = "perm-fail@simulator.notify"
temp fail = "temp-fail@simulator.notify"
2016-05-31 16:55:26 +01:00
Nicholas Staples
79dfc360f2 Notification not created in send_sms or send_email when service is in restricted mode. 2016-05-31 14:55:06 +01:00
Nicholas Staples
0fe0c1d2b4 Added job row number to the notification for csv jobs. All tests passing. 2016-05-19 10:46:03 +01:00
Adam Shimali
e1685d6a45 Was incorrectly setting from not reply to 2016-05-17 14:31:33 +01:00
Rebecca Law
4762aef1aa Merge pull request #318 from alphagov/tech-failures
New notification status types
2016-05-17 13:50:57 +01:00
Rebecca Law
191a79f27b Add new failure status for notifications. 2016-05-17 13:06:08 +01:00
Adam Shimali
642fb6f68b Process job for email uses reply to if present 2016-05-17 12:41:13 +01:00
minglis
3c751d94a4 Merge pull request #309 from alphagov/statsd-integration
Statsd integration
2016-05-16 12:29:14 +01:00
Martyn Inglis
785def9117 Timed the tasks - how long to process a task / a sms-send / a email-send 2016-05-16 11:55:00 +01:00
Martyn Inglis
3f7559b286 Added statsd integration into the API
- new client for statsd, follows conventions used elsewhere for configuration
- client wraps underlying library so we can use a config property to send/not send statsd

Added statsd metrics for:
- count of API successful calls SMS/Email
- count of successful task execution for SMS/Email
- count of errors from Client libraries
- timing of API calls to third party clients
- timing of how long messages live on the SQS queue
2016-05-13 17:15:39 +01:00
Rebecca Law
917110870d Use the template version at the time the notification is created or at the time the job is created.
Update notifications/sms|email endpoint to send the template version to the queue.
Update the process_job celery talk to send the template version to the queue.
When the send_sms|send_email task runs it will get the template by id and version.

Created a data migration script to add the template_vesion column for jobs and notifications.
The existing jobs and notifications are given the template_version of the current template.
There is a chance this is the wrong template version, but deemed okay since the application is not live.

Create unit test for the dao_get_template_versions method.
Rename /template/<id>/version to /template/<id>/versions which returns all versions for that template id and service id.
2016-05-13 16:25:05 +01:00
Rebecca Law
c8c0f95dd2 Merge branch 'master' into add-template-version
Conflicts:
	tests/app/dao/test_notification_dao.py
2016-05-12 09:49:35 +01:00
Rebecca Law
f72f5aba05 [WIP]
Start to add template_version to jobs and notification
2016-05-11 17:04:51 +01:00
Martyn Inglis
b22c52131d Fixing up a couple of pull request comments
- python style if
- renamed the relationships
2016-05-11 15:36:17 +01:00
Martyn Inglis
571686b638 Ensure that the primary provider is used in all tasks 2016-05-10 09:04:22 +01:00
Chris Hill-Scott
c96c1799f9 Process CSV files with column headers of any case
Implements and depends on:
https://github.com/alphagov/notifications-utils/pull/27

Should be merged and deployed before the same change in the admin app.
2016-04-29 17:24:00 +01:00
Rebecca Law
559496a518 Merge pull request #275 from alphagov/fix-auth-bug
This pull request fixes a bug in authentication.
2016-04-29 14:52:20 +01:00
Rebecca Law
11c4696775 Update to test as per review comments.
However, I am curious if there is evidence that using fake_uuid over creating one in the test is actually faster.
2016-04-29 14:36:10 +01:00
Nicholas Staples
f71dbe9c0f Message limit added and all tests passing. 2016-04-29 10:36:59 +01:00
Chris Hill-Scott
b282f5015b Merge pull request #262 from alphagov/email-from-name
Send emails with a friendly from name
2016-04-26 11:24:49 +01:00
Chris Hill-Scott
eeaeacbf0b Remove send email code task
We don’t send email codes any more
2016-04-26 08:55:09 +01:00
Chris Hill-Scott
ac0fa5b211 Send emails with a friendly from name
It’s nicer to have emails with a sender name, as well as the raw email
address.

Amazon SES can acheive this by using the format
```
"Sender name" <sender.name@domain.com>
```
— http://docs.aws.amazon.com/ses/latest/DeveloperGuide/email-format.html
We also have to remove all non-ASCII characters from the sender name,
because SMTP only supports 7-bit ASCII:

> A field name MUST be composed of printable US-ASCII characters (i.e.,
> characters that have values between 33 and 126, inclusive), except
> colon.

— http://www.ietf.org/rfc/rfc5322.txt

We use the service name as the sender name when:
- sending emails from the API
- sending emails from a CSV file

We use GOV.UK Notify as the sender name when:
- sending invitation emails
- sending password reset emails
2016-04-25 16:57:17 +01:00
Nicholas Staples
3b1423a2ea Provider Statistics added.
Rates command added with a test.

Updated to include added migration.
2016-04-21 13:47:04 +01:00
Nicholas Staples
bee23281fc Updated text send to user for the authorization code. 2016-04-20 12:02:17 +01:00
Nicholas Staples
f681d40366 Character count added and tests passing.
Remove sql restriction for count to not be null.
2016-04-15 16:31:13 +01:00
Chris Hill-Scott
02f903b44c Stop GOV.UK becoming a link in email clients
Implements (and depends on)
https://github.com/alphagov/notifications-utils/pull/24

Works for:
- emails we send to users, eg invite, forgot password
- email templates that users have created
2016-04-15 12:33:58 +01:00