Commit Graph

212 Commits

Author SHA1 Message Date
Martyn Inglis
001149c263 Merge branch 'master' into research-mode-implementation 2016-06-02 09:28:35 +01:00
Martyn Inglis
6ca2e588a9 Re-aligned the dao update that moved by mistake 2016-06-02 08:59:30 +01:00
Martyn Inglis
97e29b9039 Merge branch 'master' into research-mode-implementation 2016-06-01 17:06:38 +01:00
Martyn Inglis
290f416485 Various tidy ups and changes
- use new queue
- remove some new lines in AWS response data
- migration script to populate new fields
2016-06-01 16:57:57 +01:00
Nicholas Staples
276ca15919 Provider stats only updated if the provider successfully sends the message. 2016-06-01 12:43:26 +01:00
Martyn Inglis
909fac3c05 Added research mode tasks
- if a service is in research mode the don't send the notifications via the providers (MMG/SES/etc)
- instead set up a task to mimic those services callbacks - this completes the loop, and show stats, delivery receipts and so on.
- Use the "to" field to choose the response, allows users to create successful and errored notifications

temp fail sms, uses  "07833333333"
perm fail sms, uses = "07822222222"
success = "07811111111" (or anything else)

success email = "delivered@simulator.notify"
perm fail = "perm-fail@simulator.notify"
temp fail = "temp-fail@simulator.notify"
2016-05-31 16:55:26 +01:00
Nicholas Staples
79dfc360f2 Notification not created in send_sms or send_email when service is in restricted mode. 2016-05-31 14:55:06 +01:00
Nicholas Staples
0fe0c1d2b4 Added job row number to the notification for csv jobs. All tests passing. 2016-05-19 10:46:03 +01:00
Adam Shimali
e1685d6a45 Was incorrectly setting from not reply to 2016-05-17 14:31:33 +01:00
Rebecca Law
4762aef1aa Merge pull request #318 from alphagov/tech-failures
New notification status types
2016-05-17 13:50:57 +01:00
Rebecca Law
191a79f27b Add new failure status for notifications. 2016-05-17 13:06:08 +01:00
Adam Shimali
642fb6f68b Process job for email uses reply to if present 2016-05-17 12:41:13 +01:00
minglis
3c751d94a4 Merge pull request #309 from alphagov/statsd-integration
Statsd integration
2016-05-16 12:29:14 +01:00
Martyn Inglis
785def9117 Timed the tasks - how long to process a task / a sms-send / a email-send 2016-05-16 11:55:00 +01:00
Martyn Inglis
3f7559b286 Added statsd integration into the API
- new client for statsd, follows conventions used elsewhere for configuration
- client wraps underlying library so we can use a config property to send/not send statsd

Added statsd metrics for:
- count of API successful calls SMS/Email
- count of successful task execution for SMS/Email
- count of errors from Client libraries
- timing of API calls to third party clients
- timing of how long messages live on the SQS queue
2016-05-13 17:15:39 +01:00
Rebecca Law
917110870d Use the template version at the time the notification is created or at the time the job is created.
Update notifications/sms|email endpoint to send the template version to the queue.
Update the process_job celery talk to send the template version to the queue.
When the send_sms|send_email task runs it will get the template by id and version.

Created a data migration script to add the template_vesion column for jobs and notifications.
The existing jobs and notifications are given the template_version of the current template.
There is a chance this is the wrong template version, but deemed okay since the application is not live.

Create unit test for the dao_get_template_versions method.
Rename /template/<id>/version to /template/<id>/versions which returns all versions for that template id and service id.
2016-05-13 16:25:05 +01:00
Rebecca Law
c8c0f95dd2 Merge branch 'master' into add-template-version
Conflicts:
	tests/app/dao/test_notification_dao.py
2016-05-12 09:49:35 +01:00
Rebecca Law
f72f5aba05 [WIP]
Start to add template_version to jobs and notification
2016-05-11 17:04:51 +01:00
Martyn Inglis
b22c52131d Fixing up a couple of pull request comments
- python style if
- renamed the relationships
2016-05-11 15:36:17 +01:00
Martyn Inglis
571686b638 Ensure that the primary provider is used in all tasks 2016-05-10 09:04:22 +01:00
Chris Hill-Scott
c96c1799f9 Process CSV files with column headers of any case
Implements and depends on:
https://github.com/alphagov/notifications-utils/pull/27

Should be merged and deployed before the same change in the admin app.
2016-04-29 17:24:00 +01:00
Rebecca Law
559496a518 Merge pull request #275 from alphagov/fix-auth-bug
This pull request fixes a bug in authentication.
2016-04-29 14:52:20 +01:00
Rebecca Law
11c4696775 Update to test as per review comments.
However, I am curious if there is evidence that using fake_uuid over creating one in the test is actually faster.
2016-04-29 14:36:10 +01:00
Nicholas Staples
f71dbe9c0f Message limit added and all tests passing. 2016-04-29 10:36:59 +01:00
Chris Hill-Scott
b282f5015b Merge pull request #262 from alphagov/email-from-name
Send emails with a friendly from name
2016-04-26 11:24:49 +01:00
Chris Hill-Scott
eeaeacbf0b Remove send email code task
We don’t send email codes any more
2016-04-26 08:55:09 +01:00
Chris Hill-Scott
ac0fa5b211 Send emails with a friendly from name
It’s nicer to have emails with a sender name, as well as the raw email
address.

Amazon SES can acheive this by using the format
```
"Sender name" <sender.name@domain.com>
```
— http://docs.aws.amazon.com/ses/latest/DeveloperGuide/email-format.html
We also have to remove all non-ASCII characters from the sender name,
because SMTP only supports 7-bit ASCII:

> A field name MUST be composed of printable US-ASCII characters (i.e.,
> characters that have values between 33 and 126, inclusive), except
> colon.

— http://www.ietf.org/rfc/rfc5322.txt

We use the service name as the sender name when:
- sending emails from the API
- sending emails from a CSV file

We use GOV.UK Notify as the sender name when:
- sending invitation emails
- sending password reset emails
2016-04-25 16:57:17 +01:00
Nicholas Staples
3b1423a2ea Provider Statistics added.
Rates command added with a test.

Updated to include added migration.
2016-04-21 13:47:04 +01:00
Nicholas Staples
bee23281fc Updated text send to user for the authorization code. 2016-04-20 12:02:17 +01:00
Nicholas Staples
f681d40366 Character count added and tests passing.
Remove sql restriction for count to not be null.
2016-04-15 16:31:13 +01:00
Chris Hill-Scott
02f903b44c Stop GOV.UK becoming a link in email clients
Implements (and depends on)
https://github.com/alphagov/notifications-utils/pull/24

Works for:
- emails we send to users, eg invite, forgot password
- email templates that users have created
2016-04-15 12:33:58 +01:00
Nicholas Staples
1b76646170 Working placeholders in subject with upgraded notifications-utils.
Fix test.
2016-04-13 15:53:27 +01:00
Nicholas Staples
7fa33a52a9 Fixed bug with deleting the 'delivered' notifications. 2016-04-13 12:49:38 +01:00
Nicholas Staples
90f0505a3d Update limit to message_limit.
Further db changes and updates.

Remove traceback print out.

Fix bug in passing template id to a task.
2016-04-11 16:53:40 +01:00
Nicholas Staples
143d1b0db8 Updated to retrieve csv upload from new bucket.
Fix test errors.
2016-04-07 14:10:30 +01:00
Rebecca Law
340f8ceaf6 Update mmg send_sms to include cid and request headers
Use mmg to send_sms
2016-04-06 17:35:14 +01:00
Rebecca Law
323b2ff537 Use MMG client for send-sms 2016-04-06 15:56:34 +01:00
Rebecca Law
e8d5a9292a Merge branch 'master' into integrate_MMG
Conflicts:
	app/notifications/rest.py
2016-04-06 14:58:13 +01:00
Rebecca Law
4806123d5c Add process_mmg_responses
Refactor process_firetext_responses
Removed the abstract ClientResponses for firetext and mmg. There is a map for each response to handle the status codes sent by each client.
Since MMG has about 20 different status code, none of which seem to be a pending state (unlike firetext that has 3 status one for pending - network delay).
For MMG status codes, look for 00 as successful, everything else is assumed to be a failure.
2016-04-06 14:31:33 +01:00
Chris Hill-Scott
044f2b7896 Merge pull request #203 from alphagov/double-check-restricted-mode
Catch sending to restricted recipients in Celery
2016-04-06 13:10:45 +01:00
Chris Hill-Scott
eef6d80ae2 Catch sending to restricted recipients in Celery
The Celery `send_sms` and `send_email` tasks _could_ assume that all the
recipients it gets are safe, because they have been checked either:
- when the admin app processes the CSV
- in the `/notifications/sms|email` endpoint

*However*, it’s probably safer to make the check again when the Celery
task run and passes the message off to the third party.

This also means that changing a service _back_ to restricted would have
an effect on messages that were queued, as well as all subsequent
messages.

This commit:
- restores the test that was removed here:
  e56aee5d1d (diff-e5627619e387fccc04783c32a23e6859L346)
- adds checks back into the Celery tasks for sending email and SMS,
  using the `allowed_to_send_to` function that was introduced into utils
  in https://github.com/alphagov/notifications-utils/pull/16
2016-04-05 15:11:27 +01:00
NIcholas Staples
2e09ead30f Merge pull request #198 from alphagov/retain_all_notifications_for_7_days
Successful notifications are deleted after a week now. All tests pass…
2016-04-05 15:05:58 +01:00
Nicholas Staples
4cc0028b01 Remove csv after process job is finished.
Fixed new tests.
2016-04-05 14:55:03 +01:00
Nicholas Staples
4d15409781 Successful notifications are deleted after a week now. All tests passing. 2016-04-05 11:07:21 +01:00
Chris Hill-Scott
e56aee5d1d Validate recipient for restricted service w/ utils
Implements
https://github.com/alphagov/notifications-utils/pull/16

Once
https://github.com/alphagov/notifications-admin/pull/376
is merged it will no longer be possible for a user to upload a CSV file
containing recipients that they’re not allowed to send to.

So this commit also removes any restricted service checks in the task,
because any public phone numbers/email addresses no longer have any way
of reach this point if the service is restricted.
2016-04-04 20:10:19 +01:00
Rebecca Law
69697388d7 Initial set up for mmg client integration 2016-04-01 16:42:31 +01:00
Adam Shimali
ca9c886c3e [WIP] On create of notification. Upsert record for template stats
recording usages of template by day.
2016-03-31 15:57:50 +01:00
Chris Hill-Scott
e3ae81bd0a Strip trailing CSV rows, relax phone validation
Implements (and depends on):
- https://github.com/alphagov/notifications-utils/pull/14
- https://github.com/alphagov/notifications-utils/pull/15
2016-03-24 20:30:15 +00:00
Chris Hill-Scott
a0022455f0 Merge pull request #173 from alphagov/email-wrapper
Add HTML email
2016-03-18 15:42:32 +00:00
Chris Hill-Scott
f3ddf1f456 Add HTML email
Make the SES client optionally pass through HTML and text body, as per:
http://docs.aws.amazon.com/ses/latest/APIReference/API_Body.html

Gets the message as HTML from:
https://github.com/alphagov/notifications-utils/pull/12
2016-03-18 15:26:32 +00:00