Commit Graph

100 Commits

Author SHA1 Message Date
Adam Shimali
731bb19a9c Template and personalisation content is now merged and returned with
notifications, when retrieved by notification id, or service id (i.e.
all notifications for service).

There is a new element returned at top level of notification json called
body, which is the template content merged with personalisation. This
is consistent with api to endpoint to create notification which returns
what was sent as 'body' in json response.

Merging of template with personalisation is done in the
NotificationStatusSchema.

Personalisation data in encrypted before storing in db.
2016-06-20 16:49:17 +01:00
Martyn Inglis
838bf1d7d4 Merge branch 'master' into split-sms-and-retry
Conflicts:
	app/celery/tasks.py
	tests/app/celery/test_tasks.py
2016-06-17 16:39:03 +01:00
Rebecca Law
fa6ae864d6 Remove the invitation email and reset password email tasks. 2016-06-17 13:49:33 +01:00
Martyn Inglis
0a933021d6 Merge branch 'master' into split-sms-and-retry 2016-06-14 12:55:16 +01:00
Martyn Inglis
f143aade71 Added new tests/code to ensure failure stats recorded 2016-06-13 16:40:46 +01:00
Rebecca Law
a6ce109c62 Rename first send_email to send_email_v1.
Fix the tests
2016-06-13 16:16:56 +01:00
Rebecca Law
5fc14940f3 Let the send_email task set the from address 2016-06-13 14:58:00 +01:00
Martyn Inglis
bec882ba41 Merge branch 'master' into split-sms-and-retry 2016-06-13 11:39:13 +01:00
Martyn Inglis
6b5b40b953 Added tests for statsd and provider stats outcomes with the new provider stats tasks
- note large change to DAO to remove provider from create notification. Added on send now not creation.
2016-06-13 11:38:25 +01:00
Rebecca Law
50ec2eb7fc Remove task send_sms_code.
We use a notify service to send the sms code via using a template.
2016-06-09 11:54:34 +01:00
Nicholas Staples
8a1f4de217 Task added to update 'sending' notifications after 72 hours, set task to temporary-failure. 2016-06-08 15:25:57 +01:00
Martyn Inglis
92e4c0872b Pulled out the tests that send SMSs into provider_tasks file/tests
- updated tests to validate creation of next task, not sending direct to provider
2016-06-07 12:53:31 +01:00
Martyn Inglis
642aa2890e Starting to implement tests across the new tasks file 2016-06-07 11:09:54 +01:00
Martyn Inglis
754ccbe9af Removed update reference from updating the provider stats
- single focus method
- allows not to pollute DAO with research mode
2016-06-02 09:52:47 +01:00
Martyn Inglis
099c17192d Merged provider stats only on success branch
- ensures that don't raise stats in research mode
2016-06-02 09:30:01 +01:00
Martyn Inglis
b08f906662 Added 2 tests to ensure provider stats not updated when in research nmode 2016-06-02 09:28:21 +01:00
Martyn Inglis
97e29b9039 Merge branch 'master' into research-mode-implementation 2016-06-01 17:06:38 +01:00
Martyn Inglis
290f416485 Various tidy ups and changes
- use new queue
- remove some new lines in AWS response data
- migration script to populate new fields
2016-06-01 16:57:57 +01:00
Martyn Inglis
909fac3c05 Added research mode tasks
- if a service is in research mode the don't send the notifications via the providers (MMG/SES/etc)
- instead set up a task to mimic those services callbacks - this completes the loop, and show stats, delivery receipts and so on.
- Use the "to" field to choose the response, allows users to create successful and errored notifications

temp fail sms, uses  "07833333333"
perm fail sms, uses = "07822222222"
success = "07811111111" (or anything else)

success email = "delivered@simulator.notify"
perm fail = "perm-fail@simulator.notify"
temp fail = "temp-fail@simulator.notify"
2016-05-31 16:55:26 +01:00
Nicholas Staples
79dfc360f2 Notification not created in send_sms or send_email when service is in restricted mode. 2016-05-31 14:55:06 +01:00
Nicholas Staples
0fe0c1d2b4 Added job row number to the notification for csv jobs. All tests passing. 2016-05-19 10:46:03 +01:00
Adam Shimali
e1685d6a45 Was incorrectly setting from not reply to 2016-05-17 14:31:33 +01:00
Rebecca Law
4762aef1aa Merge pull request #318 from alphagov/tech-failures
New notification status types
2016-05-17 13:50:57 +01:00
Rebecca Law
191a79f27b Add new failure status for notifications. 2016-05-17 13:06:08 +01:00
Adam Shimali
3812dac3fe Missing mock 2016-05-17 12:49:37 +01:00
Adam Shimali
642fb6f68b Process job for email uses reply to if present 2016-05-17 12:41:13 +01:00
minglis
3c751d94a4 Merge pull request #309 from alphagov/statsd-integration
Statsd integration
2016-05-16 12:29:14 +01:00
Martyn Inglis
785def9117 Timed the tasks - how long to process a task / a sms-send / a email-send 2016-05-16 11:55:00 +01:00
Martyn Inglis
a520dcef47 Fixed some more pep8 2016-05-13 17:22:03 +01:00
Martyn Inglis
3f7559b286 Added statsd integration into the API
- new client for statsd, follows conventions used elsewhere for configuration
- client wraps underlying library so we can use a config property to send/not send statsd

Added statsd metrics for:
- count of API successful calls SMS/Email
- count of successful task execution for SMS/Email
- count of errors from Client libraries
- timing of API calls to third party clients
- timing of how long messages live on the SQS queue
2016-05-13 17:15:39 +01:00
Rebecca Law
917110870d Use the template version at the time the notification is created or at the time the job is created.
Update notifications/sms|email endpoint to send the template version to the queue.
Update the process_job celery talk to send the template version to the queue.
When the send_sms|send_email task runs it will get the template by id and version.

Created a data migration script to add the template_vesion column for jobs and notifications.
The existing jobs and notifications are given the template_version of the current template.
There is a chance this is the wrong template version, but deemed okay since the application is not live.

Create unit test for the dao_get_template_versions method.
Rename /template/<id>/version to /template/<id>/versions which returns all versions for that template id and service id.
2016-05-13 16:25:05 +01:00
Martyn Inglis
09b3313ce3 Merge branch 'master' into primary-provider
Conflicts:
	tests/app/dao/test_provider_rates_dao.py
2016-05-11 14:04:15 +01:00
Martyn Inglis
42c42bfdc9 Added tests to ensure pick correct provider from the database in the celery tasks. 2016-05-11 14:03:12 +01:00
Nicholas Staples
03f15d6af9 Update now to utcnow. All tests passing. 2016-05-11 10:56:24 +01:00
Martyn Inglis
571686b638 Ensure that the primary provider is used in all tasks 2016-05-10 09:04:22 +01:00
Chris Hill-Scott
b282f5015b Merge pull request #262 from alphagov/email-from-name
Send emails with a friendly from name
2016-04-26 11:24:49 +01:00
Chris Hill-Scott
eeaeacbf0b Remove send email code task
We don’t send email codes any more
2016-04-26 08:55:09 +01:00
Chris Hill-Scott
ac0fa5b211 Send emails with a friendly from name
It’s nicer to have emails with a sender name, as well as the raw email
address.

Amazon SES can acheive this by using the format
```
"Sender name" <sender.name@domain.com>
```
— http://docs.aws.amazon.com/ses/latest/DeveloperGuide/email-format.html
We also have to remove all non-ASCII characters from the sender name,
because SMTP only supports 7-bit ASCII:

> A field name MUST be composed of printable US-ASCII characters (i.e.,
> characters that have values between 33 and 126, inclusive), except
> colon.

— http://www.ietf.org/rfc/rfc5322.txt

We use the service name as the sender name when:
- sending emails from the API
- sending emails from a CSV file

We use GOV.UK Notify as the sender name when:
- sending invitation emails
- sending password reset emails
2016-04-25 16:57:17 +01:00
Nicholas Staples
3b1423a2ea Provider Statistics added.
Rates command added with a test.

Updated to include added migration.
2016-04-21 13:47:04 +01:00
Nicholas Staples
bee23281fc Updated text send to user for the authorization code. 2016-04-20 12:02:17 +01:00
Nicholas Staples
1b76646170 Working placeholders in subject with upgraded notifications-utils.
Fix test.
2016-04-13 15:53:27 +01:00
Nicholas Staples
7fa33a52a9 Fixed bug with deleting the 'delivered' notifications. 2016-04-13 12:49:38 +01:00
Nicholas Staples
90f0505a3d Update limit to message_limit.
Further db changes and updates.

Remove traceback print out.

Fix bug in passing template id to a task.
2016-04-11 16:53:40 +01:00
Nicholas Staples
143d1b0db8 Updated to retrieve csv upload from new bucket.
Fix test errors.
2016-04-07 14:10:30 +01:00
Rebecca Law
323b2ff537 Use MMG client for send-sms 2016-04-06 15:56:34 +01:00
Rebecca Law
e8d5a9292a Merge branch 'master' into integrate_MMG
Conflicts:
	app/notifications/rest.py
2016-04-06 14:58:13 +01:00
Rebecca Law
4806123d5c Add process_mmg_responses
Refactor process_firetext_responses
Removed the abstract ClientResponses for firetext and mmg. There is a map for each response to handle the status codes sent by each client.
Since MMG has about 20 different status code, none of which seem to be a pending state (unlike firetext that has 3 status one for pending - network delay).
For MMG status codes, look for 00 as successful, everything else is assumed to be a failure.
2016-04-06 14:31:33 +01:00
Chris Hill-Scott
044f2b7896 Merge pull request #203 from alphagov/double-check-restricted-mode
Catch sending to restricted recipients in Celery
2016-04-06 13:10:45 +01:00
Chris Hill-Scott
a63d6aa168 Add test for sending email while service restricted 2016-04-05 15:27:16 +01:00
Chris Hill-Scott
eef6d80ae2 Catch sending to restricted recipients in Celery
The Celery `send_sms` and `send_email` tasks _could_ assume that all the
recipients it gets are safe, because they have been checked either:
- when the admin app processes the CSV
- in the `/notifications/sms|email` endpoint

*However*, it’s probably safer to make the check again when the Celery
task run and passes the message off to the third party.

This also means that changing a service _back_ to restricted would have
an effect on messages that were queued, as well as all subsequent
messages.

This commit:
- restores the test that was removed here:
  e56aee5d1d (diff-e5627619e387fccc04783c32a23e6859L346)
- adds checks back into the Celery tasks for sending email and SMS,
  using the `allowed_to_send_to` function that was introduced into utils
  in https://github.com/alphagov/notifications-utils/pull/16
2016-04-05 15:11:27 +01:00