Commit Graph

258 Commits

Author SHA1 Message Date
Martyn Inglis
75c5e5e5d5 Merge branch 'master' into redo-queue-visibitlity-timeout
Conflicts:
	app/notifications/process_notifications.py
	app/v2/notifications/post_notifications.py
2017-05-30 10:18:18 +01:00
Imdad Ahad
6c4377bd44 Persist normalised email 2017-05-26 10:26:07 +01:00
Imdad Ahad
cbc92a6173 Store the normalised number on the notification 2017-05-26 10:26:07 +01:00
Martyn Inglis
2591d3a1df This massive set of changes uses the new queue names object throughout the app and tests.
Lots of changes, all changing the line of code that puts things into queues, and the code that tests that.
2017-05-25 10:51:49 +01:00
Rebecca Law
9f6c037530 Use iso8601 to validate scheduled_for datetime.
Added a validation method that always fails for scheduled notifications.
Comment out config for scheduled task.
The schedule notifications will be turned on once we can invite services to use it.
Waiting for the service permission story, must commit this in order to keep things from going stale.
2017-05-24 16:27:15 +01:00
Rebecca Law
751abb4b99 Update dateformat for scheduled_for to include minutes. 2017-05-22 14:15:35 +01:00
Rebecca Law
3a3161ecc4 Merge branch 'master' into schedule-api-notification
Conflicts:
	app/celery/scheduled_tasks.py
	app/v2/notifications/post_notifications.py
	tests/app/celery/test_scheduled_tasks.py
2017-05-22 14:05:57 +01:00
Imdad Ahad
0e0c18583f Fix test data and how we parse the JSON 2017-05-19 10:16:48 +01:00
Martyn Inglis
8dc7a86148 Merge branch 'master' into async-job-stats
Conflicts:
	app/dao/services_dao.py
2017-05-18 09:34:54 +01:00
Rebecca Law
973cc2c4c9 Changed the scheduled_for datetime to only send and hour of a day to send.
Also expect the date being passed in is BST. The date is converted to UTC before saving. And converted to BST when returning a notification.
2017-05-17 15:06:15 +01:00
Martyn Inglis
ee5bb5f01a Fixed issues caused by single evaluation of method signature defaults 2017-05-17 13:27:05 +01:00
Martyn Inglis
09f45332eb Fixed issues caused by single evaluation of method signature defaults 2017-05-17 13:25:40 +01:00
minglis
05a179c6ef Merge pull request #941 from alphagov/load-service-on-auth
Load service on auth
2017-05-17 10:49:00 +01:00
Martyn Inglis
83a1b1526e Merge branch 'master' into async-job-stats
Conflicts:
	app/notifications/notifications_ses_callback.py
2017-05-17 09:49:43 +01:00
Martyn Inglis
d2a7a7b3c9 Fixed error in code 2017-05-16 13:55:32 +01:00
Martyn Inglis
1dc3970595 Code tidy up as be @imdad comments. 2017-05-16 12:55:12 +01:00
Rebecca Law
3c0da9c8fd Merge branch 'master' into schedule-api-notification 2017-05-16 10:50:53 +01:00
Imdad Ahad
e6db9ffc1b Force parse JSON received from SNS:
* An SNS callback containing JSON has a plaintext header set. Using
* request.get_json() will return None if the header is not
* application/json unless the force parameter is set to True
2017-05-16 10:29:27 +01:00
Rebecca Law
f0e2713bef Add scheduled_for in the post notification request form.
Return scheduled for in get_notification requests.
2017-05-15 17:27:38 +01:00
Imdad Ahad
4003edfa67 Add DVLA callback:
* Process SNS callback, trigger the update notifications celery task
* Put autoconfirm into its own method and use in callbacks
2017-05-15 11:12:31 +01:00
Imdad Ahad
7a10a91262 Revert "Process SNS request triggered by a DVLA S3 update" 2017-05-12 17:21:07 +01:00
Martyn Inglis
8ac821fcc4 Fixed import paths 2017-05-12 16:10:00 +01:00
Imdad Ahad
4d82512ec6 Update SES callback to use autconfirm method 2017-05-12 14:24:27 +01:00
Imdad Ahad
37165e5b6a Add autoconfirm sns in dvla callback 2017-05-12 14:24:27 +01:00
Imdad Ahad
20bb91bfdd Update DVLA callback to process request and call task (if it can) 2017-05-12 14:24:26 +01:00
Martyn Inglis
19c14982a2 Call the task wrapper outcome function in the statistics_tasks file. This wraps the logic around which tasks to creates and simplifies the logic in the Callback classes. 2017-05-09 22:03:57 +01:00
Martyn Inglis
caed193647 Use the new task wrapper methods rather than creating a task directly 2017-05-09 18:17:55 +01:00
Martyn Inglis
6e888260fc Fix import 2017-05-08 16:46:44 +01:00
Martyn Inglis
0e38259cb4 Remove all DAO get service calls, replaced with reference to new service object in context. 2017-05-05 15:23:06 +01:00
Martyn Inglis
a340ed6f46 Changes to how we log to avoid unneeded DB calls. 2017-05-05 15:21:13 +01:00
Sakis
01e3a88d55 Merge pull request #920 from alphagov/autoconfirm_http_subscriptions
Autoconfirm http subscriptions
2017-05-02 17:03:41 +01:00
Martyn Inglis
b0baa1532a Added some logging around rate limiting. 2017-05-02 11:14:45 +01:00
Martyn Inglis
a9539d892c Merge branch 'master' into rate-limit-api-calls
Conflicts:
	requirements.txt
	tests/app/notifications/rest/test_send_notification.py
	tests/app/notifications/test_validators.py
	tests/app/v2/notifications/test_post_notifications.py
2017-05-02 10:56:56 +01:00
Athanasios Voutsadakis
cfed90e502 Make code a bit more defensive and add som logging 2017-04-28 16:10:41 +01:00
Athanasios Voutsadakis
74433c9335 Add tests for autoconfirmation 2017-04-28 15:11:36 +01:00
Athanasios Voutsadakis
04b003c152 Add utility function for subscription autoconfirm 2017-04-28 15:11:36 +01:00
Imdad Ahad
252c3235d7 Persist intl fields on notification 2017-04-27 13:24:36 +01:00
Martyn Inglis
dfbca8043e Fixed typo errors 2017-04-26 21:08:43 +01:00
Martyn Inglis
2a0f8c8808 Validate International phone numbers
- uses new utils methods to validate phone numbers
- defaults to International=True on validation. This ensures the validator works on all numbers
- Then check if the user can send this message to the number internationally if needed.
2017-04-26 15:56:45 +01:00
Martyn Inglis
ad8e445558 Reformatted code 2017-04-26 10:24:39 +01:00
Martyn Inglis
e977e7cee4 Wired in a simple callback to handle SNS notifications from S3
S3 will send a message when a file lands - which will trigger processing of DVLA responses.
2017-04-25 14:56:16 +01:00
Martyn Inglis
803c21865b Allow rate limiting on a per env basis
- switched off on prod by default
2017-04-25 09:54:09 +01:00
Martyn Inglis
926b8a60f9 Adds in call to new rate limit method in the redis client
- both V1 and V2 APIs
- Rate limiting wrapped into a new method - check_rate_limiting
	- delegates to the previous daily limit and the new though put limit
- Rate limiting done on key type. Each key has it's own limit (number of requests) and interval (time period of requests)
- Configured in the config. Not done on a per-env basis though could be in the future.
2017-04-24 14:15:08 +01:00
Martyn Inglis
b0e5062df2 Added the random string reference to the letter
- uses the reference field on the notifications table to store a 16char random string used to cross reference DVLA letters back to the notification
- used as letter barcode does not have space for a UUID notification id

Depends on https://github.com/alphagov/notifications-utils/pull/149

Renamed the numeric_id to notification_reference in utils and changed validation rules to match this

Note also the persist_notification method set "reference" to be "client_reference" which is confusing and they are different things, so fixed this too.
2017-04-12 17:56:55 +01:00
Martyn Inglis
ac445114c7 When using a priority template put the notification in the priority queue.
- there is now a dedicated worker to handle priority queues. We don't now bundle this in the default worker.
2017-04-04 13:44:11 +01:00
Rebecca Law
3838fb583c The redis cache for daily limits and template usage was being updated for notifications with a test api key.
This PR corrects that.
2017-03-30 13:43:44 +01:00
Martyn Inglis
7dcd3164e2 Revert "Reinstating the 2 task model for API submitted notifications."
This reverts commit 1c154c4113.
2017-03-30 10:46:23 +01:00
Martyn Inglis
385a73da2b Revert "ensure we're passing through api keys and key types from notifications"
This reverts commit 25d1777937.
2017-03-30 10:38:41 +01:00
Leo Hemsted
25d1777937 ensure we're passing through api keys and key types from notifications
when we made the change to async persist notifications, we forgot to
pass through api_key_id and key_type. in send_sms/email, for legacy
reasons, they default to None/KEY_TYPE_NORMAL, so regardless of what
your api key was set up as, we would send real messages!

TODO: Once the PaaS transition is complete and the task changes are
reverted, remove the api_key_id and key_type params from the send_*
tasks entirely, as those are only called from the csv job flow, and
don't need them
2017-03-28 13:14:46 +01:00
Martyn Inglis
1c154c4113 Reinstating the 2 task model for API submitted notifications.
This is being done for the PaaS migration to allow us to keep traffic coming in whilst we migrate the database.

uses the same tasks as the CSV uploaded notifications. Simple changes to not persist the notification, and call into a different task.
2017-03-23 14:41:00 +00:00