Commit Graph

121 Commits

Author SHA1 Message Date
Rebecca Law
f5e79302cd Remove NotificationStatistics
NotificationStatistics was added as a spike but didn't work out as expected. This is finally removing all that unused code.
I'll drop the table in the next PR
2017-11-01 15:02:50 +00:00
venusbb
9ad75ff726 Tests modified 2017-09-27 10:15:29 +01:00
venusbb
6a7013fa7a Enable Inbound sms IP blocking 2017-09-26 10:59:09 +01:00
Rebecca Law
d4422dd35f Added a proper error response if the notification type is not supported. 2017-08-23 14:56:03 +01:00
Leo Hemsted
075d2a3346 tighten key_type validation on letters api
when in research mode or test key, dont send letters via api - instead,
just put them straight to success state

when using a team key, flat out reject the request (403)
2017-08-01 15:49:00 +01:00
Leo Hemsted
8e738b783e update test_send_notification to account for new uuid mock 2017-07-31 11:10:05 +01:00
Leo Hemsted
614880f6d9 send to send-sms-tasks and send-email-tasks instead of send-tasks 2017-07-21 13:49:37 +01:00
Ken Tsang
23618a186c Further refactoring 2017-07-06 12:27:57 +01:00
Ken Tsang
46a55c1cdb Refactor code 2017-07-06 12:27:57 +01:00
Ken Tsang
542bbb2f34 Refactor code 2017-07-06 12:27:56 +01:00
Ken Tsang
c1caa4a5da Add tests for when email / sms disabled 2017-07-06 12:27:56 +01:00
Ken Tsang
b04d01ba27 Refactored code to use new service permissions only 2017-07-06 12:27:55 +01:00
Ken Tsang
98cd838510 ken-use-only-new-service-permissions 2017-07-06 12:27:55 +01:00
Leo Hemsted
ac7665bfc6 celery test cleanup
* Alter config so an error will be raised if you forget to mock out a
  celery call in one of your tests
* Remove an unneeded exception type that was masking errors
2017-06-20 12:06:49 +01:00
Leo Hemsted
a1e570dea7 persist created_by_id when using the one off notification endpoint 2017-06-20 12:06:49 +01:00
Rebecca Law
3a66027d6a Refactor ApiKeys.secret and ServiceInboundApi.bearer_token to use the same encryption method and get rid of the duplicate code. 2017-06-19 14:32:22 +01:00
Martyn Inglis
2591d3a1df This massive set of changes uses the new queue names object throughout the app and tests.
Lots of changes, all changing the line of code that puts things into queues, and the code that tests that.
2017-05-25 10:51:49 +01:00
Imdad Ahad
0e0c18583f Fix test data and how we parse the JSON 2017-05-19 10:16:48 +01:00
Imdad Ahad
e6db9ffc1b Force parse JSON received from SNS:
* An SNS callback containing JSON has a plaintext header set. Using
* request.get_json() will return None if the header is not
* application/json unless the force parameter is set to True
2017-05-16 10:29:27 +01:00
Imdad Ahad
4003edfa67 Add DVLA callback:
* Process SNS callback, trigger the update notifications celery task
* Put autoconfirm into its own method and use in callbacks
2017-05-15 11:12:31 +01:00
Imdad Ahad
7a10a91262 Revert "Process SNS request triggered by a DVLA S3 update" 2017-05-12 17:21:07 +01:00
Imdad Ahad
37165e5b6a Add autoconfirm sns in dvla callback 2017-05-12 14:24:27 +01:00
Imdad Ahad
8a5e82904e Update to pull from correct bucket and fix tests not mocking out correctly 2017-05-12 14:24:26 +01:00
Imdad Ahad
20bb91bfdd Update DVLA callback to process request and call task (if it can) 2017-05-12 14:24:26 +01:00
Martyn Inglis
a9539d892c Merge branch 'master' into rate-limit-api-calls
Conflicts:
	requirements.txt
	tests/app/notifications/rest/test_send_notification.py
	tests/app/notifications/test_validators.py
	tests/app/v2/notifications/test_post_notifications.py
2017-05-02 10:56:56 +01:00
Martyn Inglis
83dc7c2bb7 Little test updates 2017-04-27 09:58:37 +01:00
Martyn Inglis
2a0f8c8808 Validate International phone numbers
- uses new utils methods to validate phone numbers
- defaults to International=True on validation. This ensures the validator works on all numbers
- Then check if the user can send this message to the number internationally if needed.
2017-04-26 15:56:45 +01:00
Martyn Inglis
e977e7cee4 Wired in a simple callback to handle SNS notifications from S3
S3 will send a message when a file lands - which will trigger processing of DVLA responses.
2017-04-25 14:56:16 +01:00
Martyn Inglis
926b8a60f9 Adds in call to new rate limit method in the redis client
- both V1 and V2 APIs
- Rate limiting wrapped into a new method - check_rate_limiting
	- delegates to the previous daily limit and the new though put limit
- Rate limiting done on key type. Each key has it's own limit (number of requests) and interval (time period of requests)
- Configured in the config. Not done on a per-env basis though could be in the future.
2017-04-24 14:15:08 +01:00
Chris Hill-Scott
1802c84b8a Add handling for data types other than list/string
Brings in:
- [ ] https://github.com/alphagov/notifications-utils/pull/135

Also adds extra tests for:
- the exact issue that we saw in production when #867 was deployed
- what happens when `None` is passed as a placeholder value, because
  this should never get as far as the relevant bit of utils
2017-04-05 09:28:17 +01:00
Chris Hill-Scott
7943b6819b Revert "Revert "Bump utils to allow lists as placeholder values"" 2017-04-05 09:18:58 +01:00
Chris Hill-Scott
cb208473c6 Revert "Bump utils to allow lists as placeholder values" 2017-04-04 15:19:53 +01:00
Martyn Inglis
ac445114c7 When using a priority template put the notification in the priority queue.
- there is now a dedicated worker to handle priority queues. We don't now bundle this in the default worker.
2017-04-04 13:44:11 +01:00
Chris Hill-Scott
585ef8e66a Bump utils to allow lists as placeholder values
Brings in:
- [ ] https://github.com/alphagov/notifications-utils/pull/128

Also adds a test to make sure that passing a dictionary as a value in
the `personalisation` argument works.
2017-03-31 17:02:54 +01:00
Martyn Inglis
7dcd3164e2 Revert "Reinstating the 2 task model for API submitted notifications."
This reverts commit 1c154c4113.
2017-03-30 10:46:23 +01:00
Martyn Inglis
9ee26e0b67 Revert "update notification rest tests"
This reverts commit 11919d9810.
2017-03-30 10:38:51 +01:00
Leo Hemsted
11919d9810 update notification rest tests 2017-03-28 13:51:42 +01:00
Martyn Inglis
1c154c4113 Reinstating the 2 task model for API submitted notifications.
This is being done for the PaaS migration to allow us to keep traffic coming in whilst we migrate the database.

uses the same tasks as the CSV uploaded notifications. Simple changes to not persist the notification, and call into a different task.
2017-03-23 14:41:00 +00:00
Rebecca Law
78242812ef Register a before_request event for all blueprints, that defines the authentication requirement.
There are three authentication methods:
 - requires_no_auth - public endpoint that does not require an Authorisation header
 - requires_auth - public endpoints that need an API key in the Authorisation header
 - requires_admin_auth - private endpoint that requires an Authorisation header which contains the API key for the defined as the client admin user
2017-03-16 18:15:49 +00:00
Chris Hill-Scott
e507fed152 Quietly ignore extra personalisation
> If a user makes an API request with additional personalisation fields,
> we should simply discard any fields that the template doesn't have.
>
> This gives a couple of related advantages:
>
> - modifying template parameters no longer requires downtime for
>   clients - as they can pass in extra new parameters before a template
>   change, or continue passing in old unused parameters after removing
>   them from a template
>
> - services can pass in large user objects, for example, and then play
>   around with templates adding and removing fields at will
>
> we should make sure we still return an error if a user doesn't pass in
> a required parameter.

– https://www.pivotaltracker.com/story/show/140774195
2017-03-07 16:09:17 +00:00
Rebecca Law
458adefcb8 Added a redis cache for the template usage stats.
Cache expires every 10 minutes, but will help with the every 2 second query, especially when a job is running.
There is some clean up and qa to do for this yet
2017-02-13 18:47:29 +00:00
Rebecca Law
c6eb284218 Small change to include the reference that ses send us on the callback. 2017-01-30 14:09:30 +00:00
Rebecca Law
27b6aa736c Update info log to waring.
Removed duplicate test
2017-01-27 12:20:54 +00:00
Rebecca Law
67657ced26 Return success of the callback is a duplicate or for an id that does not exist
This PR changes the response to POST /notifications/sms/<mmg | firetext> from a 400 response to a 200 response.
If we get a callback for a notification more than once or for a notification we log that but we return a 200 success response to the provider.
We have found that there is a situation where the send to provider throws a timeout exception but the provider did get the message, but we still send it to them again.
In which case they send the message twice, and callback for the message twice.
Another case where we may get duplicate callbacks is that the network gave the provider two callbacks meaning they pass those two callbacks onto us.

So it is really difficult to know if we sent to the provider twice or just got two callbacks.

The test_callback has many changes because I took the opportunity to use the client conftest fixture rather than the notify_api fixture.
The only 2 tests really changed are test_mmg_callback_returns_200_when_notification_id_not_found_or_already_updated and test_firetext_callback_returns_200_when_notification_id_not_found_or_already_updated
2017-01-25 14:37:11 +00:00
Rebecca Law
b34c5f0bcd If the template.process_type = PRIROITY send the message to the notify queue.
We are using the notify queue in this iteration because that queue is a low volume queue with it's own dedicated workers. This just saves us from building a new queue at this point, and a new queue may not be necessary.
2017-01-17 13:16:26 +00:00
Rebecca Law
a8cdabcecd - Refactor v2 post_notification to use a single method for sms and email.
- Added the `simulate` notification logic to version 2. We have 3 email addresses and phone numbers that are used
to simulate a successful post to /notifications. This was missed out of the version 2 endpoint.
- Added a test to template_dao to check for the default value of normal for new templates
- in v2 get_notifications, casted the path param to a uuid, if not uuid abort(404)
2017-01-17 12:08:24 +00:00
Leo Hemsted
d550893377 update tests to use create_user instead of sample_user
note that all of these tests have to be checked to ensure that they
still call through to notify_db_session (notify_db not required) to
tear down the database after the test runs - since it's no longer
required to pass it in to the function just to invoke the sample_user
function
2017-01-10 15:04:28 +00:00
Leo Hemsted
501187a9f4 bump utils to 13.0.1
brings in a fix to InvalidEmail/Phone/AddressExceptions not being
instantiated correctly. `exception.message` is not a python standard,
so we shouldn't be relying on it to transmit exception reasons -
rather we should be using `str(exception)` instead. This involved a
handful of small changes to the schema validation
2017-01-09 16:37:58 +00:00
Chris Hill-Scott
59af44d7ab Update utils to 12.0.0
Includes:

- [x] https://github.com/alphagov/notifications-utils/pull/94 (breaking
      changes which are responsible for all the changes to the API in
      this PR)

The test for `get_sms_fragment_count` has been removed because this
method is already tested in utils here:

ac20f7e99e/tests/test_base_template.py (L140-L159)
2016-12-13 10:57:01 +00:00
Martyn Inglis
9e2ba9ee81 Pushed the cache increment into the shared code that persists notifications.
Much simpler implementation, inc code removed from tasks and V1/V2 rest clients.
2016-11-22 12:53:20 +00:00