Commit Graph

1088 Commits

Author SHA1 Message Date
Rebecca Law
7a623de171 Added boby and subject to the get_notifications schema 2016-12-15 16:19:55 +00:00
Chris Hill-Scott
2c9a3f6f57 Merge pull request #764 from alphagov/more-breaking-utils-changes
Update utils to 12.1.1
2016-12-14 17:46:15 +01:00
Chris Hill-Scott
59af44d7ab Update utils to 12.0.0
Includes:

- [x] https://github.com/alphagov/notifications-utils/pull/94 (breaking
      changes which are responsible for all the changes to the API in
      this PR)

The test for `get_sms_fragment_count` has been removed because this
method is already tested in utils here:

ac20f7e99e/tests/test_base_template.py (L140-L159)
2016-12-13 10:57:01 +00:00
Imdad Ahad
431ec8acad Retrieve notifications by reference:
* Use get_notifications instead (references may not be unique)
2016-12-12 18:04:20 +00:00
Rebecca Law
4f9b295a5b Merge pull request #766 from alphagov/fix-sms-sender
Fix the from number in the post_sms_response.
2016-12-12 16:46:00 +00:00
imdadahad
551387251c Merge pull request #762 from alphagov/v2-get-notification-by-reference
V2 get notification by reference
2016-12-12 16:14:57 +00:00
Jenny Duckett
33e74a6b07 Merge pull request #765 from alphagov/use-notify-queue-to-deliver
Send Notify's 2FA codes via only one queue
2016-12-12 15:16:59 +00:00
Rebecca Law
9973d74d09 Fix the from number in the post_sms_response. If the service.sms_sender is not set the FROM_NUMBER from the config is returned, which is the default for the notification. 2016-12-12 15:07:03 +00:00
Jenny Duckett
e569c54f45 Send Notify's 2FA codes via only the notify queue
This means that these codes won't be delayed by large jobs going through the
send-sms/email queues. send_user_sms_code now works much more like the
endpoints for sending notifications, by persisting the notification and only
using the deliver_sms task (instead of using send_sms as well).

The workers consuming the `notify` queue should be able to handle the deliver
task as well, so no change should be needed to the celery workers to support
this.

I think there's also a change in behaviour here: previously, if the Notify
service was in research mode, 2FA codes would not have been sent out, making
it impossible to log into the admin. Now, a call to this endpoint will always
send out the notification even if we've put the Notify service into research
mode, since we set the notification's key type to normal and ignore the
service's research mode setting when sending the notification to the queue.
2016-12-09 16:46:49 +00:00
Jenny Duckett
d5d079a150 Add optional queue param to send_notification_to_queue
We want to use this function for sending internal notifications to a different
queue.
2016-12-09 16:46:49 +00:00
Rebecca Law
8c9e7b2d68 Update the tests for the new text 2016-12-09 12:28:28 +00:00
Pete Herlihy
e97c3bcc73 Tweaked the wording of the expired invitation message. 2016-12-09 12:20:03 +00:00
Imdad Ahad
620a90fd7c Make get_notifications return single notification by reference 2016-12-07 14:09:56 +00:00
Imdad Ahad
d4d733f997 Add dao method to get notification by reference with tests 2016-12-07 14:08:22 +00:00
Rebecca Law
eedc2fb43e Fix test and style 2016-12-07 13:29:19 +00:00
Rebecca Law
f946b022e8 Update the invitation expiry error message so that it is nicer to read. 2016-12-07 13:05:38 +00:00
Jenny Duckett
f02869c1e6 Merge pull request #758 from alphagov/stats-toggle-test-key
Add include_from_test_key parameter to /service
2016-12-06 10:58:10 +00:00
Jenny Duckett
d2649aebc8 Add include_from_test_key parameter to /service
We want to be able to toggle the numbers on the platform admin page between
including and excluding notifications sent using test keys, so that we can see
both real use of the platform and all load on it.

This parameter defaults to True, which is the existing behaviour.
2016-12-05 17:50:52 +00:00
Jenny Duckett
7668745d8b Allow excluding test key use in the all services stats query 2016-12-05 17:35:19 +00:00
Chris Hill-Scott
d449475dd5 Incorporate breaking utils changes
The `.replaced…` methods on instances of `Template` were removed in
https://github.com/alphagov/notifications-utils/pull/84
2016-12-02 10:42:11 +00:00
Martyn Inglis
7260561ad5 Makes the app use the redis and statsd clients from utils 2016-12-01 17:20:05 +00:00
minglis
d2ce6f2808 Merge pull request #755 from alphagov/request-id-logging
Request id logging
2016-12-01 11:53:32 +00:00
minglis
0ddd8d93ae Merge pull request #753 from alphagov/logging-service-ids
Records the service ID if present in the request
2016-12-01 11:38:49 +00:00
Martyn Inglis
7fe56cf992 Add request ID logging to the API.
- adds a request ID if needed to use in all log events
- respects request-ids passed in from calling clients
2016-11-30 17:05:39 +00:00
minglis
40cf6c049f Merge pull request #746 from alphagov/record-user-agent
Statsd counter for user agents.
2016-11-30 12:51:30 +00:00
Martyn Inglis
23ca62a7ac Records the service ID if present, or if an ADMIN api call the string admin-api.
this is used later in logging of requests.
2016-11-30 10:59:55 +00:00
Rebecca Law
e8f045b352 Merge branch 'master' into dont-send-message-twice
Conflicts:
	tests/app/notifications/test_validators.py
2016-11-29 12:34:33 +00:00
Rebecca Law
94609cef79 Fix how the exception is logged. 2016-11-28 17:22:03 +00:00
Paul Craig
2ae8967a5f Merge pull request #749 from alphagov/pc-nab-numerous-notifications
Remove marshmallow schema validation + format error messages
2016-11-28 17:20:27 +00:00
Rebecca Law
3267a54ded Merge branch 'master' into dont-send-message-twice 2016-11-28 16:43:17 +00:00
Paul Craig
4082d38c0c Test invalid older_than, template_types, and bad ids
Come up with some simple tests in the routes, just to see we get
back what we expect as errors.
2016-11-28 15:56:59 +00:00
Rebecca Law
57a8f8d7fa The body of the content in the response to a POST v2/notifications was not replacing the placeholders.
This PR fixes that and adds a test for it.

I am confused as to why I had to change the test_validators test that is checking if the mock is called.
Why did this code pass on preview?
2016-11-28 15:49:29 +00:00
Paul Craig
ab990679b3 Replace marshmallow with jsonschema
Created a new schema that accepts request parameters for the
get_notifications v2 route.
Using that to validate now instead of the marshmallow validation.

Also changed the way formatted error messages are returned because
the previous way was cutting off our failing `enum` messages.
2016-11-28 14:22:51 +00:00
Paul Craig
9b1375ba84 URLs in API responses have full URL 2016-11-28 11:13:11 +00:00
Rebecca Law
be113e031f Sometimes a message is picked up twice of the SQS queue, we need to safe gaurd ourselves for that.
In this PR the id for the notification is passed in and used to created the notification, which causes a integrity error.
Normally when we get a SQLAlchemy error here we send the message to the retry queue, but if the notification already exists
we just ignore it.
2016-11-25 17:32:01 +00:00
Paul Craig
268e19bb76 Fix ServiceWhiteList __repr__
Apparently this code was unreachable.
2016-11-25 16:58:46 +00:00
Paul Craig
ffb813bb3f Add get_notifications json schema + tests
Add a schema that corresponds to our response for returning lists
of notifications, and test with a contract test.
2016-11-25 16:50:20 +00:00
Paul Craig
df7450698c Test returning notifications by "?status=failed"
Check that all failure states are returned by asking for
notifications of type "failure".
2016-11-25 15:26:40 +00:00
Paul Craig
57a0d7295d Rewrite failed statuses
There are no more notifications whose statuses are "failed", as
the "failed" status has now been replaced with statuses that are
more specific about the nature of the failure.

However, we still want to be able to filter by failing
notifications. (ie "/v2/notifications?status=failed").

Created a `.substitute_status()` method which takes a status
string or list of status strings and, if it finds 'failure',
replaces it with the other failing status types.

This way, calling for nottifications with "?status=failed" is
internally treated as
"status = ['technical-failure', 'temporary-failure', 'permanent-failure']"
2016-11-25 15:23:46 +00:00
Paul Craig
7d009915a4 Group 'completed' notification statuses
Some notification statuses assume that a notification has been
updated (ie, it cannot have been created in that state).

This caused a bug in our sample notification fixture when trying
to create a notificaiton in a 'complete' status.

This commit groups the completed statuses in a list, the way other
statuses have been grouped together so that they're more portable.
Also fixed the sample_notification fixture.
2016-11-25 15:23:04 +00:00
Paul Craig
640f51fc0a Add tests for getting back lists of notifications 2016-11-25 15:23:04 +00:00
Paul Craig
effbd315e0 Create 'v2' get notifications route
Return multiple notifications for a service.
Choosing a page_size or a page_number is no longer allowed.
Instead, there is a `next` link included with will return the
next {default_page_size} notifications in the sequence.

Query parameters accepted are:

 - template_type: filter by specific template types
 - status: filter by specific statuses
 - older_than: return a chronological list of notifications older
   than this one. The notification with the id that is passed in
   is _not_ returned.

Note that both `template_type` and `status` can accept multiple
parameters.  Thus it is possible to call
`/v2/notifications?status=created&status=sending&status=delivered`
2016-11-25 15:23:03 +00:00
Paul Craig
1fce30aaa7 Cost savings
The "cost" value was flawed for a couple of reasons.

1. Lots of messages are free, so in those instances the "cost"
   doesn't tell you anything
2. The query to get the rate was expensive and we don't have
   an obvious way to get it back very efficiently for large numbers
   of notifications.

So we scrapped it.
2016-11-25 15:23:03 +00:00
Martyn Inglis
e698d124ba Statsd counter for user agents.
We want to log the usage of the various API clients we have so that we understand when they can be cycled.

To this end we are going to count usage in statsd.

All notify clients have a suer agent, of the format: NOTIFY-API-{LANGUAGE}-CLIENT/version.number

For example, NOTIFY-API-PYTHON-CLIENT/3.0.0

We convert that into a statsd/graphite friendly string of the format: notify-api-python-client.3-0-0

So we can subdivide on client and client version on our dashboards.

Present but unknown User agents are records as "non-notify-user-agent"

Missing are presented as "unknown"
2016-11-25 13:27:36 +00:00
Paul Craig
f1ea39d4c0 Simplify _filter_query() function 2016-11-25 11:23:20 +00:00
Rebecca Law
669126d41b Merge pull request #744 from alphagov/add-default-exception-handler
Add application level error handler.
2016-11-25 09:34:23 +00:00
minglis
2e1578656b Merge pull request #735 from alphagov/caching-with-redis
Caching with redis
2016-11-25 09:06:12 +00:00
Rebecca Law
52c8596c92 Add application level error handler.
This should catch any unexcpeted exceptions.
2016-11-24 17:09:56 +00:00
Paul Craig
be5d503133 Merge pull request #743 from alphagov/stringify-cost-attr-on-v2-get-notification
Stringify the cost before passing to jsonify as it complains otherwise
2016-11-23 10:50:59 +00:00
Martyn Inglis
2a2ef34339 Merge branch 'master' into caching-with-redis
Conflicts:
	app/__init__.py
2016-11-23 09:12:11 +00:00