Commit Graph

1854 Commits

Author SHA1 Message Date
Martyn Inglis
7260561ad5 Makes the app use the redis and statsd clients from utils 2016-12-01 17:20:05 +00:00
Martyn Inglis
9667d42008 Minor bump of utils 2016-12-01 12:47:33 +00:00
minglis
d2ce6f2808 Merge pull request #755 from alphagov/request-id-logging
Request id logging
2016-12-01 11:53:32 +00:00
minglis
0ddd8d93ae Merge pull request #753 from alphagov/logging-service-ids
Records the service ID if present in the request
2016-12-01 11:38:49 +00:00
Martyn Inglis
5ae21bf1d5 Bumped requirements version 2016-12-01 11:34:38 +00:00
Martyn Inglis
d8625f9da4 Fixed tests that failed due to changes in utils brought in by latest version.
- seems phonenumber/emailaddress from the CSV are now passed in as personalisation.
- assume the renderer does the correct thing here. Will need to check with @quis
2016-11-30 17:22:03 +00:00
Martyn Inglis
7fe56cf992 Add request ID logging to the API.
- adds a request ID if needed to use in all log events
- respects request-ids passed in from calling clients
2016-11-30 17:05:39 +00:00
Andras Ferencz-Szabo
2e0ab3eba1 Merge pull request #754 from alphagov/use_http_proxy_for_docker
Use http proxy for Docker
2016-11-30 16:55:13 +00:00
bandesz
07aa4cb0b0 Use http proxy for Docker 2016-11-30 15:57:08 +00:00
minglis
40cf6c049f Merge pull request #746 from alphagov/record-user-agent
Statsd counter for user agents.
2016-11-30 12:51:30 +00:00
Rebecca Law
6b079dc2bf Merge pull request #747 from alphagov/dont-send-message-twice
Protect send_sms send_email task from being executed twice
2016-11-30 11:12:41 +00:00
Martyn Inglis
23ca62a7ac Records the service ID if present, or if an ADMIN api call the string admin-api.
this is used later in logging of requests.
2016-11-30 10:59:55 +00:00
Rebecca Law
28fe1fe049 Merge branch 'master' into dont-send-message-twice
Conflicts:
	tests/app/notifications/test_validators.py
2016-11-29 16:13:43 +00:00
Rebecca Law
6ffd97648c Merge pull request #751 from alphagov/fix-mocks
Refactored a couple of tests that were slightly testing the wrong thing.
2016-11-29 16:04:55 +00:00
Rebecca Law
e38625632c Remove print 2016-11-29 16:03:58 +00:00
Martyn Inglis
c437615661 Merge branch 'master' into fix-mocks
Conflicts:
	tests/app/notifications/test_validators.py
2016-11-29 13:32:40 +00:00
Martyn Inglis
2182b5715e Refactored a couple of tests that were slightly testing the wrong thing.
- Ensure mock used correctly.
2016-11-29 13:27:57 +00:00
Rebecca Law
e8f045b352 Merge branch 'master' into dont-send-message-twice
Conflicts:
	tests/app/notifications/test_validators.py
2016-11-29 12:34:33 +00:00
Rebecca Law
94609cef79 Fix how the exception is logged. 2016-11-28 17:22:03 +00:00
Paul Craig
2ae8967a5f Merge pull request #749 from alphagov/pc-nab-numerous-notifications
Remove marshmallow schema validation + format error messages
2016-11-28 17:20:27 +00:00
Rebecca Law
241ea80a84 Merge pull request #750 from alphagov/fix-v2-post-response-body
Fix version 2 of post notification, return placeholders
2016-11-28 17:19:42 +00:00
Rebecca Law
3267a54ded Merge branch 'master' into dont-send-message-twice 2016-11-28 16:43:17 +00:00
Paul Craig
4082d38c0c Test invalid older_than, template_types, and bad ids
Come up with some simple tests in the routes, just to see we get
back what we expect as errors.
2016-11-28 15:56:59 +00:00
Rebecca Law
57a8f8d7fa The body of the content in the response to a POST v2/notifications was not replacing the placeholders.
This PR fixes that and adds a test for it.

I am confused as to why I had to change the test_validators test that is checking if the mock is called.
Why did this code pass on preview?
2016-11-28 15:49:29 +00:00
Paul Craig
2ca675eb73 Test invalid statuses and template_types
Run through a few scenarios in the `test_schema_notifications.py`
test file, calling `.validate()` on some data instead of actually
calling the route.
2016-11-28 15:10:13 +00:00
Paul Craig
ab990679b3 Replace marshmallow with jsonschema
Created a new schema that accepts request parameters for the
get_notifications v2 route.
Using that to validate now instead of the marshmallow validation.

Also changed the way formatted error messages are returned because
the previous way was cutting off our failing `enum` messages.
2016-11-28 14:22:51 +00:00
Paul Craig
1b846f3e74 Merge pull request #745 from alphagov/pc-nab-numerous-notifications
[#131977833] Create get all notifications endpoint for V2 API
2016-11-28 13:21:49 +00:00
Paul Craig
9b1375ba84 URLs in API responses have full URL 2016-11-28 11:13:11 +00:00
Rebecca Law
be113e031f Sometimes a message is picked up twice of the SQS queue, we need to safe gaurd ourselves for that.
In this PR the id for the notification is passed in and used to created the notification, which causes a integrity error.
Normally when we get a SQLAlchemy error here we send the message to the retry queue, but if the notification already exists
we just ignore it.
2016-11-25 17:32:01 +00:00
Paul Craig
268e19bb76 Fix ServiceWhiteList __repr__
Apparently this code was unreachable.
2016-11-25 16:58:46 +00:00
Paul Craig
ffb813bb3f Add get_notifications json schema + tests
Add a schema that corresponds to our response for returning lists
of notifications, and test with a contract test.
2016-11-25 16:50:20 +00:00
Paul Craig
df7450698c Test returning notifications by "?status=failed"
Check that all failure states are returned by asking for
notifications of type "failure".
2016-11-25 15:26:40 +00:00
Paul Craig
57a0d7295d Rewrite failed statuses
There are no more notifications whose statuses are "failed", as
the "failed" status has now been replaced with statuses that are
more specific about the nature of the failure.

However, we still want to be able to filter by failing
notifications. (ie "/v2/notifications?status=failed").

Created a `.substitute_status()` method which takes a status
string or list of status strings and, if it finds 'failure',
replaces it with the other failing status types.

This way, calling for nottifications with "?status=failed" is
internally treated as
"status = ['technical-failure', 'temporary-failure', 'permanent-failure']"
2016-11-25 15:23:46 +00:00
Paul Craig
7d009915a4 Group 'completed' notification statuses
Some notification statuses assume that a notification has been
updated (ie, it cannot have been created in that state).

This caused a bug in our sample notification fixture when trying
to create a notificaiton in a 'complete' status.

This commit groups the completed statuses in a list, the way other
statuses have been grouped together so that they're more portable.
Also fixed the sample_notification fixture.
2016-11-25 15:23:04 +00:00
Paul Craig
640f51fc0a Add tests for getting back lists of notifications 2016-11-25 15:23:04 +00:00
Paul Craig
effbd315e0 Create 'v2' get notifications route
Return multiple notifications for a service.
Choosing a page_size or a page_number is no longer allowed.
Instead, there is a `next` link included with will return the
next {default_page_size} notifications in the sequence.

Query parameters accepted are:

 - template_type: filter by specific template types
 - status: filter by specific statuses
 - older_than: return a chronological list of notifications older
   than this one. The notification with the id that is passed in
   is _not_ returned.

Note that both `template_type` and `status` can accept multiple
parameters.  Thus it is possible to call
`/v2/notifications?status=created&status=sending&status=delivered`
2016-11-25 15:23:03 +00:00
Paul Craig
1fce30aaa7 Cost savings
The "cost" value was flawed for a couple of reasons.

1. Lots of messages are free, so in those instances the "cost"
   doesn't tell you anything
2. The query to get the rate was expensive and we don't have
   an obvious way to get it back very efficiently for large numbers
   of notifications.

So we scrapped it.
2016-11-25 15:23:03 +00:00
Martyn Inglis
e698d124ba Statsd counter for user agents.
We want to log the usage of the various API clients we have so that we understand when they can be cycled.

To this end we are going to count usage in statsd.

All notify clients have a suer agent, of the format: NOTIFY-API-{LANGUAGE}-CLIENT/version.number

For example, NOTIFY-API-PYTHON-CLIENT/3.0.0

We convert that into a statsd/graphite friendly string of the format: notify-api-python-client.3-0-0

So we can subdivide on client and client version on our dashboards.

Present but unknown User agents are records as "non-notify-user-agent"

Missing are presented as "unknown"
2016-11-25 13:27:36 +00:00
Paul Craig
f1ea39d4c0 Simplify _filter_query() function 2016-11-25 11:23:20 +00:00
Rebecca Law
669126d41b Merge pull request #744 from alphagov/add-default-exception-handler
Add application level error handler.
2016-11-25 09:34:23 +00:00
minglis
2e1578656b Merge pull request #735 from alphagov/caching-with-redis
Caching with redis
2016-11-25 09:06:12 +00:00
Rebecca Law
52c8596c92 Add application level error handler.
This should catch any unexcpeted exceptions.
2016-11-24 17:09:56 +00:00
bandesz
8fa337927c Exclude cache directory from pep8 2016-11-23 12:39:48 +00:00
Paul Craig
be5d503133 Merge pull request #743 from alphagov/stringify-cost-attr-on-v2-get-notification
Stringify the cost before passing to jsonify as it complains otherwise
2016-11-23 10:50:59 +00:00
Rebecca Law
97379a2904 Merge pull request #742 from alphagov/upgrade-python-client
Update python client from version 2.0.0 to 3.0.0
2016-11-23 10:48:21 +00:00
Martyn Inglis
2a2ef34339 Merge branch 'master' into caching-with-redis
Conflicts:
	app/__init__.py
2016-11-23 09:12:11 +00:00
Paul Craig
92897e20a8 Test returning a notification with a non-zero cost
Our previous test ws returning a notification without a `sent_by`
attribute, which meant that cost was always 0.
Unfortunately, this meant that returning a real value for cost was
untested and (whaddya know) it broke immediately.

Old test scenario:
- billable_units=1, sent_by=None, cost=0

New scenarios
- billable_units=0, sent_by='mmg', cost=0
- billable_units=1, sent_by='mmg', cost=1
2016-11-22 18:01:16 +00:00
Paul Craig
a17adb4707 New test for notification.cost() method
Added a test that asking for a nonexistent provider_rate blows
everything up.

Also updated existing test to use a weirder number.
2016-11-22 18:01:16 +00:00
Imdad Ahad
9b6584c8a1 Make result of notification.cost() into a float
Cost was returning a `Decimal`, which jsonify doesn't like.
Making it a float fixes the problem.

[Relevant issue on github](https://github.com/pallets/flask/issues/835).
2016-11-22 17:58:16 +00:00
Rebecca Law
3cddc64e36 Merge pull request #741 from alphagov/fix-v2-post-notification
Small fixes for the V2 post notification endpoints
2016-11-22 16:28:13 +00:00