Commit Graph

2113 Commits

Author SHA1 Message Date
bandesz
a67195a86d Set http proxy params for Docker build 2016-12-02 14:45:33 +00:00
Chris Hill-Scott
d449475dd5 Incorporate breaking utils changes
The `.replaced…` methods on instances of `Template` were removed in
https://github.com/alphagov/notifications-utils/pull/84
2016-12-02 10:42:11 +00:00
minglis
22f6161440 Merge pull request #752 from alphagov/update-readme
Update README
2016-12-02 10:14:00 +00:00
minglis
d76e7e72b1 Merge pull request #748 from alphagov/switch-redis-on
Set up REDIS config.
2016-12-02 10:13:43 +00:00
Martyn Inglis
cea48140f8 Added README to tell users how tp switch REDIS on 2016-12-02 10:11:31 +00:00
Martyn Inglis
687f217d7b Updated readme 2016-12-02 10:09:36 +00:00
Martyn Inglis
9fb378021c Bumped version to use latest utils
- note this is not yet published so will break.....
- sorts out an issue with service id logging
2016-12-01 17:24:23 +00:00
Martyn Inglis
7260561ad5 Makes the app use the redis and statsd clients from utils 2016-12-01 17:20:05 +00:00
Martyn Inglis
9667d42008 Minor bump of utils 2016-12-01 12:47:33 +00:00
minglis
d2ce6f2808 Merge pull request #755 from alphagov/request-id-logging
Request id logging
2016-12-01 11:53:32 +00:00
minglis
0ddd8d93ae Merge pull request #753 from alphagov/logging-service-ids
Records the service ID if present in the request
2016-12-01 11:38:49 +00:00
Martyn Inglis
5ae21bf1d5 Bumped requirements version 2016-12-01 11:34:38 +00:00
Jenny Duckett
84f89fb768 Update README
- the bootstrap script needs to be run before the app will run, so mention
  that first
- explain what run_celery_beat.sh is for and that it isn't required to run the
  app
2016-11-30 17:49:47 +00:00
Martyn Inglis
d8625f9da4 Fixed tests that failed due to changes in utils brought in by latest version.
- seems phonenumber/emailaddress from the CSV are now passed in as personalisation.
- assume the renderer does the correct thing here. Will need to check with @quis
2016-11-30 17:22:03 +00:00
Martyn Inglis
7fe56cf992 Add request ID logging to the API.
- adds a request ID if needed to use in all log events
- respects request-ids passed in from calling clients
2016-11-30 17:05:39 +00:00
Andras Ferencz-Szabo
2e0ab3eba1 Merge pull request #754 from alphagov/use_http_proxy_for_docker
Use http proxy for Docker
2016-11-30 16:55:13 +00:00
bandesz
07aa4cb0b0 Use http proxy for Docker 2016-11-30 15:57:08 +00:00
minglis
40cf6c049f Merge pull request #746 from alphagov/record-user-agent
Statsd counter for user agents.
2016-11-30 12:51:30 +00:00
Rebecca Law
6b079dc2bf Merge pull request #747 from alphagov/dont-send-message-twice
Protect send_sms send_email task from being executed twice
2016-11-30 11:12:41 +00:00
Martyn Inglis
23ca62a7ac Records the service ID if present, or if an ADMIN api call the string admin-api.
this is used later in logging of requests.
2016-11-30 10:59:55 +00:00
Rebecca Law
28fe1fe049 Merge branch 'master' into dont-send-message-twice
Conflicts:
	tests/app/notifications/test_validators.py
2016-11-29 16:13:43 +00:00
Rebecca Law
6ffd97648c Merge pull request #751 from alphagov/fix-mocks
Refactored a couple of tests that were slightly testing the wrong thing.
2016-11-29 16:04:55 +00:00
Rebecca Law
e38625632c Remove print 2016-11-29 16:03:58 +00:00
Martyn Inglis
c437615661 Merge branch 'master' into fix-mocks
Conflicts:
	tests/app/notifications/test_validators.py
2016-11-29 13:32:40 +00:00
Martyn Inglis
2182b5715e Refactored a couple of tests that were slightly testing the wrong thing.
- Ensure mock used correctly.
2016-11-29 13:27:57 +00:00
Rebecca Law
e8f045b352 Merge branch 'master' into dont-send-message-twice
Conflicts:
	tests/app/notifications/test_validators.py
2016-11-29 12:34:33 +00:00
Rebecca Law
94609cef79 Fix how the exception is logged. 2016-11-28 17:22:03 +00:00
Paul Craig
2ae8967a5f Merge pull request #749 from alphagov/pc-nab-numerous-notifications
Remove marshmallow schema validation + format error messages
2016-11-28 17:20:27 +00:00
Rebecca Law
241ea80a84 Merge pull request #750 from alphagov/fix-v2-post-response-body
Fix version 2 of post notification, return placeholders
2016-11-28 17:19:42 +00:00
Rebecca Law
3267a54ded Merge branch 'master' into dont-send-message-twice 2016-11-28 16:43:17 +00:00
Paul Craig
4082d38c0c Test invalid older_than, template_types, and bad ids
Come up with some simple tests in the routes, just to see we get
back what we expect as errors.
2016-11-28 15:56:59 +00:00
Rebecca Law
57a8f8d7fa The body of the content in the response to a POST v2/notifications was not replacing the placeholders.
This PR fixes that and adds a test for it.

I am confused as to why I had to change the test_validators test that is checking if the mock is called.
Why did this code pass on preview?
2016-11-28 15:49:29 +00:00
Paul Craig
2ca675eb73 Test invalid statuses and template_types
Run through a few scenarios in the `test_schema_notifications.py`
test file, calling `.validate()` on some data instead of actually
calling the route.
2016-11-28 15:10:13 +00:00
Paul Craig
ab990679b3 Replace marshmallow with jsonschema
Created a new schema that accepts request parameters for the
get_notifications v2 route.
Using that to validate now instead of the marshmallow validation.

Also changed the way formatted error messages are returned because
the previous way was cutting off our failing `enum` messages.
2016-11-28 14:22:51 +00:00
Paul Craig
1b846f3e74 Merge pull request #745 from alphagov/pc-nab-numerous-notifications
[#131977833] Create get all notifications endpoint for V2 API
2016-11-28 13:21:49 +00:00
Paul Craig
9b1375ba84 URLs in API responses have full URL 2016-11-28 11:13:11 +00:00
Martyn Inglis
7b859fcca1 Set up REDIS config.
Switched OFF on DEV and by DEFAULT
Switched ON for TESTS
Switched ON on PREVIEW - STAGING - PRODUCTION

URL set via ENV file - set in environment_test.sh. DEVS will need to update environment.sh. README updated.
2016-11-28 09:39:56 +00:00
Rebecca Law
be113e031f Sometimes a message is picked up twice of the SQS queue, we need to safe gaurd ourselves for that.
In this PR the id for the notification is passed in and used to created the notification, which causes a integrity error.
Normally when we get a SQLAlchemy error here we send the message to the retry queue, but if the notification already exists
we just ignore it.
2016-11-25 17:32:01 +00:00
Paul Craig
268e19bb76 Fix ServiceWhiteList __repr__
Apparently this code was unreachable.
2016-11-25 16:58:46 +00:00
Paul Craig
ffb813bb3f Add get_notifications json schema + tests
Add a schema that corresponds to our response for returning lists
of notifications, and test with a contract test.
2016-11-25 16:50:20 +00:00
Paul Craig
df7450698c Test returning notifications by "?status=failed"
Check that all failure states are returned by asking for
notifications of type "failure".
2016-11-25 15:26:40 +00:00
Paul Craig
57a0d7295d Rewrite failed statuses
There are no more notifications whose statuses are "failed", as
the "failed" status has now been replaced with statuses that are
more specific about the nature of the failure.

However, we still want to be able to filter by failing
notifications. (ie "/v2/notifications?status=failed").

Created a `.substitute_status()` method which takes a status
string or list of status strings and, if it finds 'failure',
replaces it with the other failing status types.

This way, calling for nottifications with "?status=failed" is
internally treated as
"status = ['technical-failure', 'temporary-failure', 'permanent-failure']"
2016-11-25 15:23:46 +00:00
Paul Craig
7d009915a4 Group 'completed' notification statuses
Some notification statuses assume that a notification has been
updated (ie, it cannot have been created in that state).

This caused a bug in our sample notification fixture when trying
to create a notificaiton in a 'complete' status.

This commit groups the completed statuses in a list, the way other
statuses have been grouped together so that they're more portable.
Also fixed the sample_notification fixture.
2016-11-25 15:23:04 +00:00
Paul Craig
640f51fc0a Add tests for getting back lists of notifications 2016-11-25 15:23:04 +00:00
Paul Craig
effbd315e0 Create 'v2' get notifications route
Return multiple notifications for a service.
Choosing a page_size or a page_number is no longer allowed.
Instead, there is a `next` link included with will return the
next {default_page_size} notifications in the sequence.

Query parameters accepted are:

 - template_type: filter by specific template types
 - status: filter by specific statuses
 - older_than: return a chronological list of notifications older
   than this one. The notification with the id that is passed in
   is _not_ returned.

Note that both `template_type` and `status` can accept multiple
parameters.  Thus it is possible to call
`/v2/notifications?status=created&status=sending&status=delivered`
2016-11-25 15:23:03 +00:00
Paul Craig
1fce30aaa7 Cost savings
The "cost" value was flawed for a couple of reasons.

1. Lots of messages are free, so in those instances the "cost"
   doesn't tell you anything
2. The query to get the rate was expensive and we don't have
   an obvious way to get it back very efficiently for large numbers
   of notifications.

So we scrapped it.
2016-11-25 15:23:03 +00:00
Martyn Inglis
e698d124ba Statsd counter for user agents.
We want to log the usage of the various API clients we have so that we understand when they can be cycled.

To this end we are going to count usage in statsd.

All notify clients have a suer agent, of the format: NOTIFY-API-{LANGUAGE}-CLIENT/version.number

For example, NOTIFY-API-PYTHON-CLIENT/3.0.0

We convert that into a statsd/graphite friendly string of the format: notify-api-python-client.3-0-0

So we can subdivide on client and client version on our dashboards.

Present but unknown User agents are records as "non-notify-user-agent"

Missing are presented as "unknown"
2016-11-25 13:27:36 +00:00
Paul Craig
f1ea39d4c0 Simplify _filter_query() function 2016-11-25 11:23:20 +00:00
Rebecca Law
669126d41b Merge pull request #744 from alphagov/add-default-exception-handler
Add application level error handler.
2016-11-25 09:34:23 +00:00
minglis
2e1578656b Merge pull request #735 from alphagov/caching-with-redis
Caching with redis
2016-11-25 09:06:12 +00:00