Commit Graph

1833 Commits

Author SHA1 Message Date
Jenny Duckett
84f89fb768 Update README
- the bootstrap script needs to be run before the app will run, so mention
  that first
- explain what run_celery_beat.sh is for and that it isn't required to run the
  app
2016-11-30 17:49:47 +00:00
Paul Craig
2ae8967a5f Merge pull request #749 from alphagov/pc-nab-numerous-notifications
Remove marshmallow schema validation + format error messages
2016-11-28 17:20:27 +00:00
Rebecca Law
241ea80a84 Merge pull request #750 from alphagov/fix-v2-post-response-body
Fix version 2 of post notification, return placeholders
2016-11-28 17:19:42 +00:00
Paul Craig
4082d38c0c Test invalid older_than, template_types, and bad ids
Come up with some simple tests in the routes, just to see we get
back what we expect as errors.
2016-11-28 15:56:59 +00:00
Rebecca Law
57a8f8d7fa The body of the content in the response to a POST v2/notifications was not replacing the placeholders.
This PR fixes that and adds a test for it.

I am confused as to why I had to change the test_validators test that is checking if the mock is called.
Why did this code pass on preview?
2016-11-28 15:49:29 +00:00
Paul Craig
2ca675eb73 Test invalid statuses and template_types
Run through a few scenarios in the `test_schema_notifications.py`
test file, calling `.validate()` on some data instead of actually
calling the route.
2016-11-28 15:10:13 +00:00
Paul Craig
ab990679b3 Replace marshmallow with jsonschema
Created a new schema that accepts request parameters for the
get_notifications v2 route.
Using that to validate now instead of the marshmallow validation.

Also changed the way formatted error messages are returned because
the previous way was cutting off our failing `enum` messages.
2016-11-28 14:22:51 +00:00
Paul Craig
1b846f3e74 Merge pull request #745 from alphagov/pc-nab-numerous-notifications
[#131977833] Create get all notifications endpoint for V2 API
2016-11-28 13:21:49 +00:00
Paul Craig
9b1375ba84 URLs in API responses have full URL 2016-11-28 11:13:11 +00:00
Paul Craig
268e19bb76 Fix ServiceWhiteList __repr__
Apparently this code was unreachable.
2016-11-25 16:58:46 +00:00
Paul Craig
ffb813bb3f Add get_notifications json schema + tests
Add a schema that corresponds to our response for returning lists
of notifications, and test with a contract test.
2016-11-25 16:50:20 +00:00
Paul Craig
df7450698c Test returning notifications by "?status=failed"
Check that all failure states are returned by asking for
notifications of type "failure".
2016-11-25 15:26:40 +00:00
Paul Craig
57a0d7295d Rewrite failed statuses
There are no more notifications whose statuses are "failed", as
the "failed" status has now been replaced with statuses that are
more specific about the nature of the failure.

However, we still want to be able to filter by failing
notifications. (ie "/v2/notifications?status=failed").

Created a `.substitute_status()` method which takes a status
string or list of status strings and, if it finds 'failure',
replaces it with the other failing status types.

This way, calling for nottifications with "?status=failed" is
internally treated as
"status = ['technical-failure', 'temporary-failure', 'permanent-failure']"
2016-11-25 15:23:46 +00:00
Paul Craig
7d009915a4 Group 'completed' notification statuses
Some notification statuses assume that a notification has been
updated (ie, it cannot have been created in that state).

This caused a bug in our sample notification fixture when trying
to create a notificaiton in a 'complete' status.

This commit groups the completed statuses in a list, the way other
statuses have been grouped together so that they're more portable.
Also fixed the sample_notification fixture.
2016-11-25 15:23:04 +00:00
Paul Craig
640f51fc0a Add tests for getting back lists of notifications 2016-11-25 15:23:04 +00:00
Paul Craig
effbd315e0 Create 'v2' get notifications route
Return multiple notifications for a service.
Choosing a page_size or a page_number is no longer allowed.
Instead, there is a `next` link included with will return the
next {default_page_size} notifications in the sequence.

Query parameters accepted are:

 - template_type: filter by specific template types
 - status: filter by specific statuses
 - older_than: return a chronological list of notifications older
   than this one. The notification with the id that is passed in
   is _not_ returned.

Note that both `template_type` and `status` can accept multiple
parameters.  Thus it is possible to call
`/v2/notifications?status=created&status=sending&status=delivered`
2016-11-25 15:23:03 +00:00
Paul Craig
1fce30aaa7 Cost savings
The "cost" value was flawed for a couple of reasons.

1. Lots of messages are free, so in those instances the "cost"
   doesn't tell you anything
2. The query to get the rate was expensive and we don't have
   an obvious way to get it back very efficiently for large numbers
   of notifications.

So we scrapped it.
2016-11-25 15:23:03 +00:00
Paul Craig
f1ea39d4c0 Simplify _filter_query() function 2016-11-25 11:23:20 +00:00
Rebecca Law
669126d41b Merge pull request #744 from alphagov/add-default-exception-handler
Add application level error handler.
2016-11-25 09:34:23 +00:00
minglis
2e1578656b Merge pull request #735 from alphagov/caching-with-redis
Caching with redis
2016-11-25 09:06:12 +00:00
Rebecca Law
52c8596c92 Add application level error handler.
This should catch any unexcpeted exceptions.
2016-11-24 17:09:56 +00:00
bandesz
8fa337927c Exclude cache directory from pep8 2016-11-23 12:39:48 +00:00
Paul Craig
be5d503133 Merge pull request #743 from alphagov/stringify-cost-attr-on-v2-get-notification
Stringify the cost before passing to jsonify as it complains otherwise
2016-11-23 10:50:59 +00:00
Rebecca Law
97379a2904 Merge pull request #742 from alphagov/upgrade-python-client
Update python client from version 2.0.0 to 3.0.0
2016-11-23 10:48:21 +00:00
Martyn Inglis
2a2ef34339 Merge branch 'master' into caching-with-redis
Conflicts:
	app/__init__.py
2016-11-23 09:12:11 +00:00
Paul Craig
92897e20a8 Test returning a notification with a non-zero cost
Our previous test ws returning a notification without a `sent_by`
attribute, which meant that cost was always 0.
Unfortunately, this meant that returning a real value for cost was
untested and (whaddya know) it broke immediately.

Old test scenario:
- billable_units=1, sent_by=None, cost=0

New scenarios
- billable_units=0, sent_by='mmg', cost=0
- billable_units=1, sent_by='mmg', cost=1
2016-11-22 18:01:16 +00:00
Paul Craig
a17adb4707 New test for notification.cost() method
Added a test that asking for a nonexistent provider_rate blows
everything up.

Also updated existing test to use a weirder number.
2016-11-22 18:01:16 +00:00
Imdad Ahad
9b6584c8a1 Make result of notification.cost() into a float
Cost was returning a `Decimal`, which jsonify doesn't like.
Making it a float fixes the problem.

[Relevant issue on github](https://github.com/pallets/flask/issues/835).
2016-11-22 17:58:16 +00:00
Rebecca Law
3cddc64e36 Merge pull request #741 from alphagov/fix-v2-post-notification
Small fixes for the V2 post notification endpoints
2016-11-22 16:28:13 +00:00
Martyn Inglis
689421aa29 Removed freeze time from test 2016-11-22 15:05:11 +00:00
Rebecca Law
8cf324de85 Update python client from version 2.0.0 to 3.0.0 2016-11-22 14:59:14 +00:00
Paul Craig
0e0dbe6e77 Merge pull request #740 from alphagov/pc-newly-nab-notifications
[#131977833] Create GET notification by ID endpoint for V2 API
2016-11-22 13:54:05 +00:00
Paul Craig
ebfac180c0 Add contract test for 'v2' email notification
We don't have a way of creating letter notifications just yet, so
this is all we can test.
2016-11-22 13:40:55 +00:00
Paul Craig
7ae427c4ef Use jsonschema validation + remove 'v2' schema
Emualated the validation methods that exist [in the python-client](620e5f7014/integration_test/__init__.py).

The `validate_v0` function loads json schemas from a local
`/schemas` directory, whereas the new `validate` function (which
we're going to use for our v2 API calls) uses the common
`get_notification_response` python schema defined in
"app/v2/notifications/notification_schemas.py".

Removed the new `v2` schema from the last commit as it's no longer
being used.

Also, refactored common code in the GET and POST contract files
so that making requests and converting responses to JSON are
pulled out into common functions.
2016-11-22 13:40:22 +00:00
Rebecca Law
fa422a85ef Fix unit test failure 2016-11-22 13:25:10 +00:00
Martyn Inglis
e7a01e979b Fixed merge conflicts 2016-11-22 13:10:12 +00:00
Martyn Inglis
0c7d9e33aa Merge branch 'master' into caching-with-redis
Conflicts:
	tests/app/notifications/test_process_notification.py
2016-11-22 13:00:37 +00:00
Martyn Inglis
9e2ba9ee81 Pushed the cache increment into the shared code that persists notifications.
Much simpler implementation, inc code removed from tasks and V1/V2 rest clients.
2016-11-22 12:53:20 +00:00
Paul Craig
fb50bb6325 Add 'v2' notification schema to contract tests
Converted python-based schema in
"app/v2/notifications/notification_schema.py" into a pure json
schema and tested it with the new "v2/" API route to confirm that
it validates.

Also refactored some common code in the public contract GET tests
that returns notifications.
2016-11-22 12:07:05 +00:00
Rebecca Law
46beece158 For the post_sms_response and post_email_response the reference property is always present but the value can be null.
Added a test for an empty reference.
Remove datetime format on the created_at attribute of a notification, it is not needed.
2016-11-21 17:32:36 +00:00
Paul Craig
c1fa5e156a Append "Z" to DATETIME_FORMAT
We're formally using the ISO 8601 UTC datetime format, and so the
correct way to output the data is by appending the timezone.
("Z" in the case of UTC*).

Unfortunately, Python's `datetime` formatting will just ignore the
timezone part of the string on output, which means we just have to
append the string "Z" to the end of all datetime strings we output.

Should be fine, as we will only ever output UTC timestamps anyway.

* https://en.wikipedia.org/wiki/ISO_8601#UTC
2016-11-21 15:59:10 +00:00
Paul Craig
82ba2cd226 Create new notification schema
This is the schema that individual notifications will conform to
when they are returned from this API.
JSON logic enforces that the right keys are set depending on the
`"type"`. (eg a schema with `"type": "sms"` must have a
`"phone_number"` value and it cannot have an `"email_address"`)
2016-11-21 15:59:10 +00:00
Paul Craig
9758b96a2b Create 'v2' get notification route
The new 'v2' API wants to return less data than the previous one,
which was sending back tons of fields the clients never used.

This new route returns only useful information, with the JSON
response dict being built up in the model's `.serialize()` method.

Note that writing the test for this was a bit painful because of
having to treat loads of keys differently. Hopefully we think this
is a good way to write this test, because if we don't, we should
start thinking of a better way to check the values are what we
expect.
2016-11-21 15:41:49 +00:00
Martyn Inglis
58bbc5a5aa Now that we have discovered that the catch all Exception handler doesn't work, I've created a custom exception (SendNotificationQueueError) that handles this case and an error handler for it.
Talked these through with @becca as an approach.
2016-11-21 15:11:19 +00:00
Martyn Inglis
7cfc58c994 Merge branch 'master' into caching-with-redis
Conflicts:
	app/celery/tasks.py
	tests/app/celery/test_tasks.py
2016-11-21 13:10:22 +00:00
Paul Craig
fddb1653ac Add .cost() to notification model
In the V2 API, the GET response for an individual notification
returns a 'cost' value, which we can get by multiplying the
billable units by the per-message rate of the supplier who
sent the message.
Any notifications with billable units > 0 but without a
corresponding `ProviderRates` entry will blow up the application,
so make sure you've got one.
2016-11-18 17:50:39 +00:00
Rebecca Law
1345c94f4b Merge pull request #736 from alphagov/implement-v2-send-email
Implemented the post email notifications endpoint for v2
2016-11-17 15:47:22 +00:00
Rebecca Law
f5e3c6f63b Remove print stmt and added assert error content 2016-11-17 14:02:44 +00:00
Rebecca Law
bc434f1736 Create new column in notifications and notification_history to store the client_reference because I remembered that reference is used for email providers. 2016-11-17 13:42:34 +00:00
Rebecca Law
dc5e21a78e Remove v2 error handler for InvalidPhoneError, no longer expect to throw this exception. 2016-11-17 09:09:24 +00:00