Commit Graph

8700 Commits

Author SHA1 Message Date
Rebecca Law
f5aac5796c Improve the error message when the service id is not the right data type.
Improve the error message is the api key is not valid.
2016-09-23 11:07:49 +01:00
Leo Hemsted
0b8c385de1 add remove_whitelist tests 2016-09-23 10:45:12 +01:00
Chris Hill-Scott
21f3448fbc Use client fixture 2016-09-23 10:41:37 +01:00
Chris Hill-Scott
ba71079f22 Reduce indentation 2016-09-23 10:41:37 +01:00
Chris Hill-Scott
d7bb83fadf Optionally get notifications created w/ test key
This is only for the method that the admin app uses; it doesn’t affect
the public get notifications endpoint.
2016-09-23 10:40:33 +01:00
Chris Hill-Scott
f9f3bb8370 Make DAO optionally return test key notifications
Developers need visibility of what their integration is doing within
the app. This includes notifications sent with a test key.

This commit adds an optional, defaults-to-false parameter to include
notifications sent from a test API key when getting notifications.
2016-09-23 10:37:26 +01:00
Martyn Inglis
124487f204 Fix from number on Load testing client 2016-09-23 10:31:18 +01:00
Martyn Inglis
54c28a4bea Updated logging as logger was parsing some JSON and erroring.
- Don't write whole JSON response. Safer not to anyway. No need.
2016-09-23 10:24:12 +01:00
Leo Hemsted
2e16be18bc Merge branch 'master' of github.com:alphagov/notifications-api into service-whitelist 2016-09-23 10:14:42 +01:00
Martyn Inglis
137aa1e2d2 Merge branch 'master' into refactor-send-tasks-into-shared-code
Conflicts:
	app/celery/provider_tasks.py
	tests/app/celery/test_provider_tasks.py
2016-09-23 09:54:53 +01:00
Leo Hemsted
281323b435 Merge pull request #683 from alphagov/paginate-dashboard
Paginate `GET /service/{}/job` query
2016-09-23 09:54:47 +01:00
Chris Hill-Scott
b36d0ff552 Make indentation more sensible
Starting arguments on their own line and putting the closing parenthesis
on it’s own line because any subsequent changes to the arguments diff
cleanly (ie without touching any other lines).
2016-09-23 09:43:25 +01:00
Martyn Inglis
376f8355cb Updated clients to have a more robust error handling
- fire text and omg much more similar. Ready to be combined.
- Error handling now for JSON valid responses
2016-09-22 17:18:05 +01:00
Leo Hemsted
af0dbd14be correct service_whitelist dao/rest functionality
additionally added dao tests
2016-09-22 17:17:34 +01:00
Rebecca Law
e757c6463c Merge pull request #682 from alphagov/remove-provider_statistics
Remove provider_statistics_dao.get_provider_statistics
2016-09-22 15:23:14 +01:00
Martyn Inglis
37a2dc7925 Implemented the REST endpoint to communicate with the new send-to-provider code
- this allows us to send a notification to a provider by means of an API call
- This is in addition to the celery code.
- idea is that we can use this method to help speed up throughput by generating API traffic by node/lambda etc to supplement the celery code in times of high load.
2016-09-22 14:01:25 +01:00
Leo Hemsted
14f889f7e1 Merge pull request #685 from alphagov/fix-build
add build dir to excldued folders for pep8
2016-09-22 13:20:24 +01:00
Leo Hemsted
faa0dd509f add build dir to excldued folders for pep8 2016-09-22 13:16:02 +01:00
Chris Hill-Scott
a34f71b5aa Merge pull request #684 from alphagov/get-service-notifications-not-include-jobs
Optionally get only notifications created by API
2016-09-22 13:08:51 +01:00
Leo Hemsted
5c5ca26630 Add ServiceWhiteList model with corresponding tests 2016-09-22 11:58:51 +01:00
Rebecca Law
30dd602dfb Merge branch 'master' into remove-provider_statistics
Conflicts:
	tests/app/notifications/rest/test_callbacks.py
2016-09-22 11:20:19 +01:00
Chris Hill-Scott
4217bd2108 Optionally get only notifications created by API
This commit adds the `include_jobs` filter to the
`GET /services/…/notifications` endpoint. It defaults to `True` (ie show
all notifications) but makes it possible to only return notifications
created by _any_ API key.

This is so that we can show a log of all notifications sent through the
API in the admin app.

It does not expose this list to the public `GET /notifications` endpoint
because this would violate our rules about keys only being able to get
notifications created with keys of the same type.
2016-09-22 11:17:46 +01:00
Leo Hemsted
203936fa84 add GET/POST rest endpoints for whitelist
GET /<service_id>/whitelist
    returns all whitelisted contacts for a service, separated into two lists

POST /<service_id>/whitelist
    removes all existing whitelisted contacts, and replaces them with the
    provided new entries

(todo: dao work + tests)
2016-09-22 10:32:32 +01:00
Leo Hemsted
3d0df9b5a7 add new service whitelist table
services can have a whitelist of phone numbers and email addresses that they
can send to in addition to team members when in trial mode. email_address
and mobile_number are nullable and app level checks will be in place to
prevent inserting blank rows. they have a created_at date so that we can
[potentially] delete them a week later to avoid keeping personally identifying
data any longer than necessary
2016-09-22 10:32:32 +01:00
Martyn Inglis
59ab5da5d3 Handle errors where the notification isn't found when executing the tasks
- Thows a NoResultFound sqlalchemy exception
- Which causes a retry. This means we give it a few goes (5, max 5 hours)  for the notification to appear.
- Should never happen, only if we get some task overlaps that are unusual that leads to tasks executed in an overlapping nature.
2016-09-22 09:52:23 +01:00
Martyn Inglis
2f36e0dbcf Refactor to make the new send_to_provider methods take a notification not a notification ID.
- Driven by the fact we won't know the type in the API call
- hence we need to load notification earlier , so pass it not the id through to the send task to avoid loading it twice.
2016-09-22 09:16:58 +01:00
Leo Hemsted
00b905bef0 use a contextmanager to set config values
so that when later tests run, the config has been restored to its
defaults and we don't end up failing later tests in confusing ways
2016-09-21 17:13:26 +01:00
Leo Hemsted
24903a19ef add pagination to GET /service/{}/job
accepts a page parameter to control what page of data

returns additional pagination fields in the response dict
* page_size: will always be 50. defined by Config.PAGE_SIZE
* total: the total amount of unpaginated records
* links: dict containing optionally prev, next, and last, links to
  other relevant pagination pages

also cleaned up some test imports
2016-09-21 16:54:02 +01:00
Leo Hemsted
4bfcfcd2fb clean up test imports 2016-09-21 15:45:26 +01:00
Leo Hemsted
29abd4c382 make dao_get_jobs_by_service_id paginated
it can return a pretty long list, especially when we run lots of
smoke tests, so make it accept pagination parameters, and return
a pagination object
2016-09-21 14:35:23 +01:00
Leo Hemsted
94b280af63 ensure conftest sets up datetimes within function bodies
this makes sure it respects any freezegun.freeze_times that might
be in place
2016-09-21 14:34:38 +01:00
Andras Ferencz-Szabo
0c5720108d Merge pull request #681 from alphagov/deploy_db_migrations
Deploy db migrations
2016-09-21 14:27:07 +01:00
Martyn Inglis
991cc884b4 Refactoring the send to provider code out of the tasks folder
- building a rest endpoint to call that code to compliment the existing task based approach.
2016-09-21 13:27:32 +01:00
bandesz
8ab7fa3395 Build db migration codedeploy artifact 2016-09-21 11:21:12 +01:00
bandesz
d69a07fed9 Delete accidentally committed notifications-api.zip 2016-09-21 11:21:12 +01:00
imdadahad
c94784b68c Merge pull request #680 from alphagov/update-return-all-notifications-schema
Update schema to validate contents of GET notifications
2016-09-21 09:52:51 +01:00
Imdad Ahad
23998a4d55 Add additionalProperties to links object 2016-09-21 09:47:50 +01:00
Martyn Inglis
bd06b18684 Refactor of celery tasks.
- Aim to move the code that contacts providers into it's own module.
- Celery tasks now call this module to send to provider
- No exceptions caught in the new module. Celery tasks now use any exception to trigger a retry.
- tests moved about - new test directory for the new class, all tests from celery test module moved, excepting the retry logic.
2016-09-20 17:24:28 +01:00
Imdad Ahad
afd381badf Update links schema to be more strict and fix import 2016-09-20 10:39:15 +01:00
Rebecca Law
51b3119a21 Remove provider_statistics_dao.get_provider_statistics
The provider_statistics table is no longer being populated, get rid of any reads from this table.
2016-09-19 17:24:26 +01:00
minglis
ffdf3272fb Merge pull request #679 from alphagov/remove-test-keys-from-admin-app
Remove test keys from admin app
2016-09-19 12:31:12 +01:00
minglis
1d28f09c22 Merge pull request #677 from alphagov/no-jobs-on-api-get
No jobs on api get
2016-09-19 12:31:04 +01:00
Chris Hill-Scott
aeba205b31 Merge pull request #678 from alphagov/better-auth-error-message
Improve error messages if you make an API call with a bad token
2016-09-19 10:15:20 +01:00
Imdad Ahad
5b0162cc60 Update schema to validate {notifications} contains a {links} object with expected properties 2016-09-16 14:56:14 +01:00
Chris Hill-Scott
cebbb2077c Merge pull request #676 from alphagov/bump-utils-fix-email-bullets
Fix HTML lists in Microsoft Outlook
2016-09-16 14:04:50 +01:00
Martyn Inglis
7c16294b75 Exclude test key notifications from:
- get all notifications by service
- template usage
- most recently used templates

Ensures that the dashboard shows no test key data. Supplements: https://github.com/alphagov/notifications-api/pull/677 which excludes CSV data. This branches from that so is dependant.
2016-09-16 13:47:09 +01:00
Chris Hill-Scott
d44a0b72bb Rewrite authentication error messages more English 2016-09-16 08:57:55 +01:00
Chris Hill-Scott
1ce91997e8 Give specifc error when service doesn’t exist
If you sign a token with a service ID that doesn’t exist (say, for
example, that you get service ID and API key mixed up) then you get
an error saying that “no API keys exist for the service”. This is wrong
because the service doesn’t even exist.

This commit adds:
- code to check if the service does exist
- a specific error message for this case

The check does mean an extra database call to look up the service.
However this only happens _after_ looping through all the API keys. So
it shouldn’t have a performance implication for anyone using a valid API
key.
2016-09-16 08:49:49 +01:00
Chris Hill-Scott
6b3c899127 Add a test for auth-ing with non-existant service
If you create a token signed with a service ID that doesn’t exist, you
will get an error (as you should).

However we didn’t have a test that explicitly checks for this. This
commit adds one.
2016-09-16 08:46:32 +01:00
Martyn Inglis
d4971cc45f Ensure GET /notifications returns JOB or API data correctly, using new boolean query param "include_jobs" 2016-09-15 16:01:26 +01:00