Commit Graph

1566 Commits

Author SHA1 Message Date
Martyn Inglis
c13ead77e4 Ensures that both the CSV processing and the API both use the new deliver_email and deliver_sms tasks not the old ones with the redundant parameter. 2016-09-28 15:05:50 +01:00
Martyn Inglis
9233ffa3d4 When processing a CSV file don't put the DB tasks into the live queue when service is in research mode. 2016-09-28 13:53:05 +01:00
minglis
9df65dfe25 Merge pull request #695 from alphagov/rationalise-queues
Removed the unused queues from the config
2016-09-28 09:46:42 +01:00
Martyn Inglis
f4f0c73fec Removed the unused queues from the config 2016-09-28 09:45:19 +01:00
minglis
9f932597a4 Merge pull request #693 from alphagov/rationalise-queues
Rationalised the queues.
2016-09-28 09:07:37 +01:00
Martyn Inglis
4f3017c88b Old config reinstated for now.
Pull requests as was would mean that we didn't read from the old queues, so whilst the new code writes to the new "notify" queue, the old code would write to the old plethera of queues. And the readers may not pick it up depending on the order of the deploy.

Safer to leave readers reading from all queues until after the deploy then doing a second deploy to tidy up the config.
2016-09-28 09:03:45 +01:00
Martyn Inglis
a4b5b486c4 Merge branch 'master' into rationalise-queues 2016-09-28 08:57:59 +01:00
minglis
b1654c8808 Merge pull request #686 from alphagov/refactor-send-tasks-into-shared-code
Refactor send tasks into shared code
2016-09-27 16:46:34 +01:00
minglis
7b69a4e53e Merge pull request #692 from alphagov/job-status-filter
add statuses filter to GET /service/{}/job
2016-09-27 15:31:50 +01:00
minglis
b2cb6891fc Merge pull request #694 from alphagov/remove-travis-image
Removed travis image
2016-09-27 14:58:37 +01:00
minglis
8dd93b350f Merge pull request #691 from alphagov/fix-template-type-bug
Fix template type bug
2016-09-27 14:51:43 +01:00
Martyn Inglis
36425cd035 Removed travis image 2016-09-27 12:39:30 +01:00
Martyn Inglis
ce5b5fbf4c Rationalised the queues.
- the internal notify activites (send sms code, forgotten passwords etc) all not in a "notify" queue.
- deleted all unused queues
2016-09-26 12:35:59 +01:00
Chris Hill-Scott
76d3f08b46 Merge pull request #690 from alphagov/add-key-name-notifications
Add details of API key to notifications
2016-09-26 08:41:02 +01:00
Chris Hill-Scott
592b0d15da Merge pull request #687 from alphagov/optionally-get-test-notifications
Optionally get notifications created with a test key
2016-09-26 08:40:54 +01:00
Leo Hemsted
d3c0c48401 fix dao_get_jobs_by_service_id
statuses now defaults to correct val
2016-09-23 17:05:42 +01:00
Leo Hemsted
fb6cb5f236 add statuses filter to GET /service/{}/job
can now pass in a query string `?statuses=x,y,z` to filter jobs based on
`Job.job_status`. Not passing in a status or passing in an empty string is
equivalent to selecting every filter type at once.
2016-09-23 16:34:13 +01:00
Rebecca Law
0ef43ab1dc Fix the wording on the message a little bit. 2016-09-23 16:03:11 +01:00
Martyn Inglis
6f7dd149e2 Tidied up pull request comments 2016-09-23 15:59:23 +01:00
Rebecca Law
82d29d470e Merge pull request #688 from alphagov/add-specific-error-for-auth
Improve authentication error message
2016-09-23 15:54:48 +01:00
Rebecca Law
09e6e6198b Fixing a bug that allows a sms notification to be sent with an email template and vice versa.
This has been resolved for the post notifications endpoint
2016-09-23 15:46:48 +01:00
Chris Hill-Scott
130f42fed2 Add details of API key to notifications
We want to show developers a log of notifications they’ve sent using the
API in the admin app. In order to indentify a notification it’s probably
helpful to know:

- who the notification was sent to (which we expose)
- when the notification was created (which we expose)
- which key was used to create the notification (which we expose, but
  only as the `id` of the key)

Developers don’t see the `id` of the API key in the app anywhere, so
this isn’t useful information. Much more useful is the `type` and `name`
of the key. So this commit changes the schema to also return these.

This commit does some slightly hacky stuff with conftest because it
breaks a lot of other tests if the sample notification has a real API
key or an API key with a non-unique name.
2016-09-23 14:44:15 +01:00
Rebecca Law
f5aac5796c Improve the error message when the service id is not the right data type.
Improve the error message is the api key is not valid.
2016-09-23 11:07:49 +01:00
Chris Hill-Scott
21f3448fbc Use client fixture 2016-09-23 10:41:37 +01:00
Chris Hill-Scott
ba71079f22 Reduce indentation 2016-09-23 10:41:37 +01:00
Chris Hill-Scott
d7bb83fadf Optionally get notifications created w/ test key
This is only for the method that the admin app uses; it doesn’t affect
the public get notifications endpoint.
2016-09-23 10:40:33 +01:00
Chris Hill-Scott
f9f3bb8370 Make DAO optionally return test key notifications
Developers need visibility of what their integration is doing within
the app. This includes notifications sent with a test key.

This commit adds an optional, defaults-to-false parameter to include
notifications sent from a test API key when getting notifications.
2016-09-23 10:37:26 +01:00
Martyn Inglis
124487f204 Fix from number on Load testing client 2016-09-23 10:31:18 +01:00
Martyn Inglis
54c28a4bea Updated logging as logger was parsing some JSON and erroring.
- Don't write whole JSON response. Safer not to anyway. No need.
2016-09-23 10:24:12 +01:00
Martyn Inglis
137aa1e2d2 Merge branch 'master' into refactor-send-tasks-into-shared-code
Conflicts:
	app/celery/provider_tasks.py
	tests/app/celery/test_provider_tasks.py
2016-09-23 09:54:53 +01:00
Leo Hemsted
281323b435 Merge pull request #683 from alphagov/paginate-dashboard
Paginate `GET /service/{}/job` query
2016-09-23 09:54:47 +01:00
Chris Hill-Scott
b36d0ff552 Make indentation more sensible
Starting arguments on their own line and putting the closing parenthesis
on it’s own line because any subsequent changes to the arguments diff
cleanly (ie without touching any other lines).
2016-09-23 09:43:25 +01:00
Martyn Inglis
376f8355cb Updated clients to have a more robust error handling
- fire text and omg much more similar. Ready to be combined.
- Error handling now for JSON valid responses
2016-09-22 17:18:05 +01:00
Rebecca Law
e757c6463c Merge pull request #682 from alphagov/remove-provider_statistics
Remove provider_statistics_dao.get_provider_statistics
2016-09-22 15:23:14 +01:00
Martyn Inglis
37a2dc7925 Implemented the REST endpoint to communicate with the new send-to-provider code
- this allows us to send a notification to a provider by means of an API call
- This is in addition to the celery code.
- idea is that we can use this method to help speed up throughput by generating API traffic by node/lambda etc to supplement the celery code in times of high load.
2016-09-22 14:01:25 +01:00
Leo Hemsted
14f889f7e1 Merge pull request #685 from alphagov/fix-build
add build dir to excldued folders for pep8
2016-09-22 13:20:24 +01:00
Leo Hemsted
faa0dd509f add build dir to excldued folders for pep8 2016-09-22 13:16:02 +01:00
Chris Hill-Scott
a34f71b5aa Merge pull request #684 from alphagov/get-service-notifications-not-include-jobs
Optionally get only notifications created by API
2016-09-22 13:08:51 +01:00
Rebecca Law
30dd602dfb Merge branch 'master' into remove-provider_statistics
Conflicts:
	tests/app/notifications/rest/test_callbacks.py
2016-09-22 11:20:19 +01:00
Chris Hill-Scott
4217bd2108 Optionally get only notifications created by API
This commit adds the `include_jobs` filter to the
`GET /services/…/notifications` endpoint. It defaults to `True` (ie show
all notifications) but makes it possible to only return notifications
created by _any_ API key.

This is so that we can show a log of all notifications sent through the
API in the admin app.

It does not expose this list to the public `GET /notifications` endpoint
because this would violate our rules about keys only being able to get
notifications created with keys of the same type.
2016-09-22 11:17:46 +01:00
Martyn Inglis
59ab5da5d3 Handle errors where the notification isn't found when executing the tasks
- Thows a NoResultFound sqlalchemy exception
- Which causes a retry. This means we give it a few goes (5, max 5 hours)  for the notification to appear.
- Should never happen, only if we get some task overlaps that are unusual that leads to tasks executed in an overlapping nature.
2016-09-22 09:52:23 +01:00
Martyn Inglis
2f36e0dbcf Refactor to make the new send_to_provider methods take a notification not a notification ID.
- Driven by the fact we won't know the type in the API call
- hence we need to load notification earlier , so pass it not the id through to the send task to avoid loading it twice.
2016-09-22 09:16:58 +01:00
Leo Hemsted
00b905bef0 use a contextmanager to set config values
so that when later tests run, the config has been restored to its
defaults and we don't end up failing later tests in confusing ways
2016-09-21 17:13:26 +01:00
Leo Hemsted
24903a19ef add pagination to GET /service/{}/job
accepts a page parameter to control what page of data

returns additional pagination fields in the response dict
* page_size: will always be 50. defined by Config.PAGE_SIZE
* total: the total amount of unpaginated records
* links: dict containing optionally prev, next, and last, links to
  other relevant pagination pages

also cleaned up some test imports
2016-09-21 16:54:02 +01:00
Leo Hemsted
4bfcfcd2fb clean up test imports 2016-09-21 15:45:26 +01:00
Leo Hemsted
29abd4c382 make dao_get_jobs_by_service_id paginated
it can return a pretty long list, especially when we run lots of
smoke tests, so make it accept pagination parameters, and return
a pagination object
2016-09-21 14:35:23 +01:00
Leo Hemsted
94b280af63 ensure conftest sets up datetimes within function bodies
this makes sure it respects any freezegun.freeze_times that might
be in place
2016-09-21 14:34:38 +01:00
Andras Ferencz-Szabo
0c5720108d Merge pull request #681 from alphagov/deploy_db_migrations
Deploy db migrations
2016-09-21 14:27:07 +01:00
Martyn Inglis
991cc884b4 Refactoring the send to provider code out of the tasks folder
- building a rest endpoint to call that code to compliment the existing task based approach.
2016-09-21 13:27:32 +01:00
bandesz
8ab7fa3395 Build db migration codedeploy artifact 2016-09-21 11:21:12 +01:00