Commit Graph

1779 Commits

Author SHA1 Message Date
Chris Hill-Scott
4217bd2108 Optionally get only notifications created by API
This commit adds the `include_jobs` filter to the
`GET /services/…/notifications` endpoint. It defaults to `True` (ie show
all notifications) but makes it possible to only return notifications
created by _any_ API key.

This is so that we can show a log of all notifications sent through the
API in the admin app.

It does not expose this list to the public `GET /notifications` endpoint
because this would violate our rules about keys only being able to get
notifications created with keys of the same type.
2016-09-22 11:17:46 +01:00
Leo Hemsted
203936fa84 add GET/POST rest endpoints for whitelist
GET /<service_id>/whitelist
    returns all whitelisted contacts for a service, separated into two lists

POST /<service_id>/whitelist
    removes all existing whitelisted contacts, and replaces them with the
    provided new entries

(todo: dao work + tests)
2016-09-22 10:32:32 +01:00
Leo Hemsted
3d0df9b5a7 add new service whitelist table
services can have a whitelist of phone numbers and email addresses that they
can send to in addition to team members when in trial mode. email_address
and mobile_number are nullable and app level checks will be in place to
prevent inserting blank rows. they have a created_at date so that we can
[potentially] delete them a week later to avoid keeping personally identifying
data any longer than necessary
2016-09-22 10:32:32 +01:00
Martyn Inglis
59ab5da5d3 Handle errors where the notification isn't found when executing the tasks
- Thows a NoResultFound sqlalchemy exception
- Which causes a retry. This means we give it a few goes (5, max 5 hours)  for the notification to appear.
- Should never happen, only if we get some task overlaps that are unusual that leads to tasks executed in an overlapping nature.
2016-09-22 09:52:23 +01:00
Martyn Inglis
2f36e0dbcf Refactor to make the new send_to_provider methods take a notification not a notification ID.
- Driven by the fact we won't know the type in the API call
- hence we need to load notification earlier , so pass it not the id through to the send task to avoid loading it twice.
2016-09-22 09:16:58 +01:00
Leo Hemsted
00b905bef0 use a contextmanager to set config values
so that when later tests run, the config has been restored to its
defaults and we don't end up failing later tests in confusing ways
2016-09-21 17:13:26 +01:00
Leo Hemsted
24903a19ef add pagination to GET /service/{}/job
accepts a page parameter to control what page of data

returns additional pagination fields in the response dict
* page_size: will always be 50. defined by Config.PAGE_SIZE
* total: the total amount of unpaginated records
* links: dict containing optionally prev, next, and last, links to
  other relevant pagination pages

also cleaned up some test imports
2016-09-21 16:54:02 +01:00
Leo Hemsted
4bfcfcd2fb clean up test imports 2016-09-21 15:45:26 +01:00
Leo Hemsted
29abd4c382 make dao_get_jobs_by_service_id paginated
it can return a pretty long list, especially when we run lots of
smoke tests, so make it accept pagination parameters, and return
a pagination object
2016-09-21 14:35:23 +01:00
Leo Hemsted
94b280af63 ensure conftest sets up datetimes within function bodies
this makes sure it respects any freezegun.freeze_times that might
be in place
2016-09-21 14:34:38 +01:00
Andras Ferencz-Szabo
0c5720108d Merge pull request #681 from alphagov/deploy_db_migrations
Deploy db migrations
2016-09-21 14:27:07 +01:00
Martyn Inglis
991cc884b4 Refactoring the send to provider code out of the tasks folder
- building a rest endpoint to call that code to compliment the existing task based approach.
2016-09-21 13:27:32 +01:00
bandesz
8ab7fa3395 Build db migration codedeploy artifact 2016-09-21 11:21:12 +01:00
bandesz
d69a07fed9 Delete accidentally committed notifications-api.zip 2016-09-21 11:21:12 +01:00
imdadahad
c94784b68c Merge pull request #680 from alphagov/update-return-all-notifications-schema
Update schema to validate contents of GET notifications
2016-09-21 09:52:51 +01:00
Imdad Ahad
23998a4d55 Add additionalProperties to links object 2016-09-21 09:47:50 +01:00
Martyn Inglis
bd06b18684 Refactor of celery tasks.
- Aim to move the code that contacts providers into it's own module.
- Celery tasks now call this module to send to provider
- No exceptions caught in the new module. Celery tasks now use any exception to trigger a retry.
- tests moved about - new test directory for the new class, all tests from celery test module moved, excepting the retry logic.
2016-09-20 17:24:28 +01:00
Imdad Ahad
afd381badf Update links schema to be more strict and fix import 2016-09-20 10:39:15 +01:00
Rebecca Law
51b3119a21 Remove provider_statistics_dao.get_provider_statistics
The provider_statistics table is no longer being populated, get rid of any reads from this table.
2016-09-19 17:24:26 +01:00
minglis
ffdf3272fb Merge pull request #679 from alphagov/remove-test-keys-from-admin-app
Remove test keys from admin app
2016-09-19 12:31:12 +01:00
minglis
1d28f09c22 Merge pull request #677 from alphagov/no-jobs-on-api-get
No jobs on api get
2016-09-19 12:31:04 +01:00
Chris Hill-Scott
aeba205b31 Merge pull request #678 from alphagov/better-auth-error-message
Improve error messages if you make an API call with a bad token
2016-09-19 10:15:20 +01:00
Imdad Ahad
5b0162cc60 Update schema to validate {notifications} contains a {links} object with expected properties 2016-09-16 14:56:14 +01:00
Chris Hill-Scott
cebbb2077c Merge pull request #676 from alphagov/bump-utils-fix-email-bullets
Fix HTML lists in Microsoft Outlook
2016-09-16 14:04:50 +01:00
Martyn Inglis
7c16294b75 Exclude test key notifications from:
- get all notifications by service
- template usage
- most recently used templates

Ensures that the dashboard shows no test key data. Supplements: https://github.com/alphagov/notifications-api/pull/677 which excludes CSV data. This branches from that so is dependant.
2016-09-16 13:47:09 +01:00
Chris Hill-Scott
d44a0b72bb Rewrite authentication error messages more English 2016-09-16 08:57:55 +01:00
Chris Hill-Scott
1ce91997e8 Give specifc error when service doesn’t exist
If you sign a token with a service ID that doesn’t exist (say, for
example, that you get service ID and API key mixed up) then you get
an error saying that “no API keys exist for the service”. This is wrong
because the service doesn’t even exist.

This commit adds:
- code to check if the service does exist
- a specific error message for this case

The check does mean an extra database call to look up the service.
However this only happens _after_ looping through all the API keys. So
it shouldn’t have a performance implication for anyone using a valid API
key.
2016-09-16 08:49:49 +01:00
Chris Hill-Scott
6b3c899127 Add a test for auth-ing with non-existant service
If you create a token signed with a service ID that doesn’t exist, you
will get an error (as you should).

However we didn’t have a test that explicitly checks for this. This
commit adds one.
2016-09-16 08:46:32 +01:00
Martyn Inglis
d4971cc45f Ensure GET /notifications returns JOB or API data correctly, using new boolean query param "include_jobs" 2016-09-15 16:01:26 +01:00
Martyn Inglis
fe5d53bd70 Ensure GET /notifications/ by service ID returns CSV and API data - used by ADMIN console.
- this is existing functionality mimicked.
2016-09-15 16:00:46 +01:00
Martyn Inglis
23062233b6 Setup code for various key types and separated notifications made by CSV and by API into separate fixtures. 2016-09-15 16:00:04 +01:00
Martyn Inglis
803820d9aa New field on GET /notifications to allow users to request CSV data 2016-09-15 15:59:34 +01:00
Martyn Inglis
9f02e94f38 Ensure that querying for notifications does not return CSV data unless:
- It's explicitly asked for AND
- you are using a LIVE key

Team and Test keys CANNOT retrieve CSV/Job derived notifications
2016-09-15 15:59:02 +01:00
Chris Hill-Scott
210ed6b912 Fix HTML lists in Microsoft Outlook
Depends on:
- [x] https://github.com/alphagov/notifications-utils/pull/69
2016-09-15 11:10:50 +01:00
imdadahad
b07bbbbe16 Merge pull request #673 from alphagov/provide-simulated-numbers-live-keys
Add capability for simulated numbers and email addresses
2016-09-14 15:23:12 +01:00
minglis
73cbfed467 Merge pull request #675 from alphagov/format-statsd
Format the stat name with environmenbt
2016-09-14 15:23:06 +01:00
Martyn Inglis
04f36fab21 Format the stat name with environmenbt 2016-09-14 15:22:50 +01:00
minglis
c67c819f0a Merge pull request #671 from alphagov/statsd-time-taken-to-be-delivered
Statsd time taken to be delivered
2016-09-14 14:53:01 +01:00
Imdad Ahad
98ef340bc6 Refactor to use .count() method instead of mocking fake UUID 2016-09-14 13:07:57 +01:00
Chris Hill-Scott
b403a3ebab Merge pull request #674 from alphagov/bump-requirements-faster-csv
Bump utils version
2016-09-14 12:37:24 +01:00
Imdad Ahad
af8cfbe78c Add checks to ensure simulated notifications are not persisted to database 2016-09-14 11:12:57 +01:00
Rebecca Law
5c41ce0ea5 Merge pull request #672 from alphagov/fix-dashboard-stats
Update the timeout_notifications scheduled tasks.
2016-09-14 10:51:37 +01:00
Rebecca Law
fc66576d0f Remove try catch, it was only logging the exception. 2016-09-14 10:38:34 +01:00
Chris Hill-Scott
e3af4b48da Bump utils version
Brings in:
- [x] https://github.com/alphagov/notifications-utils/pull/68
2016-09-14 10:29:45 +01:00
Imdad Ahad
d4b0f68a69 Refactor to re-use existing contract-dependent method 2016-09-14 10:25:09 +01:00
Rebecca Law
339d33723b Added updated_at to the update query 2016-09-13 17:35:23 +01:00
Imdad Ahad
9df24b39ce Add capability for simulated numbers and email addresses 2016-09-13 17:00:28 +01:00
Rebecca Law
cb51c69503 Update the timeout_notifications scheduled tasks.
We found that if the notifications were in created or pending they are not purged from notifications.
- New bulk update method to set all notificaitons with:
  -  a status = created|sending|pending to temporary-failure
  - and is older then today minus SENDING_NOTIFICATIONS_TIMEOUT_PERIOD (in seconds)
- the scheduled task to timeout notifications use the new bulk update query.
- the task will be more efficient
2016-09-13 16:42:53 +01:00
minglis
4f768b53e6 Merge pull request #658 from alphagov/drop-status-column
Drop the status column
2016-09-13 14:46:38 +01:00
Martyn Inglis
34bd17ce66 Record time in seconds 2016-09-13 14:40:04 +01:00