Commit Graph

1408 Commits

Author SHA1 Message Date
Leo Hemsted
2e16be18bc Merge branch 'master' of github.com:alphagov/notifications-api into service-whitelist 2016-09-23 10:14:42 +01:00
Leo Hemsted
281323b435 Merge pull request #683 from alphagov/paginate-dashboard
Paginate `GET /service/{}/job` query
2016-09-23 09:54:47 +01:00
Chris Hill-Scott
b36d0ff552 Make indentation more sensible
Starting arguments on their own line and putting the closing parenthesis
on it’s own line because any subsequent changes to the arguments diff
cleanly (ie without touching any other lines).
2016-09-23 09:43:25 +01:00
Leo Hemsted
af0dbd14be correct service_whitelist dao/rest functionality
additionally added dao tests
2016-09-22 17:17:34 +01:00
Rebecca Law
30dd602dfb Merge branch 'master' into remove-provider_statistics
Conflicts:
	tests/app/notifications/rest/test_callbacks.py
2016-09-22 11:20:19 +01:00
Leo Hemsted
203936fa84 add GET/POST rest endpoints for whitelist
GET /<service_id>/whitelist
    returns all whitelisted contacts for a service, separated into two lists

POST /<service_id>/whitelist
    removes all existing whitelisted contacts, and replaces them with the
    provided new entries

(todo: dao work + tests)
2016-09-22 10:32:32 +01:00
Leo Hemsted
29abd4c382 make dao_get_jobs_by_service_id paginated
it can return a pretty long list, especially when we run lots of
smoke tests, so make it accept pagination parameters, and return
a pagination object
2016-09-21 14:35:23 +01:00
Rebecca Law
51b3119a21 Remove provider_statistics_dao.get_provider_statistics
The provider_statistics table is no longer being populated, get rid of any reads from this table.
2016-09-19 17:24:26 +01:00
Martyn Inglis
7c16294b75 Exclude test key notifications from:
- get all notifications by service
- template usage
- most recently used templates

Ensures that the dashboard shows no test key data. Supplements: https://github.com/alphagov/notifications-api/pull/677 which excludes CSV data. This branches from that so is dependant.
2016-09-16 13:47:09 +01:00
Martyn Inglis
9f02e94f38 Ensure that querying for notifications does not return CSV data unless:
- It's explicitly asked for AND
- you are using a LIVE key

Team and Test keys CANNOT retrieve CSV/Job derived notifications
2016-09-15 15:59:02 +01:00
minglis
c67c819f0a Merge pull request #671 from alphagov/statsd-time-taken-to-be-delivered
Statsd time taken to be delivered
2016-09-14 14:53:01 +01:00
Rebecca Law
339d33723b Added updated_at to the update query 2016-09-13 17:35:23 +01:00
Rebecca Law
cb51c69503 Update the timeout_notifications scheduled tasks.
We found that if the notifications were in created or pending they are not purged from notifications.
- New bulk update method to set all notificaitons with:
  -  a status = created|sending|pending to temporary-failure
  - and is older then today minus SENDING_NOTIFICATIONS_TIMEOUT_PERIOD (in seconds)
- the scheduled task to timeout notifications use the new bulk update query.
- the task will be more efficient
2016-09-13 16:42:53 +01:00
Martyn Inglis
8541f6adcd Change DAO to return the notification from an update, not Boolean
- tests reflect this
2016-09-13 12:29:40 +01:00
Martyn Inglis
e0d46d821d Merge branch 'master' into delete-csv-file-after-7-days
Conflicts:
	app/notifications/rest.py
2016-09-12 16:33:01 +01:00
Martyn Inglis
dde22bc58b Implemented deleted notification if SQS write fails 2016-09-08 16:00:18 +01:00
Martyn Inglis
c3657839e4 New jobs dao method to get jobs that are older than a certain number of days.
Used in deleting CSV files scheduled task
2016-09-07 15:36:07 +01:00
Chris Hill-Scott
2aa8baec63 Make indentation consistent 2016-09-02 23:22:15 +01:00
Chris Hill-Scott
2c17825c12 Don’t use magic string for job status 2016-09-02 23:22:14 +01:00
Chris Hill-Scott
11a4b18451 Combine query filters when getting scheduled job
> filter_by and filter are just aliases for each other so can be
> combined together - filter is probably the better one (and then use
> == instead of keyword args)
2016-09-02 12:27:23 +01:00
Chris Hill-Scott
4a7267be8b Add an endpoint to cancel a job
If you schedule a job you might change your mind or circumstances might
change. So you need to be able to cancel it. This commit adds a `POST`
endpoint for individual jobs which sets their status to `cancelled`.

This also means adding a new status of `cancelled`, so there’s a
migration…
2016-09-02 12:27:02 +01:00
minglis
4a078e3c61 Merge pull request #635 from alphagov/scheduled-delivery-of-jobs
Scheduled delivery of jobs
2016-08-30 12:48:48 +01:00
Leo Hemsted
4f0c5fdd9e Merge pull request #623 from alphagov/test-fixes
Test fixes
2016-08-30 10:38:53 +01:00
Leo Hemsted
7835339165 Merge branch 'master' into test-fixes 2016-08-26 10:57:41 +01:00
Martyn Inglis
8b9319ac7a Merge branch 'master' into scheduled-delivery-of-jobs
Conflicts:
	app/dao/jobs_dao.py
	tests/app/dao/test_jobs_dao.py
	tests/app/job/test_rest.py
2016-08-25 14:53:00 +01:00
Martyn Inglis
05abd15026 Merge branch 'master' into remove-contested-writes 2016-08-25 14:38:04 +01:00
Martyn Inglis
5012d20d58 Merge branch 'master' into remove-contested-writes 2016-08-25 14:37:51 +01:00
Leo Hemsted
d89e9e9930 Merge pull request #628 from alphagov/notification-statistics
Platform admin notification query
2016-08-25 14:37:46 +01:00
minglis
415f2416cf Merge pull request #627 from alphagov/update-job-page-query-2
Returns the outcome statistics for the job on the API call.
2016-08-25 12:07:20 +01:00
Martyn Inglis
893164ae40 Removed contented updates the notifications stats table
- As before this is now driven from the notifications history table

- Removed from updates and create
- Signatures changes to removed unused params hits many files
- Also potential issue around rate limiting - we used to get the number sent per day from the stats table - which was a single row lookup, now we have to count this. This applies to EVERY API CALL. Probably not a good thing and should be addressed urgently.
2016-08-25 11:55:38 +01:00
Martyn Inglis
708f566c24 Removed updates to the provider stats table
- again these new come from the notifications history table
- We update this when we sent a notification, so removed from celery tasks
- tests removed also
2016-08-25 10:33:12 +01:00
Martyn Inglis
84ea173ced Removed updates to templates statistics
- on create notification we updated the templates stats to record the usage.
- this is now based on notification history
- this update and associated tests are now removed,
2016-08-25 10:12:39 +01:00
Martyn Inglis
44bc071037 Removed the updates to the job table to track delivery of notifications
Previously we kept a running total of job progress/success/failure on the job table. This causes contention, we now generate this data from notification history.

Removed these updates.
2016-08-25 09:29:53 +01:00
Martyn Inglis
9af85d9cb3 new jobs query to get all scheduled jobs 2016-08-24 16:24:30 +01:00
Leo Hemsted
29df7edaf9 tests for detailed services 2016-08-24 15:00:51 +01:00
Leo Hemsted
00d19f63f0 group results by service using itertools
allows us to nicely reuse the existing format_statistics function
2016-08-24 15:00:51 +01:00
Leo Hemsted
ebb13a1251 create initial stats query
get statistics for all services, for today only
2016-08-24 15:00:51 +01:00
Leo Hemsted
1e8ab35976 Merge branch 'master' into noti-stats-cleanup 2016-08-24 14:24:17 +01:00
Martyn Inglis
acf3c568db Tidied up the query indentation 2016-08-24 10:50:29 +01:00
Martyn Inglis
09cb94081b Returns the outcome statistics for the job on the API call. 2016-08-23 16:46:58 +01:00
Leo Hemsted
26d7675baa pep8 fixes
no idea why the build/local pep8s weren't picking them up before.

also excluded import order pep8
2016-08-23 12:05:47 +01:00
Leo Hemsted
132addb181 Merge branch 'master' into noti-stats-cleanup 2016-08-23 09:49:12 +01:00
Martyn Inglis
4e6da1ba55 Changed template stats for template id
- now returns the most recent notification history row for that template ID.
- contains all the required data for the use cases for that template
2016-08-22 14:35:04 +01:00
Martyn Inglis
fdd85b7dc0 Merge branch 'master' into template_queryies 2016-08-22 11:42:26 +01:00
Martyn Inglis
b7476a1975 Removed the group by day aspects of template stats. Not needed. Grouped by template only. 2016-08-22 10:38:44 +01:00
Martyn Inglis
f74000f548 Add query to get template usage from the Notifications History table
- groups by template Id and Day.

Returns count per day, template name, template id, template type, and day.

Ordered by day (desc) and template name (acc)
2016-08-18 12:06:00 +01:00
Leo Hemsted
5b3a0f03d3 rename actual_template to template_history
it's slightly less emotionally charged
2016-08-12 10:29:23 +01:00
Leo Hemsted
f065b08db2 remove unused dao functions 2016-08-11 16:13:53 +01:00
Leo Hemsted
4ba3745159 update schema to use template_history for accurate template details
only in the public notification endpoint so far for fear of breaking
things - in an ideal world i'd remove the template relationship
from models entirely and replace that with actual_template
2016-08-09 16:57:18 +01:00
Leo Hemsted
46c0728b12 add actual_template relationship to notification
also renamed the function to make it apparent that it'll join and grab personalisation
2016-08-09 16:57:18 +01:00