Commit Graph
4286 Commits
Author SHA1 Message Date
Martyn Inglis f3c614634f Merge branch 'master' into seperate-queues-for-sending-and-for-db 2016-08-30 14:25:14 +01:00
minglis 4a078e3c61 Merge pull request #635 from alphagov/scheduled-delivery-of-jobs
Scheduled delivery of jobs
2016-08-30 12:48:48 +01:00
Martyn Inglis b923392c49 Using UTC dates on API validation for scheduled jobs. 2016-08-30 12:47:33 +01:00
Chris Hill-Scott 7769923dda Allow test key to send irrespective of daily limit
When you use a simulate API key it should behave like a live service,
except for actually sending the messages. There should be no limits even
if the service is in trial mode.

This commit removes the restriction on sending messages when you’ve sent
up to your daily limit.
2016-08-30 11:06:38 +01:00
Chris Hill-Scott 5eaec8c235 Allow test key to send irrespective of trial mode
When you use a simulate API key it should behave like a live service,
except for actually sending the messages. There should be no limits even
if the service is in trial mode.

This commit removes the restriction on sending messages to peole outside
your team when you’re in trial mode.
2016-08-30 11:06:15 +01:00
minglis 5c8c2eb039 Merge pull request #641 from alphagov/add-job-stats-to-get-all-jobs-query
Bug fix:
2016-08-30 11:00:16 +01:00
Martyn Inglis 486697d07c New queues for the sms/email tasks
Previously there were 4 queues for sending messages

The was based on the fact that each notification has 2 actions - persist in the database and send to provider.

Two queues supported the CSV upload - for the first of these tasks
- bulk-email
- build-sms

And there were two more queues for the tasks that make the 3rd party client calls.
- sms
- email

API Calls just used the latter two queues for both tasks

Added four new queues
- db-email
- db-sms
- send-sms
- send-email

So an API call puts a notification into the db-[type] queue first, which then puts the notification into the send-[type] queue

Build queues stay as before.

This will allow us to target processing of these tasks with separate workers to manage these differently.
2016-08-30 10:42:24 +01:00
Leo Hemsted 4f0c5fdd9e Merge pull request #623 from alphagov/test-fixes
Test fixes
2016-08-30 10:38:53 +01:00
minglis 287d32c037 Merge pull request #638 from alphagov/remove-contested-writes
Remove contested writes
2016-08-30 08:56:45 +01:00
Martyn Inglis e1543e9a5c More definite create times on job - some intermittent test fails where ordering ambiguous. 2016-08-26 15:32:46 +01:00
Martyn Inglis dd7850b235 Bug fix:
- Get all jobs for service doesn't return the statistics.
- Dashboard therefore is incorrect

This is the API fix for this, returns the data.
2016-08-26 15:07:59 +01:00
Chris Hill-Scott f09704e5f4 Merge pull request #624 from alphagov/sms-sender-no-prefix
Don’t prefix text messages if sender name is set
2016-08-26 14:57:12 +01:00
Chris Hill-Scott f7391da350 Don’t prefix text messages is sender name is set
Implements:
- [x] https://github.com/alphagov/notifications-utils/pull/66
2016-08-26 14:45:01 +01:00
Leo Hemsted 7835339165 Merge branch 'master' into test-fixes 2016-08-26 10:57:41 +01:00
Martyn Inglis 8b9319ac7a Merge branch 'master' into scheduled-delivery-of-jobs
Conflicts:
	app/dao/jobs_dao.py
	tests/app/dao/test_jobs_dao.py
	tests/app/job/test_rest.py
2016-08-25 14:53:00 +01:00
Martyn Inglis 05abd15026 Merge branch 'master' into remove-contested-writes 2016-08-25 14:38:04 +01:00
Martyn Inglis 5012d20d58 Merge branch 'master' into remove-contested-writes 2016-08-25 14:37:51 +01:00
Leo Hemsted d89e9e9930 Merge pull request #628 from alphagov/notification-statistics
Platform admin notification query
2016-08-25 14:37:46 +01:00
minglis 2d2ceba2d1 Merge pull request #631 from alphagov/add-new-column-to-jobs-for-delayed-sending
Adds new job_status table and FK to jobs.
2016-08-25 12:59:16 +01:00
minglis 415f2416cf Merge pull request #627 from alphagov/update-job-page-query-2
Returns the outcome statistics for the job on the API call.
2016-08-25 12:07:20 +01:00
Martyn Inglis 893164ae40 Removed contented updates the notifications stats table
- As before this is now driven from the notifications history table

- Removed from updates and create
- Signatures changes to removed unused params hits many files
- Also potential issue around rate limiting - we used to get the number sent per day from the stats table - which was a single row lookup, now we have to count this. This applies to EVERY API CALL. Probably not a good thing and should be addressed urgently.
2016-08-25 11:55:38 +01:00
Martyn Inglis 708f566c24 Removed updates to the provider stats table
- again these new come from the notifications history table
- We update this when we sent a notification, so removed from celery tasks
- tests removed also
2016-08-25 10:33:12 +01:00
Martyn Inglis 84ea173ced Removed updates to templates statistics
- on create notification we updated the templates stats to record the usage.
- this is now based on notification history
- this update and associated tests are now removed,
2016-08-25 10:12:39 +01:00
Martyn Inglis 44bc071037 Removed the updates to the job table to track delivery of notifications
Previously we kept a running total of job progress/success/failure on the job table. This causes contention, we now generate this data from notification history.

Removed these updates.
2016-08-25 09:29:53 +01:00
Martyn Inglis 2cffed9cd4 Added new scheduled task
- runs 1 min past the hour, every hour
- looks up all scheduled jobs that have a scheduled date in the past and adds them to the normal process job queue
- these are then processed as normal
2016-08-24 17:03:56 +01:00
Martyn Inglis 1bddf66ed5 Setup code for tests for scheduled jobs. 2016-08-24 16:24:55 +01:00
Martyn Inglis 9af85d9cb3 new jobs query to get all scheduled jobs 2016-08-24 16:24:30 +01:00
Martyn Inglis 10f499805c Updates to create job to handle scheduling
- If the job JSON contains a scheduling date then the new 'job_status" column is set to "scheduled"
- the date is persisted on the JOB row in the database
- Also the job WILL NOT be placed onto the queue of jobs. This is deferred to a later celery beat task.
2016-08-24 16:00:21 +01:00
Leo Hemsted 556b69a487 still return service if they have never sent any notifications 2016-08-24 15:00:51 +01:00
Leo Hemsted 29df7edaf9 tests for detailed services 2016-08-24 15:00:51 +01:00
Leo Hemsted 00d19f63f0 group results by service using itertools
allows us to nicely reuse the existing format_statistics function
2016-08-24 15:00:51 +01:00
Martyn Inglis cf3c14dbbd Added job_status to marshmallow
- ensured statues not deleted on test runs
- returns in API call

Merge branch 'add-new-column-to-jobs-for-delayed-sending' into scheduled-delivery-of-jobs

Conflicts:
	app/models.py
2016-08-24 14:35:22 +01:00
Leo Hemsted 1e8ab35976 Merge branch 'master' into noti-stats-cleanup 2016-08-24 14:24:17 +01:00
Martyn Inglis 48089e21e5 Adding default values 2016-08-24 14:16:39 +01:00
Martyn Inglis 1e10c0485c Removed some code 2016-08-24 14:05:54 +01:00
Martyn Inglis 805da23d6b Starting to implement scheduling 2016-08-24 14:04:52 +01:00
Martyn Inglis 3006c7af54 Removed print 2016-08-24 10:59:33 +01:00
Martyn Inglis 09cb94081b Returns the outcome statistics for the job on the API call. 2016-08-23 16:46:58 +01:00
Leo Hemsted 26d7675baa pep8 fixes
no idea why the build/local pep8s weren't picking them up before.

also excluded import order pep8
2016-08-23 12:05:47 +01:00
Leo Hemsted 132addb181 Merge branch 'master' into noti-stats-cleanup 2016-08-23 09:49:12 +01:00
Martyn Inglis 4e6da1ba55 Changed template stats for template id
- now returns the most recent notification history row for that template ID.
- contains all the required data for the use cases for that template
2016-08-22 14:35:04 +01:00
Martyn Inglis fdd85b7dc0 Merge branch 'master' into template_queryies 2016-08-22 11:42:26 +01:00
Leo Hemsted 5418f65ae7 Merge pull request #599 from alphagov/version-500
fix GET /notifications 500 error
2016-08-22 10:59:46 +01:00
Martyn Inglis b7476a1975 Removed the group by day aspects of template stats. Not needed. Grouped by template only. 2016-08-22 10:38:44 +01:00
Martyn Inglis ede7d0cbea Removed old endpoint.
Going to handle the migration in the clients.
2016-08-18 14:06:12 +01:00
Martyn Inglis 7a5acea71b New endpoint for template stats to use new query. Breaking change to the returned JSON, so adding on a different url. 2016-08-18 14:01:31 +01:00
Martyn Inglis f74000f548 Add query to get template usage from the Notifications History table
- groups by template Id and Day.

Returns count per day, template name, template id, template type, and day.

Ordered by day (desc) and template name (acc)
2016-08-18 12:06:00 +01:00
Leo Hemsted 721929ca75 Merge pull request #589 from alphagov/org-rendering
Render organisational branding in emails
2016-08-16 13:26:09 +01:00
Leo Hemsted 5491668579 let users set organisation on POST /service/{id} 2016-08-15 10:54:26 +01:00
Leo Hemsted cc7ea8043c add organisation and branding to GET /service response dict 2016-08-12 11:40:57 +01:00