Commit Graph

343 Commits

Author SHA1 Message Date
minglis
8a2715e3e1 Merge pull request #980 from alphagov/change-delete-jobs-to-delete-by-type
Change delete jobs to delete by type
2017-05-24 12:35:04 +01:00
Rebecca Law
53efb4e033 Merge branch 'master' into schedule-api-notification
Conflicts:
	tests/app/v2/notifications/test_post_notifications.py
2017-05-23 13:58:50 +01:00
Martyn Inglis
a751637eaa Added the schedule into celery beat.
run the delete jobs 20 mins apart,
00:00 -> sms
00:20 -> email
00:40 -> letters
2017-05-23 13:41:22 +01:00
Leo Hemsted
3744463296 treat 40604 and GOVUK as not having a sender ID
in prep for removing the 40604-as-default, first we need to make sure
that if you either have GOVUK or None as your sms sender, then we send
GOVUK through to the provider
2017-05-22 17:10:07 +01:00
Rebecca Law
76d0783c63 Add beat config for send-scheduled-notifications task to run every 15 minutes.
Added the missing commit to the update pending
2017-05-22 16:30:45 +01:00
Martyn Inglis
8dc7a86148 Merge branch 'master' into async-job-stats
Conflicts:
	app/dao/services_dao.py
2017-05-18 09:34:54 +01:00
minglis
05a179c6ef Merge pull request #941 from alphagov/load-service-on-auth
Load service on auth
2017-05-17 10:49:00 +01:00
Martyn Inglis
83a1b1526e Merge branch 'master' into async-job-stats
Conflicts:
	app/notifications/notifications_ses_callback.py
2017-05-17 09:49:43 +01:00
Imdad Ahad
570d0ec9db Send notification to active service users with user fields (optional) 2017-05-12 14:06:29 +01:00
Martyn Inglis
02f7abcc48 Wire in the new task. Runs at 5am after the other tasks. 2017-05-11 15:23:45 +01:00
Martyn Inglis
a4549a2e26 Put the statistics tasks into a new queue. This currently has no worker in PaaS so blocks the deployment until new PaaS worker is configured. 2017-05-09 22:03:13 +01:00
Martyn Inglis
07daa369a3 Fixed some config issues 2017-05-05 15:20:34 +01:00
Martyn Inglis
97dcf766bc Switch on API rate limiting on production.
Limits POST API calls to send notifications on V1/V2 to 3000 in 60 seconds.
2017-05-03 12:13:32 +01:00
Martyn Inglis
a9539d892c Merge branch 'master' into rate-limit-api-calls
Conflicts:
	requirements.txt
	tests/app/notifications/rest/test_send_notification.py
	tests/app/notifications/test_validators.py
	tests/app/v2/notifications/test_post_notifications.py
2017-05-02 10:56:56 +01:00
Leo Hemsted
02e8f454e1 add new notification status 'sent'
for international text messages, when we may not ever receive callback info,
so we need a separate status that doesn't get timed out etc
2017-04-25 17:08:29 +01:00
Martyn Inglis
803c21865b Allow rate limiting on a per env basis
- switched off on prod by default
2017-04-25 09:54:09 +01:00
Martyn Inglis
926b8a60f9 Adds in call to new rate limit method in the redis client
- both V1 and V2 APIs
- Rate limiting wrapped into a new method - check_rate_limiting
	- delegates to the previous daily limit and the new though put limit
- Rate limiting done on key type. Each key has it's own limit (number of requests) and interval (time period of requests)
- Configured in the config. Not done on a per-env basis though could be in the future.
2017-04-24 14:15:08 +01:00
Martyn Inglis
b08ac6eb63 Ensure priority queue read by development machines. It's now a default queue in the config. 2017-04-04 13:53:01 +01:00
Martyn Inglis
787d0da7db Broke the periodic queue out from the standard worker.
- now has own worker with worker count of 2
- this is too ensure that any issues with scheduled jobs do not affect core applications
2017-04-04 11:23:54 +01:00
Martyn Inglis
8f5e1318eb Spread out the scheduled jobs.
Before the longer tasks all ran on the hour, at midnight, 1 and 2.

Now we run one at midnight, one at quarter past, half past then at quarter too. Trying to spread the load.
2017-04-03 11:17:27 +01:00
Ken Tsang
316d8fdb0d Updated config to cloudfoundry_config 2017-03-31 16:37:04 +01:00
Ken Tsang
c633e193b0 Update config to enable performance platform 2017-03-31 15:54:29 +01:00
Ken Tsang
989713b0f1 Update config for daily perf job to start 4am 2017-03-31 15:33:41 +01:00
bandesz
974b59a5b7 Add 'production' alias for live environment 2017-03-21 14:43:56 +00:00
Rebecca Law
140179b4b6 Create new task to build dvla file.
This will transform each notification in a job to a row in a file.
The file is then uploaded to S3.
The files will later be aggregated by the notifications-ftp app to send to dvla.

The method to upload the file to S3 should be pulled into notifications-utils package.
It is the same method used in notifications-admin.
2017-03-15 15:26:58 +00:00
Imdad Ahad
37341e7a62 Updates:
* Add notify user id in config
* Add dao method to get provider history versions along with tests
* BUG: Provider switching did not handle case where priorities were equal. This
* adds a fix to properly cover this case along with tests
2017-03-03 10:54:20 +00:00
Imdad Ahad
17b6c13c46 Small updates:
* Make config use new provider sms template id
* create_notification to account for created status
* Small robustness addition to test
2017-02-27 13:16:48 +00:00
Imdad Ahad
ec65575f50 Set provider switch task to run on prod according to functional tests service 2017-02-24 12:23:39 +00:00
Imdad Ahad
d058626119 Add task to run every minute that will switch provider on slow delivery notifications 2017-02-24 12:23:39 +00:00
bandesz
e3cb908c1a Remove explicit api_host_name vars from the envs 2017-02-23 11:22:58 +00:00
Rebecca Law
52342afe3f Add a limit to the number of active 2fa codes that we create. At the moment that is set to 10. 2017-02-15 16:18:05 +00:00
Rebecca Law
681f52b691 Missed some print statements and unused imports. 2017-02-14 09:59:33 +00:00
Rebecca Law
458adefcb8 Added a redis cache for the template usage stats.
Cache expires every 10 minutes, but will help with the every 2 second query, especially when a job is running.
There is some clean up and qa to do for this yet
2017-02-13 18:47:29 +00:00
Imdad Ahad
708ef48198 Reset schedule to run at 00:30 daily 2017-02-03 14:57:42 +00:00
Imdad Ahad
1e541b5079 TEMPORARY: Execute every 10 minutes 2017-02-03 13:34:09 +00:00
Leo Hemsted
1650fb0807 Merge pull request #797 from alphagov/persist-letters
Persist letters
2017-01-30 14:24:38 +00:00
Imdad Ahad
ac1ca608dd Add performance platform url directly in config (not a secret) 2017-01-27 13:03:04 +00:00
Imdad Ahad
b04c524bc3 Add celery task to run daily at 00:30 that sends notification counts to performance platform 2017-01-27 12:30:56 +00:00
Imdad Ahad
ed52f41039 Add, init and config performance platform client to prepare/upload daily notification stats 2017-01-27 12:21:08 +00:00
Leo Hemsted
70cd3fb335 ensure that the celery workers know about the new db-letter queue
this fixes running locally and on paas, a separate PR is in
notifications-aws to fix work on aws
2017-01-24 12:32:20 +00:00
Rebecca Law
8ed0979251 increase page size for API calls to 250 2017-01-20 12:26:55 +00:00
Rebecca Law
e360a1f8f6 Merge branch 'master' into priority-by-template 2017-01-17 14:08:36 +00:00
bandesz
fd66fbd719 Run API on Paas 2017-01-13 10:24:13 +00:00