darkhelm/notifications-api
1
0
Fork 0
mirror of https://github.com/GSA/notifications-api.git synced 2025-12-22 16:31:15 -05:00
Code Issues Packages Projects Releases Wiki Activity
6,807 Commits 128 Branches 60 Tags
10ea8c38cda4cea8deb8979c40f2802726b734e4
Commit Graph

225 Commits

Author SHA1 Message Date
Imdad Ahad
5b9377c697 Start populating monthly billing on a schedule 2017-08-02 15:32:32 +01:00
Leo Hemsted
3cb3cf438e remove SEND_COMBINED 2017-07-31 13:29:30 +01:00
Ken Tsang
3e6e75998b Update config to use logo cdn 2017-07-26 11:05:31 +01:00
Leo Hemsted
4d33040653 add separate send-sms and send-email queues 
we're reading from those two queues as well as teh existing send queue,
however for now we don't send anything to them
 2017-07-20 16:19:38 +01:00
Leo Hemsted
6c61a3fc2a Revert celery4 
Revert the following three pull requests:
https://github.com/alphagov/notifications-api/pull/1085
https://github.com/alphagov/notifications-api/pull/1086
https://github.com/alphagov/notifications-api/pull/1088

celery 4.0.2 looked promising, however, on staging under mild load
(5/sec api calls) the performance was actually worse than 3.1.25
 2017-07-19 15:17:19 +01:00
Martyn Inglis
0e9e8955f7 Finished celery refactor - set up config for queue prefix 
LEO notes: Also made sure the Test BROKER_URL is preserved so that
tests warn you when celery isn't mocked out
 2017-07-12 12:37:18 +01:00
Martyn Inglis
786adb5d71 Move Queuenames in with the celery code, revamp config to allow move to celery 4.x 2017-07-12 12:01:52 +01:00
Martyn Inglis
e0106eb1be hacked celery4.0.2 in. Runs and works 
- note though this version of master I branched had split head on sqlalchemey. This needs a new master merge to fix
 2017-07-12 11:58:04 +01:00
Leo Hemsted
7f883f1355 don't store non-strings to os.environ 
in tests, we were replacing os.environ with a basic dict so that
we didn't overwrite the contents of the real environment during tests.
However, os.environ doesn't accept non-str values, so this commit
changes the fixture so that it asserts all values set are strings.
We needed to change how we store ip whitelist stuff in the env because
of this.
 2017-07-11 15:41:44 +01:00
venusbb
5d57189187 changed name of test fixture 2017-07-11 09:50:09 +01:00
venusbb
5089a4d53b retrieve sms ip whitelist from credentials on paas 2017-07-10 17:03:43 +01:00
venusbb
39a8e1cfed IP address in config file uses comma rather than full stops 2017-07-06 12:30:54 +01:00
venusbb
d6c69bf437 Check inbound sms IP address stage 1 
IP address checked
 2017-07-06 12:29:37 +01:00
Leo Hemsted
ac7665bfc6 celery test cleanup 
* Alter config so an error will be raised if you forget to mock out a
  celery call in one of your tests
* Remove an unneeded exception type that was masking errors
 2017-06-20 12:06:49 +01:00
Leo Hemsted
88a479a4bb make client request fixture args conform 2017-06-20 12:06:49 +01:00
Imdad Ahad
f29ebb34f6 Add scheduled task to delete response files older than 7 days 2017-06-13 12:40:58 +01:00
Imdad Ahad
34f4829362 Merge pull request #1025 from alphagov/imdad-feat-remove-dvla-transformed-files 
Remove transformed DVLA files older than 7 days
 2017-06-13 10:23:31 +01:00
minglis
1b0ff63b1e Merge pull request #1022 from alphagov/remove-nasty-query-from-dashboard 
Remove nasty query from dashboard
 2017-06-08 10:58:35 +01:00
Imdad Ahad
c4a89b77e3 Add task to remove transformed dvla files: 
* Runs at 4:40am
* Remove files older than 7 days and only two day's worth
 2017-06-07 16:39:23 +01:00
Imdad Ahad
74a8905be9 Seperate deletion of jobs: 
* Two separate jobs, one for sms&email and another for letter
* Change celery task for delete to accept template type filter
* General refacor of tests to make more readable
 2017-06-06 16:02:01 +01:00
Martyn Inglis
75bf693f44 Add the yearly free limit to the service model. 
This allows us to reference it across the API code base and return it in the API.

But not currently attached to the service DB model - a static method on the class.
 2017-06-06 14:49:05 +01:00
Imdad Ahad
56c3f3cf7c Add task to delete inbound sms everyday at 1am 2017-06-02 16:10:27 +01:00
Chris Hill-Scott
eb493f6817 Rename ‘One-off message’ to ‘Report’ 
See 7915845cb5
 2017-06-01 13:56:47 +01:00
Chris Hill-Scott
78d071f22f Ignore one-off messages in job list on dashboard 
Same as how we ignore ‘send yourself a test’ messages (see:
d8467bfc3c). The dashboard gets clogged
up with one off messages otherwise, which affects:
- performance
- users ability to find their jobs
 2017-06-01 12:30:47 +01:00
Martyn Inglis
d74675b6d8 Removed references to old queus 2017-05-30 11:24:26 +01:00
Martyn Inglis
75c5e5e5d5 Merge branch 'master' into redo-queue-visibitlity-timeout 
Conflicts:
	app/notifications/process_notifications.py
	app/v2/notifications/post_notifications.py
 2017-05-30 10:18:18 +01:00
Martyn Inglis
29b6a918e8 Ensure dev and test builds will read from old and new queues 2017-05-25 13:43:31 +01:00
Martyn Inglis
58af4e5ee9 timeout slightly longer than retry 2017-05-25 11:20:23 +01:00
Martyn Inglis
21586c917c Created an object to hold queue names. 
- This is to be used throughout the app.
- Allows us to manage queue names centrally
- All queues renamed to allow us to change the retry processing/visibility timeout
 2017-05-25 10:50:55 +01:00
Rebecca Law
9f6c037530 Use iso8601 to validate scheduled_for datetime. 
Added a validation method that always fails for scheduled notifications.
Comment out config for scheduled task.
The schedule notifications will be turned on once we can invite services to use it.
Waiting for the service permission story, must commit this in order to keep things from going stale.
 2017-05-24 16:27:15 +01:00
Martyn Inglis
07b527bb1b All queues now managed in the same way: 
- TEST and DEV builds set up their own queue lists for reading
- PROD/STAGE/PREVIEW use the separate worker process with the -Q flag

This enables us to rename queues in due course
 2017-05-24 15:47:20 +01:00
Rebecca Law
96b0e42959 Merge branch 'master' into schedule-api-notification 
Conflicts:
	app/celery/scheduled_tasks.py
	tests/app/celery/test_scheduled_tasks.py
 2017-05-24 13:21:22 +01:00
minglis
8a2715e3e1 Merge pull request #980 from alphagov/change-delete-jobs-to-delete-by-type 
Change delete jobs to delete by type
 2017-05-24 12:35:04 +01:00
Rebecca Law
53efb4e033 Merge branch 'master' into schedule-api-notification 
Conflicts:
	tests/app/v2/notifications/test_post_notifications.py
 2017-05-23 13:58:50 +01:00
Martyn Inglis
a751637eaa Added the schedule into celery beat. 
run the delete jobs 20 mins apart,
00:00 -> sms
00:20 -> email
00:40 -> letters
 2017-05-23 13:41:22 +01:00
Leo Hemsted
3744463296 treat 40604 and GOVUK as not having a sender ID 
in prep for removing the 40604-as-default, first we need to make sure
that if you either have GOVUK or None as your sms sender, then we send
GOVUK through to the provider
 2017-05-22 17:10:07 +01:00
Rebecca Law
76d0783c63 Add beat config for send-scheduled-notifications task to run every 15 minutes. 
Added the missing commit to the update pending
 2017-05-22 16:30:45 +01:00
Martyn Inglis
8dc7a86148 Merge branch 'master' into async-job-stats 
Conflicts:
	app/dao/services_dao.py
 2017-05-18 09:34:54 +01:00
minglis
05a179c6ef Merge pull request #941 from alphagov/load-service-on-auth 
Load service on auth
 2017-05-17 10:49:00 +01:00
Martyn Inglis
83a1b1526e Merge branch 'master' into async-job-stats 
Conflicts:
	app/notifications/notifications_ses_callback.py
 2017-05-17 09:49:43 +01:00
Imdad Ahad
570d0ec9db Send notification to active service users with user fields (optional) 2017-05-12 14:06:29 +01:00
Martyn Inglis
02f7abcc48 Wire in the new task. Runs at 5am after the other tasks. 2017-05-11 15:23:45 +01:00
Martyn Inglis
a4549a2e26 Put the statistics tasks into a new queue. This currently has no worker in PaaS so blocks the deployment until new PaaS worker is configured. 2017-05-09 22:03:13 +01:00
Martyn Inglis
07daa369a3 Fixed some config issues 2017-05-05 15:20:34 +01:00
Martyn Inglis
97dcf766bc Switch on API rate limiting on production. 
Limits POST API calls to send notifications on V1/V2 to 3000 in 60 seconds.
 2017-05-03 12:13:32 +01:00
Martyn Inglis
a9539d892c Merge branch 'master' into rate-limit-api-calls 
Conflicts:
	requirements.txt
	tests/app/notifications/rest/test_send_notification.py
	tests/app/notifications/test_validators.py
	tests/app/v2/notifications/test_post_notifications.py
 2017-05-02 10:56:56 +01:00
Leo Hemsted
02e8f454e1 add new notification status 'sent' 
for international text messages, when we may not ever receive callback info,
so we need a separate status that doesn't get timed out etc
 2017-04-25 17:08:29 +01:00
Martyn Inglis
803c21865b Allow rate limiting on a per env basis 
- switched off on prod by default
 2017-04-25 09:54:09 +01:00
Martyn Inglis
926b8a60f9 Adds in call to new rate limit method in the redis client 
- both V1 and V2 APIs
- Rate limiting wrapped into a new method - check_rate_limiting
	- delegates to the previous daily limit and the new though put limit
- Rate limiting done on key type. Each key has it's own limit (number of requests) and interval (time period of requests)
- Configured in the config. Not done on a per-env basis though could be in the future.
 2017-04-24 14:15:08 +01:00
Martyn Inglis
b08ac6eb63 Ensure priority queue read by development machines. It's now a default queue in the config. 2017-04-04 13:53:01 +01:00