darkhelm/notifications-api
1
0
Fork 0
mirror of https://github.com/GSA/notifications-api.git synced 2026-01-01 13:21:44 -05:00
Code Issues Packages Projects Releases Wiki Activity
3,327 Commits 126 Branches 60 Tags
7aaef8fc090a4e727fa6596a38d5ff41d690c3f0
Commit Graph

297 Commits

Author SHA1 Message Date
Ken Tsang
4fb5e68ce7 Refactor code 
- Created TaskNames for DVLA_FILES rather than have DVLA_FILES in QueueNames
- Removed PROCESS_FTP from all_queues() as this was causing problems in picking up letter job tasks
- Created test to ensure that we don't arbitrarily add queue names to all_queues
 2017-08-23 13:22:07 +01:00
Ken Tsang
b2b74e2b5d Fixed bug in letter job schedule 2017-08-23 13:22:07 +01:00
Ken Tsang
18881cd580 Add scheduled letter jobs 2017-08-23 13:22:07 +01:00
Leo Hemsted
c2152f9cb8 remove billable-units endpoint 
it wasn't used anywhere, and the return value changed when flask was upgraded
 2017-08-21 10:57:15 +01:00
Leo Hemsted
c36e50bef1 update dependencies 2017-08-18 17:02:31 +01:00
Imdad Ahad
94605d31fa Change how we populate and retrieve MonthlyBilling totals: 
1. For both email and sms, store [] in monthly_totals if
there is no billing data (no notifications sent etc.) and
return this via the API

2. General refactoring of indentation
 2017-08-15 16:09:42 +01:00
Imdad Ahad
63e1167098 Make sure we update the correct month for billing: 
When populating the monthly billing records on a schedule, we need
to ensure the correct month is being updated.

As an example, if the current datetime is 31 Mar 2016, 23:00. The
BST equivalent is the 1st April. Therefore we need to ensure we
update billing for April, not March. This takes care of that.
 2017-08-11 16:57:17 +01:00
Rebecca Law
c1f2634c90 Removed month and year and replaced it with start_date and end_date. 
This will allow us to sort the data properly.
 2017-07-26 13:19:17 +01:00
Rebecca Law
91f29517eb Merge branch 'master' into month-billing-table 2017-07-24 15:17:36 +01:00
Rebecca Law
3e2b8190b9 - Added a scheduled task to create or update billing for the month, yesterday is used to calculate the start and end date for the month. 
- The new task has not been added to the beat application yet.
- Added an updated_at column to the monthly billing table, we may want to only calculate from the last updated date rather than the entire month.
 2017-07-24 15:13:18 +01:00
Leo Hemsted
614880f6d9 send to send-sms-tasks and send-email-tasks instead of send-tasks 2017-07-21 13:49:37 +01:00
Leo Hemsted
6c61a3fc2a Revert celery4 
Revert the following three pull requests:
https://github.com/alphagov/notifications-api/pull/1085
https://github.com/alphagov/notifications-api/pull/1086
https://github.com/alphagov/notifications-api/pull/1088

celery 4.0.2 looked promising, however, on staging under mild load
(5/sec api calls) the performance was actually worse than 3.1.25
 2017-07-19 15:17:19 +01:00
Martyn Inglis
0e9e8955f7 Finished celery refactor - set up config for queue prefix 
LEO notes: Also made sure the Test BROKER_URL is preserved so that
tests warn you when celery isn't mocked out
 2017-07-12 12:37:18 +01:00
Imdad Ahad
c8c47f44a9 Don't update older status in timeout notifications task 2017-07-10 14:09:30 +01:00
Imdad Ahad
f0ffbad77d Fix syncing issue with Notification statuses: 
The timeout_notifications job runs daily and does not correctly
update both the status fields to keep them in sync.
 2017-07-06 12:30:08 +01:00
Rebecca Law
5ef0450e37 Update the json in the post to the service for the inbound sms. 2017-06-22 10:15:08 +01:00
Rebecca Law
c608f5997b New task to send the inbound sms to the service. 
If the service has not set the url then nothing happens.
If the request to the service url returns with 500 or greater the task is retries.
The task is created when the SMS provider post the inbound SMS.
 2017-06-20 17:13:40 +01:00
Imdad Ahad
1dacc6c3a3 Updates: 
* Skip the decorated tests as that doesn't work as we expect
* Remove magic numbers in tests
 2017-06-13 15:22:31 +01:00
Imdad Ahad
f29ebb34f6 Add scheduled task to delete response files older than 7 days 2017-06-13 12:40:58 +01:00
Imdad Ahad
c6db12731a Add test to check dvla files (unexpected) are not removed 2017-06-08 13:06:24 +01:00
Imdad Ahad
c4a89b77e3 Add task to remove transformed dvla files: 
* Runs at 4:40am
* Remove files older than 7 days and only two day's worth
 2017-06-07 16:39:23 +01:00
Imdad Ahad
74a8905be9 Seperate deletion of jobs: 
* Two separate jobs, one for sms&email and another for letter
* Change celery task for delete to accept template type filter
* General refacor of tests to make more readable
 2017-06-06 16:02:01 +01:00
Imdad Ahad
56c3f3cf7c Add task to delete inbound sms everyday at 1am 2017-06-02 16:10:27 +01:00
Martyn Inglis
75c5e5e5d5 Merge branch 'master' into redo-queue-visibitlity-timeout 
Conflicts:
	app/notifications/process_notifications.py
	app/v2/notifications/post_notifications.py
 2017-05-30 10:18:18 +01:00
Rebecca Law
f555c7a73b Refactor tests to use the create_notication in tests.app.db 2017-05-25 11:41:07 +01:00
Martyn Inglis
4768f0b9fd Change retries policy. 
Before we had a long back off, now we have more, but shorter backoffs.

- PREVIOUS
When we had an error talking to a provider we retried quickly and if we still got errors we backed off more and more. Maximum attempts was 5, max delay 4hours. This was to allow us time to ship a build if that was required.

- NOW
Backing off 48 times of 5 minutes each. This gives us the same total backoff, but many more tries in that period.

- WHY
Having the long back off meant messages could be delayed 4 hours. This was happening more and more, as PaaS deploys can place things into the "inflight" state in SQS. The inflight state MUST have an expiry time LONGER than the maximum retry back off. This meant that messages would be delayed 4 hours, even when there was no app error.

By doing this we can reduce this delay to 5 minutes. Whilst still giving us time to fix issues.
 2017-05-25 11:12:40 +01:00
Martyn Inglis
2591d3a1df This massive set of changes uses the new queue names object throughout the app and tests. 
Lots of changes, all changing the line of code that puts things into queues, and the code that tests that.
 2017-05-25 10:51:49 +01:00
Rebecca Law
383dee3bb2 Updated the serialization of Notification.scheduled_for to include minutes. 2017-05-24 14:52:32 +01:00
Rebecca Law
96b0e42959 Merge branch 'master' into schedule-api-notification 
Conflicts:
	app/celery/scheduled_tasks.py
	tests/app/celery/test_scheduled_tasks.py
 2017-05-24 13:21:22 +01:00
Martyn Inglis
a5c71ea907 New task to use the new DAO method to delete the data by type 2017-05-23 13:40:36 +01:00
Rebecca Law
9bfba52f53 Add pending flag to scheduled_notifications. 
Set pending flag to false when the notification has been sent to provider task.
 2017-05-22 15:07:16 +01:00
Rebecca Law
3a3161ecc4 Merge branch 'master' into schedule-api-notification 
Conflicts:
	app/celery/scheduled_tasks.py
	app/v2/notifications/post_notifications.py
	tests/app/celery/test_scheduled_tasks.py
 2017-05-22 14:05:57 +01:00
Imdad Ahad
02db3be37c General refactor 2017-05-22 10:12:18 +01:00
Rebecca Law
973cc2c4c9 Changed the scheduled_for datetime to only send and hour of a day to send. 
Also expect the date being passed in is BST. The date is converted to UTC before saving. And converted to BST when returning a notification.
 2017-05-17 15:06:15 +01:00
Martyn Inglis
83a1b1526e Merge branch 'master' into async-job-stats 
Conflicts:
	app/notifications/notifications_ses_callback.py
 2017-05-17 09:49:43 +01:00
Rebecca Law
2e078f9fc8 Add scheduled task to send scheduled notifcations. 
Fix the query to fetch scheduled notifications.
 2017-05-16 13:47:22 +01:00
Imdad Ahad
4003edfa67 Add DVLA callback: 
* Process SNS callback, trigger the update notifications celery task
* Put autoconfirm into its own method and use in callbacks
 2017-05-15 11:12:31 +01:00
Imdad Ahad
ee484ec368 Add get_s3_file method for use in DVLA processing 2017-05-15 10:48:37 +01:00
Imdad Ahad
7a10a91262 Revert "Process SNS request triggered by a DVLA S3 update" 2017-05-12 17:21:07 +01:00
Imdad Ahad
0f7093fc38 Refactor and add filename in logging 2017-05-12 14:32:01 +01:00
Imdad Ahad
8a5e82904e Update to pull from correct bucket and fix tests not mocking out correctly 2017-05-12 14:24:26 +01:00
Imdad Ahad
f766f90207 Add task to process a DVLA response file: 
* Currently we do nothing with the parsed response. We will
* update the status of the notifications in a separate PR
 2017-05-12 14:24:26 +01:00
Martyn Inglis
1a61f06f53 Removed test as no longer make distinction on status. 2017-05-12 12:21:36 +01:00
Martyn Inglis
8818bd5dba Scheduled task to call the timeout function 2017-05-11 15:22:57 +01:00
Martyn Inglis
76494a285e New logic to: 
- increment sent only if the notifications is in a NON-finished state
- Increment outcome only if notification is in a finished state
 2017-05-11 12:10:46 +01:00
Martyn Inglis
a4549a2e26 Put the statistics tasks into a new queue. This currently has no worker in PaaS so blocks the deployment until new PaaS worker is configured. 2017-05-09 22:03:13 +01:00
Martyn Inglis
f2a47044a4 Add a wrapper method around calling the tasks to manage when to call stats tasks 2017-05-09 18:17:26 +01:00
Martyn Inglis
b4bf332ea0 Added in tasks for created initial job stats and for updating following an outcome. 2017-05-09 15:24:05 +01:00
Imdad Ahad
c4fac1d937 Revert "Revert "add DVLA organisations to API"" 2017-04-21 16:05:07 +01:00
Martyn Inglis
5550201939 Upper case only in the random notification id generator 
as DVLA need upper case only or the barcode doesn't generate.
 2017-04-20 10:55:53 +01:00