Commit Graph

1471 Commits

Author SHA1 Message Date
Martyn Inglis
02f7abcc48 Wire in the new task. Runs at 5am after the other tasks. 2017-05-11 15:23:45 +01:00
Martyn Inglis
f0e47029f2 "Created at" on the model 2017-05-11 15:23:18 +01:00
Martyn Inglis
8818bd5dba Scheduled task to call the timeout function 2017-05-11 15:22:57 +01:00
Martyn Inglis
b519321798 Adds a query to timeout the job counts.
After three days we timeout the notifications that we have not received a receipt for.

In the same way we bump the failed count to match the job count if there is a descepenecy. We do this after the same period that we do for notifications.
2017-05-11 15:22:43 +01:00
Martyn Inglis
76494a285e New logic to:
- increment sent only if the notifications is in a NON-finished state
- Increment outcome only if notification is in a finished state
2017-05-11 12:10:46 +01:00
Martyn Inglis
22a47106af Refactored the DAO to be clearer, and wrote tests for the bug whereby different types of inserts/updates caused others to reset. 2017-05-11 12:09:57 +01:00
Martyn Inglis
19c14982a2 Call the task wrapper outcome function in the statistics_tasks file. This wraps the logic around which tasks to creates and simplifies the logic in the Callback classes. 2017-05-09 22:03:57 +01:00
Martyn Inglis
a4549a2e26 Put the statistics tasks into a new queue. This currently has no worker in PaaS so blocks the deployment until new PaaS worker is configured. 2017-05-09 22:03:13 +01:00
Martyn Inglis
caed193647 Use the new task wrapper methods rather than creating a task directly 2017-05-09 18:17:55 +01:00
Martyn Inglis
f2a47044a4 Add a wrapper method around calling the tasks to manage when to call stats tasks 2017-05-09 18:17:26 +01:00
Martyn Inglis
70bc468da0 Ensure date set on creation 2017-05-09 18:16:44 +01:00
Martyn Inglis
b4bf332ea0 Added in tasks for created initial job stats and for updating following an outcome. 2017-05-09 15:24:05 +01:00
Martyn Inglis
2cbf8d29fd Removed white space for pep8 2017-05-09 14:46:15 +01:00
Martyn Inglis
4faafe01d9 Tightened the error checking.
- Looks now for a specific Integrity error, not a generic SQLAlchemy  error when checking for an insert failure
- If the update fails post an insert error, then there is an issue so raise an exception so tasks can be retried.
2017-05-09 14:41:29 +01:00
Martyn Inglis
15065c4bc7 Added DAO methods to add and update the stats table for JOBS
- create_or_update_job_sending_statistics
This will try and update an existing row. if this fails as it hasn't been created, then it will insert a row. If this fails as another process has got there first then it should try and update again.
This is a code version of upset

- update_job_stats_outcome_count
Will update the outcome states. Uses the NOTIFICATION_STATUS_TYPES_FAILED to determine if the notification failed.
Any thing in DELIVERED will be marked as delivered. Statues not in the FAILED array or delivered provoke no update to the table.
2017-05-09 14:06:27 +01:00
Martyn Inglis
a5caed5ac8 Changed job_stats table to have an update at time stamp 2017-05-09 14:04:19 +01:00
Martyn Inglis
b669c4930a Building the methods around async stats work 2017-05-09 12:30:31 +01:00
Martyn Inglis
d647640b65 Added job statistics table 2017-05-09 11:22:05 +01:00
Rebecca Law
a564b9aeb0 Update /service/{}/notifications to look for a "to" query string, if it exists call the search notifications_by_to_field method. 2017-05-08 17:20:21 +01:00
Rebecca Law
d252dc8976 New endpoint to search by "to" field of the notification.
The query ignores case and spaces.
2017-05-05 14:12:50 +01:00
Rebecca Law
93e76d2362 Update the valid_from date for the rate that is intended to start at the begining of the financial year.
It was the start of the financial year in BST, needed to convert it to UTC.
Small change to the logic to find the rates.
2017-05-03 17:11:48 +01:00
Martyn Inglis
97dcf766bc Switch on API rate limiting on production.
Limits POST API calls to send notifications on V1/V2 to 3000 in 60 seconds.
2017-05-03 12:13:32 +01:00
Ken Tsang
a1a8397f74 Update billing data to handle future dates 2017-05-02 19:23:57 +01:00
Sakis
01e3a88d55 Merge pull request #920 from alphagov/autoconfirm_http_subscriptions
Autoconfirm http subscriptions
2017-05-02 17:03:41 +01:00
minglis
2dcebc0b52 Merge pull request #917 from alphagov/rate-limit-api-calls
Rate limit api calls
2017-05-02 15:07:54 +01:00
Martyn Inglis
b0baa1532a Added some logging around rate limiting. 2017-05-02 11:14:45 +01:00
Martyn Inglis
a9539d892c Merge branch 'master' into rate-limit-api-calls
Conflicts:
	requirements.txt
	tests/app/notifications/rest/test_send_notification.py
	tests/app/notifications/test_validators.py
	tests/app/v2/notifications/test_post_notifications.py
2017-05-02 10:56:56 +01:00
Rebecca Law
3e0221adec Change get_financial_year to return ending date as 1 microsecond earlier.
That way we can write the queries as between start and end dates, making it easier to read.
This makes more sense.
2017-05-02 10:00:47 +01:00
Rebecca Law
88d92d6070 Fix the logic gettting the rates for a financial year.
The is_between_end_date_exclusive is a bit funny.
Perhaps the better way to handle it is to make the function is_between but change the financial year function return an enddate that is one millisecond less. That way we can always use the between logic and it will be easier to use.
2017-04-28 16:55:41 +01:00
Athanasios Voutsadakis
cfed90e502 Make code a bit more defensive and add som logging 2017-04-28 16:10:41 +01:00
Athanasios Voutsadakis
74433c9335 Add tests for autoconfirmation 2017-04-28 15:11:36 +01:00
Athanasios Voutsadakis
04b003c152 Add utility function for subscription autoconfirm 2017-04-28 15:11:36 +01:00
Leo Hemsted
f577e91134 treat sent as delivered in detailed service api
this is for when we fetch the large blue numbers for viewing
notifications for an entire service
2017-04-28 11:56:12 +01:00
kentsanggds
56e5045e9a Merge pull request #928 from alphagov/update-billing-api-with-rate
Update billing api with rate
2017-04-28 11:40:05 +01:00
Leo Hemsted
fd4e27bfd3 add type conversion to report downloads
"sent" is fine as an internal marker but not very obvious to the end
user that it specifically refers to international messages. We now
say "Sent internationally" in the CSV
2017-04-28 11:00:55 +01:00
Rebecca Law
3b41478a0a Updated Notification model to use Float(asdecimal=False) for rate_mutliplier.
Added test with multiple rows for a month.
2017-04-28 10:10:49 +01:00
Rebecca Law
1a64509186 Change the resultset from the yearly and monthly billing data queries.
Fix some formatting of the return objects.
2017-04-28 10:10:49 +01:00
Rebecca Law
fdbadf967e Fix the email billing data when there is no results. 2017-04-28 10:10:48 +01:00
Rebecca Law
a186fc95be Added new endpoints to return the yearly-usage and monthly-usage for a given financial year and service id.
Since the response has changed I have created new endpoints so that the deployments for Admin are more managable.

Removed print statements from some tests.
2017-04-28 10:10:48 +01:00
Rebecca Law
e1e55edd9c Add new fields to the usage queries: rate_multiplier, international, phone_prefix. 2017-04-28 10:10:48 +01:00
Rebecca Law
4c37c8bdbb New query to get billing data per month. 2017-04-28 10:10:48 +01:00
Rebecca Law
6dc336ad6c Created new queries to return the rate with the sum of billable units for the year totals.
Once we have the new columns in notifications table, the query will need to include the rate multiplier and if the number is international.
The monthly billing query will be built next.
2017-04-28 10:10:48 +01:00
Imdad Ahad
b5d4acb758 Make message more accurate 2017-04-27 16:58:00 +01:00
Imdad Ahad
e450f15b2b Fix typo and update test now that we expect sent for international 2017-04-27 16:57:04 +01:00
Imdad Ahad
c5bd685cef Don't update sent notifications (dao) 2017-04-27 16:55:39 +01:00
Martyn Inglis
349fb3529e Handle case for international SMS - use correct phone validator, and also set status correctly.
This relies on some other code so this commit has placeholder failing tests to be populated when other PRs are merged.
2017-04-27 16:27:14 +01:00
Imdad Ahad
162fc8de04 Merge pull request #925 from alphagov/feat-use-intl-provider-send-sms
Feat use intl provider send sms
2017-04-27 15:33:45 +01:00
Imdad Ahad
164ba0c6b3 as_decimal defaults to False 2017-04-27 13:50:51 +01:00
Imdad Ahad
868e102f8a Make rate_multiplier on notification return non-decimal 2017-04-27 13:24:42 +01:00
Imdad Ahad
252c3235d7 Persist intl fields on notification 2017-04-27 13:24:36 +01:00