Commit Graph

432 Commits

Author SHA1 Message Date
Rebecca Law
3a3161ecc4 Merge branch 'master' into schedule-api-notification
Conflicts:
	app/celery/scheduled_tasks.py
	app/v2/notifications/post_notifications.py
	tests/app/celery/test_scheduled_tasks.py
2017-05-22 14:05:57 +01:00
Martyn Inglis
8dc7a86148 Merge branch 'master' into async-job-stats
Conflicts:
	app/dao/services_dao.py
2017-05-18 09:34:54 +01:00
Ken Tsang
54446d5f4d Add default permissions when creating a service 2017-05-17 14:18:12 +01:00
minglis
05a179c6ef Merge pull request #941 from alphagov/load-service-on-auth
Load service on auth
2017-05-17 10:49:00 +01:00
Martyn Inglis
83a1b1526e Merge branch 'master' into async-job-stats
Conflicts:
	app/notifications/notifications_ses_callback.py
2017-05-17 09:49:43 +01:00
kentsanggds
c4964d8cf4 Merge pull request #956 from alphagov/add-service-permissions
Add service permissions DAO and refactor user service permission mock
2017-05-16 14:26:15 +01:00
Rebecca Law
2e078f9fc8 Add scheduled task to send scheduled notifcations.
Fix the query to fetch scheduled notifications.
2017-05-16 13:47:22 +01:00
Martyn Inglis
a5dae0bebd Fixed test - I had deliberately failed the test as part of debugging and not fixed the assert. Doh. 2017-05-16 12:49:20 +01:00
Ken Tsang
733c16b2bb Update to strip down DAO and clarify tests 2017-05-16 12:33:27 +01:00
Rebecca Law
56f657de9b fix style 2017-05-16 11:04:55 +01:00
Ken Tsang
54d801979c Refactored to handle single service permission 2017-05-16 10:57:57 +01:00
Rebecca Law
3c0da9c8fd Merge branch 'master' into schedule-api-notification 2017-05-16 10:50:53 +01:00
Rebecca Law
579227dfc1 Method to return scheduled notifications that are ready to send 2017-05-16 10:48:04 +01:00
Ken Tsang
114d4d84d4 Add service permissions DAO and refactor user service permission mock 2017-05-15 17:28:14 +01:00
Rebecca Law
f0e2713bef Add scheduled_for in the post notification request form.
Return scheduled for in get_notification requests.
2017-05-15 17:27:38 +01:00
Imdad Ahad
7a10a91262 Revert "Process SNS request triggered by a DVLA S3 update" 2017-05-12 17:21:07 +01:00
Martyn Inglis
fd542024a3 Merge branch 'master' into load-service-on-auth 2017-05-12 16:14:44 +01:00
Imdad Ahad
f766f90207 Add task to process a DVLA response file:
* Currently we do nothing with the parsed response. We will
* update the status of the notifications in a separate PR
2017-05-12 14:24:26 +01:00
Martyn Inglis
889c650724 Whitespace 2017-05-12 13:09:59 +01:00
Martyn Inglis
4751673f43 Fixed service test that deletes all the things to delete the job stats too. 2017-05-12 13:08:09 +01:00
Martyn Inglis
f84694fb29 updated the timeout query to base outcome on notifications
Previous:
assumed discrepancy in stats counts to be related to timeouts

Now:
If discrepancy exists do the math on the notifications for that job to work out counts based on statuses to redo stats.
2017-05-12 12:19:27 +01:00
Martyn Inglis
8d4ccc3003 Updated jobs_Dao to make a job stats row when making a job - saves the ambiguity later as to whether the row exists. 2017-05-12 12:17:34 +01:00
Martyn Inglis
d324d72926 Merge branch 'master' into async-job-stats 2017-05-11 15:41:20 +01:00
Imdad Ahad
048861b968 Add dao to get active users for service 2017-05-11 15:39:15 +01:00
Martyn Inglis
b519321798 Adds a query to timeout the job counts.
After three days we timeout the notifications that we have not received a receipt for.

In the same way we bump the failed count to match the job count if there is a descepenecy. We do this after the same period that we do for notifications.
2017-05-11 15:22:43 +01:00
Martyn Inglis
22a47106af Refactored the DAO to be clearer, and wrote tests for the bug whereby different types of inserts/updates caused others to reset. 2017-05-11 12:09:57 +01:00
Leo Hemsted
348183d93b Merge pull request #942 from alphagov/status-as-table
notification status as table
2017-05-10 13:54:49 +01:00
Martyn Inglis
70bc468da0 Ensure date set on creation 2017-05-09 18:16:44 +01:00
Martyn Inglis
4faafe01d9 Tightened the error checking.
- Looks now for a specific Integrity error, not a generic SQLAlchemy  error when checking for an insert failure
- If the update fails post an insert error, then there is an issue so raise an exception so tasks can be retried.
2017-05-09 14:41:29 +01:00
Martyn Inglis
15065c4bc7 Added DAO methods to add and update the stats table for JOBS
- create_or_update_job_sending_statistics
This will try and update an existing row. if this fails as it hasn't been created, then it will insert a row. If this fails as another process has got there first then it should try and update again.
This is a code version of upset

- update_job_stats_outcome_count
Will update the outcome states. Uses the NOTIFICATION_STATUS_TYPES_FAILED to determine if the notification failed.
Any thing in DELIVERED will be marked as delivered. Statues not in the FAILED array or delivered provoke no update to the table.
2017-05-09 14:06:27 +01:00
Martyn Inglis
b669c4930a Building the methods around async stats work 2017-05-09 12:30:31 +01:00
Martyn Inglis
064b7436db Extra call to explicitly load API keys on service load - avoids extra calls later. 2017-05-05 15:22:21 +01:00
Rebecca Law
d252dc8976 New endpoint to search by "to" field of the notification.
The query ignores case and spaces.
2017-05-05 14:12:50 +01:00
Leo Hemsted
7e52fa4d13 add new notification_status column to models.py
We now have a new column in the database, but it isn't being
populated. The first step is to make sure we update this column,
while still keeping the old enum based column up to date as well.

A couple of changes have had to happen to support this - one irritating
thing is that if we're ever querying columns individually, including
`Notification.status`, then we'll need to give that column a label,
since under the hood it translates to `Notification._status_enum`.
Accessing status through the ORM (i.e., my_noti.status = 'sending' or
similar) will work fine.
2017-05-04 17:24:28 +01:00
Rebecca Law
93e76d2362 Update the valid_from date for the rate that is intended to start at the begining of the financial year.
It was the start of the financial year in BST, needed to convert it to UTC.
Small change to the logic to find the rates.
2017-05-03 17:11:48 +01:00
Ken Tsang
a1a8397f74 Update billing data to handle future dates 2017-05-02 19:23:57 +01:00
Rebecca Law
3e0221adec Change get_financial_year to return ending date as 1 microsecond earlier.
That way we can write the queries as between start and end dates, making it easier to read.
This makes more sense.
2017-05-02 10:00:47 +01:00
Rebecca Law
88d92d6070 Fix the logic gettting the rates for a financial year.
The is_between_end_date_exclusive is a bit funny.
Perhaps the better way to handle it is to make the function is_between but change the financial year function return an enddate that is one millisecond less. That way we can always use the between logic and it will be easier to use.
2017-04-28 16:55:41 +01:00
Rebecca Law
1a64509186 Change the resultset from the yearly and monthly billing data queries.
Fix some formatting of the return objects.
2017-04-28 10:10:49 +01:00
Rebecca Law
fdbadf967e Fix the email billing data when there is no results. 2017-04-28 10:10:48 +01:00
Rebecca Law
a186fc95be Added new endpoints to return the yearly-usage and monthly-usage for a given financial year and service id.
Since the response has changed I have created new endpoints so that the deployments for Admin are more managable.

Removed print statements from some tests.
2017-04-28 10:10:48 +01:00
Rebecca Law
e1e55edd9c Add new fields to the usage queries: rate_multiplier, international, phone_prefix. 2017-04-28 10:10:48 +01:00
Rebecca Law
4c37c8bdbb New query to get billing data per month. 2017-04-28 10:10:48 +01:00
Rebecca Law
6dc336ad6c Created new queries to return the rate with the sum of billable units for the year totals.
Once we have the new columns in notifications table, the query will need to include the rate multiplier and if the number is international.
The monthly billing query will be built next.
2017-04-28 10:10:48 +01:00
Imdad Ahad
c5bd685cef Don't update sent notifications (dao) 2017-04-27 16:55:39 +01:00
Martyn Inglis
349fb3529e Handle case for international SMS - use correct phone validator, and also set status correctly.
This relies on some other code so this commit has placeholder failing tests to be populated when other PRs are merged.
2017-04-27 16:27:14 +01:00
Martyn Inglis
ba58c55c3b Filter provider details by international flags 2017-04-27 12:13:58 +01:00
Rebecca Law
e8a7762688 Merge branch 'master' into update-sms-rate 2017-04-25 10:15:54 +01:00
Rebecca Law
6c8ff80440 Rename class to be singular 2017-04-25 09:53:43 +01:00
Rebecca Law
43226dc932 Added a new rates table.
We only need one rate per channel. This reflects that. The provider_rates has been left for now, it is still not being used.
New dao has been added to select the right rate for the given notification_type and date of notificaiton.
2017-04-24 16:20:03 +01:00