Commit Graph

380 Commits

Author SHA1 Message Date
Chris Hill-Scott
bdc935a8b1 Exclude test keys when searching by recipient
The activity page doesn’t show notifications sent with a test key.

However it _does_ when you search by recipient. This is confusing
and inconsistent.
2017-08-21 17:38:21 +01:00
Leo Hemsted
c2152f9cb8 remove billable-units endpoint
it wasn't used anywhere, and the return value changed when flask was upgraded
2017-08-21 10:57:15 +01:00
Imdad Ahad
f5e38a896c Update the last template usage query to check Notification table:
* Don't check the NotificationHistory table (this can cause a timeout)
* Check template actually exists first
2017-08-07 10:19:21 +01:00
Leo Hemsted
13917c9c57 give test letter api notifications a different filename
so they can be distinguished on the frontend.

Also, some related cleanup:

* don't show test api letters on the frontpage
* make sure the subject is returned from the API for letters
* make sure the letter's address is returned for letters
2017-08-01 18:23:29 +01:00
Imdad Ahad
c8c47f44a9 Don't update older status in timeout notifications task 2017-07-10 14:09:30 +01:00
Imdad Ahad
f0ffbad77d Fix syncing issue with Notification statuses:
The timeout_notifications job runs daily and does not correctly
update both the status fields to keep them in sync.
2017-07-06 12:30:08 +01:00
Martyn Inglis
e3644bae96 Merge branch 'master' into template-query-to-not-limit-by-days 2017-06-16 14:06:56 +01:00
Martyn Inglis
1bc8073a83 Tried to make this clearer 2017-06-08 15:11:41 +01:00
Martyn Inglis
3702ebdd93 This is a bit gnarly. When querying the notifications table for template usage, we don't apply the days limit if we're trying to get exactly 7 days.
This is because 7days should be the amount of data in that table so don't restrict.

Note not all queries do this in the same way and a pivotal bug has been raised to align this. This is a bug fix as right now the numbers are out in prod.
2017-06-08 13:42:41 +01:00
Imdad Ahad
bad0ce3d86 Return personalisation in notification dump in search by to 2017-06-05 15:53:57 +01:00
Imdad Ahad
0631b6c988 Add dao to delete inbound sms after seven days 2017-06-02 16:10:27 +01:00
Chris Hill-Scott
9ada8b2753 Don’t 500 when searching with bad email address
In the future we might want to validate email addresses before
attempting to search by them. But for a first pass we can just return
no results when a user types in something that isn’t an email address
or phone number.

It definitely better than returning a 500.
2017-05-30 14:50:48 +01:00
Imdad Ahad
78c10b7d30 Search notification against normalised recipient with filter for status 2017-05-26 11:45:56 +01:00
Rebecca Law
b745e13a28 Merge branch 'master' into schedule-api-notification 2017-05-25 13:37:04 +01:00
Leo Hemsted
d23a92fbc0 Merge pull request #952 from alphagov/international-delivery-receipts
update delivery receipts for countries that return them
2017-05-25 13:13:49 +01:00
Rebecca Law
96b0e42959 Merge branch 'master' into schedule-api-notification
Conflicts:
	app/celery/scheduled_tasks.py
	tests/app/celery/test_scheduled_tasks.py
2017-05-24 13:21:22 +01:00
Martyn Inglis
aaa0f763a1 Updated delete notifications over a week old query
- PREVIOUS
based on status. so as we add new status we have some orphaned rows, as these delete queries would miss them

- NOW
based on type. In effect they do the same thing, deleting emails, sms or letters older than a week old irrespective of status. Can see is iterating on this to have more granularity say for letters, so split up. Also means that the delete action isn't so big, as we half the affected rows, by doing it by type.
2017-05-23 13:40:15 +01:00
Rebecca Law
76d0783c63 Add beat config for send-scheduled-notifications task to run every 15 minutes.
Added the missing commit to the update pending
2017-05-22 16:30:45 +01:00
Rebecca Law
9bfba52f53 Add pending flag to scheduled_notifications.
Set pending flag to false when the notification has been sent to provider task.
2017-05-22 15:07:16 +01:00
Rebecca Law
2e078f9fc8 Add scheduled task to send scheduled notifcations.
Fix the query to fetch scheduled notifications.
2017-05-16 13:47:22 +01:00
Rebecca Law
56f657de9b fix style 2017-05-16 11:04:55 +01:00
Rebecca Law
579227dfc1 Method to return scheduled notifications that are ready to send 2017-05-16 10:48:04 +01:00
Rebecca Law
f0e2713bef Add scheduled_for in the post notification request form.
Return scheduled for in get_notification requests.
2017-05-15 17:27:38 +01:00
Imdad Ahad
7a10a91262 Revert "Process SNS request triggered by a DVLA S3 update" 2017-05-12 17:21:07 +01:00
Leo Hemsted
700e3d2fa7 update delivery receipts for countries that return them
some countries don't return delivery receipts
some countries return delivery receipts when they reach the carrier

these countries, we should keep the notifications in sent (aka
sent_internatinally) for. However, for countries that have normal
delivery receipts, we should update them as we do for UK numbers
2017-05-12 15:01:55 +01:00
Imdad Ahad
f766f90207 Add task to process a DVLA response file:
* Currently we do nothing with the parsed response. We will
* update the status of the notifications in a separate PR
2017-05-12 14:24:26 +01:00
Rebecca Law
d252dc8976 New endpoint to search by "to" field of the notification.
The query ignores case and spaces.
2017-05-05 14:12:50 +01:00
Rebecca Law
3e0221adec Change get_financial_year to return ending date as 1 microsecond earlier.
That way we can write the queries as between start and end dates, making it easier to read.
This makes more sense.
2017-05-02 10:00:47 +01:00
Rebecca Law
6dc336ad6c Created new queries to return the rate with the sum of billable units for the year totals.
Once we have the new columns in notifications table, the query will need to include the rate multiplier and if the number is international.
The monthly billing query will be built next.
2017-04-28 10:10:48 +01:00
Imdad Ahad
c5bd685cef Don't update sent notifications (dao) 2017-04-27 16:55:39 +01:00
Martyn Inglis
349fb3529e Handle case for international SMS - use correct phone validator, and also set status correctly.
This relies on some other code so this commit has placeholder failing tests to be populated when other PRs are merged.
2017-04-27 16:27:14 +01:00
Leo Hemsted
935e566284 add tests for letters not getting excluded 2017-04-19 14:22:25 +01:00
Leo Hemsted
b381aeefa1 don't include letters when timing out/deleting old notifications
also refactored those functions, since they were pretty gnarly and
repeated themselves a lot
2017-04-19 11:34:00 +01:00
Leo Hemsted
c7c47a029f remove slow join from template-last-use query 2017-04-10 17:33:31 +01:00
Rebecca Law
7ba9b1d261 Update the update statements with the updated_at and sent_at timestamps. 2017-04-07 15:21:27 +01:00
Rebecca Law
2d9a449d1e Changed styling 2017-04-07 14:36:00 +01:00
Rebecca Law
61d228e797 Remove test filter 2017-04-07 11:22:03 +01:00
Rebecca Law
50a5bedcbf Refactor update to notifcations to be a bulk update. This is much better for performance. 2017-04-07 10:59:12 +01:00
Rebecca Law
d36eec652d Removed unused code in services_dao, which led to looking some of the queries.
By removing the join to services for the fetch_stats_by_date_range_for_all_services and dao_fetch_todays_stats_for_all_services
queries we get a 30% and 25% performance improvement.
2017-03-01 17:12:29 +00:00
Imdad Ahad
eafe8269ef Simplify dao method and update tests and fixtures 2017-02-24 13:39:58 +00:00
Imdad Ahad
60f63a30db Add dao method to get count of slow delivery notifications by provider 2017-02-24 12:23:39 +00:00
Rebecca Law
da07f9dde8 Performance improvement to the template usage query. 2017-02-02 09:45:06 +00:00
Chris Hill-Scott
45b505f379 Split out handling of BST queries for reuse 2017-01-31 14:16:35 +00:00
Chris Hill-Scott
56ba653f48 Add endpoint for breakdown of activity by month
This endpoint will eventualy replace the weekly breakdown one. By month
for a given financial year is better, because it gives us consistency
with the breakdown of financial usage (and eventually consistency with
the template usage).

The code to do this is a bit convoluted, in order to fill out the counts
for months and statuses where we don’t have notifications.

This will make the admin side of this easier, because we can rely on
there always being numbers available. The admin side will deal with
summing the statuses (eg `temporary-failure` > `failed`) because this
is presentational.

This commit also modifies the usage count to use `.between()` for
consistency.
2017-01-31 14:16:34 +00:00
Imdad Ahad
cc7bf45766 Move notification retrieval logic into perf platform client and dont filter by status 2017-01-27 16:39:01 +00:00
Rebecca Law
c87d0a37f3 Refactor the get_midnight functions to return the date in UTC but for the localised time.
So in June when we are in BST June 16, 00:00 BST => June 15 23:00 UTC
2017-01-27 15:57:25 +00:00
Imdad Ahad
b6cc5b00ba Add dao method to get sms and email notification counts for a date range and previous day 2017-01-27 12:22:36 +00:00
Rebecca Law
cefa00b3fa Merge branch 'master' into fix-billable-units-query
Conflicts:
	tests/app/dao/test_notification_dao.py
2017-01-10 14:24:08 +00:00
Rebecca Law
ffa54821d9 The previous commit was failing the test in the docker container.
My local version of postgres is 9.6 and docker is using 9.5 (so I am ahead).
However, we found a more simple solution that works for both versions.
2017-01-10 11:21:33 +00:00
Rebecca Law
a1d8ca9364 Use a variable for the massive month date conversion.
Document the massive date function.
Document the get_april_fools function.
2017-01-09 15:34:24 +00:00