Commit Graph
1384 Commits
Author SHA1 Message Date
Rebecca Law dae29a1b61 Update the query for the notifications activity page to return the data for the days of retention if set. 2018-07-23 09:56:04 +01:00
Rebecca Law 9674005c71 Updated the function to delete notifications over a week old to look at ServiceDataRetention to only delete notifications specified number of days for the notification type. 2018-07-23 09:56:03 +01:00
Pea Tyczynska 812f4d20dd Send complaints on to service callback APIs using an async task 2018-07-19 16:59:39 +01:00
Katie Smith b1cfa8942a Add one_off filter when getting all notifications for a service
Added the option to filter by one_off messages to the DAO function
`get_notifications_for_service`. Previously, one-off notifications
were not returned - this has changed so that the default is for
one-off notifications to be returned. Also simplified the `include_jobs`
filter for this function.

The DAO function gets used in 3 places - for the V1 and V2 API endpoints,
which will now start to return one-off messages. It also gets used by
the admin app which needs to pass in `include_one_off=False` to the
`get_all_notifications_for_service` where we don't want one-off
notifications to show, such as the API message log page.
2018-07-18 15:08:06 +01:00
Pea Tyczynska 86978c225a Filter 'get_service_callback_api_for_service' to only get status updates
Also rename it to 'get_service_delivery_status_callback_api_for_service'
2018-07-18 11:36:39 +01:00
Rebecca Law 9c99e45008 Merge pull request #1945 from alphagov/flexible-data-retention
Flexible data retention
2018-07-17 16:03:18 +01:00
Pea (Malgorzata Tyczynska) 48d1e816ef Merge pull request #1943 from alphagov/admin_platform_users_search
Add data endpoint for finding users by full or partial email
2018-07-16 15:47:23 +01:00
Pea Tyczynska a69dee5e6d Move code that escapes special chars to helper function and use it
in query get_users_by_partial_email
2018-07-13 15:47:21 +01:00
Pea Tyczynska 782a8ab9e7 Add data endpoint for finding users by full or partial email 2018-07-13 15:47:20 +01:00
Rebecca Law ab695d24d5 Added new endpoints to get service data retention for a service. 2018-07-13 15:18:27 +01:00
Rebecca Law e2a1dfeb31 New dao and endpoints to create and update service data retention. 2018-07-11 17:02:49 +01:00
Leo Hemsted d88003d143 Merge pull request #1940 from alphagov/dashboard-day-limit
Sort out the dashboard day limits
2018-07-11 16:07:31 +01:00
Katie Smith 0cb9e335b9 Delete old platform-stats route
We no longer need the `/platform-stats` route in the service blueprint,
because admin is using the new `/platform-stats` route in the platform stats
blueprint instead.
2018-07-10 14:59:24 +01:00
Katie Smith c57f33a02d Use real letter_rates in tests
Added the letter_rate table to the list of tables which does not get
deleted after each test run and changed the tests to use the real letter
rates.

Also removed the letter rate DAO since this was only being used in
tests, so was no longer needed.
2018-07-10 11:31:18 +01:00
Leo Hemsted 7fda32b7cf dashboard functions should return data for 8 days, not 7
really, it'll be somewhere btween 7 and 8 depending on what time of day
you request it at. But if today is monday, then seven days ago is last
tuesday - but we should return data for last monday as well so that
users see a full week's worth of data

also update/clarify the tests to make sure this is being honored for
all the different widgets on the dashboard
2018-07-09 12:03:53 +01:00
Leo Hemsted 6705f9d209 Merge pull request #1933 from alphagov/delete-service-stuff
Delete service apis (callback + inbound)
2018-07-06 15:19:01 +01:00
Leo Hemsted 32415b3b14 add delete functions for inbound and callback api objects
Both service api tasks work fine if the object is unexpectedly deleted
halfway through - they both check to see if the api details are still
in the DB before trying to send the request.
2018-07-05 11:09:17 +01:00
Katie Smith 7015e240e0 Merge pull request #1929 from alphagov/paginate-complaints
Paginate complaints
2018-07-04 15:34:46 +01:00
Leo Hemsted 1589bb58bb make sure test keys are filtered out of stats 2018-07-03 14:58:32 +01:00
Leo Hemsted 089056ef72 add tests for new statistics and fact notification status functions 2018-07-03 14:58:32 +01:00
Leo Hemsted 75fdb0290f remove old monthly stats function 2018-07-03 14:57:19 +01:00
Leo Hemsted 2463cd8fb5 move monthly notification stats from notification history to ft tables
it now uses the ft_notification_status table - nice and quick. if you
ask for the current tax year it'll top up the data with numbers from
the notification table.

the data is in exactly the same shape as the old endpoint - no changes
to the admin app are necessary
2018-07-03 14:56:31 +01:00
Katie Smith 3398e4cf93 Paginate the function to get all complaints
The DAO function to get all complaints has been paginated and the
endpoint which uses it has been updated to take a page number as a query
parameter.
2018-07-03 11:11:49 +01:00
Katie Smith 4228b77ebb Fix aggregate stats dao function to use start_date
Now using start_date to determine which table to query.
2018-07-03 09:26:02 +01:00
Rebecca Law db5663e429 Improve the performance of the query to get the last template usage.
By adding the service id, the query performance has improved greatly. It went from 6200ms to 0.04ms.
This should stop the 500s when a template is deleted.
2018-07-02 17:17:38 +01:00
Katie Smith cbb9f71e61 Fix bug with test notifications not showing
Test notifications are only stored in the notification table, not the
notification_history table. We were deciding which table to query for
the platform admin stats based on the start date passed to the DAO
function. This means that if the start date given was more than 7 days
ago and the end date was within the last 7 days, test notifications were
not being returned. We now use the end date to choose which table to
query which means that test notifications always get returned when they
should.
2018-06-29 16:01:45 +01:00
Katie Smith 4b030b1583 Create platform-stats blueprint
Created a platform-stats blueprint and moved the new platform stats
endpoint to the new blueprint (it was previously in the service
blueprint). Since the original platform stats route and the new platform
stats route are now in different blueprints, their view functions can
have the same name without any issues.
2018-06-29 16:01:45 +01:00
Katie Smith 5381491aae Move DAO function for new platform stats page
Moved the `fetch_new_aggregate_stats_by_date_range_for_all_services`
DAO function from the services DAO to the Notifications DAO since this
function queries the `notification` and `notification_history` tables.

Also added a test to check that the data returned from the function
takes BST into account.
2018-06-29 16:01:45 +01:00
Katie Smith 7e6076a41f Add in new endpoint with data for platform admin stats
Added in a new endpoint and DAO function to provide the data for the new
platform admin statistics page. The DAO method gets different data from
the Notifications / NotificationHistory table and also groups it differently.

The old endpoint has not been deleted yet to allow the numbers on the
old and new pages to be compared.
2018-06-29 16:01:45 +01:00
Rebecca Law 28beeebbf4 Added a query and endpoint to return the count of complaints for a date-range.
Unit tests for complaints_rest.get_complaint_count have yet to be written.
Maybe there is a better name for the new endpoint.
2018-06-27 09:11:38 +01:00
Rebecca Law e638653f11 When we get complaints we'd like to know how many we get in a day or other date range, so if there is a spike in complaints we can act on it.
Add new endpoint to return the number of complaints in a date range.

Unit tests to follow in the next commit.
2018-06-27 09:11:38 +01:00
Rebecca Law 1ab4654396 Fix codestyle 2018-06-20 16:54:12 +01:00
Rebecca Law 709a6c38c7 Created a task to update ft_notification_status for the last three days. 2018-06-20 16:45:20 +01:00
Chris Hill-Scott 27faf7912b Search for emails or texts, depending on term
The caseworking view is going to have a page which displays emails and
text messages combined together.

In order for the search to work on this page the user needs to be able
to search for an email or a text message. This commit makes it guess
what to search for when the `notification_type` isn’t known (basically
by saying ‘if the search term is only digits they’re probably looking
searching by phone number’).
2018-06-14 10:32:04 +01:00
Rebecca Law 7fa42c2cc5 When SES callback is for a complaint save that to the new complaints table.
When handling the complaint we don't want to throw an exception if the message is missing fields. Only log an exception if we are unable to tie a complaint to a notification.
2018-06-04 17:29:58 +01:00
Katie Smith c6c118fea1 Update created_at column of ft_billing to be non-nullable
`created_at` was added previously and made nullable temporarily. This
commit now populates the column, ensures that it will always have a
value, and makes `created_at` non-nullable.
2018-05-23 11:09:13 +01:00
Rebecca Law b405b131f6 Merge pull request #1886 from alphagov/ft_billing-migration-fixes
Fix how the letter rates are selected
2018-05-23 11:07:59 +01:00
Rebecca Law 1567e9bfe5 In order to correctly select the rate for letters you need the billable_unit or page count to use to match on the letter_rates.
A new column is added to the query to populate ft_billing which is only populated for letter.
2018-05-22 17:35:21 +01:00
Katie Smith 0d4fb81235 Delete ProviderStatistics model
This is not being used anywhere, so can be deleted safely.
2018-05-21 15:03:39 +01:00
Katie Smith 71796311fa Delete unused aggregate_statistics endpoint
`@service_blueprint.route('/<uuid:service_id>/fragment/aggregate_statistics')`
is not being used anywhere, so has been removed. The `provider_statistics_dao`
can also be removed, since the function this contained was only used for
the endpoint.
2018-05-21 15:03:39 +01:00
Rebecca Law e5584348ef Add rate to the primary key of ft_billing.
Use the primary key constraint in the upsert statement (easier to read than listing all the columns)
2018-05-21 14:38:25 +01:00
Rebecca Law c2b888a6af Make sure the date does not include time 2018-05-21 11:51:06 +01:00
Rebecca Law 4d8255b8d9 Merge branch 'master' into fix-ft-billing-migration 2018-05-21 11:06:31 +01:00
Rebecca Law 9fad623d91 Update the ft_billing query to only include billable notification status types.
Update test_provider_statistics dao - this is really irrelevant since the endpoint using the query is not being used. We have a PR coming to delete the unused code.
Update rate_multiplier to always be an integer
2018-05-21 10:56:16 +01:00
Rebecca Law 3c90c0ffcd Fix code style 2018-05-16 13:18:36 +01:00
Rebecca Law eb083e30ed - Only rebuild current month for monthly_billing if today is in the current year.
- Change the usage queries to a union so that billing_units is correct for all notification types. Removing the business logic from the schema.
- Added tests for different fragment counts, rates and sheet counts.
2018-05-16 12:21:59 +01:00
Rebecca Law 271ce6d76e Changed the update/insert to a postgres upsert to avoid concurrency issues. 2018-05-15 11:21:10 +01:00
Rebecca Law 3615f3d00f In order to re-run the create_nightly_billing for dates in the past, we added a condition on which table is used.
This will allow us to re-run nightly billing for those 2 days where it failed.
For the majority of time the query will run on Notiifcations.
2018-05-14 16:21:16 +01:00
Rebecca Law a640605a7c Merge pull request #1867 from alphagov/use-ft-billing-for-usage
Update ft_billing table to use integer data types
2018-05-14 13:04:52 +01:00
Chris Waszczuk 0e9fca09fb Merge pull request #1855 from alphagov/inbound-sms-max-7-days
Only get inbound messages that are a maximum of 7 days old
2018-05-14 11:15:52 +01:00