Commit Graph

3005 Commits

Author SHA1 Message Date
Katie Smith
4228b77ebb Fix aggregate stats dao function to use start_date
Now using start_date to determine which table to query.
2018-07-03 09:26:02 +01:00
Rebecca Law
db5663e429 Improve the performance of the query to get the last template usage.
By adding the service id, the query performance has improved greatly. It went from 6200ms to 0.04ms.
This should stop the 500s when a template is deleted.
2018-07-02 17:17:38 +01:00
Katie Smith
3bc9c06c3c Merge pull request #1919 from alphagov/new-platform-stats-endpoint
New endpoint for the updated platform admin stats page
2018-07-02 15:37:24 +01:00
Leo Hemsted
cb7f5a7166 fix letters being put in the wrong bucket when near midnight cut-off
another day, another timezone bug
2018-07-02 14:16:40 +01:00
Rebecca Law
0ede5d7ce9 Merge branch 'master' into replay-letters-in-error 2018-07-02 10:56:07 +01:00
Katie Smith
b4654781be Add schema and validate platform stats requests
Added a schema to check that the start_date and end_date arguments are
both in the date format.
2018-06-29 16:14:48 +01:00
Katie Smith
cbb9f71e61 Fix bug with test notifications not showing
Test notifications are only stored in the notification table, not the
notification_history table. We were deciding which table to query for
the platform admin stats based on the start date passed to the DAO
function. This means that if the start date given was more than 7 days
ago and the end date was within the last 7 days, test notifications were
not being returned. We now use the end date to choose which table to
query which means that test notifications always get returned when they
should.
2018-06-29 16:01:45 +01:00
Katie Smith
4b030b1583 Create platform-stats blueprint
Created a platform-stats blueprint and moved the new platform stats
endpoint to the new blueprint (it was previously in the service
blueprint). Since the original platform stats route and the new platform
stats route are now in different blueprints, their view functions can
have the same name without any issues.
2018-06-29 16:01:45 +01:00
Katie Smith
5381491aae Move DAO function for new platform stats page
Moved the `fetch_new_aggregate_stats_by_date_range_for_all_services`
DAO function from the services DAO to the Notifications DAO since this
function queries the `notification` and `notification_history` tables.

Also added a test to check that the data returned from the function
takes BST into account.
2018-06-29 16:01:45 +01:00
Katie Smith
7e6076a41f Add in new endpoint with data for platform admin stats
Added in a new endpoint and DAO function to provide the data for the new
platform admin statistics page. The DAO method gets different data from
the Notifications / NotificationHistory table and also groups it differently.

The old endpoint has not been deleted yet to allow the numbers on the
old and new pages to be compared.
2018-06-29 16:01:45 +01:00
Rebecca Law
f4846c88c9 Had to move the import to the method rather than the top of the file. 2018-06-29 11:12:54 +01:00
Rebecca Law
b201711705 Undo the chnages to create_invited_user 2018-06-29 10:26:14 +01:00
Rebecca Law
213e32cf6d Added a one off command to add many invited user to a service.
It would be nice to add something that can do this from the Admin app.
2018-06-28 17:09:45 +01:00
Rebecca Law
5850663ca6 Added a method to reply letters that are in the error folder on S3.
You can replay one at a time by sending in the file name. Or all the messages in the error folder.
2018-06-27 16:40:30 +01:00
Katie Smith
8c22a6afda Change schema format name of datetime format
Renamed the 'datetime' format to 'datetime_within_next_day'. This format
is used to validate the date and time of scheduled notifications, not to
check the format of a datetime.
2018-06-27 09:11:38 +01:00
Katie Smith
7f4b828aff Add tests for get_complaint_count endpoint
* Added unit tests for the get_complaint_count endpoint
* Updated the schema to use 'date' format instead of 'datetime'
* Updated the complaint endpoint to convert start_date and end_date to
be dates instead of strings
2018-06-27 09:11:38 +01:00
Rebecca Law
28beeebbf4 Added a query and endpoint to return the count of complaints for a date-range.
Unit tests for complaints_rest.get_complaint_count have yet to be written.
Maybe there is a better name for the new endpoint.
2018-06-27 09:11:38 +01:00
Rebecca Law
e638653f11 When we get complaints we'd like to know how many we get in a day or other date range, so if there is a spike in complaints we can act on it.
Add new endpoint to return the number of complaints in a date range.

Unit tests to follow in the next commit.
2018-06-27 09:11:38 +01:00
Rebecca Law
6d5eff028e Format the data for the csv. 2018-06-21 17:04:49 +01:00
Rebecca Law
1ab4654396 Fix codestyle 2018-06-20 16:54:12 +01:00
Rebecca Law
1363244a2b Added the scheduling for the task 2018-06-20 16:48:03 +01:00
Rebecca Law
709a6c38c7 Created a task to update ft_notification_status for the last three days. 2018-06-20 16:45:20 +01:00
Rebecca Law
16bb62e029 Due to a bug in 2016, there are some rows without a template id.
For that case use a default template id.
2018-06-20 14:51:24 +01:00
Rebecca Law
15e81aa7b3 Update to the command to migrate data for ft_notification_status.
Removed the initial queries - I don't think they are helpful.
2018-06-20 14:04:03 +01:00
Chris Hill-Scott
7ac1ca517c Merge pull request #1905 from alphagov/smart-phone-search
Search for emails or texts, depending on term
2018-06-14 12:37:02 +01:00
Chris Hill-Scott
27faf7912b Search for emails or texts, depending on term
The caseworking view is going to have a page which displays emails and
text messages combined together.

In order for the search to work on this page the user needs to be able
to search for an email or a text message. This commit makes it guess
what to search for when the `notification_type` isn’t known (basically
by saying ‘if the search term is only digits they’re probably looking
searching by phone number’).
2018-06-14 10:32:04 +01:00
Chris Hill-Scott
234c5c705a Add ‘caseworking’ service permission
We want to add the option for a user to have a more restricted view of
Notify.

This should be opt in/out on a per service basis, because it will be
appropriate only for a minority of teams. It shouldn’t be the default
because most teams (those without a lot of users or those only using the
API) won’t need it.

So this commit adds it as a new service permission, which means we can
roll it out:
- in prototype form initially
- then as an opt-in service setting
2018-06-11 13:23:06 +01:00
Rebecca Law
2381850d05 Refactor - extract common code into a method. 2018-06-07 14:54:40 +01:00
Rebecca Law
56c6d5101c Add some useful information to the log.
Fix unit test to actually test the right thing
2018-06-07 12:30:04 +01:00
Rebecca Law
e353a058e0 We'd like to see what the bounce message is from SES.
If there is a bounce we update the email to failed.
However, there is more than one reason for the failed message. Adding this logging will give us more details about the failure message.
2018-06-07 10:36:04 +01:00
Rebecca Law
4966da8289 Fix to handle_complaints.
The json we were getting from SES was not quite as expected, the test data now reflects what we get.
New test added, fix a test that was passing regardless.
2018-06-06 10:37:31 +01:00
Rebecca Law
c5524a3fe9 [WIP] 2018-06-05 17:23:24 +01:00
Rebecca Law
64f077f2f4 New endpoint to return data for complaints. 2018-06-05 14:25:24 +01:00
Rebecca Law
2bc6315408 Merge pull request #1896 from alphagov/handle-ses-complaints
Save complaints from SES
2018-06-05 11:05:24 +01:00
Rebecca Law
7fa42c2cc5 When SES callback is for a complaint save that to the new complaints table.
When handling the complaint we don't want to throw an exception if the message is missing fields. Only log an exception if we are unable to tie a complaint to a notification.
2018-06-04 17:29:58 +01:00
Richard Chapman
32b176ccc3 Merge pull request #1862 from alphagov/pyup-update-monotonic-1.4-to-1.5
Update monotonic to 1.5
2018-06-01 14:07:24 +01:00
Richard Chapman
aaba157587 Added Service contact_link column
This is going to be used for for the document download citizen landing
page, a service will add a contact link
e.g. https://customerservicecontactnumber.uk/dwp/ which will allow the
user to contact the sending department if there is an error or any
issues with the download.

* Added the contact link to the model
* Added db migration script to add the column to the database
2018-05-31 16:03:12 +01:00
Rebecca Law
1faba916b2 New complaints model to persist complaints from SES.
If a someone gets an email from one of our services and then complain about it (mark as spam or otherwise), we get a callback from SES.
The service needs to know about these complaints so they can remove that email from their mailing list.
2018-05-31 14:43:49 +01:00
Rebecca Law
23e6b57c26 Remove email from the log message 2018-05-30 16:45:18 +01:00
Rebecca Law
621c81a9d8 Small refactor to reduce complexity and satisfy codestyle.
This module could do with a look to check if we can simplify it. But at the moment we just want to record the complaint.
2018-05-30 16:25:49 +01:00
Rebecca Law
e82fa2c8d4 Added a way to handle complaint responses from SES.
At this point we are just logging the message so that we can confirm the contents of the SES message.

refer to: https://www.pivotaltracker.com/story/show/157969699
2018-05-30 16:16:36 +01:00
Rebecca Law
9fcfcd5127 Merge pull request #1893 from alphagov/fix-date-issue-for-letters
Fix BST bug
2018-05-30 11:21:11 +01:00
Rebecca Law
41f3293cd5 This fixes a bug where the folder name was not correctly returning the right date because date used to compare was in UTC but the datetime used for the job start time is in BST.
Units tests have been added for the affected method.
2018-05-30 10:18:37 +01:00
Katie Smith
4d27972c90 Add created_at and updated_at columns to ft_notification_status
Added created_at and updated_at to the ft_notification_status table in order
to make it easier to track down any potential issues with the data.

Also updated the command to populate the data to take created_at and
updated_at into account and to simplify the command. This can all be done in
the same commit since the table is not being used anywhere yet and can
only be populated manually.
2018-05-24 10:51:38 +01:00
Katie Smith
c6c118fea1 Update created_at column of ft_billing to be non-nullable
`created_at` was added previously and made nullable temporarily. This
commit now populates the column, ensures that it will always have a
value, and makes `created_at` non-nullable.
2018-05-23 11:09:13 +01:00
Rebecca Law
b405b131f6 Merge pull request #1886 from alphagov/ft_billing-migration-fixes
Fix how the letter rates are selected
2018-05-23 11:07:59 +01:00
Leo Hemsted
9b7cd15586 Merge branch 'master' into pyup-update-monotonic-1.4-to-1.5 2018-05-23 09:59:00 +01:00
Rebecca Law
1567e9bfe5 In order to correctly select the rate for letters you need the billable_unit or page count to use to match on the letter_rates.
A new column is added to the query to populate ft_billing which is only populated for letter.
2018-05-22 17:35:21 +01:00
Katie Smith
f6039d51e0 Add timestamp columns to ft_billing table
Added `created_at` and `updated_at` to the `ft_billing` table - having these
columns makes it easier to track down issues with the data in this table.

`created_at` is nullable initially, but will be changed to non-nullable
once the column is populated and the DAO etc. have been updated.
2018-05-22 14:32:49 +01:00
Katie Smith
0d4fb81235 Delete ProviderStatistics model
This is not being used anywhere, so can be deleted safely.
2018-05-21 15:03:39 +01:00