Commit Graph

2646 Commits

Author SHA1 Message Date
Alexey Bezhan
0dcf04def9 Add notification status for returned letters
We need to update letter notifications with a new status when DVLA
gives us a list of references for returned letters.

This adds the new status to the models and the DB.

DVLA call this 'returned mail', so I'm using it as the status name
since it seems less ambiguous than 'returned'.
2018-09-04 16:55:22 +01:00
Chris Hill-Scott
d52ee4606a Add error handler for duplicate domain 2018-09-04 14:40:24 +01:00
Chris Hill-Scott
5d5047fb13 Add unique constraint to email branding domains
Does two things:

1. Revert "Revert "Add unique constraint to email branding domain""

This reverts commit af9cb30ef3.

2. Don’t allow empty string in email branding domain

Columns with multiple `null`s can have a uniqueness constraint. Columns
with multiple empty string values are not considered unique.

This commit:
- removes any duplicate empty string values
- casts empty strings to null string any time these columns are updated

---

Squashed into this single commits because these two things are not
atomic as individual commits.
2018-09-04 11:10:46 +01:00
Chris Hill-Scott
0c47d41977 Remove govuk from possible brands
‘GOV.UK’ doesn’t make sense as a type of brand. It only made sense as
a type of branding that a service had.

Since we’ve:
- deprecated the service branding column
- made sure it’s not used as a value in the email branding table

we can remove this value from the table of possible brand types.
2018-08-30 16:36:35 +01:00
Chris Hill-Scott
337496c5bc Deprecate service branding column
We want to drop this column. First we have to stop using it anywhere.

Needs to be made nullable so we can stop writing to it.
2018-08-30 16:05:37 +01:00
Leo Hemsted
889fd796fb Merge pull request #2057 from alphagov/sanitise-precompiled
Sanitise precompiled pdfs after virus scan
2018-08-29 12:53:27 +01:00
Chris Hill-Scott
38096eb93e Merge pull request #2051 from pwright08/add-create-secret-code-unit-tests
Add unit tests for the Users DAO create_secret_code function
2018-08-29 09:49:36 +01:00
Chris Hill-Scott
2671ca3e5e Look in email_branding for brand_type, not service
https://www.pivotaltracker.com/story/show/159986276

We are now setting the type of branding on the branding itself, not on
the service.

This commit switches over from looking in the old place (on the service)
to looking in the new place (on the branding).
2018-08-28 16:22:41 +01:00
Leo Hemsted
40c537a9f5 add tests 2018-08-24 15:12:02 +01:00
Rebecca Law
562facb49c Now that EmailBranding includes the brand type we do not need two separate colours. This PR removes the new colour columns. 2018-08-24 13:51:11 +01:00
Rebecca Law
3816010c72 Add brand_type to EmailBranding.
It makes more sense to put the brand_type with EmailBranding rather than in Service.
Next step is to add the new type to the form in admin app.
2018-08-23 13:53:05 +01:00
Paul Wright
267a82758a Add unit tests for the Users DAO create_secret_code function 2018-08-23 12:18:47 +01:00
Rebecca Law
fd16d7060a Added a method to decide which colour to send to the notifications-utils HTMLEmailTemplate to create the content. 2018-08-22 12:59:06 +01:00
Rebecca Law
0b0e551626 Add banner_colour, single_id_colour, and domain to email_branding_schema.
The Admin app can now be updated to send the new columns to update/create email_branding
2018-08-20 10:57:42 +01:00
Rebecca Law
1658f4de71 Merge pull request #2045 from alphagov/email-branding-colors-and-domain
New columns for EmailBranding.
2018-08-17 12:56:32 +01:00
Rebecca Law
df2271a49f New columns for EmailBranding.
There is a requirement for the colour of the banner to be different to the single id colour. In order to accomodate that we are adding two new columns for that data.
These columns will be updated manually and not done with a data migration.

A new column called domain has been added. This will be used as a default domain, if the user that creates the service has a matching domain this email brand will be set for the service.

This PR only adds the new columns to the model and db.

https://www.pivotaltracker.com/story/show/159660295
2018-08-16 16:57:53 +01:00
Katie Smith
a87be9b74a Use new value of SMS_CHAR_COUNT_LIMIT from utils
Admin, API and utils were all defining a value for SMS_CHAR_COUNT_LIMIT.
This value has been updated in notifications-utils to allow text
messages to be 4 fragments long and notifications-api now gets the value of
SMS_CHAR_COUNT_LIMIT from notifications-utils instead of defining it in
config.

Also updated some tests to check for the higher limit.
2018-08-16 16:34:34 +01:00
Alexey Bezhan
1e3004ab3a Add service stats dao tests with variable days limit 2018-08-15 11:10:16 +01:00
Alexey Bezhan
ab428048b9 Add an endpoint to fetch service data retention by type
Admin app needs to get the service data retention for the specified
notification type, so to avoid iterating through the list of all
existing service data retention settings we restore the endpoint
to get the individual data retention period.
2018-08-13 16:45:57 +01:00
Alexey Bezhan
5f2724c429 Add limit_days argument to notification statistics endpoint
Allows getting notification counts for a given number of days to
support services with custom data retention periods (admin dashboard
page should still display counts for the last 7 days, while the
notifications page displays all stored notifications).
2018-08-13 16:45:57 +01:00
Rebecca Law
3e51b69df6 Merge pull request #2037 from alphagov/fix-email-branding
This PR fixes the check for the text key in the post data.
2018-08-13 15:17:02 +01:00
Rebecca Law
f965322f25 Fixes to the delete letter notifications.
If there are no files to delete we won't get an excpetion.
Wrap the delete file in a try/except to avoid stopping the entire task.
Fix the missing slash for the file name.
2018-08-13 14:09:51 +01:00
Rebecca Law
2a7f3faec9 Merge branch 'master' into fix-delete-s3 2018-08-13 11:44:44 +01:00
Rebecca Law
eb2c878edd Fix bug with deleting the S3 file.
Removed the duplicate method.
2018-08-13 11:33:19 +01:00
Rebecca Law
69ab649256 Use the EmailBranding.text for the bannder instead of the name. 2018-08-13 10:29:06 +01:00
Rebecca Law
febc6f4e23 This PR fixes the check for the text key in the post data. 2018-08-10 17:27:58 +01:00
Rebecca Law
ebb43082d5 Deal with letters that have not been sent, this can be test letters or letters in tech-failure. 2018-08-10 16:22:25 +01:00
Rebecca Law
85a0fd3b46 Merge pull request #2031 from alphagov/fix_data_retention
Use days of retention when deleting notifications in the nightly job.
2018-08-10 15:35:42 +01:00
Leo Hemsted
62d70dbd4d use contextmanager freeze time instead of decorator
it doesn't appear to work nicely with pytest fixtures in the newer
versions of pytest
2018-08-09 16:47:35 +01:00
Rebecca Law
d0e9ab4972 If the notifications that are being deleted are letters then we need to delete the letter from s3 as well. 2018-08-08 16:20:25 +01:00
Rebecca Law
43000b2253 Add a new test 2018-08-06 15:13:08 +01:00
Rebecca Law
517608dff5 Update method to delete notifications that are a week old, to look at the days of retention set for the service.
If the service does not have the days of retention set, then use 7 days.
Added a method to get days of retention for a service and notificaiton type
2018-08-06 13:51:54 +01:00
Rebecca Law
019c6a4e3a Revert "Purge notifications for configured days of retention" 2018-08-03 14:35:36 +01:00
Rebecca Law
78f8c17fe8 Merge pull request #1965 from alphagov/check-data-retention-before-purging-notifications
Purge notifications for configured days of retention
2018-08-03 13:49:49 +01:00
Rebecca Law
c30e58eaad Instead of deleting the data in the name field just copy it to the text field.
The admin app still needs to use the name column.
Add the text field to the post data schemas.
If the text is not in the post data, then populate it with the data in the name field.
This should make the migration to text easier, and will work until we are able to update the admin app.
2018-08-02 15:55:45 +01:00
Pea Tyczynska
d8d94b431f Find email_branding by name 2018-07-31 17:58:19 +01:00
Pea Tyczynska
c5792f4d79 Migration file for email_branding fields update 2018-07-31 17:58:19 +01:00
Rebecca Law
2f120b30e8 Fix codestyle 2018-07-31 14:13:15 +01:00
Rebecca Law
ab28691a70 Add number to migration file name 2018-07-31 14:06:29 +01:00
Rebecca Law
19890412c9 Added a cancelled notification status so that we can cancel letters before they are sent. 2018-07-31 13:52:04 +01:00
Rebecca Law
67639bf49c Merge branch 'master' into check-data-retention-before-purging-notifications 2018-07-31 11:40:58 +01:00
Rebecca Law
ff2334937c Last night we had some letter notifications that were in permanent failure and had page count = 0.
This causes the create-nighlty-billing task to fail.
This will make sure that 0 is returned in this case.
2018-07-31 11:04:48 +01:00
Pea Tyczynska
ca2b350a99 Remove references to monthly_billing table from api 2018-07-30 11:07:42 +01:00
Pea Tyczynska
c0f309a2a6 Delete scheduled task to populate monthly_billing 2018-07-30 11:06:04 +01:00
Leo Hemsted
a826f6e924 make rebuild-ft-billing-for-day remove unused rows
if the billing data was incorrect and needs to be rebuilt, we should
remove old rows. Previously we were only upserting new rows, but old
no-longer-relevant rows were staying in the database. This commit
makes the flask command remove *all* rows for that service and day,
before inserting the rows from the original notification data after.

This commit doesn't change the existing nightly task, nor does it
change the upsert that happens upon viewing the usage page. In normal
usage, there should never be a case where the number of billable units
for a rate decreases. It should only ever increase, thus, never need to
be deleted
2018-07-26 19:10:30 +01:00
Rebecca Law
a812060915 The unique constraint on SeviceCallbackApi was on service_id only.
Now that we have 2 types of api callbacks the constraint to be on service_id + callback_type.
2018-07-25 14:12:13 +01:00
Rebecca Law
4fc004b00a Increase the number of days we calculate billing from 3 to 10 days.
Log exception if the billing counts for letters are different in the dvla response file than what we collected.
2018-07-24 16:28:30 +01:00
Pea (Malgorzata Tyczynska)
1b21a12b83 Merge pull request #1971 from alphagov/add_task_to_send_complaints_on
Add and call task to send complaints on to service callback APIs
2018-07-24 15:07:09 +01:00
Rebecca Law
0675f09afb Need to be able to query the notification statistics for the right number of days.
If the request is for the big numbers on the activity page, then we need to use the number right number of days.
Added an end point to get the data retention for the service and notification type, which is needed on the activity page to say how long the report is available for.
2018-07-23 09:56:04 +01:00
Rebecca Law
dae29a1b61 Update the query for the notifications activity page to return the data for the days of retention if set. 2018-07-23 09:56:04 +01:00