Commit Graph

3094 Commits

Author SHA1 Message Date
Chris Hill-Scott
0c47d41977 Remove govuk from possible brands
‘GOV.UK’ doesn’t make sense as a type of brand. It only made sense as
a type of branding that a service had.

Since we’ve:
- deprecated the service branding column
- made sure it’s not used as a value in the email branding table

we can remove this value from the table of possible brand types.
2018-08-30 16:36:35 +01:00
Chris Hill-Scott
337496c5bc Deprecate service branding column
We want to drop this column. First we have to stop using it anywhere.

Needs to be made nullable so we can stop writing to it.
2018-08-30 16:05:37 +01:00
Leo Hemsted
889fd796fb Merge pull request #2057 from alphagov/sanitise-precompiled
Sanitise precompiled pdfs after virus scan
2018-08-29 12:53:27 +01:00
Chris Hill-Scott
2671ca3e5e Look in email_branding for brand_type, not service
https://www.pivotaltracker.com/story/show/159986276

We are now setting the type of branding on the branding itself, not on
the service.

This commit switches over from looking in the old place (on the service)
to looking in the new place (on the branding).
2018-08-28 16:22:41 +01:00
Leo Hemsted
40c537a9f5 add tests 2018-08-24 15:12:02 +01:00
Rebecca Law
562facb49c Now that EmailBranding includes the brand type we do not need two separate colours. This PR removes the new colour columns. 2018-08-24 13:51:11 +01:00
Rebecca Law
44818a91e7 Merge pull request #2054 from alphagov/add-brand-type-to-email-branding
Add brand_type to EmailBranding.
2018-08-24 10:13:44 +01:00
Leo Hemsted
001f84b0b8 Sanitise PDFs after virus scan
Notify antivirus, on success, calls the process_virus_scan_passed taks.
Previously, this task would:

* update status to created (or delivered for test keys)
* copy the file from the scan bucket to either the live or test bucket
  based on the results
* delete the old file in the scan bucket

We want it to:

* download file from scan bucket
* sanitise PDF using new template-preview functionality
* if sanitise failed, set to new status "validation-failed" and save
  the pdf somewhere.
* send new pdf to live/test bucket
* update status to created (or delivered for test keys)
* delete the original file in the scan bucket

This PR does some of that:

* download file from scan bucket
* sanitise PDF using new template-preview functionality
* if sanitise failed, just log.
* send OLD pdf to live/test bucket
* update status to created (or delivered for test keys)
* delete the original file in the scan bucket

So if sanitising fails, we won't fall over and not deliver the letter,
we'll just log a message for now. If sanitise throws an unexpected
error (as opposed to a 400), we'll retry up to fifteen times (the same
as when creating a new letter). I've added the code for using the
sanitised pdf, but it's commented out for now
2018-08-23 16:16:40 +01:00
Rebecca Law
3816010c72 Add brand_type to EmailBranding.
It makes more sense to put the brand_type with EmailBranding rather than in Service.
Next step is to add the new type to the form in admin app.
2018-08-23 13:53:05 +01:00
Athanasios Voutsadakis
1e4c2ee796 Disable header check on preview
This header was introduced to ensure that no traffic was being
directed straight to the .cloudapps.digital domain. This was especially
useful for the non-production environments where access to the proper
domains is allowed for specific IP addresses while the cloudapps.digital
ones are open.

Moving to paas custom domains [1] will allow us to stop using the paas
proxies and, as a result, unbind the cloudapps.digital domain from our
apps.

This means that the X-Custom-Forwarder will become obsolete since all
our requests will be coming directly to our domain (albeit through
cloudfront) so any IP restriction can be implemented with a route
service [2].

1: https://docs.cloud.service.gov.uk/deploying_services.html#set-up-a-custom-domain-using-the-cdn-route-service
2: https://docs.cloud.service.gov.uk/deploying_services.html#route-services
2018-08-23 13:23:14 +01:00
Rebecca Law
fd16d7060a Added a method to decide which colour to send to the notifications-utils HTMLEmailTemplate to create the content. 2018-08-22 12:59:06 +01:00
Rebecca Law
0b0e551626 Add banner_colour, single_id_colour, and domain to email_branding_schema.
The Admin app can now be updated to send the new columns to update/create email_branding
2018-08-20 10:57:42 +01:00
Rebecca Law
1658f4de71 Merge pull request #2045 from alphagov/email-branding-colors-and-domain
New columns for EmailBranding.
2018-08-17 12:56:32 +01:00
Rebecca Law
df2271a49f New columns for EmailBranding.
There is a requirement for the colour of the banner to be different to the single id colour. In order to accomodate that we are adding two new columns for that data.
These columns will be updated manually and not done with a data migration.

A new column called domain has been added. This will be used as a default domain, if the user that creates the service has a matching domain this email brand will be set for the service.

This PR only adds the new columns to the model and db.

https://www.pivotaltracker.com/story/show/159660295
2018-08-16 16:57:53 +01:00
Katie Smith
a87be9b74a Use new value of SMS_CHAR_COUNT_LIMIT from utils
Admin, API and utils were all defining a value for SMS_CHAR_COUNT_LIMIT.
This value has been updated in notifications-utils to allow text
messages to be 4 fragments long and notifications-api now gets the value of
SMS_CHAR_COUNT_LIMIT from notifications-utils instead of defining it in
config.

Also updated some tests to check for the higher limit.
2018-08-16 16:34:34 +01:00
Alexey Bezhan
ab428048b9 Add an endpoint to fetch service data retention by type
Admin app needs to get the service data retention for the specified
notification type, so to avoid iterating through the list of all
existing service data retention settings we restore the endpoint
to get the individual data retention period.
2018-08-13 16:45:57 +01:00
Alexey Bezhan
5f2724c429 Add limit_days argument to notification statistics endpoint
Allows getting notification counts for a given number of days to
support services with custom data retention periods (admin dashboard
page should still display counts for the last 7 days, while the
notifications page displays all stored notifications).
2018-08-13 16:45:57 +01:00
Rebecca Law
3e51b69df6 Merge pull request #2037 from alphagov/fix-email-branding
This PR fixes the check for the text key in the post data.
2018-08-13 15:17:02 +01:00
Rebecca Law
f965322f25 Fixes to the delete letter notifications.
If there are no files to delete we won't get an excpetion.
Wrap the delete file in a try/except to avoid stopping the entire task.
Fix the missing slash for the file name.
2018-08-13 14:09:51 +01:00
Rebecca Law
2a7f3faec9 Merge branch 'master' into fix-delete-s3 2018-08-13 11:44:44 +01:00
Rebecca Law
eb2c878edd Fix bug with deleting the S3 file.
Removed the duplicate method.
2018-08-13 11:33:19 +01:00
Rebecca Law
69ab649256 Use the EmailBranding.text for the bannder instead of the name. 2018-08-13 10:29:06 +01:00
Rebecca Law
febc6f4e23 This PR fixes the check for the text key in the post data. 2018-08-10 17:27:58 +01:00
Rebecca Law
dd4b711ba4 Merge pull request #2036 from alphagov/fix-deleted-count
Fix deleted count for logs and delete with letters that have not been sent
2018-08-10 16:53:09 +01:00
Rebecca Law
ccaa1dfeb2 Merge pull request #2034 from alphagov/remove-caseworking-permission
Remove caseworking permission
2018-08-10 16:30:50 +01:00
Rebecca Law
ebb43082d5 Deal with letters that have not been sent, this can be test letters or letters in tech-failure. 2018-08-10 16:22:25 +01:00
Rebecca Law
85a0fd3b46 Merge pull request #2031 from alphagov/fix_data_retention
Use days of retention when deleting notifications in the nightly job.
2018-08-10 15:35:42 +01:00
Rebecca Law
f844a39ea6 Move the condition outside the method, remove the notification_type variable. 2018-08-10 13:11:23 +01:00
Chris Hill-Scott
e0570c1773 Remove caseworking permission
What became ‘caseworking view’ is now available to all services.
2018-08-10 09:09:18 +01:00
Leo Hemsted
1ff729e75f read first file returned from s3
rather than looping through, just use `next`. We only expect to return
one file, so this won't actually affect the code flow. If, for whatever
reason, the file isn't on s3, whereas previously it would error when
trying to return an uninstantiated variable, now it will raise a
StopIteration
2018-08-09 16:47:35 +01:00
Rebecca Law
d0e9ab4972 If the notifications that are being deleted are letters then we need to delete the letter from s3 as well. 2018-08-08 16:20:25 +01:00
Rebecca Law
517608dff5 Update method to delete notifications that are a week old, to look at the days of retention set for the service.
If the service does not have the days of retention set, then use 7 days.
Added a method to get days of retention for a service and notificaiton type
2018-08-06 13:51:54 +01:00
Rebecca Law
019c6a4e3a Revert "Purge notifications for configured days of retention" 2018-08-03 14:35:36 +01:00
Rebecca Law
78f8c17fe8 Merge pull request #1965 from alphagov/check-data-retention-before-purging-notifications
Purge notifications for configured days of retention
2018-08-03 13:49:49 +01:00
Rebecca Law
c30e58eaad Instead of deleting the data in the name field just copy it to the text field.
The admin app still needs to use the name column.
Add the text field to the post data schemas.
If the text is not in the post data, then populate it with the data in the name field.
This should make the migration to text easier, and will work until we are able to update the admin app.
2018-08-02 15:55:45 +01:00
Pea Tyczynska
d8d94b431f Find email_branding by name 2018-07-31 17:58:19 +01:00
Pea Tyczynska
c4bfd5479e Update EmailBranding model so it has separate name and text fields 2018-07-31 17:58:19 +01:00
Rebecca Law
19890412c9 Added a cancelled notification status so that we can cancel letters before they are sent. 2018-07-31 13:52:04 +01:00
Rebecca Law
67639bf49c Merge branch 'master' into check-data-retention-before-purging-notifications 2018-07-31 11:40:58 +01:00
Rebecca Law
ff2334937c Last night we had some letter notifications that were in permanent failure and had page count = 0.
This causes the create-nighlty-billing task to fail.
This will make sure that 0 is returned in this case.
2018-07-31 11:04:48 +01:00
Pea Tyczynska
b28ec8beda Use old monthly_billing-related routes for new functions 2018-07-30 11:33:15 +01:00
Pea Tyczynska
4e49df2479 Delete MonthlyBilling model 2018-07-30 11:07:42 +01:00
Pea Tyczynska
ca2b350a99 Remove references to monthly_billing table from api 2018-07-30 11:07:42 +01:00
Pea Tyczynska
c0f309a2a6 Delete scheduled task to populate monthly_billing 2018-07-30 11:06:04 +01:00
Leo Hemsted
a826f6e924 make rebuild-ft-billing-for-day remove unused rows
if the billing data was incorrect and needs to be rebuilt, we should
remove old rows. Previously we were only upserting new rows, but old
no-longer-relevant rows were staying in the database. This commit
makes the flask command remove *all* rows for that service and day,
before inserting the rows from the original notification data after.

This commit doesn't change the existing nightly task, nor does it
change the upsert that happens upon viewing the usage page. In normal
usage, there should never be a case where the number of billable units
for a rate decreases. It should only ever increase, thus, never need to
be deleted
2018-07-26 19:10:30 +01:00
Rebecca Law
a812060915 The unique constraint on SeviceCallbackApi was on service_id only.
Now that we have 2 types of api callbacks the constraint to be on service_id + callback_type.
2018-07-25 14:12:13 +01:00
Rebecca Law
4fc004b00a Increase the number of days we calculate billing from 3 to 10 days.
Log exception if the billing counts for letters are different in the dvla response file than what we collected.
2018-07-24 16:28:30 +01:00
Pea (Malgorzata Tyczynska)
1b21a12b83 Merge pull request #1971 from alphagov/add_task_to_send_complaints_on
Add and call task to send complaints on to service callback APIs
2018-07-24 15:07:09 +01:00
Rebecca Law
0675f09afb Need to be able to query the notification statistics for the right number of days.
If the request is for the big numbers on the activity page, then we need to use the number right number of days.
Added an end point to get the data retention for the service and notification type, which is needed on the activity page to say how long the report is available for.
2018-07-23 09:56:04 +01:00
Rebecca Law
dae29a1b61 Update the query for the notifications activity page to return the data for the days of retention if set. 2018-07-23 09:56:04 +01:00