Commit Graph

2708 Commits

Author SHA1 Message Date
Katie Smith
e7f0a414ce Update service schema to add letter_logo_filename
Added the filename of a service's letter logo to the service schema. We want
this in the schema so that it is possible to call
`current_service.letter_logo_filename` from notifications-admin and to pass this value
through to template-preview.
2018-10-18 08:48:06 +01:00
Leo Hemsted
3589f1dd63 remove some use of calling fixtures directly
also experiment with giving the fixtures a different name, so we can be
sure that they won't be called in functions. Open to change on this.
2018-10-17 16:38:46 +01:00
Leo Hemsted
c63b147a2d make sure validation_failed notifications still return pdf
it's useful to see them, and previously it was crashing.
2018-10-17 16:09:30 +01:00
Leo Hemsted
7bf68e3664 fix failed sanitise flow
the move from virus scan to validation failed function was called with
the wrong variables, and had some internal logic that was slightly
wrong.

Also, Don't use `update_notification_by_id` for notifications if they
are not in `created`, `sending`, `pending`, or `sent`. It silently
doesn't update them. I didn't want to do a deeper investigation into
the reasons behind this terrifying state machine as part of this commit
so I just changed the functions to call `dao_update_notification`
manually
2018-10-16 17:30:39 +01:00
Pea Tyczynska
e22e7245fe Use sanitised pdfs for sending and handle invalid pdfs, details below:
- pass new, sanitised pdf for sending
- move invalid pdfs to a newly created bucket
- set status fro notifications that failed pdf validation to a new status validation-failed
- adjust existing tests
2018-10-16 17:30:35 +01:00
Chris Hill-Scott
8af5cbab86 Bump utils to 30.5.4 2018-10-09 15:08:54 +01:00
Katie Smith
236bbc5f28 Return postage from the monthly-usage endpoint
We were already returning the month, notification_type, billing_units
and rate from the /monthly-usage billing endpoint. This adds in the
postage too so that we can display postage details on the usage page of
admin.
2018-10-03 13:07:46 +01:00
Leo Hemsted
cffc697e7f letter estimated delivery now takes postage into account 2018-09-28 17:30:25 +01:00
Katie Smith
71d28035dd Update ft_billing with real postage data
* Changed update_fact_billing DAO function to update the table with the
real data for postage instead of hard-coding in 'second'.
* Added a test for the create nightly billing task to test that rows
with different postage are being inserted correctly.
2018-09-28 16:32:18 +01:00
Katie Smith
0aedbff750 Remove test
Removed the occasionally failing test to check how ft_billing upserts
postage data. This test will be re-added once the postage column has been
added to the primary key.
2018-09-28 15:00:29 +01:00
Katie Smith
6727f0e0f5 Update ft_billing DAO functions to use postage
* Updated the 'fetch_billing_data_for_day' DAO function to take postage into
account
* Updated the 'update_fact_billing' DAO function to insert postage for
new rows. When updating rows which are identical apart from the postage, the
original row will be kept. (This behaviour will change once postage is
added to the primary key - at this point, upserting will add a new row.)
* Also changed some fixtures / test set up functions to take postage
into account
2018-09-28 13:52:17 +01:00
Rebecca Law
a4f357499d Sometimes the page counts from DVLA do not match our page counts. Our counts have always been right. This PR does not update notification with the DVLA page count but does still let us know the difference happened. 2018-09-27 16:13:15 +01:00
Leo Hemsted
69e08fa23b fix tests to create notifications properly
migrated away from calling fixtures as functions
2018-09-25 16:10:26 +01:00
Pea Tyczynska
08fec8f928 Fix tests - update number fo letter rates in test 2018-09-25 15:10:16 +01:00
Pea Tyczynska
eed1d83187 Test that get_rate gets right rate based on post_class and date 2018-09-25 15:10:16 +01:00
Pea Tyczynska
39ebfb6826 Test filtering letter_rates by post_class 2018-09-25 15:10:16 +01:00
Pea Tyczynska
b3191dbc2a Filter letter rates by post_class in get_rate
Also adjust existing tests.
2018-09-25 15:10:15 +01:00
Rebecca Law
a54645c5a3 Set the postage in the filename based on the postage set on the notification. 2018-09-25 11:04:58 +01:00
Rebecca Law
bb97a9038f Merge branch 'master' into reduce-updates-to-notification 2018-09-20 14:48:40 +01:00
Rebecca Law
b00308d122 Removed an update statement to notifications.
It's a small change, but we should remove any db operations that are not necessary.
2018-09-20 14:47:24 +01:00
Leo Hemsted
8e19a0742f remove default postage from noti history
also add test for notification_history properly setting
2018-09-20 12:26:58 +01:00
Leo Hemsted
4dc3f829e3 give letter notifications the postage from their service 2018-09-19 17:23:17 +01:00
Leo Hemsted
9021b43eb6 use pytest.param to contain marks like xfail or skip 2018-09-19 16:33:00 +01:00
Leo Hemsted
918e4b390f add postage to notification + noti_history
if it's a letter notification, postage must equal "first" or "second",
else it must equal null
2018-09-19 16:32:58 +01:00
Rebecca Law
5f73dfdbe9 Added constraints to services.postage: not nullable and can only containt "first" or "second" 2018-09-18 17:41:25 +01:00
Rebecca Law
1c0892888f Change the name of letter_class to postage, this will match the domain language used in the documentation.
Updated the tests to answer review comments.
2018-09-18 15:22:08 +01:00
Rebecca Law
efb618e956 Fix merge conflicts for the db migration file.
Update test_services_dao to stop using fixtures. This is a massive change. The only real change in the file is to check for the letter_class.
2018-09-17 15:45:29 +01:00
Rebecca Law
79815b4f8a In order to support sending letters by first class we need a way to know how the service wants us to send the letter.
To start with this will be an attribute on the service, at the time the notification is created it will look at Service.letter_class to decide what class to use for the letter.

This PR adds Service.letter class as a nullable column.
Updated the create_service and update_service method to default the value to second.

Subsequent PRs will add the check constraint to ensure we only get first or second in the letter_class column and make that column nullable.
This can't be done all at once because it will cause an error if someone inserts or updates a service during the deploy.
2018-09-17 11:27:43 +01:00
Alexey Bezhan
c9ca720959 Change NotificationHistory.updated_at on update
Sets the updated_at to current time when the NotificationHistory
is modified (which happens occasionally for example to set status
to returned letter).
2018-09-13 14:09:09 +01:00
Rebecca Law
f1b04193ca In this PR we remove trigger-letter-pdfs-for-day scheduled task and just call collate_letter_pdfs_for_day instead.
There was a datetime bug in the query which resulted in files not being sent to the postal provider.
The trigger-letter-pdfs-for-day task is no longer needed, so rather than fix the query just call collate_letter_pdfs_for_day directly.
Less code is always better.

Deployment considerations: I realized this is strictly not backwards compatible if the scheduled job is in progress and a task is on the queue that no longer exists. This is ok since we will deploy this well before 17:50.
2018-09-12 17:16:34 +01:00
Alexey Bezhan
aacc00e3a7 Merge pull request #2075 from alphagov/process-returned-letters-list
Process returned letters list
2018-09-07 14:54:56 +01:00
Rebecca Law
8aacfb289d Add created_by_name to the Notification.serialize_for_csv.
The concern about performnace degrading has been thought through. We do not believe there will be an adverse effect since the high volume users do not send off messages.
2018-09-07 10:22:45 +01:00
Alexey Bezhan
3787e2954b Add a task to process returned letter lists
Adds an API endpoint `/letters/returned` that accepts a list of
notification references and creates a task to update their status.

Adds a new task that uses the list of references to update the status
of notifications to 'returned-letter'.

The update is currently done using a single query and logs the
number of changed records (including notification history records).
This could potentially be done within the `/letters/returned` endpoint,
but creating a job right away allows us to extend this more easily
in the future (e.g. logging missing notifications or adding callbacks).

The job is using the database tasks queue.
2018-09-06 16:39:17 +01:00
Alexey Bezhan
18ab7f3337 Add updated history count to dao_update_notifications_by_reference
For returned letter updates most notifications won't exist in the
notifications table, so in order to find out whether the reference
matches any known letters we need to check the count of updated
history records.
2018-09-04 16:55:22 +01:00
Alexey Bezhan
0dcf04def9 Add notification status for returned letters
We need to update letter notifications with a new status when DVLA
gives us a list of references for returned letters.

This adds the new status to the models and the DB.

DVLA call this 'returned mail', so I'm using it as the status name
since it seems less ambiguous than 'returned'.
2018-09-04 16:55:22 +01:00
Chris Hill-Scott
d52ee4606a Add error handler for duplicate domain 2018-09-04 14:40:24 +01:00
Chris Hill-Scott
5d5047fb13 Add unique constraint to email branding domains
Does two things:

1. Revert "Revert "Add unique constraint to email branding domain""

This reverts commit af9cb30ef3.

2. Don’t allow empty string in email branding domain

Columns with multiple `null`s can have a uniqueness constraint. Columns
with multiple empty string values are not considered unique.

This commit:
- removes any duplicate empty string values
- casts empty strings to null string any time these columns are updated

---

Squashed into this single commits because these two things are not
atomic as individual commits.
2018-09-04 11:10:46 +01:00
Chris Hill-Scott
0c47d41977 Remove govuk from possible brands
‘GOV.UK’ doesn’t make sense as a type of brand. It only made sense as
a type of branding that a service had.

Since we’ve:
- deprecated the service branding column
- made sure it’s not used as a value in the email branding table

we can remove this value from the table of possible brand types.
2018-08-30 16:36:35 +01:00
Chris Hill-Scott
337496c5bc Deprecate service branding column
We want to drop this column. First we have to stop using it anywhere.

Needs to be made nullable so we can stop writing to it.
2018-08-30 16:05:37 +01:00
Leo Hemsted
889fd796fb Merge pull request #2057 from alphagov/sanitise-precompiled
Sanitise precompiled pdfs after virus scan
2018-08-29 12:53:27 +01:00
Chris Hill-Scott
38096eb93e Merge pull request #2051 from pwright08/add-create-secret-code-unit-tests
Add unit tests for the Users DAO create_secret_code function
2018-08-29 09:49:36 +01:00
Chris Hill-Scott
2671ca3e5e Look in email_branding for brand_type, not service
https://www.pivotaltracker.com/story/show/159986276

We are now setting the type of branding on the branding itself, not on
the service.

This commit switches over from looking in the old place (on the service)
to looking in the new place (on the branding).
2018-08-28 16:22:41 +01:00
Leo Hemsted
40c537a9f5 add tests 2018-08-24 15:12:02 +01:00
Rebecca Law
562facb49c Now that EmailBranding includes the brand type we do not need two separate colours. This PR removes the new colour columns. 2018-08-24 13:51:11 +01:00
Rebecca Law
3816010c72 Add brand_type to EmailBranding.
It makes more sense to put the brand_type with EmailBranding rather than in Service.
Next step is to add the new type to the form in admin app.
2018-08-23 13:53:05 +01:00
Paul Wright
267a82758a Add unit tests for the Users DAO create_secret_code function 2018-08-23 12:18:47 +01:00
Rebecca Law
fd16d7060a Added a method to decide which colour to send to the notifications-utils HTMLEmailTemplate to create the content. 2018-08-22 12:59:06 +01:00
Rebecca Law
0b0e551626 Add banner_colour, single_id_colour, and domain to email_branding_schema.
The Admin app can now be updated to send the new columns to update/create email_branding
2018-08-20 10:57:42 +01:00
Rebecca Law
1658f4de71 Merge pull request #2045 from alphagov/email-branding-colors-and-domain
New columns for EmailBranding.
2018-08-17 12:56:32 +01:00
Rebecca Law
df2271a49f New columns for EmailBranding.
There is a requirement for the colour of the banner to be different to the single id colour. In order to accomodate that we are adding two new columns for that data.
These columns will be updated manually and not done with a data migration.

A new column called domain has been added. This will be used as a default domain, if the user that creates the service has a matching domain this email brand will be set for the service.

This PR only adds the new columns to the model and db.

https://www.pivotaltracker.com/story/show/159660295
2018-08-16 16:57:53 +01:00