Commit Graph
3512 Commits
Author SHA1 Message Date
Katie Smith 4dab4fa8ce Pass letter logo filename to template preview
We now pass `filename`, the filename of the letter logo to use, through
to Template Preview in addition to the `dvla_org_id`. Once Template
Preview has been updated to only use the `filename` we will stop
sending the `dvla_org_id`.
2018-10-19 10:07:11 +01:00
Rebecca Law 38c29d41a4 Reduce the error logging for sanitse method, since we ignore cysp.
Also fixed the spelling error, why not.
2018-10-18 16:43:14 +01:00
Rebecca Law 021a90f482 Merge branch 'master' into update-page-count-after-antivirus-scan 2018-10-18 16:01:59 +01:00
Katie Smith e7f0a414ce Update service schema to add letter_logo_filename
Added the filename of a service's letter logo to the service schema. We want
this in the schema so that it is possible to call
`current_service.letter_logo_filename` from notifications-admin and to pass this value
through to template-preview.
2018-10-18 08:48:06 +01:00
Leo Hemsted 3589f1dd63 remove some use of calling fixtures directly
also experiment with giving the fixtures a different name, so we can be
sure that they won't be called in functions. Open to change on this.
2018-10-17 16:38:46 +01:00
Leo Hemsted c63b147a2d make sure validation_failed notifications still return pdf
it's useful to see them, and previously it was crashing.
2018-10-17 16:09:30 +01:00
Leo Hemsted 7bf68e3664 fix failed sanitise flow
the move from virus scan to validation failed function was called with
the wrong variables, and had some internal logic that was slightly
wrong.

Also, Don't use `update_notification_by_id` for notifications if they
are not in `created`, `sending`, `pending`, or `sent`. It silently
doesn't update them. I didn't want to do a deeper investigation into
the reasons behind this terrifying state machine as part of this commit
so I just changed the functions to call `dao_update_notification`
manually
2018-10-16 17:30:39 +01:00
Pea Tyczynska e22e7245fe Use sanitised pdfs for sending and handle invalid pdfs, details below:
- pass new, sanitised pdf for sending
- move invalid pdfs to a newly created bucket
- set status fro notifications that failed pdf validation to a new status validation-failed
- adjust existing tests
2018-10-16 17:30:35 +01:00
Rebecca Law 4263117189 We were getting the page count for the letter before virus scan happened.
This PR moves setting the billale_units for the letter after virus scan has passed.
2018-10-16 15:08:15 +01:00
Chris Hill-Scott 8af5cbab86 Bump utils to 30.5.4 2018-10-09 15:08:54 +01:00
Katie Smith 236bbc5f28 Return postage from the monthly-usage endpoint
We were already returning the month, notification_type, billing_units
and rate from the /monthly-usage billing endpoint. This adds in the
postage too so that we can display postage details on the usage page of
admin.
2018-10-03 13:07:46 +01:00
Leo Hemsted cffc697e7f letter estimated delivery now takes postage into account 2018-09-28 17:30:25 +01:00
Katie Smith 71d28035dd Update ft_billing with real postage data
* Changed update_fact_billing DAO function to update the table with the
real data for postage instead of hard-coding in 'second'.
* Added a test for the create nightly billing task to test that rows
with different postage are being inserted correctly.
2018-09-28 16:32:18 +01:00
Katie Smith 0aedbff750 Remove test
Removed the occasionally failing test to check how ft_billing upserts
postage data. This test will be re-added once the postage column has been
added to the primary key.
2018-09-28 15:00:29 +01:00
Katie Smith 6727f0e0f5 Update ft_billing DAO functions to use postage
* Updated the 'fetch_billing_data_for_day' DAO function to take postage into
account
* Updated the 'update_fact_billing' DAO function to insert postage for
new rows. When updating rows which are identical apart from the postage, the
original row will be kept. (This behaviour will change once postage is
added to the primary key - at this point, upserting will add a new row.)
* Also changed some fixtures / test set up functions to take postage
into account
2018-09-28 13:52:17 +01:00
Rebecca Law a4f357499d Sometimes the page counts from DVLA do not match our page counts. Our counts have always been right. This PR does not update notification with the DVLA page count but does still let us know the difference happened. 2018-09-27 16:13:15 +01:00
Leo Hemsted 69e08fa23b fix tests to create notifications properly
migrated away from calling fixtures as functions
2018-09-25 16:10:26 +01:00
Pea Tyczynska 08fec8f928 Fix tests - update number fo letter rates in test 2018-09-25 15:10:16 +01:00
Pea Tyczynska eed1d83187 Test that get_rate gets right rate based on post_class and date 2018-09-25 15:10:16 +01:00
Pea Tyczynska 39ebfb6826 Test filtering letter_rates by post_class 2018-09-25 15:10:16 +01:00
Pea Tyczynska b3191dbc2a Filter letter rates by post_class in get_rate
Also adjust existing tests.
2018-09-25 15:10:15 +01:00
Rebecca Law a54645c5a3 Set the postage in the filename based on the postage set on the notification. 2018-09-25 11:04:58 +01:00
Rebecca Law bb97a9038f Merge branch 'master' into reduce-updates-to-notification 2018-09-20 14:48:40 +01:00
Rebecca Law b00308d122 Removed an update statement to notifications.
It's a small change, but we should remove any db operations that are not necessary.
2018-09-20 14:47:24 +01:00
Leo Hemsted 8e19a0742f remove default postage from noti history
also add test for notification_history properly setting
2018-09-20 12:26:58 +01:00
Leo Hemsted 4dc3f829e3 give letter notifications the postage from their service 2018-09-19 17:23:17 +01:00
Leo Hemsted 9021b43eb6 use pytest.param to contain marks like xfail or skip 2018-09-19 16:33:00 +01:00
Leo Hemsted 918e4b390f add postage to notification + noti_history
if it's a letter notification, postage must equal "first" or "second",
else it must equal null
2018-09-19 16:32:58 +01:00
Rebecca Law 5f73dfdbe9 Added constraints to services.postage: not nullable and can only containt "first" or "second" 2018-09-18 17:41:25 +01:00
Rebecca Law 1c0892888f Change the name of letter_class to postage, this will match the domain language used in the documentation.
Updated the tests to answer review comments.
2018-09-18 15:22:08 +01:00
Rebecca Law efb618e956 Fix merge conflicts for the db migration file.
Update test_services_dao to stop using fixtures. This is a massive change. The only real change in the file is to check for the letter_class.
2018-09-17 15:45:29 +01:00
Rebecca Law 79815b4f8a In order to support sending letters by first class we need a way to know how the service wants us to send the letter.
To start with this will be an attribute on the service, at the time the notification is created it will look at Service.letter_class to decide what class to use for the letter.

This PR adds Service.letter class as a nullable column.
Updated the create_service and update_service method to default the value to second.

Subsequent PRs will add the check constraint to ensure we only get first or second in the letter_class column and make that column nullable.
This can't be done all at once because it will cause an error if someone inserts or updates a service during the deploy.
2018-09-17 11:27:43 +01:00
Alexey Bezhan c9ca720959 Change NotificationHistory.updated_at on update
Sets the updated_at to current time when the NotificationHistory
is modified (which happens occasionally for example to set status
to returned letter).
2018-09-13 14:09:09 +01:00
Rebecca Law f1b04193ca In this PR we remove trigger-letter-pdfs-for-day scheduled task and just call collate_letter_pdfs_for_day instead.
There was a datetime bug in the query which resulted in files not being sent to the postal provider.
The trigger-letter-pdfs-for-day task is no longer needed, so rather than fix the query just call collate_letter_pdfs_for_day directly.
Less code is always better.

Deployment considerations: I realized this is strictly not backwards compatible if the scheduled job is in progress and a task is on the queue that no longer exists. This is ok since we will deploy this well before 17:50.
2018-09-12 17:16:34 +01:00
Alexey Bezhan aacc00e3a7 Merge pull request #2075 from alphagov/process-returned-letters-list
Process returned letters list
2018-09-07 14:54:56 +01:00
Rebecca Law 8aacfb289d Add created_by_name to the Notification.serialize_for_csv.
The concern about performnace degrading has been thought through. We do not believe there will be an adverse effect since the high volume users do not send off messages.
2018-09-07 10:22:45 +01:00
Alexey Bezhan 3787e2954b Add a task to process returned letter lists
Adds an API endpoint `/letters/returned` that accepts a list of
notification references and creates a task to update their status.

Adds a new task that uses the list of references to update the status
of notifications to 'returned-letter'.

The update is currently done using a single query and logs the
number of changed records (including notification history records).
This could potentially be done within the `/letters/returned` endpoint,
but creating a job right away allows us to extend this more easily
in the future (e.g. logging missing notifications or adding callbacks).

The job is using the database tasks queue.
2018-09-06 16:39:17 +01:00
Alexey Bezhan 18ab7f3337 Add updated history count to dao_update_notifications_by_reference
For returned letter updates most notifications won't exist in the
notifications table, so in order to find out whether the reference
matches any known letters we need to check the count of updated
history records.
2018-09-04 16:55:22 +01:00
Alexey Bezhan 0dcf04def9 Add notification status for returned letters
We need to update letter notifications with a new status when DVLA
gives us a list of references for returned letters.

This adds the new status to the models and the DB.

DVLA call this 'returned mail', so I'm using it as the status name
since it seems less ambiguous than 'returned'.
2018-09-04 16:55:22 +01:00
Chris Hill-Scott d52ee4606a Add error handler for duplicate domain 2018-09-04 14:40:24 +01:00
Chris Hill-Scott 5d5047fb13 Add unique constraint to email branding domains
Does two things:

1. Revert "Revert "Add unique constraint to email branding domain""

This reverts commit af9cb30ef3.

2. Don’t allow empty string in email branding domain

Columns with multiple `null`s can have a uniqueness constraint. Columns
with multiple empty string values are not considered unique.

This commit:
- removes any duplicate empty string values
- casts empty strings to null string any time these columns are updated

---

Squashed into this single commits because these two things are not
atomic as individual commits.
2018-09-04 11:10:46 +01:00
Chris Hill-Scott 0c47d41977 Remove govuk from possible brands
‘GOV.UK’ doesn’t make sense as a type of brand. It only made sense as
a type of branding that a service had.

Since we’ve:
- deprecated the service branding column
- made sure it’s not used as a value in the email branding table

we can remove this value from the table of possible brand types.
2018-08-30 16:36:35 +01:00
Chris Hill-Scott 337496c5bc Deprecate service branding column
We want to drop this column. First we have to stop using it anywhere.

Needs to be made nullable so we can stop writing to it.
2018-08-30 16:05:37 +01:00
Leo Hemsted 889fd796fb Merge pull request #2057 from alphagov/sanitise-precompiled
Sanitise precompiled pdfs after virus scan
2018-08-29 12:53:27 +01:00
Chris Hill-Scott 38096eb93e Merge pull request #2051 from pwright08/add-create-secret-code-unit-tests
Add unit tests for the Users DAO create_secret_code function
2018-08-29 09:49:36 +01:00
Chris Hill-Scott 2671ca3e5e Look in email_branding for brand_type, not service
https://www.pivotaltracker.com/story/show/159986276

We are now setting the type of branding on the branding itself, not on
the service.

This commit switches over from looking in the old place (on the service)
to looking in the new place (on the branding).
2018-08-28 16:22:41 +01:00
Leo Hemsted 40c537a9f5 add tests 2018-08-24 15:12:02 +01:00
Rebecca Law 562facb49c Now that EmailBranding includes the brand type we do not need two separate colours. This PR removes the new colour columns. 2018-08-24 13:51:11 +01:00
Rebecca Law 3816010c72 Add brand_type to EmailBranding.
It makes more sense to put the brand_type with EmailBranding rather than in Service.
Next step is to add the new type to the form in admin app.
2018-08-23 13:53:05 +01:00
Paul Wright 267a82758a Add unit tests for the Users DAO create_secret_code function 2018-08-23 12:18:47 +01:00