Commit Graph

3900 Commits

Author SHA1 Message Date
Chris Hill-Scott
1e82484dfb Update service branding when setting org branding
When a service asks for branding I often go and:
- set the branding on the organisation as well
- set the branding on all the organisation’s services

The latter can be quite time consuming, but it does save effort since
existing services from the same organisation won’t have to request
branding. It also improves the consistency of the communications that
users are receiving.

This commit automates that process by applying the branding update to
any services belonging to the organisation, if those services don’t
already have their own custom branding set up.
2019-09-25 15:36:11 +01:00
Rebecca Law
a1863fa419 Update all calls to get_folder_name to include the parameter name.
Use created_at date of the notification for precompiled letters.
2019-09-25 14:40:09 +01:00
Rebecca Law
f234e94572 Change the variable name to make a little more sense. 2019-09-25 13:56:10 +01:00
Rebecca Law
702d8fa85f Refactor the code that figures out what folder and filename to use for the letter pdf files.
Now we consistently use the created_at date, so we can always get the right file location and name.

The previous updates to this code were trying to solve the problem if a pdf being created at 17:29, but not ready to upload until 17:31 after the antivirus and validation check.
But in those cases we would have trouble finding the file.
2019-09-25 13:56:10 +01:00
Pea Tyczynska
8cf8d24e37 Return count of notifications in the database for a job
When we cancel a job, we need to check if all notifications are
already in the database. So far, we were querying for all
notification objects in the database and counting them in
admin app, which runs into pagination problems for large jobs,
and could time out for very large jobs.
2019-09-24 16:56:03 +01:00
Pea (Malgorzata Tyczynska)
c1b53dce3a Merge pull request #2612 from alphagov/redaction_failed_doesnt_matter_when_test_key
Don't log address redaction failure when letter sent with test key
2019-09-18 16:05:39 +01:00
Leo Hemsted
d080106cbe make sure test notifications don't get date subfolders
they just go in the test bucket's root
2019-09-18 10:24:47 +01:00
Pea Tyczynska
1279a46b8b Don't log address redaction failure when letter sent with test key 2019-09-17 15:55:26 +01:00
Leo Hemsted
99eb17fc29 Merge pull request #2610 from alphagov/get-pdf-contents-via-api
add api endpoint to get pdf for letter
2019-09-17 14:55:34 +01:00
Leo Hemsted
efaa4f2ad2 karlify the exception messages
also create a PDFNotReadyError class, separate to BadRequestError, to
imply to the end user that this is something they should handle
separately to all the other errors
2019-09-17 12:16:24 +01:00
Leo Hemsted
1ad32c9168 move get_pdf functionality to its own endpoint
it felt very awkward when the body of a pdf might be empty, might have
things in it, and whether it is empty or not can change even when the
status is the same (a created template notification might have a pdf,
but might not, we don't know).

So move it to its own endpoint, so we can hand craft some 400 errors
that appropriately explain what's going on.
2019-09-12 17:06:22 +01:00
Katie Smith
190119f4fe Change get_bucket_name_and_prefix_for_notification to use created_at
The `get_bucket_name_and_prefix_for_notification` function was looking
at the `sent_at` or `updated_at` at time of a notification to see which
bucket it was in. Precompiled letters sent through the admin app don't
have either of these times - they only have a `created_at` time, so this
lets the function check `created_at` time too.
2019-09-12 14:58:51 +01:00
Katie Smith
8347b86b44 Add endpoint to create pdf letter 2019-09-12 14:58:51 +01:00
Katie Smith
eb73ddb8b4 Add function to send PDF letter
This function checks various permissions, downloads the PDF from the
transient bucket, creates the notification then moves the letter to the
'normal' bucket.
2019-09-12 14:58:51 +01:00
Katie Smith
081543a2a9 Refactor out function to get page count
This has been moved to the letters utils file since it will be used in
more than one place. The notification parameter has been removed so that
the function can be used when we don't have a notification id.
2019-09-12 14:58:51 +01:00
Katie Smith
a241fe4a29 Add transient uploaded letters bucket to config 2019-09-12 09:56:10 +01:00
Katie Smith
0c1fa3852f Add endpoint for getting / creating a hidden template
We want a way of getting the hidden precompiled template from admin,
so this adds an endpoint which gets the template or creates it if it doesn't
exist. The function to get or create the hidden template already existed
but has been moved to the template DAO now that it is used in more
places.
2019-09-12 09:56:10 +01:00
Leo Hemsted
52f7620772 create pdfs for test templated letters
previously, we didn't create templated letters, and just marked them as
delivered straight away. However, we may need to return PDFs for these
letters, so we should create them the same as live letters. Then update
the functions so that they know where to look for these letters.
2019-09-11 15:02:12 +01:00
Leo Hemsted
22aff482a8 add flag to return pdf content via api
this is only applicable when getting a single notification by id. it's
also ignored if the notification isn't a letter.

Otherwise, it overwrites the 'body' field in the response.

If the notification's pdf is available, it returns that, base64
encoded. If the pdf is not available, it returns an empty string.

The pdf won't be available if the notification's status is:
* pending-virus-scan
* virus-scan-failed
* validation-failed
* technical-failure

The pdf will be retrieved from the correct s3 bucket based on its type
2019-09-11 15:02:12 +01:00
Pea Tyczynska
fecd7b5728 Copy original file tp redaction_failure folder when redaction fails 2019-09-10 15:10:18 +01:00
Pea (Malgorzata Tyczynska)
b5b72a83d6 Merge pull request #2604 from alphagov/new-response-from-template-preview
Handle both new and old response type from template preview's sanitise method
2019-09-09 14:58:05 +01:00
Pea Tyczynska
8460147dfa Handle both new and old response type from template preview's
sanitise endpoint

Fix tests so they accept new response handling
2019-09-06 13:18:21 +01:00
Chris Hill-Scott
c032ef215a Return org_type in list of all organisations
This will let us do some filtering of this list in the admin. It’s
better to do it there because it means the admin can use the same cached
response from Redis each time.
2019-09-05 16:04:14 +01:00
Rebecca Law
6e6b7584cf Merge pull request #2605 from alphagov/fix-get-pdf-using-created_at-date
Fix the folder returned for a pdf letter.
2019-09-05 14:21:02 +01:00
Rebecca Law
a772fb8acf Fix the folder returned for a pdf letter.
Using the created at date for the folder is not always going to work because the pdf created_at date could be just before the cut off date but virus scan and validation has yet to happen. By the time the letters is in the created state, the letter goes into the next days bucket.
It can also happen if the letters is stuck in `pending-virus-scan` and we need to restart the task, and the letters is in a different folder.
2019-09-05 11:47:31 +01:00
Leo Hemsted
70828d4d87 Merge pull request #2601 from alphagov/pyup-scheduled-update-2019-08-28
Scheduled weekly dependency update for week 34
2019-09-04 13:05:38 +01:00
Leo Hemsted
51f90c17fd pin marshmallow and update flask error handling 2019-09-04 12:14:28 +01:00
Rebecca Law
e64ae321cf The sheet count was not calculated properly (it should be billable_units/notifications_sent).
And it turns out the sheet count is not required for the report. This PR takes out the columns to resolve the group by error.
2019-09-03 13:16:08 +01:00
Leo Hemsted
d83a9fee05 Merge pull request #2599 from alphagov/billing-for-all-services
Billing for all services
2019-09-02 16:38:05 +01:00
Leo Hemsted
93e631221a use dates rather than datetimes when comparing with bst_date
bst_date is a date field. Comparing dates with datetimes in postgres
gets confusing and dangerous. See this example, where a date evaluates
as older than midnight that same day.

```
notification_api=# select '2019-04-01' >= '2019-04-01 00:00';
 ?column?
----------
 f
(1 row)
```

By only using dates everywhere, we reduce the chance of these bugs
happening
2019-09-02 11:56:56 +01:00
Leo Hemsted
5975ae2383 remove unneccessary duped lines
as per pr comments
2019-08-30 16:49:58 +01:00
Leo Hemsted
7313dbeb86 normalise join patterns across billing queries
select from service, join to org and ft_billing
2019-08-30 12:18:52 +01:00
Leo Hemsted
6f420cf066 explicitly join tables
from service, join organisation, the free_allowance_remainder subquery
and the ft_billing table. Being explicit reduces confusion about what
tables we're joining and how we're constraining those joins

also remove references to AnnualBilling since we've already got the
free sms allowance from the free_allowance_remainder subquery
2019-08-30 12:18:52 +01:00
Leo Hemsted
48e96f253b ensure fetch_sms_free_allowance_remainder always returns
if there are no rows for a service in ft_billing, we should still
return their allowance (with 0 fragments used).

To do this, we need to build the query starting from AnnualBilling and
joining onto FactBilling, rather than the other way round. Also, we
need to account for the possibility of the sums being null by
coalescing them to 0
2019-08-30 12:18:40 +01:00
Leo Hemsted
b46eed1423 sort rest response by org_name, service_name
needed to re-arrange test results for this
2019-08-30 12:18:00 +01:00
Leo Hemsted
7811010e6c fix command name and get most recent data rather than oldest
fix query to order the right way

also remove debug prints that duplicate the formatted line below
2019-08-30 12:18:00 +01:00
Leo Hemsted
e4963ca0d3 make sure all letter costs are cast from decimal to float 2019-08-30 12:18:00 +01:00
Leo Hemsted
99a008e908 explicitly join annual_billing and remove reference to service.id
service.id isn't used in this query (both tables already have
service_id), and explicitly joining makes what we're doing more obvious
2019-08-30 12:18:00 +01:00
Leo Hemsted
741d75c3a9 use func.greatest rather than case statement
simplifies the query quite a bit
2019-08-30 12:18:00 +01:00
Leo Hemsted
608738ca67 remove unneccessary str cast 2019-08-30 12:18:00 +01:00
Leo Hemsted
028b81b787 simplify validate_date_range_is_within_a_financial_year
we can use the get_financial_year_for_datetime function
to massively simplify this
2019-08-30 12:18:00 +01:00
Leo Hemsted
b7e8f1baa2 rename which_financial_year to get_financial_year_for_datetime
also fix bug on the very second of rollover, march 31st 23:00:00, and
add tests
2019-08-30 12:18:00 +01:00
Rebecca Law
1c94d6d24a Added command to populate data for annual billing based on last years values. 2019-08-30 12:17:59 +01:00
Rebecca Law
cb7fff6100 Add endpoint to return structured data 2019-08-30 12:17:59 +01:00
Rebecca Law
4c36e22e93 [WIP]
fire alarm is going off - better save :)
2019-08-30 12:17:59 +01:00
Rebecca Law
1a5763feb6 Compare result tuple rather than each element in tuple (just to make it a little shorter)
Change order of columns
2019-08-30 12:17:59 +01:00
Rebecca Law
3b16950aee Final touches on tests and query for itemised letter billing data. 2019-08-30 12:17:59 +01:00
Rebecca Law
236d111c22 Update queries for data model change (removal of organisation_to_service)
New query to get letter breakdown.
2019-08-30 12:17:59 +01:00
Chris Hill-Scott
8d9038c3e9 Give GP practices NHS branding by default
Previously we were doing it based on their email address. This will also
apply it if they self-select as a GP surgery, even if they don’t have an
NHS email address.
2019-08-28 15:33:00 +01:00
Chris Hill-Scott
edb5790b7c Use nhs_gp not gp as org type
Means we have the option of looking for anything starting with `nhs` as
a way of filtering organisations.
2019-08-28 15:24:30 +01:00