Reduce extra S3 ops when working with letter PDFs

Previously we did some unnecessary work:

- Collate task. This had one S3 request to get a summary of the object,
which was then used in another request to get the full object. We only
need the size of the object, which is included in the summary [1].

- Archive task. This had one S3 request to get a summary of the object,
which was then used to make another request to delete it. We still need
both requests, but we can remove the S3.Object in the middle.

[1]: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#objectsummary
This commit is contained in:
Ben Thorner
2021-03-16 11:57:33 +00:00
parent ff7eebc90a
commit c76e789f1e
5 changed files with 22 additions and 38 deletions

View File

@@ -28,7 +28,7 @@ from app.exceptions import NotificationTechnicalFailureException
from app.letters.utils import (
LetterPDFNotFound,
ScanErrorType,
find_letter_pdf_filename,
find_letter_pdf_in_s3,
generate_letter_pdf_filename,
get_billable_units_for_letter_page_count,
get_file_names_from_error_bucket,
@@ -237,11 +237,10 @@ def get_key_and_size_of_letters_to_be_sent_to_print(print_run_deadline, postage)
letters_awaiting_sending = dao_get_letters_to_be_printed(print_run_deadline, postage)
for letter in letters_awaiting_sending:
try:
letter_file_name = find_letter_pdf_filename(letter)
letter_head = s3.head_s3_object(current_app.config['LETTERS_PDF_BUCKET_NAME'], letter_file_name)
letter_pdf = find_letter_pdf_in_s3(letter)
yield {
"Key": letter_file_name,
"Size": letter_head['ContentLength'],
"Key": letter_pdf.key,
"Size": letter_pdf.size,
"ServiceId": str(letter.service_id)
}
except (BotoClientError, LetterPDFNotFound) as e: