mirror of
https://github.com/GSA/notifications-api.git
synced 2025-12-20 15:31:15 -05:00
Change which letters we want to be sent to look at all days
Previously, when running the `collate_letter_pdfs_for_day` task, we
would only send letters that were created between 5:30pm yesterday and
5:30 today.
Now we send letters that were created before 5:30pm today and that are
still waiting to be sent. This will help us automatically attempt to
send letters that may have fallen through the gaps and not been sent the
previous day when they should have been.
Previously we solved the problem of letters that had fallen the gap by
having to run the task with a date parameter for example
`collate_letter_pdfs_for_day('2020-02-18'). We no longer need this date
parameter as we will always look back across previous days too for
letters that still need sending.
Note, we have to change from using the pagination `list_objects_v2` to
instead getting each individual notification from s3. We reduce load by
using `HEAD` rather than `GET` but this will still greatly increase the
number of API calls. We acknowledge there will be a small cost to this,
say 50p for 5000 letters and think this is tolerable. Boto3 also handles
retries itself so if when making one of the many HEAD requests, there is
a networking blip then it should be retried automatically for us.
This commit is contained in:
@@ -19,6 +19,12 @@ def get_s3_object(bucket_name, file_location):
|
||||
return s3.Object(bucket_name, file_location)
|
||||
|
||||
|
||||
def head_s3_object(bucket_name, file_location):
|
||||
# https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.head_object
|
||||
boto_client = client('s3', current_app.config['AWS_REGION'])
|
||||
return boto_client.head_object(Bucket=bucket_name, Key=file_location)
|
||||
|
||||
|
||||
def file_exists(bucket_name, file_location):
|
||||
try:
|
||||
# try and access metadata of object
|
||||
|
||||
Reference in New Issue
Block a user