Files
notifications-api/app/dao/daily_sorted_letter_dao.py
Rebecca Law 82cc6d6bef As it turns out the s3ftp used to mount the s3 bucket to our ftp server puts the file on s3 more than once. So the SNS topic is triggered more than once.
We need to deal with this, it's ok when updating a notification status from delivered to delivered. But the DailySortedLetter counts are being doubled.
Adding the file_name to the table as a unique key to get around this issue. It will mean we have multiple rows for each billing_day, but that's ok we can aggregate that.
This will also give us a way to see which file created which count.
2018-03-14 17:04:58 +00:00

39 lines
1.3 KiB
Python

from datetime import datetime
from sqlalchemy.dialects.postgresql import insert
from app import db
from app.dao.dao_utils import transactional
from app.models import DailySortedLetter
def dao_get_daily_sorted_letter_by_billing_day(billing_day):
return DailySortedLetter.query.filter_by(
billing_day=billing_day
).first()
@transactional
def dao_create_or_update_daily_sorted_letter(new_daily_sorted_letter):
'''
This uses the Postgres upsert to avoid race conditions when two threads try and insert
at the same row. The excluded object refers to values that we tried to insert but were
rejected.
http://docs.sqlalchemy.org/en/latest/dialects/postgresql.html#insert-on-conflict-upsert
'''
table = DailySortedLetter.__table__
stmt = insert(table).values(
billing_day=new_daily_sorted_letter.billing_day,
file_name=new_daily_sorted_letter.file_name,
unsorted_count=new_daily_sorted_letter.unsorted_count,
sorted_count=new_daily_sorted_letter.sorted_count)
stmt = stmt.on_conflict_do_update(
index_elements=[table.c.billing_day, table.c.file_name],
set_={
'unsorted_count': stmt.excluded.unsorted_count,
'sorted_count': stmt.excluded.sorted_count,
'updated_at': datetime.utcnow()
}
)
db.session.connection().execute(stmt)