mirror of
https://github.com/GSA/notifications-api.git
synced 2026-09-10 10:03:38 -04:00
Only aggregate status when necessary for a service
This takes a similar approach to the nightly deletion task so that we only create sub-tasks when there are actually notifications to aggregate for a given type and day [1]. We're making this change to stop the duplication errors we're getting at the moment and ensure the task can scale to more messages and more services. There are two parts to this: - Each subtask should now run within the 5 minute visibility timeout. However, they may still be duplicated if the parent task overruns [2]. - The parent task creates a mininal number of subtasks, and the query to determine this is very fast for a normal process day (milliseconds). Since all tasks will run quickly, there should be no more duplication. In order to test this more nuanced task, I rewrote the tests: - One test checks the subtask is called correctly. - One test checks we create all the right subtasks. [1]: https://github.com/alphagov/notifications-api/pull/3381 [2]: https://docs.google.com/document/d/1MaP6Nyy3nJKkuh_4lP1wuDm19X8LZITOLRd9n3Ax-xg/edit#heading=h.q3intzwqhfzl
This commit is contained in:
@@ -794,3 +794,17 @@ def get_service_ids_that_have_notifications_from_before_timestamp(notification_t
|
||||
Notification.created_at < timestamp
|
||||
).distinct()
|
||||
}
|
||||
|
||||
|
||||
def get_service_ids_with_notifications_on_date(notification_type, date):
|
||||
return {
|
||||
row.service_id
|
||||
for row in db.session.query(
|
||||
Notification.service_id
|
||||
).filter(
|
||||
Notification.notification_type == notification_type,
|
||||
# using >= + < is much more efficient than date(created_at)
|
||||
Notification.created_at >= date,
|
||||
Notification.created_at < date + timedelta(days=1)
|
||||
).distinct()
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user