darkhelm/notifications-api
1
0
Fork 0
mirror of https://github.com/GSA/notifications-api.git synced 2025-12-20 07:21:13 -05:00
Code Issues Packages Projects Releases Wiki Activity

Optimise billing query for notification history

Browse Source 
This follows the same pattern as for status aggregations [^1]. We
haven't seen this problem for a long time because of [^2], but now
we're trying to re-run the aggregation for some incorrect rows it's
becoming apparent we need to fix it.

The following query currently fails in Production after the 30 min
SQLAlchemy timeout:

      select template_id, rate_multiplier, international, sum(billable_units), count(*)
      from notification_history
      where notification_status in ('delivered', 'sending')
      and key_type != 'test'
      and notification_type = 'sms'
      and service_id = '539d63a1-701d-400d-ab11-f3ee2319d4d4'
      and created_at >= '2021-07-07 23:00'
      and created_at < '2021-07-08 23:00'
      group by 1,2,3,4;

Running a quick "explain analyze" with this change applied returns
near immediately, but hangs without it. This is enough evidence for
me that this change will fix the issue.

[^1]: https://github.com/alphagov/notifications-api/pull/3417
[^2]: e5c76ffda7
This commit is contained in:
Ben Thorner
2022-05-17 17:24:27 +01:00
parent 177b860865
commit 4a520bce78
1 changed files with 5 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
app/dao/fact_billing_dao.py
View File

@@ -15,7 +15,8 @@ from app.dao.organisation_dao import dao_get_organisation_live_services
from app.models import (
EMAIL_TYPE,
INTERNATIONAL_POSTAGE_TYPES,
KEY_TYPE_TEST,
KEY_TYPE_NORMAL,
KEY_TYPE_TEAM,
LETTER_TYPE,
NOTIFICATION_STATUS_TYPES_BILLABLE_FOR_LETTERS,
NOTIFICATION_STATUS_TYPES_BILLABLE_SMS,
@@ -493,7 +494,7 @@ def _query_for_billing_data(table, notification_type, start_date, end_date, serv
func.count().label('notifications_sent'),
).filter(
table.status.in_(NOTIFICATION_STATUS_TYPES_SENT_EMAILS),
table.key_type != KEY_TYPE_TEST,
table.key_type.in_((KEY_TYPE_NORMAL, KEY_TYPE_TEAM)),
table.created_at >= start_date,
table.created_at < end_date,
table.notification_type == notification_type,
@@ -520,7 +521,7 @@ def _query_for_billing_data(table, notification_type, start_date, end_date, serv
func.count().label('notifications_sent'),
).filter(
table.status.in_(NOTIFICATION_STATUS_TYPES_BILLABLE_SMS),
table.key_type != KEY_TYPE_TEST,
table.key_type.in_((KEY_TYPE_NORMAL, KEY_TYPE_TEAM)),
table.created_at >= start_date,
table.created_at < end_date,
table.notification_type == notification_type,
@@ -549,7 +550,7 @@ def _query_for_billing_data(table, notification_type, start_date, end_date, serv
func.count().label('notifications_sent'),
).filter(
table.status.in_(NOTIFICATION_STATUS_TYPES_BILLABLE_FOR_LETTERS),
table.key_type != KEY_TYPE_TEST,
table.key_type.in_((KEY_TYPE_NORMAL, KEY_TYPE_TEAM)),
table.created_at >= start_date,
table.created_at < end_date,
table.notification_type == notification_type,