This was added to migrate away from the vague "billing_units" field,
but without fixing the inconsistent data behind it:
- For emails and letters, "billing_units" was just the number sent.
- For SMS, "billing_units" really was the chargeable_units.
To avoid confusion we need two fields to represent the original mix
of data - this exposes "notifications_sent" in both APIs.
This represents the number of chargeable_units that were actually
free due to the free allowance - they won't be included in "cost".
Although the existing calculations in Admin [^1][^2] will still be
correct with a change in SMS rates - it's cost that's the problem
- it makes sense to have all the knowledge about calculating usage
consistently in these two APIs.
[^1]: 474d7dfda8/app/main/views/dashboard.py (L490)
[^2]: c63660d56d/app/main/views/dashboard.py (L350)
This starts to replace the calculation in Admin [^1] and, similar
to the yearly API, also correctly attributes free allowance when
we have a rate change during a month.
[^1]: 474d7dfda8/app/templates/views/usage.html (L98)
This will replace the manual calculation in Admin [^1] for SMS and
also in API [^2] for letters.
Doing the calculation here also means we correctly attribute free
allowance to the earliest rows in the billing table - Admin doesn't
know when a given rate was applied so can't do this with the data
currently returned from the API.
Since the calculation now depends on annual billing, we need to
change all the tests to make sure a suitable row exists.
Note about "OVER" clause
========================
Using "rows=" ("ROWS BETWEEN") makes more sense than "range=" as
we want the remainder to be incremental within each group in a
"GROUP BY" clause, as well as between groups i.e
# ROWS BETWEEN (arbitrary numbers to illustrate)
date=2021-04-03, units=3, cost=3.29
date=2021-04-03, units=2, cost=4.17
date=2021-04-04, units=2, cost=5.10
vs.
# RANGE BETWEEN
date=2021-04-03, units=3, cost=4.17
date=2021-04-03, units=2, cost=4.17
date=2021-04-04, units=2, cost=5.10
See [^3] for more details and examples.
[^1]: https://github.com/alphagov/notifications-admin/blob/master/app/templates/views/usage.html#L60
[^2]: 072c3b2079/app/billing/billing_schemas.py (L37)
[^3]: https://learnsql.com/blog/difference-between-rows-range-window-functions/
These tests weren't checking anything structural about the APIs
beyond what's covered by the other tests. They represent edge
cases that we can check at a lower level instead.
It was also unclear what these tests were actually testing, as
the term "all cases" is vague. Looking at the test data, there
are variations in rates, multipliers and billable units for SMS
and letters, which I've summarised as "variable rates".
Note: I've removed part of the test data - for the first class
letter rate - as it's not clearly adding anything.
