This is covered sufficiently by the main "test_usage" assertions,
which prove the usage is broken down by postage.
I don't think we need to explicitly test the usage is broken down
by month as we already prove this for SMS and we also check the
usage is associated with the correct month in the "ordering" test.
This did more than it said and had some unconventional behaviour:
- It modified the input data. We can avoid this by computing the
postage group on-the-fly and using "sorted" instead of "sort".
- It defined a custom, named tuple. This isn't necessary as Jinja
allows us to access elements by qualification (".") already.
We can also use the same lambda function to group and sort items,
since the sort predicate is the same one we use to group them.
This is now only used for letters and represents the number sent
[^1]. We could use the chargeable_units field, but using "_sent"
is more consistent with the annual attributes [^2].
In fact, chargeable_units isn't actually used anywhere, but I've
kept it in the test data as it is part of the real API and helps
clarify the other values for SMS - free vs. charged.
Note: for SMS I've used an arbitrary "1234" for "chargeable_units"
to indicate it's not used and may be different to the number sent -
for SMS it's related to the number of fragments.
[^1]: bb62d22f25/app/dao/fact_billing_dao.py (L339)
[^2]: 3a1ac189ff/app/main/views/dashboard.py (L339)
This starts using the sms_{cost, charged, free_allowance_used}
fields in the new API to replace the "get_free_paid_breakdown"
function we had before, which could not support multiple rates.
In order to use "get_free_paid_breakdown" the calling method had
to store a "cumulative" variable to calculate the free allowance
used so far, which is now done by the API.
To calculate the data for conftest.py, I had to start from the
bottom ("April") and manually calculate the free allowance used
to emulate the API - this is what "cumulative" used to do.
It has never been possible to get multiple rows for the same month
and rate. This was making it hard to switch to the new API fields,
which will require some manual calculations. I've added the billing
units together in the remaining data so the tests still pass.
I've also moved the "April" row to the end as it was out-of-order
with all the others: it's the _start_ of the financial year.
This is also an opportunity to DRY-up the filtering of usage by
month, which we will reuse in the following commits. Doing a sum
is simple enough that it can be done inline, avoiding indirection.
So we don’t have to deploy a change on a Saturday.
In the future this could pull from the rates in the database, but while
that code is shifting around I didn’t want to touoch it. We’d also have
to think about caching so as not to have a non-authenticated route
hitting the database.
Note: I've removed the pricing assertion in the "0_free_allowance"
test as it's covered elsewhere - the value of the test is really to
check that we don't show the remainder if there never was any.
The previous, manual calculation could be incorrect depending on
which SMS rates the free allowance was attributed to.
The new field also supersedes the old "letter_total" bolt-on so we
can get cost information consistently for both types.
This adds missing assertions for email and SMS usage, as well as
letters with the help of some additional test data.
Previously we were only checking monthly usage (in other tests).
The "with_letters" was mostly a duplicate of the one before - no
change in test setup - bar the three assertions at the end.
Having the assertions in a separate test will help keep the one
above manageable as we add more assertions for the annual usage.
This will make the following changes clearer.
In the next commits we'll go into more detail about "billing_units"
and how it differs for SMS vs. emails and letters.
At the moment, we put the sms rate on the usage page for each
months billing data by taking the single sms rate for the year.
The assumption that there will be a single sms rate for the year is
no longer going to be true. Therefore, instead we take the sms
rate from the monthly data itself which tells us the rate for
a months worth of sent SMS.
This will allow both prometheis (the shared one and our own) to scrape
the /metrics endpoint, each with their own authentication
See alphagov/gds_metrics_python#1 for more details
