While the API still has a column and field for personalisation I think
it makes sense for it to consider the personalisation when serialising
the broadcast, so we should have a test for this.
This way the API still works as a coherent whole, and the admin app just
happens to be a client of the API which doesn’t implement the
personalisation feature.
If we want to remove personalisation from the API at another time we
should do it wholesale.
At the moment we display the count of scheduled jobs on the dashboard
by sending all the scheduled jobs to the admin app and letting it work
out the stats.
This is inefficient and, because the get jobs response has a page size
of 50, becomes incorrect if a service schedules more than 50 jobs.
This commit adds a separate endpoint which gives the admin app the stats
it needs directly and correctly.
For every missing row this was:
- downloading the CSV file from S3
- looping through every row in it until it found the one matching the
index of the missing row
`RecipientCSV` implements `__getitem__`[1] (which maybe it didn’t
before) so we can create it once, then index the relevant row directly.
***
1. 5ae0572d41/notifications_utils/recipients.py (L78-L79)
Since we’ve doubled the number of rows in a job, jobs can take twice as
long to insert all the notifications. We don’t check for missing rows
until we’re pretty confident that the original tasks have finished
processing. This means we need to double the time we wait to still be
as sure.
we might need to deal with a potential large volume of SMS delivery
receipts. These receipts are POSTed to oru public api to two URLs. The
actual endpoint just parses the response body and puts a task on an SQS
queue - no database connections are required or anything like that.
Split up this traffic from other traffic, so that any increase in volume
of callbacks won't affect the scaling/load/etc of the main api apps.
We've hard coded instance counts to 10 on prod for now until we get an
idea of load.
keep the ID the same as the existing prod broadcast org to keep things
simple.
Move all services with a broadcast permission across to this new
service. Note that we'll need to clear the service and organisation
cache after deploying this
we don't name letters based on the day we send them on, rather, the day
we create them on. If we process a letter for a second time for whatever
reason, even if it's a couple of days later, it'll still go in a folder
based on the created_at timestamp. There's still a slight confusion,
however - if the timestamp is after 5:30pm, the folder will be for the
day after. However, still the day after creation, so I think created_at
still makes the most sense.
Remove the term `sending_date` to try and make this relationship more
apparent.
`_now`? why would we ever use a different _now? instead say created_at,
because that's what it'll always be set to, even if we're replaying old
letters. We always set the folder name to when the letter was
created_at, or we might not know where to look to find it.
`dont_use_sending_date` doesn't really tell us what might happen if we
don't use it - the answer is we return an empty string. we ignore the
folder entirely. so lets call it that.
Also, remove use of freeze_gun in the tests, to prove that we don't use
the current time in any calculations. Also add an assert to a mock in
the get_pdf_for_templated_letter test, because we were mocking but not
asserting before, so the tests didn't fail when the function signature
changed.
We were determing the filename for precompiled letters before we had
checked if the letters were international. This meant that a letter
could have a filename indicating it was 2nd class, but once we had
sanitised the letter and checked the address we were setting the
notification to international.
This stopped these letters from being picked up to be sent to the DVLA,
since the filename and postage of the letter did not match.
We now regenerate the filename after the letter has been sanitised (and when
we know the postage) and use the updated filename when moving the letter
into the live PDF letters bucket.
It is not of the form
[[lat, long][lat, long]] as this would only hold a single polygon. It
instead needs to handle multiple polygons so instead is of the form
[[[lat, long][lat, long]]].
We don't want an exception while recording metrics to affect a user action. A KeyError exception was thrown today, that meant that a user say a 500, the action being performed was to download a document from the document-download-frontend app. By catching the error we prevent the user from seeing a 500 when a recording the connection metric fails.
Also catch the exception in the checkout event.
Our code was assuming that any notifications with `international` set to
`True` were text messages. It was then trying to look up delivery
information for a notification which wasn’t sent to a phone number,
causing an exception.
