Test notifications are only stored in the notification table, not the
notification_history table. We were deciding which table to query for
the platform admin stats based on the start date passed to the DAO
function. This means that if the start date given was more than 7 days
ago and the end date was within the last 7 days, test notifications were
not being returned. We now use the end date to choose which table to
query which means that test notifications always get returned when they
should.
Created a platform-stats blueprint and moved the new platform stats
endpoint to the new blueprint (it was previously in the service
blueprint). Since the original platform stats route and the new platform
stats route are now in different blueprints, their view functions can
have the same name without any issues.
Moved the `fetch_new_aggregate_stats_by_date_range_for_all_services`
DAO function from the services DAO to the Notifications DAO since this
function queries the `notification` and `notification_history` tables.
Also added a test to check that the data returned from the function
takes BST into account.
Added in a new endpoint and DAO function to provide the data for the new
platform admin statistics page. The DAO method gets different data from
the Notifications / NotificationHistory table and also groups it differently.
The old endpoint has not been deleted yet to allow the numbers on the
old and new pages to be compared.
The caseworking view is going to have a page which displays emails and
text messages combined together.
In order for the search to work on this page the user needs to be able
to search for an email or a text message. This commit makes it guess
what to search for when the `notification_type` isn’t known (basically
by saying ‘if the search term is only digits they’re probably looking
searching by phone number’).
When handling the complaint we don't want to throw an exception if the message is missing fields. Only log an exception if we are unable to tie a complaint to a notification.
`created_at` was added previously and made nullable temporarily. This
commit now populates the column, ensures that it will always have a
value, and makes `created_at` non-nullable.
`@service_blueprint.route('/<uuid:service_id>/fragment/aggregate_statistics')`
is not being used anywhere, so has been removed. The `provider_statistics_dao`
can also be removed, since the function this contained was only used for
the endpoint.
Update test_provider_statistics dao - this is really irrelevant since the endpoint using the query is not being used. We have a PR coming to delete the unused code.
Update rate_multiplier to always be an integer
- Change the usage queries to a union so that billing_units is correct for all notification types. Removing the business logic from the schema.
- Added tests for different fragment counts, rates and sheet counts.
This was because the rate_multiplier was being added as 1 and 1.0 which was not resolving to the same.
This updates the table to use Integrer.
Also changed the logging for the task.
Added a new DAO function which archives letter contact blocks by
setting archived to True. This raises an ArchiveValidationError if
trying to archive the default letter block for a service or the default
letter contact block for a template.
Added a new endpoint for archiving letter contact blocks.
Added a new DAO function which archives SMS senders by setting
archived to True. This raises an ArchiveValidationError, if
trying to archive a default SMS sender or an inbound number.
Added a new endpoint for archiving SMS senders.
Added a new DAO function which archives email reply_to addresses by
setting archived to True. This raises a new type of error, an
ArchiveValidationError, if trying to archive a default reply_to address.
Added a new endpoint for archiving email reply_to addresses.
Updated the DAO methods which return a single SMS sender and all SMS senders
to only return the non-archived senders. Changed the error raised in the Admin
interface from a SQLAlchemyError to a BadRequestError.
Updated the DAO methods which return a single email reply_to address and
all reply_to addresses to only return the non-archived addresses.
Changed the type of error that gets raised when using the Admin
interface to be BadRequestError instead of a SQLAlchemyError.
to get the data for a day can be reasonably slow (a few hundred
milliseconds), and if someone's viewing a service with no activity we
don't want to do that query seven times every two seconds. So if there
is no data in redis, when we get the data out of the database, we
should put it in redis so we can just grab it from there next time.
This'll happen in two cases:
* redis data is deleted
* the service sent no messages that day
additionally, make sure that we convert nicely from redis' return
values (ascii strings) to unicode keys and integer counts.
New redis keys are partitioned per service per day. New process is as
follows:
* require a count of days to filter by. Currently admin always gives 7.
* for each day, check and see if there's anything in redis. There won't
be if either a) redis is/was down or b) the service didn't send any
notifications that day
- if there isn't, go to the database and get a count out.
* combine all these stats together
* get the names/template types etc out of the DB at the end.
it's used in a few places - it should definitely know what timezones
are and return datetimes rather than dates, which are hard to work with
in terms of figuring out how tz aware they are.
we should be very careful with when we get data from
NotificationHistory - this should probably only be from scheduled
tasks. `dao_get_template_usage` is only called from the template
statistics rest endpoint, so shouldn't ever hit template history.
also, moved tests out to new file to break up the 2k test file a bit
