April 1st is in British summer time, ie 1hr ahead of UTC. The database
stores everything in UTC, so for accurate comparisions we need to make
sure that `get_financial_year()` returns a UTC, datetime-aware
timestamp that is 1hr ahead of midnight.
This also means that when we group notifications by month, the months
need to be in BST. So the line between one year and another is actually
01:00 on April 1st, _not_ 00:00 on April 1st.
There’s no way we’ve found to do this in SQLAlchemy or raw Postgres,
especially because we don’t store the timestamps with a timezone in the
database.
So the grouping and summing of the notifications has to be done in
Python.
In order to invoice people we need to know how many text message
fragments they’ve sent per month.
This should be per (government) financial year, ie April 1st to April
1st because we’ll only ever show a page for one year (because the
250,000 allowance is topped up at the start of every financial year).
This commit only does the DAO bit, not the REST bit.
Developers need visibility of what their integration is doing within
the app. This includes notifications sent with a test key.
This commit adds an optional, defaults-to-false parameter to include
notifications sent from a test API key when getting notifications.
- get all notifications by service
- template usage
- most recently used templates
Ensures that the dashboard shows no test key data. Supplements: https://github.com/alphagov/notifications-api/pull/677 which excludes CSV data. This branches from that so is dependant.
We found that if the notifications were in created or pending they are not purged from notifications.
- New bulk update method to set all notificaitons with:
- a status = created|sending|pending to temporary-failure
- and is older then today minus SENDING_NOTIFICATIONS_TIMEOUT_PERIOD (in seconds)
- the scheduled task to timeout notifications use the new bulk update query.
- the task will be more efficient
- As before this is now driven from the notifications history table
- Removed from updates and create
- Signatures changes to removed unused params hits many files
- Also potential issue around rate limiting - we used to get the number sent per day from the stats table - which was a single row lookup, now we have to count this. This applies to EVERY API CALL. Probably not a good thing and should be addressed urgently.
- again these new come from the notifications history table
- We update this when we sent a notification, so removed from celery tasks
- tests removed also
- on create notification we updated the templates stats to record the usage.
- this is now based on notification history
- this update and associated tests are now removed,
Previously we kept a running total of job progress/success/failure on the job table. This causes contention, we now generate this data from notification history.
Removed these updates.
- groups by template Id and Day.
Returns count per day, template name, template id, template type, and day.
Ordered by day (desc) and template name (acc)
please ensure that any changes to notifications table happen through either dao_create_notification or dao_update_notification.
changed the notification status update triggered by the provider callbacks to ensure that sets updated_by and can update the history table.
also re-added the character_count so we can reconstruct billing data if needed.
triggered via calls in dao_create_notification and dao_update_notification - if you don't use those functions (eg update in bulk) you'll have to update the history table yourself!
set to 'normal' for all existing notifications, and all job notifications also created as 'normal' - so if your eg reporting service hits notify, it gets notifications created from both API calls and front-end csv jobs.
It seems like an oversight not to include the notification type in the notifcation.
When updating statistics a query to the template table is required to get the type, this update will mean that query does not have to happen.
If the notification ends up in the retry queue and the delivery app is restarted the notification will get sent twice.
This is because when the app is restarted another message will be in the retry queue as message available which is a
duplicate of the one in the queue that is a message in flight.
This should complete https://www.pivotaltracker.com/story/show/121844427
It’s going to be useful to see all the notifications for a job that are
failed/delivered/etc.
The API seems to support this behaviour already, but it doesn’t seem to
be tested.
This commit adds some testsfor the DAO and REST layers.
In order to set a message as temporary-failure, we check if it is in pending status first.
Otherwise a delivery receipt for failure is set to permanent failure.
If the notification has a status == sending then update the status otherwise do not update the status.
In other words do not change the status more than once.
