In addition to the existing provider data, we also want return the number of
billable units (muliplied by the rate multiplier) that each SMS provider sent
this month. This will be used on the platform admin providers page.
Since we can no longer get all the information we need from the provider details
schema, this makes a new DAO function to get all the data for the endpoint.
This data includes service and org name, consent to research,
contact details and both intended and factual notifications
volumes by notification type.
This query was created to get data for a csv report for our
platform admins.
This PR adds a function to upsert (insert or update if exists) NotificationHistory all the rows from Notification that we are about to delete in the nightly task. This will happen just before the delete function. Since it is a upsert query the function can be called more than once.
This should allow us remove all the insert/updates to NotificationHistory.
However, there is a consern that this will double the length of time the tasks take. So do we do these upserts in a separate task or in the same one?
This relationship is via the `Organisation` now; we don’t use this
column to fudge a relationship based on the user’s email address and the
matching something in these columns.
The NHS is a special case because it’s not one organisation, but it does
have one consistent brand. So anyone working for an NHS organisation
should have their default branding set when they create a service, even
if we know nothing about their specific organisation.
The behaviour of stacking the version decorators does not work as
expected.
What you would expect to happen is that each decorator causes a history
row to be written for its respective model object.
What actually happens is that the first decorator adds history records
to the database session, but then causes the database session to commit.
This means that subsequent uses of this decorator find a clean session,
and therefore no changes to copy to their respective history tables.
This commit changes the intended use of the decorator so that it is only
used once per function, and accepts multiple definitions of what to
record history for. This way it can record everything that needs to go
into the history before doing anything that would risk flushing the
session.
This is fiendishly difficult error to discover on your own.
It’s caused when, during the creation of a row in the database, you run
a query on the same table, or a table that joins to the table you’re
inserting into. What I think is happening is that the database is forced
to flush the session before running the query in order to maintain
consistency.
This means that the session is clean by the time the history stuff comes
to do its work, so there’s nothing for it to copy into the history
table, and it silently fails to record history.
Hopefully raising an exception will:
- prevent this from failing silently
- save whoever comes across this issue in the future a whole load of
time
the create_nightly_notification_status task runs at 00:30am UK time,
however this means that in summer datetime.today() will return the
wrong date as the server (which runs on UTC) will run the task at
23:30 (populating the wrong row in the table).
fix this to use nice tz aware functions
The previous query was including all notifications regardless of notification_status. I don't think that's right, it shouldn't include things like technical-failure or validation-failed. Thoughts?
I also need to remove the query that's no longer being used.
also, it should default to last 7 days, not last 6 days. also change
count_inbound_sms to have the days passed in, so that it's more
explicit at the endpoint that we only return 7 days regardless of your
service's data retention
When creating a service it should inherit it’s organisation’s branding,
if that organisation has branding.
This wasn’t working because we were referring to the ID of the branding
when making the association, not the branding itself.
This sets the folder permissions for a user when adding them to a
service. If a user is being added to a service after accepting an
invite, we need to account for the possibility that the folders we are
trying to add them to have been deleted before they accepted the invite.
If the new folder has a parent folder, it inherits user permissions
from its parent. Else if the new folder is at root level, all users
will have a permission to view it.
When triggered by an admin request `dao_remove_user_from_service`
raised an IntegrityError since the user_to_service delete query was
issued before the folder permissions one, violating the foreign key
constraint on the folder permissions table.
For some reason this isn't caught by the tests in test_services_dao
that check that folder permissions are removed properly.
If we had organisations for GDS and Cabinet Office, then we’d always
want someone whose email address ends in `@cabinet-office.gov.uk` to
match to `cabinet-office.gov.uk` before matching to
`digital.cabinet-office.gov.uk`.
Sorting the list by shortest first addresses this.
Currently we have
- a thing in the database called an ‘organisation’ which we don’t use
- the idea of an organisation which we derive from the user’s email
address and is used to set the default branding for their service and
determine whether they’ve signed the MOU
We should make these two things into one thing, by storing everything
we know about an organisation against that organisation in the database.
This will be much less laborious than storing it in a YAML file that
needs a deploy every time it’s updated.
An organisation can now have:
- domains which we can use to automatically associate services with it
(eg anyone whose email address ends in `dwp.gsi.gov.uk` gets services
they create associated to the DWP organisation)
- default letter branding for any new services
- default email branding for any new services
code inspired by the delete notification code, but with some clean up
since we don't deal with different types etc, and only need to run the
query for services with inbound numbers
also, update tests.app.db.create_inbound_sms to create inbound numbers
and assign them to services to ensure the test db is always accurate
and reflects real world usage
Updated the endpoint for `.set_permissions` to update a user's folder
permissions as well as permissions for a service. User folder
permissions are optional for now, since Admin is not currently passing
this data through.
