Commit Graph

2958 Commits

Author SHA1 Message Date
Rebecca Law
b405b131f6 Merge pull request #1886 from alphagov/ft_billing-migration-fixes
Fix how the letter rates are selected
2018-05-23 11:07:59 +01:00
Rebecca Law
1567e9bfe5 In order to correctly select the rate for letters you need the billable_unit or page count to use to match on the letter_rates.
A new column is added to the query to populate ft_billing which is only populated for letter.
2018-05-22 17:35:21 +01:00
Katie Smith
f6039d51e0 Add timestamp columns to ft_billing table
Added `created_at` and `updated_at` to the `ft_billing` table - having these
columns makes it easier to track down issues with the data in this table.

`created_at` is nullable initially, but will be changed to non-nullable
once the column is populated and the DAO etc. have been updated.
2018-05-22 14:32:49 +01:00
Katie Smith
0d4fb81235 Delete ProviderStatistics model
This is not being used anywhere, so can be deleted safely.
2018-05-21 15:03:39 +01:00
Katie Smith
71796311fa Delete unused aggregate_statistics endpoint
`@service_blueprint.route('/<uuid:service_id>/fragment/aggregate_statistics')`
is not being used anywhere, so has been removed. The `provider_statistics_dao`
can also be removed, since the function this contained was only used for
the endpoint.
2018-05-21 15:03:39 +01:00
Rebecca Law
e5584348ef Add rate to the primary key of ft_billing.
Use the primary key constraint in the upsert statement (easier to read than listing all the columns)
2018-05-21 14:38:25 +01:00
Rebecca Law
e4e253ce44 Remove where clause for notification_status 2018-05-21 11:54:49 +01:00
Rebecca Law
c2b888a6af Make sure the date does not include time 2018-05-21 11:51:06 +01:00
Rebecca Law
4d8255b8d9 Merge branch 'master' into fix-ft-billing-migration 2018-05-21 11:06:31 +01:00
Rebecca Law
9fad623d91 Update the ft_billing query to only include billable notification status types.
Update test_provider_statistics dao - this is really irrelevant since the endpoint using the query is not being used. We have a PR coming to delete the unused code.
Update rate_multiplier to always be an integer
2018-05-21 10:56:16 +01:00
Rebecca Law
64aec4a64c Update the command to migrate data for ft_billing.
There is a massive performance improvement.
Another commit will need to alter the pk to include rate.
2018-05-18 17:22:51 +01:00
Rebecca Law
4b36fe0d9e Use created_at for the rate, there are some rows that do not have a sent_at set. 2018-05-17 13:37:55 +01:00
Rebecca Law
1969c83f57 Fix letter rate bug in migtion 2018-05-17 10:06:58 +01:00
Rebecca Law
3c90c0ffcd Fix code style 2018-05-16 13:18:36 +01:00
Rebecca Law
eb083e30ed - Only rebuild current month for monthly_billing if today is in the current year.
- Change the usage queries to a union so that billing_units is correct for all notification types. Removing the business logic from the schema.
- Added tests for different fragment counts, rates and sheet counts.
2018-05-16 12:21:59 +01:00
Rebecca Law
40d8f78b2b Convert the day_start from a string to a datetime. 2018-05-15 14:00:06 +01:00
Rebecca Law
271ce6d76e Changed the update/insert to a postgres upsert to avoid concurrency issues. 2018-05-15 11:21:10 +01:00
Rebecca Law
3615f3d00f In order to re-run the create_nightly_billing for dates in the past, we added a condition on which table is used.
This will allow us to re-run nightly billing for those 2 days where it failed.
For the majority of time the query will run on Notiifcations.
2018-05-14 16:21:16 +01:00
Rebecca Law
a640605a7c Merge pull request #1867 from alphagov/use-ft-billing-for-usage
Update ft_billing table to use integer data types
2018-05-14 13:04:52 +01:00
Chris Waszczuk
0e9fca09fb Merge pull request #1855 from alphagov/inbound-sms-max-7-days
Only get inbound messages that are a maximum of 7 days old
2018-05-14 11:15:52 +01:00
Rebecca Law
2ae4cf18e8 Merge branch 'use-ft-billing-for-usage' of github.com:alphagov/notifications-api 2018-05-14 09:16:40 +01:00
Katie Smith
13f3662051 Add command to populate data 2018-05-14 08:51:32 +01:00
Katie Smith
c9dc6f724d Add FactNotificationStatus model
Added the FactNotificationStatus model for the ft_notification_status table.
2018-05-14 08:51:32 +01:00
Rebecca Law
d98581cfe6 Added a new endpoint for yearly usage totals using ft_billing. 2018-05-11 16:25:16 +01:00
Rebecca Law
99d1357c37 Fix the logging message in the nightly task 2018-05-10 17:13:38 +01:00
Rebecca Law
d50a0d4cb4 Migration script
Change to command.
2018-05-10 16:41:24 +01:00
Rebecca Law
8028f6cc28 We found that the reporting task failed twice because of integrity constraints.
This was because the rate_multiplier was being added as 1 and 1.0 which was not resolving to the same.
This updates the table to use Integrer.
Also changed the logging for the task.
2018-05-10 15:35:58 +01:00
chrisw
2abbb590ab Only get inbound messages that are a maximum of 7 days old 2018-05-10 14:00:55 +01:00
Leo Hemsted
95475d130b Merge pull request #1865 from alphagov/separate-stats-endpoint
create new stats endpoint
2018-05-09 14:28:46 +01:00
Leo Hemsted
c30f13ea67 create new stats endpoint
the admin app currently calls get_detailed_service, which gets
notification stats, adds them on to the rest of the detailed service
endpoint, and returns them. However, the admin app then only looks at
the stats - it doesn't look at the rest of the service object.

This is called in a few high profile places - the dashboard, the
notification summary page, and when you send a job. By creating a
separate endpoint that ignores the rest of the service object (and no
marshmallow too!), the hope is that we'll improve some slowness we've
been seeing.

Note: The old detailed function will still need to stay - it's used
by get_services(detailed=True) for the platform admin page.
2018-05-09 12:02:56 +01:00
Rebecca Law
d52e65d89a Merge pull request #1863 from alphagov/use-ft-billing-for-usage
Use ft billing for usage
2018-05-08 17:00:56 +01:00
Rebecca Law
844f5b2e5d Added logging message if the comparison failed. 2018-05-08 16:32:28 +01:00
Rebecca Law
3e3b885bdc Realised that it's best to cast the Month as date. 2018-05-08 13:53:44 +01:00
Rebecca Law
fd6e5f39cf Changes as per requested from code review
Move the serialize method to the billing_schema
Update variable names
Improve tests
Fix bug in command
2018-05-08 12:09:29 +01:00
Rebecca Law
ea3523199a New endpoint to get monthly billing usage from the ft_billing table.
New command to compare the results of monthly billing to ft_billing.
2018-05-04 13:09:14 +01:00
Chris Hill-Scott
54d0e64582 Merge pull request #1853 from alphagov/pass-letter-date-to-template-preview
Pass date of letter to template preview
2018-05-04 10:52:47 +01:00
Rebecca Law
18c2b9a56d Use better date function to get the first of each month.
Build the json object to return for the new endpoint.
2018-05-02 09:36:13 +01:00
Rebecca Law
0fb9c1d318 Add notification_type to query 2018-05-02 09:36:13 +01:00
Rebecca Law
a9c5ba7674 Remove the truncate date function for the query. 2018-05-01 16:48:34 +01:00
Rebecca Law
93c7ab6251 Because dates are confusing we need to fix this query once more.
The template statistics are returning 7 days inclusive, however the big numbers on the dashboard are for 8 days.
This PR fixes that.
2018-05-01 10:47:47 +01:00
Chris Hill-Scott
8ae800b117 Pass date of letter to template preview
Otherwise all letters will show the current date.

Also beefed up the tests around this part of the code a bit.
2018-04-30 15:47:49 +01:00
Katie Smith
f2d4bc795e Add DAO function and endpoint for archiving letter contact blocks
Added a new DAO function which archives letter contact blocks by
setting archived to True. This raises an ArchiveValidationError if
trying to archive the default letter block for a service or the default
letter contact block for a template.

Added a new endpoint for archiving letter contact blocks.
2018-04-30 15:25:17 +01:00
Katie Smith
472b86f3f4 Add DAO function and endpoint for archiving SMS senders
Added a new DAO function which archives SMS senders by setting
archived to True. This raises an ArchiveValidationError, if
trying to archive a default SMS sender or an inbound number.

Added a new endpoint for archiving SMS senders.
2018-04-30 15:25:17 +01:00
Katie Smith
5f43fe23a7 Add DAO function and endpoint for archiving email reply_to addresses
Added a new DAO function which archives email reply_to addresses by
setting archived to True. This raises a new type of error, an
ArchiveValidationError, if trying to archive a default reply_to address.

Added a new endpoint for archiving email reply_to addresses.
2018-04-30 15:25:17 +01:00
Katie Smith
663021e494 Only return active SMS senders
Updated the DAO methods which return a single SMS sender and all SMS senders
to only return the non-archived senders. Changed the error raised in the Admin
interface from a SQLAlchemyError to a BadRequestError.
2018-04-30 15:25:17 +01:00
Katie Smith
f810daa3c5 Only return non-archived letter contact blocks
Changed DAO functions which return one letter contact block and all
letter contact blocks for a service to only return non-archived ones.
2018-04-30 15:25:17 +01:00
Katie Smith
a57b2f50c2 Only return non-archived email reply_to addresses
Updated the DAO methods which return a single email reply_to address and
all reply_to addresses to only return the non-archived addresses.

Changed the type of error that gets raised when using the Admin
interface to be BadRequestError instead of a SQLAlchemyError.
2018-04-30 15:25:17 +01:00
Katie Smith
be28f2e2de Add 'archived' column to the 3 reply_to models
Added a new boolean column, `archived`,  with a default of False to the
three models which are used to specify the 'reply to' address for
notifications:
 * ServiceEmailReplyTo
 * ServiceSmsSender
 * ServiceLetterContact
2018-04-30 15:25:17 +01:00
Leo Hemsted
238e85ea9e Merge pull request #1833 from alphagov/read-redis
Read template usage stats from new redis keys
2018-04-30 15:15:16 +01:00
Leo Hemsted
e35c1bde77 ensure tests have test_ prefix
also remove unused code branch

thanks, coverage!
2018-04-30 13:23:02 +01:00