Commit Graph

5169 Commits

Author SHA1 Message Date
Rebecca Law
c2b888a6af Make sure the date does not include time 2018-05-21 11:51:06 +01:00
Rebecca Law
4d8255b8d9 Merge branch 'master' into fix-ft-billing-migration 2018-05-21 11:06:31 +01:00
Rebecca Law
9fad623d91 Update the ft_billing query to only include billable notification status types.
Update test_provider_statistics dao - this is really irrelevant since the endpoint using the query is not being used. We have a PR coming to delete the unused code.
Update rate_multiplier to always be an integer
2018-05-21 10:56:16 +01:00
Rebecca Law
64aec4a64c Update the command to migrate data for ft_billing.
There is a massive performance improvement.
Another commit will need to alter the pk to include rate.
2018-05-18 17:22:51 +01:00
Rebecca Law
4b36fe0d9e Use created_at for the rate, there are some rows that do not have a sent_at set. 2018-05-17 13:37:55 +01:00
Rebecca Law
1969c83f57 Fix letter rate bug in migtion 2018-05-17 10:06:58 +01:00
Katie Smith
4f2412056a Merge pull request #1877 from alphagov/stop-update-for-previous-year
Refactor usage queries and stop rebuilding monthly_billing when not needed.
2018-05-16 14:11:23 +01:00
Rebecca Law
3c90c0ffcd Fix code style 2018-05-16 13:18:36 +01:00
Rebecca Law
eb083e30ed - Only rebuild current month for monthly_billing if today is in the current year.
- Change the usage queries to a union so that billing_units is correct for all notification types. Removing the business logic from the schema.
- Added tests for different fragment counts, rates and sheet counts.
2018-05-16 12:21:59 +01:00
Chris Hill-Scott
de95a28b7c Merge pull request #1876 from alphagov/fire-police-logos
Add letter logos for TWFRS and Thames Valley Police
2018-05-16 11:07:18 +01:00
Chris Hill-Scott
f6b8611b18 Add letter logos for TWFRS and Thames Valley Police 2018-05-16 11:00:05 +01:00
Rebecca Law
e3179dee99 Merge pull request #1875 from alphagov/fetch-data-from-right-table
Convert the day_start from a string to a datetime.
2018-05-15 14:06:33 +01:00
Rebecca Law
40d8f78b2b Convert the day_start from a string to a datetime. 2018-05-15 14:00:06 +01:00
Rebecca Law
61f1ac54ee Merge pull request #1874 from alphagov/fetch-data-from-right-table
Conditional table to fetch data for billing.
2018-05-15 13:12:30 +01:00
Rebecca Law
271ce6d76e Changed the update/insert to a postgres upsert to avoid concurrency issues. 2018-05-15 11:21:10 +01:00
Rebecca Law
3615f3d00f In order to re-run the create_nightly_billing for dates in the past, we added a condition on which table is used.
This will allow us to re-run nightly billing for those 2 days where it failed.
For the majority of time the query will run on Notiifcations.
2018-05-14 16:21:16 +01:00
Rebecca Law
a640605a7c Merge pull request #1867 from alphagov/use-ft-billing-for-usage
Update ft_billing table to use integer data types
2018-05-14 13:04:52 +01:00
Rebecca Law
828ebb6079 Fix test 2018-05-14 12:48:42 +01:00
Chris Waszczuk
0e9fca09fb Merge pull request #1855 from alphagov/inbound-sms-max-7-days
Only get inbound messages that are a maximum of 7 days old
2018-05-14 11:15:52 +01:00
Rebecca Law
56d7a7242b Fix merge conflict in db migration 2018-05-14 09:49:24 +01:00
Rebecca Law
2ae4cf18e8 Merge branch 'use-ft-billing-for-usage' of github.com:alphagov/notifications-api 2018-05-14 09:16:40 +01:00
Katie Smith
8e3466f717 Merge pull request #1866 from alphagov/ft-notification-status-migration
Add ft_notification_status table and data migration command
2018-05-14 09:06:08 +01:00
Katie Smith
13f3662051 Add command to populate data 2018-05-14 08:51:32 +01:00
Katie Smith
c9dc6f724d Add FactNotificationStatus model
Added the FactNotificationStatus model for the ft_notification_status table.
2018-05-14 08:51:32 +01:00
Katie Smith
bb9eb367b7 Migration to create the ft_notification_status table
Added the migration for the `ft_notification_status` table which will be
used to look up the counts for notifications by various different
fields.

The `job_id` column is non-nullable because we want to include it in the
composite primary key. For notifications which weren't part of a job,
the `job_id` will be set to `0000-00000-0000-00000` when populating the
data.
2018-05-14 08:51:32 +01:00
Rebecca Law
d98581cfe6 Added a new endpoint for yearly usage totals using ft_billing. 2018-05-11 16:25:16 +01:00
Leo Hemsted
a9bd189a24 Merge pull request #1868 from alphagov/pyup-update-requests-mock-1.4.0-to-1.5.0
Update requests-mock to 1.5.0
2018-05-11 14:07:28 +01:00
Leo Hemsted
9ac511e725 Merge pull request #1871 from alphagov/revert-1854-pyup-update-gunicorn-19.7.1-to-19.8.1
Revert "Update gunicorn to 19.8.1"
2018-05-11 11:29:32 +01:00
Leo Hemsted
307fd24072 Revert "Update gunicorn to 19.8.1" 2018-05-11 10:57:04 +01:00
pyup-bot
178a851d81 Update requests-mock from 1.4.0 to 1.5.0 2018-05-10 23:43:16 +01:00
Rebecca Law
99d1357c37 Fix the logging message in the nightly task 2018-05-10 17:13:38 +01:00
Rebecca Law
d50a0d4cb4 Migration script
Change to command.
2018-05-10 16:41:24 +01:00
Rebecca Law
8028f6cc28 We found that the reporting task failed twice because of integrity constraints.
This was because the rate_multiplier was being added as 1 and 1.0 which was not resolving to the same.
This updates the table to use Integrer.
Also changed the logging for the task.
2018-05-10 15:35:58 +01:00
chrisw
2abbb590ab Only get inbound messages that are a maximum of 7 days old 2018-05-10 14:00:55 +01:00
Rebecca Law
20129a2b14 Merge pull request #1854 from alphagov/pyup-update-gunicorn-19.7.1-to-19.8.1
Update gunicorn to 19.8.1
2018-05-10 12:01:15 +01:00
Rebecca Law
b4883fdd89 Merge pull request #1836 from alphagov/pyup-update-sqlalchemy-1.2.6-to-1.2.7
Update sqlalchemy to 1.2.7
2018-05-10 12:01:01 +01:00
Rebecca Law
87161b147f Merge pull request #1805 from alphagov/pyup-update-celery-3.1.25-to-3.1.26.post2
Update celery to 3.1.26.post2
2018-05-10 12:00:47 +01:00
Leo Hemsted
95475d130b Merge pull request #1865 from alphagov/separate-stats-endpoint
create new stats endpoint
2018-05-09 14:28:46 +01:00
Leo Hemsted
c30f13ea67 create new stats endpoint
the admin app currently calls get_detailed_service, which gets
notification stats, adds them on to the rest of the detailed service
endpoint, and returns them. However, the admin app then only looks at
the stats - it doesn't look at the rest of the service object.

This is called in a few high profile places - the dashboard, the
notification summary page, and when you send a job. By creating a
separate endpoint that ignores the rest of the service object (and no
marshmallow too!), the hope is that we'll improve some slowness we've
been seeing.

Note: The old detailed function will still need to stay - it's used
by get_services(detailed=True) for the platform admin page.
2018-05-09 12:02:56 +01:00
Rebecca Law
d52e65d89a Merge pull request #1863 from alphagov/use-ft-billing-for-usage
Use ft billing for usage
2018-05-08 17:00:56 +01:00
Rebecca Law
844f5b2e5d Added logging message if the comparison failed. 2018-05-08 16:32:28 +01:00
Rebecca Law
3e3b885bdc Realised that it's best to cast the Month as date. 2018-05-08 13:53:44 +01:00
Rebecca Law
dd113a8e86 Update the tests so that they ignore timezone in the month returned by query.
The timezone added is that of the database - locally a db will get timezone=GB, but the servers are using UTC (which is right)
This date is trucated to month, and is BST, here time and timezone is irrelevant.
2018-05-08 13:44:06 +01:00
Rebecca Law
fd6e5f39cf Changes as per requested from code review
Move the serialize method to the billing_schema
Update variable names
Improve tests
Fix bug in command
2018-05-08 12:09:29 +01:00
Rebecca Law
ea3523199a New endpoint to get monthly billing usage from the ft_billing table.
New command to compare the results of monthly billing to ft_billing.
2018-05-04 13:09:14 +01:00
Chris Hill-Scott
54d0e64582 Merge pull request #1853 from alphagov/pass-letter-date-to-template-preview
Pass date of letter to template preview
2018-05-04 10:52:47 +01:00
Chris Hill-Scott
da43780d10 Merge pull request #1859 from alphagov/pyup-update-pytest-mock-1.9.0-to-1.10.0
Update pytest-mock to 1.10.0
2018-05-04 10:27:19 +01:00
Chris Hill-Scott
f8f1507ab1 Merge pull request #1861 from alphagov/cadw
Add letter organisation for CADW
2018-05-03 12:08:45 +01:00
Chris Hill-Scott
545127966b Add letter organisation for CADW 2018-05-03 11:50:53 +01:00
Rebecca Law
18c2b9a56d Use better date function to get the first of each month.
Build the json object to return for the new endpoint.
2018-05-02 09:36:13 +01:00