Commit Graph

5159 Commits

Author SHA1 Message Date
Rebecca Law
eb083e30ed - Only rebuild current month for monthly_billing if today is in the current year.
- Change the usage queries to a union so that billing_units is correct for all notification types. Removing the business logic from the schema.
- Added tests for different fragment counts, rates and sheet counts.
2018-05-16 12:21:59 +01:00
Rebecca Law
e3179dee99 Merge pull request #1875 from alphagov/fetch-data-from-right-table
Convert the day_start from a string to a datetime.
2018-05-15 14:06:33 +01:00
Rebecca Law
40d8f78b2b Convert the day_start from a string to a datetime. 2018-05-15 14:00:06 +01:00
Rebecca Law
61f1ac54ee Merge pull request #1874 from alphagov/fetch-data-from-right-table
Conditional table to fetch data for billing.
2018-05-15 13:12:30 +01:00
Rebecca Law
271ce6d76e Changed the update/insert to a postgres upsert to avoid concurrency issues. 2018-05-15 11:21:10 +01:00
Rebecca Law
3615f3d00f In order to re-run the create_nightly_billing for dates in the past, we added a condition on which table is used.
This will allow us to re-run nightly billing for those 2 days where it failed.
For the majority of time the query will run on Notiifcations.
2018-05-14 16:21:16 +01:00
Rebecca Law
a640605a7c Merge pull request #1867 from alphagov/use-ft-billing-for-usage
Update ft_billing table to use integer data types
2018-05-14 13:04:52 +01:00
Rebecca Law
828ebb6079 Fix test 2018-05-14 12:48:42 +01:00
Chris Waszczuk
0e9fca09fb Merge pull request #1855 from alphagov/inbound-sms-max-7-days
Only get inbound messages that are a maximum of 7 days old
2018-05-14 11:15:52 +01:00
Rebecca Law
56d7a7242b Fix merge conflict in db migration 2018-05-14 09:49:24 +01:00
Rebecca Law
2ae4cf18e8 Merge branch 'use-ft-billing-for-usage' of github.com:alphagov/notifications-api 2018-05-14 09:16:40 +01:00
Katie Smith
8e3466f717 Merge pull request #1866 from alphagov/ft-notification-status-migration
Add ft_notification_status table and data migration command
2018-05-14 09:06:08 +01:00
Katie Smith
13f3662051 Add command to populate data 2018-05-14 08:51:32 +01:00
Katie Smith
c9dc6f724d Add FactNotificationStatus model
Added the FactNotificationStatus model for the ft_notification_status table.
2018-05-14 08:51:32 +01:00
Katie Smith
bb9eb367b7 Migration to create the ft_notification_status table
Added the migration for the `ft_notification_status` table which will be
used to look up the counts for notifications by various different
fields.

The `job_id` column is non-nullable because we want to include it in the
composite primary key. For notifications which weren't part of a job,
the `job_id` will be set to `0000-00000-0000-00000` when populating the
data.
2018-05-14 08:51:32 +01:00
Rebecca Law
d98581cfe6 Added a new endpoint for yearly usage totals using ft_billing. 2018-05-11 16:25:16 +01:00
Leo Hemsted
a9bd189a24 Merge pull request #1868 from alphagov/pyup-update-requests-mock-1.4.0-to-1.5.0
Update requests-mock to 1.5.0
2018-05-11 14:07:28 +01:00
Leo Hemsted
9ac511e725 Merge pull request #1871 from alphagov/revert-1854-pyup-update-gunicorn-19.7.1-to-19.8.1
Revert "Update gunicorn to 19.8.1"
2018-05-11 11:29:32 +01:00
Leo Hemsted
307fd24072 Revert "Update gunicorn to 19.8.1" 2018-05-11 10:57:04 +01:00
pyup-bot
178a851d81 Update requests-mock from 1.4.0 to 1.5.0 2018-05-10 23:43:16 +01:00
Rebecca Law
99d1357c37 Fix the logging message in the nightly task 2018-05-10 17:13:38 +01:00
Rebecca Law
d50a0d4cb4 Migration script
Change to command.
2018-05-10 16:41:24 +01:00
Rebecca Law
8028f6cc28 We found that the reporting task failed twice because of integrity constraints.
This was because the rate_multiplier was being added as 1 and 1.0 which was not resolving to the same.
This updates the table to use Integrer.
Also changed the logging for the task.
2018-05-10 15:35:58 +01:00
chrisw
2abbb590ab Only get inbound messages that are a maximum of 7 days old 2018-05-10 14:00:55 +01:00
Rebecca Law
20129a2b14 Merge pull request #1854 from alphagov/pyup-update-gunicorn-19.7.1-to-19.8.1
Update gunicorn to 19.8.1
2018-05-10 12:01:15 +01:00
Rebecca Law
b4883fdd89 Merge pull request #1836 from alphagov/pyup-update-sqlalchemy-1.2.6-to-1.2.7
Update sqlalchemy to 1.2.7
2018-05-10 12:01:01 +01:00
Rebecca Law
87161b147f Merge pull request #1805 from alphagov/pyup-update-celery-3.1.25-to-3.1.26.post2
Update celery to 3.1.26.post2
2018-05-10 12:00:47 +01:00
Leo Hemsted
95475d130b Merge pull request #1865 from alphagov/separate-stats-endpoint
create new stats endpoint
2018-05-09 14:28:46 +01:00
Leo Hemsted
c30f13ea67 create new stats endpoint
the admin app currently calls get_detailed_service, which gets
notification stats, adds them on to the rest of the detailed service
endpoint, and returns them. However, the admin app then only looks at
the stats - it doesn't look at the rest of the service object.

This is called in a few high profile places - the dashboard, the
notification summary page, and when you send a job. By creating a
separate endpoint that ignores the rest of the service object (and no
marshmallow too!), the hope is that we'll improve some slowness we've
been seeing.

Note: The old detailed function will still need to stay - it's used
by get_services(detailed=True) for the platform admin page.
2018-05-09 12:02:56 +01:00
Rebecca Law
d52e65d89a Merge pull request #1863 from alphagov/use-ft-billing-for-usage
Use ft billing for usage
2018-05-08 17:00:56 +01:00
Rebecca Law
844f5b2e5d Added logging message if the comparison failed. 2018-05-08 16:32:28 +01:00
Rebecca Law
3e3b885bdc Realised that it's best to cast the Month as date. 2018-05-08 13:53:44 +01:00
Rebecca Law
dd113a8e86 Update the tests so that they ignore timezone in the month returned by query.
The timezone added is that of the database - locally a db will get timezone=GB, but the servers are using UTC (which is right)
This date is trucated to month, and is BST, here time and timezone is irrelevant.
2018-05-08 13:44:06 +01:00
Rebecca Law
fd6e5f39cf Changes as per requested from code review
Move the serialize method to the billing_schema
Update variable names
Improve tests
Fix bug in command
2018-05-08 12:09:29 +01:00
Rebecca Law
ea3523199a New endpoint to get monthly billing usage from the ft_billing table.
New command to compare the results of monthly billing to ft_billing.
2018-05-04 13:09:14 +01:00
Chris Hill-Scott
54d0e64582 Merge pull request #1853 from alphagov/pass-letter-date-to-template-preview
Pass date of letter to template preview
2018-05-04 10:52:47 +01:00
Chris Hill-Scott
da43780d10 Merge pull request #1859 from alphagov/pyup-update-pytest-mock-1.9.0-to-1.10.0
Update pytest-mock to 1.10.0
2018-05-04 10:27:19 +01:00
Chris Hill-Scott
f8f1507ab1 Merge pull request #1861 from alphagov/cadw
Add letter organisation for CADW
2018-05-03 12:08:45 +01:00
Chris Hill-Scott
545127966b Add letter organisation for CADW 2018-05-03 11:50:53 +01:00
Rebecca Law
18c2b9a56d Use better date function to get the first of each month.
Build the json object to return for the new endpoint.
2018-05-02 09:36:13 +01:00
Rebecca Law
0fb9c1d318 Add notification_type to query 2018-05-02 09:36:13 +01:00
pyup-bot
c722e57657 Update pytest-mock from 1.9.0 to 1.10.0 2018-05-01 18:44:58 +01:00
Rebecca Law
8e2e94ca12 Merge pull request #1858 from alphagov/really-fix-it
Remove the truncate date function for the query.
2018-05-01 16:55:35 +01:00
Rebecca Law
a9c5ba7674 Remove the truncate date function for the query. 2018-05-01 16:48:34 +01:00
Alexey Bezhan
a2d5e77cf3 Merge pull request #1857 from alphagov/document-download-paas-manifest-env
Add document-download-api environment variables to PaaS instances
2018-05-01 15:37:36 +01:00
Alexey Bezhan
860304933c Add document-download-api environment variables to PaaS instances
Document Download keys have been added to the PaaS environment
credentials so we can use the manifest to set them on the app
instances.
2018-05-01 15:17:07 +01:00
Rebecca Law
b02a7030cc Merge pull request #1856 from alphagov/fix-date-bug
Dates are hard to get right
2018-05-01 15:00:49 +01:00
Katie Smith
6544666cd2 Merge pull request #1846 from alphagov/pyup-update-marshmallow-2.15.0-to-2.15.1
Update marshmallow to 2.15.1
2018-05-01 14:47:59 +01:00
Katie Smith
2619bc1002 Merge pull request #1843 from alphagov/pyup-update-notifications-python-client-4.8.1-to-4.8.2
Update notifications-python-client to 4.8.2
2018-05-01 14:47:48 +01:00
Rebecca Law
93c7ab6251 Because dates are confusing we need to fix this query once more.
The template statistics are returning 7 days inclusive, however the big numbers on the dashboard are for 8 days.
This PR fixes that.
2018-05-01 10:47:47 +01:00