Commit Graph

6582 Commits

Author SHA1 Message Date
Leo Hemsted
48e96f253b ensure fetch_sms_free_allowance_remainder always returns
if there are no rows for a service in ft_billing, we should still
return their allowance (with 0 fragments used).

To do this, we need to build the query starting from AnnualBilling and
joining onto FactBilling, rather than the other way round. Also, we
need to account for the possibility of the sums being null by
coalescing them to 0
2019-08-30 12:18:40 +01:00
Leo Hemsted
b46eed1423 sort rest response by org_name, service_name
needed to re-arrange test results for this
2019-08-30 12:18:00 +01:00
Leo Hemsted
7811010e6c fix command name and get most recent data rather than oldest
fix query to order the right way

also remove debug prints that duplicate the formatted line below
2019-08-30 12:18:00 +01:00
Leo Hemsted
e4963ca0d3 make sure all letter costs are cast from decimal to float 2019-08-30 12:18:00 +01:00
Leo Hemsted
99a008e908 explicitly join annual_billing and remove reference to service.id
service.id isn't used in this query (both tables already have
service_id), and explicitly joining makes what we're doing more obvious
2019-08-30 12:18:00 +01:00
Leo Hemsted
741d75c3a9 use func.greatest rather than case statement
simplifies the query quite a bit
2019-08-30 12:18:00 +01:00
Leo Hemsted
608738ca67 remove unneccessary str cast 2019-08-30 12:18:00 +01:00
Leo Hemsted
028b81b787 simplify validate_date_range_is_within_a_financial_year
we can use the get_financial_year_for_datetime function
to massively simplify this
2019-08-30 12:18:00 +01:00
Leo Hemsted
b7e8f1baa2 rename which_financial_year to get_financial_year_for_datetime
also fix bug on the very second of rollover, march 31st 23:00:00, and
add tests
2019-08-30 12:18:00 +01:00
Rebecca Law
2abcda47a3 Remove print 2019-08-30 12:17:59 +01:00
Rebecca Law
1c94d6d24a Added command to populate data for annual billing based on last years values. 2019-08-30 12:17:59 +01:00
Rebecca Law
cb7fff6100 Add endpoint to return structured data 2019-08-30 12:17:59 +01:00
Rebecca Law
4c36e22e93 [WIP]
fire alarm is going off - better save :)
2019-08-30 12:17:59 +01:00
Rebecca Law
1a5763feb6 Compare result tuple rather than each element in tuple (just to make it a little shorter)
Change order of columns
2019-08-30 12:17:59 +01:00
Rebecca Law
3b16950aee Final touches on tests and query for itemised letter billing data. 2019-08-30 12:17:59 +01:00
Rebecca Law
236d111c22 Update queries for data model change (removal of organisation_to_service)
New query to get letter breakdown.
2019-08-30 12:17:59 +01:00
Leo Hemsted
d3449c37da Merge pull request #2593 from alphagov/split-up-nightly-tasks
split up reporting tasks in to separate tasks per day
2019-08-21 15:58:17 +01:00
Leo Hemsted
1d9fd775d3 move delete tasks to 4am
just to make sure they definitely run after the create tasks
2019-08-21 11:15:49 +01:00
Rebecca Law
d40cbf8d74 Merge pull request #2594 from alphagov/update-alert-for-letters-in-created
Update alert for letters in created status
2019-08-20 13:49:27 +01:00
Pea (Malgorzata Tyczynska)
009406898f Merge pull request #2596 from alphagov/lighter-serialize-for-users
Leaner serialize for user model for displaying list of users
2019-08-20 11:40:07 +01:00
Leo Hemsted
8f13697cf1 Revert "trigger nightly delete tasks from the create notification status task"
This reverts commit 58f24a0a83.
2019-08-19 16:06:25 +01:00
Leo Hemsted
36dd750637 split up reporting tasks in to separate tasks per day
to try and speed up overall time by parallelising
2019-08-19 16:06:25 +01:00
Rebecca Law
ad389e7252 Merge pull request #2591 from alphagov/remove-org-to-service-table
Remove the organisation_to_service table.
2019-08-19 15:15:00 +01:00
Rebecca Law
8fdf700b90 Remove print statement 2019-08-19 14:51:21 +01:00
Pea Tyczynska
b2f811662d Leaner serialize for user model for displaying list of users 2019-08-19 13:31:29 +01:00
Pea (Malgorzata Tyczynska)
25819d0690 Merge pull request #2595 from alphagov/serialize-for-lighter-query
Use serialize instead of marshmallow schema for lighter query
2019-08-19 11:40:31 +01:00
Pea Tyczynska
252be67436 Use serialize instead of marshmallow schema for lighter query
Find services by name feature was timing out for big collections
of results.
2019-08-16 17:30:04 +01:00
Pea (Malgorzata Tyczynska)
7e43b5bbc3 Merge pull request #2587 from alphagov/find_services_by_partial_name
Find services by partial name
2019-08-16 11:21:19 +01:00
Rebecca Law
f097abe82b Change the query to get the notifications for the check_templated_letter_state.
Now looking at the updated_at date, we are getting the alert if the notification was created_at:17:29 updated to created status at 17:30, so the letter is in the next days bucket.

Not sure if I want to make this change, there isn't an index on updated_at, so the query might be slow.
2019-08-16 10:37:51 +01:00
Leo Hemsted
92d78956be Merge pull request #2592 from alphagov/reporting-worker
Add reporting worker
2019-08-15 17:22:27 +01:00
Leo Hemsted
e5c76ffda7 reduce days to process from 10 to 4
to try and speed it up temporarily.
2019-08-15 17:06:38 +01:00
Leo Hemsted
3a0bf2b23e Add reporting worker
also remove references to unused statistics queue
2019-08-15 16:42:15 +01:00
Leo Hemsted
689d76d820 Merge pull request #2590 from alphagov/delete-tasks
trigger nightly delete tasks from the create notification status task
2019-08-15 15:39:00 +01:00
Rebecca Law
b37de7785c Remove the organisation_to_service table.
This table is no longer used or referenced in the code.
We can remove this mapping table now that the organisation to service relationship is handled by the foreign key in Services.
2019-08-15 15:17:53 +01:00
Rebecca Law
5ae77efbe0 Merge pull request #2586 from alphagov/add-org-id-to-services-2
Add organisation_id to Service.
2019-08-15 10:03:52 +01:00
Leo Hemsted
58f24a0a83 trigger nightly delete tasks from the create notification status task
the nightly tasks need to run after the create nightly notification
status task - so that test notifications are still there to record
stats for, and to stop the risk of deleting notificaitons part-way
through recording stats for them.
2019-08-14 18:04:45 +01:00
Rebecca Law
fdee6cbfb2 As per PR comments:
- Remove Organisation.services
- added backref='services' to Services.organisation relationship and removed redundant foreign_keys argument.
2019-08-14 13:23:20 +01:00
Sakis
c6d6c1ac7d Merge pull request #2588 from alphagov/enable-statsd-exporter-production
Enable statsd exporter for production
2019-08-14 12:38:33 +01:00
Athanasios Voutsadakis
02f50975b9 Run make freeze-requirements 2019-08-14 11:52:32 +01:00
Athanasios Voutsadakis
cd936d2e71 Enable statsd exporter for production
Also bump the utils version to include a fix on the error handling logic
when we fail to send a metric.
2019-08-14 11:42:13 +01:00
Rebecca Law
c4127d0508 Improve the data migration and downgrade script. 2019-08-14 11:01:22 +01:00
Rebecca Law
55dc7184cc Add organisation_id to Service.
This is the second commit in the series to add organisation_id to Service.
- Data migration to update services.organisation_id from data in organisation_to_service
 (The rollback will lose any updates to organisation unless the script is updated to set organistion_to_service from service.organisation_id )
- Update Service.organisation relationship to a ForeignKey relationship to Organisation.
- Update Organisation.services to a backref relationship to Service.
2019-08-13 17:25:30 +01:00
Pea Tyczynska
52a1316a7e Find services by partial name 2019-08-13 17:20:37 +01:00
Leo Hemsted
b07944d9f5 Merge pull request #2585 from alphagov/config-typo
fix typo in config
2019-08-13 15:28:22 +01:00
Leo Hemsted
7b8028d03f fix typo in config
was a `,`, not a `:`, so 'options' was a set rather than a dictionary.
2019-08-13 15:19:28 +01:00
Rebecca Law
515d6602c1 Merge pull request #2583 from alphagov/add-org-id-to-services
Add org id to services
2019-08-13 11:05:29 +01:00
Rebecca Law
cc7008c129 Merge pull request #2584 from alphagov/fix-error-for-tech-failure
Update NotificationTechnicalFailureException
2019-08-13 09:59:57 +01:00
Rebecca Law
ae1bc54f9e Update NotificationTechnicalFailureException
- Change the NotificationTechnicalFailureException so that it only inherits from Exception.
- The notify_celery task should create the logging message on failure.
- Fix unit tests
- Remove named parameter when raising exception.
2019-08-12 16:51:39 +01:00
Rebecca Law
12f305bf86 - Migration to add Service.organisation_id
- Add oranisation_id to Service data model.
- Update methods to create service and associate service to organisation to set the organisation_id on the Service.
- Create the missing test, if the service user email matches a domain for an organisation then associate the service to the organisation and inherit crown and organisation_type from the organisation.
2019-08-12 16:03:02 +01:00
Katie Smith
6ea849298e Merge pull request #2580 from alphagov/set-bad-email-addresses-to-perm-fail
Set bad email addresses to permanent-failure
2019-08-12 16:03:01 +01:00