Commit Graph

274 Commits

Author SHA1 Message Date
Rebecca Law
e4e253ce44 Remove where clause for notification_status 2018-05-21 11:54:49 +01:00
Rebecca Law
9fad623d91 Update the ft_billing query to only include billable notification status types.
Update test_provider_statistics dao - this is really irrelevant since the endpoint using the query is not being used. We have a PR coming to delete the unused code.
Update rate_multiplier to always be an integer
2018-05-21 10:56:16 +01:00
Rebecca Law
64aec4a64c Update the command to migrate data for ft_billing.
There is a massive performance improvement.
Another commit will need to alter the pk to include rate.
2018-05-18 17:22:51 +01:00
Rebecca Law
4b36fe0d9e Use created_at for the rate, there are some rows that do not have a sent_at set. 2018-05-17 13:37:55 +01:00
Rebecca Law
1969c83f57 Fix letter rate bug in migtion 2018-05-17 10:06:58 +01:00
Rebecca Law
40d8f78b2b Convert the day_start from a string to a datetime. 2018-05-15 14:00:06 +01:00
Rebecca Law
2ae4cf18e8 Merge branch 'use-ft-billing-for-usage' of github.com:alphagov/notifications-api 2018-05-14 09:16:40 +01:00
Katie Smith
13f3662051 Add command to populate data 2018-05-14 08:51:32 +01:00
Rebecca Law
d50a0d4cb4 Migration script
Change to command.
2018-05-10 16:41:24 +01:00
Rebecca Law
844f5b2e5d Added logging message if the comparison failed. 2018-05-08 16:32:28 +01:00
Rebecca Law
fd6e5f39cf Changes as per requested from code review
Move the serialize method to the billing_schema
Update variable names
Improve tests
Fix bug in command
2018-05-08 12:09:29 +01:00
Rebecca Law
ea3523199a New endpoint to get monthly billing usage from the ft_billing table.
New command to compare the results of monthly billing to ft_billing.
2018-05-04 13:09:14 +01:00
Rebecca Law
0520dfdb60 Added a command to rebuild the data for a given service_id and date 2018-04-26 15:39:04 +01:00
Rebecca Law
a91b46dfa9 Added the else so the entire table does not rebuild. 2018-04-23 16:16:41 +01:00
Rebecca Law
12796051f5 We need to rebuild the monthly billing for one service for the month of March.
This change makes that possible without affecting any other data in the table.
2018-04-23 13:53:47 +01:00
Leo Hemsted
ac0b2d79a2 sys.exit rather than throwing an assertion error
Assertions should only be used in tests - they can be disabled at
runtime by setting the python flag -O (though I don't believe we use
that flag under normal circumstances).

also clean up test asserts - mock_redis is the redis object, so its
`called` property will always be false, because we never say
`redis_store()`. Rather, we should use the `mock_calls` property to
see all calls to all of its children
2018-04-09 11:45:43 +01:00
Leo Hemsted
ba612b0f5b make sure redis command filters dates properly
also write the tests that i should have written a while ago for this
2018-04-05 15:11:29 +01:00
Leo Hemsted
6e554188bd add command to backfill template usage
The command takes a service id and a day, grabs the historical data for
that day (potentially out of notification_history), and pops it in
redis (for eight days, same as if it were written to manually).

also, prefix template usage key with "service" to make clear that it's
a service id, and not an individual template id.
2018-04-03 16:14:47 +01:00
venusbb
2262db4f62 Database tweak and BST bug fix 2018-03-27 10:37:56 +01:00
Rebecca Law
fc21121764 Check input is a UUID 2018-03-23 16:11:43 +00:00
Rebecca Law
6dad9b43db New command to create-pdf-letter task for a given notification id.
After a notificaiton is created we create a task to create the pdf and save it to S3,
if for some reason that task does not run we are left with notifications that are not sent.
This should not happen, but if it does we have a way to continue sending the letter.
2018-03-23 15:47:01 +00:00
venusbb
444365faa5 add statd 2018-03-22 17:17:03 +00:00
venusbb
9aa2536997 use sql parametrize rather than python format 2018-03-21 17:04:51 +00:00
venusbb
4b25654cbf update record rather than ignore when duplicate 2018-03-21 15:37:49 +00:00
venusbb
b9953dd005 Command to migrate data to ft_billing 2018-03-21 15:21:16 +00:00
Rebecca Law
a3d04ca672 Improve log message 2018-03-09 12:01:08 +00:00
Rebecca Law
e95740a6b5 There was a problem with the worker that was sending the service updates for the notification.
The problem has been resolved but we need to replay the messages that are missing. We have been sent a file containing client_references for all the notificaitons that the service would needs updates for.
2018-03-09 11:06:47 +00:00
Chris Hill-Scott
5db3eef8f2 Add warning to Performance Platform command 2018-03-06 15:42:38 +00:00
Chris Hill-Scott
ca167206d5 Add command to backfill Performance Platform totals
We don’t have any way of playing back the totals we send to performance
platform.

This commit copies the command used to backfill the processing time and
adapts it to backfill the totals instead. Under the hood it uses the
same code that we use in the scheduled tasks to update performance
platform on a daily basis. I had to modify this code to take a `day`
argument because it was hardcoded to only work for ‘yesterday’.
2018-03-05 17:02:33 +00:00
Rebecca Law
28925da575 Fix codestyle 2018-02-21 11:52:52 +00:00
Rebecca Law
9207325501 Added a message about the file format. 2018-02-21 10:30:34 +00:00
Rebecca Law
953a545aea Added a command to insert inbound numbers from file.
Expected file format is one number per line, each phone number should be 07... not 447..
2018-02-21 10:00:17 +00:00
Rebecca Law
995d70740a Add letter type when getting billing data for financial year 2017-12-18 10:40:13 +00:00
Rebecca Law
e0e64d51d5 [WIP]
Fix the query to count rather than sum the billing units.
Need to fix the query that returns the monhtly billing, there is only one row but there should be two if there are two rates.
2017-12-15 17:29:32 +00:00
Rebecca Law
012d80003a Update montly billing with letters 2017-12-14 17:17:05 +00:00
Leo Hemsted
a4e2a40134 update dao tests to pass in free sms fragment limit, remove command 2017-12-06 14:45:43 +00:00
Leo Hemsted
5b7118973e take advantage of click's type validation/coercion
(saves us having to write the stuff ourselves). Also adds a small
click plugin to do datetime parsing.

Sample output:

```
[leohemsted:~/dev/api]$ flask command create_provider_rates --help
Usage: flask command create_provider_rates [OPTIONS]

  Backfill rates for a given provider

Options:
  -p, --provider_name [mmg|firetext|ses]
                                  [required]
  -c, --cost FLOAT                Cost (pence) per message including decimals
                                  [required]
  -d, --valid_from DATE           [required]
  --help                          Show this message and exit.
[leohemsted:~/dev/api]$ flask command create_provider_rates -p ses -c 1.234 -d invalid
Usage: flask command create_provider_rates [OPTIONS]

Error: Invalid value for "-d" / "--valid_from": Could not parse datetime string "invalid" formatted as %Y-%m-%dT%H:%M:%S
```
2017-11-28 12:50:29 +00:00
Leo Hemsted
721202b44f use more click functionality to reduce our own codes' complexity 2017-11-28 10:35:46 +00:00
Leo Hemsted
4c14e3279f ensure the app context is included in every single flask command 2017-11-24 12:01:28 +00:00
Leo Hemsted
cd6f85281c any apps that use current_app need to be in a context
to achieve this, the decorator flask.cli.with_appcontext is used. Not sure why it
hasn't applied by default /shrug
2017-11-23 17:04:58 +00:00
Leo Hemsted
e2e9db8c97 add docstrings for all custom commands 2017-11-23 17:04:58 +00:00
Leo Hemsted
9f56dccdee Remove flask-script, move commands to click
click (http://click.pocoo.org/) is used by flask to run its cli args.
In removing flask_script (it's unmaintained), we had to migrate all our
commands to use click. This is a change for the better in my eyes - you
don't need to define the command in several places, and it makes
managing options a bit easier.

View diff with whitespace turned off unless you're a masochist.
2017-11-23 17:04:58 +00:00
Rebecca Law
c8a210afd1 Added a command to re-run the build_dvla_file for a given job id.
There was an instance where the file failed to create because the address in the personalisation had punctuation which caused the address lines to merge into one.
Utils has been updated to fix that problem, this task will allow us to re-run the task so that the notifications can be sent.
2017-10-31 15:20:14 +00:00
venusbb
b2fa7cdd83 create new table annual_billing 2017-10-18 15:35:51 +01:00
Venus Bailey
9b60d69931 Revert "Revert "[#151837054] Add new column free_sms_fragment_limit in the Services table"" 2017-10-16 16:24:34 +01:00
Venus Bailey
616a6f8ef8 Revert "[#151837054] Add new column free_sms_fragment_limit in the Services table" 2017-10-16 12:43:05 +01:00
Katie Smith
9322ed0ca3 Add command to populate the free_sms_fragment_limit
Added a one-off command to set values of the free_sms_fragment_limit in
the services table and the services_history table to the default of
250000.
2017-10-12 13:54:43 +01:00
Imdad Ahad
1dd3298290 Add command to populate new service letter contact table from existing data 2017-10-02 11:46:19 +01:00
Rebecca Law
f2928e6c55 Migration server command to create one row in Service_sms_senders for each service. 2017-09-22 16:27:23 +01:00
Rebecca Law
9bd3ed2dac Use a unique id 2017-09-12 09:30:55 +01:00