Commit Graph

5306 Commits

Author SHA1 Message Date
Rebecca Law
9680946fbf Merge pull request #1931 from alphagov/fix-nightly-status
Fix task name for notification status updates
2018-07-04 15:23:06 +01:00
Rebecca Law
317ab149f4 The periodic task to populate ft_notification_status was calling the wrong task, this fixes that. 2018-07-04 14:12:47 +01:00
Leo Hemsted
732106119a Merge pull request #1930 from alphagov/monthly-statuses-page
Monthly statuses page from ft_notification_status
2018-07-04 13:36:56 +01:00
Leo Hemsted
be52adc68f add tests for monthly notification stats 2018-07-03 14:58:32 +01:00
Leo Hemsted
00e9fee471 make sure we combine stats that are already there
eg we have yesterdays data and want to top it up with todays deltas

'use a defaultdict to make this painless'
2018-07-03 14:58:32 +01:00
Leo Hemsted
1589bb58bb make sure test keys are filtered out of stats 2018-07-03 14:58:32 +01:00
Leo Hemsted
22496d9161 move stats endpoints to new file
(no new tests for new endpoint included in this commit)
2018-07-03 14:58:32 +01:00
Leo Hemsted
089056ef72 add tests for new statistics and fact notification status functions 2018-07-03 14:58:32 +01:00
Leo Hemsted
75fdb0290f remove old monthly stats function 2018-07-03 14:57:19 +01:00
Leo Hemsted
2463cd8fb5 move monthly notification stats from notification history to ft tables
it now uses the ft_notification_status table - nice and quick. if you
ask for the current tax year it'll top up the data with numbers from
the notification table.

the data is in exactly the same shape as the old endpoint - no changes
to the admin app are necessary
2018-07-03 14:56:31 +01:00
Rebecca Law
e3b3a407c9 Merge pull request #1918 from alphagov/wrap-get_json-in-try-except
Wrap exception to return a sensible message to the client
2018-07-03 11:03:01 +01:00
Rebecca Law
6da552a00b Added a message before the error description, to explain that it's the input arguments that have caused a problem. 2018-07-03 09:41:02 +01:00
Katie Smith
d2f8a206a8 Merge pull request #1928 from alphagov/platform-stats-fix
Fix aggregate stats DAO function to use start_date
2018-07-03 09:33:05 +01:00
Katie Smith
4228b77ebb Fix aggregate stats dao function to use start_date
Now using start_date to determine which table to query.
2018-07-03 09:26:02 +01:00
Rebecca Law
13155e24dc We are getting 500 errors when a POST notification has characters that can not be decoded or the json is malformed.
The exception is wrapped and a sensible error message is returned to the client.
2018-07-03 09:04:33 +01:00
Rebecca Law
e1a7d99c72 Merge pull request #1927 from alphagov/improve-last-usage-query
Improve the performance of the query to get the last template usage.
2018-07-03 08:55:47 +01:00
Rebecca Law
db5663e429 Improve the performance of the query to get the last template usage.
By adding the service id, the query performance has improved greatly. It went from 6200ms to 0.04ms.
This should stop the 500s when a template is deleted.
2018-07-02 17:17:38 +01:00
Katie Smith
3bc9c06c3c Merge pull request #1919 from alphagov/new-platform-stats-endpoint
New endpoint for the updated platform admin stats page
2018-07-02 15:37:24 +01:00
Leo Hemsted
3b5eefdd3d Merge pull request #1923 from alphagov/pyup-update-sqlalchemy-1.2.8-to-1.2.9
Update sqlalchemy to 1.2.9
2018-07-02 14:30:42 +01:00
Leo Hemsted
956822ee94 Merge pull request #1925 from alphagov/bump-utils-for-email-validation
Updated notifications-util to the latest version
2018-07-02 14:30:31 +01:00
Leo Hemsted
cb8b17e7d3 Merge pull request #1926 from alphagov/letter-timezone-bug
fix letters being put in the wrong bucket when near midnight cut-off
2018-07-02 14:28:59 +01:00
Leo Hemsted
cb7f5a7166 fix letters being put in the wrong bucket when near midnight cut-off
another day, another timezone bug
2018-07-02 14:16:40 +01:00
Rebecca Law
08dcc920a2 Updated notifications-util to the latest version.
There has been an update to the email validation code.
Added an extra test for email validation.
2018-07-02 13:48:40 +01:00
Rebecca Law
69c01ce8a0 Merge pull request #1924 from alphagov/replay-letters-in-error
Added a method to reply letters that are in the error folder on S3.
2018-07-02 11:05:13 +01:00
Rebecca Law
c1a84fb1c4 Added tests 2018-07-02 10:59:55 +01:00
Rebecca Law
0ede5d7ce9 Merge branch 'master' into replay-letters-in-error 2018-07-02 10:56:07 +01:00
pyup-bot
cb0971dcce Update sqlalchemy from 1.2.8 to 1.2.9 2018-06-29 19:09:51 +01:00
Katie Smith
b4654781be Add schema and validate platform stats requests
Added a schema to check that the start_date and end_date arguments are
both in the date format.
2018-06-29 16:14:48 +01:00
Katie Smith
cbb9f71e61 Fix bug with test notifications not showing
Test notifications are only stored in the notification table, not the
notification_history table. We were deciding which table to query for
the platform admin stats based on the start date passed to the DAO
function. This means that if the start date given was more than 7 days
ago and the end date was within the last 7 days, test notifications were
not being returned. We now use the end date to choose which table to
query which means that test notifications always get returned when they
should.
2018-06-29 16:01:45 +01:00
Katie Smith
4b030b1583 Create platform-stats blueprint
Created a platform-stats blueprint and moved the new platform stats
endpoint to the new blueprint (it was previously in the service
blueprint). Since the original platform stats route and the new platform
stats route are now in different blueprints, their view functions can
have the same name without any issues.
2018-06-29 16:01:45 +01:00
Katie Smith
5381491aae Move DAO function for new platform stats page
Moved the `fetch_new_aggregate_stats_by_date_range_for_all_services`
DAO function from the services DAO to the Notifications DAO since this
function queries the `notification` and `notification_history` tables.

Also added a test to check that the data returned from the function
takes BST into account.
2018-06-29 16:01:45 +01:00
Katie Smith
7e6076a41f Add in new endpoint with data for platform admin stats
Added in a new endpoint and DAO function to provide the data for the new
platform admin statistics page. The DAO method gets different data from
the Notifications / NotificationHistory table and also groups it differently.

The old endpoint has not been deleted yet to allow the numbers on the
old and new pages to be compared.
2018-06-29 16:01:45 +01:00
Rebecca Law
fa4ad93d40 Merge pull request #1920 from alphagov/bulk-invites-to-service
Command to add invited users from a CSV file
2018-06-29 13:20:53 +01:00
Alexey Bezhan
5eef8bb650 Merge pull request #1922 from alphagov/tail-celery-multi-logs
Stream delivery worker logs to stdout when running on PaaS
2018-06-29 12:56:03 +01:00
Alexey Bezhan
676e3ec39a Stream delivery worker logs to stdout when running on PaaS
Our application servers and celery workers write logs both to a
file that is shipped to CloudWatch and to stdout, which is picked
up by CloudFoundry and sent to Logit Logstash.

This works with gunicorn and single-worker celery deployments, however
celery multi daemonizes worker processes, which detaches them from
stdout, so there's no log output in `cf logs` or Logit.

To fix this, we start a separate tail process to duplicate logs written
to a file to stdout, which should be picked up by CloudFoundry.
2018-06-29 11:49:02 +01:00
Rebecca Law
f4846c88c9 Had to move the import to the method rather than the top of the file. 2018-06-29 11:12:54 +01:00
Rebecca Law
a2aa4934fb Merge branch 'master' into bulk-invites-to-service 2018-06-29 10:33:49 +01:00
Rebecca Law
b201711705 Undo the chnages to create_invited_user 2018-06-29 10:26:14 +01:00
Chris Hill-Scott
2ad703b89b Merge pull request #1921 from alphagov/ofgem
Add Ofgem letter logo
2018-06-29 10:21:50 +01:00
Chris Hill-Scott
1c960272a6 Add Ofgem letter logo 2018-06-29 10:09:18 +01:00
Rebecca Law
213e32cf6d Added a one off command to add many invited user to a service.
It would be nice to add something that can do this from the Admin app.
2018-06-28 17:09:45 +01:00
Rebecca Law
5850663ca6 Added a method to reply letters that are in the error folder on S3.
You can replay one at a time by sending in the file name. Or all the messages in the error folder.
2018-06-27 16:40:30 +01:00
Katie Smith
17a971cb77 Merge pull request #1910 from alphagov/summary-complaints-api
Add endpoint to show number of complaints
2018-06-27 09:25:19 +01:00
Katie Smith
8c22a6afda Change schema format name of datetime format
Renamed the 'datetime' format to 'datetime_within_next_day'. This format
is used to validate the date and time of scheduled notifications, not to
check the format of a datetime.
2018-06-27 09:11:38 +01:00
Katie Smith
7f4b828aff Add tests for get_complaint_count endpoint
* Added unit tests for the get_complaint_count endpoint
* Updated the schema to use 'date' format instead of 'datetime'
* Updated the complaint endpoint to convert start_date and end_date to
be dates instead of strings
2018-06-27 09:11:38 +01:00
Rebecca Law
28beeebbf4 Added a query and endpoint to return the count of complaints for a date-range.
Unit tests for complaints_rest.get_complaint_count have yet to be written.
Maybe there is a better name for the new endpoint.
2018-06-27 09:11:38 +01:00
Rebecca Law
e638653f11 When we get complaints we'd like to know how many we get in a day or other date range, so if there is a spike in complaints we can act on it.
Add new endpoint to return the number of complaints in a date range.

Unit tests to follow in the next commit.
2018-06-27 09:11:38 +01:00
Leo Hemsted
639a3f42b2 Merge pull request #1911 from alphagov/pyup-update-pytest-3.6.1-to-3.6.2
Update pytest to 3.6.2
2018-06-26 16:07:22 +01:00
Leo Hemsted
1f2f12238c Merge pull request #1909 from alphagov/pyup-update-flask-migrate-2.2.0-to-2.2.1
Update flask-migrate to 2.2.1
2018-06-26 16:07:10 +01:00
Leo Hemsted
01fbafd41e Merge pull request #1908 from alphagov/pyup-update-psycopg2-binary-2.7.4-to-2.7.5
Update psycopg2-binary to 2.7.5
2018-06-26 16:06:00 +01:00