Commit Graph

2826 Commits

Author SHA1 Message Date
Leo Hemsted
3e21f57481 fix platform admin stats row-order bug
now that we're reading from two tables (ft_notification_status and
notifications) for stats, we'll get a couple of rows for each
notification type. If a service doesn't have any rows in one of those
tables, the query will return a row with nulls for the notification
types and counts. Some services will have history but no stats from
today, others will have data from today but no history.

This commit acknowledges that any row might have nulls, not just the
first row.
2019-01-09 11:43:40 +00:00
Rebecca Law
8fbe60bb90 Remove unused query 2019-01-07 15:37:26 +00:00
Rebecca Law
4522547753 Merge branch 'master' into optimise-platform-admin-live-services 2019-01-07 15:25:50 +00:00
Rebecca Law
bd9a6352fd Optimise the query for getting the platform statistics for all services. The page should render for all time after this change.
This is one step closer to eliminating the need to read from NotificationHistory.
2019-01-04 16:45:39 +00:00
Pea Tyczynska
eea324a19d Add flag on create_service to decide whether we should also check if service exists 2019-01-03 10:57:29 +00:00
Pea Tyczynska
f79c0b03e7 Sample job uses create_template instead of sample template 2019-01-03 10:52:14 +00:00
Pea Tyczynska
cde30de100 Use create_template instead of sample_template in sample_notification 2019-01-03 10:52:14 +00:00
Pea Tyczynska
154257027f Nothing in conftest uses sample_service now :) 2019-01-03 10:52:14 +00:00
Pea Tyczynska
d367daaf6e Some more conftest fixtures use create_service instead of sample_service 2019-01-03 10:52:14 +00:00
Pea Tyczynska
a3310c2da6 sample_job uses create_service instead of sample_service 2019-01-03 10:52:14 +00:00
Pea Tyczynska
0bcf13d85c sample_api_key uses create_service instead of sample_service 2019-01-03 10:52:14 +00:00
Pea Tyczynska
e8ce669b72 Use create_service instead of sample_service when creating sample_email_template for tests 2019-01-03 10:52:14 +00:00
Pea Tyczynska
95115e7ae6 Use create_service instead of sample_service when creating service permission for tests 2019-01-03 10:52:14 +00:00
Pea Tyczynska
3306b9fc97 use conditional in create_service to establish user 2019-01-03 10:52:14 +00:00
Pea Tyczynska
923703120b Check if test service exists before it gets created 2019-01-03 10:52:14 +00:00
Pea Tyczynska
c12594949f Refactor service_factory 2019-01-03 10:49:38 +00:00
Rebecca Law
39963d9784 Created a query to get the notification status counts per notification type and service for all service for a given date range.
The query follows the same pattern as the other queries, getting the statistics from the fact_notification_status table for dates older than today and union that with today.
Tests required.
2018-12-31 16:08:08 +00:00
Pea Tyczynska
fb1ca9b20d Test postage setting on happy path for create and update template 2018-12-24 12:24:17 +00:00
Pea Tyczynska
686c58acee Test post letter request sets notification postage correctly 2018-12-24 11:27:29 +00:00
Pea Tyczynska
e6524af89c Choose postage when persisting a notification 2018-12-21 16:37:52 +00:00
Pea Tyczynska
19f7678b05 Don't allow to set postage per template if no service permission 2018-12-21 16:37:52 +00:00
Rebecca Law
67cffc22b6 Oops it missed the limit 2018-12-20 17:03:46 +00:00
Rebecca Law
d8a0a3f5ab Added a test to make sure the loop exits. 2018-12-20 17:02:49 +00:00
Katie Smith
e9fb60f05c Send extra headers to Template Preview /precompiled/sanitise endpoint
We want to send two new headers, ServiceId and NotificationId to the
template preview /precompiled/sanitise endpoint. This is to allow us to log
errors from this endpoint in template preview with all the information needed,
instead of needing to pass the information back to notifications-api and
to log it there.
2018-12-19 13:49:27 +00:00
Pea (Malgorzata Tyczynska)
c87e95d7ff Merge pull request #2264 from alphagov/choose_postage
Add choose_postage service permission and add postage to Template
2018-12-18 10:31:11 +00:00
Leo Hemsted
251aecab1b base job start of processing_started rather than created_at
otherwise scheduled jobs will be viewed as old, and we'll pull stats
from the statistics tables, even if they might have not even started
yet
2018-12-17 15:47:16 +00:00
Leo Hemsted
2bd4f74ad0 Merge pull request #2258 from alphagov/dashboard-jobs-speedup
stop dashboard reading notification_history
2018-12-17 14:52:06 +00:00
Pea Tyczynska
86c3d96a83 Test postage constraints on TemplateBase model 2018-12-17 10:38:09 +00:00
Pea Tyczynska
52a7dcf86c Add choose_postage service permission and add postage to Template 2018-12-14 18:12:26 +00:00
Chris Hill-Scott
2f4f381fa4 Include live/trial mode in list of services
We need this so we can check in the admin whether users have any live
services.
2018-12-13 11:09:47 +00:00
Leo Hemsted
b80beab76c use ft_notification_status and notifications for job statistics
we previously always read from NotificationHistory to get the
notification status stats for a job. Now, if the job is more than three
days old read from ft_notification_status table, otherwise read from
the notifications table (to keep live updates).
2018-12-12 13:27:19 +00:00
Leo Hemsted
e555a7595b move job rest tests to use db.py instead of conftest directly 2018-12-12 12:59:55 +00:00
Leo Hemsted
63b3a3849f move job dao tests to use db.py instead of conftest directly 2018-12-12 12:59:55 +00:00
Leo Hemsted
bf62d3ad5f infer template/service from job for notification/ft_noti_status in db.py 2018-12-12 12:59:55 +00:00
Rebecca Law
4bb68c3e52 Merge pull request #2249 from alphagov/improve-performance-of-platform-admin-page
Improve performance of platform stats page
2018-12-12 12:20:13 +00:00
Pea Tyczynska
abe01c0bc0 Revert "Switch providers on slow delivery only produces logs"
This reverts commit 6938600ab8.
2018-12-11 15:14:08 +00:00
Pea (Malgorzata Tyczynska)
d7fcd564e0 Merge pull request #2250 from alphagov/switch_providers_update
Update switch providers on slow delivery method and query
2018-12-11 10:27:29 +00:00
Pea Tyczynska
5ed7564066 Remove unused config variables
We don't use FUNCTIONAL_TEST_PROVIDER_SERVICE_ID or
UNCTIONAL_TEST_PROVIDER_SMS_TEMPLATE_ID anymore so we can safely
delete them from config and tests.
2018-12-10 17:25:53 +00:00
Rebecca Law
5b90fd6fb0 Removed unused method 2018-12-10 16:27:59 +00:00
Leo Hemsted
474acc5bba new chillmaid approved error messages
these get shown on the front-end so make sure their content is good
2018-12-10 12:59:16 +00:00
Pea Tyczynska
9a76d6706e Update tests to match the new csv data 2018-12-07 11:24:15 +00:00
Pea Tyczynska
6938600ab8 Switch providers on slow delivery only produces logs 2018-12-05 15:56:16 +00:00
Pea Tyczynska
418060fbdb Update switch provider on slow delivery task to change max once evey 10 minutes 2018-12-05 15:56:16 +00:00
Pea Tyczynska
39ca5b9525 New query for finding if provider is slow
The delivery for provider is slow if more than threshold (currently
we pass in threshold 10%) either took x (for now 4) minutes to deliver,
or are still sending after that time. We look at all notifications
for current provider which are delivered or sending, and are not under
test key, for the last 10 minutes.

We are using created_at to establish if notifications are from last
10 minutes because we have an index on it, so the query is faster.

Also write tests for new is_delivery_slow_for_provider query
2018-12-05 15:51:40 +00:00
Rebecca Law
76ea46bc70 Added test for platform stats using query rather than mock. 2018-12-04 13:55:56 +00:00
Katie Smith
90d9135fcf Return cancelled letters for dashboard
Added cancelled letters to the number of failed letters in the statistics
that get used for the dashboard. At some point, we want to stop
including cancelled letters in the stats, but for now this keeps things
consistent with our current letter failure state, permanent-failure.
2018-12-03 17:51:09 +00:00
Katie Smith
1d67b55b16 Add endpoint for cancelling letters 2018-12-03 17:51:08 +00:00
Rebecca Law
c766febe94 Update /platform-stats to return the data from ft_notification_status, that way the request should not time out for a long date range.
Next steps is to update the query for platform admin stats for all services.
2018-12-03 15:46:44 +00:00
Katie Smith
365c462e93 Update get_notification_by_id to take an optional service_id
It can be useful to get a notification by id while checking that the
notification belongs to a given service. This changes the
get_notification_by_id DAO function to optionally also filter by
service_id so that we can check this.
2018-12-03 12:05:23 +00:00
Katie Smith
902e1b403a Update update_notification_status_by_id DAO function
Replaced `.with_for_lockmode()`, which is now deprecated, with
`.with_for_update() - https://docs.sqlalchemy.org/en/latest/orm/query.html#sqlalchemy.orm.query.Query.with_lockmode

The function should update any statuses that are not 'final', so added
`pending-virus-check` to the list of statuses that the function can
update.
2018-12-03 12:05:23 +00:00