Commit Graph

340 Commits

Author SHA1 Message Date
Leo Hemsted
b4036e062d rework query to use count() 2017-02-16 12:44:40 +00:00
Rebecca Law
ed4b9d34a6 Changes as per code review comments.
Fix my backward date math :P
2017-02-15 17:41:07 +00:00
Rebecca Law
52342afe3f Add a limit to the number of active 2fa codes that we create. At the moment that is set to 10. 2017-02-15 16:18:05 +00:00
Rebecca Law
c4bf7a1331 Merge branch 'master' into stronger-2fa-security 2017-02-15 14:20:32 +00:00
Rebecca Law
53b7ad0961 Moved the cache key to the utils module.
Renamed the dao method.
2017-02-14 14:22:52 +00:00
Rebecca Law
5f48367ee5 Set the expiry time on a verify code (2fa) to 10 minutes.
When the verify code is wrong or expired increment the failed to login count for the user.
When the verify code is successfully used reset the failed login count to 0.
2017-02-14 14:04:11 +00:00
Rebecca Law
1c6cfb9bc8 Got the SqlAlchemy query to work 2017-02-14 11:05:23 +00:00
Rebecca Law
681f52b691 Missed some print statements and unused imports. 2017-02-14 09:59:33 +00:00
Rebecca Law
2346634fec Some code clean up, removed print statements. 2017-02-14 09:54:37 +00:00
Rebecca Law
458adefcb8 Added a redis cache for the template usage stats.
Cache expires every 10 minutes, but will help with the every 2 second query, especially when a job is running.
There is some clean up and qa to do for this yet
2017-02-13 18:47:29 +00:00
imdadahad
7417662fc1 Merge pull request #819 from alphagov/fix-auto-provider-switch-update-history
Fix: Update history when auto switching provider
2017-02-09 10:50:01 +00:00
Chris Hill-Scott
8cefe6a71b Merge pull request #820 from alphagov/fix-monthly-miscount
Fix miscount with monthly totals
2017-02-08 13:05:43 +00:00
Chris Hill-Scott
0dfa4a93d5 Make sure status dictionary is assinged each time
The status dictionary was being assigned once, and then subsequent
uses of it were by reference. This meant that each template type was
pointing at the same dictionary, and updating one meant updating all.

This commit adds a dictionary comprehension, which gets evaluated once
for each template type, so each template type has its own `dict` of
statuses.

Before
--
```
Email     SMS       Letter
  |        |           |
{'sending':, 'failed', …}
```

After
--
```
Email                SMS                   Letter
   |                  |                      |
{'sending':,      {'sending':,            {'sending':,
'failed',          'failed',               'failed',
…}                 …}                      …}
```
2017-02-07 13:14:58 +00:00
Imdad Ahad
c512b31afd Update provider history when auto switching provider 2017-02-07 12:38:56 +00:00
Imdad Ahad
ac48e4c944 Add separate method to update user password 2017-02-07 11:05:15 +00:00
Rebecca Law
30ca9e7e78 Merge pull request #811 from alphagov/check-service-is-active
Check service is active
2017-02-02 11:59:28 +00:00
Rebecca Law
f56824adde Cancel job if the service is inactive.
Update the PermissionsDao.get_permissions_by_user_id to only return permissions for active services,
this will make the admin app return a 403 if someone (otherthan platform admin) tries to look at an inactive service.
Removed the active flag in sample_service the dao_create_service overiddes this attribute.
2017-02-02 11:34:00 +00:00
Rebecca Law
da07f9dde8 Performance improvement to the template usage query. 2017-02-02 09:45:06 +00:00
Chris Hill-Scott
45b505f379 Split out handling of BST queries for reuse 2017-01-31 14:16:35 +00:00
Chris Hill-Scott
56ba653f48 Add endpoint for breakdown of activity by month
This endpoint will eventualy replace the weekly breakdown one. By month
for a given financial year is better, because it gives us consistency
with the breakdown of financial usage (and eventually consistency with
the template usage).

The code to do this is a bit convoluted, in order to fill out the counts
for months and statuses where we don’t have notifications.

This will make the admin side of this easier, because we can rely on
there always being numbers available. The admin side will deal with
summing the statuses (eg `temporary-failure` > `failed`) because this
is presentational.

This commit also modifies the usage count to use `.between()` for
consistency.
2017-01-31 14:16:34 +00:00
Rebecca Law
87556687ab This is the first PR in the story to create a suspend service feature.
- Renaming /service/<id>/deactivate to /service/<id>/archive to match language on the UI.
  - Will need to update admin before deleting the deactive service method
- Created dao and endpoint methods to suspend and resume a service.
- I confirm the use of suspend and resume with a couple people on the team, seems to be the right choice.

The idea is that if you archive a service there is no coming back from that.
To suspend a service is marking it as inactive and revoking the api keys. To resume a service is to mark the service as active, the service will need to create new API keys.
It makes sense that if a service is under threat that the API keys should be renewed.

The next PR will update the code to check that the service is active before sending the request or allowing any actions by the service.
2017-01-30 16:32:44 +00:00
imdadahad
eec7564475 Merge pull request #804 from alphagov/feat-add-perf-platform-client-and-job
Add client and job to update the performance platform daily
2017-01-27 17:03:38 +00:00
Imdad Ahad
cc7bf45766 Move notification retrieval logic into perf platform client and dont filter by status 2017-01-27 16:39:01 +00:00
Rebecca Law
c87d0a37f3 Refactor the get_midnight functions to return the date in UTC but for the localised time.
So in June when we are in BST June 16, 00:00 BST => June 15 23:00 UTC
2017-01-27 15:57:25 +00:00
Imdad Ahad
b6cc5b00ba Add dao method to get sms and email notification counts for a date range and previous day 2017-01-27 12:22:36 +00:00
Chris Hill-Scott
8ad0236f28 Allow filtering of billing info by financial year
We already filter the usage-by-month query by financial year. When we
show the total usage for a service, we should be able to filter this
by financial year.

Then, when the two lots of data are put side by side, it all adds up.
2017-01-27 11:13:22 +00:00
Imdad Ahad
e1d1769345 Refactor to switch properly when providers have equal priority with test 2017-01-23 13:36:04 +00:00
Imdad Ahad
d466265a9d Refactor provider switch methods in dao and tests 2017-01-20 16:13:13 +00:00
imdadahad
6a99b89d2e Fix long line 2017-01-17 17:01:47 +00:00
Imdad Ahad
2896ed27f2 Add dao methods to retrieve providers, switch, toggle along with tests 2017-01-17 15:38:54 +00:00
Rebecca Law
cefa00b3fa Merge branch 'master' into fix-billable-units-query
Conflicts:
	tests/app/dao/test_notification_dao.py
2017-01-10 14:24:08 +00:00
minglis
0c6193e2e9 Merge pull request #775 from alphagov/do-not-write-test-data-to-the-history-table
Do not write test data to the history table
2017-01-10 13:05:06 +00:00
Rebecca Law
ffa54821d9 The previous commit was failing the test in the docker container.
My local version of postgres is 9.6 and docker is using 9.5 (so I am ahead).
However, we found a more simple solution that works for both versions.
2017-01-10 11:21:33 +00:00
Rebecca Law
a1d8ca9364 Use a variable for the massive month date conversion.
Document the massive date function.
Document the get_april_fools function.
2017-01-09 15:34:24 +00:00
Rebecca Law
88e1e58f24 Fix the get_april_fools method to use March 31 23:00 as the timestamp.
The financial year start April 1, 00:00 BST and our dates are stored as UTC.
Added a test for get_april_fools.
Added some test more test data for get_billable_unit_count_per_month.
2017-01-09 14:54:25 +00:00
Rebecca Law
17767dfa62 Calculating the billing units per month was taking too long for a services with a lot of data.
The query now does the sum per month.
2017-01-09 11:13:24 +00:00
Rebecca Law
1de022f005 Update the query to execute immediately.
Fix indent.
Left a comment as to why start and end date are not set.
2016-12-29 13:28:55 +00:00
Rebecca Law
8ad078b663 Added a date range filter for the get all services end point.
When the start_date and end_date query argruments exists in the request,
the query will return the results from the NotificationHistory table for the given date range.
We will need to check the performance of this query, but this will only be used by the platform admin page.
2016-12-28 15:39:55 +00:00
Rebecca Law
c6df84a8f3 Merge branch 'master' into do-not-write-test-data-to-the-history-table
Conflicts:
	app/dao/notifications_dao.py
2016-12-21 09:57:17 +00:00
Leo Hemsted
f1899c6d53 add updated_at to provider_details
it's set to utcnow from dao_update_provider_details
2016-12-19 17:45:46 +00:00
Leo Hemsted
9d1b1328af add version to provider_details and _history
set all existing rows to have a version of 1 (also copy across values
to populate the new provider_details_history table in the upgrade
script)

in dao_update_provider_details bump the provider_details.version by 1
and then duplicate into the history table as a new row
(done manually as opposed to the decorator used in template_history
since this is only edited in this one place and the decorator is icky)
2016-12-19 17:13:10 +00:00
Leo Hemsted
a197754906 add new provider_details_history table
* to be used for auditing changes to provider details
* not hooked in yet
* also made provider_details.active non-nullable. set all existing null
  provider_details.active to FALSE. none are set false on live so
  should hopefully be fairly uncontroversial
* refactored NotificationHistory.from_notification and update from noti
  to share with provider_details_history
2016-12-19 17:13:10 +00:00
Martyn Inglis
0f37824b0c Ensure updates on a research mode service or test key don't touch the history table
- note this is an unexpectedly big change.
- When we create a service we pass the service id to the persist method. This means that we don't have the service available to check if in research mode.
- All calling methods (expecting the one where we use the notify service) have the service available. So rather than reload it I changed the method signature to pass the service, not the ID to persist.
- Touches a few places.

Note this means that the update or create methods will fall over on a null service. But this seems correct.

Goes back to the story which we need to play to make the service available as the API user so that the need to load and pass around services is minimised.
2016-12-19 16:45:18 +00:00
Martyn Inglis
35aa888658 When creating a notification don't create a history row if:
1) research mode service
2) test mode key

Stop test data getting into history and subsequently into stats and so on.
2016-12-19 13:57:06 +00:00
Rebecca Law
40dca7f9c7 Changed the timeout notications update query to set notifications still in created to a technical-failure and notifications still in sending or pending are set to a temporary-failure 2016-12-16 11:40:58 +00:00
Rebecca Law
a6da1ac864 Update the timeout notifications to use technical-failure rather than temporary failure 2016-12-14 16:18:43 +00:00
Imdad Ahad
431ec8acad Retrieve notifications by reference:
* Use get_notifications instead (references may not be unique)
2016-12-12 18:04:20 +00:00
Imdad Ahad
d4d733f997 Add dao method to get notification by reference with tests 2016-12-07 14:08:22 +00:00
Jenny Duckett
7668745d8b Allow excluding test key use in the all services stats query 2016-12-05 17:35:19 +00:00
Paul Craig
4082d38c0c Test invalid older_than, template_types, and bad ids
Come up with some simple tests in the routes, just to see we get
back what we expect as errors.
2016-11-28 15:56:59 +00:00