Commit Graph

5088 Commits

Author SHA1 Message Date
Leo Hemsted
e35c1bde77 ensure tests have test_ prefix
also remove unused code branch

thanks, coverage!
2018-04-30 13:23:02 +01:00
Leo Hemsted
0efa223fb2 rename days_ago to midnight_n_days_ago
also add some more timezone boundary tests and minor code cleanup
2018-04-30 11:50:56 +01:00
Leo Hemsted
310b8eda4c refactor dao_get_template_usage now that limit_days no longer used
doesnt need to have any conditional stuff now - always requires a date
2018-04-30 11:13:21 +01:00
Leo Hemsted
7dc34fc3a4 set 0'd dict in redis if no data exists in redis
to get the data for a day can be reasonably slow (a few hundred
milliseconds), and if someone's viewing a service with no activity we
don't want to do that query seven times every two seconds. So if there
is no data in redis, when we get the data out of the database, we
should put it in redis so we can just grab it from there next time.

This'll happen in two cases:
* redis data is deleted
* the service sent no messages that day

additionally, make sure that we convert nicely from redis' return
values (ascii strings) to unicode keys and integer counts.
2018-04-30 11:13:21 +01:00
Leo Hemsted
85fd7c3869 add new tests for template statistics 2018-04-30 11:13:21 +01:00
Leo Hemsted
9e8b6fd00d refactor template stats endpoint to read from new redis keys
New redis keys are partitioned per service per day. New process is as
follows:

* require a count of days to filter by. Currently admin always gives 7.
* for each day, check and see if there's anything in redis. There won't
  be if either a) redis is/was down or b) the service didn't send any
  notifications that day
  - if there isn't, go to the database and get a count out.
* combine all these stats together
* get the names/template types etc out of the DB at the end.
2018-04-30 11:13:21 +01:00
Leo Hemsted
5e702449cb move days_ago to utils and make it tz aware
it's used in a few places - it should definitely know what timezones
are and return datetimes rather than dates, which are hard to work with
in terms of figuring out how tz aware they are.
2018-04-30 11:13:21 +01:00
Leo Hemsted
67019fc5a1 add day filter to template usage dao, and remove NotificationHistory
we should be very careful with when we get data from
NotificationHistory - this should probably only be from scheduled
tasks. `dao_get_template_usage` is only called from the template
statistics rest endpoint, so shouldn't ever hit template history.

also, moved tests out to new file to break up the 2k test file a bit
2018-04-30 11:13:21 +01:00
Leo Hemsted
901383777b Merge pull request #1844 from alphagov/zendesk
zendesk instead of deskpro
2018-04-27 16:59:08 +01:00
Leo Hemsted
897ab93148 zendesk instead of deskpro 2018-04-27 16:36:39 +01:00
Rebecca Law
e41834c36a Merge pull request #1848 from alphagov/update-ft-billing-for-month-and-service
Added a command to rebuild the data for a given service_id and date
2018-04-27 11:29:50 +01:00
Rebecca Law
0520dfdb60 Added a command to rebuild the data for a given service_id and date 2018-04-26 15:39:04 +01:00
Chris Waszczuk
cc6e4fabe1 Merge pull request #1839 from alphagov/make-service-stats-7-days-max
Adjust get_service_stats to get counts for a maximum of 7 days ago
2018-04-26 15:10:40 +01:00
Rebecca Law
f3dc290bcf Merge pull request #1845 from alphagov/use_ft_billing_for_usage_page
Use ft billing for usage page
2018-04-26 09:47:03 +01:00
Rebecca Law
51af6b27a0 Update name of method to clarify the meaning.
Update group by to use label.
Update test.
2018-04-25 17:00:19 +01:00
Rebecca Law
cff731c52d Set filter on query 2018-04-25 16:17:56 +01:00
Rebecca Law
e32debbca1 Add more tests 2018-04-25 15:40:44 +01:00
Rebecca Law
d00614205e Adding test for fetch_biling_data_for_day. 2018-04-25 14:45:51 +01:00
Rebecca Law
16ef133aa5 Refactor reporting task so that methods can be used for the annual usage page.
Still a work in progress, tests are coming.
2018-04-25 14:45:51 +01:00
Rebecca Law
bd91aac763 Write some pseudo code for ft billing 2018-04-25 14:45:51 +01:00
Rebecca Law
8cd4e1be29 Query ft_billing aggregate for a month 2018-04-25 14:45:51 +01:00
Rebecca Law
731f29f2f7 Merge pull request #1842 from alphagov/fix-letter-in-sending-alert
Fix alert for letters still in sending
2018-04-25 14:44:44 +01:00
Rebecca Law
cb2a910469 Update variable name 2018-04-25 14:29:39 +01:00
Rebecca Law
b1ecf88283 Merge pull request #1841 from alphagov/pyup-update-pytest-3.5.0-to-3.5.1
Update pytest to 3.5.1
2018-04-25 11:01:21 +01:00
Rebecca Law
6e1417481c Remove print stmt 2018-04-25 10:16:01 +01:00
Rebecca Law
167f7a18e3 Fix the query that raises the alert for letters still in sending.
If Monday or Tuesday check for letters still sending after 4 days.
If Saturday or Sunday do nothing
If Wed, Thurs, Fri check for letters still sending after 2 days

Added test for Tuesday, corrected tests after the correction to query.
2018-04-25 10:10:25 +01:00
pyup-bot
95c39f6508 Update pytest from 3.5.0 to 3.5.1 2018-04-25 00:02:47 +01:00
chrisw
c157de82c4 Adjust get_service_stats to get counts for a maximum of 7 days ago 2018-04-24 16:05:43 +01:00
Rebecca Law
0ceed13bee Merge pull request #1837 from alphagov/rebuild-monthly-billing
Update populate_monthly_billing to take a service_id and month
2018-04-23 16:42:42 +01:00
Rebecca Law
a91b46dfa9 Added the else so the entire table does not rebuild. 2018-04-23 16:16:41 +01:00
Chris Hill-Scott
258ee6c498 Merge pull request #1835 from alphagov/pyup-update-moto-1.3.1-to-1.3.3
Update moto to 1.3.3
2018-04-23 15:48:22 +01:00
Rebecca Law
12796051f5 We need to rebuild the monthly billing for one service for the month of March.
This change makes that possible without affecting any other data in the table.
2018-04-23 13:53:47 +01:00
Katie Smith
fb6c07573f Merge pull request #1834 from alphagov/migration-to-add-active-column-to-replyto-tables
Add is_active column for the 3 reply_to tables
2018-04-23 12:34:05 +01:00
pyup-bot
08fbd04eb7 Update moto from 1.3.1 to 1.3.3 2018-04-19 00:11:32 +01:00
Katie Smith
c401314072 Add is_active column for the 3 reply_to tables
Added a new boolean column, `is_active` to these tables
* service_email_reply_to
* service_sms_senders
* service_letter_contacts

This has a database default of True in order to backfill the data, but
this default will be replaced with a model default later.
2018-04-17 16:23:46 +01:00
Chris Hill-Scott
e75c84a88c Merge pull request #1831 from alphagov/normalise-newlines-sms
Bump utils to normalise newlines in text messages
2018-04-12 15:05:08 +01:00
Chris Hill-Scott
308511e0ed Bump utils to normalise newlines in text messages
Brings in:
- [ ] https://github.com/alphagov/notifications-utils/pull/460
2018-04-12 11:13:02 +01:00
Sakis
a025e2cc9c Merge pull request #1811 from alphagov/add-http-healthchecks
Add http health-check for the api app
2018-04-11 15:51:05 +01:00
Athanasios Voutsadakis
0766e68099 Revert "Add 5 second sleep before stopping old api instances"
This reverts commit 23cfe73c48.
2018-04-11 15:38:41 +01:00
Athanasios Voutsadakis
23cfe73c48 Add 5 second sleep before stopping old api instances
This is to allow the new instances to start responding to the http
healthchecks and start serving traffic.
2018-04-11 15:01:45 +01:00
Athanasios Voutsadakis
e2a05efae0 Merge branch 'master' into add-http-healthchecks 2018-04-11 13:45:20 +01:00
Chris Hill-Scott
87ac1f15b7 Merge pull request #1829 from alphagov/bump-utils-plain-text-email-formatting
Bump utils to improve plain text email formatting
2018-04-10 11:37:40 +01:00
Chris Hill-Scott
c9882e2f9c Bump utils to improve plain text email formatting
Brings in:
- [x] https://github.com/alphagov/notifications-utils/pull/438
- [x] https://github.com/alphagov/notifications-utils/pull/450
- [x] https://github.com/alphagov/notifications-utils/pull/454

Changes:
- https://github.com/alphagov/notifications-utils/compare/25.3.0...26.2.0
2018-04-10 11:14:48 +01:00
Alexey Bezhan
84b99a9796 Merge pull request #1823 from alphagov/post-notification-with-document-upload
Post notification with document upload
2018-04-10 10:33:34 +01:00
Katie Smith
abade902e1 Merge pull request #1827 from alphagov/change-letter-still-sending-notification-date
Add extra day before raising letter still sending alert
2018-04-10 09:36:30 +01:00
Katie Smith
5ab46c737d Merge pull request #1830 from alphagov/pyup-update-pytest-mock-1.8.0-to-1.9.0
Update pytest-mock to 1.9.0
2018-04-10 09:36:22 +01:00
Katie Smith
417d382d1b Add extra day before raising letter still sending alert
We now want to wait an extra day before sending the alert that letters
are still sending.
2018-04-10 09:29:29 +01:00
pyup-bot
1e1e6c3550 Update pytest-mock from 1.8.0 to 1.9.0 2018-04-10 04:28:17 +01:00
Alexey Bezhan
89f2f8b9a7 Test that document download requests set the auth header 2018-04-09 16:30:16 +01:00
Alexey Bezhan
c189f09687 Update template content values from notification personalisation
We need to set template_with_content placeholder values with an
updated personalisation dictionary after processing the document
uploads in order to avoid returning base64 file data instead of
the document URLs in the response contents.
2018-04-09 16:30:16 +01:00