Commit Graph

1779 Commits

Author SHA1 Message Date
Rebecca Law
6065ed57cf Fix for the job status
- It seems that when we changed the name of the job.status column that we didn't update the code to use job.job_status.
- Therefore none of the jobs since then have had the job status updated.
- Now that this is fix we can show the job status when there is an error like "sending exceeds limits"
  - This could happen if a job is scheduled to run at the top of the hour, so at the time of the job creation the limit was not exceed, but at the time of processing the job the limit is exceed.
2016-10-05 14:56:32 +01:00
Pete
8aca768c30 Merge pull request #702 from alphagov/petes-readme-update
README + bootstrap update
2016-10-05 10:13:19 +01:00
Chris Hill-Scott
7e2131d2b2 Merge pull request #699 from alphagov/monthly-billable-units
Add DAO and endpoint for getting billable units/financial year
2016-10-04 16:08:31 +01:00
Peter Chamberlin
b3c213406d Changes after review, and further thinking... 2016-10-04 15:12:04 +01:00
Chris Hill-Scott
826eaaf5b3 Remove unecessary brackets when returning tuple 2016-10-04 13:05:15 +01:00
Chris Hill-Scott
219d5943c1 Return iterator from query for speed 2016-10-04 13:04:42 +01:00
Chris Hill-Scott
d352c0eed9 Really fix the timezones
Two main changes:

- uses `astimezone` instead of `replace` because `replace` doesn’t
  handle daylight savings time correctly [1]
- create the notifications one second before midnight in BST, because
  midnight is actually counted as being start of the _next_ day, month,
  etc

1. http://www.saltycrane.com/blog/2009/05/converting-time-zones-datetime-objects-python/#add-timezone-localize
2016-10-04 13:00:37 +01:00
Pete
f8db1a9731 Spelling 2016-10-04 12:30:55 +01:00
Pete
555163aa48 AWS region should probably be user-defined too 2016-10-04 12:30:37 +01:00
Peter Chamberlin
df2d081c14 Removed extraneous back ticks 2016-10-04 12:05:46 +01:00
Peter Chamberlin
b0d0a5627d Use make to generate version file 2016-10-04 12:03:07 +01:00
Peter Chamberlin
26325ff5e7 Corrected repo name/link for func tests 2016-10-04 11:18:43 +01:00
Peter Chamberlin
1539a527e6 Updated bootstrap to copy version.py.dist as required 2016-10-04 11:13:43 +01:00
Peter Chamberlin
214071707b Updated README based on my experience getting things running 2016-10-04 11:12:55 +01:00
Rebecca Law
7adc35b94d Merge pull request #701 from alphagov/change-email-template
Update the database to match the application models
2016-10-04 10:54:37 +01:00
Rebecca Law
6f83258004 Update Template.version and TemplateHistory.version nullable=False to match the db 2016-10-04 10:47:34 +01:00
Rebecca Law
81a390f9c3 There are differences in the models to the database, this script
gets the database up to speed with the models.
2016-10-04 10:00:00 +01:00
Chris Hill-Scott
76d5f14952 Filter unbillable notifications before calculating
Notifications with a `billable_units` count of `0` wont have any effect
on the result, but including them in the query will slow down the
grouping and summing of the results because it’ll have to loop over more
rows.
2016-10-04 09:51:41 +01:00
Chris Hill-Scott
7abe40b506 Make billing year aware of British Summer Time
April 1st is in British summer time, ie 1hr ahead of UTC. The database
stores everything in UTC, so for accurate comparisions we need to make
sure that `get_financial_year()` returns a UTC, datetime-aware
timestamp that is 1hr ahead of midnight.

This also means that when we group notifications by month, the months
need to be in BST. So the line between one year and another is actually
01:00 on April 1st, _not_ 00:00 on April 1st.

There’s no way we’ve found to do this in SQLAlchemy or raw Postgres,
especially because we don’t store the timestamps with a timezone in the
database.

So the grouping and summing of the notifications has to be done in
Python.
2016-10-04 09:08:27 +01:00
Chris Hill-Scott
621e015f5f Test that a year with no notifications returns empty list 2016-10-03 15:50:55 +01:00
Chris Hill-Scott
def1d253aa Add endpoint to get billable units/financial year
`/services/ef7a665d-11a4-425a-a180-a67ca00b69d7/billable-units?year=2016`

Pretty much just passes through to the DAO layer. Validates that year
is:

- present (there’s no need for unbounded queries on this endpoint)
- an integer
2016-10-03 15:50:55 +01:00
Chris Hill-Scott
6a5e947220 Add DAO for getting billable units/financial year
In order to invoice people we need to know how many text message
fragments they’ve sent per month.

This should be per (government) financial year, ie April 1st to April
1st because we’ll only ever show a page for one year (because the
250,000 allowance is topped up at the start of every financial year).

This commit only does the DAO bit, not the REST bit.
2016-10-03 15:50:54 +01:00
Rebecca Law
8d188f4664 Merge pull request #700 from alphagov/refactor-send_notifications
Refactor send notifications
2016-10-03 14:54:25 +01:00
Rebecca Law
c66d4c3c59 fix checkstyle 2016-10-03 14:48:06 +01:00
Rebecca Law
f0772fe06b Updates for review comments.
Simplify mocker in parameterised tests.
Fix duplicate test name.
2016-10-03 14:26:36 +01:00
Rebecca Law
9e7a7b1857 Merge branch 'master' into refactor-send_notifications
Conflicts:
	tests/app/notifications/rest/test_send_notification.py
2016-10-03 11:40:31 +01:00
Rebecca Law
e3d418506a Create query to sum notificaitons for the message limit check, this is a bit more efficient.
Refactored send_notifications method so that it is more readible.
Refectored the test_send_notificaitons so that it uses parametrized test to avoid duplication.
2016-10-03 10:57:10 +01:00
minglis
1d9e8c03b1 Merge pull request #697 from alphagov/run-research-mode-queue-in-own-worker
Research mode will have own worker so removed from default config.
2016-09-30 14:48:16 +01:00
minglis
e760d0617e Merge pull request #698 from alphagov/fixing-the-build
Fixed build
2016-09-30 13:40:57 +01:00
Martyn Inglis
ad5222442a Fixed build
- passing a single param to a celery task must be done as a list not a tuple.
2016-09-30 13:34:44 +01:00
minglis
f7242e007e Merge pull request #696 from alphagov/research-mode-csv-file-queues
Research mode queue should be used for research-mode services
2016-09-30 12:39:14 +01:00
Martyn Inglis
19f2a6cb65 Added stop for research mode worker 2016-09-30 11:25:12 +01:00
Martyn Inglis
6ce85878d8 Added start for research mode worker 2016-09-30 11:23:54 +01:00
Martyn Inglis
a504343623 Research mode will have own worker so removed from default config.
this means normal celery workers WON'T process this queue. Requires a dedicated celery worker to do this.

- note development and test configs add it in so DEV and TEST builds require no change.
2016-09-30 11:14:36 +01:00
Martyn Inglis
e3ed8fad1a Merge branch 'master' into research-mode-csv-file-queues
Conflicts:
	tests/app/celery/test_tasks.py
	tests/app/notifications/rest/test_send_notification.py
2016-09-30 11:07:32 +01:00
Martyn Inglis
4e03f81fcf Updated mock import 2016-09-30 10:31:50 +01:00
bandesz
d27e1595d5 Always pull base Docker image before building 2016-09-29 17:16:16 +01:00
imdadahad
c2b57b79ee Merge pull request #689 from alphagov/service-whitelist
Service whitelist
2016-09-28 17:20:06 +01:00
Imdad Ahad
a7d42896cd Refactor tests 2016-09-28 17:03:17 +01:00
Imdad Ahad
8f74d9a122 Fix issue with test where team key does not check sending elegibility 2016-09-28 17:02:57 +01:00
Imdad Ahad
db608a05d2 Refactor sending elegibility function and update across files 2016-09-28 17:00:17 +01:00
Martyn Inglis
b9f0659eaa Ensure that when an API call is made on
- a TEST key
- a research-mode service

We put the task into the research-mode queue NOT the production delivery queue.
2016-09-28 15:48:12 +01:00
Martyn Inglis
3618a3967c Creating notification tasks should write send tasks to the research-mode queue if research mode service. 2016-09-28 15:29:10 +01:00
Martyn Inglis
20ccdab3bd Contract tests mock the new deliver_email|sms tasks 2016-09-28 15:28:45 +01:00
Martyn Inglis
c13ead77e4 Ensures that both the CSV processing and the API both use the new deliver_email and deliver_sms tasks not the old ones with the redundant parameter. 2016-09-28 15:05:50 +01:00
Martyn Inglis
9233ffa3d4 When processing a CSV file don't put the DB tasks into the live queue when service is in research mode. 2016-09-28 13:53:05 +01:00
Imdad Ahad
1222a6ddf3 Refactor to increase readability for getting whitelist objects 2016-09-28 10:16:10 +01:00
minglis
9df65dfe25 Merge pull request #695 from alphagov/rationalise-queues
Removed the unused queues from the config
2016-09-28 09:46:42 +01:00
Martyn Inglis
f4f0c73fec Removed the unused queues from the config 2016-09-28 09:45:19 +01:00
minglis
9f932597a4 Merge pull request #693 from alphagov/rationalise-queues
Rationalised the queues.
2016-09-28 09:07:37 +01:00