Commit Graph

839 Commits

Author SHA1 Message Date
Leo Hemsted
d8467bfc3c filter out test jobs from the GET /service/{}/job endpoint
this is so that the filtering, which we do on the admin side, is applied
before pagination - so that the pages returned are all valid displayable
jobs. unfortunately this means that another config value has to be copied
to the server side but it's not the end of the world
2016-10-11 14:30:40 +01:00
Chris Hill-Scott
717e73a9f7 Loosen key restriction on get notification
Currently getting a single notification by ID is restricted to
notifications created with the same key type.

This makes things awkward for the functional tests now we’ve removed the
ability to create live keys in trial mode. So this commit removes the
restriction, so that any key can get any notification, no matter how it
was created.

And you’re never going to guess a UUID, so the chances of this giving
you privileged access to someone’s personal information is none.

This does not change the get all notifications endpoint, which
absolutely should be restricted by key type.
2016-10-10 17:29:38 +01:00
Chris Hill-Scott
88362cb075 Merge pull request #706 from alphagov/sort-jobs-by-processed-time-first
Sort jobs by processed time first
2016-10-10 14:03:27 +01:00
Chris Hill-Scott
ac1f5fcc91 Fix impossible test notification
No way a notification can be processed before it’s created.
2016-10-10 13:12:50 +01:00
Chris Hill-Scott
a2ea0db393 Make test easier to read
Explicity labelling what the numbers mean makes it easier to understand
what’s going on rather than having to refer back to the function call.
2016-10-10 13:12:40 +01:00
Chris Hill-Scott
df0504ddda Refactor job order test
- uses 4 rather than 8 entries to test the sort (2 notifications × 2
  columns on which we’re sorting)
- makes sure we test for when a scheduled job was created before a job
  that’s been processed already
- removes any relative datetimes so the tests are independant of
  database speed
2016-10-10 12:45:48 +01:00
Imdad Ahad
9bbd4713e4 Fix PEP issueFix PEP issueFix PEP issueFix PEP issueFix PEP issueFix PEP
issueFix PEP issueFix PEP issue
2016-10-10 10:27:57 +01:00
Imdad Ahad
ed203e33ea Refactor test for sending to non-whitelist member 2016-10-10 10:24:33 +01:00
Chris Hill-Scott
b4291684b7 Sort jobs by processed time first
Say you have a dashboard with some jobs you sent. Normally looks like:

job | sent
--- | ---
file.csv | **5pm**
file.csv | 3pm
file.csv | 1pm
file.csv | 11am

However if your 5pm job was scheduled at lunchtime, then it will look
like this:

job | sent
--- | ---
file.csv | 3pm
file.csv | 1pm
file.csv | **5pm**
file.csv | 11am

This is because the jobs are sorted by when they were created, not when
they were sent. It looks wrong.

**For jobs that have already been sent**

This commit changes the sort order to be based on `processed_at`
instead.

**For upcoming jobs**

If a job doesn’t have a `processed_at` time then it’s scheduled, but
hasn’t started yet. Only in this case should we still be sorting by
`created_at`.
2016-10-10 09:36:53 +01:00
Chris Hill-Scott
74be99e7c7 Let team key send to whitelist
There is an overlap between team key/trial mode/whitelist. But it’s not
a complete overlap. So it’s hard to understand all the different
permutations of which key lets you send to which people when.

This commit tries to reduce the differences between these concepts. So
for a user of the API

**In trial mode**

- You can send to anyone in your team or whitelist, using the team key
- You can simulate sending to anyone, using the simulate key

**When you’re live**

- You can send to anyone in your team or whitelist, using the team key
- You can simulate sending to anyone, using the simulate key
- You can send to anyone with the live key

So doing a `git diff` on that list, the only difference between being in
trial mode and live mode is now:

`+` You can send to anyone with the live key

**(How trial mode used to work)**
- You can send to anyone in your team or whitelist, using the normal key
- You can simulate sending to anyone, using the simulate key
- You can send to _just_ people in your team using the team key
2016-10-07 15:38:36 +01:00
Leo Hemsted
bdb4da4976 tests n stuff 2016-10-07 13:08:41 +01:00
Leo Hemsted
27c7a08523 use assert rather than relying on mock functions
mocks create any property you access, so calling functions on them is
inherently risky due to typos quietly doing nothing. instead assert
`.called is False`, which will fail noisily if you typo
2016-10-07 12:48:58 +01:00
Rebecca Law
6065ed57cf Fix for the job status
- It seems that when we changed the name of the job.status column that we didn't update the code to use job.job_status.
- Therefore none of the jobs since then have had the job status updated.
- Now that this is fix we can show the job status when there is an error like "sending exceeds limits"
  - This could happen if a job is scheduled to run at the top of the hour, so at the time of the job creation the limit was not exceed, but at the time of processing the job the limit is exceed.
2016-10-05 14:56:32 +01:00
Chris Hill-Scott
d352c0eed9 Really fix the timezones
Two main changes:

- uses `astimezone` instead of `replace` because `replace` doesn’t
  handle daylight savings time correctly [1]
- create the notifications one second before midnight in BST, because
  midnight is actually counted as being start of the _next_ day, month,
  etc

1. http://www.saltycrane.com/blog/2009/05/converting-time-zones-datetime-objects-python/#add-timezone-localize
2016-10-04 13:00:37 +01:00
Chris Hill-Scott
7abe40b506 Make billing year aware of British Summer Time
April 1st is in British summer time, ie 1hr ahead of UTC. The database
stores everything in UTC, so for accurate comparisions we need to make
sure that `get_financial_year()` returns a UTC, datetime-aware
timestamp that is 1hr ahead of midnight.

This also means that when we group notifications by month, the months
need to be in BST. So the line between one year and another is actually
01:00 on April 1st, _not_ 00:00 on April 1st.

There’s no way we’ve found to do this in SQLAlchemy or raw Postgres,
especially because we don’t store the timestamps with a timezone in the
database.

So the grouping and summing of the notifications has to be done in
Python.
2016-10-04 09:08:27 +01:00
Chris Hill-Scott
621e015f5f Test that a year with no notifications returns empty list 2016-10-03 15:50:55 +01:00
Chris Hill-Scott
def1d253aa Add endpoint to get billable units/financial year
`/services/ef7a665d-11a4-425a-a180-a67ca00b69d7/billable-units?year=2016`

Pretty much just passes through to the DAO layer. Validates that year
is:

- present (there’s no need for unbounded queries on this endpoint)
- an integer
2016-10-03 15:50:55 +01:00
Chris Hill-Scott
6a5e947220 Add DAO for getting billable units/financial year
In order to invoice people we need to know how many text message
fragments they’ve sent per month.

This should be per (government) financial year, ie April 1st to April
1st because we’ll only ever show a page for one year (because the
250,000 allowance is topped up at the start of every financial year).

This commit only does the DAO bit, not the REST bit.
2016-10-03 15:50:54 +01:00
Rebecca Law
c66d4c3c59 fix checkstyle 2016-10-03 14:48:06 +01:00
Rebecca Law
f0772fe06b Updates for review comments.
Simplify mocker in parameterised tests.
Fix duplicate test name.
2016-10-03 14:26:36 +01:00
Rebecca Law
9e7a7b1857 Merge branch 'master' into refactor-send_notifications
Conflicts:
	tests/app/notifications/rest/test_send_notification.py
2016-10-03 11:40:31 +01:00
Rebecca Law
e3d418506a Create query to sum notificaitons for the message limit check, this is a bit more efficient.
Refactored send_notifications method so that it is more readible.
Refectored the test_send_notificaitons so that it uses parametrized test to avoid duplication.
2016-10-03 10:57:10 +01:00
Martyn Inglis
ad5222442a Fixed build
- passing a single param to a celery task must be done as a list not a tuple.
2016-09-30 13:34:44 +01:00
Martyn Inglis
e3ed8fad1a Merge branch 'master' into research-mode-csv-file-queues
Conflicts:
	tests/app/celery/test_tasks.py
	tests/app/notifications/rest/test_send_notification.py
2016-09-30 11:07:32 +01:00
Martyn Inglis
4e03f81fcf Updated mock import 2016-09-30 10:31:50 +01:00
imdadahad
c2b57b79ee Merge pull request #689 from alphagov/service-whitelist
Service whitelist
2016-09-28 17:20:06 +01:00
Imdad Ahad
a7d42896cd Refactor tests 2016-09-28 17:03:17 +01:00
Imdad Ahad
8f74d9a122 Fix issue with test where team key does not check sending elegibility 2016-09-28 17:02:57 +01:00
Martyn Inglis
b9f0659eaa Ensure that when an API call is made on
- a TEST key
- a research-mode service

We put the task into the research-mode queue NOT the production delivery queue.
2016-09-28 15:48:12 +01:00
Martyn Inglis
3618a3967c Creating notification tasks should write send tasks to the research-mode queue if research mode service. 2016-09-28 15:29:10 +01:00
Martyn Inglis
20ccdab3bd Contract tests mock the new deliver_email|sms tasks 2016-09-28 15:28:45 +01:00
Martyn Inglis
c13ead77e4 Ensures that both the CSV processing and the API both use the new deliver_email and deliver_sms tasks not the old ones with the redundant parameter. 2016-09-28 15:05:50 +01:00
Martyn Inglis
9233ffa3d4 When processing a CSV file don't put the DB tasks into the live queue when service is in research mode. 2016-09-28 13:53:05 +01:00
Martyn Inglis
a4b5b486c4 Merge branch 'master' into rationalise-queues 2016-09-28 08:57:59 +01:00
Imdad Ahad
f9b539d71e Fix conflict 2016-09-27 16:54:00 +01:00
minglis
b1654c8808 Merge pull request #686 from alphagov/refactor-send-tasks-into-shared-code
Refactor send tasks into shared code
2016-09-27 16:46:34 +01:00
minglis
7b69a4e53e Merge pull request #692 from alphagov/job-status-filter
add statuses filter to GET /service/{}/job
2016-09-27 15:31:50 +01:00
minglis
8dd93b350f Merge pull request #691 from alphagov/fix-template-type-bug
Fix template type bug
2016-09-27 14:51:43 +01:00
Imdad Ahad
af2cbaa9c5 Fix PEP8 issues 2016-09-27 14:16:35 +01:00
Imdad Ahad
d258e29edc Fix conflict 2016-09-27 14:05:29 +01:00
Imdad Ahad
99cfa7bae2 Fix url to get from whitelist endpoint 2016-09-27 13:51:43 +01:00
Imdad Ahad
3b592c8619 Update creating a sample service whitelist based on new data model 2016-09-27 13:50:23 +01:00
Imdad Ahad
7b0cbca89b Enable sending to whitelist users if using a normal api key 2016-09-27 13:48:51 +01:00
Imdad Ahad
fbf266be87 Update ServiceWhitelist dao to conform to new data model 2016-09-27 13:45:36 +01:00
Imdad Ahad
8184eff15a Update ServiceWhitelist to conform to new data model 2016-09-27 13:44:29 +01:00
Martyn Inglis
ce5b5fbf4c Rationalised the queues.
- the internal notify activites (send sms code, forgotten passwords etc) all not in a "notify" queue.
- deleted all unused queues
2016-09-26 12:35:59 +01:00
Chris Hill-Scott
76d3f08b46 Merge pull request #690 from alphagov/add-key-name-notifications
Add details of API key to notifications
2016-09-26 08:41:02 +01:00
Chris Hill-Scott
592b0d15da Merge pull request #687 from alphagov/optionally-get-test-notifications
Optionally get notifications created with a test key
2016-09-26 08:40:54 +01:00
Leo Hemsted
fb6cb5f236 add statuses filter to GET /service/{}/job
can now pass in a query string `?statuses=x,y,z` to filter jobs based on
`Job.job_status`. Not passing in a status or passing in an empty string is
equivalent to selecting every filter type at once.
2016-09-23 16:34:13 +01:00
Rebecca Law
0ef43ab1dc Fix the wording on the message a little bit. 2016-09-23 16:03:11 +01:00