Commit Graph

850 Commits

Author SHA1 Message Date
Leo Hemsted
a45c62d41d add can_send_letters flag to services
defaults to false
2016-10-26 13:01:40 +01:00
Leo Hemsted
6e015b5a7a Merge pull request #713 from alphagov/email-exception
Don't retry invalid emails
2016-10-17 11:26:50 +01:00
Chris Hill-Scott
2a65bf3bb5 Merge pull request #711 from alphagov/extend-scheduling-to-96-hours
Let jobs be scheduled up to 96 hours in the future
2016-10-14 15:33:44 +01:00
Leo Hemsted
c69d2aa778 add tests for ses client 2016-10-13 16:07:32 +01:00
Leo Hemsted
a2c3d265de remove unused former send_sms_to_provider and send_sms_to_email functions
they were superceded by deliver_sms and deliver_email in the same file 3 wks ago
2016-10-13 15:53:01 +01:00
Leo Hemsted
a095aa41f3 don't retry task if InvalidEmailError
just record it as a technical error - retrying wont fix a bad email
2016-10-13 15:27:47 +01:00
Rebecca Law
d9ced9f912 Merge pull request #712 from alphagov/change-email-confirmation
Change email confirmation
2016-10-13 14:01:23 +01:00
Rebecca Law
9ffdf66c49 Rename the endpoints.
Increase test coverage to include the encrypted message sent to the task.
2016-10-13 11:59:47 +01:00
Chris Hill-Scott
9f36b363f2 Let jobs be scheduled up to 96 hours in the future
If you want to send a job on Monday morning, you should be able to
schedule it on Friday. You shouldn’t need to work on the weekend.

96 hours is a full 4 days, so you can schedule a job at any time on
Friday for any time on Monday.

We’ve checked with the information assurance people, and they’re OK with
us holding the data for this extra amount of time.
2016-10-12 13:38:12 +01:00
Rebecca Law
b37eef974c Change email confirmation
- New endpoint to send a user an email to verify the email address when they want to change it.
2016-10-12 13:06:39 +01:00
Leo Hemsted
487554c025 use partial to make tests more legible 2016-10-11 14:40:23 +01:00
Leo Hemsted
d8467bfc3c filter out test jobs from the GET /service/{}/job endpoint
this is so that the filtering, which we do on the admin side, is applied
before pagination - so that the pages returned are all valid displayable
jobs. unfortunately this means that another config value has to be copied
to the server side but it's not the end of the world
2016-10-11 14:30:40 +01:00
Chris Hill-Scott
717e73a9f7 Loosen key restriction on get notification
Currently getting a single notification by ID is restricted to
notifications created with the same key type.

This makes things awkward for the functional tests now we’ve removed the
ability to create live keys in trial mode. So this commit removes the
restriction, so that any key can get any notification, no matter how it
was created.

And you’re never going to guess a UUID, so the chances of this giving
you privileged access to someone’s personal information is none.

This does not change the get all notifications endpoint, which
absolutely should be restricted by key type.
2016-10-10 17:29:38 +01:00
Chris Hill-Scott
88362cb075 Merge pull request #706 from alphagov/sort-jobs-by-processed-time-first
Sort jobs by processed time first
2016-10-10 14:03:27 +01:00
Chris Hill-Scott
ac1f5fcc91 Fix impossible test notification
No way a notification can be processed before it’s created.
2016-10-10 13:12:50 +01:00
Chris Hill-Scott
a2ea0db393 Make test easier to read
Explicity labelling what the numbers mean makes it easier to understand
what’s going on rather than having to refer back to the function call.
2016-10-10 13:12:40 +01:00
Chris Hill-Scott
df0504ddda Refactor job order test
- uses 4 rather than 8 entries to test the sort (2 notifications × 2
  columns on which we’re sorting)
- makes sure we test for when a scheduled job was created before a job
  that’s been processed already
- removes any relative datetimes so the tests are independant of
  database speed
2016-10-10 12:45:48 +01:00
Imdad Ahad
9bbd4713e4 Fix PEP issueFix PEP issueFix PEP issueFix PEP issueFix PEP issueFix PEP
issueFix PEP issueFix PEP issue
2016-10-10 10:27:57 +01:00
Imdad Ahad
ed203e33ea Refactor test for sending to non-whitelist member 2016-10-10 10:24:33 +01:00
Chris Hill-Scott
b4291684b7 Sort jobs by processed time first
Say you have a dashboard with some jobs you sent. Normally looks like:

job | sent
--- | ---
file.csv | **5pm**
file.csv | 3pm
file.csv | 1pm
file.csv | 11am

However if your 5pm job was scheduled at lunchtime, then it will look
like this:

job | sent
--- | ---
file.csv | 3pm
file.csv | 1pm
file.csv | **5pm**
file.csv | 11am

This is because the jobs are sorted by when they were created, not when
they were sent. It looks wrong.

**For jobs that have already been sent**

This commit changes the sort order to be based on `processed_at`
instead.

**For upcoming jobs**

If a job doesn’t have a `processed_at` time then it’s scheduled, but
hasn’t started yet. Only in this case should we still be sorting by
`created_at`.
2016-10-10 09:36:53 +01:00
Chris Hill-Scott
74be99e7c7 Let team key send to whitelist
There is an overlap between team key/trial mode/whitelist. But it’s not
a complete overlap. So it’s hard to understand all the different
permutations of which key lets you send to which people when.

This commit tries to reduce the differences between these concepts. So
for a user of the API

**In trial mode**

- You can send to anyone in your team or whitelist, using the team key
- You can simulate sending to anyone, using the simulate key

**When you’re live**

- You can send to anyone in your team or whitelist, using the team key
- You can simulate sending to anyone, using the simulate key
- You can send to anyone with the live key

So doing a `git diff` on that list, the only difference between being in
trial mode and live mode is now:

`+` You can send to anyone with the live key

**(How trial mode used to work)**
- You can send to anyone in your team or whitelist, using the normal key
- You can simulate sending to anyone, using the simulate key
- You can send to _just_ people in your team using the team key
2016-10-07 15:38:36 +01:00
Leo Hemsted
bdb4da4976 tests n stuff 2016-10-07 13:08:41 +01:00
Leo Hemsted
27c7a08523 use assert rather than relying on mock functions
mocks create any property you access, so calling functions on them is
inherently risky due to typos quietly doing nothing. instead assert
`.called is False`, which will fail noisily if you typo
2016-10-07 12:48:58 +01:00
Rebecca Law
6065ed57cf Fix for the job status
- It seems that when we changed the name of the job.status column that we didn't update the code to use job.job_status.
- Therefore none of the jobs since then have had the job status updated.
- Now that this is fix we can show the job status when there is an error like "sending exceeds limits"
  - This could happen if a job is scheduled to run at the top of the hour, so at the time of the job creation the limit was not exceed, but at the time of processing the job the limit is exceed.
2016-10-05 14:56:32 +01:00
Chris Hill-Scott
d352c0eed9 Really fix the timezones
Two main changes:

- uses `astimezone` instead of `replace` because `replace` doesn’t
  handle daylight savings time correctly [1]
- create the notifications one second before midnight in BST, because
  midnight is actually counted as being start of the _next_ day, month,
  etc

1. http://www.saltycrane.com/blog/2009/05/converting-time-zones-datetime-objects-python/#add-timezone-localize
2016-10-04 13:00:37 +01:00
Chris Hill-Scott
7abe40b506 Make billing year aware of British Summer Time
April 1st is in British summer time, ie 1hr ahead of UTC. The database
stores everything in UTC, so for accurate comparisions we need to make
sure that `get_financial_year()` returns a UTC, datetime-aware
timestamp that is 1hr ahead of midnight.

This also means that when we group notifications by month, the months
need to be in BST. So the line between one year and another is actually
01:00 on April 1st, _not_ 00:00 on April 1st.

There’s no way we’ve found to do this in SQLAlchemy or raw Postgres,
especially because we don’t store the timestamps with a timezone in the
database.

So the grouping and summing of the notifications has to be done in
Python.
2016-10-04 09:08:27 +01:00
Chris Hill-Scott
621e015f5f Test that a year with no notifications returns empty list 2016-10-03 15:50:55 +01:00
Chris Hill-Scott
def1d253aa Add endpoint to get billable units/financial year
`/services/ef7a665d-11a4-425a-a180-a67ca00b69d7/billable-units?year=2016`

Pretty much just passes through to the DAO layer. Validates that year
is:

- present (there’s no need for unbounded queries on this endpoint)
- an integer
2016-10-03 15:50:55 +01:00
Chris Hill-Scott
6a5e947220 Add DAO for getting billable units/financial year
In order to invoice people we need to know how many text message
fragments they’ve sent per month.

This should be per (government) financial year, ie April 1st to April
1st because we’ll only ever show a page for one year (because the
250,000 allowance is topped up at the start of every financial year).

This commit only does the DAO bit, not the REST bit.
2016-10-03 15:50:54 +01:00
Rebecca Law
c66d4c3c59 fix checkstyle 2016-10-03 14:48:06 +01:00
Rebecca Law
f0772fe06b Updates for review comments.
Simplify mocker in parameterised tests.
Fix duplicate test name.
2016-10-03 14:26:36 +01:00
Rebecca Law
9e7a7b1857 Merge branch 'master' into refactor-send_notifications
Conflicts:
	tests/app/notifications/rest/test_send_notification.py
2016-10-03 11:40:31 +01:00
Rebecca Law
e3d418506a Create query to sum notificaitons for the message limit check, this is a bit more efficient.
Refactored send_notifications method so that it is more readible.
Refectored the test_send_notificaitons so that it uses parametrized test to avoid duplication.
2016-10-03 10:57:10 +01:00
Martyn Inglis
ad5222442a Fixed build
- passing a single param to a celery task must be done as a list not a tuple.
2016-09-30 13:34:44 +01:00
Martyn Inglis
e3ed8fad1a Merge branch 'master' into research-mode-csv-file-queues
Conflicts:
	tests/app/celery/test_tasks.py
	tests/app/notifications/rest/test_send_notification.py
2016-09-30 11:07:32 +01:00
Martyn Inglis
4e03f81fcf Updated mock import 2016-09-30 10:31:50 +01:00
imdadahad
c2b57b79ee Merge pull request #689 from alphagov/service-whitelist
Service whitelist
2016-09-28 17:20:06 +01:00
Imdad Ahad
a7d42896cd Refactor tests 2016-09-28 17:03:17 +01:00
Imdad Ahad
8f74d9a122 Fix issue with test where team key does not check sending elegibility 2016-09-28 17:02:57 +01:00
Martyn Inglis
b9f0659eaa Ensure that when an API call is made on
- a TEST key
- a research-mode service

We put the task into the research-mode queue NOT the production delivery queue.
2016-09-28 15:48:12 +01:00
Martyn Inglis
3618a3967c Creating notification tasks should write send tasks to the research-mode queue if research mode service. 2016-09-28 15:29:10 +01:00
Martyn Inglis
20ccdab3bd Contract tests mock the new deliver_email|sms tasks 2016-09-28 15:28:45 +01:00
Martyn Inglis
c13ead77e4 Ensures that both the CSV processing and the API both use the new deliver_email and deliver_sms tasks not the old ones with the redundant parameter. 2016-09-28 15:05:50 +01:00
Martyn Inglis
9233ffa3d4 When processing a CSV file don't put the DB tasks into the live queue when service is in research mode. 2016-09-28 13:53:05 +01:00
Martyn Inglis
a4b5b486c4 Merge branch 'master' into rationalise-queues 2016-09-28 08:57:59 +01:00
Imdad Ahad
f9b539d71e Fix conflict 2016-09-27 16:54:00 +01:00
minglis
b1654c8808 Merge pull request #686 from alphagov/refactor-send-tasks-into-shared-code
Refactor send tasks into shared code
2016-09-27 16:46:34 +01:00
minglis
7b69a4e53e Merge pull request #692 from alphagov/job-status-filter
add statuses filter to GET /service/{}/job
2016-09-27 15:31:50 +01:00
minglis
8dd93b350f Merge pull request #691 from alphagov/fix-template-type-bug
Fix template type bug
2016-09-27 14:51:43 +01:00
Imdad Ahad
af2cbaa9c5 Fix PEP8 issues 2016-09-27 14:16:35 +01:00