Commit Graph

8700 Commits

Author SHA1 Message Date
Martyn Inglis
fe5d53bd70 Ensure GET /notifications/ by service ID returns CSV and API data - used by ADMIN console.
- this is existing functionality mimicked.
2016-09-15 16:00:46 +01:00
Martyn Inglis
23062233b6 Setup code for various key types and separated notifications made by CSV and by API into separate fixtures. 2016-09-15 16:00:04 +01:00
Martyn Inglis
803820d9aa New field on GET /notifications to allow users to request CSV data 2016-09-15 15:59:34 +01:00
Martyn Inglis
9f02e94f38 Ensure that querying for notifications does not return CSV data unless:
- It's explicitly asked for AND
- you are using a LIVE key

Team and Test keys CANNOT retrieve CSV/Job derived notifications
2016-09-15 15:59:02 +01:00
Chris Hill-Scott
210ed6b912 Fix HTML lists in Microsoft Outlook
Depends on:
- [x] https://github.com/alphagov/notifications-utils/pull/69
2016-09-15 11:10:50 +01:00
imdadahad
b07bbbbe16 Merge pull request #673 from alphagov/provide-simulated-numbers-live-keys
Add capability for simulated numbers and email addresses
2016-09-14 15:23:12 +01:00
minglis
73cbfed467 Merge pull request #675 from alphagov/format-statsd
Format the stat name with environmenbt
2016-09-14 15:23:06 +01:00
Martyn Inglis
04f36fab21 Format the stat name with environmenbt 2016-09-14 15:22:50 +01:00
minglis
c67c819f0a Merge pull request #671 from alphagov/statsd-time-taken-to-be-delivered
Statsd time taken to be delivered
2016-09-14 14:53:01 +01:00
Imdad Ahad
98ef340bc6 Refactor to use .count() method instead of mocking fake UUID 2016-09-14 13:07:57 +01:00
Chris Hill-Scott
b403a3ebab Merge pull request #674 from alphagov/bump-requirements-faster-csv
Bump utils version
2016-09-14 12:37:24 +01:00
Imdad Ahad
af8cfbe78c Add checks to ensure simulated notifications are not persisted to database 2016-09-14 11:12:57 +01:00
Rebecca Law
5c41ce0ea5 Merge pull request #672 from alphagov/fix-dashboard-stats
Update the timeout_notifications scheduled tasks.
2016-09-14 10:51:37 +01:00
Rebecca Law
fc66576d0f Remove try catch, it was only logging the exception. 2016-09-14 10:38:34 +01:00
Chris Hill-Scott
e3af4b48da Bump utils version
Brings in:
- [x] https://github.com/alphagov/notifications-utils/pull/68
2016-09-14 10:29:45 +01:00
Imdad Ahad
d4b0f68a69 Refactor to re-use existing contract-dependent method 2016-09-14 10:25:09 +01:00
Rebecca Law
339d33723b Added updated_at to the update query 2016-09-13 17:35:23 +01:00
Imdad Ahad
9df24b39ce Add capability for simulated numbers and email addresses 2016-09-13 17:00:28 +01:00
Rebecca Law
cb51c69503 Update the timeout_notifications scheduled tasks.
We found that if the notifications were in created or pending they are not purged from notifications.
- New bulk update method to set all notificaitons with:
  -  a status = created|sending|pending to temporary-failure
  - and is older then today minus SENDING_NOTIFICATIONS_TIMEOUT_PERIOD (in seconds)
- the scheduled task to timeout notifications use the new bulk update query.
- the task will be more efficient
2016-09-13 16:42:53 +01:00
minglis
4f768b53e6 Merge pull request #658 from alphagov/drop-status-column
Drop the status column
2016-09-13 14:46:38 +01:00
Martyn Inglis
34bd17ce66 Record time in seconds 2016-09-13 14:40:04 +01:00
Martyn Inglis
1a4b303c43 Only record the elapsed time at provider for notifications with a sent_at date,
could be that have a problem with state, misaligned tasks, we should just not record as edge case and this is aggregate.
2016-09-13 13:57:06 +01:00
Martyn Inglis
517cec173b Logs time between sent_at and the provider callback into statsd 2016-09-13 13:04:44 +01:00
Martyn Inglis
8541f6adcd Change DAO to return the notification from an update, not Boolean
- tests reflect this
2016-09-13 12:29:40 +01:00
minglis
d2b936e631 Merge pull request #667 from alphagov/delete-csv-file-after-7-days
Delete csv file after 7 days
2016-09-13 09:03:48 +01:00
Rebecca Law
94e78c9c33 Merge pull request #669 from alphagov/accept-invite-tests
Fix a bug in the accept invitation that was throwing a 500.
2016-09-12 16:52:32 +01:00
Rebecca Law
f1b9702dfb Fix codestyle 2016-09-12 16:41:48 +01:00
Martyn Inglis
e0d46d821d Merge branch 'master' into delete-csv-file-after-7-days
Conflicts:
	app/notifications/rest.py
2016-09-12 16:33:01 +01:00
Rebecca Law
bab9360519 Fix a bug in the accept invitation that was throwing a 500.
- If the token for invitation was expired, the method would throw a 500 while creating the erorr message.
- Added unit tests.
2016-09-12 16:16:02 +01:00
minglis
af70d6d15c Merge pull request #666 from alphagov/write-to-postgres-not-sqs
Write to postgres not sqs
2016-09-09 11:29:16 +01:00
Chris Hill-Scott
3ed97f15a1 Merge pull request #668 from alphagov/update-api-client
Update Python client
2016-09-09 09:33:25 +01:00
Martyn Inglis
5f72034577 Renamed and fixed the migration script to be in sequence 2016-09-08 16:12:15 +01:00
Martyn Inglis
e099edae51 Merge branch 'master' into drop-status-column 2016-09-08 16:11:18 +01:00
Chris Hill-Scott
1e235fc0cf Update Python client
Just so that nobody else has to do it.

Implements:
- [x] https://github.com/alphagov/notifications-python-client/pull/29

Which is a breaking change requiring the renaming of method arguments.
2016-09-08 16:08:30 +01:00
Martyn Inglis
a6bfc48478 Merge branch 'master' into write-to-postgres-not-sqs 2016-09-08 16:00:31 +01:00
Martyn Inglis
dde22bc58b Implemented deleted notification if SQS write fails 2016-09-08 16:00:18 +01:00
Martyn Inglis
a3f3f3e279 Changed the FROM_NUMBER on preview and stage to be shorter 2016-09-08 10:44:14 +01:00
minglis
c50a0200d4 Merge pull request #664 from alphagov/rationalise-config
Simplified the config. Aim is to get the actual secrets in credstash …
2016-09-08 10:27:31 +01:00
Martyn Inglis
9ee5c661b0 Changed as per @chs comment. Using cls based not explicity calling type 2016-09-08 09:21:32 +01:00
Martyn Inglis
b4d40dcd18 Merge branch 'master' into write-to-postgres-not-sqs
Conflicts:
	app/notifications/rest.py
	tests/app/notifications/rest/test_send_notification.py
2016-09-08 09:15:07 +01:00
Martyn Inglis
bdc1d464bf Fixed up a few things as per pull request review:
- Moved email domain and css bucket into dev config, not default

- deleted duplicated property

- removed unused property
2016-09-08 09:07:43 +01:00
imdadahad
b1013d8d31 Merge pull request #660 from alphagov/update-readme
Update README to include additonal instructions
2016-09-07 16:05:24 +01:00
Martyn Inglis
852e207478 Adds new scheduled task to delete CSV files.
Deletes files for jobs older than 7 days, matching delete process for the actual notifications

Runs 1 minutes past the hour at midnight, 1 and 2 am.
2016-09-07 15:36:59 +01:00
Martyn Inglis
c3657839e4 New jobs dao method to get jobs that are older than a certain number of days.
Used in deleting CSV files scheduled task
2016-09-07 15:36:07 +01:00
Martyn Inglis
f506f6751e Removed unused import 2016-09-07 15:35:25 +01:00
Martyn Inglis
0ecd6f2737 Removed the 'delete job' tasks from the tasks.py file
- this was called after processing a job.
- now to be called  on a schedule.
2016-09-07 15:35:12 +01:00
imdadahad
6cbd5396ea Merge pull request #662 from alphagov/remove-send-notification-tests
Remove rate limiting tests
2016-09-07 15:18:44 +01:00
Martyn Inglis
cf107d649f Merge branch 'master' into write-to-postgres-not-sqs
Conflicts:
	app/celery/tasks.py
2016-09-07 13:51:21 +01:00
Martyn Inglis
770b0c2838 Update rest end point to wrote to the DB not the queue.
We wrote to the queue as a performance optimisation , however dev found it confusing to not have immediate access to the notification as it may be perished some minutes later under periods of load. Additionally we had a couple of DB issues which led to us dropping notifications.

Pushing the DB write to earlier in the flow makes the system a little  more robust in the early days, we may want to change this when the traffic increases.
2016-09-07 13:47:22 +01:00
Martyn Inglis
255ffbd7f6 Tasks now use the new builder method to construct the DB object. 2016-09-07 13:45:37 +01:00