Commit Graph

102 Commits

Author SHA1 Message Date
Martyn Inglis
b8638a891f Merge branch 'master' into research-mode-implementation
Conflicts:
	tests/app/conftest.py
2016-06-02 11:53:06 +01:00
Leo Hemsted
a51d6fae47 Merge pull request #363 from alphagov/send-sms-tests
send_sms tests
2016-06-02 11:27:57 +01:00
Martyn Inglis
001149c263 Merge branch 'master' into research-mode-implementation 2016-06-02 09:28:35 +01:00
Leo Hemsted
a3b847bf64 tests for send_sms through mmg/firetext 2016-06-01 15:59:44 +01:00
Nicholas Staples
276ca15919 Provider stats only updated if the provider successfully sends the message. 2016-06-01 12:43:26 +01:00
Martyn Inglis
909fac3c05 Added research mode tasks
- if a service is in research mode the don't send the notifications via the providers (MMG/SES/etc)
- instead set up a task to mimic those services callbacks - this completes the loop, and show stats, delivery receipts and so on.
- Use the "to" field to choose the response, allows users to create successful and errored notifications

temp fail sms, uses  "07833333333"
perm fail sms, uses = "07822222222"
success = "07811111111" (or anything else)

success email = "delivered@simulator.notify"
perm fail = "perm-fail@simulator.notify"
temp fail = "temp-fail@simulator.notify"
2016-05-31 16:55:26 +01:00
Rebecca Law
e4eecb894a Update job.notifications_delivered and job.notifications_failed count when updating the status of a notification. 2016-05-24 11:31:44 +01:00
Nicholas Staples
0fe0c1d2b4 Added job row number to the notification for csv jobs. All tests passing. 2016-05-19 10:46:03 +01:00
Rebecca Law
917110870d Use the template version at the time the notification is created or at the time the job is created.
Update notifications/sms|email endpoint to send the template version to the queue.
Update the process_job celery talk to send the template version to the queue.
When the send_sms|send_email task runs it will get the template by id and version.

Created a data migration script to add the template_vesion column for jobs and notifications.
The existing jobs and notifications are given the template_version of the current template.
There is a chance this is the wrong template version, but deemed okay since the application is not live.

Create unit test for the dao_get_template_versions method.
Rename /template/<id>/version to /template/<id>/versions which returns all versions for that template id and service id.
2016-05-13 16:25:05 +01:00
Rebecca Law
c8c0f95dd2 Merge branch 'master' into add-template-version
Conflicts:
	tests/app/dao/test_notification_dao.py
2016-05-12 09:49:35 +01:00
Rebecca Law
f72f5aba05 [WIP]
Start to add template_version to jobs and notification
2016-05-11 17:04:51 +01:00
Martyn Inglis
8669a169eb Merge branch 'master' into primary-provider
Conflicts:
	app/dao/notifications_dao.py
	app/dao/provider_statistics_dao.py
	app/schemas.py
	tests/app/conftest.py
2016-05-10 09:13:02 +01:00
NIcholas Staples
f06aaa924e Merge pull request #287 from alphagov/statistics_group_by_week_backup
Group by Notification Statistics added and all tests working.
2016-05-09 10:17:37 +01:00
Nicholas Staples
d40d9f76d4 Purge functional tests command added with passing test.
Code quality fix.
2016-05-06 11:16:30 +01:00
Martyn Inglis
57e05feafb Updated the Provider stats and rates DAO objects to query based on the identifier in the ProviderDetails object.
- updated all tests
- changed teardown to leave provider details rows on end of individual tests
2016-05-06 09:09:47 +01:00
Nicholas Staples
f465066a69 Group by Notification Statistics added and all tests working.
Fixed tests for upgraded client.
2016-05-05 11:04:20 +01:00
Nicholas Staples
bedc20d0ff Fragment count endpoint added and all tests working. 2016-04-28 12:01:57 +01:00
Rebecca Law
94aab20317 Add created_by_id to jobs table.
The user logged in will be the user marked as creating the job.
The admin code already sends the created_by user when creating a job.
2016-04-26 16:15:34 +01:00
Nicholas Staples
e6cc3b1724 Added functionality to archive a template.
Renamed migration file.
2016-04-26 10:11:18 +01:00
Chris Hill-Scott
eeaeacbf0b Remove send email code task
We don’t send email codes any more
2016-04-26 08:55:09 +01:00
Nicholas Staples
b56e324a4c Working tests and provider stats table.
Fix for tests and import error.

Added tests and updated for code review comments.
2016-04-25 12:20:06 +01:00
Adam Shimali
74ac5b5f30 Added version history to api keys. This needed a bit of change
to create history to handle foreign keys better. There may yet
be a better way of doing this that I have not found yet in sqlalchemy
docs.
2016-04-21 14:47:58 +01:00
Nicholas Staples
3b1423a2ea Provider Statistics added.
Rates command added with a test.

Updated to include added migration.
2016-04-21 13:47:04 +01:00
Adam Shimali
a6a18c1a6f This approach uses part of sqlalchemy example history_meta code
adapted to recording inserts and updates.

This removes need to manually create history tables.

Our code still remains in control of when history records are
created.
2016-04-19 17:13:26 +01:00
Nicholas Staples
1b76646170 Working placeholders in subject with upgraded notifications-utils.
Fix test.
2016-04-13 15:53:27 +01:00
Nicholas Staples
90f0505a3d Update limit to message_limit.
Further db changes and updates.

Remove traceback print out.

Fix bug in passing template id to a task.
2016-04-11 16:53:40 +01:00
Nicholas Staples
c4b316bde6 Rebased migrations, all tests working. 2016-04-08 13:34:54 +01:00
Nicholas Staples
143d1b0db8 Updated to retrieve csv upload from new bucket.
Fix test errors.
2016-04-07 14:10:30 +01:00
Nicholas Staples
4cc0028b01 Remove csv after process job is finished.
Fixed new tests.
2016-04-05 14:55:03 +01:00
Rebecca Law
ec180980ee Merge branch 'master' into remove_email_safe 2016-04-01 13:45:21 +01:00
Nicholas Staples
514d490d2f No limit for live services. 2016-04-01 11:12:44 +01:00
Rebecca Law
8df4919029 The admin app now sends the email from when creating a service and when updating the service name.
This PR removes the need for the email_safe function. The api does not create the email_from field for the service.
Tests were updated to reflect this change.
2016-03-31 17:46:18 +01:00
Nicholas Staples
090babaaeb Added endpoint for removing user from service, all tests passing. 2016-03-22 13:14:23 +00:00
Nicholas Staples
327f169575 Filtering added and tests working. 2016-03-21 12:37:34 +00:00
Adam Shimali
2d1d883283 Added task for sending email verification links out on intial
registration.

Left original email code endpoint in as it is still used for things like
email change.
2016-03-17 15:21:04 +00:00
Martyn Inglis
b88b0499b9 Order jobs on dashboard page 2016-03-14 16:31:57 +00:00
Martyn Inglis
62a7b8bcd0 Update notification status by message reference
- SES sends a reference to allow us to identify the notification
- use this to update status

If source of email is one of our internal emails (invites or validations) - don't try and update a notification.
2016-03-11 10:19:40 +00:00
Martyn Inglis
fd7eb54d10 Merge branch 'master' into delete-expired-things
Conflicts:
	app/celery/tasks.py
	tests/app/celery/test_tasks.py
	tests/app/dao/test_notification_dao.py
2016-03-10 09:48:29 +00:00
Martyn Inglis
c8a5366484 Scheduled tasks to clean up the database
- tasks run hourly
- uses celery beat to schedule the tasks

4 new tasks
- delete verify codes (after 1 day)
- delete invitations (after 1 day)
- delete successful notifications  (after 1 day)
- delete failed notifications (after 7 days)

Delete methods in the DAO classes
2016-03-09 17:46:01 +00:00
Martyn Inglis
593d753ed5 Updated tests 2016-03-09 14:52:07 +00:00
Martyn Inglis
2a13fe4a54 Merge branch 'master' into rate-limit
Conflicts:
	app/celery/tasks.py
	tests/app/celery/test_tasks.py
2016-03-09 14:16:59 +00:00
Martyn Inglis
61a0cf32c8 Ensure clients have rate limit enforced
- rate limiting is a hard number per day
- not limited in terms of rate of request
- limit is a single number held against the service
- every notification counts against the limit, regardless of type
- return a 429 if limit exceeded.
2016-03-09 11:06:37 +00:00
Chris Hill-Scott
589b4de5f9 Accept CSV files with additional columns
Currently when the Celery task processes a CSV it will call the API with the
values for all the non-recipient columns in the `personalisation` field. This
means that those API calls would fail, even though the CSV has been processed
‘successfully’.

This was not being caught by the tests, so this commit adds extra tests to check
what data the task is passing to the API call.

It then updates utils to version 2.0.1 which brings in this fix:
https://github.com/alphagov/notifications-utils/pull/10
2016-03-09 09:52:39 +00:00
NIcholas Staples
528f570ab6 Merge pull request #141 from alphagov/capture-aggregate-data
Capture aggregate data
2016-03-08 17:54:39 +00:00
Martyn Inglis
f5f50e00ff New notification stats table
- to capture the counts of things that we do
- initial commit captures when we create an email or sms

DOES NOT know about ultimate success only that we asked our partners to ship the notification

Requires some updates when we retry sending in event of error.
2016-03-08 15:23:19 +00:00
Chris Hill-Scott
157b385327 Use validation of recipients from utils
This was added to utils in 5914da74f1

This means that:
- we are doing the exact same validation in the API and admin app
- we are actually validating phone numbers for the correct format (hence all the
  changes to the tests)
2016-03-08 09:47:21 +00:00
Martyn Inglis
800afc9e44 Ensure restricted service are respected by tasks:
This is checked on 3rd party API calls, but jobs (CSV files) were able expected to only allow valid files.

Change in tack means we want to have restricted notification failures reported in the UI.
2016-03-03 12:05:18 +00:00
Nicholas Staples
ebec54cb80 Permission enums corrected, all tests passing. 2016-03-02 15:34:26 +00:00
Rebecca Law
3fd3aa5b3e Merge pull request #119 from alphagov/update_permission_endpoints
Update permission endpoints
2016-03-02 14:33:18 +00:00
Nicholas Staples
01f616eeac Fix all tests. 2016-03-02 11:10:52 +00:00