Commit Graph
306 Commits
Author SHA1 Message Date
NIcholas Staples 964fd5ac35 Merge pull request #377 from alphagov/single_from_number
Replaced mmg from number and firetext from number with single from nu…
2016-06-09 10:41:39 +01:00
Martyn Inglis 92e4c0872b Pulled out the tests that send SMSs into provider_tasks file/tests
- updated tests to validate creation of next task, not sending direct to provider
2016-06-07 12:53:31 +01:00
Nicholas Staples fe7d894420 Replaced mmg from number and firetext from number with single from number.
Fix merge mistake.

Fix tests from merge.

Update config to include correct staging and live names.
2016-06-06 16:41:33 +01:00
Rebecca Law be9fde1420 Fix tests for sending sms codes.
Since the unit tests delete the data in between tests I need to add the template data for the test for send sms code.
2016-06-06 11:51:12 +01:00
Martyn Inglis b8638a891f Merge branch 'master' into research-mode-implementation
Conflicts:
	tests/app/conftest.py
2016-06-02 11:53:06 +01:00
Leo Hemsted a51d6fae47 Merge pull request #363 from alphagov/send-sms-tests
send_sms tests
2016-06-02 11:27:57 +01:00
Martyn Inglis 001149c263 Merge branch 'master' into research-mode-implementation 2016-06-02 09:28:35 +01:00
Leo Hemsted a3b847bf64 tests for send_sms through mmg/firetext 2016-06-01 15:59:44 +01:00
Nicholas Staples 276ca15919 Provider stats only updated if the provider successfully sends the message. 2016-06-01 12:43:26 +01:00
Martyn Inglis 909fac3c05 Added research mode tasks
- if a service is in research mode the don't send the notifications via the providers (MMG/SES/etc)
- instead set up a task to mimic those services callbacks - this completes the loop, and show stats, delivery receipts and so on.
- Use the "to" field to choose the response, allows users to create successful and errored notifications

temp fail sms, uses  "07833333333"
perm fail sms, uses = "07822222222"
success = "07811111111" (or anything else)

success email = "delivered@simulator.notify"
perm fail = "perm-fail@simulator.notify"
temp fail = "temp-fail@simulator.notify"
2016-05-31 16:55:26 +01:00
Rebecca Law e4eecb894a Update job.notifications_delivered and job.notifications_failed count when updating the status of a notification. 2016-05-24 11:31:44 +01:00
Nicholas Staples 0fe0c1d2b4 Added job row number to the notification for csv jobs. All tests passing. 2016-05-19 10:46:03 +01:00
Rebecca Law 917110870d Use the template version at the time the notification is created or at the time the job is created.
Update notifications/sms|email endpoint to send the template version to the queue.
Update the process_job celery talk to send the template version to the queue.
When the send_sms|send_email task runs it will get the template by id and version.

Created a data migration script to add the template_vesion column for jobs and notifications.
The existing jobs and notifications are given the template_version of the current template.
There is a chance this is the wrong template version, but deemed okay since the application is not live.

Create unit test for the dao_get_template_versions method.
Rename /template/<id>/version to /template/<id>/versions which returns all versions for that template id and service id.
2016-05-13 16:25:05 +01:00
Rebecca Law c8c0f95dd2 Merge branch 'master' into add-template-version
Conflicts:
	tests/app/dao/test_notification_dao.py
2016-05-12 09:49:35 +01:00
Rebecca Law f72f5aba05 [WIP]
Start to add template_version to jobs and notification
2016-05-11 17:04:51 +01:00
Martyn Inglis 8669a169eb Merge branch 'master' into primary-provider
Conflicts:
	app/dao/notifications_dao.py
	app/dao/provider_statistics_dao.py
	app/schemas.py
	tests/app/conftest.py
2016-05-10 09:13:02 +01:00
NIcholas Staples f06aaa924e Merge pull request #287 from alphagov/statistics_group_by_week_backup
Group by Notification Statistics added and all tests working.
2016-05-09 10:17:37 +01:00
Nicholas Staples d40d9f76d4 Purge functional tests command added with passing test.
Code quality fix.
2016-05-06 11:16:30 +01:00
Martyn Inglis 57e05feafb Updated the Provider stats and rates DAO objects to query based on the identifier in the ProviderDetails object.
- updated all tests
- changed teardown to leave provider details rows on end of individual tests
2016-05-06 09:09:47 +01:00
Nicholas Staples f465066a69 Group by Notification Statistics added and all tests working.
Fixed tests for upgraded client.
2016-05-05 11:04:20 +01:00
Nicholas Staples bedc20d0ff Fragment count endpoint added and all tests working. 2016-04-28 12:01:57 +01:00
Rebecca Law 94aab20317 Add created_by_id to jobs table.
The user logged in will be the user marked as creating the job.
The admin code already sends the created_by user when creating a job.
2016-04-26 16:15:34 +01:00
Nicholas Staples e6cc3b1724 Added functionality to archive a template.
Renamed migration file.
2016-04-26 10:11:18 +01:00
Chris Hill-Scott eeaeacbf0b Remove send email code task
We don’t send email codes any more
2016-04-26 08:55:09 +01:00
Nicholas Staples b56e324a4c Working tests and provider stats table.
Fix for tests and import error.

Added tests and updated for code review comments.
2016-04-25 12:20:06 +01:00
Adam Shimali 74ac5b5f30 Added version history to api keys. This needed a bit of change
to create history to handle foreign keys better. There may yet
be a better way of doing this that I have not found yet in sqlalchemy
docs.
2016-04-21 14:47:58 +01:00
Nicholas Staples 3b1423a2ea Provider Statistics added.
Rates command added with a test.

Updated to include added migration.
2016-04-21 13:47:04 +01:00
Adam Shimali a6a18c1a6f This approach uses part of sqlalchemy example history_meta code
adapted to recording inserts and updates.

This removes need to manually create history tables.

Our code still remains in control of when history records are
created.
2016-04-19 17:13:26 +01:00
Nicholas Staples 1b76646170 Working placeholders in subject with upgraded notifications-utils.
Fix test.
2016-04-13 15:53:27 +01:00
Nicholas Staples 90f0505a3d Update limit to message_limit.
Further db changes and updates.

Remove traceback print out.

Fix bug in passing template id to a task.
2016-04-11 16:53:40 +01:00
Nicholas Staples c4b316bde6 Rebased migrations, all tests working. 2016-04-08 13:34:54 +01:00
Nicholas Staples 143d1b0db8 Updated to retrieve csv upload from new bucket.
Fix test errors.
2016-04-07 14:10:30 +01:00
Nicholas Staples 4cc0028b01 Remove csv after process job is finished.
Fixed new tests.
2016-04-05 14:55:03 +01:00
Rebecca Law ec180980ee Merge branch 'master' into remove_email_safe 2016-04-01 13:45:21 +01:00
Nicholas Staples 514d490d2f No limit for live services. 2016-04-01 11:12:44 +01:00
Rebecca Law 8df4919029 The admin app now sends the email from when creating a service and when updating the service name.
This PR removes the need for the email_safe function. The api does not create the email_from field for the service.
Tests were updated to reflect this change.
2016-03-31 17:46:18 +01:00
Nicholas Staples 090babaaeb Added endpoint for removing user from service, all tests passing. 2016-03-22 13:14:23 +00:00
Nicholas Staples 327f169575 Filtering added and tests working. 2016-03-21 12:37:34 +00:00
Adam Shimali 2d1d883283 Added task for sending email verification links out on intial
registration.

Left original email code endpoint in as it is still used for things like
email change.
2016-03-17 15:21:04 +00:00
Martyn Inglis b88b0499b9 Order jobs on dashboard page 2016-03-14 16:31:57 +00:00
Martyn Inglis 62a7b8bcd0 Update notification status by message reference
- SES sends a reference to allow us to identify the notification
- use this to update status

If source of email is one of our internal emails (invites or validations) - don't try and update a notification.
2016-03-11 10:19:40 +00:00
Martyn Inglis fd7eb54d10 Merge branch 'master' into delete-expired-things
Conflicts:
	app/celery/tasks.py
	tests/app/celery/test_tasks.py
	tests/app/dao/test_notification_dao.py
2016-03-10 09:48:29 +00:00
Martyn Inglis c8a5366484 Scheduled tasks to clean up the database
- tasks run hourly
- uses celery beat to schedule the tasks

4 new tasks
- delete verify codes (after 1 day)
- delete invitations (after 1 day)
- delete successful notifications  (after 1 day)
- delete failed notifications (after 7 days)

Delete methods in the DAO classes
2016-03-09 17:46:01 +00:00
Martyn Inglis 593d753ed5 Updated tests 2016-03-09 14:52:07 +00:00
Martyn Inglis 2a13fe4a54 Merge branch 'master' into rate-limit
Conflicts:
	app/celery/tasks.py
	tests/app/celery/test_tasks.py
2016-03-09 14:16:59 +00:00
Martyn Inglis 61a0cf32c8 Ensure clients have rate limit enforced
- rate limiting is a hard number per day
- not limited in terms of rate of request
- limit is a single number held against the service
- every notification counts against the limit, regardless of type
- return a 429 if limit exceeded.
2016-03-09 11:06:37 +00:00
Chris Hill-Scott 589b4de5f9 Accept CSV files with additional columns
Currently when the Celery task processes a CSV it will call the API with the
values for all the non-recipient columns in the `personalisation` field. This
means that those API calls would fail, even though the CSV has been processed
‘successfully’.

This was not being caught by the tests, so this commit adds extra tests to check
what data the task is passing to the API call.

It then updates utils to version 2.0.1 which brings in this fix:
https://github.com/alphagov/notifications-utils/pull/10
2016-03-09 09:52:39 +00:00
NIcholas Staples 528f570ab6 Merge pull request #141 from alphagov/capture-aggregate-data
Capture aggregate data
2016-03-08 17:54:39 +00:00
Martyn Inglis f5f50e00ff New notification stats table
- to capture the counts of things that we do
- initial commit captures when we create an email or sms

DOES NOT know about ultimate success only that we asked our partners to ship the notification

Requires some updates when we retry sending in event of error.
2016-03-08 15:23:19 +00:00
Chris Hill-Scott 157b385327 Use validation of recipients from utils
This was added to utils in https://github.com/alphagov/notifications-utils/commit/5914da74f1f948e2fbcd7597b7a26476de1e2eb3

This means that:
- we are doing the exact same validation in the API and admin app
- we are actually validating phone numbers for the correct format (hence all the
  changes to the tests)
2016-03-08 09:47:21 +00:00