Commit Graph

75 Commits

Author SHA1 Message Date
Ken Tsang
ab72d6f555 Call the create letters pdf task in api calls for services with letters as pdf 2017-12-12 12:07:55 +00:00
Richard Chapman
344e5c1f5d dded statsd to the endpoint and removed it from a letters only method
as we are not interested int he letters performance at the moment.
2017-12-01 15:46:31 +00:00
Richard Chapman
5f25d3162a Disable Redis for Staging added statsd for post methods
- Disable Redis as there is a current connection limit of 256 which
 could slow down the request if they are all used
 - Added statd to methods in the post to help spot any bottlenecks
2017-12-01 15:41:53 +00:00
Rebecca Law
8aa94af5d7 Stop populating the notification to sender mapping table when creating a notificaiton. 2017-11-29 16:47:23 +00:00
Rebecca Law
12024e9619 Update save_letter task to save the default letter contact block when creating the notification. 2017-11-27 11:33:04 +00:00
Rebecca Law
d779751bae Set reply_to_text for letter notifications posted to /notifications/letter 2017-11-24 16:17:25 +00:00
Rebecca Law
e0d0b2f4fd Merge branch 'add-reply-to-notifications' of github.com:alphagov/notifications-api into add-reply-to-notifications 2017-11-23 16:57:17 +00:00
Rebecca Law
75d99ea397 [WIP] 2017-11-23 16:01:05 +00:00
Rebecca Law
df2d3a95ae [WIP] 2017-11-23 14:55:49 +00:00
Rebecca Law
fe6bafcfb2 Update the from number in the response of post notification. 2017-11-10 14:17:29 +00:00
Rebecca Law
4eec11b633 Added an optional parameter in the form for POST /v2/notifications/sms and /service/<service_id>/send-notification to pass in the SMS sender id.
The send_sms_to_provider still needs to use the SMS sender being passed in to the POST.

As part of https://www.pivotaltracker.com/story/show/152106587
2017-10-30 13:36:49 +00:00
Rebecca Law
8d6c38e1fe Now that we allow multiple SMS senders for a service, update the sender for SMS to be the default SMS sender, regardless if it is inbound number or not 2017-10-24 13:37:17 +01:00
Richard Chapman
068ecd80a1 Changes after review. Style changes to ensure consistency with the rest of the code base to ensure good maintainability in future. 2017-10-05 16:50:18 +01:00
Richard Chapman
7974e59eb7 Changes after review. Updates to style and import order to be more consistent with other code.
Added clarification to an error message to give better debugging information.
Removed using dao_get_reply_to_by_service_id in tests to be more consistent with other code and use the test db functions or remove the need for a call altogether making the code less complex.
2017-10-05 16:50:18 +01:00
Richard Chapman
d2168b7985 Added the mapping between notification and reply to email to the database and persisted the mapping when the request is received by the end point. the end point also checks if the reply to email id exists and if not returns an error. Also added tests to test the functionality. 2017-10-05 16:50:17 +01:00
Richard Chapman
2fbe492d5c [2/10] Allow API calls to specify the reply address option (#1291)
* Added service_email_reply_to_id to the POST /v2/notifications/email and a test to test the validator

* Caught NoResultFound exception in check_service_email_reply_to_id as it was not being caught when there there was no valid service_id or reply_to_id. Fixed failing tests which were not passing due to the NoResultFound exception and added further tests to check for the good path through the code and an test to check for an invalid service_id

* Added service_email_reply_to_id to the POST /v2/notifications/email and a test to test the validator

* Caught NoResultFound exception in check_service_email_reply_to_id as it was not being caught when there there was no valid service_id or reply_to_id. Fixed failing tests which were not passing due to the NoResultFound exception and added further tests to check for the good path through the code and an test to check for an invalid service_id

* Fixed code style in validators.py to confirm with rules
Update the name of email_reply_to_id to conform better with other attributes in the schema and the resultant code in post_notifications.py
Fixed code style in test_validators.py to confirm with rules
Added tests to test_post_notifications.py to test the email_reply_to_id being present and being incorrect, it being optional is being tested by other tests.

* Added service_email_reply_to_id to the POST /v2/notifications/email and a test to test the validator

* Added service_email_reply_to_id to the POST /v2/notifications/email and a test to test the validator

* Caught NoResultFound exception in check_service_email_reply_to_id as it was not being caught when there there was no valid service_id or reply_to_id. Fixed failing tests which were not passing due to the NoResultFound exception and added further tests to check for the good path through the code and an test to check for an invalid service_id

* Caught NoResultFound exception in check_service_email_reply_to_id as it was not being caught when there there was no valid service_id or reply_to_id. Fixed failing tests which were not passing due to the NoResultFound exception and added further tests to check for the good path through the code and an test to check for an invalid service_id

* Fixed code style in validators.py to confirm with rules
Update the name of email_reply_to_id to conform better with other attributes in the schema and the resultant code in post_notifications.py
Fixed code style in test_validators.py to confirm with rules
Added tests to test_post_notifications.py to test the email_reply_to_id being present and being incorrect, it being optional is being tested by other tests.

* Minor update after manual merge to fix check style rule break in test_validators.py where a single space was introduced.

* Updates after code review. Moved the template from the exception message as it was not required and updated the error message to match the field name in the sschema for better debugging and error identification.

* Fixed test after update of exception message
2017-10-04 14:34:45 +01:00
Leo Hemsted
b1928b928c update process_letter tests 2017-09-26 11:28:54 +01:00
Leo Hemsted
f3db920c71 remove jobs from letter api calls
we now no longer create a job. At the end of the post there is no
action, as we don't have any tasks to queue immediately - if it's a
real notification it'll get picked up in the evening scheduled task.

If it's a test notification, we create it with an initial status of
sending so that we can be sure it'll never get picked up - and then we
trigger the update-letter-notifications-to-sent-to-dvla task to sent
the sent-at/by.
2017-09-26 09:57:36 +01:00
Leo Hemsted
17ba8db97f remove jobs from letter api and make success/error ftp callback tasks
1. No longer create jobs when creating letters from api 🎉
2. Bulk update notifications based on the notification references after
   we send them to DVLA - either as success or as error
2017-09-26 09:57:35 +01:00
Leo Hemsted
2c758cbccd use new send-jobs-to-dvla task instead of send-files-to-dvla
the tasks (on the ftp app) do the same thing, but to improve clarity
i've renamed it to jobs, because we'll be adding a notifications one
soon
2017-09-12 17:28:07 +01:00
Rebecca Law
0a21f1f3e8 Small refactor to how notification_schemas are tested.
My local build was not always getting the optional requirement for the jsonschema uri format checker (rfc3987).
The requirement has been added to the requirements_for_test file.
I changed the tests to validate the response schema from the post_notifications tests, this way we can tell if we are breaking the contract.
This showed that the email_from was not returning the entire email address but just the username, that has been corrected here.
Removed the response schema validation tests since they are not being testing in the post notification tests.
2017-09-11 11:10:45 +01:00
Rebecca Law
19f964a90b Added a check that the call is not using a test api key.
Removed the tests for trial mode service for the scheduled tasks and the process job.
Having the validation in the POST notification and create job endpoint is enough.
Updated the test_service_whitelist test because the order of the array is not gaurenteed.
2017-09-04 17:24:41 +01:00
Ken Tsang
8f82642422 Prevent v2 notifications POST when in trial mode 2017-08-30 16:04:10 +01:00
Ken Tsang
667ee57a35 Refactor code to use inbound_numbers if set 2017-08-16 12:50:44 +01:00
Leo Hemsted
13917c9c57 give test letter api notifications a different filename
so they can be distinguished on the frontend.

Also, some related cleanup:

* don't show test api letters on the frontpage
* make sure the subject is returned from the API for letters
* make sure the letter's address is returned for letters
2017-08-01 18:23:29 +01:00
Leo Hemsted
075d2a3346 tighten key_type validation on letters api
when in research mode or test key, dont send letters via api - instead,
just put them straight to success state

when using a team key, flat out reject the request (403)
2017-08-01 15:49:00 +01:00
Leo Hemsted
f528236eda make job.created_by nullable
Since letter jobs from the API aren't created by any single individual,
lets make created_by nullable. Note: We'll have to make sure that we
update the admin app to handle these jobs nicely
2017-07-27 12:58:13 +01:00
Leo Hemsted
2746bf0318 process letters from api traffic
there are three steps to this

1. Create a job
  * Starts in status 'ready to send'
  * Created by None, since it's from the API
  * original file name 'letter submitted via api'
2. Create a single notification for that job
  * job_row_number 0
  * client reference if provided
  * address line 1 as recipient
3. Trigger the build_dvla_file task
  we know that all the notifications have been created for this job
  (since we just created them ourselves synchronously), so this will
  just create the dvla-format file for the job, and upload it to s3.
2017-07-27 11:12:09 +01:00
Leo Hemsted
79e33073c9 raise 404 when unknown url
consistent with other endpoints.

also refactor of notification_schema to separate some fns to a different file
2017-07-20 15:23:51 +01:00
Leo Hemsted
2be194d9ce refactor post_notification to separate sms/email and letter flows 2017-07-19 17:02:19 +01:00
Leo Hemsted
7f4eec79e4 add POST letter schema
similar to sms/email, however, for consistency with response values
and internal storage, rather than supplying an "email_address" field
or a "phone_number" field, supply an "address_line_1" and "postcode"
field within the personalisation object.
2017-07-19 17:02:19 +01:00
Leo Hemsted
6c61a3fc2a Revert celery4
Revert the following three pull requests:
https://github.com/alphagov/notifications-api/pull/1085
https://github.com/alphagov/notifications-api/pull/1086
https://github.com/alphagov/notifications-api/pull/1088

celery 4.0.2 looked promising, however, on staging under mild load
(5/sec api calls) the performance was actually worse than 3.1.25
2017-07-19 15:17:19 +01:00
Martyn Inglis
786adb5d71 Move Queuenames in with the celery code, revamp config to allow move to celery 4.x 2017-07-12 12:01:52 +01:00
Ken Tsang
72000c38aa Refactor to abstract out post notification errors 2017-07-06 12:27:58 +01:00
Ken Tsang
23618a186c Further refactoring 2017-07-06 12:27:57 +01:00
Ken Tsang
0b3277b8a4 Refactored to make code clearer 2017-07-06 12:27:57 +01:00
Ken Tsang
e0fbcb0dc6 Add permission check in for v2 post notification 2017-07-06 12:27:57 +01:00
Leo Hemsted
10b851b456 add send_notification file for handling sending one off messages
currently, they're made by creating a one-line job, but we want to
reduce task/csv file noise so we're moving them to persist in the
same vein as API usage. However, we can't just call through to that
since there are some differences:

* no api keys
* tighter control over API format
* no scheduling
* no client references

etc.

So, re-using as much of the v2 validation stuff as possible, I've
created this file that just does basic validation, and then calls
through to persist_notification and schedules a task. Woo.
2017-06-20 11:51:36 +01:00
Rebecca Law
40aadf2748 Merge pull request #991 from alphagov/invite-service-to-schedule
Invite the service to schedule notifications from the API
2017-06-01 12:37:32 +01:00
Martyn Inglis
75c5e5e5d5 Merge branch 'master' into redo-queue-visibitlity-timeout
Conflicts:
	app/notifications/process_notifications.py
	app/v2/notifications/post_notifications.py
2017-05-30 10:18:18 +01:00
Rebecca Law
3dc70b8c39 Check service.permissions for the existence of schedule_notifications if the notications is being created with a scheduled_for param. 2017-05-26 15:41:14 +01:00
Martyn Inglis
2591d3a1df This massive set of changes uses the new queue names object throughout the app and tests.
Lots of changes, all changing the line of code that puts things into queues, and the code that tests that.
2017-05-25 10:51:49 +01:00
Rebecca Law
9f6c037530 Use iso8601 to validate scheduled_for datetime.
Added a validation method that always fails for scheduled notifications.
Comment out config for scheduled task.
The schedule notifications will be turned on once we can invite services to use it.
Waiting for the service permission story, must commit this in order to keep things from going stale.
2017-05-24 16:27:15 +01:00
Rebecca Law
3a3161ecc4 Merge branch 'master' into schedule-api-notification
Conflicts:
	app/celery/scheduled_tasks.py
	app/v2/notifications/post_notifications.py
	tests/app/celery/test_scheduled_tasks.py
2017-05-22 14:05:57 +01:00
Martyn Inglis
1dc3970595 Code tidy up as be @imdad comments. 2017-05-16 12:55:12 +01:00
Rebecca Law
f0e2713bef Add scheduled_for in the post notification request form.
Return scheduled for in get_notification requests.
2017-05-15 17:27:38 +01:00
Martyn Inglis
0e38259cb4 Remove all DAO get service calls, replaced with reference to new service object in context. 2017-05-05 15:23:06 +01:00
Martyn Inglis
a9539d892c Merge branch 'master' into rate-limit-api-calls
Conflicts:
	requirements.txt
	tests/app/notifications/rest/test_send_notification.py
	tests/app/notifications/test_validators.py
	tests/app/v2/notifications/test_post_notifications.py
2017-05-02 10:56:56 +01:00
Martyn Inglis
2a0f8c8808 Validate International phone numbers
- uses new utils methods to validate phone numbers
- defaults to International=True on validation. This ensures the validator works on all numbers
- Then check if the user can send this message to the number internationally if needed.
2017-04-26 15:56:45 +01:00
Martyn Inglis
926b8a60f9 Adds in call to new rate limit method in the redis client
- both V1 and V2 APIs
- Rate limiting wrapped into a new method - check_rate_limiting
	- delegates to the previous daily limit and the new though put limit
- Rate limiting done on key type. Each key has it's own limit (number of requests) and interval (time period of requests)
- Configured in the config. Not done on a per-env basis though could be in the future.
2017-04-24 14:15:08 +01:00