Switches on authentication checks for Firetext inbound SMS callbacks.
This should only be released once Firetext callback URLs have been
updated with authentication details.
Checks authentication header value on inbound SMS requests from
Firetext against a list of allowed API keys set in the application
config.
At the moment, we're only logging the attempts without aborting the
requests. Once this is rolled out to production and we've checked
the logs we'll switch on the aborts and add the tests for 401 and 403
responses.
this involved:
* moving that task to callback_tasks to prevent circular imports
* updating the dummy research mode callbacks (with actual SNS messages from the
ses simulator emails)
* refactoring tests
One of our providers gives us messages with special characters escaped,
ie a newline comes through as `\n`, not a literal newline. We shouldn’t
be showing these backslashes to any of our users. We also have examples
of real inbound messages containing `👍` and `’`, so we should continue
to display these properly.
It’s a bit tricky, because the strings we get from this provider are a
mixture of escape sequences (eg `\n`) and unicode characters (eg `😨`).
So we have to first convert the unicode character `😨` into an escape
sequence, `\U0001f628` in this example. We do this by encoding with
the `raw_unicode_escape` codec:
> Latin-1 encoding with \uXXXX and \UXXXXXXXX for other code points.
> Existing backslashes are not escaped in any way. It is used in the
> Python pickle protocol.
– https://docs.python.org/3/library/codecs.html#text-encodings
Then we turn this back into a string using the `unicode_escape` codec,
which transforms all escape sequences into their literal representations
(eg `\U0001f628` becomes `😨` and `\n` becomes a newline).
NotificationStatistics was added as a spike but didn't work out as expected. This is finally removing all that unused code.
I'll drop the table in the next PR
- Updated the retry and max_retries of the process_ses_results celery
task to be the same as other retry strategies in that file
- Provided a message with the 200 to be similar to how other responses
are handled
In preparation for moving the SNS notification to an SES queue remove
the HTTP errors codes and arguments as the method will now be run by
a celery task. Also made the callback http method return more generic
codes as this will be removed in the longer term.
- Removed errors and arguments returned from process_ses_response
- Updated tests
In other places the text we use for this error message is "Missing personalisation: name, date, thing". See:
- 72b108b694/app/template/rest.py (L125)
- 717c0510a3/app/notifications/rest.py (L206)
- 05a179c6ef/app/v2/template/post_template.py (L38)
For some reason this part of the codebase says "Template missing personalisation: …". This is inconsistent, and also confusing because it’s the API call that’s missing the personalisation, not the template itself.
This commit changes the error message to be consistent with the majority of the codebase, which uses the less confusing wording.
Added clarification to an error message to give better debugging information.
Removed using dao_get_reply_to_by_service_id in tests to be more consistent with other code and use the test db functions or remove the need for a call altogether making the code less complex.
* Added service_email_reply_to_id to the POST /v2/notifications/email and a test to test the validator
* Caught NoResultFound exception in check_service_email_reply_to_id as it was not being caught when there there was no valid service_id or reply_to_id. Fixed failing tests which were not passing due to the NoResultFound exception and added further tests to check for the good path through the code and an test to check for an invalid service_id
* Added service_email_reply_to_id to the POST /v2/notifications/email and a test to test the validator
* Caught NoResultFound exception in check_service_email_reply_to_id as it was not being caught when there there was no valid service_id or reply_to_id. Fixed failing tests which were not passing due to the NoResultFound exception and added further tests to check for the good path through the code and an test to check for an invalid service_id
* Fixed code style in validators.py to confirm with rules
Update the name of email_reply_to_id to conform better with other attributes in the schema and the resultant code in post_notifications.py
Fixed code style in test_validators.py to confirm with rules
Added tests to test_post_notifications.py to test the email_reply_to_id being present and being incorrect, it being optional is being tested by other tests.
* Added service_email_reply_to_id to the POST /v2/notifications/email and a test to test the validator
* Added service_email_reply_to_id to the POST /v2/notifications/email and a test to test the validator
* Caught NoResultFound exception in check_service_email_reply_to_id as it was not being caught when there there was no valid service_id or reply_to_id. Fixed failing tests which were not passing due to the NoResultFound exception and added further tests to check for the good path through the code and an test to check for an invalid service_id
* Caught NoResultFound exception in check_service_email_reply_to_id as it was not being caught when there there was no valid service_id or reply_to_id. Fixed failing tests which were not passing due to the NoResultFound exception and added further tests to check for the good path through the code and an test to check for an invalid service_id
* Fixed code style in validators.py to confirm with rules
Update the name of email_reply_to_id to conform better with other attributes in the schema and the resultant code in post_notifications.py
Fixed code style in test_validators.py to confirm with rules
Added tests to test_post_notifications.py to test the email_reply_to_id being present and being incorrect, it being optional is being tested by other tests.
* Minor update after manual merge to fix check style rule break in test_validators.py where a single space was introduced.
* Updates after code review. Moved the template from the exception message as it was not required and updated the error message to match the field name in the sschema for better debugging and error identification.
* Fixed test after update of exception message
we now no longer create a job. At the end of the post there is no
action, as we don't have any tasks to queue immediately - if it's a
real notification it'll get picked up in the evening scheduled task.
If it's a test notification, we create it with an initial status of
sending so that we can be sure it'll never get picked up - and then we
trigger the update-letter-notifications-to-sent-to-dvla task to sent
the sent-at/by.
1. No longer create jobs when creating letters from api 🎉
2. Bulk update notifications based on the notification references after
we send them to DVLA - either as success or as error
so they can be distinguished on the frontend.
Also, some related cleanup:
* don't show test api letters on the frontpage
* make sure the subject is returned from the API for letters
* make sure the letter's address is returned for letters
there are three steps to this
1. Create a job
* Starts in status 'ready to send'
* Created by None, since it's from the API
* original file name 'letter submitted via api'
2. Create a single notification for that job
* job_row_number 0
* client reference if provided
* address line 1 as recipient
3. Trigger the build_dvla_file task
we know that all the notifications have been created for this job
(since we just created them ourselves synchronously), so this will
just create the dvla-format file for the job, and upload it to s3.
when functions get as big as that, it's confusing to try and work out what
things are what. By including a * as the first arg, we require that anyone
calling the function has to use kwargs to reference the parameters
