As we gradually move from statsd to prometheus, we change the metric to
be a prometheus metric rather than statsd.
The change worth pointing out is that we have dropped the 'successful'
and 'failed' statuses from the metrics. I don't think it's useful to
have these statuses. It's very rare for an inbound message to fail when
we receive it and when it does, we raise an error and see it in our
logs. We aren't going to be looking at a graph of it as it's a rare
event, not typical behaviour that we want to monitor with a graph.
This PR tries to parse the date, if that throws an error return now as the datereceived. This will at least allow the message to be persisted. Typically the DateReceived, provider_date, and the created_at date in the inbound_sms table are within a second of each other.
Checks authentication header value on inbound SMS requests from
MMG against a list of allowed API keys set in the application
config.
At the moment, we're only logging the attempts without aborting the
requests. Once this is rolled out to production and we've checked
the logs we'll switch on the aborts and add the tests for 401 and 403
responses.
This work has already been done for Firetext in a previous PR:
https://github.com/alphagov/notifications-api/pull/1409
Switches on authentication checks for Firetext inbound SMS callbacks.
This should only be released once Firetext callback URLs have been
updated with authentication details.
One of our providers gives us messages with special characters escaped,
ie a newline comes through as `\n`, not a literal newline. We shouldn’t
be showing these backslashes to any of our users. We also have examples
of real inbound messages containing `👍` and `’`, so we should continue
to display these properly.
It’s a bit tricky, because the strings we get from this provider are a
mixture of escape sequences (eg `\n`) and unicode characters (eg `😨`).
So we have to first convert the unicode character `😨` into an escape
sequence, `\U0001f628` in this example. We do this by encoding with
the `raw_unicode_escape` codec:
> Latin-1 encoding with \uXXXX and \UXXXXXXXX for other code points.
> Existing backslashes are not escaped in any way. It is used in the
> Python pickle protocol.
– https://docs.python.org/3/library/codecs.html#text-encodings
Then we turn this back into a string using the `unicode_escape` codec,
which transforms all escape sequences into their literal representations
(eg `\U0001f628` becomes `😨` and `\n` becomes a newline).
Refactor tests/db/create_service() to behave more like the real world.
Created new create_service_with_inbound_number and create_service_with_defined_sms_sender() test/db methods.
This will need to be refactored after the deployment of api and admin and after the update script for existing services using inbound numbers has been executed.
If the service has not set the url then nothing happens.
If the request to the service url returns with 500 or greater the task is retries.
The task is created when the SMS provider post the inbound SMS.
rather than using the `normalise_phone_number` function, use the
`validate_and_format_phone_number` function - this will also convert
all numbers to international format, which means we won't need to
worry about whether the user enters internaional or UK phone numbers
when searching
the DateRecieved field from MMG comes in with +s instead of spaces,
and uriencoded (the same as how they format their messages)
Make sure we decode this, and then convert to a UTC timestamp
