One of our providers gives us messages with special characters escaped,
ie a newline comes through as `\n`, not a literal newline. We shouldn’t
be showing these backslashes to any of our users. We also have examples
of real inbound messages containing `👍` and `’`, so we should continue
to display these properly.
It’s a bit tricky, because the strings we get from this provider are a
mixture of escape sequences (eg `\n`) and unicode characters (eg `😨`).
So we have to first convert the unicode character `😨` into an escape
sequence, `\U0001f628` in this example. We do this by encoding with
the `raw_unicode_escape` codec:
> Latin-1 encoding with \uXXXX and \UXXXXXXXX for other code points.
> Existing backslashes are not escaped in any way. It is used in the
> Python pickle protocol.
– https://docs.python.org/3/library/codecs.html#text-encodings
Then we turn this back into a string using the `unicode_escape` codec,
which transforms all escape sequences into their literal representations
(eg `\U0001f628` becomes `😨` and `\n` becomes a newline).
Refactor tests/db/create_service() to behave more like the real world.
Created new create_service_with_inbound_number and create_service_with_defined_sms_sender() test/db methods.
This will need to be refactored after the deployment of api and admin and after the update script for existing services using inbound numbers has been executed.
If the service has not set the url then nothing happens.
If the request to the service url returns with 500 or greater the task is retries.
The task is created when the SMS provider post the inbound SMS.
rather than using the `normalise_phone_number` function, use the
`validate_and_format_phone_number` function - this will also convert
all numbers to international format, which means we won't need to
worry about whether the user enters internaional or UK phone numbers
when searching
the DateRecieved field from MMG comes in with +s instead of spaces,
and uriencoded (the same as how they format their messages)
Make sure we decode this, and then convert to a UTC timestamp
