At the moment there’s nothing in Notify that says the logo on letter
templates can be changed from ‘HM Government’. While some people guess
that it’s possible, or are motivated enough to enquire, others might be
assuming that the branding can’t be changed (and thus deciding Notify
letters aren’t for them).
In the future we should make this process slicker (eg with a file
upload) but the easiest, quickest improvement to make for now is:
- let people know that the branding can be changed
- direct them to support if they do want to change it
This is better than just keying into the JSON because it means you get
an exception straight away when looking up a key that doesn’t exist
(which via mocking you could ordinarily miss).
Currently we have a bunch of users who aren’t signed in asking us for
the agreement.
This is bad because:
- it’s slower (for them) than just being able to download it
- it creates work for us
We can’t just offer the agreement to anyone, but we can offer to it to
anyone who’s signed in because we now let people self-select which
version to download when we can’t tell which one to give them.
p1 == "should notify team be alerted of this (via pagerduty)"
urgent == "should the user be told we'll look at it"
* If it's in office hours, it's always urgent. It's never a P1 because
we'll notice it anyway
* If it's outside of office hours, it's urgent and P1 if it's severe,
otherwise it's neither
I don’t think it’s a massive risk (we’re certainly mitigating against
any XSS), but having a page on a GOV.UK domain where you can prefill
text on the page from a query string probably isn’t great.
So this commit restricts prefilling the support form to a set of
named questions.
Done using isort[1], with the following command:
```
isort -rc ./app ./tests
```
Adds linting to the `run_tests.sh` script to stop badly-sorted imports
getting re-introduced.
Chosen style is ‘Vertical Hanging Indent’ with trailing commas, because
I think it gives the cleanest diffs, eg:
```
from third_party import (
lib1,
lib2,
lib3,
lib4,
)
```
1. https://pypi.python.org/pypi/isort
The GDS Way™[1] recommends using Flake8 to lint Python projects.
This commit takes the Flake8 config from Digital Marketplace API[2] and
removes the bits we don’t need.
It changes the `max_complexity` setting to 14, which is the most complex
code we have in this repo currently (we shouldn’t be writing code _more_
complex than what we already have).
This commit also fixes the errors found by Flake8, which includes 6(!)
tests which were never getting run because they had the same names as
existing tests.
Here is a full list of the errors that were found and fixed:
```
./app/__init__.py:2:1: F401 're' imported but unused
./app/__init__.py:4:1: F401 'json' imported but unused
./app/__init__.py:8:1: F401 'dateutil' imported but unused
./app/__init__.py:11:1: F401 'flask.escape' imported but unused
./app/__init__.py:41:1: F401 'app.proxy_fix' imported but unused
./app/__init__.py:129:5: F821 undefined name 'proxy_fix'
./app/__init__.py:221:19: F821 undefined name 'highlight'
./app/__init__.py:221:35: F821 undefined name 'JavascriptLexer'
./app/__init__.py:221:54: F821 undefined name 'HtmlFormatter'
./app/config.py:2:1: F401 'datetime.timedelta' imported but unused
./app/event_handlers.py:2:1: F401 'flask_login.current_user' imported but unused
./app/utils.py:11:1: F401 'dateutil.parser' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.two_factor' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.notifications' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.add_service' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.forgot_password' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.inbound_number' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.styleguide' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.organisations' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.letter_jobs' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.verify' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.conversation' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.api_keys' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.send' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.dashboard' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.jobs' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.manage_users' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.sign_in' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.sign_out' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.code_not_received' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.invites' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.platform_admin' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.providers' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.service_settings' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.index' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.new_password' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.user_profile' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.feedback' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.choose_service' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.templates' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.register' imported but unused
./app/main/forms.py:12:1: F401 'wtforms.SelectField' imported but unused
./app/main/views/api_keys.py:37:29: E241 multiple spaces after ':'
./app/main/views/feedback.py:3:1: F401 'flask.flash' imported but unused
./app/main/views/feedback.py:122:17: E123 closing bracket does not match indentation of opening bracket's line
./app/main/views/inbound_number.py:1:1: F401 'flask.url_for' imported but unused
./app/main/views/inbound_number.py:1:1: F401 'flask.session' imported but unused
./app/main/views/inbound_number.py:1:1: F401 'flask.redirect' imported but unused
./app/main/views/inbound_number.py:1:1: F401 'flask.request' imported but unused
./app/main/views/inbound_number.py:13:1: F401 'flask.jsonify' imported but unused
./app/main/views/jobs.py:31:1: F401 'app.utils.get_template' imported but unused
./app/main/views/letter_jobs.py:1:1: F401 'datetime' imported but unused
./app/main/views/letter_jobs.py:6:1: F401 'app.format_datetime_24h' imported but unused
./app/main/views/manage_users.py:111:9: E123 closing bracket does not match indentation of opening bracket's line
./app/main/views/notifications.py:121:5: F841 local variable 'status_args' is assigned to but never used
./app/main/views/organisations.py:1:1: F401 'flask.request' imported but unused
./app/main/views/service_settings.py:77:9: E123 closing bracket does not match indentation of opening bracket's line
./app/main/views/service_settings.py:82:9: E123 closing bracket does not match indentation of opening bracket's line
./app/main/views/service_settings.py:420:13: E123 closing bracket does not match indentation of opening bracket's line
./app/main/views/sign_in.py:12:1: F401 'flask_login.confirm_login' imported but unused
./app/main/views/sign_in.py:17:1: F401 'app.service_api_client' imported but unused
./app/main/views/sign_in.py:62:13: E123 closing bracket does not match indentation of opening bracket's line
./app/main/views/templates.py:4:1: F401 'flask.json' imported but unused
./app/main/views/templates.py:17:1: F401 'notifications_utils.formatters.escape_html' imported but unused
./app/main/views/templates.py:23:1: F401 'app.utils.get_help_argument' imported but unused
./app/main/views/templates.py:64:13: E123 closing bracket does not match indentation of opening bracket's line
./app/notify_client/service_api_client.py:6:1: F401 '.notification_api_client' imported but unused
./app/notify_client/user_api_client.py:1:1: F401 'uuid' imported but unused
./app/notify_client/user_api_client.py:3:1: F401 'flask.session' imported but unused
./tests/__init__.py:1:1: F401 'csv' imported but unused
./tests/app/main/test_asset_fingerprinter.py:2:1: F401 'os' imported but unused
./tests/app/main/test_asset_fingerprinter.py:4:1: F401 'unittest.mock' imported but unused
./tests/app/main/test_asset_fingerprinter.py:98:9: F841 local variable 'string_with_unicode_character' is assigned to but never used
./tests/app/main/test_errorhandlers.py:2:1: F401 'flask.url_for' imported but unused
./tests/app/main/test_permissions.py:26:13: F841 local variable 'response' is assigned to but never used
./tests/app/main/test_placeholder_form.py:3:1: F401 'wtforms.Label' imported but unused
./tests/app/main/test_placeholder_form.py:11:10: F841 local variable 'req' is assigned to but never used
./tests/app/main/test_two_factor_form.py:10:67: F841 local variable 'req' is assigned to but never used
./tests/app/main/test_two_factor_form.py:23:65: F841 local variable 'req' is assigned to but never used
./tests/app/main/test_two_factor_form.py:37:48: F841 local variable 'req' is assigned to but never used
./tests/app/main/test_two_factor_form.py:51:67: F841 local variable 'req' is assigned to but never used
./tests/app/main/test_two_factor_form.py:65:67: F841 local variable 'req' is assigned to but never used
./tests/app/main/views/test_accept_invite.py:356:5: F841 local variable 'element' is assigned to but never used
./tests/app/main/views/test_activity.py:11:1: F811 redefinition of unused 'mock_get_notifications' from line 11
./tests/app/main/views/test_activity.py:18:1: F401 'datetime.datetime' imported but unused
./tests/app/main/views/test_activity.py:102:5: F841 local variable 'content' is assigned to but never used
./tests/app/main/views/test_activity.py:104:5: F841 local variable 'notification' is assigned to but never used
./tests/app/main/views/test_activity.py:337:5: F841 local variable '_notifications_mock' is assigned to but never used
./tests/app/main/views/test_activity.py:373:13: E126 continuation line over-indented for hanging indent
./tests/app/main/views/test_activity.py:378:9: E121 continuation line under-indented for hanging indent
./tests/app/main/views/test_activity.py:404:13: E126 continuation line over-indented for hanging indent
./tests/app/main/views/test_activity.py:407:9: E121 continuation line under-indented for hanging indent
./tests/app/main/views/test_api_keys.py:354:5: F841 local variable 'response' is assigned to but never used
./tests/app/main/views/test_conversation.py:5:1: F401 'bs4.BeautifulSoup' imported but unused
./tests/app/main/views/test_conversation.py:198:5: F841 local variable 'mock_get_inbound_sms' is assigned to but never used
./tests/app/main/views/test_dashboard.py:53:5: F841 local variable 'mock_template_stats' is assigned to but never used
./tests/app/main/views/test_dashboard.py:72:5: F841 local variable 'mock_template_stats' is assigned to but never used
./tests/app/main/views/test_jobs.py:2:1: F401 'uuid' imported but unused
./tests/app/main/views/test_jobs.py:3:1: F401 'urllib.parse.urlparse' imported but unused
./tests/app/main/views/test_jobs.py:3:1: F401 'urllib.parse.quote' imported but unused
./tests/app/main/views/test_jobs.py:3:1: F401 'urllib.parse.parse_qs' imported but unused
./tests/app/main/views/test_jobs.py:9:1: F401 'app.main.views.jobs.get_status_filters' imported but unused
./tests/app/main/views/test_jobs.py:10:1: F401 'tests.notification_json' imported but unused
./tests/app/main/views/test_letters.py:6:1: F401 'tests.service_json' imported but unused
./tests/app/main/views/test_notifications.py:5:1: F401 'app.utils.REQUESTED_STATUSES' imported but unused
./tests/app/main/views/test_notifications.py:5:1: F401 'app.utils.DELIVERED_STATUSES' imported but unused
./tests/app/main/views/test_notifications.py:5:1: F401 'app.utils.SENDING_STATUSES' imported but unused
./tests/app/main/views/test_notifications.py:5:1: F401 'app.utils.FAILURE_STATUSES' imported but unused
./tests/app/main/views/test_platform_admin.py:242:13: E126 continuation line over-indented for hanging indent
./tests/app/main/views/test_platform_admin.py:247:13: E126 continuation line over-indented for hanging indent
./tests/app/main/views/test_send.py:3:1: F401 'unittest.mock.Mock' imported but unused
./tests/app/main/views/test_send.py:18:1: F811 redefinition of unused 'mock_get_service' from line 18
./tests/app/main/views/test_send.py:18:1: F401 'tests.conftest.multiple_letter_contact_blocks' imported but unused
./tests/app/main/views/test_send.py:18:1: F401 'tests.conftest.no_sms_senders' imported but unused
./tests/app/main/views/test_send.py:18:1: F401 'tests.conftest.multiple_sms_senders' imported but unused
./tests/app/main/views/test_send.py:18:1: F401 'tests.conftest.no_letter_contact_blocks' imported but unused
./tests/app/main/views/test_send.py:102:5: F841 local variable 'response' is assigned to but never used
./tests/app/main/views/test_send.py:870:5: F841 local variable 'response' is assigned to but never used
./tests/app/main/views/test_send.py:1367:5: F841 local variable 'service_id' is assigned to but never used
./tests/app/main/views/test_send.py:1451:13: E126 continuation line over-indented for hanging indent
./tests/app/main/views/test_send.py:1620:80: E226 missing whitespace around arithmetic operator
./tests/app/main/views/test_send.py:1909:13: E126 continuation line over-indented for hanging indent
./tests/app/main/views/test_send.py:1912:9: E121 continuation line under-indented for hanging indent
./tests/app/main/views/test_service_settings.py:13:1: F811 redefinition of unused 'no_reply_to_email_addresses' from line 13
./tests/app/main/views/test_service_settings.py:13:1: F401 'tests.conftest.single_reply_to_email_address' imported but unused
./tests/app/main/views/test_service_settings.py:28:5: E123 closing bracket does not match indentation of opening bracket's line
./tests/app/main/views/test_service_settings.py:104:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:166:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:186:5: F841 local variable 'mocked_get_fn' is assigned to but never used
./tests/app/main/views/test_service_settings.py:217:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:237:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:257:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:307:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:340:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:466:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:555:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:615:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:719:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:874:5: F841 local variable 'page' is assigned to but never used
./tests/app/main/views/test_service_settings.py:902:5: F841 local variable 'page' is assigned to but never used
./tests/app/main/views/test_service_settings.py:954:5: F841 local variable 'page' is assigned to but never used
./tests/app/main/views/test_service_settings.py:986:5: F841 local variable 'page' is assigned to but never used
./tests/app/main/views/test_service_settings.py:1101:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1121:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1271:1: F811 redefinition of unused 'test_set_letter_contact_block_saves' from line 1189
./tests/app/main/views/test_service_settings.py:1433:5: F841 local variable 'page' is assigned to but never used
./tests/app/main/views/test_service_settings.py:1495:5: F841 local variable 'mocked_get_fn' is assigned to but never used
./tests/app/main/views/test_service_settings.py:1540:5: F841 local variable 'mocked_get_fn' is assigned to but never used
./tests/app/main/views/test_service_settings.py:1570:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1589:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1621:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1641:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1658:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1676:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1697:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1759:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1775:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_templates.py:3:1: F401 'uuid' imported but unused
./tests/app/main/views/test_templates.py:11:1: F401 'tests.conftest.mock_get_user' imported but unused
./tests/app/main/views/test_templates.py:514:1: F811 redefinition of unused 'mock_get_user' from line 11
./tests/app/main/views/test_templates.py:672:1: F811 redefinition of unused 'mock_get_user' from line 11
./tests/app/main/views/test_templates.py:795:1: F811 redefinition of unused 'mock_get_user' from line 11
./tests/app/main/views/test_templates.py:835:1: F811 redefinition of unused 'mock_get_user' from line 11
./tests/app/main/views/test_two_factor.py:67:13: E126 continuation line over-indented for hanging indent
./tests/app/notify_client/test_notification_client.py:79:5: F841 local variable 'mock_post' is assigned to but never used
```
1. https://gds-way.cloudapps.digital/manuals/programming-languages/python/linting.html#how-to-use-flake8
2. d5ab8afef4/.flake8
Cool URIs don’t change 😎https://www.w3.org/Provider/Style/URI
We still have links to `/feedback` in our emails. These will live in people’s inboxes forever.
The support flow was using `yes` and `no` to mean emergency/not
emergency. But not in all places – in one place it was using
`True`/`False` instead.
We were treating anything other than `yes`/`no` as a non-answer, which
means ask the question again. Because of the `True`/`False` thing, there
was no way of the user providing a valid `yes`/`no` answer. Which means
that we just kept asking them the question again and again and they got
stuck in a loop.
`severe` can mean one of three things:
- `yes` – user has told us this is an emergency
- `no` – user has told us this isn’t an emergency
- Anything else – user hasn’t been asked the question or has
hacked/mangled the URL
This commit adds some stricter sanitisation of the `severe` query
parameter and does so up front, rather than spreading it across multiple
functions.
`replace` doesn’t convert a time from one timezone to another. It just
changes the label that says what timezone a time is in 😬
`.localize` is how we handle these kind of issues in the API (see
d0b467b2fb/app/utils.py (L42-L44) )
So this commit changes the calculation to use `.localize`, and makes the
tests timezone aware to check we’re doing this right.
Using and/or over any/all has a couple of advantages:
- it's a bit quicker
- it won't evaluate the second half at all if the first half fails – if
it is in business hours, and convert_to_boolean would raise, with your
use of all we'd throw a 500, whereas if we had or, business_hours
would trip and we'd skip over the second half without worrying about
exceptions
any and all are designed for use with variable length args eg
`any(x for x in thing())`
If you report a problem we want to be able to get back to you to find
out more information, or to update you on the status of a fix. So it
shouldn’t be possible to report a problem without providing an email
address.
This commit makes `email_address` a required field when `ticket_type` is
problem.
This requires a bit of fiddling with the tests which weren’t expecting
to have to provide an email address. So the tests now either:
- pass an email address
- check for an error when they don’t pass an email address
This is a real edge case, but it seems worth handling.
How you’d get to this case:
- it’s 5:29pm and you start to describe the problem you’re having
- it’s 5:31pm and you click ‘submit’
- you’re redirected to the triage page because we’re now out of hours
- you click ‘this is a serious problem’
What would be bad thing to happen:
- you’re back on the message page and all the stuff you’ve written is
gone
What would be a good thing to happen:
- we save the message in a session so that you can check it again before
sending it
Generally I prefer confirmation pages to the flash message thing
(they’re harder to miss). So this commit adds one.
It also adds some logic to this page, so that, depending what the user
has told us about the thing they’ve submitted, we can tell them how
quickly to expect a response.
TL;DR, as much as possible we should work out how to prioritise tickets
and not put that burden on the user. However, there are some cases where
we can’t.
In business hours all tickets are high priority, ie we will at least
acknowledge them within 30 mins.
If we are not in business hours then we need to know if a ticket is
serious enough to get someone out of bed. Only the user can tell us
this, but we can give them some examples to help them decide.
In addition, out-of-hours tickets are only a priority if the user has
live services. Normally we can determine this and do the
priority-setting in the background.
If they can’t log in then we can’t determine what services they have. So
in this case they will need to use the emergency email address, which
only users with live services will have.
The logic for this gets fairly complex. It might be to easier to
understand what’s going on by walking through the test cases, which are
a bit more declarative.
N.B. Deskpro’s ‘urgency’ is descending, eg 10 is the most urgent and 1
is the least.
If a user is logged in then we already know their name and email
address. So there’s no need for them to fill them again on the support
form.
One concern we might have about this is the user not realising we’re
doing this, and the feedback form looking like a bit of a black hole.
So we’re replaying their email address on this page to reassure them
that:
- we know who they are
- and that they’ll get a reply
The kind of communications we’re getting at the moment can broadly be
broken down into:
- problems
- questions and feedback
We will need to triage problems differently, because they could
potentially be urgent/severe/emergency/P1/whatever language we use.
Questions or feedback will never be P1.
Two reasons for making the user categorise their tickets themselves:
- Outside of hours we can’t get someone out of bed in order to decide if
a ticket is a problem or just feedback
- We can tailor the subsequent pages to whether it’s a problem or
feedback (eg showing a link to the status page if the user is having
a problem)
This commit let’s users make the choice with a pair of radio buttons.
It also cleans up a bunch of the tests and parameterizes them so we’re
testing the flow for both ticket types.
Our support process is about to get more fully fledged so we’ll need
an index page to route people properly.
We reckon that users will also want to know what the support process is,
so let’s explain it on this page.
This commit:
- moves things around a bit on the request to go live page
- sticks a textbox in there
So when someone click the big green button, we will get a support ticket
that looks something like:
```
From Test User <test@user.gov.uk> on behalf of Test Service
(6ce466d0-fd6a-11e5-82f5-e0accb9d11a6)
---
We’ll send about 1000 text messages in the first month, and then 10,000
text messages per month after that. Usage of our service is about 50%
higher in March, at the end of the tax year.
```
