this endpoint should probably only be used for the choose-service page
also create an OrganisationBrowsableItem to aid rendering of them
in the front-end.
We were counting users who had the `manage_settings` permission. This
is the old name for it, therefore there would never be any users with
this permission, so the tick would never go green.
The new name for the permission is `manage_service`. This commit fixes
the error, and adds an extra safeguard against something like this
happening again.
rather than allow admins to do everything specifically, we should
only block them from things we conciously don't want them to do.
This is "Don't let platform admins send letters from services they're
not in". Everything else the platform admins can do.
This is step one, adding a restrict_admin_usage flag, and setting that
for those restricted endpoints around creating api keys, uploading CSVs
and sending one-off messages.
Also, this commit separates the two use cases for permissions:
* user.has_permission for access control
* user.has_permission_for_service for user info - this is used for
showing checkboxes on the manage-users page for example
With this, we can remove the admin_override flag from the permission
decorator.
in the db, we have several rows for single permissions - we separate
`send_messages` into `send_texts`, `send_emails` and `send_letters`,
and also `manage_service` into `manage_users` and `manage_settings`.
But on the front end we don't do anything with this distinction. It's
unhelpful for us to have to think about permissions as groups of things
when we can never split them up at all. So we should combine them. This
commit makes sure:
* when user models are read (from JSON direct from the API), we
should transform them from db permissions into roles.
* when permissions are persisted (editing permissions, and creating
invites), we should send db permissions to the API.
All other interaction with permissions (should just be the endpoint
decorator and checks in html templates generally) should use admin
roles.
when added to a service, all users are given the view_activity
permission. So, if that's included in the list, we don't need `any_`,
and we don't need any of the other permissions.
Currently requests to the API made from the admin app are going from
PaaS admin app to the nginx router ELB, which then routes them back
to the api app on PaaS.
This makes sense for external requests, but for requests made from
the admin app we could skip nginx and go directly to the api PaaS
host, which should reduce load on the nginx instances and
potentially reduce latency of the api requests.
API apps on PaaS are checking the X-Custom-Forwarder header (which
is set by nginx on proxy_pass requests) to only allow requests going
through the proxy.
This adds the custom header to the API client requests, so that they
can pass that header check without going through nginx.
One of the things we need to know for a service to go live is whether
they have at least two users with the ‘manage service’ permission.
So this commit adds a method to the client to count how many users have
a given permission. We can do logic on this count later. But having the
counting done in the client feels like a cleaner separation of concerns.
Meant some refactoring of the way `service_id` is extracted from the
request, in order to make it easier to mock.
the update_user fn was used in two places, for things that are handled
fine by update_user_attribute. Reduce complexity in the API by killing
the PUT, which is more dangerous (might silently overwrite things that
shouldn't be, like "last_logged_in_at" etc).
Had to change the code not received mobile number form, and the
activate user function.
the update_user fn was used in two places, for things that are handled
fine by update_user_attribute. Reduce complexity in the API by killing
the PUT, which is more dangerous (might silently overwrite things that
shouldn't be, like "last_logged_in_at" etc).
Had to change the code not received mobile number form, and the
activate user function.
The GDS Way™[1] recommends using Flake8 to lint Python projects.
This commit takes the Flake8 config from Digital Marketplace API[2] and
removes the bits we don’t need.
It changes the `max_complexity` setting to 14, which is the most complex
code we have in this repo currently (we shouldn’t be writing code _more_
complex than what we already have).
This commit also fixes the errors found by Flake8, which includes 6(!)
tests which were never getting run because they had the same names as
existing tests.
Here is a full list of the errors that were found and fixed:
```
./app/__init__.py:2:1: F401 're' imported but unused
./app/__init__.py:4:1: F401 'json' imported but unused
./app/__init__.py:8:1: F401 'dateutil' imported but unused
./app/__init__.py:11:1: F401 'flask.escape' imported but unused
./app/__init__.py:41:1: F401 'app.proxy_fix' imported but unused
./app/__init__.py:129:5: F821 undefined name 'proxy_fix'
./app/__init__.py:221:19: F821 undefined name 'highlight'
./app/__init__.py:221:35: F821 undefined name 'JavascriptLexer'
./app/__init__.py:221:54: F821 undefined name 'HtmlFormatter'
./app/config.py:2:1: F401 'datetime.timedelta' imported but unused
./app/event_handlers.py:2:1: F401 'flask_login.current_user' imported but unused
./app/utils.py:11:1: F401 'dateutil.parser' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.two_factor' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.notifications' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.add_service' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.forgot_password' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.inbound_number' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.styleguide' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.organisations' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.letter_jobs' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.verify' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.conversation' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.api_keys' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.send' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.dashboard' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.jobs' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.manage_users' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.sign_in' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.sign_out' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.code_not_received' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.invites' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.platform_admin' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.providers' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.service_settings' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.index' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.new_password' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.user_profile' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.feedback' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.choose_service' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.templates' imported but unused
./app/main/__init__.py:5:1: F401 'app.main.views.register' imported but unused
./app/main/forms.py:12:1: F401 'wtforms.SelectField' imported but unused
./app/main/views/api_keys.py:37:29: E241 multiple spaces after ':'
./app/main/views/feedback.py:3:1: F401 'flask.flash' imported but unused
./app/main/views/feedback.py:122:17: E123 closing bracket does not match indentation of opening bracket's line
./app/main/views/inbound_number.py:1:1: F401 'flask.url_for' imported but unused
./app/main/views/inbound_number.py:1:1: F401 'flask.session' imported but unused
./app/main/views/inbound_number.py:1:1: F401 'flask.redirect' imported but unused
./app/main/views/inbound_number.py:1:1: F401 'flask.request' imported but unused
./app/main/views/inbound_number.py:13:1: F401 'flask.jsonify' imported but unused
./app/main/views/jobs.py:31:1: F401 'app.utils.get_template' imported but unused
./app/main/views/letter_jobs.py:1:1: F401 'datetime' imported but unused
./app/main/views/letter_jobs.py:6:1: F401 'app.format_datetime_24h' imported but unused
./app/main/views/manage_users.py:111:9: E123 closing bracket does not match indentation of opening bracket's line
./app/main/views/notifications.py:121:5: F841 local variable 'status_args' is assigned to but never used
./app/main/views/organisations.py:1:1: F401 'flask.request' imported but unused
./app/main/views/service_settings.py:77:9: E123 closing bracket does not match indentation of opening bracket's line
./app/main/views/service_settings.py:82:9: E123 closing bracket does not match indentation of opening bracket's line
./app/main/views/service_settings.py:420:13: E123 closing bracket does not match indentation of opening bracket's line
./app/main/views/sign_in.py:12:1: F401 'flask_login.confirm_login' imported but unused
./app/main/views/sign_in.py:17:1: F401 'app.service_api_client' imported but unused
./app/main/views/sign_in.py:62:13: E123 closing bracket does not match indentation of opening bracket's line
./app/main/views/templates.py:4:1: F401 'flask.json' imported but unused
./app/main/views/templates.py:17:1: F401 'notifications_utils.formatters.escape_html' imported but unused
./app/main/views/templates.py:23:1: F401 'app.utils.get_help_argument' imported but unused
./app/main/views/templates.py:64:13: E123 closing bracket does not match indentation of opening bracket's line
./app/notify_client/service_api_client.py:6:1: F401 '.notification_api_client' imported but unused
./app/notify_client/user_api_client.py:1:1: F401 'uuid' imported but unused
./app/notify_client/user_api_client.py:3:1: F401 'flask.session' imported but unused
./tests/__init__.py:1:1: F401 'csv' imported but unused
./tests/app/main/test_asset_fingerprinter.py:2:1: F401 'os' imported but unused
./tests/app/main/test_asset_fingerprinter.py:4:1: F401 'unittest.mock' imported but unused
./tests/app/main/test_asset_fingerprinter.py:98:9: F841 local variable 'string_with_unicode_character' is assigned to but never used
./tests/app/main/test_errorhandlers.py:2:1: F401 'flask.url_for' imported but unused
./tests/app/main/test_permissions.py:26:13: F841 local variable 'response' is assigned to but never used
./tests/app/main/test_placeholder_form.py:3:1: F401 'wtforms.Label' imported but unused
./tests/app/main/test_placeholder_form.py:11:10: F841 local variable 'req' is assigned to but never used
./tests/app/main/test_two_factor_form.py:10:67: F841 local variable 'req' is assigned to but never used
./tests/app/main/test_two_factor_form.py:23:65: F841 local variable 'req' is assigned to but never used
./tests/app/main/test_two_factor_form.py:37:48: F841 local variable 'req' is assigned to but never used
./tests/app/main/test_two_factor_form.py:51:67: F841 local variable 'req' is assigned to but never used
./tests/app/main/test_two_factor_form.py:65:67: F841 local variable 'req' is assigned to but never used
./tests/app/main/views/test_accept_invite.py:356:5: F841 local variable 'element' is assigned to but never used
./tests/app/main/views/test_activity.py:11:1: F811 redefinition of unused 'mock_get_notifications' from line 11
./tests/app/main/views/test_activity.py:18:1: F401 'datetime.datetime' imported but unused
./tests/app/main/views/test_activity.py:102:5: F841 local variable 'content' is assigned to but never used
./tests/app/main/views/test_activity.py:104:5: F841 local variable 'notification' is assigned to but never used
./tests/app/main/views/test_activity.py:337:5: F841 local variable '_notifications_mock' is assigned to but never used
./tests/app/main/views/test_activity.py:373:13: E126 continuation line over-indented for hanging indent
./tests/app/main/views/test_activity.py:378:9: E121 continuation line under-indented for hanging indent
./tests/app/main/views/test_activity.py:404:13: E126 continuation line over-indented for hanging indent
./tests/app/main/views/test_activity.py:407:9: E121 continuation line under-indented for hanging indent
./tests/app/main/views/test_api_keys.py:354:5: F841 local variable 'response' is assigned to but never used
./tests/app/main/views/test_conversation.py:5:1: F401 'bs4.BeautifulSoup' imported but unused
./tests/app/main/views/test_conversation.py:198:5: F841 local variable 'mock_get_inbound_sms' is assigned to but never used
./tests/app/main/views/test_dashboard.py:53:5: F841 local variable 'mock_template_stats' is assigned to but never used
./tests/app/main/views/test_dashboard.py:72:5: F841 local variable 'mock_template_stats' is assigned to but never used
./tests/app/main/views/test_jobs.py:2:1: F401 'uuid' imported but unused
./tests/app/main/views/test_jobs.py:3:1: F401 'urllib.parse.urlparse' imported but unused
./tests/app/main/views/test_jobs.py:3:1: F401 'urllib.parse.quote' imported but unused
./tests/app/main/views/test_jobs.py:3:1: F401 'urllib.parse.parse_qs' imported but unused
./tests/app/main/views/test_jobs.py:9:1: F401 'app.main.views.jobs.get_status_filters' imported but unused
./tests/app/main/views/test_jobs.py:10:1: F401 'tests.notification_json' imported but unused
./tests/app/main/views/test_letters.py:6:1: F401 'tests.service_json' imported but unused
./tests/app/main/views/test_notifications.py:5:1: F401 'app.utils.REQUESTED_STATUSES' imported but unused
./tests/app/main/views/test_notifications.py:5:1: F401 'app.utils.DELIVERED_STATUSES' imported but unused
./tests/app/main/views/test_notifications.py:5:1: F401 'app.utils.SENDING_STATUSES' imported but unused
./tests/app/main/views/test_notifications.py:5:1: F401 'app.utils.FAILURE_STATUSES' imported but unused
./tests/app/main/views/test_platform_admin.py:242:13: E126 continuation line over-indented for hanging indent
./tests/app/main/views/test_platform_admin.py:247:13: E126 continuation line over-indented for hanging indent
./tests/app/main/views/test_send.py:3:1: F401 'unittest.mock.Mock' imported but unused
./tests/app/main/views/test_send.py:18:1: F811 redefinition of unused 'mock_get_service' from line 18
./tests/app/main/views/test_send.py:18:1: F401 'tests.conftest.multiple_letter_contact_blocks' imported but unused
./tests/app/main/views/test_send.py:18:1: F401 'tests.conftest.no_sms_senders' imported but unused
./tests/app/main/views/test_send.py:18:1: F401 'tests.conftest.multiple_sms_senders' imported but unused
./tests/app/main/views/test_send.py:18:1: F401 'tests.conftest.no_letter_contact_blocks' imported but unused
./tests/app/main/views/test_send.py:102:5: F841 local variable 'response' is assigned to but never used
./tests/app/main/views/test_send.py:870:5: F841 local variable 'response' is assigned to but never used
./tests/app/main/views/test_send.py:1367:5: F841 local variable 'service_id' is assigned to but never used
./tests/app/main/views/test_send.py:1451:13: E126 continuation line over-indented for hanging indent
./tests/app/main/views/test_send.py:1620:80: E226 missing whitespace around arithmetic operator
./tests/app/main/views/test_send.py:1909:13: E126 continuation line over-indented for hanging indent
./tests/app/main/views/test_send.py:1912:9: E121 continuation line under-indented for hanging indent
./tests/app/main/views/test_service_settings.py:13:1: F811 redefinition of unused 'no_reply_to_email_addresses' from line 13
./tests/app/main/views/test_service_settings.py:13:1: F401 'tests.conftest.single_reply_to_email_address' imported but unused
./tests/app/main/views/test_service_settings.py:28:5: E123 closing bracket does not match indentation of opening bracket's line
./tests/app/main/views/test_service_settings.py:104:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:166:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:186:5: F841 local variable 'mocked_get_fn' is assigned to but never used
./tests/app/main/views/test_service_settings.py:217:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:237:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:257:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:307:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:340:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:466:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:555:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:615:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:719:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:874:5: F841 local variable 'page' is assigned to but never used
./tests/app/main/views/test_service_settings.py:902:5: F841 local variable 'page' is assigned to but never used
./tests/app/main/views/test_service_settings.py:954:5: F841 local variable 'page' is assigned to but never used
./tests/app/main/views/test_service_settings.py:986:5: F841 local variable 'page' is assigned to but never used
./tests/app/main/views/test_service_settings.py:1101:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1121:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1271:1: F811 redefinition of unused 'test_set_letter_contact_block_saves' from line 1189
./tests/app/main/views/test_service_settings.py:1433:5: F841 local variable 'page' is assigned to but never used
./tests/app/main/views/test_service_settings.py:1495:5: F841 local variable 'mocked_get_fn' is assigned to but never used
./tests/app/main/views/test_service_settings.py:1540:5: F841 local variable 'mocked_get_fn' is assigned to but never used
./tests/app/main/views/test_service_settings.py:1570:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1589:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1621:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1641:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1658:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1676:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1697:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1759:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_service_settings.py:1775:1: F811 redefinition of unused 'single_reply_to_email_address' from line 13
./tests/app/main/views/test_templates.py:3:1: F401 'uuid' imported but unused
./tests/app/main/views/test_templates.py:11:1: F401 'tests.conftest.mock_get_user' imported but unused
./tests/app/main/views/test_templates.py:514:1: F811 redefinition of unused 'mock_get_user' from line 11
./tests/app/main/views/test_templates.py:672:1: F811 redefinition of unused 'mock_get_user' from line 11
./tests/app/main/views/test_templates.py:795:1: F811 redefinition of unused 'mock_get_user' from line 11
./tests/app/main/views/test_templates.py:835:1: F811 redefinition of unused 'mock_get_user' from line 11
./tests/app/main/views/test_two_factor.py:67:13: E126 continuation line over-indented for hanging indent
./tests/app/notify_client/test_notification_client.py:79:5: F841 local variable 'mock_post' is assigned to but never used
```
1. https://gds-way.cloudapps.digital/manuals/programming-languages/python/linting.html#how-to-use-flake8
2. d5ab8afef4/.flake8
Because we’re setting the API key and service ID after calling the
`__init__` method of the client it wasn’t doing the thing where it
splits the combined key into the two individual UUIDs. So we still need
to set them directly, individually on the client.
The Notify API client changed in version 4 to take two arguments, not
three (service ID was removed in favour of the combined API key).
This gets a bit gnarly because the API key has to be at least a certain
length so it can be substringed internally.
The user has 10 tries at the password, after which the account is locked.
The same is true for the verify code, the user will have 10 tries before the user account is locked.
specifically, the 2FA page when you first create an account is different to the login 2FA page
and also the 2FA page when you change your phone number is different as well
when a user enters their 2FA code, the API will store a random UUID
against them in the database - this code is then stored on the cookie
on the front end.
At the beginning of each authenticated request, we do the following
steps:
* Retrieve the user's cookie, and get the user_id from it
* Request that user's details from the database
* populate current_user with the DB model
* run the login_required decorator, which calls
current_user.is_authenticated
is_authenticated now also checks that the database model matches the
cookie for session_id. The potential states and meanings are as follows:
database | cookie | meaning
----------+--------+---------
None | None | New user, or system just been deployed.
| | Redirect to start page.
----------+--------+---------
'abc' | None | New browser (or cleared cookies). Redirect to
| | start page.
----------+--------+---------
None | 'abc' | Invalid state (cookie is set from user obj, so
| | would only happen if DB is cleared)
----------+--------+---------
'abc' | 'abc' | Same browser. Business as usual
----------+--------+---------
'abc' | 'def' | Different browser in cookie - db has been changed
| | since then. Redirect to start
This PR changes the flow to change an email address.
Once the user enter their password, they are told "Check your email".
An email has been sent to them containing a link to notify which contains an encrypted token.
The encrypted token contains the user id and new email address. Once the link is clicked the user's email address is updated to the new email address.
They are redirected to the /user-profile page.
Also in this commit is an update from flask.ext.login to flask_login.
The clients never get passed useful values to their `__init__` methods.
Rather the real values are passed through later using the `init_app`
method.
So it should be an error if the client is relying on the values that
get passed to it’s init method. Easiest way to ensure this is by making
the `__init__` method not expect any arguments and passing fake values
to the `Super` call.
when visited sends sms code for second step of account verification.
At that second step user enters just sms code sent to users mobile
number.
Also moved dao calls that simply proxied calls to client to calling
client directly.
There is still a place where a user will be a sent a code for
verification to their email namely if they update email address.
