Commit Graph

6102 Commits

Author SHA1 Message Date
Chris Hill-Scott
28ea75728c Return domains in get organisation response
We need this so we can disply them in the admin app.
2019-03-20 11:56:17 +00:00
Pea (Malgorzata Tyczynska)
a46e7d62f2 Merge pull request #2404 from alphagov/test-managing-multiple-folder-users-simultaneously
Test update_template_folder can manage multiple users at once
2019-03-19 14:11:41 +00:00
Pea Tyczynska
09731c109e Test update_template_folder can manage multiple users at once 2019-03-19 11:48:37 +00:00
Pea (Malgorzata Tyczynska)
7b8a478b7a Merge pull request #2401 from alphagov/update-folder-viewing-permissions
Update folder viewing permissions when updating a folder
2019-03-19 10:59:58 +00:00
Pea Tyczynska
dabcbc20db Update folder viewing permissions when updating a folder 2019-03-18 17:49:47 +00:00
Chris Hill-Scott
89d5a640b2 Merge pull request #2403 from alphagov/bump-utils-non-plussed-lists
Bump utils
2019-03-18 16:11:31 +00:00
Chris Hill-Scott
f961110603 Bump utils
Brings in:
- [ ] https://github.com/alphagov/notifications-utils/pull/592
2019-03-18 16:01:08 +00:00
Rebecca Law
69e1b47b36 Merge pull request #2402 from alphagov/reduce-zip-file-further
Reduce number of files per zip file
2019-03-18 13:23:34 +00:00
Rebecca Law
be9daf3454 Until we can fix it properly, changing the max number of files to 500. Hopefully the task will finish in less than 5 minutes. 2019-03-18 13:13:05 +00:00
Katie Smith
71c1e2e1be Merge pull request #2395 from alphagov/change-permissions-data-format
Allow add_user_to_service endpoint to accept new data format
2019-03-18 10:03:44 +00:00
Rebecca Law
2a711390da Merge pull request #2399 from alphagov/reduce-zip-file-size
Reduce the number of files sent to the zip-and-send-letter-pdfs task
2019-03-15 16:07:36 +00:00
Rebecca Law
94e0b8b4eb Reduce the number of files sent to the zip-and-send-letter-pdfs
The ftp application is struggling, running out of CPU. This is attempt to help with that.
2019-03-15 15:46:09 +00:00
Katie Smith
d4757c9208 Allow add_user_to_service endpoint to accept new data format
The data posted to the `add_user_to_service` endpoint is currently sent as a
list of permissions:
`[{'permission': MANAGE_SETTINGS}, {'permission': MANAGE_TEMPLATES}]`.

This endpoint is going to also be used for folder permissions, so the
data now needs to be nested:

`{'permissions': [{'permission': MANAGE_SETTINGS}, {'permission': MANAGE_TEMPLATES}]}`

This changes the add_user_to_service endpoint to accept data in either
format. Once admin is sending data in the new format, the code can be
simplified.
2019-03-14 16:59:48 +00:00
Leo Hemsted
6f4b415426 Merge pull request #2397 from alphagov/slow-provider-switch
lower provider switch threshold from 20% to 30%
2019-03-14 16:22:52 +00:00
Leo Hemsted
3a746dd4a2 Merge pull request #2396 from alphagov/service-error-handler
fix constraint name check in service error handler
2019-03-14 16:15:12 +00:00
Leo Hemsted
2f94e1d9bc lower provider switch threshold from 20% to 30%
make it less likely to switch on slow messages to allow more manual
control of provider balance
2019-03-14 16:11:59 +00:00
Leo Hemsted
1efee1fa66 fix constraint name check in service error handler 2019-03-14 12:09:29 +00:00
Alexey Bezhan
2825c3d384 Merge pull request #2394 from alphagov/fix-remove-user-from-service-folder-integrity-error
Change dao_remove_user_from_service query to avoid IntegrityError
2019-03-14 10:54:03 +00:00
Pea (Malgorzata Tyczynska)
aee7dfefb6 Merge pull request #2393 from alphagov/create-folder-with-user-permissions
Set user permissions when creating a folder
2019-03-12 16:36:25 +00:00
Pea Tyczynska
64c0c19419 Set user permissions when creating a folder
If the new folder has a parent folder, it inherits user permissions
from its parent. Else if the new folder is at root level, all users
will have a permission to view it.
2019-03-12 16:11:20 +00:00
Alexey Bezhan
6dc93ffd08 Change dao_remove_user_from_service query to avoid IntegrityError
When triggered by an admin request `dao_remove_user_from_service`
raised an IntegrityError since the user_to_service delete query was
issued before the folder permissions one, violating the foreign key
constraint on the folder permissions table.

For some reason this isn't caught by the tests in test_services_dao
that check that folder permissions are removed properly.
2019-03-11 17:30:06 +00:00
Rebecca Law
5089ae07f1 Merge pull request #2387 from alphagov/remove-fk-to-users
Removing the foreign from to users from Notification_history.
2019-03-11 13:48:39 +00:00
Rebecca Law
2cbcf2b805 Fix merge conflicts 2019-03-11 11:43:16 +00:00
Rebecca Law
d43a274562 Merge branch 'master' into remove-fk-to-users 2019-03-11 11:25:17 +00:00
Chris Hill-Scott
1a1c5abc62 Merge pull request #2389 from alphagov/fix-domain-migration
Fix organisation domain migration
2019-03-08 15:08:28 +00:00
Chris Hill-Scott
98b389baa2 Fix organisation domain migration
Have to drop constraint before dropping columns.
2019-03-08 15:03:00 +00:00
Chris Hill-Scott
cf2c032c90 Merge pull request #2368 from alphagov/richer-orgs
Use ‘organisations‘ table to store info about organisations (not YAML)
2019-03-08 14:57:21 +00:00
Chris Hill-Scott
73ca8b73f9 Make sure domains are always lowercased
Because otherwise we might get garbage duplicate data.
2019-03-08 14:51:23 +00:00
Chris Hill-Scott
6e8ce78603 Choose most specific domains first
If we had organisations for GDS and Cabinet Office, then we’d always
want someone whose email address ends in `@cabinet-office.gov.uk` to
match to `cabinet-office.gov.uk` before matching to
`digital.cabinet-office.gov.uk`.

Sorting the list by shortest first addresses this.
2019-03-08 13:30:51 +00:00
Chris Hill-Scott
c0fb9267bd Automatically associate new service with an org
This is the same thing we do in the admin app at the moment with YAML:
2f4e933b65/app/utils.py (L556-L562)
2019-03-08 13:30:42 +00:00
Chris Hill-Scott
d7e03e00d3 Storing more info about an organisation
Currently we have
- a thing in the database called an ‘organisation’ which we don’t use
- the idea of an organisation which we derive from the user’s email
  address and is used to set the default branding for their service and
  determine whether they’ve signed the MOU

We should make these two things into one thing, by storing everything
we know about an organisation against that organisation in the database.
This will be much less laborious than storing it in a YAML file that
needs a deploy every time it’s updated.

An organisation can now have:
- domains which we can use to automatically associate services with it
  (eg anyone whose email address ends in `dwp.gsi.gov.uk` gets services
  they create associated to the DWP organisation)
- default letter branding for any new services
- default email branding for any new services
2019-03-08 13:30:32 +00:00
Rebecca Law
1625371106 Merge pull request #2381 from alphagov/inbound-sms-retention
Inbound sms now deletes according to data retention
2019-03-08 10:58:01 +00:00
Rebecca Law
06facb7576 Removing the foreign from to users from Notification_history.
The relationship is not used and it prevents us from deleting users. It would be good to delete users that are never activated.
2019-03-07 17:20:42 +00:00
Chris Hill-Scott
46abcfd96d Merge pull request #2374 from alphagov/pyup-update-flake8-3.7.6-to-3.7.7
Update flake8 to 3.7.7
2019-03-07 09:36:00 +00:00
Alexey Bezhan
c43b03634e Merge pull request #2386 from alphagov/missing-ses-receipt-warning
Downgrade log level for missing notifications in SES receipt
2019-03-06 12:50:33 +00:00
Alexey Bezhan
6f5822ae5b Downgrade log level for missing notifications in SES receipt
The timestamps available in the SES receipt don't always correspond
to the time the notification has been sent. We've seen callbacks with
a current timestamp in both 'mail' and 'bounce' objects that referenced
a notification sent a week ago, which means we can't rely on it to skip
archived notifications.

One possible approach would be to look up the notification reference in
the notification_history table, but this goes against our plans to stop
relying on it in the future.

This changes the SES receipts logic to retry missing notifications once
(if the callback timestamp is within the last 5 minutes the task will
retry after a 5 minute delay) to capture callbacks arriving before the
notification reference has been persisted to the DB. Otherwise, we log
the missing notification as a warning instead of error.
2019-03-06 11:35:32 +00:00
Chris Hill-Scott
a355a98442 Merge pull request #2383 from alphagov/consent_to_research_null
Make `consent_to_research` nullable
2019-03-01 15:31:21 +00:00
Chris Hill-Scott
b27bcc1d80 Make consent_to_research nullable
It should be nullable so we can tell whether someone has answered the
question already or not.

No real users have entered data into this column yet, so it’s fine to
wipe it.
2019-03-01 15:21:49 +00:00
Leo Hemsted
7683e340cc get all inbound sms should default to 7 days, not 6
to be consistent with other checks.
2019-02-28 15:28:13 +00:00
Leo Hemsted
b4d1a590b7 rename days_ago_to_start to limit_days
consistency with the rest of the app
2019-02-28 13:59:59 +00:00
Leo Hemsted
88303140ec give dao_get_inbound_sms_for_service optional day limit
defaults to 6 to preserve backwards compatibility
2019-02-27 17:17:35 +00:00
Leo Hemsted
e7b8f83fc8 make inbound sms honour data retention
code inspired by the delete notification code, but with some clean up
since we don't deal with different types etc, and only need to run the
query for services with inbound numbers

also, update tests.app.db.create_inbound_sms to create inbound numbers
and assign them to services to ensure the test db is always accurate
and reflects real world usage
2019-02-27 15:53:55 +00:00
Katie Smith
27a0ac289b Merge pull request #2375 from alphagov/user-folder-permissions
Add user folder permissions
2019-02-27 15:46:53 +00:00
Leo Hemsted
26243cd2b0 Merge pull request #2377 from alphagov/skip-antivirus
stub out antivirus in dev
2019-02-27 13:38:01 +00:00
Katie Smith
cd240f9212 Delete user folder permissions when user is removed from a service 2019-02-27 13:04:14 +00:00
Katie Smith
70cdcfff55 Include list of users_with_permission in all folders response
This adds a list of user_ids of the users who have permission to view a folder
to the `get_template_folders_for_service` endpoint.
2019-02-27 12:56:26 +00:00
Katie Smith
d981abf18e Allow user folder permissions to be updated
Updated the endpoint for `.set_permissions` to update a user's folder
permissions as well as permissions for a service. User folder
permissions are optional for now, since Admin is not currently passing
this data through.
2019-02-27 12:56:26 +00:00
Katie Smith
33166f3fb2 Add migration for user_folder_permissions table
Changed the user_to_service mapping table into a model called
ServiceUser. When looking at users who have permission for a folder
we are only interested in users for a particular service, not all users,
so we can use the ServiceUser model to access folder permissions.

Added a user_folder_permissions table which contains the service_id,
user_id and template_folder_id. There are links between
user_folder_permissions and TemplateFolder, and between
user_folder_permissions and ServiceUser.
2019-02-27 11:50:36 +00:00
Leo Hemsted
653f1ab6b9 stub out antivirus in dev
antivirus is sometimes tough to get running locally - now in dev
antivirus is skipped unless `ANTIVIRUS_ENABLED=1` is set on the command
line. on all other environments it is always enabled.
2019-02-27 10:59:31 +00:00
Leo Hemsted
38f0ea6cca remove functions to not talk about 7 days
remind us that data retention is flexible
2019-02-26 17:57:35 +00:00