Commit Graph

411 Commits

Author SHA1 Message Date
Chris Hill-Scott
eb41ce7304 Make service created by platform admin non-counted
If a service has been created by someone on our team, it’s probably a
test service, which shouldn’t be included in the count of live services.
2019-03-25 13:30:13 +00:00
Chris Hill-Scott
ef515400f3 Fix automatic inheritance of org’s branding
When creating a service it should inherit it’s organisation’s branding,
if that organisation has branding.

This wasn’t working because we were referring to the ID of the branding
when making the association, not the branding itself.
2019-03-22 15:57:20 +00:00
Katie Smith
2aa14bc41c Set folder permissions when adding a user to a service
This sets the folder permissions for a user when adding them to a
service. If a user is being added to a service after accepting an
invite, we need to account for the possibility that the folders we are
trying to add them to have been deleted before they accepted the invite.
2019-03-22 09:30:39 +00:00
Katie Smith
b0d3bd9046 Update add_user_to_service endoint to only handle new data format
Updated the add_user_to_service endpoint to only handle data in the
'new' format (`{"permissions": [...]}` instead of `[permission_1, permission_2]`)
since Admin has been updated to send data the new way.

This change means that we no longer need the Marshmallow Permission
schema, so it can be deleted.
2019-03-22 09:30:39 +00:00
Katie Smith
d4757c9208 Allow add_user_to_service endpoint to accept new data format
The data posted to the `add_user_to_service` endpoint is currently sent as a
list of permissions:
`[{'permission': MANAGE_SETTINGS}, {'permission': MANAGE_TEMPLATES}]`.

This endpoint is going to also be used for folder permissions, so the
data now needs to be nested:

`{'permissions': [{'permission': MANAGE_SETTINGS}, {'permission': MANAGE_TEMPLATES}]}`

This changes the add_user_to_service endpoint to accept data in either
format. Once admin is sending data in the new format, the code can be
simplified.
2019-03-14 16:59:48 +00:00
Chris Hill-Scott
6e8ce78603 Choose most specific domains first
If we had organisations for GDS and Cabinet Office, then we’d always
want someone whose email address ends in `@cabinet-office.gov.uk` to
match to `cabinet-office.gov.uk` before matching to
`digital.cabinet-office.gov.uk`.

Sorting the list by shortest first addresses this.
2019-03-08 13:30:51 +00:00
Chris Hill-Scott
c0fb9267bd Automatically associate new service with an org
This is the same thing we do in the admin app at the moment with YAML:
2f4e933b65/app/utils.py (L556-L562)
2019-03-08 13:30:42 +00:00
Chris Hill-Scott
b27bcc1d80 Make consent_to_research nullable
It should be nullable so we can tell whether someone has answered the
question already or not.

No real users have entered data into this column yet, so it’s fine to
wipe it.
2019-03-01 15:21:49 +00:00
Rebecca Law
331414d379 Merge pull request #2357 from alphagov/remove-edit-folder-permission
Remove the need for the edit_folders service permission.
2019-02-15 13:19:48 +00:00
Chris Hill-Scott
f679731af8 Also store consent to research against a service
It makes most sense to collect this at the same time as the estimated
volumes. Which means we need to store it somewhere; we can’t put it
straight into the ticket.
2019-02-15 10:08:22 +00:00
Chris Hill-Scott
ecdf076506 Store anticipated volumes as integers
This will make it easier to do analysis on the data. Almost all users
are submitting data in a numerical format now anyway, because we ask the
question in a sensible way.
2019-02-15 10:08:22 +00:00
Chris Hill-Scott
17e32fa5f6 Add fields to record a service’s estimated volumes
When a service go live we ask people for their estimated sending
volumes. At the moment we only put this in the ticket, and store it in
a spreadsheet.

This means that a service can
- say they want to go live
- say they are sending 100,000 emails per year
- not have created any email templates
- still see ‘create templates’ as ‘completed’ in the go live checklist

If we store this data against the service we can collect it earlier, and
then use it to determine automatically what kind of templates the user
needs to create before their go live checklist can be considered
complete.
2019-02-15 10:08:22 +00:00
Leo Hemsted
53ecaa3230 remove dvla_organisation
it's been superceeded by letter branding
2019-02-13 15:02:18 +00:00
Leo Hemsted
afc5c96927 Don't fallback to dvla_organisation if letter branding unset
The template preview app now accepts a null value for the `filename` 
parameter. If a service doesn't have a letter branding option set, 
previously we defaulted to their dvla_organisation (probably HM 
Government). Now, we pass through None, so that we generate letters 
without any logo or branding.
2019-02-13 11:58:54 +00:00
Pea Tyczynska
8ef581de69 Remove service.postage and choose_postage permission from the code and tests 2019-02-11 18:20:34 +00:00
Rebecca Law
38a599c758 Remove the need for the edit_folders service permission.
We need another PR to remove it from ServicePermissionTypes and remove the data in service_permissions.
2019-02-11 17:26:17 +00:00
Leo Hemsted
478de8bb8e fix new services being assigned to wrong letter branding
when creating a service, the api accepts a `service_domain` field that
it uses to populate the letter branding - if the service domain is
known to match an existing letter branding option, use that
automatically. However, the admin currently doesn't know about this
field yet so doesn't pass anything through - the api erroneously
searches the DB for letter branding with a domain of None - which they
currently all have.

This meant that when services were created, their letter branding was
set to the most recent row in the DB (that matched None).
2019-02-11 11:46:33 +00:00
Chris Hill-Scott
4b1336b405 Give folders to new services by default
Step 1 of 2 of turning on folders for all services.

We think it’s a feature which will be useful for the majority of
services, and we think we’ve done enough research to know that it’s
mature enough to release to all services.
2019-01-29 12:30:41 +00:00
Rebecca Law
e030c2be88 Removing platform_default as a concept. No service actually wants to send letters with the default hm-government logo so we are going to remove it as a constraint.
However, until we can create a letter without a logo, we will still default to hm-government, because the dvla_organisation is set on the service.
This does simplify the code.
Also removed the inserts to letter_branding in the data migration file, because we can deploy this before the rest of the work is finished. But we will need to do it later.
2019-01-25 15:03:01 +00:00
Rebecca Law
f11aa55e0b Adding new data models for letter branding. 2019-01-22 17:27:00 +00:00
Pea (Malgorzata Tyczynska)
276a9a3828 Merge pull request #2293 from alphagov/choose_postage_for_precompiled
Choose postage on POST request for precompiled letters
2019-01-16 14:13:26 +00:00
Pea Tyczynska
5ebeb9937a Avoid call to database to get template in persist_notifications 2019-01-14 17:53:06 +00:00
Rebecca Law
efad58edd8 There is no need to have a separate table to store template monthly statistics. It's easy enough to aggregate the stats from ft_notification_status.
This removes the nightly task, and all the dao methods.
The next PR will remove the table.
2019-01-14 16:30:36 +00:00
Rebecca Law
92460fbd38 Merge branch 'master' into use-ft-notification-statu-for-monthly-template-usage 2019-01-11 17:11:23 +00:00
Pea Tyczynska
bd5126481c Remove cancelled from requested statuses in service statistics 2019-01-11 14:27:54 +00:00
Rebecca Law
507138cc94 Create a new query for template monthly stats. 2019-01-10 16:24:51 +00:00
Pea (Malgorzata Tyczynska)
c5e5c86982 Merge pull request #2290 from alphagov/its_not_a_failure_to_cancel
Cancelled notifications don't show as failures in statistics
2019-01-10 15:09:30 +00:00
Alexey Bezhan
1719f31909 Merge pull request #2284 from alphagov/add-count-pages-flag-to-service-notifications
Don't return pagination links for API Message log requests
2019-01-09 14:36:36 +00:00
Pea Tyczynska
e179e1e4a4 Update test 2019-01-09 13:30:28 +00:00
Leo Hemsted
3e21f57481 fix platform admin stats row-order bug
now that we're reading from two tables (ft_notification_status and
notifications) for stats, we'll get a couple of rows for each
notification type. If a service doesn't have any rows in one of those
tables, the query will return a row with nulls for the notification
types and counts. Some services will have history but no stats from
today, others will have data from today but no history.

This commit acknowledges that any row might have nulls, not just the
first row.
2019-01-09 11:43:40 +00:00
Alexey Bezhan
47c403f6ab Don't return pagination links for API Message log requests
Flask-SQLAlchemy paginate function issues a separate query to get
the total count of rows for a given filter. This query (with
filters used by the API integration Message log page) is slow for
services with large number of notifications.

Since Message log page doesn't actually allow users to paginate
through the response (it only shows the last 50 messages) we can
use limit instead of paginate, which requires passing in another
flag from admin to the dao method.

`count` flag has been added to `paginate` in March 2018, however
there was no release of flask-sqlalchemy since then, so we need
to pull the dev version of the package from Github.
2019-01-08 13:22:27 +00:00
Rebecca Law
bd9a6352fd Optimise the query for getting the platform statistics for all services. The page should render for all time after this change.
This is one step closer to eliminating the need to read from NotificationHistory.
2019-01-04 16:45:39 +00:00
Katie Smith
90d9135fcf Return cancelled letters for dashboard
Added cancelled letters to the number of failed letters in the statistics
that get used for the dashboard. At some point, we want to stop
including cancelled letters in the stats, but for now this keeps things
consistent with our current letter failure state, permanent-failure.
2018-12-03 17:51:09 +00:00
Katie Smith
1d67b55b16 Add endpoint for cancelling letters 2018-12-03 17:51:08 +00:00
Chris Hill-Scott
f5ea77ffa0 Add reference to one off letters
Letters should always have a reference, because that’s what DVLA use to
tell us when they’ve sent a letter.

If a letter has a reference of `None` then DVLA say they’ve sent a
letter with a reference of `'None'`. This means we can never reconcile
the letter, which means it stays in `created`, which means it never
gets billed.

We don’t think this has affected any real letters yet, just ones that
we’ve sent as tests.
2018-12-03 11:34:14 +00:00
Pea (Malgorzata Tyczynska)
d46caa184e Merge pull request #2211 from alphagov/fix_notification_statistics_on_dashboard
Fix notification statistics on dashboard
2018-11-06 16:22:00 +00:00
Pea Tyczynska
5d806c2437 Adjust tests to new way of querying for notification stats 2018-11-06 14:40:40 +00:00
Rebecca Law
d15de2e753 Merge pull request #2193 from alphagov/fix-failed-statuses
Add new failed stats to statistic counts
2018-11-01 13:19:10 +00:00
Chris Hill-Scott
674cdbd265 Allow the admin app to send one-off letters
This commit modifies the code paths the admin app uses to send one off
emails and text messages to also accept letters.

This mostly worked already, the two changes were:
- making sure that one-off letters are processed by the correct task,
  from the correct queue
- one-off letters sent from a service in research mode don’t get put on
  a queue and go straight to `delivered` (because we don’t want to send
  them for real)
2018-10-31 16:48:43 +00:00
Rebecca Law
00e7b337df Update unit test to be more realistic with failure types 2018-10-30 16:01:23 +00:00
Rebecca Law
80d02c434b Add validation-failed and virus-scan-failed in the failed counts for stats 2018-10-29 16:57:06 +00:00
Leo Hemsted
ac5c19c3d5 remove some pytest warnings 2018-10-26 17:54:53 +01:00
Leo Hemsted
5693d1181e add notify_db_session to test funcs that hit db 2018-10-26 17:51:50 +01:00
Katie Smith
e7f0a414ce Update service schema to add letter_logo_filename
Added the filename of a service's letter logo to the service schema. We want
this in the schema so that it is possible to call
`current_service.letter_logo_filename` from notifications-admin and to pass this value
through to template-preview.
2018-10-18 08:48:06 +01:00
Rebecca Law
1c0892888f Change the name of letter_class to postage, this will match the domain language used in the documentation.
Updated the tests to answer review comments.
2018-09-18 15:22:08 +01:00
Rebecca Law
79815b4f8a In order to support sending letters by first class we need a way to know how the service wants us to send the letter.
To start with this will be an attribute on the service, at the time the notification is created it will look at Service.letter_class to decide what class to use for the letter.

This PR adds Service.letter class as a nullable column.
Updated the create_service and update_service method to default the value to second.

Subsequent PRs will add the check constraint to ensure we only get first or second in the letter_class column and make that column nullable.
This can't be done all at once because it will cause an error if someone inserts or updates a service during the deploy.
2018-09-17 11:27:43 +01:00
Chris Hill-Scott
337496c5bc Deprecate service branding column
We want to drop this column. First we have to stop using it anywhere.

Needs to be made nullable so we can stop writing to it.
2018-08-30 16:05:37 +01:00
Katie Smith
a87be9b74a Use new value of SMS_CHAR_COUNT_LIMIT from utils
Admin, API and utils were all defining a value for SMS_CHAR_COUNT_LIMIT.
This value has been updated in notifications-utils to allow text
messages to be 4 fragments long and notifications-api now gets the value of
SMS_CHAR_COUNT_LIMIT from notifications-utils instead of defining it in
config.

Also updated some tests to check for the higher limit.
2018-08-16 16:34:34 +01:00
Alexey Bezhan
ab428048b9 Add an endpoint to fetch service data retention by type
Admin app needs to get the service data retention for the specified
notification type, so to avoid iterating through the list of all
existing service data retention settings we restore the endpoint
to get the individual data retention period.
2018-08-13 16:45:57 +01:00
Rebecca Law
019c6a4e3a Revert "Purge notifications for configured days of retention" 2018-08-03 14:35:36 +01:00