Commit Graph

506 Commits

Author SHA1 Message Date
Rebecca Law
5f73dfdbe9 Added constraints to services.postage: not nullable and can only containt "first" or "second" 2018-09-18 17:41:25 +01:00
Rebecca Law
1c0892888f Change the name of letter_class to postage, this will match the domain language used in the documentation.
Updated the tests to answer review comments.
2018-09-18 15:22:08 +01:00
Rebecca Law
79815b4f8a In order to support sending letters by first class we need a way to know how the service wants us to send the letter.
To start with this will be an attribute on the service, at the time the notification is created it will look at Service.letter_class to decide what class to use for the letter.

This PR adds Service.letter class as a nullable column.
Updated the create_service and update_service method to default the value to second.

Subsequent PRs will add the check constraint to ensure we only get first or second in the letter_class column and make that column nullable.
This can't be done all at once because it will cause an error if someone inserts or updates a service during the deploy.
2018-09-17 11:27:43 +01:00
Alexey Bezhan
c9ca720959 Change NotificationHistory.updated_at on update
Sets the updated_at to current time when the NotificationHistory
is modified (which happens occasionally for example to set status
to returned letter).
2018-09-13 14:09:09 +01:00
Alexey Bezhan
aacc00e3a7 Merge pull request #2075 from alphagov/process-returned-letters-list
Process returned letters list
2018-09-07 14:54:56 +01:00
Rebecca Law
8aacfb289d Add created_by_name to the Notification.serialize_for_csv.
The concern about performnace degrading has been thought through. We do not believe there will be an adverse effect since the high volume users do not send off messages.
2018-09-07 10:22:45 +01:00
Alexey Bezhan
3787e2954b Add a task to process returned letter lists
Adds an API endpoint `/letters/returned` that accepts a list of
notification references and creates a task to update their status.

Adds a new task that uses the list of references to update the status
of notifications to 'returned-letter'.

The update is currently done using a single query and logs the
number of changed records (including notification history records).
This could potentially be done within the `/letters/returned` endpoint,
but creating a job right away allows us to extend this more easily
in the future (e.g. logging missing notifications or adding callbacks).

The job is using the database tasks queue.
2018-09-06 16:39:17 +01:00
Alexey Bezhan
0dcf04def9 Add notification status for returned letters
We need to update letter notifications with a new status when DVLA
gives us a list of references for returned letters.

This adds the new status to the models and the DB.

DVLA call this 'returned mail', so I'm using it as the status name
since it seems less ambiguous than 'returned'.
2018-09-04 16:55:22 +01:00
Chris Hill-Scott
5d5047fb13 Add unique constraint to email branding domains
Does two things:

1. Revert "Revert "Add unique constraint to email branding domain""

This reverts commit af9cb30ef3.

2. Don’t allow empty string in email branding domain

Columns with multiple `null`s can have a uniqueness constraint. Columns
with multiple empty string values are not considered unique.

This commit:
- removes any duplicate empty string values
- casts empty strings to null string any time these columns are updated

---

Squashed into this single commits because these two things are not
atomic as individual commits.
2018-09-04 11:10:46 +01:00
Chris Hill-Scott
af9cb30ef3 Revert "Add unique constraint to email branding domain" 2018-09-04 10:55:49 +01:00
Chris Hill-Scott
f095fa1ddf Add unique constraint to email branding domain
Brandings with a domain set should be considered canonical. It doesn’t
make sense to have the same domain set on multiple different email
brands – you can’t tell which one to use.
2018-09-04 09:46:50 +01:00
Chris Hill-Scott
0c47d41977 Remove govuk from possible brands
‘GOV.UK’ doesn’t make sense as a type of brand. It only made sense as
a type of branding that a service had.

Since we’ve:
- deprecated the service branding column
- made sure it’s not used as a value in the email branding table

we can remove this value from the table of possible brand types.
2018-08-30 16:36:35 +01:00
Chris Hill-Scott
337496c5bc Deprecate service branding column
We want to drop this column. First we have to stop using it anywhere.

Needs to be made nullable so we can stop writing to it.
2018-08-30 16:05:37 +01:00
Rebecca Law
562facb49c Now that EmailBranding includes the brand type we do not need two separate colours. This PR removes the new colour columns. 2018-08-24 13:51:11 +01:00
Rebecca Law
3816010c72 Add brand_type to EmailBranding.
It makes more sense to put the brand_type with EmailBranding rather than in Service.
Next step is to add the new type to the form in admin app.
2018-08-23 13:53:05 +01:00
Rebecca Law
df2271a49f New columns for EmailBranding.
There is a requirement for the colour of the banner to be different to the single id colour. In order to accomodate that we are adding two new columns for that data.
These columns will be updated manually and not done with a data migration.

A new column called domain has been added. This will be used as a default domain, if the user that creates the service has a matching domain this email brand will be set for the service.

This PR only adds the new columns to the model and db.

https://www.pivotaltracker.com/story/show/159660295
2018-08-16 16:57:53 +01:00
Chris Hill-Scott
e0570c1773 Remove caseworking permission
What became ‘caseworking view’ is now available to all services.
2018-08-10 09:09:18 +01:00
Pea Tyczynska
c4bfd5479e Update EmailBranding model so it has separate name and text fields 2018-07-31 17:58:19 +01:00
Rebecca Law
19890412c9 Added a cancelled notification status so that we can cancel letters before they are sent. 2018-07-31 13:52:04 +01:00
Pea Tyczynska
4e49df2479 Delete MonthlyBilling model 2018-07-30 11:07:42 +01:00
Rebecca Law
a812060915 The unique constraint on SeviceCallbackApi was on service_id only.
Now that we have 2 types of api callbacks the constraint to be on service_id + callback_type.
2018-07-25 14:12:13 +01:00
Katie Smith
af5c4d2872 Include the notification sender name when serializing notifications
The `serialize` method of the Notification model now includes the
`created_by_name`. If a notification was sent as a one-off message this
value will now be the name of the person who sent the notification. If
a notification was sent through the API or by a CSV upload we don't
record the id of the sender, so `created_by_name` will be `None`.

This change affects the data that gets returned from these endpoints:
* /v2/notifications/<notification_id>
* /v2/notifications
2018-07-20 09:37:50 +01:00
Pea Tyczynska
183aa160c6 Add callback_type column to service_callback_api table
Also add service_callback_type table with allowed types
2018-07-17 16:54:00 +01:00
Rebecca Law
e2a1dfeb31 New dao and endpoints to create and update service data retention. 2018-07-11 17:02:49 +01:00
Rebecca Law
df0df55402 Added unique constraint for service_id + notification_type
Change updated_at to have onupdate rather than default.
2018-07-11 10:09:33 +01:00
Rebecca Law
b71f6777e9 Create a table SERVICE_DATA_RETENTION to store the number of days the notification is to be kept for. 2018-07-11 10:09:33 +01:00
Chris Hill-Scott
234c5c705a Add ‘caseworking’ service permission
We want to add the option for a user to have a more restricted view of
Notify.

This should be opt in/out on a per service basis, because it will be
appropriate only for a minority of teams. It shouldn’t be the default
because most teams (those without a lot of users or those only using the
API) won’t need it.

So this commit adds it as a new service permission, which means we can
roll it out:
- in prototype form initially
- then as an opt-in service setting
2018-06-11 13:23:06 +01:00
Rebecca Law
64f077f2f4 New endpoint to return data for complaints. 2018-06-05 14:25:24 +01:00
Richard Chapman
aaba157587 Added Service contact_link column
This is going to be used for for the document download citizen landing
page, a service will add a contact link
e.g. https://customerservicecontactnumber.uk/dwp/ which will allow the
user to contact the sending department if there is an error or any
issues with the download.

* Added the contact link to the model
* Added db migration script to add the column to the database
2018-05-31 16:03:12 +01:00
Rebecca Law
1faba916b2 New complaints model to persist complaints from SES.
If a someone gets an email from one of our services and then complain about it (mark as spam or otherwise), we get a callback from SES.
The service needs to know about these complaints so they can remove that email from their mailing list.
2018-05-31 14:43:49 +01:00
Katie Smith
4d27972c90 Add created_at and updated_at columns to ft_notification_status
Added created_at and updated_at to the ft_notification_status table in order
to make it easier to track down any potential issues with the data.

Also updated the command to populate the data to take created_at and
updated_at into account and to simplify the command. This can all be done in
the same commit since the table is not being used anywhere yet and can
only be populated manually.
2018-05-24 10:51:38 +01:00
Katie Smith
c6c118fea1 Update created_at column of ft_billing to be non-nullable
`created_at` was added previously and made nullable temporarily. This
commit now populates the column, ensures that it will always have a
value, and makes `created_at` non-nullable.
2018-05-23 11:09:13 +01:00
Katie Smith
f6039d51e0 Add timestamp columns to ft_billing table
Added `created_at` and `updated_at` to the `ft_billing` table - having these
columns makes it easier to track down issues with the data in this table.

`created_at` is nullable initially, but will be changed to non-nullable
once the column is populated and the DAO etc. have been updated.
2018-05-22 14:32:49 +01:00
Katie Smith
0d4fb81235 Delete ProviderStatistics model
This is not being used anywhere, so can be deleted safely.
2018-05-21 15:03:39 +01:00
Rebecca Law
9fad623d91 Update the ft_billing query to only include billable notification status types.
Update test_provider_statistics dao - this is really irrelevant since the endpoint using the query is not being used. We have a PR coming to delete the unused code.
Update rate_multiplier to always be an integer
2018-05-21 10:56:16 +01:00
Rebecca Law
2ae4cf18e8 Merge branch 'use-ft-billing-for-usage' of github.com:alphagov/notifications-api 2018-05-14 09:16:40 +01:00
Katie Smith
c9dc6f724d Add FactNotificationStatus model
Added the FactNotificationStatus model for the ft_notification_status table.
2018-05-14 08:51:32 +01:00
Rebecca Law
8028f6cc28 We found that the reporting task failed twice because of integrity constraints.
This was because the rate_multiplier was being added as 1 and 1.0 which was not resolving to the same.
This updates the table to use Integrer.
Also changed the logging for the task.
2018-05-10 15:35:58 +01:00
Katie Smith
be28f2e2de Add 'archived' column to the 3 reply_to models
Added a new boolean column, `archived`,  with a default of False to the
three models which are used to specify the 'reply to' address for
notifications:
 * ServiceEmailReplyTo
 * ServiceSmsSender
 * ServiceLetterContact
2018-04-30 15:25:17 +01:00
Alexey Bezhan
04a643a777 Add a service permission for uploading documents to Document Download
Service permission allows attaching a file to a notification API request
that gets uploaded to Document Download API.
2018-03-27 10:16:56 +01:00
venusbb
af975b89ec primary key for ft_billing should use notification_type 2018-03-21 15:02:08 +00:00
venusbb
7e2947790f merged master and up migration version 2018-03-16 10:57:23 +00:00
Rebecca Law
8251766cc0 Merge pull request #1766 from alphagov/user-orgs-and-services
User orgs and services
2018-03-15 16:01:36 +00:00
venusbb
ea70c6454a Fine-tuning DB model and create taks for data migration
- Removed unused columns in ft_billing
- Create tasks for nightly data migration
2018-03-14 17:07:33 +00:00
Rebecca Law
82cc6d6bef As it turns out the s3ftp used to mount the s3 bucket to our ftp server puts the file on s3 more than once. So the SNS topic is triggered more than once.
We need to deal with this, it's ok when updating a notification status from delivered to delivered. But the DailySortedLetter counts are being doubled.
Adding the file_name to the table as a unique key to get around this issue. It will mean we have multiple rows for each billing_day, but that's ok we can aggregate that.
This will also give us a way to see which file created which count.
2018-03-14 17:04:58 +00:00
Leo Hemsted
5871dee606 use joinedload to only hit the database once per request
also:

* only include active orgs
* write lots of tests
2018-03-14 15:39:31 +00:00
Leo Hemsted
91fa475645 add new endpoint to get organisations and services for a user
contains orgs, and unmapped services.

the orgs contain nested services - services the user is a part of that
belong to that org.

the unmapped services are any services that the user is a part of that
either don't have an org or have one that the user doesn't know about
2018-03-14 15:39:31 +00:00
Leo Hemsted
efec57db01 replace user_schema with serialize method on user model
this is so that we can filter out inactive organisations and services

note: can't remove user schema completely, as we still use it in
POST /user to create new users
2018-03-14 15:39:31 +00:00
Katie Smith
43c63a1644 Add new virus scan statuses
Added the following new notification statuses:
* pending_virus_check
* virus_scan_failed

If we decide to remove these statuses in future, we will need to replace
them with a different status in the notifications and
Notification_history tables where they are referenced, so
pending-virus-check will be replaced with sending, and virus-scan-failed
will be replaced with permanent-failure.
2018-03-13 12:03:14 +00:00
venusbb
59898b7349 Add billing models and small alteration to billing tables 2018-03-12 18:19:26 +00:00