Commit Graph

284 Commits

Author SHA1 Message Date
Imdad Ahad
a9c1338873 Remove Notification, NotificationHistory status labels:
Replace labels by adding a key kwarg in the model for status.

We still need this as sqlalchemy attmempts to look for `notification_status`
on the model (Notification/NotificationHistory). To achieve true ORM mapping
(map status -> notification_status) we need the key kwarg.

More here:
http://docs.sqlalchemy.org/en/latest/core/metadata.html#sqlalchemy.schema.Column#key
2017-07-10 14:09:30 +01:00
Imdad Ahad
ac9eb29707 Revert "Revert "1/4 Stop updating old Notification status column"" 2017-07-10 14:09:30 +01:00
Imdad Ahad
ae388a3cfc Make Noti and NotiHis status column nullable:
This is in preparation for deprecating this old column
2017-07-10 12:09:02 +01:00
Imdad Ahad
b94a463911 Revert "1/4 Stop updating old Notification status column" 2017-07-06 12:30:08 +01:00
Imdad Ahad
cd36d7b77c Stop updating old column and return new column for .status 2017-07-06 12:30:07 +01:00
Leo Hemsted
38bb4ad6c2 notification.personalisation now always returns an empty dict
There are a variety of ways a notification can be created - via the
API, via CSV, via one-off messages, via the same but sent as a test -
the point is, there are lots of entry points, and lots of inconsistency
about how personalisation may be sent in. If there are no
personalisation options in the template, we may get either `None` or
`{}` - if the value is None, to avoid an error in our encryption lib,
we just store None in the database.

THIS ENDS NOW!

We've had some problems on the front-end that is caused by
notifications having the `None` value. This commit changes the
personalisation property getter and setter to store/return `{}`
rather than None.
2017-07-06 12:30:07 +01:00
Ken Tsang
98cd838510 ken-use-only-new-service-permissions 2017-07-06 12:27:55 +01:00
Leo Hemsted
bd71ee9d02 add redact to notification with template schema.
So that when the admin gets notifications, the template they return
also has a "redact_personalisation" boolean attached to it. Note, it
won't do the redacting on the api - that'll be part of the admin.

Under the hood, this uses an association_proxy, which is essentially
black magic. But it proxies the `redact_personalisation` property of
`TemplateRedacted` onto the `Template` object, so that Marshmallow
can pick it up.

Note: NOT currently added to NotificationWithTemplateHistory
2017-06-28 16:15:03 +01:00
Leo Hemsted
29fc81090e add template personalisation redaction
If passing in `redact_personalisation` to the template update endpoint,
we should mark that template permanently as redacted - this means that
we won't ever return the personalisation for any notifications for it.

This is to be used with templates containing one time passwords, 2FA
codes or other sensitive information that you may not want service
workers to be able to see.

This is implemented via a separate table, `template_redacted`, which
just contains when the template was redacted.
2017-06-28 15:53:08 +01:00
Leo Hemsted
350133e6db ensure created_by_id is being persisted correctly
(also make sure it's well tested 🎉 )
2017-06-23 15:56:47 +01:00
Leo Hemsted
a1e570dea7 persist created_by_id when using the one off notification endpoint 2017-06-20 12:06:49 +01:00
Leo Hemsted
9f307fd1c5 add created_by to notifications
this is so one-off notifications can be tied to a user
(jobs have a created_by, and api notifications don't make sense
 to have one)
2017-06-20 11:51:17 +01:00
Rebecca Law
3a66027d6a Refactor ApiKeys.secret and ServiceInboundApi.bearer_token to use the same encryption method and get rid of the duplicate code. 2017-06-19 14:32:22 +01:00
Rebecca Law
6202da7dea Update model to remove the string length restriction.
Moved logic to the dao from the endpoint.
2017-06-19 12:25:05 +01:00
Rebecca Law
effb99dca8 Add fetch request for service inbound api.
Add unique constraint on service_id for service_inbound_api.
2017-06-15 16:19:12 +01:00
Rebecca Law
828d5cd493 New table to store the inbound api information for a service. The table is versioned.
There is a new endpoint to create the inbound api and one to update it.
2017-06-15 11:32:51 +01:00
Rebecca Law
b186cad046 Add a new table to store the api information for a service inbound sms message.
Including:
 - url to push the inbound sms to
 - bearer_token to be added to the header of the request.

The services will be expected to manage these properties.
2017-06-13 15:27:13 +01:00
Martyn Inglis
9bbb3c6a56 Merge branch 'master' into firetext-inbound-sms-not-null-provider
Conflicts:
	tests/app/db.py
2017-06-09 11:56:32 +01:00
Rebecca Law
03c6e74c46 Merge pull request #1024 from alphagov/job-stats-for-dashboard
Change the job_statistics table structure
2017-06-08 13:19:15 +01:00
Rebecca Law
1b4097cb16 Add three new columns to job_statistics for sent, delivered and failed.
A job only ever has one notification type.
This is the first deploy, where the columns are added and populated.

Next a data migration will happen to populate these new columns for the older jobs that do not have the values set.
Then we stop populating the old columns and remove them.
This refactoring of the table structure will make the queries to the table much easier to handle.
2017-06-07 11:15:05 +01:00
Martyn Inglis
75bf693f44 Add the yearly free limit to the service model.
This allows us to reference it across the API code base and return it in the API.

But not currently attached to the service DB model - a static method on the class.
2017-06-06 14:49:05 +01:00
Martyn Inglis
687b8443d1 Merge branch 'populate_provider_column' into firetext-inbound-sms-not-null-provider 2017-06-05 11:57:39 +01:00
Martyn Inglis
f15d235d1e Merge branch 'master' into firetext-inbound-sms 2017-06-05 11:33:53 +01:00
Martyn Inglis
400096520d Not null the provider column on the inbound SMS table. 2017-06-02 16:51:27 +01:00
Martyn Inglis
012f8d2675 Adds provider onto the inbound sms table so we know where this came from. 2017-06-02 16:37:57 +01:00
Leo Hemsted
ef52337d85 add inbound sms api
two endpoints:
* get all inbound sms for a service (you can limit to the X most
  recent, or filter by user's phone number [which will be normalised])
* get a summary of inbound sms for a service - returns the count of
  inbound sms in the database, and the date that the most recent was
  sent
2017-06-02 15:20:18 +01:00
Rebecca Law
9d80bdc70f Merge branch 'master' into letters-billing-table
Conflicts:
	app/models.py
2017-06-02 14:47:28 +01:00
Martyn Inglis
c57e2a6894 Merge branch 'master' into inbound-sms
Conflicts:
	app/notifications/receive_notifications.py
	tests/app/notifications/test_receive_notification.py
2017-06-01 15:56:33 +01:00
Martyn Inglis
a60d40bbc0 Merge branch 'master' into remove-nasty-query-from-dashboard
Conflicts:
	tests/app/service/test_rest.py
2017-06-01 14:47:00 +01:00
Leo Hemsted
3e47519045 Merge pull request #1006 from alphagov/revert-1001-revert-988-non-null-again
Revert "Revert "Remove nulls from sms_sender""
2017-06-01 13:14:03 +01:00
Martyn Inglis
7f65aa3eef Merge branch 'master' into remove-nasty-query-from-dashboard
Conflicts:
	app/service/rest.py
	tests/app/service/test_rest.py
2017-06-01 13:11:54 +01:00
Leo Hemsted
d33698216c Revert "Revert "Remove nulls from sms_sender"" 2017-06-01 11:00:26 +01:00
Leo Hemsted
ea0ba8d87a Revert "Remove nulls from sms_sender" 2017-05-31 14:52:48 +01:00
Rebecca Law
0b642623fb Added table and model for letter rates.
The rates for the letters are per page, therefore it seemed better to build a different table.
2017-05-31 13:34:54 +01:00
Leo Hemsted
eb6edf06a3 add upgrade script to remove non-null values from the sender column 2017-05-30 10:47:15 +01:00
Leo Hemsted
4a85818c34 add inbound sms table 2017-05-30 10:47:01 +01:00
Rebecca Law
3dc70b8c39 Check service.permissions for the existence of schedule_notifications if the notications is being created with a scheduled_for param. 2017-05-26 15:41:14 +01:00
Ken Tsang
f7a18f77cf Update model to cascade permissions assoc proxy 2017-05-25 17:48:09 +01:00
Rebecca Law
b745e13a28 Merge branch 'master' into schedule-api-notification 2017-05-25 13:37:04 +01:00
Leo Hemsted
57b7cb8d24 Merge pull request #986 from alphagov/create-service-bug
separate service deserialization from validation
2017-05-25 10:52:38 +01:00
Leo Hemsted
4b8b6ca91e add test to ensure that updating other things doesnt affect sms sender 2017-05-25 10:37:17 +01:00
Rebecca Law
9f6c037530 Use iso8601 to validate scheduled_for datetime.
Added a validation method that always fails for scheduled notifications.
Comment out config for scheduled task.
The schedule notifications will be turned on once we can invite services to use it.
Waiting for the service permission story, must commit this in order to keep things from going stale.
2017-05-24 16:27:15 +01:00
Leo Hemsted
554a193cff separate service deserialization from validation
Marshmallow validates and deserialises - BUT, when it deserialises,
it explicitly sets `sms_sender=None`, even when you haven't passed
sms_sender in. This is problematic, because we wanted to take advantage
of sqlalchemy's default value to set sms_sender to `GOVUK` when the
actual DB commit happens.

Instead, still use marshmallow for validating, but manually carry out
the json deserialisation in the model class.

This fixes a bug that only manifested when the database was upgraded,
but the code hadn't updated. 🎉
2017-05-24 16:27:12 +01:00
Imdad Ahad
c986b88e21 Merge pull request #979 from alphagov/imdad-migration-add-normalised-to-notification
Add normalised_to field to notification
2017-05-24 16:21:15 +01:00
Rebecca Law
383dee3bb2 Updated the serialization of Notification.scheduled_for to include minutes. 2017-05-24 14:52:32 +01:00
Martyn Inglis
511e143ace toString on the rates object 2017-05-24 08:57:11 +01:00
Leo Hemsted
e9b1fd4dde Revert "Non null sender" 2017-05-23 16:32:03 +01:00
Rebecca Law
53efb4e033 Merge branch 'master' into schedule-api-notification
Conflicts:
	tests/app/v2/notifications/test_post_notifications.py
2017-05-23 13:58:50 +01:00
Martyn Inglis
e987668b0c Merge branch 'master' into remove-nasty-query-from-dashboard 2017-05-23 13:55:30 +01:00
Leo Hemsted
47560515dc add upgrade script to remove non-null values from the sender column 2017-05-23 11:33:43 +01:00