Commit Graph

259 Commits

Author SHA1 Message Date
Leo Hemsted
566ae798df Speed up GET services endpoints
Add additional relationships to exclude in the ServiceSchema metaclass.
Marshmallow by default lazily loads relationships when dumping, so any
relationships we know we won't need, we can exclude and avoid a DB call.
Lots of tables are linked to services, so it loads a lot of tables.
So don't load statistics tables, since they're clearly not needed.

We *do* however want to return the users for the service - they're used
in a few places. If we're returning all services, then we don't want to
make separate queries for these users, so we modify the services_dao
queries to load users the first time round. This should speed up all
GET queries to the services endpoints, most notably pages that get many
services (platform_admin, choose service, login)
2016-07-15 12:04:38 +01:00
Rebecca Law
b2da3f563b Merge branch 'master' into email-registered-users 2016-07-08 14:53:38 +01:00
Rebecca Law
f4976539e4 Make email a required field for the email_data_schema. 2016-07-08 10:57:20 +01:00
Chris Hill-Scott
a8a556d02a Use PassThrough renderer
Implements and depends on:
- [ ] https://github.com/alphagov/notifications-utils/pull/49
2016-07-07 22:48:07 +01:00
Adam Shimali
c29dd23702 Add sms sender to service to be used in sms templates
in place of default numeric short code.

If not present default short code is used.
2016-07-01 15:27:54 +01:00
Leo Hemsted
4c44f0c9e9 remove default key_type 2016-06-27 17:00:02 +01:00
Leo Hemsted
523215c8fa Merge pull request #473 from alphagov/api-key-type
key_types table
2016-06-27 15:02:52 +01:00
Leo Hemsted
9eedb19fd4 default to KEY_TYPE_NORMAL to ensure backwards compatibility
also cleaned up tests around api_keys - fixed imports, reduced fixture usage
and added an additional (temporary) test for default test type
2016-06-24 16:33:51 +01:00
Leo Hemsted
e9482c7fe1 add new key_type table
* single-column static data table that currently contains two types: 'normal' and 'team'
* key_type foreign-keyed from api_keys
  - must be not null
  - existing rows set to 'normal'
* key_type foreign-keyed from notifications
  - nullable
  - existing rows set to null
* api_key foreign-keyed from notifications
  - nullable
  - existing rows set to null
2016-06-24 15:56:54 +01:00
Adam Shimali
0e2e99f64e notifications_utils Template constructor accepts None for
personalisation data therefore None check not needed.

If personalisation is None in in db it will get passed through to
template which returns content.
2016-06-23 15:35:35 +01:00
Adam Shimali
3423c0c44d Added subject for email templates 2016-06-23 15:21:03 +01:00
Adam Shimali
f65b86cfc9 Body of notification without placeholder should have been present. 2016-06-23 10:25:07 +01:00
Leo Hemsted
c92138d5ab return replaced subject back from get notifications API 2016-06-22 16:24:34 +01:00
Adam Shimali
731bb19a9c Template and personalisation content is now merged and returned with
notifications, when retrieved by notification id, or service id (i.e.
all notifications for service).

There is a new element returned at top level of notification json called
body, which is the template content merged with personalisation. This
is consistent with api to endpoint to create notification which returns
what was sent as 'body' in json response.

Merging of template with personalisation is done in the
NotificationStatusSchema.

Personalisation data in encrypted before storing in db.
2016-06-20 16:49:17 +01:00
Rebecca Law
39c9d9ebe9 Merge branch 'master' into refactor-query-filters
Conflicts:
	app/permission/rest.py
2016-06-16 10:26:17 +01:00
Rebecca Law
f8262b13c1 Refactor permissions dao - removed custom query filter
Removed permissions rest - not being used.
2016-06-15 16:32:52 +01:00
Adam Shimali
b33312b855 Change endpoint responses where there are marshalling, unmarshalling
or param errors to raise invalid data exception. That will cause
those responses to be handled in by errors.py, which will log
the errors.

Set most of schemas to strict mode so that marshmallow will raise
exception rather than checking for errors in return tuple from load.

Added handler to errors.py for marshmallow validation errors.
2016-06-15 14:37:51 +01:00
Leo Hemsted
7083be429a exclude notifications from job
was being lazily loaded to get notification ids, so every time a job is loaded it would potentially select thousands of notifications from the database
2016-06-10 15:37:05 +01:00
Leo Hemsted
7801efd5ca don't create job if template has been archived 2016-05-24 13:08:02 +01:00
Leo Hemsted
81e5eb2c0a return all zeros when NotificationStatistics dont exist 2016-05-20 15:24:57 +01:00
Nicholas Staples
1797d9360b Update template json to return created_at and updated_at. Also fixed a bug where updated_at was not being added to the templates_history model. 2016-05-19 17:02:55 +01:00
Rebecca Law
992f9d78f9 There is a problem where columns on the templates table were not set.
It is also discovered that columns that have a default value and use the version mixin must set the value when creating the db object before the insert otherwise the history table will be missing the default value.
Updated the templates_history.created_by_id with a value where missing, using the current version of the template for this value.
Update templates_history.archived to false. This is okay as we do not yet have a way to set this value to true.
Removed the versions attribute from the TemplateSchema, there is not a need for this column.
2016-05-16 16:16:14 +01:00
Martyn Inglis
09b3313ce3 Merge branch 'master' into primary-provider
Conflicts:
	tests/app/dao/test_provider_rates_dao.py
2016-05-11 14:04:15 +01:00
Rebecca Law
eb9f47af68 Merge branch 'master' into add-user-to-job 2016-05-11 12:08:39 +01:00
Rebecca Law
aff0abb78c Add a load only and dump only for the created_by attribute of the JobSchema 2016-05-11 12:03:25 +01:00
Nicholas Staples
083d3d75ae Add user details to template schema dump. 2016-05-10 14:55:59 +01:00
Martyn Inglis
8669a169eb Merge branch 'master' into primary-provider
Conflicts:
	app/dao/notifications_dao.py
	app/dao/provider_statistics_dao.py
	app/schemas.py
	tests/app/conftest.py
2016-05-10 09:13:02 +01:00
Martyn Inglis
571686b638 Ensure that the primary provider is used in all tasks 2016-05-10 09:04:22 +01:00
NIcholas Staples
f06aaa924e Merge pull request #287 from alphagov/statistics_group_by_week_backup
Group by Notification Statistics added and all tests working.
2016-05-09 10:17:37 +01:00
Nicholas Staples
9b3d4a6087 Template history endpoint added. All tests passing.
Code quality fix.
2016-05-06 15:47:13 +01:00
Nicholas Staples
f465066a69 Group by Notification Statistics added and all tests working.
Fixed tests for upgraded client.
2016-05-05 11:04:20 +01:00
NIcholas Staples
dcd2856c67 Merge pull request #276 from alphagov/created_by_export
Update created_by to work like other relationship fields.
2016-05-03 11:07:34 +01:00
NIcholas Staples
7121a50cd1 Merge pull request #273 from alphagov/provider_statistics
Fragment count endpoint added and all tests working.
2016-04-29 12:28:54 +01:00
Nicholas Staples
09d3ae27b6 Update created_by to work like other relationship fields. 2016-04-29 12:24:43 +01:00
Adam Shimali
4e81695c84 Added limit_days filter to get notifications dao. 2016-04-28 16:51:19 +01:00
Nicholas Staples
bedc20d0ff Fragment count endpoint added and all tests working. 2016-04-28 12:01:57 +01:00
Adam Shimali
f71b626089 Merge pull request #267 from alphagov/events
Endpoint for recording events in api.
2016-04-27 11:37:33 +01:00
Adam Shimali
dacbbfbf2f Endpoint for recording events in api.
An event has an id, a type and a blob of json attached.
2016-04-27 10:36:59 +01:00
Rebecca Law
3f7bae3428 Removed update_job endpoint because it is not being used 2016-04-27 09:46:45 +01:00
Nicholas Staples
b56e324a4c Working tests and provider stats table.
Fix for tests and import error.

Added tests and updated for code review comments.
2016-04-25 12:20:06 +01:00
Adam Shimali
7378352c26 This is a temporary endpoint to get history of service and api key
updates. This will be surfaced on a simple log page on admin app
until product team work out who/how/when etc. of viewing history
data.
2016-04-21 16:32:20 +01:00
Nicholas Staples
31978bd987 Added page_size parameter for notifications api. All tests passing.
Add page_size and total parameters to all calls for notifications.
2016-04-19 11:26:21 +01:00
Nicholas Staples
1b76646170 Working placeholders in subject with upgraded notifications-utils.
Fix test.
2016-04-13 15:53:27 +01:00
Nicholas Staples
c4b316bde6 Rebased migrations, all tests working. 2016-04-08 13:34:54 +01:00
Adam Shimali
0d06be05e1 [WIP] Added dao method and rest endpoint for getting template
statistics by service.

Some cosmetic changes to imports.

Added fix for job rest not correctly returning errors.
2016-04-04 14:38:54 +01:00
NIcholas Staples
e2d0d39ea7 Merge pull request #192 from alphagov/update_filter_for_multiple_arguments
Added functionality to allow filtering by multiple arguments.
2016-04-04 14:29:53 +01:00
Nicholas Staples
fac34aff10 Added functionality to allow filtering by multiple arguments.
Removed commented out code.
2016-04-04 13:21:51 +01:00
Rebecca Law
8493e29acc Added some more tests.
Removed the validation in the schema - it was adding complexity, let the unique constraint on the db throw the exception. This should only ever happen on a race condition which seems unlikely (two people changing a service to the same name at the same time)
Do no set debug=true to the test config. If debug=true it changes the behaviour of the error handlers, throwing the exception rather than returning a 500.
2016-04-01 13:42:11 +01:00
Chris Hill-Scott
e3ae81bd0a Strip trailing CSV rows, relax phone validation
Implements (and depends on):
- https://github.com/alphagov/notifications-utils/pull/14
- https://github.com/alphagov/notifications-utils/pull/15
2016-03-24 20:30:15 +00:00
Nicholas Staples
327f169575 Filtering added and tests working. 2016-03-21 12:37:34 +00:00