Commit Graph

688 Commits

Author SHA1 Message Date
Adam Shimali
cfd42a8a05 Statsd timings were being passed a timedelta instead for float for
milliseconds.

Bugfix for https://www.pivotaltracker.com/story/show/126852733
2016-07-22 12:13:24 +01:00
Chris Hill-Scott
c38962e91a Only put a character limit on SMS notifications
We limit SMS to be a maximum of 495 characters at the point of an
API call being made.

We were also applying this limit to emails, which is wrong.
2016-07-21 16:33:21 +01:00
Leo Hemsted
964ea1954c Merge pull request #538 from alphagov/services-speed-up
Speed up GET services endpoints
2016-07-21 14:03:44 +01:00
Rebecca Law
c8ad5362eb Moved mmg_url to configs.
Fix up the tests
2016-07-20 10:40:50 +01:00
Rebecca Law
aca23472e3 Use the new mmg api 2016-07-19 15:27:52 +01:00
Leo Hemsted
566ae798df Speed up GET services endpoints
Add additional relationships to exclude in the ServiceSchema metaclass.
Marshmallow by default lazily loads relationships when dumping, so any
relationships we know we won't need, we can exclude and avoid a DB call.
Lots of tables are linked to services, so it loads a lot of tables.
So don't load statistics tables, since they're clearly not needed.

We *do* however want to return the users for the service - they're used
in a few places. If we're returning all services, then we don't want to
make separate queries for these users, so we modify the services_dao
queries to load users the first time round. This should speed up all
GET queries to the services endpoints, most notably pages that get many
services (platform_admin, choose service, login)
2016-07-15 12:04:38 +01:00
Adam Shimali
9f9040869d Daily message limit imposed for service api calls. 2016-07-14 11:25:45 +01:00
Leo Hemsted
4ef084464d update notification_history table from notification_dao create/update functions
please ensure that any changes to notifications table happen through either dao_create_notification or dao_update_notification.
changed the notification status update triggered by the provider callbacks to ensure that sets updated_by and can update the history table.
also re-added the character_count so we can reconstruct billing data if needed.
2016-07-12 13:28:45 +01:00
Leo Hemsted
722699a72a Update notification_history table on insert/update of notifications
triggered via calls in dao_create_notification and dao_update_notification - if you don't use those functions (eg update in bulk) you'll have to update the history table yourself!
2016-07-12 13:27:31 +01:00
Leo Hemsted
47ef63adbe add notification_history table
table will be used for storing archival versions of notifications. It's an exact duplicate of notifications table, but with the following modifications:
* removed _personalisation
* removed to
* removed content_char_count

All foreign keys to other tables still exist. additionally, removed defaults (for id and created_at) since they'll be set when we create, and we should ensure that we don't forget about them when inserting/udpating this table.
2016-07-12 13:27:31 +01:00
Rebecca Law
f9004f983a Merge pull request #524 from alphagov/email-registered-users
Email registered users
2016-07-12 10:51:17 +01:00
Rebecca Law
8acb373f07 Merge pull request #523 from alphagov/fix-temp-failure-job
Add a lockmode for the update_notifications_status_by_id to prevent t…
2016-07-12 09:49:00 +01:00
Rebecca Law
b2da3f563b Merge branch 'master' into email-registered-users 2016-07-08 14:53:38 +01:00
Rebecca Law
1bfd25107b Add a lockmode for the update_notifications_status_by_id to prevent the timeout task from updating the same notification more than once.
This happens because more than one beat process was creating the timeout task, resulting in multiple workers running the same queries at the same time.
2016-07-08 14:48:07 +01:00
Rebecca Law
f4976539e4 Make email a required field for the email_data_schema. 2016-07-08 10:57:20 +01:00
Chris Hill-Scott
a8a556d02a Use PassThrough renderer
Implements and depends on:
- [ ] https://github.com/alphagov/notifications-utils/pull/49
2016-07-07 22:48:07 +01:00
Rebecca Law
36ecdca04c Add new email template for the GOV.UK Notify service, to send an email to users that register with the same email address.
Add a new endpoint to send the email.
2016-07-07 17:23:07 +01:00
Chris Hill-Scott
824085ead8 Bring in changes to template and CSV processing
Functional changes:
- adds the blue bar

Performance changes
- faster CSV processing

Depends on:

- [ ] https://github.com/alphagov/notifications-utils/pull/47
- [ ] https://github.com/alphagov/notifications-utils/pull/48

Also brings in some breaking changes, which do not affect utils (apart
from a weird import).
2016-07-07 15:44:30 +01:00
Rebecca Law
a1209ed9a4 Merge branch 'master' into fix-count 2016-07-06 14:45:10 +01:00
Rebecca Law
8bea8b6a8f Fix bug where all notifications were getting a type = email.
Includes a fix to notification and notification_statistics data.
2016-07-06 14:31:52 +01:00
Leo Hemsted
96513442fe notifications from test api keys dont send for real - a la research_mode
also added a bunch more tests around api keys to help ensure we're behaving in at least a consistent way
2016-07-05 16:29:53 +01:00
Leo Hemsted
a5e488c035 add test api key type 2016-07-05 16:20:02 +01:00
Leo Hemsted
242be97bfa remove reply_to_addresses from task kwargs
also added a test for aws_ses.send_email to prove it handles reply_to_address correctly
2016-07-04 17:29:41 +01:00
Leo Hemsted
f5e14f043d dont pass reply-to-addresses about
dont send reply_to_addresses around from process_job and send_email -
take it from the service in send_email_to_provider. also clean up
the kwarg in aws_ses.send_email to more accurately reflect what we
might pass in
2016-07-04 15:04:43 +01:00
Adam Shimali
19f2ccb594 Merge pull request #498 from alphagov/set-sms-sender-api
Set sms sender on service
2016-07-04 10:32:04 +01:00
Leo Hemsted
4e2a14ed28 Merge branch 'master' into notifications-get-api-key-filter 2016-07-01 17:12:03 +01:00
Adam Shimali
b32f0ab2cd Resolve conflicts 2016-07-01 16:53:12 +01:00
Leo Hemsted
f992240192 make notification.key_type not nullable
set to 'normal' for all existing notifications, and all job notifications also created as 'normal' - so if your eg reporting service hits notify, it gets notifications created from both API calls and front-end csv jobs.
2016-07-01 16:36:11 +01:00
Adam Shimali
c29dd23702 Add sms sender to service to be used in sms templates
in place of default numeric short code.

If not present default short code is used.
2016-07-01 15:27:54 +01:00
Rebecca Law
3aff22cf87 Improve the logging message 2016-07-01 14:42:40 +01:00
Leo Hemsted
2cf1d22748 add filters to GET /notifications endpoints to only return for provided key_type
if api_key used to access endpoint is type team, endpoints only return that type -
will 404 if you provide a different ID. Same applies for normal (normal api keys
cannot see team notifications)
also, for convenience, set sample_notification to supply key_type of KEY_TYPE_NORMAL
by default
2016-07-01 14:35:02 +01:00
Leo Hemsted
01419e7894 store api_key_id and key_type on notification
pass through from POST /notification/<type> to the celery task
also removed a couple of asserts that can fail (based on unfrozen time comparisons)
2016-07-01 14:35:02 +01:00
Leo Hemsted
5ae7ed1acb only send to team emails/phones if POST /notificaiton/<type> with team api_key
uses same restriction as a service in trial mode
2016-07-01 14:33:25 +01:00
Rebecca Law
f52755742c Split send_email task into one task to create the notification and one to send it to the provider.
Is there is an exception the task will go to the retry queue.
2016-07-01 14:14:28 +01:00
Leo Hemsted
11093530f9 Merge pull request #485 from alphagov/api_user-cleanup
Api user cleanup
2016-07-01 10:58:18 +01:00
Rebecca Law
3f11447bc8 A small refactor to use the SMS_TYPE and EMAIL_TYPE in code rather that 'sms' or 'email' 2016-06-30 15:41:51 +01:00
Chris Hill-Scott
a26d2a2551 Merge pull request #482 from alphagov/swagger-spike
Publish a Swagger specification
2016-06-30 13:41:33 +01:00
Leo Hemsted
3838715077 refactored the requires_auth handler to raise exceptions
hopefully cleans up code flow and readability [a tiny bit].
raise an AuthException in auth.py, and catch it in errors.py to
save on returning error_repsonse values throughout the function
2016-06-30 10:44:21 +01:00
Leo Hemsted
39519e3f36 attach api_key to app
we previously attached the service id and the key's secret
also more refactoring of auth.py
2016-06-30 10:44:21 +01:00
Leo Hemsted
adbe02783d refactor authentication code
moved api_key secret manipulation (generating and getting) into
authentiation/utils, and added a property on the model, to facilitate
easier matching of authenticated requests and the api keys they used
2016-06-30 10:44:21 +01:00
Leo Hemsted
18b30de452 fix calling init_app twice
this was causing flask decorators like auth check to be mounted twice
2016-06-30 10:44:21 +01:00
Leo Hemsted
2b645f490a move get_all_notifications_for_service and get_all_notifications_for_job
moved from notifications/rest -> service/rest and job/rest respectively
endpoint routes not affected
removed requires_admin decorator - that should be set by nginx config
as opposed to python code
2016-06-30 10:44:21 +01:00
Rebecca Law
1ce3495fc7 Remove reference to template.template_type for the stats update query.
Removes a query during the stats update.
2016-06-29 12:12:35 +01:00
Chris Hill-Scott
7c3d25a87a Publish a Swagger specification
Adds a new endpoint (`/spec`) which returns a the specification of the
API in Swagger-formatted JSON.

This means we will have something to point frontends at, so we can
evaluate which ones we like.

Right now it’s all hand-defined. If we were consistent about our use of
Marshmallow we could generated the spec from the Marshmallow schemas.
2016-06-29 12:07:32 +01:00
Rebecca Law
25db1bce74 Use the notification types enum for the notifications.notification_type.
Reuse EMAIL_TYPE in template_types and notification_types.
2016-06-29 11:50:54 +01:00
Rebecca Law
60e159e3c0 Add notification_type to notification table.
It seems like an oversight not to include the notification type in the notifcation.
When updating statistics a query to the template table is required to get the type, this update will mean that query does not have to happen.
2016-06-29 11:23:02 +01:00
Rebecca Law
340abae82c Merge pull request #478 from alphagov/make-password_changed_at-not-null
Users.password_changed_at not nullable
2016-06-28 17:00:22 +01:00
Rebecca Law
abb9135e35 Password changed at is defaulted to the current date when the user is created.
This PR set the users.password_changed_at column to not be nullable.
2016-06-28 11:24:08 +01:00
Leo Hemsted
4c44f0c9e9 remove default key_type 2016-06-27 17:00:02 +01:00
Rebecca Law
6626771335 Merge branch 'master' into notification-created-status 2016-06-27 15:36:42 +01:00