Commit Graph

1268 Commits

Author SHA1 Message Date
Leo Hemsted
d38fdb2d11 add organisation and branding models
a service now has branding and organisation_id columns, and two new
tables have been aded to reflect these:
* branding is a static types table referring to how a service wants
  their emails to be branded:
    * 'govuk' for GOV UK branding (default)
    * 'org' for organisational branding only
    * 'both' for co-branded output with both
* organisation is a table defining an organisation's branding. this
  contains three entries, all of which are nullable
    * colour - a hex code for a coloured bar on the logo's left
    * logo - relative path for that org's logo image
    * name - the name to display on the right of the logo
2016-08-09 16:03:04 +01:00
Leo Hemsted
c7d909be03 add UKVI as first organisation branding 2016-08-09 15:59:22 +01:00
Leo Hemsted
e631333a34 add GET /organisation and GET /organisation/id endpoints 2016-08-09 15:59:22 +01:00
Leo Hemsted
8e46cd44fd make history meta handle nullable and default better
* can now handle nullable foreign keys - would previously raise
  AttributeError
* can now handle default values - we would previously not insert
  default values that hadn't been generated yet, which if the
  field is not nullable will result in IntegrityErrors. We were
  deliberately removing 'default' attributes from columns. Only
  remove them if nullable is set to true now.
2016-08-09 15:59:22 +01:00
Leo Hemsted
a6cd490942 add organisation and branding models
a service now has branding and organisation_id columns, and two new
tables have been aded to reflect these:
* branding is a static types table referring to how a service wants
  their emails to be branded:
    * 'govuk' for GOV UK branding (default)
    * 'org' for organisational branding only
    * 'both' for co-branded output with both
* organisation is a table defining an organisation's branding. this
  contains three entries, all of which are nullable
    * colour - a hex code for a coloured bar on the logo's left
    * logo - relative path for that org's logo image
    * name - the name to display on the right of the logo
2016-08-09 15:59:16 +01:00
Martyn Inglis
99a8a4f7ce Bumped utils version 2016-08-08 12:29:01 +01:00
minglis
d67e43a6d0 Merge pull request #583 from alphagov/stats-db-updates
Stats db updates
2016-08-08 11:48:01 +01:00
Martyn Inglis
fe54fa9f73 Final pass through existing statsd endpoints to ensure they match new naming strategy.
Updates accordingly.
2016-08-08 11:23:58 +01:00
Martyn Inglis
0fa614d812 Removed unused import 2016-08-08 11:09:07 +01:00
Martyn Inglis
6b12c397a1 Ensure that configs know which environment it is.
This allows us to prefix metrics with the environment to allow stats from staging and live to go to the same statsd, and alls us to filter in the dashboard by environment.
2016-08-08 10:20:33 +01:00
Martyn Inglis
365019461b Updated utils version 2016-08-05 13:53:46 +01:00
Martyn Inglis
f223446f73 Refactor statsd logging
Removed all existing statsd logging and replaced with:

- statsd decorator. Infers the stat name from the decorated function call. Delegates statsd call to statsd client. Calls incr and timing for each decorated method. This is applied to all tasks and all dao methods that touch the notifications/notification_history tables

- statsd client changed to prefix all stats with "notification.api."

- Relies on https://github.com/alphagov/notifications-utils/pull/61 for request logging. Once integrated we pass the statsd client to the logger, allowing us to statsd all API calls. This passes in the start time and the method to be called (NOT the url) onto the global flask object. We then construct statsd counters and timers in the following way

	notifications.api.POST.notifications.send_notification.200

This should allow us to aggregate to the level of

	- API or ADMIN
	- POST or GET etc
	- modules
	- methods
	- status codes

Finally we count the callbacks received from 3rd parties to mapped status.
2016-08-05 10:44:43 +01:00
Leo Hemsted
bae8655b63 Merge pull request #579 from alphagov/usage
get usage stats from notification_history
2016-08-05 10:22:08 +01:00
Leo Hemsted
d86af3ce83 fix syntax errors in downgrade script 2016-08-04 12:00:26 +01:00
Martyn Inglis
3128e79e7c Merge branch 'master' into stats-db-updates 2016-08-04 11:02:15 +01:00
Martyn Inglis
4234052ccf Removed debug from db script 2016-08-03 17:33:49 +01:00
Leo Hemsted
143cfb526c change update_provider_stats to use billable_units
updated tests etc, and removed some old tests that are no longer relevant
2016-08-03 17:22:20 +01:00
Leo Hemsted
527a5c4eaa calculate billable units when sending an sms
don't calculate it if we're in research mode
* added tests to prove this
* removed last code referring to content_char_count
2016-08-03 16:46:05 +01:00
Leo Hemsted
2793541b9c add billable_units column to notifications table
this replaces content_char_count, by performing the additional
steps to calculated billable units at insert time, rather than
read time. This means we can take into account whether the
service was in research mode or using a test api key when the
notification was sent :tada
2016-08-03 16:41:06 +01:00
Leo Hemsted
4ca23b2282 bring models in line with alembic
prevents new alembic scripts being pre-populated with index downgrades
2016-08-03 16:41:06 +01:00
Leo Hemsted
9a9ebf0886 filter on key types to avoid research mode (that dont actually send) 2016-08-03 16:41:04 +01:00
Leo Hemsted
1617f058e2 rework get_fragment_count to not use ProviderStatistics
use NotficationHistory instead. Unfortunately this means the SQL
gets a bit gnarly, as we have to repeat notifications_utils'
`get_sms_fragment_count` functionality inside a SELECT 😱
2016-08-03 16:29:30 +01:00
Martyn Inglis
bac27d688b Update travis for live 2016-08-03 13:47:05 +01:00
Martyn Inglis
8a4b16fa98 COnfigured CSV bucket in the config 2016-08-03 11:50:24 +01:00
Martyn Inglis
e6347d99bd Merge branch 'master' into stats-db-updates
Conflicts:
	tests/app/conftest.py
2016-08-03 11:46:40 +01:00
Martyn Inglis
376561e684 travis file for preview/master 2016-08-03 09:10:06 +01:00
Martyn Inglis
3df71bf467 Added all deploy groups 2016-08-02 15:31:18 +01:00
Martyn Inglis
6bf4a9b0d4 Change application name 2016-08-02 15:18:53 +01:00
Martyn Inglis
be2babb050 Travis deploy for staging 2016-08-02 15:04:22 +01:00
minglis
ba93fdc0ed Merge pull request #573 from alphagov/new-staging
New staging
2016-08-02 14:53:58 +01:00
Martyn Inglis
f939e9220d reverted postgres version 2016-08-02 14:40:56 +01:00
Martyn Inglis
610d37cf47 latest postgres version 2016-08-02 14:35:17 +01:00
Martyn Inglis
0d1732d46d Removed deploy steps from new staging.
to be added back once merged to master.
2016-08-02 14:29:27 +01:00
Martyn Inglis
6eb210fae9 Merge branch 'master' into new-staging 2016-08-02 14:24:32 +01:00
Martyn Inglis
61aaa36f9b Rewiring how we do statsd
- decorater added to the DAO for notifications
2016-08-02 14:23:47 +01:00
Leo Hemsted
7471e3ac62 Merge pull request #571 from alphagov/fix-tests
use freeze_time whenever we mess with dates
2016-08-02 11:14:22 +01:00
Leo Hemsted
05ef4911b1 use freeze_time whenever we mess with dates
also fixed a gotcha where an object was created in a fixture, so the freezetime decorator didn't apply
2016-08-02 11:02:59 +01:00
Leo Hemsted
cb19e6769d Merge pull request #563 from alphagov/week-agg
New weekly aggregate function
2016-08-02 10:27:15 +01:00
Martyn Inglis
7a77312e04 Merge branch 'master' into new-staging 2016-08-02 09:02:32 +01:00
Martyn Inglis
daf85e7787 Reorder the SQL scripts as clash 2016-08-01 15:19:16 +01:00
minglis
54a63c0351 Merge pull request #544 from alphagov/job-notification-history
Create notification history from jobs
2016-08-01 15:04:15 +01:00
Chris Hill-Scott
6d03933dd1 Merge pull request #566 from alphagov/fix-newlines-in-html-emails
Fix newlines in HTML emails
2016-08-01 14:36:34 +01:00
Chris Hill-Scott
482a435e66 Fix newlines in HTML emails
Implements:
- [ ] https://github.com/alphagov/notifications-utils/pull/60
2016-08-01 14:23:10 +01:00
Leo Hemsted
5aa00d8d21 Merge pull request #559 from alphagov/get-stats-for-today
Get service with stats for today
2016-08-01 13:38:38 +01:00
Martyn Inglis
cfb29d4404 Ensure the downgrade script targets only DMP services 2016-08-01 12:20:06 +01:00
Leo Hemsted
6edb9324da add indexes to notifications and notification_history 2016-08-01 11:34:20 +01:00
Chris Hill-Scott
5f405f234e Merge pull request #564 from alphagov/fix-markdown-oddities
Fix Markdown/HTML email oddities
2016-08-01 11:31:51 +01:00
Chris Hill-Scott
d8ab82b139 Fix Markdown/HTML email oddities
Depends on:
- [ ] https://github.com/alphagov/notifications-utils/pull/59
2016-08-01 11:16:18 +01:00
Leo Hemsted
717f1fd4a0 Merge branch 'master' into get-stats-for-today 2016-08-01 10:36:56 +01:00
Martyn Inglis
c665653859 Longer timeout on install as fails on staging 2016-07-29 10:49:48 +01:00