Commit Graph

696 Commits

Author SHA1 Message Date
Martyn Inglis
4e6da1ba55 Changed template stats for template id
- now returns the most recent notification history row for that template ID.
- contains all the required data for the use cases for that template
2016-08-22 14:35:04 +01:00
Martyn Inglis
fdd85b7dc0 Merge branch 'master' into template_queryies 2016-08-22 11:42:26 +01:00
Leo Hemsted
5418f65ae7 Merge pull request #599 from alphagov/version-500
fix GET /notifications 500 error
2016-08-22 10:59:46 +01:00
Martyn Inglis
b7476a1975 Removed the group by day aspects of template stats. Not needed. Grouped by template only. 2016-08-22 10:38:44 +01:00
Martyn Inglis
ede7d0cbea Removed old endpoint.
Going to handle the migration in the clients.
2016-08-18 14:06:12 +01:00
Martyn Inglis
7a5acea71b New endpoint for template stats to use new query. Breaking change to the returned JSON, so adding on a different url. 2016-08-18 14:01:31 +01:00
Martyn Inglis
f74000f548 Add query to get template usage from the Notifications History table
- groups by template Id and Day.

Returns count per day, template name, template id, template type, and day.

Ordered by day (desc) and template name (acc)
2016-08-18 12:06:00 +01:00
Leo Hemsted
721929ca75 Merge pull request #589 from alphagov/org-rendering
Render organisational branding in emails
2016-08-16 13:26:09 +01:00
Leo Hemsted
5491668579 let users set organisation on POST /service/{id} 2016-08-15 10:54:26 +01:00
Leo Hemsted
cc7ea8043c add organisation and branding to GET /service response dict 2016-08-12 11:40:57 +01:00
Leo Hemsted
f065b08db2 remove unused dao functions 2016-08-11 16:13:53 +01:00
Leo Hemsted
fff81b4910 remove unused notification-statistics endpoints 2016-08-11 11:55:28 +01:00
Chris Hill-Scott
ebfaa5dac2 Show separate error messages for team key
Although using a team key is functionally the same as your service being
restricted, conflating the two errors is not helpful. What we typically
saw in research was that someone was using a team key, got the error,
used a live key and got the _same_ error.

This commit adds a new error message that specifically mentions the type
of API key that you’re using.
2016-08-10 16:20:01 +01:00
Chris Hill-Scott
0a429e18b4 Give better error when sending to non-team member
Scenario we saw in research:
- trying to send a message to someone outside your team
- service is in trial mode

Result:
- error message was terrible, no-one understood it

Solution:
- better error message
2016-08-10 16:17:47 +01:00
Leo Hemsted
4dd924d438 Merge branch 'master' into version-500 2016-08-10 14:55:07 +01:00
Leo Hemsted
f184bb7996 add test for notification status return value
to ensure we never break compatibility
2016-08-09 17:31:38 +01:00
Leo Hemsted
5b7af00295 re-add content_char_count to public GET /notification endpoints
we shouldn't remove stuff from endpoints or we'll break clients
2016-08-09 17:30:38 +01:00
Leo Hemsted
648b4a17c8 remove xfail flag - the test passes properly now! 🎉 2016-08-09 16:57:18 +01:00
Leo Hemsted
4ba3745159 update schema to use template_history for accurate template details
only in the public notification endpoint so far for fear of breaking
things - in an ideal world i'd remove the template relationship
from models entirely and replace that with actual_template
2016-08-09 16:57:18 +01:00
Leo Hemsted
46c0728b12 add actual_template relationship to notification
also renamed the function to make it apparent that it'll join and grab personalisation
2016-08-09 16:57:18 +01:00
Leo Hemsted
c820938ced fix schema and primary key
* version is an additional primary key so we need to indicate that
* schema no longer relies on Template model, and uses nested user
2016-08-09 16:56:43 +01:00
Leo Hemsted
049514d4b2 remove history-meta for templates, replace with hand-made history table
history-meta's dynamic magic is insufficient for templates, where we
need to be able to refer to the specific history table to take
advantage of sqlalchemy's relationship management (2/3rds of an ORM).
So replace it with a custom made version table.

Had to change the version decorator slightly for this
2016-08-09 16:56:43 +01:00
Leo Hemsted
e88624ed4a attempt to pull logos from the admin app's static images directory
(this is configured by a config value)
2016-08-09 16:03:04 +01:00
Leo Hemsted
ebb8947290 look at service's organisation for branding to pass through to email renderer 2016-08-09 16:03:04 +01:00
Leo Hemsted
e631333a34 add GET /organisation and GET /organisation/id endpoints 2016-08-09 15:59:22 +01:00
Leo Hemsted
8e46cd44fd make history meta handle nullable and default better
* can now handle nullable foreign keys - would previously raise
  AttributeError
* can now handle default values - we would previously not insert
  default values that hadn't been generated yet, which if the
  field is not nullable will result in IntegrityErrors. We were
  deliberately removing 'default' attributes from columns. Only
  remove them if nullable is set to true now.
2016-08-09 15:59:22 +01:00
Leo Hemsted
a6cd490942 add organisation and branding models
a service now has branding and organisation_id columns, and two new
tables have been aded to reflect these:
* branding is a static types table referring to how a service wants
  their emails to be branded:
    * 'govuk' for GOV UK branding (default)
    * 'org' for organisational branding only
    * 'both' for co-branded output with both
* organisation is a table defining an organisation's branding. this
  contains three entries, all of which are nullable
    * colour - a hex code for a coloured bar on the logo's left
    * logo - relative path for that org's logo image
    * name - the name to display on the right of the logo
2016-08-09 15:59:16 +01:00
minglis
d67e43a6d0 Merge pull request #583 from alphagov/stats-db-updates
Stats db updates
2016-08-08 11:48:01 +01:00
Martyn Inglis
fe54fa9f73 Final pass through existing statsd endpoints to ensure they match new naming strategy.
Updates accordingly.
2016-08-08 11:23:58 +01:00
Martyn Inglis
f223446f73 Refactor statsd logging
Removed all existing statsd logging and replaced with:

- statsd decorator. Infers the stat name from the decorated function call. Delegates statsd call to statsd client. Calls incr and timing for each decorated method. This is applied to all tasks and all dao methods that touch the notifications/notification_history tables

- statsd client changed to prefix all stats with "notification.api."

- Relies on https://github.com/alphagov/notifications-utils/pull/61 for request logging. Once integrated we pass the statsd client to the logger, allowing us to statsd all API calls. This passes in the start time and the method to be called (NOT the url) onto the global flask object. We then construct statsd counters and timers in the following way

	notifications.api.POST.notifications.send_notification.200

This should allow us to aggregate to the level of

	- API or ADMIN
	- POST or GET etc
	- modules
	- methods
	- status codes

Finally we count the callbacks received from 3rd parties to mapped status.
2016-08-05 10:44:43 +01:00
Leo Hemsted
143cfb526c change update_provider_stats to use billable_units
updated tests etc, and removed some old tests that are no longer relevant
2016-08-03 17:22:20 +01:00
Leo Hemsted
527a5c4eaa calculate billable units when sending an sms
don't calculate it if we're in research mode
* added tests to prove this
* removed last code referring to content_char_count
2016-08-03 16:46:05 +01:00
Leo Hemsted
9a9ebf0886 filter on key types to avoid research mode (that dont actually send) 2016-08-03 16:41:04 +01:00
Leo Hemsted
1617f058e2 rework get_fragment_count to not use ProviderStatistics
use NotficationHistory instead. Unfortunately this means the SQL
gets a bit gnarly, as we have to repeat notifications_utils'
`get_sms_fragment_count` functionality inside a SELECT 😱
2016-08-03 16:29:30 +01:00
Martyn Inglis
e6347d99bd Merge branch 'master' into stats-db-updates
Conflicts:
	tests/app/conftest.py
2016-08-03 11:46:40 +01:00
Martyn Inglis
61aaa36f9b Rewiring how we do statsd
- decorater added to the DAO for notifications
2016-08-02 14:23:47 +01:00
Leo Hemsted
05ef4911b1 use freeze_time whenever we mess with dates
also fixed a gotcha where an object was created in a fixture, so the freezetime decorator didn't apply
2016-08-02 11:02:59 +01:00
Leo Hemsted
cb19e6769d Merge pull request #563 from alphagov/week-agg
New weekly aggregate function
2016-08-02 10:27:15 +01:00
Leo Hemsted
717f1fd4a0 Merge branch 'master' into get-stats-for-today 2016-08-01 10:36:56 +01:00
Leo Hemsted
e5b0d568fa ensure stats returned for lifespan of service
even if they've never sent a notification for realsies
2016-07-28 15:24:21 +01:00
Leo Hemsted
8ad47481d7 add GET /service/<id>/notifications/weekly
moved format_statistics to a new service/statistics.py file, and
refactored to share code. moved tests as well, to try and enforce
separation between the restful endpoints of rest.py and the logic/
data manipulation of statistics.py
2016-07-28 13:48:39 +01:00
Leo Hemsted
444132ad66 rewrite weekly aggregate function to honor week boundaries
(and not use the stats table, and also be easier to read)
2016-07-26 17:23:59 +01:00
Leo Hemsted
91393bdb0d ensure /notifications rather than service/:id/notifications in tests
for content merging
2016-07-26 15:09:25 +01:00
Leo Hemsted
48eff9a2ee add today_only flag to GET /service/:id
if both detailed=True and today_only=True are passed in, the stats
returned will only be for today.

if detailed is false or not specified, today_only has no effect
2016-07-26 14:35:29 +01:00
Leo Hemsted
fd96c854e1 add dao function to get notification stats for a server for toady only 2016-07-26 14:35:29 +01:00
Leo Hemsted
c81b30dba1 separated schemas once more into "with template" and "with personalisation"
"with personalisation" should only be used by the public notification api
"with template" should be used when we want template name, etc details.

also added an xfail test for correctly constructing notification
personalisation
2016-07-26 14:34:59 +01:00
Leo Hemsted
b28e7efd14 dont load template contents for job page
rename the notification_status_schema to make it apparent that it
involves the template, and then don't use it on the job page - the
job page doesn't do anything with the data. won't somebody think of
the cpu cycles! (also means it ignores problems with template
versions)
2016-07-26 14:34:59 +01:00
minglis
95af32f289 Merge pull request #556 from alphagov/fix-expired-token-error
Fix bug with expired token error response
2016-07-26 11:25:09 +01:00
minglis
cd4e271dcf Merge pull request #557 from alphagov/fix-inconsistent-tests
make tests run consistently
2016-07-26 11:24:44 +01:00
Leo Hemsted
3df0e8ee2d make tests run consistently
* tests on API endpoints that we do not explicitly sort should
  either sort the results or compare results in an orderless way
  (e.g. converting to a set)
* tests that touch the provider_details should reset values after
  running, since the provider_details table is not torn down and
  re-created between tests (unlike most tables)
2016-07-25 09:47:38 +01:00