Commit Graph

1066 Commits

Author SHA1 Message Date
Martyn Inglis
71289d36db Change to research mode.
Previously research mode created a task to fake the callback from the providers. This meant there is an extra task for each notification than would be generated in a live situation.

This PR changes that, so that a research mode/test key notification calls the callback API rather than make a task to do that .

This ensures that the flow for research mode more closely mimics that of live, and removes a task from the process so we can more accurately test throughput,
2017-01-24 14:24:58 +00:00
imdadahad
c3a9d6d5ed Merge pull request #795 from alphagov/switch-providers-on-error
Auto-switch providers on error
2017-01-24 10:58:50 +00:00
Rebecca Law
f0215a5596 Merge pull request #800 from alphagov/increase-page-size
Increase page size for API calls to 250
2017-01-23 14:39:13 +00:00
Imdad Ahad
e1d1769345 Refactor to switch properly when providers have equal priority with test 2017-01-23 13:36:04 +00:00
Rebecca Law
74cf85bfbd Merge branch 'master' into fix-simulate-logic 2017-01-23 10:48:37 +00:00
Rebecca Law
fb27a2d4c9 Changed the body of the request to use the right phone number, the phone number is not currently used in the callback logic but if it was then this may cause a problem.
Changed the test to use a 077009 series phone number.
2017-01-23 10:43:33 +00:00
Leo Hemsted
0d415d9469 Merge pull request #796 from alphagov/dont-strip-html-on-save
Don’t strip HTML when saving templates
2017-01-20 16:30:05 +00:00
Imdad Ahad
0a277b26b6 Switch providers ONLY on provider exception 2017-01-20 16:14:29 +00:00
Imdad Ahad
d466265a9d Refactor provider switch methods in dao and tests 2017-01-20 16:13:13 +00:00
Rebecca Law
5e3bb08860 The notification.to field is being formatted to be +447...
This meant that the research mode task would never work.
Also the way we mark data as "temporary-failure" with firetext is with first a pending status callback then a declined callback.
This PR changes the research mode task to account for that situation.
2017-01-20 13:17:53 +00:00
Rebecca Law
8ed0979251 increase page size for API calls to 250 2017-01-20 12:26:55 +00:00
Rebecca Law
a3f080ef81 Fix tests, oops! 2017-01-19 15:11:02 +00:00
Chris Hill-Scott
6e6d471cda Don’t strip HTML when saving templates
Right now we strip HTML from templates at the point of saving them. This
also converts stuff like ampersands to their entity form (eg &) and
this is what we save in the database.

This is a bad idea when you’re sending a text message or a letter, in
which an HTML entity makes no sense. But we still need to encode HTML in
the body of HTML emails.

The right place to do this is when rendering the templates. The code to
do this is now in utils. So this commit:
- pull in this new utils code
- removes the old
- adds some integration tests to make sure that everything is working
  as expected (more thorough unit tests are happening in utils)
2017-01-19 12:05:28 +00:00
Rebecca Law
d0bea35069 Merge pull request #794 from alphagov/priority-by-template
Priority by template
2017-01-18 10:28:53 +00:00
Rebecca Law
f66670c558 - Added logging to indicate that we created the notification and that we sent the notification to a delivery queue.
- Small updates as recommended by review comments.
2017-01-18 09:56:26 +00:00
Rebecca Law
41b49eb8e0 Make the update template endpoint work when process_type is present. 2017-01-17 15:48:51 +00:00
Imdad Ahad
c221118669 Auto switch providers if exception is returned on sms delivery 2017-01-17 15:46:02 +00:00
Imdad Ahad
2896ed27f2 Add dao methods to retrieve providers, switch, toggle along with tests 2017-01-17 15:38:54 +00:00
Chris Hill-Scott
cbe6a20d4e Merge pull request #793 from alphagov/better-expired-error
Give a more helpful error when token has expired
2017-01-17 14:48:57 +00:00
Rebecca Law
e360a1f8f6 Merge branch 'master' into priority-by-template 2017-01-17 14:08:36 +00:00
Rebecca Law
b34c5f0bcd If the template.process_type = PRIROITY send the message to the notify queue.
We are using the notify queue in this iteration because that queue is a low volume queue with it's own dedicated workers. This just saves us from building a new queue at this point, and a new queue may not be necessary.
2017-01-17 13:16:26 +00:00
Rebecca Law
a8cdabcecd - Refactor v2 post_notification to use a single method for sms and email.
- Added the `simulate` notification logic to version 2. We have 3 email addresses and phone numbers that are used
to simulate a successful post to /notifications. This was missed out of the version 2 endpoint.
- Added a test to template_dao to check for the default value of normal for new templates
- in v2 get_notifications, casted the path param to a uuid, if not uuid abort(404)
2017-01-17 12:08:24 +00:00
Andras Ferencz-Szabo
e10c5e28cd Merge pull request #763 from alphagov/cloudfoundry
Run on Paas
2017-01-17 11:28:38 +00:00
Chris Hill-Scott
b6b9b3b225 Give a more helpful error when token has expired
We’ve seen quite a few developers encounter the `Invalid token: expired`
error message when they’re getting started using the Notify API. When
this happens they either raise a support ticket or ask for help on
Slack.

In every case this has been because the clock on their
machine/environment/container isn’t accurate. The error message doesn’t
help them figure this out.

This commit adds extra detail to the error message so they can fix the
problem without having to come to us for help.
2017-01-17 10:44:00 +00:00
Rebecca Law
61b6b1bab9 Merge branch 'master' into add-templates-process_type 2017-01-13 12:30:31 +00:00
Rebecca Law
6c79ddbe41 This is the first deploy in series of deploys to give certain templates priority in processing.
If the template is marked as priority the notification will be sent using the `notify` queue.
The `notify` queue is a low volume queue, messages here will not be queue behind a large job and should be delivered with in a more consistent time frame.

- Added templates.process_type and templates_history.process_type column.
- Added a template_process_type table to handle the enum for templates.process_type, initial values are normal and priority

https://www.pivotaltracker.com/story/show/135429147
2017-01-13 12:14:34 +00:00
Leo Hemsted
48424b32aa Merge pull request #782 from alphagov/conftest-cleanups
Conftest cleanups
2017-01-13 11:57:55 +00:00
bandesz
fd66fbd719 Run API on Paas 2017-01-13 10:24:13 +00:00
Leo Hemsted
0f327a0995 remove pytest.mark.usefixtures decorator
while it's nice to use the decorator to signify fixtures with side
effects, it has unfortunate problems of completely overriding any
fixtures you've declared in the funcargs - so isn't really suitable
for our usecase where we often have other fixtures we rely on to
return values to us.

So for consistency, let's remove this and stick to using funcargs
to define our fixtures
2017-01-10 15:04:28 +00:00
Leo Hemsted
44a8526807 use pytest.mark.usefixtures instead of funcargs
when you invoke the fixture `sample_user`, it does two things: it
creates the user in the database, but also returns the user, a useful
object that you may want to manipulate or reference in your test.

however, when you invoke the fixture `notify_db_session`, it doesn't
do anything - rather, it *promises* to clear up the database tables
at the end of the test run.

because we have no need of the notify_db_session object in our tests
(indeed, for a long time this fixture just returned `None`), using
`pytest.mark.usefixtures('notify_db_session')` brings attention to the
fact that this is a side-effect fixture rather than a data setup
fixture. Functionally it is identical to passing as a parameter
2017-01-10 15:04:28 +00:00
Leo Hemsted
996bd2579a fix failing tests
make sure we're using notify_db_session to ensure test independence
2017-01-10 15:04:28 +00:00
Leo Hemsted
d550893377 update tests to use create_user instead of sample_user
note that all of these tests have to be checked to ensure that they
still call through to notify_db_session (notify_db not required) to
tear down the database after the test runs - since it's no longer
required to pass it in to the function just to invoke the sample_user
function
2017-01-10 15:04:28 +00:00
Leo Hemsted
aeb79a0de9 refactor sample_user to create a new create_user function
this create_user function can be imported for use in creating specific
users in your tests, for example

```
from tests.app.db import create_user
def test_create_user_persists_to_database(notify_db_session):
    user = create_user()
    assert User.query.count() == 1
```

this has the benefit of not requiring you to pass the notify_db and
notify_db_session fixtures around, and separating custom object
creation from the fixture dependency trees to aid clarity

I started with sample_user since it has no downstream dependencies, but
the intention is to push this out to all db fixtures eventually. This
is a total conversion, but can be rolled out in a non-breaking manner
by keeping arguments in the fixture, and passing them through to the
new db function - then tests can be updated to use the create_* instead
of sample_* functions as and when you want to
2017-01-10 15:04:28 +00:00
Rebecca Law
cefa00b3fa Merge branch 'master' into fix-billable-units-query
Conflicts:
	tests/app/dao/test_notification_dao.py
2017-01-10 14:24:08 +00:00
minglis
0c6193e2e9 Merge pull request #775 from alphagov/do-not-write-test-data-to-the-history-table
Do not write test data to the history table
2017-01-10 13:05:06 +00:00
Rebecca Law
34a9b8aba4 Merge branch 'master' into fix-billable-units-query 2017-01-10 08:55:23 +00:00
Leo Hemsted
501187a9f4 bump utils to 13.0.1
brings in a fix to InvalidEmail/Phone/AddressExceptions not being
instantiated correctly. `exception.message` is not a python standard,
so we shouldn't be relying on it to transmit exception reasons -
rather we should be using `str(exception)` instead. This involved a
handful of small changes to the schema validation
2017-01-09 16:37:58 +00:00
Rebecca Law
a1d8ca9364 Use a variable for the massive month date conversion.
Document the massive date function.
Document the get_april_fools function.
2017-01-09 15:34:24 +00:00
Rebecca Law
88e1e58f24 Fix the get_april_fools method to use March 31 23:00 as the timestamp.
The financial year start April 1, 00:00 BST and our dates are stored as UTC.
Added a test for get_april_fools.
Added some test more test data for get_billable_unit_count_per_month.
2017-01-09 14:54:25 +00:00
Rebecca Law
17767dfa62 Calculating the billing units per month was taking too long for a services with a lot of data.
The query now does the sum per month.
2017-01-09 11:13:24 +00:00
Rebecca Law
bbdbb9079e Fixed the template uri for the post notification responses.
It was optimistic to think the v2 templates would be complete.
2017-01-04 11:22:50 +00:00
Rebecca Law
7f9111fd90 Removed the oneOf validation in the get_notification_response schema. 2017-01-04 11:00:47 +00:00
Rebecca Law
c8b11ffc8a Merge pull request #783 from alphagov/add-date-filter-to-get-services
Added a date range filter for the get all services end point.
2017-01-03 10:29:23 +00:00
Rebecca Law
0ec84ff5e8 Refactor the get_detailed_services so that the start and end date are not defaulted to None.
Set the start and end date to today's date if they are not set in the request.args
2016-12-29 13:50:41 +00:00
Rebecca Law
1de022f005 Update the query to execute immediately.
Fix indent.
Left a comment as to why start and end date are not set.
2016-12-29 13:28:55 +00:00
Leo Hemsted
d4f23f1e44 don't use cloudfront on preview or staging
cloudfront caching isn't set up on those environments yet, so continue
to use flask for now - in the future we'll want to revert this once
those environments are up and running properly
2016-12-29 12:15:47 +00:00
Leo Hemsted
9e27574298 Merge pull request #779 from alphagov/switch-to-cloudfront
use cloudfront instead of flask to serve static images
2016-12-29 11:35:05 +00:00
Rebecca Law
8ad078b663 Added a date range filter for the get all services end point.
When the start_date and end_date query argruments exists in the request,
the query will return the results from the NotificationHistory table for the given date range.
We will need to check the performance of this query, but this will only be used by the platform admin page.
2016-12-28 15:39:55 +00:00
Leo Hemsted
8c5c712f2b Merge pull request #781 from alphagov/subject-placeholder
fix placeholders not appearing in email subject
2016-12-22 15:35:44 +00:00
Rebecca Law
b2c293e53f Merge pull request #778 from alphagov/refactor-notify-user-endpoints
Refactor notify user endpoints
2016-12-22 14:50:12 +00:00