Commit Graph

5277 Commits

Author SHA1 Message Date
Rebecca Law
e3a75d1b7d Notification_type is a required parameter, admin app always passes it in.
Normalise for notificaiton type.
Throw InvalidRequest exception is the notification type is invalid.
2018-03-09 09:53:05 +00:00
Rebecca Law
5f25fc0db4 The template type should be passed in when doing a search by recipent.
It is possible to search for a phone number when from the email notification page and get a SMS message in return.
This also helps to optimise the query.
2018-03-09 09:53:05 +00:00
Chris Hill-Scott
faaea75a99 Do more normalisation for better partial matching
Phone numbers sometimes contain stuff we normalise out. This matches
perfectly if we have a full phone number, because we can normalise the
thing we’re searching for in the same way as the search term.

With partial search terms we can’t do this completely, because we can’t
work out if ‘123’ is part of a UK number, an international number, the
start of the phone number, the last 3 digits, etc.

What we can do is remove some stuff that we can know will cause partial
search terms to not match:
- leading pluses
- leading `0`s
- any brackets
- any spaces
2018-03-09 09:51:50 +00:00
Chris Hill-Scott
209d75efd9 Match on partial email addresses in search
Users expect the search to work on partial email addresses ‘similar to
Gov.Pay’. We can’t have a situation where Pay are doing something better
than us 😜
2018-03-09 09:51:49 +00:00
venusbb
01ff06fc09 Rename tables, column and added indexes 2018-03-09 08:17:02 +00:00
Sakis
4767379fc7 Merge pull request #1753 from alphagov/document-new-worker-process
Describe process for creating a new worker app
2018-03-08 16:40:02 +00:00
Rebecca Law
00b17b5ad7 When we sent the service the status callback for a notification, we have all the information we need.
Which means we can remove the need to request the data from the database.
In order for the PR to be backwards compatible I have added an optional parameter "encrypted_status_update".
If this is not None then the new code is called.

The next PR will send the encrypted data to this task.

A final PR will remove the code that uses the database to get the notification and service callback api.
2018-03-08 16:17:41 +00:00
venusbb
5b0d047de3 Merge branch 'master' of https://github.com/alphagov/notifications-api into vs_billing_tables 2018-03-08 15:51:26 +00:00
kentsanggds
b0b0062b35 Merge pull request #1732 from alphagov/ken-hidden-in-json-response
Return `is_precompiled_letter` field as part of json for notification by id
2018-03-08 15:06:10 +00:00
Leo Hemsted
a777e13e4d Merge pull request #1754 from alphagov/revert-service-callback-eventlets
Revert service callback eventlets
2018-03-08 14:16:40 +00:00
Leo Hemsted
651c3062b9 retry service callbacks if the db queries fail
we don't expect them to fail, but they might if we accidentally
exhaust our connection pool. Just in case, lets retry.
2018-03-08 14:08:56 +00:00
Athanasios Voutsadakis
19129a1313 Describe process for creating a new worker app 2018-03-08 14:02:08 +00:00
chrisw
e600a199a1 handle malformed invite tokens 2018-03-08 13:58:45 +00:00
Leo Hemsted
22f86aa1b5 Revert service callback worker eventlets
We have seen problems with the service callback workers due to the db
connection pool being exhausted. When the worker picks up the task, it
makes a db query to get the notification, a query to get the callback
url, and then closes the session before it makes the 3rd party request.

However, even closing the session before the (potentially lengthy)
web request wasn't enough - we've seen significant amounts of
`sqlalchemy.exc.TimeoutError`s.

This reverts commit 2dfbd93c7e
2018-03-08 13:35:53 +00:00
Chris Hill-Scott
f902ba476c Bring in refactored CSV processing
Shouldn’t be any functional changes here, just things being named more
clearly.
2018-03-08 13:12:00 +00:00
Chris Waszczuk
1a63847988 Merge pull request #1746 from alphagov/add-is-org-unique-rest-endpoint
Add endpoint to check for unique organisation names
2018-03-08 12:11:06 +00:00
Chris Hill-Scott
c029927fde Merge pull request #1733 from alphagov/remove-dvla
Remove anything to do with DVLA format letters
2018-03-08 10:58:34 +00:00
Richard Chapman
e6a3d63d3d Merge pull request #1749 from alphagov/pyup-update-sqlalchemy-1.2.4-to-1.2.5
Update sqlalchemy to 1.2.5
2018-03-08 09:56:11 +00:00
Richard Chapman
94dea42f6f Merge pull request #1747 from alphagov/add_precompiled_letters
Fixed bug only where only data being passed to TemplatePreview when JSON is required
2018-03-08 09:56:03 +00:00
Ken Tsang
23ce36dc48 Update response to return is_precompiled_letter 2018-03-07 23:03:03 +00:00
Ken Tsang
7011b90bd4 Refactor is_precompiled_letter to model 2018-03-07 23:03:03 +00:00
venusbb
a4c054124f work in progress 2018-03-07 17:23:52 +00:00
Chris Hill-Scott
ae0113a5de Merge pull request #1748 from alphagov/quis-patch-1
Add warning to Performance Platform command
2018-03-07 17:13:46 +00:00
Sakis
1782d824c8 Merge pull request #1751 from alphagov/fix-dockerfile
Install libffi-dev and python-dev in docker
2018-03-07 16:10:40 +00:00
Athanasios Voutsadakis
a84a70d791 Install libffi-dev and python-dev in docker
This is to allow us to build cffi
2018-03-07 15:56:41 +00:00
venusbb
5082d9d83c Increase datetime number of years from 20 to 50 2018-03-07 12:57:19 +00:00
venusbb
effaecbd49 added ft_billing and dm_datetime tables for report purpose 2018-03-07 12:46:11 +00:00
Katie Smith
f78cde61e7 Merge pull request #1743 from alphagov/remove-job-stats-step-1
Delete functions which call the job statistics tasks
2018-03-07 11:11:39 +00:00
Katie Smith
a85f5d3acf Merge pull request #1741 from alphagov/pyup-update-freezegun-0.3.9-to-0.3.10
Update freezegun to 0.3.10
2018-03-07 11:11:30 +00:00
Richard Chapman
d60e802f35 Removed the superfluous variable and pass through as it pulls it
automatically out of sys.exc_info.
2018-03-07 09:51:58 +00:00
Richard Chapman
7f32b06615 Replaced the JSONDecodeError with a ValueError as it was failing on
Jenkins.
2018-03-07 09:32:56 +00:00
Katie Smith
7f2e9f507e Delete functions which call the job statistics tasks
The JobStatistics table is going to be deleted. There are currently
3 tasks which use the JobStatistics model via the Statistics DAO, so we
need to make sure that these tasks aren't being used before they are
deleted in a separate PR.

This commit deletes:
* The `create_initial_notification_statistic_tasks` function which gets
used to call the `record_initial_job_statistics` task.
* The `create_outcome_notification_statistic_tasks` function which gets
used to call the `record_outcome_job_statistics` task.
* And the scheduling of the `timeout-job-statistics` scheduled task.
2018-03-07 09:23:29 +00:00
Katie Smith
0c1d5c12a3 Merge pull request #1699 from alphagov/persist-sorted-in-letter-response-file
Create and populate DailySortedLetter table
2018-03-07 09:06:33 +00:00
Katie Smith
f1db3c25eb Merge pull request #1740 from alphagov/pyup-update-pytest-3.4.1-to-3.4.2
Update pytest to 3.4.2
2018-03-07 09:06:21 +00:00
pyup-bot
e2a308a703 Update sqlalchemy from 1.2.4 to 1.2.5 2018-03-06 19:17:29 +00:00
Richard Chapman
cf2a506b3b Removed the import and directly used json.decoder.JSONDecodeError
as it was failing on Jenkins.
2018-03-06 18:20:45 +00:00
Richard Chapman
3f9a37d8a3 Updated the import for JSONDecodeError to import from from json.decoder
as it was failing on Jenkins.
2018-03-06 17:31:08 +00:00
Athanasios Voutsadakis
0220839f4d Merge branch 'master' into support-celery-multi 2018-03-06 16:23:53 +00:00
Richard Chapman
e1e7f13f23 Added tests to ensure the parameter to the mock post request are the
correct type for the call as there are some that require json and some
binary. The additional checks ensure that that json decode either fails
or succeeds in the correct case.
2018-03-06 16:16:29 +00:00
Athanasios Voutsadakis
d365921093 Address PR comments
Make timeout 9s on both files
Use more descriptive variable name
Declare the logs dir as a constant
2018-03-06 16:11:42 +00:00
Chris Hill-Scott
5db3eef8f2 Add warning to Performance Platform command 2018-03-06 15:42:38 +00:00
Richard Chapman
77a3397ce5 Added option flag to the _get_png_preview method to determine if the
post method content is data or json format.
2018-03-06 15:35:00 +00:00
chrisw
a487293bf0 Add check org name is unique endpoint 2018-03-06 15:34:39 +00:00
Richard Chapman
ed9936bba0 Fixed bug where the content header was not being passed onto the post
request. Changed data => json.

Added extra logging to display the error with more detail
2018-03-06 14:42:53 +00:00
Richard Chapman
d0df85a602 Fixed bug where the content header was not being passed onto the post
request. Changed data => json.

Added extra logging to display the error with more detail
2018-03-06 14:24:30 +00:00
Ken Tsang
28136734e4 Refactor to use is_precompiled_letter in letters/utils.py 2018-03-06 13:04:57 +00:00
Ken Tsang
bca858f4a8 Return precompiled_letter flag rather than hidden 2018-03-06 12:46:06 +00:00
Ken Tsang
b5a2dbb24e Update schema to dump precompiled_letter
- to be a precompiled letter, the template must be a letter, be hidden, and have a matching template name to the one expected in config['PRECOMPILED_TEMPLATE_NAME']
2018-03-06 12:46:06 +00:00
Ken Tsang
5dd3c62b5c Fix template caching tests 2018-03-06 12:46:06 +00:00
Ken Tsang
564504bf97 Add template hidden field in response 2018-03-06 12:46:06 +00:00