Commit Graph

194 Commits

Author SHA1 Message Date
Ken Tsang
7e70b44113 Error in task when letter template and in trial mode 2017-08-30 16:04:10 +01:00
Leo Hemsted
c2152f9cb8 remove billable-units endpoint
it wasn't used anywhere, and the return value changed when flask was upgraded
2017-08-21 10:57:15 +01:00
Leo Hemsted
c36e50bef1 update dependencies 2017-08-18 17:02:31 +01:00
Leo Hemsted
614880f6d9 send to send-sms-tasks and send-email-tasks instead of send-tasks 2017-07-21 13:49:37 +01:00
Rebecca Law
5ef0450e37 Update the json in the post to the service for the inbound sms. 2017-06-22 10:15:08 +01:00
Rebecca Law
c608f5997b New task to send the inbound sms to the service.
If the service has not set the url then nothing happens.
If the request to the service url returns with 500 or greater the task is retries.
The task is created when the SMS provider post the inbound SMS.
2017-06-20 17:13:40 +01:00
Martyn Inglis
2591d3a1df This massive set of changes uses the new queue names object throughout the app and tests.
Lots of changes, all changing the line of code that puts things into queues, and the code that tests that.
2017-05-25 10:51:49 +01:00
Imdad Ahad
02db3be37c General refactor 2017-05-22 10:12:18 +01:00
Imdad Ahad
4003edfa67 Add DVLA callback:
* Process SNS callback, trigger the update notifications celery task
* Put autoconfirm into its own method and use in callbacks
2017-05-15 11:12:31 +01:00
Imdad Ahad
ee484ec368 Add get_s3_file method for use in DVLA processing 2017-05-15 10:48:37 +01:00
Imdad Ahad
7a10a91262 Revert "Process SNS request triggered by a DVLA S3 update" 2017-05-12 17:21:07 +01:00
Imdad Ahad
0f7093fc38 Refactor and add filename in logging 2017-05-12 14:32:01 +01:00
Imdad Ahad
8a5e82904e Update to pull from correct bucket and fix tests not mocking out correctly 2017-05-12 14:24:26 +01:00
Imdad Ahad
f766f90207 Add task to process a DVLA response file:
* Currently we do nothing with the parsed response. We will
* update the status of the notifications in a separate PR
2017-05-12 14:24:26 +01:00
Imdad Ahad
c4fac1d937 Revert "Revert "add DVLA organisations to API"" 2017-04-21 16:05:07 +01:00
Martyn Inglis
5550201939 Upper case only in the random notification id generator
as DVLA need upper case only or the barcode doesn't generate.
2017-04-20 10:55:53 +01:00
minglis
ddda2bd158 Merge pull request #902 from alphagov/task-to-set-job-to-error
Task to set job to error
2017-04-18 16:07:32 +01:00
Martyn Inglis
2b4043515f fixed tests that used the new state. 2017-04-18 14:40:48 +01:00
Martyn Inglis
8956338d31 Add a new task to update a job to error - note it leaves the notifications in whatever state they were in. 2017-04-18 11:42:48 +01:00
Martyn Inglis
739ff2dceb Extra test on personilisation block 2017-04-18 11:07:11 +01:00
Martyn Inglis
1894c96f3a updated test to use random string not numeric id formats 2017-04-13 12:52:49 +01:00
Martyn Inglis
b0e5062df2 Added the random string reference to the letter
- uses the reference field on the notifications table to store a 16char random string used to cross reference DVLA letters back to the notification
- used as letter barcode does not have space for a UUID notification id

Depends on https://github.com/alphagov/notifications-utils/pull/149

Renamed the numeric_id to notification_reference in utils and changed validation rules to match this

Note also the persist_notification method set "reference" to be "client_reference" which is confusing and they are different things, so fixed this too.
2017-04-12 17:56:55 +01:00
Rebecca Law
e022fe0403 Update build_dvla_file with a mocked notify_celery.send_task.
Check the task is being called with the right parameters.
2017-04-11 11:13:47 +01:00
Rebecca Law
cf3eeb2120 Merge branch 'master' into task-to-update-letter-job
Conflicts:
	tests/app/celery/test_tasks.py
2017-04-06 17:18:09 +01:00
Rebecca Law
add0cfa62f Add a task for the FTP app to call that will update the job to sent to dvla and all the notifications for that job as sending. 2017-04-06 17:16:08 +01:00
Chris Hill-Scott
8bd386028b Merge pull request #893 from alphagov/dont-output-to_name_2
Build DVLA file with blank `TO_NAME_2` field
2017-04-06 16:54:02 +01:00
Chris Hill-Scott
7a6d748c6a Build DVLA file with blank TO_NAME_2 field
Problem: we were sending the first line of the address in the
`TO_NAME_2` field. This meant that they couldn’t do any PAF lookups with
it, because it wasn’t where they were expecting.

The first line of the address is the second line of what our users give
us. We need to give this to DVLA as the _third_ line of the address
output, which they call `TO_ADDRESS_LINE_1`.
2017-04-06 13:12:05 +01:00
Rebecca Law
4f6a66c904 Merge branch 'master' into update-job-status
Conflicts:
	app/dao/jobs_dao.py
	tests/app/dao/test_jobs_dao.py
2017-04-06 11:11:37 +01:00
Chris Hill-Scott
61a8097ca5 Merge pull request #886 from alphagov/revert-883-revert-867-bump-utils-lists-as-placeholder-values
Revert "Revert "Bump utils to allow lists as placeholder values""
2017-04-05 13:14:57 +01:00
Chris Hill-Scott
f98f220e74 Update utils to fix letter output
Includes:
- [x] https://github.com/alphagov/notifications-utils/pull/136
- [ ] https://github.com/alphagov/notifications-utils/pull/131
2017-04-05 12:35:28 +01:00
Rebecca Law
c49217b1bf Update job status to ready to send, indicating the file for the job has been uploaded to S3. 2017-04-05 11:57:56 +01:00
Chris Hill-Scott
e47a008dd9 Pass through contact block from service to letter
Whatever a user has entered for their service’s contact block should
appear in the right place in the file we give to DVLA.

The work to output in the right fields in the DVLA file has already been
done. We just weren’t passing it through. This commit passes it through.
2017-04-03 13:27:47 +01:00
Chris Hill-Scott
eb5f9421dd Check for what DVLALetterTemplate is called with
Extends the test to make sure that the thing that builds each line of
the file is getting called with the right template, personalisation and
numeric ID. Will be helpful the more complicated the call to the
template gets.
2017-04-03 13:25:22 +01:00
Martyn Inglis
7dcd3164e2 Revert "Reinstating the 2 task model for API submitted notifications."
This reverts commit 1c154c4113.
2017-03-30 10:46:23 +01:00
Martyn Inglis
385a73da2b Revert "ensure we're passing through api keys and key types from notifications"
This reverts commit 25d1777937.
2017-03-30 10:38:41 +01:00
Leo Hemsted
25d1777937 ensure we're passing through api keys and key types from notifications
when we made the change to async persist notifications, we forgot to
pass through api_key_id and key_type. in send_sms/email, for legacy
reasons, they default to None/KEY_TYPE_NORMAL, so regardless of what
your api key was set up as, we would send real messages!

TODO: Once the PaaS transition is complete and the task changes are
reverted, remove the api_key_id and key_type params from the send_*
tasks entirely, as those are only called from the csv job flow, and
don't need them
2017-03-28 13:14:46 +01:00
Rebecca Law
31979c3272 Use 500 for org id 2017-03-27 17:49:09 +01:00
minglis
d7adfcaf87 Merge pull request #866 from alphagov/no-downtime-hosting-migration
Reinstating the 2 task model for API submitted notifications.
2017-03-27 13:54:43 +01:00
Rebecca Law
37a5ce601a Update the version of notifications-utils.
There was a problem with the address block for the DVLATemplate.

Added a countdown to the build_dvla_file.
2017-03-24 15:25:02 +00:00
Martyn Inglis
1c154c4113 Reinstating the 2 task model for API submitted notifications.
This is being done for the PaaS migration to allow us to keep traffic coming in whilst we migrate the database.

uses the same tasks as the CSV uploaded notifications. Simple changes to not persist the notification, and call into a different task.
2017-03-23 14:41:00 +00:00
Rebecca Law
140179b4b6 Create new task to build dvla file.
This will transform each notification in a job to a row in a file.
The file is then uploaded to S3.
The files will later be aggregated by the notifications-ftp app to send to dvla.

The method to upload the file to S3 should be pulled into notifications-utils package.
It is the same method used in notifications-admin.
2017-03-15 15:26:58 +00:00
Rebecca Law
f56824adde Cancel job if the service is inactive.
Update the PermissionsDao.get_permissions_by_user_id to only return permissions for active services,
this will make the admin app return a 403 if someone (otherthan platform admin) tries to look at an inactive service.
Removed the active flag in sample_service the dao_create_service overiddes this attribute.
2017-02-02 11:34:00 +00:00
Leo Hemsted
70cd3fb335 ensure that the celery workers know about the new db-letter queue
this fixes running locally and on paas, a separate PR is in
notifications-aws to fix work on aws
2017-01-24 12:32:20 +00:00
Leo Hemsted
4f238d241a persist_letter saves address correctly to database
the `to` field stores either the phone number or the email address
of the recipient - it's a bit more complicated for letters, since
there are address lines 1 through 6, and a postcode. In utils, they're
stored alongside the personalisation, and we have to ensure that when
we persist to the database we keep as much parity with utils to make
our work easier. Aside from sending, the `to` field is also used to
show recipients on the front end report pages - we've decided that the
best thing to store here is address_line_1 - which is probably going to
be either a person's name, company name, or PO box number

Also, a lot of tests and test cleanup - I added create_template and
create_notification functions in db.py, so if you're creating new
fixtures you can use these functions, and you won't need to pass
notify_db and notify_db_session around, huzzah!

also removed create param from sample_notification since it's not used
anywhere
2017-01-24 12:32:20 +00:00
Leo Hemsted
c904025ee9 add comprehensive tests for process_row
in the future we can probably remove some of the slower databasey
process_job tests, but that's out of scope for this
2017-01-24 12:22:31 +00:00
Leo Hemsted
542f08d0b5 refactor exception handling code in job tasks code
it's almost entirely duplicated so share it across.

also clean up retrying - `task.retry(...)` raises a
celery.exceptions.Retry object, so you do not need to `raise` its
response. additionally, cleaned up tests around that since raising
Exception and asserting Exception is raised is dangerous as it could
mask actual programming errors
2017-01-24 12:22:31 +00:00
Leo Hemsted
0324ba02ff split process_job up into process_job and process_row
makes tests a bit cleaner, and makes it much easier to test letter
functionality. haven't moved many tests around, just changed a couple of mock calls
2017-01-24 12:22:31 +00:00
Leo Hemsted
d550893377 update tests to use create_user instead of sample_user
note that all of these tests have to be checked to ensure that they
still call through to notify_db_session (notify_db not required) to
tear down the database after the test runs - since it's no longer
required to pass it in to the function just to invoke the sample_user
function
2017-01-10 15:04:28 +00:00
Leo Hemsted
3ed97151ee fix placeholders not appearing in email subject
it now switches utils.template.Template type, since the base Template
type now no longer has a subject attribute.

updated test case to use `sample_email_template_with_placeholders`
instead of `sample_email_template`
2016-12-22 14:40:33 +00:00
Martyn Inglis
d8625f9da4 Fixed tests that failed due to changes in utils brought in by latest version.
- seems phonenumber/emailaddress from the CSV are now passed in as personalisation.
- assume the renderer does the correct thing here. Will need to check with @quis
2016-11-30 17:22:03 +00:00