Commit Graph

32 Commits

Author SHA1 Message Date
Rebecca Law
b73bf4220f Refactor process_letter_notification to make the code easier to read.
- Separated the logic of precompiled and template letters.
- Remove the check for research mode, research mode is not relevant to api calls. The test key is used for testing.
Refactor upload_pdf_letter to accept a precompile boolean to save a query to template.
2018-04-09 13:56:44 +01:00
Richard Chapman
023862dfdc Refactored code to be more Pythonic and make the code more readable 2018-03-27 13:32:46 +01:00
Richard Chapman
3299055a09 Refactored the shared code between
move_scanned_pdf_to_test_or_live_pdf_bucket and
move_failed_pdf to consolidate some code so it is easier to maintain in
future as so that _move_s3_object can be used for any new methods.
2018-03-27 10:32:40 +01:00
Richard Chapman
8b6d28d3b0 Added a new task to handle any error cases with the anti-virus
application. If the Anti-virus app fails due to s3 errors or ClamAV
so does not scan (even after retries) the file at all an error needs
to be raised and the notification set to technical-failure.

Files should be moved to a 'folder' a separate one for ERROR and FAILURE.

* Added new letter task to process the error
* Added a new method to letter utils.py to move a file into an error or
failure folder based on the input
* Added tests to test the task and the utils.py method
2018-03-26 14:18:44 +01:00
Ken Tsang
1a9bc2a5cf Move test letters to test letters bucket without date folder name 2018-03-23 14:59:48 +00:00
Ken Tsang
0ee5c33084 Add antivirus check on precompiled letters sent with test key
- precompiled PDFs sent by test key uploaded to scan bucket
- set status to VIRUS-SCAN-FAILED for pdfs failing virus scan rather than PERMANENT-FAILURE
- Make call to AV app for precompiled letters sent via a test key, and set notification status to PENDING-VIRUS-SCAN
2018-03-23 12:04:37 +00:00
Ken Tsang
d6f6669610 Add comments for copying s3 objects 2018-03-21 16:33:49 +00:00
Ken Tsang
0c102f0727 Update letter utils for methods to handle virus process
- add function to get reference from filename
- add function to move pdf from scan folder to process folder
- add function to delete pdfs from scan bucket for failed virus scans
2018-03-20 10:12:59 +00:00
Ken Tsang
8733d84e75 Upload precompiled letter pdfs to letters-scan bucket 2018-03-20 10:11:36 +00:00
Ken Tsang
d40bc28f2a Add test-letters bucket for precompiled test keys
- uses moto to mock out boto3 for testing
2018-03-14 21:27:07 +00:00
Ken Tsang
c4e1d56492 Update letter utils to handle precompiled letters sent by test API keys
- precompiled letters sent using a test key should be put into a test-letters bucket
- also refactored code from test_letter_pdf_tasks
2018-03-14 18:15:00 +00:00
Ken Tsang
7011b90bd4 Refactor is_precompiled_letter to model 2018-03-07 23:03:03 +00:00
Ken Tsang
28136734e4 Refactor to use is_precompiled_letter in letters/utils.py 2018-03-06 13:04:57 +00:00
Richard Chapman
a4feaba309 Added tests to tests for precompiled flow and refactored a little
* Added is_precompiled_letter method to letter/utils.py
* Added tests for letter/utils.py
* Added tests for the rest endpoint
* Moved the Precompiled name to a central location
* Added hidden field to the test method to create a template
2018-03-05 14:11:37 +00:00
Richard Chapman
a9a67ce542 Updated API to handle pre-compiled pdfs
* added a method to letter/utils.py to get the PDF document from the S3
bucket
* added the logic to return the pdf or to produce a png of the pdf
2018-03-02 14:54:28 +00:00
Rebecca Law
891a80addf Added notification id to the log message for upload pdf to make it easier to search for the letter notification in the logs. 2018-03-01 10:37:07 +00:00
Alexey Bezhan
8971a5adce Upload pre-compiled letter PDF to S3
Pre-compiled letter endpoint uploads PDF contents to S3 directly
instead of creating a letter task to generate PDF using template
preview.

This moves some of the utility functions used by existing letter
celery tasks to app.letters.utils, so that they can be reused by
the API endpoint.
2018-02-23 17:52:25 +00:00
Leo Hemsted
2c758cbccd use new send-jobs-to-dvla task instead of send-files-to-dvla
the tasks (on the ftp app) do the same thing, but to improve clarity
i've renamed it to jobs, because we'll be adding a notifications one
soon
2017-09-12 17:28:07 +01:00
Ken Tsang
4fb5e68ce7 Refactor code
- Created TaskNames for DVLA_FILES rather than have DVLA_FILES in QueueNames
- Removed PROCESS_FTP from all_queues() as this was causing problems in picking up letter job tasks
- Created test to ensure that we don't arbitrarily add queue names to all_queues
2017-08-23 13:22:07 +01:00
Ken Tsang
18881cd580 Add scheduled letter jobs 2017-08-23 13:22:07 +01:00
Ken Tsang
8c4a084a0d Refactor letters filename 2017-08-23 13:22:07 +01:00
Leo Hemsted
6c61a3fc2a Revert celery4
Revert the following three pull requests:
https://github.com/alphagov/notifications-api/pull/1085
https://github.com/alphagov/notifications-api/pull/1086
https://github.com/alphagov/notifications-api/pull/1088

celery 4.0.2 looked promising, however, on staging under mild load
(5/sec api calls) the performance was actually worse than 3.1.25
2017-07-19 15:17:19 +01:00
Martyn Inglis
786adb5d71 Move Queuenames in with the celery code, revamp config to allow move to celery 4.x 2017-07-12 12:01:52 +01:00
Martyn Inglis
2591d3a1df This massive set of changes uses the new queue names object throughout the app and tests.
Lots of changes, all changing the line of code that puts things into queues, and the code that tests that.
2017-05-25 10:51:49 +01:00
Ken Tsang
8792461998 Changed response to 201 for send_letter_jobs 2017-04-11 16:42:08 +01:00
Ken Tsang
ad6bdbcc97 Removed redundant json import 2017-04-11 15:52:47 +01:00
Ken Tsang
155e173363 Update send letter jobs to return json 2017-04-11 15:44:33 +01:00
Rebecca Law
f7ab49246a Added an endpoint that returns all letter jobs, ordered by created_at desc. 2017-04-07 11:50:56 +01:00
Rebecca Law
a9e429b09f Fix task name, should have dashes not underscores 2017-04-06 15:26:50 +01:00
Rebecca Law
1d46dd0be1 Remove bucket-name from the task. 2017-04-06 15:03:06 +01:00
Rebecca Law
7bd0a07f0d Added bucket name to task arguments. 2017-04-06 12:18:34 +01:00
Rebecca Law
970a4e7b4e New endpoint to send a list of job ids to a queue.
The task will be picked up by the FTP app. Given the list of job ids the tasks will get all the files from s3, aggregate them then send to dvla
2017-04-06 12:14:17 +01:00