Commit Graph

6670 Commits

Author SHA1 Message Date
Leo Hemsted
e094dd4bfd remove loadtesting from providers
we don't use it since we wrote our own provider stubs for performance
tests.

this removes it from the api - it's still in the DB and will be
retrieved by queries, but is set to disabled on prod
2019-10-23 11:45:07 +01:00
Leo Hemsted
496b6f4737 Merge pull request #2627 from alphagov/letter-alert-v3
Letter alert
2019-10-22 13:26:35 +01:00
Pea (Malgorzata Tyczynska)
35602b96fc Merge pull request #2631 from alphagov/add-validation-message-to-s3-as-metadata
Put pdf letter validation failed message in S3 metadata
2019-10-18 10:29:54 +01:00
Rebecca Law
ad14f96b8d A small change to make the code just a little bit clearer. 2019-10-17 15:17:43 +01:00
Pea Tyczynska
6ee7ac6cac Refactor and harmonise metadata for invalid letters with those sent from admin app 2019-10-16 14:11:50 +01:00
Leo Hemsted
0340927a6e Merge pull request #2634 from alphagov/startup-timeout
set startup timeout to 10 minutes
2019-10-15 14:18:48 +01:00
Leo Hemsted
46b0cce17b set startup timeout to 10 minutes
5 minutes isn't long enough to scale api at load, leading to failed deploys
2019-10-15 12:09:06 +01:00
Leo Hemsted
dc02060de2 Merge pull request #2632 from alphagov/scaling-fix
scale up api to 25 instances before deploy on production
2019-10-15 11:07:14 +01:00
Katie Smith
c9743d70b4 Merge pull request #2633 from alphagov/update-noti-history-query
Update insert_update_notification_history to take a query limit
2019-10-14 17:01:03 +01:00
Katie Smith
fb80a9c92e Update insert_update_notification_history to take a query limit
The nightly job to delete email notifications was failing because it was
timing out (`psycopg2.errors.QueryCanceled: canceling statement due to statement timeout`).

This adds a query limit to the query which inserts or updates
notification history so that it only updates a maximum of 10000 rows at
a time.
2019-10-14 16:51:46 +01:00
Leo Hemsted
9c2ded00c1 scale up api to 25 instances before deploy on production
deploys take up to five minutes, during which notify-paas-autoscaler
can't scale the app. We saw 502s due to a large volume of traffic
coming in during that time, and we couldn't react cos we were
deploying.

scale up to 25 instances, the autoscaler won't be able to downscale
until after the deploy has finished.
2019-10-14 16:12:35 +01:00
Leo Hemsted
b886127c80 Merge pull request #2630 from alphagov/clean-dockerfile
clean up docker and makefile
2019-10-14 11:37:40 +01:00
Pea Tyczynska
0b65e75fe9 Format metadata correctly and use MetadataDirective to put new metadata in S3 object 2019-10-14 11:11:32 +01:00
Pea Tyczynska
0a617379c4 Put pdf letter validation failed message in S3 metadata
So it can be used to tell the user why the letter has failed validation
2019-10-11 17:24:48 +01:00
Leo Hemsted
9e64dcbb2f clean up docker and makefile
* remove gosuuser - this means we can upgrade the base image to
  something more modern and not have to faff around with gpg
* remove unnecessary commands - some things need to exist in the
  makefile to keep jenkins happy
* remove concept of building separately - pip install requirements.txt
  in the dockerfile
2019-10-11 13:55:21 +01:00
Leo Hemsted
8285ef5f89 only check for dvla response files on mon/weds/fri
dvla don't process 2nd class files on tues and thurs
2019-10-08 18:16:45 +01:00
David McDonald
0152dcf254 Merge pull request #2626 from alphagov/drop-datetime-dimension-table
Delete dm_datetime database table
2019-10-08 14:37:26 +01:00
David McDonald
4bc71f36bf Delete dm_datetime database table
No longer needed
2019-10-08 14:02:43 +01:00
David McDonald
546ffe5094 Merge pull request #2625 from alphagov/drop-dm-dimension
Drop datetime dimension model
2019-10-08 10:33:11 +01:00
David McDonald
a0ace380b1 Drop datetime dimension model
We have a table for datetime dimension that we no longer use and
I believe can be dropped.

First step is to remove the model and release the change. The next
step will be to then add a database migration to drop the actual
table. I believe we need to do it in this order and it can't
be done as a single PR.
2019-10-08 10:15:39 +01:00
Leo Hemsted
6965478afc explicitly name days in celery unit tests
make it easier to understand what's happening
2019-10-07 08:58:09 +01:00
Rebecca Law
2c41d6130c Merge pull request #2617 from alphagov/notification-count-for-job
Return count of notifications in the database for a job
2019-10-03 16:54:30 +01:00
Rebecca Law
7fc7d99dac Update the new endpoint to return a 404 if the job or service id are not found.
All our endpoint should perform a check that the params are valid - this is an easy whay to check that and is standard for our endpoints.
I reverted the query to just filter by job id.
2019-10-03 14:58:49 +01:00
Chris Hill-Scott
8f6b0baba0 Merge pull request #2620 from alphagov/update-service-branding-when-updating-org
Update the branding of services branding when updating their organisation’s branding
2019-10-03 10:53:34 +01:00
Rebecca Law
dd93cdc93a Merge pull request #2621 from alphagov/new-letter-rates
letter rates go up by 5p as of 1st october 2019
2019-09-27 11:57:10 +01:00
Rebecca Law
ab5de4053d Merge pull request #2622 from alphagov/process-row-from-job
Added a command to process a row from a job.
2019-09-26 14:41:30 +01:00
Rebecca Law
44b7b36acd Added a command to process a row from a job. 2019-09-26 14:19:09 +01:00
Leo Hemsted
d4c2a2b284 fix failing test
it failed after 5:30pm. pin the time to 4am, when the task actually runs
2019-09-26 11:54:22 +01:00
Leo Hemsted
ddfc722537 letter rates go up by 5p as of 1st october 2019 2019-09-25 16:33:53 +01:00
Pea Tyczynska
c48aa77dd5 Use service_id in the query to make it safer, also use named parameters 2019-09-25 16:32:27 +01:00
Chris Hill-Scott
f63565695a Refactor for reuse
Makes one resuable function, instead of 3 repetetive ones.
2019-09-25 15:36:55 +01:00
Chris Hill-Scott
1e82484dfb Update service branding when setting org branding
When a service asks for branding I often go and:
- set the branding on the organisation as well
- set the branding on all the organisation’s services

The latter can be quite time consuming, but it does save effort since
existing services from the same organisation won’t have to request
branding. It also improves the consistency of the communications that
users are receiving.

This commit automates that process by applying the branding update to
any services belonging to the organisation, if those services don’t
already have their own custom branding set up.
2019-09-25 15:36:11 +01:00
Rebecca Law
3f35c634cd Merge pull request #2619 from alphagov/use-created-at-2
Use created at 2
2019-09-25 15:34:49 +01:00
Rebecca Law
1d0ddeb17e Fix changes made with the merge conflicts. 2019-09-25 15:11:15 +01:00
Rebecca Law
a1863fa419 Update all calls to get_folder_name to include the parameter name.
Use created_at date of the notification for precompiled letters.
2019-09-25 14:40:09 +01:00
Rebecca Law
f234e94572 Change the variable name to make a little more sense. 2019-09-25 13:56:10 +01:00
Rebecca Law
702d8fa85f Refactor the code that figures out what folder and filename to use for the letter pdf files.
Now we consistently use the created_at date, so we can always get the right file location and name.

The previous updates to this code were trying to solve the problem if a pdf being created at 17:29, but not ready to upload until 17:31 after the antivirus and validation check.
But in those cases we would have trouble finding the file.
2019-09-25 13:56:10 +01:00
Pea Tyczynska
8cf8d24e37 Return count of notifications in the database for a job
When we cancel a job, we need to check if all notifications are
already in the database. So far, we were querying for all
notification objects in the database and counting them in
admin app, which runs into pagination problems for large jobs,
and could time out for very large jobs.
2019-09-24 16:56:03 +01:00
Leo Hemsted
a10aaddbcc Merge pull request #2614 from alphagov/test-updates
Make a few fixtures Pytest 4 compliant and fix tests with errors
2019-09-23 10:22:00 +01:00
Pea (Malgorzata Tyczynska)
c1b53dce3a Merge pull request #2612 from alphagov/redaction_failed_doesnt_matter_when_test_key
Don't log address redaction failure when letter sent with test key
2019-09-18 16:05:39 +01:00
Katie Smith
a2c25cec9f Make a couple of fixtures Pytest 4 compliant
By stopping them from calling other fixtures as functions.
2019-09-18 12:01:20 +01:00
Katie Smith
62735ea125 Remove sample_notification_with_api_key fixture
This was only used once and was not Pytest 4 compatible.
2019-09-18 11:04:24 +01:00
Katie Smith
09e8ac9644 Fix assertions when we catch an error in the tests
Code that is within a `with Python.raises(...)` context manager but
comes after the line that raises the exception doesn't get evaluated.
We had some assertions that we never being tested because of this, so
this ensures that they will always get run and fixes them where
necessary.
2019-09-18 11:04:24 +01:00
Leo Hemsted
f9f9092b5c Merge pull request #2613 from alphagov/test-letter-subfolder-fix
make sure test notifications don't get date subfolders
2019-09-18 10:47:05 +01:00
Leo Hemsted
d080106cbe make sure test notifications don't get date subfolders
they just go in the test bucket's root
2019-09-18 10:24:47 +01:00
Pea Tyczynska
1279a46b8b Don't log address redaction failure when letter sent with test key 2019-09-17 15:55:26 +01:00
Leo Hemsted
99eb17fc29 Merge pull request #2610 from alphagov/get-pdf-contents-via-api
add api endpoint to get pdf for letter
2019-09-17 14:55:34 +01:00
Leo Hemsted
efaa4f2ad2 karlify the exception messages
also create a PDFNotReadyError class, separate to BadRequestError, to
imply to the end user that this is something they should handle
separately to all the other errors
2019-09-17 12:16:24 +01:00
Leo Hemsted
1ad32c9168 move get_pdf functionality to its own endpoint
it felt very awkward when the body of a pdf might be empty, might have
things in it, and whether it is empty or not can change even when the
status is the same (a created template notification might have a pdf,
but might not, we don't know).

So move it to its own endpoint, so we can hand craft some 400 errors
that appropriately explain what's going on.
2019-09-12 17:06:22 +01:00
Katie Smith
281040945c Merge pull request #2607 from alphagov/upload-precompiled-letters
Allow PDF letters to be uploaded
2019-09-12 15:14:16 +01:00