During database upgrades and database fail overs there has been errors
because the database connection stays open, when a query is run the
query fails and the connection is re-established. To avoid these errors
shorter timeouts have been used to keep the connections from getting
stale.
- SQLALCHEMY_POOL_TIMEOUT timeout idle connections after 30 secs
- Updated SQLALCHEMY_POOL_RECYCLE to recycle the connection every 5 mins
See guide on optimistic disconnect handling - using the pool recycle
as a way to manage this:
http://docs.sqlalchemy.org/en/latest/core/pooling.html#disconnect-handling-optimistic
If is_letter_contact_blank then the user has set the letter contact block to be blank on purpose
ELSE IF is_letter_contact_blank is false THEN use the template default
IF template default is blank THEN the service_letter_contact is blank use the service default
Grouping the letters into a maximum number of files is necessary because
the SQS task needs to be under a certain size. We also compress the task
when sending.
When we’re doing user research we often:
- start the task by inviting the participant to a service on Notify
- have them use a prototype version of the admin app, hosted on a
different domain
Currently we can’t do both of these things together, because the invite
emails always send people to `notifications.service.gov.uk` (because
it’s the API that sends the emails, and the prototype admin app points
at the production API).
This commit changes the API to optionally allow an instance of the admin
app to specify which domain should be used when generating invite links.
add collate-letter-pdfs task (name pending). This retrieves a list of
letter pdf files (just the metadata, not the actual data) from s3, and
loops through them, calling the ftp task zip-and-send-letter-pdfs. It
groups them up by adding them to lists while counting the total
filesize, if it gets over a certain filesize (currently set to 500mb)
it breaks at that chunk, sends off that list of files to the ftp app,
and then starts building up a new list.
DVLA have a hard 2gb limit on how big the zip files we can send is -
however we're going to be limited by the amount of memory on the ftp
app well before we get around to handling 2gb of pdf data - so the
limit is 500mb for now. We'll adjust it after we see how ftp performs.
SQL Alchemy config changes were made to decrease the downtime of the
application. The last test only had 1 min of downtime in the upgrade
period i.e. 40 mins. Tested without the config changes to double
check the change had the desired effect. Adding back in so we can test
the changes under load and performance test outside of upgrade.
SQL Alchemy config changes were made to decrease the downtime of the
application. The last test only had 1 min of downtime in the upgrade
period i.e. 40 mins. Reverting the changes so that the same process
can be followed to ensure the changes had the desired effect.
example log line:
```
API AuthError: AuthError({'token': ['Invalid token: signature, api token is not valid']}, 403, service_id=3e1ed7ea-8a05-4b4e-93ec-d7bebfea6cae, api_key_id=None)"
```
Returns either template's reply_to text if set or the related field
from the default service record.
Return value can be used as default for `Notification.reply_to_text`
when per-notification value is not provided.
Checks that email/sms/letter reply to object has the same service_id
as the template it's being attached to, to make sure it's not possible
to retrieve data about return addresses for other services.
Validators check that service_letter_contact_id belongs to the
same service as the notification/template.
Generic reply_to validator calls the correct function for the given
type (for either notification or template). It can be used by the
template API endpoints to verify that given reply_to ID has the same
service_id as the template itself.
The original approach was to create a DB foreign key constraint,
but this caused issues with the `version_class` decorator saving
related Service objects without creating a history record.
If someone receives an invitation email for Notify the reply-to address
of the email was the GOV.UK Notify email address. This has been changed
to be the email address of the user who sent the invite.
Pivotal story: https://www.pivotaltracker.com/story/show/153094646
