Rebecca Law 841a4fc22f Mark letters as validation-failed if the templated letter is too long.
It is possible that the personalisation for a templated letter can make the letter exceed 10 pages or 5 sheets. We are not validating the letters posted via the API for this validation error. It is only possible to validate the letter once we create the PDF in notifications-template-preview. This means that the letter can only get a validation-failed status after the client has received a 201 from the POST to /v2/notifications.
NOTE: we only validate the preview row of a CSV for this validation error, this change will mean that it is possible for a letter to be marked as validation-failed after a successful file upload.

A new task to update the notification to `validation-failed` has been added to the API. If we find that the letter is too long once we have created the PDF we call the `update-validation-failed-for-templated-letter` task rather than `update-billable-units-for-letter` task.

New work flow for a letter in brief:
API - receives POST /v2/notifications
:: save to db
:: put CREATE_LETTERS_PDF task on queue for template preview to consume
TEMPLATE-PREVIEW - consumes task CREATE_LETTERS_PDF
:: create PDF
:: count pages of PDF
:: IF page count exceeds 10 pages
	 put in the letters-invalid-pdf S3 bucket with metadata (similar to the precompiled letters)
	 put `update-validation-failed-for-templated-letter` task on the queue for the API to consume
   ELSE
     put PDF in the `letters-pdf` bucket
     put `update-billable-units-for-letter` task on the queue
API - consumes `update-billable-units-for-letter` OR `update-validation-failed-for-templated-letter` task
:: IF `update-billable-units-for-letter` task:
   	update billable units for notification as usual
:: ELSE `update-validation-failed-for-templated-letter`:
   	update notification_status = `validation-failed`
ADMIN - view notification page for letter
:: show validation letter for templated letter

There will be 3 PRs in order to make this change, one for the API, template-preview and the admin app.

Deployment plan

Deploy Admin first
Deploy API
Deploy template-preview

Related PRs:
alphagov/notifications-template-preview#619
alphagov/notifications-admin#4107

https://www.pivotaltracker.com/story/show/169209742
2022-01-19 08:29:48 +00:00
2020-05-12 16:04:18 +01:00
2021-12-14 14:02:28 +00:00
2019-08-02 12:41:03 +01:00
2021-11-11 13:54:21 +00:00
2019-05-16 17:06:34 +01:00
2021-03-08 17:44:48 +00:00
2021-12-20 16:45:47 +00:00
2021-12-20 16:45:47 +00:00
2019-05-16 17:06:34 +01:00
2021-11-11 13:54:14 +00:00

GOV.UK Notify API

Contains:

  • the public-facing REST API for GOV.UK Notify, which teams can integrate with using our clients
  • an internal-only REST API built using Flask to manage services, users, templates, etc (this is what the admin app talks to)
  • asynchronous workers built using Celery to put things on queues and read them off to be processed, sent to providers, updated, etc

Setting Up

Python version

We run python 3.9 both locally and in production.

pycurl

See https://github.com/alphagov/notifications-manuals/wiki/Getting-started#pycurl

AWS credentials

To run the API you will need appropriate AWS credentials. See the Wiki for more details.

environment.sh

Creating and edit an environment.sh file.

echo "
export NOTIFY_ENVIRONMENT='development'

export MMG_API_KEY='MMG_API_KEY'
export FIRETEXT_API_KEY='FIRETEXT_ACTUAL_KEY'
export NOTIFICATION_QUEUE_PREFIX='YOUR_OWN_PREFIX'

export FLASK_APP=application.py
export FLASK_ENV=development
export WERKZEUG_DEBUG_PIN=off
"> environment.sh

Things to change:

  • Replace YOUR_OWN_PREFIX with local_dev_<first name>.
  • Run the following in the credentials repo to get the API keys.
notify-pass credentials/providers/api_keys

Postgres

Install Postgres.app.

Currently the API works with PostgreSQL 11. After installation, open the Postgres app, open the sidebar, and update or replace the default server with a compatible version.

Note: you may need to add the following directory to your PATH in order to bootstrap the app.

export PATH=${PATH}:/Applications/Postgres.app/Contents/Versions/11/bin/

Redis

To switch redis on you'll need to install it locally. On a OSX we've used brew for this. To use redis caching you need to switch it on by changing the config for development:

    REDIS_ENABLED = True

To run the application

# install dependencies, etc.
make bootstrap

# run the web app
make run-flask

# run the background tasks
make run-celery

# run scheduled tasks (optional)
make run-celery-beat

To test the application

# install dependencies, etc.
make bootstrap

make test

To run one off tasks

Tasks are run through the flask command - run flask --help for more information. There are two sections we need to care about: flask db contains alembic migration commands, and flask command contains all of our custom commands. For example, to purge all dynamically generated functional test data, do the following:

Locally

flask command purge_functional_test_data -u <functional tests user name prefix>

On the server

cf run-task notify-api "flask command purge_functional_test_data -u <functional tests user name prefix>"

All commands and command options have a --help command if you need more information.

Further documentation

Description
The API powering Notify.gov
Readme 58 MiB
Languages
Python 98.5%
HCL 0.6%
Jinja 0.5%
Shell 0.3%
Makefile 0.1%