2018-10-30 13:30:46 +00:00
# GOV.UK Notify API
2015-11-20 10:51:08 +00:00
2018-10-30 13:30:46 +00:00
Contains:
- the public-facing REST API for GOV.UK Notify, which teams can integrate with using [our clients ](https://www.notifications.service.gov.uk/documentation )
- an internal-only REST API built using Flask to manage services, users, templates, etc (this is what the [admin app ](http://github.com/alphagov/notifications-admin ) talks to)
- asynchronous workers built using Celery to put things on queues and read them off to be processed, sent to providers, updated, etc
2015-11-20 10:51:08 +00:00
2016-10-04 15:09:58 +01:00
## Setting Up
2018-10-30 13:33:56 +00:00
### Python version
2020-01-03 15:36:54 +00:00
At the moment we run Python 3.6 in production. You will run into problems if you try to use Python 3.5 or older, or Python 3.7 or newer.
2018-10-30 13:33:56 +00:00
2016-10-04 15:09:58 +01:00
### AWS credentials
2016-10-04 11:12:55 +01:00
2021-02-17 16:56:20 +00:00
To run the API you will need appropriate AWS credentials. See the [Wiki ](https://github.com/alphagov/notifications-manuals/wiki/aws-accounts#how-to-set-up-local-development ) for more details.
2016-10-04 15:09:58 +01:00
2018-07-30 15:25:48 +01:00
### `environment.sh`
2016-10-04 15:09:58 +01:00
2021-02-17 17:04:28 +00:00
Creating and edit an environment.sh file.
2016-02-17 10:20:40 +00:00
2016-02-16 15:25:46 +00:00
```
2016-02-17 09:49:36 +00:00
echo "
2016-10-04 15:09:58 +01:00
export NOTIFY_ENVIRONMENT='development'
2018-02-21 18:12:03 +00:00
2016-09-07 09:35:31 +01:00
export MMG_API_KEY='MMG_API_KEY'
2016-10-04 15:09:58 +01:00
export FIRETEXT_API_KEY='FIRETEXT_ACTUAL_KEY'
export NOTIFICATION_QUEUE_PREFIX='YOUR_OWN_PREFIX'
2018-02-21 18:12:03 +00:00
2017-11-06 12:34:29 +00:00
export FLASK_APP=application.py
2021-02-18 09:10:13 +00:00
export FLASK_ENV=development
2017-11-06 12:34:29 +00:00
export WERKZEUG_DEBUG_PIN=off
2016-02-17 09:49:36 +00:00
"> environment.sh
```
2021-02-17 17:04:28 +00:00
Things to change:
* Replace `YOUR_OWN_PREFIX` with `local_dev_<first name>` .
* Run the following in the credentials repo to get the API keys.
2016-02-17 10:20:40 +00:00
2021-02-17 17:04:28 +00:00
```
notify-pass credentials/providers/api_keys
```
2016-02-23 12:28:10 +00:00
2016-10-04 15:09:58 +01:00
### Postgres
2016-10-04 11:12:55 +01:00
2021-02-18 15:19:40 +00:00
Install [Postgres.app ](http://postgresapp.com/ ).
2016-02-23 12:28:10 +00:00
2021-02-18 15:19:40 +00:00
Currently the API works with PostgreSQL 11. After installation, open the Postgres app, open the sidebar, and update or replace the default server with a compatible version.
**Note:** you may need to add the following directory to your PATH in order to bootstrap the app.
```
export PATH=${PATH}:/Applications/Postgres.app/Contents/Versions/11/bin/
```
2018-11-07 16:33:22 +00:00
2016-12-02 10:09:36 +00:00
### Redis
2016-12-02 10:11:31 +00:00
To switch redis on you'll need to install it locally. On a OSX we've used brew for this. To use redis caching you need to switch it on by changing the config for development:
REDIS_ENABLED = True
2016-12-02 10:09:36 +00:00
2016-02-23 12:28:10 +00:00
## To run the application
```
2021-02-17 17:06:58 +00:00
# install dependencies, etc.
2021-02-17 17:22:45 +00:00
make bootstrap
2021-02-17 17:06:58 +00:00
# run the web app
2021-02-17 16:49:05 +00:00
make run-flask
2016-02-23 12:28:10 +00:00
2021-02-17 17:06:58 +00:00
# run the background tasks
2021-02-17 16:49:05 +00:00
make run-celery
2016-02-23 12:28:10 +00:00
2021-02-17 17:06:58 +00:00
# run scheduled tasks (optional)
2021-02-17 16:49:05 +00:00
make run-celery-beat
2016-03-18 12:58:17 +00:00
```
2016-05-11 15:52:49 +01:00
2016-08-24 14:30:46 +01:00
## To test the application
```
2021-02-17 17:06:58 +00:00
# install dependencies, etc.
2021-02-17 17:22:45 +00:00
make bootstrap
2021-02-17 17:06:58 +00:00
2016-08-24 14:30:46 +01:00
make test
```
Pin all application requirements in requirements.txt
The list of top-level dependencies is moved to requirements-app.txt,
which is used by `make freeze-requirements` to generate the full
list of requirements in requirements.txt.
This is based on alphagov/digitalmarketplace-api#615, so rationale
from that PR applies here.
We had a problem with unpinned packages on new deployments leading
to failed tests (e.g. alphagov/notifications-admin#2144) which is
why we're implementing this now.
After re-evaluating pipenv again, this still seems like the least
disruptive approach:
* pyup.io has experimental support for Pipfile, but doesn't respect
version ranges or updating hashes in the lock file
* CloudFoundry buildpack recognizes and supports Pipfiles out of the
box, but the support is relatively new. For example until recently
CF would install dev packages during deployment. It's also based on
generating a requirements file from the Pipfile, which doesn't
properly support pinning VCS dependencies (eg it doesn't set the
#egg= version, meaning pip will not upgrade the package if it's
already installed).
* pipenv has a strict dependency resolution algorithm, which doesn't
appear to be well documented and can cause some unexpected failures.
For example, pipenv doesn't seem to be able to install `awscli-cwlogs`
package at all, believing it to have a version conflict for `botocore`
(which it doesn't list as a direct dependency) while neither `pip` nor
`pip-tools` highlight any issues with it.
* While trying out `pipenv install` on our list of dependencies it would
regularly fail to install utils with a "Will try again." message.
While the installation succeeds after a retry, this doesn't inspire
confidence.
* The switch to Pipfile and pipenv-managed virtualenvs requires a series
of changes to `make` targets and scripts - replacing `pip install` with
`pipenv`, removing references to requirements files and prefixing
commands with `pipenv run`. While it's likely to simplify the overall
process of managing dependencies, it would require time to properly
implement across our applications and environments (Jenkins, PaaS,
docker containers, and dev machines).
2018-07-10 14:50:30 +01:00
## To update application dependencies
`requirements.txt` file is generated from the `requirements-app.txt` in order to pin
versions of all nested dependencies. If `requirements-app.txt` has been changed (or
we want to update the unpinned nested dependencies) `requirements.txt` should be
regenerated with
```
make freeze-requirements
```
`requirements.txt` should be committed alongside `requirements-app.txt` changes.
2016-08-24 14:30:46 +01:00
2017-11-22 16:36:09 +00:00
## To run one off tasks
2016-05-11 15:52:49 +01:00
2017-11-22 16:36:09 +00:00
Tasks are run through the `flask` command - run `flask --help` for more information. There are two sections we need to
care about: `flask db` contains alembic migration commands, and `flask command` contains all of our custom commands. For
example, to purge all dynamically generated functional test data, do the following:
2016-05-11 15:52:49 +01:00
Locally
```
2017-11-22 16:36:09 +00:00
flask command purge_functional_test_data -u <functional tests user name prefix>
2016-05-11 15:52:49 +01:00
```
On the server
```
2017-11-22 16:36:09 +00:00
cf run-task notify-api "flask command purge_functional_test_data -u <functional tests user name prefix>"
2016-05-11 15:52:49 +01:00
```
2017-11-22 16:36:09 +00:00
All commands and command options have a --help command if you need more information.
2018-03-08 14:02:08 +00:00
2021-09-15 15:33:42 +01:00
## Further documentation
2018-03-08 14:02:08 +00:00
2021-09-15 17:09:04 +01:00
- [Writing public APIs ](docs/writing-public-apis.md )