notifications-api/README.md

US Notify API

This project is the core of Notify. It's cloned from the brilliant work of the team at GOV.UK Notify, cheers!

This repo contains:

• A public-facing REST API for Notify, which teams can integrate with using API clients built by UK
• An internal-only REST API built using Flask to manage services, users, templates, etc., which the admin UI talks to)
• Asynchronous workers built using Celery to put things on queues and read them off to be processed, sent to providers, updated, etc.

Our other repositories are:

• notifications-admin
• notifications-utils
• us-notify-compliance
• notify-python-demo

Local setup

Common steps

On MacOS, using Homebrew for package management is highly recommended. This helps avoid some known installation issues.

Note: If brew is not found: Verify that Homebrew is in your system's PATH. You may need to follow additional instructions to add Homebrew to your PATH in /Users/homefolder/.zprofile:

1. Install pre-requisites for setup:

   • jq: brew install jq
   • terraform: brew install terraform or brew install tfenv and use tfenv to install terraform ~> 1.4.0
   • cf-cli@8: brew install cloudfoundry/tap/cf-cli@8
   • postgresql: brew install postgresql@15 (Homebrew requires a version pin, but any recent version will work)
   • redis: brew install redis
   • pyenv: brew install pyenv
   • poetry: brew install poetry

2. Log into cloud.gov: cf login -a api.fr.cloud.gov --sso

3. Ensure you have access to the notify-local-dev and notify-staging spaces in cloud.gov

4. Run the development terraform (Be sure to clone the repository first and navigate to the terraform directory):

       ```
       $ cd terraform/development
       $ ./run.sh
       ```
   

5. If you want to send data to New Relic from your local develpment environment, set NEW_RELIC_LICENSE_KEY within .env

6. Start Postgres && Redis

    ```
    brew services start postgresql@15
    brew services start redis
    ```
   

Install

1. Run the project setup (If there are errors, check for Python versions >=3.9,<3.12. Ensure that you have the required database set up and migrations)

   make bootstrap

2. Run the web server and background workers

   make run-procfile

• Or run them individually:

• Run Flask (web server)

  make run-flask

• Run Celery (background worker)

  make run-celery

Python dependency management

We're using Poetry for managing our Python dependencies and local virtual environments. When it comes to managing the Python dependencies, there are a couple of things to bear in mind.

For situations where you manually manipulate the pyproject.toml file, you should use the make py-lock command to sync the poetry.lock file. This will ensure that you don't inadvertently bring in other transitive dependency updates that have not been fully tested with the project yet.

If you're just trying to update a dependency to a newer (or the latest) version, you should let Poetry take care of that for you by running the following:

poetry update <dependency> [<dependency>...]

You can specify more than one dependency together. With this command, Poetry will do the following for you:

• Find the latest compatible version(s) of the specified dependency/dependencies
• Install the new versions
• Update and sync the poetry.lock file

In either situation, once you are finished and have verified the dependency changes are working, please be sure to commit both the pyproject.toml and poetry.lock files.

Keeping the notification-utils dependency up-to-date

The notifications-utils dependency references the other repository we have at https://github.com/GSA/notifications-utils - this dependency requires a bit of extra legwork to ensure it stays up-to-date.

Whenever a PR is merged in the notifications-utils repository, we need to make sure the changes are pulled in here and committed to this repository as well. You can do this by going through these steps:

• Make sure your local main branch is up-to-date
• Create a new branch to work in
• Run make update-utils
• Commit the updated poetry.lock file and push the changes
• Make a new PR with the change
• Have the PR get reviewed and merged

Known installation issues

On M1 Macs, if you get a fatal error: 'Python.h' file not found message, try a different method of installing Python. Installation via pyenv is known to work.

A direct installation of PostgreSQL will not put the createdb command on your $PATH. It can be added there in your shell startup script, or a Homebrew-managed installation of PostgreSQL will take care of it.

Documentation

• Infrastructure overview
  • GitHub Repositories
  • Terraform
  • AWS
  • New Relic
  • Onboarding
  • Setting up the infrastructure
• Using the logs
• Testing
  • CI testing
  • Manual testing
  • To run a local OWASP scan
• Deploying
  • Egress Proxy
  • Managing environment variables
  • Sandbox environment
• Database management
  • Initial state
  • Data Model Diagram
  • Migrations
  • Purging user data
• One-off tasks
• How messages are queued and sent
• Writing public APIs
  • Overview
  • Documenting APIs
  • New APIs
• API Usage
  • Connecting to the API
  • Postman Documentation
  • Using OpenAPI documentation
• Queues and tasks
  • Priority queue
  • Celery scheduled tasks
• US Notify
  • System Description
• Run Book
  • Alerts, Notifications, Monitoring
  • Restaging Apps
  • Smoke-testing the App
  • Configuration Management
  • DNS Changes
  • Exporting test results for compliance monitoring
  • Known Gotchas
  • User Account Management
  • SMS Phone Number Management
• Data Storage Policies & Procedures
  • Potential PII Locations
  • Data Retention Policy

License && public domain

Work through commit e604385 is licensed by the UK government under the MIT license. Work after that commit is in the worldwide public domain. See LICENSE.md for more information.

Contributing

As stated in CONTRIBUTING.md, all contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.