darkhelm/notifications-api
1
0
Fork 0
mirror of https://github.com/GSA/notifications-api.git synced 2026-01-22 10:31:55 -05:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #481 from GSA/stvnrlly/docs-91823

Browse Source 
Doc updates for logs and env vars
This commit is contained in:
Steven Reilly
2023-09-22 22:12:13 -04:00
committed by GitHub
parent bc39606e3c 6c6762b2ce
commit 9eb7c92461
2 changed files with 92 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files

96
README.md
View File

@@ -100,53 +100,55 @@ A direct installation of PostgreSQL will not put the `createdb` command on your
## Documentation
- [Infrastructure overview](#infrastructure-overview)
- [GitHub Repositories](#github-repositories)
- [Terraform](#terraform)
- [AWS](#aws)
- [New Relic](#new-relic)
- [Onboarding](#onboarding)
- [Setting up the infrastructure](#setting-up-the-infrastructure)
- [Testing](#testing)
- [CI testing](#ci-testing)
- [Manual testing](#manual-testing)
- [To run a local OWASP scan](#to-run-a-local-owasp-scan)
- [Deploying](#deploying)
- [Egress Proxy](#egress-proxy)
- [Sandbox environment](#sandbox-environment)
- [Database management](#database-management)
- [Initial state](#initial-state)
- [Data Model Diagram](#data-model-diagram)
- [Migrations](#migrations)
- [Purging user data](#purging-user-data)
- [One-off tasks](#one-off-tasks)
- [How messages are queued and sent](#how-messages-are-queued-and-sent)
- [Writing public APIs](#writing-public-apis)
- [Overview](#overview)
- [Documenting APIs](#documenting-apis)
- [New APIs](#new-apis)
- [API Usage](#api-usage)
- [Connecting to the API](#connecting-to-the-api)
- [Postman Documentation](#postman-documentation)
- [Using OpenAPI documentation](#using-openapi-documentation)
- [Queues and tasks](#queues-and-tasks)
- [Priority queue](#priority-queue)
- [Celery scheduled tasks](#celery-scheduled-tasks)
- [US Notify](#us-notify)
- [System Description](#system-description)
- [Run Book](#run-book)
- [ Alerts, Notifications, Monitoring](#-alerts-notifications-monitoring)
- [ Restaging Apps](#-restaging-apps)
- [ Smoke-testing the App](#-smoke-testing-the-app)
- [ Configuration Management](#-configuration-management)
- [ DNS Changes](#-dns-changes)
- [Exporting test results for compliance monitoring](#exporting-test-results-for-compliance-monitoring)
- [ Known Gotchas](#-known-gotchas)
- [ User Account Management](#-user-account-management)
- [ SMS Phone Number Management](#-sms-phone-number-management)
- [Data Storage Policies \& Procedures](#data-storage-policies--procedures)
- [Potential PII Locations](#potential-pii-locations)
- [Data Retention Policy](#data-retention-policy)
- [Infrastructure overview](./docs/all.md#infrastructure-overview)
- [GitHub Repositories](./docs/all.md#github-repositories)
- [Terraform](./docs/all.md#terraform)
- [AWS](./docs/all.md#aws)
- [New Relic](./docs/all.md#new-relic)
- [Onboarding](./docs/all.md#onboarding)
- [Setting up the infrastructure](./docs/all.md#setting-up-the-infrastructure)
- [Using the logs](./docs/all.md#using-the-logs)
- [Testing](./docs/all.md#testing)
- [CI testing](./docs/all.md#ci-testing)
- [Manual testing](./docs/all.md#manual-testing)
- [To run a local OWASP scan](./docs/all.md#to-run-a-local-owasp-scan)
- [Deploying](./docs/all.md#deploying)
- [Egress Proxy](./docs/all.md#egress-proxy)
- [Managing environment variables](./docs/all.md#managing-environment-variables)
- [Sandbox environment](./docs/all.md#sandbox-environment)
- [Database management](./docs/all.md#database-management)
- [Initial state](./docs/all.md#initial-state)
- [Data Model Diagram](./docs/all.md#data-model-diagram)
- [Migrations](./docs/all.md#migrations)
- [Purging user data](./docs/all.md#purging-user-data)
- [One-off tasks](./docs/all.md#one-off-tasks)
- [How messages are queued and sent](./docs/all.md#how-messages-are-queued-and-sent)
- [Writing public APIs](./docs/all.md#writing-public-apis)
- [Overview](./docs/all.md#overview)
- [Documenting APIs](./docs/all.md#documenting-apis)
- [New APIs](./docs/all.md#new-apis)
- [API Usage](./docs/all.md#api-usage)
- [Connecting to the API](./docs/all.md#connecting-to-the-api)
- [Postman Documentation](./docs/all.md#postman-documentation)
- [Using OpenAPI documentation](./docs/all.md#using-openapi-documentation)
- [Queues and tasks](./docs/all.md#queues-and-tasks)
- [Priority queue](./docs/all.md#priority-queue)
- [Celery scheduled tasks](./docs/all.md#celery-scheduled-tasks)
- [US Notify](./docs/all.md#us-notify)
- [System Description](./docs/all.md#system-description)
- [Run Book](./docs/all.md#run-book)
- [ Alerts, Notifications, Monitoring](./docs/all.md#-alerts-notifications-monitoring)
- [ Restaging Apps](./docs/all.md#-restaging-apps)
- [ Smoke-testing the App](./docs/all.md#-smoke-testing-the-app)
- [ Configuration Management](./docs/all.md#-configuration-management)
- [ DNS Changes](./docs/all.md#-dns-changes)
- [Exporting test results for compliance monitoring](./docs/all.md#exporting-test-results-for-compliance-monitoring)
- [ Known Gotchas](./docs/all.md#-known-gotchas)
- [ User Account Management](./docs/all.md#-user-account-management)
- [ SMS Phone Number Management](./docs/all.md#-sms-phone-number-management)
- [Data Storage Policies \& Procedures](./docs/all.md#data-storage-policies--procedures)
- [Potential PII Locations](./docs/all.md#potential-pii-locations)
- [Data Retention Policy](./docs/all.md#data-retention-policy)
## License && public domain

46
docs/all.md
View File

@@ -5,12 +5,14 @@
- [New Relic](#new-relic)
- [Onboarding](#onboarding)
- [Setting up the infrastructure](#setting-up-the-infrastructure)
- [Using the logs](#using-the-logs)
- [Testing](#testing)
- [CI testing](#ci-testing)
- [Manual testing](#manual-testing)
- [To run a local OWASP scan](#to-run-a-local-owasp-scan)
- [Deploying](#deploying)
- [Egress Proxy](#egress-proxy)
- [Managing environment variables](#managing-environment-variables)
- [Sandbox environment](#sandbox-environment)
- [Database management](#database-management)
- [Initial state](#initial-state)
@@ -85,13 +87,17 @@ In addition to terraform directories in the api and admin apps above:
## Terraform
We use Terraform to manage our infrastructure, providing consistent setups across the environments.
Our Terraform configurations manage components via cloud.gov. This means that the configurations should work out of the box if you are using a Cloud Foundry platform, but will not work for setups based on raw AWS.
### Development
There are several remote services required for local development:
* s3
* ses
* sns
* S3
* SES
* SNS
Credentials for these services are created by running:
@@ -205,6 +211,20 @@ Example answers for toll-free registration form
![example answers for toll-free registration form](./toll-free-registration.png)
# Using the logs
If you're using the `cf` CLI, you can run `cf logs notify-api-ENV` and/or `cf logs notify-admin-ENV` to stream logs in real time. Add `--recent` to get the last few logs, though logs often move pretty quickly.
For general log searching, [the cloud.gov Kibana instance](https://logs.fr.cloud.gov/) is powerful, though quite complex to get started. For shortcuts to errors, some team members have New Relic access.
The links below will open a filtered view with logs from both applications, which can then be filtered further. However, for the links to work, you need to paste them into the URL bar while *already* logged into and viewing the Kibana page. If not, you'll just be redirected to the generic dashboard.
Production: https://logs.fr.cloud.gov/app/discover#/view/218a6790-596d-11ee-a43a-090d426b9a38
Demo: https://logs.fr.cloud.gov/app/discover#/view/891392a0-596e-11ee-921a-1b6b2f4d89ed
Staging: https://logs.fr.cloud.gov/app/discover#/view/73d7c820-596e-11ee-a43a-090d426b9a38
Once in the view, you'll likely want to adjust the time range in the upper right of the page.
# Testing
```
@@ -304,6 +324,26 @@ application to a select list of allowed domains.
Update the allowed domains by updating `deploy-config/egress_proxy/notify-api-<env>.allow.acl`
and deploying an updated version of the application throught he normal deploy process.
## Managing environment variables
For an environment variable to make its way into the cloud.gov environment, it *must* end up in the `manifest.yml` file. Based on the deployment approach described above, there are 2 ways for this to happen.
### Secret environment variables
Because secrets are pulled from GitHub, they must be passed from our action to the deploy action and then placed into `manifest.yml`. This means that they should be in a 4 places:
- [ ] The GitHub secrets store
- [ ] The deploy action in the `env` section using the format `{secrets.SECRET_NAME}`
- [ ] The deploy action in the `push_arguments` section using the format `--var SECRET_NAME="$SECRET_NAME"`
- [ ] The manifest using the format `SECRET_NAME: ((SECRET_NAME))`
### Public environment variables
Public env vars make up the configuration in `deploy-config`. These are pulled in together by the `--vars-file` line in the deploy action. To add or update one, it should be in 2 places:
- [ ] The relevant YAML file in `deploy-config` using the format `var_name: value`
- [ ] The manifest using the format `((var_name))`
## Sandbox environment
There is a sandbox space, complete with terraform and `deploy-config/sandbox.yml` file available