darkhelm/notifications-api
1
0
Fork 0
mirror of https://github.com/GSA/notifications-api.git synced 2026-02-04 10:21:14 -05:00
Code Issues Packages Projects Releases Wiki Activity

change to actually downloading file

Browse Source
This commit is contained in:
Kenneth Kehl
2024-08-01 08:05:12 -07:00
parent 86c9065a44 b3e56cb28b
commit 0ab937346d
28 changed files with 461 additions and 509 deletions
Download Patch File Download Diff File Expand all files Collapse all files

81
docs/all.md
View File

@@ -60,9 +60,13 @@
- [Data Storage Policies \& Procedures](#data-storage-policies--procedures)
- [Potential PII Locations](#potential-pii-locations)
- [Data Retention Policy](#data-retention-policy)
- [Debug messages not being sent](#debug-messages-not-being-sent)
- [Getting the file location and tracing what happens](#getting-the-file-location-and-tracing-what-happens)
- [Viewing the csv file](#viewing-the-csv-file)
- [Troubleshooting](#troubleshooting)
- [Debug messages not being sent](#debug-messages-not-being-sent)
- [Getting the file location and tracing what happens](#getting-the-file-location-and-tracing-what-happens)
- [Viewing the csv file](#viewing-the-csv-file)
- [Deployment / app push problems](#deployment--app-push-problems)
- [Routes cannot be mapped to destinations in different spaces](#routes-cannot-be-mapped-to-destinations-in-different-spaces)
- [API request failed](#api-request-failed)
# Infrastructure overview
@@ -432,21 +436,35 @@ Rules for use:
### Deploying to the sandbox
If this is the first time you have used Terraform in this repository, you will first have to hook your copy of Terraform up to our remote state. Follow [Retrieving existing bucket credentials](https://github.com/GSA/notifications-api/tree/main/terraform#retrieving-existing-bucket-credentials).
:anchor: The Admin app depends upon the API app, so set up the API first.
1. Set up services:
```
```bash
$ cd terraform/sandbox
$ ../create_service_account.sh -s notify-sandbox -u <your-name>-terraform -m > secrets.auto.tfvars
$ terraform init
$ terraform plan
$ terraform apply
```
1. start a poetry shell as a shortcut to load `.env` file variables: `$ poetry shell`
Check [Terraform troubleshooting](https://github.com/GSA/notifications-api/tree/main/terraform#troubleshooting) if you encounter problems.
1. Change back to the project root directory: `cd ../..`
1. Start a poetry shell as a shortcut to load `.env` file variables by running `poetry shell`. (You'll have to restart this any time you change the file.)
1. Output requirements.txt file: `poetry export --without-hashes --format=requirements.txt > requirements.txt`
1. Deploy the application:
1. Ensure you are using the correct CloudFoundry target
```bash
cf target -o gsa-tts-benefits-studio -s notify-sandbox
```
1. Deploy the application:
```bash
cf push --vars-file deploy-config/sandbox.yml --var NEW_RELIC_LICENSE_KEY=$NEW_RELIC_LICENSE_KEY
```
The real `push` command has more var arguments than the single one above. Get their values from a Notify team member.
1. Visit the URL(s) of the app you just deployed
* Admin https://notify-sandbox.app.cloud.gov/
* API https://notify-api-sandbox.app.cloud.gov/
# Database management
@@ -1324,11 +1342,12 @@ Seven (7) days by default. Each service can be set with a custom policy via `Ser
Data cleanup is controlled by several tasks in the `nightly_tasks.py` file, kicked off by Celery Beat.
# Troubleshooting
# Debug messages not being sent
## Debug messages not being sent
## Getting the file location and tracing what happens
### Getting the file location and tracing what happens
Ask the user to provide the csv file name. Either the csv file they uploaded, or the one that is autogenerated when they do a one-off send and is visible in the UI
@@ -1337,7 +1356,7 @@ Starting with the admin logs, search for this file name. When you find it, the
In the api logs, search by job_id. Either you will see evidence of the job failing and retrying over and over (in which case search for a stack trace using timestamp), or you will ultimately get to a log line that links the job_id to a message_id. In this case, now search by message_id. You should be able to find the actual result from AWS, either success or failure, with hopefully some helpful info.
## Viewing the csv file
### Viewing the csv file
If you need to view the questionable csv file on production, run the following command:
@@ -1352,7 +1371,7 @@ locally, just do:
poetry run flask command download-csv-file-by-name <file location in admin logs>
```
## Debug steps
### Debug steps
1. Either send a message and capture the csv file name, or get a csv file name from a user
2. Using the log tool at logs.fr.cloud.gov, use filters to limit what you're searching on (cf.app is 'notify-admin-production' for example) and then search with the csv file name in double quotes over the relevant time period (last 5 minutes if you just sent a message, or else whatever time the user sent at)
@@ -1360,3 +1379,45 @@ poetry run flask command download-csv-file-by-name <file location in admin logs>
4. To get the csv file contents, you can run the command above. This command currently prints to the notify-api log, so after you run the command,
you need to search in notify-api-production for the last 5 minutes with the logs sorted by timestamp. The contents of the csv file unfortunately appear on separate lines so it's very important to sort by time.
5. If you want to see where the message actually failed, search with cf.app is notify-api-production using the job_id that you saved in step #3. If you get far enough, you might see one of the log lines has a message_id. If you see it, you can switch and search on that, which should tell you what happened in AWS (success or failure).
## Deployment / app push problems
### Routes cannot be mapped to destinations in different spaces
During `cf push` you may see
```
For application 'notify-api-sandbox': Routes cannot be mapped to destinations in different spaces
```
:ghost: This indicates a ghost route squatting on a route you need to create. In the cloud.gov web interface, check for incomplete deployments. They might be holding on to a route. Delete them. Also, check the list of routes (from the CloudFoundry icon in the left sidebar) for routes without an associated app. If they look like a route your app would need to create, delete them.
### API request failed
After pushing the Admin app, you might see this in the logs
```
{"name": "app", "levelname": "ERROR", "message": "API unknown failed with status 503 message Request failed", "pathname": "/home/vcap/app/app/__init__.py", ...
```
And you would also see this in the Admin web UI
```
Sorry, we can't deliver what you asked for right now.
```
This indicates that the Admin and API apps are unable to talk to each other because of either a missing route or a missing network policy. The apps require [container-to-container networking](https://cloud.gov/docs/management/container-to-container/) to communicate. List `cf network-policies`; you should see one connecting API and Admin on port 61443. If not, you can create one manually:
```bash
cf add-network-policy notify-admin-sandbox notify-api-sandbox --protocol tcp --port 61443
```
### Service instance not found
This error encounted after `cf push` indicates you may be using the wrong CloudFoundry target
```
For application 'notify-api-sandbox': Service instance 'notify-api-rds-sandbox' not found
```
Run `cf target -o gsa-tts-benefits-studio -s notify-sandbox` before pushing to the Sandbox