darkhelm/notifications-api
1
0
Fork 0
mirror of https://github.com/GSA/notifications-api.git synced 2026-02-04 10:21:14 -05:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #519 from GSA/stvnrlly/setup-streamline

Browse Source 
Streamline setup instructions
This commit is contained in:
Carlo Costino
2023-09-29 11:02:57 -04:00
committed by GitHub
parent 6ae3417ac5 cd37c242ba
commit 66494f9540
9 changed files with 9 additions and 368 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
README.md
View File

@@ -29,6 +29,7 @@ On MacOS, using [Homebrew](https://brew.sh/) for package management is highly re
* [postgresql](https://www.postgresql.org/): `brew install postgresql@15` (Homebrew requires a version pin, but any recent version will work) * [postgresql](https://www.postgresql.org/): `brew install postgresql@15` (Homebrew requires a version pin, but any recent version will work)
* [redis](https://redis.io/): `brew install redis` * [redis](https://redis.io/): `brew install redis`
* [pyenv](https://github.com/pyenv/pyenv): `brew install pyenv` * [pyenv](https://github.com/pyenv/pyenv): `brew install pyenv`
* [poetry](https://python-poetry.org/docs/#installation): `brew install poetry`
1. [Log into cloud.gov](https://cloud.gov/docs/getting-started/setup/#set-up-the-command-line): `cf login -a api.fr.cloud.gov --sso` 1. [Log into cloud.gov](https://cloud.gov/docs/getting-started/setup/#set-up-the-command-line): `cf login -a api.fr.cloud.gov --sso`
1. Ensure you have access to the `notify-local-dev` and `notify-staging` spaces in cloud.gov 1. Ensure you have access to the `notify-local-dev` and `notify-staging` spaces in cloud.gov
1. Run the development terraform with: 1. Run the development terraform with:
@@ -39,19 +40,19 @@ On MacOS, using [Homebrew](https://brew.sh/) for package management is highly re
``` ```
1. If you want to send data to New Relic from your local develpment environment, set `NEW_RELIC_LICENSE_KEY` within `.env` 1. If you want to send data to New Relic from your local develpment environment, set `NEW_RELIC_LICENSE_KEY` within `.env`
1. Follow the instructions for either `Direct installation` or `Docker installation` below 1. Start Postgres && Redis
### Direct installation ```
brew services start postgresql@15
1. Set up Postgres && Redis on your machine brew services start redis
```
1. Install [poetry](https://python-poetry.org/docs/#installation)
1. Install
1. Run the project setup 1. Run the project setup
`make bootstrap` `make bootstrap`
1. Run the web server and background worker 1. Run the web server and background workers
`make run-procfile` `make run-procfile`
@@ -65,33 +66,6 @@ On MacOS, using [Homebrew](https://brew.sh/) for package management is highly re
`make run-celery` `make run-celery`
### VS Code && Docker installation
If you're working in VS Code, you can also leverage Docker for a containerized dev environment
1. Uncomment the `Local Docker setup` lines in `.env` and comment out the `Local direct setup` lines.
1. Install the Remote-Containers plug-in in VS Code
1. With Docker running, create the network:
`docker network create notify-network`
1. Using the command palette (shift+cmd+p) or green button thingy in the bottom left, search and select “Remote Containers: Open Folder in Container...” When prompted, choose **devcontainer-api** folder (note: this is a *subfolder* of notifications-api). This will start the container in a new window, replacing the current one.
1. Wait a few minutes while things happen 🍵
1. Open a VS Code terminal and run the Flask application:
`make run-flask`
1. Open another VS Code terminal and run Celery:
`make run-celery`
NOTE: when you change .env in the future, you'll need to rebuild the devcontainer for the change to take effect. VS Code _should_ detect the change and prompt you with a toast notification during a cached build. If not, you can find a manual rebuild in command pallette or just `docker rm` the notifications-api container.
### Known installation issues ### Known installation issues
On M1 Macs, if you get a `fatal error: 'Python.h' file not found` message, try a different method of installing Python. Installation via `pyenv` is known to work. On M1 Macs, if you get a `fatal error: 'Python.h' file not found` message, try a different method of installing Python. Installation via `pyenv` is known to work.
@@ -156,4 +130,4 @@ Work through [commit `e604385`](https://github.com/GSA/notifications-api/commit/
## Contributing ## Contributing
As stated in [CONTRIBUTING.md](CONTRIBUTING.md), all contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest. As stated in [CONTRIBUTING.md](CONTRIBUTING.md), all contributions to this project will be released under the CC0 dedication. By submitting a pull request, you are agreeing to comply with this waiver of copyright interest.

49
devcontainer-api/.devcontainer.json
View File

@@ -1,49 +0,0 @@
{
"name": "notification-api",
"dockerComposeFile": "../docker-compose.devcontainer.yml",
"service": "dev",
"workspaceFolder": "/workspace",
"shutdownAction": "stopCompose",
"remoteEnv": {
"PATH": "/home/vscode/.local/bin:${containerEnv:PATH}" // give our installed Python modules precedence
},
"settings": {
"[python]": {
"editor.formatOnSave": true
},
"python.linting.enabled": true,
"python.linting.pylintEnabled": true,
"python.defaultInterpreterPath": "/usr/bin/python3",
"python.linting.pylintPath": "/usr/local/share/pip-global/bin/pylint",
"python.analysis.extraPaths": [
"/home/vscode/.local/lib/python3.9/site-packages",
"/home/vscode/.local/bin"
]
},
"features": {
"docker-from-docker": {
"version": "latest",
"moby": true
}
},
"extensions": [
// "ms-python.python",
"donjayamanne.python-extension-pack",
// "ms-azuretools.vscode-docker",
"ms-python.vscode-pylance",
"eamodio.gitlens",
"wholroyd.jinja",
"pmbenjamin.vscode-snyk",
"visualstudioexptteam.vscodeintellicode",
"yzhang.markdown-all-in-one",
"ms-ossdata.vscode-postgresql",
// "ms-vsliveshare.vsliveshare",
// "mtxr.sqltools",
// "mtxr.sqltools-driver-pg"
],
"forwardPorts": [
6011
],
"postCreateCommand": "notify-dev-entrypoint.sh",
"remoteUser": "vscode"
}

40
devcontainer-api/Dockerfile
View File

@@ -1,40 +0,0 @@
FROM mcr.microsoft.com/vscode/devcontainers/python:0-3.9
RUN apt-get update \
&& apt-get -y install --no-install-recommends \
apt-utils \
postgresql-client \
2>&1 \
&& apt-get -y install \
curl \
emacs \
exa \
fd-find \
git \
iproute2 \
less \
libsodium-dev \
lsb-release \
man-db \
manpages \
net-tools \
openssh-client \
procps \
sudo \
tldr \
unzip \
vim \
&& apt-get autoremove -y \
&& apt-get clean -y \
&& rm -rf /var/lib/apt/lists/*
# Upgrade pip
RUN pip install --upgrade pip
COPY devcontainer-api/scripts/notify-dev-entrypoint.sh /usr/local/bin/
COPY devcontainer-api/scripts/notify-worker-entrypoint.sh /usr/local/bin/
ENV SHELL /bin/zsh
EXPOSE 8000
EXPOSE 6011

18
devcontainer-api/initdb/notify-db-entrypoint.sh
View File

@@ -1,18 +0,0 @@
#!/usr/bin/env bash
set -ex
###################################################################
# This script will get executed *once* the Docker container has
# been built. Commands that need to be executed with all available
# tools and the filesystem mount enabled should be located here.
#
# The PostgreSQL Docker image has an extension mechanism that does
# not necessitate to override the entrypoint or main command. One
# simply has to copy a shell script into the
# /docker-entrypoint-initdb.d/ initialization folder.
###################################################################
# Notify database setup.
createdb --user=postgres notification_api
wall "The db container entrypoint setup is complete!"

42
devcontainer-api/scripts/notify-dev-entrypoint.sh
View File

@@ -1,42 +0,0 @@
#!/bin/bash
set -ex
###################################################################
# This script will get executed *once* the Docker container has
# been built. Commands that need to be executed with all available
# tools and the filesystem mount enabled should be located here.
###################################################################
# Define aliases
echo -e "\n\n# User's Aliases" >> ~/.zshrc
echo -e "alias fd=fdfind" >> ~/.zshrc
echo -e "alias l='ls -al --color'" >> ~/.zshrc
echo -e "alias ls='exa'" >> ~/.zshrc
echo -e "alias l='exa -alh'" >> ~/.zshrc
echo -e "alias ll='exa -alh@ --git'" >> ~/.zshrc
echo -e "alias lt='exa -al -T -L 2'" >> ~/.zshrc
# # Kubectl aliases and command autocomplete
# echo -e "alias k='kubectl'" >> ~/.zshrc
# echo -e "alias k-staging='aws eks --region ca-central-1 update-kubeconfig --name notification-canada-ca-staging-eks-cluster'" >> ~/.zshrc
# echo -e "alias k-prod='aws eks --region ca-central-1 update-kubeconfig --name notification-canada-ca-production-eks-cluster'" >> ~/.zshrc
# echo -e "source <(kubectl completion zsh)" >> ~/.zshrc
# echo -e "complete -F __start_kubectl k" >> ~/.zshrc
cd /workspace
# Warm up git index prior to display status in prompt else it will
# be quite slow on every invocation of starship.
git status
make generate-version-file
poetry install --dev
# Install virtualenv to support running the isolated make freeze-requirements from within the devcontainer
pip3 install virtualenv
# Upgrade schema of the notification_api database
poetry run flask db upgrade
# Run flask server
# make run-flask

39
devcontainer-api/scripts/notify-worker-entrypoint.sh
View File

@@ -1,39 +0,0 @@
#!/bin/bash
set -ex
###################################################################
# This script will get executed *once* the Docker container has
# been built. Commands that need to be executed with all available
# tools and the filesystem mount enabled should be located here.
###################################################################
# Define aliases
echo -e "\n\n# User's Aliases" >> ~/.zshrc
echo -e "alias fd=fdfind" >> ~/.zshrc
echo -e "alias l='ls -al --color'" >> ~/.zshrc
echo -e "alias ls='exa'" >> ~/.zshrc
echo -e "alias l='exa -alh'" >> ~/.zshrc
echo -e "alias ll='exa -alh@ --git'" >> ~/.zshrc
echo -e "alias lt='exa -al -T -L 2'" >> ~/.zshrc
# # Kubectl aliases and command autocomplete
# echo -e "alias k='kubectl'" >> ~/.zshrc
# echo -e "alias k-staging='aws eks --region ca-central-1 update-kubeconfig --name notification-canada-ca-staging-eks-cluster'" >> ~/.zshrc
# echo -e "alias k-prod='aws eks --region ca-central-1 update-kubeconfig --name notification-canada-ca-production-eks-cluster'" >> ~/.zshrc
# echo -e "source <(kubectl completion zsh)" >> ~/.zshrc
# echo -e "complete -F __start_kubectl k" >> ~/.zshrc
cd /workspace
# Warm up git index prior to display status in prompt else it will
# be quite slow on every invocation of starship.
git status
make generate-version-file
poetry install --dev
# Install virtualenv to support running the isolated make freeze-requirements from within the devcontainer
pip3 install virtualenv
# Run Celery worker
# make run-celery

42
devcontainer-worker/.devcontainer.json
View File

@@ -1,42 +0,0 @@
{
"name": "notification-api",
"dockerComposeFile": "../docker-compose.devcontainer.yml",
"service": "worker",
"workspaceFolder": "/workspace",
"shutdownAction": "stopCompose",
"remoteEnv": {
"PATH": "/home/vscode/.local/bin:${containerEnv:PATH}" // give our installed Python modules precedence
},
"settings": {
"[python]": {
"editor.formatOnSave": true
},
"python.linting.enabled": true,
"python.linting.pylintEnabled": true,
"python.pythonPath": "/usr/local/bin/python"
},
"features": {
"docker-from-docker": {
"version": "latest",
"moby": true
}
},
"extensions": [
"ms-python.black-formatter",
"donjayamanne.python-extension-pack",
"ms-azuretools.vscode-docker",
"ms-python.vscode-pylance",
"eamodio.gitlens",
"wholroyd.jinja",
"pmbenjamin.vscode-snyk",
"visualstudioexptteam.vscodeintellicode",
"yzhang.markdown-all-in-one",
"ms-ossdata.vscode-postgresql",
"GitHub.copilot",
"ms-vsliveshare.vsliveshare",
"mtxr.sqltools",
"mtxr.sqltools-driver-pg",
],
"postCreateCommand": "notify-worker-entrypoint.sh",
"remoteUser": "vscode"
}

71
docker-compose.devcontainer.yml
View File

@@ -1,71 +0,0 @@
version: '3'
services:
db:
container_name: db
image: postgres:13.4
volumes:
- ./devcontainer-api/initdb:/docker-entrypoint-initdb.d
environment:
PGGSSENCMODE: disable
POSTGRES_USER: postgres
POSTGRES_PASSWORD: chummy
POSTGRES_HOST_AUTH_METHOD: trust
ports:
- "5432:5432"
expose:
- "5432"
command:
- "postgres"
- "-c"
- "listen_addresses=*"
restart: always
redis:
container_name: redis
image: redis:6.2
restart: always
command: sh -c "rm -f /data/dump.rdb && redis-server --port 6380" # disable persistence
ports:
- "6380:6380"
expose:
- "6380"
dev:
container_name: dev
image: dev-notification-api
build:
context: .
dockerfile: devcontainer-api/Dockerfile
env_file: .env
volumes:
- .:/workspace:cached
command: sleep infinity
ports:
- 6011:6011
expose:
- "6011"
depends_on:
- db
- redis
links:
- db
worker:
container_name: worker
image: dev-notification-api
build:
context: .
dockerfile: devcontainer-api/Dockerfile
env_file: .env
volumes:
- .:/workspace:cached
command: sleep infinity
depends_on:
- db
- redis
- dev
links:
- db
networks:
default:
name: notify-network
external: true

32
docker/Dockerfile
View File

@@ -1,32 +0,0 @@
FROM python:3.9-slim-bullseye as parent
ENV PYTHONUNBUFFERED=1
ENV DEBIAN_FRONTEND=noninteractive
RUN echo "Install base packages" && apt-get update \
&& apt-get install -y --no-install-recommends \
build-essential \
git \
&& echo "Install binary app dependencies" \
&& apt-get install -y --no-install-recommends \
libcurl4-openssl-dev \
libssl-dev \
&& apt-get -y clean \
&& rm -rf /var/lib/apt/lists/* /tmp/*
RUN pip install --upgrade pip
WORKDIR /home/vcap/app
COPY requirements.txt ./
# RUN useradd celeryuser
RUN \
echo "Installing python dependencies" \
&& pip install -r requirements.txt
COPY app app
COPY run_celery.py .
COPY environment.sh .
COPY Makefile .