add script to run celery from within docker

as a team we primarily develop locally. However, we've been experiencing
issues with pycurl, a subdependency of celery, that is notoriously
difficult to install on mac. On top of the existing issues, we're also
seeing it conflict with pyproj in bizarre ways (where the order of
imports between pyproj and pycurl result in different configurations of
dynamically linked C libraries being loaded.

You are encouraged to attempt to install pycurl locally, following these
instructions: https://github.com/alphagov/notifications-manuals/wiki/Getting-Started#pycurl

However, if you aren't having any luck, you can instead now run celery
in a docker container.

`make run-celery-with-docker`

This will build a container, install the dependencies, and run celery
(with the default of four concurrent workers).

It will pull aws variables from your aws configuration as boto would
normally, and it will attempt to connect to your local database with the
user `postgres`. If your local database is configured differently (for
example, with a different user, or on a different port), then you can
set the SQLALCHEMY_DATABASE_URI locally to override that.

Makefile

@@ -177,3 +177,8 @@ disable-failwhale: ## Disable the failwhale app and enable api
 	cf unmap-route notify-api-failwhale ${DNS_NAME} --hostname api
 	cf stop notify-api-failwhale
 	@echo "Failwhale is disabled"
+
+.PHONY: run-celery-with-docker
+run-celery-with-docker: ## Run celery in Docker container (useful if you can't install pycurl locally)
+	docker build -f docker/Dockerfile -t notifications-api .
+	./scripts/run_with_docker.sh make run-celery

app/config.py

@@ -427,7 +427,7 @@ class Development(Config):
     NOTIFY_LOG_PATH = 'application.log'
     NOTIFY_EMAIL_DOMAIN = "notify.tools"
 
-    SQLALCHEMY_DATABASE_URI = 'postgresql://localhost/notification_api'
+    SQLALCHEMY_DATABASE_URI = os.getenv('SQLALCHEMY_DATABASE_URI', 'postgresql://localhost/notification_api')
     REDIS_URL = 'redis://localhost:6379/0'
 
     ANTIVIRUS_ENABLED = os.getenv('ANTIVIRUS_ENABLED') == '1'

docker/Dockerfile

@@ -0,0 +1,32 @@
+FROM python:3.9-slim-bullseye as parent
+
+ENV PYTHONUNBUFFERED=1
+ENV DEBIAN_FRONTEND=noninteractive
+
+RUN echo "Install base packages" && apt-get update \
+    && apt-get install -y --no-install-recommends \
+    build-essential \
+    git \
+    && echo "Install binary app dependencies" \
+    && apt-get install -y --no-install-recommends \
+    libcurl4-openssl-dev \
+    libssl-dev \
+    && apt-get -y clean \
+    && rm -rf /var/lib/apt/lists/* /tmp/*
+
+RUN pip install --upgrade pip
+
+WORKDIR /home/vcap/app
+
+COPY requirements.txt ./
+
+# RUN useradd celeryuser
+
+RUN \
+    echo "Installing python dependencies" \
+    && pip install -r requirements.txt
+
+COPY app app
+COPY run_celery.py .
+COPY environment.sh .
+COPY Makefile .

scripts/run_with_docker.sh

@@ -0,0 +1,13 @@
+#!/bin/bash
+DOCKER_IMAGE_NAME=notifications-api
+
+source environment.sh
+
+docker run -it --rm \
+  -e AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID:-$(aws configure get aws_access_key_id)} \
+  -e AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY:-$(aws configure get aws_secret_access_key)} \
+  -e SQLALCHEMY_DATABASE_URI=${SQLALCHEMY_DATABASE_URI:-postgresql://postgres@host.docker.internal/notification_api} \
+  -v $(pwd):/home/vcap/app \
+  ${DOCKER_ARGS} \
+  ${DOCKER_IMAGE_NAME} \
+  ${@}