Files
notifications-api/scripts/run_tests.sh
Alexey Bezhan 75940c9566 Pin all application requirements in requirements.txt
The list of top-level dependencies is moved to requirements-app.txt,
which is used by `make freeze-requirements` to generate the full
list of requirements in requirements.txt.

This is based on alphagov/digitalmarketplace-api#615, so rationale
from that PR applies here.

We had a problem with unpinned packages on new deployments leading
to failed tests (e.g. alphagov/notifications-admin#2144) which is
why we're implementing this now.

After re-evaluating pipenv again, this still seems like the least
disruptive approach:

* pyup.io has experimental support for Pipfile, but doesn't respect
  version ranges or updating hashes in the lock file
* CloudFoundry buildpack recognizes and supports Pipfiles out of the
  box, but the support is relatively new. For example until recently
  CF would install dev packages during deployment. It's also based on
  generating a requirements file from the Pipfile, which doesn't
  properly support pinning VCS dependencies (eg it doesn't set the
  #egg= version, meaning pip will not upgrade the package if it's
  already installed).
* pipenv has a strict dependency resolution algorithm, which doesn't
  appear to be well documented and can cause some unexpected failures.
  For example, pipenv doesn't seem to be able to install `awscli-cwlogs`
  package at all, believing it to have a version conflict for `botocore`
  (which it doesn't list as a direct dependency) while neither `pip` nor
  `pip-tools` highlight any issues with it.
* While trying out `pipenv install` on our list of dependencies it would
  regularly fail to install utils with a "Will try again." message.
  While the installation succeeds after a retry, this doesn't inspire
  confidence.
* The switch to Pipfile and pipenv-managed virtualenvs requires a series
  of changes to `make` targets and scripts - replacing `pip install` with
  `pipenv`, removing references to requirements files and prefixing
  commands with `pipenv run`. While it's likely to simplify the overall
  process of managing dependencies, it would require time to properly
  implement across our applications and environments (Jenkins, PaaS,
  docker containers, and dev machines).
2018-07-10 14:59:04 +01:00

36 lines
724 B
Bash
Executable File

#!/bin/bash
#
# Run project tests
#
# NOTE: This script expects to be run from the project root with
# ./scripts/run_tests.sh
set -o pipefail
function display_result {
RESULT=$1
EXIT_STATUS=$2
TEST=$3
if [ $RESULT -ne 0 ]; then
echo -e "\033[31m$TEST failed\033[0m"
exit $EXIT_STATUS
else
echo -e "\033[32m$TEST passed\033[0m"
fi
}
if [[ -z "$VIRTUAL_ENV" ]] && [[ -d venv ]]; then
source ./venv/bin/activate
fi
make test-requirements
display_result $? 1 "Requirements check"
flake8 .
display_result $? 1 "Code style check"
# run with four concurrent threads
py.test --cov=app --cov-report=term-missing tests/ --junitxml=test_results.xml -n4 -v --maxfail=10
display_result $? 2 "Unit tests"