Commit Graph

49 Commits

Author SHA1 Message Date
Chris Hill-Scott
d36b742e14 Automatically set environment vars before tests
Sometimes you just wanna run some tests directly using the `pytest`
command. But you’re in a new shell, and have forgotten to do
`source environment_test.sh`. The screen fills with red, and your day
just got a little bit worse.

This commit will stop this from ever happening again, by making the
setting environment variables part of running Pytest. It does this with
a plugin called pytest-env[1].

pytest.ini is the standard way of configuring pytest. Creating this file
where it didn’t exist before changes the behaviour of pytest, in that
it will now look for tests in the same directory as the file, rather
than defaulting to the `tests/` directory. So we also have to explicitly
configure pytest[2] to tell it that it should only look in this
directory. Otherwise it gets lost in the weeds of `node_modules`.

This also changes the way that `SQLALCHEMY_DATABASE_URI` is overriden to
the convention used by this plugin.

1. https://github.com/MobileDynasty/pytest-env
2. https://docs.pytest.org/en/latest/customize.html#confval-testpaths

fixup! Remove environment_test.sh
2018-02-14 12:19:12 +00:00
Alexey Bezhan
466166c8fb Fix a typo in api-db-migration deployment make task 2018-01-09 11:45:01 +00:00
Alexey Bezhan
f2e6018512 Use generated manifest when deploying apps to PaaS
Adds a new `make generate-manifest` target which is called by
deployment steps.
2018-01-09 10:45:03 +00:00
Athanasios Voutsadakis
806e880ec2 Fix .PHONY reference 2017-12-28 18:29:31 +00:00
Athanasios Voutsadakis
e06b15aa3b Remove unused cf-build and cf-build-with-docker
We are using the non-cf versions of these commands
2017-12-28 18:28:51 +00:00
Athanasios Voutsadakis
4cc8bfb174 Remove unused DNS_NAME variable 2017-12-28 18:28:02 +00:00
Athanasios Voutsadakis
d487d57509 Remove targets related to AWS deployments
Also remove checks for unused variables
2017-12-28 18:24:36 +00:00
Athanasios Voutsadakis
2b829e1f17 Remove upload-codedeploy-artifact target 2017-12-28 18:21:39 +00:00
Athanasios Voutsadakis
a833bbf6d5 Remove build-codedeploy-artifact target
Also remove appspec files that are not used in the paas world.
2017-12-28 18:18:32 +00:00
Leo Hemsted
7d1c4ea722 fix makefile syntax 2017-12-14 15:16:43 +00:00
Leo Hemsted
c6e6fad01f if apps crash on startup, then fail deploy process
we saw an issue where the app started, then immediately crashed due to
a setup error. However, jenkins had already returned positively, and
the deploy continued.

cf-deploy should fail if the app doesn't start up.

We do this by looking through the cloudfoundry events, and aborting
if there are any `app.crash` events for the new GUID.
2017-12-14 14:23:32 +00:00
Leo Hemsted
bcee95214e Add sleep after stopping the old app instances on deploy
sleep for 10 seconds to try and make sure that all worker threads
(either web api or celery) have finished before we delete when we
delete the DB is unbound from the app, which can cause
"permission denied for relation" psycopg2 errors.
2017-11-27 13:46:39 +00:00
Leo Hemsted
4d75f032c6 remove cf stop to try and improve deploy robustness
Rationale:
Sometimes, when deploying, we've seen errors while stopping the old
apps: "(psycopg2.ProgrammingError) permission denied for relation notifications".

When you call cf stop, it may not be entirely synchronous. Under the
hood, cloudfoundry has to do a whole bunch of things when you stop an
app - it has its own internal db of what app states are, and also has
to remove it from any load balancers etc, and also it has to actually
stop the app. We're not sure if the `cf stop` command guarantees that
your process has already terminated by the time that the command
returns.

In our Makefile, we call `cf stop`, followed by `cf delete`. One
posisble theory is that the process is still running when `cf stop`
exits, and then `cf delete` unbinds that service from the database,
removing all of it's users' permissions.

This isn't confirmed, however, this commit removes the `cf stop`
command to see if it solves the issue. PaaS team confirmed that
it's redundant - `cf delete` will carry out the same tasks under
the hood.
2017-11-24 10:53:16 +00:00
Leo Hemsted
d30a8b83c1 update readme and ensure makefile up to date 2017-11-23 17:04:58 +00:00
Athanasios Voutsadakis
2e65417dc2 Explicitly target environment on cf-push 2017-10-31 16:51:49 +00:00
Athanasios Voutsadakis
2c590acff5 Also unbind notify-config and notify-aws 2017-10-02 16:25:49 +01:00
Athanasios Voutsadakis
912728713a Unbind the db service before pushing the app
This should cause the app to get new credentials for the database upon
pushing.
2017-09-27 12:11:44 +01:00
Imdad Ahad
6da3d3ed0b Remove wheels-ing on deployment 2017-07-21 14:26:59 +01:00
Imdad Ahad
726371269a Use python3 on jenkins otherwise SSL erorr occurs 2017-05-31 15:38:57 +01:00
Imdad Ahad
a58e724d21 Add a script and make command to detect if there are any migration changes 2017-05-26 16:44:23 +01:00
bandesz
92c95fe66a Run the db migration script without the run_app_paas.sh script, because it is not failing if the migration script fails 2017-03-23 17:45:16 +00:00
bandesz
c032ea5eee PaaS rollback: check if the rollback app is in a started state 2017-03-16 12:33:10 +00:00
bandesz
1e83f7d8e2 Remove Docker commands for CF, clean up artifact creation 2017-02-28 12:32:01 +00:00
bandesz
34f1eff84d Remove Jenkinsfiles, add PaaS build/upload artifact tasks 2017-02-28 10:47:11 +00:00
bandesz
e3cb908c1a Remove explicit api_host_name vars from the envs 2017-02-23 11:22:58 +00:00
imdadahad
978d99f708 Merge pull request #836 from alphagov/fix-remove-old-wheelhouse-contents
Delete old wheels on build
2017-02-20 16:01:56 +00:00
bandesz
9082aeffa3 Make Docker container names unique for PaaS deployment 2017-02-20 13:21:40 +00:00
Imdad Ahad
28944e3952 Delete old wheels on build 2017-02-20 13:09:30 +00:00
Imdad Ahad
952110577f Remove target dir before build and include any dotfiles in the zips 2017-02-20 10:09:59 +00:00
bandesz
72dc93c6db Add awslogs logging for PaaS 2017-02-16 17:38:30 +00:00
Andras Ferencz-Szabo
e10c5e28cd Merge pull request #763 from alphagov/cloudfoundry
Run on Paas
2017-01-17 11:28:38 +00:00
bandesz
e9a90b46b0 Add Makefile commands to control autoscaling processes 2017-01-16 17:53:11 +00:00
bandesz
ac0e1f1def Add gosu and host user to Docker 2017-01-13 10:24:13 +00:00
bandesz
c9f3005392 Add quotes around Docker volume definitions 2017-01-13 10:24:13 +00:00
bandesz
fd66fbd719 Run API on Paas 2017-01-13 10:24:13 +00:00
bandesz
eda06228d6 Enable SSE for S3 upload 2016-12-05 16:58:20 +00:00
bandesz
07aa4cb0b0 Use http proxy for Docker 2016-11-30 15:57:08 +00:00
Imdad Ahad
ebe2ec4341 Install pip wheel in the dockerfile 2016-10-28 17:48:31 +01:00
Imdad Ahad
c8be8db742 Use venv for installation 2016-10-28 17:43:27 +01:00
Imdad Ahad
497f56bc81 Move wheel dependency installation to docker build 2016-10-28 17:27:37 +01:00
Imdad Ahad
72993311b3 Install wheel when building artifact 2016-10-24 16:23:18 +01:00
Imdad Ahad
177515de8f Use python wheels to speed up AS deployment:
* Create wheels from python dependencies on building codedeploy artifact
* Update script to install wheels and default to pypi if not found
2016-10-21 16:16:10 +01:00
bandesz
8ab7fa3395 Build db migration codedeploy artifact 2016-09-21 11:21:12 +01:00
bandesz
df388ec33d Finish coveralls task 2016-08-30 12:06:05 +01:00
Leo Hemsted
7835339165 Merge branch 'master' into test-fixes 2016-08-26 10:57:41 +01:00
bandesz
135ac6f23e Rename development env to preview 2016-08-24 15:55:29 +01:00
bandesz
f113309713 Fix version file generation, remove master deploys from travis 2016-08-23 13:44:34 +01:00
Leo Hemsted
a3843053cd improved local test-running ability
call generate-version-file before tests, since they'll fall over if the version file isn't present
use /Users/leohemsted/.virtualenvs/api rather than looking for ./venv/ - if there's some other venv already active, then don't try and look for a current venv to activate
2016-08-23 12:17:56 +01:00
bandesz
68c1135379 Create Docker build image, build project with Docker 2016-08-22 17:25:41 +01:00