When generating a new migration we give it a number that increments
the latest existing migration on master. This means that when there
are multiple PRs open containing a migration and one of them gets
merged the others need to be updated to move their migration files
to apply on top of the recently merged one.
This requires renaming the file and changing migration references
for both the migration revision and down_revision.
If a PR introduced more than 1 migration they all need to be updated
one after another since each one needs to be renamed.
This adds a script to simplify this process. `./scripts/fix_migrations.py`
will check for any branch points If it finds exactly one (which
should be the common case), it asks which migration should be moved
and renames / updates references to move the selected branch on top
of the other one.
It won't resolve any conflicts within migrations themselves (eg if
both branches modified the same column) and it won't try to resolve
cases with more than 1 branch.
click (http://click.pocoo.org/) is used by flask to run its cli args.
In removing flask_script (it's unmaintained), we had to migrate all our
commands to use click. This is a change for the better in my eyes - you
don't need to define the command in several places, and it makes
managing options a bit easier.
View diff with whitespace turned off unless you're a masochist.
previously they were using sample_service fixture under the hood, but
with full permissions added - this works fine, **unless** there's
already a service with the name "sample service" in the database. This
can happen for two reasons:
* A previous test didn't tear down correctly
* This test already invoked the sample_service fixture somehow
If this happens, we just return the existing service, without modifying
its values - values that we might change in tests, such as
research mode or letters permissions.
In the future, we'll have to be vigilant! and aware! and careful! to
not use sample_service if we're doing tests involving letters, since
they create a service with a different name now
previously we didn't do this because the tests all used the same DB
(test_notifications_api), however @minglis shared a snippet that simply
creates one test db per thread.
previously in run_app_paas.sh, we captured stdout from the app and
piped that into the log file. However, this came up with a bunch of
problems, mainly:
* exceptions with stack traces often weren't formatted properly,
and kibana could not parse them
* celery logs were duplicated - we'd collect both the json logs and
the human readable stdout logs.
instead, with the updated utils library, we can use that to log json
straight to the appropriate directory directly.
Stopping celery via upstart issues a TERM signal to the whole process group, which immediately kills the worker threads. This can lead to inconsistentcy in the state of a notification as an API call, for example, can take several seconds consequently the likelihood of the call being lost due to a restart is relatively high.
We need to ensure that we stop cleanly to ensure code deploys and autoscale events do not cause issues.
this script is called as part of a pre-stop hook in upstart, which passes a pid file location to the script. The script will then issues the TERM signal to only the master celery process. which will cleanly shut down the workers and exit.
the script will loop based on the existence of a /proc/{pid} for the celery master process. Once the file is gone the script will exit. Upstart blocks on this script before issuing kill on whatever processes may be left hanging around.
the length of the loop is a maximum 15 minutes - which is the tested max time a task may take. In reality it will exit much quicker.
PEP8 was renamed to pycodestyle; this issue explains why:
https://github.com/PyCQA/pycodestyle/issues/466
This commit changes our tests to use pycodestyle instead of pep8.
It also means:
- making a couple of whitespace changes to appease the linter
- disabling warnings for bare `Except`s (ie `Except` instead of `Except
ValueError`) – this seems like a sensible thing to catch but I’m not
going to make meaningful code changes in this commit
call generate-version-file before tests, since they'll fall over if the version file isn't present
use /Users/leohemsted/.virtualenvs/api rather than looking for ./venv/ - if there's some other venv already active, then don't try and look for a current venv to activate
