* unused variables
* variables in loops overshadowing imports
* excepts with no defined exc type (tried to avoid `except Exception` too)
* history mapper is still too complex
* default variables should never be mutable
(saves us having to write the stuff ourselves). Also adds a small
click plugin to do datetime parsing.
Sample output:
```
[leohemsted:~/dev/api]$ flask command create_provider_rates --help
Usage: flask command create_provider_rates [OPTIONS]
Backfill rates for a given provider
Options:
-p, --provider_name [mmg|firetext|ses]
[required]
-c, --cost FLOAT Cost (pence) per message including decimals
[required]
-d, --valid_from DATE [required]
--help Show this message and exit.
[leohemsted:~/dev/api]$ flask command create_provider_rates -p ses -c 1.234 -d invalid
Usage: flask command create_provider_rates [OPTIONS]
Error: Invalid value for "-d" / "--valid_from": Could not parse datetime string "invalid" formatted as %Y-%m-%dT%H:%M:%S
```
this reduces the amount of error messages we log (we'll no longer log
at error level when build-dvla-file-for-job retries while waiting for
the task to finish), and make sure we retry in those cases above - db
or s3 having temporary troubl
This causes an issue when it hits the max retry limit, and tries to
throw your exception to let you deal with it - at this point it was
moaning because we pass in a string
if it's not defined, and we're inside an exception block celery uses
that instead.
sleep for 10 seconds to try and make sure that all worker threads
(either web api or celery) have finished before we delete when we
delete the DB is unbound from the app, which can cause
"permission denied for relation" psycopg2 errors.
Removed the REST endpoint and the DAO that it uses as the endpoint is
no longer used by the Admin UI and the DAO is not reused anywhere
else.
- Removed REST endpoint
- Removed DAO which gets the stats
- Removed associated tests of both methods
Rationale:
Sometimes, when deploying, we've seen errors while stopping the old
apps: "(psycopg2.ProgrammingError) permission denied for relation notifications".
When you call cf stop, it may not be entirely synchronous. Under the
hood, cloudfoundry has to do a whole bunch of things when you stop an
app - it has its own internal db of what app states are, and also has
to remove it from any load balancers etc, and also it has to actually
stop the app. We're not sure if the `cf stop` command guarantees that
your process has already terminated by the time that the command
returns.
In our Makefile, we call `cf stop`, followed by `cf delete`. One
posisble theory is that the process is still running when `cf stop`
exits, and then `cf delete` unbinds that service from the database,
removing all of it's users' permissions.
This isn't confirmed, however, this commit removes the `cf stop`
command to see if it solves the issue. PaaS team confirmed that
it's redundant - `cf delete` will carry out the same tasks under
the hood.
When generating a new migration we give it a number that increments
the latest existing migration on master. This means that when there
are multiple PRs open containing a migration and one of them gets
merged the others need to be updated to move their migration files
to apply on top of the recently merged one.
This requires renaming the file and changing migration references
for both the migration revision and down_revision.
If a PR introduced more than 1 migration they all need to be updated
one after another since each one needs to be renamed.
This adds a script to simplify this process. `./scripts/fix_migrations.py`
will check for any branch points If it finds exactly one (which
should be the common case), it asks which migration should be moved
and renames / updates references to move the selected branch on top
of the other one.
It won't resolve any conflicts within migrations themselves (eg if
both branches modified the same column) and it won't try to resolve
cases with more than 1 branch.
Switches on authentication checks for Firetext inbound SMS callbacks.
This should only be released once Firetext callback URLs have been
updated with authentication details.
instead of using wsgi, we now use "application" - this tells gunicorn
to look inside the python module application (application.py) for a
wsgi app - and by default that is also called application so rename
the variable.
Also, when running tasks, we're not using gunicorn so need to set the
flask variable in the manifest so that `flask command ...` finds the
app properly.
Remove server_commands as it's not used any more.
