`create_history` used to initialise an empty model instance and
assign attributes to it one by one. Assigned attributes are
collected by iterating over internal state of the modified record
and stored in a dictionary, so the order in which they are set on
the history instance is undefined.
This works fine for most model instances, but recent changes to the
Template models require `template_type` to be set before `reply_to`,
otherwise Template doesn't know which column to use to persist the
`reply_to` value.
This means that we need to order the attributes when they're being
assigned. We achieved this by defining a constructor for Template
objects that assigns `template_type` first.
In order to make it work with `create_history` we replace individual
`setattr` calls with a single call to the Template constructor which
passes the entire data dictionary as keyword arguments.
This works because all persisted attributes are defined as columns
and SQLAlchemy accepts all column values as constructor arguments.
Increasing the Gunicorn workers to 10 actually made the API decrease
in performance from ~200 requests/sec with with 20 API instances to
150 requests per/sec. The base value is 5 so this is to test how the API
performs with a slight increase.
Increased Gunicorn threads to 10 on staging to test if there is a
performance increase when testing request per second. Increased to 10.
Gunicorn recommend 2 x cores + 1 however on PaaS the number of cores
is not consistent.
* unused variables
* variables in loops overshadowing imports
* excepts with no defined exc type (tried to avoid `except Exception` too)
* history mapper is still too complex
* default variables should never be mutable
(saves us having to write the stuff ourselves). Also adds a small
click plugin to do datetime parsing.
Sample output:
```
[leohemsted:~/dev/api]$ flask command create_provider_rates --help
Usage: flask command create_provider_rates [OPTIONS]
Backfill rates for a given provider
Options:
-p, --provider_name [mmg|firetext|ses]
[required]
-c, --cost FLOAT Cost (pence) per message including decimals
[required]
-d, --valid_from DATE [required]
--help Show this message and exit.
[leohemsted:~/dev/api]$ flask command create_provider_rates -p ses -c 1.234 -d invalid
Usage: flask command create_provider_rates [OPTIONS]
Error: Invalid value for "-d" / "--valid_from": Could not parse datetime string "invalid" formatted as %Y-%m-%dT%H:%M:%S
```
this reduces the amount of error messages we log (we'll no longer log
at error level when build-dvla-file-for-job retries while waiting for
the task to finish), and make sure we retry in those cases above - db
or s3 having temporary troubl
This causes an issue when it hits the max retry limit, and tries to
throw your exception to let you deal with it - at this point it was
moaning because we pass in a string
if it's not defined, and we're inside an exception block celery uses
that instead.
