Flask-SQLAlchemy paginate function issues a separate query to get
the total count of rows for a given filter. This query (with
filters used by the API integration Message log page) is slow for
services with large number of notifications.
Since Message log page doesn't actually allow users to paginate
through the response (it only shows the last 50 messages) we can
use limit instead of paginate, which requires passing in another
flag from admin to the dao method.
`count` flag has been added to `paginate` in March 2018, however
there was no release of flask-sqlalchemy since then, so we need
to pull the dev version of the package from Github.
Bumped notifications-utils to 3.7.0. Version 3.7.0 includes the
`convert_utc_to_bst` and `convert_bst_to_utc` functions and the
`LETTER_PROCESSING_DEADLINE` constant, so these have been removed from
this repo and anywhere using these has now been updated to get these
from `notifications-utils`.
Also bumped pytest by a patch version to bring in a bug fix.
Updated jsonschema to Draft7, this allowed a conditional validation on subject, if template_type == 'email' or 'letter' then subject is required.
This version is backward compatible with Draft4.
When creating TempalteRedacted, I've built the dict depending on if the created_by or created_by_id exists.
itsdangerous v1 uses sha512 instead of sha1 to sign and unsign its
strings. Pin to 0.24 until we figure a migration plan, since it's
used in a few places (In the DB, in email tokens, and sending blobs
to celery at least).
also pyup ignore awscli and botocore because that complex mesh of
dependency hell doesn't play will with pyup opening a new PR every time
that botocore/awscli update (every few days)
you need to `pip install celery[sqs]` to get the additional
dependencies that celery needs to use SQS queues - there are two libs -
boto3 and pycurl.
pycurl is a bunch of python handles around curl, so needs to be
installed from source so it can link to your curl/ssl libs. On paas and
in docker this works fine (needed to add `libcurl4-openssl-dev` to the
docker container), but on macos it can't find openssl. We need to pass
a couple of flags in:
* set the environment variable PYCURL_SSL_LIBRARY=openssl
* pass in the global options `build_ext` and `-I{openssl_headers_path}`.
As shown here:
https://github.com/pycurl/pycurl/issues/530#issuecomment-395403253
Env var is no biggie, but using any install-option flags disables
wheels for the whole pip install run. (See
https://github.com/pypa/pip/issues/2677 and
https://github.com/pypa/pip/issues/4118 for more context on the
install-options flags). A whole bunch of our dependencies don't
install nicely from source (but do from wheel), so this commit installs
pycurl separately as an initial step, with the requisite flags, and
then installs the rest of the requirements as before.
I've updated the makefile and bootstrap.sh files to reflect this, but
if you run `pip install -r requirements.txt` from scratch you will run
into issues.
awscli has a requirement of a new version of botocore
moto has a requirement of an old version of boto3, which requires an
old version of botocore
We had to pin boto to an older version, because of the moto issues.
this commit pins awscli to the version currently deployed on prod, so
that it plays nice with that older version of boto/botocore
