Changes:
53.0.0
---
* `notifications_utils.columns.Columns` has moved to
`notifications_utils.insensitive_dict.InsensitiveDict`
* `notifications_utils.columns.Rows` has moved to
`notifications_utils.recipients.Rows`
* `notifications_utils.columns.Cell` has moved to
`notifications_utils.recipients.Cell`
52.0.0
---
* Deprecate the following unused `redis_client` functions:
- `redis_client.increment_hash_value`
- `redis_client.decrement_hash_value`
- `redis_client.get_all_from_hash`
- `redis_client.set_hash_and_expire`
- `redis_client.expire`
51.3.1
---
* Bump govuk-bank-holidays to cache holidays for next year.
we previously pinned cryptography to versions less than 3.4 since after
that point, cryptography started using rust as a dependency. This isn't
an issue if you install from wheel, but we found that the version of pip
bundled with the python buildpack was too old to support this. However,
since upgrading from python 3.6 to python 3.9, the pip version has been
bumped and we now no longer need to pin cryptography as it installs
correctly.
two vulnerabilities in <4.6.5 (GHSL-2021-1037 and GHSL-2021-1038)
https://github.com/lxml/lxml/blob/master/CHANGES.txt
also removes docopt as we don't use it except for a dev script (which we
might not need anyway)
This includes performance improvements for RecipientCSV, which may
reduce the processing time in some edge cases - this depends on if
the Admin app rejects CSVs with these edge cases.
Just so other people don’t have to merge these changes.
The breaking changes don’t affect this repo because the API doesn’t:
- check the service guestlist before sending a message
- do any visual preview of emergency alert messages
> **51.0.0**
> - Initial argument to RecipientCSV renamed from whitelist to guestlist, in other words consuming code should call RecipientCSV(guestlist=['test@example.com'])
> - RecipientCSV.whitelist property renamed to RecipientCSV.guestlist
>
> **50.0.0**
> - Make icon in broadcast_preview_template.jinja2 an inline SVG (requires changes to the CSS of consumer code)
>
> **49.1.0**
> Add ttl_in_seconds argument to RequestCache.set to let users specify a custom TTL
This commit also changes the format of the line in the requirements
file, copying https://github.com/alphagov/notifications-admin/pull/4074/files
This new version of utils implements the transformation of our polygons
to a Cartesian plane. In other words, it converts them from being
defined in spherical degrees to metres.
For the API this means our simplification will be slightly more
accurate.
Note that the new base class doesn't include a bespoke feature we
had here: 'log_on_worker_shutdown'. We've agreed it's reasonable
to remove it for now as it was introduced many years ago and its
use case is unclear - we can always add it back if needed.
This is purely by elimination: I couldn't see anything in the logs
to indicate the cause of the crashes, just that the processes were
exiting. The crash seemed to happen immediately after the AWS logs
part of the wrapper script, which was a small indicator it might be
something AWS-related. Since this package is no longer included by
other dependencies, we need to include it explicitly.
Most of these are due to dependency changes in celery / kombu:
-boto==2.49.0
9b2a172078
+cached-property==1.5.2
560518287a
+click-didyoumean==0.3.0
+click-plugins==1.1.1
+click-repl==0.2.0
f462a437e3/requirements/default.txt
+pycurl==7.43.0.5
59d88326b8/requirements/extras/sqs.txt
+vine==5.0.0
f6c3b3313f
I'm not sure about the following, but neither are critical so
I don't think it's worth tracking down where they came from.
+prompt-toolkit==3.0.21
+wcwidth==0.2.5
There are several other changes we need to make in order to install
the new version. For more context, see:
- 208e90e40f
- e3d1993a58
- 7e93611fce
In the next commits we'll look at tidying up the config and other
dependencies so the change is deployable.
This is so we can clear the diff prior to upgrading to Celery 5,
which has a number of transitive package changes associated with
it. It makes sense for this to be a separate change in case it
causes issues of its own. However, the only major difference in
this commit is pyparsing [1].
[1]: https://github.com/pyparsing/pyparsing/blob/master/docs/whats_new_in_3_0_0.rst
This is the newest version.
Pyup is complaining about vulnerabilities in version 1.0.1, specifically
> Werkzeug version 2.0.2 improves the security of the debugger cookies.
> "SameSite" attribute is set to "Strict" instead of "None", and the
> secure flag is added when on HTTPS.
Previously we were using whatever version of Werkzeug that Flask
specified this pins it to get rid of the vulnerability without having to
upgrade everything at once.
We’ve done this for the admin app already:
https://github.com/alphagov/notifications-admin/pull/4042/files
I suspect the memory usage issues we saw with version 2.0.0 have been
fixed in 2.0.2, per this line in the changelog:
> Fix memory usage for locals when using Python 3.6 or pre 0.4.17 greenlet versions.
> https://github.com/pallets/werkzeug/pull/2212
— https://werkzeug.palletsprojects.com/en/2.0.x/changes/
This new version of utils implements the transformation of our polygons
to a Cartesian plane. In other words, it converts them from being
defined in spherical degrees to metres.
For the API this means our simplification will be slightly more
accurate.
We observe high memory usage since we bumped it (along with other
things) and because it only appears on the API and not on the workers
the hypothesis is that Werkzeug is responsible for it.
See https://github.com/alphagov/notifications-utils/pull/878 for
details.
Changes we had to make for our app and tests to work correctly
after the dependency updates:
1. Update emergency alerts polygons test because we changed
how exact we are with locations of the points on the map.
2. Use Flask's g object to set additional request attributes
So far we have been storing them in _request_ctx_stack which is
an innard for Flask's request context.
Because of major update to Werkzeug dependency, which Flask relies
on, the way we were using it stopped working, so we had a new
way to set those values.
The way we set those values now, by using g object, seems to also
be favoured in Flask documentation:
https://flask.palletsprojects.com/en/1.1.x/reqcontext/#how-the-context-works
We haven't bumped the test version for a while.
Also bumped the version of Flask and itsdangerous.
In order to fix flask warnings I needed to changed how the blueprints were registerd.
for the service.
The letters rates for cronw and non crown are the same. It would be nice
to remove the need for crown but for now this is a quick fix.
The maximum content count of a broadcast varies depending on its
encoding, so we can’t simply validate it against a schema. This commit
moves to using the validation from `notifications-utils`, and raising a
custom error response.
One of our dependencies has a dependency on cryptography, which has
recently released version 3.4.
This version introduced a circular import error
(pyca/cryptography#5756) which was fixed in
3.4.1.
However, 3.4.1 has a different error where it fails because it cannot
find a rust compiler.
The suggested
solutions are:
Install a newer version of pip which will install a pre-compiled
cryptography wheel OR
Have rust installed and available on our PATH so that it can be used
to build the package.
Since we can't change the buildpack's pip version and we cannot install
rust ourselves, the only we're left with is to avoid upgrading to 3.4 -
at least until PaaS updates their python buildpacks.
