This changeset pulls in all of the notification_utils code directly into the API and removes it as an external dependency. We are doing this to cut down on operational maintenance of the project and will begin removing parts of it no longer needed for the API.
Signed-off-by: Carlo Costino <carlo.costino@gsa.gov>
This deletes a big ol' chunk of code related to letters. It's not everything—there are still a few things that might be tied to sms/email—but it's the the heart of letters function. SMS and email function should be untouched by this.
Areas affected:
- Things obviously about letters
- PDF tasks, used for precompiling letters
- Virus scanning, used for those PDFs
- FTP, used to send letters to the printer
- Postage stuff
Marshmallow v3 has changed the way that DateTimes get serialized
(https://marshmallow.readthedocs.io/en/stable/upgrading.html#datetime-leaves-timezone-information-untouched-during-serialization).
In order to avoid breaking anything, we want to keep the existing way of
handling DateTimes for now - this could be changed later. We can't just
pass a `format` argument to a DateTime field with the old format, which
looked like this `2017-09-19T00:00:00+00:00`. When we tried that,
Marshmallow then expected data that we are loading to also have that
format, which it doesn't.
This adds a new field, which serializes data in the old format but which
doesn't require data that is being deserialized to have such a precise
format.
Due to a difference in marshmallow 3, when calling
`service_schema.load()` with permissions data the permissions data was
being dropped. We could fix this by allowing the ServiceSchema to
include any unknown keys using but that have unexpected consequences.
Instead, this change adds a method so that the schema knows how to
deserialize permissions. This uses the same code that the `pre_load`
method uses, but there were errors when it wasn't included in both places.
When we nest the `TemplateSchema` as a field on the
`NotificationWithTemplateSchema`, we want to include the
`is_precompiled_letter` field. However, we don't want the
`is_precompiled_letter` in any of the other places that we use the
`TemplateSchema`.
The way we had the code was giving errors in version 3 of Marshmallow
since `is_precompiled_letter` was not defined on the `TemplateSchema`.
Instead of writing complicated logic around when the field should be
included or excluded, this adds a new schema which we can use in the one
place where we do want to include `is_precompiled_letter`.
https://marshmallow.readthedocs.io/en/stable/upgrading.html#schemas-are-always-strict
`.load` doesn't return a `(data, errors)` tuple any more - only data is
returned. A `ValidationError` is raised if validation fails. The code
now relies on the `marshmallow_validation_error` error handler to handle
errors instead of having to raise an `InvalidRequest`. This has no
effect on the response that is returned (a test has been modified to
check).
Also added a new `password` field to the `UserSchema` so that we don't
have to specially check for password errors in the `.create_user` endpoint
- we can let marshmallow handle them.
We had the `only` defined in the Meta class, and this wasn't working -
any extra fields were also being loaded. This moves it to the point
where the class is instantiated, which now works.
We have a lot of cases in the schemas where we're excluding a field that
doesn't actually exist on the schema anyway. This is often because a
model has been deleted, and the schema has not been updated. These
excluded fields have no effect at the moment, but Marshmallow 3 does
raise an error if you try and exclude non-existent fields.
There should be no change to what gets (de)serialized after this change.
