This changeset pulls in all of the notification_utils code directly into the API and removes it as an external dependency. We are doing this to cut down on operational maintenance of the project and will begin removing parts of it no longer needed for the API.
Signed-off-by: Carlo Costino <carlo.costino@gsa.gov>
This deletes a big ol' chunk of code related to letters. It's not everything—there are still a few things that might be tied to sms/email—but it's the the heart of letters function. SMS and email function should be untouched by this.
Areas affected:
- Things obviously about letters
- PDF tasks, used for precompiling letters
- Virus scanning, used for those PDFs
- FTP, used to send letters to the printer
- Postage stuff
Flake8 Bugbear checks for some extra things that aren’t code style
errors, but are likely to introduce bugs or unexpected behaviour. A
good example is having mutable default function arguments, which get
shared between every call to the function and therefore mutating a value
in one place can unexpectedly cause it to change in another.
This commit enables all the extra warnings provided by Flake8 Bugbear,
except for:
- the line length one (because we already lint for that separately)
- B903 Data class should either be immutable or use `__slots__` because
this seems to false-positive on some of our custom exceptions
- B902 Invalid first argument 'cls' used for instance method because
some SQLAlchemy decorators (eg `declared_attr`) make things that
aren’t formally class methods take a class not an instance as their
first argument
It disables:
- _B306: BaseException.message is removed in Python 3_ because I think
our exceptions have a custom structure that means the `.message`
attribute is still present
Matches the work done in other repos:
- https://github.com/alphagov/notifications-admin/pull/3172/files
add `datetime` format (note, not the built-in `date-time`) to our json
schemas. this uses the iso8601 library to try and parse the string.
also, move `strict-rfc3339` and `rfc3987` (used by jsonschema to
validate `date-time` and `uri` formats respectively from test
requirements to regular requirements. if they're not installed,
validation silently succeeds, so validation wouldnt reject anything bad
on prod, only in unit tests.
This is part of moving away from `postcode` and towards `address line 7`
We think it will be easier for people to map their existing data to our
API if we let them fill in any 3 lines, instead of requiring at least
1, 2, and postcode specifically.
By adding `force=True` to request.get_json() the mime type is ignore. If the data is not valid json the method will return a `BadRequestError` we catch that and throw our own error with a clear error message "Invalid JSON supplied in POST data".
If the json is valid return the json data or an empty dict if None is passed in.
This PR improves the error messages if the json is invalid, previously, the error message was "None object type" message which is not very helpful.
Updated jsonschema to Draft7, this allowed a conditional validation on subject, if template_type == 'email' or 'letter' then subject is required.
This version is backward compatible with Draft4.
When creating TempalteRedacted, I've built the dict depending on if the created_by or created_by_id exists.
* create template folder
* rename template folder
* get list of template folders for service (not nested/presented in any
particular way)
* delete template folder
Also removed `lazy=dynamic` from the `template_folder.templates`
relationship. lazy=dynamic returns a query object (which you can then
filter further). We just want to return the entire fetched list, at
least for now.
Renamed the 'datetime' format to 'datetime_within_next_day'. This format
is used to validate the date and time of scheduled notifications, not to
check the format of a datetime.
Switched to using a isinstance check on the string.
Added an order by clause to dao_get_template_usage_stats_by_service, it was causing an itermitten failure in the tests.
similar to sms/email, however, for consistency with response values
and internal storage, rather than supplying an "email_address" field
or a "phone_number" field, supply an "address_line_1" and "postcode"
field within the personalisation object.
Added a validation method that always fails for scheduled notifications.
Comment out config for scheduled task.
The schedule notifications will be turned on once we can invite services to use it.
Waiting for the service permission story, must commit this in order to keep things from going stale.
jsonschema states:
> A format attribute can generally only validate a given set of
> instance types. If the type of the instance to validate is not in
> this set, validation for this format attribute and instance SHOULD
> succeed.
We were not checking for the type of the input, and our validators were
behaving in unexpected manners (throwing TypeErrors etc etc). Despite
us declaring that the phone_number field is of type `str`, we still
need to make sure the validator passes gracefully, so that the inbuilt
type check can be the bit that catches if someone passes in a non-str
value. We've seen this with people passing in integers instead of strs
for phone numbers. This'll make them receive a nice 400 error
(e.g. "phone_number 12345 is not of type string"), rather than us
having a 500 internal server error
- uses new utils methods to validate phone numbers
- defaults to International=True on validation. This ensures the validator works on all numbers
- Then check if the user can send this message to the number internationally if needed.
brings in a fix to InvalidEmail/Phone/AddressExceptions not being
instantiated correctly. `exception.message` is not a python standard,
so we shouldn't be relying on it to transmit exception reasons -
rather we should be using `str(exception)` instead. This involved a
handful of small changes to the schema validation
* Ensure we dont raise exception if e.cause does not contain a message
* Ensure we handle case where e.path may be empty
* Refactor existing tests to conform to new format
Created a new schema that accepts request parameters for the
get_notifications v2 route.
Using that to validate now instead of the marshmallow validation.
Also changed the way formatted error messages are returned because
the previous way was cutting off our failing `enum` messages.
This is the schema that individual notifications will conform to
when they are returned from this API.
JSON logic enforces that the right keys are set depending on the
`"type"`. (eg a schema with `"type": "sms"` must have a
`"phone_number"` value and it cannot have an `"email_address"`)
Update the format_checkers to raise the specific exception that why the validator can handle multiple messages.
Which led to a refactor of build_error_message.
Start building up the validators required for post notificaiton.
The app/v2/errors.py is a rough sketch, will be passed a code, the error can look up the message and link for the error message.
