This changeset pulls in all of the notification_utils code directly into the admin and removes it as an external dependency. We are doing this to cut down on operational maintenance of the project and will begin removing parts of it no longer needed for the admin.
Signed-off-by: Carlo Costino <carlo.costino@gsa.gov>
The responses we get to paginated queries from the API are fairly
consistent, so we should be able to reuse the code that takes JSON
from the API and turns it into Python objects. This commits factors out
that code so that it is reusable (by inheriting from it).
If a subclass of `JSONModel` defines a property then we shouldn’t try
to override it with the value from the underlying dictionary.
Rather than silently fail we should raise an exception because it will
help keep our list of `ALLOWED_PROPERTIES` nice and tidy.
We wrote custom `__getattr__` and `__getitem__` implementation to make
it easier to find code that was accessing fields in the dictionary that
we weren’t aware of. See this commit for details:
b48305c50d
Now that no view-layer code accesses the service dictionary directly we
don’t need to support this behaviour any more. By removing it we can
make the model code simpler, and closer to the `SerialisedModel` it
inherits from.
It still needs some custom implementation because:
- a lot of our test fixtures are lazy and don’t set up all the expected
fields, so we need to account for fields sometimes being present in
the underlying dictionary and sometimes not
- we often implement a property that has the same name as one of the
fields in the JSON, so we have to be careful not to try to override
this property with the value from the underlying JSON
Brings in:
- re-usable `SerialisedModel`
- speed improvements to processing CSVs against email templates
I chose not to rename `JSONModel` or `ModelList` to keep the diff nice
and small.
`dir(object)` is a useful Python function that tells you what attributes
and methods an object has. It’s also used by tools like iPython and IDEs
for code completion.
Some of the attributes of a `JSONModel` are dynamic, based on what
fields we expect in the underlying JSON. Therefore they don’t
automatically end up in the result of calling `dir`. To get around this
we can implement our own `__dir__` method, which also returns the names
of the fields we’re expecting the the JSON.
Inspired by this Raymond Hettinger tweet:
> #python tip: If you add attributes to an API with __getattr__() or
> __getattribute__(), remember to update __dir__() to make the extension
> introspectable.
— https://twitter.com/raymondh/status/1249860863525146624
Because ModelList implements `__add__` we can do the following:
```python
ImmediateJobs() + ScheduledJobs()
ImmediateJobs() + []
```
Both of these call the `__add__` method of `ImmediateJobs`.
What we can’t do is this:
```python
[] + ScheduledJobs()
```
That tries to call the `__add__` method of list, which doesn’t know what
to do with an instance of `ModelList`.
The Pythonic way to deal with this is to implement `__radd__` (right
add) which is invoked when our instance is on the right hand side of the
addition operator.
I’m hoping that if I can design something that clearly differentiates
them then we won’t need to do so by putting them in separate tables,
which then need labelling, which would clutter up the page.
The property doesn’t represent the whole client, but just one method on
it. So this commit renames the property to better describe what it is
designed to store.
Flake8 Bugbear checks for some extra things that aren’t code style
errors, but are likely to introduce bugs or unexpected behaviour. A
good example is having mutable default function arguments, which get
shared between every call to the function and therefore mutating a value
in one place can unexpectedly cause it to change in another.
This commit enables all the extra warnings provided by Flake8 Bugbear,
except for the line length one (because we already lint for that
separately).
It disables:
- _B003: Assigning to os.environ_ because I don’t really understand this
- _B306: BaseException.message is removed in Python 3_ because I think
our exceptions have a custom structure that means the `.message`
attribute is still present
We store our audit history in two ways:
1. A list of versions of a service
2. A list of events to do with API keys
In the future there could be auditing data which we want to display that
is stored in other formats (for example the event table).
This commit adds some objects which wrap around the different types of
auditing data, and expose a consistent interface to them. This
architecture will let us:
- write clean code in the presentation layer to display these events on
a page
- add more types of events in the future by subclassing the `Event` data
type, without having to rewrite anything in the presentation layer
`__getattr__` is called whenever an attribute error is raised.
This means that if something deep inside a property on a model raised
an attribute error, that error would be caught by `__getattr__`, which
would then raise an exception that looked like the property itself
didn’t exist. Very confusing.
The solution seems to be to override `__getattribute__` instead, which
handles _all_ attributes, not just those that aren’t explicitly defined.
We then only intervene if the desired attribute is one of the
`ALLOWED_PROPERTIES`, otherwise falling through to the built in methods
of the underlying `object`.
Our other models inherit from `JSONModel`, rather than manually doing
lookups of the JSON in the `__init__` method. This commit changes the
user model to work in the same way.
I had to add a new concept (`DEFAULTS`) to account for some properties
not always being present in the (mocked) JSON. In reality it might be
that the API does always return these values. This should be looked at
in future work, to see which defaults can be safely removed. At least
now they:
- do not mean any changes are needed to the existing tests
- are explicitly separated from the attributes that we do expect to
always be in the JSON response
The data flow of other bits of our application looks like this:
```
API (returns JSON)
⬇
API client (returns a built in type, usually `dict`)
⬇
Model (returns an instance, eg of type `Service`)
⬇
View (returns HTML)
```
The user API client was architected weirdly, in that it returned a model
directly, like this:
```
API (returns JSON)
⬇
API client (returns a model, of type `User`, `InvitedUser`, etc)
⬇
View (returns HTML)
```
This mixing of different layers of the application is bad because it
makes it hard to write model code that doesn’t have circular
dependencies. As our application gets more complicated we will be
relying more on models to manage this complexity, so we should make it
easy, not hard to write them.
It also means that most of our mocking was of the User model, not just
the underlying JSON. So it would have been easy to introduce subtle bugs
to the user model, because it wasn’t being comprehensively tested. A lot
of the changed lines of code in this commit mean changing the tests to
mock only the JSON, which means that the model layer gets implicitly
tested.
For those reasons this commit changes the user API client to return
JSON, not an instance of `User` or other models.
This is the first step of replacing the `domains.yml` file.
In order to replicate the same functionality we get from the
`domains.yml` file and its associated code this commit adds a
`Organisation` model. This model copies a lot of methods from the
`AgreementInfo` class which wrapped the `domains.yml` file.
It factors out some stuff that would otherwise be duplicated between the
`Organisation` and `Service` model, in such a way that could be reused
for making other models in the future.
This commit doesn’t change other parts of the code to make use of this
new model yet – that will come in subsequent commits.
