A comment on the pull request for this branch
pointed out that it's not clear why the 'items'
list is deleted and then reassigned in
extend_params:
https://github.com/alphagov/notifications-admin/pull/3770#pullrequestreview-573067465
The simple reason is that we want to use
merge_jsonlike to merge params and
param_extensions (passed in as extensions) but
merge_jsonlike doesn't merge lists correctly.
I realised that if we just make merge_jsonlike
merge lists correctly, we can use it for
everything extend_params does.
This commit does that, and replaces all calls to
extend_params with merge_jsonlike.
Because extend_params is used across many form
field classes, and so many pages, I took the
following precautions after making those changes:
1. found every use of param_extensions
2. looked at the merges onto params that each would
cause and deduped them to a final list of 6(!)
3. tested pages containing fields from that list
4. added new testcases to the merge_jsonlike tests
for any merges that exist in our codebase but
not in our tests
Current behaviour is to check item-against-item
and merge based on whether items match, irrelevant
of position. This doesn't produce the results we
need for our usecases (merging data to send to
GOVUK Frontend components).
We actually want:
- items to be compared based on their position
- new primitive items at the same position to
overwrite existing ones
- dicts or lists at the same position to be merged
For example,
Starting with this list:
[{"name": "option-1", "value": "1"}]
Merging in this list:
[{"hint": {"text": "Choose one option"}}]
You currently get this:
[
{"name": "option-1", "value": "1"},
{"hint": {"text": "Choose one option"}}
]
We want to get this:
[
{
"name": "option-1", "value": "1",
"hint": {"text": "Choose one option"}
}
]
The OrganisationAgreementSignedForm class has a
bug causing it to render different HTML when the
page loads to when you subsequently refresh it.
This commit proposes a change to the extend_params
function to fix it.
extend_params, is used by the
OrganisationAgreementSignedForm, as well as all
the other WTForms field classes we added to wrap
GOVUK Frontend components. Fixing it should
therefore fix any similar bugs with them.
All of these fields send a dict of configuration
data to the GOVUK Frontend component when they
call it, at render time. This dict is 'JSON-like',
meaning it's values can be all the primitives as
well as lists and dicts. This also means it can go
quite deep.
Extending the default configuration
The classes have a default dict of this data kept
privately in the params variable. They let you
change it by passing in an argument called
param_extensions on instantiation, after that,
through an attribute of the same name and at
render time as the same argument (in templates).
The extend_params function
The param_extensions dict is used as a collection
of changes to make to the default params dict.
The changes are applied by the extend_params
function. Its code deletes part of the
param_extensions, a side effect that didn't seem a
problem because it isn't used after the function
has run.
The bug
The bug was only with the part of the HTML that
got its data from the part of the param_extensions
dict that was deleted by extend_params. The class
with the bug set param_extensions when the field
is instantiated, as part of its parent form
definition.
My guess is that param_extensions was stored in
memory, as part of the form class, and reused
when the page refreshed. At that point,
extend_params had deleted part of its data,
causing the bug.
This command freezes all our requirements into the `.txt` files.
We want these files version controlled so that our builds are exactly
reproducible from environment to environment.
We only want PyUp to be checking for the dependencies we specify
directly, not any sub-dependencies.
By telling it to now look only at `.in` files we preserve this existing
behaviour.
In the past we've avoided using out-of-the-box solutions for Python
dependency resolution because a) they haven't been very mature and b)
we've had lots of issues with version conflicts. See [[1]], [[2]] for
details. Instead, we've been using a custom Python script that
under-the-hood runs `pip freeze` and saves the output to
`requirements.txt`.
This script works well for us, but it doesn't integrate well with other
tools. On the other hand [`pip-tools`](https://github.com/jazzband/pip-tools)
as of 2020 seems to be well-supported by its maintainers and other
tools; for instance, GitHub's automated update service
[Dependabot](https://dependabot.com) supports `requirements.in` files.
This commit replaces our `freeze-requirements` make command with
`pip-compile`.
The Digital Marketplace team have made this change and seem happy with
the results.
We’re going to move to using pip-tools for freezing requirements.
pip-tools uses `.in` files for the un-frozen list of requirements, and
then generates `.txt` equivalents.
This commit just copies our existing `.txt` files, keeping the same name
but giving them a `.in` extension ready for pip-tools to use.
jQuery.attr returns `undefined` if an element does not have an
attribute. We want an empty string, rather than the default of coercing
`undefined` to the string `'undefined'`.
The content length message was making the page jumpy and causing reflows
in three ways. This commit addresses each of those ways:
As the user scrolled
---
The footer went from fixed to sticky and the spacing around the message
changed. This change in spacing was needed so that the message looked
right in both contexts.
I think the best way to resolve this is to not use the sticky footer
when editing text message or broadcast templates.
On my 1440×900 screen I can fit a 5 fragment text message, plus the
‘will be charged as 5 text messages’ message, plus the save button.
Our top 10 screen resolutions according to our analytics are:
Position | Resolution | Percentage of users
---------|------------|--------------------
1 | 1920x1080 | 27.37%
2 | 1280×720 | 11.07%
3 | 1366×768 | 8.88%
4 | 1536×864 | 5.79%
5 | 1440×900 | 4.52%
6 | 1600×900 | 3.71%
7 | 1280×1024 | 3.10%
8 | 1680×1050 | 2.42%
9 | 1920×1200 | 2.33%
10 | 2560×1440 | 1.99%
When the page first loaded
---
The message is empty so takes up no space, then the javascript fires
and inserts the message, taking up a line of space.
This is resolved by making the empty message take up space with a
non-breaking space character.
When the user first typed
---
We previously didn’t show any message until the user started typing.
This meant that, with the above fix, there was a larger than normal
empty space between the textarea and the save button.
This is resolved by always showing the message, even when the user
hasn’t typed anything yet.
***
These are design decisions which made sense when the message was
displayed along side the button, but we’ve had to change now that the
message is above the button.
We feel that this is more appropriate because it’s part of the
information you’re agreeing to before you hit submit.
Sometimes users can missing information that doesn’t start left-aligned
to the column they’re interacting with.
It also makes it closer to the Design System component.
We’re keeping it in the sticky footer, so that it’s always visible no
matter where in the message you’re scrolled to (this means you won’t
have to edited to content then scroll down to check whether you’ve
made it fit).
The `<textarea>` should have its existing `aria-describedby` point to a
hint. Pointing to the label is bad practice as it would duplicate the
accessible name into the accessible description. It’s good not to have
that in the tests in case anyone copies the code elsewhere.
This looks tidy, and because of the sticky footer it means the message
is always visible, even if your template is quite long. So no matter
where you’re scrolled to in the template you don’t have to scroll to the
bottom to see the count update.
The endpoint that count characters should be pretty low-load because it
won’t talk to the database (unless, on the first request, the user and
service aren’t cached in Redis).
The response size is also very small, only one line of text wrapped in a
single `<span>`, so won’t be as CPU-intensive to render as a whole page.
Still, we don’t want to completely hammer the server if a user types
very quickly.
This commit adds some throttling, so that we wait until there’s a
certain amount of delay between keystrokes before firing off the request
to the backend.
I’ve set the delay at 150ms. At normal typing speed this makes the lag
feel fairly imperceptible – it feels like you get an updated count in
response to most keystrokes. It’s only if you really mash the keyboard
that the count won’t update until you take a breath.
This commit copies the same ARIA attributes that are added to the
character count component[1] in the GOV.UK Design System.
This means that screen reader users will hear the count message when
they stop typing.
1. https://design-system.service.gov.uk/components/character-count/
This commit adds some Javascript that makes AJAX requests as the users
changes the content of their template.
It then takes the content returned by the backend and inserts it in the
page.
Users sending text messages are sometimes unaware that long messages
will cost more.
Users sending broadcast messages need to be aware that there’s a
character limit, so they can take this into account when planning their
messages.
This commit adds an endpoint which counts the number of characters in
some template content, and returns a snippet of useful info about how
long the message is.
In subsequent commits we’ll be able to use AJAX to fetch this snippet as
the user types.
There’s a surprising amount of complexity in counting the length of
messages. So we’ll need to do this in Python because it would be too
convoluted to re-implement the length counting in client side code, let
alone ensuring it had parity with its Python equivalent.
As formatters we can use them in Jinja or Python code.
It also means we don’t need to import them every time we want to use
them – they’re always available in the template context.
For now this doesn’t remove the macros, it just aliases them to the
formatters. This gives us confidence that the formatters are working the
same way the old macros did, and reduces the diff size of each commit.
We have lots of functions for converting various types of data into
strings to be displayed to the user somewhere.
This commit collects all these functions into their own module, rather
than having them cluttering up `app/__init__.py` or buried amongst
various other things that have ended up in `app/utils.py`.
`app/utils.py` is a bit of a dumping ground for things we don’t have a
better place for.
We now have a place and structure for storing ‘model’ code (‘model’ in
the model, view, controller (MVC) sense of the word).
This commit moves the spreadsheet model to that place.
We added an extra, hidden, <input> to our /sign-in
and /register forms to stop Chrome's form
heuristics filling the fields in wrong.
See the commit that added it:
33b15cdec6
This removes it after testing with the following
Chrome/OS combinations and all working without the
hack:
- Chrome 87, Windows 7
- Chrome 86, Windows 7
- Chrome 86, Windows 10
- Chrome 85, Windows 7
- Chrome 80, Windows 10
- Chrome 81, Windows 7
- Chrome 52, Windows 7
- Chrome 68, Windows 7
- Chrome 78, Windows 8.1
- Chrome 86, Windows 8.1
These combinations were based on the most-used
versions recorded in our analytics for the last 3
months.
