We removed whitespace in the HTML of the copy to clipboard component
in https://github.com/alphagov/notifications-admin/pull/4236/files
When the Javascript on the page loads it re-renders the component,
using HTML which is embedded in the .js file.
This means we also need to apply the same change to the .js file
to remove any extraneous whitespace.
In user research we quite often saw people accidentally scroll the
branding preview `<iframe>` when trying to scroll the page.
This is suboptimal because they:
- were confused why the page wasn’t scrolling
- lost visibility of the branding they were trying to preview because
it scrolled outside the bounds of the `<iframe>`
In previous iterations of the classPersister, we
found issues with the implementation meant classes
it should have added back to elements were also
added to other elements. This adds tests for this
scenario to ensure it doesn't happen again.
Also includes changes to fix a linting error with
the JS which complained about a function being
defined in a loop while referencing variables in
the outer scope.
The assumption that the classes you want to
persist will always have parity with the elements
that have those classes, at that point, won't
always be true.
Because of that, this changes the way elements
with those classes are stored, to be in a map
between classes and the elements with them (at
that point).
Also includes an extra test for a scenario where
more than one updating component is in the page
with classes that need to persist through updates.
If the :has pseudo-class becomes available in the
browsers we support in future, the code this
branch/pull request adds will be redundant as its
behaviour will be possible only using CSS.
This adds comments to note this to the parts of
the code you'd need to remove.
Includes the following:
Guard for adding duplicate classNames
Stops the code that allows classNames to persist
across updates to the component HTML from adding a
className multiple times to the list of those to
persist. From this comment on the associated pull
request:
https://github.com/alphagov/notifications-admin/pull/4155#discussion_r804639058
Add comment explaining guard for operations on
elements no longer in the DOM
The value of this guard can be unclear and why it
is needed so we add a comment to explain this.
From this comment on the associated pull request:
https://github.com/alphagov/notifications-admin/pull/4155#discussion_r804697189
We can't guarantee that elements we stored a
reference to with `classesToPersist.remove` will
still exist so we need to guard against this.
Note: it checks for whether the node is still
attached to the DOM rather than whether it exists
because the standard way to delete a node just
detaches it from the DOM and relies on garbage
collection to delete it from memory.
The current updateContent JS replaces the in-page
HTML with the HTML from the server the first time
an AJAX request is fired, even if the HTML from
the server has no changes. This is because the
code that compares the two operates on two
different things:
The HTML in the page is the component HTML, with
all the data attributes and the partial HTML
(marked with the 'ajax-block-container' class) as
its first child:
```
<div data-module="update-content" data-url="...">
<div class="ajax-block-container">
...
</div>
</div>
```
The HTML from the server only contains the
partial:
```
<div class="ajax-block-container">
...
</div>
```
The diffing code just sees them as different at
the top level so replaces the page HTML with the
partial from the server. This means all subsequent
diffs are between partial HTML and partial HTML so
only update on actual changes.
These replace the component with the partial, as
part of the component initialising. This means all
code that runs on an AJAX response will only
compare like-for-like so will result in actual
changes (or none at all), not just swapping one
element out for another.
Note: this commit also removes the
aria-live="polite" from the ajax_block component.
It has always been overwritten by the first
response so never announces anything to assistive
technologies. Removing it makes this more clear.
Includes:
- JS to add a class to the heading when the link
is focused
- CSS to apply the enlarged focus style via a
selector which uses that class
- changes to the partial to hook in the JS to
track focus on links and to tell the
updateContent JS to persist the classes added
between updates to the HTML
Wrap the code that updates the HTML with changes
from the server with code that stores and
re-applies specified classes.
This is to allow other JS to add classes which
change the visual state of the HTML without them
being considered by the code that diffs our
in-page HTML against that from the server.
They are called classesToPersist because this
should make the visual state they create persist
between updates.
Includes the addition of tests for updateContent
that cover the addition/deletion of elements so we
can write a test for classNames persisting through
updates. The existing tests only cover updates
that change the content of elements. Just adding
the test for these changes to those would simulate
a scenario that doesn't exist in the app. Writing
extra tests for the kind of updates these changes
act on keeps them in line with the app code.
We added domdiff to replace the DiffDOM library
here:
87f54d1e88
DiffDOM had updated its code to be written to the
ECMAScript 6 (ES6) standard and so needed extra
work to work with the older browsers in our
support matrix. This was recorded as an issue
here:
https://www.pivotaltracker.com/n/projects/1443052/stories/165380360
Domdiff didn't work (see below for more details)
so this replaces it with the morphdom library.
Morphdom supports the same browsers as us and is
relied on by a range of large open source
projects:
https://github.com/patrick-steele-idem/morphdom#what-projects-are-using-morphdom
It was tricky to find alternatives to DiffDOM so
if we have to source alternatives in future, other
options could be:
- https://github.com/choojs/nanomorph
- https://diffhtml.org/index.html (using its
outerHTML method)
Why domdiff didn't work
Turns out that domdiff was replacing the page HTML
with the HTML from the AJAX response every time,
not just when they differed. This isn't a bug.
Domdiff is bare bones enough that it compares old
DOM nodes to new DOM nodes with ===. With our
code, this always results to false because our new
nodes are made from HTML strings from AJAX
response so are never the same node as the old
one.
Previously all our error messages had the class `error-message`.
Where we are using the components from GOV.UK Frontend they have the
class `govuk-error-message`.
This makes the code which jumps focus to the first error work in both
cases.
In 674c27a693 we updated the autofocus
Javascript to be compatible with GOV.UK Frontend textboxes, which have
the `data-module` attribute set on the `input` element, rather than on
a wrapper element.
However we still have some `<textarea>`s and `<input>`s which haven’t
moved to GOV.UK Frontend and therefore aren’t getting picked up by the
Javascript which is supposed to focus them.
This commit makes the Javascript work with both kinds of textbox, which
is needed until we move entirely to GOV.UK Frontend.
This is currently spelt incorrectly though it
seemed to work nonetheless. Can only assume this
is a common error, for this attribute or all
attribute names, so browsers work it out.
This makes the spelling match the spec:
https://www.w3.org/TR/wai-aria/#aria-describedby
A while ago diffDOM moved its code to use ES6
modules and started using various language
features specific to ES6. These two things
happened independently btw.
The result of this is that the version of diffDOM
suitable for our build pipeline, structured as an
immediately invoked function evocation (IIFE),
now requires polyfills of some ES6 features to
work in the older browsers we support, like IE11.
It's also worth noting that in the move to ES6
the maintainers of diffDOM have adopted a process
whereby users who need to support older browsers
now have to add polyfill code for any ES6 features
they choose to use.
This commmit proposes a move to the domdiff
library instead because:
- it runs on all javascript runtimes with no
polyfills
- it is 2KB instead of diffDOM's 25KB
Domdiff takes a different approach to diffDOM, in
that it compares existing nodes and new nodes and
replaces the existing ones with the new ones if
there are differences. By contrast, diffDOM will
make in-place changes to nodes if there are enough
similarities. In other words, in most situations,
diffDOM won't change the node in $component
whereas domdiff will.
Because of this, I've had to change the
updateContent.js code to cache the data-key
attribute's value so we don't lose access to it by
overwrite the $component variable with a different
jQuery selection.
since we are hard-coding a generic error message on the front-end, we
have no need to do anything on the back end. This is also nice as it
standardises the two flows to behave more like each other (rather than
previously where one would `flash` an error message and the other would
return CBOR for the js to decode).
Note that the register flow returns 400 while the auth flow returns 403.
The js for both just checks `response.ok` so will handle both. The JS
completely discards any body returned if the status isn't 200 now.
turns out that we're only using errorBanner with a static message, and
it's also full of rich html content. This means that it's probably
better to put it in the html templates with other content, rather than
hidden away in js files if we can help it.
Since there are two places, had to dupe the error message but i think
that's fine as i don't anticipate this error message being used in
significantly more places.
making it a string is a bit gross and means we don't get nice syntax
highlighting on it, but as it needs to be passed in to a jinja macro
that's the way it has to go unfortunately.
the banner is a nicer user experience, and consistent with how we
display errors elsewhere in notify. For now pass through the error
message from JS, but we'll probably want to change that since the erorr
messages themselves are often a bit cryptic and unhelpful
this ensures it's reusable by other components, and easier to unit test
by isolating the separate concerns
note: this is not in Modules since that's designed for classes that are
then bound to an element in the DOM as indicated by a data-module
attribute. This will just live at the window.GOVUK level since we want
there to only ever be one `.banner-dangerous` warning.
the next url comes from sign in via a query param, and needs to go to
the POST /webauthn/authenticate endpoint. That endpoint logs the user
in and returns the redirect to the browser, and will take the next from
the request query params to get there.
also moving the window mocks to beforeEach/afterEach ensures that
promise callbacks from previous tests aren't still associated in future
tests to ensure good test isolation.
unfortunately i couldn't get mocking location for a single js test to
work, but by changing the global config i was able to add some query
params that i can expect to be passed through. Don't love this at all
but not quite sure of a good way round this. I think we're not
practicing very good hygiene and best practices with our mocking and
it's really confounding me here.
notably i had to change `window.location = foo` to
`window.location.assign` so that i could have something to spy on with
jest. mocking sucks. Otherwise this is pretty similar to the
registerSecurityKey.test.js file.
the js fetch function is really not designed to work with 302s. when it
receives a 302, it automatically follows it and fetches the next page.
This is awkward because I don't want js to do all this in ajax, I want
the browser to get the new URL so it can load the page.
A better approach is to view the admin endpoint as a more pure API: the
js sends a request for authentication to the admin app, and the admin
app responds with a 200 indicating success, and then a payload of
relevant data with that.
The relevant data in this case is "Which URL should I redirect to", it
might be the user's list of services page, or it might be a page telling
them that their email needs revalidating.
the js `fetch` function will follow redirects blindly and return you the
final 200 response. when there's an error, we don't want to go anywhere,
and we want to use the flask `flash` functionality to pop up an error
page (the likely reason for seeing this is using a yubikey that isn't
associated with your user). using `flash` and then
`window.location.reload()` handles this fine.
However, when the user does log in succesfully we need to properly log
them in - this includes:
* checking their account isn't over the max login count
* resetting failed login count to 0 if not
* setting a new session id in the database (so other browser windows are
logged out)
* checking if they need to revalidate their email access (every 90 days)
* clearing old user out of the cache
This code all happens in the ajax function rather than being in a
separate redirect, so that you can't just navigate to the login flow. I
wasn't able to unit test that function due how it uses the session and
other flask globals, so moved the auth into its own function so it's
easy to stub out all that CBOR nonsense.
TODO: We still need to pass any `next` URLs through the chain from login
page all the way through the javascript AJAX calls and redirects to the
log_in_user function
The flow of the code is roughly as follows:
user clicks button on webauthn page
js sends GET request
python reads GET request, sets up login challenge
python returns login challenge in response
js reads GET response, passes login challenge to browser
browser asks user to touch yubikey
browser returns yubikey challenge response data to js
js sends POST request with yubikey challenge response data
python reads yubikey challenge and compares with users creds from db
if its a match, python signs user in
The login challenge is a PublicKeyCredentialRequestOptions: [1]
The browser function we call is navigator.credentials.get(): [2]
The response to the challenge from the browser is a PublicKeyCredential: [3]
The python server does all the work setting those up and tearing them
back down again (and checking them against the values we have stored in
the database), but we need to do work to convert them to-and-from CBOR.
[1] https://developer.mozilla.org/en-US/docs/Web/API/PublicKeyCredentialRequestOptions
[2] https://developer.mozilla.org/en-US/docs/Web/API/CredentialsContainer/get
[3] https://developer.mozilla.org/en-US/docs/Web/API/PublicKeyCredential
if user has `webauthn_auth` as their auth type, then redirect them to an
interstitial that prompts them to click on a button which right now just
logs to the JS console, but in a future commit will open up the webauthn
browser prompt
content is unsurprisingly not final.
Previously we would raise a 500 error in a variety of cases:
- If a second key was being registered simultaneously (e.g. in a
separate tab), which means the registration state could be missing
after the first registration completes. That smells like an attack.
- If the server-side verification failed e.g. origin verification,
challenge verification, etc. The library seems to use 'ValueError'
for all such errors [1] (after auditing its 'raise' statements, and
excluding AttestationError [2], since we're not doing that).
- If a key is used that attempts to sign with an unsupported
algorithm. This would normally raise a NotImplemented error as part
of verifying attestation [3], but we don't do that, so we need to
verify the algorithm is supported by the library manually.
This adds error handling to return a 400 response and error message
in these cases, since the error is not unexpected (i.e. not a 500).
A 400 seems more appropriate than a 403, since in many cases it's
not clear if the request data is valid.
I've used CBOR for the transport encoding, to match the successful
request / response encoding. Note that the ordering of then/catch
matters in JS - we don't want to catch our own throws!
[1]: 142587b3e6/fido2/server.py (L255)
[2]: c42d9628a4/fido2/attestation/base.py (L39)
[3]: c42d9628a4/fido2/cose.py (L92)
This adds Yubico's FIDO2 library and two APIs for working with the
"navigator.credentials.create()" function in JavaScript. The GET
API uses the library to generate options for the "create()" function,
and the POST API decodes and verifies the resulting credential. While
the options and response are dict-like, CBOR is necessary to encode
some of the byte-level values, which can't be represented in JSON.
Much of the code here is based on the Yubico library example [1][2].
Implementation notes:
- There are definitely better ways to alert the user about failure, but
window.alert() will do for the time being. Using location.reload() is
also a bit jarring if the page scrolls, but not a major issue.
- Ideally we would use window.fetch() to do AJAX calls, but we don't
have a polyfill for this, and we use $.ajax() elsewhere [3]. We need
to do a few weird tricks [6] to stop jQuery trashing the data.
- The FIDO2 server doesn't serve web requests; it's just a "server" in
the sense of WebAuthn terminology. It lives in its own module, since it
needs to be initialised with the app / config.
- $.ajax returns a promise-like object. Although we've used ".fail()"
elsewhere [3], I couldn't find a stub object that supports it, so I've
gone for ".catch()", and used a Promise stub object in tests.
- WebAuthn only works over HTTPS, but there's an exception for "localhost"
[4]. However, the library is a bit too strict [5], so we have to disable
origin verification to avoid needing HTTPS for dev work.
[1]: c42d9628a4/examples/server/server.py
[2]: c42d9628a4/examples/server/static/register.html
[3]: 91453d3639/app/assets/javascripts/updateContent.js (L33)
[4]: https://stackoverflow.com/questions/55971593/navigator-credentials-is-null-on-local-server
[5]: c42d9628a4/fido2/rpid.py (L69)
[6]: https://stackoverflow.com/questions/12394622/does-jquery-ajax-or-load-allow-for-responsetype-arraybuffer
This adds a new platform admin settings row, leading a page which
shows any existing keys and allows a new one to be registered. Until
the APIs for this are implemented, the user API client just returns
some stubbed data for manual testing.
This also includes a basic JavaScript module to do the main work of
registering a new authenticator, to be implemented in the next commits.
Some more minor notes:
- Setting the headings in the mapping_table is necessary to get the
horizontal rule along the top (to match the design).
- Setting caption to False in the mapping_table is necessary to stop
an extra margin appearing at the top.
The fieldset that wraps the collapsible checkboxes
has an aria-describedby to make the summary its
accessible description.
This needs to point to the id of the summary but
the summary didn't have one.
These changes add the id and fix a fixture in the
tests for this module.
The fullscreenTable component has 2 layers to the
table you see onscreen:
1. the actual data table
2. a clone, with only the first column showing,
that sits on top so the row numbers stay in
place while you scroll
Table 1. has an id attribute on its caption. The
region wrapping it has an aria-describedby
attribute with the id as its value. This makes the
caption the description for the region.
This isn't needed for the clone and
makes the HTML invalid because ids should be
unique.
This removes the id from the cloned table.
Includes changing the code so that the radios
aren't split into two columns in the HTML present
when the page loads. This layout is now added by
the JS.
After talking with the reviewer, it was decided
that:
1. the JS could do with some comments to explain
its structure and what various functions do
better
2. some CSS selectors in the tests don't need to
be as complex and simplifying them makes the
test easier to read
Makes focus shift to the first time in the range
when you select a day.
Also rewrites the code for controlling focus so it
explains itself better, now it has different
settings.
All buttons that open or close a region of the
component should have aria-expanded attributes to
show:
- they have that control
- the state of the region
jQuery.attr returns `undefined` if an element does not have an
attribute. We want an empty string, rather than the default of coercing
`undefined` to the string `'undefined'`.
The endpoint that count characters should be pretty low-load because it
won’t talk to the database (unless, on the first request, the user and
service aren’t cached in Redis).
The response size is also very small, only one line of text wrapped in a
single `<span>`, so won’t be as CPU-intensive to render as a whole page.
Still, we don’t want to completely hammer the server if a user types
very quickly.
This commit adds some throttling, so that we wait until there’s a
certain amount of delay between keystrokes before firing off the request
to the backend.
I’ve set the delay at 150ms. At normal typing speed this makes the lag
feel fairly imperceptible – it feels like you get an updated count in
response to most keystrokes. It’s only if you really mash the keyboard
that the count won’t update until you take a breath.
This commit copies the same ARIA attributes that are added to the
character count component[1] in the GOV.UK Design System.
This means that screen reader users will hear the count message when
they stop typing.
1. https://design-system.service.gov.uk/components/character-count/
This commit adds some Javascript that makes AJAX requests as the users
changes the content of their template.
It then takes the content returned by the backend and inserts it in the
page.
The previewPane JS used selectors that targeted
the old form of radios HTML.
The JS tests also contained selectors like this
and fragments of HTML, used for fixtures, modelled
on the old radios HTML.
