I emailed the Geography team at the ONS:
> Hi geography team,
>
> I work on GOV.UK Notify, which is a service run by Government Digital Service (part of the Cabinet Office). I was given your email address by [redacted] who’s been helping answer some of my questions on the cross-government Slack.
>
> We’re using some of the boundary datasets from the Open Geography Portal, and mostly they’ve been excellent.
>
> In the abstract, the problem we’re trying to solve is, given a point outside an area, what is the minimum distance to a point within that area. So, for example, if a crow was somewhere in Cardiff, what’s the shortest distance it would have to fly to reach somewhere in the Bristol local authority district?
>
> We’ve noticed some problems with the data that means our calculations would be wrong. We’ve noticed this around Torquay, Norwich and Bristol. Here are some screenshots of Bristol, from the generalised and full resolution boundaries:
>
> The artefacts I’ve highlighted are closer to Cardiff than any actual part of the land area of Bristol. They are either:
> - in the sea
> - land that’s part of North Somerset
>
> I suspect that this is being caused by the process of clipping the actual region of Bristol (which, unusually, extends into the water) to the mean high water line.
>
> I’ve worked around this by filtering out any polygons that are smaller than ~7,500m². It’s a bit hacky because parts of the Scilly Isles start disappearing. That’s not a problem for what I’m working on, but it would be nice to not need the hack.
>
> So my questions would be:
>
> - Is there a better way to remove these artefacts than filtering by area?
> - Is there a plan to remove these artefacts from the data in future releases?
>
> Thanks in advance,
> Chris
They emailed back to say:
> Hi Chris
>
> Thank you for your enquiry.
>
> We have completed the amendments to the LAD MAY 2020 BFC and BGC boundaries as mentioned so you should be able to download them from the portal now.
>
> Hope this helps.
>
> Kind regards
> [redacted]
This commit brings in the files they’ve updated. We still have to do
some filtering (but now at a higher resolution) because they haven’t
fixed Norwich yet. I’ll email them separately about that.
We filter out very small polygons from the original data to remove
glitches. These glitches are caused by trying to subtract the water from
a polygon that includes some land and some water, but using two
different definitions or resolutions of mean high water line.
If we don’t do this then we end up with a bunch of very small polygons
which lie far outside the understood area of a place, causing large
overspill.
We need to increase the threshold for this process because we’re still
seeing this problem around Bristol and Norwich.
This does mean we lose a few very small polygons in places like Shetland
and the Scilly Isles, but not in such a way that we would avoid
broadcasting to them (because they’d still be caught by the
simplification and overspill).
Since the key relies on visual association between the shapes on the
maps and the styling of the key, it won’t work for non-visual users.
An alternative way of giving them the same information is by providing
the size of the area numerically.
We have a bunch of stuff for doing lat/long transformation in the
`BroadcastMessage` class. This is not a good separation of concerns, now
that we have a separate class for dealing with polygons and coordinates.
This commit does two things:
- uses our new polygon-simplifying library to process the polygons
before storing them, rather than processing them in real time
- stores only the polygons in the database, rather than the whole
GeoJSON feature, because we don’t need any of the other information
about the feature
Simplifying polygons means reducing the number of points used to render
them. This commit implements simplification such that, for any given
input polygons, the combined point count of the simplified polygons is
less than 100.
When simplifying the polygons we are trying to get the smallest number
of points while meeting these two rules:
1. No part of the area the user has chosen can be cut off
2. The area of the simplified polygon should be as small as possible
This commit introduces two techniques we weren’t using before:
1. Dilating and eroding the area to fill in concave details of the
shape, like inlets and harbours[1]
2. Making the simplification threshold proportionate to the perimeter of
all polygons, so bigger and crinklier polygons get more
simplification applied
It also shows the estimated bleed as a separate polygon. This lets us
make it bigger (so it’s more closer the the approximate bleed) without
having to send a bigger area to the CBC and compounding the amount of
actual bleed.
1. Inspired by this blog post about ‘removing the crinkley bits’ from
Vancouver Island:
http://blog.cleverelephant.ca/2010/11/removing-complexities.html
