I emailed the Geography team at the ONS:
> Hi geography team,
>
> I work on GOV.UK Notify, which is a service run by Government Digital Service (part of the Cabinet Office). I was given your email address by [redacted] who’s been helping answer some of my questions on the cross-government Slack.
>
> We’re using some of the boundary datasets from the Open Geography Portal, and mostly they’ve been excellent.
>
> In the abstract, the problem we’re trying to solve is, given a point outside an area, what is the minimum distance to a point within that area. So, for example, if a crow was somewhere in Cardiff, what’s the shortest distance it would have to fly to reach somewhere in the Bristol local authority district?
>
> We’ve noticed some problems with the data that means our calculations would be wrong. We’ve noticed this around Torquay, Norwich and Bristol. Here are some screenshots of Bristol, from the generalised and full resolution boundaries:
>
> The artefacts I’ve highlighted are closer to Cardiff than any actual part of the land area of Bristol. They are either:
> - in the sea
> - land that’s part of North Somerset
>
> I suspect that this is being caused by the process of clipping the actual region of Bristol (which, unusually, extends into the water) to the mean high water line.
>
> I’ve worked around this by filtering out any polygons that are smaller than ~7,500m². It’s a bit hacky because parts of the Scilly Isles start disappearing. That’s not a problem for what I’m working on, but it would be nice to not need the hack.
>
> So my questions would be:
>
> - Is there a better way to remove these artefacts than filtering by area?
> - Is there a plan to remove these artefacts from the data in future releases?
>
> Thanks in advance,
> Chris
They emailed back to say:
> Hi Chris
>
> Thank you for your enquiry.
>
> We have completed the amendments to the LAD MAY 2020 BFC and BGC boundaries as mentioned so you should be able to download them from the portal now.
>
> Hope this helps.
>
> Kind regards
> [redacted]
This commit brings in the files they’ve updated. We still have to do
some filtering (but now at a higher resolution) because they haven’t
fixed Norwich yet. I’ll email them separately about that.
This is a better name for the module because it’s:
- not just constants, there’s a method in here now
- only stuff to do with populations, not other kinds of constants
We need to give people a better feel for the consequences of
broadcasting an alert. We’ve seen in research that some users will
assume it is subscription based, or opt-in, rather than going to every
phone in the area.
I reckon that the most effective way to communicate this is to put some
numbers next to the areas, to give people an idea of how many people
will get alerted.
We can estimate how many phones are in an area by:
- taking the population of all electoral wards in that area
- multiplying it by the percentage of people who own an internet
connected phone[1]
The Office for National Statistics publish both these datasets.
The number of people who own an intenet connected phone varies a lot by
age. Since the population data for each ward is broken down by age we
can factor this in. Simplified, the calculation looks like this:
- take the _Abbey_ ward of _Barking and Dagenham_
- in this ward there are 26 people aged 80
- 40% of people over 65 have an internet-connected phone
- therefore 10 of these 80-year-olds would be likely to receive a
broadcast
- (repeat for all other ages)
These numbers won’t be exact, but should be enough to give people a feel
for the severity of what they’re about to do. We can see if they acheive
this aim in user research.
1. This is a proxy for the number of people who are likely to have a 4G
capable phone, because only 4G capable phones will be receiving
broadcasts to begin with
they're in both the local authority and County & Unitary Authority data
sets, so we should only add them once. Keep the LA one so the ctyua19
dataset is only used for counties
map lower tier local authorities (districts within rural counties) to
upper tier local authorities (administrative counties) using the new
ctyua19 dataset.
nb: unitary authorities (eg: Southwark) are both lower tier AND upper
tier. For this first pass they turn up twice in the db, eg Southwark is
in there as `lad20-E09000028` and `ctyua19-E09000028`. For now, the
electoral wards within Southwark are mapped to the pre-existing
`lad20-E09000028`. Both lad20 and ctyua19 have data sets in
broadcast_area_features.
We have a bunch of stuff for doing lat/long transformation in the
`BroadcastMessage` class. This is not a good separation of concerns, now
that we have a separate class for dealing with polygons and coordinates.
This commit does two things:
- uses our new polygon-simplifying library to process the polygons
before storing them, rather than processing them in real time
- stores only the polygons in the database, rather than the whole
GeoJSON feature, because we don’t need any of the other information
about the feature
It’s been superceded by the ‘Local’ library (formerly ‘Electoral wards
in the United Kingdom’).
The latter is better because:
- it’s covers all 4 nations, not just England and Wales
- it has electoral wards as well as local authorities which group them,
so there’s more flexibility when choosing an area to broadcast to
We’ve observed people using ‘national’ and ‘local’ during user research.
It has less tongue-twisting ambiguity than county vs country.
But we think that maybe just getting rid of ‘counties’ is enough to
disambiguate them. So this commit just takes the ‘local’ concept.
This commit also gives the libraries and areas new IDs, which means if
we want to rename them in the future it won’t be a breaking change.
Broadcasting is not a precise technology, because:
- cell towers are directional
- their range varies depending on whether they are 2, 3, 4, or 5G
(the higher the bandwidth the shorter the range)
- in urban areas the towers are more densely packed, so a phone is
likely to have a greater choice of tower to connect to, and will
favour a closer one (which has a stronger signal)
- topography and even weather can affect the range of a tower
So it’s good for us to visually indicate that the broadcast is not as
precise as the boundaries of the area, because it gives the person
sending the message an indication of how the technology works.
At the same time we have a restriction on the number of polygons we
think and area can have, so we’ve done some work to make versions of
polygons which are simplified and buffered (see
https://github.com/alphagov/notifications-utils/pull/769 for context).
Serendipitously, the simplified and buffered polygons are larger and
smoother than the detailed polygons we’ve got from the GeoJSON files. So
they naturally give the impression of covering an area which is wider
and less precise.
So this commit takes those simple polygons and uses them to render the
blue fill. This makes the blue fill extend outside the black stroke,
which is still using the detailed polygons direct from the GeoJSON.
It made for a good early demo to show how we could have different
libraries, but we’d don’t think there’s a strong user need for being
able to broadcast to a region of England.
Regions also have the problem that:
- they are ambiguous – both England and Scotland have a region called
‘South east’
- Northern Ireland doesn’t have formal regions
This commit removes the regions library.
