By using the simplified polygons instead of the full resolutions ones
we:
- query less data from SQLite
- pass less data around
- give Shapely a less complicated shape to do its calculations on
This makes it faster to calculate how much of each electoral ward a
custom area overlaps.
For the two areas in our tests:
Place represented by custom area | Before | After
---------------------------------|--------|--------
Bristol | 0.07s | 0.02s
Skye | 0.02s | 0.01s
Previously this was hidden away in an anonymous __init__.py file.
I did think about splitting the models into individual files, like
we do with the top-level models for the app. Since the models are
only imported in one place - i.e. are all used together - it didn't
seem worth the hassle, so I've kept them in one file.
We want to know how many phones are in a user-supplied polygon, so we
can show the impact of a broadcast, in the same way that we do when
users pick areas from our library.
We already know how many phones are in each electoral ward. But there
are challenges with an arbitrary polygon:
- where it does overlap a ward, the overlap could be partial
- it could overlap more than one ward
- finding out which wards it overlaps by brute force (looping through
all the wards and seeing which ones intersect with our polygon) would
be way to slow to do in real time
Instead we can use a data structure called an R-tree[1] to build an
index which provides a much, much faster way of looking up which
polygons overlap another. We can build this tree in advance and save it
somewhere, which means there’s a lot of computation we don’t need to do
in real time.
The R-tree returns a set of objects (ward IDs) which we can go and look
up in our library of electoral wards. These wards will be the ones that
might have some overlap with our custom polygon.
Once we have this small set of wards which might overlap our ward, we
can look at the size of the area of overlap (relative to the size of the
whole ward) and multiply that by the known count of phones in that ward
to get an approximation of the count of phones in the overlap area.
Summing these approximations give an estimate for the whole area of the
custom polygon.
1. https://en.wikipedia.org/wiki/R-tree
This allows MNOs to test delivery to multiple non-adjacent cells without
risk of sending a broadcast on the public network. This will also support
testing of multiple polygon geometries in a single message.
Test polygons are all non-UK (northern Finland).
Signed-off-by: Richard Baker <richard.baker@digital.cabinet-office.gov.uk>
If an area has a `count_of_phones` value of `0` it means we don’t have
data about the population.
This means we can’t do the maths to work out the estimated bleed. So we
should return the default amount of bleed of 1,500m instead, which is
something in between what we’d expect for a built up area and a rural
area.
This prevents us from giving unrealistically large or small bleed
estimates in case we have areas which are more dense or less dense than
the most/least dense areas we currently have.
Also means we don’t have to treat City of London as a special case.
There are basically two kinds of 4G masts:
Frequency | Range | Bandwidth
----------|-------------|----------------------------------
800MHz | Long (500m) | Low (can handle a bit of traffic)
1800Mhz | Short (5km) | High (can handle lots of traffic)
The 1800Mhz masts are better in terms of how much traffic they can
handle and how fast a connection they provide. But because they have
quite short range, it’s only economical to install them in very built up
areas†.
In more rural areas the 800MHz masts are better because they cover a
wider area, and have enough bandwidth for the lower population density.
The net effect of this is that cell broadcasts in rural areas are likely
to bleed further, because the masts they are being broadcast from are
less precise.
We can use population density as a proxy for how likely it is to be
covered by 1800Mhz masts, and therefore how much bleed we should expect.
So this commit varies the amount of bleed shown based on the population
density.
I came up with the formula based on 3 fixed points:
- The most remote areas (for example the Scottish Highlands) should have
the highest average bleed, estimated at 5km
- An town, like Crewe, should have about the same bleed as we were
estimating before (1.5km) – Pete D thinks this is about right based on
his knowledge of the area around his office in Crewe
- The most built up areas, like London boroughs, could have as little as
500m of bleed
Based on these three figures I came up with the following formula, which
roughly gives the right bleed distance (`b`) for each of their population
densities (`d`):
```
b = 5900 - (log10(d) × 1_250)
```
Plotted on a curve it looks like this:
This is based on averages – remember that the UI shows where is _likely_
to receive the alert, based on bleed, not where it’s _possible_ to
receive the alert.
Here’s what it looks like on the map:
---
†There are some additional subtleties which make this not strictly true:
- The 800Mhz masts are also used in built up areas to fill in the gaps
between the areas covered by the 1800Mhz masts
- Switching between masts is inefficient, so if you’re moving fast
through a built up area (for example on a train) your phone will only
use the 800MHz masts so that you have to handoff from one mast to
another less often
At the moment there are some areas which have:
- a `count_of_phones` value of `None`
- no sub-areas
This is wrong, but until we fix the data the phone counting code needs
to handle this.
This commit:
- adds the `or 0` in the right place (where it will catch these areas
with missing data)
- adds a test which checks these areas, and compares them to other kinds
of areas
This is a better name for the module because it’s:
- not just constants, there’s a method in here now
- only stuff to do with populations, not other kinds of constants
What was previously ward -> local authority is now a ward -> local
authority -> county. County only covers rural counties and not
metropolitan boroughs and other unitary authorities. Previously, there
was a page full of local authorities (unitary authorities and
districts), and each one of those would have a list of electoral wards.
However, now there are counties that contain a list of districts - so
this needs a new page - a checkbox for "select the county" and then a
list of links to district pages.
If you want to select multiple districts, you'll need to go into each
one of those sub-sections in turn and click select all.
Needed to tweak the query to retrieve the list of areas in a list for a
library. Previously, it just returned anything at top level (ie: didn't
have a parent). However, rural districts now have parents (the rural
counties themselves). So the query now returns "everything that isn't a
leaf node", or in more specific terms, everything that has at least
other row referring to it as a parent. So no electoral wards, since
they dont have any children, but yes to districts and counties.
We have a bunch of stuff for doing lat/long transformation in the
`BroadcastMessage` class. This is not a good separation of concerns, now
that we have a separate class for dealing with polygons and coordinates.
This commit does two things:
- uses our new polygon-simplifying library to process the polygons
before storing them, rather than processing them in real time
- stores only the polygons in the database, rather than the whole
GeoJSON feature, because we don’t need any of the other information
about the feature
It’s been superceded by the ‘Local’ library (formerly ‘Electoral wards
in the United Kingdom’).
The latter is better because:
- it’s covers all 4 nations, not just England and Wales
- it has electoral wards as well as local authorities which group them,
so there’s more flexibility when choosing an area to broadcast to
We’ve observed people using ‘national’ and ‘local’ during user research.
It has less tongue-twisting ambiguity than county vs country.
But we think that maybe just getting rid of ‘counties’ is enough to
disambiguate them. So this commit just takes the ‘local’ concept.
This commit also gives the libraries and areas new IDs, which means if
we want to rename them in the future it won’t be a breaking change.
It made for a good early demo to show how we could have different
libraries, but we’d don’t think there’s a strong user need for being
able to broadcast to a region of England.
Regions also have the problem that:
- they are ambiguous – both England and Scotland have a region called
‘South east’
- Northern Ireland doesn’t have formal regions
This commit removes the regions library.
If a library has lots of items then the first 3 should be shown, with
a count of how many more there are, for a total of 4 list items:
> a, b, c, and 23 more
If the library only has 4 items then all 4 should be shown, with
consistent use of conjunction and Oxford comma[1]:
> a, b, c, and d
This keeps the lengths of the examples nice and consistent.
1. We use an Oxford comma because it helps disambiguate when an area
itself has a comma or ‘and’ in it, for example ‘Armagh City, Banbridge
and Craigavon’
When you click through to the page for a library you see the available
areas in alphabetical order. The examples given for each library should
match this.
The given examples should match the choices offered when you visit the
next page. The choices offered on the next page are either the areas
(when a library is not grouped) or the groups (when a library is
grouped).
This commit makes the examples match the choices by excluding sub-areas,
ie those that have a grouping ID.
