Why adding uncertainty into regional GVA is a good thing (maybe)
What’s this doc?
This is a ‘chunk’ analysing what difference the inclusion of uncertainty could make to UK regional GVA analysis, by taking error rates from the Annual Business Survey and applying them to regional output.
Chunks are designed as small elements in self-contained projects hosted on github. This project is here.
Here’s a blog post [update when final] on the rationale for chunks. Briefly: less finished than working papers, more solid than a blog post, meant to be good for test / learn / iterate / imperfection / open feedback.
See here on how to comment / feed back - any input gets an acknowledgement on the github site (unless you ask not to be).
Headlines
Including uncertainty in UK regional GVA data leads to quite different, useful ways of thinking that aren’t possible with point estimates. It allows filtering signal from noise.
The ONS rules out error rates for regional GVA as ‘too complex’ - which it is, within the restrictions of a national accounts framework (globally agreed so extremely slow to change) (Coyle 2014). But it’s quite possible to test methods outside those restrictions.
This report applies UK Annual Business Survey uncertainty rates to region-by-industry GVA data to explore what the impact of uncertainty could be.
It argues uncertainty is powerful because it short-circuits many unhelpful ways of thinking about growth and industrial strategy, including spurious rank-building. Error bars direct us back towards our regions, what we concretely know about them, and how that knowledge is built into the choices and structures we make.
This first piece provides tools for showing what difference uncertainty can make, and asks what this might mean for regional economic thinking, with the aim of seeking feedback for a follow up article asking: “If we accept uncertainty into regional growth data, what are the decision-making implications?”
Resources/links
There are three interactive web pages for each of the GVA sources discussed below. I’ll refer back to these with links.
Chained volume GVA: What is the growth signal if error rates are included in regional data?
Current prices GVA: What happens to location quotients if error rates are included?
Triangle plot viewer for both growth slopes of chained volume GVA and LQ differences.
The repository for this project is here. This includes all code (which mostly downloads data directly to be processed) and writing, as well as a record of any Claude-Code conversations used in development. See this draft set of guidelines on how to clearly keep one’s own work separable from LLM use, treating it no differently to any other source (with some complications).
A section at the end explains how LLMs were (minimally) used in this article and points to other sources outlining how they were (more extensively) used in code production.
Introduction: how to avoid chasing vampires off cliff edges
In the movie ‘The Lost Boys’, David and his west-coast vampire buddies race motorbikes across a misty landscape. Michael (as yet unaware of the blood-drinking habits of his new associates) tries to keep up with them. David eggs Michael on - but then Michael spots the faint beam of a lighthouse, puts two and two together and barely avoids hurtling over a cliff edge.
One might say David was claiming ‘incredible certitude’ (Manski 2020) about the cliff’s location. “Don’t worry Micheal, there’s no cliff. Why would I be racing you otherwise? It’s miles away. Definitely.”
Had Michael not seen the lighthouse through the fog, he may have made a type II error: believed David’s false negative - “no cliff here” - and raced on. (See Table 1 for a full type I / II error breakdown of the scene.)
The fog itself is information. Without David’s certitude, Michael would do the the same as the rest of us - slow down, check his surroundings and steer more carefully.
I am of course going to apply this clunky metaphor to regional economic policy. Decisions are being constantly made on the basis of David-level certitude. But what are the implications of that for steering through the economic fog? What if we took the fog seriously? The issue is already well-explored (see below), but there has been little attempt to measure the fog and think through its policy effects at UK regional level (that I have found, let me know if I’ve missed anything).
This is especially vexatious for place-based policies, including industrial strategy. As devolution deepens, the impact of spurious accuracy will only increase. Uncertainty points us back towards what we really know about our economies - and that draws attention to the systems we use to get that knowledge.
Cited reasons for the dominance of single-data-point estimates usually come down to the complexity of identifying uncertainty in a national accounts framework. It’s true that the scale of the task is intimidating and onerous. But we don’t have to jump straight in at the deep end. Here, I argue it’s useful to explore the implications using some plausible, evidence-based ‘if/then’ scenarios about regional economic uncertainty.The results are imperfect, but arguably some error in our error is better than false certitude.
From that, it should be possible to iteratively explore policy implications.
In this opening piece, I lay out the problem with ‘incredible certitude’ (as others have done before) and then present two examples of what applying uncertainty to regional GVA data could look like.
I then make some suggestions and ask some questions about what the policy and decision-making implications could be. I’ll leave those questions open to try and get feedback for the next chunk of this work: fleshing out how uncertainty could affect regional decision-making.
A quick note on attitude
A few words on the spirit of this work, going back to an old argument about how we should approach criticism of economic ideas. I’ve just been re-reading a 2023 post by Richard Murphy about imputed rent (the rent that would be paid if owner-occupier houses were rented). The post starts with a ‘gotcha’ claim: “10% of GDP is made up – it simply does not exist in the real world”. Just another example, he says, showing how ridiculous national accounts are.
A commenter replies:
I really do wish you’d dial back the language a bit! GDP are an honest attempt to do something that is conceptually and technically very difficult – and there are honest debates about what to include and what not to include.
I want to mirror that respect for work done by much smarter people than me, in the ONS and other places. The ONS has come under immense pressure (example below) while doing phenomenal work with less money and a much harder survey landscape.
Getting away from perfect sums
National accounts are a powerful, mature system that has grown to become an essential part of global governance structures. As the latest SNA (2025) front page says, the goal is to:
“… ensure the compilation of internationally comparable national accounts statistics according to best practice and in a consistent way, allowing policy makers to benchmark their economies.”
National accounts are just that - national. Methods are designed around national estimates. Whatever issues those estimates have apply equally to smaller geographies - but re-purposing national accounts methods for UK sub-regions introduces new problems.
Estimating output for sub-regions is a top-down process (Black et al. 2025) that divides national values across the UK so they sum correctly. As the ONS say in their breakdown of “observed/estimated/modelled components of regional GVA”:
“Each of the components used in the measurement of regional gross value added (GVA) starts with a value for the UK as a whole, which is taken from the latest UK National Accounts Blue Book dataset. We then use the most appropriate available regional indicator to allocate the national total to parts of the UK, in a top-down hierarchical process. In this way we ensure that all of the regions sum to the published UK total and all sub-regions sum to their respective region total.”
This top-down approach is an unavoidable outcome of the point-estimate accounting framework it is part of. The ONS use “hundreds of input datasets to represent individual components of GVA” to achieve this accounting balance - in a separate regional GVA methods quality document, they say of this:
“The complex process by which GVA estimates are produced means that it is not currently possible to define the accuracy of the estimates… for example, through their standard errors. Therefore, the reliability of the estimates is measured by the extent of revisions.”
Black et al. (2025) discuss many of the problems and challenges with this complexity. But it isn’t the complexity itself that makes uncertainty estimates impossible. ONS statisticians would have no problems navigating that challenge. It is the complexity that the constriction of national accounts imposes. Chiefly, as with any accounts, everything has to sum correctly - the ledger of inputs and outputs must all balance exactly, across all places and sectors. Outside of that framework, it would be quite possible to examine what error rates were like in any administration or survey sources.
Why does this matter? Because policy action is taken on the basis of this certitude. Coyle (2017) digs into this issue (open PDF here), discussing the long history of the smallest GDP shifts having huge political effects. Here’s a recent example from November 2025. Ostensibly, the UK grew by exactly 0.1% in the three months to November 2025. An earlier 0.1% contraction led to predictably mature responses in the press (BBC example), with the shadow chancellor accusing the government of having “misled the British public” over what is almost certainly statistical noise.
As Coyle explains, accounting certainty combines with political demand for precision (however spurious) to keep us stuck here. But as she says:
“… neither the degree of underlying uncertainty nor the everyday practice of ignoring it seems sustainable, even though this situation has lasted for decades.” (Coyle 2017, p.228)
Attempts have been made to move away from this spurious precision. The ONS has experimented with some uncertainty, but it isn’t derived from the data itself. It takes this form: “How much have revisions to official numbers moved in the past? What bounds does that imply?” Figure 1 is an example of their answer (source). But this doesn’t explicitly include any actual sample error.
Work has also been done recently on public perceptions of national GDP uncertainty (Galvão and Mitchell 2024) (online here) which directly tested people’s understanding of - and reaction to - uncertainty. Using a variety of error visualisations and approaches, Galvão and Mitchell found strong support for explicitly including well-designed uncertainty information: it increases insight and does not affect trust. Not only do people understand why point estimates can be misleading and why uncertainty can be less so, they find that explicit inclusion of uncertainty can innoculate against confusion:
“Absent communication of data uncertainty, the public’s probabilistic perceptions of GDP data uncertainty are dispersed and inaccurate. When the public is treated with quantitative communication tools, we find that the public’s perceptions become better aligned with objective estimates of data uncertainty.”
They note how this can affect policy indirectly through public expectations.
But if policymakers are affected in the same way, the implications are wider: those ‘dispersed and inaccurate’ perceptions may feed directly into steering regional economies, creating potential for following-vampire-over-cliff levels of decision problems.
Adding error rates helps because it short-circuits those inaccurate ways of thinking. For example, using national accounts methods lends itself to making ranks of places and perceiving regional economics as a who’s up / who’s down jostle for position (a problem Professor Richard Harris has pointed out also applies to university rankings, where he shows it is equally spurious).
This competitive view has deep roots in national accounts approaches, originating as they do in managing wartime economies. This can be seen, for example, in how national accounts methods were used by the CIA to model the U.S/ Soviet economies as part of Cold War strategy. The stakes might not be quite so high in competition between Yorkshire and Lancashire, but point estimates make this kind of thinking difficult to avoid.
There are counter-arguments that point estimates are the only plausible tool in many policy areas such as funding allocation (see the final section below). I am not trying to suggest anything lacking uncertainty is invalid. But equally, why continue to rely solely on point estimates if we don’t need to? As discussed below, including uncertainty points policy toward iterative, slow-test improvements - careful steering in the fog.
Making a data-driven guess about the level of regional GVA uncertainty
This section presents the method and results for making a data-driven educated guess at the level of uncertainty in UK region-by-sector GVA numbers. The ONS region by industry GVA data is used for the central estimate numbers.
The Annual Business Survey (ABS) is used to get uncertainty values. It is an ideal source for reasonable confidence bounds around regional/sectoral GVA data. It is designed to capture GVA at a reasonable resolution, and - for the sectors it covers - gets its data direct from firms. It can be matched cleanly against the region by industry data’s ITL1 geographies and most 2-digit sectors (once it is aligned to the ONS’ bespoke SIC groupings in the region-by-industry data).
The ABS is also the most important input into regional GVA calculations. As the ONS say in their detailed breakdown of “observed/estimated/modelled components of regional GVA”:
“Of all the data sources used in regional GVA, the ABS has the greatest overall impact, representing around 71% of GVA(P) and 22% of GVA(I)… It includes elements corresponding to all three of the categories of data we wish to analyse: directly collected from businesses operating in a single region; weighted to represent non-sampled businesses; and apportioned to regions from UK-wide company information.”
Using the ABS means there is much less reliance on some of the more heroic assumptions that go into other aspects of regional GVA - though it isn’t without its own assumptions, a key one being that, for firms with multiple sites across regions, GVA is allocated in proportion to site employment count geographically. But it is still survey-based, making it a perfect source for this section’s question: if the regional GVA data had the same error rates as the ABS, what would it look like and what would the implications for analysis and policy be?
Note, single-point GVA values in the ABS can be quite different to the official region-by-industry GVA. Figure 7 summarises the average percent difference per sector between these two data sources (where single-sector matches exist). This figure is a visual reflection of the differences arising from the top-down national accounts balancing process, compared to survey data; understanding the differences would require a clearer view into the complexity the ONS cite as the barrier to ever including error rates.
(The difference in some sectors are more easily explainable than others, especially mostly public ones. For example, health is 92% lower in the ABS, as the region-by-industry data includes a proxy for public sector output, where GVA is assumed to be just regional job count multiplied by average wage, balanced against national totals. It’s thus much higher than the purely ‘survey data from private health firms’ ABS data).
Why graft the uncertainty from the ABS onto the region-by-industry data, then, rather than just examine ABS GVA? To carry out an ‘if/then’ exercise that lets us compare to the GVA numbers we already use for policy analysis, and ask how uncertainty could change their meaning.
Is this a valid exercise? I”m arguing yes, on the basis of two things. First, there is definitely uncertainty in the GVA data. An educated guess at error rates may be smaller or larger than the true uncertainty, but including it is very likely more accurate than the alternative - working with point estimates as if they were exact, true values.
Second, seeing what difference error makes provides a way to explore its implications. Incredible certitude cuts off that chance, and as mentioned lends itself to horse-race / rank style thinking that may be serving us poorly. I also argue below that it can contribute to focusing minds where they should be - improving regional knowledge systems, not relying purely on handed down data.
Another argument against doing this is straightforwardly methodological, and that has two parts. First, the statistical approach used here is too basic. I’d argue that’s fine for this stage, where the goal is to explore what uncertainty might mean for regional policy. If the principle stands, the statistics can always be improved. As already mentioned, the ONS itself has a wealth of expertise that could achieve this, alongside partners testing the role of uncertainty in policy. (The code includes options for testing both Newey West and AR(1) time series approaches; see below for other possible additions.)
Second, this approach does nothing to draw on the complexity of national accounts production. That is part of the point, however. If including uncertainty is useful, it should be possible to refine our understanding of that uncertainty, including other sources used in national accounts balancing. But the task is not to try and re-create national accounts approaches, which have very different purposes. If we want to know what the landscape of regional output really is, that may require its own, more region-friendly, approach.
Method for using the Annual Business Survey
The publically available ABS sources are broken into the central estimate data and error data1 giving GVA values and standard errors at ITL1 geography level and 2-digit SIC sectors. Here, I convert GVA and standard errors from the ABS to coefficients of variation (proportion of error) and then apply them to the regional/sectoral GVA national accounts data at the same geographical scale, and where sectors are present in both sources.
The region-by-industry GVA data uses bespoke groupings of SIC sectors (different groupings for each geography level, to keep to disclosure requirements). The ABS has single-digit SICs, for those sectors in the survey. In order to match the small number of grouped sectors to the ABS, each sector group (such as SIC sectors 5 to 8, covering four of the five mining/quarrying/fossil fuels sectors in SIC section B) is used to get an average coefficient of variation, weighted by the sector’s GVA value from the ABS. There are 73 sectors that match once ABS sector categories are processed in this way. Only years available across both sources are used - 2012 to 2023.
Below, I’ll explore the implications of these error rates when applied both available types of regional GVA data:
- Chained volume (CV) data that aims to capture ‘real’ output changes over time. With this, we can ask, “Did this sector grow or shrink in this place?”
- Current prices (CP) data, which (being prices as they were in the year they were counted) can be summed, and used to calculate location quotients (LQs) - how concentrated a sector is in each place. This provides a good sense of the how the UK’s economic structure changes. We’ll test adding error rates to LQs.
Chained volume GVA: growth over time
This section asks how much the growth signal changes for individual sectors in specific ITL1s if error rates are included. This interactive web page provides a way to look at the results, for individual sectors across all twelve ITL1 zones.
Chained volume (CV) data is used to assess real economic change over time, accounting for inflation and quality changes in goods and services. The ONS calculate separate deflators for each sector within each region, with a single year set as the point where the chained volume and ‘prices that year’ amounts are the same. (How deflator calculations work and their potential effect on uncertainty is a vital topic for a later article.) Other years are adjusted relative to that, for each sector/place combination. As a result, each sector/place time series must be treated separately and -unlike the current prices data - cannot be summed.
The assumptions used to add the ABS error rates to this data (on top of the if/then above) are: 95% confidence intervals (see below for addition of 90% CIs) and assuming that GVA change isn’t clearly present if the possible minimum from one year is lower than the possible maximum from another year, and vice versa. (This is a conservative assumption that the true value2 could be at both confidence extremes between timepoints - in the penultimate section, a tighter version is used based on z-test slope comparisons.)
The inclusion of error rates immediately makes it very rare for year-to-year changes to be distinguishable from zero, if looking at individual 2-digit SIC sectors in specific ITL1 zones. So instead, what each grid plot does in the visualisations here is compare each year to every other year, to show where each is clearly separable.
Let’s talk through the default sector in the interactive (and in Figure 2): fabricated metals, to make more sense of that. What this shows:
The top part of the figure plots the time series with added uncertainty bounds. (You can see the same data in the interactive by hovering over Yorkshire and Humber to see its growth over time.) The grid below that compares each year to every other year.
Blue squares in the grid indicate that the column year was clearly higher than the row year it is compared to (‘clearly’ meaning, as described above, 95% mins and maxes don’t overlap).
We can see that 2020 to 2023 are clearly higher than the earlier years in the data, from 2012 to 2016, though the high peak in 2021 is the only year consistently separable from others. 2020 to 2023 are years where the confidence interval minimum is separable from the earlier CI maxima.
The rest of the grey area is showing that, for the first half of the data from 2012 to 2017 - on the assumptions we have here - no clear growth signal is coming through.
Note, the grid is mirrored along the diagonal. So orange squares are showing the same thing in reverse - earlier column years are clearly lower than some later ones.
Looking on the interactive, Scotland shows the opposite pattern, which can be seen from its growth slope clearly too - bottom left orange squares indicate clearly lower values in the later part of the data.
Figure 3 shows fabricated metals in the East Midlands. While the later value increases appear separable from the low point around 2017, it can’t be so easily distinguished from where it was from 2012 to 2016, despite the central estimates looking higher (they could well have been higher in the earlier period and lower in the later one). That is, recent apparent growth could just be a return to the pre-Brexit status quo, not a step change. The difference to Figure 2 is mainly driven by East Midlands’ larger standard errors - the underlying GVA data is more uncertain in the ABS. Other sources could be explored to triangulate. Have job numbers risen, for instance? What is known about any productivity changes in the sector here?
Several other ITLs for fabricated metals in the interactive are purely grey. Looking at any of those shows combinations of wide error rates and fairly flat change over time, considering the range of those confidence intervals. The apparent drop in the North West after 2017, for example, isn’t clearly separable, though it looks strong.
Within this way of looking, the data signal of the COVID19 pandemic comes straight through for land transport (view on interactive page), as it does for accommodation (here) and food services (here). Other sectors where one might expect a pandemic impact (arts/entertainment and museums/culture) are short on data. Mid-pandemic drops show up as an orange ‘column year lower than most other years’ band. London stands out as the one place that hasn’t yet bounced back to its pre-pandemic state for land transport, remaining significantly lower than other ITL1s.
Current prices GVA: location quotients
Location quotients (LQs) are an intuitive way to assess the relative strength of sectors in regions. An LQ is the ratio of a sector’s proportion of a region’s whole economy versus that sector’s proportion of the UK as a whole. If, say, fabricated metals is 20% of a region’s GVA but only 10% nationally, it is twice as concentrated in that region - the LQ is two3.
Current price (CP) GVA data can be used for LQs because (unlike CV) it can be summed, as it is just the money value of each sector’s output recorded in that year. CP data can’t show real growth over time, but it can be used to examine economic structural change - how much a sector/place combination has relatively grown in concentration (which could be due to nominal growth in that sector, or other sectors increasing in value faster).
LQs are (in theory) perfect for thinking about regional specialisms, and for comparison and ranking, making them an obvious fit for industrial strategy thinking. But they also carry the “single accurate value” issue over from the underlying GVA data.
Putting error bars around an LQ is trickier than for direct GVA because it is a derived value - it doesn’t make sense to apply confidence intervals to the LQ directly. The solution used here is to simulate what the range of the underlying GVA values could be using the applied uncertainty, and then calculate LQs based on those simulated values.
Each simulation pulls a sample from a normally distributed range based on the standard errors applied from the ABS, adjusted to make sure no negative values arise as these could break the LQ formula. This is then repeated 500 times to produce a range of possible LQs.
Here is the interactive page for looking at the results, starting again with fabricated metals. There are LQs for each ITL1, including 95% confidence intervals (Figure 4 is the same plot for fabricated metals).
Considering overlapping error rates, the North East, East Midlands and Yorkshire & the Humber might be seen as one ‘high concentration’ group not really separable from each other, with London and the South East separately at the ‘low concentration’ end. West Midlands looks like it has clearly the highest concentration, above all other ITL1s. Other places in the centre are not so separable, and four have error bars crossing zero - if these are reasonable, there is no way to say if fabricated metals is more or less concentrated than the UK average.
The plot also has an option to compare LQs across five years, between 2017 and 2023. Overlapping error bars suggest the LQ for fabricated metals may not have significantly changed between those years in many places. The East Midlands (real LQ increase), Scotland and the South East (a plausible decrease in both) do suggest genuine structural change over time.
Consider what difference the overall error rate can make for LQs. Some sectors have very high uncertainty, carried over from the ABS. The error rates for construction of buildings, for example, make it clear that any single-point differences need treating with much caution. Only the North East (relative growth) and London (shrinking) appear to have a clear signal. This is also true for food services - huge uncertainty here (seen also in the CV data) also suggests being careful with claims about the service economy from this data (though this doesn’t account for any further data contributions going into the complex national accounts numbers).
Error bars for sectors like arts/entertainment show uncertainty varying a lot across ITL1s. Clearly there is better ABS data for Wales, the North East and Northern Ireland for this sector. Elsewhere, many places are not distinguishable from the UK average concentration. London is also quite uncertain but because its arts/entertainment is so concentrated, it is easy to separate (large effect size, in statistical terms).
Triangle plots for both chained volume and LQ data
This section runs through a way to quickly look at comparisons of LQs (from current prices data) and real growth (from chained volume data) to separate signal from noise, and in the case of the ‘real growth’ chained volume data, easily compare growth slopes with and without the extra error4.
Let’s start with location quotients and this triangle plot for London’s ITL1 zone (see also Figure 5 below) comparing LQs between 2019 and 2023. What we have here:
Each grid square compares LQs and their uncertainty, for each sector in London with that sector in every other ITL zone (x axis).
The bottom-left triangles in each square show if London’s LQ in 2019 was lower (blue) or higher (orange)5 than that sector in the other ITL1s, or not separable (using a z test to compare the difference between the two LQ values). The top-right triangles show the same for 2023.
Grey triangles - and especially whole grey blocks - are exactly the ‘fog is information’ point. Using only LQ point data, there would be no grey here. That has two uses. First, it acts to show where there’s signal in the separable LQ triangles. Second, the grey can be its own signal - for example, construction of buildings (fourth from bottom of top half) is showing its country-wide uncertainty here. London moved from ‘more concentrated from five places’ in 2019 to ‘unclear there is any difference’ in 2023.
The ‘most relative position shifts’ plot in the top half shows the top 15 sectors that saw LQs change most often between low/high/not separable. For example, gambling and betting in London has gone from an LQ of 0.65 to 1.23 (LQ values are on the y axis) between 2019 and 2023, and this shows up in the triangles: many have moved from blue (lower concentration than the other ITL1) to orange (higher). Only the North East continues to have a separably higher concentration than London.
The ‘fewest relative position shifts’ plot in the lower half shows where LQs remained the most unchanged between 2019 and 2023. The solid blocks of blue and orange are sectors in London that have been consistently either higher or lower concentration there. No other ITL1 has this level of fixed structure over the whole dataset (the South East is closest, though that line of higher LQs to London mostly in manufacturing is telling; West Midlands is third). See also the 2012 to 2023 comparison for London - mostly the same sectors have remained unchanged.
There is also the option to order LQ differences by how many are significantly higher or lower overall. Here is an example for Yorkshire and The Humber, comparing LQs between 2012 and 2023, where it’s possible to confirm what Figure 4 shows above for fabricated metals. Seventh from the bottom of the top plot, fabricated metal concentration in Yorkshire and The Humber has been consistently more concentrated than most places - except the East Midlands, where the LQs aren’t separable (at 95% confidence), and the West Midlands, which - as Figure 4 also confirms - has a definite higher LQ, and did in 2012 too.
Next, we use the triangle plot approach to look at ‘real’ (inflation/quality adjusted) growth over time using chained volume data. This is the plot for London; Figure 6 shows the output here.
This time, the top and bottom triangles in each grid are ‘slopes differ at 90% and 95% confidence levels’. 90% is indicated with a triangle in the bottom left of each grid square. This is a lower bar. The stricter 95% level is top right. So for example: looking near the top of the bottom half, warehousing and transport support has each triangle full, across all ITL1 comparators - and they are blue, so slopes are separably lower than those places, at both 90 and 95% levels. This sector does look to have shrunk, and clearly more so than anywhere else.
Y axis labels show the actual average yearly change (from the linear slopes) for each sector, including 95% confidence intervals. Again, blue indicates the sector shrunk on average, orange shows it grew. If a sector crosses zero (i.e. it is not statistically clear if it grew or shrunk) the colour is faded. Note in the y axis label, warehousing and transport support’s average yearly change was -4.6%.
The top and bottom groups of 15 sectors are ordered by uncertainty inflation - that is, how much adding extra uncertainty from ABS could impact confidence in slope differences. The top 15 have the widest uncertainty difference (i.e. adding uncertainty from the ABS has made the biggest difference); the bottom 15 have the smallest (so warehousing and transport support looks solidly correct across both approaches).
The top 15 are showing which sectors are most affected by the added uncertainty. To pick a striking sector, where scientific R&D looks to have separably higher growth in London than many other ITL1s (third from top), with added uncertainty, only one slope remains separable (at 90%, from the North West). On the right hand side, showing results with added uncertainty, the y axis label also shows this sector’s growth bounds (at 95%) cross zero. Unlike the ‘OLS only’ result, it’s not separable from zero linear growth over the whole dataset.
So what?
“We demand rigidly defined areas of doubt and uncertainty.” Vroomfondel, Amalgamated Union of Philosophers, Sages, Luminaries and Other Thinking Persons. (Hitchhiker’s Guide to the Galaxy)6
This chunk has laid out some working examples of how regional output could look when uncertainty is applied. So, should we also be demanding rigidly defined areas of doubt and uncertainty?
The alternative is point estimates, the precision of which we know nothing about, except that we know it’s very likely not precise.
Often, we have to work with point estimates. Sometimes, it’s optimal to do so, if no alternative exists. Allocation of funding is an obvious example - whether through the UK Index of Multiple Deprivation at small scale, or European objective 1 funding given to NUTS2 zones with GDP per head below 75% of the EU average (no longer in the UK of course), point estimates can seem like the only practical approach. Issues are entirely acknowledged, but spurious accuracy is weighed against the ‘merit of simplicity’ (Bofinger 2003, p.9), potentially a strong policy advantage if it aids efficiency and legibility.
But why do this if we don’t have to? And how might decision-making be affected if we take uncertainty seriously? I’m saving a full dig into that for part 2, but I’ll end here with two connected points to bounce off.
- How to think about error and uncertainty?
There’s plenty of theory on this topic (e.g. “planning under conditions of deep uncertainty” Haasnoot et al. (2013), game-theoretic ideas) but it may not be the best place to start. Manski’s application of minimax regret Stoye (2012), for example, sounds sensible - quantifying the worst risks, comparing and minimising them. But as this excellent analysis of uncertainty in energy planning points out, approaches like this demand their own forms of ‘incredible certitude’ to specify their exact trade-off calculations:
“… especially in the context of long-term decision-making in which deep uncertainties are present, there is no ‘automated’ method of analysis for [their management] so as to arrive at an optimal decision. Rather… judgements are required at many stages in the decision-making process.” (Dent, French, and Zachary 2021, p.16)
They point back towards thinking through how decision processes are designed to be iterative and flexible. More on that below.
It is, however, useful to hold in mind the wrong directions we might steer towards: things we mistakenly believe to be true (type I errors) and things we may be missing or are overlooking (type II). This report naturally lends itself to the ‘false alarm’ type I, as added uncertainty can turn a positive into an “unsure” very easily (every ITL1 has at least one sector that goes from signal to noise in this data). And these error types tend to be easily noticed when they happen. Johnson et al. (2013) argue we have a built-in bias towards type I ‘false alarms’ mattering more, for good reason:
“We sometimes think sticks are snakes (which is harmless), but rarely that snakes are sticks (which can be deadly).”
This is embodied in the precautionary principle: “Extra precaution is justified when false negatives are worse than false positives” (Persson 2016). In the chasing-vampires example, Michael is best using this - it’s safer to assume David is leading him off a cliff, even if he isn’t.
Spurious accuracy can make the precautionary principle seem unappealing, however. Precise claims of the ‘x region is growing fastest / x advanced sector is largest in the UK’ type, in isolation, can be hard to resist. Uncertainty around those would allow a little more precaution where it could be vital.
But while the opposite type II false negatives can be deadly serious (“there is no cliff oh argh yes there is”), more often their cumulative effect can be more insidious. They can be a symptom of how data is used. The ‘streetlight effect’ captures this perfectly (Kirman 1992, p.134). This is akin to:
“… the person who, having dropped his keys in a dark place, chose to look for them under a streetlight since it was easier to see there.”
False negatives - failing to see what’s there - can become systemic when we mistake a narrow data source for reality. More subtle and systemic errors can result - for example, if our hypothesis is that foundational sectors in the creative or building economies are downstream of growth choices, their lack of economic signal may make them even more likely to fall from our thinking.
But we shouldn’t default to seeing with what data there is, just because it’s what is available. Again - the fog is information. It suggests slowing down to think through our information systems. As with Michael following vampires toward cliff edges, we can check surroundings and steer more carefully.
So there are reasons to suspect uncertainty might help to steer between both types of blindspot. But in what ways, exactly? In day to day policy choices, what would be different? Thoughts please.
- Spuriously accurate GVA data blinds us to the need for deeper regional knowledge structures built bottom-up.
Coyle and Muhtar Coyle and Muhtar (2023) explore this brilliantly. They identify a structural ‘failure to learn’ at regional level in the UK. Due to an historic lack of bottom-up development (they suggest Japan and the U.S. as places where this has happened successfuly) there is an “absence of mechanisms for feedback and learning from local outcomes” which they identify as a “fundamental institutional weakness”.
A plausible stab at uncertainty helps focus minds towards the weaknesses of top-down data and the relative immaturity of data/policy sinews below national level they identify. Error makes the fog visible, and so directs us back towards our regions, what we concretely know about them, and how that knowledge is built into the choices we make.
A more regionalised sense-checking process is precisely what catches mistakes in spuriously accurate top-down data and encourages development of deeper, more devolved knowledge networks.
What next?
I’ll aim to talk to people about ways uncertainty like this might affect thinking and decisionmaking, as well as what errors I’m making in the approach (there will be many). Again - any and all feedback welcome.
It would be useful to think about other ways to include uncertainty. Of course, it doesn’t have to be - and shouldn’t be - limited to GVA data, or to ITL1 scale. Smaller geographies with added uncertainty could actually produce more accurate insight, rather than building on average values spread over entire regions (that is clear, for example, from the subregional LQ numbers in Yorkshire and Humber, even without error). Error from the annual population survey, BRES data or Companies House could help understand jobs and productivity.
There are related aspects of regional economic data that tie in as well - I’ve listed some of those on the project’s github front page here and will add more as they arise.
LLM use
In this article:
- Claude code added the LQ formula and the lines from “where GVA…” to “below 1, less so” in Current prices GVA: location quotients.
- ChatGPT made Table 1 for me, from this prompt. I tweaked some of the text. “Consider the following two bits of text. (1)”Never confuse Type I and II errors again: Just remember that the Boy Who Cried Wolf caused both Type I & II errors, in that order. First everyone believed there was a wolf, when there wasn’t. Next they believed there was no wolf, when there was. Substitute “effect” for “wolf” and you’re done.” And (2) “In the movie ‘The Lost Boys’, David and his west-coast vampire buddies race motorbikes across a foggy landscape. Michael (unaware of the blood-drinking habits of his new associates) tries to keep up with them. They egg him on - but then he spots the faint beam of a lighthouse, puts two and two together and barely avoids hurtling over a cliff edge. One might say David was claiming ‘incredible certitude’ (Manski) about the cliff’s location.”Don’t worry Micheal, it’s definitely, definitely miles from here.” Can you (1) put the lost boys quote in a type1/2 error context for me? And (2) search for other online sources/links that discuss examples of uncertainty and ‘incredible certitude’ and their implications?”
In this project as whole:
The main R code script here has sections clearly marked where Claude Code (CC) was used to output code. This is mostly for visualisations, but CC also built the Monte Carlo randomisation simulation to get a spread of LQ values.
CC was used to output the two HTML/Javascript interactives, built around saved images. The prompt back-and-forth for that process is here.
CC drafted a first version of the project folder structure that I then tweaked. That built on this source-synthesis from CC on ‘modular open publishing workflow’.
There is a CC-drafted source summary exploring why the ABI and region-by-industry GVA values differ here.
Appendix
Table: Lost Boys cliff scene in type I / II terms
Figure: GVA from the ABS versus GVA from the ONS region-by-industry data, average percent difference
References
Black, James, Kevin Connolly, Sharada Nia Davidson, and Mairi Spowage. 2025. “The Use and Misuse of Regional Data: Perspectives, Challenges and Policy Implications from Across the UK.” Regional Studies 59 (1): 2464102. https://doi.org/10.1080/00343404.2025.2464102.
Bofinger, Peter. 2003. “The Stability and Growth Pact Neglects the Policy Mix Between Fiscal and Monetary Policy.” Intereconomics 38 (1): 4–7. https://doi.org/10.1007/BF03031831.
Coyle, Diane. 2014. GDP: A Brief but Affectionate History. Princeton University Press.
———. 2017. “The Future of the National Accounts: Statistics and the Democratic Conversation.” Review of Income and Wealth 63 (s2): S223–37. https://doi.org/10.1111/roiw.12333.
Coyle, Diane, and Adam Muhtar. 2023. “Levelling up Policies and the Failure to Learn.” Contemporary Social Science 18 (3-4): 406–27. https://doi.org/10.1080/21582041.2023.2197877.
Dent, Chris, Simon French, and Stan Zachary. 2021. “Decision Making for Future Energy Systems | Ofgem.” https://www.ofgem.gov.uk/publications/decision-making-future-energy-systems.
Galvão, Ana Beatriz, and James Mitchell. 2024. “Communicating Data Uncertainty: Multiwave Experimental Evidence for UK GDP.” Journal of Money, Credit and Banking 56 (1): 81–114. https://doi.org/10.1111/jmcb.13044.
Haasnoot, Marjolijn, Jan H. Kwakkel, Warren E. Walker, and Judith ter Maat. 2013. “Dynamic Adaptive Policy Pathways: A Method for Crafting Robust Decisions for a Deeply Uncertain World.” Global Environmental Change 23 (2): 485–98. https://doi.org/10.1016/j.gloenvcha.2012.12.006.
Johnson, Dominic D. P., Daniel T. Blumstein, James H. Fowler, and Martie G. Haselton. 2013. “The Evolution of Error: Error Management, Cognitive Constraints, and Adaptive Decision-Making Biases.” Trends in Ecology & Evolution 28 (8): 474–81. https://doi.org/10.1016/j.tree.2013.05.014.
Kirman, Alan P. 1992. “Whom or What Does the Representative Individual Represent?” Journal of Economic Perspectives, Journal of economic perspectives, 6 (2): 117–36. http://ideas.repec.org/a/aea/jecper/v6y1992i2p117-36.html.
Manski, Charles F. 2020. “The lure of incredible certitude.” Economics and philosophy 36 (2): 216245. https://doi.org/10.1017/S0266267119000105.
Persson, Erik. 2016. “What Are the Core Ideas Behind the Precautionary Principle?” Science of The Total Environment 557-558 (July): 134–41. https://doi.org/10.1016/j.scitotenv.2016.03.034.
Stoye, Jörg. 2012. “Minimax Regret Treatment Choice with Covariates or with Limited Validity of Experiments.” Journal of Econometrics, Annals issue on “identification and decisions,” in honor of chuck manski’s 60th birthday, 166 (1): 138–56. https://doi.org/10.1016/j.jeconom.2011.06.012.
Footnotes
code/comments here run through combining those two sources. A final joined CSV is here, to save others the pain, though it’s just for GVA, not every ABS value.↩︎
‘True value’ being used as shorthand for the cumbersome ‘95% confidence interval would contain the true value 95% of the time if the survey sample was taken again’. [ONS source…]↩︎
Because (A/B)/(C/D) is equivalent to (A/C)/(B/D), the LQ actually captures two related ways of seeing the same thing: how relatively concentrated sectors are across a whole geography like the UK, and how concentrated within a subgeography like South Yorkshire they are. (See the desciption and table in the ONS’ older LQ document.)↩︎
These plots do not, at the moment, provide options to adjust for the fact that we’re examining many separate ‘1 in 20’ or ‘1 in 10’ threshold pairs, where each could through pure chance change from positive to negative. Adjustments like Benjamini-Hochberg are imperfect but could make a bit more robust, potentially.↩︎
These are the two colours most likely to be clearly distinguishable for colour-blind people.↩︎
The stats / greek crossover being what it is, this quote has been used for the same purpose in this Nature Methods article, this PhD thesis (p.31), this Medium article, this teaching handout and probably many other places. I see no reason not to continue this time-honoured tradition.↩︎