Off the back of presenting at the ONS subnational data conference, this post collects the open data / code I used in the slides, as well as a few extra bits mentioned in there.
The presentation talks about the huge value and power of ONS data for the UK: how it can help us understand where we’ve come from and where we are now – and so help us work out we want to go.
There’s a mix here of step by step data walkthroughs and raw code: I want to work on getting more of this into a form that’s as useable as possible, ideally through testing what actually is useful and iterating.
I’ll add in some to-do notes on things that need updating / changing / improving and change this page as those get done.
Data and code used
- For the 1971-81 ‘scarring’ work (used in ‘Steel City: Deindustrialisation and Peripheralisation in Sheffield’ with Jay Emery and Gwilym Pryce):
- Harmonised Census data 1971 to 2011: country of birth and employment variables harmonised along with consistent geography. Full explanation in the readme there talks through how to get the data for country of birth (and further down the page there’s a link to the employment data). POSSIBLE TO-DO: HARMONISE WITH 2021 (and 1961???)
- That data is used in this RMarkdown output that produces the plots used (from this repo with general data stitching code). The data for the RMarkdown output, using the harmonised datasets, is processed in this R Script.
- For the sector proportion plots, and other code on processing ONS regional GVA data for location quotients, mapping and other bits, see this code and data stepthrough on the regecontools site.
- The productivity “GVA vs JOBS percent change” plots and map don’t have a good walkthrough yet - the code (including code to update to latest BRES and regional GVA data) is here in the repo for the first tranche of sector analysis work done for SYMCA, and is fairly readable and self-contained there. That BRES data is automatically extracted using the super-useful NOMISR package in this script and processed in this script (where it’s linked to the LCREE dataset, along with GVA data - work done in this script and then collated for a report in this Quarto doc). TO-DO: MAKE WALKTHROUGH FOR PROD PLOTS
- The GVA per hour plot is part of this walkthrough on the regecontools page looking more broadly at some GVA per capita / per hour worked.
- The Beatty / Fothergill rank change plot is from this fuller breakdown of their data, with code walkthrough, on the regecontools site.
Other links
- The Y-PERN website.
- SYMCA Plan for Good Growth page, which includes the M-D economic analysis and my own sectoral 3-pager summary.
- Write up / blog post of the 2019 Sheffield Data for Good hack day.
- Liverpool City Region Civic Data Coop
- Story on Sheffield Neighbourhood Mapping project (link to current map version).
- Centre for Cities LA Evidential report.
References from the presentation:
- Rice / Venables: “The persistent consequences of adverse shocks: how the 1970s shaped UK regional inequality” here
- Sarah Willams, Data Action: Using Data for Public Good.
- Martin A. Schwartz, “The Importance of Stupidity in Scientific Research.” Journal of Cell Science 121.
- Peter Tennant on Bluesky talking about how we grow and why we need an open mind and be willing to be wrong.
Bits I didn’t manage to cram in the slides
- Analysis of ONS business demography data that links local authorities across the dataset, including business ‘efficiency’ (balance of births and deaths) showing something shifted in more recent years in the south. (I write about automating your way out of an Excel data hole for this project here
- The incredible Dutch secure data service data used in our Rotterdam project - paper here, supplementary material with a map here. Individual-level data! 100m^2, track over time! Link to other survey data! Secure, trustworthy, easy to use!
- Northern Irish Census data - summarised down to 100m^2. Allows you to e.g. see Belfast like this (interactive map).
