Big Data and Public Policy: What It Changed
Where large-scale behavioural data has genuinely improved public decisions, where it has produced expensive dashboards, and how to tell the difference in advance.
Read →
Research theme
Data and Society: Evidence, Policy and Its Limits
Administrative records, platform logs and sensor data describe populations at a resolution that survey research never reached. They also describe them with biases that survey research spent a century learning to correct for.
The optimistic version of the data-and-society story is straightforward: more measurement means better decisions. The version that survives contact with the data is more careful. Every large behavioural dataset is a record of an instrument, not of a population. Mobile phone records describe people who carry phones in the way the operator bills them. Platform logs describe engagement as the platform defines it. None of that makes the data useless, but it does mean that the first analytical step is always reconstructing what the sample actually is.
Where it works
Coverage and frequency are the real gains. Official statistics arrive quarterly and revised; digital traces arrive continuously. For anything where timing matters more than precision - tracking mobility after a disruption, spotting a shift in demand, mapping how a rumour moves - that trade is worth making. The big data and public policy page covers the cases where this has actually changed decisions, and the more common cases where it has not.
Where it breaks
It breaks when the instrument changes. A platform adjusts its ranking algorithm and your time series develops a discontinuity that has nothing to do with the world. It breaks when the measurement becomes a target. And it breaks, in a way that is harder to fix, when re-identification turns an aggregate dataset into a personal one; see digital traces and privacy.
An applied case
Health is where these tensions are sharpest, because the data is the most sensitive and the potential gains are the largest. Digital health and access looks at what happens when care itself moves online, using the European mail-order pharmacy market as a worked example.
In this theme
Notes under Data and society
Digital Traces and Privacy: Why Anonymisation Fails
Behavioural data is far more identifying than it appears. What re-identification research established, and what it means for research using platform data.
Read →Digital Health and Access to Medicines
What happens to access, price and safety when pharmacy moves online - the European mail-order market as a worked case in digital service transitions.
Read →Related
Other themes
- Complexity economics Why treating an economy as a system in permanent motion rather than a machine that settles into equilibrium changes what you measure, model and forecast.
- Agent-based modelling How agent-based models are built, what they are good for, where they fail, and which software people actually use.
- Social networks Network structure, how information and opinions move through it, and what the measures actually mean.