Social data and analysis are not the most important issues at the moment (to put it mildly!), but for those of us who aren’t key workers, this is where we can contribute. And data are genuinely important: good decision-making and political accountability require an understanding of the social consequences of the pandemic, the effectiveness of the social policy response, and a debate about what we can do better.
With this in mind, I’ve put together an open Google Doc on COVID-19 social data – please do use / contribute / share, so this becomes a collective resource! (And thanks to those whose contributions helped me write the first draft). But as well as this, I wanted to reflect on the need for MORE data than we currently have.
The limitations of government data
Given that most major govt surveys are face-to-face, and this can’t happen any more, ONS have done a frankly incredible job of trying to keep data collection going (as the Google Doc explains in detail). They’ve introduced two new surveys – the first starting fieldwork on 20th March! – and have already moved fieldwork for many other surveys to telephone/online.
However, there are four reasons to think that policymakers (and the rest of us) still need more:
• Speed: there is usually a delay before ONS data are made available, partly reflecting the long duration of fieldwork and the extensive data cleaning/post-processing. Even the Labour Force Survey – the quickest in normal times – isn’t likely to publish any data covering April’s labour market until September, and most surveys will be substantially slower. The speed of ONS data is remarkable in usual times, but in this period of crisis, it just isn’t fast enough to help policymakers make difficult decisions.
ONS’s new surveys are stepping into this breach, with the first publication from the 20-30th March Opinion & Lifestyles Survey coming out on 16th April, and the new Labour Market Survey (LMS) hopefully providing rapid updates – again, a great response by ONS. However, unless things change, the underlying data are not available (e.g. there’s a two-year lag for the Opinion & Lifestyles Survey to reach the Data Service). Still, while these two surveys only scratch the surface of the data we need, but are still a fantastic development.
• Detail: in usual times, the welter of detail from the major government surveys allow us to unpick the most crucial aspects of life in Britain. Yet the questions we want to ask at the moment are different from the usual ones; and while ONS are adding some COVID-19 questions into their surveys, it still seems likely that there will be gaps. (There’s a list of these at the end of the Google Doc).
• Sample size for a changing picture: many government surveys aim to produce a representative picture over a year, and their ability to capture a point in time is limited. The LFS and particularly the LMS are the major exceptions here – their role will be crucial. But this is going to be a major limitation of other datasets in showing what happens during lockdown.
• Comparability: to truly understand what life during (and after) the pandemic is like, we need to know how it compares to life beforehand. This is going to be tricky, because the changes in the way that data are collected will introduce discontinuities into many survey series.
There is one dataset that looks particularly valuable here: the LFS longitudinal data. This had a pre-existing sample (of people who completed an interview before the pandemic hit), who will be followed over the next year by phone, just as they were before. I’m planning to use this, and I hope many others will do so too (if so, get in touch so we can compare notes!).
None of these concerns disparage the work done by ONS so far; I think they’ve done a brilliant job, and it must have required a huge effort by the ONS teams involved (for which we owe them a massive thanks!). But I still feel that these data aren’t enough; we need to collect more data, and to do it quickly.
The limitations of COVID-19 surveys so far
We have already seen a flurry of nationally representative social surveys during COVID-19 (seven online panel surveys so far; the Google Doc has an ongoing list). One of the advantages of online panel surveys is that they can produce (weighted) results very quickly, which is a major benefit during a time of crisis when speed is of the essence. These surveys nearly always generate considerable media attention, and are becoming one of the crucial ways that we understand the world that we are currently living in.
But these also have limitations:
• Detail: we are yet to see a rapid survey that goes into any detail on people’s lives. The surveys so far include a few (1-4) questions on a specific issue, throwing an important light on their central focus (e.g. food insecurity), but little or nothing else. There’s also numerous topics that have not yet been covered in a survey, particularly those that are trickier or more expensive to conduct online surveys on (again, see the list at the end of the Google Doc).
• Quality/reflection: I am a big fan of online panel surveys – I’ve even commissioned a couple – but it’s important to be aware of their limitations. As the inquiry into the 2015 British general election polls makes clear, these do a pretty good job of providing a representative picture of the British population, but this can go wrong. I’m a little disappointed that the rapid reports so far don’t discuss these issues at all. People can cope with uncertainty, as long as you take the time to express it clearly.
• Open data: As far as I know, none of the existing surveys are making their underlying data available for anyone else to use. In a time of crisis, collaboration is crucial, as the Wellcome-brokered open data consensus statement makes clear. Lots of COVID-19 data are already being made publicly available globally; we should make sure that social data from the UK are part of this joint effort.
Again, this isn’t to disparage the work done to date – it’s remarkable that as of today (16th April), we already have six published surveys with fieldwork on 24th March or later. Nevertheless, we need more efforts, which are bigger and more detailed, and which rapidly make their (well-documented) data publicly available.
A collaborative way forward
I know that some organisations and academics are currently planning social surveys on the response to COVID-19, and I’m sure there are many others I don’t know about.
What I would urge everyone to do is to collaborate as much as possible. If you’re thinking of doing a survey, consult as widely as you can (including with the people that are affected by whatever you’re looking at, rather than just policy/academic elites), to ensure that you’re covering the things that matter, and not duplicating what other people are doing. And make sure that the data are publicly available as quickly as possible (in a proper repository, with full documentation).
If you disagree (or what to extend this argument), please reply under this post or get in touch if you want to write a separate blog here – like everything, the best opinions are the ones that emerge through debate.
Inequalities 