There are some buzzwords that can win an argument all by themselves. You can’t describe yourself as against ‘fairness’ or ‘freedom’, for instance, or object to ‘social justice’ – however wrong-headedly they’re being used. And for policy-focused researchers, our clinching buzzword of choice is ‘evidence-based policy’.
So when I say I’m about to argue that evidence-based policy (EBP) is dangerous, I know I’m facing an uphill battle. But over 2011 I want to explain the many failings of EBP – and perhaps even persuade you to be more sceptical about the unthinking use of ‘evidence’.
Work stress, capitalism and evidence
For this post, I just want to give you two quick examples of one problem with EBP, which frustrated me in the closing months of 2010.
At the end of October, I found myself in a historic building on Great Carlton Street, one of the more exclusive addresses on the edge of the royal parks in the heart of London. Tarani Chandola – a really excellent sociologist/epidemiologist - was launching his British Academy report on work stress, to an audience of the great and the good (including Michael Marmot, and the head of the Trades Union Congress). Before describing the many harms caused by job strain, Tarani began with the following graph:
In other words, Britain has seen an astonishing rise in job strain over the past 20 years. (Strain here meaning a high-demands, low-control job). You’d therefore expect the recommendations from the report to talk about tackling the likely causes of this – which for private sector jobs are structural changes in British capitalism, and for public sector jobs is the doctrine of ‘New Public Management’. (More on both of these in future, as this is the topic of my PhD thesis…).
But such recommendations were nowhere to be seen. Instead, there was an exclusive focus on interventions that target either individual people or individual workplaces, together with health & safety law. When I asked Tarani why his policy recommendations didn’t match his most striking graph, he simply said, ‘there’s no evidence on what interventions would work at the societal level, so while I wanted to include it, I had to keep the conclusions evidence-based’.
Tarani Chandola is right, of course. The fault isn’t with him (and I really recommend his report as an accessible and thorough overview of the work stress literature). The fault is with what evidence-based policy has become. Before bringing this point out, though, another example.
Evidence-based reductions in health inequalities
I see the same problem when we try and talk about ‘evidence-based interventions’ to tackle health inequalities. Late in 2010, we had a report on inequalities in life expectancy from probably the most powerful groups of MPs outside Government – the Public Accounts Select Committee. Their report was highly critical about New Labour’s record, noting that “inequality in health has increased” and that targets have been consistently missed.
Health inequalities obviously depend on a variety of upstream social factors, which we can call ‘inequality-generating mechanisms’. Did the Committee’s report mention these at all? Well, no. They actually focused on delays and implementation issues in the government’s health inequalities strategy, with their most useful suggestions concerning the commissioning strategies in primary care.
But while their suggestions are sensible, the idea that this would have been sufficient to meet the Government’s health inequalities targets is dubious. I’ve previously written on Inequalities about Labour’s unflattering record on inequality-generating mechanisms. Yet the Public Accounts Committee’s role on evidence-based, value-for-money-focused policy restricts it evaluating policies in terms of micro-management – rather than whether the policy could ever actually succeed in its aims.
How to tackle inequality
In critiquing the idea of evidence-based policy, I’m obviously not calling for a return to ‘opinion-based policy’ – if I didn’t believe that social science had a valuable role in policymaking, I wouldn’t dedicate my life to it. Yet much as I love experiments, the most important determinants of inequalities are often impossible to evaluate using the logic of conventional evaluation research. The current model of EBP is therefore flawed; not just wrong, but actively damaging to attempts to reduce inequalities.
The question is: if we want to provide evidence with a wider scope, what do we replace EBP with? The recent Marmot Review on health inequalities in England reassuringly uses research on a broader, societal-level canvas – including on work stress. But the review also struggles with the role of evidence, glossing over major research gaps, recommending radical policies that go way beyond the science, and generally giving the appearance of a polemic rather than an evidence-gathering exercise. (An unfortunate impression, as there’s enormous amounts of great research in there). This illustrates further problems with the current model of evidence & policy, which I’ll come back to on another occasion.
Like those other powerful, symbolically loaded terms, ‘evidence-based policy’ is too useful a term to be consigned to the dustbin. The point is rather to contest its meaning, and to turn into something that has a realistic chance of reducing inequalities. Battles over terminology are not at the glamorous end of social science – but their importance in setting the direction of research and policy cannot be underestimated.