There’s a time-honoured tradition of getting headlines by putting countries in a league table. This week it was the turn of UNICEF’s report on inequality in child well-being, which showed the UK struggling in lower mid-table obscurity – and with the US, Greece and Italy at the bottom. This follows a well-publicised paper last year that put the UK at 24th out of 29 EU countries for its overall level of child wellbeing. But while the headlines rarely go beyond the overall ranking, to actually understand these we need to dig deeper to understand what they’re actually showing – and to ask whether these league tables can be any use at all.
(A confession: I’m as guilty of creating these league table as anyone, back in the days when I was a full-time alcohol researcher… So I understand how tempting this is!).
How UNICEF created their league table
The ‘overall record’ league table is shown to the right of the page, based on a simple 1-3 score of whether a country is below, at, or above average for inequalities in each of material wellbeing, education and health. The danger, though, is that it’s almost impossible to understand this without knowing how this is calculated – which to be fair to UNICEF, they explain very well if you’re prepared to dig into the report.
Let’s start with inequality in material well-being. This is based on an equally-weighted average of three standardised scores: post-tax income, educational resources (e.g. internet connection, a quiet place to study), and number of rooms per person in families with children.
This is transparent – but I’m not convinced this is a sensible scale. ‘Number of rooms’ and ‘educational resources’ simply don’t seem as important as income here. Otherwise Sweden – which does brilliantly on income equality – is ‘close to the average’, simply because it has fewer rooms per person than other countries. (If any Swedes can explain whether Swedish rooms are unusually large, btw, that would be helpful…). Spain, conversely, does terribly on income inequality but pretty well on rooms-per-person. What this means – and why you would put these measures together – isn’t obviously clear to me.
Reassuringly, the measure of educational inequality makes perfect sense: it sensibly combines reading, maths and science literacy.
But the scale for health wellbeing is muddled too. It combines inequalities in (i) number of self-reported ill-health symptoms, (ii) eating fruit & vegetables, and (iii) physical activity outside school. These are just not the same sorts of thing: ill-health symptoms are a health outcome; while healthy eating and physical activity are health risk factors.
There are also other, more technical issues here – for example, about their measures of inequality for the categorical outcomes. (For everything except income and education, they compare the median score with the average score of those below the median – which is defensible, but not that easy to interpret). Moreover, it would be a brave researcher who would argue that 11-15 year olds answer questions on ‘feeling irritable’ in the same way across all these different countries.
But even if we could create a perfect league table, the question remains: what do league tables tell us?
Going beyond league tables
Politicians and newspapers are suckers for league tables. Newspapers love them because anything that is the best/worst is automatically a good headline – and in the UK, they also feed our masochistic urge to say how terrible our country is. More helpfully, politicians love league tables because they seem to show that we’re failing in some way, and that a better world is possible. For those on the right, this often means envious glances over the Atlantic to the US; for those on the left, it’s dreaming of Sweden (which, as ever, is near the top on the UNICEF table).
The trouble is that it’s by no means clear what we should actually do in response to league tables. UNICEF follow-up their table by talking about a range of policies, but nothing here is as hard-hitting as the original ranking – and it certainly isn’t based on naively imitating top-ranked Denmark or Switzerland. Often there are a range of reasons behind a country’s superior performance, some of which are cultural factors that are simply not possible to replicate. Americans will never become Swedes; the Danish flexicurity model may simply require too much goodwill for a Britain that is riven with stereotypes and distrust around benefits. And for the top-ranked countries, league tables argue for a policy of smug complacency, rather than the possibility for every country to do more.
None of which is to say that league tables should be scrapped. They have their place. But it would be much more useful to focus our knowledge on what we could actually do to reduce inequalities, and how this compares internationally. We desperately need more results that estimate the effect of a whole range of policies across different countries, showing in which countries the greatest gains could be made – and by extension, where the political class is showing the greatest failures. Colleagues of mine in the alcohol field are working on just such models. Over the coming fortnight, I’ll try and explain these in more detail – and why even creative experiments need this kind of work alongside them.