When social policy goes wrong

When social policy experts create a new intervention to solve some social problem or make people’s lives better, there’s two possible outcomes they’re expecting: either the intervention works, or it doesn’t. But what we forget is that there’s a third option – that our well-meaning intervention actually makes people’s lives worse.

This isn’t just an idle fear. Shockingly, this seems to be true of the biggest randomised controlled trial of an intervention that has ever been conducted in the UK (and with legs in the US) – the Employment Retention and Advancement (ERA) project. 

The ERA project

The findings come from new research by Richard Dorsett & Andrew Oswald at a NatCen/NIESR event (so come with the caveat that they’re not fully published in a peer-reviewed journal yet). I’ve blogged about the ERA project before when summarising its final results - the key features of the interventions are:

    • Extra help and guidance from personal advisers to encourage both retention and advancement;
    • Extra financial support like a training budget and an emergency fund to cover anything that might help them stay in work;
    • Cash bonuses£400 three times a year for 2yrs for staying infull-time work, plus £1k for completing training.

The idea was to get provide both help and incentives to get people to progress at work, breaking out of the low-pay no-pay cycle. And prima facie, it’s difficult to see how such help could be a bad thing. But…

The effects of ERA on wellbeing & financial security

Dorsett & Oswald’s study focuses on single mothers – the group whose earnings were raised while the programme was running, but who went back to the same situation as the control group after the programme stopped (see the summary on the blog) . They look at the situation 2 years after ERA started (when the programme was still running), and 5 years afterwards (at least 2 years after the programme had stopped). It’s worth pointing out that they find slightly more positive results on earnings than Gregg and Riccio in my previous summary, so that even after 5 years, single mothers who’d been randomly assigned to ERA had slightly higher earnings than those who weren’t (but bear in mind the difficulty of looking at earnings on a changing group of people in work).

The really striking findings are in the table below:

In words:

  • If you look at the row in pink, you can see that people in ERA had lower life satisfaction five years later than those who weren’t on ERA.  It’s only a small effect (0.07 points on a 1-5 scale), but it’s statistically significant. When the programme was actually running, there was no effect on life satisfaction.
  • If you look at the section in red, you can see that people in ERA had more money struggles five years later than those who weren’t on ERA.  Again, these effects are generally small but statistically significant – but in the case of running out of money, are slightly larger (0.2 points on a 1-6 scale).

 

What the hell is going on here?

To me, this was an incredibly surprising finding – it simply hadn’t occurred to me that such a well-motivated intervention could backfire in the long-term. But on reflection, there are a few things that could explain this:

  1. Loss aversion – Dorsett & Oswald point out that people are often more sensitive to losses than gains. So if they receive extra money and have this taken away, they feel less satisfied than if they’d never had the money at all.  Another version of this is that these single mothers wanted to work, but struggled to make work pay (e.g. due to childcare problems). During the programme they were significantly more likely to work full-time, but after the programme finished they were no more likely to work than mothers who hadn’t participated in ERA. So perhaps their lower life satisfaction was partly because they were no longer able to work the hours they wanted to. This could explain the life satisfaction results, but not the negative impact on financial struggles.
  2. Sticky consumption – if people started spending more money when they were given the bonus payments, then they may have struggled to adjust their consumption patterns back when this money later disappears. This is Dorsett & Oswald’s main explanation for the financial struggles findings, and it fits the fact that the biggest negative effect of ERA is on whether people ‘run out of money’ at the end of the month.
  3. Frustrated ambitions – a final possibility is that these single mothers went into the programme hopeful that they could really progress at work if they tried hard and got additional skills/qualifications, ending up in much better jobs down the line. These ambitions don’t seem to have been realised for this group of participants – perhaps leaving them disappointed with both their career advancement and the associated financial rewards.

Learning from ERA

Overall, I think the ‘sticky consumption’ explanation is the most convincing (although like Dorsett & Oswald, I think we need more work on this). And this has crucial implications for the design of social security policies. Historically, the welfare state has been about income-smoothing – particularly from working life to retirement, but also in the face of risks like disability or unemployment. To the extent that it achieves income-smoothing, then it’s likely to increase people’s wellbeing. BUT when policies increase income fluctuations rather than smooth them – e.g. through time-limited interventions, or by trying to claim back tax credit overpayments – then they can actually create problems for people’s ability to match their spending to their income, with knock-on impacts on financial strains and wellbeing.

So the lesson I take from this is that benefit payments are not just about moving money from one group of people to another; they’re about helping people create stability in the face of all the other changes in their lives. And this also suggests that leaving people with the perpetual threat of having their benefits taken away may counteract any positive impacts of the money itself on their financial security. As Dorsett & Oswald say, more research is really needed here, but it’s striking and essential stuff.

About these ads

About Ben Baumberg

I am currently a Lecturer in Sociology and Social Policy at the School of Social Policy, Sociology and Social Research (SSPSSR) at the University of Kent. I also helped set up the collaborative research blog Inequalities, where I regularly write articles and short blog posts. I have a wide range of (too many...) research interests, at the moment focusing on disability, the workplace, inequality, deservingness and the future of the benefits system, and the relationship between evidence and policy. You can find out more about me at http://www.benbaumberg.com
This entry was posted in Blog posts and tagged , , , , . Bookmark the permalink.

9 Responses to When social policy goes wrong

  1. Hugely interesting, Ben. If your hypotheses are correct, then it seems that the negative results could have been avoided, or even reversed, if the intervention also included more financial counseling — both to manage expectations and to help program participants budget their money (coaching them on income smoothing). That’s just to say that before we give up on these kinds of interventions, we need to see whether there are some tweaks that produce the hoped-for benefits.

    • Ben Baumberg says:

      That’s true – although I’ve done some voluntary financial counselling myself, and I’m not convinced of its value. But as a general principle, thinking about ways of minimising the problems of withdrawing interventions is a really important point.

  2. Ben Baumberg says:

    A couple of important things I forgot to write in the original post:

    1. Security (rather than absolute levels of income) seems to be important when we look at related studies on benefits and wellbeing. On the one hand, Sjöberg 2010 finds that higher unemployment benefits increase the wellbeing of workers, suggesting that stability is helping even people who don’t receive the money (free WP here). On the other hand, a 2008 Cochrane review by Luca et al found no consistent impacts of family financial support on child wellbeing in developed countries [h/t for both to Pega et al 2012]. This isn’t surprising – we know that one of the prime reasons for social security is helping people deal with risk – but it’s something we often seem to forget these days.

    2. In the previous post where I summarised the ERA results, there was a bigger, more consistent effect of ERA on long-term unemployed people than on single mothers. It’s not clear where Dorsett & Oswald don’t look at wellbeing in this group – the data might not be there, the results might be different etc. – so this is something that would be interesting to check too. It could even be the case that policies can harm some groups while helping others…

  3. Charlotte Cavaille says:

    what i get out of this is that social policy that is aimed at adapting people to the market (“come on work harder, if you want you can get this promotion, I can even teach you how! I ll even pay you for your motivation and hard work”) don’t work as well as macro policy that is aimed at adapting the market to the people, by providing good jobs with good wages. We have a growth model issue in the West, improvement is limited to a small groups of medium to highly skilled. That kind of policy is a “fig-leaf”…

  4. Daniel Sage says:

    Really interesting post Ben. My own PhD research is looking at whether or not welfare-to-work interventions can be successful in improving social outcomes such as life satisfaction, so this is an interesting finding. A lot of literature – particularly from more continental social policy, as well as disciplines like social psychology – predicts a positive effect of activation policies on well-being. These kinds of arguments often seen to ignore (or at least underplay) the idea that welfare-to-work could actually be psychologically damaging, in the event that (a) it is done badly and (b) it is stigmatising. This finding seems to give some meat to this second, more critical view of welfare-to-work. Thanks for posting.

    • Ben Baumberg says:

      I didn’t realise your PhD topic was so closely related – hopefully we can get you to post your own results on the blog in time! Might be worth speaking before that though in terms of whether your research fits into a cumulative impact assessment on the cuts (esp for disabled people) – will speak to you about this at the event in September if not before.

  5. Ben, great post and apologies for the delay in responding here. Four very quick thoughts:
    1.) There are hundreds of small ‘interventions’ delivered by local authorities and voluntary organisations across the UK (and, of course, elsewhere) every day and we really don’t know what the effect of them are, individually or collectively. Most aren’t evaluated in any robust way. I’d bet that some have a negative effect longer term and many have litte or no impact.
    2.) Isn’t there reasonably good evidence that ‘extrinsic rewards’ have a far shorter (and potentially negative longer term) effect than ‘intrinsic rewards’, certainly amongst children – see for example http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&id=1974-10497-001
    3.) We are seeing ‘incentives to work’ being introduced in welfare reform policies where the explicit intended (short-term) outcome is to make peoples lives worse, at least in financial terms, with the expectation that it will lead to a longer term improvement. Very interesting and links to Steven Luke’s theory of power and real or imagined interests. (I think)
    4.) Finally, came across this today – fascinating. Looks (at a very brief 1st glance) like a similar project – “the study aims to measure the reduction in child poverty that could result from engagement in a few hours work while receiving a modest charitable payment designed to support low income families” – http://www.ofmdfmni.gov.uk/equalityresearch
    Got to be worth a follow up blog……

    Regards,

    Steve

  6. Ben Baumberg says:

    Another programme which measures the impact on wellbeing is New Hope in Milwaukee – while the two-year impacts on financial worries were positive (see p166) when the programme was still running, the five-year impacts show little impact (p79) – there’s a possible beneficial impact of the programme on depression, but this really looks like random fluctuations rather than anything substantive.

  7. willemdax says:

    Reblogged this on Surf, Sink or Swim! and commented:
    Conservatives focus on results. Liberals focus on good intentions. No one ever said “The road to hell is paved with good results.”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s