Some big lessons from Troubled Families

David Cameron and the former head of the Troubled Families programme, Louise Casey

In future years, people may look back on the government's main response to the 2011 urban riots as a clear case study of what happens when evidence clashes with short-term politics.

An evaluation of the policy has concluded that the programme has not been effective - but the government is refusing to listen.

Ministers have decided to ignore the bleak results of an official evaluation which has found that the first wave of the £400m "Troubled Families" programme to "turn around" 120,000 households did not work. Lord Bourne, the minister put up to respond, has simply stated that he still "believes" the programme works. The government seems to be ploughing on with it.

This is despite the work of researchers from the National Institute for Economic and Social Research, who wrote: "Across a wide range of outcomes, covering the key objectives of the Troubled Families Programme - employment, benefit receipt, school attendance, safeguarding and child welfare - we were unable to find consistent evidence that the programme had any significant or systematic impact."

• Watch Chris Cook's report on Troubled Families (August 2016)

The success of Troubled Families is also not simply of academic concern. The initial Troubled Families scheme, which was examined by the research, sought to turn around 120,000 households with multiple disadvantages at a cost of around £400m. An ongoing second wave covering another 400,000 families will eventually cost £900m.

The newly published research (whose contents were revealed by Newsnight in August) was built on a large amount of data - 56 local authorities submitted 18 months of data. It was based on the first wave of the programme, which was better funded per family than the second tranche. Their conclusion was that families enrolled in the programme were no more likely to see their lots improve on these measures than similar families who had not been enrolled.

The trouble with Troubled Families

A large fire in Croydon in August 2011 during the riots

The researchers' dispiriting results are not terribly surprising. As Newsnight has reported repeatedly, Troubled Families was always oddly designed. Existing policy ideas were modified into a rushed response to the 2011 riots.

Local authorities were set a quota for the number of "troubled families" that central government believed to exist in their area. For each such family that they identified in the first wave, the councils were paid £3,200. Unsurprisingly, cash-strapped councils filled their quotas rapidly, recruiting families into the scheme. Families enrolled in the scheme were in real need; researchers found they had an average of seven distinct major problems.

But local authorities could get a further £800 for each of those recruited family that they could say had been "turned around". The bar for that was low, however, so they could easily find reasons to declare victory with individual families, claim the extra cash and move on.

A temporary lull in truancy by a child, for example, could be enough for a council to claim a successful turnaround. That is why, nationally, councils claimed a 98.9% turnaround rate when the average family on the scheme still had five major problems when they left it.

Put bluntly, local authorities could sign up families, wait for time to heal a problem or two and then claim the cash.

Ministers seemed not to ask too many questions about the idea that you could get near-100% success rates at rapidly turning around the lives of a large group of vulnerable people who had already been the subject of repeated (unsuccessful) interventions. The extraordinary success of Troubled Families should have been a red flag.

It is important to stress that effective social work was funded by the scheme - councils seem to have spent the Troubled Families money in the designated areas. But it may have been more efficient to simply give grants to the local authorities to pay social workers, rather than making them jump through the hoops attached to Troubled Families cash.

The Troubled Families programme was a response to the riots in 2011

The trouble with Westminster

Looking beyond Troubled Families, this case raises another big problem. These results have been known to government since last year. Newsnight then published them in August and our report then triggered a select committee investigation. That, in turn, seems to have forced the publication of this full suite of documents.

This series of research papers might not have seen the light of day this week but for that sequence of events. And this is not a one-off - the government suppresses unhelpful research all the time and promotes useful findings. This creates an unhealthy case of "publication bias", akin to the problem that afflicts pharmaceutical research.

This is particularly pernicious in public policy, because the government is often the only agency that can do work on policy. It is the sole custodian of lots of administrative data. If the government is systematically refusing to admit when ministers are making mistakes, it may not be possible for outsiders to spot it.

Perhaps we need to, in effect, create a system where primary research must be published at regular intervals - and bar ministers from suppressing such work.

There is no reason to believe this would damage policy-making. Indeed, openness about research would reveal the extent to which departments have analytical biases in one direction or another. Fixing those would help improve decision-making.

More simply, though, if a minister spends your money investigating a topic, surely you should get to see what they find?

Chris Cook is policy editor for BBC Newsnight