Audit Scotland calls for bold decisions on poor quality data

• Scotland's health secretary Humza Yousaf and Home Secretary Suella Braverman are being put under pressure by numbers. Data is the modern economy's gold or oil, and in the political arena, it is dangerous material.

• However, watchdogs of government spending are warning about the poor quality of data they have to work with, notably in social security in Scotland, and the limited use of it to achieve better outcomes in public services.

• Audit Scotland is calling for bold decisions and radical innovation, to change data from being seen as a time-consuming burden on public sector workers to the means by which services can be re-shaped and improved. Accountability, however, can be a challenge to those not used to it.

When Aneurin Bevan launched the National Health Service in 1948, he wanted to emphasise that its political leadership should be held accountable for its performance.

"If a hospital bedpan is dropped in a hospital corridor in Tredegar, the reverberations should echo around Whitehall," the health secretary (is reputed to have) said.

A lot has changed in nearly 75 years, and not just that Tredegar's hospitals are now funded through the Welsh Senedd. Accountability can now be achieved through setting targets for bedpan-dropping, counting and reporting the number dropped, the decibels of reverberation and the subjective impact on patients' experience.

You can then analyse the wards and NHS staff where bedpan-dropping is most prevalent, cross-reference to other performance indicators, followed by a bonus payment for the hospital manager if bedpan-dropping incidents come in on target.

Maybe not bedpans, admittedly, but such data is taking over. We're told digits are the new gold or oil. If you can sift, harness and interpret data, you have power in the modern economy: controlling robotics, autonomous vehicles and artificial intelligence, but more commonly measuring staff productivity, and getting to know your customers better than they know themselves.

Its uses are driving change in the private sector. But in the public sector, it tends to be a different story, and one with which public sector workers are far from happy.

Form-filling

What gets counted becomes what matters. And if politicians are elected on a manifesto that specifies maximum class sizes or police officer numbers, achieving those targets can be at the expense of better education or policing, or of the support staff and other resources that teachers and detectives need.

The key performance indicators that measure the quality of delivery of a public service are harder to gauge accurately, so there's a risk they are pushed down the list of priorities.

In the health service, you hear anecdotally that hard-pressed staff are running to catch up with data, at the expense of what they often judge would be a better service. Teachers complain of form-filling that has taken over their working lives.

With political accountability, data carries risk if it falls into opponents' hands, or is misinterpreted in the media. And when it's measuring a public service that is not working, regular publication puts those who run the service under immense strain.

NHS waiting time data puts pressure on Health Secretary Humza Yousaf

For health secretary Humza Yousaf, there is the political pressure of frequent updates on waiting lists, which he can then use to pressure health boards on their performance. When he was transport minister, the Glasgow MSP was both the victim and deployer of ScotRail reliability stats in much the same way.

• One in six patients should not be in hospital

The pressure on the Home Secretary, Suella Braverman, is piled on by the numbers on immigration and refugees. The smarter opposition politicians are being shown to be the ones who can deploy statistics most effectively.

But from Audit Scotland on Tuesday comes a warning - or, indeed, two warnings. One is that the costing data on social care in Scotland is a long way wide of a reliable mark. The other is that the use of data in public services is a long way from where it should be.

'A bit of a mess'

An unusually frank blog has been published on the spending watchdog's website quoting a recent round-table discussion in which the general view seemed to be: "It's a bit of a mess".

"People producing data are often stuck in a cycle of reporting for reporting's sake," says the blog. "Often those people capturing data are on the frontline, already hard-pressed, and don't see its wider benefits, leading to missing or poor-quality data. There are also concerns that data will be misused if shared."

It goes on: "Public sector leaders, too, are not clear on what data they have and how to use it. Or they find that the data they want is simply missing or doesn't exist. This all uses precious staff resources but without delivering the value and insights we need".

You might think this is a predictable gripe of those who live by data of a financial nature and lots of other key performance indicators, and then base their judgements on that.

But they're not alone. Economists also complain that it's hard to know what's really happening in the engine room of the Scottish economy because there are significant gaps in available statistics - on pay, prices, imports and trade in services, for instance.

The Reform Scotland think tank recently pointed to the number of targets set by the Scottish government to reduce climate changing emissions and reach net zero, but which have little data back-up to see if anyone is on track to reach those targets.

Sex and gender

The Scottish Fiscal Commission regularly updates on the data it needs but can't get hold of. Two months ago, it published a 36-page report on its data needs, reflecting some improvements in tax collection, but with more pressure to get greater clarity with spending budgets, and a significant list of issues in its dealings with the new operations of Social Security Scotland.

The budget watchdog says it is not getting the same quality of data on Scottish claimants that it gets from Whitehall's Department of Work and Pensions. It has been told that is due to a lack of resources, and that it is reviewing its use of data.

Unless that is sorted out soon, and as Whitehall data becomes older, the Scottish commission says its forecasts of the multi-billion pound welfare budget at Holyrood will become less reliable.

Just one of its problems with Social Security Scotland is that data collection does not include male and female categorisation, apparently because sex and gender has become such a sensitive subject for some. The Commission points out that, without that basic level of data, how can it forecast the benefits bill for a disability that is more common among men than women, or girls than boys?

If that seems problematic, it may be nothing compared with the potentially gaping holes in data arising from the very low compliance with this year's Scottish census. Groups - demographic, geographic or ethnic - that tended not to engage with the census process are at risk of losing out on the distribution of resources. Or put the other way round, government may find demand for services is far greater in some ways than the data say it should be.

Bold and radical

Audit Scotland wants to offer an incentive for the public sector to do better with data. If done right, it can help inform decisions about how best to use resources, to deliver better outcomes and value for money for the public. It can help identify users' needs and issues that are being overlooked. All of this is with the proviso that it has to be done in a secure, transparent and ethical way, which takes public trust on the journey.

It is pointed out that the Covid vaccination programme used data in innovative ways, and achieved more than many thought possible, though it also identified gaps. Scandinavian researchers and health workers join forces with their data and their understanding of how to use it so that they get better social outcomes.

But back in Scotland: "Often, we, and other public sector bodies, simply cannot get the specific evidence we need to decide whether money has been well spent".

The answer, says Audit Scotland's blog author Gemma Diamond, lies in "bold" decisions and even "radical" departures.

Try starting with an end to duplication, she suggests. Get IT systems that speak to one another. Getting to a data-driven culture in public services will require investment and resources, she says, and it will be necessary to stop doing or counting some things in order to do or count new ones.

It may also get awkward. What about the school which has all the data it needs on the exam under-performance in one teacher's class when set against that of another? Is the head teacher or department head going to use the data to drive better performance?

They can, but will they, for fear of resistance or non-cooperation from those who feel threatened? What if that level of data was made available to parents, as it has become possible to see some data about patient outcomes from different services?

Data opens up possibilities to make public services more accountable, and some won't like it.