How accurate is hospital A&E data?

It's an important and high profile benchmark for NHS performance - the number of patients having to wait more than four hours in A&E to be treated, assessed or for a decision to be admitted to a hospital.

Government ministers and NHS leaders are held to account on performance against the target of 95% of patients dealt with in four hours. This winter most hospitals around the UK have struggled to get near it.

But how much can we trust the data? This week has seen major questions raised about how the figures are compiled.

This follows a BBC story revealing that some trusts in England had started including data last October from minor injuries centres run by other providers. This had resulted, in some cases, in an improvement by several percentage points in their performance. But they had not made clear what they were doing.

The plot thickened with the BBC revelation of emails from the regulator NHS Improvement appearing to encourage hospitals to pull in figures from these care centres.

In response, the regulator said it was up to local systems to review their reporting and there was no intention to artificially inflate A&E performance figures.

Credit is due to my colleague Faye Kirkland for her investigation of this. The stats watchdog UKSA called for explanations and said the practice could have resulted in "misleading conclusions".

'Seriously inconsistent'

Now NHS Improvement has written to all trusts in England calling on them to review how they report their figures. The letter from the chief executive Ian Dalton refers to new pathways for emergency care (for example GP-led care centres) and says they should be "appropriately reflected" in national figures and that consistent principles are needed to decide whether they should count towards the current A&E target.

It adds that the trusts should list separately figures from minor injuries and urgent care centres which are not located on the same sites as the hospitals.

Ambulance A&E delays hit one in eight

Long A&E waits: 3m a year wait over four hours

The letter reveals the concern felt at the highest level of NHS Improvement that data reporting around the country may be seriously inconsistent:

"It is extremely important that local and national A&E data accurately reflects evolving clinical practices. We must also maintain the integrity of A&E attendance data, which is used to understand the pressures you are under, and ensure that we can understand and explain how this data has changed over time."

If you can't see the NHS Tracker, click or tap here.

The problems focus on how to report patients who arrive at the front door of A&E and are referred quickly to a GP-led centre. Some trusts don't include these patients at all in their data returns, but others do.

A trust might even record that it has "assessed" a patient within four hours, even though that individual subsequently waits several hours more to see a doctor in an urgent care centre. The background to all this is the evolving nature of these "pathways" with increasingly sophisticated streaming of patients. No longer is A&E the only place to go with every kind of injury or illness.

'Real headache'

It is admirable that NHS Improvement is trying to achieve clarity and consistency. But the letter raises more questions than it answers. If hospitals list separately all patients seen at these centres will they still count towards their overall performance against the four-hour target?

How far back should hospitals go to achieve a consistent set of data? Some hospitals might have only started including data from the centres last October while others were doing so for much longer.

It is a muddle and a real headache for the trusts and NHS Improvement. Their nightmare scenario is the possibility that the data reported monthly through this winter and for months before has been wrong.

Labour's Jonathan Ashworth has called for Health Secretary Jeremy Hunt to carry out an urgent investigation into the lack of transparency over the changes and to update MPs on whether the winter has been worse than currently understood.

NHS leaders want patients to be directed to urgent care centres and minor injuries units when appropriate as this takes the pressure off A&E and allows those units to concentrate on the patients most in need of emergency care.

But they need to be recorded somewhere. Above all, NHS leaders, politicians and patients need to be sure that data on the performance of their local hospital and the system as a whole is robust and trustworthy.