Surgeon data will lead to alerts over high death rates

When data was released last week about the death rates for individual vascular surgeons, the report from the Royal College of Surgeons stressed reassuringly that "all surgeons are performing within the range expected".

Indeed each doctor named in the document was listed with a green triangle symbol to confirm this status.

But in fact, I have discovered, the few surgeons with the worst apparent records are to be flagged as an "alert", their performance will be investigated and their work will be reviewed.

At this stage it is not possible to say how many doctors, or which ones, this applies to. This is because it has been realised that the published statistics contain some errors. The figures are being checked and updated, and a new analysis should be issued soon by the RCS.

Poor results

On the basis of the data as disclosed last Friday, it is probable that eight surgeons would be flagged as an alert because of their mortality rate on one kind of major surgery, the repair of a highly dangerous bulge in the aorta (an abdominal aortic aneurysm).

And three surgeons would probably get an alert flag due to their poor results on another procedure, to correct the narrowing of an artery in the neck (carotid endarterectomy). It is not clear whether these two groups of surgeons would overlap.

This is out of more than 450 consultants in the UK who perform operations of this sort on major blood vessels.

When the initial data was released last week, the Royal College proclaimed that no individual's performance was an "outlier" because none lay outside an outer limit line on graphs known to statisticians as "funnel plots". The report's graphs show this as the "99.8% limits".

Source: Royal College of Surgeons

This criterion is equivalent to being worse than the average performance by three standard deviations, a statistical measure of distance from the mean or average.

However, when I read the RCS and Vascular Society policy on outliers, I found that it actually states that a surgeon or unit should be flagged as an "alert", prompting further investigation and review, at the threshold of only two, rather than three, standard deviations from the mean. This is broadly equivalent to the 95% limit line drawn on the funnel plots.

'Robust'

It is clear from looking at the funnel plots that several individual surgeons were within this range between the two limit lines, although it is difficult to identify with certainty who they might be due to the way in which the specific data has been provided.

When I put this to David Mitchell, a consultant with North Bristol NHS Trust who chairs the Vascular Society's audit and quality improvement committee, he confirmed that when the data is revised, anyone found to be over two standard deviations worse than the mean will indeed be flagged as an "alert".

The surgeons involved would be notified, and this would result in further scrutiny of their performance data to see if there is any error or justifiable explanation for the outcomes. If the data is confirmed and no explanation is provided, further action would be up to local NHS Trusts and their senior staff responsible for the quality of care.

NHS England has moved to open up health service performance information

"If the data is correct, we will inform their clinical governance leads," says Mr Mitchell. "We can't tell them what to do. Some will take a robust view, others won't."

"Beyond the alert line, our advice would be that there should be a review of the Trust's policy on undertaking this service."

'Exceedingly naive'

The funnel plots are based on risk-adjusted data, which reflect the age of the patients and the nature of the operations. However, the figures published for each named surgeon are not risk-adjusted and therefore are less valid as a basis for comparison.

These statistics are being released as part of a broad drive by NHS England towards opening up health service performance information. While similar statistics have been available for several years for individual heart surgeons, figures on clinical outcomes are now being disclosed for several other surgical specialisms, where national audits are also already established.

The vascular consultants have gone further towards transparency than some of their colleagues in other fields. The vascular data lists all surgeons in one document, whereas others such as the bariatric surgeons (who treat obesity) have only issued look-up tables, where you can look up individual doctors one by one and compare their results to national averages.

Patients now have increasing access to data about their surgeons

Some other specialists are said to regard the Vascular Society as being "exceedingly naive" in this level of openness. Indeed, there was much nervousness among some vascular surgeons about releasing outcomes by doctor rather than by hospital, and six refused to have their individual figures published.

False reassurance

But Mr Mitchell is convinced it is the right approach: "Everything is laid out there in a way you couldn't get from other audits that are reporting. We're allowing people to make a choice. That's the point of data."

Some patients may now feel they were provided with false reassurance over last week's figures when they were told that no surgeon's record was outside expectations, and they may find that disturbing.

But in another view patients may also feel more reassured by the discovery that the relevant professional bodies are not complacently accepting that all the surgeons involved are performing acceptably - and instead they are challenging those who, the evidence suggests, may not be.