Death data 'not good enough to spot poor surgeons'

Heart surgery is one area where individual data is robust, the study says

Publishing death rates for individual surgeons will not spot poor performers in certain fields because too few operations take place, a study finds,

In June, the NHS in England published the first data, including death rates, for vascular surgeons, with nine more specialist areas to follow.

But the paper says it would be better if some data was shown per hospital.

NHS England said the reason for the initiative was transparency - not to try to spot poorly performing doctors.

'Statistically reliable?'

The team from the London School of Hygiene and Tropical Medicine used hospital data from England to look at death rates in four specialist areas,

These were adult heart surgery, bowel cancer resection (removing part of the colon or rectum), oesophagectomy (removing the oesophagus) and gastrectomy (taking out all or part of the stomach), and hip fracture surgery.

They then assessed how many procedures would be necessary for reliable detection of poor performance, and how many surgeons in English NHS hospitals actually do that number of operations.

The researchers concluded enough heart and hip operations were carried out to make death rate data statistically reliable.

But they suggest the same is not true for the other categories they studied.

For example, the median number of bowel cancer resections carried out by an individual surgeon was nine - meaning the number was too small and too easily affected by a range of other factors, such as one surgeon doing more complex operations than their peers on sicker patients.

In addition, death rates after hip surgery are extremely low and so would not give a fair reflection of the procedure's success.

The researchers suggest post-operative quality of life might be a better measure for this specialty.

Other factors

They conclude that in many areas, individual performance data would not be robust - and may give a false sense of complacency because "an absence of evidence could be falsely interpreted as evidence of acceptable performance".

Dr Jenny Neuburger, who led the research, said: "The reporting of results for individual surgeons should be based on outcomes that are fairly frequent, and fortunately, from the point of view of patients, mortality is not one of them.

"For specialties in which most surgeons do not perform sufficient numbers of operations to reliably assess their outcomes, reporting should be at the level of the surgical team or hospital, and not the surgeon."

Prof Jan van der Meulen, who also worked on the paper, said there should be as much transparency as possible.

But he said: "It's not just the surgeon who has an impact - it's the whole care package, the whole team, the pre and post-operative care."

Prof Norman Williams, president of the Royal College of Surgeons, said: "The driving force behind the publication of surgeon-level outcomes data is to improve surgical performance and allow patients to find out more about the quality of care provided by hospitals and individual surgeons."

He said it was "an extremely challenging project" and particularly complex for cancer operations where there were "low volumes of surgery".

But Prof Williams said: "This is the beginning of a new approach to transparency in healthcare.

"Auditing some areas of surgery is undeniably easier than others.

"This is by no means a reason not to measure outcome as long as the complexities are understood and interpreted correctly - and the profession is committed to doing so, no matter how difficult it may be."

And Prof Sir Bruce Keogh, medical director at NHS England, said: "The authors have misunderstood the purpose of public disclosure of information.

"It is not a statistical exercise to identify poor performers. Rather it is an exercise to provide information on activity and outcomes, to focus minds on improving results and to assure the public that the NHS offers high quality surgery.

"But it does raise the question as to whether it is better to have your complex operation performed by a surgeon doing a lot of that operation or only a few."