10 February 2012
Volume and outcome studies provide unreliable evidence used in proposals to centralise hospital services
Proposals for centralising hospital services which are founded solely on studies linking the volume of hospital activity to the outcomes achieved are based on unsafe evidence. That is the conclusion drawn in a paper published today in the journal Health Services Management Research (HSMR). Most such studies fail to demonstrate a causal link between volume and outcome and consequently should not be used to justify centralisation.
Calling for new guidelines for assessing the case for centralisation, the paper's author, Anthony Harrison, Research Associate, King's Fund, said: "Systematic reviews of the evidence linking volume of activity of a hospital or clinician to the outcomes they achieve have confirmed that it is unreliable, yet official documents continue to use volume and outcome studies in support of the changes they propose."
What appear to be statistically sound relationships between volume and activity may be misleading because they do not adjust for confounding factors such as differences in case-mix. These may arise because of differences in the characteristics of the catchment areas of different hospitals or because healthier people with higher incomes are more likely to travel to a more distant hospital. When differences in case-mix are taken into account, the apparent benefits of scale may be reduced or even disappear.
Mr Harrison said: "Even when the case-mix is accounted for, the overall result may hide the fact that some large units or surgeons carrying out a large number of operations produce poor results. There is also evidence that performance varies widely even among experienced surgeons." Even with a well-founded relationship between volume and outcome in statistical terms, Mr Harrison warns against assuming that better outcomes will result from centralisation. If applied across the board, centralisation could result in high-performing smaller units closing and poor-performing larger units expanding.
"In the light of the evidence it is hard to understand why volume and outcome studies continue to be used as a guide to action" said Mr Harrison. "One reason may be that it seems intuitively reasonable that volume and outcome should be positively related, on the grounds that 'practice makes perfect'." However, most volume and outcome studies cannot prove that this is the case since very few use data extending over different time periods and consequently cannot identify whether performance improves as experience accumulates.
A key question is whether factors which result in better outcomes in hospital units carrying out higher volumes of activity can be reproduced in hospitals with lower volumes of activity. These factors include specially trained staff, effective organisation and the use of defined protocols for delivery of care during and after surgery. The answer could lie in the effective transfer of knowledge and expertise between hospitals. However, in the case of emergency services such as major trauma or hyperacute stroke, where skilled staff and equipment need to be available at all times, smaller units may not be able to provide such availability at an acceptable cost.
"Proposals for centralising services continue to cherry pick the evidence or overstate the strength of the evidence from volume and outcome studies," said Mr Harrison. "They should instead be based on a much broader-based analysis and at a minimum include measures such as outcomes actually achieved, an assessment of the implications for access, short-run and long-run changes in costs, the availability of other factors, particularly skilled staff, and finally the implications for other services both at the losing and gaining institutions."
Notes for editors
Assessing the relationship between volume and outcome in hospital services: implications for service centralization by Anthony Harrison is published in the February issue of Health Services Management Research (HSMR). The paper will be published online at 00.01hrs Thursday 9 February. Please make sure you mention or link to the journal in your piece. Many thanks.
For further information or a copy of the paper please contact:
Royal Society of Medicine
media @ rsm.ac.uk
DL:+44 (0) 1580 764713
M: +44 (0) 7785 182732