420 likes | 548 Views
Does transparency improve quality? lessons learnt from cardiac surgery . BCIS meeting 2006 Ben Bridgewater SMUHT. History of cardiac surgical audit. Cardiac surgery register since 1977. History of cardiac surgical audit. Cardiac surgery register since 1977 UK database since 1994.
E N D
Does transparency improve quality?lessons learnt from cardiac surgery BCIS meeting 2006 Ben Bridgewater SMUHT
History of cardiac surgical audit • Cardiac surgery register since 1977
History of cardiac surgical audit • Cardiac surgery register since 1977 • UK database since 1994
History of cardiac surgical audit • Cardiac surgery register since 1977 • UK database since 1994 • Dr Foster/The Times 2001
History of cardiac surgical audit • Cardiac surgery register since 1977 • UK database since 1994 • Dr Foster/The Times 2001 • Named unit mortality SCTS 2001
History of cardiac surgical audit • Cardiac surgery register since 1997 • UK database since 1994 • Dr Foster/The Times 2001 • Named unit mortality SCTS 2001 • SCTS individual ‘standards’ 2003
History of cardiac surgical audit • Cardiac surgery register since 1997 • UK database since 1994 • Dr Foster/The Times 2001 • Named unit mortality SCTS 2001 • SCTS individual ‘standards’ 2003 • Guardian named surgeon data 2005 • Freedom of Information Act
History of cardiac surgical audit • Cardiac surgery register since 1977 • UK database since 1994 • Dr Foster/The Times 2001 • Named unit mortality SCTS 2001 • SCTS individual ‘standards’ 2003 • Guardian named surgeon data 2005 • Healthcare commission named surgeon data 2006
History of cardiac surgical audit • Cardiac surgery register since 1997 • UK database since 1994 • Dr Foster/The Times 2001 • Named unit mortality SCTS 2001 • SCTS individual ‘standards’ 2004 • Guardian named surgeon data 2005 • Healthcare commission named surgeon data 2006
History of cardiac surgical audit • Cardiac surgery register since 1997 • UK database since 1994 • Dr Foster/The Times 2001 • Named unit mortality SCTS 2001 • SCTS individual ‘standards’ 2004 • Guardian named surgeon data 2005 • Healthcare commission named surgeon data 2006
History of cardiac surgical audit • Cardiac surgery register since 1997 • UK database since 1994 • Dr Foster/The Times 2001 • Named unit mortality SCTS 2001 • SCTS individual ‘standards’ 2004 • Guardian named surgeon data 2005 • Healthcare commission named surgeon data 2006
History of cardiac surgical audit • Cardiac surgery register since 1997 • UK database since 1994 • Dr Foster/The Times 2001 • Named unit mortality SCTS 2001 • SCTS individual ‘standards’ 2004 • Guardian named surgeon data 2005 • Healthcare commission named surgeon data 2006
Issues • Has public accountability improved quality?
Issues • Has public accountability improved quality? • Is there now a culture of ‘risk-averse’ behaviour?
Has public accountability improved quality? Mortality significantly higher than average – Dr Foster Mortality significantly lower than average – Healthcare commission
Risk adjusted mortality – National data – isolated CABG Increased predicted risk Decreased observed mortality
Hawthorn effect Public disclosure • New York state database • Pennsylvania report cards • SCTS database • Northern New England Cardiovascular study group • VA database • NW regional audit project 1997 to 2001 No disclosure
Why is public reporting important? Because it has driven data collection and use Clinicians managers support staff professional organisations
Is there now a culture of risk averse behaviour? • Newsnight survey of UK cardiac surgeons 2000 • 80% surgeons in favour of public accountability • 90% felt that high risk cases would be turned down • Only 6% felt that available algorithms adjusted appropriately for risk See also Burack 1999, Schneider and Epstein 1996, Narins 2005
Existing data • Little ‘hard’ statistical data investigating the influence of public accountability on cardiac surgical practice • NY experience suggests conflicting data • Hannan 1996 • Dranove 2003
Is there risk averse behaviour in the UK? • Very difficult to measure surgical ‘turndowns’ • If there was significant risk averse behaviour you would expect to see a decrease in the number of high risk cases coming to surgery • Complex issues with respect to surgical case mix due to PCI developments
Northwest data 1997 to 2005 • 25,730 patients under 30 surgeons • Isolated CABG alone • Observed and predicted mortality • Number of low risk, high risk and very high patients each year • 2 time periods • 1997 to 2001 – prior to public disclosure • 2001 to 2005 – post public disclosure
Results • Significant decrease in observed mortality • Significant increase in overall predicted mortality • Significant decrease in risk adjusted mortality
Results • Significant decrease in observed mortality • Significant increase in overall predicted mortality • Significant decrease in risk adjusted mortality
Is there now a culture of risk averse behaviour? • No overall effect • May be transient or individual effects • Important that this is ‘mopped up’
Is there now a culture of risk averse behaviour? • What is perceived by someone as risk-averse behaviour is perceived by another as good clinical decision making
Is there now a culture of risk averse behaviour? • What is perceived by someone as risk-averse behaviour is perceived by another as good clinical decision making • Transparency may have focussed the multidisciplinary team on optimising treatment strategies for individual patients
Risk adjustment • ‘No model is perfect – some are useful’
Risk adjustment • ‘No model is perfect – some are useful’ • Need clarity around ‘fit for purpose’
Risk adjustment • ‘No model is perfect – some are useful’ • Need clarity around ‘fit for purpose’ • Arguments about models can paralyse developments
Risk adjustment • ‘No model is perfect – some are useful’ • Need clarity around ‘fit for purpose’ • Arguments about models can paralyse developments • Model ‘drift’ • Calibration and weightings
Risk adjustment • ‘No model is perfect – some are useful’ • Need clarity around ‘fit for purpose’ • Arguments about models can paralyse developments • Model ‘drift’ • Calibration and weightings • Progress will be too slow for some and too quick for others