Ethical QI A Rational Approach to Review and Oversight, or How to Avoid the Big Chi ll

1. Ethical QIA Rational Approach to Review and Oversight, orHow to Avoid the Big Chill Don Goldmann, M.D. Senior Vice President Institute for Healthcare Improvement Professor of Pediatrics Harvard Medical School

2. Bedrock Principles of Ethical QI • Make the decision regarding the need for IRB review as simple as possible by • Respecting individual privacy • Assuring confidentiality, regardless of whether data are used for QI or research • Assuring that the QI project poses no more than minimal risk

3. Review Criteria That Don’t Make Sense • “Projects with the intent to produce generalizable knowledge or publications are research and require IRB review” • The goal of sound QI is to build and share knowledge about how to improve quality and safety. Indeed, it is our ethical obligation to share and publish our insights

4. Sensible Criteria for an IRB Exemption (At The Most, Expedited Review) • Projects are exempt from IRB review if they are designed to improve care so as to conform more reliably to established or accepted standards (evidence-based or supported by strong consensus) • Evaluation is intrinsic to improvement • It is counterintuitive to suggest that evaluating QI efforts is research • Failure to evaluate is incompatible with learning and may be unethical • But, feedback of data (both process and outcome data) in real time is essential • Withholding data from participants so as not to “contaminate” the evaluation converts QI to research

5. Surveys • Surveys designed to gauge the opinions and perceptions of external customers, patients, staff, and trainees are considered integral to an organization’s quality oversight activities • Privacy and confidentiality must be respected • To avoid perception of coercion or possible repercussions, include language such as: • “This is an anonymous survey. Results will be presented only as aggregate data, with complete protection of individual anonymity. Completion is entirely voluntary.” • If there is intent to publish, those being surveyed are informed that they can opt out by returning a blank survey

6. Who Provides Oversight if the IRB Does Not? • Hospitals must have a robust process for insuring that QI projects are sound and have the potential to produce new knowledge • Poorly design QI projects waste time, energy, and resources • Such projects are at best futile and at worst unethical • Potential harm and unintended consequences must be considered carefully and monitored aggressively • Example: active surveillance tests for MRSA

7. Hospital Oversight • IRBs have years of operating experience and learning, often with strong quality control • Very few hospitals have the experience, knowledge, trained personnel, processes, and infrastructure to assure that QI projects: • Have appropriate, meaningful goals • Are well designed • Have a sound measurement and evaluation strategy • Promote shared learning and spread of successful strategies • These basic principles apply to small tests of change in microsystems, as well as to more ambitious, larger scale projects and collaboratives

8. Weighing the Evidence • How much evidence is required before deciding to spread change? • What kind of evidence is appropriate? • Randomized controlled trials • Cluster randomized trials • “Adaptive,” “Dynamic” randomized trials • Quasi-experimental studies • Statistical process control • Time-series analysis • Qualitative studies • Behavioral science, Sociology, Anthropology • Mixed methods • Even if the evidence looks strong, thoughtfully monitor unintended consequences

9. The Case of Rapid Response Teams • “Early trials of medical emergency teams suggested a large potential benefit – to the point that some observers regarded further study as unethical. However, a large, randomized trial subsequently showed that medical emergency teams had no effect on patient outcomes.” Auerbach, et al., NEJM 2007:357:608-613

10. The MERIT Cluster Randomized Trial • 23 Australian hospitals randomized • 2-month baseline, 4-month preparation period, 6-month intervention • Superb statistical analytic plan • More inter- and intra-hospital variance than expected, much lower event rate than expected • Increased call rate in intervention hospitals, but no effect on outcomes • Impressive reduction in mortality in both arms of study • Sub-optimal team activation in patients with call criteria MERIT Study Investigators, Lancet 2005;365:2091-2097

11. What If…. • Baseline data were used to adjust power • Study would have been declared “futile” • Performance data were fed back in real time • QI were encouraged to improve performance • Mixed methods were used to understand context and outcomes in individual sites

12. Lessons • Every QI “experiment” should use the most appropriate evaluation method for the question and context • The broadest possible palette of methods should be utilized • No opportunity to learn should be wasted • “Futile” QI projects, whether large or small are wasteful and perhaps unethical