Comments and Reactions: Four Evidence-based MH QI projects

1. Comments and Reactions:Four Evidence-based MH QI projects Tim Cuerdon, PhD Director of Measurement and Evaluation Office of Mental Health Services, VHA December 11, 2008

2. Background / Context • Participated in, and evaluated Quality Improvement projects for about 20 years at: • Program Evaluation and Methodology Division (PEMD) – GAO • QIO (nee PRO) program at CMS (nee HCFA) • NIMH • OQP & OMHS - VHA

3. Results • Fairly typical of most QI efforts • Some, weak, but disappointing amounts of improvement. • Improvements tend to be incremental, not tectonic or quantum • Despite good evidence-based processes and high enthusiasm

4. What have we learned? • Not much • Shot gun approaches seem to work better than single, rifle shot interventions • Effective, charismatic champions seem to be associated with most successful QI efforts

5. How did we get here? • “Just do it” mentality • Moral imperative to deliver the right or best care to all patients has caused a sense of urgency • “We’re not doing research here” • We don’t need “analysis paralysis”

6. What’s Missing? • Theory of the Status Quo – why are things the way they are, when everyone knows there’s a better way? • Identification and appreciation of the forces that cause inertia / resistance to change

7. Implicit Theories • Every QI project reflects an implicit theory, based on the nature of the interventions used. • But because the theory is left unstated, it usually cannot be directly tested, or compared to a competing theory

8. Review of today’s four projects • Rosenheck and Sernyak Cost-Effectiveness Study • Typical of many early QI efforts • Lack of knowledge causes today’s pattern of care • Spreading the good news will lead to positive change • Earlier Patient Education literature challenges the utility of information as a change agent

9. Review of today’s four projects • Owen et al Study • Moves information closer to point of care delivery – reminders / pocket cards. Makes the information more salient • Adds performance monitoring and feedback – suggesting that increased self-monitoring and/or competition might induce change • Team QI vs. Opinion leader – nexus of change “ownership” / motivation

10. Review of today’s four projects • Young et al Study • Clozapine training • Family referrals • Weight / wellness referrals • Feedback reports • Evaluation of QI components also incorporated in the design • Burnout • Cross-service boundaries identified as barriers

11. Review of today’s four projects • Resnick Study • Model sites developed • Other sites come to model to learn by observing • Fidelity to model site features emphasized • But not all model sites are exemplery, and some can do, but not teach

12. Where do we go from here? • Make explicit the cause and effect connection between the intervention and the targeted QI outcome • Use designs that can test out this hypothesized connection • Include a run out, or post-QI phase to detect Hawthorne effects and delayed onsets

13. Where do we go from here? • Debrief QI participants / collaborators (or “victims”) • Conduct post-hoc examinations of most and least successful sites. • Begin to describe and then test the efficacy of the characteristics that make a successful champion / opinion leader

14. Invitation • Implementation of the MH Uniform Services Handbook will generate lots of study opportunities • Psychotherapy vs. pharmacotherapy • Delivery of MH services in specialty clinics vs. integrated and primary care clinics • Delivery of MH services in CBOCs • Delivery of MH services via Tele-medicine.