Working Group: Can Six Blind Men Find Apples & Oranges? Measuring Variable Implementation of QI Interventions Using

1. Working Group:Can Six Blind Men FindApples & Oranges?Measuring Variable Implementation of QI Interventions Using Multiple Data Sources

2. Presenters • Alexander S. Young, MD, MSHS • Elizabeth (Becky) Yano, PhD, MSPH • Lisa V. Rubenstein, MD, MSPH • Alison Hamilton, PhD

3. Overview • 90 minutes: presentations • 60 minutes: group discussion and breakout groups • 30 minutes: group consensus on priorities, suggested next steps, directions • Working group moves to plenary • 5 minute summary presented

4. Overview of Working Group • Presentations • Introduction and overview (Alex) • QUITS smoking cessation trial (Becky) • TIDES depression collaborative care (Lisa) • EQUIP evidence-based practice in schizophrenia (Alison)

5. Goal of Presentations • Brief orientation to example QI intervention • How context matters  sets us up for variable QI intervention deployment • Process for intentional adaptation of evidence into context of local practice • Types of data sources brought to bear on measuring implementation • including development of a fidelity score • Triangulation of data sources to tell story

6. QI Intervention (QII) Examples • QUITS (Quality Improvement Trial for Smoking cessation) • evidence-based quality improvement to implement smoking cessation guidelines • Scott Sherman MD & Becky Yano PhD (co-PIs) • TIDES (Translating Interventions for Depression into Evidence-based Solutions) • depression collaborative care model • Lisa Rubenstein MD & Ed Chaney PhD (co-PIs)

7. QI Intervention Examples (cont’d) • EQUIP (Enhancing QUality of care In Psychosis) • evidence-based quality improvement to implement effective treatments in schizophrenia • Alex Young MD & Amy Cohen PhD (co-PIs)

8. TIDES Depression Collaborative Care • Evidence base: • >20 RCTs • Depression • toolkit Provider/patient education Depression care manager EBQI QI Informatics support Performance feedback “adaptation” “priority-setting” Leadership support

9. Context Matters: Design for It • TIDES • 2:1 intervention-to-control sites x 3 VISNs (6 intervention + 3 control sites total) • VISN leaders chose sites, we randomized within network (block on network characteristics) • QUITS • regional concentration in southwest (3 VISNs) • matched on size/academic affiliation within VISN • we chose sites and randomized within network • EQUIP • 1:1 intervention-to-control sites x 4 diverse VISNs • sites chosen based on leadership interest

10. Context Matters: Input from Sites • Attitudes / beliefs / experiences • perceived need for the intervention • competing demands • staff openness to innovation • Resources • perceived time to use program and participate in implementation • organizational structure, staffing, prior QI experience, informatics support Source: Kirchner JE, Parker LE, Yano EM, COVES evaluation (2007).

11. Multiple Data Sources: Measuring Implementation

12. Triangulation • Critical to collect information about implementation from multiple sources • be prepared for disagreement • perspectives and opportunities for observation differ for managers, providers vs. patients • Recognize differences between “exposed” sample and practice population • does the “enrolled” group represent the practice? • did the intervention penetrate among all providers?