Applying Multiple Frameworks and Theories in Implementation Research

1. Applying Multiple Frameworks and Theories in Implementation Research Jeffrey Smith Implementation Research Coordinator Mental Health QUERI

2. “In theory there is no difference between theory and practice… in practice there is.” Yogi Berra

3. Definitions • Formative evaluation (FE) • Rigorous assessment process designed to identify potential and actual influences on the progress and effectiveness of implementation efforts(Stetler et al, JGIM 2006; 21:(Suppl 2):S1-8.) • Summative evaluation • Systematic process of collecting data on impacts, outputs, products, outcomes for a study • Used to assess success, effectiveness, or goal achievement

4. MH QUERI Approach to Implementation • Design interventions based on theory and/or results from formative evaluation • Conduct formative evaluation • identify determinants of care • tailor intervention design and implementation to local context • assess barriers to implementation • Use external facilitation techniques • Identify and test new approaches and methods for overcoming barriers • Conduct summative or impact evaluation

5. An Approach to Using Theory for Implementation Planning Select interventionsthat fit with plannedstrategies (based on theory) Select theory of planned behaviorchange Identify potential strategies for achieving change Assess fit with initial theory Evaluate effectivenessof intervention,strategies, tools Launch interventionusing identified toolsand strategies Identify interventiontools that fit bothstrategy and theory From: Sales A, Smith JL, Curran G, Kochevar L. Models, strategies and tools: The role of theory in implementing evidence-based findings into health care practice. Journal of General Internal Medicine 2006; 21:S43-49.

6. Antipsychotic Treatment Improvement Program (ATIP) • Goals • Improve adherence to CPGs for psychosis • Reduce use of very high doses of antipsychotic medications • Encourage use of newer “atypical” antipsychotic medications for non-responders to conventional antipsychotic medications

7. ATIP Application of Frameworks, Theory and Planning Models (cont) • Social Influence Theory, Diffusion of Innovation, Social Cognitive Theory • Utilize influential local clinician leaders (opinion leaders) to inform other clinical staff about evidence-based antipsychotic medication management, model-targeted prescribing behaviors, and motivate practice change • Complexity Theory • Initial conditions in HCOs are important in intervention planning, but HCOs are adaptive and change over time • Need mechanism for assisting clinical partners in modifying or adapting initial intervention strategy based on changing circumstances, initial success

8. ATIP Application of Frameworks, Theory and Planning Models (cont) • Promoting Action Research Implementation in Health Services (PARiHS) • Use external facilitation techniques to maintain contact with clinical opinion leader to assist with problem-solving and addressing challenges to intervention implementation • PRECEDE Planning Model (Predisposing, Reinforcing, and Enabling Constructs in Education Diagnosis and Evaluation) • Address predisposing factors (eg, knowledge, attitudes) that influence EBP adoption • Enable providers to follow guideline recommendations at the point of care • Apply social incentives through performance audit/feedback to reinforce providers’ implementation of EBPs

9. “Refined” Description of Facilitation Based on VA QUERI Experience* Facilitation is a process of interactive problem-solving and support to meet specific implementation goals, which occurs in the context of a recognized need for improvement and a supportive interpersonal relationship. * Stetler CB, et al. Implementation Science 2006; 1:23.

10. Operationalization of External Facilitation in MH QUERI • Facilitator • Maintains regular contact w/ local QI leader / team • Email communication • Telephone • Participate in QI Team meetings (as feasible) • Assist in goal-setting; monitor implementation of project tools/strategies; performance on clinical QI goals • Identify and problem-solve barriers to implementation • Assist in adapting tools/strategies as needed or suggested by site partners to meet project goals

11. ATIP Intervention Components

12. Formative Evaluation: Barriers and Lessons Learned • Physicians do not always agree on who is an opinion leader; some sites may have no opinion leader • Some sites had poorly developed formal and informal social networks among physicians • A focus on physicians only as agents of change • How much directive should be given to the opinion leaders concerning how to influence attitudes and behaviors? From: Curran GM, Thrush CR, Smith JL, et al. Jt Commission J Qual Safety 2005; 31(12):700-707.

13. Summative Evaluation • ATIP intervention improved antipsychotic medication management in concordance with CPGs • Reduced pharmacy costs for antipsychotics • Participating clinicians reported positive experiences with ATIP educational materials and clinical support tools

14. Summary Application of multiple frameworks/theories in guiding intervention design and implementation can be successful in implementing EBPs accommodates tailoring to setting when combined with formative evaluation acknowledges there are generally multi-level determinants to complex, clinical QI issues organizational-level team-level interpersonal-level individual-level

15. Strengths • useful in designing multifaceted interventions to influence multi-level determinants of care (flexible) • allows integration of theory, knowledge, methods from multiple disciplines (multidisciplinary) • Limitation • Can be unwieldy… need to provide rationale for applying multiple theory approach, and rationale for selecting the specific frameworks / theories applied • Key Guidance on Evaluation • combine with rigorous formative evaluation • conduct summative (impact) evaluation to assess intervention effectiveness on key study outcomes • confirm, refute or propose refinements to selected theory(ies) based on study findings Multiple theory approach

17. Stages of FE Pre-Implementation Post-Implementation Implementation • Interpretive • Assess intervention usefulness/value from stakeholder perspective • Elicit stakeholder recommendations for further intervention refinements • Assess satisfaction with intervention and implementation process • Identify additional barriers / facilitators • Implementation-Focused • Assess discrepancies between implementation plan and execution, exploring issues of fidelity, intensity, exposure • Understand and document nature and implications of local adaptation • Developmental • Identify determinants of current practice • Identify potential barriers / facilitators • Assess feasibility of proposed intervention • Integrate findings into intervention design and refinement prior to implementation • Progress-Focused • Monitor impacts and indicators of progress toward project goals • Use data to inform need for modifying or refining original strategy • Provide positive reinforcement to high performers; negative reinforcement to low performers