1 / 28

Implementation, Evaluation and Getting to the Triple Aim

This article explores the implementation and evaluation of the Triple Aim approach, focusing on population health and patient experience. It discusses the challenges of translating research into practice and highlights the importance of context in evaluating interventions. The RE-AIM framework is introduced as a tool for planning, evaluating, and reporting studies. Case studies, such as the Advancing Care Together program, demonstrate the use of RE-AIM in integrated care initiatives. The article also discusses the benefits of integrated care in improving patient outcomes and provider experience.

hadsell
Download Presentation

Implementation, Evaluation and Getting to the Triple Aim

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Implementation, Evaluation and Getting to the Triple Aim Deborah J. Cohen. PhD Oregon Health & Science University Russell E. Glasgow, PhD University of Colorado School of Medicine

  2. Overview • Implementation and Implementation Science • Evaluation, Learning and the Triple Aim • Population Health • Patient Experience • Take Home Points

  3. Implementation Science Pragmatic Challenge: Much Research is Not Relevant to Practice • Traditional RCTs study the effectiveness of treatments delivered to carefully selected populations under ideal conditions. • This makes it difficult to translate results to the real world. • Even when we do implement a tested intervention into everyday clinical practice, we often see a “voltage drop”—a dramatic decrease in effectiveness. “If we want more evidence-based practice, we need more practice-based evidence.” Green LW. Am J Pub Health 2006 Rothwell PM. External validity of randomised controlled trials…Lancet 2005;365:82-93.

  4. Implementation Science, Multi-level Research-Practice Contextual Systems Approach Organization Evidence-Tested Program Program as Tested Delivery Site(s) Fit Program Delivery Staff Critical Elements Non-critical Packaging Program as Marketed Partnership Design Appropriate for Question Research Design Team And Adaptive Design Adapted from Estabrooks P. et. al. AJPM, 2005, 31: S45

  5. Models for Implementation Science Tabak RG et al, Bridging Research and Practice: Models for Dissemination and Implementation Research Am J Prev Med, 2012, 43: 337-350; • Models – theories and frameworks • What can they do: • Make the spread and uptake of interventions more likely • Provide systematic structure for the development, management, and evaluation of interventions/D&I efforts • Wealth of existing models for D&I • 61 identified by Tabak et al in a recent review • Additional models with practitioner focus

  6. RE-AIM To Help Plan, Evaluate and Report Studies R Increase Reach E Increase Effectiveness A Increase Adoption I Increase Implementation M Increase Maintenance Glasgow, Klesges, Dzweltowksi, et al. Ann Behav Med 2004;27(1):3-12

  7. Precision (Personalized) Medicine Questions in IS • PawsonR, et al. J Health Serv Res Policy 2005;10(S1):S21-S39. • Gaglio B, Glasgow RE. Evaluation approaches…In: Brownson R, Colditz G, Proctor E, (Eds).Dissemination and implementation research in health: Translating science to practice. New York: Oxford University Press; 2012. Pages 327-356. Determine: What percent and types of patients are Reached; For whom among them is the intervention Effective;in improving what outcomes; with what unanticipated consequences; In what percent and types of settings and staff is this approach Adopted; How consistently are different parts of it Implementedat what cost to different parties; • And how well are the intervention components and their effects Maintained?

  8. RE-AIM Focus on Context • Broaden the criteria used to evaluate programs to include external validity • Evaluate issues relevant to program adoption, implementation, and sustainability • Help close the gap between research studies and practice by: • Suggesting standard reporting criteria • Informing design and evaluation of interventions • Focus on contextual factors that may impact results

  9. Advancing Care Together (ACT) – An example of use of RE-AIM in an integration program • Program funded by The Colorado Health Foundation • 11 practices funded to do demonstration projects • 9 primary care practices • 2 community mental health centers • All focused on integrating behavioral health and primary care • We had the privilege of following their journey; RE-AIM informed our work

  10. Better Outcomes • Systematic reviews and other rigorous, peer-reviewed studies show that integrated care leads to better patient outcomes1-5 for: • Depression Panic Disorder • Tobacco cessation Alcohol Misuse • Diabetes IBS • GAD Chronic Pain • Primary Insomnia Somatic Complaints 1. Butler et al., AHRQ Publication No. 09- E003. Rockville, MD. AHRQ. 2008. 2. Craven et al., Canadian Journal of Psychiatry. 2006;51:1S-72S. 3. Gilbody et al., British Journal of Psychiatry, 2006;189:484-493. 4. Williams et al., General Hospital Psychiatry, 2007; 29:91-116. 5. Hunter et al., Integrated Behavioral Health in Primary Care: American Psychological Association, 2009

  11. Improved Patient and Provider Experience • With a shift to integrated delivery models, patient experience with healthcare delivery improves1-5 • With a shift to integrated delivery models, primary care provider experience improves too6,7 • Chen et al., American Journal of Geriatric Psychiatry. 2006; 14:371-379. • Unutzer et al., JAMA. 2002; 288:2836-2845. • Katon et al., JAMA. 1995; 273:1026-1031. • Katon et al., Archives of General Psychiatry. 1999; 56:1109-1115. • Katon et al., Archives of General Psychiatry. 1996; 53:924-932. • Gallo et al.,Annals of Family Medicine. 2004; 2:305-309. • Levine et al., General Hospital Psychiatry. 2005; 27:383-391. 11

  12. Lower Cost When Treated • Multifaceted Diabetes and Depression Program– medical savings of $39 PMPM observed over 18 months • Pathways program for diabetes & depression - $46 PMPM saved, or about 5% over 2 years • IMPACT program for depression among the elderly - $70 PMPM saved over 4-year period, or about 10% • Missouri CMHC health homes in 2012 – independent living increased by 33%, vocational activity increased by 44%, overall healthcare costs decreased by 8% • Observed savings of between 9% and 16% of value opportunity Melek, SP, Norris, DT, Paulus, J. Economic impact of integrated medical-behavioral healthcare: Implications for psychiatry. Prepared by Milliman. for the American Psychological Association, April 2014

  13. Findings from an Analysis of Comorbid Chronic Medical & Behavioral Conditions in Insured Populations Melek, SP, Norris, DT, Paulus, J. Economic impact of integrated medical-behavioral healthcare: Implications for psychiatry. Prepared by Milliman. for the American Psychological Association, April 2014

  14. Learning Evaluation – A RE-AIM Informed Evaluation Approach • RE-AIM provides a framework for collecting relevant data when implementing an innovation • Implementation of a new innovation occurs through rapid, short cycles of improvement • Data are essential to • Monitoring and refining the change process • Ensuring that implemented changes result in expected outcomes • Taking the innovation to to scale

  15. What Data Do We Collect • Practice Characteristics • Practice Information Form • Implementation / Context • Documentation • Online diaries • Phone calls • Site visits • Reach • % of target screened • % of target positive • % of target received services • Effectiveness • Patient outcomes • Expenditures • Utilization

  16. What We Do With the Data We Collect

  17. RE-AIM Dimensions and Definitions for Population Health Participation rate among eligible individuals Representativeness of participants REACH (Individual Level) Participation rate among invited settings and staff Representativeness of participating settings and staff ADOPTION (Setting Level) www.re-aim.org

  18. Key Lessons Learned about Reach and Adoption • Both focus on importance of denominators • Frequently confused: Same principles at different levels • If exact denominator is unknown, estimate!

  19. Types of Interventions Provided to Patients • Brief counseling in primary care • Referrals for traditional long-term counseling • Within clinic • From partner clinic • Outside clinic • Warm hand-off • Joint PC and BH counseling

  20. Screening Tools Used by ACT Innovators

  21. Reach The ACT program included: • Over a 12 month period: • 84,645 target patients • 13,168 were screened • 6,845 screened positive • Over a three month period, on average: • 21,149 target patients • 3,292 were screened • 1,711 screened positive 15.5% screened 52% of those screened, screened positive

  22. Screening Systematic Screening Clinical Discretion

  23. Two stories • Documenting referral or receipt of further counseling for patients who screened positive • Mental model for how data is used in quality improvement process

  24. RE-AIM Dimensions and Definitions for Patient Experience Effects on primary outcome of interest Impact on quality of life, any negative outcomes Effectiveness (Individual Level) Long-term effects of intervention Sustained delivery and modification of intervention (setting level) Maintenance (Setting Level) www.re-aim.org

  25. Key Lessons Learned about Effectiveness and Maintenance • Are often unintended consequences of programs, can be either positive or negative • Some of the programs that have the greatest effectiveness have the lowest reach (and vice versa) • Setting level Maintenance is seldom sustained, but almost never is a program continued in the exact same way

  26. Effectiveness – A Patient’s Perspective Patrick’s health story https://www.youtube.com/watch?v=7CObVLYUORc Sandy’s health story https://www.youtube.com/watch?v=9CRf3Ttrsk0

  27. Take Home Points • “All models are wrong” • Importance of ongoing evaluation, monitoring quality, learning from a range of data • Interrelationship of RE-AIM dimensions • Value of RE-AIM when implementing integration innovations

  28. Questions?

More Related