Phyllis C. Panzano, Ph.D. , PI Dee Roth, M.A., Co-PI

1. The Innovation Diffusion and Adoption Research Project (IDARP) Funded by the ODMH & the Mac Arthur Foundation Phyllis C. Panzano, Ph.D. , PI Dee Roth, M.A., Co-PI Bev Seffrin, Ph.D, Senior Consultant Dushka Crane-Ross, Ph.D., Project Manager Decision Support Services, Inc. Ohio Dept of Mental Health, OPER ODMH RESEARCH RESULTS BRIEFING 2003

2. Ohio’s Quality Agenda Best Practices QI Outcomes

3. Evidence Base Salience

4. Coordinating Centers of Excellence (CCOEs) • SAMI-IDDT • MST • Family Psychoeducation • Cluster-Based Planning Evidence Base • OMAP • MH/Criminal Justice • MH/Schools • Advance Directives Salience

5. Structure of CCOEs • University or local partnership • One Best Practice per CCOE • Statewide service area

6. Role of CCOEs • Promotion of Best Practices • Education & training • Capacity development • Fidelity measurement • Cross-system sharing

7. Research Question What factors and processes influence the adoption, assimilation, and impact of evidence-based practices by mental health provider organizations?

8. Independent Variables • Characteristics of the Best Practice • Adoption Decision & Implementation Process • Adopting Organization • Adopting Organization – CCOE Relationship

9. Research Team • Decision Support Services, Inc. • Phyllis Panzano, Ph.D • Beverly Seffrin, Ph.D. • Sheri Chaney, M.A. • Vandana Vadyanathan, M.A. • Sheau-yuen Yeo, M.A. • Ohio Dept of Mental Health • Dee Roth, M.A. • Dushka Crane-Ross, Ph.D. • Rick Massatti, M.A. • Carol Carstens, Ph.D. • Ohio State University: Fisher College of Business • Department of Psychology

10. Theoretical Background • Numerous literatures are relevant • Resulting Assumptions: • EBPs are innovations • Scientific evidence necessary but not sufficient • Upper Echelon Theory relevant • Implementation effectiveness  Innovation effectiveness • Factors at many “levels” impact outcomes • 3 phases: initiation; decision; implementation

11. COMPLEXITY idarp 100-PIECE JIGSAW PUZZLE

12. IDARP Models

13. Model 1:Adoption Decision –Decision making under risk

14. Phase 1: Decision Under Risk LIKELIHOOD OF IMPLEMENTING Perceived Risk of Adopting ANTECEDENTS - • IMPLEMENT • ADOPTER • WAIT & SEE • NEVER More Likely + Capacity to Manage or Absorb Risk Less Likely + Risk-taking Propensity

15. Model 2: Multi-level Influences on Implementation Success

16. Interested in Two Classes of Outcomes • Measures of Implementation effectiveness: • Accurate, committed and consistent use of practice • by targeted employees (assimilation, fidelity, etc.) • Measures of Innovation effectiveness: Benefits that accrue to an organization and its stakeholders as a result of implementing an innovative practice (positive consequences for clients, staff, etc.)

17. Expected Link Between Two Classes of Outcomes INNOVATION EFFECTIVENESS IMPLEMENTATION EFFECTIVENESS

18. For example: INNOVATION EFFECTIVENESS IMPLEMENTATION EFFECTIVENESS POSITIVE OUTCOMES FIDELITY

19. Variables at multiple levels are expected to impact these two classes of outcomes

20. Examples of Variables by Level Level Example

21. Model 2 Level 5: Environment Level 4: Inter-organizational Level 3:Adopting organization Level 2: Project level Level 1: Innovation level • Dependent Variables: • Implementation effectiveness • Innovation effectiveness

22. Model 3: Cross-Phase Effects on Implementation Outcomes

23. Model 3: Cross-phase effects Decision Outcomes INITIATION IMPLEMENTATION Time

24. Model 3: Examples of Cross-phase Effects Positive Consequences Experiential Evidence Objective Process Access to Technical Assistance Initiation Decision Implementation Time

25. Model 4: Effects of Implementation Variables on Outcomes Over Time

26. Model 4: Effects of Implementation Variables Over Time PAST Implementation PRESENT Implementation PRESENT OUTCOMES TIME

27. Model 4: Examples of Effects of Implementation Variables Over Time PAST Access to Technical Assistance PRESENT Dedicated Resources PRESENT OUTCOMES TIME

28. Methods & Progress to Date

29. Four CCOEs Participating • Selection criteria maximize generalizability • Cluster-Based Planning Alliance • Multi-systemic Therapy (CIP) • Ohio Medication Algorithm Project • Integrated Dual Disorder Treatment (IDDT) –New Hampshire - Dartmouth model

30. Research Design • Longitudinal study • Organizations at different stages of adoption • Multiple key informants at each organization • Quantitative and qualitative data • Interviews, surveys & archival data

31. Participating Projects*by Type of Innovation Cluster Alliance IDDT/ SAMI MST OMAP *18 organizations involved in multiple projects; Total of 74 organizations with 91 projects under study.

32. Participating Projectsby Stage of Adoption at Time One Never Wait & see Adopter De-adopter Implementer N = 91

33. Participating Projectsby Stage of Adoption at Time Two Never Adopter Wait & See De-adopter Implementer N = 50

34. Key Informants by Level at Time One CCOE Community Collaborative Decision maker CFO/QA Implementer N = 369

35. Key Informants by Level at Time Two CCOE Community Collaborative Decision maker Implementer N = 135

36. Strongly disagree $ 37,500 Findings 6 22 agree Very satisfied

37. Do the data support our four models?

38. THE TIP….OF THE TIP

39. POSITIVE CORRELATION As the value of one variable increases, the value of a second variable also increases ___________________________________ + correlation (r = +1.00) Higher Median Income Lower Years of Formal Education Less More

40. NEGATIVE CORRELATION As the value of one variable increases, the value of a second variable decreases ___________________________________ Higher - correlation (r = -1.00) Unemployment Rate Lower Years of Formal Education Less More

41. ZERO ‘0’ CORRELATION The relationship between the value of one variable and the value of a second variable is random ___________________________________ Zero Correlation (r = 0.00) Taller x x x x x x x x Height x x x x x x x x x Shorter Years of Formal Education Less More

42. CORRELATION CANNOT BE DETERMINED because the value of one (or both) variable(s) is constant or almost constant ___________________________________ Higher Unemployment Rate Lower Years of Formal Education = BA, BS

43. The Adoption Decision (Model 1)

44. Time 1/First contact data

45. Phase 1: A Decision Under Risk Perceived Risk of Adopting -.51 • Likelihood of implementing as indicated by Stage • Implementer • Adopter • Wait & See • Never Capacity to Manage or Absorb Risk +.38 +.20 Risk-taking Propensity

46. Antecedents to Risk Perceptions ANT E C E D ENT S Perceived Risk of Adopting -.51 • Likelihood of implementing as indicated by Stage • Implementer • Adopter • Wait & See • Never Capacity to Manage Risk +.38 +.20 Risk-taking Propensity

47. Antecedents to Perceived Risk • Innovation Level Factors • Relative Advantage • Scientific Evidence • Experiential Evidence -.51 -.20 -.30 Perceived Risk • Org-Level Factors • Knowledge Set -. 43 • Environmental Factors • Norms for Adoption -.45

48. Antecedents to Risk Management EBP–Level Factors Ease of Use + .45 + .25 Craft Skills Capacity to Manage Risk Org–Level Factors Top Mgmt. Support + .50 + .63 Dedicated Resources Environmental Factors Environmental uncertainty - .22

49. Antecedents to Risk Propensity • Organization-Level Factors • Learning Encouragement • Managerial Attitude • About Change +.71 Risk Propensity +.23

50. Summing Up: Model 1 3. Antecedents have implications for action 1. Adoption decision is a decision involving risk 2. Organizations are more likely to adopt if: • Perceived risk of adopting is low • Capacity to manage risk is high • Propensity to take risks is high