Voice and Choice: The Consumer / Family Role in Transformation

1. Voice and Choice:The Consumer / Family Role in Transformation Dean L. Fixsen, Ph.D. Karen A. Blase, Ph.D. National Implementation Research Network Louis de la Parte Florida Mental Health Institute Kentucky Meeting, 2006

2. Transformation Agenda • Effective services • Implementation of effective services • Consumer, family, advocacy role in transformation • Discussion

3. Transformation Goals • Mental Health Care Is Consumer and Family Driven • Disparities in Mental Health Services Are Eliminated • Early Mental Health Screening, Assessment, and Referral to Services Are Common Practice • Excellent Mental Health Care Is Delivered New Freedom Commission on Mental Health, 2003

4. Evidence-Based Programs The integration of best practice evidence with clinical expertise and patient value. Institute of Medicine, 2001

5. Evidence-Based Programs • Leading edge of MH System Change • Restructure therapeutic, administrative, and financial environments to enable MH systems that are: • More dynamic and adaptable • More accountable • More effective

6. Use of “Evidence-Based”(Hoagwood & Johnson, 2003) 1900-1990 0 1990-1995 86 1995-1999 5,525 Google (10/2004): 2,320,000 Google (01/2006): 16,600,000

7. Consumer & Family Voice • Several EBPs have included the consumer & family voice to help guide the development of their programs • ACT, Supported Employment, Teaching-Family Model, Multisystemic Therapy, Functional Family Therapy, and others

8. 7 6 5 4 3 2 1 Current License Yes No Correcting Problems Cooperation Serving Community Needs Correcting Problems Cooperation Serving Community Needs Home Environment Correcting Problems Cooperation Follow Guidelines Community Comments Cooperation Correcting Problems Communication Cooperation Effective In Helping Child Communication Fairness of T-Ps Concern of T-Ps Effectiveness of T-Ps In Correcting Problems Pleasantness of T-Ps Helpfulness of T-Ps Good Treatment Program Social Skills of Youths Teaching Skills Self-government Motivation System Family-Style Living Condition of Home (T-P Res.) Record Keeping Condition of Home (Board Res.) X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X Court Social Service Board Schools Parents Youths Professional Evaluation T-Ps ______________________ Address: __________________ Evaluation Type: ___________ 7=Completely Satisfied 6=Satisfied 5=Slightly Satisfied 4= Neither Satisfied nor Dissatisfied 3=Slightly Dissatisfied 2=Dissatisfied 1=Completely Dissatisfied Shaded area representsacceptable rangeof performance X Certified Teaching-Parents

9. Effective Programs • Nurse-Family Partnership • Parent Management Training • Functional Family Therapy • Positive Behavior Support • Multi-Systemic Therapy • Multidimensional TFC • Teaching-Family Model

10. Effective Programs • Medication algorithms • Family psychoeducation • Illness management • Integrated dual disorders treatment • Supported employment • Assertive Community Treatment • Fountain House Lodges

11. What does NOT Work? • Non-behavioral interventions for disruptive behavior disorders and/or ADHD (Weisz et al., 1995; Pelham et al., 1998) • Group, peer-based interventions for disruptive disorders (GGI, PPC) • Instead may facilitate “deviancy training”: increases in rates of substance abuse, delinquent behaviors, and violence post-intervention (Dishion et al., 1999).

12. What does NOT Work? • DARE (5th and 6th grade curriculum) • Gun Buyback programs • Boot Camps • Peer counseling programs • Summer job programs (at risk youth) • Home detention with electronic monitoring • Wilderness / challenge programs • Casework / counseling

13. What is HARMFUL? What produces negative (iatrogenic) outcomes? • Waivers to adult (criminal courts) • Scared Straight • Shock Probation / Parole

14. A Scale of Usefulness Evidence-based programs have demonstrated effectiveness through research Promising practices have some evidence for benefits to consumers (Jim Wotring & Kay Hodges in Michigan). Common practices have no data to support their use but over many years they have been built into a series of laws, regulations, funding mechanisms, professional and organizational accreditation standards, etc. that sustain them. Harmful practices have evidence indicating harm to clients but often still are supported as a common practice

15. Implementation • The act of accomplishing some aim or executing some order • To put into practical effect; carry out • Pursue to a conclusion – Dictionary.com A lot of hard work!

16. Consumer & Family Choice Implementation = Access to effective services = Benefits to you and your family

17. The NIRN • Craft knowledge • EBP purveyors (program developers) • EBP implementation site managers • Implementation researchers • Survey of EBP program developers • Scientific information • CRUSK, Follow Through, Lodges, et al. • Program development and replication data • Synthesis of the implementation evaluation and research literature

18. Implementation Review • Human service prevention and treatment programs (e.g. substance abuse, adult / children’s MH, justice, health, education) • Advanced manufacturing technologies • Research-based clinical guidelines • Engineering: bridge maintenance • Hotel service management • National franchise operations • Cancer prevention & treatment

19. Implementation Attempts • Homebuilders national implementation grants: $500 million, 5 years • Program / system change grants plus TA for systems of care, state transformation, school reform, medical clinical guidelines, etc: $?? billion a year

20. Implementation Attempts • In business, change initiatives that are heavily dependent on people (reengineering, TQM, culture change) fail 80-90% of the time • About 10% of what is taught in training gets transferred to the job • "Up to 70% of the failures in business are not due to poor strategy or a lack of good ideas, but to flawed execution." R. W. Rogers, 2002

21. Implementation Attempts Successful programs do not contain the seeds of their own replication. Lisbeth Schorr, 1993 Author of Within Our Reach

22. Implementation Success • PMTO program in Norway: National implementation of parent management program (NIDA funded research) • DOTS program in India: National implementation of TB treatment protocol, 1 million patients in 4 years • MST program in USA: National implementation of homebased interventions for juvenile offenders

23. Common to Success • Good science and technology • Baseline / decision support data • Expanding commitment & flexibility • Purposeful workforce development • Organizational change & supports • Alignment of system functioning • Knowledgeable purveyor, constant monitoring, feedback, intervention Khatri & Frieden, 2002

24. Good News • Knowledge of successful implementation strategies can benefit all human services

25. Consumer Outcomes Effective intervention practices + Effective implementation practices = Good outcomes for consumers

26. Consumer Outcomes IMPLEMENTATION Effective NOT Effective Performance Implementation Paper / Procedure Implementation Effective INTERVENTION NOT Effective

27. Paper Implementation • Review of TQM adoptions in 5,492 hospitals • "If organizations can minimize evaluation and inspection of their internal operations by external constituents through adoption alone, they may neglect implementation altogether, decoupling operational routines from formally adopted programs." Westphal, Gulati, & Shortell (1997)

28. Consumer Outcomes IMPLEMENTATION Effective NOT Effective High Fidelity Low Fidelity EBP INTERVENTION NOT Effective

29. Teaching-Family Model Bedlington, et al., 1979 2.0 50% rs = – .94 1.9 1.8 40% 1.7 Delinquency 1.6 30% Mean Self ReportedDelinquency Ratings PercentParental-Teaching With Youths 1.5 1.4 20% 1.3 Teaching 1.2 10% 1.1 1.0 0% 1 2 3 4 5 6 Homes

30. What you need to know An evidence-based program is one thing Implementation of an evidence-based program is a very different thing

31. Stages of Implementation • Implementation is not an event • A mission-oriented process involving multiple decisions, actions, and corrections

32. 2 – 4 Years Stages of Implementation Implementation occurs in stages: • Exploration • Installation • Initial Implementation • Full Implementation • Innovation • Sustainability Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

33. Implementation Stages:Non-Linear Processes Installation Initial Implementation

34. Fidelity Measure Implementation Framework Organizational Structures/Culture Practitioner Purveyor Evidence-based Practices Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

35. EBPs: The Source • The usability of a program has little to do with the quality or weight of the evidence regarding that program • Evidence on intervention effectiveness for specific populations helps us choose what to implement • Evidence on the effectiveness of the intervention does not help implement the program or practice successfully

36. EBPs: The Source Core intervention components • Clearly described (what/how) • Practical measure of fidelity • Fully operationalized (do/say) • Field tested (iterative revision) • Contextualized (org./systems fit)

37. Implementation Site: The Destination • Defined Need • Identified Resources • Fully informed consent re: innovation and implementation • Top level commitment to change and agreement re: access

38. Implementation Site: The Destination • Practitioners impact consumers • It is the job of directors, managers, and funders to align policies and structures to facilitate effective practitioner practices • There is no such thing as an “administrative decision” – they are all treatment decisions

39. What you need to know Be a part of the process of choosing what to implement (ask about core intervention components, match with strengths and needs, costs to consumers & families) Advocate with MH directors and legislators (what is the plural form of anecdote?) Be informed (ACMH Guide for Families, Malisa Pearson & Tiffany Leischner; www.acmh-mi.org)

40. Implementation of EBPs • Changing the behavior of adult human service professionals • Changing organizational structures, cultures, and climates • Changing the thinking of system directors and policy makers Leading edge of MH System Change New Freedom Commission

41. Human Service Systems “Systems trump programs.” – Patrick McCarthy, Annie E. Casey Foundation“ Advocate at the systems levels to enable longer lasting local impacts.

42. Purveyor Excellent experimental evidence for what does not work • Dissemination of information by itself does not lead to successful implementation (research literature, mailings, promulgation of practice guidelines) • Having information and knowledge is helpful to decision making, but it is NOT implementation

43. Purveyor Excellent experimental evidence for what does not work • Training alone, no matter how well done, does not lead to successful implementation Information dissemination and training by themselves repeatedly have been shown to be ineffective in human services, education, health, business, and manufacturing

44. Active Purveyor Role • Successful implementation on a useful scale requires a purveyor • An individual or group of individuals representing a program or practice who actively work to implement that practice or program with fidelity and good effect • Purveyors accumulate data & experiential knowledge -- more effective and efficient over time (information economics, K. Arrow)

45. Active Purveyor Role • Change the behavior of adult human service professionals • Change organizational structures, cultures, and climates • Change the thinking of system directors and policy makers Successful and sustainable implementation of evidence-based practices and programs always requires organizational change.

46. Practitioner • Organization • Management (leadership, policy) • Administration (HR, structure) • Supervision (nature, content) System of care State policies Active Purveyor Role Simultaneous, Multi-Level Interventions Purveyor

47. Keys to Implementation STAFF PERFORMANCE EVALUATION CONSULTATION & COACHING DECISION SUPPORT DATA SYSTEMS INTEGRATED & COMPENSATORY FACILITATIVE ADMINISTRATIVE SUPPORTS PRESERVICE TRAINING RECRUITMENT AND SELECTION SYSTEMS INTERVENTIONS

48. OUTCOMES (% of Participants who Demonstrate Knowledge, Demonstrate new Skills in a Training Setting, and Use new Skills in the Classroom) TRAINING COMPONENTS Knowledge Skill Demonstration Use in the Classroom Theory and Discussion 10% 5% 0% ..+Demonstration in Training 30% 20% 0% …+ Practice & Feedback in Training 60% 60% 5% …+ Coaching in Classroom 95% 95% 95% Joyce and Showers, 2002

49. What you need to know An evidence-based program is one thing Implementation of an evidence-based program is a very different thing Good outcomes occur ONLY when evidence-based practices and programs are implemented well. As a consumer of evidence-based practices and programs, you need to know how to assess the quality of implementation.

50. What you need to know • Fidelity is important. • Fidelity means actually using the treatment procedures as they were tested and found to be effective. • Implementation with fidelity is required to achieve advertised outcomes