Evidence Based Practices=Achieving Outcomes

1. Evidence Based Practices=Achieving Outcomes Center for Innovative Practices Institute for the Study and Prevention of Violence Kent State University Acknowledgement to: Karen A. Blase, Ph.D. National Implementation Research Network Frank Porter Graham Child Development Institute at UNC- Chapel Hill • Targeted RECLAIM Initiative • May 14, 2009

2. Rationale for Evidence Based Programs • Science supports outcomes • High level of training and coaching • Clear set of program standards • Support with implementation • Ultimately can be ‘cost effective’ • All systems are moving to an ‘outcome based’ approach to funding

3. What Do We Mean By Evidence-Based? A program or practice that has been demonstrated through scientific studies to be effective in improving outcomes for a specific population.

4. A Scale of Usefulness Evidence-based programs have demonstrated effectiveness through research Promising practices have some evidence for benefits to consumers Common practices have no data to support their use but over many years they have been built into a series of laws, regulations, funding mechanisms, professional and organizational accreditation standards, etc. that sustain them Harmful practices have evidence indicating harm to clients but often still are supported as a common practice

5. Levels of ‘Evidence’ • No evidence • Expert consensus • Formal Evaluation • Comparison studies • Randomized clinical trials • Meta-analysis • Replicable in real world settings

6. Core Components of Evidence Based and Promising Programs • Some higher Level of Evidence of effectiveness • Reliability of Treatment Application • Treatment Manual and Uniform Training • Treatment Fidelity • Coaching, Consultation, Quality Assurance and Support • Validity: Strong theoretical and research basis • Consumer responsiveness; Salience • True clinical sample of intended treatment population • Ethical delivery of service • A balanced focus on strengths as well as needs

7. A Sample of Programs/Practices Relevant to Juvenile Justice and Behavioral Health • Multisystemic Therapy = MST • Intensive Home Based Treatment = IHBT • Multidimensional Treatment Foster Care = MTFC • Integrated Co-Occurring Treatment = ICT • Wrap Around = WA • Functional Family Therapy = FFT • Cognitive Behavior Therapy = CBT • Appropriate medication intervention But all need to be based on: • Multi factored Screening, Assessment, and Diagnosis for MH and Substance Abuse

8. A Short List of Effective Practices in Ohio • Integrated Co-Occurring Treatment-Intensive home and community integrated MH & SA treatment • Nurse-Family Partnership- Home visiting model by RNs for newborns focusing on child-parent interaction and child development • WrapAround-A process technique to develop en ecological treatment plan for families with multiple systems needs • Columbia Teen Screen-Science based voluntary youth screening tool for depression and other mental health • School based Mental Health Strategies-Specifically focusing on school based approaches to prevention and intervention • Strengthening Families: Ages 10-14, designed to reduce substance abuse and other problematic behaviors in youth, is a seven week course for parents and youth • MST-Intensive family based intervention for youth with externalizing behavior • IHBT-Intensive home based treatment model for families with youth with a wide range of emotional and behavioral disorders • Incredible Years-Parent and Teacher modeling and behavior management skills • Functional Family Therapy-Family therapy and structure model for youth with disruptive behaviors • disorders • Positive Behavior Supports-Youth specific and environment general techniques to minimize problem behaviors • Multidimensional Treatment Foster Care-Specialized foster care linking foster and home to focus on disruptive behaviors in youth

9. MST Teams Affiliated with CIP Provider Counties Served # of Teams Crisis Intervention Center * Stark 1 Family Resource Centers Hancock 1 ApplewoodCenters * Lorain and Cuyahoga 2 Family Life Counseling Richland 1 Children’s Hospital * Franklin 2 Buckeye Ranch Franklin 2 Rosemont Center Franklin 2 Counseling Center Columbiana 1 Family and Community Svcs Portage 1 Justice Affairs Cuyahoga 1 TBD Lucas 1 Total 9 counties 15 * (piloting an MST-PSB team) Franklin, Stark, Lorain, Cuyahoga 4

10. Ohio MST Outcomes

11. Estimated Cost Comparisons

12. Implementation Issues

13. Explore • Who are the key populations we are trying to address • What are the key problem or challenge areas we are trying to address • What is already in our continuum • What is the capacity of that continuum • Do we need more of the same • Do we need something different • Do we need to stop doing some things

14. Explore Assess thePotential Match Equation Community Needs + Cross System Support + Community Resources + Readiness + Options for evidence-based/best practice = Decision to proceed (or not).

15. Options • What process do we use to identify the possible service options • What other partners do we want/need at the table • What role do other youth serving systems have in our exploration

16. Decision Points • What are the services, strategies, practices that might address our identified needs and build local continuum • What criteria do we use to ‘nominate’ or identify potential services • What outcomes are the most important • What level of evidence do we want for programs we select • What are the reliable lists/sources of information on effective practices

17. Selection • What services or practices ‘match’ our criteria and targeted outcomes • What details about the treatment do we need information on: • Cost • Training • Licensing requirements • Coaching • Level of research • Workforce needs • Feasibility factors • Outcomes

18. Example of One Good Resource for Exploration/Selection • http://www.colorado.edu/cspv/blueprints/modelprograms.html

19. Model Programs as Evaluated by the Blueprints for Violence Prevention project • Midwestern Prevention Project (MPP) • Big Brothers Big Sisters of America • Functional Family Therapy (FFT) • Life Skills Training (LST) • Multisystemic Therapy (MST) • Nurse-Family Partnership (NFP) • Multidimensional Treatment Foster Care (MTFC)Olweus Bullying Prevention Program • Promoting Alternative THinking Strategies (PATHS) • The Incredible Years: Parent, Teacher and Child Training Series (IYS) • Project Towards No Drug Abuse (Project TND)

20. Promising Programs • ATLAS (Athletes Training and Learning to Avoid Steroids) • Behavioral Monitoring and Reinforcement Program • Brief Strategic Family Therapy (BSFT) • CASASTART • FAST TrackBPP06 • Good Behavior Game (GBG) • Guiding Good Choices (GGC) • I Can Problem Solve (ICPS) • Linking the Interests of Families and Teachers (LIFT)

21. Promising Programs • Perry Preschool Project BPP12 • Preventive Treatment Program (PTP) • Project ALERT • Project Northland • BASICS (Brief Alcohol and Intervention of College Students) • Seattle Social Development Project (SSDP) Strengthening Families Program For Parents and Youth 10-14 • Strong African American Families (SAAF) Program

22. Implementation Reality • Not an event but an ongoing process • Typically takes 2-4 years to take hold • The treatment effectiveness is in direct proportion to the effectiveness of implementation at ALL levels

23. Effective intervention practices and programs + Effective implementation practices = Good outcomesfor children and their families No other combination of factors reliably produces desired outcomes for children, families, and caregivers

24. IMPLEMENTATION Effective NOT Effective Paper & Process Implementation (Low or No Fidelity) Performance Implementation (High Fidelity) Effective INTERVENTION NOT Effective It’s hard to land here and harder to stay here!

25. Implemenation Anticipation • What will implementation of an EBP mean to the local system of care • What organizational changes will need to be made to accommodate • What intersystem relationships/processes need to be created/changed/modified • Can we identify the challenges that will be inevitable • Do our system stakeholders have the patience needed for successful implementation

26. Sustainability • Can we afford it without ‘special’ funding and/or when special funding ends • Can we identify ways to embed the service within the local financing • Can we ‘repurpose’ existing funds • What is the collective public youth serving systems’ investment strategy to keep their money and keep their kids, at home • What is the cost-benefit

27. Reality: Considerable Challenges • Clinical & Programmatic • Systemic • Financial

28. Clinical & Programmatic Challenges • Changing practices for both clinicians and organizations • Rigorous supervision/coaching; ongoing training • Focus on Quality Improvement and Assurance • Staffing; training; retraining • Outcomes driven

29. Systemic Challenges • The allure of ‘the list’ • Identifying and selecting practices within a context of a community planning process • Shift to an outcomes /results based (qualitative) system • Developing local evaluation capacity • Going to scale…making effective practices the rule

30. Financial Challenges • ‘Bridge’ funding to finance start up • Reinvesting current resources to new programs • Anticipating all the costs: what funds will pay for what, e.g., Medicaid does not pay for staff training • Mechanics of reimbursement and limitations of fee for service • Potential ‘conflict’ with productivity approach

31. Factors that Inhibit Development • ‘Over-promise’ of the intervention • Expectation of “instant” results • Lack of adequate advance strategic planning • Provider is the only champion • Limited stakeholders at the table • High level of resistance to change • Workforce issues • Organizational issues • Short term plan for financing

32. Factors that Facilitate Development • Interventions selected on solid data • Outcomes that impact JJ, BH and CW • Feasibility: what can and will the local system support • ‘Real world’ data to capture clinical and cost effectiveness • Diversion from more costly, more restrictive level of care • Systems that are ‘saving’ reinvest some funds back into the program • Full table of participants

33. Summary • Investigate need • Identify change agents, stakeholders, and champions • Identify Programs and Practices that look promising to meet target needs • Establish key outcomes across stakeholders • Measure qualitatively and quantitatively, including $$ • Begin ‘sustainability planning’ from the beginning • Use multiple funding sources from multiple systems

34. Systems Intervention Systems Can Trump Programs! - Patrick McCarthy, Annie E. Casey Foundation Goes on at: • practice • program • agency • and systems levels

35. Resources and Tools • Center for Innovative Practices www.cipohio.org • National Center for MH/JJ www.ncmhjj.org • OJJDP Model Programs Guide www.dsgonline.com/mpg_non_flash/mpg_index.htm • Federation of Families www.ffcmh.org • NAMI www.nami.org/youth • CMHS-Center for Mental Health Services www.mentalhealth.org • Surgeon General www.surgeongeneral.gov • Center for the Study and Prevention of Violence www.colorado.edu/cpsv • OJJDP-Comp Strategies www.ncjrs.org/strategy/index/html • GAINS Center www.prainc.com/gains

36. Resources and Tools • Washington State Institute of Public Policy www.wsipp.wa.gov • OJJDP Parent Resource www.parentingresource.ncjrs.org • Children’s Defense Fund www.childrensdefense.org • National MH Assoc. www.nmha.org • Annie E. Casey Foundation www.aecf.org • Center for Effective Collaboration www.air-dc.org/cecp • Search Institute www.searchinstitute.org • Nat’l Academies of Science www.academies.org • Institute for the Study And Prevention of Violence www.kent.edu/violence

37. Resources and Tools • Rand www.promisingpractices.org • CWLA www.cwla.org • MST www.mstservices.org • Oregon Social Learning Center (Multidimensional TFC) www.oslc.or • Functional Family Therapy www.fftinc.com • SAMHSA Model EBP Project www.modelprograms.samhsa.gov/template • Dept. of Ed www.ed.gov • Nat’l Center on MH and JJ www.ncmhjj.org • Center for Education, Disabilities and JJ www.edjj.org • Data Trends (Resource Center) www.rtc.pdx.edu/DataTrends • Georgetown Childrens MH TA Center www.georgetown.ed/research/gucdc.cassp

38. Resources and Tools • Florida MH Research Center www.rtckids.fmhi.usf.edu • President’s Freedom Commission www.mentalhealthcommission.gov • Chestnut AOD www.chestnut.org • NIDA www.drugabuse.gov/nidahome • Tech Assist Collab-TAC (EBP Manual) www.tacinc.org • School Based Mental Health http://smhp.psych.ucla.edu/resource.htm • State of New York articles on EBP: http://www.omh.state.ny.us/omhweb/EBP/practicesforchildren.htm • Date Trends Newsletter: http://www.childtrendsdatabank.org / • DHHS Best Practices: http://www.osophs.dhhs.gov/ophs/BestPractice/ • SAMHSA Model Programs: • http://www.modelprograms.samhsa.gov/template.cfm?CFID=540873&CFTOKEN=20095080

39. Review of Evidence Based Practices • Psychosocial therapies • Chambless and Hollon (1998). Defining empirically-supported therapies. Journal of Consulting & Clinical Psychology, 66, 7-18 • Kazdin, Psychotherapy for children and adolescents • Oxford, 2000 • Weisz & Jensen, Mental Health Services Research, 1999 • School-Based Interventions • Rones & Hoagwood, School-based mental health services, • Clinical Child and Family Psychology Review, 2000 • Psychopharmacology • Journal of the Am. Academy of Child/Adol. Psychiatry special issue on psychopharmacology, 1999 • Weisz & Jensen, Mental Health Services Research, 1999

40. Reviews of Evidence-based Interventions Community-based Interventions • Surgeon General’s Mental Health Report, 1999 • Surgeon General’s Youth Violence Report, 2001 • Burns & Hoagwood (2002) Eds. Community treatments for youth. Oxford University Press • Burns, Hoagwood, & Mrazek (2000) Effective treatments for mental disorders in children and adolescents. Child Clinical and Family Psychology Review, 2000 • Greenberg, et al., Review of the Effectiveness of Prevention Programs, 1999 (CMHS) • Olds et al., Review of Preventive Interventions, 1999 (CMHS)

41. For More Information • Patrick J. Kanary pkanary@kent.edu Center for Innovative Practices www.cipohio.org