Washington State’s Experience With Research-Based Juvenile Justice Programs

1. Washington State’s Experience With Research-Based Juvenile Justice Programs May 19, 2005 Robert Barnoski, Ph.D. (Barney) Washington State Institute for Public Policy www.wsipp.wa.gov

2. Washington State Institute for Public Policy Created in 1983 by the state Legislature Mission: Carry out non-partisan research on projects assigned either by the legislature or the Institute’s Board of Directors • 8 legislators • 4 higher education provosts or presidents • 4 state agency directors

3. What We Will Talk About Today This presentation describes Washington State's experiences implementing research-based programs • Some History • Outcome Evaluations • Quality Assurance • Cost Benefit • Cost Benefit Meta-Analysis

4. Juvenile Justice in Washington State • Determinant sentencing since 1977; based on prior record and current offense • Two systems: county vs. state • Each county runs juvenile court, detention, and probation • State runs juvenile correctional institutions, parole, and distributes state funds to counties (Juvenile Rehabilitation Administration – JRA) • Juvenile courts also manage non-offender cases: dependency, youth-at-risk, truancy, etc.

5. History of Legislation 1995 Funding of intensive probation program. 1996Preliminary outcomes are not good. 1997 Determine if research-based programs exist and can be implemented in Washington State juvenile courts. Eliminate parole but establish intensive parole in JRA. 1998 Switch funding from intensive probation to research-based programs in juvenile courts. Re-establish parole in JRA. 2002 Preliminary research-based program outcomes are positive and point to competent delivery. Parole and intensive parole outcomes are not good. 2003 Develop adherence and outcome standards for research-based programs. JRA moves toward research-based and Functional Family Parole (FFP). Cost-benefit analyses of prevention and intervention programs beyond juvenile offenders.

6. Initiation of Research-Based Effort in Washington State 1997: Community Juvenile Accountability Act (CJAA) • Legislature said it would provide funding if research-based programs could be identified • State, juvenile court administrators, and the Institute met regularly to identify “research-based” programs • Six programs were identified, four were recommended 1998: Legislature funded the programs ~$3.3 million/year 1999: Programs started 2001: Evaluation groups filled 2003: Evaluation completed

7. Four Things Were Needed to Implement Research-Based Programs • Find research-based programs scientifically shown to work • Develop an assessment to identify the most appropriate program for each youth • Implement quality assurance to ensure services are delivered as designed • Conduct a valid outcome evaluation

8. First Thing: Research-Based Program List • Functional Family Therapy (FFT) • Aggression Replacement Training (ART) • Coordination of Services • Multi-Systemic Therapy (MST) • Multidimensional Therapeutic Foster Care • Adolescent Diversion

9. Functional Family Therapy($2,100 per youth) • Blueprint Program: University of Colorado’s Center for the Study and Prevention of Violence • Structured family-based intervention to enhance protective and reduce risk factors in the family • FFT is a three-phase program • Motivate family toward change • Teach family how to change a specific critical problem • Help family generalize their problem-solving skills • Trained therapists have caseloads of 10 to 12 families • Involves about 12 visits during a 90-day period

10. Aggression Replacement Training($700 per youth) • Group training of 10 youth by two instructors • Classes meet for one hour three times per week for 10 weeks • Aggression cycle • Skill development • Moral reasoning • Training, not therapy, that uses guided discussion, modeling, role play, and homework

11. Coordination of Services($400 per youth) • Educational program for low-risk juvenile offenders and their parents. The goals: • Describe the consequences of delinquent behavior • Stimulate goal setting • Review the strengths of youth and family • Explain what resources are available • Participants given vehicle to open lines of communication and make shifts in thinking • Community groups present participants with information concerning services they provide

12. Second Thing: Build the Washington Juvenile Court Assessment • Pre-screen for level of risk – more intensive service to higher risk youth and families; minimal for low risk • Comprehensive assessment only for higher risk youth – 10 life domains of risk and protective factors • Motivational interview to engage youth and family • Re-assess dynamic factors for progress – individual and groups • Common language for talking about youth and families

13. Mapping Problem to Intervention for Moderate to High Risk Youth

14. Case Management:Sequence to Self Reliance If family has problems: Family => Engage & motivate family => Specific Problems => Generalization If no family problems: Engage & motivate youth => Attitudes & Skills => Specific Problems => Generalization

15. Role of Probation in Rehabilitation • Assess youth’s strengths and weaknesses • Assign youth to right caseload (low risk caseload) • Assign youth to right program(s) • Engage and motivate youth and family • Courts: motivational interview and change cycle • JRA: Functional Family Probation • Support and help youth and family generalize what is learned during research-based program • Deflect enabling undesirable behavior • Monitor changes in risk and protective factors to see if approach is making a difference and adjust • Does not make job easier, but might make it more effective

16. Assessment Proliferation • Must be something to WSJCA, since it is being used in at least a dozen states (including New York, Illinois, Florida, Utah, N. Dakota) and multiple courts – some in California • Service providers particularly seem to like it • We encourage continual refinement and improvement of the assessment; capitalizing on what has been done • It is evolving to be a systematic collection of relevant social file information

17. Third Thing: Quality Assurance Infrastructure • Steering committee • Contract with program designers • Statewide program experts • Regional consultants • Initial training • On-going consultation, feedback, and training • Assessment of competent program delivery

18. Fourth Thing: Evaluation Design • A “rigorous” test • Not enough funding for everyone, so “Waiting List” youth assigned to a control group by court staff • Includes completers and non-completers • Youth’s eligibility for program is based on risk assessment • Level of risk for re-offending • Profile of risk factors • Tests if probation plus a research-based program reduces recidivism more than probation alone

19. FFT Results Control Group (N=313) Individual Therapists (N=387) 63 Group Mean (Average) 55 47 4 7 43 42 34 33 33 31 28 26 26 23 23 22 20 18 18 17 17 17 17 14 14 14 12 11 8 0 C 1 2 3 4 5 6 7 M 8 9 10 11 12 13 M 14 15 16 17 18 M 19 20 21 22 23 24 25 M Not Competent Borderline Competent Highly Competent 18-Month Unadjusted Felony Recidivism Percentage

20. ART Results Control (N=525) Not Competent (N=203) Competent (N=501) 50% 49% 45% 27% 25% 19% 7% 7% 6% Misdemeanor and Felony Recidivism Felony Recidivism Violent Felony Recidivism 18-Month Recidivism Percentage

21. Coordination of Services Results Control (N=171) COS (N=171) 19% 17% 12-Month Recidivism Percentage 1% 3% Misdemeanor and Felony Felony Adjusted 12-Month Recidivism

22. Findings From Initial Studies • FFT delivered competently • Reduces 18-month felony recidivism by 38 percent • Benefit to cost ratio of $7.69 • ART delivered competently • Reduces 18-month felony recidivism by 24 percent • Benefit to cost ratio of $12.60 • Coordination of Services • Reduces 12-month felony recidivism by 57 percent • Benefit to cost ratio of $13.58 • MST • No findings because of problems implementing the Institute’s evaluation design

23. A Clear Lesson… The key to reducing recidivism with research-based programs is competently delivering the service.

24. Additional JRA Programs We Were Asked to Evaluate • Parole • Intensive Parole • Local Parole • Local Commitment • Group Homes ------------------------------------------ • Basic Training Camp • Dialectic Behavior Therapy • Mentoring Program • Family Integrated Transitions

25. Cost Benefit Estimates for Washington State Programs as of September 17, 2004 Measured Benefits and Costs Per Youth Benefits Costs Benefits per Dollar of Cost Benefits Minus Costs Dialectical Behavior Therapy $32,087 $843 $38.05 $31,243 Washington Basic Training Camp $14,778 -$7,586 n/a $22,364 Family Integrated Transitions $28,215 $8,968 $3.15 $19,247 Functional Family Therapy $16,455 $2,140 $7.69 $14,315 Aggression Replacement Training $9,564 $759 $12.60 $8,805 Interagency Coordination $5,536 $408 $13.58 $5,128 Mentoring $11,544 $6,471 $1.78 $5,073 Intensive Probation Supervision $0 $1,500 $0.00 -$1,500 Intensive Parole $0 $5,992 $0.00 -$5,992 Regular Parole (v. not having parole) -$10,379 $2,098 -$4.95 -$12,478

26. 2003 Legislature Starts to Repeat Process for Other Intervention and Prevention Programs • Benefits and Costs of Prevention and Early Intervention Programs for Youth • September 2004 • Steve Aos et al. • Washington State Institute for Public Policy • www.wsipp.wa.gov

27. •Lower substance abuse, •Improve educational outcomes, •Decrease teen pregnancy, •Reduce teen suicides? •Lower child abuse or neglect, or • Question: Are there “research-based” programs or policies with a “real world” ability to: Research Questions & Methods • Reduce crime, • We screened evaluations written in English for: • The six outcomes • The quality of a study’s research design • Whether the program is “real world” • We computed effects (meta-analytically) • We then calculated monetary benefits and costs

28. Over 30 well-researched studies, mostly of programs for 3 & 4 year olds from low income families. Key findings: • improved education outcomes, increased high school graduation higher test scores lower special education lower grade repetition • reduced crime, • reduced child abuse & neglect. Evidence of decay in early test score outcomes, but still statistically significant by high school graduation. Selected Findings Early Childhood Education$17,202$7,301$9,901 A home visitation program, with active nationwide dissemination. NFP isdelivered by nurses and isfor low income, soon-to-be first time mothers. Evidence of: • reduced crime for mothers and children, • reduced child abuse & neglect, • improved education outcomes. Website: www.nursefamilypartnership.org/ Nurse Family Partnership$26,298$9,118$17,180 Functional Family Therapy$16,455$2,140$14,315 Multi-Systemic Therapy$14,996$5,681$9,316 Life Skills Training$746$29$717 Two programs for juvenile offenders and their families, conducted by trained therapists. FFT has been implemented statewide in Washington State’s juvenile courts. Evidence of reduced crime when the model is followed. Websites: www.fftinc.com/ and www.mstservices.com/ Seattle Soc. Dev. Project$14,246$4,590$9,837 Guiding Good Choices$7,605$687$6,918 Multi-D Treat. Foster Care $26,748$2,459 $24,290 A three-year program for middle school youth designed to prevent tobacco, alcohol, and marijuana use. Delivered by classroom teachers. Key findings: Delayed initiation of • tobacco, • alcohol, • illicit drugs. Website: www.lifeskillstraining.com/ Intensive Juv. Supervision $0$1,482 -$1,482 Big Brothers/Sisters (all costs)$4,058$4,010$48 (taxpayer costs only)$4,058$1,283$2,775 A multi-year grade school and middle school training program for parents (family management training) and teachers (classroom management, interactive teaching) for children with low socioeconomic status. Evidence of:• reduced crime, • increased high school graduation, • reduced grade repetition Website: http://depts.washington.edu/sdrg/ A multimedia training program (parenting skills, peer pressure refusal skills for students) implemented with families of middle school children. Evidence of:• reduced crime, • reduced alcohol initiation Website:www.channing-bete.com/positiveyouth/pages/FTC/FTC-GGC.html Multidimensional Treatment Foster Care is an alternative to group facilities for youth with chronic & severe criminal behavior. Delivered by trained families, the goal is to return the youth to the family; the primary family also receives therapy. Evidence of:reduced crime Website: www.oslc.org/ HF We meta-analyzed 19 studies of intensive supervision programs for juvenile offenders. No statistically significant effect on recidivism rates. Mentoring program. Evidence: increased test scores, delayed alcohol and drug initiation.Website: www.bbbsa.org

29. Managing Competent Program Delivery Invest in increasing skills, not accountability • Pick good people • Provide required training • Enhance skills • Retain only competent providers

30. Initial Provider Training • Written knowledge test and feedback after initial training • Follow-up skill training and consultations • Certification at end of probationary period

31. Continually Enhance Provider Skills • Form regional support teams • On-site direct observation by an expert, including audio- or video-taping to improve competence • Case-by-case consultations • Give feedback

32. Skill Feedback Please place a check in the box that best describes what happened in this session. YES▼ NO ▼ • Was a positive climate established through welcoming students? • Were group norms reviewed, emphasizing positive participation? • Were all ACT concepts covered to this point reviewed? • Were issues from the last anger control training session reviewed? • Did all youth complete the hassle log(s)?

33. Ensure Competent Providers Are Retained • Establish process for corrective action • Corrective action taken when needed • Building morale; taking pride that we are good!

34. Annual Legislative Accountability • Youth eligible, assigned, and completing programs (75%) • Changes in targeted risk and protective factors • Program provider competency ratings • Meeting expected recidivism rates

35. Some of You May Be Thinking… “Let’s get going on one of these programs.” -or- “We already do a great job, so why would we be interested in this?”

36. If You Are Not Comfortable With the Research Approach, a Possibility… • Understand what is currently being done • Understand what is being done in research-based programs • Examine the differences • Are you already doing it • Does it seem like an improvement • What needs to be tested • Implement quality assurance for current practice • Have competent researcher do valid outcome evaluations

37. Essential Ingredients • Leadership • Legislation • Funding pilot research-based programs • De-funding ineffective programs • Infrastructure for program fidelity • Funding and de-funding contingent on continued outcomes • Juvenile court involvement (administrators, supervisors, and staff) • Local provider involvement • Skilled independent researchers • Assessment • Research-based programs • Valid outcome evaluations • Cost-benefit analysis