Reforming California’s Juvenile Corrections System: An Evidence-Based Change Strategy*

1. Reforming California’s Juvenile Corrections System: An Evidence-Based Change Strategy* James M. Byrne, Ph.D. Professor Department of Criminal Justice University of Massachusetts, Lowell *Plenary presentation at the Transfer of Knowledge Workshop, “Parole Re-Entry: A New Vision”, sponsored by The Division of Juvenile Justice Parole Operations, Nov. 1,2005 Pomona, California

2. What is Evidence-based Practice? • It is the development and implementation of programs based on a systematic review of “what works” • There are three basic approaches to Evidence-based practice: • Strategy 1: Conduct a comprehensive review of all available research on a particular topic: • Examples: • The systematic reviews conducted by the Campbell Collaboration Crime and Justice Group (Sherman et. al, 2005; Sherman, et. al, 1997) • The systematic reviews using meta-analytic methods including experimental and quasi-experimental research (Gendreau, et. al, 1990)

3. What is Evidence-based Practice? (continued) • Strategy 2: Examine only a subset of all available research studies, using randomized field experiments as the “Gold Standard”, e.g. Farrington and Welsh’s recent review of all randomized experiments (2005) • Strategy 3: Conduct a nonscientific review, simply say “evidence based”, and then offer your own listing of best practices. • Reexamine/reposition scientific reviews • Only include a subset of all available research, often supporting either liberal or conservative ideology (Farabee, 2005; Cullen, 2002) • No specific identification of review procedures, inclusion/exclusion criteria, etc.

4. What is Meta-Analysis? • Meta-Analysis is a statistical technique designed to synthesize empirical relationships across a large number of studies. • Meta-Analysis allows us to determine both the Size and Direction of effects across studies. • Meta-Analysis techniques can be translated into summary statistics that public policy makers can easily understand (e.g. a 10% recidivism reduction effect for all correctional interventions using multi-systemic strategies) Sources: Wilson (2001); Rossi, Freeman, and Lipsey (2001)

5. When Should Meta-Analysis be Used? • Meta-Analysis should only be used to summarize research findings from methodologically rigorous evaluation designs. • Meta-Analysis should not be used when there are only a small number of studies (10 or less) being included in the review. • Meta-Analysis is most appropriate for studies examining the relationship between clearly defined independent and dependent variables.

6. What Review Criteria are Used in The Campbell Collaborative’s Systematic Reviews?

7. Study Inclusion Criteria For Systematic Reviews The scientific methods scale ranks evaluation studies from 1=weakest to 5=strongest on overall internal validity: • What Works: For a program to be classified as working, there must be a minimum of two level 3 studies with significance tests showing effectiveness and the preponderance of evidence in the same direction. • What Does Not Work: For a classification of not working, there must be a minimum of two level 3 studies with significance tests showing ineffectiveness and the preponderance of evidence in the same direction. • What is Promising: For the classification of promising, at least one level 3 study is required with significance tests showing effectiveness and preponderance of evidence in support of the same conclusion. • What is Unknown: Any program not classified in one of the threeabove categories is considered to have unknown effects. Source: Welsh and Farrington, (2003: 169-170)

8. Criteria for Assessing The Quality of Evaluation Studies: • Level 5: Well-designed randomized field experiment • Level 4: Randomized field experiment of moderate quality; well-designed quasi-experiment • Level 3: Quasi-experimental designs (moderate quality) • Level 2: Quasi-experimental designs (minimal quality) • Level 1: Non-experimental designs

9. How do Systematic Reviews (and Meta-Analyses) help us answer the following public policy question: • Should current sentencing and corrections policies – both institution-based and community-based – focus on offender rehabilitation or offender control?

10. The evidence in favor of rehabilitation is found in systematic reviews of correction research that estimate that the provision of treatment (in sufficient dosages and duration) is cost-effective and results in modest offender change (10% reduction) • The evidence opposed to rehabilitation is found in these same systematic reviews, which reveal that the vast majority of individual research studies do not find statistically significant differences between experimental and control groups in recidivism.

12. Current evidence-based reviews highlight the limitations inherent in offender-based change strategies: • Only incremental, short-term changes in offender behavior should be expected from the full implementation of evidence-based practices in adult and juvenile corrections • If we are interested in long-term offender change, we need to focus our attention on the community context of offender behavior • There is a growing body of research on the need to integrate individual and community-level change strategies (Sampson, et. al. 2005; Bursik, 2005; Carr, 2003)

13. The Need for Reform In California’s Juvenile Corrections System: • The Youth Incarceration Rate in California is among the highest in the country • Institutional costs represent a significant proportion (75%) of all juvenile corrections expenditures (400 million in FY ’04) • Institutionalization is having a negative effect on both juvenile offenders and on the communities in which juvenile offenders reside • Recycling/Churning: • Juvenile offenders released from CYA facilities are being rearrested and returned to prison/CYA at unacceptably high rates • Control-based community supervision strategies have failed to produce the desired recidivism reduction effects, either short-term or long-term

14. Source: Nadel-Hayes and Macallair, 2005: 32

15. Three Basic Organizational Change Strategies: • Empirical Rational Strategies: based on the notion that information/education can lead to behavior change • Normative Reeducation Strategies: based on the notion that attitudes/values must change before behavior will change • Power Coercive Strategies: based on the notion that formal, coercive, sanction-based strategies can induce behavior change

16. Common Characteristics of Successful Normative Reeducative Change Strategies: • At The Individual Level: • Cognitive Restructuring • Emphasize Positive Lifestyle Change (Employment, Marriage, Relocation) • Focus on Informal Social Controls • At The Organizational Level: • Leadership, Partnership, and Ownership • Evidence-Based Strategic Plan • Focus on Culture Change (Moral Performance) • New Ways of Doing Business: Performance Measurement

17. The Challenges for Community Corrections Managers Involved In Large-Scale Organizational Change • Design a reform strategy that is defensible – based on “best practices”/evidence-based reviews • Implement a reform strategy that is compatible with the values, skills, and capacity of line staff (and Mgt.) • Evaluate a reform strategy from the outset, based on the premise that “what gets measured gets done”

18. Resistance to Change in Community Corrections:A Typology • Conformists – Accept organization’s goals and follow standard policies and procedures • Ritualists – Reject organization’s goals but still follow policies and procedures, often at minimal compliance level • Artful Dodgers – May or may not accept organization’s goals but do not follow existing policies and procedures (catch me if you can) • Loose Cannons – May or may not accept organization’s goals but often directly challenge leadership on policies and procedures (you can not touch me)

19. A Research Agenda for California’s Division of Juvenile Parole Operations • Survey line staff on attitudes – Focus on organizational change initiatives/culture/resistance to change • Conduct workload study based on newly defined roles and responsibilities for line staff and management • Examine the reliability and validity of classification system (e.g. inter-rater reliability; prediction accuracy) • Monitor compliance with new policies and procedures using a clearly articulated performance measurement system • Evaluate both the implementation and impact of specific strategies (and programs) using level 3 (and higher) research designs (i.e. experiments and quasi-experiments)

20. NIC’s Eight Principles for The Utilization of Evidence-Based Practices in Community Corrections: • Assess Actuarial Risk/Needs • Enhance Intrinsic Motivation • Target Interventions • Risk Principle: Prioritize supervision and treatment resources for higher risk offenders • Need Principle: Target interventions to criminogenic needs • Responsivity Principle: Be responsive to temperament, learning style, motivation, culture, and gender when assigning programs. • Dosage: Structure 40-70% of high-risk offenders’ time for 3-9 months • Treatment: Integrate treatment into the full sentence/sanction requirements • Skill Train with Directed Practice (use Cognitive Behavioral treatment methods) • Increase Positive Reinforcement • Engage Ongoing Support in Natural Communities • Measure Relevant Processes/Practices • Provide Measurement Feedback

21. 6 months Structured Reentry Institution Surveillance ● Services ● Outreach Sign Behavioral Contract Change Education Treatment in Prison In-Prison Meeting with Community Meeting with Classification for Risk, Treatment, and Control Job Family Change Education Drug/ Alcohol Police/Community Reintegration Team Treatment In-Community Job Family Move Target Population to Select Facility Drug/ Alcohol Job Community Housing Victim Outreach Family Reentry Partnership Continuum Community/ Reintegration

22. Redefining The Criteria for A Successful Juvenile Corrections System The moral performance of institutional and community corrections needs to be examined along with cost, and offender change.

23. Relationships: Respect Humanity Relationships Trust Support Social Structure: Power/authority Social relations Regime: Fairness Order Safety Well-being Personal Development Family Contact Decency Other: Meaning Quality of life Defining The Moral Performance of Prisons: Source Adapted From: Allison Liebling (2005)

24. Cullen’s “Liberal but not stupid” approach to crime control • (1) This approach is informed by a general belief that people’s choices are bounded by social constraints, including constraints that are rooted in social inequalities—with many such inequalities tied to economic processes that may produce wealth but also have untoward, if not unfair, consequences. • (2) Knowing that harsh living conditions are implicated in crime, it is irresponsible not to pursue social welfare policies that ameliorate these disadvantages. • (3) Within the criminal justice system—a system that will affect millions of lives annually—it is irresponsible not to pursue policies that improve the plight of offenders, victims, and potential future victims.

25. Cullen’s “Liberal but not stupid” approach to crime control (continued) • (4) For those with progressive sentiments, the key issue is not nonintervention but rather the quality of the intervention that takes place in the criminal justice system. • (5) Beyond the values that inform policy choices, criminal justice interventions should not be based on prejudice, custom, or fashion but on “what works”—on what the data show are the most effective policies to pursue. • (6) Liberal welfare approaches to crime control that are revealed to be ineffective should not be defended but relinquished.

26. Concluding Comments • The focus of the next wave of adult and juvenile corrections programs should be on addressing the problems of both high-risk offenders and high-risk communities • At minimum, correctional interventions should be designed to do no harm, either to offenders or communities • It is not just adult and juvenile offenders that need to change; • It is the moral performance of corrections – both institutions and community corrections – that needs to change as well.