DJJDP s Comprehensive Delinquency Prevention Intervention Strategy Buddy Howell Pinehurst, NC buddyhowellnc.rr

1. DJJDP’s Comprehensive Delinquency Prevention & Intervention Strategy Buddy HowellPinehurst, NCbuddyhowell@nc.rr.com Much of the material in this presentation appears in Howell, J.C. (2003). Preventing and Reducing Juvenile Delinquency: A Comprehensive Framework. Thousand Oaks, CA: Sage Publications. Much of the material in this presentation appears in Howell, J.C. (2003). Preventing and Reducing Juvenile Delinquency: A Comprehensive Framework. Thousand Oaks, CA: Sage Publications.

2. The Need For a Comprehensive Strategy Poor matching of prevention programs with risk factors for delinquency Poor targeting of serious, violent and chronic offenders Little use of risk and needs assessments Poor matching of offenders with the level of service Over-use of detention and incarceration This slide discusses the need for a Comprehensive Strategy. It highlights common problems in juvenile justice systems across the US. NC had all of these problems to some degree before it adopted the Comp Strat in the 1998 JJ Reform Act. We’ll see later how the NC JJS has been improved as a result of implementing the Comp Strat framework. Detention is used excessively in NC. There were nearly 9,000 admissions to detention centers in 2002 with an average length of stay of 11 days (DJJDP, 2002 Annual Report). Nearly half of them were under 15 years of age, and more than 50 were age 10 and younger. Minority youth are adversely affected by each of these problems. Overrepresentation of Black youth is a serious problem in NC. According to DJJDP’s 2002 Annual Report, nearly half (48%) of youths on whom risk assessments were made that year were African Americans; they represented more than half (55%) of detention center admissions. Nearly two-thirds of YDC admissions in 2001 were Black youth. The use of SDM tools will help alleviate these problems. It is suggested that one or two of the bullets be discussed and the participants given an opportunity to ask additional questions on this slide. Proceed if none come up.This slide discusses the need for a Comprehensive Strategy. It highlights common problems in juvenile justice systems across the US. NC had all of these problems to some degree before it adopted the Comp Strat in the 1998 JJ Reform Act. We’ll see later how the NC JJS has been improved as a result of implementing the Comp Strat framework. Detention is used excessively in NC. There were nearly 9,000 admissions to detention centers in 2002 with an average length of stay of 11 days (DJJDP, 2002 Annual Report). Nearly half of them were under 15 years of age, and more than 50 were age 10 and younger. Minority youth are adversely affected by each of these problems. Overrepresentation of Black youth is a serious problem in NC. According to DJJDP’s 2002 Annual Report, nearly half (48%) of youths on whom risk assessments were made that year were African Americans; they represented more than half (55%) of detention center admissions. Nearly two-thirds of YDC admissions in 2001 were Black youth. The use of SDM tools will help alleviate these problems. It is suggested that one or two of the bullets be discussed and the participants given an opportunity to ask additional questions on this slide. Proceed if none come up.

3. North Carolina’s Comprehensive Strategy is modeled after the OJJDP Comprehensive Strategy for Serious, Violent, and Chronic Juvenile Offenders (Wilson & Howell, 1993). There is considerable evidence that North Carolina’s Comprehensive Strategy is working, but several improvements need to be made. Disproportionate representation of Black youths in the JJS remains a serious problem. In addition, detention is used excessively. The disposition matrix and risk assessment instruments are producing improvements in the placements of offenders. Admissions to YDCs have dropped by two-thirds (without an increase in arrests or court complaints across the state). Court referrals have leveled off, but the decrease in admissions to YDCs puts additional pressure on court programs and increased use of detention is seen. Needs assessments should be used more along with risk assessments to achieve a better match between offenders and programs. In the next phase of this project, we are going to work with JCPCs and Juvenile Courts to evaluate programs and strengthen NCs Comp Strat and make improvements to it. This is a unique opportunity to integrate the prevention and intervention components, beginning with JCPC programs that serve court adjudicated youths. North Carolina’s Comprehensive Strategy is modeled after the OJJDP Comprehensive Strategy for Serious, Violent, and Chronic Juvenile Offenders (Wilson & Howell, 1993). There is considerable evidence that North Carolina’s Comprehensive Strategy is working, but several improvements need to be made. Disproportionate representation of Black youths in the JJS remains a serious problem. In addition, detention is used excessively. The disposition matrix and risk assessment instruments are producing improvements in the placements of offenders. Admissions to YDCs have dropped by two-thirds (without an increase in arrests or court complaints across the state). Court referrals have leveled off, but the decrease in admissions to YDCs puts additional pressure on court programs and increased use of detention is seen. Needs assessments should be used more along with risk assessments to achieve a better match between offenders and programs. In the next phase of this project, we are going to work with JCPCs and Juvenile Courts to evaluate programs and strengthen NCs Comp Strat and make improvements to it. This is a unique opportunity to integrate the prevention and intervention components, beginning with JCPC programs that serve court adjudicated youths.

4. Source: Howell, 2003, p. 88. (Copyrighted by Sage Publications) This figure zooms into the Comp Strat. The bottom track shows the main parts of the program and sanctions continuum. The middle section shows the approximate ages at which illustrative problem behaviors develop. The third track in the CS is added--risk and protective factors, and shows the approximate order in which they come into play from the standpoint of a child’s development over time. It also shows the age-related timeline along which problem behaviors develop, and the continuum of programs and strategies that is needed to effectively prevent and reduce delinquency. This figure illustrates the importance of early intervention with less costly and more effective programs. The cost to society of a delinquent’s criminal career that extends 10 years is from $1.7 to $2.3 million. Each year that the criminal career is reduced represents a large cost savings. The total cost of prevention and treatment court-based programs range from less than $1,000 to about $20,000. The average annual cost of incarcerating a juvenile is about $50,000. Therefore, prevention and intervention programs that succeed at least some of the time are likely to pay for themselves many times over. This figure also points to the need to integrate services across child and family service systems. Because the JCPCs fund early intervention and immediate intervention programs, we have a unique opportunity to integrate the Prevention & Graduated Sanctions components. Source: Howell, 2003, p. 88. (Copyrighted by Sage Publications) This figure zooms into the Comp Strat. The bottom track shows the main parts of the program and sanctions continuum. The middle section shows the approximate ages at which illustrative problem behaviors develop. The third track in the CS is added--risk and protective factors, and shows the approximate order in which they come into play from the standpoint of a child’s development over time. It also shows the age-related timeline along which problem behaviors develop, and the continuum of programs and strategies that is needed to effectively prevent and reduce delinquency. This figure illustrates the importance of early intervention with less costly and more effective programs. The cost to society of a delinquent’s criminal career that extends 10 years is from $1.7 to $2.3 million. Each year that the criminal career is reduced represents a large cost savings. The total cost of prevention and treatment court-based programs range from less than $1,000 to about $20,000. The average annual cost of incarcerating a juvenile is about $50,000. Therefore, prevention and intervention programs that succeed at least some of the time are likely to pay for themselves many times over. This figure also points to the need to integrate services across child and family service systems. Because the JCPCs fund early intervention and immediate intervention programs, we have a unique opportunity to integrate the Prevention & Graduated Sanctions components.

5. Integrated Prevention and Intervention Risk/protective factors in the individual, family, peer group, school, neighborhood Source: Howell, 2003, p. 195 (copyrighted by Sage Publications)Source: Howell, 2003, p. 195 (copyrighted by Sage Publications)

6. The CS is based on research on juvenile delinquency. It’s driven by data on risk and protective factors (the linchpin activities), and measurable outcomes (reductions in risk factors and delinquency). The SPEP is also research-based, data-driven, and outcome-focused, as we shall see in this training. It is based on prior program evaluations; it is driven by data in the Client Tracking Form; it’s purpose is to improve program outcomes. The CS is based on research on juvenile delinquency. It’s driven by data on risk and protective factors (the linchpin activities), and measurable outcomes (reductions in risk factors and delinquency). The SPEP is also research-based, data-driven, and outcome-focused, as we shall see in this training. It is based on prior program evaluations; it is driven by data in the Client Tracking Form; it’s purpose is to improve program outcomes.

7. Source: Snyder, 1998; reprinted in Howell, 2003, p. 59 (copyrighted by Sage Publications) This is a representation of 151,000 juvenile offender’s court careers in a large study of court referrals in Phoenix and surrounding areas (Maricopa Co.), that spanned 16 birth cohorts’ full period of eligibility for juvenile court jurisdiction (ages 10-17 in AZ). (See pp. 59-60 of Howell’s book). It illustrates 1) that two-thirds never became serious, violent, and chronic offenders (SVC) (i.e., low risk) and that 2) we need to focus especially on the 4% that are serious, violent & chronic offenders. Comparable NC numbers are currently available in preliminary data. In a DJJDP analysis of court risk assessments and dispositions during the last half of 2001 (p. 41 of the 2001 Annual Report), nearly 60% were neither serious, violent, nor chronic offenders; 27% were serious offenders (i.e., had a serious property complaint), only 10% were chronic (i.e. had 4 or more court referrals), and only 2% were referred to court for a violent offense. (The % that is SV&C was not determined, but it is probably around 1% because only 2% were violent offenders and studies show that only about half of the violent juvenile offenders are also chronic offenders). Source: Snyder, 1998; reprinted in Howell, 2003, p. 59 (copyrighted by Sage Publications) This is a representation of 151,000 juvenile offender’s court careers in a large study of court referrals in Phoenix and surrounding areas (Maricopa Co.), that spanned 16 birth cohorts’ full period of eligibility for juvenile court jurisdiction (ages 10-17 in AZ). (See pp. 59-60 of Howell’s book). It illustrates 1) that two-thirds never became serious, violent, and chronic offenders (SVC) (i.e., low risk) and that 2) we need to focus especially on the 4% that are serious, violent & chronic offenders. Comparable NC numbers are currently available in preliminary data. In a DJJDP analysis of court risk assessments and dispositions during the last half of 2001 (p. 41 of the 2001 Annual Report), nearly 60% were neither serious, violent, nor chronic offenders; 27% were serious offenders (i.e., had a serious property complaint), only 10% were chronic (i.e. had 4 or more court referrals), and only 2% were referred to court for a violent offense. (The % that is SV&C was not determined, but it is probably around 1% because only 2% were violent offenders and studies show that only about half of the violent juvenile offenders are also chronic offenders).

8. Source: Loeber et al., 1999 (copyrighted by Taylor & Francis, Inc.) These are developmental pathways in disruptive/delinquent behavior. Loeber's three-pathway model has strong empirical support, even in it's applicability to racial/ethnic minorities. It has been validated in youth populations in Denver, Rochester, Chicago, and also in a national sample of youth. For the vast majority of youths, the development of antisocial behavior appears to be congruent with the sequences in Loeber's model. Most important, compared with general delinquents, a larger proportion of serious and violent offenders followed the pathways. Thus the model holds promise for differentiating serious, violent, and chronic juvenile offenders from others. Source: Loeber et al., 1999 (copyrighted by Taylor & Francis, Inc.) These are developmental pathways in disruptive/delinquent behavior. Loeber's three-pathway model has strong empirical support, even in it's applicability to racial/ethnic minorities. It has been validated in youth populations in Denver, Rochester, Chicago, and also in a national sample of youth. For the vast majority of youths, the development of antisocial behavior appears to be congruent with the sequences in Loeber's model. Most important, compared with general delinquents, a larger proportion of serious and violent offenders followed the pathways. Thus the model holds promise for differentiating serious, violent, and chronic juvenile offenders from others.

9. Risk Factors for Delinquency

15. Percent of All Serious Violent Offenses Committed by Gang Members Source: Thornberry, 1998; figure appears in Howell, 2003, p. 84 (copyrighted by Sage Publications). The Denver, Rochester, and Seattle studies show that gang members, who make up only a small proportion of these large adolescent samples, commit the overwhelming majority of violent offenses committed by the entire samples. In general, gang members’ violent offense rates are up to seven times higher than the violent crime rates of adolescents who are not in gangs. In the Rochester adolescent sample, two-thirds (66%) of the chronic violent offenders were gang members. Source: Thornberry, 1998; figure appears in Howell, 2003, p. 84 (copyrighted by Sage Publications). The Denver, Rochester, and Seattle studies show that gang members, who make up only a small proportion of these large adolescent samples, commit the overwhelming majority of violent offenses committed by the entire samples. In general, gang members’ violent offense rates are up to seven times higher than the violent crime rates of adolescents who are not in gangs. In the Rochester adolescent sample, two-thirds (66%) of the chronic violent offenders were gang members.

16. 8th Graders’ Position in the Gang Most young youth gang members are not strongly bonded to their gang (The strongest bonding is indicated by gang members’ having placed themselves within the inner circle). This research suggests that interventions have a chance to work with most gang members. Most young youth gang members are not strongly bonded to their gang (The strongest bonding is indicated by gang members’ having placed themselves within the inner circle). This research suggests that interventions have a chance to work with most gang members.

17. A Graduated Sanctions Model This figure illustrates how risk offenders can be stepped up (as delinquency increases) and stepped down (as delinquency decreases) in a graduated sanctions system. Of course, most youth do not climb all of the steps. The majority of court referrals never become SV&C offenders. Some of them do not because of the effectiveness of early intervention and court/correctional rehabilitation programs. DJJDP has a has instituted the beginnings of a step-up & step-down system of graduated sanctions. It is seen in the three levels of court supervision. Court supervision of adjudicated youngsters across the state is structured in three graduated levels: standard supervision, modified supervision, and intensive supervision. Intensive Supervision, to which youth with elevated recidivism risk are assigned, uses an interdisciplinary team in designing and implementing intensive supervision and services. Step-down supervision and services are also provided as the offender's behavior improves. Some urban courts have designed even more highly-structured intensive probation supervision programs, coupled with comprehensive treatment plans, for offenders at elevated risk of becoming serious, violent, chronic offenders such as those who have committed a felony or serious misdemeanor offenses. This figure illustrates how risk offenders can be stepped up (as delinquency increases) and stepped down (as delinquency decreases) in a graduated sanctions system. Of course, most youth do not climb all of the steps. The majority of court referrals never become SV&C offenders. Some of them do not because of the effectiveness of early intervention and court/correctional rehabilitation programs. DJJDP has a has instituted the beginnings of a step-up & step-down system of graduated sanctions. It is seen in the three levels of court supervision. Court supervision of adjudicated youngsters across the state is structured in three graduated levels: standard supervision, modified supervision, and intensive supervision. Intensive Supervision, to which youth with elevated recidivism risk are assigned, uses an interdisciplinary team in designing and implementing intensive supervision and services. Step-down supervision and services are also provided as the offender's behavior improves. Some urban courts have designed even more highly-structured intensive probation supervision programs, coupled with comprehensive treatment plans, for offenders at elevated risk of becoming serious, violent, chronic offenders such as those who have committed a felony or serious misdemeanor offenses.

18. Structured Decision Making Tools Detention screening instruments Intake screening instruments Research-based risk risk assessments Objective assessments of youth and family strengths and needs A placement matrix for recommending court dispositions Standardized case plans Routine assessment of case plan progress Each of these is a very useful tool (see pp. 262-74 of Howell’s book). A detention screening instrument is badly needed in NC because of the excessive use of detention. DJJDP has taken early steps toward developing an intake screening instrument to help with the diversion of low-risk cases. As noted earlier, better use needs to be made of needs/strengths assessments in the development of treatment plans.Each of these is a very useful tool (see pp. 262-74 of Howell’s book). A detention screening instrument is badly needed in NC because of the excessive use of detention. DJJDP has taken early steps toward developing an intake screening instrument to help with the diversion of low-risk cases. As noted earlier, better use needs to be made of needs/strengths assessments in the development of treatment plans.

19. Key DJJDP SDM Tools DJJDP has a validated risk assessment instrument DJJDP has a needs/strengths assessment instrument The JJ Reform Act provided a Disposition Matrix The Disposition Matrix and risk assessment instrument are functioning well in guiding offender placements The next few slides illustrate these points. DJJDP also has put into use a Court Services Policies and Procedures manual. It will help standardize use of SDM tools and provide other benefits, such as consistency in case management. See pp. 251-257 of Howell’s book for a discussion of how well the SDM tools are working in NC. The DJJDP has implemented court services policies and procedures to ensure more consistency in court services and to enhance court effectiveness.The next few slides illustrate these points. DJJDP also has put into use a Court Services Policies and Procedures manual. It will help standardize use of SDM tools and provide other benefits, such as consistency in case management. See pp. 251-257 of Howell’s book for a discussion of how well the SDM tools are working in NC. The DJJDP has implemented court services policies and procedures to ensure more consistency in court services and to enhance court effectiveness.

20. Disposition Matrix A disposition matrix organizes sanctions and programs by risk level and offense severity. It places offenders along a continuum of programs and sanctions Research shows that a reliable risk assessment instrument predicts different recidivism rates at various risk levels. These are the basic functions of a Structured Decision Making system. See pp. 267-68 of Howell’s book for an explanation of risk assessment and referenced studies of risk assessment instruments that predicted different recidivism rates at various risk levels. These are the basic functions of a Structured Decision Making system. See pp. 267-68 of Howell’s book for an explanation of risk assessment and referenced studies of risk assessment instruments that predicted different recidivism rates at various risk levels.

21. Key Points of the Disposition Matrix Low risk offenders are placed in community programs with minimal supervision Medium risk offenders are typically placed in more structured community programs with intensive probation supervision High risk offenders may be placed in Youth Development Centers This is the general distribution of juvenile offenders in the NC JJS, guided by the Disposition Matrix.This is the general distribution of juvenile offenders in the NC JJS, guided by the Disposition Matrix.

22. A point system—factoring in chronic offending and whether or not the current offense was committed while the offender was on probation—guides determination of the risk level. The intersection of the risk level and current offense governs placement of an offender at Level 1, 2, or 3. For example, an offender with a serious and chronic offense history (i.e., high risk) who has been adjudicated for a violent offense would earn a Level 3 disposition, commitment to a Youth Development Center. Level 2 is an intermediate disposition, and Level 1 is a community disposition. Risk assessment instruments are used in some cases by the court to make a final determination (when the dispo matrix indicates that two levels are permissible) of the level into which offenders are to be placed. The Risk and Needs/Strength instruments need to be used more widely to determine program placements in cases that straddle boxes in the matrix. Considering that nearly 60% appear not to be serious, violent, and chronic offenders, these instruments can be used to great advantage to link low and medium risk offenders with appropriate sanctions and programs. An important next step is to link risk & needs profiles with disposition options the legislature provided for the 3 disposition levels (p. 253 of Howell’s book). These can be arrayed in the matrix. An example from Indiana is displayed on p. 272 of the book. A point system—factoring in chronic offending and whether or not the current offense was committed while the offender was on probation—guides determination of the risk level. The intersection of the risk level and current offense governs placement of an offender at Level 1, 2, or 3. For example, an offender with a serious and chronic offense history (i.e., high risk) who has been adjudicated for a violent offense would earn a Level 3 disposition, commitment to a Youth Development Center. Level 2 is an intermediate disposition, and Level 1 is a community disposition. Risk assessment instruments are used in some cases by the court to make a final determination (when the dispo matrix indicates that two levels are permissible) of the level into which offenders are to be placed. The Risk and Needs/Strength instruments need to be used more widely to determine program placements in cases that straddle boxes in the matrix. Considering that nearly 60% appear not to be serious, violent, and chronic offenders, these instruments can be used to great advantage to link low and medium risk offenders with appropriate sanctions and programs. An important next step is to link risk & needs profiles with disposition options the legislature provided for the 3 disposition levels (p. 253 of Howell’s book). These can be arrayed in the matrix. An example from Indiana is displayed on p. 272 of the book.

23. These data are based on all juvenile court referrals throughout the state during the last quarter of 2001—a total of more than 2,000 delinquent and “undisciplined” youths. A validation study by the Jordan Institute found that the risk assessment instrument and the current offense reliably classified youths in the three placement levels, as the disposition data in this slide suggest. Note (in the totals at the bottom) that most youths referred to juvenile courts in NC are low risk offenders. The low and medium risk categories combined constitute the bulk of the cases in the matrix boxes in which “wiggle room” is provided for courts to achieve the best match between offenders, sanctions, and services. The data displayed in this table suggest that North Carolina’s structured decision-making model is functioning very much as intended. Nearly two-thirds (65%) of the low-risk offenders received a Level 1 (community) disposition, and 70% of the high-risk offenders received a Level 3 (commitment) disposition. Thus the classification matrix and risk assessment instrument appear to be functioning quite well in guiding disposition decisions. Notice the large differences in the proportions of high-risk offenders who received Level 1 (3%), Level 2 (26%), and Level 3 (70%) dispositions. These differences are a good indication of the discriminatory power of the classification scheme. However, there is room for improvement in the case of medium risk offenders. Many of these fall into the “wiggle room” boxes of the matrix.These data are based on all juvenile court referrals throughout the state during the last quarter of 2001—a total of more than 2,000 delinquent and “undisciplined” youths. A validation study by the Jordan Institute found that the risk assessment instrument and the current offense reliably classified youths in the three placement levels, as the disposition data in this slide suggest. Note (in the totals at the bottom) that most youths referred to juvenile courts in NC are low risk offenders. The low and medium risk categories combined constitute the bulk of the cases in the matrix boxes in which “wiggle room” is provided for courts to achieve the best match between offenders, sanctions, and services. The data displayed in this table suggest that North Carolina’s structured decision-making model is functioning very much as intended. Nearly two-thirds (65%) of the low-risk offenders received a Level 1 (community) disposition, and 70% of the high-risk offenders received a Level 3 (commitment) disposition. Thus the classification matrix and risk assessment instrument appear to be functioning quite well in guiding disposition decisions. Notice the large differences in the proportions of high-risk offenders who received Level 1 (3%), Level 2 (26%), and Level 3 (70%) dispositions. These differences are a good indication of the discriminatory power of the classification scheme. However, there is room for improvement in the case of medium risk offenders. Many of these fall into the “wiggle room” boxes of the matrix.

24. The Jordan Institute validation study took the next step of determining whether or not youths placed in different levels actually had lower or higher recidivism rates. This is the “validity” issue: Do the Disposition Matrix and RA instrument serve to produce different recidivism rates at different risk levels? They do indeed. Overall, about one-third of the adjudicated offenders recidivated (with a new court complaint) within 9 months. Whereas only 20% of the low-risk offenders recidivated with new offenses, 34% of the medium-risk offenders recidivated, and 45% of the high-risk offenders had new offenses. These data show that the DJJDP risk assessment instrument reliably predicts recidivism at various risk levels. A DJJDP recidivism study found that 21% of A-E felony adjudications recidivated within 2-3 years. The study excluded offenders that were in YDCs, because they couldn’t recidivate. Juvenile courts issued an average of 4.5 court orders for treatment for these juvenile offenders. This is evidence of the “never give up” philosophy of NC juvenile courts.The Jordan Institute validation study took the next step of determining whether or not youths placed in different levels actually had lower or higher recidivism rates. This is the “validity” issue: Do the Disposition Matrix and RA instrument serve to produce different recidivism rates at different risk levels? They do indeed. Overall, about one-third of the adjudicated offenders recidivated (with a new court complaint) within 9 months. Whereas only 20% of the low-risk offenders recidivated with new offenses, 34% of the medium-risk offenders recidivated, and 45% of the high-risk offenders had new offenses. These data show that the DJJDP risk assessment instrument reliably predicts recidivism at various risk levels. A DJJDP recidivism study found that 21% of A-E felony adjudications recidivated within 2-3 years. The study excluded offenders that were in YDCs, because they couldn’t recidivate. Juvenile courts issued an average of 4.5 court orders for treatment for these juvenile offenders. This is evidence of the “never give up” philosophy of NC juvenile courts.

25. Admissions to YDCs have been reduced by two-thirds since the 98 JJ Reform Act.Admissions to YDCs have been reduced by two-thirds since the 98 JJ Reform Act.

26. This figure shows the changing composition of offender in the YDCs since the 98 JJ Reform Act. The legislature sought to achieve a key objective of the Comprehensive Strategy: reserve secure confinement for serious, violent, and chronic juvenile offenders. The Matrix is working to reserve costly YDC beds (nearly $50,000 per year) for the SVC offenders.This figure shows the changing composition of offender in the YDCs since the 98 JJ Reform Act. The legislature sought to achieve a key objective of the Comprehensive Strategy: reserve secure confinement for serious, violent, and chronic juvenile offenders. The Matrix is working to reserve costly YDC beds (nearly $50,000 per year) for the SVC offenders.

27. A Practical Approach To Evaluating and Improving Juvenile Justice Programs Utilizing The Standardized Program Evaluation Protocol  Dr. Lipsey and Dr. Howell are co-directors of the project. Dr. Lipsey and Dr. Howell are co-directors of the project.

28. Participating Organizations Department of Juvenile Justice and Delinquency Prevention Vanderbilt Institute for Public Policy This project is funded by the Governor’s Crime Commission & DJJDP.

29. Project Team Dr. Mark Lipsey (Vanderbilt Univ.) Dr. James “Buddy” Howell (NC) Dr. Simon Tidd (Vanderbilt Univ.) Mr. Ron Mangum, M.A. (NC) Dr. James “Jim” Palmer (NC) DJJDP Ms. Susan Whitten, State Administrator, Intervention & Prevention Division Ron Mangum is the Training Coordinator, and Jim Palmer is a professional trainer. Simon Tidd, is a meta-analyst and key resource person on Dr. Lipsey’s staff, and Susan Whitten provides valuable guidance in integrating the project with DJJDP activities. Ron Mangum is the Training Coordinator, and Jim Palmer is a professional trainer. Simon Tidd, is a meta-analyst and key resource person on Dr. Lipsey’s staff, and Susan Whitten provides valuable guidance in integrating the project with DJJDP activities.

30. Pilot Counties These are the 8 pilot counties in the SPEP project.These are the 8 pilot counties in the SPEP project.

31. DJJDP & JCPC Evaluation Requirements in 1998 Juvenile Justice Reform Act DJJDP and the JCPCs share the responsibility for evaluating every program funded by JCPCs. Reducing recidivism is one of six outcome measures for JCPC programs. DJJDP has responsibility for identifying research-based programs (“best practices”) for use across the state. The Lipsey-Howell project provides a user-friendly method for comparing current programs with the most effective evaluated programs.DJJDP and the JCPCs share the responsibility for evaluating every program funded by JCPCs. Reducing recidivism is one of six outcome measures for JCPC programs. DJJDP has responsibility for identifying research-based programs (“best practices”) for use across the state. The Lipsey-Howell project provides a user-friendly method for comparing current programs with the most effective evaluated programs.

32. North Carolina’s Practical Approach to Improving Juvenile Justice System Programs Most juvenile justice programs reduce recidivism--at least slightly. The most practical and cost-effective approach is to improve existing programs. This can be done by applying research-based knowledge of the features of effective programs. We’ll see later in this training that most JJ programs reduce recidivism--at least slightly. The main objective of the practical approach DJJDP is taking is to improve existing programs rather than “throwing the baby out with the bath water” and chasing “magic bullets.” Communities can make existing programs more effective by configuring them around the program features that produce large recidivism reductions. This approach is an alternative to the current popular approach of attempting to replicate effective Research and Demonstration (R&D) programs. This can be a very costly and difficult enterprise. Few of them would be feasible in NC’s 92 rural counties. We’ll see later in this training that most JJ programs reduce recidivism--at least slightly. The main objective of the practical approach DJJDP is taking is to improve existing programs rather than “throwing the baby out with the bath water” and chasing “magic bullets.” Communities can make existing programs more effective by configuring them around the program features that produce large recidivism reductions. This approach is an alternative to the current popular approach of attempting to replicate effective Research and Demonstration (R&D) programs. This can be a very costly and difficult enterprise. Few of them would be feasible in NC’s 92 rural counties.

33. Standardized Program Evaluation Protocol Development: The Evidence Base Dr. Mark W. Lipsey Vanderbilt University This training module begins with discussion of Dr. Lipsey’s pioneering meta-analyses (statistical syntheses of studies--see p. 198 of the book for an explanation of meta-analysis procedures). This training module begins with discussion of Dr. Lipsey’s pioneering meta-analyses (statistical syntheses of studies--see p. 198 of the book for an explanation of meta-analysis procedures).

34. Source: Lipsey, 2000 (copyrighted) This figure is based on Dr. Lipsey’s meta-analysis of 556 juvenile justice system program evaluations. It shows that most juvenile justice system programs (57%) reduce recidivism. This slide shows that there is a preponderance of positive effects from JJS programs, but at the individual study level they are not necessarily statistically significant (largely due to small sample size). At the meta-analysis level, we clearly see that most of the research shows positive effects.Dr. Lipsey has been instrumental in debunking the “nothing works” myth with respect to juvenile offenders (p. 197 of Howell’s book) and also the myth that juvenile courts are not effective (p. 199). (See also pp. 203-04) Source: Lipsey, 2000 (copyrighted) This figure is based on Dr. Lipsey’s meta-analysis of 556 juvenile justice system program evaluations. It shows that most juvenile justice system programs (57%) reduce recidivism. This slide shows that there is a preponderance of positive effects from JJS programs, but at the individual study level they are not necessarily statistically significant (largely due to small sample size). At the meta-analysis level, we clearly see that most of the research shows positive effects.Dr. Lipsey has been instrumental in debunking the “nothing works” myth with respect to juvenile offenders (p. 197 of Howell’s book) and also the myth that juvenile courts are not effective (p. 199). (See also pp. 203-04)

35. These are the four main predictors of recidivism reductions in Dr. Lipsey’s meta-analysis : Primary intervention--the effectiveness of the primary intervention, independent of its use with another intervention; Supplementary services--adding another service component to an intervention may, but often does not, increase its effectiveness; Amount of Service--the amount and quality of service provided, as indicated in service frequency, program duration, and extent of implementation; and Characteristics of the juvenile clients--some programs are more effective for high-risk juveniles than low-risk offenders and vice versa; others are more effective for older or younger offenders. These are the four main predictors of recidivism reductions in Dr. Lipsey’s meta-analysis : Primary intervention--the effectiveness of the primary intervention, independent of its use with another intervention; Supplementary services--adding another service component to an intervention may, but often does not, increase its effectiveness; Amount of Service--the amount and quality of service provided, as indicated in service frequency, program duration, and extent of implementation; and Characteristics of the juvenile clients--some programs are more effective for high-risk juveniles than low-risk offenders and vice versa; others are more effective for older or younger offenders.

36. Comparison of Programs with Varying Numbers of Favorable CharacteristicsProportion of practical programs with different numbers of favorable characteristics and associated change in recidivism rates relative to control group Source: Lipsey, 1999 Dr. Lipsey analyzed nearly 200 evaluations of practical (everyday) juvenile justice system programs, most of which are juvenile court programs, many of which are developed by staff. Programs that had none of the four characteristics (7% of the programs) increased recidivism somewhat. Those that had only one of the four characteristics (50% of the programs) reduced recidivism insignificantly (only 2%). However, those that had two of the four characteristics (27% of the programs) reduced recidivism 10%, those with three of the four characteristics (15% of the programs) reduced recidivism 21%, and those with all four of the highly effective program characteristics (2% of the programs) reduced recidivism 25%. Thus the presence of three or four of the desired program characteristics produced a very substantial impact on recidivism. Dr. Lipsey’s analysis shows that there is much room for improvement in everyday juvenile justice system programs. This is a main purpose of the SPEP implementation process.Source: Lipsey, 1999 Dr. Lipsey analyzed nearly 200 evaluations of practical (everyday) juvenile justice system programs, most of which are juvenile court programs, many of which are developed by staff. Programs that had none of the four characteristics (7% of the programs) increased recidivism somewhat. Those that had only one of the four characteristics (50% of the programs) reduced recidivism insignificantly (only 2%). However, those that had two of the four characteristics (27% of the programs) reduced recidivism 10%, those with three of the four characteristics (15% of the programs) reduced recidivism 21%, and those with all four of the highly effective program characteristics (2% of the programs) reduced recidivism 25%. Thus the presence of three or four of the desired program characteristics produced a very substantial impact on recidivism. Dr. Lipsey’s analysis shows that there is much room for improvement in everyday juvenile justice system programs. This is a main purpose of the SPEP implementation process.

37. The Standardized Program Evaluation Protocol(SPEP) This approach to “evidence-based” or “research-based” practices has the potential for moving the juvenile justice field ahead of other human service fields, which still attempt to replicated R&D programs. The points assigned to programs in the SPEP rating process correspond to the contribution (added increment) that each program characteristic makes to recidivism reduction in prior studies.This approach to “evidence-based” or “research-based” practices has the potential for moving the juvenile justice field ahead of other human service fields, which still attempt to replicated R&D programs. The points assigned to programs in the SPEP rating process correspond to the contribution (added increment) that each program characteristic makes to recidivism reduction in prior studies.

38. The SPEP cont’d Dr. Lipsey has amassed, by far, the most comprehensive database of delinquency prevention and juvenile justice system program evaluations. He received an OJJDP award a few years ago to expand his database. It consists of all known published and unpublished studies. JCPC Program Agreements were coded into his database in the same manner that evaluated programs had been coded. Cluster analysis was used to identify the evaluated programs that were most similar to the unevaluated NC programs represented in the JCPC Program Agreements. The SPEP was created from analyses of the similar evaluated programs.Dr. Lipsey has amassed, by far, the most comprehensive database of delinquency prevention and juvenile justice system program evaluations. He received an OJJDP award a few years ago to expand his database. It consists of all known published and unpublished studies. JCPC Program Agreements were coded into his database in the same manner that evaluated programs had been coded. Cluster analysis was used to identify the evaluated programs that were most similar to the unevaluated NC programs represented in the JCPC Program Agreements. The SPEP was created from analyses of the similar evaluated programs.

39. What the SPEP is NOT Program evaluations rarely examine outcomes beyond recidivism. As more studies accumulate that examine other outcomes, such as improved school performance, better family relations, etc., future meta-analyses will be done that cover such other outcomes. The SPEP guides juvenile justice managers in the use of main and supplementary interventions for client categories. The details of treatment plan development is left to professionals.Program evaluations rarely examine outcomes beyond recidivism. As more studies accumulate that examine other outcomes, such as improved school performance, better family relations, etc., future meta-analyses will be done that cover such other outcomes. The SPEP guides juvenile justice managers in the use of main and supplementary interventions for client categories. The details of treatment plan development is left to professionals.

40. These are the most widely used effective interventions among all JCPC and court delinquency services programs. Each of them has been found to be effective elsewhere. The central question is whether or not they are administered effectively in NC. SPEPs have not been developed for two categories of programs: 1) those that are not effective (such as shock incarceration) and 2) those for which there is not sufficient research to specify what works (such as sex offender treatment).These are the most widely used effective interventions among all JCPC and court delinquency services programs. Each of them has been found to be effective elsewhere. The central question is whether or not they are administered effectively in NC. SPEPs have not been developed for two categories of programs: 1) those that are not effective (such as shock incarceration) and 2) those for which there is not sufficient research to specify what works (such as sex offender treatment).

41. SPEPs have not been developed for these four intervention types because they are not used frequently enough in JCPC programs to make it worthwhile to produce and use them.SPEPs have not been developed for these four intervention types because they are not used frequently enough in JCPC programs to make it worthwhile to produce and use them.

42. This listing shows the relative effectiveness of the interventions most commonly used in JCPC prevention programs. Note that all of them are effective types of interventions. Some have proved to be more effective than others in prior studies. Note also that restitution is not included here. It is not often used for prevention purposes (in working with youth who have not been adjudicated delinquent or have been diverted from court).This listing shows the relative effectiveness of the interventions most commonly used in JCPC prevention programs. Note that all of them are effective types of interventions. Some have proved to be more effective than others in prior studies. Note also that restitution is not included here. It is not often used for prevention purposes (in working with youth who have not been adjudicated delinquent or have been diverted from court).

43. This listing shows the relative effectiveness of the interventions most commonly used in NC court delinquency supervision programs. Note the differences in the relative degree of effectiveness of intervention types when used for adjudicated delinquents versus when used as a prevention intervention. Note also that restitution is included in this court supervision group. This intervention type is far more commonly used in NC court supervision programs than as a prevention measure.This listing shows the relative effectiveness of the interventions most commonly used in NC court delinquency supervision programs. Note the differences in the relative degree of effectiveness of intervention types when used for adjudicated delinquents versus when used as a prevention intervention. Note also that restitution is included in this court supervision group. This intervention type is far more commonly used in NC court supervision programs than as a prevention measure.

44. There will be 3 sets of SPEP program rating forms for the 3 sections of the NC juvenile justice continuum, permitting evaluation of every program that DJJDP funds, statewide.There will be 3 sets of SPEP program rating forms for the 3 sections of the NC juvenile justice continuum, permitting evaluation of every program that DJJDP funds, statewide.

45. Source: Copyrighted by the DJJDP, and CERM, Vanderbilt University. This is a “standardized program evaluation protocol” (SPEP) for parent training/counseling interventions that are used for prevention purposes (i.e., for non-adjudicated youth). It contains the 4 features of the most effective evaluated programs of this type in Dr. Lipsey’s database. We’ll look next at each section of the SPEP instrument. This rating scheme corresponds with what research studies show about various mixtures of program features. Dr. Lipsey has adjusted the incremental contributions that each of the 4 features of the most effective evaluated programs makes to recidivism outcomes to reflect their value in terms of a numerical score. Recall the grouping of the 9 most commonly used prevention interventions into “effective, but above average,” “effective, but average,” and “effective, but below average” categories. The “effective, but above average” programs are allocated a maximum of 100 points “effective, but average” programs can score a maximum of 90 points and “effective, but below average” programs can reach a maximum of 80 points. This means that there is a maximum of 40 points that any program can gain from the remaining three features of effective programs. It is important to note that each of the SPEP program ratings can be done entirely from the Client Tracking Form.Source: Copyrighted by the DJJDP, and CERM, Vanderbilt University. This is a “standardized program evaluation protocol” (SPEP) for parent training/counseling interventions that are used for prevention purposes (i.e., for non-adjudicated youth). It contains the 4 features of the most effective evaluated programs of this type in Dr. Lipsey’s database. We’ll look next at each section of the SPEP instrument. This rating scheme corresponds with what research studies show about various mixtures of program features. Dr. Lipsey has adjusted the incremental contributions that each of the 4 features of the most effective evaluated programs makes to recidivism outcomes to reflect their value in terms of a numerical score. Recall the grouping of the 9 most commonly used prevention interventions into “effective, but above average,” “effective, but average,” and “effective, but below average” categories. The “effective, but above average” programs are allocated a maximum of 100 points “effective, but average” programs can score a maximum of 90 points and “effective, but below average” programs can reach a maximum of 80 points. This means that there is a maximum of 40 points that any program can gain from the remaining three features of effective programs. It is important to note that each of the SPEP program ratings can be done entirely from the Client Tracking Form.

46. Source: Copyrighted by the DJJDP, and CERM, Vanderbilt University. Note first of all, that when parent training/counseling is used as a court delinquency supervision program (50 points), it is not as effective overall as when it is used for prevention clients (60 points). Again, this is a research-based difference. On average, parent training/counseling is somewhat less effective for adjudicated delinquents than for non-adjudicated and diverted youth. Other differences are apparent--especially in supplemental services that boost the effectiveness of parent training/counseling. When used as a prevention program for non-adjudicated and diverted youth, parent training/counseling combines very well with tutoring or mentoring. However, when it is used as a treatment program for adjudicated delinquents, it combines best with family counseling and cognitive-behavioral training. Source: Copyrighted by the DJJDP, and CERM, Vanderbilt University. Note first of all, that when parent training/counseling is used as a court delinquency supervision program (50 points), it is not as effective overall as when it is used for prevention clients (60 points). Again, this is a research-based difference. On average, parent training/counseling is somewhat less effective for adjudicated delinquents than for non-adjudicated and diverted youth. Other differences are apparent--especially in supplemental services that boost the effectiveness of parent training/counseling. When used as a prevention program for non-adjudicated and diverted youth, parent training/counseling combines very well with tutoring or mentoring. However, when it is used as a treatment program for adjudicated delinquents, it combines best with family counseling and cognitive-behavioral training.

47. Source: Dr. Mark Lipsey, CERM, Vanderbilt Univ.unpublished. This analysis by Dr. Lipsey illustrates the incremental gains in expected recidivism for prevention programs with successively more favorable intervention characteristics. It shows that the average recidivism rate for control groups is 30% versus 27% for juveniles in the average prevention program for which Dr. Lipsey has an evaluation in his database. When one of the most effective program interventions is used, the average recidivism rate drops to 25%. Lower recidivism rates are associated with each addition of a favorable program characteristic, from 20% for those programs using a best supplemental service to only 13% for those with all four favorable program characteristics. Again, the more favorable characteristics a program has, the more it reduces recidivism in the studies in Dr. Lipsey’s database. These data suggest that recidivism in the average prevention program can potentially be reduced by 17% (from 30% to 13%). Incremental improvements can add up to an overall significant recidivism reduction. This is an innovative approach to program improvement that has a great deal of practical application. Source: Dr. Mark Lipsey, CERM, Vanderbilt Univ.unpublished. This analysis by Dr. Lipsey illustrates the incremental gains in expected recidivism for prevention programs with successively more favorable intervention characteristics. It shows that the average recidivism rate for control groups is 30% versus 27% for juveniles in the average prevention program for which Dr. Lipsey has an evaluation in his database. When one of the most effective program interventions is used, the average recidivism rate drops to 25%. Lower recidivism rates are associated with each addition of a favorable program characteristic, from 20% for those programs using a best supplemental service to only 13% for those with all four favorable program characteristics. Again, the more favorable characteristics a program has, the more it reduces recidivism in the studies in Dr. Lipsey’s database. These data suggest that recidivism in the average prevention program can potentially be reduced by 17% (from 30% to 13%). Incremental improvements can add up to an overall significant recidivism reduction. This is an innovative approach to program improvement that has a great deal of practical application.

48. Expected Recidivism with Features of Effective Court Delinquency Supervision Programs Source: Dr. Mark Lipsey, CERM, Vanderbilt Univ.unpublished. This analysis by Dr. Lipsey illustrates the incremental gains in expected recidivism for juvenile court programs with successively more favorable intervention characteristics. It shows that the average recidivism rate for control groups is 40% versus 34% for juveniles in the average court supervision program for which Dr. Lipsey has an evaluation in his database. When one of the most effective program interventions is used, the average recidivism rate drops to 32%. Lower recidivism rates are associated with each addition of a favorable program characteristic, from 28% for those programs using a best supplemental service to only 21% for those with all four favorable program characteristics. Again, the more favorable characteristics a program has, the more it reduces recidivism in the studies in Dr. Lipsey’s database. These data suggest that recidivism in the average court supervision program can potentially be reduced by 13% (from 34% to 21%). Incremental improvements can add up to an overall significant recidivism reduction. This is an innovative approach to program improvement that has a great deal of practical application. The average recidivism rate in NC court supervision programs is unknown. NC-JOIN will soon be able to answer this question for us. However, note that in the first full year that risk assessments were done (2002), 65% of the cases on which risk assessments were done had no prior adjudication (p. 18 of the DJJDP 2002 Annual Report). Dr. Lipsey’s calculation of the average recidivism rate for court delinquency supervision programs is noteworthy for another reason. This low rate further refutes the myth that juvenile courts are ineffective (see p. 199 of the book). However, there is much room for improvements, which can be made using the SPEP process. Source: Dr. Mark Lipsey, CERM, Vanderbilt Univ.unpublished. This analysis by Dr. Lipsey illustrates the incremental gains in expected recidivism for juvenile court programs with successively more favorable intervention characteristics. It shows that the average recidivism rate for control groups is 40% versus 34% for juveniles in the average court supervision program for which Dr. Lipsey has an evaluation in his database. When one of the most effective program interventions is used, the average recidivism rate drops to 32%. Lower recidivism rates are associated with each addition of a favorable program characteristic, from 28% for those programs using a best supplemental service to only 21% for those with all four favorable program characteristics. Again, the more favorable characteristics a program has, the more it reduces recidivism in the studies in Dr. Lipsey’s database. These data suggest that recidivism in the average court supervision program can potentially be reduced by 13% (from 34% to 21%). Incremental improvements can add up to an overall significant recidivism reduction. This is an innovative approach to program improvement that has a great deal of practical application. The average recidivism rate in NC court supervision programs is unknown. NC-JOIN will soon be able to answer this question for us. However, note that in the first full year that risk assessments were done (2002), 65% of the cases on which risk assessments were done had no prior adjudication (p. 18 of the DJJDP 2002 Annual Report). Dr. Lipsey’s calculation of the average recidivism rate for court delinquency supervision programs is noteworthy for another reason. This low rate further refutes the myth that juvenile courts are ineffective (see p. 199 of the book). However, there is much room for improvements, which can be made using the SPEP process.

49. Let’s examine the programs and sanctions in the NC Disposition Levels (HANDOUT). That is not a very precise array of disposition options. It would be difficult to place them in the Disposition Matrix in a way that we could agree upon. The second row in this figure contains several of the “program types” that are used across the state. The third level is effective interventions that can be provided in particular program types. DJJDP is working with the Area Consultants to refine this continuum. The next step, when we return in August or September, will be to work with you in adapting that statewide continuum to reflect the actual continuum in your particular county. We will soon have SPEPs for all three sections of the NC continuum. We can use the SPEPs to help determine the most effective program type in which an effective intervention should be located. Dr. Lipsey’s analysis of risk and needs assessments will provide a profile of risk levels and pressure points in treatment needs. This information, along with the SPEPs will help strengthen the overall continuum.Let’s examine the programs and sanctions in the NC Disposition Levels (HANDOUT). That is not a very precise array of disposition options. It would be difficult to place them in the Disposition Matrix in a way that we could agree upon. The second row in this figure contains several of the “program types” that are used across the state. The third level is effective interventions that can be provided in particular program types. DJJDP is working with the Area Consultants to refine this continuum. The next step, when we return in August or September, will be to work with you in adapting that statewide continuum to reflect the actual continuum in your particular county. We will soon have SPEPs for all three sections of the NC continuum. We can use the SPEPs to help determine the most effective program type in which an effective intervention should be located. Dr. Lipsey’s analysis of risk and needs assessments will provide a profile of risk levels and pressure points in treatment needs. This information, along with the SPEPs will help strengthen the overall continuum.

50. Next Steps in thePilot Counties (September-October) SPEP evaluation of individual JCPC programs using client tracking information Engage service providers in making program improvements to conform more closely with best practices This is the first priority in the pilot counties.This is the first priority in the pilot counties.

51. Next Steps cont.(September-October) This is the second priority in pilot counties.This is the second priority in pilot counties.

52. Statewide Roll-out (October-March) Presentations on the Lipsey-Howell Project at the 4 Area Meetings with Chief Court Counselors is a beginning point of information dissemination. The remaining details of rollout across the state are “to be determined” (TBD). Susan Whitten has announced that rollout in the JCPCs in the remaining 92 counties will consist of only program ratings in the upcoming fiscal year. We will continue to engage the 8 pilot counties in continuum building in the upcoming fiscal year. Presentations on the Lipsey-Howell Project at the 4 Area Meetings with Chief Court Counselors is a beginning point of information dissemination. The remaining details of rollout across the state are “to be determined” (TBD). Susan Whitten has announced that rollout in the JCPCs in the remaining 92 counties will consist of only program ratings in the upcoming fiscal year. We will continue to engage the 8 pilot counties in continuum building in the upcoming fiscal year.