Training the Trainers: Juvenile Sexual Offender Risk Assessment with the JSORRAT-II

1. Training the Trainers: Juvenile Sexual Offender Risk Assessment with the JSORRAT-II Douglas L. Epperson, Ph.D. Washington State University (epperson@wsu.edu) Sacramento, CA April 8, 2009

2. Co-Authors of the JSORRAT-II Chris Ralston (Iowa State University) Dave Fowers and John DeWitt (Utah Juvenile Justice Services)

3. Overview

4. Necessity of Accurate Risk Assessment • Risk reduction (treatment) and risk management must be differential to be effective and affordable treatment or risk management intervention with the level of risk presented • Matching requires the accurate assessment of risk • Treatment also requires psychological assessment

5. Specific Values of Accurate Risk Assessment with Juveniles • Segregation of low risk juveniles from higher risk offenders to avoid contagion effects • More efficient use of limited resources • Matching treatment intensity with risk • Matching placement, programming, and supervision with risk • Better treatment and risk management outcomes • Empirical basis for limiting application of adult laws and policies to juveniles

6. Two Measurement Approaches to Risk Assessment Assessment of Empirically Linked Risk Indicators Relevant Past Events and Behavior Including Sexual Offenses Future Behavior (Risk) Relatively Stable Characteristics Assessment of Risk Factors

7. JSORRAT-II • Like most other actuarial risk assessment tools, JSORRAT-II is based on the measurement of empirically linked risk indicators • Correlated with risk • More readily and accurately assessed and serve as behavioral proxies for underlying complex interactions

8. Research to Develop an Actuarial Juvenile Risk Assessment Tool (JSORRAT-II) • Began a collaborative study about 5 years ago with Utah Juvenile Justice Services • This study required • a large sample that was fully representative of the broad spectrum of juveniles who offend sexually • a reliable measure of juvenile sexual recidivism to use as the criterion • a pool of potential predictor variables

9. Sample Details • Large (N = 636) • Essentially exhaustive sample of primarily 12-17 year olds adjudicated for a sex offense in Utah between 1990-92 • Fully representative of the complete spectrum of sexual offending behavior • Geographically limited to Utah

10. Ethnic Distribution in Sample • 76.4% Caucasian • 7.7% Hispanic/Latino • 2.2% African American • 1.6% Asian American • 1.4% Native American • 1.1% Multiethnic or other • 9.6% Unspecified

11. Sexual Recidivism Criterion and Rates • Juvenile sexual recidivism = charge for a new sex offense prior to age 18 • Base rate = 13.2% (84) • Adult sexual recidivism = charge for a new sex offense as adult (prior to 2004) • Base rate = 9.1% (58) • Anytime sexual recidivism = charge for a new sex offense at anytime (prior to 2004) • Base rate = 19.8% (126)

12. Data Collection for Potential Predictors • Files edited to appear as they did when the adolescent exited the juvenile criminal justice system for the 1990-92 index offense • Trained ISU students reviewed the files and extracted data into a series of codebooks • Didactic training • Group scoring and discussion of same cases • Pairs scoring and discussion of same cases • Subsequent questions directed to lab coordinator and recorded in a log book

13. Analyses • Followed a complex analytic procedure designed to identify the optimal set of variables that was most predictive of juvenile sexual recidivism • Simplified the model and confirmed that there was little information loss in using the simplified model over the more complex model • Subsequently assessed the ability of this set of variables to predict adult sexual recidivism and anytime sexual recidivism

14. Final 12 Variables that Comprise the JSORRAT-II • Analyses identified 12 variables that optimally predicted juvenile sexual recidivism • Principal components analysis suggested a four factor structure for these items

15. Factor 1: Persistence of Offending 1. Number of adjudications for sexual offenses 3. Length of offending history based on charged sexual offenses 2. Number of different victims in charged sexual offenses

16. Factor 2: Antisocial Orientation (Unwillingness or Inability to Follow Rules) 12. Number of adjudications for non-sexual offenses 11. Number of education time periods with discipline problems 4. Commission of a charged sexual offense while under court-ordered supervision 10. Placement in special education for any reason

17. Factor 3: Abuse History / Treatment Needs 9. Number of officially documented physical abuse incidents with JSO as victim 8. Number of officially documented “hands-on” sexual abuse incidents with JSO as victim 7. Prior sexual offender specific treatment failures

18. Factor 4: Risk Taking ?? 5. Commission of a charged, felony-level, “hands-on” sexual offense in a public place 6. Use of deception or grooming in any charged sexual offense

19. Simple Categorical Item Scoring System • Examined recidivism rates in the development sample • Set item level with lowest rate at a score of zero • Added one point for each item level with a meaningfully higher rate of recidivism • Note that the rates of recidivism are from the development sample, for which the tool is optimized – cannot be considered to be representative

20. Simple Categorical Scoring of Items • Number of SO adjudications

21. Number of different victims in charged sexual offenses

22. Number of officially documented hands-on, sexual abuse incidents in which the offender was the victim

23. Utah Development Sample Score Distribution

24. Predictive Accuracy of the Simplified Model • Area under ROC curve = .89 • 95% CI: .85 to .92

25. JSORRAT-II Score & Associated Juvenile Sexual Recidivism Rates

26. Performance of the JSORRAT – II with Anytime & Adult Sexual Recidivism • ROC-AUC for anytime sexual recidivism using the same 12 items and scoring was .79 • Driven by accuracy for juvenile sexual recidivism • ROC-AUC for adult sexual recidivism using the same 12 items and scoring was .64

27. Comparison of Juvenile and Adult Sexual Recidivism Rates

28. Reliability with the JSORRAT-II • Coders in our lab • Scored the same 16 cases selected by stratified (on expected score) random selection • Singular ICC for absolute agreement was .96

29. Reliability with the JSORRAT-II (cont.) • Collaborative study with Michelle Gourley and colleagues • Seven state evaluators who had attended a one-day training session • Scored the same 17 cases (stratified random selection) over the next couple of weeks • Singular ICC for absolute agreement was .91, which is excellent

30. Validation Studies • Currently conducting validations studies with independent samples from several states • Utah: completed for juvenile sexual recidivism but not for adult recidivism • Iowa: completed for juvenile sexual recidivism but not for adult recidivism • Georgia: in progress for juvenile sexual recidivism • California in preparation

31. Validation Studies (continued) • Research methods in validation studies • Same file preparation as in the development study • Same training methods for coders, but using a coding grid instead of the old coding books • 12 JSORRAT-II variables plus about 30 research variables

32. Utah Validation Study • Nearly exhaustive sample of 538 boys • adjudicated sexual offense in 1996 or 1997 • between the ages of 11.00 and 16.99 years • Complete data sample of 406 boys meeting criteria above and having complete data for all 12 JSORRAT-II items • Juvenile sexual recidivism base rates • Total sample: 12.8% (69/538) • Complete data sample: 12.3% (50/406) • Only real difference from development sample is temporal cohort

33. Utah Validation Sample Score Distribution

34. Utah Validation Study (continued) • Areas and under the ROC curves and 95% confidence intervals

35. Predicted Probabilities: Total Sample (N = 538) and the Complete Data Sample (N = 408)

36. Utah Predicted Probabilities of Recidivism

37. Optimal Risk Levels from Utah Validation Data

38. Iowa Validation Study • Nearly exhaustive sample of 366 boys • Adjudicated for a sexual offense in 1998-2000 • Between the ages 11 and 16.99 years • Base rate for juvenile sexual recidivism was 7.1% (26/366) • ROC-AUC = .65 (95% CI of .55 - .75)

39. Iowa Score Distribution

40. Iowa Predicted Probability of Sexual Recidivism

41. Iowa Predicted Probability of Sexual Recidivism

42. Iowa Predicted Probability of Sexual Recidivism

43. Optimal Risk Categories from Iowa Data

44. Summary • JSORRAT-II has been successfully validated in two states • Although the predictive validity was statistically significant in both studies, it was less accurate than in the development sample • This suggests that there may have been considerable “capitalization on chance” in the development sample -- but

45. Additional Likely Explanations • Given that both Utah samples are exhaustive samples of JSO’s that differ only temporally (1990-92 and 1996-97) AND • Given the much lower number of prior sex offense charges & adjudications in the validation samples AND • Given the dramatically lower recidivism rate for the Iowa sample (consistent with national trends)

46. Additional Likely Explanations (cont.) • There may have been systemic changes between 1990-92 and 1996-97 and again between the mid 1990’s and early 2000’s in the way JSO’s were charged, adjudicated, and managed • Recent discussions with Utah JJS officials confirms that beginning in about 1995 more minor offenses were charged and processed through the juvenile court • More first-time offenders with minor offenses and little history. More minor offenses as recidivating offenses

47. Additional Likely Explanations (cont.) • In Iowa, there is a possibility of a decreased likelihood of adjudicating sex offenses as sex offenses because of the consequences of registration and community notification • Risk management (i.e., supervision) and reduction (i.e., treatment) strategies may be working

48. Future Research • Revisit Utah validation data to explore time at risk • Perform additional analyses in both the Utah and the Iowa validation samples controlling for severity of index and recidivating offenses

49. Future Research (continued) • Complete validation studies in other states • Will provide multiple data points and ability to look for patterns (e.g., are Utah and Iowa validation samples outliers or part of a larger pattern) • If part of a larger pattern, evaluate research items for possible substitution to increase accuracy across states

50. Future Research (continued) • Collect and analyze adult recidivism for Utah an Iowa validation samples • Continue to follow both Utah samples (development and validation samples) and the Iowa sample (n = 1540 for three samples combined) further into adulthood • Examine temporal patterns in offending • Further assess ability, if any, to predict sexual offending as adults