Tim Brennan Ph.D Chief Scientist Northpointe , Inc. tbrennan38@earthlink.net

1. How can Risk/Needs assessment (LSI, COMPAS) support Court Stakeholders with Case Description, Interpretation and Sentencing?MAY 21, 2013 ATLANTA, GEORGIA Tim Brennan Ph.D Chief Scientist Northpointe, Inc. tbrennan38@earthlink.net

2. Objectives 1. Review basics of Risk-Need Assessment (RNA) 2. Identifying best predictors of recidivism 3. Brief review of evidence based practice (EBP) to enhance correctional outcomes 4. Explore how Risk/Needs may support sentencing decisions (Using LSI or COMPAS)

3. 5. Support for Case interpretation 1. Systematic Case Description (RNA) Risk assessment Comprehensive Needs Assessment 2. Case Interpretation (Understanding, RNA) 3. Select Sentencing goals, Select Interventions 4. Monitoring acase (Progress and Outcomes) 6. SomeImplementation challenges

4. Part 1 - The Context Problems and challenges in Sentencing?

5. Recent Trends requiring EBP& RNA in Courts 1. Legislative statutes, Financial pressure, Research pressure requiring Risk needs assessment (RNA) & EBP in Sentencing: • Indiana - State Statutes requiring use of RNA and EBP; Michigan • Oregon (2003) DOC funding for EBP programs and RNA • Washington Legislature 2005 directed Institute for Policy Study to study fiscal savings of EBP treatment programs. 2. Pressures to improve/upgrade sentencing • Crime reduction must be a component of sentencing • Judges must consider impact of sentences in reducing future crime • Strong focus on effectiveness of sentencing, rehabilitation • Rapid growth of Problem Solving Courts and “Smart Sentencing” 3. Rehabilitation strongly resurrected – research last 10 years 1. From “Nothing Works” to “What works”; Meta-analytic findings, EBP 2. Broad “buy-in” to Risk-Need-Responsivity and EBP inCorrections

6. Part 2 Overview of Risk & Needs Assessment • The problem of Decision Errors • Comparing Actuarial methods tohuman judgment • What are the most “valid/powerful” risk / need factors • Scientific criteria for acceptable RNA tools

7. EBP – Research Consensus on best Predictors of Recidivism The big 4 (or 5 or six) • Criminal History • Criminal Peers • Criminal Opportunity / High Risk Lifestyle • Criminal Personality • Criminal Thinking (Anti-Social Cognitions) -------------------------------------------------------------------- 6. Drug Abuse (early onset; dependency) 7. Work/Educational Failure - Low Social Capitol 8. Family Risk Factors (Parenting failure, parent crime/drugs, family support) 9. Environmental Factors: Hi crime area, Stress, gangs, unemployment, etc Both COMPAS and LSI cover most of these factors 9 9

8. WHAT ARE THE MOST POWERFUL AND RELEVANT VARIABLES REGARDING RECIDIVISM(20 yrs. of Meta-analytic predictive studies - EBP) 10

9. KEY FACTORS FOR WOMEN’S ASSESSMENT SOCIAL EXCLUSION / Economic Marginalization Poverty - housing safety - educational/job resources Economic dependency PARENTING ISSUES - Especially Single Parenting Parenting involvement - Parenting stress/anxiety PSYCHOLOGICAL - MENTAL HEALTH ISSUES Mental Illness - depression - anxiety - Low self efficacy - Anger Self esteem, Self efficacy Trauma, stemming from Sexual abuse, Battering in curr. relations FAMILY & RELATIONSHIP - SIGNIFICANT OTHER ISSUES Family support - Family conflict – Physical and sexual abuse Sig.Other support/conflict - Victimization (physical, sexual) Sig.Other dominance/exploitation (drug prostitution) CRIMINAL OFFENDING PATTERNS OF WOMEN Typically less serious and less violent than males, mostly drug and minor property

10. EBP in Risk Level Assessment: Professional Judgment or Actuarial Risk Assessment, or both? 1. Data Collection Phase 2. Data Integration Phase 3. Decision Phase 4. Action Phase Human Judgment Risk/Needs Assessment Background Data Crim. History Data Reach Decisions Action Outcomes Statistical Risk Assessment Kinds of Errors: 1. Over Classification 2. Under Classification 3. Misconstruing Cases 12

12. COMPAS Composite (default) Norm for Supervision Recommendations Minimum Supervision Recommendation 38.5% Medium Supervision Recommendation 24% High Supervision Recommendation 6.5% Medium (with override consideration to High) Supervision Recommendation 31%

13. Additional Factors requestedby clients • Women’s Gender Responsive inventory (NIC, Van Voorhis) • Stages of Change Instrument (Readiness for Treatment) • Drug inventories • LSI inventory – included within the COMPAS suite …….and others, as requested by clients Offense Specific Instruments Standard risk assessment are questionable, need to be bolstered by crime specific factors • Sex offense predictive tools • DUI specific Module • Domestic Violence Module (DVSI, Williams)

14. Standards: Validity/reliability of Risk/Needs Assessment Tools • Content Validity: Are the “right/relevant”factors selected? (Anything missing?) • Theory Guided Assessment (Farrington’s model - below) • Empirical support for selected factors (Meta-analysis, Validation Res) • Validation of risk models • Predictive validity • Concurrent validity • Construct validity • Theoretical/Substantive validity • Reliability, Consistency and clearmeaning of each scale • Unidimensionalstructure of each scale (Factor analysis) • Internal consistency reliability • Test-Retest reliability (stability of measurement) • Inter-rater reliability (across different interviewers)

15. Design Criteria for Correctional Risk/Needs Assessment Scientific Predictive Validity Theoretical Coherence Construct Validity Reliability of Measurement Content Validity Legal Rational Logical Validated Data Objective Data Consistency All Relevant Factors Practical Ease of Use (time) Ease of Training Consistency (w/ staff skills) Easy to Understand Provides Clear Explanations Fairness Provides Guidance (for decisions making, Interventions)

16. PREDICTIVE VALIDITY 1:NEW ARREST AT 24 MTHS (NY Probation) AUC = 0.76 (R = 0.44) Note: AUC’s are similar to other recent researchers on recidivism (Cottle, Lee, & Heilbrun, 2001; Gran, Belfrage, & Tengstrom 2000; Quinsey et al., 1998). 20

17. Michigan: Cumulative incidence of a felony conviction offense within the levels of the General Recidivism Risk Scale (first releases, n=17,913)

18. Can you trust a RNA system?What evidence should you demand?COMPAS and LSI both satisfy the following criteria Any RNA method should be evaluated for several quality, validity and reliability tests: 1. Peer Reviewed Publications: A key test of scientific quality • Brennan et al 2009 New York Probation (All races, both Genders) • Brennan et al 2012 (Women’s COMPAS) 2. Validation Studies by “independent scientists” - a severe test (3 studies of COMPAS, many for LSI). All had high AUC > 0.70) and high scale reliabilities • Lansing et al 2012 New York Parolees • Farabeeet al (2010) California (Compared LSI v. COMPAS on test-retest reliability) • Mann et al 2012 Florida Jail and Pretrial populations 3. Local (State) norming and validation study. A prudent test of local samples for validity and reliability • Michigan, NY, Cal, Wyoming all conducted multiple norming and validation studies 4. Large Samples, multi-year follow up predictive validation studies. Cross-validation on diverse samples.

19. Predictive validation: COMPAS Recidivism Risk Scale Over 1 year outcome 4000+ Prison Releases: Confidence Intervals for Low, Medium, High levels

20. COMPAS MATRIX LEVELS - DIFFERENCESCriminal history profiles of the risk matrix levels in the COMPAS normative data (n=7,381)

21. COMPAS Reliability: Alpha Coefficients for Men and Women in CDCR Sample

22. Part 3EVIDENCE BASED PRACTICE (EBP) + RISK-NEED-RESPONSIVITY PRINCIPLES • EBP – Evaluation research on effective programs • R-N-R principles – Findings from “What Works” Lit. • Risk Assessment is foundational for both • Judge Roger Warren “Evidence-Based Practices and State Sentencing Policy: Ten Policy Initiatives to Reduce Recidivism.”

23. What works: Clear sanctions/rewards Pro-social modeling Cognitive-Behavioral Tx Skills practice Emphasize intrinsic motivation Motivational Interviewing Family/Community support Recurring continuous Progs Relapse Prevention Progs Fidelity with program design is critical (many progs. poorly implemented) What doesn’t work: Punishment/sanctions Incarceration (research on criminogenic effects) Self Esteem programs Fear based programs (scared straight) Military “boot camps” Intensive supervision w/o treatment Talking cures/ Freudian …..and others EBP findings: What Programs work? Recent Findings - see cautions below Cautions 1. These lists change with new studies, often updated 2. These findings do not address specific responsivity matching

24. Main R-N-R principles 1. Risk Principle (Who to Target for Hi Intensity Tx) • Give Low/minimal intensity Tx to Low Risk Groups) • Targetstronger intensity Tx for medium andhigh risk Gps • Goal 1: Avoid Criminogenic effects on Lo risk Gps • Goal 2: Optimize efficiency of treatment resources 2. Need Principle (What factors to Target) • Target changeable criminogenicneeds (e.g. Big 5/8) • Avoid targeting non-criminogenic factors (e.g. self esteem) • What is a “criminogenic factor” (Dynamic + Changeable) 3. Responsivity(IndividualMatching of offenders to Tx) • Match treatment plans to offender’s uniqueneeds • Specific Responsivity (Design Individualized plans; Typologies) 4. Use validated RNA to support above principles • But allow Judicial Discretion, aided by RNA results • Incorporate EBP and RNA curriculum intoprofessional training programs for judges, probation officers, prosecutors, and defense bar.

25. SUMMARY: Potential Benefits of EBP/What Works for Sentencers • More accurate risk predictionof Higher and Lower Risk cases • Higher Risk Assessment accuracy than professional judgment • More comprehensive assessment of criminogenicneeds • Valid guidance/support for PSI construction • Guidance on crime reduction decisions • Valid identification of treatment goals • Guidance on whatprograms are “effective” (see EBP chart) • Guidance formatching offenders to appropriate Txdosage/intensity levels • Smarter Sentencing (if Judges accept, and understand the RNA model) • Information to “match” offenders with appropriate / relevant programs • Improved Outcomes • More effective rehabilitation, Less recidivism, Less crime. • Less incarceration, Less crowding, Lower costs

26. Part 4: IMPLEMENTATION ISSUES • Training challenges • Cultural Shift: • Collaboration vs. Adversarial competitive stance • Incorporate RNA into decision process • Incorporate EBP into decision process • Funding challenges • Legislative challenges

27. Risk Needs Assessment and EBP enters a “contested” environment • Risk Needs Assessment has mostly avoided the sentencing arena • Avoid the “Guilt/Innocence” decisions • Avoid being implicated in punishment/sanctioning decisions • Mainfocus on treatment, rehabilitation and crime reduction • Different stakeholders in Courts differ in goals/purposes and hold differing values re. crime/offenders (Conceptualize cases in diff. ways) • Scientific EBP and validated RNA will impose pressures on Judges, Defense, Prosecutors to follow it’s implications in their decisions: • All can gain in reaching well supported decisions • Substantial learning and training in how best to use RNA • High possibility of resistance, conflict, non-collaboration Judges- discretion; Defense - only to support client; Prosecutoronly to support their goals

28. Implementation Tasks 1. Building “Buy-In” (In “Contested” environment) • Stakeholders must “see” advantages of EBP and ACTUARIAL RISK ASSESSMENT • Stakeholder Collaboration vs. Antagonisms in (SADO issue w/ Arrests) • Will Judicial resistance be an issue? - “Judicial Discretion” (Offer as “support” only) • See Problem Solving Courts (for useful models of collaboration – Greene and Heilbrun 2011) 2. Building a Design – Does COMPAS need any design revisions to meet your requirements • Identify and fix any design problems (Pilot tests across States) • Information Structure (What do Users want, What do they need to know?) • MUST meet all “scientific criteria” (See “Standards” – Daubert case) • See: Faigman and Monahan (2009), “Standards for Legal Admissibility and their Implications for Psychological Science” 3. Building Competencies among Court StakeholderS • Training deficits of different stakeholders (See Problem-Solving Courts) • See Greene and Heilbrun (2011) “Psychology and the Legal System” Wadsworth. • Willingness/openness to learn AMONG JUDGES, DA’S, DEFENSE, PROBATION OFFICERS

29. (Continued)Implementation Tasks 4. Achieving Collaboration in a “Contested” setting • Will Prosecutors and Defense accept risk and need findings? • Or constantly fight over “admissibility” and “validity” issues • Problem solving courts may offer a viable collaborative model 5. Resist using Junk Science • Risk and needs assessment has attracted many who do not follow scientific criteria • May sabotage the field by marketing poor designs yet may achieve acceptance if users lack scientific training

30. Current Implementation status • Slow shift - Many judges prefer traditional “subjective discretion” • Don’t trust actuarial risk assessments • Some acknowledge a sentencing crisis (Warren, Marcus) • Collaboration problems - Values/Cultural clashes between stakeholders • Judges desire for “judicial discretion”may dislike constraints of using EBP/RN logic • Defense may attack risk/needs assessment if not supporting their positions, making client look good • Prosecutors may attack many low risk labels • Training problems for different stakeholders • Most Court Stakeholders not trained or knowledgeable in EBP/WW, Criminological theory, or RNR principles • Some PSI/PO writers lack expertise in EBP/WW • Case formulation analysis and writing is critical • Staff training in Case Formulation almost non-existent • Psychologists/social workers often untrained in crim theory

31. Part 5 CASE FORMULATION

32. FROM ASSESSMENT TO TREATMENT: EBP input at three main tasks of sentencers

33. What is achieved by case formulation? • The “dots are connected”. An interpretation reached. • The unique case is conceptualized, INTERPRETED & UNDERSTOOD • How is a case contextualized – Two strategies: • 1) What “kind” of case is this? Common case-types or offender typologies • 2) What “causal process(es)” is this? Common explanations of crime • The “new”becomes familiar – A Recurring pattern is recognized: This new case is transformed, contextualized, linked to your prior knowledge • Knowledge of prior well known recurring case TYPES • Knowledge ofwell known THEORIESof Crime Causation The “beginner-to-expert” literature reflects these two strategies • Decisions can be made & JUSTIFIED, WISELY &QUICKLY • Choice of specific intervention goals (What needs to Target) • Choice of rehabilitation strategies, programs and control levels (Use EBP). • Case formulation offers abasis to justify & explain sentence decisions • Rationally links causal structure to treatment so that outcome evaluation has more focus / precision (are social bonds stronger?).

34. 1. Contextualizing a case by recognizing what “kind” of case it is!Incorporating Criminological Research • 50 years of research of criminal types • Contemporary research on criminal “Patterns” • Contemporary software ability to classify cases into their appropriate “kind” • Male Patterns • Female Patterns CAVEAT: These diagnostic decisions are “Probabilistic” and thus are open to challenge

35. Typological structure based on current literature(Lykken 1995, Van Voorhis 1994, Mealey1995Moffitt 1993)

36. OFFENDER TYPOLOGY 7 RISK/NEEDS PATTERNS SITUATIONAL PSYCHOPATHIC ACCIDENTAL SOCIOPATHIC DRUGS EMOTIONAL PROBS FEW RISK FACTORS Weak Socialization ADDICTION Lykken 1995 ADUSTMENT PRBS Lykken 1995 HI CRIM HIST Hare's Psychopath SOC-EXCLUSION Van Voorhis 1994 Lykken 1995 Mealey 1995 MINOR CRIMES Alcohol/Dom V Fighting Type 7 Type 6 Type 5 Type 8 Type 1 Type 3 Type 2 Type 4 Highest drugs Social Exclusion Highest ciminality Young males Early drug onset Habitual crime Older DUI/Alcohol Lowest crimes Habitual crimes Socialization prbs Highest recidivism Few risk Factors High Crim Hist Mostly Minor/NV Few risk factors Lowest recidivism Mostly non-Viol Crim Attitudes Multiple Risk Factors Good soc-bonds Mostly non-Viol Poor/Transient Low recidivism risk Fewest Risk factors Poor / Transient Crim Peers/Family Crim. Personality Low CrimHist Younger/Not poor Jobless/Homeless Domestic Violence **Lie alerts Older/Isolated Crim Area Crim Opportunity Fighting Not isolated Adjustment prbs Good soc-bonds Soc Marginalized Young/Gangs Crim Attitudes...etc Hi Crim Peers Mental Health? Violent NATIONAL 14% 15% 15% 12% 12% 10% 11% 8% Wyoming 15% 12% 10% 12% 14% 7% 10% 13% Male Offender Patterns

37. Patterns of persistent women offenders

38. 2. Recognizing “causal process(es)” that drive anti-social behavior • What causal processes are exhibited by this case • Each causal process implies specific rehabilitative & crime reduction interventions

39. Criminological Theories: Connecting the dots on causes of Crime?

40. COMPONENT 1 CASE DESCRIPTION • COMPAS CHART (1 page) • Risks: • Violent recidivism risk • General recidivism risk • Absconding/FTA • Technical Failures/Revocations • Needs: Select as needed • What “kind” of offender (Pattern) • What “criminal history” (pattern) • Demographics • Case Narratives (Detailed) • Strengths • Total details in Narrative Format (Must be edited by data collector)

41. Problemsof Assessments • Irrelevant factors (unknown or minimal validity) • Incompleteassessment,or omission of key criminogenic factors • Disorganized or unsystematic presentationof information • Too much data - Cognitive overload – Difficult to digest – Stressful • Unreliable data (poor data collection, incompetent staff) • Confirmation bias among human assessors when compiling data for the case • Tend to notice ONLY what SUPPORTS pre-conceived notions • Producesconflict with otherswho look at the same case through different “lenses”

42. What Court Stakeholders want in a Case Descriptions? • Succinct & well organized (Big picture view, 1 page) • Time demands • Avoid information overload • *Comprehensive coverage of key factors (don’t omit key issues) – EBP • All Relevant info/factors (exclude irrelevant info. ) – EBP • Validity – trustworthy, well supported by background research • Standardized Consistent and reliable scoring • Guidance to understand/interpret a case • How to interpret the profile of risk/needs • Guidance for making decisions (Valid risk and Needs) • What Level of surveillance/control, placement? • What needs should be targeted? At what intensity? • Patterns of needs – What holistic/wraparound programs are needed * This is acommon deficiency

44. COMPONENT 2INTERPRETATIONandCASE FORMULATION SIMPLE APPROACH – AVOID INTERPRETATION - JUST LIST THE MOST SEVERE KEY FACTORS INTERPRETATIVE SYNTHESIS – UNDERSTAND BY CONNECTING DOTS, TWO SCIENTIFIC APPROACHES COMMON “PROCESS PATHWAYS” TO CRIME (CRIMINOLOGICAL THEORIES) TYPICAL PATTERNS OR “KINDS” OF OFFENDERS; WHAT “KIND” OF A CASE IS THIS

45. Case Interpretation IT’S ALL ABOUT PATTERNSCONNECTING THE DOTS HIGHLY RECURRING PATTERNS COMMON “KINDS” OF OFFENDERS HAVE YOU SEEN THIS “KIND” BEFORE!! Examples: 1. Older Poor, marginalized, substance abusing, transient, MH 2. Young, gang member, drug trafficking, weapons offender, homeboy, sub-cultural social learning (Lykken) 3. Middle class, employed, stable address, repeat DUI, sex offender

46. Why is Case Formulation so difficult? • Time pressures/enormous Caseloads on decision makers • Information overload - for any specific case • Case materials can accumulate, human limits on information processing, etc • Training gaps • Case formulation is rarely taught in criminal justice curricula • Few techniques for “connecting the dots” to teach case interpretation • Inadequate training in linking “ Assessment – Case Formulation -Treatment plan” • Clinical Professional judgment often idiosyncratic, personal • Differences in Background training and education, specialist training • Differences in personal experience (different “mental models”) • Personal stereotyping, implicit assumptions • Personal values, biases, beliefs, errors of commission, errors of ommission

47. Common Errors in Case Formulation 1. Human cognitive limitations lead to cognitive shortcuts Streamlining as a coping response Rubber stamping - Fast routinization of cases, No individualized justice…. “Assembly line justice” (minimal “thinking”) 2. Immobilized by Overload – Ignore the data – Premature closure Fall back on one’s personal intuitive assumptions – Often Wrong Construals Focus on 1 or 2 major risk factors for Rehabilitation Liberal Assumptions - Blame environment as cause Conservative Assumptions - Blame person as cause 3. Group Think and Stereotyping– If a strong “COURT CULTURE”exists, cases and “war stories” often discussed, people learn how to handle cases Emergence of shared cognitive stereotypes, shared assumptions 4. Confirmation bias (Seeing cases ONLY through one’s own mental “lenses”) People“find/notice” ONLY the evidence confirming preconceived notions COGNITIVE filtering to “fit” one’s pre-conceptions, of a “kind” of case Often based on own“discipline perspective”(e.g. Sociologists, psychologists, geneticists, etc) 5. Availability heuristic (illusory correlations, false classifications) Highly memorable but rare cases easily remembered and retrieved from memory We falsely assign cases to the memorable category on basis of insufficient factors – Case isassign to the “wrong kind”! These produce conflicts since people see the world differently

48. Summary: How COMPAS (EBP + RNA) support Case Conceptualization • Comprehensive: Emphasizes ALL known criminogenic factors • Validated: Offers repeatedly validated Risk Assessment – higher accuracy than human decision makers. • Prevents Data Overload – Identifies all key factors in 1 page rept. • Prevents Premature closure: Reduces human tendency to use Availability Heuristic &Premature closure • COMPAS forces full set of criminogenic factors to be considered • COMPAS forces several alternative explanatory theories to be considered; • COMPAS force several “categories/kinds” to be considered • Introduces criminological theory into Case Conceptualization 1. What causal process underlies the case (RNA – alerts you to major theoretical explanations of crime) 2. What “kind of case” is this? Allows identification of most COMMONLY occurring offender case types • Antidote to Groupthink • RNA offers classifications and theories that offer “Devil’s Advocate” options for more open discussion and challenges

49. COMPONENT 3 DECISION MAKING(Sentence Design) • Basic Goals • Errors/Failures and their causes • Error protection • How might RNA/COMPAS help?

50. Common errors in Decision making COMPONENT 1.Decision Errors resulting from OVERLOAD – Decisions are streamlined “routinized”to reduce cognitive load - “One size fits all / Cookie Cutter” treatment Plan • Rubber stamping (passively accept prior assessments or decisions ) • “Assembly Line” justice (see quotes) • Do what you did last time 2. Misconstruals/Misinterpretations leading to decision errors • Not enough time to process or deliberate about the data • Prioritize wrong factors / solve wrong problems, Fix wrong cause - Disciplinary or causal biases – Impose MY favorite theory or interpretation 3. False Positive Tendency Better safe/than sorry… leads to false +veerrors 4. Excessive use of overrides – especially for Risk Levels: - Ignore RNA & replace with personal judgment (errors) - Decision makers often overuse discretionary overrides (more errors) 5. Assign cases to inappropriate or ineffective programs Plethora of programs, many not evaluated for effectiveness 6. Can’t handle complexity of cases - many co-occurring factors, can’t connect dots, cant reach clear “interpretation”