210 likes | 373 Views
Researching Sex Offenders: A Workshop on Conceptualizing and Implementing Sex Offender Research Projects. R. Karl Hanson Public Safety Canada Presentation at the 13 th Annual Conference of the NYATSA, Saratoga Springs, NY, May 14, 2008. Big Questions.
E N D
Researching Sex Offenders: A Workshop on Conceptualizing and Implementing Sex Offender Research Projects R. Karl Hanson Public Safety Canada Presentation at the 13th Annual Conference of the NYATSA, Saratoga Springs, NY, May 14, 2008
Big Questions • Impact of criminal justice and social policy interventions (e.g., community notification, residency restrictions, civil commitment, risk assessment) • Assessment of reduced risk in high risk offenders • Active components in sexual offender treatment • Social policy for prevention on onset
Evaluation of Sex Offender Treatment Programs
Collaborative Outcome Data Committee • Formed in 1997 Goals: • Define standards for research on treatment outcome for sexual offenders (develop consensus) • Organize existing sexual offender outcome studies • Promote high quality evaluations
Anthony Beech Darren Bisshop Guy Bourgon Dawn Fisher R. Karl Hanson Andrew Harris Calvin Langton Roxanne Lieb Janice Marques Michael Miner William Murphy Michael Seto Vernon Quinsey David Thornton Pamela Yates CODC contributors
CODC Study Quality Guidelines • Structured rating scale • Definition of study quality • “…judgement of minimal bias can be made with high confidence.” • 20 items (plus 1 additional item rated for cross-institutional designs) • Items fall under 7 categories • Items assess either confidence OR bias (including direction of bias)
I) Administrative control of independent variables • Defining treatment (confidence) • Defining comparison group (confidence) • Miscellaneous incidental factors (bias)
II) Experimenter expectancies • Experimenter involvement (bias) • Blinding in data management (bias)
III) Sample size • Sample size of treatment (confidence) • Sample size of comparison (confidence) • Sample size of institutions (confidence) (for cross-institutional designs only)
IV) Attrition • Subject selection (bias) • Program attrition (bias) • Intent-to-treat (bias) • Attrition in follow-up (bias)
V) Equivalency of groups • A priori equivalency of groups (bias) • Adequacy of search of differences (confidence) • Findings on group differences (bias)
VI) Outcome variables • Length of follow-up (confidence) • Validity/reliability of recidivism information (confidence) • Equivalency of follow-up (bias)
VII) Correct comparison conducted • Data dredging (confidence) • Effectiveness of statistical controls (confidence) • Computation of least bias comparison (bias)
Global Rating • All items considered in making overall judgment of bias and confidence • Same three-point scales as individual items • Bias and confidence are considered separately
Global Rating Categories • Strong • High confidence AND negligible bias • Good • High confidence and some bias, OR • Some confidence and negligible bias • Weak • Some confidence and some bias • Reject • Little confidence, OR • Considerable bias
Reliability Study 1: Students • 2 senior undergrad students • Approximately one week training (8 practice studies) • 10 real studies rated independently
Results: Global Ratings • Overall 9/10 (ICC = 0.95) • Global confidence 10/10 (ICC = 1.00) • Global bias 9/10 (ICC = 0.69) • Direction of bias 7/10
Reliability Study 2: Experts • 12 Experts in sex offender research evaluation • No training on guidelines • 10 hypothetical studies ranging in quality • Rated 1-6 studies each (3 ratings per study)
Guy Bourgon Andrew Harris Grant Harris Niklas Langstrom Roxanne Lieb Ruth Mann Robert McGrath William Murphy Vernon Quinsey Marnie Rice David Thornton Pamela Yates Expert Raters
Expert Reliability • Some agreement on individual items • No agreement on global ratings
www.publicsafety.gc.ca • Collaborative Data Outcome Committee. (2007). Sex offender treatment outcome research: Guidelines for Evaluation (CODC Guidelines). Part 1: Introduction and overview. Corrections User Report No 2007-02. Ottawa: Public Safety Canada. • Collaborative Data Outcome Committee. (2007). The Collaborative Outcome Data Committee’s Guidelines for the evaluation of sexual offender treatment outcome research. Part 2: CODC Guidelines. Corrections User Report No 2007-03. Ottawa: Public Safety Canada.