1 / 21

Evaluation Design Challenges on the East Coast: A Tale of Two Drug Courts

Evaluation Design Challenges on the East Coast: A Tale of Two Drug Courts. American Evaluation Association November 3, 2006 Kimberly Pukstas, PhD. Research Team. Jodi Brekhus, MS Shannon Carey, PhD Dave Crumpton, MPA Michael Finigan, PhD Bob Linhares, MA Juliette Mackin, PhD

cale
Download Presentation

Evaluation Design Challenges on the East Coast: A Tale of Two Drug Courts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation Design Challenges on the East Coast: A Tale of Two Drug Courts American Evaluation Association November 3, 2006 Kimberly Pukstas, PhD

  2. Research Team • Jodi Brekhus, MS • Shannon Carey, PhD • Dave Crumpton, MPA • Michael Finigan, PhD • Bob Linhares, MA • Juliette Mackin, PhD • Kimberly Pukstas, PhD • Judy Weller, BS

  3. Research Aims Two juvenile drug courts in Maryland were selected: • Process Evaluation - What are the policies and procedures? • Outcome Evaluation – What were the results? • Cost Evaluation – What were the costs?

  4. Drug Court #1 Inner City Setting Began 1997 Capacity: 200 Marijuana 95% Male 99% African American Drug Court #2 Suburban Setting Began 2000 Capacity: 30 Marijuana 75% Male 88% Caucasian Study Sites

  5. Early Concerns • Drug Court # 1 – lack of variability among participants • Drug Court # 2 – small sample sizes • Drug Court # 2 – younger drug court program may still be in flux

  6. Drug Court #1 11key stakeholders interviewed 1 focus group held with parents (n=3) 2 focus groups held with juveniles (n=10) Drug Court #2 10 key stakeholders interviewed 1 focus group held with parents (n=5) 1 focus group held with juveniles (n=12) 1 individual interview w/ discharged youth Process Evaluation

  7. Process Evaluation Challenges • Overall challenges were similar for both drug courts • Program staff busy, distracted • Difficulties recruiting discharged youth • Difficulties recruiting parents (court #1) • Participants in different stages attending same focus groups • Location – need privacy, comfort

  8. Process Evaluation Solutions • Increase timeline for process evaluation • Improve communication – point person • Allow for phone interviews when focus groups are not possible • Create more meaningful incentives for participants • Delegate more responsibilities to regional employees

  9. Drug Court #1 Program had undergone recent challenges Frustration apparent but also a strong commitment to the program Less adherence to the 10 key components Drug Court #2 Staff had anxiety about the evaluation Strong staff commitment to the program observed Greater adherence to the 10 key components Process Evaluation Results

  10. Drug Court #1 Lotus (?) Program Hard Copy Reports Untrained Staff 6 variables consistently entered Drug Court #2 Access Program Password Protected CD Highly Trained Staff Over 125 variables consistently entered Outcome Evaluation

  11. Drug Court #1 Sex, Race Name,DOB ??? ??? ??? ??? Start Date, Status Drug Court #2 Demographics Personal Identifiers Baseline Health/MH School/Employment AOD History Family History Program Participation Outcome Evaluation

  12. Outcome Evaluation Solutions Drug Court #1 • Arrange Site Visit • Obtain Paper Case Files • Create Electronic Database for Research Purposes • Transfer Data from Paper Files to Electronic database

  13. Results of Paper File Search

  14. New Concerns Drug Court #1 • No Central MIS System • Lots of Missing Records • Key Variables Missing • Several Different Forms in Use • Inconsistent Data Entry • Program is in Period of Upheaval • Program Functioning Far Below Capacity

  15. Evaluation Solutions Drug Court #1 • Prioritize Process Evaluation • Recommend TA for program staff • Develop a corrective action plan • Postpone Outcome & Cost Evaluation

  16. Outcome Evaluation Drug Court #2 – Results Searched: • Drug Court Database • Juvenile Justice Database • Adult Criminal Justice Database • Treatment Database • Local Detention Center Data

  17. Outcome Evaluation: Results * p<.05

  18. Cost Evaluation Drug Court #2 • More than 25 Interviews Conducted • 21 Transactions Identified • Transactions Include: Drug court appearances, Case mgt, drug testing individual & group sessions, mh evaluations, educational counseling

  19. Cost Results Drug Court #2 • Ave program cost per drug court participant is $11,689($41 per person per day) • Ave cost of criminal justice system outcomes in the year following program involvement was 60% less than the cost of the comparison group ($3,409 vs. $8,481)

  20. Lessons Learned • Expect the unexpected!!! • Capacity does not equal sample size • Be prepared to alter your evaluation strategy • Missing data are still an interesting research finding • Make the most of the data that are available

  21. Questions? • Email Dr. Kimberly Pukstas: pukstas@npcresearch.com • Final Reports are available: www.npcresearch.com

More Related