Process Evaluation of the King County Family Treatment Court Preliminary Results

1. Process Evaluation of the King County Family Treatment CourtPreliminary Results Eric J. Bruns, Tracy Jones Justin D. Smith, Eric Trupin University of Washington Division of Public Behavioral Health and Justice Policy March 30, 2006

2. Components of the Evaluation • Data element and data collection assessment • Process evaluation • Interviews with team members • Comparison study of short-term outcomes for KCFTC participants vs. matched comparison group • Parent interviews • Record reviews (KCFTC, TARGET, and CAMIS) • Outcomes evaluation design • Cost-benefit analysis design

3. Process Evaluation • KCFTC PROCESSES AND FUNCTIONS • Comprehensive SB Assessment • High quality, appropriate CD Services • Timely/effective MH and other services • Effective care planning and management • Expanded and more frequent visitation • Consistent, timely incentives & sanctions • Random UA Screens • Effective pre-hearing case conferences • Effective judicial interaction • SHORT-TERM OUTCOMES • Eligibility/enrollment completed quickly • Enrollment in appropriate CD services • Parents compliant with/complete treatment • Parents ultimately able to remain sober • Parents/children more fully engaged in svcs • Parents/children receive needed services • Decreased placement disruptions • Parents compliant with court orders • Less negative effect on child well-being • Less disruption of child-parent bonds • Increased family reunification rates • Earlier determination. of alternate placement options • NATIONAL BEST PRACTICES • Communication bw Court and providers • Judge plays active role in Tx process • Judge responds to positive & noncompliant beh. • Mechanisms for shared decision making • Accountability for Tx services • Strategy for responding to noncompliance • MIS allow data to be assembled/reviewed • Enhancement of due process • Team members provided adeq resources

4. Method • Interviews with KCFTC staff and key informants (e.g., advisory group members) • N=35 initially identified; 5 deemed not appropriate • N=9 added through staff hires and identification by Evaluation Advisory Committee • Total N=39; 34 completed, 5 scheduled or pending • Average interview time = 63 minutes (range: 40-80 min)

5. Staff and key informant interview • Respondent information • General questions about KCFTC goals, target population, and overall success • Eligibility and referral process • KCFTC process and functions • Adherence to best practices (child welfare and treatment courts) • Short-term outcomes • KCFTC teamwork and collaboration • Ratings of individual team members’ effectiveness • Open-ended questions on strengths, weaknesses, and areas for improvement

6. Respondents

7. Respondents (total N=34) Team member interviews conducted 19 (56%) Key informant interviews conducted 15 (44%)

8. Survey Respondents Time Spent Directly with KCFTC Court or Families (n=33) 22 respondents 21 respondents

9. Results, part 1:Major Questions

10. Overall Success“How successful do you feel the KCFTC has been in accomplishing its goals overall?”

11. Overall Success (n=33)“How successful do you feel the KCFTC has been in accomplishing its goals overall?” Between Group Comparison: Front-line (>25% with KCFTC) versus Other (<25%) Team Members Time <25%: Mean 3.71 Std Dev. .772 Time >25%: Mean 3.19 Std. Dev. .834

12. Overall Success (compared to regular court)“Compared to regular dependency court process, how successful do you feel the KCFTC is in accomplishing its overall goals for participating families?” (n=33)

13. Overall Success (compared to regular court)“Compared to regular dependency court process, how successful do you feel the KCFTC is in accomplishing its overall goals for participating families?” (n=33) Between Group Comparison Time <25%: Mean 5.12 Std Dev. 1.317 Time >25%: Mean 4.73 Std. Dev. 1.335

14. Success Serving Target Population“How successful do you believe the KCFTC has been in serving its target population?” (n=33)

15. Success Serving Target Population“How successful do you believe the KCFTC has been in serving its target population?” (n=32) Between Group Comparison Time <25%: Mean 3.69 Std Dev. .793 Time >25%: Mean 3.19 Std. Dev. .981

16. Success Serving Representative Population“How would you describe the KCFTC’s success in serving a population that reflects the race, ethnicity, and gender of the general population of parents involved in the DCFS dependency system?” (n=30)

17. Success Serving Representative Population“How would you describe the KCFTC’s success in serving a population that reflects the race, ethnicity, and gender of the general population of parents involved in the DCFS dependency system?” (n=29) Between Group Comparison Time <25%: Mean 2.14 Std Dev. 1.027 Time >25%: Mean 3.07 Std. Dev. 1.280

18. Summary of Major Questions

19. KCFTC Capacity“In your opinion is the KCFTC currently serving too many families for its capacity, too few, or just about right?” (n=32)

20. Success of Referral and Eligibility Process“Overall, how successful do you think the KCFTC referral and eligibility process has been?” (n=30)

21. FTC Collaboration“A primary component of the KCFTC is “non-adversarial teamwork, in which team members know each other as individuals.” How successful have the KCFTC team members and agencies been in achieving this ideal?” (n=34)

22. FTC Shared Vision“Please rate the extent to which there is a shared vision among the different FTC team members. (n=34)

23. Results, part 2:KCFTC Processes and Functions

24. Summary of Process and Function Questions Comprehensive SB Assessment High quality, appropriate CD Services Timely/effective other services (parents) Timely/effective other services (children) Care planning and management Expanded and more freq visitation Consistent, timely incentives & sanctions Random UA Screens Effective pre-hearing case conferences Training and education for FTC staff Effective judicial interaction FTC staff collab with other agencies A little bit successful Somewhat Moderately Extremely

25. Results, part 3:Achievement of Best Practices

26. Summary of Best Practices Questions Court+providers maintain communication Judge plays active role in Tx process Judge responds to positive & neg behavior Mechanisms for shared decision making Accountability for Tx services AOD testing frequent in first months Strategy for responding to noncompliance MIS allow data to be assembled/reviewed Children protected from abuse and neglect Children kept safely in their homes possible Enhancement of due process Team members provided adeq resources About the same Little more successful Somewhat more Good deal more

27. Results, part 4:Proposed KCFTC Outcomes

28. Summary of Outcomes Questions Eligibility/enrollment completed quickly Enrollment in appropriate CD services Parents compliant with/complete Tx Ultimately able to be and remain sober Parents/childr more fully engaged in svcs Parents/ch receive services they need Decreased plcement disruptions Parents compliant with court orders Less negative effect on child well-being Less disruption of child-parent bonds Increased family reunification rates Earlier determ. of alternate plcmt options About the same Little more successful Somewhat more Good deal more

29. Results, part 5:Open ended questions

30. Gaps in Resources, Barriers, and other issues that need to be addressed.

31. Gaps continued…

32. Gaps continued…

33. General Strengths of the KCFTC

34. Strengths continued…

35. Greatest Challenges or Weaknesses of the KCFTC

36. Challenges continued…

37. Challenges continued…

38. Additional Results:Ratings of Individual Team Members’ Effectiveness Effectiveness at being a non-adversarial team member Overall effectiveness at achieving positive outcomes for enrolled families

39. JudgeNon-Adversarial Team Member Overall Effectiveness (n=19)

40. DSHS Social WorkerNon-Adversarial Team Member Overall Effectiveness (n=19)

41. Judge DSHS Social Workers Social Work Supervisor CASA CASA Supervisor/Mgr KCFTC Program Mgr KCFTC Specialist PCAP Representatives Treatment provider Tx provider supervisor Child’s attorney Parents’ attorney Wraparound facilitator Team members rated

42. SUMMARY OF RESULTS • 88% believe the KCFTC is at least “a little bit more successful” than the regular dependency court at achieving outcomes • Overall, 94% of respondents believe the KCFTC has been at least “somewhat successful” overall • Advisors and those with less overall contact with KCFTC participants gave higher ratings • Respondents less confident that goal of serving a representative population is being met • Respondents overwhelmingly believe current client load is about right for capacity • Mixed opinions on the amount of shared vision among team members

43. Relative strengths • Processes, functions, and best practices • Visitation • Random UA screens and AOD testing • Judicial interaction • Communication between court and providers • Outcomes (compared to regular dependency court) • Parents’ ability to remain sober • Reduction of negative effects on child • Parents’ compliance with court orders • Enrollment in appropriate CD services

44. Relative weaknesses • Processes, functions, and best practices • Strengths-based assessments • Timely and effective MH and other services for parents • Consistent and timely sanctions and incentives • Staff training and education • MIS • CD Treatment accountability • Outcomes (compared to regular dependency court) • Compliance with and completion of treatment • Full engagement in services • Earlier determination of alternative placement options

45. Areas that need improvement(From open-ended questions) • Resources and Services for clients • Housing • Visitations • Transportation • Day Care • Treatment services • Greater individualization and choice for services • Improved Communication and Collaboration • Team Building • Staffings • Increased Funding and Sustainability • Training Needs – Especially to ensure understanding of the model and to facilitate shared vision • More fully established policies and procedures • MIS and data

46. Next steps • Complete surveying and analysis • Complete analysis of Open-ended questions • Develop reports and presentations based on needs • Move ahead with Participant-level component of evaluation