Risk Assessment, Domestic Violence, and Differential Response in Child Protection

1. Risk Assessment, Domestic Violence, and Differential Response in Child Protection ACWA Australia 2010 Aron Shlonsky Associate Professor Factor-Inwentash Chair in Child Welfare University of Toronto Factor-Inwentash Faculty of Social Work

2. Rights, Safety, and Evidence • International Rights of the Child • Ethical obligation to Provide Services • Effective • Available • Desired outcomes • Larger Society • Individual Culture • Individual Preference • Transparent • Responsive • Continuous evaluation REQUIREMENT: CRITICAL DIALOGUE BETWEEN PARTIES WITH COMPETING / CONFLICTING INTERESTS

3. Context of Decisions and Their Biases • Individual level • Agency level • Policy level • Systemic level

4. The Five Major Problems • Over-inclusion (specificity) • Under-inclusion (sensitivity) • Capacity • Service Delivery • Service Orientation Waldfogel, 1998

5. Differential ResponseDefinition • Customized • Community-based • Informal and natural helpers • Low or fairly low risk • Less adversarial Waldfogel, 2008

6. Evidence • Walter R. McDonald (2003) • US national study • Comparability issues (range in services and screening procedures) • Greater discretion • Same ole, same ole? • Hollinshead et al. (2005) • More services • Same recurrence rate • Institute of Applied Research (Loman & Siegel) • RCT • Decreased maltreatment recurrence • Decreased days in foster care • Increased services • Increased satisfaction • Overall cost savings

7. Challenges • Philosophy or clearly defined practice??? • Hinges on potentially unreliable clinical decision • Lack of effective services

8. How do you know which way to go?

9. Problems in Clinical Decision Making • Large number of “risk factors” • Severe time constraints • Increased information does not improve prediction (Dawes, 1989) • Subjective first impression • Selected risk factors may not be predictive

10. Safety assessment Safety assessments are intended to accurately identify children who have recently been or are currently being maltreated, or are at risk of imminent harm; and, to determine the nature and type of harm, its severity, and its potential consequences for the child. By definition, these children are already at elevated levels of risk to their health, safety, and well-being Rycus & Hughes, 2003

11. Risk Assessment “The classic theory of risk assessment is a venerable concept. Regardless of field of application, it always analyzes two factors when attempting to determine potential risk: 1) what is the likelihood that a harmful event will occur, and, 2) if it occurs, what is the potential severity of that harm. Any analysis that asks these two questions with respect to a factor or combination of factors can be called a risk assessment” Rycus & Hughes, 2003

12. Three Types of Risk Assessment in Child Protective Services • Clinical Judgment • Consensus Risk Assessment • Actuarial Risk Assessment

13. “ . . . the question of whether a statistical or clinical approach is superior has been the subject of extensive empirical investigation . . . The results have been uniform. Even fairly simple statistical methods outperform clinical judgment . . . . . . objections [to using statistical models] ignore the data from well over 100 studies . . . The objections to using statistics also ignore the ethical mandate that, for important social purposes such as protecting children, decisions should be made in the best way possible. If relevant statistical information exists, use it. If it doesn’t exist, collect it.” Robyn Dawes, Carnegie Mellon University The Chronicle of Higher Education, June 1993

14. Two types of risk assessment estimates Prediction:A specific behavioral outcome is declared in advance, e.g., the family will re-abuse their child.Classification:Families are placed in risk classification groups based on their estimated probability of future maltreatment. Human behavior is extremely difficult to predict. Classification recognizes the limitations of risk assessment estimates. Used appropriately, it can help improve worker decisions and help agencies develop more effective case management policies.

15. Decision Making Goals Reduce subsequent harm to children served by child welfare agencies • Reduce subsequent abuse/neglect substantiations, child injuries, and foster care placements among families agencies serve. • Reduce time to permanency for children in foster care

16. Actuarial v.Consensus-based:New Substantiation Rate Baird & Wagner, 2000

17. California Family Risk Classification by Follow-Up Investigation N = 10.097 18 month post-investigation follow-up (NCCD-CRC)

18. California Family Risk Classification by Follow-Up Substantiation N = 10.097 18 month post-investigation follow-up (NCCD-CRC)

19. California Family Risk Classification by Child Placement N = 10.097 18 month post-investigation follow-up (NCCD-CRC)

20. What risk assessment does not do • Risk assessment does not assist in case specific clinical decisions • Risk assessment does not inform the worker about anything more than the likelihood of future maltreatment

21. The role of clinical judgment in risk assessment • Clinical judgment not suspended (Meehl, 1954) • Find the optimal combination of actuarial method and clinical judgment (Holt, 1958). “A risk estimate is like a weather forecast. The forecast may indicate a 60% or higher probability of rain the next day. That forecast may prove to be wrong but since it rained 60% of the time on similar days in the past, it may be a good idea to carry an umbrella” (Gottfredson and Tonry, 1987).

22. Is SDM Transportable Across Contexts? • SDM implemented in Ontario April 2007 • 6-month cohort January 2008-June 2008 • Five children’s aid societies • 7,239 investigations • Followed for 18 months • New investigation • New substantiated investigation

23. Ontario Risk Assessment StudyInterim Results • SDM implemented in April 2007 • 6-month cohort January 2008-June 2008 • Five children’s aid societies • 7,239 investigations • Followed for 18 months • New investigation • New substantiated investigation

24. 18 Month Recurrence Performance

25. Implementation Issues • Theme 1: (Mis)understanding of the Risk Assessment • The Ontario Risk Assessment Model (ORAM) as a frame of reference • Static versus dynamic tool • Theme 2: Training for the Use of Risk Assessment • What do we mean by risk? • Theme 3: Problematic Elements of the Tool • Theme 4: Inconsistencies across the Board • Theme 5: How to Deal with Domestic Violence

26. Implementation Issues • Theme 6: How the Tool is Used • Being uncomfortable with uncertainty • thinking what if? • Usage of the tool • The use of override decisions • How the tool is perceived • Theme 7: Integration of the SDM tools in Court • Theme 8: Clinical Autonomy • Theme 9: Guidelines for Consistency

27. Implementation Issues • Theme 10: Differential Response and How Workers were Prepared • Training and preparing community partners • Theme 12: Differential Response: From Philosophy to Implementation • Funding formula

28. Implications • Transportable, but… • Needs calibration • Needs good implementation • Needs ongoing supervision and training • Link risk level to services • Link services to evidence • Be sure not to lose clinical skills in the process

29. SDM strength and needs assessment instruments • Consensus assessment instrument to evaluate child and family functioning. • Assists in: Case plan development Choice of service interventions Screening for specialized assessment Monitoring service interventions Highlight major case issues • Provides data for improving service delivery • Clinical judgments still required.

30. Discussion

31. Double Jeopardy: DV and Child Maltreatment • Quantitative data collection from a sample of investigated cases (N=936) across five participating agencies • Quantitative data analysis using OCANDS (N=4,776) • Qualitative - Content analysis (N=75 randomly selected closed cases) • Qualitative - Interviews (N=12) and focus group (n=8) of women with closed CPS cases involving DV

32. Ontario DRDV Study - Results • DV involved cases were more often referred by police DV families tended to be younger • More non-white families are investigated for DV than other forms of maltreatment • 71 percent of DV cases are substantiated in the child welfare system (vs 39.6% for all other forms of child maltreatment) • 34 percent of DV involved cases remain open after investigation (vs 23.5% for all other forms of child maltreatment)

33. Ontario DRDV Study - Results • More DV cases were referred to community based services mostly to victims services and parent counseling (vs 47.6% for all other forms of CM) • One-third (34.7 percent) of children are assessed to have been emotionally harmed (vs 23.1% of all other form of child maltreatment • The overall rate of recurrence is about 28 percent at 18 months, with DV cases recurring about 26 percent of the time and non-DV cases recurring at about 31 percent

34. Ontario DRDV Study - Results • Tendency is towards more intensive child welfare involvement in DV cases in index (rst opening) investigations of DV • Longer involvement in cases where there is a concern that the victim parent (most often the mother) will stay with or return to the abusive partner • Initial child welfare interventions are more intensive to support the mother • Very little focus on perpetrating fathers and very little information on fathers in the workers’ narratives beyond the assault event

35. Ontario DRDV Study - Results • CPS involvement is aimed at child and maternal safety and the securing of services • Referrals beyond the child welfare system focused on services for the victim parent (i.e., victims services, parent counselling) • Subsequent CPS involvement is short-term and CPS involvement diminishes over time • CPS involvement is shorter and less intense when the client chooses to receive services • immediately, especially when this involves a shelter stay, CPS involvement is shorter and less intense when caseworkers are assured the mother is not going to return to the perpetrator

36. Select DV Risk Assessment Instruments • Danger Assessment - validated for fatalities • Spousal Assault Risk Assessment Guide (SARA) - Consensus tool for DV recurrence • Ontario Domestic Assault Risk Assessment (ODARA) - Actuarial tool for DV recurrence • Kingston Screening Instrument (K-SID) - Actuarial screener for DV

37. Substantiated Allegation Assigned for Investigation -Assess Family Risk Received Investigation Completed -Assess Family Strengths and Needs Unsubstantiated -Assign Service Priority Response Safety -Develop Case Plan Priority Assessment Unfounded Risk Assessment Timing:CM

38. Case Closed DV DV Assessment Assessment -Unfounded/ U nsubstantiated -No service Investigation Child Protection CPS Substantiation Case Open Ongoing Safety Decision Services Assessment -CPS -Substantiated Actuarial Risk -Contextual -Unsubstantiated Assessment Family Assessment -Service Intensity Decision -Case planning Risk Assessment Timing:CM and DV

39. Proposed Decision Aid:CM and DV

40. Controversial Discussions • What use is a prediction if it can’t tell me what to do? • Given the tension, what are some solutions? • Are there commonalities between them? • Are philosophical differences too big?

41. Discussion Tension between child protection and domestic violence services Audience Participation

42. PERPetual Problems: Batterer Intervention Programs Evidence from Campbell Collaboration systematic reviews • Smedslund et al (2007). Cognitive behavioural therapy for men who physically abuse their female partner • Little or no evidence of effectiveness • Need more research • Feder and Wilson (2008). Court-mandated interventions for individuals convicted of domestic violence • Modest benefit for officially reported outcomes (i.e., police reports) • No effect for victim reported outcomes • Quasi-experimental studies using a no-treatment comparison had inconsistent findings indicating an overall small harmful effect • Quasi-experimental that did not control for dropout had a large positive effect • Davis et al (2008). Effects of second responder programs on repeat incidents of family abuse • Slightly increased likelihood of reporting subsequent incident • No difference in rate of subsequent incident

43. Ramsay et al (2009) Advocacy interventions to reduce or eliminate violence and promote the physical and psychosocial well-being of women who experience intimate partner abuse.Campbell Collaboration Systematic Review Methods and Results • Ten (10) RCT’s involving 1,527 women • provision of legal, housing and financial advice • Facilitating access to and use of community resources such as refuges or shelters, emergency housing, and psychological interventions • Provision of safety planning advice • Ongoing support and informal counselling.

44. Ramsay et al (2009) Advocacy interventions to reduce or eliminate violence and promote the physical and psychosocial well-being of women who experience intimate partner abuse.Campbell Collaboration Systematic Review Results • Possible decrease in physical abuse more than one to two years after the intervention for women already in refuges • Inconsistent evidence for a positive impact on emotional abuse. • Equivocal evidence for the positive effects of intensive advocacy on depression, quality of life and psychological distress • Evidence that brief advocacy increases the use of safety behaviours by abused women

45. PERPetual Problems: Is Engagement the Answer? • Lack of evidence • Mutual violence • Low frustration tolerance • Poor affect regulation • Can philosophy drive services successfully? • How can we empower men? • Caring Dads?

46. What is the optimal combo of RA and DR? • Reliable, valid instruments • Decision aid for: • DR track • Range of services • Permanency planning • BEST if used PRIOR to decisions

47. What you don’t want to see

49. Evidence-Based Social Work Is A Concept Awaiting Implementation • Much discussed • Required in most jurisdictions • Not clearly Defined • Not yet implemented in pure form “--- the conscientious, explicit and judicious use of current best evidence in making decisions regarding the welfare of service-users and carers” (Sheldon, 2003, p. 1).

50. EBP, EBM, EIP, EST’s, EBP’s, PG’s, and BP’s – huh? • Evidence-based Practice (EBP) is based on Evidence-based Medicine (EBM) Sackett et al (1997) at McMaster • Evidence-informed practice • Describes EBP and EBM • Empirically Supported Treatments(EST’s or EBP’s) – Use of interventions with evidence of their efficacy and/or effectiveness • Practice guidelines (PG’s) andBest Practices (BP’s) are lists of (hopefully) validated interventions