The Generation and Use of Evidence in Child Welfare

1. The Generation and Use of Evidence in Child Welfare Aron Shlonsky Visiting Scholar, Haruv Institute Associate Professor Factor-Inwentash Chair in Child Welfare University of Toronto Factor-Inwentash Faculty of Social Work

2. Evidence-Informed Social Work Is a Concept Awaiting Implementation • Much discussed • Required in most jurisdictions • Not clearly Defined • Not yet implemented in pure form “--- the conscientious, explicit and judicious use of current best evidence in making decisions regarding the welfare of service-users and carers” (Sheldon, 2003, p. 1).

3. The Need for EBP in Social Work • Overwhelming and unreliable sources of information • Changing state of knowledge • Potential to cause harm • Opportunity costs • Social justice / informed choice • Systematic rubric

4. What is EIP?

5. EBP, EBM, EIP, EST’s, EBP’s, PG’s, and BP’s – huh? • Evidence-based Practice (EBP) is based on Evidence-based Medicine (EBM) Sackett et al (1997) at McMaster • Evidence-informed practice • Describes EBP and EBM • Empirically Supported Treatments(EST’s or EBP’s) – Use of interventions with evidence of their efficacy and/or effectiveness • Practice guidelines (PG’s) andBest Practices (BP’s) are lists of (hopefully) validated interventions

6. Clinical state and circumstances Clinical Expertise Client Preferences and actions Current Best Evidence The EBP Model: A Philosophical and Practical Approach Haynes, Devereaux, and Guyatt (2002)

7. Steps of EBP (based on Sackett et al., 2000) Become Motivated to Apply EBP • Convert information need (prevention, assessment, treatment, risk) into an answerable question. • Track down the best evidence to answer the question. • Critically appraise the evidence for its validity (closeness to the truth) impact (size of the effect) and applicability (usefulness in our practice). • Integrate critical appraisal with our practice experience, client’s strengths, values circumstances. • Evaluate effectiveness and efficiency in exercising steps 1-4 and seek ways to improve them next time. • Teach others to follow the same process Gibbs, 2003

9. Challenges to EBP • Philosophical straw men • Lack of evidence • Access to evidence • Complexity of social services • Lack of time • Translation across different contexts • Decision-making in an uncertain world • Cognitive dissonance • No manual for integrating evidence with context

10. Political Context Organizational Mandate Client State and and and Circumstances Socio / Historical Context Community Professional Context Professional Training / Supervision Expertise Expertise Client Client Client Research Research Research Preferences Preferences Preferences Evidence Evidence Evidence and Actions Organizational Resources / Constraints Economic Context Regehr, Stern & Shlonsky (2007)

11. D NO DETERMINISM Evidence in NOT Deterministic • Evidence-informed • Systematic process • Inclusive • Critical • Flexible

12. Tensions Within the North American Child Welfare System • Child welfare or child protection? • Child welfare worker - friend or foe? • System - should it be residual or preventive? • How do you balance safety, culture, community, family rights, and the individual needs of children? • How do you assess the risk the system poses to the child?

13. Decision Points in Child Welfare • Report / Screening • investigate • Investigation • Verify • close • Open for ongoing • Ongoing Services (in-home) • Close Case • Out-of-home care • Ongoing Services (out-of-home) • Reunification • LTFC • Guardianship • Adoption Risk / Prognostic Tools Diagnostic Tools Effective Treatment Monitoring Evaluation

14. Proper blend: Clinical Skills and Evidence • Requires a structure that teaches clinical skills and constantly creates and evaluates evidence. • Ongoing struggle • Responsibility to both find and evaluate evidence • Find ways to integrate • No set recipe

15. Types of Evidence in CW • Clinical Experience • Research literature • Administrative Data • Survey • Evaluation • Quantitative • Qualitative

16. Clinical state and circumstances Contextual assessment Contextual assessment Clinical Expertise Client Preferences and actions Current Best Evidence The Cycle of EBP Appropriate for this client? Actuarial risk assessment Other valid assessment measures? Client’s preferred course or at least willing to try? Effective services Barriers (e.g., cultural conflict) Adapated from Haynes, Devereaux, and Guyatt, 2002; as presented in Shlonsky and Wagner, 2005.

18. Campbell Collaboration Evolution • Systematic Reviews of Evidence • Importance of question • Importance of non-RCT’s • Econometric • Observational • Risk/diagnosis • Increasing number of reviews • Equity Group • User Abstracts • Translation

19. California Clearinghouse Scientific Ratings of 1 • Cognitive Therapy • Coping Cat • Coping Power • Coping with Depression for Adolescents • Interpersonal Psychotherapy (Adults & Children of Depressed Adults) • Mindfulness Based Cognitive Therapy (Adults) • Motivational Interviewing (MI) • Multidimensional Treatment Foster Care - Adolescents (MTFC-A) • Multisystemic Therapy?? • Nurse-Family Partnership (NFP) • Parent-Child Interaction Therapy (PCIT) • The Incredible Years • Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) • Triple P-Positive Parenting Program http://www.cebc4cw.org/scientific-rating/scale Slide courtesy of Richard P. Barth

20. Critical Thinking is Key Eight Critical Thinking Guidelines • Ask Questions: Be willing to wonder. • Define Your Terms: Key to Research • Examine the Evidence • Analyze Assumptions and Biases • Avoid Emotional Reasoning • Don’t Oversimplify • Consider Other Interpretations • Tolerate Uncertainty http://cla.calpoly.edu/~cslem/Invit/1/Chp1c.html

21. A Well Cultivated Critical Thinker • Raises vital questions and problems; • Formulates them clearly and
precisely; • Gathers and assesses relevant information • Uses abstract ideas to
interpret information; • Comes to well-reasoned conclusions and solutions; • Tests them against relevant criteria and standards; • Thinks openmindedly; • Recognizes and assesses assumptions, implications, and practical consequences; • Communicates effectively Elder, 2007

23. Knowledge needs to be managed much more effectively than it has been in the past The challenge is to rethink our view of what knowledge is & how best to facilitate its rapid generation, sharing, and application in a manner that closes the policy-research-practice gap

24. Discussion