CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis

1. CPS Recidivism Associated with a Home Visiting Program: A Quasi Experimental Analysis Ed Byrnes, Ph.D. Eastern Washington University Michael Lawson, M.S. University of California at Davis

2. Evaluation Questions • What is the CPS recidivism rate for Birth & Beyond families in comparison to similar families who were involved with CPS? • How does the time to onset of CPS recidivism differ between Birth & Beyond families and similar families who were involved with CPS?

3. Evaluation Design • Quasi Experimental • External Comparison Group • Fixed Follow Up Periods • Data Sources • B & B Database • Sacramento County CPS Database • Between Groups Comparability • Multiple Points of Comparison • Outcomes • CPS Recidivism and Onset of Recidivism • Client Predictors of Outcomes • Demographic Characteristics • Assessment Results

4. Birth and Beyond Group • 384 B & B Families • All Had A Prior CPS Referral • All Had At Least One B & B Home Visit • All Had A First Home Visit On Or Before July 11, 2006 For One Year Follow Up

5. CPS Comparison Group • 327 Families • NOT B & B Clients • All Had A First CPS Referral Between January 1, 1999 and July 11, 2006 • Sample Trimming • Children’s Ages • Most Severe CPS Allegation • Children’s Language and Race • Stratified Random Sampling • Most Severe CPS Allegation • Children’s Gender, Language and Race

6. Between Groups Comparability • No Significant Differences in • Children’s Race • Children’s Primary Language • Children’s Gender • Very Unlikely to Influence Results • Children’s Ages Differed Significantly • CPS Comparison Group Had More Neonates • CPS Comparison Group Was Then Older • Children’s Age and Group Membership Correlated at .25, a Weak Relationship • Significance An Artifact of Sample Size of 711 Families • If Children’s Ages Have Any Influence on Results It Is Quite Small

7. Between Groups Comparability

8. Between Groups Comparability

9. Between Groups Comparability

10. Between Groups Comparability

11. CPS Recidivism • Data From CPS Database • All Referrals With A Definitive Finding • Substantiated, Inconclusive and Unfounded • Reflect Contact With The CPS System • One Year Follow Up Period • From First Home Visit for B & B Group • From First CPS Referral for Comparison Group • Contrast Voluntary Additional Services with Usual CPS Processing

12. CPS Recidivism • Of the 384 B & B families, 140 (36%) had a new CPS referral during the one year follow up period • Of the 327 CPS Comparison Group families 159 (49%) had a new CPS referral during the one year follow up period • This difference was statistically significant • B & B participation versus regular CPS services was weakly correlated with CPS Recidivism (Phi = .12) • Participating in B & B home visiting influenced CPS recidivism in the desired direction

13. CPS Recidivism

14. CPS Recidivism Onset • Data From CPS Database • Same Criteria As CPS Recidivism • One Year Follow Up Period • Survival Analysis • Time to First CPS Referral • From First Home Visit for B & B Group • From First CPS Referral for Comparison Group

15. CPS Recidivism Onset • Average time elapsed from the beginning of services to their first CPS referral during the one year follow up period • For B & B families was 279 days (sd = 6.5) • For CPS Comparison Group families was 256 days (sd = 7.45) • This difference was statistically significant • Relationship between group membership and time to the onset of CPS recidivism was moderate-to-strong • Magnitudes of the test statistics • Participating in B & B home visiting influenced the time to the onset of CPS recidivism in the desired direction

16. CPS Recidivism Onset

17. CPS Recidivism Onset • B & B and CPS Comparison Groups have similar patterns of cumulative CPS recidivism rates through the first 150 days • After 150 days the cumulative CPS recidivism curves diverge • The relationship between participating in B & B home visiting and a delayed onset of CPS recidivism is more beneficial as time progresses

18. Predicting CPS Outcomes • Additional Evaluation Questions • What demographic, service and assessment characteristics of cases are strongly related to having a new CPS referral within 150 days of the first home visit? • What demographic, service and assessment characteristics of cases are strongly related to having a new CPS referral within one year of the first home visit? • Members of the CPS comparison group never engaged in B & B services they could not be included in these analyses

19. Predicting CPS Outcomes • Logistic Regression Primer • Predict the occurrence of an event • Risk and Protection • Can be quantified using Logistic Regression • Odds Ratios • Value of 1 means no relationship between a factor and an event • Value greater than 1 means a risk factor for an event • Value less than 1 means a protective factor against an event

20. Predicting CPS Outcomes • Outcome Variables • New CPS Referral Within 150 Days • New CPS Referral Within 1 Year • Same Definitions As Recidivism and Onset • Predictor Variables • Demographics • Services • Assessment

21. Predicting CPS Outcomes • Model Building Process • Two Stages • Model by Variable Category for Each Outcome • Final Model for Each Outcome • Criteria For Final Model Inclusion • Statistical Significance • Odds Ratio Greater Than 2 • Evaluating Models • Goodness of Fit, Improved Case Classification, Outliers In The Solution v. Over-identification • Sample Size • Explained Variance: How Much Variability In The Outcome Can Be Attributed To Variables In The Model – 20% or higher is acceptable

22. Predicting CPS Outcomes • Predicting 150 Day CPS Recidivism • No Service Variables Met Final Model Criteria • Predictors At Intake • Being a first time parent • Being pregnant • Being under the age of 25 • Being a member of the Black race • Being a member of the White race • Being a member of the Hispanic race • Meeting the criteria for clinical depression • Having a high total score on the APPI (indicating more skillful parenting) • Having a low score on the MSSI (indicating a low level of social support)

23. Predicting CPS Outcomes • Predicting 150 Day CPS Recidivism • 85 Cases With Complete Data • 36% Explained Variance • Met Other Model Quality Criteria • Risk Factors • Pregnant at the Time of Intake • Being Black • Having Low Levels of Social Support • Protective Factor • Being Under Age 25

24. Predicting CPS Outcomes

25. Predicting CPS Outcomes • Predicting 1 Year CPS Recidivism • No Service Variables Met Final Model Criteria • Predictors At Intake • Being fluent in English • Being pregnant • Having a low total score on the APPI (indicating less skillful parenting) • Experiencing severe domestic violence • Predictors From Follow Up Assessment • Having a high score on the CAGE questionnaire (indicating a higher likelihood of alcohol dependence) • Experiencing severe domestic violence

26. Predicting CPS Outcomes • Predicting 1 Year CPS Recidivism • 93 Cases With Complete Data • 25% Explained Variance • Met Other Model Quality Criteria • Risk Factors • Pregnant at the Time of Intake • Having a Higher Final CAGE Questionnaire Score • Having Low Parenting Skills at Intake • No Protective Factors

27. Predicting CPS Outcomes

28. Dynamic Risk • Assessment Results Are Not Very Stable Over Time • Pretest – Posttest Correlations Are Weak To Modest • Risk and Protective Factors That B & B Clients Experience Are Dynamic Rather Than Static • B & B Home Visitors Must View Assessment As An Ongoing Process

29. Dynamic Risk

30. CPS Recidivism Results • B & B participation was associated with lower aggregate CPS recidivism rates • B & B participation was associated with a delayed onset of CPS recidivism • Given the broad ecologies of child maltreatment, these results can be viewed as promising

31. Discussion Questions • What challenges have you encountered in conducting quasi experimental evaluation research, and how have you addressed these? • Since California has a county administered child welfare system what efforts can we make to encourage uniformity of data for greater generalizability of results?

32. Discussion Questions • What are the trade offs between using matched (case control) designs versus designs that rely on inclusion and exclusion algorithms for comparison groups? • Since California and it's counties, like so many other states and locales, are facing deep budget shortfalls, how can we continue to promote evidence based practice as a priority?

33. Dissemination Plan • Incorporate feedback about our study from this symposium into a final manuscript. • Submit the manuscript for publication. • Peer reviewed journals being considered include: • Children and Youth Services Review; • Research on Social Work Practice.

34. Follow Up Contact Ed Byrnes, Ph.D. Email: ebyrnes@mail.ewu.edu (509) 455-3422 Michael Lawson, M.S. Email: malawson@ucdavis.edu