Foster Care Reunification: The use of hierarchical modeling to account for sibling and county correlation

1. Foster Care Reunification:The use of hierarchical modeling to account for sibling and county correlation Emily Putnam-Hornstein, MSW Center for Social Services Research, UC Berkeley Terry V. Shaw, PhD, MSW, MPH Ruth H. Young Center for Children and Families University of Maryland

2. Hierarchical Child Welfare Data?

3. People or events hierarchically structured within the same higher level unit tend to be systematically more similar than those drawn from another unit Further homogeneity occurs over time due to shared experience within clustering unit Nested Structure:

4. Research Questions: Modeling Reunification as an outcome: 1) How much variability is there within a given group of siblings? Can multilevel modeling be used to adjust for the non-independence of siblings within families? 2)Is multilevel modeling necessary to adjust for unobservable and (potentially) systematic differences across California’s 58 counties?

5. The “sibling” question: • 2007: Two-thirds of children in California’s foster care system had at least one other sibling in care • 2001 cohort of first entries, 52% entered with or followed a sibling into care • Can hierarchical modeling be used?

6. The “county” question: • California relies on a county administered child welfare system • Wide range of performance across counties • Much debate…but to what extent does performance actually vary across counties?

7. County Variation… 85 % 37 %

8. Attempts to address non-independence in child welfare data: Within Family (sibling) Correlation: • Ignore it • Bias induced by the unadjusted inclusion of non-independent units (siblings) • Random selection of a sibling • Is being a member of a sibling group of a X-size random? • Always want to avoid throwing away data! • General Estimating Equation • Treat dependence as a “nuisance” rather than modeling it Within County (family) Correlation: • Ignore it • Are families randomly distributed across counties? • Are child welfare reunification decisions random within a county?

9. Hierarchical Modeling: • A generalization of linear and generalized linear modeling in which regression coefficients are themselves given a model (Gelman, 2006) • Estimates & associated inferences are purged of shared variance defined by their group/cluster membership

10. The Basic Multilevel Model: If we fail to cluster? Correlated residuals. Fixed Part Random Part

11. Data: • Data were drawn from a longitudinal extract of the California Child Welfare Services/Case Management System (CWS/CMS) • Cohort of children first entering foster care in the 2001 calendar year • Child/Family Data were merged with a file of county level data compiled from public administrative reports

12. Dataset:

13. Dependent Variable: yijk=reunified within 24m (0/1) Level 1 Covariates: Age Dummies (5) Race Dummies (6) Removal Dummies (4) Gender (0/1) Kinship Care (0/1) Level 2 Covariates: Drug/Alcohol (0/1) Sibling Count Dummies (4) Level 3 Covariates: Entry Rate % Population Black % Births to Teen Moms Variables:

14. Level 1: Descriptive Stats

15. Levels2 & 3: Descriptive Stats

16. Model Specification:

17. Three Stage Formulation:(The three-stage formulation specifies the same model and distributional assumptions for the random intercepts.) Level One Model (child level): Level Two Model (family level): Level Three Model (county level):

18. Analysis • STATA • GLLAMM (generalized linear latent and mixed models) • Adaptive Quadrature: performs well for dichotomous responses with a wide range of cluster sizes and ICC (Rabe-Hesketh, Skrondal, & Pickles, 2005) • Modeling: iterative process of using ordinary quadrature with a small number of integration points, passing those values forward, increasing the number of points, and so forth until our model had converged and our estimates stabilized.

19. Intraclass Correlations (ICC)(y* assumes latent-response model specification) ICC derived for children i and i’ from the same family j (and same county k): ICC derived for children i and i’ from different families j and j’ and different counties k and k’

20. Model 1– Unconditional Model 2– Level 1 Covariates (child) Model 3– Level 2 Covariates (family) Model 4– Level 3 Covariates (county) Model Building:

21. Results: Intraclass Correlations

22. Odds of Reunification (sample data): Marginal v. Conditional Interpretations

23. Results for Research Question 1: • Within Family (Intraclass) Correlation = .90 - .96 • Very difficult to model… • Findings largely support the assumptions behind randomly selecting one sibling from a family when the question is reunification (i.e., the reunification outcome is the same for all children in a given family – either all kids go home or none do) • Deterministic? Misspecification to assume random family influence?

24. Results for Research Question 2: • Within County (Intraclass) Correlation = .00 - .04 • Introduction of covariates at all three levels reduced the within county variation to null • Evidence that adjusting for demographic differences matters…but it is not unreasonable to assume that the odds of reunification within a given county (in CA) are not confounded by unobservable county-level factors • Aggregation bias / Ecological Fallacy?

25. Next steps… • What is a meaningful way to define a sibling relationship? • Reunification may be the most consistent practice across California’s child welfare system (it is the “default” plan)…would more pronounced differences may be found for other outcomes? • What factors predict a mixed family reunification outcome? Presence of an infant?

26. A closer look at mixed outcomes…

27. eputnamhornstein@berkeley.edutshaw@ssw.umaryland.edu Thank you to our colleagues at the Center for Social Services Research, the California Department of Social Services and the Stuart Foundation