Daniel Webster, PhD Barbara Needell, PhD Terry Shaw, PhD Center for Social Services Research

1. California Child Welfare Outcomes and Accountability Legislation:Evolving Toward System Improvement with Longitudinal Data & Analysis Panel on Increasing Impact and Redistributional Efficiency in Public Child Welfare Daniel Webster, PhD Barbara Needell, PhD Terry Shaw, PhD Center for Social Services Research University of California, Berkeley Association for Public Policy Analysis & Management 28th Annual Research Conference Madison, WI November 2, 2006 Slides in this presentation originally created by Barbara Needell, Lynn Usher, and Emily Putnam-Hornstein The Performance Indicators Project at CSSR is supported by the California Department of Social Services and the Stuart Foundation.

2. Presentation Outline • Background on Assembly Bill 636 • A Review of Progress • Relative Change on 20 Outcome Indicators • Multivariate Models on Permanency • The Future of AB636

3. 3 Key Approaches to Data Data

4. Caseload Snapshots Versus Entry Cohorts Jan. 1, 2005 Jan. 1, 2004 Jan. 1, 2006

5. The Cycle of Experiences in the Child Welfare System Rate of Referrals/ Substantiated Referrals Home-Based Services vs. Out-of-Home Care Reentry to Care Permanency Through Reunification, Adoption, or Guardianship Counterbalanced Indicators of System Performance Use of Least Restrictive Form of Care Shorter Lengths Of Stay Maintain Positive Attachments To Family, Friends, and Neighbors Stability Of Care Source:Usher, C.L., Wildfire, J.B., Gogan, H.C. & Brown, E.L. (2002). Measuring Outcomes in Child Welfare. Chapel Hill:  Jordan Institute for Families,

6. Assembly Bill 636Child Welfare Outcomes and Accountability System • Legislation passed: October 2001 • Baseline Quarterly Report: January 2004 • Provisions of the Act • County self-assessments & self-improvement plans • Quarterly data reports to state and county officials • Public-private collaboration to support reform efforts • Longitudinal data publicly available

7. The Double-Edged Nature ofPublic Data • PROS: • Greater performance accountability • Community awareness and involvement, encourages public-private partnerships • Ability to track improvement over time, identify areas where programmatic adjustments are needed - County/County and County/State collaboration • CONS: • Potential for misuse, misinterpretation, and misrepresentation • Available to those with agendas or looking to create a sensational headline • Misunderstood data can lead to the wrong policy decisions

8. California:AB636 Federal Measures, Percent IMPROVEMENT from Baseline to Most Recent Report Period Safety Permanency Note: (+) indicates a measure where a % increase equals improvement. (-) indicates a measure where a % decrease equals improvement.

9. California:AB636 State-Enhanced Measures, Percent IMPROVEMENT from Baseline to Most Recent Report Period Note: (+) indicates a measure where a % increase equals improvement. (-) indicates a measure where a % decrease equals improvement. indicates a measure where performance declined. Safety & Participation Permanency Well-Being

10. Study Limitations & Next Steps • Shortcomings of Administrative Data • Study Time Period • Examine All Performance Indicators • Analyze Interplay Between Outcomes • Further Specification for Multivariate Models

11. Whither AB636 ? • Early Indication of Positive Change • Results of present study • Attitude shift in public child welfare staff • A New Landscape with CFSR Round 2 • Composites, components, & measures (oh my!) • Rethink and re-tool quarterly report • Educate child welfare agency staff • Turning Data into Knowledge • Familiarity/Use of data throughout agency • Making the link from outcome to practice • Developing human capital to use data