Orientation to Measuring Child and Family Outcomes for New People

1. Orientation to Measuring Child and Family Outcomes for New People Kathy Hebbeler, ECO at SRI Lynne Kahn, ECO at FPG/UNC Early Childhood Outcomes Meeting Baltimore, Maryland August 2007 Early Childhood Outcomes Center

2. We will cover • Why the federal government is interested in outcomes • How these outcomes were selected • Review the SPP/APR indicators and what they mean • Questions encouraged throughout!! Early Childhood Outcomes Center

3. How Looking at Child Indicators for EI/ECSE Became Important Early Childhood Outcomes Center

4. Focus on Accountability • Followed an era of focusing on and measuring actions/activities • Call for interest in the “ultimate result”: child and family outcomes • Cut across both the public and private sectors • Funders want data to determine whether a program is doing what it is supposed to do Early Childhood Outcomes Center

5. Critical Events in EI/ECSE Accountability • 1992 –Osborne and Graebler, Reinventing Government • 1993 – GPRA (Government Performance and Results Act) passed • Intervening years…. • 2002 – PART (Program Assessment Rating Tool) finds there are no data on outcomes for Part C or Part B Preschool • 2003 – OSEP begins to ask states for child outcome data (and funds the ECO Center!) • 2005, 2006 – OSEP releases revisions to the reporting requirements Early Childhood Outcomes Center

6. PART Review for Part C and Part B Preschool • Results Not Demonstrated Part C “While the program has met its goal relating to the number of children served, it has not collected information on how well the program is doing to improve the educational and developmental outcomes of infants and toddlers served.” Part B Preschool “The Department has no performance information on preschool children with disabilities served by this program.” • Read more at Expectmore.gov Early Childhood Outcomes Center

7. What was happening in the intervening years? • Special Education – • National study found poor outcomes for special education adolescents Results • Push to include students with disabilities in statewide assessment systems • Early Childhood • Debate about whether child outcomes should be measured at all • Discussion of the many problems in trying to measure outcomes for young children with disabilities Early Childhood Outcomes Center

8. Measuring Child Indicators for EI/ECSE • The PART findings put an end to the debate about whether or not to do it • Unfortunately, almost no progress had been made in the intervening years as to HOW to do it Early Childhood Outcomes Center

9. Why These Outcomes? Early Childhood Outcomes Center

10. Early Childhood Outcomes Center • Funded in October 2003 • Promote the development and implementation of child and family outcome measures for infants, toddlers and preschoolers with disabilities that can be used in national and state accountability systems Early Childhood Outcomes Center

11. Origin of the Child Outcomes • ECO Center convened stakeholder meetings followed by public comment period (2004) • First, collected themes and ideas • Then, drafted and re-drafted wording based on input from stakeholders • Made recommendation to OSEP (2005) Early Childhood Outcomes Center

12. Themes from the Stakeholders: Child Outcomes • Consistent with IDEA and legislative intent • Reflect what EI and ECSE are trying to do • One set for birth to 5 • Reflect what is known about development and learning Early Childhood Outcomes Center

13. Themes from the Stakeholders • Be compatible with best practice (esp. transdisciplinary service models, functional behaviors) • Do not base them on domains • Have potential to influence practice in a positive way • Incorporate universal design • Be readily understood Early Childhood Outcomes Center

14. Make Outcomes/Indicators Functional • Functional refers to things that are meaningful to the child in the context of everyday living • Refers to an integrated series of behaviors or skills that allow the child to achieve the outcomes • Not discrete skills; not domain-based Early Childhood Outcomes Center

15. Recommendations • Decision: One set of indicators birth to 5 • Decision: Functional outcomes • Decision: Global, not specific Early Childhood Outcomes Center

16. The need for an introductory statement • Family and child outcomes are linked • There are overarching goals for children and family that cut across the outcomes • Helpful to frame the outcomes with these overarching goals Early Childhood Outcomes Center

17. Concepts in the introduction: Goal for children The ultimate goal is for young children to be active and successful participants now and in the future in a variety of settings– in their homes, in their child care, preschool or school programs, and in the community http://www.fpg.unc.edu/~eco/pdfs/eco_outcomes_4-13-05.pdf Early Childhood Outcomes Center

18. Concepts in the introduction: Goal for children • Active and successful participants • Now and in the future • In variety of settings Early Childhood Outcomes Center

19. Concepts in the introduction: Goal for families • Enable families to provide appropriate care for their child • Have resources they need to participate in community activities Early Childhood Outcomes Center

20. Concepts in the preface: Outcomes and accountability • Outcomes determined by variety of factors • Not all families and children will achieve all outcomes • BUT, system should still strive to achieve them Early Childhood Outcomes Center

21. ECO Child Outcomes • Children have positive social relationships • Children acquire and use knowledge and skills • Children take appropriate action to meet their needs Early Childhood Outcomes Center

22. Understand their child’s strengths, abilities, and special needs Know their rights and advocate effectively for their children Help their children develop and learn Have support systems Access desired services, programs, activities in their community ECO Family Outcomes Early Childhood Outcomes Center

23. What kind of data should be reported on child outcomes? • Had extensive discussions with stakeholders about status vs. progress data • Status – where child is • Progress – how child has changed • Progress data alone is weak evidence • How much progress should be considered “good progress” or “enough progress?” Early Childhood Outcomes Center

24. ECO Recommendations to OSEP • ECO submitted recommendations to OSEP in May 2005 on the kind of percentages that should be requested from states. http://www.fpg.unc.edu/~eco/pdfs/ECO_response_to_OSEP_5-9-05.pdf Early Childhood Outcomes Center

25. The APR/SPP Requirements Related to Outcomes Early Childhood Outcomes Center

26. OSEP Reporting Requirements: Child Outcomes • Positive social emotional skills (including positive social relationships) • Acquisition and use of knowledge and skills (including early language/ communication [and early literacy]) • Use of appropriate behaviors to meet their needs Part C #3; Part B #4 Early Childhood Outcomes Center

27. OSEP Reporting Categories Percentage of children who: a. Did not improve functioning b. Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers c. Improved functioning to a level nearer to same-aged peers but did not reach it d. Improved functioning to reach a level comparable to same-aged peers e. Maintained functioning at a level comparable to same-aged peers 3 outcomes x 5 “measures” = 15 numbers Early Childhood Outcomes Center

28. Additional information about child indicators • Child must be in program at least 6 months • Data must be collected at (near) entry and (near) exit • Entry data for year July 1, 2005 to June 30, 2006 to be reported in February 2007 • Progress data to be reported in February 2008 and therafter Early Childhood Outcomes Center

29. Implications for Measurement • Compares entry to exit data for each child- • 2 measurement points are needed for each child for the data analysis. • Individual children’s entry data will need to be maintained and matched to exit data • Information about children’s functioning in each area needs to be compared to age level expectations Early Childhood Outcomes Center

30. Implications for Measurement(continued) • Outcomes are broad • Functional outcomes summarize each child’s current functioning across settings and situations • Not asking for domains • Best practice for assessing young children recommends the use of multiple measures Early Childhood Outcomes Center

33. Comparing to Same Aged Peers? • Deficit model? Not individualized? • Progress alone is weak evidence • Goal of EI/ECSE: Active and successful participation now and in the future • Kindergarten, school readiness, having friends, community participation • Setting high expectations Early Childhood Outcomes Center

34. Understand their child’s strengths, abilities, and special needs Know their rights and advocate effectively for their children Help their children develop and learn Have support systems Access desired services, programs, activities in their community Percent of families participating in Part C who report that EI services have helped the family Know their rights Effectively communicate their children’s needs Help their children develop and learn Comparison of OSEP requirements to ECO recommendations: Family outcomes (Part C only) ECO OSEP (#4) Early Childhood Outcomes Center

35. Percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities ECO recommends that states collect additional information about families of children 3-5 OSEP APR Requirements for Families for Part B Preschool (#8) Early Childhood Outcomes Center

36. Additional information about family indicators • Timing is not specified (e.g., every May, at exit, etc.) • States have options with regard to how they collect the information • Survey, interview • Which tool Early Childhood Outcomes Center

37. Family Outcome Issue • OSEP reporting requirements do not call for data on family outcomes for C or B Early Childhood Outcomes Center

38. Family Outcomes Issues • All states are doing some kind of data collection on family issues • OSEP requirements • + satisfaction +/or parent involvement +/or outcomes, etc. • Some states are opting to collect data on family outcomes Early Childhood Outcomes Center

39. Tools for Indicator Measurement • Child Outcomes Summary Form (COSF) • Family Outcomes Survey (FOS) • NCSEAM Survey Early Childhood Outcomes Center

40. Finding out more • Additional information about OSEP requirement and state activities • ECO and other resources including training materials for the COSF, Family Outcomes Survey, resources on assessment and typical child development, and materials for parents www.the-eco-center.org Early Childhood Outcomes Center