Early Childhood Outcomes: Using Data for Program Improvement

1. Early Childhood Outcomes: Using Data for Program Improvement Kathy Hebbeler ECO at SRI International Robin Rooney and Christina Kasprzak ECO at Frank Porter Graham Institute Illinois October, 2008

2. Objectives • Understand the purposes of the child outcomes data collection • Be familiar with key considerations related to accurately completing the Child Outcomes Summary Form • Be able to evaluate child outcomes data for accuracy as part of supervision • Understand the use of child outcomes data for program improvement, including for instruction, resource allocation, and professional development Early Childhood Outcomes Center

3. Why are we doing this? Early Childhood Outcomes Center

4. Keeping our eye on the prize: High quality services for children and families that will lead to good outcomes. Early Childhood Outcomes Center

5. Goal of early childhood special education “…To enable young children to be active and successful participants during the early childhood years and in the future in a variety of settings – in their homes with their families, in child care, in preschool or school programs, and in the community.” (from Early Childhood Outcomes Center, http://www.fpg.unc.edu/~eco/pdfs/eco_outcomes_4-13-05.pdf) Early Childhood Outcomes Center

6. High Quality Data on Outcomes • Data are a piece of a system that helps to achieve overarching goals for children and families • Data yield • Evidence that allows you to make an inference that should lead to specific actions to improve the system. Early Childhood Outcomes Center

7. System for Producing Good Child and Family Outcomes Adequate funding Good outcomes for children and families High quality services and supports for children 0-5 and their families Good Federal policies and programs Good State policies and programs Good Local policies and programs Strong Leadership • Prof’l Development • Preservice • Inservice

8. The Vision: Using Data as a Tool for Program Improvement • Illinois will have quality data available on an ongoing basis about multiple components of the system • Outcomes for children and families • Programs and services provided • Personnel (types, qualifications, etc.) • Etc. Early Childhood Outcomes Center

9. Driving Force for Data on Child Goals Comes from the Federal Level • Government Performance and Results Act (GPRA) • Program Assessment Rating Tool (PART) • Individuals with Disabilities Education Act (IDEA) Early Childhood Outcomes Center

10. Government Performance and Results Act (GPRA) passed in 1993 • Requires goals and indicators be established for IDEA • Indicators and data collection further along for school age population than for EC • Previously, for early childhood data had been collected on: • Number of children served • Settings Early Childhood Outcomes Center

11. PART evaluation results (2002) • 130 programs examined in 2002; 50% programs had no performance data • Programs looking at inputs, not results • Part C and Section 619 • No long-term child outcome goals or data • Need to develop a strategy to collect annual performance data in a timely manner Early Childhood Outcomes Center

14. Federal Funding for Preschool Special Education Early Childhood Outcomes Center

15. Early Childhood Outcomes Center

16. Individuals with Disabilities Education Act SEC. 616. <<NOTE: 20 USC 1416.>> MONITORING, TECHNICAL ASSISTANCE, AND ENFORCEMENT. ``(a) Federal and State Monitoring.-….. ….. ``(2) Focused monitoring.--The primary focus of Federal and State monitoring activities described in paragraph (1) shall be on-- ``(A) improving educational results and functional outcomes for all children with disabilities; Early Childhood Outcomes Center

17. How Office of Special Education (OSEP) responded • Required states to submit outcome data in their Annual Performance Report (APR) • Funded the Early Childhood Outcomes (ECO) Center to do research, make recommendations, and assist states Early Childhood Outcomes Center

18. Where are we now: Federal reporting requirements Early Childhood Outcomes Center

19. OSEP Reporting Requirements: the Outcomes • Positive social emotional skills (including positive social relationships) • Acquisition and use of knowledge and skills (including early language/ communication [and early literacy]) • Use of appropriate behaviors to meet their needs Early Childhood Outcomes Center

20. OSEP Reporting Categories Percentage of children who: a. Did not improve functioning b. Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers c. Improved functioning to a level nearer to same-aged peers but did not reach it d. Improved functioning to reach a level comparable to same-aged peers e. Maintained functioning at a level comparable to same-aged peers 3 outcomes x 5 “measures” = 15 numbers Early Childhood Outcomes Center

21. Due February 2008 Data in reporting categories at exit for all children who have been in the program for at least 6 months Must be reported for the year beginning July 1, 2006 Repeat with next year’s data in 2009, etc. Reporting Schedule 2010 • States must set targets • Summary statements for targets recommended; not yet official* • States must report data to public by school districts for these target numbers *See http://www.fpg.unc.edu/~ECO/meetings.cfm for more information on recommended targets Early Childhood Outcomes Center

22. Where are we now: State decisions and activities Early Childhood Outcomes Center

23. WHY? (State Version) Purpose To have data for program improvement and to respond to federal reporting requirements To respond to federal reporting requirements Early Childhood Outcomes Center

24. WHY? (Local Version) Purpose To have data for program improvement and to provide data to the state To provide data to the state Early Childhood Outcomes Center

25. State approaches • Most states have embraced outcomes measurement and are collecting outcomes data for their own purposes. • Many states are building bigger systems than needed to produce the federal data. • Go to www.the-eco-center.org for more information about what other states are doing Early Childhood Outcomes Center

26. How are states collecting child outcomes data? Possible state approaches to collection of child data • Child Outcomes Summary Form (COSF) • Publisher’s online assessment system • Single assessment statewide • Other approaches Early Childhood Outcomes Center

27. State Approaches to Measuring Child Outcomes

28. Checking in – How is it going? Implementing the Child Outcomes Summary Form Process • What is going well? • What is not going well? • Are there other issues or concerns that need to be addressed about the process? Early Childhood Outcomes Center

29. Let’s Review: Child Outcome Summary Form Early Childhood Outcomes Center

30. Learning from Each Other • Did you attend a training on the COSF? • Have you participated in a COSF team rating process? • For more than 10 children? 20? 30? • Have you reviewed COSF forms completed by others? • Have you provided training on the COSF? • How confident are you in your knowledge of the COSF process? Early Childhood Outcomes Center

31. Essential Knowledge for Completing the Child Outcomes Summary Form Between them, team members must: • Know about the child’s functioning across settings and situations • Understand age-expected child development • Understand the content of the three child outcomes • Know how to use the rating scale • Understand age expectations for child functioning within the child’s culture Early Childhood Outcomes Center

32. Important point • It is not necessary that all team members be knowledgeable in all 5 areas • Especially, no expectation that parents understand the rating scale or typical child development • But the professionals have to! Early Childhood Outcomes Center

33. Essential Knowledge for Completing the Child Outcomes Summary Form Between them, team members must: • Know about the child’s functioning across settings and situations • Understand age-expected child development • Understand the content of the three child outcomes • Know how to use the rating scale • Understand age expectations for child functioning within the child’s culture Early Childhood Outcomes Center

34. 1. Know about the child’s functioning across settings and situations How we learn about the child’s functioning across settings and situations: Good assessment Early Childhood Outcomes Center

35. DEC* recommended practices for assessment • Involve multiple sources • Examples: family members, professional team members, service providers, caregivers • Involve multiple measures • Examples: observations, criterion- or curriculum-based instruments, interviews, norm-referenced scales, informed clinical opinion, work samples *Division for Early Childhood Early Childhood Outcomes Center

36. Assessment practices appropriate for outcomes measurement: ASHA* ASHA recommended practices: • Gather information from families, teachers, other service providers • Collect child-centered, contextualized, descriptive, functional information (*American Speech-Language-Hearing Association) Early Childhood Outcomes Center

37. Assessment the tool vs. assessment the process Assessment tools can inform us about children’s functioning in each of the three outcome areas Challenge: There is no assessment tool that assesses the three outcomes directly Assessment instruments Early Childhood Outcomes Center

38. Essential Knowledge for Completing the Child Outcomes Summary Form Between them, team members must: • Know about the child’s functioning across settings and situations • Understand age-expected child development • Understand the content of the three child outcomes • Know how to use the rating scale • Understand age expectations for child functioning within the child’s culture Early Childhood Outcomes Center

39. Resources for understanding age-expected child development • ECO link http://www.fpg.unc.edu/~eco/pdfs/Age-expected_child_dev_9-5-07.pdf (under “ECO Tools”) • New course coming soon • Watch ECO web site www.the-eco-center.org Early Childhood Outcomes Center

40. Essential Knowledge for Completing the Child Outcomes Summary Form Between them, team members must: • Know about the child’s functioning across settings and situations • Understand age-expected child development • Understand the content of the three child outcomes • Know how to use the rating scale • Understand age expectations for child functioning within the child’s culture Early Childhood Outcomes Center

41. Outcomes Jeopardy $100 $100 $100 $200 $200 $200 $300 $300 $300 Early Childhood Outcomes Center

42. Children have positive social relationships • Involves: • Relating with adults • Relating with other children • For older children, following rules related to groups or interacting with others Includes areas like: • Attachment/separation/autonomy • Expressing emotions and feelings • Learning rules and expectations • Social interactions and play Early Childhood Outcomes Center

43. Children acquire and use knowledge and skills • Involves • Thinking • Reasoning • Remembering • Problem solving • Using symbols and language • Understanding physical and social worlds • Includes: • Early concepts—symbols, pictures, numbers • Imitation • Object permanence • Expressive language and communication • Early literacy Early Childhood Outcomes Center

44. Children take appropriate action to meet their needs • Involves: • Taking care of basic needs • Getting from place to place • Using tools (e.g., fork, toothbrush, crayon) • In older children, contributing to their own health and safety • Includes: • Integrating motor skills to complete tasks • Self-help skills (e.g., dressing, feeding, grooming, toileting, household responsibility) • Acting on the world to get what one wants Early Childhood Outcomes Center

45. Essential Knowledge for Completing the Child Outcomes Summary Form Between them, team members must: • Know about the child’s functioning across settings and situations • Understand age-expected child development • Understand the content of the three child outcomes • Know how to use the rating scale • Understand age expectations for child functioning within the child’s culture Early Childhood Outcomes Center

46. The two COSF questions a. To what extent does this child show age-appropriate functioning, across a variety of settings and situations, on this outcome? (Rating: 1-7) b. Has the child shown any new skills or behaviors related to [this outcome] since the last outcomes summary? (Yes-No) Early Childhood Outcomes Center

47. 7 – Completely • The child shows behaviors and skills expected in all or almost all everyday situations that are part of the child’s life • Home, store, park, child care, with strangers, etc. • The child’s functioning is considered appropriate for his/her age • No one has significant concerns about the child’s functioning in this outcome area Early Childhood Outcomes Center

48. 6 – Between completely and somewhat • The child’s functioning generally is considered appropriate for his or her age • but there are somesignificant concerns about the child’s functioning in this outcome area Early Childhood Outcomes Center

49. 5 – Somewhat • The child shows functioning expected for his/her age some of the time and/or in some situations • The child’s functioning is a mix of age-appropriate and not appropriate functioning • The child’s functioning might be described as like that of a slightly younger child Early Childhood Outcomes Center

50. 4 – Between a 5 and a 3 • Child shows some age appropriate functioning some of the time or in some situations or settings but most of the child’s functioning would be described as not yet age appropriate. • The child’s functioning might be described as like that of a younger child Early Childhood Outcomes Center