National Picture – Child Outcomes for Early Intervention and Preschool Special Education

1. National Picture – Child Outcomes for Early Intervention and Preschool Special Education Kathleen Hebbeler Abby Winer Cornelia Taylor August 26, 2014

2. On Today’s Call • Brief review of the national data • Data quality • Completeness of data • State-to-state variation • Change over time

3. State Approaches to Measuring Child Outcomes – 2012-13

4. Method for Calculating National Estimates • Weighted average of states that met minimum quality criteria • Minimum Quality Criteria: • Reporting data on enough children • Part C – 28% or more of exiters • Part B Preschool – 12% or more of child count • Within expected patterns in the data • category ‘a’ not greater than 10% • category ‘e’ not greater than 65%

5. Number of States that Met Criteria for Inclusion in the National Analysis

6. Part C: Reason States Were Excluded from Analyses (out of 51)

7. Part B Preschool: Reason States Were Excluded from Analyses (out of 51)

12. What We See • Continuing to see consistency over time • Increasing number of states who meet minimum quality criteria for national analysis • Increasing number of children in the child outcomes data

13. Current Emphasis of State Requests • Data Quality • Increasing the number of children/families in the data • Pattern checking to identify data quality issues • Training, guidance, supervision, etc. • Using Data for Program Improvement • SSIP data analysis • Identifying trends in the data • Identifying areas of low and high performance • Identifying meaningful differences

14. Part C: Percent of States by Completeness of Child Outcomes Data* * Completeness = (total with outcomes data/total exiters) 14

15. Part B Preschool: Percent of States by Completeness of Child Outcomes Data* * Completeness = (total with outcomes data/child count) 15

16. State Level Variation and Patterns

17. Part C: Positive Social Emotional Skills Progress Category ‘b’ – All States National: 21%

18. Part C: State Variation: Greater than Expected Growth – Social Emotional, 2012-2013, All States National: 66%

19. Part B Preschool: Knowledge and Skills Progress Category ‘b’ – All States National: 15%

20. Part B Preschool: State Variation: Exited within Age Expectations – Knowledge and Skills, 2012-2013, All States National: 53%

21. Part C: Exited within Age Expectations by State Percent of ExitersNot Eligible for Part B* 2012-13 - All States

22. Part C: Average Percentage Who Exited within Age Expectations by State Percent Served*, 2012-13 – All States *http://therightidea.tadnet.org/assets/2514

23. Part C: Average Percentage Who Exited within Age Expectations by ITCA Eligibility Category, 2012-13, All States 23

24. Part B Preschool: Average Percentage Who Exited within Age Expectations by State Percent Served, 2012-13, All States

25. Variation Over Time

26. What Types of Change are Important?

27. Part C: Longitudinal Patterns All StatesOutcome 1 Summary Statement 1

28. Part C: Longitudinal Patterns All States, Last 3 Years, Outcome 1 Summary Statement 1 28

29. Part C: Longitudinal Patterns States Included in National Estimate, Last 3 Years Outcome 1 Summary Statement 1 (30 states) 29

30. Part B 619: Longitudinal Patterns All StatesOutcome 1 Summary Statement 1 30

31. Part B 619: Longitudinal Patterns All States,Last 3 Years, Outcome 1 Summary Statement 1 31

32. Part B 619: Longitudinal Patterns States Included in National Estimate, Last 3 Years Outcome 1 Summary Statement 1 (25 states) 32

33. Part C: Statistically Significant* Change between 2011-12 and 2012-13 : 41 States That Met the Criteria *p<.01

34. Part B Preschool: Statistically Significant* Change between 2011-12 and 2012-13: 39 States** That Met the Criteria *p<.01 ** Two states excluded because there was insufficient data from 2011-12 to make the comparison

35. Conclusions • The data continue to be used by the federal government to justify funding. • Results Driven Accountability is shining a spotlight on each state’s child outcomes data. • States can expect more scrutiny around data quality. • The criteria used for the national analyses do not set a high bar for data quality.

36. How We Can Help! • State data quality profiles for FFY 2012-13 were sent out • Email Abby Winer with questions: abby.winer@sri.com • Contact us for help with data quality analysis and quality assurance activities • Contact us for help with program improvement planning and data analysis

37. State Child Outcomes Data Quality Profiles FFY 2012-13 38

38. Updated National Graphing Template • http://www.ectacenter.org/eco/pages/summary.asp#summarygraph

39. Data Quality: Pattern Checking http://www.ectacenter.org/eco/assets/pdfs/Pattern_Checking_Table.pdf Training materials on looking at data: http://www.ectacenter.org/eco/assets/ppt/LookingAtData_revised.ppt 40

40. Other Resources • Additional data quality resources • http://www.ectacenter.org/eco/pages/quality_assurance.asp • Data analysis for program improvement http://www.ectacenter.org/eco/pages/usingdata.asp • SSIP-related Resources • http://www.ectacenter.org/~calls/2014/ssip/ssip.asp

41. Reminders & Announcements • We hope to see you at the Improving Data, Improving Outcomes Conference! • The Outcomes for Children and Family Outcomes flyers will be available at the conference • Stay tuned for an upcoming presentation of Family Data: Indicator C4Highlights

42. Find More Resources At: http://www.ectacenter.org/eco