Maryland’s Approach to Converting Preschool Outcomes Data to OSEP Reporting Categories

1. Maryland’s Approach to Converting Preschool Outcomes Data to OSEP Reporting Categories Nancy M. Vorobey, M.Ed. Maryland State Department of Education nvorobey@msde.state.md.us

2. Maryland’s Choice of Assessment for Preschool Children • Since the mid 1990s, Maryland has had in place the Maryland Model for School Readiness (MMSR), “a framework to assist early educators in instructing and assessing young children in the knowledge, skills and behaviors they need to be prepared for the learning demands of formal schooling”. • State general funds have been made available through noncompetitive grants to MD’s 24 local school systems to train all K and Pre-K (4 yr. old programs) teachers in the Work Sampling System, the instrument of choice for the MMSR.

3. MMSR & Child Outcomes • Since 2000, State law has required that all children, including children with IEPs, enrolled in their kindergarten year, have data on 30 sample K-level WSS indicators collected and made public in the form of an annual report, “Children Entering School Ready to Learn.” • Choice of WSS to address Indicator #7 was made in order to be consistent with and align statewide assessment for children with and without disabilities.

4. Work Sampling System • Decision made to collect data on all indicators at each age level of the WSS • 66 for K checklist • 55 for 4 year old checklist • 49 for 3 year old checklist • Each indicator for each checklist cross walked to one or more of the 3 broad child outcomes • WSS Indicator ratings entered through a web-based system

5. Single Service Children • Observations and progress notes from SLP, OT, and/or PT collected over 6-8 weeks of initiation of services • Information gathered from several sources may be used to inform ratings for non-IEP related WSS indicators: • For children transitioning from Infants and Toddlers, information from the child’s early intervention record provided to Part B for eligibility determination may be used to help inform ratings • Parent/Care Provider Interviews (example, Ages and Stages)

6. WSS  Child Outcomes • Each WSS indicator may received a rating of: • Proficient (3) • In Process (2) • Needs Development (1) • For the February 1, 2007 revised SPP submission to OSEP for Indicator #7, a composite score of 2.5 was used as the cut-off for answering the question of performance comparable to same age peers (Status At Entry).

7. WSS Scores  OSEP’s Reporting Categories • ECO summary form will not be used • Working with a consultant to examine 2 approaches to translating WSS scores into OSEP reporting categories • GOAL: As a result of calculations, there will be indices that will be linked to the OSEP categories

8. Intervention Efficacy Index (IEI) • The IEI is a mechanism for relating changes in a child’s capabilities to time spent in a program. • It yields an index that can describe individual progress in terms of average developmental gains within and across (curricular) domains for each month’s participation in an intervention program.

9. What does the IEI look like? IEI = Developmental gain (in months) Time in Intervention (in months) MD IEI = Mean WSS Domain Exit - Mean WSS Domain Entry Time in Intervention in Months

10. IEI Example: Mean WSS Cog Exit 2.91 - Mean WSS Cog Entry 2.75 Time in Intervention - 12 months 0.16 / 12 = 0.013 Note: This IEI index is interpreted as an average of 0.013 month of gain for each month of participation in the program/services. Decision Remaining: Which indices (ranges) will be linked with each of the OSEP reporting categories.

11. Proportional Change Index (PCI) • The PCI: • Controls for children’s developmental status preintervention, which is not accounted for with the IEI. • PCI calculation is a ratio of a child’s rate of development at pretesting to the rate of development during intervention defined at posttesting. • Recommended to be used with measures from all developmental domains.

12. What does the PCI look like? PCI = Developmental gain Pretest Dev Age Time in Intervention Pretest Chron Age Note: the first ratio of the PCI is the same as the IEI; what is different is controlling for preintervention developmental rate, which is the second ratio created from the Entry data.

13. MD PCI Example: Mean WSS Cog Exit (2.91) - Mean WSS Cog Entry (2.75) Time in Intervention in Months (12) / Mean WSS Cog Entry (2.75) Age in Months at Entry (36) 0.013 / 0.076 = 0.171

14. PCI interpretation* • Children who continue to develop during intervention as they did prior to intervention receive a PCI score of 1.0 • Children whose rates of development appear to be slower during as compared to prior to intervention receive a PCI score of less than 1.0 • Children whose rates of development appear to accelerate during intervention receive a PCI score greater than 1.0; this category would likely be divided in half for the OSEP categories. (*guidelines for PCI interpretation offered by Wolery)

15. Pros, Cons & Challenges • IEI - with IEI calculations, all children make some progress; there is no consideration of dev age at entry or disability. • PCI - takes both dev age at entry and impact of disability on rate of progress over time into consideration; may yield more relevant information for program improvement. • Either approach takes us into the realm of the unknown, as we’ve never done this before!