Using the PMRN, ECI and EI Indices, and Other Information to Make Effective Decisions

1. Using the PMRN, ECI and EI Indices, and Other Information to Make Effective Decisions Elizabeth Crawford, MS, CCC-SLP Director of Interventions The Florida Center for Reading Research

2. Agenda • Risk Levels and Instructional Levels of DIBELS • Use Data to: • interpret ECI and EI tables • calculate ECI and EI indices at the class level • manage school level intervention resources • monitor progress of specific groups of students • Concluding Thoughts and Resources

3. Risk Levels and Instructional Levels of DIBELS

4. Risk Level Key

6. SY0607 Florida DIBELS Recommended Level of Instruction Decision Rules http://www.fcrr.org/assessment/pdf/ril_decisionrules_0607.pdf

7. Various Uses of Data

8. Use Data to… • Interpret ECI and EI tables • Calculate ECI and EI indices at Class Level • Manage school level intervention resources • Monitor progress of specific groups of students

9. School Level Indices

10. ECI Definition • Effectiveness of Core Instruction (ECI) — this index shows the percentage of students who began the year reading “at grade level” and are continuing to meet grade level expectations at the mid-year or end of year assessment. This index tells you whether the core instruction is sufficiently powerful to produce a year’s growth in reading skill for a year’s worth of instruction for all students who begin the year at grade level.

11. ECI – “School” .97 x 75 = 73 children 73 out of 75 children remained on grade level, 2 children did not remain on grade level 91% of the 45 kindergarteners who started the year on grade level remained on grade level .91 x 45 = 41 children 41 out of 45 children remained on grade level In first grade, my school is doing equal to or better than 96% of the RF schools on keeping our 1st grade students on grade level.

12. EI Definition • Effectiveness of Interventions (El)* —this index shows the percentage of your students who began the year at some level of risk for reading difficulties (reading below grade level) but who have grown rapidly enough to advance to a lower level of risk by the mid year or end of year assessment (i.e. they move from intensive to strategic, or from strategic to grade level). This index, and the two explained below, provide information about how effectively intervention programs are working to accelerate the development of students who are lagging behind in reading growth.

13. EI – “School” .23 x 30 = 7 students 7 out of 30 3rd graders who started at some level of risk have moved to a lower level of risk. 23 students are still at some risk. 81% of the 42 kindergartners who started the year at some level of risk, have made gains and are now at a lower level of risk. .81 x 42 = 34 students In first grade, my school is doing equal to or better than 44% of the RF schools in moving children to strategic or grade level. T

14. EI-S Definition • Effectiveness of Interventions for Strategic Students(EI-S) — this index shows the percentage of students that moved from the “strategic” risk level to grade level.

15. EI – S “School” In 3rd grade, 62% of the 16 children (10) who started at “strategic” are now at “grade level” In 1st grade, when compared to the other 583 RF schools in FL, We are doing equal to or better than 48% of them at moving first grade “strategic” students to “grade level” .56 x 27 kindergarten students who started the year at “strategic” moved to “grade level” = 15 27-15 = 12 are still “strategic”

16. EI-I Definition • Effectiveness of Interventions for Intensive Students (EI-I) — this index shows the percentage of students that moved from the “intensive” risk category to the “strategic” risk level or to grade level.

17. EI – I “School” In all grades, K-3, of the 46 students who started at “intensive” 50% (or 23) of them made growth to “strategic” or “grade level.” 93% of our 14 kindergarten students who started the year at “intensive” made progress to a lower risk level. .93 x 14 = 13 students made growth, 1 is still “intensive” In second grade, we are doing an equal or better job than 47% of the RF schools at moving our children out of the “intensive” category.

18. State Averages for ECI and EI Indices: Numbers to consider when comparing data across years • Due to different level of difficulty with passages from assessment to assessment and year to year, it is sometimes difficult to compare indices across assessments and years.

19. Background • For all subtests of DIBELS, the same forms are used at each assessment from year to year, except Oral Reading Fluency (ORF). • In order to decrease familiarity with passages, a rotation system was used – passages rotated on a four year cycle.

20. Background • This rotation systems began in 2003 and was created based on the assumption that all DIBELS ORF passages were roughly the same level of difficulty. • Also, three passages are used with a median score to further to protect against difficulty.

21. ECI State Averages

22. EI State Averages

23. EI_S State Averages

24. EI_I State Averages

25. Points to Remember • We recognize that this variability in difficulty makes it difficult to compare classroom and school growth in ORF from one year to next so we are taking steps to correct this in the next set of early progress monitoring measures being developed for use in Florida.

26. Points to Remember • If your school wants to know how you are doing in improving your ECI and EI indices, the best way is to examine your percentile rank compared to other Reading First schools in the state. • If your percentile rank increased from last year to this year, then you are doing better than average in improving your students’ performance on these indices. • Perhaps more important than the year to year change in ECI and EI, though, is relative standing on these measures. If you have a very low percentile rank on any of these indices, that means that your instruction is relatively weak when compared to that provided in other Reading First schools.

27. This school year, 08-09, PMRN will generate this School RLI report with your ECI and EI data. Percentile Ranks will only be generated once all Reading First School Data is entered. • The tables on this report are very similar to the tables that were explained above.

29. Calculating the Class Level ECI, EI-I, and EI-S • How do you do it? • AND…what does it all mean?? • Refer to “DIBELS Data Analysis Conference” Worksheet Handout

31. Key ‘follow-up’ sections on worksheet • Items Discussed • Roles and Responsibilities • Teacher will… • Reading Coach will… • Principal will…

32. Class Recommended Level of Instruction Report This report provides a summary of the students’ overall progress. It can be used to get an overall sense of instructional levels in the class and to calculate the Effectiveness of Core Instruction (ECI) index and the three Effectiveness of Intervention (EI) indices.

33. Class Recommended Level of Instruction Report

34. Class Recommended Level of Instruction Report This is the same type of report with “all” students included. Students 5, 6, 7, and 19 would not be used to calculate the ECI and EI indices because they were not in the school for all three assessment periods. This is why we select the full year only function for calculation.

35. Class Recommended Level of Instruction Report You can use the pie charts to help you fill out the top part of the DIBELS worksheet Initial Level = Low risk Strategic Level = Moderate Risk Intensive Level = High Risk Assessment #3 % Initial = 44 % Strategic = 50 % Intensive = 6

36. Class Recommended Level of Instruction Report To calculate ECI – Effectiveness of Core Instruction for the year, use only those students that were at Initial Instruction during the 1st assessment and remained in the class throughout the year (that is why we select full year only). Divide the number of the students that finished at Initial Instruction for the 3rd assessment (5) by the number of students that were at Initial Instruction for the 1st assessment (9). The higher the percentage the better.

37. Class Recommended Level of Instruction Report Effectiveness of Core Instruction # stayed initial 5 --------------------- = ------- # started initial 9 5 --- = 56% ECI = 56% 9 Target Students: Students 3, 6, 8, 9 (Initial to Strategic)

38. Class Recommended Level of Instruction Report To calculate EI – Effectiveness of Intervention for the year, use only those students that were in the class for the first assessment and the last assessment (full year only). Take the number of students who finished the year at ‘initial’ or ‘strategic’ (if they started at intensive) level by the students who started the year at some level of risk (intensive or strategic) The higher the percentage the better.

39. Class Recommended Level of Instruction Report Effectiveness of Intervention # moved to initial or strategic from Intensive 2 --------------------- = ------- # started strategic & intensive 7 2 --- = 29% EI = 29% 7 Target Students: Students 4, 7, and 10 (strategic to intensive during assessment 2)

40. Class Recommended Level of Instruction Report Effectiveness of Intervention-Strategic # moved to initial 1 --------------------- = ------- # began strategic 5 1 --- = 20% EI-S = 20% 5 Target Students: Students 4, 7, and 10 (strategic to intensive during assessment 2)

41. Class Recommended Level of Instruction Report Effectiveness of Intervention - Intensive # moved to strategic or initial 1 --------------------- = ------- # began intensive 2 1 --- = 50% EI-I = 50% 2 Target Students: Student 5

42. What do all of these indices mean for my school and teachers? • This particular teacher in 1st grade is struggling both with maintaining children at grade level and reaching her struggling readers: • ECI = 56% • EI = 29% • EI-S = 20% • EI-I = 50% • You may want to first determine if other first grade teachers have similar indices and if so, you want to examine your Core Reading Program and especially your Intervention Programs • You can look at the School Recommended Level of Instruction Report to see how different grade levels are doing by comparing the pie charts from one assessment period to the next • The goal is to see the green increase and the red decrease – to make the pie chart look more like an olive!!

43. Use Data to… monitor progress of specific groups of students

45. School Demographic Reports Demographic data is available, by assessment period, at the school and district levels for individual measures and the Recommended Level of Instruction. The data file allows all of the data to be downloaded so that Users can complete their own analyses.

48. School Demographic Reports You can also display a bar graph for a specific demographic group from this report. The next slides shows a bar graph for kindergarten students at the third assessment by Ethnicity. The total number of students in each category is in ( ) next to the group name. For example Hispanic (17)