Curriculum Based Measurement and Language Proficiency in English Language Learners

1. Curriculum Based Measurement and Language Proficiency in English Language Learners Ella Farmer, M.Ed.; Laura Swanlund, M.Ed, NCSP; Kathy Pluymert, Ph.D, NCSP Community Consolidated School District #15, Palatine IL NASP 2010

2. The need for this study • Large, elementary district with a diverse student body • CBMs introduced at the district level in the 2008-2009 school year as part of the transition to RTI • We wanted to see if CBMs were a reliable indicator of growth rates for ELLs • We wanted to understand the rate of growth for ELLs in relation to instruction

3. What led up to this study • Data showed that some ELL students were not transferring literacy skills into English • By 4,5,6th grade the students were not making progress on standardized achievement tests (Measures of Academic Progress) • Performing poorly in the classroom • We need a process that looks at dual discrepancy compared to the correct peer group. • Teachers and administrators questioned whether CBMs were a reliable measure for ELLs

4. Research on this topic • Students who enter kindergarten with limited English proficiency lag behind their peers in reading achievement through 5th grade. Controlling for risk factors such as SES reduces the effect of LEP status on achievement (Kieffer, 2008) • Reading CBM assessments have been found to be reliable, valid, and sensitive measures to use with bilingual Hispanic students (Baker & Good, 1995) • Moderate correlations have also been found between ELL students’ performance on reading CBM measures and a state standards test (Wiley & Deno, 2005) • When using both Spanish and English CBMs with bilingual students, rates of improvement in both languages were significantly lower for ELLs than that which is seen in the overall population (Dominguez de Ramirez and Shapiro, 2006)

5. Our research questions • Is there a significant difference in initial R-CBM scores or rate of growth when comparing ELLs and English-speaking peers? • Is there a correlation between English language proficiency and R-CBM or rate of improvement? • Is there a difference in initial R-CBM score or rate of growth depending on SES, reading intervention, or type of ESL instruction?

6. The data • Large, suburban school district with 15 elementary schools • Four schools selected based on concentration of ELLs and available CBM data • N= 1,197 students, Grades 1-3 • Measures included: • R-CBM scores for fall, winter, and spring • ROI from fall to spring • English Language Proficiency (ACCESS for ELLs composite score) • SES (free or reduced lunch status) • ELL status • Reading intervention • Type of ESL instruction (Bilingual or Resource)

7. Descriptive data

8. Average R-CBM Scores and ROI

9. Correlations between R-CBM and ACCESS

10. HLM • What we did: • For each grade looked at R-CBM and predictor (i.e. SES and ELL status) • Looked at initial CBM status and rate of growth • Intercept for each grade = Average fall R-CBM score • Slope for each grade = Rate of growth • Looked at the above based on SES, ELL status, and Reading Intervention • Separate analysis for ELL students only, looking at ESL instruction, Reading Intervention, and SES as predictors • Because there were moderate to strong correlations between ACCESS scores and R-CBM, this variable was not added as a predictor

11. Level 1 Analysis At all levels there was significant variation around the intercept and slope, which means that since there is variance we can look at factors that may contribute to the variance, such as SES status, ELL status, and intervention status.

12. Fall Initial Score difference from students with higher SES and non-ESL by grade

13. Fall Growth Rate Score difference from students with no intervention, higher SES, and non-ELL by grade

14. Fall Initial Score difference from ELL students in Resource and higher SES

15. Fall Growth Rate Score difference from ELL students in Resource and higher SES

16. Summary of the findings HLM- All students • Students in grades 1-2 show significant growth on CBM measures. This is true of ELL students as well as English –speaking students. While most students in grade 3 demonstrate significant growth over the school year, some do not. • ELL students and lower SES students have lower initial (Fall) CBM scores at all grade levels, when compared to peers with no risk factors. • ELL students improve at a lower rate than their peers in first grade. However, this reverses and their rate of improvement is significantly higher than their peers in second grade. In third grade, there is no significant difference in rates of improvement when comparing ELL students and English-speaking students. • In third grade, students who are lower SES improve at a higher rate than their peers who are not at risk. • Students who are receiving reading intervention have a significantly lower rate of improvement than do their peers who are not receiving intervention. • For all students in all grades, correlations between slope and intercept were weak to moderate.

17. Summary HLM- ELL Students • ELL students from lower SES backgrounds in grade 1 have lower initial R-CBM scores. • ELL students who are lower SES in grade 2 have significantly lower rates of growth than their ELL peers of higher SES. • ELL students in grade 1 who receive reading intervention have significantly lower rates of growth than their ELL peers who are not receiving intervention. • In grades 2 and 3, ELL students who are in a bilingual program have lower initial R-CBM scores than do their peers who receive ESL resource support. This is consistent with the finding that R-CBM scores are highly correlated with ACCESS scores. • There are no significant differences in ROI depending on the type of ESL program (bilingual or resource) that students receive.

18. Implications for ELLs • When looking for a dual discrepancy under RTI, ELL students’ initial R-CBM scores should be compared to ELL peers. • A lower initial Fall R-CBM score is expected for this group when compared to English Speaking peers, therefore, it is best to compare initial scores to ELL peers • Compare rate of improvement to ELL peers • The results suggest that in all grade levels ELL students should be making consistent progress on the R-CBM during the year • ELL progress should be considered in terms of ACCESS, SES status, ESL status, and academic year

19. Limitations • Did not examine early literacy measures for Grade 1. Research suggests that NWF may be a more reliable and valid measure than RCBM (Vanderwood, Linklater, & Healy, 2008) • ELL students in this study were primarily Spanish-speaking • Did not examine predictive validity • Did not examine Spanish RCBM scores

20. Use of this data for our district • Triangulation of data • Administration, teachers, and specialists received training on how to integrate multiple data sources for ELL students • Used of the results in order to determine the ELL peer comparison expected scores and growth rates • Examined the curriculum and appropriate tier 2 interventions • Data analysis occurred at multiple levels • Program evaluation • School-wide data digs • Quarterly grade level data meetings • Individual problem solving

21. Future directions • Need to examine predictive validity of R-CBM and correlation with other measures, such as MAP and ISAT • Need to examine data for grades 4-8 • Need to examine early literacy data for grades K-1

22. References Baker, S. K., & Good, R. (1995). Curriculum-based measurement of English reading with bilingual Hispanic students: A validation. School Psychology Review, 24(4), 561. De Ramírez, R. D., & Shapiro, E. S. (2006). Curriculum-Based Measurement and the evaluation of reading skills of Spanish-speaking English language learners in bilingual education classrooms. School Psychology Review, 35(3), 356. Kieffer, M. J. (2008). Catching up or falling behind? Initial English proficiency, concentrated poverty, and the reading growth of language minority learners in the united states. Journal of Educational Psychology, 100(4), 851-868. Vanderwood, M. L., Linklater, D., & Healy, K. (2008). Predictive accuracy of Nonsense Word Fluency for English language learners. School Psychology Review, 37(1), 5. Wiley, H. I., & Deno, S. L. (2005). Oral reading and maze measures as predictors of success for English learners on a state standards assessment. Remedial and Special Education, 26(4), 207.