Similar English Learner Students, Different Results

1. Similar English Learner Students, Different Results Why Do Some Schools Do Better? EdSource www.edsource.org Trish Williams, Kenji Hakuta, Ed Haertel, Mike Kirst, Jesse Levin, Robert Linquanti, Noli Brazil, Mary Perry, Isabel Oregon

2. Castañeda v. Pickard Sound theory examine revise reform Results Implementation evaluate

3. Castañeda v. Pickard Sound theory examine revise reform Results Implementation evaluate

4. Reading Comprehension 12 12 11 10 9 9 Age Equivalent Score 8 7 6 6 5 Program 4 Bilingual 3 3 English-Only 0 0 1 1 2 2 3 3 4 4 5 5 6 6 GRADE

5. Opportunity Costs

6. AIR / WestEd Study Staff capacity to address EL needs; Schoolwide focus; Shared priorities and expectations; and Systematic assessment and data-driven decision-making.

7. The Research Question Why do some schools serving similarly challenged student populations vary in their performance on the API by as many as 250 points? Extended analysis in 2006-07: Why do some schools serving similarly challenged EL student populations vary in their performance on the EL-API by as many as 250 points?

8. 2005 English Learner Base Academic Performance Index (API) vs. School Characteristics Index (SCI)

9. Survey Development • Review of effective schools literature updated for current CA standards-based policy environment • Focused on concrete, actionable items, neutrally phrased • Initial surveys reviewed by academics, state policymakers, K-12 educators • Piloted study in 11 elementary schools

10. Size of SSDR Sample is Impressive • 257 elementary schools from 145 districts • 80% of K-5 teachers at each school • 5,500 total completed teacher surveys • All principals from each school • About 50% of the schools from the 25-35 band • Teacher survey was 31 pages with 63 multi-part questions, totaling 370 items each • The principal survey had 442 items

11. Geography of SSDR Sample Schools

12. SSDR Sample Student Demographics

13. Restricting the Sample to Look at EL:

14. Restricting the Sample to Look at EL:

15. Survey Research Domains • Prioritizing student achievement • Implementing a coherent, standards-based instructional program • Using assessment data to improve student achievement and instruction • Ensuring availability of instructional resources • Enforcing high expectations for student behavior • Encouraging teacher collaboration and providing professional development • Involving and supporting parents

16. Findings Which schoolwide and district practices differentiated schools with higher EL-APIs from schools with lower EL-APIs, serving similar EL student populations?

17. Findings: Use of Assessment Data • Teachers report receiving CST data in variety of formats • Principals report directly using data from CST and other tests in many ways related to student learning and instructional improvement • Principals report that district evaluates them upon and provides support for site level planning related to improving student achievement

18. Findings: Ensuring Availability of Instructional Resources • Teacher competence is strong: • teachers report 5 years experience and a standard credential • principal reports staff have strong skills and positive attitudes • Both report school has current texts for every student, adequate materials, maintained facility

19. Findings: Curriculum and Instructional Program is Coherent • Based on CA academic content standards, especially math and ELA • Consistent within each grade and aligned from one grade to the next • District provides the grade by grade curriculum, principal provides the oversight, and teachers make it happen

20. Findings: Prioritizing Student Achievement • School vision = student learning outcomes • Principal makes expectations clear • Teachers embrace responsibility • Plans for improvement are well defined • API/AYP goals are a priority • Measurable goals for exceeding the API subgroup growth targets

21. Effect Size Predicted variation in API scores (based on standard deviations) for each domain in order of importance. *Based on findings published in Similar Students, Different Results, 2005

22. Observation: Vertical Alignment District to Classroom • Strong district leadership, accountability, and support appear to influence schoolwide and EL API • Principal leadership is being redefined to focus on effective management of the school improvement process • Teachers get clear signals and support, work together to implement curriculum, and take responsibility for student achievement

23. Analysis of EL-Specific Survey Questions

24. Findings • Higher school EL–API was correlated with principals who affirmed that in the last four years, their school has implemented a new program for EL students. • Also more positively correlated with higher EL-API was response by a school’s teachers that explicit English Language Development instruction was delivered to the teacher’s EL students through a pull out program (e.g. resource teacher).

25. Findings • The response by the teachers at a school that their EL students were taught mathematics using ESL or immersion techniques (SDAIE) was also more positively correlated with a higher schoolwide weighted mean scale score for math on the CST. • In addition, as was also found in our original analysis, schools with higher EL–API more often had teachers who agreed strongly that their district addresses the instructional needs of English Learner students at their school.

26. Positive with AMAO • School principal responses indicating that a new EL program had been implemented at the school within the past four years was positively correlated with AMAO.

27. Negative with AMAO • Negatively and significantly correlated with AMAO 1 and 2 were strong school-level teacher responses that explicit ELD instruction was delivered to EL students by the classroom teacher herself, or by ELD level through teacher teaming. • Also negatively correlated were teacher reports that they receive CELDT assessment data and that, besides CELDT, teachers use no other type of ELD assessment for their EL students.

28. An Interpretation We interpret these results to complement those discussed elsewhere in this report. Namely, having in place an explicit program of instruction for ELs is fundamentally important to their English language development needs, and enhancing EL students’ access to and comprehension of core academic subjects such as math may also yield benefits to their academic language development in English.

29. Some Surprises • We were surprised that the presence in a school of more teachers with CLAD/BLCAD certification did not have a relationship, in our analysis, to schools’ EL–API scores.

30. Some Surprises • Another surprise was that the number of minutes teachers reported devoting to explicit ELD instruction at the school did not correlate with EL–API.

31. An Interpretation Quality trumps quantity.

32. Bigger Picture: Castañeda v. Pickard Sound theory examine revise reform Results Implementation evaluate

33. The Daunting Challenge • The rich get richer.

34. THANK YOUFor more information:hakuta@stanford.eduwww.edsource.orgEdSource520 San Antonio Road, Ste. 200Mountain View, CA 94040-1217 (650) 917-9481