Loyola Marymount University School of Education The Center for Equity for English Learners (CEEL)

1. Assessment for English Language Learners: Issues, Trends, and Policy Implications Loyola Marymount UniversitySchool of Education The Center for Equity for English Learners (CEEL) Jamal Abedi CRESST/ University of California, Davis Thursday October 8, 2009

2. Why ELL students need the utmost attention? Here are 10 evidence-based points/facts (based on 65 published papers and 87 papers presented at the professional organizations)

3. Facts About ELLs: #1 They are the fastest growing population

4. Facts About ELLs: #2 These students perform substantially lower than their non-ELL peers

5. Performance Gap Between ELL and Non-ELL Students

6. Facts About ELLs: #3 • There is absolutely no evidence to suggest these students are less able to learn academic content

7. Performance Gap in Reading and Math Between ELL Students at Different Level of English Proficiency

8. Facts About ELLs: #4 There are serious issues concerning the classification of these students

9. The Relationship Between Language Proficiency Test Scores and ELL Classification One would expect a perfect correlation between English language proficiency test scores and ELL classification (ELL versus non-ELL)

10. Misclassification of ELL Students - A total of 738 eighth-grade students from 34 classrooms at 10 public schools in California participated in this study - There were 117 students (15.9%) with disabilities and 621 were students without disabilities - Of the 117 students who were classified as students with disabilities, 79 or 67.6% were ELL - This trend, is consistent with the literature that ELL students at the lower level of English proficiency have a much higher chance of being classified as having a learning disability than non-ELL students (see for example, Abedi, 2006; Artiles, Rueda, Salazar, & Higareda, 2005).

11. Facts About ELLs: #5 The ELL subgroup is the most unstable among other subgroups

12. Facts About ELLs: #6 These students, their teachers, and their parents are often blamed for the lack of progress in school

13. Facts About ELLs: #7 Assessments that are constructed and field tested for native English speakers many not be relevant for ELL students

14. ELL Students’ Performance Outcomes Suffer From Lower Reliability

15. ELL Assessment Outcomes Suffer From a Construct-irrelevant Dimension (linguistic complexity)

16. Are the Currently Used Standardized Achievement Tests Appropriate for ELLs? The Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 1999) elaborated on this issue: For all test takers, any test that employs language is, in part, a measure of their language skills. This is of particular concern for test takers whose first language is not the language of the test.

17. Facts About ELLs: #8 Classical test theory may not apply in the assessment of ELL students

18. How Classical Test Theory Works in the Assessment of ELL Students? s2X = s2T + s2E X: Observed ScoreT: True ScoreE: Error Score ρXX’=σ2T /σ2X σ2X = σ2T +σ2E +σ2s +σES

19. High-stakes accountability decisions are made based on assessments that are: • Linguistically and culturally biased • Are based on different psychometric properties • Are based on different scaling concepts and principles • Are assessed in Title I AYP when they are not proficient enough in English to take such assessments

20. Examining Complex Linguistic Features in Content-Based Test Items • Unnecessary complex linguistic features slow down the reader, make misinterpretation more likely, and add to the reader’s cognitive load; thus interfering with concurrent tasks. These features include: • Concrete vs. abstract or impersonal presentations • Item length • Unfamiliar Vocabulary • Nominal heaviness • Relative clause • Conditional clause • Passive voice • Long noun phrases • Subordinate clauses • Conditional clauses • Relative clauses • Concrete vs. abstract or impersonal presentatio • Negation

21. Example Original:The census showed that three hundred fifty-six thousand, ninety-seven people lived in Middletown. Written as a number, that is:A. 350,697B. 356,097C. 356,907D. 356,970 Modified: Janet played a video game. Her score was three hundred fifty-six thousand, ninety-seven. Written as number, that is:  A. 350,697B. 356,097C. 356,907D. 356,970

22. Linguistic Modification Concerns • Familiarity/frequency of non-math vocabulary: • Unfamiliar or infrequent words changed A certain reference file > Mack’s company Original:A certain reference file contains approximately six billion facts. About how many millions is that?A. 6,000,000B. 600,000C. 60,000D. 6,000E. 600 Modified:Mack’s company sold six billion pencils. About how many millions is that? A. 6,000,000 B. 600,000 C. 60,000 D. 6,000 E. 600

23. Facts About ELLs: #9 Accommodations are provided to level the playing field for ELL students; however, many of these accommodations are not relevant

24. Sample Accommodations Currently Used for ELL Students Enlarged answer sheets provided Breaks provided Test individually administered Test administered in small group Test administered in location with minimal distraction Native language testing

25. Facts About ELLs: #10 ELL students have less opportunity to learn

26. Impact of ELL Research on Policy and Practice • Moving toward a more positive impression of ELLs • L1 + L2 = A+ (Carlos Garcia) • Develop assessments that are more accessible for ELL students

27. Performance Gap Between ELL and Non-ELL Students Reduces as the Level of Language Demand of Assessment Decreases

28. Impact of ELL Research on Policy and Practice - State assessments & accommodations - Test Publishers - No Child Left Behind - Legislation in General

29. National and state assessments are either using or considering using linguistic modification approachU.S. Department of Education recommended the use of linguistic modification as an appropriate accommodationMany of the test publishers use cultural and linguistic bias reviews in their test development process Impact of ELL Research on Policy and Practice

30. Impact of ELL Research on Policy and Practice - Less emphasis on punitive action - More emphasis in supportive action - More attention to Title III assessment as an important Prerequisite for Title I - Impact on reauthorization

31. Research-Based Recommendations • Provide sufficient opportunity to learn for ELL students • Formative assessments play an extremely important role for ELL students; conduct formative assessments to inform instruction and assessments for these students • Use multiple measures for ELL assessment, accountability, classification, and promotion • Make sure that assessments are free from unnecessary linguistic complexity and cultural biases • Reducing the level of unnecessary linguistic complexity is not “dumbing down” the assessments, it is “cutting down” the bias • Use only those accommodations that are relevant, provide reliable and valid assessment outcomes, and are feasible

32. What is a Fair Testing Practice? -Expect all students to perform the same regardless of their backgrounds and academic needs? -Or be mindful of students’ backgrounds characteristic and their academic needs? -Should the assessments be accessible to all students? -If so, what accessibility features should be considered for ELL students?