Robert Linquanti Project Director & Senior Research Associate

1. Strengthening the Signal: Thoughts on Leveraging RTTT Assessment Program To Improve Outcomes for English Language Learners Robert Linquanti Project Director & Senior Research Associate RTTT AP ELL Public Input Meeting Denver, CO December 2, 2009

2. Purposes of this session • Touch on a (very) few key aspects of English Learner status relevant to topic • Examine approaches for RTTT assessment program to improve assessment (and instruction) of ELLs at different ELP levels • Consider role and feasibility of primary language content assessments • Suggest what it will take to get there

3. Which ELs are we talking about? Not a monolithic group: More variation within EL than to non-EL • US-born vs. 1.5G/Recent Immigrant • Time in US school/age-grade on entry • L1 literate vs. not (Prior schooling) • School attendance/consistency • Beginner vs. Intermediate vs. Advanced ELP • Overall composite vs. subskills • Socio-cultural/dialectical differences

4. Skimming Effect and the Redesignation Dilemma • Successful ELs typically exit EL cohort, leaving lower-performing and newly arrived • Those remaining not meeting criteria for exit: Which criteria? Linguistic? Academic? • Better performing by definition, exited ELs may have ongoing linguistic, academic needs • Suggests continuum of progress, needs and supports (vs. binary categories, funding)

5. Academic language proficiency • ESL/ELD necessary but not sufficient for academic success • Many academic tasks mediated by language • Necessary academic language skills central to performing sophisticated content area tasks • Academic language is developed across the curriculum -- Every teacher must model and teach using the language of their discipline

6. If an English learner performs poorly on academic content assessment… is it due to… • Insufficient academic language proficiency to demonstrate content knowledge? • Lack of content knowledge? (opportunity to learn content?) • Construct-irrelevant interference? (Unnecessarily complex language) • Other sources of bias or error? (Cultural distance, dialectical variation, rater misinterpretation)

7. Problems assessing ELs’ academic content knowledge & skills… =they cannot learn that content = we don’t teach them grade-level content = until they have sufficient English, it's all noise

8. Opportunities for ELs in RTTT AP • Revisit content standards with ELs in mind • Re-envision how range of content assessments could… • support teacher practices that strengthen learning • Require students (& teachers) to engage and reflect • Require (& promote) sophisticated language use • Provide richer, clearer signals on what ELs know • Advance new understandings of validity, utility

9. Common core/consortium content standards in R/LA require: • More explicit delineation of academic language forms, functions, and communicative competence features • Hypothesizing, defining, persuading, comparing, etc. • lexical, grammatical/syntactic, pragmatic, discourse, strategic competencies • More explicit descriptions of performance expectations in listening and speaking (linguistic means of demonstrating content knowledge & skills)

10. Common core/consortium content standards will require • Revisiting ELP standards and assessments: • Stronger Alignment of ELP to ELA & other content standards (breadth, depth, complexity) • Stronger linkage of content assessments & ELP assessments

11. ELA content assessments & ELP Level For lowest-ELP level EL students: • Progress on ELP assessment may provide predictive validity of ELA performance • Assumes strong ELP-ELA alignment & linkage • Accountability: Until what ELP level, for how long? • Computer adaptive testing may also more reliably estimate content knowledge

12. Promising possibilities for assessing Math/Science at Beginning ELP Use emerging technologies to assess content knowledge of ELLs at lower ELP levels • computer-based dynamic item types • graphic/representational models • simulation of context and target stimuli. [parallel form] • more expensive, best used at lowest ELP

13. Strengthening Accommodations for ELLs • Accommodations research findings mixed • 20-25% gap reduction using English dictionaries/glossaries • Plain language approaches vary in impact • Hard to disentangle reasons for this: • Necessary academic language complexity of learning strand? • Aggregate EL findings mask which accommodations work for which ELs? (Imprecise/inappropriate application) • More promising: Student profiles used to assign configurations of accommodations tailored to need • Emerging evidence of increased effectiveness

14. Potential roles of primary language (L1) in content assessments • ELs vary in L1 proficiency for academic uses: Not all ELs can better demonstrate knowledge on L1 content assessments • Need info on schooling background, exposure to L1 academic language • Test in language of instruction: Program goals aim for… • Bilingualism & biliteracy? Dual immersion w/ SLLs? • Academic core content achievement in two languages? • Transition to all English instruction? At what grade? • Content knowledge of more recent immigrants

15. Potential roles of primary language (L1) in content assessment Technical requirements: • Simultaneous test development in two languages is key • Concurrent development & piloting • Avoid translating English tests • Construct validity threats are significant • English-Spanish most feasible (80%)

16. Enormous Professional Development Imperative • Build assessment literacy of teachers and administrators • Esp. formative assessment development and use • Link to diagnosis of language and content learning needs, priorities for instructional capacity-building • Are curricular objectives clear and strong enough? • Curriculum-embedded assessments fit?

17. Integrate Expertise Up-Front Educational linguists, EL assessment and instructional experts must be involved in every phase of content standards development, curricular framework & materials development, assessment development, professional development • Much less effective as afterthought/retrofit

18. Careful Analysis and Reporting is Crucial • “Ever-EL” cohort = Current ELs + former Reclassified ELs • Helps to correct skimming bias • Current ELs: Progress expectations are Key • ELP Level by time in program (expected progress) • Trajectories vary by ELP level and grade • Academic performance by ELP level and Time in program • Program goals and expectations matter

19. Bottom Line • Adopt probabilistic rather than deterministic view of assessing ELs • Use assessments carefully (recognize limitations, ambiguity) – strengthen local assessment • Don’t lower expectations, create separate tracks • Provide useful information about student performance to improve instruction, programs • Ignoring or downplaying unavoidable problems in assessment yields misleading information • increases likelihood of harming students we’re aiming to help