SEC-ELL Enhanced Assessment Grant: Aligning Instruction to Standards and Assessments

1. SEC-ELL Enhanced Assessment Grant: Aligning Instruction to Standards and Assessments for English Language Learners Language Demands and Complexity Edynn Sato Peter Worth Minneapolis, MN August 5, 209

2. Overview • Motivation • Questions • Goals and objectives • Preliminary findings • Continued examination • For the future

3. Motivation for the study What is academic English language and how can we support its acquisition? • How can we define it in a way that is coherent with theory and accessible to teachers? • How can we support the development and implementation of integrated curricula?

4. Study Question What is the relationship of English language development standards and assessments to state academic standards?

5. By “relationship” we are interested in… the nature of the language that students need to meaningfully engage with and achieve academic content? That is… Given the expected student outcome (content, learning), how do we expect students to use (receive, produce) language to acquire and demonstrate understanding of the content?

6. Study Goals and Objectives • Overall Purpose: Improve academic achievement of English language learner students • Use Data in Ed improvement: Use SEC data tool/methods—augmented with linguistic analysis--to measure and report the degree of alignment between instruction, state standards, and student assessments for English language learners in a sample of classrooms and schools in the Consortium states. • R&D ELL Assessment/standards: Advance and improve the SEC alignment procedures and tool to increase capacity for applying the method to English language development practices and materials. • Technical assistance/Training (PD) State level: Provide expert technical assistance for Consortium States to apply alignment analysis results to improve the quality and validity of assessment instruments for ELL students.

7. What do the data suggest regarding the relationships? Standards Curriculum/ Instructional Practice Assessment Content topics Cognitive demand Language demand Language complexity

8. Content and Academic Language What is the relationship? Content Academic Language Content Academic Language Note: We recognize the importance of linguistic skills and socio-functional language; however, the focus is on academic English language. Content Academic Language

9. Language Demands 2008

10. Language Complexity—Density 2008

11. Language Complexity—Construction 2008

12. Activity…

13. Use your knowledge and understanding of the language of your content (ELD, ELA, mathematics, science) • On your worksheet (handout), either prioritize (1-17) or note the relative emphasis of the language demands (0-100%) [You can have more than one “#1” priorities or more than one language demand at, for example, 30%.] • For each grade range • Consider progression across grade ranges • Consider the questions: • What is “academic English language” for your content area? • What is the relationship between English language development standards and academic content standards (in terms of academic language)? • What (additional) information do you need to define the academic English language for your content area?

14. What we’re finding… [see spreadsheet] Guiding questions: • Consider your responses to the previous activity and compare to/consider the following. • Look at the information in each cell in the spreadsheet – the relative emphasis of each language demand for each grade level/span and each content area (Is this expected? Intended? On what is this relative emphasis based?) • Look at the range of language demands represented for a grade level/span and content area (Is this expected? Intended? On what is this relative emphasis based?) • Look at the information for a given language demand across the grade levels/spans (e.g., Is there continuity? progression?) • Look at the information for a given language demand across content areas (e.g., Do the similarities/differences/relationships make sense? On what are the similarities/relationships/differences based?) NOTE: These data are based on standards documents only. The state may have additional information (e.g., curriculum guides, professional development materials) that may have more information related to intended language demands; however, these additional documents were not included in this analysis.

15. Some additional questions…

16. On what are the standards based (theory, research)? • What is the state’s expectation of the role of academic English language in student access to the curriculum? • What is the state’s expectation of the role of academic English language in students’ demonstration of proficiency in a content area (to what degree do students need to be able to speak, write, listen, and read about content in order to demonstrate proficiency in the content)?

17. Are state standards documents the only documents that should be examined? • Re: the language intended to enact the skills/knowledge reflected in the standards • How do teachers and test developers “know” what language (demands, complexity) is appropriate and necessary for achievement of the content? • How do students “know” what language is appropriate and necessary to engage with (e.g., read, listen) and demonstrate mastery (e.g., write, speak) of the content?

18. How much of a difference makes a difference (instruction, assessment, student achievement)? • In the relative emphasis of language demands within a grade/grade span • In the consistency of emphasis of language demands across grades/grade spans • In the level of complexity of the language (How necessary is high-density, complex language for higher-order thinking?) • What makes a difference (student achievement)? • Language functions • Language complexity • Grammar • Vocabulary

19. Other general findings… 1. A common language for language 2. Attention to mode (receptive, productive) matters 3. Language complexity varies (density, construction) 4. Language demands and complexity interact 5. Taxonomy more clearly applied to assessment; need additional considerations re: standards and instruction

20. Continue Examination For a grade level and across grade levels: • Typical/common associations between • Content topic and language demand(s) • Content topic and language complexity • Cognitive demand and language demand(s) • Cognitive demand and language complexity • Typical/common clusters of language demands • Typical/common progressions of language • Existing instructional practices that address the language needs of our EL students

21. Still need to ask… • What is intended? • What is appropriate? • On what are determinations of language demands and complexity based (definition, progression, range, relationships)?

22. For the future (research)… Do we need to refine our construct definition (language for achievement)? SEC work Language Demands work Expert Judgments (Teachers, administrators, researchers, other specialists) -Grade/grade span expectations. -Relationships between ELD and academic language of content -Range of complexity (density, construction) SEC-ELL EAG (current) -Extended SEC model to include language dimension. -Applied/tried out the language taxonomy. Review Additional Literature -Support, refute judgments/SEC-ELL EAG findings Additional Research -Examine other state/local documents -Examine student work samples -Observations, interviews -Impact, consequences on achievement, practice, policy, etc. -Etc.

23. For the future (application)… • (Taxonomy) • Consider grammatical and lexical dimensions (in addition to functional) of academic language that may contribute to student engagement with and achievement of content • Consider “content-obligatory vs. content-compatible” language (e.g., Genesee) • Consider “topic-focused vs. process-focused” language • (Process) • Consider additional documents to evaluate (e.g., curriculum guides, professional development materials re: standards implementation) • Consider how to more clearly associate/connect content topics with language demands • Refine coding and analyses (inter-rater consistency, unit of analysis, reporting)

24. For the future (application)… • (Survey) • Revisit and refine questions related to enactment of academic language in the classroom (as guided by standards/standards-based materials) • Consider whether factors influencing teacher reports of classroom practice are adequately accounted for in the survey, in order to help with the interpretation of survey results • e.g., “Understanding” of language in content areas in addition to “enactment” of languageCharacteristics of ELL student population (e.g., proficiency, literacy)Nature and quality of student engagement of students • Consider whether critical “readiness” factors (and their indicators) for effective instruction of ELL students (for language and content specialists) should be identified and incorporated into the survey

25. For the future (application)… • (Interpretation) • Continue to discuss and determine (1) what is intended and (2) what is appropriate re: language and content access and achievement • e.g., What language demands and levels of complexity are necessary to support higher order thinking and content achievement of students?What range and emphasis of language demands are needed to support student achievement