Aligning Instruction to Standards and Assessments for English Language Learners (M eeting #1)

1. Aligning Instruction to Standards and Assessments for English Language Learners (Meeting #1) Iowa SEC-ELL Consortium U.S. Department of Education, Title I, Enhanced Assessment Grant December 6-7, 2007

2. Study Objectives • Do ELLs have opportunity to learn academic content and skills in state standards? • What is relationship of state ELD standards and assessments to state academic standards? • What instructional practices/strategies are used to teach English language skills? • What is relationship of alignment of instruction (to standards) with student achievement?

3. Major Steps • Planning & Instrument Development • Alignment content analysis --workshop • Data collection – in schools w/teachers • Produce data analysis reports -- from project • Technical assistance, training—state, local • Achievement analysis, Further data (options) • Report and Disseminate

4. SEC-ELL ConsortiumIowa, Florida, Idaho, Maine, Minnesota, Ohio, Utah, Virginia, Wisconsin, Nevada CCSSO – Rolf Blank Carlise Smith, Adam Peterman Carolyn Karatzas, Lani Seikaly, consultants edCount Ellen Forte WCER – John Smithson Jaime Usma WestEd -- Edynn Sato Peter Worth, Merle Longenecker Evaluator Joe McCrary (WestEd) Advisers Jamal Abedi, Gary Cook, Robert Linquanti, Charlene Rivera, Andrew Porter, Phoebe Winter

5. Meeting #1 Objectives • Review study design and decide how project will meet state needs and education community • Develop ELL component or extension of SEC instrument for ELL alignment • Project schedule and implementation plan

6. Tools for Aligning Instruction, Standards, & Assessments Surveys of Enacted Curriculum Orientation

7. SEC Orientation—Leaders, Educators • What? … are the Survey of Enacted Curriculum tools? • How?… are data collected, analyzed, reported? • Why?… are SEC data useful to educators, leaders, researchers? • Now what?…how do we implement locally and lead the use of SEC in schools?

8. Rationale Standards Assessment Curriculum

9. Applications • Alignment analysis --instruction, standards, assessments • Instructional improvement in schools • Needs assessment/ Evaluation • Indicators – monitoring change over time

10. Key Question -- SEC ToolsResearch into Practice • How can Educators obtain reliable, valid data to determine Alignment of instruction with required standards and assessments?

11. Survey Sections • Instructional Readiness • Teacher Opinions • Professional Development • Types, Frequency • Content , Active, • Collegial, Coherence • Instructional Content • Topic x Cog. Demand • School & Class Description • Instructional Activities • General • Problem Solving Activities • Pairs & Small Group Work • Use of Hands-on Materials • Use of Calculators/Computers & other Ed. Tech. • Assessment Use • Instructional Influences

13. Content Maps

14. Steps in SEC Development • R&D studies of curriculum-- Early ‘90s • Models:TIMSS, NAEP, Analyze assessment • OTL interest of States, Porter/ Smithson research • CCSSO Science Assessment project • Develop Math/Science surveys-’98-’01 (NSF) • 11-State field study, Reports format, Alignment method

15. Multi-State Collaborative • Build Knowledge of leaders • Share Strategies • Produce joint products, tools

16. SEC Collaborative Members 2007-08 • Delaware • Idaho • Illinois • Indiana • Iowa • Kansas • Maine • Michigan • • Mississippi • • Montana • • Ohio • Oklahoma • Oregon • Wisconsin • • Duval/Jax, FL • • Minnesota • • Vermont

17. Social Studies SEC Development 2005-2007 • Arizona • Delaware • Idaho • Maine • • Ohio • • Oregon • • Vermont • • Wisconsin • • Minnesota • • Nevada • U of Wis-WCER • NCSS • Civics Education • National Geographic • GENIP • NCEE • CCSSO

18. SEC Collaborating Organizations • Council of Chief State School Officers www.SECsurvey.org • Wisconsin Center for Education Research www.SEConline.org • Learning Point Associates/NCREL www.SECsupport.org • TERC Regional Alliance DEC Project www.ra.terc.edu/DEC • Measured Progress (assessment development and PD)

19. SEC Collaborative offers • Core Components of SEC tools/services • Surveys with teachers – report instruction • Alignment analysis – content code standards and assessments • How to use Data – knowledge, skills 4) Leader development – 2-3 meetings/yr • In-state workshops: a) orientation b) use of data

20. SEC by the Numbers: ’06-07 10,393 SEC Surveys 06-07 4674 Math 1993 Science 3646 ELAR 80 Soc Stud 131 Standards/Assess. Content analysis 1000 est. schools--Data use

21. SEC by the Numbers (2) • By Comparison 1996 207 Science Teachers (5 states) 1999 626 M/S Teachers (11 states) 2005 5,414 E, M, S teachers 2006 10,200 E, M, S teachers 2007 10,400 E, M, S, SSt (27 states)

22. SEC Collaborative How SEC Collaborative works for members • Training and Assistance to project leaders • Project Planning w/ budget– meet local objectives • Each state/district makes decisions on services & project plan • Multi-state benefits of collaborative e.g., learning from experience, sharing ideas, training

23. Accomplishments06-07SEC Collaborative • In-state Leader workshops, Project Planning • PPTs for Orientation, Survey Admin, Data use • Leader Development Standards document • Website revisions, improvements (SECsurvey.org; SEConline.org) • E.g., State Projects, All Alignment results, Report generator

24. More AccomplishmentsSEC Collaborative • “Using Data to Improve Instruction” St. Louis 9/07 conference – 200 leaders, 32 sessions (30 states) • Social Studies Survey development – • 10 states, 5 prof. orgs., • NAEP Alignment analysis – 2007 Math items • Research/evaluation: • MSP study of professional development– multi-site study • Ohio case study of district/school implementation of SEC • Follow-up data tool –to track school level data • Validity study—longitudinal data for teachers and students • Ohio reading study • John demonstrated SEC comparisons in Feb 07

25. Goals SEC 07-08 • State/local projects—From pilots to broad application, local leadership training • Integrate SEC tools/data with Improvement Initiatives, especially data-driven • Research/Evaluation --Broader use of Surveys, Alignment • Assist States in planning with Service delivery agencies; • Social Studies SEC complete & implement

26. (2) Steps in SEC • Data as PD in urban schools – DEC study • Evaluation of PD effects on instruction (MSP study, 4 sites) • English/ Language Arts survey (‘03 – 04) • SEC Collaborative –Operational—15 to 20 states – since ‘04 • Social Studies survey • Using Data leadership and PD • SEC–ELL development • See www.secsurvey.org/ Resources

27. Questions Addressed by SEC Data • How can in-depth data on content of instruction be collected and reported (not topic checklists), to analyze teaching content in relation to standards, assessments, achievement? • How can methods of teaching practices be compared across classrooms, schools, districts, and states? • How can enacted curriculum data be reported in a manner to encourage use by teachers to improve instruction?

28. Key Education Questions (cont’d) • How can we measure the effects of standards-based initiatives on instructional practices and curriculum in classrooms? • How can we analyze effectiveness of professional development on changes in teachers’ instructional practices? (i.e. determine the quality of professional development)

29. Content Matrix

30. Science content matrix

31. A neutral content grid with cognitive demand Surveys of Enacted Curriculum The intended curriculum: State content standards—What students should learn The assessed curriculum: State (and other) assessments—tested learning The learned curriculum: Student outcomes based on school learning The enacted curriculum: What teachers teach

32. The Enacted Curriculum

35. SEC Websites • CCSSO, SEC Collaborative www.SECsurvey.org WCER, SEC Online Survey and Reports www.SEConline.org