Tools for Aligning Instruction, Standards, & Assessments

1. Tools for Aligning Instruction, Standards, & Assessments Surveys of Enacted Curriculum SEC Project Planning & Leader Training February 2005 SEC State Collaborative Tampa, Florida

2. New Member Orientation • What? … are the Survey of Enacted Curriculum tools? • How?… are the collected, analyzed, reported? • Why?… are they useful to educators, leaders, researchers? • Now what?… do we do to implement and apply to our programs and to schools?

3. Rationale Standards Assessment Curriculum

4. What SEC offers • Core Components of SEC tools/services • Surveys with teachers – instruction for year • Alignment analysis – content code standards and assessments • Data reports – pre-designed charts • Workshops – a) train for orientation b) use of data

5. SEC Collaborative 15 States, 3 Districts CCSSO – Project Management Rolf Blank, Carlise Greenfield WCER – Surveys, reporting, analysis John Smithson, Alissa Minor, Eric Osthoff Training & Technical Assistance Learning Point Associates, TERC Regional Alliance, Consultants Diana Nunnaley Carolyn Karatzas Jennifer Unger Bryan Chumbley Lani Seikaly

6. SEC Collaborative Functions/ Assumptions (04-05) • Training and assistance to state leaders • State SEC budget to meet state objectives – supported by SEC Collaborative • Each state makes decisions • on services & project plan • Multi-state benefits of collaborative

7. Multi-State Collaborative • Build Knowledge of leaders • Share Strategies • Produce joint products, tools

8. Key Question -- SEC ToolsResearch into Practice • How can Educators obtain reliable, valid data to determine Alignment of instruction with required standards and assessments?

9. Survey Sections • Instructional Readiness • Teacher Opinions • Professional Development • Types • Content Focus • Active Learning • Collegial Participation • Coherence • Time Span • Teacher Characteristics • Instructional Content • School & Class Description • Instructional Activities • General • Problem Solving Activities • Pairs & Small Group Work • Use of Hands-on Materials • Use of Calculators/Computers & other Ed. Tech. • Assessment Use • Instructional Influences

10. Steps in SEC Development • R&D studies curriculum-- Early ‘90s • Models:TIMSS, NAEP, Analyze assessment • OTL interest of States, Porter/ Smithson research, CCSSO Science Assessment project • State collaborative –Develop surveys-’98-’01 (NSF) 11-State field study, Reports, Alignment, USI • Use of data experiment – DEC (NSF/ROLE:‘01-04) • Evaluation: MSP RETA PD Study (‘02 – 05, NSF) • English/ Language Arts survey (‘03 – 04) • Current projects: 15 States, 4 Districts, 5 MSPs

11. Questions Addressed by SEC Data • How can in-depth data on content of instruction be collected and reported–not topic checklists, to analyze teaching content in relation to standards, assessments, achievement? • How can methods of teaching practices be compared across classrooms, schools, districts, and states? • How can enacted curriculum data be reported in a manner to encourage use by teachers to improve instruction?

12. Key Education Questions (cont’d) • How can we measure the effects of standards-based initiatives on instructional practices and curriculum in classrooms? • How can we analyze effectiveness of professional development on changes in teachers’ instructional practices? (i.e. determine the quality of professional development)

13. Applications • Alignment analysis --instruction, standards, assessments • Instructional improvement in schools • Needs assessment/ Evaluation • Indicators – monitoring change over time

14. What if… [for Intro of SEC to Educators] You could use data on instructional quality and content to guide professional development? You could have consistency across grade levels? You could know how well aligned your state standards were to the state assessment? You could compare how you teach content compared to how others across the nation teach? You could use anonymous teacher data to start a powerful school discussion about what the teacher needs are?

16. Content Matrix

17. Content Maps

18. A neutral content grid with cognitive demand Surveys of Enacted Curriculum The intended curriculum: State content standards—What students should learn The assessed curriculum: State (and other) assessments—tested learning The learned curriculum: Student outcomes based on school learning The enacted curriculum: What teachers teach

19. What teachers teach What standards require Curriculum Analysis:EnactedIntended

20. Full Alignment Analysis Intended— standards Enacted— taught Assessed— state tests

21. SEC Content TopicsMath, Science, ELA The SEC provides a neutral, research-based language to describe content of English language arts, mathematics, and science. NCLB anyone?

27. The Enacted Curriculum

34. Descriptive results can be reported using a variety of graphic displays

35. SEC Collaborative Members Current States participating: • New Hampshire • North Carolina • Ohio • Oklahoma • Oregon • Wisconsin • Districts • Miami-Dade • Milwaukee • Winston-Salem • Alabama • Idaho • Indiana • Iowa • Kansas • Maine • Michigan • Mississippi • Montana

36. SEC Collaborative Services  Surveys, Data Reports  Alignment Analysis – content coding  Planning, Training-- workshops  Leader materials, e.g. Orientation Education Leaders Survey Administrators Explaining SEC  Using Data on Enacted Curriculum (Model)

38. Data on Enacted Curriculum PROJECT DESIGN 5 Urban Districts—40 Middle Schools Teacher & School Data Reports Teacher Data Measure Change Standards Assessment Curriculum Improved Student Achievement Science/Math Initiatives Professional Development Assist Schools/ Professional Development Assist Schools/ Professional Development

39. SEC Collaborating Organizations • Council of Chief State School Officers www.SECsurvey.org • Wisconsin Center for Education Research www.SEConline.org • Learning Point Associates/NCREL www.SECsupport.org • TERC Regional Alliance DEC Project www.ra.terc.edu/DEC • •American Institutes for Research — MSP RETA Study – PD Evaluation

40. Longitudinal study design Spring 2003 Spring 2005 2002-03 2003-04 2004-05 Longitudinal study design Spring 2003 Spring 2005 • Baseline Data: • Teacher Survey Instruction Practices, Content, Alignment Preparation; PD • Visits to 5 sites • Baseline Data: • Teacher Survey Instruction Practices, Content, Alignment Preparation; PD • Visits to 5 sites • PD Log w/ Teachers, Interviews-- • Measure PD Quality • Type - Content • Duration • Collective • Active learning • Coherence • Content focus • PD Log w/ Teachers, Interviews-- • Measure PD Quality • Type - Content • Duration • Collective • Active learning • Coherence • Content focus • After Participation: • Teacher Survey • Compare practices, Content, Alignment • Measure change • After Participation: • Teacher Survey • Compare practices, Content, Alignment • Measure change Professional Dev. [5 MSP Sites] A) MSP Activities (treatment) B) Other PD in sites (control) Professional Dev. [5 MSP Sites] A) MSP Activities (treatment) B) Other PD in sites (control) Empirical Study of Effects of PD on Instruction

42. The teacher survey tool… • Collects data on what content is taught and how it is taught. • Produces reports indicating the degree of alignment between the taught (enacted) curriculum and state assessments or standards. • Collects data on teacher beliefs, readiness to teach the content and readiness to instruct special groups of students. • Provides a rich source of information to support teachers analysis of student learning challenges.

43. The SEC can help your schools… • Align curriculum, instruction, and assessment including vertical alignment • Improve instruction within and across grade levels, • Determine professional development needs • Focus teacher planning and development on instructional practice and its effects on student performance • Focus coaching and mentoring activity around content and practice

44. The SEC can … • Help teachers monitor changes in practice as a result of implementing new curricula. • Help teachers and leaders determine areas of need when planning for professional development. • Help teachers see areas for deepening content expertise and knowledge of how students learn that content. • Help teachers identify gaps/redundancies in curriculum –weak materials to support instruction

45. SEC Survey Administration Procedures for completing on-line surveys Procedures for completing paper surveys

46. Approximately 60-90 minutes to compete May be completed in multiple sittings Data is saved as each section is completed Online Survey Administration

47. SEC Online Registration www.seconline.org

48. SEC Online Registration Registrar Area

49. Your data is saved each time you click on a Submit button.

50. You may log-off at any time by simply closing your browser.