Teacher Performance Assessment (TPA): Scorer Training Overview

1. Teacher Performance Assessment (TPA): Scorer Training Overview An Overview… by Donna Hanby donna.hanby@wright.edu

2. Setting the Context

3. House Bill 1 • Transfers responsibility for approving teacher preparation programs from the State Board to the Chancellor of the Board of Regents • Directs the Chancellor, jointly with the State Superintendent, to: (1) establish metrics and educator preparation programs for the preparation of educators and other school personnel, and (2) provide for inspection of the institutions. • Through HB1, Ohio is first in the nation to require a four-year induction program (Resident Educator)

5. Career Ladder Teacher Performance Assessment Model

6. Ohio Comprehensive System of Educator Accountability Metrics Performance Outcome More coursework or enter different area of study Pre-Service Not Effective • Content Knowledge: Praxis II • Performance Assessment: TPA Effective Recommended for resident educator license Employment terminated Not Effective Teacher Residency PAR Program Not Effective • Formative assessment coupled with goal setting and coaching • Annual summative assessment based on multiple measures of educator effectiveness including student growth Effective Effective Continue with Residency Recommended for Five Year Professional License Employment terminated Not Effective Annual Teacher Evaluation Not Effective PAR Program • Formative assessments that inform PD and coaching support • Annual summative assessment based on multiple measures of educator effectiveness including student growth Effective Informs decisions: retention, dismissal, tenure, promotion, compensation Effective Continue as Teacher

7. TPA Background • Three-year grant to create a National Teacher Performance Assessment • Based upon the Performance Assessment for Teacher Candidates (PACT) from California (http://www.pacttpa.org) • Co-PIs ~ Linda Darling-Hammond & Ray Pecheone

8. Project Partners • AACTE (American Association of Colleges for Teacher Education) • CCSSO (Council of Chief State School Officers) • Stanford University • Lead IHE in Each State (Wright State, for Ohio) • IHEs in Each State (University of Dayton, University of Cincinnati & The Ohio State University) • SEAs (ODE & OBR)

10. National Progress to Date • Commitments from 20+ participating pilot states, including State Ed Agencies (OBR/ODE) and Institutions of Higher Education IHEs • Field-based state and IHE review of the California PACT assessment methodology, upon which this initiative is based • Conducted an alignment study with INTASC teaching standards, as well as state developed teaching standards • Implemented tasks last spring in accelerated IHEs and provided feedback • Convened design team of leading measurement experts and researchers to inform the development of the TPA

11. Ohio’s TPA Timeline • Ohio became one of 20+ States engaged in TPA in spring of 2009. • Ohio became one of five accelerated states in late spring of 2010. Others are Massachusetts, Minnesota, Tennessee and Washington. • Summer 2010, Four Year timeline completed with the TPAC team at Stanford University (3 IHE reps and 2 OBR reps attended). • Nov. 2010, State TPA meeting with 47 IHEs attending. • Dec. 2010, MOUs distributed to all 51 IHEs by Chancellor (4 yr timeline) 70% returned to date. • Jan. 2011, Ohio trained 12 Trainers-of-Training. • April 2011, Scorer Training to be held (300+ assessments).

12. April – May 2011 • April ~ State scorer training • April/May – calibrate scorers & assess portfolios • May 30 - Submit Results to Stanford • Fall 2011 – All Ohio IHEs will complete the full assessment cycle • Becomes high stakes in 2013 as one of House Bill 1’s Metrics for Teacher Education Institutions

13. A MULTIPLE MEASUREMENT ASSESSMENT The Capstone Teaching Event Embedded Signature Assessments — examples — • Teaching Event • Demonstrates : • Planning • Instruction • Assessing • Reflecting • Academic Language Child Case Studies Analyses of Student Learning Curriculum /Teaching Analyses Observation/Supervisory Evaluation & Feedback RP 13 13

14. Content Assessments There are six content assessments currently being utilized nationwide: • Elementary Literacy • Elementary Math • Secondary Math • Secondary English/Lang. Arts • Secondary Science • Secondary History/Social Science

15. What do Candidates Do for TPA? A Content-Specific Teacher Work Sample that includes: • Planning Instruction and Assessment – Task 1 • Instructing & Engaging Students In Learning – Task 2 • Assessing Student Learning – Task 3 • Retrospective Reflection – Task 4 • Academic Language • Theory into Practice application

16. What is submitted and by what means? • An electronic portfolio with the three to five days of planning, instructing ( including a 15-20 minute video), assessment, and reflection • Unique IDs will be assigned to each T. candidate and calibrated scorer for each portfolio to be scored • Portfolio artifacts will be kept on an IHEs home assessment system (Chalk & Wire, Taskstream etc.) for accreditation needs AND portfolio constructed/submitted to a separate server to protect identity and to be assessed • Results returned to candidate and program end of term

17. How is it completed? • During the student teaching experience in 3-5 lessons the teacher candidate will be teaching • It includes planning, instructing, assessing, and reflecting over 3-5 consecutive lessons to be taught (such as within a unit of study) • A faculty/supervisor may assist to determine a timeline to complete the portfolio • It will then be completed electronically (like other key assessments) and scored by a trained/calibrated assessor

18. Task 1 Purpose The Planning Instruction & Assessment task asks the candidate to: • Describe plans for the learning segment and explain how they are appropriate for the students and the content being taught; • Demonstrate the ability to organize curriculum, instruction, and assessment to help diverse students meet standards for the content; and, • Develop academic language related to the content. • Evidence ~ to select, adapt, or design learning tasks and materials that offer students equitable access to the content.

19. TPA TASK # 1: Planning Instruction and Assessment • Describe school and classroom context.  • Describe students’ academic development, language abilities, social and emotional development, and family and community assets. • Describe modifications planned for students with specific learning needs. • Identify what will be taught in the content area. • Describe plans to teach content in ways to support student learning.  • Describe how student learning will be monitored while teaching.  • Identify the language needed to process and express the content.

21. Commentary 7-9 single-spaced pages • Summarize content focus (big idea/essential question) • Describe how knowing Ss influences choices of instructional strategies for learning • How do plans support student learning of content and academic language • Monitoring student learning during the learning segment • Reflection (research/theory guiding strategies)

22. Evidence provided: • Context for Learning form • Context for Learning commentary • Lesson plans • Copies of handouts, assessments, and other materials needed to understand the learning and assessment tasks • Planning commentary

23. Task 2: Instructing and Engaging Students in Learning Purpose • The Instructing and Engaging Students in Learning task asks to demonstrate how one facilitates students’ developing understanding of skills and strategies to comprehend or compose text. • Evidence of engaging students in meaningful tasks, monitor understanding, and use responses tostudents to guide their learning.

24. Task 2 Instructing & Engaging Students in Learning • Identify lessons where students are engaged in using relevant skills and strategies to comprehend and/or compose text. One lesson is selected for filming. • Collect permission forms from parents and prepare for filming. • Video the lesson. • Review the video to identify one or two video clips that meet requirements. • Respond to commentary prompts to analyze your teaching and your students’ learning in the video clip(s). To Submit:  Video Clip(s) ~ Video Label Form ~ Instruction Commentary

25. Video Taping Specs

26. Video Guidelines • A video clip should be continuous and unedited, with no interruption in the events. If two clips are used, the two clips must come from the same lesson. • The clip(s) can feature either the whole class or a targeted group of students. • Both the teacher candidate and their students should be visible and clearly heard on the video submitted. • Tips for recording a class on video should be available from one’s program. • Before videotaping, gain the appropriate permission from parents/guardians of the teacher candidate’s students and from adults that appear on the video.

27. Instruction Commentary • 2-4 single-spaced pages • Describe strategies used to engage Ss in learning tasks to develop skills & strategies to build their understandings of the specific content. • Cite language supports seen in the clip to help Ss understand content and/or build their academic language skills. • Describe strategies for eliciting student thinking and how ongoing responses further their learning. • Reflection – student learning of concepts and academic lang. If done over, what might have been done to take advantage of missed opportunities or to improve the learning of Ss with diverse learning needs and characteristics.

28. Task 3: Assessing Student Learning Purpose • The Assessment of Student Learning task asks to assess student achievement, diagnose student learning strengths and needs, and inform instruction. Provide evidence to: • 1) develop evaluation criteria aligned with big idea or essential question, standards, and learning objectives; • 2) analyze student performance on an assessment in relation to student needs and the identified learning objectives; • 3) provide feedback to students; and • 4) use the analysis to identify next steps in instruction for the whole class and individual students.

29. Task #3: Assessing Student Learning • Describe the variation in student learning based on whole class and individual student performance. • Describe how your feedback specifically addressed your students’ learning needs. • Analyze what students learned based on an assessment that is aligned with the content learning goals.

30. Task 3 Commentary • Refer to lesson plans & cite specific standards/objectives that are measured by the assessment chosen for analysis • Create a summary of student learning for the whole class relative to the evaluation criteria • Discuss what most students appear to understand well, and, if relevant, any misunderstandings, confusions, or needs for greater challenge apparent for some or most. Cite evidence to support analysis from the three student work samples you selected. • For two specific Ss – strengths/challenges, conclusions from the work sample, feedback influencing their Individual needs and opportunities for them to apply the feedback given • Reflection on student performance on assessment, next steps, individual next steps for the two students

31. Task 3 Assessing Student Learning • Analyze student performance across the class from one assessment completed during the learning segment. • Identify three student work samples that illustrate class trends in student understanding. • Select two focus students from the class whose learning you will analyze in more depth, and for whom you will document feedback on their work. • Respond to commentary prompts to analyze the extent to which the whole class met the standards/objectives, analyze the individual learning of two focus students and describe your feedback to them, and identify next steps in instruction based on your analysis. What to submit:  • Evaluation Criteria Student Work Samples • Evidence of Feedback Assessment Commentary

32. Retrospective Reflection • Reflection is incorporated throughout TPA • During Planning, Instruction and Assessing • There is also a culminating Retrospective Reflection (Task 4) incorporated at the end of the total TPA portfolio

33. Academic Language Academic language is the oral and written language needed by students to understand and communicate in the academic disciplines for specific purposes and audiences. Academic language genres include the specialized vocabulary, linguistic features, and textual resources associated with genres within a field (e.g., literary criticism, explanations of historical phenomena, lab reports). It also includes instructional language needed to participate in learning and assessment tasks. (See Appendix A for Content-Specific Academic Language).

34. Scoring • Scoring in accelerated states should include: • Independent scoring by scorers who have not supervised the candidate • Random selection of 10-15% of portfolios submitted for double-scoring • Double scoring of any portfolio receiving more than 4 of 11 scores at rubric level “1” • The portfolio can be a requirement to be passed during student teaching (like BCI, FBI etc.) but should not be “graded” as part of the student teaching course… Passing is determined by the calibrated scorer(s) not the instructor of the student teaching course.

35. Ohio Spring TPAC Timeline

36. Thanks! For Participating In This National Initiative!!!

37. Next Steps… Room Assignments: • Check-in & meals The Capitol Room (#6) • Elementary Literacy The Worthington Room (#7) • Elementary Mathematics The Ohio Room (#10) • Secondary English-Language Arts The Westerville Room (#2) • Secondary History-Social Science The Powell Room (#1) • Secondary Mathematics The Board Room (#8) • Secondary Science The Dublin Room (#9)