1 / 53

Teacher Performance Assessment Consortium

Teacher Performance Assessment Consortium. November 10, 2010 Andrea Whittaker and Nicole Merino Stanford University. 1. Three Sponsoring Organizations. AACTE – overall project management, communication with programs Stanford University – assessment development and technical support

lavonn
Download Presentation

Teacher Performance Assessment Consortium

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Teacher Performance Assessment Consortium November 10, 2010Andrea Whittaker and Nicole MerinoStanford University 1

  2. Three Sponsoring Organizations • AACTE – overall project management, communication with programs • Stanford University – assessment development and technical support • Council of Chief State School Officers – policy development and support, communication with state education agencies

  3. TPAC State Participation Requirements • Sign off from state education agencies • Participation of preparation programs • Up to 3 funded through grant • Alignment and policy studies • Adaptation of TPA to address future state program and policy needs

  4. Lineage of TPAC • National Board for Professional Teaching Standards (NBPTS) portfolio assessments – accomplished teachers • Connecticut BEST assessment – teachers at end of induction • Performance Assessment for California Teachers (PACT) – pre-service teachers source: SCALE @ Stanford University

  5. APPLE CriteriaThe National Board for Professional Teaching Standards Administratively feasible Publicly credible Professionally acceptable Legally defensible Economically affordable

  6. Design Principles: Educative Assessment • Discipline Specific • Integrated Assessment • Student Centered • Analytic Feedback and Support • Represents a Complex View of Teaching: Multiple Measures 7

  7. The TPAC Assessment System Assessments Embedded in Local Programs — examples — The Capstone Assessment • TPAC Assessment • Demonstrates : • Planning Instruction • and Assessment • Engaging Students • and Supporting • Learning • Assessing Student • Learning • Reflecting • Academic Language Child Case Studies Analyses of Student Learning Curriculum /Teaching Analyses Observation/Supervisory Evaluation & Feedback 8

  8. TPAC Assessment Records of Practice* 9

  9. Guiding Questions and Analytic Rubrics PLANNING Planning Focused, Sequenced Instruction Using Knowledge of Students to Inform Teaching Planning Assessments to Monitor and Support Student Learning ENGAGING Engaging Students in Learning Deepening Student Learning During Instruction • ASSESSMENT • Analyzing Student Work • Using Assessment to Inform Instruction • Using Feedback to Guide Further Learning • REFLECTION • Monitoring Student Progress and Adjusting Instruction • ACADEMIC LANGUAGE • Understanding Language Demands and Resources • Developing Students’ Academic Language Repertoire 10 10

  10. Rubric Levels • Is the candidate ready for independent teaching (i.e., to be the teacher of record)? • Rubric Levels • Level 1 – Some skill but needs more practice to be teacher-of-record • Level 2 – Acceptable level to begin teaching • Level 3 – Solid foundation of knowledge and skills • Level 4 – Stellar candidate, in the top 5% of candidates

  11. Questions and Answers ???

  12. PACT ScoresInter-rater Reliability Sample Size · 2,580 Spearman-Brown Reliability Estimate · 0.88 13

  13. PACT Validity Studies • Content validity • Development teams, Program directors, Program faculty, & Leadership team • TPE alignment study • Concurrent validity • Evaluation of score validity • Decision Consistency · Holistic vs. Analytic ratings • Bias and fairness review • Construct validity • Factor Analysis(2002-03 Pilot Year): • Reflection & Assessment • Instruction • Planning • Predictive Validity (Carnegie/CT Study) • Consequential Validity 14

  14. Research Plans • Value-added analysis • Observations and interviews of candidates completing assessment + first year of teaching • Surveys and interviews with faculty and program administrators on use of assessment data

  15. Assessment Development • 2009-10 Small-scale tryout tasks & feedback from users • 2010-11 Development of pilot prototypes based on feedback and piloting in at least 2 credential areas per institution. User feedback to guide revisions • 2011-12 National field test of prototypes, producing a technical report with reliability and validity studies and a bias and sensitivity review. National standard setting. • 2012-13 Adoption of validated assessment

  16. Overview of Tasks and Rubrics • Artifacts and Commentaries • What looks familiar? • Level Three Rubric Descriptors • What gets assessed? • Tips for implementation

  17. Handbook Overview

  18. Context Matters • Candidates describe what they know about their students as learners: • Academic development • Academic Language development • Social-emotional development • Family, community and cultural assets • Special needs • Candidates use information about students to plan appropriate instruction and assessment. • Scoring rubrics examine extent to which needs of students are addressed

  19. Guiding Questions and Analytic Rubrics PLANNING Planning Focused, Sequenced Instruction Using Knowledge of Students to Inform Teaching Planning Assessments to Monitor and Support Student Learning ENGAGING Engaging Students in Learning Deepening Student Learning During Instruction • ASSESSMENT • Analyzing Student Work • Using Assessment to Inform Instruction • Using Feedback to Guide Further Learning • REFLECTION • Monitoring Student Progress and Adjusting Instruction • ACADEMIC LANGUAGE • Understanding Language Demands and Resources • Developing Students’ Academic Language Repertoire 20 20

  20. EM1: Planning Focused, Sequenced Instruction • Standards/objectives, learning tasks, and assessments are clearly aligned to a big idea or essential question. The learning tasks and assessments represent differing depths of understanding • Candidate plans how to make clear connections among mathematical facts, computations/procedures, concepts, and reasoning.  • Learning tasks build on each other to promote an understanding of the designated mathematical concepts, computations/procedures, and reasoning skills. Learning tasks (or their adaptation) are justified by explaining their appropriateness for students with references to relevant research and/or theory.

  21. EM2: Using Knowledge of Students to Inform Teaching • Learning tasks and materials draw upon students’ academic and social/emotional development, including strengths, as well as experiences and interests to help students reach the learning objectives. • Planned support consists of strategically selected or modified tasks/materials and/or scaffolding of instruction that is closely tied to specific learning objectives. It is appropriate for specific individuals or subgroups.

  22. EM3: Planning Assessments that Monitor and Support Student Learning • The set of assessments will provide evidence of student learning relative to the standards/objectives for each lesson. At least one lesson’s assessments provide evidence of student learning that extends beyond the formulaic application of computations or procedures.  • Assessments are aligned to clearly defined benchmarks or criteria for student performance. • Assessments are modified or adapted to be appropriate for students having difficulty demonstrating their learning.

  23. EM4: Engaging Students in Learning • Strategies for intellectual engagement seen in the clip(s) offer structured opportunities for students to develop their own understanding of mathematical concepts through discourse. These strategies reflect attention to students’ academic or language development, social/emotional development, and/or cultural and lived experiences. • Candidate identifies successful and unsuccessful teaching practices. The proposed improvements are reasonable and address the learning of a subgroup or individual students.

  24. EM5: Deepening Student Learning During Instruction • Candidates and/or other students build on what students are saying and/or doing, using reasoning to improve understanding of mathematical concepts. • Candidate and/or other students prompt students to make connections between and among mathematical concepts and representations of content.

  25. EM6: Analyzing Student Work • Criteria are well-defined and reflect the depth of understanding stated in of the indicated standards/objectives from the learning segment. • The analysis focuses on patterns of student errors, skills, and understandings in relation to standards and learning objectives. The analysis uses these patterns to understand student thinking. • The analysis is supported by work samples and the summary of learning. Specific patterns are identified for individuals or subgroup(s) in addition to the whole class.

  26. EM7: Using Assessment to Inform Instruction • Next steps follow from an accurate analysis of student learning and aim at improving student understanding of important features of the standards/learning objectives. • Next steps focus on improving student performance through targeted support to individuals and groups to address specific identifiedneeds. • Next steps are based on whole class patterns of performance and some patterns for individuals and/or subgroups and are described in sufficient detail to understand them.

  27. EM8: Using Feedback to Guide Learning • Specific and accurate feedback helps the student understand what s/he did well, and provides guidance for improvement. • Candidate describes how students will use feedback to improve their work or their understanding.

  28. EM9: Monitoring Student Progress and Adjusting Instruction • Daily reflections identify what students could or could not do within each lesson and consider the implications for meeting the standards/objectives at the end of the learning segment. • Adjustments to instruction are appropriate and focused on addressing some individual and collective learning needs.

  29. EM10: Understanding Language Demands and Resources • Candidate describes academic language strengths and needs of students at different levels of academic language proficiency. • The language genre(s) discussed are clearly related to the academic purpose of the learning segment and language demands are identified. One or more linguistic features and/or textual resources of the genre are explicitly identified. • Candidate identifies essential vocabulary for students to actively engage in specific language tasks.

  30. EM11: Developing Students’ Academic Language Repertoire • The candidate’s use of scaffolding or other support provides access to core content while also providing explicit models, opportunities for practice, and feedback for students to develop further language proficiency for selected genres and key linguistic features. • Candidate articulates why the instructional strategies chosen are likely to support specific aspects of students’ language development for different levels of language proficiency.

  31. Potentially New Competencies • Academic language development • Interpreting student work and identifying trends across the class • Designing next steps in instruction based on assessment results • Providing feedback to students

  32. Issues to Think about for Spring 2011 Pilot • Match with curriculum • Which faculty? Which credential areas? • Candidate support • Timeline, from introducing assessment through scoring • Preparation for videotaping • Recruiting scorers • Consideration of results

  33. What did successful PACT programs do? • Broad and collegial involvement in decision making • Inquiry and program improvement rather than compliance • Affirm and maintain program values and identity • Broad involvement in assessment activities • Regular practices and protocols for data analysis • Learned from individual “case” studies

  34. Questions and Answers ???

  35. Academic Language Overview Three “F” Words…

  36. Academic Language Academic language development is making the language explicit to expand students’ control over language and improve their language choices according to the purpose/FUNCTIONand audience for the message. Academic language also offers structures/FORMSfor developing as well as expressing explanations, evaluations, and analyses. Developing students’ FLUENCY in academic language forms and functions provides access to the “language of school” and academic success

  37. Academic Language The purposes of Academic Language areto clearly and explicitly define, classify, analyze, explain, argue, interpret and evaluate ideas for distant audiences.

  38. Academic Language The purposes of Academic Language areto clearly and explicitly define, classify, analyze, explain, argue, interpret and evaluate ideas for distant audiences. = FUNCTION

  39. Academic Language • Language for academic purposes differs greatly from every day purposes • These differences include • a better-defined system of FORMS (genres) with explicit expectations about how texts are organized to achieve academic purposes • precisely-defined vocabulary to express abstract concepts and complex ideas • more complex grammar in order to pack more information into each sentence • a greater variety of conjunctions and connective words and phrases to create coherence among multiple ideas • formatting conventions, graphics and organizational titles and headings to guide understanding of texts

  40. Academic Language Academic Language also includes instructional language needed to participate in learning and assessment tasks, including discussing ideas and asking questions, summarizing instructional and disciplinary texts, following and giving instructions, listening to a mini-lesson, explaining thinking aloud, giving reasons for a point of view, writing essays to display knowledge on tests.

  41. Academic Languagein TPAC • Vocabulary • Genre/forms with particular functions ▪ Linguistic features (grammar, organization) ▪ Textual resources (headers, table of contents, illustrations, graphs, charts)

  42. Vocabulary • Technical vocabulary: triangle, metaphor, metabolize • Words whose technical meaning is different than everyday language: “balance” in chemistry, “plane” in mathematics, “ruler” in history/social science, “force” in science • Connector words: and, but, because, therefore, however

  43. Genres/Forms • Have a general structure – e.g., narratives, explanations, arguments • Structure of an explanation - Description of what is being explained - Statements of cause-effect relationships - Sometimes ending with an interpretation judgment

  44. Subject-Specific Genres/FORMS • Representing word problems mathematically • Procedures for a science experiment • Literary interpretation • Argument proposing causes of an historical event

  45. Structures ofArgument Genre/FORM • Mental verbs used to express opinions: like, believe • Move from personal to impersonal voice • Connectives used for logical relations and to link points • Temporal connectives: first, next • Causal conditional connectives: because • Comparative connectives to introduce counterpoints: consequently, therefore

  46. Structure of Arguments (from Knapp & Watkins, 2005) • Simple argument: point/proposition, elaboration • “I like The Simpsons because they make me laugh.” • Argument with evidence: Proposition, argument, conclusion • Discussion: statement of issue, arguments for, arguments against, recommendation • Elaborated discussion: statement of issue, preview of pro/con, several iterations of point/elaboration representing arguments against, several iterations of point/elaboration representing arguments for, summary, conclusion

  47. Characteristics of Advanced Academic Language • Abstractions: government, electron, linear equation, acid • Nominalizations, verbs or adjectives becoming nouns to enable more dense text or more cohesive text: organize into…→ this organization… …were revealed. The trigger for this revelation was… • More precise connector words and phrases, going beyond “and” or “but” to “in contrast” or “Given this, it follows that…”.

  48. Academic LanguageCompetencies Measured (Rubric 10) • Understanding language demands and resources for instructional emphasis • Identification of linguistic features of a genre/form/function addressed within instruction • Relation of vocabulary identified to content and to students’ academic language proficiencies • Description of student language strengths and needs

  49. Academic LanguageCompetencies Measured (Rubric 11) • Expanding students’ academic language repertoires • Making key linguistic features related to genre purpose (form and function) visible to students • Modeling vocabulary and linguistic features and providing opportunities for practice (developing fluency) • (at higher levels) appropriateness of models for students at different levels of language proficiency

More Related