Developing and Selecting Student Growth Measures for Use in Teacher Evaluation Systems

1. AACC Mini Conference June 8-9, 2011 Developing and Selecting Student Growth Measures for Use in Teacher Evaluation Systems Joan Herman and Pete Goldschmidt CCSSO National Conference on Student Assessment Orlando, Florida June 21, 2011

2. Source of Presentation Developing and Selecting Assessments of Student Growth for Use in Teacher Evaluation Systems Joan L. Herman, Margaret Heritage, and Pete Goldschmidt

3. Focus of Attention • Sophisticated statistical models proposed to estimate the relative value individual teachers add to students’ performance • Little attention paid to the quality of the student assessments models use to estimate student growth

4. Quality of Measures Matters • Generating student growth scores requires at least two assessments of student learning • Carefully designed and validated to provide trustworthy evidence for evaluating teacher effectiveness • Guidance to states and districts as they develop and/or select student growth measures

5. Purpose of Guidance • Recognize the pressure on states and districts • Provide an understanding of what developing/selecting quality assessment entails • Support short-term and long-term planning for continuous improvement of measures

6. BASIC ARGUMENT JUSTIFYING USE IN TEACHER EVALUATION

7. Validity • Validity is overarching concept that defines quality in educational measurement • Concerns the extent to which an assessment measures what it is intended to measure and provides sound evidence for specific decision-making purposes. • Validation involves evaluating or justifying a specific interpretation(s) or use(s) of the scores

8. Validity Framework • Establishes the basic argument that justifies the use of student growth measures as part of teacher evaluation • Lays out the essential claims within the argument that need to be justified • Suggests sources of evidence for substantiating the claims • Uses accumulated evidence to evaluate and improve score validity

9. ARGUMENTS AND PROPOSITIONS

10. Propositions to Justify Use of Measures for Evaluating Teacher Effectiveness

11. CLAIMS AND EVIDENCE

12. Propositions and Claims to the Validity Evaluation

13. Propositions and Claims to the Validity Evaluation

14. Propositions and Claims to the Validity Evaluation

15. Propositions and Claims to the Validity Evaluation

16. Propositions and Claims to the Validity Evaluation

17. Propositions and Claims to the Validity Evaluation

18. ACCUMULATED EVIDENCE

19. Propositions and Claims to the Validity Evaluation • Validity is a matter of degree • Appraisal of all claims and evidence • Strengths and weaknesses • Assessment and validity evidence can always be improved

20. GETTING STARTED

21. Where to start? • In the beginning- will not meet all or even many of the criteria • Start by being clear on learning expectations • Ensure assessments developed or selected are aligned • Collect evidence for propositions during operational administrations • Develop long-term agenda to improve assessments

22. Finally….. • A single assessment cannot adequately capture the multi-faceted domain of teacher effectiveness • Multiple measures are essential