Assessing Your Standards: Standards and Assessment Alignment

1. AdultEducation Content Standards Consortia Meeting Assessing Your Standards: Standards and Assessment Alignment Regie Stites July 22, 2005 U.S. Department of Education, Division of Adult Education and Literacy American Institutes for Research

2. Assessment is …?

3. Assessment is … • “Rocket science” • A critical component of effective teaching • Not all that it needs to be to support standards-based educational improvement

4. True or False • A test is a test is a test • A score is a score is a score • Two tests that measure the same thing can be equated • Multiple choice questions only measure recall • You can tell a good item by looking at it • Multiple-choice test = Standardized test = High-stake test

5. Assessment Literacy • All educational assessments are observations and interpretations of defined constructs (models of cognition) • “(A)n assessment result is an estimate, based on samples of knowledge and performance from the much larger universe of everything that a person knows and can do” (from Knowing What Students Know: The Science and Design of Educational Assessment, NRC, 2001, p. 37)

6. Best Assessment Practices • Assessments should be used for the purpose they were designed to serve – for different purposes use different tests • Purpose is defined by when you give the test (before, during, or after instruction) and by how you will use the test results (formative or summative)

7. Formative Assessment • Key purpose is to use test results (usually before or during instruction) to plan, monitor, and improve teaching and learning • Should be curriculum-based and aligned with standards • May or may not be standardized • Can be selected or locally-developed

8. Summative Assessment • Key purpose is to use results as evidence of achievement (usually pre/post or post instruction) • May or may not be high-stakes for the learner • Should be a standardized test to ensure reliability and fairness • Can be used as partial evidence of instructional quality (and to guide improvement)

9. Seeing Standardized Tests In A New Light • Recognize their true utility – don’t ignore results and don’t over-interpret them • Be patient - help everyone understand that standardized test results are just one of many sources of evidence of learning on standards • Question the experts – ask for the help you need to improve alignment of standards and assessments

10. Webb’s Criteria • Categorical concurrence • Extent to which assessment items address same content as standards • Depth-of-knowledge consistency • Extent to which cognitive complexity of responses to assessment tasks matches what learners should know and be able to do as stated in the standard • Range-of-knowledge correspondence • Proportion of indicators of standards covered by assessment items • Balance of representation • Extent to which assessment items are distributed evenly across indicators

11. Alignment Resources • Norman L. Webb (1999). Alignment of Science and Mathematical Standards and Assessments in Four States http://www.ccsso.org/content/pdfs/AlignmentPaper.pdf • CCSSO Alignment Analysis Web page http://www.ccsso.org/Projects/Alignment_Analysis/

12. Guided Practice 1 • Select high priority standards that are most suitable for standardized testing (with the test you have) • With the Webb criteria in mind evaluate the likely alignment of these standards to your current test and options for additional assessment if needed

13. Guided Practice 2 • Discuss the need to organize an assessment review or a formal alignment analysis of your state standards and the test you use for program accountability • Map out plans for a review of assessment options or a formal alignment analysis