PAY ATTENTION! There is going to be a test.

1. PAY ATTENTION!There is going to be a test. Standardized Tests and Education Reform Keith Clay, GRCC

2. Our love of tests, part I “Are we going to be tested on this?” An inservice teacher at a biotechnology workshop at UC San Francisco

3. Our love of tests: Part II Check the web. The WASL is: • “stupid” • “evil” • “AWFL” • “child abuse” • “terrorism”

4. Tests and Education Reform • Why Standardized Tests? • The Art and Science of Testing • The state of Washington • Do tests help with education reform? • Do education reforms help on the tests?

5. Why standardized tests? Some critics argue that “High-Stakes Testing” is just a bad idea… …but it’s not… …It’s the law.

6. Why standardized tests? • Assessment of curricula • Core Knowledge? FOSS? PBI? • Assessment of systems • Seattle School District? Mrs. Nelson’s class? • Assessment of individuals • Formative? Summative? Instructive?

7. Assessment of Curricula

8. Assessment of systems • Identify substandard schools • Identify outstanding schools • Then what? • Allow students to leave weak schools? • Financially reward low performance?

9. Assessment of individuals • “Formative” or “Diagnostic” • Identify strengths and weaknesses • Construct strategies for improvement • “Summative” or “Evaluative” • Grades • Graduation Requirement • 10th Grade WASL in 2008 (5th graders now)

10. Summative Eval Question: “Should a PhD historian be able to pass a state physics test without ever taking a physics class?” Bernie Khoury, AAPT • Should a “state physics test” be physics content free? • Should a PhD historian be allowed to graduate from high school?

11. Assessment as Teaching Tool “Instructive” or “reflective” assessment • Students study the results of their own assessment throughout learning • White and Frederiksen, 2000 • “Inquiry + reflective assessment” produced better results than “Inquiry + discussion” • Differences were greatest for weakest students and hardest material

12. Design of assessment tools: • Curriculum assessment probes for “student averaged” content weaknesses • Systems assessment probes “student and content averaged” weaknesses • Individual summative tests probe for “content averaged” weakness • Individual formative tests probe for specific content and student weakness

13. High-Stakes Assessment: • NCLB mandated assessments often try to do all four things at once. • Nat’l Assessment of Educ. Progress (NAEP) is “high-bandwidth, low-fidelity.” • “Measurement at the level of individual students is poor.”NRC, Knowing What Students Know, 2001

14. The art and science of testing THE ASSESSMENT TRIANGLE (KWSK, 2001)

15. The art and science of testing Psychometrics: • For statistical analysis, correct responses (cr) must outnumber random chance (rp) • Usual requirement: cr > 2 rp • A,B,C,D choice, cr > 50% • A,B,C,D,E choice, cr > 40% • Open ended: Average > 40% to 50%

16. The art and science of testing Compromises: Can we get 40% correct? Q: Throw a ball straight up into the air and it comes back down to your hand. Define “up” to be the “positive direction.” When the ball is at the maximum height, is has an acceleration of: A) 9.8 m/s2 D) -4.9 m/s2 B) -9.8 m/s2 E) zero C) 4.9 m/s2 • Most students choose E, so psychometricians suggest removing it from the list of options.

17. Can we get 40% correct, if… We frame the same question but ask… 1) …at maximum height, is has a velocity of: A) 9.8 m/sD) -4.9 m/s B) -9.8 m/sE) zero C) 4.9 m/s 2) …at max. height, is has an acceleration of: A) 9.8 m/s2 D) -4.9 m/s2 B) -9.8 m/s2 E) zero C) 4.9 m/s2 • 40% of students get #2 right if asked #1 first.

18. The state of Washington: EALR, WASL, SCIF, SALT, and OIAs • EALR: Ess. Acad. Learning Requirement • WASL: Wa. Assess’t of Student Learning • SCIF: Science Curr. Instr. Frameworks (they write the EALRs) • SALT: Sci. Assess’t Leadership Team (They write the WASL) • OIAs: Other Important Acronyms

19. The state of Washington: • SCIF and SALT talk to each other (this is unusual) • Aligned on content and level (Bloom) • From SCIF: EALR and Nat’l Standards alignment document in progress • From SALT: content specialists may override psychometricians

20. The state of Washington: K8 Science according to SCIF and SALT: INQUIRY 30% Ecosystems, Circuits, Rock cycle, Nerves… Rockets, Solar system, Spiders… SYSTEMS: 40% 30% DESIGN

21. Do tests help education reform? “Whereas teaching directly to the items on a test is not desirable, teaching to the theory of cognition and learning that underlies a test can provide a positive direction for instruction.” NRC, Knowing What Students Know, 2001

22. Do tests help education reform? “Large-scale standardized assessments can communicate across time and place, but by so constraining the content and timeliness of the message that they often have limited utility in the classroom.” NRC, Knowing What Students Know, 2001

23. Do tests help education reform? The Frame and the Tapestry Thompson and Zeuli, 1999 • Michigan Educ. Assessm’t Prog. (MEAP) • MEAP has both goals and assessments • Most local school districts aligned their curricula with MEAP and NSES

24. Do tests help education reform? “…considering how deep-seated most teachers’ ideas are about subject matter, teaching, and learning, one would not expect these… substantially aligned documents to produce conceptually transformative teacher learning on a broad scale. Our classroom-level research confirms that they have not.” Thompson and Zeuli, 1999

25. Does reform help the tests? • Education reform for teachers is a necessary precondition for success “It is now widely accepted that in order to realize recently proposed reforms in what is taught and how it is taught in math and science, teachers will have to unlearn much of what they believe, know, and know how to do, while also forming new beliefs, developing new knowledge, and mastering new skills.” Thompson and Zeuli, 1999

26. Teaching reform requires… For teachers: • Cognitive dissonance • Time for reflection • Connection with their classrooms • A repertoire of techniques • Continuing support For the system: • A scalable, sustainable process Thompson and Zeuli, 1999

27. How long will this take? “This is a generation-long process.” Pinky Nelson, WWU & Project 2061