Testing in the classroom: Using tests to promote learning

1. Testing in the classroom: Using tests to promote learning Richard P. Phelps Universidad Finis Terrae, Santiago, Chile January 7, 2014

2. Q. What is a standardized test? A. An assessment with at least one aspect – in its content or administration – standardized. Q. What is the key advantage of standardized testing? A. It is standardized.

3. Meta-analysis • A method for summarizing a large research literature, with a single, comparable measure. World Association of Education Research, 17th Congress, Reims, June, 2012

4. John Hattie’s meta-analyses of meta-analyses

5. John Hattie’s list 1. 21. Student self-assessment/self-grading Response to intervention Teacher credibility Providing formative assessments Classroom discussion Teacher clarity Feedback Reciprocal teaching Teacher-student relationships fostered Spaced vs. mass practice Concept mapping Cooperative vs individualistic learning Direct instruction Tactile stimulation programs Mastery learning Worked examples Visual-perception programs Peer tutoring Cooperative vs competitive learning Phonics instruction 11. Acceleration Classroom behavioral techniques Vocabulary programs Repeated reading programs Creativity programs Student prior achievement Self-questioning by students Study skills Problem-solving teaching Not labeling students 31. Student-centered teaching Classroom cohesion Pre-term birth weight Peer influences Classroom management techniques Outdoor-adventure programs Home environment Socio-economic status

6. The effect of testing on student achievement: 1910-2010 Richard P. PHELPS

7. The effect of testing on student achievement • 12-year long study • analyzed close to 700 separate studies, and more than 1,600 separate effects • 2,000 other studies were reviewed and found incomplete or inappropriate • lacking sufficient time and money, hundreds of other studies will not be reviewed World Association of Education Research, 17th Congress, Reims, June, 2012

8. Studies included in the meta-analyses • …when: • a test is newly introduced, or newly removed • quantity of testing is increased or reduced • test stakes are introduced or increased, or removed or reduced World Association of Education Research, 17th Congress, Reims, June, 2012

9. Number of studies of effects, by methodology type World Association of Education Research, 17th Congress, Reims, June, 2012

10. Effect size: Interpretation • d between 0.25 & 0.50  weak effect • d between 0.50 et 0.75  medium effect • d more than 0.75  strong effect World Association of Education Research, 17th Congress, Reims, June, 2012

11. Which predictors matter? World Association of Education Research, 17th Congress, Reims, June, 2012

12. Students tend to study more, and learn more, when: they know they will be tested, but not precisely what will be tested (e.g.) Experiment comparing gains of students with “take-home tests” to those with “in class tests” -- the latter learned substantially more. when there is reinforcement of material already studied Mastery learning experiments of 1960s—1980s: Students learn more when asked to recall what they have learned. Up to a point, the more students are made to actively process information, and describe it to others, the better they learn. Why tests?

13. Surveys and opinion polls: Regular standardized tests, performance tests World Association of Education Research, 17th Congress, Reims, June, 2012

14. Qualitative studies: Effect on student achievement 244 studies conducted in the past century in over 30 countries World Association of Education Research, 17th Congress, Reims, June, 2012

15. “Repeated retrieval during learning is the key to long-term retention.”

16. 10 benefits of testing and their applications to education Roediger, Putnam and Smith Direct effects of testing Retrieval practice during tests enhances retention of the retrieved information (relative to not testing or even to studying) -- the “testing effect” Repeated retrieval produces knowledge that can be retrieved flexibly and transferred to other situations On open-ended assessments (e.g., essay tests) retrieval practice induced by tests helps students organize information into a coherent knowledge base. Repeated retrieval leads to easier retrieval of related information SOURCE: Roediger, Putnam, & Smith, Ten benefits of testing and their applications to educational practice, Psychology of Learning and Motivation, 55, 2011.

17. 10 benefits of testing and their applications to education Roediger, Putnam and Smith Indirect effects of testing Students tested frequently study more and with more regularity. Tests permit students to discover gaps in their knowledge and adjust their study efforts to focus on difficult material. Students who study after taking a test learn more than if they had not taken a test. Students who self-test or are tested more frequently in class learn more. SOURCE: Roediger, Putnam, & Smith, Ten benefits of testing and their applications to educational practice, Psychology of Learning and Motivation, 55, 2011.

18. 10 benefits of testing and their applications to education Roediger, Putnam and Smith Benefit 1: The Testing Effect: Retrieval Aids Later Retention Benefit 2: Testing Identifies Gaps in Knowledge Benefit 3: Testing Causes Students to Learn More from the Next Study Episode Benefit 4: Testing Produces Better Organization of Knowledge Benefit 5: Testing Improves Transfer of Knowledge to New Contexts Benefit 6: Testing can Facilitate Retrieval of Material That was not Tested Benefit 7: Testing Improves Metacognitive Monitoring Benefit 8: Testing Prevents Interference from Prior Material when Learning New Material Benefit 9: Testing Provides Feedback to Instructors Benefit 10: Frequent Testing Encourages Students to Study SOURCE: Roediger, Putnam, & Smith, Ten benefits of testing and their applications to educational practice, Psychology of Learning and Motivation, 55, 2011.

19. 10 benefits of testing and their applications to education Roediger, Putnam and Smith Benefit 1: The Testing Effect: Retrieval Aids Later Retention Benefit 2: Testing Identifies Gaps in Knowledge Benefit 3: Testing Causes Students to Learn More from the Next Study Episode Benefit 4: Testing Produces Better Organization of Knowledge Benefit 5: Testing Improves Transfer of Knowledge to New Contexts Benefit 6: Testing can Facilitate Retrieval of Material That was not Tested Benefit 7: Testing Improves Metacognitive Monitoring Benefit 8: Testing Prevents Interference from Prior Material when Learning New Material Benefit 9: Testing Provides Feedback to Instructors Benefit 10: Frequent Testing Encourages Students to Study SOURCE: Roediger, Putnam, & Smith, Ten benefits of testing and their applications to educational practice, Psychology of Learning and Motivation, 55, 2011.

20. 10 benefits of testing and their applications to education Roediger, Putnam and Smith Most teachers should be testing much more frequently, …with smaller, shorter, less consequential tests. Students learns more when they test. But learn best when the tests are “spaced”. What is the optimal lapse of time between tests? The best time to test again is just before students start forgetting the information. This time lapse is shorter with discrete material, like mathematics, than with other subjects. Some studies suggest that math students should be tested at least once a week.

21. The more high-stakes decision points, the better the student performance ? SOURCE: Phelps, Benchmarking to the best in mathematics, Evaluation Review, 2001

22. Quality control has proportionally greater effect in poorer countries SOURCE: Phelps, Benchmarking to the best in mathematics, Evaluation Review, 2001

23. What testing skills do teachers need… …for interpreting information from large-scale tests? Basic understanding of statistics: - distributions, mean, median, skewness, kurtosis - sampling error, measurement error - type 1 / type 2 error, statistical power - sampling (size, representativeness) Protocols to help them explain tests to others: - to students - to parents - to the media

24. What testing skills do teachers need… …for developing and administering classroom tests? Practice (with each other) in writing items / prompts / rubrics : - unambiguous, relevant, un-biased • Understand that useful assessment can be very simple: • e.g., save the last few minutes of each class to assess by asking students to record 2-3 concepts they learned that day Learn the optimal frequency, spacing of tests for your subject field and grade level.

25. It is easy to know what you are teaching. But, you can only know what students are learning if you assess.