The Benefits of Standardized Tests in Education

1. Why standardized tests? Richard P. Phelps University of Bucharest 4 June 2018 Why Standardized Tests? – U. Bucharest

2. “If a thing exists, it exists in some amount. If it exists in some amount, then it is capable of being measured.” −−René Descartes, Principles of Philosophy, 1664 Why Standardized Tests? – U. Bucharest

3. Learning Curve Why Standardized Tests? – U. Bucharest

4. Forgetting Curve (1870s) Why Standardized Tests? – U. Bucharest

5. Why Standardized Tests? – U. Bucharest

6. Ebbinghaus: “Learning usually requires rehearsal or repetition” Why Standardized Tests? – U. Bucharest

7. Working Memory Capacity George Miller, 1950s Cognitive Load Theory John Sweller, 1980s Why Standardized Tests? – U. Bucharest

8. Working Memory: Ability to temorarily hold and manipulate information for cognitive tasks Working Memory is challenged by: new, unfamiliar information and quantity of discrete bits of information Why Standardized Tests? – U. Bucharest

9. Two centuries of research on learning concludes… “…repeated retrieval during learning is the key to long-term retention.” — Henry L. “Roddy” Roediger Why Standardized Tests? – U. Bucharest

10. 10 benefits of testing and their applications to education Roediger, Putnam and Smith Benefit 1: The Testing Effect: Retrieval Aids Later Retention Benefit 2: Testing Identifies Gaps in Knowledge Benefit 3: Testing Causes Students to Learn More from the Next Study Episode Benefit 4: Testing Produces Better Organization of Knowledge Benefit 5: Testing Improves Transfer of Knowledge to New Contexts Benefit 6: Testing can Facilitate Retrieval of Material That was not Tested Benefit 7: Testing Improves Metacognitive Monitoring Benefit 8: Testing Prevents Interference from Prior Material when Learning New Material Benefit 9: Testing Provides Feedback to Instructors Benefit 10: Frequent Testing Encourages Students to Study SOURCE: Roediger, Putnam, & Smith, Ten benefits of testing and their applications to educational practice, Psychology of Learning and Motivation, 55, 2011. Why Standardized Tests? – U. Bucharest

11. 10 benefits of testing and their applications to education Roediger, Putnam and Smith Indirect effects of testing Students tested frequently study more and with more regularity. Tests permit students to discover gaps in their knowledge and adjust their study efforts to focus on difficult material. Students who study after taking a test learn more than if they had not taken a test. Students who self-test or are tested more frequently in class learn more. SOURCE: Roediger, Putnam, & Smith, Ten benefits of testing and their applications to educational practice, Psychology of Learning and Motivation, 55, 2011. Why Standardized Tests? – U. Bucharest

12. John Hattie’s meta-analyses of meta-analyses Why Standardized Tests? – U. Bucharest

13. John Hattie’s list of education interventions, in order of effectiveness ( those with testing ) Student self-assessment/self-grading Response to intervention Teacher credibility Providing formative assessments Classroom discussion Teacher clarity Feedback Reciprocal teaching Teacher-student relationships fostered Spaced vs. mass practice Concept mapping Cooperative vs individualistic learning Direct instruction Tactile stimulation programs Mastery learning Worked examples Visual-perception programs Peer tutoring Cooperative vs competitive learning Phonics instruction 1. 21. 11. Acceleration Classroom behavioral techniques Vocabulary programs Repeated reading programs Creativity programs Student prior achievement Self-questioning by students Study skills Problem-solving teaching Not labeling students 31. Student-centered teaching Classroom cohesion Pre-term birth weight Peer influences Classroom management techniques Outdoor-adventure programs Home environment Socio-economic status Why Standardized Tests? – U. Bucharest

14. The effect of testing on student learning • 12-year study, read >3,000 documents • analyzed close to 700 separate studies, and more than 1,600 separate effects • 2,000 other studies were reviewed and found incomplete or inappropriate • hundreds of other studies remain to be reviewed Why Standardized Tests? – U. Bucharest

15. The effect of testing on student learning 245 Qualitative studies 813 Surveys or Polls 640 Quantitative Studies: Experiments: School- and classroom-level Multivariate studies: Large-scale testing programs Why Standardized Tests? – U. Bucharest

16. Meta-analysis A method for summarizing a large research literature, with a single, comparable measure. ( 0.5 effect size ≈ 1 grade level of learning ) Why Standardized Tests? – U. Bucharest

17. Survey study effect sizes average >1.0 Over 90% of qualitative studies positive For quantitative studies, effect sizes vary between 0.55 and 0.88: + testing more frequently + testing with stakes + testing with feedback Findings from Phelps (2012): Why Standardized Tests? – U. Bucharest

18. Cognitive Scientists’ 6 Strategies for Effective Learning Retrieval Practice Spaced Practice Dual Coding Interleaving Concrete Examples Elaboration Why Standardized Tests? – U. Bucharest

19. Retrieval Practice Why Standardized Tests? – U. Bucharest

20. 10 benefits of testing and their applications to education Roediger, Putnam and Smith Most teachers should be testing much more frequently, …with smaller, shorter tests. Students learn more when they test. But learn best when the tests are “spaced”. What is the optimal lapse of time between tests? Why Standardized Tests? – U. Bucharest

21. Why Standardized Tests? – U. Bucharest

22. Why Standardized Tests? – U. Bucharest

23. Implications for Teachers 1 Most teachers should test more frequently, …with smaller, shorter, low-stakes tests Understand that useful assessment can be short and simple. Why Standardized Tests? – U. Bucharest

24. Implications for Teachers 2 • Does the test format matter? • multiple-choice? • essay? • short answer? • oral? • demonstration? • …etc.? • Not so much. Why Standardized Tests? – U. Bucharest

25. Implications for Teachers 3 Tests provide feedback to teachers about what works and what does not Just like students can learn by testing each other; teachers can help each other by reviewing each others’ tests. Why Standardized Tests? – U. Bucharest

26. WhyStandardizedtests? In some places, the only objective measure available to the public (i.e., not under the control of insiders). Why Standardized Tests? – U. Bucharest

27. Studies of the reliability of teacher grading, 1890s to 1920s • e.g., Starch & Elliot, 1912 • Twoactual English examination papers • Sent to 142 teachers to grade • Grades ranged from 50 to 98% • One paper: 14 grades < 80% & 14 > 94% Why Standardized Tests? – U. Bucharest

28. Studies of the reliability of teacher grading, 1890s to 1920s • Starch & Elliot, 1912 • Twoactual Geometry examination papers • Sent to 116 teachers to grade • Grades ranged from 28 to 92% • One paper: 20 grades < 60% & 9 > 85% Why Standardized Tests? – U. Bucharest

29. How can those outside a school or classroom judge the quality of a school, its instruction, or its students? Schools vary in quality Courses vary in quality Grade comparisons are not reliable Standardized tests’ most important feature is standardization. Why Standardized Tests? – U. Bucharest

30. Most respond to both intrinsic and extrinsic motivators and the proportion varies from individual to individual. High-stakes tests provide both forms of inducement. High-stakes tests tend to be taken more seriously and administered with tighter security. Whyhigh-stakestests? Why Standardized Tests? – U. Bucharest

31. Findings from Phelps & Silva Meta-Regression (2018) To raise achievement: Test more often + Test more often with more feedback ++ (formative testing) Test more often with more stakes ++ (summative testing) Test more often with feedback AND stakes? – NO (they are less effective together than separate) Formative & summative differ; serve different purposes. Why Standardized Tests? – U. Bucharest

32. Cognitive Psychology experiments were conducted with “formative” tests in schools and classrooms Why Standardized Tests? – U. Bucharest

33. The more systemwide decision points, the better ? SOURCE: Phelps, Benchmarking to the best in mathematics, Evaluation Review, 2001 Why Standardized Tests? – U. Bucharest

34. Quality control has proportionally greater effect in poorer countries SOURCE: Phelps, Benchmarking to the best in mathematics, Evaluation Review, 2001 Why Standardized Tests? – U. Bucharest

35. What about systemwide, large-scale tests? First priority: do no harm to the formative testing programs in schools and classrooms Why Standardized Tests? – U. Bucharest

36. Besides, systemwide tests are needed for other purposes, such as… …selection to programs with limited number of places …monitoring and system diagnosis …workforce planning …accountability …credentialing That’s enough! Why Standardized Tests? – U. Bucharest

37. Some large-scale test advantages On per-student basis, inexpensive Cognitive laboratory pre-testing possible Standardization offers comparisons across schools and regions. May produce high-quality items that schools and teachers can use. MOST IMPORTANT: provides reliable, comparative information to all those not involved in a particular school Why Standardized Tests? – U. Bucharest

38. Effect of scale on testing achievement benefits from Phelps & Silva (2018) • size of study population • small +0.34 over large • scale of test administration • small-scale +0.14 over large-scale • responsible level of government • local tests +0.29 over state tests Why Standardized Tests? – U. Bucharest

39. Large-scale test, tight security Why Standardized Tests? – U. Bucharest

40. Large-scale test, lax security Why Standardized Tests? – U. Bucharest

41. Why Standardized Tests? – U. Bucharest

42. “Teaching to the Test” Teachers will teach only material that will appear on a standardized test. Rebuttal If high-stakes tests are kept behind lock and key until the day of test administration, teachers will not know what material will be on the test, except in the most general terms. In the absence of common standards and tests, the curriculum becomes arbitrary and of uncertain origin. Why Standardized Tests? – U. Bucharest

43. “Teaching to the Test” Teachers will teach only material that will appear on a standardized test. Counter-argument In the absence of common standards and tests, the curriculum becomes arbitrary and of uncertain origin. Why is that better than teaching to a required curriculum? Why Standardized Tests? – U. Bucharest

44. “Narrowing the Curriculum” A common curriculum prescribed by standards has less content than a teacher-made curriculum. Rebuttal A school year’s fixed number of hours and days render it unlikely that a common curriculum will have less content than a teacher-arbitrary curriculum. I.e., if a teacher drops one topic when standards are introduced, she necessarily has added another. Why Standardized Tests? – U. Bucharest

45. “Narrowing the Curriculum” A common curriculum prescribed by standards has less content than a teacher-made curriculum. Counter-argument What teachers and schools do in the classroom without common standards is not necessarily “broader.” In fact, it can often be “narrower”—governed in the absence of other criteria by personal preferences. Why Standardized Tests? – U. Bucharest

46. Cheating by Students High-stakes standardized testing increases students’ incentive to cheat. Rebuttal Cheating is far easier to prevent and detect with standardized tests. Different forms used in the same classroom can make copying unrewarding. Computer programs run after the fact can look for telltale patterns. Why Standardized Tests? – U. Bucharest

47. Cheating by Students High-stakes standardized testing increases students’ incentive to cheat. Counter-argument Cheating in regular classroom work is common. The overwhelming majority of US students admit to cheating in polls. Teachers and schools are ill-equipped to monitor or detect most cheating. Meanwhile, the Internet makes cheating far easier than in the past. Why Standardized Tests? – U. Bucharest

48. Cheating by Teachers Many teachers have been caught cheating on high-stakes standardized tests. Rebuttal The fact of detection may be evidence of how easily such cheating can be detected. Why Standardized Tests? – U. Bucharest

49. Cheating by Teachers Many teachers have been caught cheating on high-stakes standardized tests. Counter-argument Social promotion and grade inflation provide evidence that nonstandardized testing and grading are far from infallible. And consider that in surveys, the majority of teachers surveyed invariably claim overwhelming pressure to award high grades to undeserving students. Why Standardized Tests? – U. Bucharest

50. Preferred Instructional Methods Classrooms governed by standards are barren, dreary places where only factoids are learned. Rebuttal A curriculum will always rely on some sort of standard or criteria for inclusion. The question is, Do we want formal, open standards, openly arrived at, or should their origins be more obscure or idiosynchratic to each teacher? Why Standardized Tests? – U. Bucharest