1 / 34

The effect of testing on student achievement: 1910-2010

Comprehensive meta-analysis of the impact of testing on student achievement over 100 years, including quantitative and qualitative studies, effect sizes, and moderators that influence outcomes.

mvue
Download Presentation

The effect of testing on student achievement: 1910-2010

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The effect of testing on student achievement: 1910-2010 Richard P. PHELPS International Test Commission, 8th Conference, Amsterdam, July, 2012

  2. Meta-analysis A method for summarizing a large research literature, with a single, comparable measure. International Test Commission, 8th Conference, Amsterdam, July, 2012

  3. The effect of testing on student achievement 12-year long study analyzed close to 700 separate studies, and more than 1,600 separate effects 2,000 other studies were reviewed and found incomplete or inappropriate lacking sufficient time and money, hundreds of other studies will not be reviewed International Test Commission, 8th Conference, Amsterdam, July, 2012

  4. Looking for studies to include in the meta-analyses Included only those studies that found an effect from testing on student achievement or on teacher instruction… International Test Commission, 8th Conference, Amsterdam, July, 2012

  5. Studies included in the meta-analyses …when: a test is newly introduced, or newly removed quantity of testing is increased or reduced test stakes are introduced or increased, or removed or reduced International Test Commission, 8th Conference, Amsterdam, July, 2012

  6. Studies included in the meta-analyses3. …plus previous research summaries (e.g.) Kulik, Kulik, Bangert-Drowns, & Schwalb (1983-1991) on: mastery testing, frequency of testing, and programs for high-risk university students Basol & Johanson (2009) on testing frequency Jaekyung Lee (2007) on cross-state studies W.J. Haynie (2007) in career-tech ed International Test Commission, 8th Conference, Amsterdam, July, 2012

  7. Number of studies of effects, by methodology type International Test Commission, 8th Conference, Amsterdam, July, 2012

  8. Effect size: Cohen’s d d = (YE - YC) / Spool YE = mean, experimental group YC = mean, control group Spooled = standard deviation International Test Commission, 8th Conference, Amsterdam, July, 2012

  9. Effect size: Other formulae d= t*((n1+n2/n1*n2)^0.5 d= 2r/(1-r²)^0.5 d= (YE pre-YE post-YC pre+ YC post)/Spooled post International Test Commission, 8th Conference, Amsterdam, July, 2012

  10. Effect size: Interpretation d between 0.25 & 0.50  weak effect d between 0.50 et 0.75  medium effect d more than 0.75  strong effect International Test Commission, 8th Conference, Amsterdam, July, 2012

  11. Quantitative studies(population coverage ≈ 7 million persons) International Test Commission, 8th Conference, Amsterdam, July, 2012

  12. Quantitative studies: Effect size “Bare bones” calculation: d ≈ +0.55 …a medium effect Bare bones effect size adjusted for measurement error d ≈ +0.71 …a stronger effect Using same-study-author aggregation d ≈ +0.88 …a strong effect International Test Commission, 8th Conference, Amsterdam, July, 2012

  13. Which predictors matter? International Test Commission, 8th Conference, Amsterdam, July, 2012

  14. More Moderators – Source of Test International Test Commission, 8th Conference, Amsterdam, July, 2012

  15. More Moderators – Sponsor of Test International Test Commission, 8th Conference, Amsterdam, July, 2012

  16. More Moderators - Study Design International Test Commission, 8th Conference, Amsterdam, July, 2012

  17. More Moderators – Scale of Analysis International Test Commission, 8th Conference, Amsterdam, July, 2012

  18. More Moderators – Scale of Administration International Test Commission, 8th Conference, Amsterdam, July, 2012

  19. Surveys and opinion polls International Test Commission, 8th Conference, Amsterdam, July, 2012

  20. Percentage of survey items, by respondent group and type of survey International Test Commission, 8th Conference, Amsterdam, July, 2012

  21. Number and percent of survey items,by test stakes and target group International Test Commission, 8th Conference, Amsterdam, July, 2012

  22. Opinion polls, by year • 244 between 1958--2008, in the U.S. & Canada • 813 unique question-response combinations • close to 700,000 individual respondents International Test Commission, 8th Conference, Amsterdam, July, 2012

  23. Surveys and opinion polls: Regular standardized tests, performance tests International Test Commission, 8th Conference, Amsterdam, July, 2012

  24. Qualitative studies: Summary (One cannot calculate an effect size.) International Test Commission, 8th Conference, Amsterdam, July, 2012

  25. Qualitative studies, by methodology type International Test Commission, 8th Conference, Amsterdam, July, 2012

  26. Qualitative studies: Effect on student achievement 244 studies conducted in the past century in over 30 countries International Test Commission, 8th Conference, Amsterdam, July, 2012

  27. Qualitative studies: Testing improves student achievement and teacher instruction International Test Commission, 8th Conference, Amsterdam, July, 2012

  28. Qualitative studies: Variation by rigor and test stakes International Test Commission, 8th Conference, Amsterdam, July, 2012

  29. Qualitative studies: Regular standardized tests and performance tests International Test Commission, 8th Conference, Amsterdam, July, 2012

  30. An enormous research literature But, assertions that it does not exist at all are common Some claims are made by those who oppose standardized testing, and may be wishful thinking Others are “firstness” claims International Test Commission, 8th Conference, Amsterdam, July, 2012

  31. Dismissive research reviews With a dismissive research literature review, a researcher assures all that no other researcher has studied the same topic International Test Commission, 8th Conference, Amsterdam, July, 2012

  32. Firstness claims With a firstness claim, a researcher insists that he or she is the first to ever study a topic International Test Commission, 8th Conference, Amsterdam, July, 2012

  33. Social costs are enormous Research conducted by those without power or celebrity is dismissed -- ignored and lost Public policies are skewed, based exclusively on the research results of those with power or celebrity Society pays again and again for research that has already been done International Test Commission, 8th Conference, Amsterdam, July, 2012

  34. The effect of testing on student achievement: 1910-2010 Richard P. PHELPS International Test Commission, 8th Conference, Amsterdam, July, 2012

More Related