1 / 34

The effect of testing on student achievement: 1910-2010

World Association of Education Research, 17th Congress, Reims, June, 2012<br>

richardpphelps
Download Presentation

The effect of testing on student achievement: 1910-2010

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The effect of testing on student achievement: 1910-2010 Richard P. PHELPS World Association of Education Research, 17th Congress, Reims, June, 2012

  2. Meta-analysis A method for summarizing a large research literature, with a single, comparable measure. World Association of Education Research, 17th Congress, Reims, June, 2012

  3. The effect of testing on student achievement 12-year long study analyzed close to 700 separate studies, and more than 1,600 separate effects 2,000 other studies were reviewed and found incomplete or inappropriate lacking sufficient time and money, hundreds of other studies will not be reviewed World Association of Education Research, 17th Congress, Reims, June, 2012

  4. Looking for studies to include in the meta-analyses Included only those studies that found an effect from testing on student achievement or on teacher instruction… World Association of Education Research, 17th Congress, Reims, June, 2012

  5. Studies included in the meta-analyses …when: a test is newly introduced, or newly removed quantity of testing is increased or reduced test stakes are introduced or increased, or removed or reduced World Association of Education Research, 17th Congress, Reims, June, 2012

  6. Studies included in the meta-analyses3. …plus previous research summaries (e.g.) Kulik, Kulik, Bangert-Drowns, & Schwalb (1983-1991) on: mastery testing, frequency of testing, and programs for high-risk university students Basol & Johanson (2009) on testing frequency Jaekyung Lee (2007) on cross-state studies W.J. Haynie (2007) in career-tech ed World Association of Education Research, 17th Congress, Reims, June, 2012

  7. Number of studies of effects, by methodology type World Association of Education Research, 17th Congress, Reims, June, 2012

  8. Effect size: Cohen’s d d = (YE - YC) / Spool YE = mean, experimental group YC = mean, control group Spooled = standard deviation World Association of Education Research, 17th Congress, Reims, June, 2012

  9. Effect size: Other formulae d= t*((n1+n2/n1*n2)^0.5 d= 2r/(1-r²)^0.5 d= (YE pre-YE post-YC pre+ YC post)/Spooled post World Association of Education Research, 17th Congress, Reims, June, 2012

  10. Effect size: Interpretation d between 0.25 & 0.50  weak effect d between 0.50 et 0.75  medium effect d more than 0.75  strong effect World Association of Education Research, 17th Congress, Reims, June, 2012

  11. Quantitative studies(population coverage ≈ 7 million persons) World Association of Education Research, 17th Congress, Reims, June, 2012

  12. Quantitative studies: Effect size “Bare bones” calculation: d ≈ +0.55 …a medium effect Bare bones effect size adjusted for measurement error d ≈ +0.71 …a stronger effect Using same-study-author aggregation d ≈ +0.88 …a strong effect World Association of Education Research, 17th Congress, Reims, June, 2012

  13. Which predictors matter? World Association of Education Research, 17th Congress, Reims, June, 2012

  14. More Moderators – Source of Test World Association of Education Research, 17th Congress, Reims, June, 2012

  15. More Moderators – Sponsor of Test World Association of Education Research, 17th Congress, Reims, June, 2012

  16. More Moderators - Study Design World Association of Education Research, 17th Congress, Reims, June, 2012

  17. More Moderators – Scale of Analysis World Association of Education Research, 17th Congress, Reims, June, 2012

  18. More Moderators – Scale of Administration World Association of Education Research, 17th Congress, Reims, June, 2012

  19. Surveys and opinion polls World Association of Education Research, 17th Congress, Reims, June, 2012

  20. Percentage of survey items, by respondent group and type of survey World Association of Education Research, 17th Congress, Reims, June, 2012

  21. Number and percent of survey items,by test stakes and target group World Association of Education Research, 17th Congress, Reims, June, 2012

  22. Opinion polls, by year • 244 between 1958--2008, in the U.S. & Canada • 813 unique question-response combinations • close to 700,000 individual respondents World Association of Education Research, 17th Congress, Reims, June, 2012

  23. Surveys and opinion polls: Regular standardized tests, performance tests World Association of Education Research, 17th Congress, Reims, June, 2012

  24. Qualitative studies: Summary (One cannot calculate an effect size.) World Association of Education Research, 17th Congress, Reims, June, 2012

  25. Qualitative studies, by methodology type World Association of Education Research, 17th Congress, Reims, June, 2012

  26. Qualitative studies: Effect on student achievement 244 studies conducted in the past century in over 30 countries World Association of Education Research, 17th Congress, Reims, June, 2012

  27. Qualitative studies: Testing improves student achievement and teacher instruction World Association of Education Research, 17th Congress, Reims, June, 2012

  28. Qualitative studies: Variation by rigor and test stakes World Association of Education Research, 17th Congress, Reims, June, 2012

  29. Qualitative studies: Regular standardized tests and performance tests World Association of Education Research, 17th Congress, Reims, June, 2012

  30. An enormous research literature But, assertions that it does not exist at all are common Some claims are made by those who oppose standardized testing, and may be wishful thinking Others are “firstness” claims World Association of Education Research, 17th Congress, Reims, June, 2012

  31. Dismissive research reviews With a dismissive research literature review, a researcher assures all that no other researcher has studied the same topic World Association of Education Research, 17th Congress, Reims, June, 2012

  32. Firstness claims With a firstness claim, a researcher insists that he or she is the first to ever study a topic World Association of Education Research, 17th Congress, Reims, June, 2012

  33. Social costs are enormous Research conducted by those without power or celebrity is dismissed -- ignored and lost Public policies are skewed, based exclusively on the research results of those with power or celebrity Society pays again and again for research that has already been done World Association of Education Research, 17th Congress, Reims, June, 2012

  34. The effect of testing on student achievement: 1910-2010 Richard P. PHELPS World Association of Education Research, 17th Congress, Reims, June, 2012

More Related