Presentation Transcript

1. Township High School District 214 The ACT College Readiness Benchmarks and the Fabrication of a Crisis steve.cordogan@d214.org www.d214.org/research
2. What Are theCollege Readiness Benchmarks?(chart below is from an ACT document) The benchmarks are scores on the ACT subject-area tests that represent the level of achievement required for students to have a 50% chance of obtaining a B or higher or about a 75% chance of obtaining a C or higher in corresponding credit-bearing first-year college courses. These college courses include English composition, college algebra, introductory social science courses, and biology. Based on a nationally representative sample of 98 institutions and more than 90,000 students, the Benchmarks are median course placement values for these institutions and as such represent a typical set of expectations. The 2010 ACT College Readiness Benchmarks are:
3. ACT Puts the Benchmarks on Almost All of Its Reports(form excerpt below is from an ACT document)
4. Test Question #1:Which of the FollowingDoes Not Belong? 1 2 3 4 Answer: #4 - there is much less empirical evidence to support believing in the Benchmarks. College Readiness Benchmarks
5. And According to ACT… Only 25%of the students in the United States are ready for college! Only 23% of Illinois students are ready for college!
6. Outrageously Negative News Makes Great Headlines Every year, ACT makes headlines with the failure of students to meet its benchmarks. This year was ACT’s most publicized yet, with an Associated Press article that went out across the country, including NPR, EdWeek, & ASCD SmartBriefs. The story made headlines in every metropolitan area in Illinois on TV, radio and newspapers.
7. So How Badly Are Illinois Schools Performing? Beginning of Universal Testing
8. ACT’s Illinois High School to College Success Report: Benchmarks Defined Only 29.8% of District 214 Students As College-Ready
9. ACT’s Illinois High School to College Success Report:Colleges and Universities Defined74.0% As College Ready
10. ACT’s Illinois High School to College Success Report:And 82.4% Persisted BeyondTheir First Year of College
11. So How Can Only 23% of Illinois Students Be Ready for College?(text below is from an ACT document)
12. How Did ACT Get It So Wrong?Incredibly Flawed Research! ACT used a non-random convenience sample for its study. The only participants were users of an ACT college placement service. The science benchmark was generated from a sample of science majors only! The science test predicts very little in terms of freshman biology class (or any other science class) performance.
13. How Powerful Is the Science Test in Predicting Student Performance?
14. Quoting the National Bureau of Economic Research(in a 2011 Report on Predicting College Performance) “after controlling for Mathematics and English scores, Reading and Science provide essentially no predictive power regarding college outcomes”
15. Quoting The College Board(in a 2010 Report on the Creation ofMultiple-Measure Benchmarks) “Outcomes based solely on one predictor (e.g., test scores or grades) have consistently proven to be inferior to prediction models that employ multiple predictors.” “This research has led us to question the efficacy of any college readiness models based solely on test scores.”
16. Though Also Flawed, the College Board Benchmark Classifies 72% More Students As College Ready Percent of Students Nationally Meeting Each Test Vendor’s College Readiness Benchmark
17. ACT’s Own Research Shows That 65% of All Students Who Met NONE of the Benchmarks Persisted Beyond Their Freshman Year(graph below is from an ACT document)
18. Why Did ACT Do It? ACT is a corporation (albeit a technically not-for-profit one) marketing tests, services, and learning materials ACT has been expanding their market share substantially in recent years. The creation of the perception of a crisis, and the revisiting of that crisis annually, is a great strategy to stay in the eye of the media and establish oneself as an educational watchdog and leader.
19. In Closing… ACT is as good as any large-scale test currently available for evaluating students and schools, but… ACT is a major player in destroying the credibility of our schools and our students. ACT has far overstepped the bounds of legitimate educational research in its efforts to market itself. ACT is guilty of grossly misleading the American public and betraying its trust in them.
20. References ACT College Readiness Benchmarks, Retention, and First-Year College GPA: What’s the Connection? (2005). ACTResearch and Policy Issues. Retrieved from http://www.act.org/research/policymakers/pdf/2005-2.pdf Adams, C. (2011). SAT Scores Fall as Number of Test-Takers Rises. Education Week:31(4). Allen, J. & Sconing, J. (2005). Using ACT Assessment Scores to Set Benchmarks for College Readiness. Research and Policy Issues: ACT Research and Policy Reports. Retrieved from http://www.act.org/research/researchers/reports/pdf/ACT_RR2005-3.pdf Bettinger, E., Evans, B., Pope, D. (2011, June). Improving College Performance and Retention the Easy Way: Unpacking the ACT Exam.National Bureau of Economic Research. Retrieved from http://www.nber.org/papers/w17119 Olsen, J. & House, J. (1997) Student Achievement in Science and Mathematics General Education Courses:An Assessment of the Effects of Academic Background and Noncognitive Variables. Paper presented at the Illinois Association for Institutional Research. Wiley, A., Wyatt, J. & Camara, W. (2010). The Development of a Multidimensional College Readiness Index. College Board Research Report #2010 3. Retrieved from http://professionals.collegeboard.com/profdownload/pdf/10b_2084_DevMultiDimenRR_WEB_100618.pdf