1 / 18

RESEARCH nEWS: Standardized Testing: a Case study

RESEARCH nEWS: Standardized Testing: a Case study. Tim Higgins April 15, 2013. Citation:.

phoebe
Download Presentation

RESEARCH nEWS: Standardized Testing: a Case study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. RESEARCH nEWS:Standardized Testing: a Case study Tim Higgins April 15, 2013

  2. Citation: • Feinman, J. (2008). High Stakes, but Low Validity? A Case Study of Standardized Tests and Admissions into New York City Specialized High Schools. Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/publication/high-stakes-but-low-validity

  3. Great Quote • There are lies, damned lies and statistics. Mark Twain (attributed to)

  4. The Specialized High School Admission Test (SHSAT) • State law (1971) – Admission to specialized public high schools based solely on this test given yearly. • Pool of applicants large, over 25,000 yearly. • Approximately 18% of all test takers were offered a seat at one of the schools with only half admitted to their first choice school. • Test consists of 100 items split between verbal and math • Four versions are given randomly.

  5. The Issues • No study has ever been done that: • Compares SHS admissions procedure with generally accepted psychometric standards and practice. • Examines the statistical properties of the SHSAT itself: distribution of scores, test reliability, confidence intervals around scores, etc. • This study sought to remedy these deficiencies. • Mirrors national debate on standardized testing: • Critics contend that these tests are narrow measures of students’ abilities. • Difficult to distinguish among candidates with similar abilities and biased along racial, class, and gender lines.

  6. Key Findings • Highly regarded and selective NYC public high schools base their admission on only this one test – the SHSAT. • Thousands of students are being denied admission whose scores are statistically indistinguishable from thousands who are admitted. • Unusual scoring feature: high score in one section and poor score in other yields better chance. • The scoring system may not be eliminating differences among the test versions. • This test may contain prediction bias across gender and ethnic groups. • Predictive validity studies are needed to link SHSAT scores to outcomes.

  7. Admissions – Data analysis • Tables and Figures: • A. Table 1 – Summary Statistics, Figure 1:Frequency Distribution – Verbal Raw Scores (2006) • B. Table 2 – Summary Statistics, Figure 2:Frequency Distribution – Math Raw Scores (2006) • C. Grading the Test: Conversion • D. Figure 7: Balanced vs Unbalanced Scorers near cutoff (2006, Stuyvesant only) • Table 4 – Example of missed cut (2006, Stuyvesant only) • Figure 11: SHSAT Results by gender • Figure 12: School Enrollment by Ethnicity

  8. Grading the Test: Conversion to Scaled Scores

  9. Favoring unbalanced scores

  10. Potential Bias

  11. In the news New York Times 3/23/13 Education News 10/1/12

  12. Recommendations • Formal predictive validity studies need to be carried out. At a minimum, these studies should look at the ability of SHSAT scores (separate verbal and math) and middle school grades to predict high school performance. • They should also test for prediction bias across gender and ethnic groups. • If this study concludes that it is best to use additional admissions criteria besides a standardized test, the New York State law—which says that admissions to these schools must be based solely on a test—would need to be changed.

  13. Recommendations continued…. • Findings such as those presented in this study, and the particular choices of admissions procedures for these schools, should be discussed and deliberated in New York. • An informed decision should be made about future practices. Whatever admissions procedures are established, all applicants should know all of their implications. • These findings should also contribute to the broader national debate on standardized tests, school admissions, and high-stakes testing such as exit exams.

  14. Implications – Social Justice

  15. Discussion Questions 1) What could it be about the SHSAT that yields results contrary to most other studies about gender-based intelligences? 2) Is it simply a very difficult test that accomplishes a very competitive mission?

More Related