1 / 21

Setting a college-readiness cut score for the CRMT

Setting a college-readiness cut score for the CRMT. Background Pilot test summary Current MPT-I placement cut scores Contrasting groups Group discussion. CRMT Working Group August 19, 2009 1:30-3:30. CRMT Background. Original Legislation .

beata
Download Presentation

Setting a college-readiness cut score for the CRMT

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Setting a college-readiness cut score for the CRMT • Background • Pilot test summary • Current MPT-I placement cut scores • Contrasting groups • Group discussion CRMT Working GroupAugust 19, 20091:30-3:30

  2. CRMT Background Original Legislation • Common college readiness test in math for WA public higher education with common performance standard • Fall 2009: high schools must make CRMT available to high school students (subject to funding) Systems Agreements • COP (four-year schools) agreement • SBCTC (two-year schools) agreement

  3. 2008-2009 CRMT Pilot Study 1 Purpose To relate MPT test scores to subsequent course grades to assist in setting CRMT college readiness cut score Research Questions • How does student performance on the MPT-G compare to performance on the MPT-I? • How well do student test scores predict student grades? 1 McGhee D., N. Lowell, J. Gillmore, and J. Peterson (2009) 2009 General Mathematics Placement Test (MPT-G) Pilot , OEA Report 09-03 [http://www.washington.edu/oea/pdfs/reports/OEAReport0903.pdf]

  4. 2008-2009 CRMT Pilot Study Method • Tests administered between October 2008 and June 2009 Edmonds CC (n = 89) Spokane Falls CC (n = 137) • Students at four-year universities took the MPT-G (n = 557) or the MPT-I (n = 692) for placement into courses • Students at other campuses were offered the opportunity to use test scores for placement at four-year universities TESC (n = 34) 21 high schools (n = 2220)

  5. 2008-2009 CRMT Pilot Study Results Test Reliability Both the MPT-G and MPT-I showed excellent internal consistency ( = .84 and .85, respectively) Test Difficulty The MPT-G was more difficult than the MPT-I (Mns = 18.4 and 20.4, respectively; percentage equivalents = 52.6% and 58.3%)

  6. 2008-2009 CRMT Pilot Study Results Test Difficulty (continued) The difference in test difficulty was observed for all three educational sectors, but was most pronounced at four-year institutions Average total score by test type and institution type

  7. 2008-2009 CRMT Pilot Study Results Test Difficulty (continued) Students enrolled or enrolling in college level courses scored 6 points higher than students taking courses below college level Average total score by test type and course level

  8. 2008-2009 CRMT Pilot Study Results Course Grades MPT-G and MPT-I scores were significantly correlated with subsequent math course grades both at high schools and four-year schools (r  .4)

  9. 2008-2009 CRMT Pilot Study Results Course Grades (continued) Students who passed their math course (grade  2.0) tended to have scored significantly higher on the MPT than did students who did not pass Average total score by course level and grade (four-year schools)

  10. 2008-2009 CRMT Pilot Study Results Course Grades (continued) The probability of passing a course generally increased with total test score Observed rates of success in non-precalculus/calculus college level courses as a function of total test score (four-year schools)

  11. 2008-2009 CRMT Pilot Study Conclusions • Both the MPT-G and MPT-I show excellent reliability • Both tests show good discriminant validity in the superior performance of students taking college level math courses over those in pre-college level courses • Both tests show good predictive validity by the significant correlations between total test scores and mathematics course grades

  12. Current Intermediate Math Placement Test (MPT-I) Placement Cutoffs (July 2009)

  13. Contrasting Groups Definition A family of methods to use actual performance data of known groups to set cut scores Process 1. Define two groups (“Masters” and “Non-Masters”) 2. Analyze data to find score(s) that best differentiate(s) between groups a. Select the point of intersection of two frequency distributions (visual inspection) Options b. Find the midpoint between average scores (computation) c. Find score at which probability of group membership is .50 (using logistic regression)

  14. Contrasting Groups Defining the Groups You want the groups to be meaningful, defensible, and clearly distinct (All) high school students vs. (All) post-secondary students Not distinct enough High school Algebra 1 students vs. Post-secondary Calculus students More extreme than necessary

  15. Contrasting Groups Possible Groups 1. any non-college, < 2.0 vs. any college,  2.0 2. any non-college, < 2.0 vs. post-secondary, college,  2.0 3. post-secondary, non-college vs. post-secondary, college 4. post-secondary, non-college, < 2.0 vs. post-secondary, college,  2.0 5. high school, non-college vs. post-secondary, college 6. high school, non-college , < 2.0 vs. post-secondary, college,  2.0

  16. Comparison 1

  17. Comparison 2

  18. Comparison 3

  19. Comparison 4

  20. Comparison 5

  21. Comparison 6

More Related