1 / 54

Predicting Success

Predicting Success. Kris Campbell and Dan Kernler Elgin Community College Elgin, IL. Want to follow along on your device? Download the presentation: http://tinyurl.com/AMATYC-S178. Outline. Part 1: Background

hiroko
Download Presentation

Predicting Success

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Predicting Success Kris Campbell and Dan Kernler Elgin Community College Elgin, IL Want to follow along on your device? Download the presentation: http://tinyurl.com/AMATYC-S178

  2. Outline • Part 1: Background • Part 2: Initial question – Do placement scores, ACT Math sub-scores, and HS GPA predict success in Intermediate Algebra? • Part 3: Do they predict success at the transfer level? • Part 4: What effect does high school coursework have on placement and success?

  3. Part 1: Background

  4. Elgin Community College (ECC) • 18,088 students (Fall 2013) • 69% part-time • Average age is 28

  5. Background – Curriculum Transfer Level Math Courses Mth099 Combined Basic & Intermediate Algebra Mth098Intermediate Algebra Mth097Geometry Mth096Basic Algebra Mth090Pre-Algebra

  6. Background – Curriculum Mth126 Calc. for Business Mth110 Math for Elem. Teach. I Mth112 College Algebra Mth113 Math for Elem. Teaching II Mth114 Trigonometry Mth101 Quantitative Literacy Mth102 Gen. Ed. Statistics Mth133 Calculus I Mth120 Statistics I Mth125 Finite Math Developmental Math Courses

  7. Background – Placement • COMPASS Exam • Written by ACT • Calculators allowed • Can be retaken once • Can place students up through Calculus I • 4 Domains • Branching Pre-Algebra Start Algebra College Algebra Trigonometry

  8. Background – Placement College Algebra Pre-Algebra Algebra College Ready College Ready College Ready 55-100 38-54 Int. Algebra 33-37 Combined Basic & Int. Algebra Basic Algebra 0-32 44-100 BasicAlgebra Basic Algebra 21-43 Pre-Algebra 0-20 Arithmetic Pre-Algebra Arithmetic

  9. Part 2: Initial Question

  10. Background – Initial Data • New students, all recent high school graduates • Fall 2008, 2009, and 2010 only • Valid placement • Highest COMPASS score only • Initially focused on Intermediate Algebra only • Initial analysis considered only three districts due to an extreme weighting system in the fourth

  11. Method • Binomial logistic regression using COMPASS score as predictor for course success (C or better) • Add ACT Math sub-score and HS Cumulative GPA to regression model

  12. Results • COMPASS score was a statistically significant predictor. • High school GPA significantly and dramatically increased the accuracy of the prediction of success. • ACT Math sub-score did not have a unique contribution to the model.

  13. Results COMPASS only (Algebra domain) COMPASS alone is a significant predictor (p<.001). Each additional point on the exam increases the odds of success by 1.08.

  14. Results COMPASS, GPA, and ACT Math sub-score HS GPA is a significant predictor (p<.001), while the ACT Math sub-score is not (p=.901). Each additional point increase in GPA increases the odds of success by nearly 10. (Roughly double the effect of COMPASS once the scale is taken into consideration.)

  15. Results Residuals Recall: residual = observed – predicted Low COMPASS score and GPA, but were successful. High COMPASS score and GPA, but were not successful.

  16. Results Pseudo R2 Recall: R2 is the percent of variation in success explained by the model. COMPASS only Cox & Snell R2 = 0.041 (Algebra domain) Nagelkerke R Square = 0.055 COMPASS, GPA, and Cox & Snell R2= 0.260 ACT Math sub-score NagelkerkeR Square = 0.351

  17. Success by COMPASS 69% success C or better 66% success D, W, or F 57% success 49% success

  18. Success by GPA 30% 92% 10% 53% 81% C or better D, W, or F

  19. C or Better Distribution

  20. C or Better Distribution

  21. C or Better Distribution 85% C or better 39% of the population

  22. B or Better Distribution

  23. B or Better Distribution 72% A/B 12% of the population

  24. C or Better Distribution

  25. C or Better Distribution

  26. Part 3: Transfer Level Success

  27. Transfer Level Success HS GPA is a clear predictor of success in developmental courses. What about at the transfer level?

  28. Transfer Level Success • Consider only introductory transfer level courses: • Mth101 Quantitative Literacy • Mth102 General Education Statistics • Mth110 Math for Elementary Teachers I • Mth112 College Algebra • Focus on students with a COMPASS Algebra score only. • Reported GPA in database (some are weighted)

  29. Success by COMPASS & HS GPA Intermediate Algebra and Introductory Transfer Level Courses C or better D, W, or F 1.5 2.0 2.5 3.0 3.5 4.0

  30. Success by COMPASS & HS GPA Intermediate Algebra and Introductory Transfer Level Courses C or better D, W, or F 1.5 2.0 2.5 3.0 3.5 4.0

  31. Success by COMPASS & HS GPA Intermediate Algebra and Introductory Transfer Level Courses C or better D, W, or F 1.5 2.0 2.5 3.0 3.5 4.0

  32. COMPASS CSEM • COMPASS Reference Manual reports a conditional standard error of measurement (CSEM) • Can be interpreted similarly to confidence intervals • Measures the difference between actual score and expected score if the test were repeated • Depends on the score and test (hence conditional)

  33. COMPASS CSEM

  34. COMPASS CSEM

  35. COMPASS CSEM C or better D, W, or F 1.5 2.0 2.5 3.0 3.5 4.0

  36. COMPASS CSEM C or better D, W, or F +10 GPA: 3.6 COMPASS: 50 -10 1.5 2.0 2.5 3.0 3.5 4.0

  37. COMPASS CSEM C or better D, W, or F 1.5 2.0 2.5 3.0 3.5 4.0

  38. Adjusting Placement • Should we consider high school GPA as a mitigating factor in placement? • A student who places into Intermediate Algebra but has a high HS GPA might do fine in an introductory transfer level course. • Based on similar analysis, a student who places into Basic Algebra but has a high HS GPA might do fine in Combined Basic and Intermediate Algebra.

  39. Background – Placement College Algebra Pre-Algebra Algebra College Ready College Ready 55-100 38-54 Int. Algebra 33-37 Combined Basic &Int. Algebra 0-32 44-100 Basic Algebra Basic Algebra 21-43 0-20 Pre-Algebra Arithmetic

  40. Adjusting Placement – Proposal • Two principles: • Redefine “College Ready” • Accelerate completion of the developmental sequence 55-100 College Ready COMPASS Algebra Score College Ready 38-54 Intermediate Algebra Comb. Basic & Int. Algebra 33-37 Combined Basic & Int. Algebra Basic Algebra 0-32 2.0 2.5 3.0 3.5 Cumulative HS GPA

  41. Adjusting Placement – Fall 2014 College Ready(Eligible for Mth101, Mth102, Mth110, or Mth112) 55-100 College Ready* Intermediate Algebra (Mth098) 38 - 54 COMPASS Algebra Score 33 - 37 Combined Basic and Int. Algebra (Mth099) 0 - 32 Basic Algebra (Mth096) 1.5 2.0 2.5 3.0 3.5 High School Cumulative GPA * Assuming C or better both semesters of a second-year HS algebra course.

  42. Strengths and Weaknesses • Strengths • Research-based • Uses multiple measures • Should improve success and retention • Weaknesses • Complex – not as transparent to students as a single score • Could encourage more student complaints • Still does not include factors that have been shown to be significant predictors

  43. Concerns & Complications • Not all districts report on a 4.0 scale. • One district only reports weights, and has weights up to 6.5 on a 4.0 scale. • How should current students be treated? • Should returning students be under the same placement criteria? • What concerns would you have?

  44. Part 4: High School Coursework

  45. HS Coursework Questions • Does taking a math course senior year predict placement? • Does taking a math course senior year predict success?

  46. Results Does a fourth year math class impact college readiness? p < 0.0001 n = 1112 Note: For those who placed via COMPASS, the comparison is 10% vs 28%.

  47. Results Does a fourth year math class impact college placement ? p < 0.0001 n = 1085

  48. Results • Does a fourth year math class impact success in the first college math course? p =0.006 n = 990 4th Year Math Class

  49. Results • Does a fourth year math class impact success?

  50. Results • Does a fourth year math class impact success? p = 0.06 n = 3193 p = 0.03 n = 34 p = 0.05 n = 55

More Related