Using Survival Analysis to Better Understand Factors that Determine Student Success

1. Using Survival Analysis to Better Understand Factors that Determine Student Success Russell LongPurdue University Youngkyoung MinThe Korea Foundation for the Advancement of Science and Creativity Guili Zhang East Carolina University Timothy J. AndersonUniversity of Florida Matthew W. Ohland Purdue University

2. Key Research Questions • Does the profile of risk for students leaving engineering differ among cohorts and groups with different cognitive factors (SAT math and verbal scores) and the non-cognitive individual characteristics (gender and ethnicity)? • When are students most likely to leave engineering as a major? • Is SAT score a good predictor of the risk of leaving engineering?

3. Data Source • Longitudinal data • Multiple-Institution Database for Investigating Engineering Longitudinal Development (MIDFIELD) • 1987/88 – 2003/04 • 883,020 student-level records of all undergraduate students enrolled. • 100,179 engineering students.

4. Southeastern MIDFIELD Institutions • Clemson University • Florida Agricultural and Mechanical University • Florida State University • Georgia Institute of Technology • North Carolina Agricultural and Technical State University • North Carolina State University • University of Florida • University of North Carolina Charlotte • Virginia Polytechnic Institute and State University

5. MIDFIELD • 6of the 50 largest U.S. undergraduate engineering programs. • 12% of all U.S. engineering undergraduate degrees. • 1988-2004 cohorts include 20,782 (20.7%) female engineering students. • 25% of all U.S. African-American engineering B.S. degree recipients each year. • Graduation percentage of Hispanics (regardless of gender) is representative of other U.S. programs. • All other ethnic populations are representative of a national sample.

6. Frequency of Gender by Ethnicity

7. Nonparametric Survival Analysis • Class of statistical methods for studying the occurrence and timing of events. • Often applied to death studies – originally used to analyze cancer data. • Engineering literature: reliability or failure time analysis • Better for timed events than multiple regression. • Probability of an event occurring at a particular time.

8. Methodological Definitions • A non-failure: A student who did not leave an engineering major, i.e., a student who either graduated with an engineering degree or is still attending school and had not changed major from engineering to any other non-engineering major. • A failure: A student who left an engineering major, i.e., a student who changed major from engineering to another discipline or left the university. • A major change from one engineering major to another one at the same institution does not constitute failure. • A student who leaves engineering but returns to engineering at the same institution (enrolled or subsequently graduated with an engineering degree) also does not constitute failure.

9. Statistical Methodology • SAS PROC LIFETEST. • Life-table method for large numbers of observation • A student is regarded as censored if he or she does not leave engineering in each time period. • Tests of homogeneity for survival functions: Log-rank tests (Later survival times) and Wilcoxon tests (early survival time) • Hazard functions indicate the risk of loss of engineering students as a function of semester.

10. PROC LIFETEST proclifetest data= dataset name method=lt intervals=(0 to 12by 1) plots=(s,ls,h); time semester*engdrop (0); strata institution/nodetail; run;

11. Life-Table Survival Estimates for Entire Population

12. First-time-in-college Students Matriculating in Engineering

13. First-time-in-college Students Matriculating in Engineering

14. First-time-in-college Students Matriculating in Engineering

15. By Cohort Group

16. By Cohort Group

17. By Gender

18. By Gender

19. By Ethnicity

20. By Ethnicity

21. By SAT Math Score Group

22. By SAT Math Score Group

23. By SAT Verbal Score Group

24. By SAT Verbal Score Group

25. Conclusions • There are no significant differences among cohort subgroups for long survival times, but there are significant differences between cohort subgroups for early survival times, as well as for gender, ethnicity, and SAT math and verbal scores subgroups.  • Females show higher risk of leaving engineering in semesters 3 to 5 than males, while the risks are similar during other semesters.   • White students tend to leave engineering slightly more than Minority students, which leave engineering more than Asians, which leave engineering more than Other students. The Minority and Other categories show an increase in hazard rate for the 9th semester and beyond, possibly related to financial or other pressures. • Except for groups with SAT math <550, engineering college students have the highest hazard rate during the third semester, which in part may due to probationary periods offered in earlier periods.   • SAT math score better predicts the risk of ‘failure’ than SAT verbal score. That is, the lower a student’s SAT math score the more likely that student is to leave engineering.   • Engineering college students with SAT verbal score between 200 and 500 are slightly more likely to survive than the students whose SAT verbal is between 500 and 600.

26. Acknowledgements This material is based on work supported by the National Science Foundation Grant No. REC-0337629 (now DRL- 0729596) and EEC-0646441, funding the Multiple-Institution Database for Investigating Engineering Longitudinal Development (MIDFIELD). The opinions expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation.