Kevin Eagan Gina Garcia Felisha Herrera Juan Garibay Sylvia Hurtado , Principal Investigator

1. Making a Difference in Science Education for Underrepresented Students: The Impact of Undergraduate Research Programs Kevin Eagan Gina Garcia Felisha Herrera Juan Garibay Sylvia Hurtado, Principal Investigator Mitchell Chang, Principal Investigator Higher Education Research Institute, UCLA 2010 AIR Annual Forum Chicago, Illinois

2. Introduction • Graduate enrollment in science and engineering has been increasing • However, URM enrollment continues to lag behind • Proportion of URMs in graduate programs during 2006-2007 academic year • American Indian 0.4% • Black 4.9% • Latina/o 3.6% • STEM completion rates remain low (esp. for URMs) • Huang, Taddese, & Walter (2000) • Higher Education Research Institute (2010)

3. Purpose • To examine the effects of undergraduate research programs on students’ intentions to enroll in graduate school through the use of advance statistical techniques on multi-institutional data. • Propensity score matching

4. Background • Graduate School Enrollment • Prior academic achievement • Race/socioeconomic status • Parent education • Institutional selectivity • Level of involvement • Student faculty interaction

5. Background • Benefits of undergraduate research programs

6. Conceptual Framework Social and Cultural Capital • Capital inherited through social position and family background • Social capital acquired in college complements the capital that students bring with them Science Identity • Fostering knowledge growth • Opportunities to display scientific knowledge & practices • Acknowledgement of being a science person

7. Research Questions • What pre-college experiences and characteristics of entering college students predict their likelihood of participating in a structured undergraduate research program during college? • After accounting for students’ chances of participating in an undergraduate research program, what effect does participation in such a program have on students’ intention to enroll in graduate/professional school, particularly in a STEM field?

8. Methods: Sample • CIRP Longitudinal Sample (n=4,212) • 2004 Freshman Survey (TFS) • 2008 College Senior Survey (CSS) • Targeted institutions: • Strong reputations in STEM graduation rates • Undergraduate research programs funded by NSF and NIH • Minority-serving institutions

9. Methods: Variables • DV: 3-part variable representing post-college intentions: • Enroll in graduate/professional STEM program • Enroll in graduate/professional non-STEM program • No intentions to pursue graduate/professional degree • IVs • Undergraduate research participation • Science identity • Career focus in 2008 • College GPA • College experiences • Pre-college preparation • Demographics • Institutional characteristics

10. Methods: Analyses • Missing data • Propensity score matching • Discussion of the counterfactual • Estimation of the propensity score related to participation in an undergraduate research program • Multinomial hierarchical generalized linear modeling

11. Methods: Analyses • Issues of selection bias/endogeneity • Counterfactual framework • “a potential outcome, or the state of affairs that would have happened in the absence of the cause” (Guo & Fraser, 2010, p. 24) • Comparing a “treated” individual with a “non-treated” individual • Propensity score estimation

12. Methods: Analyses • Reweighting of the data with derivations of the propensity score • Average treatment effect • Average treatment of the untreated (ATU) effect • Average treatment of the treated (ATT) effect • Multinomial hierarchical generalized linear modeling

13. Limitations • Secondary data analysis • Limited DV: intentions and combination of graduate and professional school • Unobservable variables affecting undergraduate research participation • Weighting adjustment using propensity score rather than matching by propensity score

14. Findings: Predictors of Participating in Undergraduate Research Programs • Major: Physical sciences (10.77%), Life sciences (7.34%), Health sciences (4.90%) • Race: Black (5.71%) • Participated in pre-college research program (4.03%) • Degree aspiration in 2004: Ph.D. (3.54%) • Composite SAT score (100-point change): 2.27%

15. Findings: Effects of Undergraduate Research Program Participation on Graduate/Professional School Enrollment Intentions

16. Discussion • Confirmation of results from prior studies • “Effect” is much more modest than prior studies might suggest • UG research programs attract students who already identify as scientists • Average treatment of the untreated (ATU) effect • Expand the reach of these programs • Ensure programs not only harvest talent but develop it, too

17. Conclusion and Directions for Future Research • Follow these students into graduate school and examine matriculation patterns • Investigate via qualitative methods the quality of students’ research experiences • UG research programs as wise investments

18. Contact Information Faculty and Co-PIs: Sylvia Hurtado Mitchell Chang Postdoctoral Scholars: Kevin Eagan Josephine Gasiewski Administrative Staff: Aaron Pearl Graduate Research Assistants: Christopher Newman Minh Tran Jessica Sharkness Monica Lin Gina Garcia Felisha Herrera Cindy Mosqueda Juan Garibay Papers and reports are available for download from project website: http://heri.ucla.edu/nih Project e-mail: herinih@ucla.edu Acknowledgments: This study was made possible by the support of the National Institute of General Medical Sciences, NIH Grant Numbers 1 R01 GMO71968-01 and R01 GMO71968-05 as well as the National Science Foundation, NSF Grant Number 0757076. This independent research and the views expressed here do not indicate endorsement by the sponsors.