1 / 17

Beyond Triage: A Randomized Experiment in Sustained Pre-College Advising

Explore the effects of continuous pre-college advising on high school students' enrollment in higher education. Study covers various models and outcomes with a focus on low-income and minority students.

bjustice
Download Presentation

Beyond Triage: A Randomized Experiment in Sustained Pre-College Advising

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. PRELIMINARY. PLEASE DO NOT CITE OR TWEET WITHOUT AUTHORS’ PERMISSION Beyond Triage: A Randomized Experiment in Sustained Pre-College Advising Eric P. BettingerBrent J. Evans With Anthony Antonio, jesse foster, Eileen Horng, riekijima UC Davis – 15 March 2015

  2. Motivation • Non-financial barriers related to information deter students from preparing for and enrolling in higher education, especially among low-income and minority students (Avery & Kane, 2004) • Gauntlets • Financial aid (ACSFA, 2005) • Admission process (Klasik, 2012) • High school counselors are overloaded and cannot meet the demand in their schools. • Huge investment by college access programs to provide information and guide students through the process • TRIO (> $800 million) • Countless small local programs • Models vary substantially

  3. Literature • Information and guidance matter at very specific stages in the process • H&R Block Study (Bettinger, Long, Oreopoulos, & Sanbonmatsu, 2012) • Information and guidance related to financial aid improves aid receipt and college enrollment • 2-8 percentage point effect • Summer Melt Studies (Castleman, Page, & Schooley, 2014; Castleman, Arnold, & Wartman, 2012) • Outreach to students in the summer between high school completion and college enrollment improves likelihood of enrolling • 3-14 percentage point effect • Holistic advising • Mathematica’s Upward Bound Study (Myers et al., 2004) • Primarily academic supports to promote college enrollment, but also provide information • RCT found no effects on college enrollment although some evidence of shift from two-year to four-year • College Possible (Avery, 2013) • Targeted ACT prep, information, and admission support • High-touch intervention-360 hours • Large effects (15 pp) on shifting from two-year to four-year college but no overall enrollment effects • Dartmouth Outreach (Carrell and Sacerdote, 2013)

  4. Need for evidence of different models • Different models of providing information and guidance • Targeted versus school wide • In school versus out of school • Near-peer • By investigating wide range of models, we can start to tease apart mechanisms • Need well identified, causal estimates of efficacy • Randomized controlled trial evidence should be privileged where possible

  5. Advise TX • Part of the College Advising Corps model (active in 15 states 350-400 high schools) • Goal is to help low-income, first-generation college students attend and succeed in college • Recent college graduates serve as full-time near-peer advisers to entire student body at a high school • Full school college advising intervention-concentrate on seniors but some time spent developing college plans for underclassmen • Encourage attendance; provide information; assist with selection of colleges, college applications, and FAFSA completion • Program pilot in 2010-2011 in 15 schools with rapid expansion to over 120 schools in 2011-2012 school year

  6. Experimental Design • School level randomization • Schools invited to apply • > 35% FRL • < 70% graduates attend college within one year (average is 45% in 2009) • < 55% students undertaking a “distinguished” college-prep curriculum • Schools were ranked on these 3 criteria and a qualitative “fit” component • 84 schools automatically selected among 237 who applied • Next 111 schools eligible for random assignment (experimental sample) • 36 randomly selected for treatment assignment, others assigned control status (no program) • Blocked on region: individual lotteries were held within each region (23)

  7. Data • School level data from Texas Education Agency (TEA) • Student level data from Texas Higher Education Coordinating Board (THECB) • 2 years of enrollment outcome data • From 2011-2012 (2012) and 2012-2013 (2013) high school graduating classes • All in-state public postsecondary institutions • Our results are lower bounds of effect of program if the program had any effect on out of state or private enrollments • Outcomes: 2-year and 4-year enrollment and enrollment in Fall after high school graduation Preliminary results from student surveys conducted in treatment/control schools

  8. Treatment-Control Balance

  9. Descriptive Statistics & Balance • Relative to all TX high schools, experimental schools • Are much larger (1683 to 728) • Have a higher percentage of low-income status (64% to 54%) • Have a higher share of minority students (77.5% to 60%) • Have same graduation rate (80%) • Treatment schools relative to control schools • Have more Black and less Hispanic students (8 percentage points difference) • Have slightly more Asian students (1 percentage point difference) • Same graduation rates and college enrollment rates • We control for race in all models to account for this imbalance

  10. Compliance • 5 of 36 treatment schools declined to participate or eventually left the program • 9 of 75 control schools eventually received treatment • Although we had a randomly determined waitlist, it was not consistently adhered to by program staff • E.g. Austin loved the program and offered to pay for all control and treatment schools to participate after the first year. • Treatment on the treated estimates are approximately 33% higher than Intent to treat estimates

  11. Main Impacts

  12. Intent to treat estimates • 1.2 – 1.5 pp impacts on enrolling in fall after high school completion (significant at 10% level) • Impacts are observed in first year of treatment • enrollment effects on two-years become more significant if just examine first year • If we pool the data, small and insignificant effects (~ 1 pp) on overall enrollment • Concentrated at 2-years • Point estimates at 4-years close to 0 • All second year impacts attenuated

  13. Impacts on Subgroups

  14. Subgroup Analysis • Impact on Low-income students • More likely to enroll overall, split between two-years and four-years (2-3 pp) • Impact on Hispanic students • More likely to enroll overall, mostly at two-years (2-3 pp) • Impact on Black students • Less likely to enroll in a two-year (3 pp) • Some evidence of substitution into four-year, but four-year enrollments not significant • Impact on Female students • Less likely than men to enroll in four-year and start at a four-year in fall after high school (1.6 pp)

  15. Survey Sample • Surveys conducted in spring 2014 • All treatment schools (36 + additional randomized group from 2013) • 42 control schools • Treatment/Control Balance • Less representation among Hispanic students (as in the administrative data) • Surveys conducted in spring 2014 • All treatment schools (36 + additional randomized group from 2013) • 42 control schools

  16. Preliminary Survey Results • Key Differences • Degree expectations improve • Plans to work full- or part-time after college or to enter the military decrease • Similarly likely to participate in “soft” college prep activities (college visits, website, test prep) • More applications submitted and greater likelihood of receiving help • More likely to attempt AP exams or to retake college entrance exams • More likely to complete FAFSA (self-reported) • More likely to have submitted a deposit • Less likely to have talked to a high school counselor or teacher about admissions

  17. Discussion • Unsurprising that treatment effects are small at the school level • Advisers likely spend time with inframarginal students who are already going to college as opposed to targeting support at students who need it most • Degree of complementarity is potentially important part of the story • Cost-benefit analysis • Scalability • Future work • Will examine persistence outcomes to observe whether advisers have effect on “fit”

More Related