Using Institutional Data to Improve Retention and Graduation Rates

1. Using Institutional Data to Improve Retention and Graduation Rates The University of North Carolina at Chapel Hill Lynn Williford, Assistant Provost, Institutional Research & Assessment Presented to the Reinvention Center Annual Meeting October 24, 2013

2. A Decade of Data-Driven Improvements to Retention and Graduation Rates at Carolina • Interest in retention and graduation intensified in early 2000s due to: • Observed disparities in outcomes by race/ethnicity • University system mandate to set improvement targets • Chancellor’s vision • Two formal studies commissioned by the Enrollment Policy Advisory Committee • Ongoing -- a wide variety of projects, short- and long-term, on specific issues or populations, evaluations of special programs, projections. • Undergraduate Education and Institutional Research are primary partners, but shared sense of ownership for retention and graduation exists across campus • “Low Walls” at Carolina enhance information-sharing

3. Phase 1: 2002-04 • Two-year study of factors related to retention and graduation, including • Statistical analyses of factors associated with non-completion: demographic, academic, entry characteristics, financial need, attitudes, academic achievement, etc. • Peer comparisons of policies • Tracked status of “drop-outs,” identified transfer-outs • Special surveys and qualitative analyses of internal documents

4. Process • Working group -- practitioners as well as administrators from Undergraduate Education and Enrollment Management • Advising, Academic Counseling, Minority Affairs, Admissions, Financial Aid, Student Affairs, faculty, student government, guests, etc. • Institutional Research & Assessment held bi-weekly meetings to ask questions, share data, invite “story-telling,” interpret findings • Set the stage for information-sharing discussions across offices that continue today

5. Outcomes of the Study • Some major recommendations that were implemented based on evidence gathered: • Early Warning System • New academic eligibility standards; raised the bar, implemented a new probationary status • More intensive, dedicated support services (a retention coordination office) • Strategies for more regular data gathering and review, development of longitudinal databases

6. Phase 2: 2009-10 • Formal follow-up study to 2004 report to: • Review changes in retention/graduation rates and standing among peers • Report on implementation status of 2004 recommendations • Evaluate the effectiveness (to date) of new programs and policies • Repeat some of the original analyses • Drill down into some of the factors identified in 2004 as significant predictors • Focus on outcomes for specific sub-populations: • Underrepresented minorities, particularly males • First generation

7. Outcomes of Second Study • Validated earlier statistical findings • Heightened recognition of special needs of subpopulations, leading to additional questions and analyses • Confirmed the usefulness of the interventions, which helped protect resources for those services during budget cuts • Identified additional service needs and justified requests for new resources

8. Current Areas of Research • STEM programming initiatives and grant proposals • Diversity issues and campus climate • Alumni perspectives • “Flipped” classrooms • Military veterans experiences and challenges • Transfer students from various origins

9. Further Enhancing the Work of Undergraduate Education and Institutional Research on Retention/Graduation Analyses • Continuing practitioners discussions about specific issues (Brown Bag lunches, etc.) • Re-establishing informal research working group meetings to refine questions, identify priorities, share data resources • Undergraduate Education’s continuing support for Institutional Research: • Improving data collection capability, e.g., coding, numbering courses, flagging student participation in academic activities, etc. • Providing political support for better IT resources and reporting capability • Advocating for staff resources (particularly analysts) for IR

10. Exchanging Retention/Graduation Data with Peers • AAU Data Exchange: • Undergraduate time to degree • Retention and graduation rates • Survey data: • National Survey of Student Engagement (NSSE) • Student in the Research University (SERU) • Ad hoc queries in response to specific questions • Publicly available data: • The Common Data Set • US News & World Report • Descriptive information – provides some clues about why we might observe differences between institutions

11. Potentially Useful Exchanges Between Institutional Research Offices to Promote Retention/Graduation Research • Identifying questions best answered with peer data • Agreeing upon common definitions: • First generation, drop-out, stop-out, STEM majors • Establishing methodology for measuring retention and time to degree for transfer and non-cohort-based students • More work on measuring retention/graduation in individual majors – when does someone become a major? • Developing an AAUDE repository of reports on campus-based retention/graduation research to serve as analytical models for others; eventually for doing some comparisons of results

12. Sharing Successful Practices for: • Collecting individual level data that can be used in longitudinal analyses: • Out-of-class academic experiences • Use of academic support services • Finding stop-outs and making contact to find out why they left and what they need to finish • Techniques for disseminating results publicly (dashboards, etc.) and creating customized reports for specific functional areas