Our research on what students are learning in college

1. Our research on what students are learning in college • Wabash National Study • 17,000 students at 49 colleges and universities • Longitudinal, following students for four years • Students entered in 2006, 2007, and 2008 • Multiple measures of outcomes, experiences, academic performance • Critical thinking, moral reasoning, openness to and interest in diversity, political and social involvement, etc. • Two parts to the study • Research on student learning • Helping institutions use Wabash Study data to improve student learning

2. Findings to date • Students don’t improve enough over four years of college • Moderate improvement on five outcomes, no change or decline on seven • Well known “good practices” have an impact, but are not pervasive • Institutions have data, but most are not able to use it for improving student learning

3. The perfect evaluation system • Includes both measures of student learning and the environment in which students learn, including what teachers are doing • Longitudinal • More time, expertise, and energy devoted to using the data for improvement

4. What findings or conclusions about evaluating learning have been reported thoroughly and accurately and which ones have been misconstrued or misleadingly presented? • Overall good, but occasional problems with important details • Like the banking crises - sometimes arcane facts matter • Huffington Post versus Academically Adrift • "45% of students don't learn much in college" • “we observe no statistically significant gains in critical thinking, complex reasoning, and writing skills for at least 45% of the students in our study”.

5. What findings or conclusions about evaluating learning have been reported thoroughly and accurately and which ones have been misconstrued or misleadingly presented? • 4/6/2011 Inside Higher Ed “NSSE’s Validity Questioned” • "A trustee," they write, "could be listening to a glowing report about how well her institution is doing in terms of student engagement, not realizing that high NSSE scores might actually be negatively associated with outcomes she considers desirable for the institution and its students. After all, doesn't engagement take away from the 'time on task' that a student would normally devote to coursework and studying?" Such a statement is consistent with the message of the new book Academically Adrift, which argues that academic rigor, not engagement, is the key to improving student learning. • Reports on VSA/K-12 high-stakes testing that do consider whether the tests actually measure change

6. What questions do you think reporters are missing in their coverage of postsecondary accountability and student learning? • Accountability fallacy? • Will things change once the evidence is public? • Do institutions know how to change? • Change • How much change do we want? • From a cognitive science standpoint, how much change is realistic in four years?

7. More information • About the Wabash Study • http://www.centerofinquiry.org • About the Good Practices • http://www.centerofinquiry.org/study-research/ • About the Teagle Assessment Scholar Program • http://www.centerofinquiry.org/assessment-scholars/ • About the Higher Education Data Sharing Consortium • www.hedsconsortium.org