NSSE: A Window into Student and Institutional Performance George Kuh University of Maine Trustees

1. NSSE: A Window into Student and Institutional Performance George Kuh University of Maine Trustees Farmington ME March 19, 2007

2. Commission on Future of Higher Education • Improve student academic preparation and make more financial aid available • Simplify & restructure student financial aid system using incentives to control costs • Cultivate a "robust culture of accountability & transparency“: invent new student learning measures, make comparable information public, focus accreditation on outcomes • Embrace continuous innovation and quality improvement. • Target federal investments to critical areas such as math, science, and foreign languages. • Develop a national strategy for lifelong learning

3. Association of American Colleges and Universities

4. We value what we measure Wise decisions are needed about what to measure in the context of campus mission, values, and desired outcomes.

5. Overview • Why student engagement matters • What we’ve learned from NSSE • Unsolicited advice for trustees • Conversation

6. Student Success in College Academic achievement, engagement in educationally purposeful activities, satisfaction, acquisition of desired knowledge, skills and competencies, persistence, attainment of educational objectives, and post-college performance

7. Academic preparation Ability and college-level skills Family education and support Financial wherewithal Pre-college Characteristics Associated with Student Success

8. 32.N162FG12 Postsecondary Education OPPORTUNITY Association for Institutional Research – May 16, 2006 – Chicago, Illinois

9. Goal realization Psycho-social fit Credit hours completed Academic and social support Involvement in the “right” kinds of activities Early College Indicators of Persistence and Success

10. Factors That Threaten Persistence and Graduation from College • academically underprepared for college-level work • first-generation college student • gap between high school and college • 30+ hours working per week • part-time enrollment • single parent • financially independent • children at home

11. What Really Matters in College: Student Engagement Because individual effort and involvement are the critical determinants of impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement. Pascarella & Terenzini, How College Affects Students, 2005, p. 602

12. Foundations of Student Engagement Time on task (Tyler, 1930s) Quality of effort (Pace, 1960-70s) Student involvement (Astin, 1984) Social, academic integration (Tinto,1987, 1993) Good practices in undergraduate education (Chickering & Gamson, 1987) College impact (Pascarella, 1985) Student engagement (Kuh, 1991, 2005)

13. Good Practices in Undergraduate Education(Chickering & Gamson, 1987; Pascarella & Terenzini, 2005) • Student-faculty contact • Active learning • Prompt feedback • Time on task • High expectations • Experiences with diversity • Cooperation among students

14. The Student Engagement Trinity • What students do -- time and energy devoted to educationally purposeful activities • What institutions do -- using effective educational practices to induce students to do the right things • Educationally effective institutions channel student energy toward the right activities

15. NSSE’s Survey InstrumentThe College Student Report Student Behaviors Student Learning & Development Institutional Actions & Requirements Reactions to College Student Background Information

16. National Survey of Student Engagement(pronounced “nessie”)Community College Survey of Student Engagement(pronounced “cessie”) College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development

17. Effective Educational Practices Level of Academic Challenge Active & Collaborative Learning Student Faculty Interaction Supportive Campus Environment Enriching Educational Experiences

18. NSSE Project Scope • 1,000,000+ students from 1,100+ different schools • 80% of 4-yr U.S. undergraduate FTE • 50 states, Puerto Rico, Canada • 70+ consortia

19. NSSE Core Purposes Institutional Improvement Public Advocacy Documenting Good Practice

20. Customized Institutional Report • Overview • Institutional data • Means and frequencies • 1st year students and seniors • Comparisons by peers, Carnegie, national • National benchmarks • Data use tips • CD with raw data, etc. • And more!

21. What have we learned so far?

22. Grades, persistence, student satisfaction, and engagement go hand in hand

23. Does institutional size matter to engagement? Yes, size matters. Smaller isgenerally better.

24. Benchmark Scores for All Students by Undergraduate Enrollment

25. Academic Challenge, Active Learning, Student-Faculty Interaction by Enrollment

26. Student engagement varies more within than between institutions.

27. Supportive Campus Environment: Seniors at Master's Institutions Percentile 10 Percentile 50 Percentile 90 100 80 60 40 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Master's Institutions

28. Academic Challenge by Institutional Type 75 75 Seniors First-Year Students 70 70 65 65 60 60 55 55 50 50 Benchmark Scores 45 45 40 40 35 35 30 30 25 25 Bac LA Bac Gen Doc Ext Doc Int Bac LA Bac Gen Nation Doc Ext Doc Int MA MA Nation

29. Behold the compensatory effects of engagement

33. Association of American Colleges and Universities

34. Narrow Learning is Not Enough—The Essential Learning Outcomes • Knowledge of Human Cultures and the Physical & Natural World • Intellectual and Practical Skills • Personal and Social Responsibility • Integrative Learning

35. Integrating ideas or information from various sources Included diverse perspectives in class discussions/writing Put together ideas from different courses Discussed ideas with faculty members outside of class Discussed ideas with others outside of class Analyzing the basic elements of an idea, experience, or theory Essential Learning Outcome:Integrative Learning NSSE DEEP LEARNING SCALE • Synthesizing & organizing ideas, info., or experiences • Making judgments about the value of information • Applying theories to practical problems or in new situations • Examined the strengths and weaknesses of your own views • Tried to better understand someone else's views • Learned something that changed how you understand an issue

36. Essential Learning Outcome:Intellectual and Practical Skills NSSE SELF-REPORTED GAINS • Acquiring job or work-related knowledge and skills • Writing clearly and effectively • Speaking clearly and effectively • Thinking critically and analytically • Analyzing quantitative problems • Using computing and information technology • Working effectively with others • Solving complex real-world problems

37. Essential Learning Outcome:Personal & Social Responsibilities NSSE SELF-REPORTED GAINS • Voting in local, state, or national elections • Learning effectively on your own • Understanding yourself • Understanding people of other racial and ethnic backgrounds • Developing a personal code of values and ethics • Contributing to the welfare of your community • Developing a deepened sense of spirituality

38. Very much Quite a bit Some Very little

39. Very much Quite a bit Some Very little

40. Effective Educational Practices • First-Year Seminars and Experiences  • Common Intellectual Experiences • Learning Communities • Writing-Intensive Courses • Collaborative Assignments and Projects • “Science as Science Is Done”; Undergraduate Research • Diversity/Global Learning • Service Learning, Community-Based Learning • Internships • Capstone Courses and Projects

41. First-year Senior Standardized Y- Standardized Y- Regression Standardized Regression Standardized Coefficient Sig. Effect size Coefficient Sig. Effect size Engagement Activities .16 .32 .12 .28 Academic Efforts *** *** .20 .40 .15 .35 Higher Order Thinking *** *** .19 .39 .16 .38 Academic Integration *** *** .26 .53 .24 .54 Active and Collaborative Learning *** *** .30 .60 .22 .51 Interactions with Faculty *** *** .21 .41 .16 .36 Diversity Experiences *** *** Perception of Campus Environment .12 .23 .08 .17 Quality of Academic Advising *** *** .19 .37 .14 .32 Supportive Campus Environment *** *** .13 .25 .10 .23 Satisfaction *** *** Learning Outcomes .24 .48 .18 .40 Gains in Personal and Social *** *** .22 .45 .16 .36 Gains in Practical Competence *** *** .18 .36 .11 .24 General Education Gains *** *** Effects of Learning Communities on Engagement

43. Faculty Time Devoted to Research and Probability of Students Doing Research 0.50 0.38 Probability of Undergraduate Research 0.25 0.13 0 2.02 6.04 10.07 14.09 18.11 Faculty Research (hours)

44. Faculty View of Importance of Doing Research with Students and Probability of Students Doing Research 0.50 0.38 0.25 Probability of Undergraduate Research 0.13 0 1.87 2.25 2.62 2.99 3.37 Importance of Undergraduate Research

45. AVG STUDENT Acad emic Act ive - Diversity Stu dent - AVG FACULTY c hall enge c ollab e xperiences f ac ulty A c ad emic chall enge ü ü ü emph asis Act ive - collab ü ü ü ü practices Emph asis on diversity ü ü ü experiences Emph asis on h igher ü ü ü order thinking Imp ortance enriching ü ü ü ed uc e xp eriences Faculty Priorities and Student Engagement

46. What to Make of This? • When faculty members emphasize certain educational practices, students engage in them to a greater extent than their peers elsewhere. • Good things go together

47. Effective Educational Practices • First-Year Seminars and Experiences  • Common Intellectual Experiences • Learning Communities • Writing-Intensive Courses • Collaborative Assignments and Projects • “Science as Science Is Done”; Undergraduate Research • Diversity/Global Learning • Service Learning, Community-Based Learning • Internships • Capstone Courses and Projects

48. Effective Educational Practices Increase Odds That Students Will: • Invest time and effort • Interact with faculty and peers about substantive matters • Experience diversity • Get more frequent feedback • Discover relevance of their learning through real-world applications

49. Institutional Reflection Areas of Effective Educational Practice Areas of Question or Improvement

50. Merge NSSE data with school records Identify engagement patterns by student characteristics Predict retention, degree attainment, grades, other outcomes Track student engagement year to year Compare vs. peer, aspirational, same-region schools Program evaluation Accreditation Consortium and system data sharing Student outcomes research Public reporting A national reporting template?!? Possibilities