NSSE: Retrospective and Prospective George Kuh AAC&U General Education and Assessment Conference

1. NSSE: Retrospective and Prospective George Kuh AAC&U General Education and Assessment Conference March 2, 2007

2. National Survey of Student Engagement(pronounced “nessie”)Community College Survey of Student Engagement(pronounced “cessie”) College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development

3. We value what we measure Wise decisions are needed about what to measure in the context of campus mission, values, and desired outcomes.

4. Overview • Why student engagement matters • What we’ve learned • Conversation

5. What Really Matters in College: Student Engagement Because individual effort and involvement are the critical determinants of impact, institutions should focus on the ways they can shape their academic, interpersonal, and extracurricular offerings to encourage student engagement. Pascarella & Terenzini, How College Affects Students, 2005, p. 602

6. Foundations of Student Engagement Time on task (Tyler, 1930s) Quality of effort (Pace, 1960-70s) Student involvement (Astin, 1984) Social, academic integration (Tinto,1987, 1993) Good practices in undergraduate education (Chickering & Gamson, 1987) College impact (Pascarella, 1985) Student engagement (Kuh, 1991, 2005)

7. Good Practices in Undergraduate Education(Chickering & Gamson, 1987; Pascarella & Terenzini, 2005) • Student-faculty contact • Active learning • Prompt feedback • Time on task • High expectations • Experiences with diversity • Cooperation among students

8. The Student Engagement Trinity • What students do -- time and energy devoted to educationally purposeful activities • What institutions do -- using effective educational practices to induce students to do the right things • Educationally effective institutions channel student energy toward the right activities

9. NSSE’s Survey InstrumentThe College Student Report Student Behaviors Student Learning & Development Institutional Actions & Requirements Reactions to College Student Background Information

10. Effective Educational Practices Level of Academic Challenge Active & Collaborative Learning Student Faculty Interaction Supportive Campus Environment Enriching Educational Experiences

11. NSSE Project Scope • 1,000,000+ students from 1,100+ different schools • 80% of 4-yr U.S. undergraduate FTE • 50 states, Puerto Rico, Canada • 70+ consortia

12. NSSE Core Purposes Institutional Improvement Public Advocacy Documenting Good Practice

13. Customized Institutional Report • Overview • Institutional data • Means and frequencies • 1st year students and seniors • Comparisons by peers, Carnegie, national • National benchmarks • Data use tips • CD with raw data, etc. • And more!

14. What have we learned so far?

15. Student engagement varies more within than between institutions.

16. Supportive Campus Environment: Seniors at Master's Institutions Percentile 10 Percentile 50 Percentile 90 100 80 60 40 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Master's Institutions

17. Academic Challenge by Institutional Type 75 75 Seniors First-Year Students 70 70 65 65 60 60 55 55 50 50 Benchmark Scores 45 45 40 40 35 35 30 30 25 25 Bac LA Bac Gen Doc Ext Doc Int Bac LA Bac Gen Nation Doc Ext Doc Int MA MA Nation

18. Behold the compensatory effects of engagement

22. Association of American Colleges and Universities

23. Narrow Learning is Not Enough—The Essential Learning Outcomes • Knowledge of Human Cultures and the Physical & Natural World • Intellectual and Practical Skills • Personal and Social Responsibility • Integrative Learning

24. Integrating ideas or information from various sources Included diverse perspectives in class discussions/writing Put together ideas from different courses Discussed ideas with faculty members outside of class Discussed ideas with others outside of class Analyzing the basic elements of an idea, experience, or theory Essential Learning Outcome:Integrative Learning NSSE DEEP LEARNING SCALE • Synthesizing & organizing ideas, info., or experiences • Making judgments about the value of information • Applying theories to practical problems or in new situations • Examined the strengths and weaknesses of your own views • Tried to better understand someone else's views • Learned something that changed how you understand an issue

25. Essential Learning Outcome:Intellectual and Practical Skills NSSE SELF-REPORTED GAINS • Acquiring job or work-related knowledge and skills • Writing clearly and effectively • Speaking clearly and effectively • Thinking critically and analytically • Analyzing quantitative problems • Using computing and information technology • Working effectively with others • Solving complex real-world problems

26. Essential Learning Outcome:Personal & Social Responsibilities NSSE SELF-REPORTED GAINS • Voting in local, state, or national elections • Learning effectively on your own • Understanding yourself • Understanding people of other racial and ethnic backgrounds • Developing a personal code of values and ethics • Contributing to the welfare of your community • Developing a deepened sense of spirituality

27. Very much Quite a bit Some Very little

28. Very much Quite a bit Some Very little

29. Effective Educational Practices • First-Year Seminars and Experiences  • Common Intellectual Experiences • Learning Communities • Writing-Intensive Courses • Collaborative Assignments and Projects • “Science as Science Is Done”; Undergraduate Research • Diversity/Global Learning • Service Learning, Community-Based Learning • Internships • Capstone Courses and Projects

30. First-year Senior Standardized Y- Standardized Y- Regression Standardized Regression Standardized Coefficient Sig. Effect size Coefficient Sig. Effect size Engagement Activities .16 .32 .12 .28 Academic Efforts *** *** .20 .40 .15 .35 Higher Order Thinking *** *** .19 .39 .16 .38 Academic Integration *** *** .26 .53 .24 .54 Active and Collaborative Learning *** *** .30 .60 .22 .51 Interactions with Faculty *** *** .21 .41 .16 .36 Diversity Experiences *** *** Perception of Campus Environment .12 .23 .08 .17 Quality of Academic Advising *** *** .19 .37 .14 .32 Supportive Campus Environment *** *** .13 .25 .10 .23 Satisfaction *** *** Learning Outcomes .24 .48 .18 .40 Gains in Personal and Social *** *** .22 .45 .16 .36 Gains in Practical Competence *** *** .18 .36 .11 .24 General Education Gains *** *** Effects of Learning Communities on Engagement

32. Faculty Time Devoted to Research and Probability of Students Doing Research 0.50 0.38 Probability of Undergraduate Research 0.25 0.13 0 2.02 6.04 10.07 14.09 18.11 Faculty Research (hours)

33. Faculty View of Importance of Doing Research with Students and Probability of Students Doing Research 0.50 0.38 0.25 Probability of Undergraduate Research 0.13 0 1.87 2.25 2.62 2.99 3.37 Importance of Undergraduate Research

34. AVG STUDENT Acad emic Act ive - Diversity Stu dent - AVG FACULTY c hall enge c ollab e xperiences f ac ulty A c ad emic chall enge ü ü ü emph asis Act ive - collab ü ü ü ü practices Emph asis on diversity ü ü ü experiences Emph asis on h igher ü ü ü order thinking Imp ortance enriching ü ü ü ed uc e xp eriences Faculty Priorities and Student Engagement

35. What to Make of This? • When faculty members emphasize certain educational practices, students engage in them to a greater extent than their peers elsewhere. • Good things go together

36. Effective Educational Practices • First-Year Seminars and Experiences  • Common Intellectual Experiences • Learning Communities • Writing-Intensive Courses • Collaborative Assignments and Projects • “Science as Science Is Done”; Undergraduate Research • Diversity/Global Learning • Service Learning, Community-Based Learning • Internships • Capstone Courses and Projects

37. Effective Educational Practices Increase Odds That Students Will: • Invest time and effort • Interact with faculty and peers about substantive matters • Experience diversity • Get more frequent feedback • Discover relevance of their learning through real-world applications

38. Institutional Reflection Areas of Effective Educational Practice Areas of Question or Improvement

39. Merge NSSE data with school records Identify engagement patterns by student characteristics Predict retention, degree attainment, grades, other outcomes Track student engagement year to year Compare vs. peer, aspirational, same-region schools Program evaluation Accreditation Consortium and system data sharing Student outcomes research Public reporting A national reporting template?!? Possibilities

40. NSSE Virtues • Research based • Focus on educationally purposeful activities • Results point to areas where improvement is desirable • Compelling face validity • Established psychometrics • Transparent operations • Third party administration

41. NSSE Virtues • Random sampling • Targeted sampling • Flexibility: consortium question, modules • Benchmarks for peer, national and other comparisons • Link/integrate with other measures -- BCSSE, FSSE, CLA, MAPP, portfolios, graduation rates etc.

42. Other Data Sources • Locally-developed measures • National instruments • Cooperative Institutional Research Program (CIRP) • Your First College Year (YFCY) • College Student Experiences Questionnaire (CSEQ) • Noel Levitz Student Satisfaction Inventory • ETS MAPP and Major Field Tests • ACT Collegiate Assessment of Academic Proficiency • Collegiate Learning Assessment (CLA) • Institutional data -- GPA, financial aid, transcripts, retention, certification tests, alumni surveys, satisfaction surveys… • Faculty Survey of Student Engagement (FSSE) and Beginning College Survey of Student Engagement (BCSSE)

43. NSSE Accreditation Toolkit • Links Between NSSE & Accreditation • Mapping NSSE to Accreditation Standards • Vignettes of Institutional Usage

44. NSSE Cautions • Only one source of information • Corroborate results • Engagement skeptics • Not an outcomes measure • Denial in face of less-than-desirable results • Eschew rankings

45. Discussion www.iub.edu/~nsse