Student Engagement at WSU: What can we Learn From Five years of NSSE Information

1. Office of Institutional Research Student Engagement at WSU: What can we Learn From Five years of NSSE Information

2. NSSE: National Study of Student Engagement • A national survey of first year students and seniors in 4-year colleges and universities. • It was designed to measure “engagement” or how students perceive their involvement in the total academic experience • Engagement is defined by George Kuh as: • What students do -- time and energy devoted to educationally purposeful activities both in and out of the classroom • What institutions do-- using effective educational practices to induce students to do the right things

3. NSSE: National Study of Student Engagement • First full data collection in 2000 • 1300 different 4-year schools have participated • Averages 600+ schools for the last five years • Survey of a random sample of freshmen and seniors conducted in spring each year

4. Questions • Students are asked to respond to questions about their experiences at the university including academics, university services, and various out of classroom activities. • 85 items with varying response options such as • How often have you… 1 Never to 4 very often • How would you rate… 1 very unfavorable to 7 highly favorable • How would you evaluate…1 poor to 4 excellent • How much emphasis… 1 very little to 4 very much

5. Seven Principles of Good Practice in Undergraduate Education* • Student-faculty contact • Active learning • Prompt feedback • Time on task • High expectations • Experiences with diversity • Cooperation among students * Chickering & Gamson, 1987; Pascarella & Terenzini, 2005

6. NSSE Indicators ofEffective Educational Practice Active & Collaborative Learning Level of Academic Challenge Enriching Educational Experiences Supportive Campus Environment Student – Faculty Interaction

7. The WSU Experience Several administrations Pooling data on several administrations Providing college level results with sufficient sample size

8. Background Information • NSSE has now been administered at WSU six times (2002, 2004, 2005, 2007, 2008 and 2009) • Although part of two recent KBOR Performance Agreements, use by the academic colleges or other areas has been minimal. • The Foundation of Excellence Project last year did use the results for the first year students. • Academic colleges have expressed concern that the sample size in a single year is insufficient for their use.

9. Thus … The University Assessment asked us to see if we could legitimately combine several years’ data and then consider differences among the academic colleges.

10. Purposes of Current Analyses • Reviewed data over several years to identify any trends or changes. • Using the data from several survey administrations, disaggregate the WSU NSSE data to individual colleges by major with sample sizes large enough to encourage confidence in results. • Provide information about groups of items that are conceptually and empirically confirmed as related (i.e. benchmarks and scalelets). • Allow colleges and other campus groups the opportunity to use the NSSE as another component of their total outcome assessments. • Limited to seniors since most logical at the college level.

11. Step 1: Merging data over multiple survey administrations Validated item consistency Determined differences, if any, for WSU data over administrations

12. Item consistency over time:Review using NSSE materials • Reviewed the detailed report produced by the NSSE staff for item changes over each administration since 2002. • Sufficient number of changes before 2004 that deletion of 2002 data seemed essential. • One item changed in 2004 but this item not used in our analysis • The wording of the survey items and the response options on the surveys are consistent over the years since 2004.

13. Item consistency over time: Review using WSU Data • Compared each item by class and year using Chi-Square or Analysis of Variance. • Used complete responses (e.g. 1-4) and pooled responses (e.g. often/very often) as another means to validate any differences in an item over time. • Removed 2004 and repeated the analyses. No additional meaningful differences noted. • With this large number of cases, occurrence of significant differences by chance alone are highly probable. • Reviewed items for meaningful differences if statistically significant.

14. Item consistency over time: Summary • Differences in items over administrations were minimal. • Fluctuations in scores rather than progressive pattern of changed was noted for some items. • Responses from seniors were remarkably stable • NSSE developers indicate that unless deliberate and systemic university-wide interventions were made, differences should be minimal.

15. Two interesting items • E-mail as a method for communication with faculty • Increased from 49% in 2004 to 72% In 2009 as reported by freshmen. • No differences for seniors • University encouraging students to vote quite a bit or very much. • Increases were noted in the two presidential election years i.e. 2005 and 2009. • In 2005 and 2009, freshmen reported encouragement at 30 % and 37% respectively but only 18-19% in other years. • In 2005 and 2009, seniors reported encouragement at 24% and 27% respectively but only 18-19% in other years.

16. Demographics Sample versus total seniors Characteristics of Students for sample versus total seniors

17. Comparison of Sample and Total Enrollment by College (N=2173) *Fall 2008

18. Demographics of Seniors

19. A few Individual Items

20. Rating for Academic Advising and Satisfaction Entire Educational Experience

21. Interrelated items

22. Benchmarks represent Areas of Effective Practice Active & Collaborative Learning Level of Academic Challenge Enriching Educational Experiences Supportive Campus Environment Student – Faculty Interaction

23. Scalelets versus Benchmarks • Scalelets include a small number of items related to specific concept that have empirically validated. (Pike,2002,2006) • 14 scalelets with 12 as subsections of the benchmarks • Advantages over benchmarks • More specific and meaningful concepts • More congruence between items and concept • Benchmark and scalelet scores are converted to 0-100 scale. • The mean for the items in each scalelet and benchmark for each student were calculated if he/she answered 3/5 or 60% of items

24. Academic Challenge Benchmark: Academic Challenge Scalelets: Course Challenge and Writing

25. Level of Academic Challenge: Items • Preparing for class (studying, reading, writing, doing homework or lab work, etc. related to academic program) • Number of assigned textbooks, books, or book-length packs of course readings • Number of written papers or reports of 20 pages or more • Number of written papers or reports of between 5 and 19 pages • Number of written papers or reports of fewer than 5 pages • Coursework emphasizes: Analysis of the basic elements of an idea, experience or theory • Coursework emphasizes: Synthesis and organizing of ideas, information, or experiences into new, more complex interpretations and relationships • Coursework emphasizes: Making of judgments about the value of information, arguments, or methods • Coursework emphasizes: Applying theories or concepts to practical problems or in new situations • Working harder than you thought you could to meet an instructor's standards or expectations • Campus environment emphasizes: Spending significant amount of time studying and on academic work.

26. Level of Academic Challenge: Mean Scores

27. Course challenge

28. Writing

29. Higher-Order thinking Skills

30. Academic Challenge: Related Scalelets

31. Active and Collaborative Learning Collaborative Learning Active Learning

32. Active & Collaborative Learning Asked questions in class or contributed to class discussions Made a class presentation Worked with other students on projects during class Worked with classmates outside of class to prepare class assignments Tutored or taught other students (paid or voluntary) Participated in a community-based project (e.g., service learning) as part of a regular course Discussed ideas from your readings or classes with others outside of class (students, family members, co-workers, etc.)

33. Active Learning

34. Collaborative Learning

35. Active and Collaborative Learning

36. Faculty/Staff – Student Interaction Course Interaction Out of Classroom Interaction

37. Course Interaction

38. Out-of-Class Interaction

39. Course Interaction and Outside Class Interaction

40. Enriching Educational Experiences Varied Experiences Information Technology Diversity

41. Varied Experiences(Part I)

42. Varied Experiences(Part II)

43. Technology

44. Diversity

45. Varied Experiences, Technology, Diversity

46. Supportive Campus Environment Support for Student Success Interpersonal Environment

47. Support for Student Success

48. Interpersonal Environment

49. Support for Student Success and Supportive Interpersonal Environment

50. Outcome Measures Practical Skills General Education