1 / 46

Using National Studies of Student Engagement to Support Institutional Change Nathan Marti, CCSSE

Using National Studies of Student Engagement to Support Institutional Change Nathan Marti, CCSSE Todd Chamberlain, NSSE FAIR Conference June 23, 2004. What Really Matters in College: Student Engagement.

roz
Download Presentation

Using National Studies of Student Engagement to Support Institutional Change Nathan Marti, CCSSE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using National Studies of Student Engagement to Support Institutional Change Nathan Marti, CCSSE Todd Chamberlain, NSSE FAIR Conference June 23, 2004

  2. What Really Matters in College:Student Engagement “The research is unequivocal: students who are actively involved in both academic and out-of-class activities gain more from the college experience than those who are not so involved.” Ernest T. Pascarella & Patrick T. Terenzini, How College Affects Students

  3. Lessons from the Research • What matters most to desired outcomes is what students do, not who they are • A key factor for student learning is the quality of effort students devote to educationally purposeful activities

  4. National Survey of Student Engagement(pronounced “nessie”)Community College Survey of Student Engagement(pronounced “sessie”) College student surveys that assess the extent to which students engage in educational practices associated with high levels of learning and development

  5. Penetrating NSSE/CCSSE Findings • Overview of Reports • Start Broad, then Dig Deeper • Identify Significant Items • Focus on What is Important to YOUR Institution’s Priorities

  6. Customized Institutional Report • Respondent Characteristics • National benchmarks • Institutional data • Means and frequencies • Subpopulations • Comparisons by Carnegie/size, national

  7. Respondent Characteristics

  8. Respondent Characteristics:Does This Represent Your Campus? • Response Rate • Sampling Error • Student Characteristics • Gender • Race/Ethnicity • Class Level • Enrollment Status • Comparisons by Carnegie/size, national and consortium when relevant

  9. Benchmarks: High Level Views of the Data NSSE/CCSSE developed five indicators, or benchmarks, to represent the multi-dimensional nature of student engagement at the institutional, sector and national levels Supportive Campus Environment/ Support for Learners Enriching Educational Experiences/ Student Effort Level of Academic Challenge Active & Collaborative Learning Student- Faculty Interaction

  10. Benchmark Report

  11. Mean Summary ReportFrequencyDistributionReport

  12. Means & Frequency Reports • Look at the Items that Make Up Each Benchmark • Which Items Have Significantly Higher/Lower Responses than Comparison Groups? • Practical Significance: Identify Standardized Effect Sizes greater than .2

  13. Activity: Highlight Key Findings • Review Benchmark and Means Summary Report • Small groups by benchmark • Examine a benchmark • Which items differ?

  14. Digging Deeper: Using the Data • Identify important subpopulations • Determine the outcome that matters • Determine the factors that influence the outcome

  15. Principles for Data-Driven Learning-Centered Change

  16. 1. Get the ideas right • Focus on a real problem (e.g., persistence, raising expectations, success in major field courses) • Concentrate on effective educational practices

  17. Organizational culture valuing High expectations Respect for diverse talents Emphasis on early years of study Characteristics of Educationally Effective Colleges

  18. Curriculum Coherence in learning Synthesizing experiences Integrating education and experience Ongoing practice of learned skills Characteristics of Educationally Effective Colleges

  19. Instruction Active learning Assessment and feedback Collaboration Adequate time on task Out-of-class contact with faculty Characteristics of Educationally Effective Colleges

  20. 2. Get grass roots buy-in • Leaders endorse, but don’t dictate • Structures not (nearly) as important as relationships • Validate pockets of quality • The 10% rule

  21. 2. Get grass roots buy-in Examples • Ask deans about their concerns • Focus groups • Get students “engaged” in the improvement effort (Illinois State, Oregon State) • Faculty version of NSSE survey

  22. 3. Keep the stakes and volume low • Avoid “winners” & “losers” • Suspend disbelief • Denial management • Go public later than sooner

  23. 4. It’s the culture (stupid) • Culture is (almost) always (at least) part of the problem • Focus on “reculturing” and “revisioning” • Use familiar (or at least understandable) language

  24. 5. Think and act systemically • Link innovations and change efforts from different parts of the campus (e.g., Greater Expectations, Gen Ed reform, SOTL, NSSE, service learning, diversity)

  25. Presenting NSSE/CCSSE Findings • Potential Audiences? • Consider Audiences Before Administration—It May Influence Decisions • Internal Use & External Reporting • Be Strategic

  26. Internal Audiences Source: NSSE 2003 Report Card

  27. External Audiences Source: NSSE 2003 Report Card

  28. Ways to Communicate Results • Customize Report to Reach Campus Audiences • Organize NSSE data by colleges, depts, programs • Focus on items of interest to stakeholder and communicate understandable percentages • Use data to start discussions • Use data to correct misunderstandings • Assemble and distribute detailed information on method and credibility of survey

  29. Activity: Connecting Findings with Relevant Audiences • What was the most interesting finding? • Who needs to know about this? • How can you let them know?

  30. National Context Putting Your Results in Context: • How good is good enough? • What would it mean to have 80% of your students indicate that they are satisfied with the college?

  31. Sampling Methods Sampling Consistency Across Colleges: • CCSSE takes a stratified random sample of courses • NSSE takes random samples of first- and fourth-year students • Results are representative of the populations from which they are drawn • Over-sampling for targeted populations

  32. Comparing Your Results Sampling Consistency Across Colleges: • Because results are comparable across institutions, institutions can situate their results • Anything on the survey can potentially be benchmarked

  33. Benchmarking Two Approaches: • Normative - compares your students’ responses to those of students at other colleges and universities. • Criterion - compares your school’s performance against a predetermined value or level appropriate for your students, given your institutional mission, size, curricular offerings, funding, and so forth.

  34. Criterion Referenced Approach • Most valued activities Ask faculty & staff what is most valued in institution, department, then present relevant data • Eliminate “Nevers” Work on reducing or eliminating reports by students of never doing specific engagement activities

  35. Assessment Purposes • Accountability • Improvement

  36. Activity: Identify Appropriate Benchmarks • Internal Differences: How do part- and full-time students differ? • External Examples: How does your institution differ from similar colleges?

  37. Principles for Good Practice in Undergraduate Education(Chickering & Gamson, 1987) • Student-faculty contact • Active learning • Prompt feedback • Time on task • High expectations • Respect for diverse learning styles • Cooperation among students

  38. Lessons from the Research • What matters most is what students do, not who they are • A key factor is the quality of effort students expend • Educationally effective institutions channel student energy toward the right activities

  39. Principles For Promoting Student Engagement • Intentionality - Not leaving serendipity to chance • Alignment (mission, curriculum, student experiences) • Collaboration (pull many levers) • Assessment and feedback to guide/document improvement

  40. How is Engagement Measured? • Switch benchmarks • How would you measure this construct?

  41. Psychometric Findings: Reliability • Both instruments have been shown to be reliable and valid • Good construct reliability: benchmarks items measure the same construct • Structural equation models have been used to demonstrate that there is strong consistency across subpopulations (gender & part- v. full-time) and across years

  42. Psychometric Findings: Validity

  43. Psychographic Data • NSSE data used to form clusters of students • http://mypage.iu.edu/~chuzhao/air03dpi.pdf • Psychographic data consistently was more effective at predicting outcomes than demographic data

  44. The Florida Opportunity • How does engagement relate to outcome? • Persistence • Goal Attainment

  45. Activity: Expectations and Desires in Student Engagement • How many students responded often or very often? • What would you like to see? • How did they respond? • How can you increase engagement?

  46. Questions and Suggestions • Questions for NSSE and CCSSE • Suggestions for NSSE and CCSSE

More Related