Download
1 / 39
390 likes | 508 Views
An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement. Alexander C. McCormick Center for Postsecondary Research Indiana University Bloomington, USA HEPI-HEA Conference The British Academy, London 22 May 2012. Overview.
E N D
An Experience-Based View of University Quality in the USA: The National Survey of Student Engagement Alexander C. McCormick Center for Postsecondary Research Indiana University Bloomington, USA HEPI-HEA Conference The British Academy, London 22 May 2012
Overview • Context: Conceptions of quality in U.S. tertiary education • The National Survey of Student Engagement (NSSE) • Low stakes, high yield assessment • Student engagement trends • Controversies and cautions
Context Higher education in the USA Limited understanding of quality Student engagement as a new approach
Higher Education in the USA • No national system of higher education • Autonomy of institutions • Diversity of institutions • 4,600 degree-granting tertiary institutions • Public and private • Universities (undergraduate and postgraduate) • Colleges (undergraduate only) • 4-year bachelor’s degree • 2-year associate’s degree (community colleges)
Higher Education in the USA (2) Diversity of students • Full- and part-time • Traditional college-age • Older and returning students • Widely varying levels of preparation
Quality Assessment in US Higher Education • Accreditation • Limited public understanding • Limited disclosure • Traditional emphasis on capacity and resources • Limited attention to teaching & learning (until recently) • Newsmagazine rankings • Emphasize reputation, resources, other inputs • Research rankings • Blind to undergraduate education
Origins of the “National” Survey of Student Engagement (NSSE) • A reaction to the dominant discourse about university quality in the USA • Shift the focus to teaching and learning • Promote evidence-based improvement
The Big Idea • Ask students about their experience • Focus on behavior: empirically confirmed effective practices
Why a Student Survey? Direct assessment of learning is problematic • Measurement challenges • Inferential challenges • Cost challenges • Motivation challenges Survey approach offers advantages • Cost-effective and efficient • Diagnostic utility of behavioral data • Students are the experts on their experience
NSSE’s Key Aims Enrich the impoverished national discourse about university quality • Shift the focus to teaching & learning, through the lens of effective practices Provide diagnostic informationthat can be used to improve undergraduate education • Actionable information based on valid & reliable measures • Meaningful comparisons to other institutions
What is Student Engagement? The extent to which students engage in and are exposed to effective educational practices • Activities and practices known to be related to desired outcomes
OK, but what is Engagement? • Challenging academic work • Active learning • Deep approaches to learning • High-impact practices • Involvement with instructors In a campus context that promotes and supports success
Conceptual & Empirical Foundations Time on task (Tyler, 1930s) Quality of effort (Pace, 1960s & 70s) Student involvement (Astin, 1984) Academic & social integration (Tinto, 1975 & 1987) Good practices in undergraduate education (Chickering & Gamson, 1987) College impact (Pascarella, 1985) Student engagement (Kuh, 1991, 2005)
Why Does Engagement Matter? The impact of college is largely determined by individual effort and involvement in the academic, interpersonal, and extra-curricular offerings on a campus. It is important to focus on the ways an institution can shape its offerings to encourage student engagement. Paraphrased from Pascarella & Terenzini (2005), p. 602
How NSSE Works • Paper or Web-based survey • First- & final-year undergraduates • Uniform, centralized survey administration • Institutions receive detailed reports and an identified student data file • Customizable comparison groups • Results are confidential • Supported by institution fees
Participation • 550 to 770 U.S. & Canadian institutions per year • About 1,500 since NSSE’s launch in 2000 • Most repeat every 1-4 years
Survey Content Academic activities & experiences Academic challenge Reading and writing Active learning Cognitive tasks in courses Enriching educational experiences Time use Study, work, socializing… Co-curricular activities Institutional emphases Quality of campus relationships Cognitive and noncognitive gains Satisfaction Demographic & enrollment characteristics
Sample Questions • Asked questions in class or contributed to class discussions? • Made a class presentation? • Prepared two or more drafts of a paper or assignment before turning it in? In your experience at your institution during the current school year, about how often have you…
Sample Questions • Put together ideas or concepts from different courses when completing assignments or during class discussions? • Discussed ideas from your readings or classes with faculty members outside of class? In your experience at your institution during the current school year, about how often have you…
Sample Questions • Received prompt written or oral feedback from faculty on your academic performance? • Worked harder than you thought you could to meet an instructor’s standards or expectations? In your experience at your institution during the current school year, about how often have you…
Sample Questions • Memorisingfacts, ideas, or methods… • Analysingthe basic elements of an idea… • Synthesisingand organizing ideas, information… • Making judgments about the value of information… • Applyingtheories or concepts… During the current school year, how much has your coursework emphasized the following mental activities?
Sample Questions • Examined the strengths or weaknesses of your own views on a topic or issue? • Learned something that changed the way you understand an issue or concept? During the current school year, about how often have you done each of the following?
Using NSSE Results How are we doing? How can we improve?
Using NSSE Results • Diagnostic purpose: analysing the undergraduate experience • Identify strengths and weaknesses, areas where improvement is needed • Stimulates conversations about quality • Informs decision-making with “actionable” information
Many Ways to Use Results Peer comparisons • How do we measure up to others? Self comparison • Are we improving? Absolute standard • How do we measure up to our ideals? Internal variation • Who is least engaged? What to look at and how to look at it
NSSE results as a catalyst for campus discussions about teaching & learning
Comparing Student andInstructor Perceptions Faculty Survey Courses emphasisedmemorisation“Quite a bit” or “Very much”
Comparing Student andInstructor Perceptions Courses emphasisedmemorisation“Quite a bit” or “Very much”
Results at institutions with several NSSE administrations show many with positive trends
Summary of Findings • 41% of institutions demonstrated a pattern of improvement in at least one measure for first-year students • 28% did so for final-year students • Trends found across many institutional types • Control (public & private), Size, Research emphasis • Ratio of positive to negative trends: 7:1
Cautions & Controversies NSSE is no “magic bullet” The risks of high-stakes uses “Participation” vs. “use”
Cautions • NSSE is no “magic bullet”: Best used in combination with other information • NSSE’s research foundation is based on studies of US university students • Most variation is between students, not institutions
Controversies • Confidentiality of results: Institutions control disclosure • Attaching high stakes to NSSE is risky • Institutions will decline to participate ifpublic relations cost > institutional benefit • Reliability of responses could be compromised • Mere participation is not sufficient • Results must be analysedand used
Understanding Improvement Organizations that improve… create and nurture agreement on what is worth achieving, and they set in motion the internal processes by which people progressively learn how to do what they need to do in order to achieve what is worthwhile. (Elmore in Fullan, 2001)
