Student Goals, Engagement and Outcomes: The Path to Success

Student Goals, Engagement and Outcomes: The Path to Success Mark Troy YunheeBae Texas A&M University AIR Forum, May 21, 2013

Goals and engagement are dimensions that facilitate student learning and lead to student success • How are these dimensions related? • Are some more effective than others at facilitating learning? • What combinations of goals and engagement lead to what types of learning?

IEO Assessment Model

Instrument • Student Experience in the Research University (SERU) • Administered Spring 2012 • 42263 students in Six universities • Minnesota • Pittsburgh • Rutgers • South Carolina • Texas A&M • Virginia • 7603 responses from TAMU

Methods • Exploratory factor analysis and confirmatory factor analysis to specify: • three goal factors (epistemic goals, intrinsic goals, and instrumental goals), • four engagement factors (attention and interest, class preparation, consolidated learning activities, and extra interaction with faculty), • three outcomes factors (knowledge/comprehension, research skills, and communication skills). • Multiple Indicators and Multiple Causes (MIMIC) CFA to validate criterion validities through interrelationships between academic engagement constructs and academic outcomes including the three outcome factors and GPA • Structural Equation Modeling (SEM) to identify relationships between factors

I: Inputs • Knowledge and skill upon entering the university • Student goals • Epistemic goal • Instrumental goal • Intrinsic goal • Background information • Parents’ education • Parents’ household income

Measurement Constructs:Students’ College Goal Orientations (n=6,782)

E: Engagement • Academic Engagement • Attention and interest in class • Class responsibilities • Consolidated learning activities • Extra interactions with faculty • Cognitive Strategies • Elaboration • Critical Thinking • Rehearsal

Measurement Constructs:Academic Engagement (n=33,880)

Measurement Constructs:Academic Engagement (Continued)

Measurement Constructs:Emphasis on Cognitive Strategies in the Major(n = 32,034)

O: Outcomes • Knowledge and Comprehension Skills • Research Skills • Communication Skills • GPA

Measurement Constructs:Students’ Perceived Learning Outcomes (n=28,681)

Criterion Validation • MIMIC (Multiple Indicators Multiple Causes) • Correlations between factor scores and criteria variables • Parents’ education • Parents’ household income • GPA

Correlations Between Factors • and Criterion Variables Note. ** : p < .001; * p < .01

Correlations Between Factors • and Criterion Variables (continued) Note. ** : p < .001; * p < .01

Correlations Between Factors • and Criterion Variables Note. ** : p < .001; * p < .01

Summary • Cronbach’s alphas (.53 ≤α≤.85; mean = .72) were acceptable for all constructs indicating good internal consistency. • CFAs on each measurement model of AE, SCG, SRLO, ECSM yielded relevant factor loadings (.32 ≤ λ ≤ .94; mean = .64) with good or acceptable model fits, supporting the construct validity of AE, SCG, and SPLO . • The relationship of GPA with “Class Responsibilities” ,”Instrumental Goal”, “Knowledge and Comprehension” at the present, and “Elaboration” were the strongest in AE, SCG, SPLO, and ECSM respectively. • Parents’ annual income is most likely to influence “Knowledge and Comprehension” at present, while, Parent Education is most likely to be related with students’ “Knowledge and Comprehension” at starting indicating a greater impact of parent income than parent education on students’ college learning outcomes • Necessary for the investigation of the relationships among AE, SCG, SPLO, ECSM and GPA to understand the paths to the learning outcomes.

Paths to Student Success in the SERU : The Relationships among 5 domains of students’ Self-Reported Constructs and their cumulative GPA through Structural Equation Modeling (SEM) • To investigate the relationships among Academic Engagement, Students’ College Goal, Learning Outcomes, and Cognitive Strategies.--> How do those constructs lead to each learning outcome including student’s perceived learning outcomes (Knowledge & Comprehension, Research Skills, and communication Skills) and GPA ? • To better understand students’ college experiences and to develop productive policies and programs to heighten students’ competencies for their paths to success.

Students’ Perceived Learning Outcomes at Starting Students’ College Goal Orientations Academic Engagement Students’ Perceived Learning Outcomes Now Pre-Knowledge & Comprehension Attention & Interest Epistemic Goal Class Responsibilities Pre- Research Skills Post- Knowledge & Comprehension Intrinsic goal Consolidated Learning Pre- Communication Skills Instrumental Goal Extra-Interaction with Faculty Emphasis on Cognitive Strategies in the Major Elaboration Critical Thinking Rehearsal

Students’ Perceived Learning Outcomes at Starting Students’ College Goal Orientations Academic Engagement Students’ Perceived Learning Outcomes Now Pre-Knowledge & Comprehension Attention & Interest R2 = .21 .54 Epistemic Goal R2 = .01 Class Responsibilities R2 = .08 Pre- Research Skills Post- Knowledge & Comprehension .59 Intrinsic goal R2 = .41 R2 = .02 Consolidated Learning .58 R2 = .18 Pre- Communication Skills Instrumental Goal Extra-Interaction with Faculty R2 = .02 R2 = .15 Emphasis on Cognitive Strategies in the Major Elaboration Critical Thinking Rehearsal .67 .55 .46

Students’ Perceived Learning Outcomes at Starting Students’ College Goal Orientations Academic Engagement Students’ Perceived Learning Outcomes Now Pre-Knowledge & Comprehension .05 Attention & Interest R2 = .21 -.06 .33 .16 -.10 .18 .54 -.11 -.03 .11 Epistemic Goal .00(ns) .00 -.01(ns) -.01 R2 = .01 .02 .13 Class Responsibilities R2 = .08 .49 .00(ns) .00 Pre- Research Skills .16 .06 Post- Knowledge & Comprehension .07 .02 .10 .59 .16 .00(ns) .00 .01(ns) .05 -.04 .00(ns) .17 .03 .07 Intrinsic goal .08 R2 = .41 .09 -.07 .09 R2 = .02 Consolidated Learning .58 .25 R2 = .18 -.04 .10 .07 .14 .11 .04 .04 .09 Pre- Communication Skills .05 Instrumental Goal -.01(ns) -.01 Extra-Interaction with Faculty .06 -.03 R2 = .02 R2 = .15 .28 .14 .13 -.07 Emphasis on Cognitive Strategies in the Major .01(ns) .01 .07 .01(ns) .01 Elaboration Critical Thinking Rehearsal .67 .55 .46

Students’ Perceived Learning Outcomes Now Students’ Perceived Learning Outcomes at Starting Students’ College Goal Orientations Academic Engagement Pre-Knowledge & Comprehension .05 Attention & Interest .33 R2 = .21 .16 -.06 -.10 .18 .54 -.11 -.03 .11 Epistemic Goal .00(ns) -.01(ns) R2 = .01 .02 .13 Class Responsibilities .64 R2 = .08 .06 .00(ns) .06 .07 Pre- Research Skills .10 Post- Research Skills .02 .59 .12 .00(ns) .00(ns) .05 -.04 .04 .17 .04 .07 Intrinsic goal .01(ns) .10 R2 = .50 -.07 .25 .09 R2 = .02 Consolidated Learning .10 .58 .07 R2 = .18 -.04 .14 .11 .04 .04 .09 Pre- Communication Skills .05 Instrumental Goal -.01(ns) Extra-Interaction with Faculty .28 .06 -.03 .14 R2 = .02 R2 = .15 -.07 .13 Emphasis on Cognitive Strategies in the Major .01(ns) .07 .01(ns) Elaboration Critical Thinking Rehearsal .67 .55 .46

Students’ Perceived Learning Outcomes Now Students’ Perceived Learning Outcomes at Starting Students’ College Goal Orientations Academic Engagement Pre-Knowledge & Comprehension .05 .33 Attention & Interest R2 = .21 .16 -.10 -.06 .18 .54 -.11 -.03 .11 Epistemic Goal .00(ns) .00 -.01(ns) -.01 R2 = .01 .02 .13 Class Responsibilities .58 R2 = .08 .06 .00(ns) .07 .00 .10 Pre- Research Skills .12 Post- Communication Skills .02 .59 .10 .00(ns) .00 -.05 .05 -.04 .02 .17 .09 .07 Intrinsic goal .04 R2 = .49 .11 -.07 .25 R2 = .02 .09 .10 .58 Consolidated Learning .07 R2 = .18 -.04 .14 .11 .04 .04 .09 Pre- Communication Skills .05 Instrumental Goal -.01(ns) -.01 .28 Extra-Interaction with Faculty .06 .14 -.03 R2 = .02 R2 = .15 -.07 .13 Emphasis on Cognitive Strategies in the Major .01(ns) .01 .07 .01(ns) .01 Elaboration Critical Thinking Rehearsal .67 .55 .46

Students’ Perceived Learning Outcomes Now Post- Knowledge & Comprehension Students’ Perceived Learning Outcomes at Starting Students’ College Goal Orientations Academic Engagement .33 Post- Research Skills .16 -.10 Pre-Knowledge & Comprehension .05 Attention & Interest -.06 R2 = .21 .18 Post- Communication Skills -.11 .54 -.03 .11 Epistemic Goal .00(ns) .00 -.01(ns) -.01 .06 .07 .26 .10 .02 R2 = .01 .13 Class Responsibilities -.10 R2 = .08 .00(ns) -.10 .00 Pre- Research Skills .04 Cumulative GPA .02 .01(ns) .00(ns) .59 .00 .11 .05 -.04 -.08 .17 -.02 .07 Intrinsic goal .02 .25 R2 = .07 .10 .08 .07 -.07 .09 Consolidated Learning R2 = .02 .58 -.04 R2 = .18 .14 .11 .04 .04 .09 Pre- Communication Skills .28 .05 .14 Instrumental Goal -.07 -.01(ns) -.01 Extra-Interaction with Faculty .06 -.03 R2 = .02 R2 = .15 .13 Emphasis on Cognitive Strategies in the Major .01(ns) .01 .07 .01(ns) .01 Elaboration Critical Thinking Rehearsal .67 .55 .46

NOTE: • Model Fit - Χ2/DF: 3250.59/39; CFI: .99; RMSEA: .04 • Each construct value (factor score) was composed of 1) standardized item scores to address the difference in item-response scale, 2) the mean of item scores weighted by factor loadings, and 3) the resulting scores standardized into a mean of 5 and a standard deviation of 2 . • The SEM includes the error-correlations, but not shown in the figure . • The color of each construct box is correspondent with that of each path coefficient box • The Italic numbers followed by “(ns)” indicate non-significant path coefficients • The numbers following “ “ indicate the strongest impact of each domain on each construct .

Conclusion Structural equation modeling (SEM) yielded several insights into the dimensions that have the greatest impacts on student learning. • Not surprisingly, for any learning outcome, the student’s incoming ability has the greatest effect on current ability. • Following that, the greatest effect on “knowledge and comprehension” is found on the path from “pre-communication skills”, to “intrinsic goal”, to “attention and interest in class”, and from “elaboration”. • The most important factors leading to GPA were found to be the SPLO of “knowledge and comprehension,” and the AE of “class responsibilities”. • “Elaboration” considered a high level of cognitive strategy did not contribute to GPA. • The effect of “critical thinking” on GPA was significant, but negative. • “Elaboration” is the strongest factor of ECSM on all of SPLO.

Conclusion, continued • There are multiple paths to any desired outcome. • Not all engagement is equal. Engagement activities need to be appropriate to the desired outcome. • Goals are important

Contact Us • Mark Troy • metroy@tamu.edu • YunheeBae • BYH777@gmail.com • To download this presentation, go to: • http://dars.tamu.edu/SERU

Student Goals, Engagement and Outcomes: The Path to Success

Student Goals, Engagement and Outcomes: The Path to Success

Presentation Transcript

Facebook Follies: Serious Lessons for Students in Civic Engagement, Social Networking and Free Speech

The context for today’s workshop: A continued focus on student learning, engagement and success.

Technology Tools for Facilitating Learning Outcomes Assessment

Data for Student Success Examining Student Work to Inform Instruction

Student Engagement: Measuring and Studying this Critical Factor for School Improvement and Student Achievement

Data for Student Success Examining Student Work to Inform Instruction

Allies in the Path to Student Success Part I

Student Success

RP Group’s Strengthening Student Success Conference October 7, 2009

Keys to Success

Maximizing The Impact Of Advising On Student Success

What Matters to Student Success in the First Year of University?

Student Handbook

Response to Instruction and Intervention: A Standards-aligned System for Student Success

Data for Student Success Examining Student Work II

Improving Student Outcomes through Progress Monitoring

What Matters to Student Success

Graphs, Goals and NPCs

George D. Kuh Maricopa Student Success Conference September 29, 2006

Developing the Fall 08-09 Educational Plan for Student Success (EPSS)

Student Learning Goals