1 / 20

The Study of Examinee Motivation

The Study of Examinee Motivation. AAHE Assessment Forum Denver, Colorado: June, 2004 Donna L. Sundre, Executive Director, Center for Assessment and Research Studies www.jmu.edu/assessment. Center for Assessment and Research Studies.

hogan
Download Presentation

The Study of Examinee Motivation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Study of Examinee Motivation AAHE Assessment Forum Denver, Colorado: June, 2004 • Donna L. Sundre, • Executive Director, • Center for Assessment and Research Studies • www.jmu.edu/assessment

  2. Center for Assessment and Research Studies • Seeks to become a nationally recognized standard of excellence for assessment programs • Designs sophisticated and innovative instruments to directly measure learning and develop outcomes • Produces graduates who will become national leaders • Contributes to scholarship of assessment • Assists faculty, student affairs, and administrative staff in the process of assessing learning and developmental outcomes

  3. The Study of Examinee Motivation Why this is an important area to study Introduction to theoretical frames What the literature tells us What we have discovered What next?

  4. Why This Is an Important Area to Study • Learning and achievement are critical variables for all educators, the public, policy makers, and researchers • Tests play the primary role as achievement indicators • Development of sound measures is the first step • What other factors can impact performance?

  5. Accountability and Policy Uses A few examples: • International Education Association • National Assessment of Educational Progress • No Child Left Behind • State, District, and sometimes additional local tests • Accreditation of Every School and College • Disciplinary Accreditation

  6. Assessment Uses A few examples: • Program Improvement • Curricular Improvement • Teaching and Instructional Delivery • Academic Program Review • Restructuring and Reallocation of Resources

  7. Research and Development Uses • Test Development • Item Analysis • Development of Norms • Standard Setting • Item Parameter Estimation • Research for Theory Testing and Building

  8. Are These Tests Capable of Bearing the Weight of Such Decisions? Given the many factors that can influence test scores, it’s obvious to us that we have several serious validity concerns However, many stakeholders have not considered these threats We know remarkably little about many of these validity threats operate or how to control them

  9. Theoretical Frames for Linking Motivation, Self-Regulation and Achievement • Eccles-Parsons, et al. (1983): Expectancy-Value • Pintrich and DeGroot (1991): Adapted Expectancy-Value • Snow (1994): Metatheory of Aptitude

  10. Eccles-Parsons et al., 1983 • Expectancy- • Belief in ability to succeed at a task • Value- • Importance- Perceived importance and personal interest • Utility- Perceived uses of engagement (means to an end; prerequisite in a major; empowering) • Cost- Perceived difficulty; cost to benefit analysis

  11. Pintrich and DeGroot, 1990 • Expectancy and Value are Insufficient • Students May Not Be Motivated to Use Their Self-Regulation Strategies • An Additional Component is Necessary • Adapted Expectancy- Value Theory • Expectancy • Value • Affect- Emotional Reaction to the Task

  12. Richard Snow, 1994; Others follow • Metatheory of Aptitude • Broadening the Concept of Aptitude to Include: • Cognitive Processes- Abilities, Strategies, Knowledge • Affective Processes- Temperament, Emotions • Conative Processes- Motivation, Volition • Dynamic Person-Situation Interaction • “Aptitude Complex”-environment and individual • Whole-Person in Context • Demands of the Performance Situation (challenge, resources) • Dynamic Inner Psychological & Outer Task Environment • “Fit” Between Resources and Situational Demand is Important

  13. What the Literature Tells Us • Motivation is Correlated with Performance • Test Consequences Impact Motivation and Performance: Consequences Matter • High-Stakes: Anxiety is the Prime Contaminant • Low-Stakes: Many Other Factors are Introduced • Test Modality Matters: Selected vs. Constructed Response • Task Demand Matters: Arduous Tasks Result in Lower Motivation and Performance • Feedback doesn’t matter-even choice of feedback (Wise, 2004)

  14. Assessment Accountability Climate • We Must Learn More About These Phenomena • Our Systematic Data Collection • Our work with • Achievement Goal Theory • Examinee Motivation, coupled with • Direct Measures of Student Learning Measures • We Are Right on Target to Contribute to this Conversation

  15. A Sampler of Research Findings • Development/Refinement of SOS • Lisa Wolf’s Dissertation- Motivation Scale: k=8 • Is examinee motivation a unidimensional construct? • Wolf and Smith say 1 factor; others say two • Revised Wolf’s Instrument- create Student Opinion Scale • Total and Two Factors: Importance k=5; Effort k=5 • Cronbach alphas all in mid to high .80’s • Successful CFAs of SOS conducted (Sundre & Finney, 2002) • Validity Evidence Mounts

  16. Empirical Results Support Theory • Consequences Impact Motivation • Motivation Explains Additional Variance in Test Performances Beyond SAT and Grades • Situation-Specific Variables Impact Motivation and Performance • Administrative Errors Reduce • Unmotivated Proctors Impact Both Motivation and Performance • Task Demand Impacts Motivation

  17. More Empirical Results • Motivation Impacts Psychometric Properties (Sundre & Wise, 2003) • “Motivation Filtering” Study Reliability is Impacted • Validity is Impacted • Self-Regulation and Motivation Behave Differently (Sundre & Kitsantas, 2004) • Self-Regulation is a Strong Predictor When There Are Consequences • Self-Regulation Plays a Greater Role in No Consequence Conditions on Selected Response • Motivation Steals the Show on Constructed Response

  18. Sundre and Wise, 2003— “Motivation Filtering” Statistics for the Scientific ReasoningTest, by Motivation Filter SOS w • Motivation Filter N Mean SD Alpha SEM SATr SATT • Total Test Score • No Filter 777 58.11 12.89 .77 6.18 .45 .02 • Suspect SOS 680 58.70 12.33 .75 6.17 .46 .01 • Suspect SOS or SOS ≤20 632 59.47 11.72 .73 6.09 .51 .03 • Suspect SOS or SOS ≤25 562 59.93 11.31 .71 6.09 .54 .06 • Suspect SOS or SOS ≤30 371 61.26 10.72 .68 6.06 .59 .05 • Suspect SOS or SOS ≤35 174 62.93 10.72 .68 6.06 .64 .01

  19. Sundre and Kitsantas (2004) Contemporary Educational Psychology

  20. What Next? • Expansion of the conception of motivation • Affective and conative components included • Affective: Feelings and emotions as legitimate • Conative: Motivation and Volition- persistence • Learning goals as motivation • Do these change over time? • How contextual are they? • Experimental Studies • How can we impact motivation positively? • Response Time Indicators in CBT (Wise and Kong) • Innovative Item Types • Self-Adaptive Testing (Wise) • Data Filtering for Reporting Purposes • Motivation Filtering

More Related