120 likes | 231 Views
Modeling but NOT Measuring Engagement in Computer Games. Mark Chen w ith Beth Kolko , Elisabeth Cuddihy , Eliana Medina @ mcdanger | markchen@uw.edu markdangerchen.net. Methods. Compare two games: The Curse of Monkey Island 89.9% metascore on gamerankings.com
E N D
Modeling but NOT Measuring Engagement in Computer Games Mark Chen with Beth Kolko, Elisabeth Cuddihy, Eliana Medina @mcdanger | markchen@uw.edu markdangerchen.net
Methods • Compare two games: • The Curse of Monkey Island • 89.9% metascore on gamerankings.com • The Oregon Trail 5th Edition • assumed worse than a successful commercial game
Methods • Lab studies • Immersive Tendencies Questionnaire • e.g., Do you ever become so involved in doing something that you lose all track of time? • Gaming Engagement Questionnaire • e.g.,Wereyou involved in the game to the extent that you lost track of time? • (based on Witmer & Singer, 1998, Presence Questionnaire) • observation • during-game flow questions (Csikszentmihalyi, 1990) • post-game interviews
Big Heap o’ Fail Adrian Bruce http://adrianbruce.com/teacher-toolbox/the-fail-whale/
Big Heap o’ Fail The Oregon Trail >The Curse of Monkey Island !! Curse’s score didn’t reflect metascore Pilot test participants were frustrated with Curse much more so than with The Oregon Trail
What happened? • Wrong assumptions: • metascore from game reviews = engagement • reviews done after hours of play • reviewers have deep knowledge of gaming culture and game genres • educational titles must be worse than commercial titles • didn’t select The Oregon Trail based on metascore • testing methods over emphasize immersion and usability • We were noobs.
(de)motivation • conference win, paper fail (guilt/shame) • presented at DiGRA, but not a full paper • never completed study or full paper • group parted ways (re: noobness)
It’s still useful! • not that bad; our model is still useful and relevant, our questionnaires still useful (Bianchi-Berthouze, Kim, & Patel, 2007)
What now? • Hall of Fail as win-win! • need to include ecologically valid methods to studying engagement • capture affective aspects of engagement
Modeling but NOT Measuring Engagement in Computer Games Mark Chen with Beth Kolko, Elisabeth Cuddihy, Eliana Medina @mcdanger | markchen@uw.edu markdangerchen.net