Presentation Transcript

1. LREC 2012 Tutorial Kostas Karpouzis National Technical University of Athens kkarpou@cs.ntua.gr

   Using games to model and recognize interactions

2. schedule User models for game play Modellingprototypical player and non-player characters Emotion and games: inducing and expressing emotions User, environment and interaction context Modellingand measuring player satisfaction
3. schedule Game-based corpora Elements of captured user expressivity Design issues in game-based corpora Inducing emotion with games Measuring behavioural, affective and cognitive aspects during game play
4. schedule Emotion, affect and behaviour Available modalities in natural human-computer interaction Your body (and face, hand, speech) is the controller Behavioural cues, non-verbal interaction What else can we recognize? What can’t we?
5. schedule Lessons learned CALLAS: capturing affect in arts and entertainment apps Serious games for observing and learning social skills: the Siren project Games for corpora collection the platformer case: Super Mario Bros clone the FPS case: Cube 2 the racing case: TORCS
6. yours truly Post-doc in Humaine Network of Excellence Since: CALLAS (emotion in arts and entertainment) Feelix Growing (affect-aware robots) Currently: Siren (serious games for conflict resolution in school environments) ILearnRW (serious games for children with dyslexia and dysorthographia)
7. why games? Toys provide our first human-“machine” interface! Games are everywhere these days! Computers, consoles/TV, mobile phones Browser and Facebook/Google+ do not require installation ‘Freemium’ business model attractive to both gamers and developers Huge interest in industry and academia Lots of funding from EU projects 
8. why games? Constrained environment  easier to record people Novel modalities: gestures, body movement, speech (lexical and prosody) Ideally associated with what happen in the game (Possibly) novel interaction paradigms, i.e. not WIMP
9. why games? The concept of Flow a state of concentration or complete absorption with the activity at hand and the situation. It is a state in which people are so involved in an activity that nothing else seems to matter (Csikszentmihalyi,1990) “Being completely involved in an activity for its own sake. The ego falls away. Time flies. Every action, movement, and thought follows inevitably from the previous one, like playing jazz. Your whole being is involved, and you're using your skills to the utmost”
10. while in flow… Gamers are more expressive May show less (or no) social inhibition Spontaneous expressivity Include expressive modalities not usual in other HCI contexts (e.g. gesturing) Positive or negative expressions may be exaggerated
11. while in flow…
12. or(if things go wrong)…
13. while in flow…

14. a bit of theory

    the quest for Flow (and fun)
15. a theory of fun RaphKoster lead designer of Ultima Online creative director of Star Wars Galaxies http://www.theoryoffun.com/theoryoffun.pdf http://www.raphkoster.com
16. a theory of fun ‘all games are edutainment’ ‘some games teach spatial relationships (e.g. Tetris)’ ‘some teach you to explore (Super Mario)’ ‘some teach you how to aim (FPSs)’ some teach you to share/cooperate (Farmville) ‘players seeking to advance in a game will always try to optimize what they are doing’
17. a theory of fun
18. a theory of fun
19. a theory of fun ‘We talk so much about emergent gameplay, non-linear storytelling, or about player-entered content. They’re all ways of increasing the possibility space, making self-refreshing puzzles’ So, what is it that makes a game ‘fun’?
20. flow revisited the ‘holy grail’ of game design just the right amount of challenge making a game very hardgamers quit making a game very easygamers bored
21. flow revisited it’s not about the graphics or the controller or the franchise (e.g. sports games) just ask Rovio makers of Angry Birds $80M/yr, 600M dl’s
22. flow revisited ‘smart’ games adapt to player skill and engagement keeping them coming back for more at the end of the day…
23. that’s flow!
24. generating content we want to make games harder or easier to match player skill predefined levels (e.g. ‘easy’ / ‘expert’) still we have to define what ‘easy’ means
25. generating content game level as a multi-parameter function produce game content procedurally e.g. number and size of gaps, number of opponents, etc.
26. generating content Java implementation Mario AI Championship www.marioai.org cf. Yannakakis & Togelius @ IEEE TAC
27. what about serious games? ‘the opposite of play is not work, it’s depression’ Dr. Stuart Brown Serious Play 2008 (hosted @ TED.com)
28. you may want to watch… Jane McGonigal: Gaming can make a better world, Feb 2010
29. you may want to watch… A gamer just before an ‘epic win’!
30. what about serious games? our first activity as children ‘fun’ and ‘flow’ are a given
31. what about serious games? our first activity as children ‘fun’ and ‘flow’ are a given best material to teach social skills
32. what about serious games? our first activity as children ‘fun’ and ‘flow’ are a given best material to teach social skills but schools fail to capitalize on that
33. gamification the best one way to influence player behaviour include game design elements in non-game contexts
34. gamification image by Sebastian Deterding Bunchball white paper: http://info.bunchball.com/gamification-101/
35. gamification in Fourquare, users earn points for check-ins and other activities leaderboards and badge display enhance competition
36. gamification replace ‘check-in’ with, e.g., ‘recycle’ gamification in the real world
37. gamification replace ‘check-in’ with, e.g., ‘recycle’ gamification in the real world what happens when badges and rewards are taken away? open research question
38. in a nutshell games provide challenge and fun to players or should be adapted to do so fun not always equal to entertainment the case of serious/learning games player experience: function of skill, performance and challenge

39. user modelling

40. player personalities Richard Bartle co-creator of MUD1 (the first MUD) Bartle Test of Gamer Psychology series of questions to players of MMOs into categories based on gaming preferences
41. player personalities
42. player personalities Achievers:  players who prefer to gain points, levels, equipment and other concrete measurements of succeeding in a game Explorers: players who prefer discovering areas, creating maps and learning about hidden places
43. player personalities Socializers: players who choose to play games for the social aspect, rather than the actual game itself Killers: players who like to …‘club’ other players They thrive on competition with other players, and prefer fighting them to scripted computer-controlled opponents.
44. player personalities Bartle Quotient totals 200% across all categories, with no single category exceeding 100% A person may score "100% Killer, 50% Socializer, 40% Achiever, 10% Explorer“ indicating preference to fight people compared to other aspects of gameplay but…
45. player personalities these are self-reported characteristics mostly refer to what players would prefer to do and not necessarily what they actually do when playing game genre-specific and different for single- and multi-player gaming offer little in terms of adaptation mostly refer to game mechanics and features
46. back to the drawing board what can we model? and how? definition of ‘affective computing’ ‘affective Computing is computing that relates to, arises from, or deliberately influences emotion or other affective phenomena’ -- Roz Picard, 1995 ‘a set of observable manifestations of a subjectively experienced emotion’ -- Merriam-Webster’s dictionary
47. observable manifestations
48. observable manifestations
49. hypothesis ‘shallow’ treatment i.e. not as far as ‘personality’, sticking to ‘affect’ identify/track user reactions facial expressions and gestures, body movements and stance, hand and body expressivity (for whole-body interaction) relate those to events in the game
50. hypothesis ideally, we could identify the players’ stress level (via the ‘observable manifestations’) and their skill level (via their performance) and cluster those to identify player types for the particular game genre! or use them to adapt the game make it easier for players ‘in distress’ or harder for players in the verge of boredom
51. hypothesis why bother with both affect and performance? why are players standing still? is it flow (immersion) or boredom? or why do they move around? is it immersion (e.g. in a racing game) or lack of engagement? remember: Flow  skill AND engagement
52. hypothesis
53. pitfalls! happy face  low stress? may indicate irony sad face  high stress  need to adapt? some serious games may need to induce negative emotions to get things running e.g. in a conflict resolution game more on this later
54. we’ve covered affect;what else is there? cognitive models (Gray, 2007) evaluate why players behave in the way that they do, or conversely to control computer-driven AI (Funge, 1999) cultural model “collective programming of the mind which distinguishes the members of one group or category of people from another” (Hofstede, 1996) not necessarily related to player origin or descent (e.g. sub-cultures)
55. we’ve covered affect;what else is there? group models differences of equal treatment (favourability) when interacting with individuals from the in-group (i.e. the group the agent belongs to) or out-group
56. we’ve covered affect;what else is there? learner model students motivation strongly linked to learning (Malone and Lepper, 1987) demographic information and personality  understand and predict the student’s learning behavior demographics (gender), personality (Big 5: openness, conscientiousness, extraversion, agreeableness, neuroticism), goal orientation (performance based on result or mastery on skill), and presence (involvement with the system) (McQuiggan et al., 2010)
57. in a nutshell games provide an ideal medium to induce and capture affective interactions well-designed games bring out different (and valuable!) reactions from players gaming is a multi-faceted activity thus, player models are usually detailed player affect tells us a lot about the game

58. lessons learned

    the CALLAS and Siren projects
59. capturing affect in artsand entertainment The aim of the CALLAS project was to identify the role of affect in artistic and entertainment apps Close the affective loop between user and ‘machine’
60. your body is the controller(™)
61. your body is the controller(™) ‘Whole Body Interaction’ is a novel, affect-aware medium OK, MS Kinect admittedly took it to the masses mainstream exertion games have been around since the 90s (e.g. Konami's Dance Dance Revolution) image by Exertion Games Lab / Floyd Mueller
62. your body is the controller(™) sometimes it’s not about which gesture you produce unless it carries affective meaning (e.g. ‘thumbs-up’) but about how you produce it greet a stranger vs your significant other
63. your body is the controller(™) Kinect (or simple image processing techniques)  track motion of hands or whole body Caridakis, Castellano, etc. study ‘expressivity features’ speed, spatial extend, fluidity, etc.
64. your body is the controller(™) perceptual studies indicated strong correlation with human annotators especially wrt activation levels culturally dependent? initial studies verify cultural clichésabout expressivity
65. conflict resolution games Siren aims to produce a conflict resolution serious game for 10-14 y.o. children in school environments
66. conflict resolution games
67. conflict resolution games The life cycle of conflict (Swanstrom and Weissmann, 2005)
68. conflict resolution games during escalation, negative emotions are present cannot use neg. emotions to indicate stress  adaptation
69. conflict resolution games rather, use estimated emotion to identify where players are in this figure (which phase)
70. conflict resolution games and produce content to ‘push’ users towards descalation learning objective of the game!
71. conflict resolution games sensed affect can be used to identify player performance i.e. whether players actually ‘move’ towards resolving the conflict but which emotions are relevant? negative vs positive is that enough for all game genres?
72. the Siren database Super Mario revisited Idea: correlate facial expressivity with game events facial expressions, cues (e.g. lip biting), head movement (expressivity, eye gaze)
73. the Siren database self-reported frustration and engagement for pairs of games no direct (objective) measurement of difficulty level
74. the Siren database about 80 volunteers mostly students not necessarily game players interesting results experienced players move less men move less
75. the Siren database about 80 volunteers mostly students not necessarily game players interesting results experienced players move less men move less paper in ‘Multimodal corpora’ workshop
76. the Siren database prototypical player types? e.g. ‘Iceman’ who hardly moves but performs superbly or ‘dancers’ who ‘flow’ along with Super Mario’s motion culture-related?
77. other game genres Cube 2 is an open Source (ZLIB license) First-Person Shooter capture game events create game content on the fly http://sauerbraten.org
78. other game genres TORCS (The Open Racing Car Simulator) is a portable multi platform car racing simulation competitions for faster/smoother driving, demolition derby! http://torcs.sourceforge.net
79. other game genres TORCS (The Open Racing Car Simulator) is a portable multi platform car racing simulation competitions for faster/smoother driving, demolition derby! http://torcs.sourceforge.net
80. in a nutshell player affect is genre-dependent reflects many qualities from the user model many open research questions single- vs multi-player easy to find people to play games yay! large AV datasets difficult to put together as always 

81. thank you!

    Kostas Karpouzis kkarpou@cs.ntua.gr