Introduction to Cognition and Gaming

1. Introduction to Cognition and Gaming • 11/17/02: User Interface

2. User Interface is Important

3. The Importance of a Good Interface • Without a fluid and effective control scheme, the best game feature is reduced to an unplayable source of frustration • Without displaying proper information, the player becomes a helpless victim of circumstance • The interface is the designer’s sole means of communicating with the player – in a sense, it defines the game

4. The Importance of a Good Interface • In some genres, the interface is the game – sometimes mastering intricate keystroke combinations is the heart of the gameplay • A bad interface can ruin a great game – e.g. if reaching a state in which gameplay can occur takes up a measurable share of the total game session, the UI is flawed • Games are holistic – The experience is diminished if the UI clashes with the visual style or rhythm of the game • The interface requires significant resources – Flight sims can have half the screen taken up by the interface – verbal UIs may need dedicated hardware

5. Visual Display Rules (P&A) • Stimuli away from the center of focus should be based on variations of luminance instead of color, and vice versa • The larger the display, the higher the frame rate that you need • Only 25% of the display should be devoted to significant information

6. Visual Display Rules (P&A) • The eye is most sensitive to changes in the upper-right quadrant of the screen, least sensitive to lower-right. • Different screen layout for rookies and veterans of games using keyboard interfaces • The eye can quickly process up to 5 different colors, 4 bars of different length, and 24 different angles

7. Input Sequences • A computer keyboard has over 100 keys, a typical console controller, ~12 • PC games can support many different commands, but requires a great deal of hand movement • Console games are more likely to require multiple-key commands, context-sensitive commands, or both • For a game with a large control set, initiating the command will take longer on the PC, but completing the command will be faster

9. Map

10. Map and some other stuff

11. Map and some other stuff

12. The Six Most Important Things to Know • 1 – Consistency • Frame rate, art style, sound style, model/texture quality, object silhouettes, and lighting/shadows should all be consistent throughout the gaming experience • People are very good at noticing things that are out of place, which hurts, if not kills, the illusion of immersion

13. The Six Most Important Things to Know • 2 – Rhythm • Rhythmic patters often emerge during gameplay – DDR exploits this directly • Attack patterns in Doom • Can you play Donkey Kong with your eyes closed? • Design the control timing to match aural and visual cues in the game • Flying a dragon

14. The Six Most Important Things to Know • 3 – Expressiveness • Humans have a need for self-expression • Try to allow players to express themselves • Custom player names, clan affiliations, logos, preset phrases, colors, model type, taunts, salutes, weapons, and chat • The Sims takes this to an extreme, and is one of the key features that led to its success

15. The Six Most Important Things to Know • 4 – Orthogonality • Distinct actions should be kept separately controllable without interfering with each other • On a control pad, the Up and Right buttons are orthogonal, but the Up and Down buttons are not • i.e. “fire” and “dodge” bad, “jump” and “crawl” good • Be sure to map non-orthogonal commands to non-orthogonal control states

16. The Six Most Important Things to Know • 5 – Context • If your control scheme is limited, you may need to have one button do multiple actions based on context • In Hubie, Up will climb when there’s a wall nearby, otherwise Up will jump • Max Payne overloads the “use” key with the “zoom” key when the sniper rifle is equipped • In Eternal Darkness, the B button both shoots the gun and opens doors - problem!

17. The Six Most Important Things to Know • 6 – Fluidity • As games become more complex, it becomes more and more important to keep the interface fluid and natural • How best to implement an item selection screen when the list becomes huge? • Most players like the “flower” layout in The Sims, which would map well to a directional controller

19. A Fork in the Tale • Released January 1997 • Written by Advance Reality • Published by AnyRiver Entertainment • Distributed by Electronic Arts • Hired UI expert Jeff Johnson as consultant to solve UI design problems reported in playtesting

20. A Fork in the Tale • First-person FMV game • Follows “web” design of a Choose Your Own Adventure (as opposed to “tree” design) • Real-time – icons appear briefly every few seconds for player to make choice before a default is chosen automatically

21. The Problem • The symbols for controlling the movie action were much to complex for players • Symbols were non-intuitive, unsystematic, and numerous • Players needed to choose symbol quickly, so there had to be near-instant recognition

22. The Assignment • Devise a new control scheme • Organize and represent protagonist actions • Keep the game’s mysteries and puzzles challenging, but make operating the game as simple as possible

23. The Initial Analysis • Too many symbols – set into color, shape, and texture categories – over 200 total! • More semantic resolution than necessary – each action situation was different • Similar representation of different actions – hindered recognition of type of action • Flawed implementation – different editors used only symbols they liked, and added more

24. Requirements/Constraints • Keep clickable action symbols that appear at the bottom of the screen during choice points in the game • All available actions are presented simultaneously • No text • 3D look for icons – artistic integrity

25. First Recommendations • Categorize protagonist actions into six categories: navigate, look, interact, speak, think, memorize • Group into superordinate classes – physical movements and speech/thought actions • Physical actions would be represented by semitransparent icons that “float” over the scene • Speech/thought would be represented by cartoon speech balloons containing a symbol representing specific action

26. Redesign Physical Actions

27. Physical Actions Second Generation

28. Redesign Speech and Thought • 12 speech actions: statement, question, accept, refuse, offer help, aggressive/insult, sarcastic, humorous, frustrated/needing help, flatter/praise/thank, grovel/beg/plead, remember • For thought balloons, use a ‘?’ for Question, comedy mask for Humorous, and a shaking fist for Aggressive • Designer preferred common visual theme for speech symbols • Compromise was to use human figures making mime gestures

29. Initial Sketches

30. Testing the Icons

31. Testing the Icons

32. Speech/Thought Second Generation

33. Final Icons

35. The Moral of the Story • Finding the right picture to convey a verbal concept can be very hard. Some verbal concepts are best expressed verbally! • If symbols in a set depict their meanings well, users can discriminate and recognize them even if the set is large • A universal emotion scale may not exist – designs that map color to emotion are risky

36. The Future of Interface • Lately, we have force feedback technology, steering wheels, flight sticks, Mech suits (sort of), dance pads, fake snowboards, etc. • Better speech recognition and NLP • Better speech synthesis, with changes in tone, rhythm, modulation, and prosody (metrics) • What would you think if a synthetic character wouldn’t make eye contact with you, and repeatedly scratched his nose?

37. Homework #4 • Design a specific interface for your game • Visual layout • General “button” commands • Due Thursday 11/20

38. Final Project • Due Thursday 12/4 • Either: • Instruction manual for a board game, at least equal to the complexity of Clue. Include board diagram • or: • Design a digital, rule-based structured game. Explain what you think the emergent properties would be. 3-4 pages • or: • 6-8 page Design doc for a story/character based game. Same restrictions as Homework #3

39. IGDA meeting • Wednesday, 11/19, 7PM • The Larkin, 199 Lark St., Albany(upper level) • Chapter game project and production stages