1 / 60

The Science of Play Testing: EA’s Methods for User Research

The Science of Play Testing: EA’s Methods for User Research. The Science of Play Testing: EA’s Methods for User Research. Veronica Zammitto Game User Researcher. Outline:. Game User Experience Evaluations Case Study 1: NBA Live 10 Case Study 2: NHL 11 Take Away Q&A.

chibale
Download Presentation

The Science of Play Testing: EA’s Methods for User Research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Science of Play Testing: EA’s Methods for User Research The Science of Play Testing: EA’s Methods for User Research Veronica Zammitto Game User Researcher

  2. Outline: • Game User Experience Evaluations • Case Study 1: NBA Live 10 • Case Study 2: NHL 11 • Take Away • Q&A

  3. Game User Experience Evaluation

  4. Mixed Method Approach for Evaluating Sports Games • In-depth understanding of User Experience (UX). • Support design decisions. • Triangulating: Eye Tracking Interviews Survey UX Telemetry Psychophysiology

  5. Case Study 1: NBA Live10 • NBA gameplay issues  to identify: • successful and unsuccessful gameplay aspects. • emotional profile of the player • engagement and emotions • attentional focus • UX throughout the game, and for certain events.

  6. Eye Tracking • Hardware + software  X, Y on screen • Tracking users’ gaze can reveal the player’s focus. • DIYS, ~US$ 4,000 to 80,000

  7. Eye Tracking • By using ET we can identify where players’ attention is. • Fixation • Saccades • Gaze Movement • Patterns

  8. Play Styles video

  9. Triangulating Eye Tracking & Survey

  10. Lay Up Dunk Pass Call Time Out Switch Players Steal Telemetry • Hooks in the game engine that flag and time stamp pre-defined events. • Players’ in-game behavior • Statistical analysis • Visualizations • Machine learning algorithms

  11. Events Performed by Players Through Timein NBA 10

  12. Events Performed by Players Through Timein NBA 10

  13. Passes sent by players

  14. Players’ Scoring Location 58.6 % 91.4 % of the shots

  15. AI Scoring Location 75 %

  16. What Went Right – NBA study • Better understanding of : • players styles and demographics. • In-game behavior • Identification of emotions (next slides) • The new techniques were proven to provide useful data to development. • Rethink the role of game elements. I.e., coach. • Create tutorials for court observation based on eye tracking data • Worthy of further investment to continue with studies

  17. What Went Wrong • Large scope of the NBA study • Impacted synchronization of the usability study with production’s delivery schedule. • The study should have been subdivided into mini assessments to achieve a quicker turn around. • Low involvement of production in the project. • Manual coding: • Time consuming.

  18. Case Study 2: NHL 11 • Same techniques used for NBA • Adjustments from lessons learnt: • Narrower focus: “Game Presentation” • Front-End Visualizations (Overlays). • I.e.: Do players look at information provided in the UI? • Cut Scenes (NIS): • Are they watched or skipped? • High involvement with the development team • Meetings with development for a ‘statement of work’. • Iterative process with development • Helps to define the root of usability questions.

  19. Front End: Overlays

  20. Overlays Distribution

  21. Only ¼ of the overlays are actually observed, the other 75% are ignored.

  22. Percentage of Observed and Ignored Overlays during different game sections. • Quality and sensitive information that helps the player has more chances to be looked during NISes.

  23. Overlay “Map” in NHL 11with Observed and Ignored proportions

  24. How people observed the overlays.

  25. Non-Interactive Sequence (NIS) Event NISes Scripts - Subsets

  26. NIS’ Map

  27. NISes by event and type

  28. The effectiveness of NIS is a combination of its type (Canned, Context Sensitive, and Replay), the event that triggers the sequence, and its frequency.

  29. Psychophysiology (Biometrics) arousal • Infer emotions from physiological data. • Arousal: engagement, excitement, magnitude of emotions. • Valence: positive (fun) or negative (frustration) Frustrating Exciting Emotional Valence Boring Fun

  30. Galvanic Skin Response (GSR) • Psychological arousal. • Skin’s conductance increases when a person becomes excited, stressed, or anxious.

  31. Electromyography (EMG) • Emotional valence. • Sensors capture and amplify muscles’ contractions. • Facial muscles: • Smiling (zygomatic) = Positive emotions • Frowning (corrugator) = Negative emotions (or) Cognitive load

  32. Facial EMG

  33. Player’s positive emotional reaction when scoring in NBA Live 10

  34. Isomorphism between telemetry and GSR

  35. Emotional Profiling of NHL 11 Arousal = exciting, engagement.

More Related