1 / 39

The Turing Test for Game AI

The Turing Test for Game AI. Bing Gordon Electronic Arts June 2005. “I wish someone would…”. …develop a 'metric' for computer npc ai. Kind of an eliza scorecard.”. 1. Context Powers of 10…. 1981 Atari VCS 128 bytes 1986 NES 2K + 2K 1989 Sega Genesis 64K+64K

thiery
Download Presentation

The Turing Test for Game AI

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Turing Test for Game AI Bing Gordon Electronic Arts June 2005

  2. “I wish someone would…” • …develop a 'metric' for computer npc ai. • Kind of an eliza scorecard.”

  3. 1. Context Powers of 10… • 1981 Atari VCS 128 bytes • 1986 NES 2K + 2K • 1989 Sega Genesis 64K+64K • 1994 Sony Playstation 2 MB + 512K • 2001 Playstation 2 32MB +4MB • 2006 PSX3/Xbox 2 512MB • 2012? PS4/Xbox 3? 5G?

  4. Context Powers of polish … • 75% of 90+ rated games are hits. • 25% of 90+ rated movies are hits. Source: Metacritic.com

  5. Aspirations Tiger swing Gameface Franchise mode Dynamic sports news Assymmetrical sides Badges Anti-grav gun Auction House Living city Crash mode Freestyle Console online voice and custom maps Powered by Feature Innovation

  6. Quality bars of the near-future • Procedurally generated content • “Emergent” behaviors, collisions • Believable characters • 100x physics • Portable avatars, persistent assets • Communities • Economies and money • Camera POV and LOD drives gameplay • Collaborative and dynamic intelligences

  7. AI could be a “killer app” feature of next gen • Characters: • Awareness • Memory • Complex motives, simple commands • 100x RAM allocation • Procedural responses • Parsers • Stitching • Pathfinding and cover • Must be co-developed with animators! • Game AI must be acted out and seen • Expressions & gestures> pathing

  8. EA Learning We are Influenced by the success of our “Killer Apps” • Pinball Construction Set -- floppy disk • Deluxe Paint -- graphics co-processor • EA Sports – 16-bit processor • Interactive movies – cd-rom • MMO’s – servers • Sims – object-oriented programming • Spore – procedural animation • Etc.

  9. 2. EA Learning EA Sports is a tech story • 16-bit (1990) Sports simulations • Individual ratings • Dynamic simulations • Playstation (1994) 3D Sports • Z-axis passing, tackling • Playstation 2 (2000) Articulation • Skeletal modeling and animation

  10. What EA learned from John Madden • The Oakland Raiders playbook • Matchup strategy • 5 zones of field-position • One Knee Equals Two Feet • Player ratings • All-Madden team

  11. The Madden Test (of game AI) • 1985: “That’s not football!” • 1990: “I’d fire the coach!” • 1995: “What are those guys doing?” • 2000: “Rookie, you’re cut!” • 2005: “That’s the way I designed it!”

  12. Madden Football Genesis23%

  13. Madden 97 Playstation54%

  14. Madden 2001 Playstation 263%

  15. Madden 2005 PS283%

  16. 97 -- Grand Theft Auto III 97 -- Tony Hawk's Pro Skater 3 96 -- Metal Gear Solid 2: Sons of Liberty 95 -- Grand Theft Auto: Vice City 95 -- Grand Theft Auto: San Andreas 95 -- Gran Turismo 3 A-Spec 95 -- Madden NFL 2003 95 -- Tony Hawk's Pro Skater 4 95 -- Devil May Cry 94 -- God of War 94 -- Madden NFL 2002 94 -- Madden NFL 2004 94 -- Burnout 3: Takedown 94 -- NCAA Football 2004 94 -- Virtua Fighter 4 93 -- Pro Evolution Soccer 93 -- SSX 93 -- SSX 3 93 -- World Soccer Winning Eleven 6 International 93 -- NFL2K 3. Reviewers What EA learned from www.metacritic.com

  17. 3. Reviewers AI is not critical…yet • AI cited for 6/20 top PS2 games. • Metal Gear, NFL x 4, Soccer • AI cited for 3/10 top PC games. • Half Life x 2, Civilization

  18. When AI is applauded… • Appropriate npc behaviors • Dynamic adjustments • Satisfying mistakes • Coordinated attacks/retreats • Challenging opponent

  19. When AI is punished… • Too easy • No cover • “Money plays” • Too dumb • Low awareness • “Deer in headlights”

  20. "Do NOT duck inside those doors, men! Honorable warriors need no cover!" Pointless Waste of Time.com

  21. 4. An experiment A Turing Test for Game AI(version 1.0) • Clips from 5 games • Madden, FIFA, Fight Night, Mortal Kombat • Many others could not be tested • “Player or AI controlled?” • App by Karim Mahrous, UCD phd candidate

  22. Format of test • Watch a clip, multiple replays • Player or AI control for specific character. • Rationale: • Pathing • Reaction • Timing • Effectiveness • Orientation • Awareness • 80 respondents

  23. App

  24. Madden 2001

  25. Madden 2001Results

  26. Madden 2005 – Clip 1

  27. Madden 2005 – Clip 1Results

  28. Madden 2005 – Clip 2

  29. Madden 2005 – Clip 2Results

  30. Results are discouraging (so far) • 60% accurate • Clip > game or technology • 0 correlation between rationale, correctness • Possible improvement in “orientation” AI between Madden 01 and 05.

  31. 5. For example Feature metacritic planning for

  32. Metal Gear Solid 2 95 metacritic overall Cited by 4/5 top reviews #2-4 feature mentioned “communication chain” “perceptive reinforcements” “teams of soldiers will find you” “frighteningly intelligent” “Outclassing every game on PS2” Half Life 2 95 metacritic overall Cited by 2/3 top reviews #5 feature mentioned “use cover effectively” “run away from grenades” “actively hunt players down and flank you ” “if you hurt them, they set up traps” “Nothing less than incredible” Competitive quality barEnemy Combat AI

  33. Civilization 2 94 overall Cited by 3/3 top reviews #2-5 feature mentioned “you pay for your mistakes” “clever and resourceful in negotiation” “respects your power in battles” “may act like a bonehead at times” Madden Football 94 overall Cited by 3/6 top reviews #2-6 feature mentioned “Geek out the AI to the viciousness you like” “Players make plays all over the field” “Tough half-time adjustments” “Gone are the massive scores…” “AI is advanced to another level” Hall of fame feature Opponent command AI

  34. Hall of Fame FeaturePathfinding and formations • Rome: Total War • 92 overall • Cited by 2/4 top reviews • #5 feature mentioned • “enjoyable and convincing” • “centered around troop morale” • “moments of divine inspiration” • “on occasion, cretinous”

  35. Recommended Spore AI goals • Half Life 2: enemy hunting AI • Metal Gear Solid 2: communication chains • Civilization 2: city/star combat and diplomacy • Madden Football: “half time adjustments” • Rome Total War: pathfinding and formations • Plus: • Procedural combat • Collaborative filtering

  36. 10 predictions Areas of discovery for next-gen features

  37. My bookshelf • Confessions of an Advertising Man, Ogilvy • Positioning, Ries and Trout • The Fifth Discipline, Senge • How Hits Happen, Farrell • Net Gain, Hagel & Armstrong • The Innovator’s Dilemma, Christenson • The Medici Effect, Frans Johansson

  38. Thanks • bgordon@ea.com

More Related