General Game Playing (GGP)

1. General Game Playing (GGP) Marco Adelfio CMSC 828N – Spring 2009

2. Classic Game Playing AI Deep Blue TD-Gammon Poki

3. General Game Playing AI GGP Agent

4. General Game Playing • GGP Goals: • Create systems to play arbitrary games (given formal game definitions) • Eliminate game-specific strategies • Emphasize generic strategy formulation • Competition created by Stanford Logic Group • Hosted during AAAI conference since 2005 • $10,000 Grand Prize

5. General Game Playing • Questions: • What additional challenges arise for GGP agents? • How should a GGP agent evaluate game states? • Can a GGP agent transfer knowledge between games?

6. General Game Playing • Finitely many players, states • Game play controlled by Game Manager over network • Players act synchronously (noops allowed) • Time limits enforced • Basic agent must: • Understand rule specification • Respond to game states with legal actions • Recognize a terminal state and its payoffs

7. Game Definition Language • A game definition must logically define: • Set of states in the game • Legal actions for each player from a given game state • Transition function • Initial state • Terminal states and their payoffs

8. Game Definition Language - Example (role p1) (role p2) (init (cell 1 1 b)) (init (cell 1 2 b)) … (init (control p1) … (<= (legal ?w (mark ?x ?y)) (true (cell ?x ?y b)) (true (control ?w))) … (<= (next (cell ?m ?n x)) (does p1 (mark ?m ?n)) (true (cell ?m ?n b))) … (<= (row ?m ?x) (true (cell ?m 1 ?x)) (true (cell ?m 2 ?x)) (true (cell ?m 3 ?x))) … (<= (line ?x) (row ?m ?x)) (<= (line ?x) (column ?m ?x)) (<= (line ?x) (diagonal ?x)) … (<= (goal p1 100) (line x)) (<= (goal p1 0) (line o) … (<= terminal (line x))

9. Game Communication

10. General Game Playing • Design Challenges: • Indeterminacy • Size • Multi-game Commonalities • Opponent Recognition

11. AAAI Competition – Past Winners • 2005 - ClunePlayer (UCLA) • 2006 - FluxPlayer (Technical University of Dresden) • 2007 - CADIA (Reykjavik University) • 2008 - CADIA (Reykjavik University)

12. Agent 1: ClunePlayer • Approach: Minimax • Problem: Needs to assign values to intermediate game states in arbitrary games. • Solution: • Calculate a vector of generic features at each node • Simulate games to determine which features are “stable” and correlated with either payoff or control • When running minimax, use a combination of those scores as the evaluation heuristic

13. Agent 2: CADIA-Player • Approach: UCT (Variant of Monte Carlo simulation) • Monte Carlo: • Pick random actions for each player to descend the tree • After reaching a terminal state, update expected payoff Q(s,a) for each visited state s and action a • Introduces explore/exploit tradeoff

14. Agent 2: CADIA-Player • UCT (Upper Confidence bound for Trees) • Balance exploration and exploitation • Give “bonus” to less travelled paths

15. Agent 3: UTexas LARG • Approach: Knowledge Transfer • Uses lessons from past games to improve play in new games • War Games! • Determines whether a new game is isomorphic or similar to a previous game. If so, transfer estimated rewards

16. Summary • General Game Playing introduces a different set of challenges than designing game-specific AI • Biggest challenge is evaluating states in a novel game • Better understanding of general strategy formation has many applications

17. References • GGP Website: http://games.stanford.edu/ • Hilmar Finnson. CADIA-Player: A General Game Playing Agent. MSc Thesis, School of Computer Science, Reykjavik University. 2007. • Kuhlmann, Gregory and Peter Stone. Graph-Based Domain Mapping for Transfer Learning in General Games. Lecture Notes in Computer Science, Volume 4701/2007.