Turn-Based Games

1. Turn-Based Games • sources: • http://www.game-research.com/ • www.gamespot.com • Wikipedia.org • Russell & Norvig AI Book; Chapter 5 (and slides) • Jonathan Schaeffer’s AAW 05 presentation • My own Héctor Muñoz-Avila

2. Turn-Based Strategy Games • Early strategy games was dominated by turn-based games • Derivate from board games • Chess • The Battle for Normandy (1982) • Nato Division Commanders (1985) • Turn-based strategy: • game flow is partitioned in turns or rounds. • Turns separate analysis by the player from actions • “harvest, build, destroy” in turns • Two classes: • Simultaneous • Mini-turns

3. Turn-Based Games Continues to be A Popular Game Genre • At least 3 sub-styles are very popular: • “Civilization”-style games • Civilization IV came out last week • Fantasy-style (RPG) • Heroes of Might and Magic series • Poker games • Poker Academy

4. Some Historical Highlights • 1952 Turing design a chess algorithm. Around the same time Claude Shannon also develop a chess program • 1956 Maniac versus Human • 1970 Hamurabi. A game about building an economy for a kingdom • The Battle for Normandy (1982) • 1987 Pirates! • 1990 Civilization • 1995 HoMM • 1996 Civilization II • The best game ever? • … • 2005 Civilization IV • 2006 HoMM V

5. Side-tracking: Game Design: Contradicting Principles • Principle: All actions can be done from a single screen. • Classical example: Civ IV • But: HoMM uses two interfaces: HoMM IV

6. Coming back: How to Construct Good AI? • Idea: Lets just use A* and define a good heuristic for the game • Search space: a bipartite tree • After all didn’t we use it with the 9-puzzle game? • Problems with this idea: • Adversarial: we need to consider possible moves of our opponent (s) • Time limit: (think Chess)

7. Types of AdversarialTBGs (from AI perspective) Chance Deterministic Chess, Go, rock-paper-scissors Perfect information Backgammon, monopoly Bridge, Poker Imperfect information Battleships, Stratego Civilization, HoMM

8. Game tree (2-player, deterministic, turns) • Concepts: • State: node in search space • Operator: valid move • Terminal test: game over • Utility function: value for outcome of the game • MAX: 1st player, maximizing its own utility • MIN: 2nd player, minimizing Max’s utility

9. Minimax • Finding perfect play for deterministic games • Idea: choose move to position with highest minimax value = best achievable payoff against best play • E.g., 2-play game:

10. Minimax algorithm

11. Properties of minimax • Complete? • Optimal? • Time complexity? • b: branching factor • m: # moves in a game Yes (if tree is finite) Yes (against an optimal opponent) O(bm) • For chess, b ≈ 35, m ≈100 for "reasonable" gamesTherefore, exact solution is infeasible

12. Cutoff-test(state) evaluationFunction(state) Cutoff-test(state) evaluationFunction(state) Minimax algorithm with Imperfect Decisions

13. Chess • weight: Piece  Number • (w1) Pawn  1 • (w2) Knight  3 • (w3) Bishop  3 • (w4) Rook  5 • (w5) Queen  9 • Function; state  Number • f1 = #(pawns,w)  #(pawns,b) • f2 = #(knight,w)  #(knight,b) • f3 = #(bishop,w)  #(bishop,b) • f4 = #(rook,w)  #(rook,b) • f5 = #(knight,w)  #(knight,b) Evaluation Function • Evaluation Function • Is an estimate of the actual utility • Typically represented as a linear function: EF(state) = w1f1(state) + w2f2(state) + … + wnfn(state) • Example:

14. Example: Evaluation Function “all things been equal” White moves, Who is winning? Is this consistent with Evaluation function? Black Yes!

15. Evaluation Function (2) • Obviously, the quality of the AI player depends on the evaluation function • Conditions for evaluation functions: • If n is a terminal node, • Computing EF should not take long • EF should reflect chances of winning EF(n) = Utility(n) If EF(state) > 3 then is almost-certain that blacks win

16. Cutting Off Search • When to cutoff minimax expansion? • Potential problem with cutting off search: Horizon problem • Solution: • Fixed depth limit • Iterative deepening until times runs out • Decision made by opponent is damaging but cannot be “seen” because of cutoff • Quiescent: states that are unlikely to exhibit wild swings in the values of the evaluation functions

17. Example: Horizon Problem “all things been equal” White moves, Who is winning? Is this consistent with Evaluation function? Black No!

18. α-β pruning: Motivation • A good program may search 1000 positions per second • In a chess tournament, a player gets 150 seconds per move • Therefore, the program can explore 150,000 positions per move • With a branching factor of 34, this will mean a look ahead of 3 or 4 moves • Facts: • 4-turns ≈ human novice • 8-turns ≈ typical PC, human master • 12-turns ≈ Deep Blue, Kasparov • How to look ahead more than 4 turns? Use α-β pruning

19. Example: • Finding perfect play for deterministic games • Idea: choose move to position with highest minimax value = best achievable payoff against best play • E.g., 2-play game:

20. α-β pruning

21. α-β pruning example

22. α-β pruning example

23. α-β pruning example

24. α-β pruning example

25. α is the value of the best (i.e., highest-value) choice found so far at any choice point along the path for max If v α, max will avoid it Therefore, prune that branch β is the lowest-value found so far at any choice point along the path for min If v α, min will avoid it Therefore, prune that branch Principle of α-β Prunning

26. The α-β algorithm

27. The α-β algorithm

28. Properties of α-β • Pruning preserves completeness and optimality of original minimax algorithm • Good move ordering improves effectiveness of pruning • With "perfect ordering," time complexity = O(bm/2) Therefore, doubles depth of search • Used in PC games today (9 moves look-ahead, Grand Master level)

29. Deterministic games in practice • Checkers: Chinook ended 40-year-reign of human world champion Marion Tinsley in 1994. Used a precomputed endgame database defining perfect play for all positions involving 8 or fewer pieces on the board, a total of 444 billion positions. • Chess: Deep Blue defeated human world champion Garry Kasparov in a six-game match in 1997. Deep Blue searches 200 million positions per second, 24 processors, quiescent identified with help of human grand masters • Othello: human champions refuse to compete against computers, who are too good. • Go: human champions refuse to compete against computers, who are too bad. In go, b > 300, so most programs use pattern knowledge bases to suggest plausible moves.

30. Additional Notes • The next 5 slides are form David W. Aha (NRL) presentation at Lehigh University in Fall’04

31. Example Game: FreeCiv(Chance, adversarial, imperfect information game) Civilization II(MicroProse) • Civilization II (1996-): 850K+ copies sold • PC Gamer: Game of the Year Award winner • Many other awards • Civilization series (1991-): Introduced the civilization-based game genre FreeCiv (Civ II clone) • Open source freeware • Discrete strategy game • Goal: Defeat opponents, or build a spaceship • Resource management • Economy, diplomacy, science, cities, buildings, world wonders • Units (e.g., for combat) • Up to 7 opponent civs • Partial observability http://www.freeciv.org

32. FreeCiv Scenario General description • Game initialization: Your only unit, a “settler”, is placed randomly on a random world (see Game Options below). Players cyclically alternate play • Objective: Obtain highest score, conquer all opponents, or build first spaceship • Scoring: “Basic” goal is to obtain 1000 points. Game options affect the score. • Citizens: 2 pts per happy citizen, 1 per content citizen • Advances: 20 pts per World Wonder, 5 per “futuristic” advance • Peace: 3 pts per turn of world peace (no wars or combat) • Pollution: -10pts per square currently polluted • Top-level tasks (to achieve a high score): • Develop an economy • Increase population • Pursue research advances • Opponent interactions: Diplomacy and defense/combat

33. FreeCiv Concepts Concepts in an Initial Knowledge Base • Resources: Collection and use • Food, production, trade (money) • Terrain: • Resources gained per turn • Movement requirements • Units: • Type (Military, trade, diplomatic, settlers, explorers) • Health • Combat: Offense & defense • Movement constraints (e.g., Land, sea, air) • Government Types (e.g., anarchy, despotism, monarchy, democracy) • Research network: Identifies constraints on what can be studied at any time • Buildings (e.g., cost, capabilities) • Cities • Population Growth • Happiness • Pollution • Civilizations (e.g., military strength, aggressiveness, finances, cities, units) • Diplomatic states & negotiations

34. FreeCiv Decisions Civilization decisions • Choice of government type (e.g., democracy) • Distribution of income devoted to research, entertainment, and wealth goals • Strategic decisions affecting other decisions (e.g., coordinated unit movement for trade) City decisions • Production choice (i.e., what to create, including city buildings and units) • Citizen roles (e.g., laborers, entertainers, or specialists), and laborer placement • Note: Locations vary in their terrain, which generate different amounts of food, income, and production capability Unit decisions • Task (e.g., where to build a city, whether/where to engage in combat, espionage) • Movement Diplomacy decisions • Whether to sign a proffered peace treaty with another civilization • Whether to offer a gift

35. FreeCiv CP Decision Space Variables • Civilization-wide variables • N: Number of civilizations encountered • D: Number of diplomatic states (that you can have with an opponent) • G: Number of government types available to you • R: Number of research advances that can be pursued • I: Number of partitions of income into entertainment, money, & research • U: #Units • L: Number of locations a unit can move to in a turn • C: #Cities • Z: Number of citizens per city • S: Citizen status (i.e., laborer, entertainer, doctor) • B: Number of choices for city production Decision complexity per turn (for a typical game state) • O(DNGRI*LU*(SZB)C) ; this ignores both other variables and domain knowledge • This becomes large with the number of units and cities • Example: N=3; D=5; G=3; R=4; I=10; U=25; L=4; C=8; Z=10; S=3; B=10 • Size of decision space (i.e., possible next states): 2.5*1065 (in one turn!) • Comparison: Decision space of chess per turn is well below 140 (e.g., 20 at first move)