Turn-Based Games

1. Turn-Based Games • sources: • http://www.game-research.com/ • www.gamespot.com • Wikipedia.org • Russell & Norvig AI Book; Chapter 5 (and slides) • My own Héctor Muñoz-Avila

2. Turn-Based Strategy Games • Early strategy games was dominated by turn-based games • Derivate from board games • Chess • The Battle for Normandy (1982) • Nato Division Commanders (1985) • Turn-based strategy: • game flow is partitioned in turns or rounds. • Turns separate analysis by the player from actions • “harvest, build, destroy” in turns • Two classes: • Simultaneous • Mini-turns

3. Turn-Based Games Continues to be A Popular Game Genre • At least 3 sub-styles are very popular: • “Civilization”-style games • Civilization IV came out last week • Fantasy-style (RPG) • Heroes of Might and Magic series • Poker games • Poker Academy

4. Some Historical Highlights • 1952 Turing design a chess algorithm. Around the same time Claude Shannon also develop a chess program • 1956 Maniac versus Human • 1970 Hamurabi. A game about building an economy for a kingdom • The Battle for Normandy (1982) • 1987 Pirates! • 1990 Civilization • 1995 HoMM • 1996 Civilization II • The best game ever? • … • 2005 Civilization IV • 2006 HoMM V

5. Side-tracking: Game Design: Contradicting Principles • Principle: All actions can be done from a single screen. • Classical example: Civ IV • But: HoMM uses two interfaces: HoMM IV

6. Coming back: How to Construct Good AI? • Idea: Lets just use A* and define a good heuristic for the game • Search space: a bipartite tree • After all didn’t we use it with the 9-puzzle game? • Problems with this idea: • Adversarial: we need to consider possible moves of our opponent (s) • Time limit: (think Chess)

7. Types of AdversarialTBGs (from AI perspective) Chance Deterministic Chess, Go, rock-paper-scissors Perfect information Backgammon, monopoly Bridge, Poker Imperfect information Battleships, Stratego Civilization, HoMM

8. Game tree (2-player, deterministic, turns) • Concepts: • State: node in search space • Operator: valid move • Terminal test: game over • Utility function: value for outcome of the game • MAX: 1st player, maximizing its own utility • MIN: 2nd player, minimizing Max’s utility

9. Minimax • Finding perfect play for deterministic games • Idea: choose move to position with highest minimax value = best achievable payoff against best play • E.g., 2-play game:

10. Minimax algorithm

11. Properties of minimax • Complete? • Optimal? • Time complexity? • b: branching factor • m: # moves in a game Yes (if tree is finite) Yes (against an optimal opponent) O(bm) • For chess, b ≈ 35, m ≈100 for "reasonable" gamesTherefore, exact solution is infeasible

12. Cutoff-test(state) evaluationFunction(state) Cutoff-test(state) evaluationFunction(state) Minimax algorithm with Imperfect Decisions

13. Chess • weight: Piece  Number • (w1) Pawn  1 • (w2) Knight  3 • (w3) Bishop  3 • (w4) Rook  5 • (w5) Queen  9 • Function; state  Number • f1 = #(pawns,b)  #(pawns,w) • f2 = #(knight,b)  #(knight,w) • f3 = #(bishop,b)  #(bishop,w) • f4 = #(rook,b)  #(rook,w) • f5 = #(knight,b)  #(knight,w) Evaluation Function • Evaluation Function • Is an estimate of the actual utility • Typically represented as a linear function: EF(state) = w1f1(state) + w2f2(state) + … + wnfn(state) • Example:

14. Evaluation Function (2) • Obviously, the quality of the AI player depends on the evaluation function • Conditions for evaluation functions: • If n is a terminal node, • Computing EF should not take long • EF should reflect chances of winning EF(n) = Utility(n) If EF(state) > 3 then is almost-certain that blacks win

15. Cutting Off Search

16. α-β pruning example

17. α-β pruning example

18. α-β pruning example

19. α-β pruning example

20. α-β pruning example

21. Properties of α-β • Pruning does not affect final result • Good move ordering improves effectiveness of pruning • With "perfect ordering," time complexity = O(bm/2) doubles depth of search • A simple example of the value of reasoning about which computations are relevant (a form of metareasoning)

22. α is the value of the best (i.e., highest-value) choice found so far at any choice point along the path for max If v is worse than α, max will avoid it  prune that branch Define β similarly for min Why is it called α-β?

23. The α-β algorithm

24. The α-β algorithm

25. Resource limits Suppose we have 100 secs, explore 104 nodes/sec106nodes per move Standard approach: • cutoff test: e.g., depth limit (perhaps add quiescence search) • evaluation function = estimated desirability of position

26. Evaluation functions • For chess, typically linear weighted sum of features Eval(s) = w1 f1(s) + w2 f2(s) + … + wn fn(s) • e.g., w1 = 9 with f1(s) = (number of white queens) – (number of black queens), etc.

27. Cutting off search MinimaxCutoff is identical to MinimaxValue except • Terminal? is replaced by Cutoff? • Utility is replaced by Eval Does it work in practice? bm = 106, b=35  m=4 4-ply lookahead is a hopeless chess player! • 4-ply ≈ human novice • 8-ply ≈ typical PC, human master • 12-ply ≈ Deep Blue, Kasparov

28. Deterministic games in practice • Checkers: Chinook ended 40-year-reign of human world champion Marion Tinsley in 1994. Used a precomputed endgame database defining perfect play for all positions involving 8 or fewer pieces on the board, a total of 444 billion positions. • Chess: Deep Blue defeated human world champion Garry Kasparov in a six-game match in 1997. Deep Blue searches 200 million positions per second, uses very sophisticated evaluation, and undisclosed methods for extending some lines of search up to 40 ply. • Othello: human champions refuse to compete against computers, who are too good. • Go: human champions refuse to compete against computers, who are too bad. In go, b > 300, so most programs use pattern knowledge bases to suggest plausible moves.

29. Summary • Games are fun to work on! • They illustrate several important points about AI • perfection is unattainable  must approximate • good idea to think about what to think about