Adversarial Games

1. Adversarial Games

2. Two Flavors • Perfect Information • everything that can be known is known • Chess, Othello • Imperfect Information • Player’s have each have partial knowledge • Poker: dispute is settled by revealing the contents of one’s hand

3. Two Approaches to Perfect Information Games • Use simple heuristics and search many nodes • Use sophisticated heuristics and search few nodes Cost of calculating the heuristics might outweigh the cost of opening many nodes The closer h is to h*, the better informed it is. But information can be expensive

4. MiniMax on exhaustively searchable graphs Two Players • min: tries to achieve an outcome of 0 • max: tries to achieve an outcome of 1

5. You are max at node A Expand the entire search space MAX A 0 C D Min B 1 Max G E F Min K 1 0 J H I Max O N 1 1 L M 0 0 R 0 P Q Min 1

6. Wins • Max: F,J,N,Q,L • Min: D,H,P,R,M Propagating Scores: A first pass • Min’s Turn a Node I • Go to M to win • So assign I a 0 • Max’s turn at node O • Go to Q to win • So assign 1 to node O

7. Conclusion • If faced with two labeled choices, you would choose 0 (if min) or 1 (if max) • Assume you’re opponent will play the same way

8. Propagating Scores For each unlabeled node in the tree • If it’s max’s turn, give it the max score of its children • If it’s min’s turn, give it the min score of its children Now label the tree Conclusion: Max must choose C at the first move or lose the game

9. Nim • 7 coins • 2 players • Players divide coins into two piles at each move, such that • Piles have an unequal number of coins • No pile is empty • Play ends when a player can no longer move

10. Start of Game min 7 4,3 5,2 6,1 5,1,1 4,2,1 Complete the game to see that min wins only if max makes a mistake at 6,1 or 5,2

11. α-β Pruning Problem • For games of any complexity, you can’t search the whole tree Solution • Look ahead a fixed number of plys (levels) • Evaluate according to some heuristic estimate • Develop a criterion for pruning subtrees that don’t require examination

12. Recast Instead of representing wins, numbers represent the relative goodness of nodes. At any junction • Max chooses highest • Min chooses lowest Higher means better for max Lower means better for min

13. Example: Max’s Turn 8 Max g 4 8 j k 9 t beta n m z Alpha 4 8 12 r p q Min 7 3 9

14. Situation Max’s turn at node g • Left subtree of g has been explored • If max chooses j, min will choose m • So the best max can do by going left is 8. Call this α Now Examine K and its left subtree n with a value of 4 • If max chooses k, the worst min can do is 4. • Because, T may be < 4. If it is min will choose it. If not, min will choose 4 • So the worst min can do, if max goes right is 4. Call this β

15. Question Should max expand the right subtree of k. No. Because min is guaranteed 4. But if max chose j, min is only guaranteed 8. Val(k) = min(4, val(t)) <= 4 Val(g) = max(8,val(k)) = max(8, min(4,val(t)) = 8

16. Leads to Max Principle Search can be stopped below any min node where β <= α of its max ancestor

17. Example: Min’s Turn 4 min k 9 4 n t d e 4 p q r Min Beta 4 3 7 9 alpha 3

18. Situation Min’s turn at node k • Left subtree of k has been explored • If min chooses n, max will choose d • So the best min can do by going left is 4. Call this β Now examine T and its left subtree P with a value of 7 • If min chooses T, the worst max can do is 7. • Because, Q or R may be > 7. If it is Max will choose it. If not, min will choose 7. • So the worst max can do, if min goes right is 7. Call this α

19. Question • Should min explore Q and R • No • Max is guaranteed 7 if min chooses T • But if min chooses N, max gets only 4 • Val(T) = max(7,val(Q), val(R)) >= 7 • Val(k) = min(4,val(T)) = 4

20. Leads to min principle Search can be stopped below any max node where α >= β of its min ancestor

21. To Summarize • Max’s turn • β is min’s guaranteed score (the worst min can do) • α is best max can do • Max principle • Search can be stopped below any min node where β <= α of its max ancestor • Min’s turn • α is max’s guaranteed score (the worst max can do) • β is best min can do • Min principle • Search can be stopped below any max node where α >= β of its min ancestor

30. Efficiency D = 2, B = 2: Terminal Nodes = 22 In general, Terminal Nodes = BD Suppose a tree has depth, D, and branching factor B

31. Ordering Ordering of nodes in a tree clearly affects the number than can be pruned using alpha/beta. Call ND the number of terminal nodes Can be shown that with alpha/beta best case performance is: ND = 2BD/2 – 1 for even D ND = 2B(D+1)/2 + B(D-1)/2 for odd D

32. Example Suppose B = 5, D = 6 w/out alpha/beta ND= 56 = 15625 w/alpha/beta ND= 2* 53 - 1 = 249 Approx. 1.6% of worst case

33. Reduction in Branching Factor Without alpha/beta pruning: ND = BD With alpha/beta pruning: ND = 2BD/2 - 1 for even D Reducing the branching factor used in computing ND from B to B1/2 (since: 1/2= BD/2

34. Average Performance On average, the peformance reduces B to B3/4 Suppose B = 5, Bab = 3.34 Suppose D = 6 Then ND = 56 = 15625 NDAB = 3.346 = 1388 ≈ 8.8% of worst case

35. Combinatorial Explosion • Key Idea: branching factor makes optimal solution intranctable • Sum of Subsets Problem • 2 • Traveling Salesperson • (N + 1)/2 • 8 puzzle • 2.67 • 15 puzzle • Approximately 4 Let B = average branching factor Let T = total nodes Let D = depth of search Then T = B + B2 + B3 + … + BD = B(BD – 1)/(B – 1) + 1

36. Sum of Subsets Given a set, S, of positive integers, find all subsets whose sum is m. E.G. S = {7,11,13,24} m = 31 Solutions S-1 = {7,11,13} S-2 = {7,24}

37. Problem Representation Solution is a sequence of 1s and 0s, indicating that elements of S have been chosen or not. Rep of S-1 (1,1,1,0) Rep of S-2 (1,0,0,1) State space is a tree where left turn indicates a 1 and right turn indicates a 0

38. Partial Space S-1 Three left turns and a right turn to get to S-1 is the sequence (1,1,1,0) Clearly the branching factor is 2

39. TSP Can also be represented as a state space search. Suppose 4 cities 4 Choices at level 0 3 Choices at level 1 2 Choices at level 2 1 Choice at level 3 T = 4*3*2*1 = 4P4 = 4!/(4-4)! = 4!

40. Branching Factor B_F = (sum of choices at each level)/#of levels = (4+3+2+1)/4 = 2.5 Clearly this increases with the size of the tour: (1+2+3+…+n)/n = (n(n+1)/2)/n = (n+1)/2

41. Relationship between branching factor and nodes in the tree Whenever the branching factor >= 2, we have an exponentially complex problem Let T = number of nodes in a full binary tree T = 20 + 21 + 22 + … + 2d-1 = 2d – 1 Easily proved through induction

42. Replace 2 by branching factor, B T = B0 +B1 + B2 + … + BL = B(BL -1)/(B-1) + 1 Where L is d-1, d being the depth of the tree Proof Basis: T = B(B0 – 1)/(B-1) + 1 = 1 Inductive hypothesis: T = B0 +B1 + B2 + … + BL = B(BL -1)/(B-1) + 1

43. Show that this is true at level L+1 That is, Show = B0 +B1 + B2 + … + BL+1 = B(BL+1 -1)/(B-1) + 1 B0 +B1 + B2 + … + BL + BL+1 = B(BL -1)/(B-1) + BL+1 + 1 = (B(BL -1) + BL+1(B-1))/(B-1) + 1 %common D = (B( (BL -1) + BL(B-1))/(B-1) + 1 %factor B out = (B(BL -1 + BL+1 –BL )/(B-1) + 1 %multiply b = B(BL+1 -1)/(B-1) + 1 %subtract Which is what we were trying to prove

44. Two Concepts • B – average number of descendents that emerge from any state in the space • Total nodes = B(BL -1)/(B-1) + 1 Where L is the deepest level (or, the depth of the search)

45. Another Problem: 8 Puzzle A B C 1 2 3 1 2 1 2 8 4 3 4 5 3 4 5 7 6 5 6 7 8 6 7 8 A: 4 moves for blank * 1 position = 4 B: 2 moves for blank * 4 positions = 8 C: 3 moves for blank * 4 positions = 12 B = (4 + 8 + 12) /(1 + 4 + 4) = 2.67 Does the branching factor of larger (15, 24) puzzles approach 4 as the puzzles get larger?