Game Theory

1. Game Theory Impartial Games,Nim, Composite Games, Optimal Play in Impartial games Learning & Development Team http://academy.telerik.com Telerik Software Academy

2. Table of Contents (1) • What is Game Theory? • Types of combinatorial games and plays • Impartial, Partisan • Normal & misère play • Game States • Turns; P, N positions • Optimal play • "Scoring" game example

3. Table of Contents (2) • The Game Nim • Gameplay, winning positions • Composite games • Tweedledum & Tweedledeeprinciple • Sprague-Grundy Theorem • Calculating Grundy Numbers • Minimal excludant

4. WTF Game Theory? What are The Facts on Game Theory

5. What is Game Theory? • Study of decision making • Mathematical models • Of conflict • Of cooperation • Between intelligent decision makers • Started with study of zero-sum games • One player winning leads to other player losing • Evolved into "decision theory" • General decision making strategies

6. Game Theory – Concepts • Players, Actions, Payoffs, Information (PAPI) • Key concepts to describing a game • Solution strategy is based on PAPI • Solution strategy + PAPI = predictable, deterministic outcomes to games • Game theory fields of application • Politics, Economics • Phycology, Biology, Logic • All fields needing info on behavioral relations

7. Game Theory – Game Types • Notable general game classifications • Perfect vs. imperfect information • Cooperative vs. competitive • Symmetric vs. asymmetric • Combinatorial • Infinitely long • Discrete vs. continuous, differential, population, stochastic, metagames…

8. Combinatorial Games Impartial, Partizan and Play Types

9. Combinatorial Games • Definition of a combinatorial game • Perfect information – players know at any time: • Game state • All possible player moves • No chance devices • Actions are not random and do not depend on random events Two players move alternatively No chance devices Perfect information The game must eventually end Winner depends on who moves last – no draws

10. Combinatorial Game Examples • Examples of combinatorial games • Chess (partly) • Hex 7 • Checkers • Take-away & Subtraction games • Silver-dollar game

11. Partisan vs. Impartial • Partisan games • Different moves available to different players • Chess, Checkers, Tic-Tac-Toe… are partisan • Impartial games • Different players have the same moves • Examples – Nim, Quarto • Impartial games can be generalized and analyzed • Very strong base for studying other games • Many partisan games, but with similar principles

12. Combinatorial Games • Play types – determine which player wins • Based on who moves last • Normal play • Player which makes last possible move wins • E.g. player who takes the last coin and wins • Misère play • Player which makes last possible move loses • Aim to force the other player into the last move

13. Game States & Positions Turns, P and N Positions in Optimal Play

14. Game States – Chips • Game “chips” (or “pieces”) • A chip is something a player controls • E.g. figures in chess, stack(s) of coins, numbers to operate on, etc. • Chips are always in some state (i.e. position) • A position can be winning or losing • Or, good for next or for previous player

15. Game States – Turns • Impartial games are played in turns • Turns can be broken down like so: • Positions are usually the "thinks" stage • Impartial game states – positions & count of chips Blue moves Yellow moves Blue moves Blue "thinks" Yellow "thinks" Blue "thinks" Yellow "thinks"

16. Game States – Player Position • Position of a player • Depends on the positions of all the chips • Considered in the "think" stage • Immediately after the other player moved • Immediately before the current player moves • Two types of positions • Good for current player (i.e. next to move) • Good for previous player (i.e. who just moved) • Usually noted P (previous) and N (next/current)

17. Game States – Optimal Play • Impartial games are deterministic • Optimal play always exists • Player position – either winning or losing • Optimal play in impartial games • Best actions by player, in order to win • Optimal play + winning position = victory • So, N and P positions for the current player: • N – Winning positions • P – Losing position

18. How to Play Optimally • Which positions to choose to win? • Impartial games are zero-sum • Positions are either winning or losing • Try to force other player into losing position • i.e. a P-position (good for previous player -> us) • If we are in a losing position, we can’t win • if other player plays optimally

19. Game of Scoring • Scoring is a game where a single chip is moved • Right to left, starting at a numbered position • Moves have maximal step – e.g. 3 positions • Movechip by no less than 1 and no more than 3 • 0 (zero) is the leftmost position • Can’t move from that position • Player which cannot move is the loser

20. Game of Scoring – Demo • Let’s try to determine P and N positions • Max step is 3

21. Game of Scoring – Positions • 0 – losing (P-position) • 1, 2, 3– winning positions (N-positions) • We can immediately move the chip to 0 • 4 – losing position (P-position) • We place chip at 1, 2 or 3 and other player wins • 5, 6, 7 – winning positions (N-positions) • Can place other player in a losing position (4) • Scoring has a direct formula for P-positions

22. Generalization of Positions • For all impartial games • A position is winning if it canleadto at least one losingposition • A position is losing, if it leadsonlyto winningpositions • This regards single-chipgames

23. Solving Scoring in C++ Live Demo

24. The Game Nim Gameplay, Positions, Nimbersand Nim-sum

25. Nim • Ancient, but fundamental game • Impartial, Many variations • Nim theory developed early 20th century • Important related terms – nimbers, Nim-sum • Rules • Several heaps (piles) of stuff (coins/stones/...) • Player takes 1 or more elements from 1 pile • Normal play – player taking last element wins

26. Nim – formal notation • Nim state is easy to express formally • If the number of heaps is k • Then (n0, n1, …, nk) is the state • n0 is the number of items in the first heap, etc. • State is often referred to as position • If we have 3 heaps of sizes 3, 4 and 2 • Then we are at position(3, 4, 2) • Position (0) is losing • In normal play

27. Nim – Demo • Let’s play Nim

28. Nim – Observations • What happens for Nim with 1-element heaps? • (1) – obviously, first player wins (N) • (1, 1) – first player takes a heap, second wins (P) • (1, 1, 1) – first player forces second player to case b), so second player will lose (N) • (1, 1, 1, 1) – first player goes into case c), so second player will win (P) • Nim with k 1-element heaps • Winning position if k is odd

29. Nim – More Observations • What happens for Nim with 1-element heaps mixed with 2-element heaps? • (1, 2) – first player wins (winning position) • first player can reduce 2 to 1 • player 2 forced into (1, 1), which is losing • (2, 2) – other player wins (losing position) • First player’s moves are to (1, 2) or (0, 2) * • First option leads second player to case a) • Second option – second player takes the heap *(0, 2) and (2, 1) are also possible, but with the same outcome, so we will not consider them

30. Nim – General Observations • We could go on generating positions • That would take too much (exponential) time • Another pattern can be noticed • Even same-sized heap count – bad position • Odd same-sized heap count – good position • b) + one larger heap – good position • can move to position a) by reducing larger heap • Other similar even-odd considerations • Can we easily filter through good/bad positions?

31. Nim-sum • Nim-sum (aka XOR, ^, addition modulo two) • The Nim-sum of a position (n0, …, nk) isn0 ^ n1 ^ … ^ nk • Accredited to Charles L. Boutonas part of the solution of the game • Mathematical notation typically:n0 ⊕ n1 ⊕ … ⊕ nk • Heap sizes are usually called nimbers A non-zero Nim-sum denotes a winning position A zero Nim-sum denotes a losing position

32. Nim-sum – Why it Works • Position (0) has a Nim-sum = 0 • No moves can be made – position is losing • Position (k) has a Nim-sum > 0 (k > 0) • Player takes the entire heap – position is winning • Changes to a position with Nim-sum = 0 • Always lead to positions with Nim-sum > 0 • Changes to a position with Nim-sum > 0 • At least one change leading to Nim-sum = 0 • So, we can always force a losing position • From a winning position

33. Nim-sum – Finding Positions • Considering we are in a winning position • Need to search for positions with Nim-sum zero • Finding a zero Nim-sum position • Decrease numbers, which make the Nim-sum > 0 • Achieve even number of 1’s in each bit column Calculate the current Nim-sum Finds its leftmost bit equal to 1 (denotes a column with an odd number of 1’s) Find any number N (heap size) having a 1 at that bit Nullify that number N Calculate the Nim-sum again Set the nullified number N to the new Num-sum

34. Solving Nim in C++ Live Demo

35. NimImportance • What’s the big deal with Nim? • The Sprague-Grundy theorem • Who? • R. P. Sprague & P. M. Grundy in 1935 & 1939 • Independently discovered the theorem • The Sprague-Grundy theorem states: • Layman’s terms: every impartial game is a Nim in disguise or some variant of it Every impartial game, under normal play is equivalent to a nimber

36. Nim Importance • Consider the game Nimble • Several coins, at some positions • Must move exactly one coin at least 1 position • Coins move only right to left (towards zero) • Player who moves the last coin to 0 wins • Seem familiar?

37. Nim Importance • How about now? • The solutions of Nimble and Nim are equivalent • After indices are turned into heap sizes

38. Composite Games Dividing into Subgames, Tweedledum& Tweedledee Principle, Sprague-Grundy

39. Game of Scoring – Demo • Let’s play Scoring again • This time with 2 chips • Max step is 3 • Are the marked P/N (make them win/lose if you want) positions correct in this case?

40. Composite Games – 2 Chips • Tweedledum & Tweedledee Principle • 2-chip game, the second player can mimic moves • If the two chips are in the same position • Leading to victory for the second player • In positions which would be winning for the first player in a single-chip game • Losing positions remain losing, problem is with winning ones • Similar cases occur in games with more chips • And more equivalent positions

41. Composite Games – Solving • Composite impartial games • Equivalent to nimbers (as other impartial games) • As per the Sprague-Grundy theorem • Yep, these guys again • A game has k chips, each with a position, • Game position – set of k numbers (n0, …, nk) • Equivalent to a position in Nim, hence • Nim-sum of the position = 0 -> position is losing • How do we get those numbers/nimbers?

42. Calculating the Sprague-Grundy Function Follower positions, Minimal excludent

43. Calculating SG function • Sprague-Grundy function • Recursively generates "Grundy" values/numbers • Numbers correspond to heap sizes in Nim • Adapt to specific game through • Follower function • Mex function

44. Calculating SG function • Sprague-Grundy function – formal description: • g – Sprague-Grundy function • F – follower function • Mex – minimal excludant • p and q – two game positions g(p) = Mex(g(q)); q ∈F(p)

45. Calculating SG function • Follower function F(position) • Returns positions, reachable in 1 move from given position • i.e. "followers" of that position • Minimal excludantMex(numbers) • Returns the minimal non-negative number • Not belonging to a given set • i.e. first number different than any of the given numbers

46. Calculating SG function • So, thisreads as follows: • The Grundy value of position p • Is the minimal non-negative integer • Which is NOT a Grundy value of • Any of the followers of position p g(p) = Mex(g(q)); q ∈F(p)

47. Calculating SG – example • Grundy values with Scoring • Losingpositions ina game haveg() = 0 • Here, can’tmove from 0, so we lose there and g(0) = 0 g(0) = 0, by definition. g(1) = 1 since F(1) = {0}. g(2) = 2 since F(2) = {0, 1}. g(3) = 3 since F(3) = {0, 1, 2}. g(4) = 0 since F(4) = {1, 2, 3}. - The values of g on F(4) are {1, 2, 3} and the minimum that does not appear there is 0. g(5) = 1 since F(5) = {2, 3, 4}. - The values of g on F(5) are {2, 3, 0} and the minimum that does not appear there is 1...

48. Calculating SG – pseudocode • Grundy values pseudocode • Walk positionsrecursively • Generate setof Grundy values for followers • Take the Mexof the Grundyvalues of the followers int GrundyNumber(position pos) { moves[] = Followers(pos); set s; for (all x in moves) { insert into s GrundyNumber(x); } //Mex – return smallest non-zero //integer, not in s; int mexCandidate=0; while (s.contains(mexCandidate)) { mexCandidate++; } return mexCandidate; }

49. Solving Composite Games Using SG & Nim • Determining a position as winning or losing • We have the positions of the chips in the game • We have the Grundy values of those positions • Hence, we have the Nim position: • (g(p0), g(p1), … g(pk)) • Where p0 … p1 are the chip positions • So, we can compute the Nim-sum • If it is zero, we are in a losing position • If it is non-zero, we are in a winning position

50. Solving Multi-Chip Scoring in C++ Live Demo