On the Borel and von Neumann Poker Models

1. On the Borel and von Neumann Poker Models

2. Comparison with Real Poker • Real Poker: • Around 2.6 million possible hands for 5 card stud • Hands somewhat independent for Texas Hold ‘em • Let’s assume probability of hands comes from a uniform distribution in [0,1] • Assume probabilities are independent

3. The Poker Models • La Relance Rules: • Each player puts in 1 ante before seeing his number • Each player then sees his/her number • Player 1 chooses to bet B/fold • Player 2 chooses to call/fold • Whoever has the largest number wins. • von Neumann Rules: • Player 1 chooses to bet B/check immediately • Everything else same as La Relance

4. The Poker Models • http://www.cs.virginia.edu/~mky7b/cs6501poker/rng.html

5. La Relance • Who has the edge, P1 or P2? Why? • Betting tree:

6. La Relance • The optimal strategy and value of the game: • Consider the optimal strategy for player 2 first. It’s no reason for player 2 to bluff/slow roll. • Assume the optimal strategy for player 2 is: • Bet when Y>c • Fold when Y<c • Nash’s Equilibrium

7. La Relance • P2 should choose appropriate c so that P1’s decision does not affect P2: • If PI has some hand X<c, the decision he makes should not affect the game’s outcome. • Suppose PI bets B • P1 wins 1 if P2 has Y<c (since he folds ‘optimally’) • P1 loses B+1 if P2 has Y>c (since he calls ‘optimally’) • Suppose P1 folds • P1 wins -1 Which yields:

8. La Relance • We knew the optimal strategy for P2 is to always bet when Y>c. Assume the optimal strategy for player I is: • Bet when X>c (No reason to fold when X>c since P2 always folds when Y<c) • Bet with a certain probability p when X<c (Bluff) • Now PI should choose p so that P2’s decision is indifferent: Using Bayes’ theorem:

9. La Relance • Consider P2’s Decision at Y=c: • If P2 calls with Y=c, he/she wins pot if X<c and loses if X>c: • If P2 folds, Value for P2 is -1. • Solve the equation: We get:

10. La Relance • Now we can compute the value of the game as we did in AKQ game: • Result shows the game favors P2.

11. La Relance • When to bluff if P1 gets a number X<c? • Intuitively, P1 bluffs with c2<X<c, (best hand not betting), bets with X>c and folds with X<c2. • Why? • If P2 is playing with the optimal strategy, how to choose when to bluff is not relevant. • This penalizes when P2 is not following the optimal strategy.

12. La Relance • What if player / opponent is suboptimal? • Assumed Strategy • player 1 should always bet if X > m, fold otherwise • player 2 should always call if Y > n, fold otherwise, Also call if n > m is known (why?) • Assume decisions are not random beyond cards dealt • Alternate Derivations Follow

13. La Relance

14. La Relance (Player 2 strategy)

15. La Relance (Player 2 strategy) • What can you infer from the properties of this function? • What if m ≈ 0? What if m ≈ 1?

16. La Relance (Player 1 response) • Player 1 does not have a good response strategy (why?)

17. La Relance (Player 1 Strategy) • Let’s assume player 2 doesn’t always bet when n > m • This function is always increasing, is zero at n = β / (β + 2) • What should player 1 do?

18. La Relance (Player 1 Strategy) • If n is large enough, P1 should always bet (why?) • If n is small however, bet when m > • What if n = β / (β + 2) exactly?

19. Von Neumann • Betting tree:

20. Von Neumann

21. Von Neumann • Since P1 can check, • now he gets positive value out of the game • P1 now bluff with the worst hand. Why? • On the bluff part, it’s irrelevant to choose which section of (0,a) to use if P2 calls (P2 calls only when Y>c) • On the check part, it’s relevant because results are compared right away.

22. Von Neumann • Nash’s equilibrium: • Three key points: • P1’s view: P2 should be indifferent between folding/calling with a hand of Y=c • P2’s view: P1 should be indifferent between checking and betting with X=a • P2’s view: P1 should be indifferent between checking and betting with X=b

23. Von Neumann • What if player / opponent is suboptimal? • Assumed Strategy • Player 1 Bet if X < a or X > b, Check otherwise • Player 2 Call if Y > c, fold otherwise • If c is known, Player 1 wants to keep a < c and b > c

24. Von Neumann

25. Von Neumann

26. Von Neumann (Player 1 Strategy) • Find the maximum of the payoff function • a = • b = • What can we conclude here?

27. Von Neumann (Player 2 Response) • Player 2 does not have a good response strategy

28. Von Neumann (Player 2 Strategy) • This analysis is very similar to Borel Poker’s player 1 strategy, won’t go in depth here… • c =

29. Bellman & Blackwell • Bet tree Where • Borel: B1= B2 • Von Neumann: B1= 0

30. Bellman & Blackwell mL mH b1 b3 High B High B Low B Fold Low B b2

31. Bellman & Blackwell • Where Or if

32. La Relance: Non-identical Distribution • Still follows the similar pattern • Where F and G are distributions of P1 and P2, c is still the threshold point for P2. π is still the probability that P1 bets when he has X<c. • What if ?

33. La Relance:(negative) Dependent hands • X and Y conforms to FGM distribution • Marginal distributions are still uniform. • is correlation factor. means negative correlation.

34. La Relance:(negative) Dependent hands • Player 1 bets when X > l • P(Y < c | X = l) = B / (B + 2) • Player 2 bets when Y > c • (2*B + 2)*P(X > c|Y = c) = (B + 2)*P(X > l|Y = c) • Game Value: • P(X > Y) – P(Y < X) • + B * [ P(c < Y < X) – P(l < X < Y AND Y > c) ] • + 2 * [ P(X < Y < c AND X > l) – P(Y < X < l) ]

35. Von Neumann:Non-identical Distribution • Also similar to before (just substitute the distribution functions) • a | (B + 2) * G(c) = 2 * G(a) + B • b | 2 * G(b) = G(c) + 1 • c | (B + 2) * F(a) = B * (1 – F(b))

36. Von Neumann:(negative) Dependent hands • Player 2 Optimal Strategy: • Player 1 Optimal Strategy:

37. Discussion / Thoughts / Questions • Is this a good model for poker?