The challenge of poker

1. The challenge of poker NDHU CSIE AI Lab 羅仲耘

2. Outline • Introduction • Texas Hold’em rules • Poki’s architecture • Betting Strategy • Pre-flop betting strategy • Basic betting strategy • Reference the challenge of poker

3. Introduction • Bridge and poker involve imperfect information • Traditional methods like deep search can’t play these game well the challenge of poker

4. Texas Hold’em rules (1) 1. Opening deal – Each player is dealt two cards face down, which are known as hole cards or pocket cards. 2. First round of betting – Starting with the player to the left of the big blind, each player can call the big blind, raise, or fold. The big blind has the option to raise an otherwise unraised pot. the challenge of poker

5. Texas Hold’em rules (2) 3. The flop – The dealer burns a card, and then deals three community cards face up. The first three cards are referred to as the flop, while all of the community cards are collectively called the board. the challenge of poker

6. Texas Hold’em rules (3) 4. Second round of betting – Starting with the player to the left of the dealer button, each player can check or bet. Once a bet has been made, each player can raise, call, or fold. the challenge of poker

7. Texas Hold’em rules (4) 5. The turn – The dealer burns another card, and then adds a fourth card face-up to the community cards. This fourth card is known as the turn card, or fourth street. the challenge of poker

8. Texas Hold’em rules (5) 6. Third round of betting – It follows the same format as the second round, but the size of the bets have usually doubled in limit games. the challenge of poker

9. Texas Hold’em rules (6) 7. The river – The dealer burns another card, and then adds a fifth and final card to the community cards. This fifth card is known as the river card, or fifth street. the challenge of poker

10. Texas Hold’em rules (7) 8. Final round of betting – It follows the same format as the second and third rounds. 9. The showdown – Using the best five-card combination of their hole cards and the community cards, the remaining players show their hands, with the bettor or last raiser showing first. The highest five-card hand wins the pot. (In case of a tie, the pot is evenly split among the winning hands.) the challenge of poker

11. Poki’s architecture the challenge of poker

12. Pre-flop betting strategy(1) • Income rate: base on roll simulation • Roll simulation: an off-line computation that consists of playing several million hands (trials) where all players call the first bet, and then all the remaining cards are dealt out without any further betting • David Sklansky wrote books on this game and classify the hand cards. the challenge of poker

13. Pre-flop betting strategy(2) the challenge of poker

14. Pre-flop betting strategy(3) • Prefer roll-out simulation than David Sklansky’s classification • Roll-out simulation information is quantitative rather than qualitative • Can apply many difference specific situation the challenge of poker

15. Basic betting strategy(1) • Hand strength (HS): is the probability that a given hand is better than that of an active opponent • For example: hand is A♦-Q♣ and the flop is J♥-4♣-3♥.47 remaining unknown cards and therefore {47 choose 2} = 1081 possible hands an opponent might hold. 444 cases are better than hand, 9 are equal, and 628 are worse than hand. • There is a 58.5% chance that hand is better than a random hand. • If there are 5 opponents, the HS5 is 0.5855 = 6.9% the challenge of poker

16. Basic betting strategy(2) • Hand potential • Positive potential (PPot): Chance that a hand which is not currently the best improves to win at the showdown • Negative potential (NPot): Chance that a currently leading hand ends up losing • All possible opponent cards is {47 choose 2} = 1081, therefore remains {45 choose 2} = 990 possible turn and river cards to consider for each opponent the challenge of poker

17. Basic betting strategy(3) • For example: if the hand A♦-Q♣ is ahead against one opponent after 5 cards, then after 7 cards there is a 449005 / 621720 = 72% chance of still being ahead the challenge of poker

18. Basic betting strategy(4) • Effective hand strength (EHS): combines hand strength and potential to give a single measure of the relative strength of Poki’s hand against an opponent P(win) = P(ahead) x P(opponent does not improve) + P(behind) x P(we improve) = HS x (1 - NPot) + (1 - HS) x PPot the challenge of poker

19. Basic betting strategy(5) • Weighting the enumerations • The probability of each hand being played to a particular point in the game will very • For example: the probability that the opponent holds Ace-King is much higher than 7-2 after the flop, because most will fold 7-2 before the flop the challenge of poker

20. Basic betting strategy(6) • To account for this, Poki maintains a weight table for each opponent • The weights have a value in the range zero to one the challenge of poker

21. Basic betting strategy(7) • For example:[ A♠K♣, 0.40 ],…[ Q♦2♦, 0.20 ],…the PredictOpponentAction procedure generates probability distributions { P( fold), P( check / call), P( bet / raise)} A♠ - K♣ { 0.0, 0.7, 0.3 }Q♦ - 2♦ { 0.0, 0.1, 0.9 } • If the observed player action is a bet, after reweighting, the new table entry for A♠ - K♣ will be 0.4 x 0.3 = 0.12Q♦ - 2♦ will be 0.2 x 0.9 = 0.18 the challenge of poker

22. Basic betting strategy(8) • Probability triple PT = {f, c, r}, such that f + c + r = 1.0, representing the probability distribution that the next betting action in a given context is fold, call, or raise, respectively the challenge of poker

23. Reference • The challenge of Poker/ Darse Billings, Aaron Davidson, Jonathan Schaeffer, Duane Szafron / Artificial Intelligence 134 (2002) • http://www.cardplayer.com/rules-of-poker/texas-holdem.php • http://www.ultimatebet.com/rules-strategy/texas-holdem.html the challenge of poker

24. Thanks for your attention! the challenge of poker