1 / 28

SARTRE: System Overview A Case-Based Agent for Two-Player Texas Hold'em

Jonathan Rubin & Ian Watson University of Auckland Game AI Group http://www.cs.auckland.ac.nz/research/gameai/. SARTRE: System Overview A Case-Based Agent for Two-Player Texas Hold'em. Overview. Introduction Texas Hold'em Approaches to Computer Poker Sartre: System Overview Results

iago
Download Presentation

SARTRE: System Overview A Case-Based Agent for Two-Player Texas Hold'em

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Jonathan Rubin & Ian Watson University of Auckland Game AI Group http://www.cs.auckland.ac.nz/research/gameai/ SARTRE: System OverviewA Case-Based Agent for Two-Player Texas Hold'em

  2. Overview • Introduction • Texas Hold'em • Approaches to Computer Poker • Sartre: System Overview • Results • Future Work

  3. Texas Hold'em • Two-player Limit Hold'em • Much different to full-table game • Chance events • Hidden Information

  4. Approaches to Computer Poker • Near-Equilibrium Strategy • Exploitative Strategy

  5. Near-Equilibrium Strategy • Nash Equilibrium • Assumes the opponent makes no mistakes • Attempts to minimise its loses against this perfect opponent • Near-Equilibrium • As game tree is too large • Plays not to lose

  6. Exploitative Strategy • Exploitative Strategy • Opponent Modelling • Attempts to punish weaknesses in the opponents strategy • Plays off the equilibrium • Plays to win

  7. Sartre: System Overview • Similarity Assessment Reasoning for Texas hold'em via Recall of Experience • Our entry for the 2009 Computer Poker Competition • Case-base was constructed from past CPC games

  8. Sartre: System Overview • Hand picked by authors • Case Features • Previous betting for the hand • Hand Category • Board Category

  9. 1. Previous betting for the hand • Currently represented as a string • f = fold • c = check/call • r = bet/raise • Examples • r • rrc-r • rc-crrc-rc-cr

  10. 1. Previous betting for the hand

  11. 2. Hand Category • Rule-based System

  12. 2. Hand Category • Two components • Hand Category • Hand Potential • Examples • Missed • One-Pair, Two-Pair, Three-of-a-kind • Flush-draw, Straight-draw

  13. 3. Board Category • Captures information about potential • Flush Draws or, • Straight Draws • Information that is likely to be noticed by an good player

  14. 3. Board Category • Flush Highly Possible

  15. 3. Board Category • Straight Possible

  16. Similarity • Currently either all or nothing • If a collection of cards maps to the same category they are assigned a similarity of 1.0, otherwise 0.

  17. Case Overview • Case Features • 1. Previous betting for the hand • 2. Hand Category • 3. Board Category • Solution • f, c, r • Outcome • +/- value • + Profit • - Loss

  18. Case Overview • Solution + Outcome • Recorded from equilibrium approaching bots from previous AAAI Computer Poker Competition • Separate case-bases for preflop, flop, turn & river • Approx. 250,000 cases in each case-base.

  19. Decision Making • Retrieved cases can have different decisions • Three different versions • 1. Probability Triple • 2. Majority rules • 3. Outcome-based

  20. Decision Making • Probability Triple • Proportion of times that the solution indicated to fold, call or raise • (f, c, r) • Majority Rules • Decision made the most is reused • Outcome-Based • Dependant on adjusted average outcome values for each decision • If a call or raise decision was never made, it's outcome is unknown and is given a value of +infinity

  21. Duplicate Matches • Experimental results derived using duplicate matches • Play N poker hands • Reset each players memory • Reverse the position of each player and deal the same N hands • Forward + Reverse Directions • Reduces variance

  22. Self-Play Experiments • Small bets per hand (sb/h) • Assuming a $10/$20 game • Sartre-Probability Vs. Sartre-Outcome • Sartre-Probability wins 0.168 sb/h • On average $1.68 profit per hand • Sartre-Probability Vs. Sartre-Majority • Sartre-Majority wins 0.039 sb/h • On average $0.39 per hand

  23. Self-Play Experiments • Chose Sartre – Majority Rules. • Results not transitive • Makes Sartre more predictable and hence more exploitable by strong opposition

  24. 2009 Computer Poker Competition Results • Duplicate match structure • 3000 hands in forward & reverse direction • Multiple matches against each opponent until statistical significance obtained • Sartre placed 7th out of 13 entrants in limit competition

  25. 2009 Computer Poker Competition Results

  26. 2009 Computer Poker Competition Results • Overall profit of +0.097 sb/h • Assuming a $10/$20 game • $0.97 per hand profit

  27. Future Work • Investigate loosening of all-or-nothing similarity • CBR and adaptive poker agents • Opponent modelling • Learning • Better solution adaptation • Combination of decision + outcome

  28. The End!

More Related