1 / 28

Applied machine learning in game theory

Applied machine learning in game theory. Dmitrijs Rutko Faculty of Computing University of Latvia. Joint Estonian-Latvian Theory Days at Rakari, 2010. Topic outline. Game theory Game Tree Search Fuzzy approach Machine learning Heuristics Neural networks

sallystewart
Download Presentation

Applied machine learning in game theory

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Applied machine learning in game theory Dmitrijs Rutko Faculty of Computing University of Latvia Joint Estonian-Latvian Theory Days at Rakari, 2010

  2. Topic outline • Game theory • Game Tree Search • Fuzzy approach • Machine learning • Heuristics • Neural networks • Adaptive / Reinforcement learning • Card games

  3. Research overview • Deterministic / stochastic games • Perfect / imperfect information games

  4. Finite zero-sum games

  5. Topic outline • Game theory • Game Tree Search • Fuzzy approach • Machine learning • Heuristics • Neural networks • Adaptive / Reinforcement learning • Card games

  6. Game trees

  7. max 8 min 2 8 max 2 7 8 9 1 2 7 4 3 6 8 9 5 4 √ √ √ Χ Χ √ √ √ Χ Χ Classical algorithms • MiniMax • O(wd) • Alpha-Beta • O(wd/2)

  8. Advanced search techniques • Transposition tables • Time efficiency / high cost of space • PVS • Negascout • NegaC* • SSS* / DUAL* • MTD(f)

  9. max ≥5 min <5 ≥5 max <5 ? ≥5 ≥5 1 2 7 4 3 6 8 9 5 4 √ √ Χ Χ Χ √ Χ √ Χ Χ Fuzzy approach • O(wd/2) • More cut-offs

  10. α β 2 8 X1 X3 X2 Geometric interpretation • 1) X2 - successful separation • 2) X1 or X3 - reduced search window • α = X1 • β = X3

  11. BNS enhancement through self-training • Traditional statistical approach

  12. Two dimensional game sub-tree distribution

  13. Statistical sub-tree separation

  14. Experimental results. 2-width trees

  15. Experimental results. 3-width trees

  16. Future research directions in game tree search • Multi-dimensional self-training • Wider trees • Real domain games

  17. Topic outline • Game theory • Game Tree Search • Fuzzy approach • Machine learning • Heuristics • Neural networks • Adaptive / Reinforcement learning • Card games

  18. Games with element of chance

  19. Expectiminimax algorithm • Expectiminimax(n) = • Utility(n) • If n is a terminal state • Max s Successors(n) Expectiminimax(s) • if n is a max node • Min s Successors(n) Expectiminimax(s) • if n is a min node • s Successors(n) P(s) * Expectiminimax(s) • if n is a chance node • O(wdcd)

  20. Perfomance in Backgammon *-Minimax Performance in Backgammon, Thomas Hauk, Michael Buro, and Jonathan Schaeer

  21. Backgammon • Evaluation methods • Static – pip count • Heuristic – key points • Neural Networks

  22. Temporal difference (TD) learning • Reinforcement learning • Prediction method

  23. Experimental setup • Multi-layer perceptron • Representation encoding • Raw data (27 inputs) • Unary (157 inputs) • Extended unary (201 inputs) • Binary (201 input) • Training game series – 400 000 games

  24. Learning results

  25. Program “DMBackgammon”

  26. Topic outline • Game theory • Game Tree Search • Fuzzy approach • Machine learning • Heuristics • Neural networks • Adaptive / Reinforcement learning • Card games

  27. Artificial Intelligence and Poker* * Joint work with Annija Rupeneite

  28. Questions ? dim_rut@inbox.lv

More Related