1 / 33

Monte-Carlo Search Algorithms

Monte-Carlo Search Algorithms. Daniel Bjorge and John Schaeffer. Problem. Where it needs to be recognized. Important stuff. Solution. Best early prediction?. Which is faster?. Search Algorithms. Best β parameter?. Comparing Search Algorithms. Option 1: Computer Go.

dennis-leach
Download Presentation

Monte-Carlo Search Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Monte-Carlo Search Algorithms Daniel Bjorge and John Schaeffer

  2. Problem Where it needs to be recognized Important stuff

  3. Solution Best early prediction? Which is faster? • Search Algorithms Best β parameter?

  4. ComparingSearch Algorithms

  5. Option 1: Computer Go

  6. Option 2: Artificial Trees 010 110 010 110 010 110 010 110 010 110 010 110 010 110 010 110 010 110

  7. Existing Frameworks

  8. Existing Frameworks

  9. Gomba Search Framework(Go Multi-Armed Bandit Analysis)

  10. Eager Game Tree Generation (Small trees) • Tree Generator • Memory • Search Algorithm

  11. Eager Game Tree Generation (Big trees) • Memory • Tree Generator

  12. Lazy Game Tree Generation (Any trees) • Tree Generator • Search Algorithm • Memory

  13. Minimax Values Minimizing Maximizing 7 0 1 -4 9 6 3 4 -9

  14. Minimax Values Minimizing Maximizing 4 4 7 9 7 0 1 -4 9 6 3 4 -9

  15. Minimax Values Minimizing Maximizing 4 4 7 9 7 0 1 -4 9 6 3 4 -9 Go:

  16. Downward Minimax Evaluation 4 4 7 9 0 7 0 1 4 9 6 3 4

  17. Example Minimizing Maximizing 0 0 5 5 1

  18. Difficulty Minimax Optimal Easier White to play

  19. Quantified Difficulty Minimizing Maximizing 0 0 5 5 1

  20. Application to Search Algorithms

  21. Many-Armed Bandit Solutions 10 8 3 7 ... 0 1 0 0 ... 14 19 -27 -13 ... 1 2 2 2 ... -12 -4 -2 -8 …

  22. (KocsisandSzepesvári, 2006) UCTUpper Confidence Bound for Trees Optimal arm choice rate

  23. (Hartland et al., 2006) UCT-FPUUCT with First Play Urgency Optimal arm choice rate

  24. Infinitely-Armed Bandit Solutions … 10 8 3 7 ... -12 -4 -2 -8 …

  25. Infinite-Armed Bandit SolutionsAdapting to Trees • K-Failure • …for trees • UCB-V () • UCT-V () • UCB-V () AIR • UCT-V () AIR (Wang, Audibert, and Munos, 2006)

  26. K-Failure Optimal arm choices (out of 2000)

  27. UCT-V () … … … … … … … … …

  28. UCT-V () with AIR … … … … … … … … …

  29. UCT-V () with AIR: Gomba Optimal arm choices (out of 1000)

  30. In Fuego

  31. Wrapping Up

  32. Questions?

  33. Köszönjük szépen!

More Related