1 / 19

Parallel Monte-Carlo Tree Search with Simulation Servers

Parallel Monte-Carlo Tree Search with Simulation Servers. H IDEKI K ATO †‡ and I KUO T AKEUCHI † † The University of Tokyo ‡ Fixstars Corporation November 7 th , 2008. Contents. Computer Go Monte-Carlo Tree Search Parallel Monte-Carlo Tree Search Client-Server Approach

apria
Download Presentation

Parallel Monte-Carlo Tree Search with Simulation Servers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Parallel Monte-Carlo Tree Search with Simulation Servers HIDEKI KATO†‡ and IKUO TAKEUCHI† † The University of Tokyo ‡ Fixstars Corporation November 7th, 2008

  2. Contents Computer Go Monte-Carlo Tree Search Parallel Monte-Carlo Tree Search Client-Server Approach Experiments and Discussion Conclusion and Future Work

  3. Computer Go • The game of Go • Task par excellence for AI (H. Berliner 1978) • Most challenging; largest search space • 19 x 19 10171, 9 x 9 1038 cf. Chess 1050 • Minimax tree search and a static evaluation function with domain knowledge was used so far without major success • The Monte-Carlo Go revolution • MoGo beat an 8-dan professional player on 9 x 9 • Crazy Stone beat a 4-dan professional player with 8 stones handicap

  4. Repeat until time-up Descend tree from root to leaf Add a node Simulate a game Update values of the moves Play most visited move in root Monte-Carlo Tree Search (MCTS)

  5. Parallel MCTS (PMCTS) • Symmetrical multi-thread (SMT) PMCTS • Commonly used straightforward implementation • MCTS threads share a search tree Thread 2 Thread 1 Lock Thread 4 Thread 3 Search tree (shared)

  6. Related Work • S. Gelly et al. introduced SMT PMCTS for shared-memory SMP systems (2006) • T. Cazenave et al. proposed and evaluated three PMCTS algorithms on a 16 Intel Pentium-4 MPI cluster (2007) • G. Chaslot et al. evaluated root, leaf and treeparallelization on 2 x 8 core IBM Power5 (2008) • S. Gelly et al. proposed SMT PMCTS for MPI clusters of shared-memory SMP nodes (2008)

  7. Problems • Number of processors • Shared tree PMCTS can run only on shared-memory systems; currently up to 16 or 32 processors • PMCTS algorithms for clusters of computers connected through networks is necessary • Longer communication time decreases performance like other parallel applications • Increasing the threads increases the overhead of the locks to share search tree

  8. MoGo’s Solution • Combine fine and coarse grain PMCTS • For MPI clusters with shared-memory SMP nodes (S. Gelly et al. 2008) • Runs SMT PMCTS on each node • Periodically exchanges and merges values in the tree • Excellent performance • MoGoTitan beat an 8-dan Korean Professional Go player with 9 stones handicap (2008) Huygens super computer at SARA in Amsterdam, the Netherlands 25 out of 104 SMP nodes were used Each node consists of 16 dual core Power6 processors at 4.7 GHz

  9. MoGo’s Solution (cont’d) • Disadvantages • Expensive • High speed network interfaces such as InfiniBand are very expensive (so are the clusters) • Lack of flexibility • MPI does not allow to add or remove computers on the fly • MPI requires special setup; must be pre-configured • Applicable to non-MPI clusters on moderate speed networks? • Nobody tried yet

  10. Client-Server Approach • Recent success of grid computing • Folding@home achieved one petaflop with major benefits by 41,145 Sony Playstation 3 consoles all over the world (2007) • Less expensive massive parallel approach • Applicable to PMCTS? • Basic idea • Separate tree search part and simulation part • Broadcast positions to be simulated using UDP/IP • Don’t wait the end of slow simulations

  11. Client-Server Approach (cont’d) • Client-server PMCTS • A client searches tree and send a position; a server simulates a game from the position and sends back the result • Runs on a cluster of loosely-coupled computers • Servers can run on small memory computers even if the tree is going to be huge • No special set-up for servers; just a small application • Longer communication time due to moderate speed networks • Performance? Scales well?

  12. Client-Server PMCTS Client Repeat until time-up Search tree Descend tree from root to leaf Server 2 Server 1 Repeat forever Repeat forever Add a node Receive positions Receive positions Broadcast the position Receive a result (no wait) Simulate a game Simulate a game Update values of the moves Send the result Send the result Loop Select most visited move in root

  13. Experimental System PC1 (1 client and 3 servers) PC2 (4 servers) CPU: Q9550/3GHz (400 x 7.5) OS: Ubuntu Linux 8.04 M/B: ASUS P5K-VM (G33) RAM: PC3200 4GiB NIC: Intel EXP9300PT (PCI-Ex x1) CPU: Q6600/3GHz (333 x 9) OS: Ubuntu Linux 8.04 M/B: ASUS P5K-VM (G33) RAM: PC3200 4GiB NIC: Intel EXP9300PT (PCI-Ex x1) RTT: 154±20 ms @ 1 kB Switch Allied Telesis GS908XL Switching delay: 2.2 ms @ 64 byte PC3 (4 servers) PC4 (4 servers) CPU: Q6600/3GHz (333 x 9) OS: Ubuntu Linux 8.04 M/B: ASUS P5WDG2-WS Pro (975X) RAM: PC3200 4GiB NIC: Intel EXP9300GT (PCI) RTT: 159±22 ms @ 1 kB CPU: Q9550/3GHz (400 x 7.5) OS: Ubuntu Linux 8.04 M/B: DFI LP JR P45-T2RS (P45) RAM: PC3200 4GiB NIC: Intel EXP9300PT (PCI-Ex x1) RTT: 151±22 ms @ 1 kB

  14. Experiments • A tree searcher or a simulator exclusively uses a core • One core or other on the client computer is used for a tree searcher or a simulator thread, respectively • The simulators on the server computers run as individual processes • All results are ELO ratings against GNU Go 3.7.11 level 0

  15. How to evaluate the results? • Simulations per second? • Commonly used for shared memory SMP systems but not a good measure for clusters • The benefits of simulations are not the same • Use equivalent-strength speed-up • The ratio of time-per-move settings that give the same strength at different number of simulators • “Equivalent speed-up” for short • Number of simulators or cores • The number of simulators is used to evaluate scalability while the number of all cores is used to evaluate performance

  16. 300 200 100 0 -100 -200 -300 1/16 1/8 1/4 1/2 1 2 4 8 Equivalent Speed-up 16 core (9 x 9) 16 core (13 x 13) ELO rating 4 core (9 x 9) 4 core (13 x 13) Time per move (s)

  17. Performance (4 core vs. 16 core) 13 x 13 9 x 9

  18. Scalability 9 x 9 (0.08 s/move) ELO rating 13 x 13 (0.4 s/move) 1 2 3 4 5 7 8 11 15 Number of simulators

  19. Conclusion and Future Work • Client-server parallel Monte-Carlo tree search • Runs on a cluster of loosely coupled computers • Small memory computers such as game consoles can be used for simulation servers • Allows servers to connect or disconnect on-the-fly • Reduced communication by broadcasting • No overhead to share search tree • Scales well on 13 x 13 with 15 simulators • Future work • Multiple clients for single or multiple users

More Related