A Survey on Computer Game Players

1. A Survey onComputer Game Players M1 Hirokazu Ishii Chikayama & Taura Lab

2. Agenda • Introduction • Computer Game Players in General • Computer Go Players • Features of Go • Implementations • Conclusion

3. Introduction

4. Computer Game Players • Has been studied for many years • Easy to evaluate • Suitable to try out various basic technologies • Was successful in some games • Othello, backgammon, chess ... • Great challenges remain in some games • Shogi, go, ...

5. Computer Game Players in General

6. Game Players • Important factors • Recognizing the position • Generating candidate moves • Searching candidate moves • Game tree search • Minimax • Alpha-beta • Evaluating candidate moves • Evaluation function • Selecting the best move

7. Computer Go Players

8. Features of Go • Simple and clear rules • Stones enclosed by opponent's are taken off. • Sizes of obtained area decide the winner. • Require both strategy and tactics. • One of the most complex board games.

9. Difficulties of Go • Large search space • Criteria of evaluation are not clear • Other difficulties • Existence of Ko • Obscure definition of the end of a game… • While a chess program had beaten the human champion, go programs has stopped at an amateur level (1 Dan).

10. Large Search Space • Large branching factor • Full-board search is too costly • Decompose the board into a number of games called subgame

11. Criteria of Evaluation • A single stone does not have its own value. • Roles of stones changes according to circumstances. • Influence distribution

12. Existing Programs • ‘Gifu challenge’ is one of the most prosperous competition. Result

13. Structure Description Current position Data structure Candidate moves The best move Next position

14. Structure Description Current position Data structure Candidate moves The best move Next position

15. Data Structure • Meaningful unit of stones • String, group • Other data • Moyo, liberty, eye, ... String Group

16. Structure Description Current position Data structure Candidate moves The best move Next position

17. Generating Candidate Moves • Local evaluation • Full-board evaluation • Knowledge based • Pattern Matching • Search • Minimax • Alpha-beta

18. HARUKA • Knowledge based • Fuseki routine • Joseki routine • Search • Life-and-death search routine • Semeai search routine • Center search routine • Local search routine

19. HARUKA • Data structure • Probability of capturing stones • Probability of cutting stones • Influence distribution • Eye shape …

20. HARUKA • Generate candidate moves • Alpha-beta search • Previous opponent move • Previous own move • Area surrounding the point which is influenced by two moves • Evaluate the move • Influence distribution

21. Katsunari • Generate candidate moves • Knowledge based • Shape move • Joseki move • Search • Life-and-death move

22. Katsunari • Memory-based reasoning • Data structure • Shape • The move which High-level human played • Emergency • Compare shape data with current position • Criteria of similarity • Difference of stone • Distance from candidate move to surrounding stones • Distance from candidate move to the nearest stone

23. Katsunari • Value of shape move • Emergency • Similarity • Distance from the nearest stone

24. GNU Go • Generate candidate moves • Move reasons • Attacking and defending stones • Cutting and connecting stones • Making or destroying eyes… • Evaluate candidate moves • Correct move reasons

25. GNU Go • Knowledge based • Pattern matching • Fuseki • Joseki • Connecting and cutting stones • Search • Pattern-based minimax search • Life-and-death of string and group

26. NeuroGo • Full-board search • 1-ply search • Evaluation function • Neural network 1-ply search

27. NeuroGo • Input • The go position • Some local features of the position • A number of stones • A number of liberties • Output • Probability to become alive at each point

28. NeuroGo Example of output

29. NeuroGo • Learning • Self-play and TD (0) learning • Backpropagation • Reinforcement learning • Single-point eye • Connection • Alive

30. NeuroGo • Network Architecture (a number of neurons per point) • Input layer ( one or more ) • The activations is set to 0 or 1 according to whether a input feature is true or not. • First hidden layer ( one or more ) • The number of neurons is parameter of the network architecture. • Second hidden layer ( one or more ) • The number of neurons is parameter of the network architecture. • Simple eyes layer ( 2 (black 1, white 1) ) • The activation is a prediction of whether that color is able to create a single point eye at this point. • It receives a reinforcement signal.

31. NeuroGo • Network Architecture (a number of neurons per point) • Local connections layer ( 18 (black 9, white 9 ) • The activation is a prediction of whether that dolor is able to create a connection from this point to each of the 9 points in a 3×3 window around this point. • It receives a reinforcement signal. • Global connectivity layer ( 2×(board size)2 ) • The activation is a prediction whether each color is able to create a connection from this point to any point on the board. • Evaluation layer ( 1 ) • The activation is a prediction whether this point will be alive for Black ( activation 1 ) or White (activation 0 ).

32. NeuroGo • 3 connections • Receptive fields • 3×3 square window centered at this point. • Connectivity pathfinder • Creating a global connectivity map from local connections layer. • Connectivity-based weight selection • Assigning weights from global connectivity layer.

33. Comparison • There is no program which excels other programs crushingly.

34. Conclusion

35. Conclusion • I introduced ... • Brief of general game • System description of go • Four go players • There is no complete solution. • Game tree search • Learning • Parallel computing