Learning to rank moves in mahjong using SVM with tree kernels

1. Learning to rank moves in mahjong using SVM with tree kernels Chikayama & Taura Lab. M1 Ayato Miki

2. Outline • Introduction • Related work • Proposed method • SVM with kernels • Tree kernels in mahjong • Learning to rank using SVM • Experiment • Conclusion

3. 1. Introduction • Features = elements to evaluate positions in games • e.g. Numbers and arrangements of pieces in Shogi • Difficulty in creating features • Require expert knowledge for the game • Simple linear combinations are insufficient • e.g. XOR

4. Objective • Use kernels for evaluation features in games • Simple inputs • Tree structure • Expect to work as non-linear features • Implicit classification in high level feature space

5. Method overview • Classification of moves in mahjong using SVM with kernels • “Evaluation functions > search” in mahjong • Kernel method is effective • Tree kernels for tree structures of mahjong hands • Similarity representation by kernel functions • Use expert game records • Learn “expert moves > other moves” with SVM

6. Outline • Introduction • Related work • Proposed method • SVM with kernels • Tree kernels in mahjong • Learning to rank using SVM • Experiment • Conclusion

7. 2. Related work • Machine learning using simple features • TD-Gammon [Tesauro, 1992] • Research about mahjong • Learning from expert records [Kitagawa, 2007]

8. Machine learning in mahjong [Kitagawa, 2007]

9. Mahjong features[Kitagawa, 2007]

10. Outline • Introduction • Related work • Proposed method • SVM with kernels • Tree kernels in mahjong • Learning to rank using SVM • Experiment • Conclusion

11. Outline • Introduction • Related work • Proposed method • SVM with kernels • Tree kernels in mahjong • Learning to rank using SVM • Experiment • Conclusion

12. Support Vector Machine (SVM)[Vapnik, 1965] • 2-class linear classifier Maxmizemargin

13. SVM with kernels[Cortes and Vapnik, 1995] • Method for non-linear classification Explicit Replace

14. Outline • Introduction • Related work • Proposed method • SVM with kernels • Tree kernels in mahjong • Learning to rank using SVM • Experiment • Conclusion

15. Tree structure 手牌 孤立牌 面子候補 面子 暗刻 明刻 暗順 … リャンメン カンチャン ペンチャン トイツ Specific cards as leaves

16. Example 孤立牌 リャンメン カンチャン ペンチャン トイツ 暗刻 暗順

17. Tree kernels[Moschitti, 06] 手牌 手牌 孤立牌 面子候補 面子 孤立牌 面子候補 面子 リャンメン カンチャン 暗順 リャンメン カンチャン 暗順 Count common subtrees SST 手牌 リャンメン リャンメン 面子候補 面子候補 面子 … … … … … … リャンメン リャンメン 暗順

18. Subtrees weight[Moschitti, 06] • Deep subtreesare not very important 面子 暗順 暗順 暗刻

19. Tree kernel function Set of nodes in tree t Subtreeset Depth of subtreefi If fi is rooted at node n otherwise

20. Outline • Introduction • Related work • Proposed method • SVM with kernels • Tree kernels in mahjong • Learning to rank using SVM • Experiment • Conclusion

21. Learning to rank[Shen et al. 03] • SVMis just a 2-class classifier • How learn to rank • If you want to know ranks of three moves… Rank Order classifier( > or < ) Input data -> Pairs of moves

22. Learn and classify orders • One training example has two tree instances • Label +1 when tl > tr • Label -1 when tl < tr • Define kernel function for relative order • Classify “tl > tr” and “ tl< tr”

23. Outline • Introduction • Related work • Proposed method • SVM with kernels • Tree kernels in mahjong • Learning to rank using SVM • Experiment • Conclusion

24. 4. Experiment • Experiment of proposed method • Error analysis • Comparison with related work • Practical player

25. Environment • Machine spec • Dual-Core AMD Opteron 2.4GHz • 32GB RAM • Implementation • SVM-Light-TK [Moschitti, 2004] • Soft margin trade-off parameterC=0.1 • Optimization thresholdε=0.1

26. Method of experiment • Learn from tsumo positions in expert records • “Offensive” positions only • Nobody declares “li-zhi” • Nobody calls 3 or more “chi”, “pon” or “kan” • Using records of Totugeki Tohoku • ~285 games (~13,000training positions) • Evaluation • Accuracy rates of trained classifiers • Are expert moves ranked as the bests ? • 4-fold cross validation

27. Accuracy rates

28. Typical mistakes (1) • “Defensive” positions

29. Typical mistakes (2) • Positions require “yaku”(=poker hands) knowledge

30. Comparison with SVM using linear features • Using features designed in [Kitagawa, 2007] • Including board status information • Implementation • Ranking SVM in SVM-Light [Joachims, 2002]

31. Accuracy rates with linear features

32. Practical player ? • Core2 Duo 1.06GHz • 91819 support vectors • 700ms for classification of one tree pair • 7 seconds for deciding one move • 4 seconds in dual threading • Good enough for playing against human players

33. Outline • Introduction • Related work • Proposed method • SVM with kernels • Tree kernels in mahjong • Learning to rank using SVM • Experiment • Conclusion

34. 5. Conclusion • Classified ranks of moves with tree kernels • Possible with simple input • 57% accuracy • Despite the lack of information of field and opponents • Increasing… • Fine accuracy with permissible cost

35. Future work • Classification analysis • Positions that linear combinations cannot classify • Refine tree structure • Other information • Hands information with other kernels • String kernels • Information of field and opponents • Add as linear combinations or other kernels • Heavy computing cost • Classification time increases with a number of training positions • Indispensable in other games