Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)

Graham KendallAutomated Scheduling, Optimisation and Planning Research Group (ASAP) MIU, July 2004

Contents • Checkers: Why was it considered “beaten”? • Two approaches to Checkers • Poker (if time)

1959. Arthur Samuel started to look at Checkers2 • The determination of weights through self-play • 39 Features • Included look-ahead via mini-max 2 Samuel A. Some studies in machine learning using the game of checkers. IBM J. Res. Develop. 3 (1959), 210-229

Samuels’s program defeated Robert Nealy, although the victory is surrounded in controversy • Was he state champion? • Did he lose the game or did Samuel win?

Checkers Starting Position 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32

Checkers Moves 1 2 4 3 5 6 8 7 Pieces can only move diagonally forward 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32

Jumps are forced Checkers Forced Jumps 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32

K 1 Red (Samuel’s Program) 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 Getting to the back row gives a King 23 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

Red (Samuel’s Program) 1 2 4 3 5 6 8 7 Forced Jump 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

Red (Samuel’s Program) 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

Strong (Try to keep) Trapped Only advance to Square 28 Red (Samuel’s Program) 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

Red (Samuel’s Program) 1 2 4 3 What Move? 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

This was a very poor move. • It allowed Samual to retain his “Triangle of Oreo” • AND.. By moving his checker from 19 to 24 it guaranteed Samuel a King • This questioned how strong a player Nealy really was

K 23 Red (Samuel’s Program) : After Move 25 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

K 23 Red (Samuel’s Program) : After Move 25 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 24 21 22 25 28 26 What Move (5, 13 or 16)? 27 29 30 31 32 White (Nealey)

K 23 Computers & Game Playing : A Potted History Red (Samuel’s Program) : After Move 25 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

16-12 then 5-1, Chinook said would be a draw K 23 Red (Samuel’s Program) : After Move 25 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

K 23 Computers & Game Playing : A Potted History Red (Samuel’s Program) : After Move 25 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

K Red (Samuel’s Program) : After Move 25 1 2 4 3 5 6 8 7 11 9 10 This checker is lost 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

K Red (Samuel’s Program) : After Move 25 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

K K Red (Samuel’s Program) : After Move 25 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 What Move (3, 6 or 19)? 29 30 31 32 White (Nealey)

K K Red (Samuel’s Program) : After Move 25 1 2 4 3 5 6 8 7 11 9 10 12 This checker could run (to 10) but.. 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

K K Red (Samuel’s Program) : After Move 25 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

K K Red (Samuel’s Program) : After Move 25 1 2 Forced Jump 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

K K Red (Samuel’s Program) : After Move 25 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

Victory K Red (Samuel’s Program) : After Move 25 1 2 4 3 5 6 8 7 11 9 10 12 15 16 14 13 17 18 19 20 23 24 21 22 25 28 26 27 29 30 31 32 White (Nealey)

Two Mistakes by Nealy • Allowing Samuel to get a King • Playing a move that led to defeat when there was a draw available

The next year a six match rematch was won by Nealy 5-1. • Three years later (1966) the two world championship challengers (Walter Hellman and Derek Oldbury) played four games each against Samuel’s program. They won every game.

Checkers • Chinook • Blondie 24 (aka Anaconda)

Types of Games • Perfect • Each Player has complete knowledge of the game state • Usually only two players, who take alternate turns • Examples include Chess, Checkers, Awari, Connect-Four, Go, Othello

Types of Games • Imperfect • Some of the game state is hidden • Examples include Poker, Cribbage, Bridge

Types of Games • Games with an element of chance • The game moves have some stochastic element • For example, Backgammon

Types of Games 6 Jaap van den Herik H., Uiterwijk and van Rijswijck J. Games Solved: Now and in the future. Artificial Intelligence 134 (2002) 277-311

Case Study 1: Checkers • Samuel’s work, perhaps, restricted the research into Checkers until 1989 when Jonathan Schaeffer began working on Chinook • He had two aims • To develop the worlds best checkers player • To “solve” the game of checkers

Case Study 1: Checkers • Chinook, at its heart, had an evaluation function • Piece count (+30% for a King) • Runaway checker • “Dog Hole” • The weights were hand-tuned

Case Study 1: Checkers • Opening game database from published work (with corrections they found) • Initially 4000 openings, leading to an eventual 40,000 • “Cooks” – innovative lines of play that could surprise an opponent • The aim was to take opponents into unknown territory

Case Study 1: Checkers • Endgame database: Started writing in May 1989 • The 8-piece endgame database finished on February 20th 1994

Case Study 1: Checkers

Graham Kendall Automated Scheduling, Optimisation and Planning Research Group (ASAP)