Search applied to a problem against an adversary

1. Game playing • Search applied to a problem against an adversary • some actions are not under the control of the problem-solver • there is an opponent (hostile agent) • Since it is a search problem, we must specify states & operations/actions • initial state = current board; operators = legal moves; goal state = game over; utility function = value for the outcome of the game • usually, (board) games have well-defined rules & the entire state is accessible

2. Basic idea • Consider all possible moves for yourself • Consider all possible moves for your opponent • Continue this process until a point is reached where we know the outcome of the game • From this point, propagate the best move back • choose best move for yourself at every turn • assume your opponent will make the optimal move on their turn

3. Problem • For interesting games, it is simply not computationally possible to look at all possible moves • in chess, there are on average 35 choices per turn • on average, there are about 50 moves per player • thus, the number of possibilities to consider is 35^100 http://www.youtube.com/watch?v=y9UMt-8gfW8

4. Types of Games Chance Deterministic Perfect Information Imperfect Information

5. Deterministic Two-person Games • Two players take turn. • Each Player knows what the other has done to this point and can do. • One of the players wins and the other loses (or there is a draw).

6. MAX MIN Games as SearchGrundy’s Game (Nilsson 1980, Page 112) • Start with a stack of coins • Each player divides one of the current stacks into two unequal stacks (one having more coins than the other). • The game ends when every stack contains one or two coins • The first player who cannot play loses.

7. 6, 1 5, 2 4, 3 Max’s turn Min’s turn 3, 2, 2 5, 1, 1 4, 2, 1 3, 3, 1 4, 1, 1, 1 3, 2, 1, 1 2, 2, 2, 1 Max’s turn MAX Loses 3, 1, 1, 1, 1 2, 2, 1, 1, 1 Min’s turn Min Loses 2, 1, 1, 1, 1, 1 Max’s turn MAX Loses Grundy’s Game: States Can Max devise a strategy to always win? Min’s turn 7

8. Utility function (objective function or payoff function) A numeric value for the terminal states. For chess it is 1, 0, -1 for win loss, draw. For backgammon 192 to -192.

9. X O X X O O e(n) = 4 - 3 = 1 e(n) = 6 - 4 = 2 Conduct a depth first search until a terminal node is reached- a breadth first search until all nodes at level 2 are generated Evaluation function example: tic-tac-toe for a node n e(n)=(number of complete lines- row, column or diagonal that are still open for max) - (number of complete lines- row, column or diagonal that are still open for min)

10. General Minimax Algorithm on a Game Tree For each move by the computer 1. perform depth-first search as far as the terminal state 2. assign utilities at each terminal state 3. propagate upwards the minimax choices if the parent is a minimizer (opponent) propagate up the minimum value of the children if the parent is a maximizer (computer) propagate up the maximum value of the children 4. choose the move (the child of the current node) corresponding to the maximum of the minimax values of the children minimax values are gradually propagated upwards as depth-first search proceeds, i.e., minimax values propagate up the tree in a “left-to-right” fashion minimax values for sub-tree are propagated upwards “as we go”, so only O(bd) nodes need to be kept in memory at any time