AIMA Code: Adversarial Search

CSE-391: Artificial IntelligenceUniversity of Pennsylvania Matt Huenerfauth February 2005 AIMA Code: Adversarial Search

Game Class classGame: . . . defterminal_test(self, state): "Return True if this is a final state for the game.“ defutility(self, state, player): "Return the value of this final state to player." defto_move(self, state): "Return the player whose move it is in this state.“ defsuccessors(self, state): "Return a list of legal (move, state) pairs."

MINIMAX • Search a game tree all the way to the end. • Start at initial game state. • Exhaustively expand children nodes (all the possible moves in the game from each board). • When you hit an “end state,” decide if it was good/win or bad/loss and give it an appropriate value. • Then pass up the MIN or the MAX of the values of the children at each level (depending on who’s turn it was). • At the top level, make the best move: follow the child that passes up the best value.

argmax(list,func) – see utils.py defargmax(list, func): "Return element of list with highest func(list[i]) score" best = list[0]; best_score = func(best) for x in list: x_score = func(x) if x_score > best_score: best, best_score = x, x_score return best

Use of argmax(list,func)… We’ll see this line of code on the next slide… state – will be a state of the game… call_min – is a function we’ll define later… argmax( game.successors(state), lambda ((a, s)): call_min(s) ) • Successors is a list of children of this state: (action, state) pairs. • The lambda function created takes two arguments. • It ignores the first and passes the second to call_min(). • Applied to (action, state) pairs, it calls call_min on each state. • Argmax returns pair with highest value for call_min(state).

At the root of the MINIMAX search at some state in a game. defminimax_decision(state, game): # In a state in a game. player = game.to_move(state) # Who’s turn is it? action, state = \ argmax(game.successors(state), # What successor returns lambda ((a, s)): call_min(s)) # maximum call_min()? return action # Move to that state. We first decide whose turn it is. Then our function will tell us what move to make that will move us to the successor state that:- has the MAXIMUM return value - when we call the CALL_MIN function on it. We’re going to see a trend here pretty soon…

Take the MAX of your MINs, and the MIN of your MAXs…

defcall_max(state): if game.terminal_test(state): # Check if game over. return game.utility(state, player) v = -infinity for (a, s) in game.successors(state): # Return the maximum v = max(v, call_min(s)) # of the call_min’s. return v defcall_min(state): if game.terminal_test(state): # Check if game over. return game.utility(state, player) v = infinity for (a, s) in game.successors(state): # Return the minimum v = min(v, call_max(s)) # of the call_max’s. return v

= take maximum= take minimum Initial call to minimax_decision def call_max(state): if game.terminal_test(state): return game.utility(state, player) v = -infinity for (a, s) in game.successors(state): v = max(v, call_min(s)) return v def call_min(state): if game.terminal_test(state): return game.utility(state, player) v = infinity for (a, s) in game.successors(state): v = min(v, call_max(s)) return v GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER

Depth Limited MINIMAX • Most games are so complex that searching all of the possible moves in the game until you reach “GAME OVER” on every branch takes too long. • So, we set a limit on how far down to look. • Now, we’ll need a function that can look at a game state that’s not yet finished. It needs to guess how good it thinks that state of the game is. • As we discussed in class, an “Evaluation Function” looks at non-terminal state, and returns a value of how promising that state looks.

Depth Limited MINIMAX So, we add a depth limit. Each time we make a call to the next level down, we’ll decrement this depth limit. def minimax_decision(state, game, d): # We’re in some state during a game. player = game.to_move(state) # Who’s turn is it? action, state = \ # Return the successor of current argmax(game.successors(state), # board that returns the highest lambda ((a, s)): call_min(s, d-1)) # value when we call the return action # call_min function on it.

def call_max(state, d): ifd == 0: # Use eval. func. if hit limit. return game.evaluation_function(state, player) if game.terminal_test(state): # Check if game over. return game.utility(state, player) v = -infinity for (a, s) in game.successors(state): # Return the maximum v = max(v, call_min(s, d-1)) # of the call_mins. return v def call_min(state, d): ifd == 0: # Use eval. func. if hit limit. return game.evaluation_function(state, player) if game.terminal_test(state): # Check if game over. return game.utility(state, player) v = infinity for (a, s) in game.successors(state): # Return the minimum v = min(v, call_max(s, d-1)) # of the call_maxs. return v

= take maximum= take minimum Initial call to minimax_decision = call_max = call_min GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER

= take maximum= take minimum Initial call to minimax_decision = call_max = call_min GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER GAMEOVER EVALUATIONFUNCTION!!!

Alpha Beta Search Doing a little pruning…

Alpha Beta • Think about this momentof the search process… 23 16 23 5 Imagine we already looked at all the stuff down this part of tree… 24 Do we even need tobother looking at thesetwo last red ones here?Could their blue parentever get picked up top?

Alpha Beta • The blue node will return at least a 24.So, it’s branch willnever get picked bythe top-level node. 23  24 16 23 5 Imagine we already looked at all the stuff down this part of tree… 24 Do we even need tobother looking at thesetwo last red ones here?Could their blue parentever get picked up top?

Alpha Beta • So, don’t bother lookingat the blue node’s otherchildren. They just don’tmatter. 23  24 16 23 5 Imagine we already looked at all the stuff down this part of tree… 24

Alpha Beta • Continue with the rest of the search. 23  24 16 23 5 Imagine we already looked at all the stuff down this part of tree… 24

Alpha Beta Search • A version of MINIMAX that incorporates this handy trick. • It lets us “prune” branches from our tree. • This saves us a lot of searching through useless parts of the tree. • How do we modify the MINIMAX code to do this? (For now, let’s forget about the depth limit stuff… easy to add later.)

Implementing Alpha Beta First Step: Thread alpha and beta values through all the calls… Alpha: highest value seen in any call_max on the path from root to the current node. Beta: lowest value seen in any call_min on the path from root to current node.

def call_max(state, alpha, beta): if game.terminal_test(state): return game.utility(state, player) v = -infinity for (a, s) in game.successors(state): v = max(v, call_min(s, alpha, beta)) if v >= beta: return v # stop checking kids if any  beta alpha = max(alpha, v) # alpha= highest value seen in a max return v # no pruning trick if we got to here def call_min(state, alpha, beta): if game.terminal_test(state): return game.utility(state, player) v = infinity for (a, s) in game.successors(state): v = min(v, call_max(s, alpha, beta)) if v <= alpha: return v# stop checking kids if any  alpha beta = min(beta, v) # beta= lowest value seen in a min return v # no pruning trick if we got to here

At the root… def alpha_beta(state, game): player = game.to_move(state) action, state = \ argmax(game.successors(state), lambda ((a, s)): call_min(s, -infinity, infinity)) return action Alpha: Starts at –infinity so we know it will get set to new value first time it’s “max”ed inside a call_max function. Beta: Starts at infinity. Same reason for min.

Wrap Up • Game Class • Utility(), Evaluation_Function(), Is_Move() • MINIMAX • Search the whole game tree to the end each time. • Depth-Limit MINIMAX • Stop before hitting “GAME OVER” in all branches. • Alpha-Beta • Cool pruning trick to eliminate useless parts of tree.

AIMA Code: Adversarial Search