Artificial Intelligence in Game Design

1. Artificial Intelligence in Game Design Heuristics and Other Ideas in Board Games

2. Good and Bad Heuristics • Heuristic for evaluating board must be accurate • Directly related to “likelihood” of win • Inversely related to “distance” from win • Example: TicTacToe heuristicH(board) = 2 × # of possible rows/columns/diagonals where X could win in one move + + 1 × # of possible rows/columns/diagonals where X could win in two moves + - 2 × # of possible rows/columns/diagonals where O could win in one move + - 1 × # of possible rows/columns/diagonals where O could win in two moves • Gives these two boards same measure Guaranteed loss for AI -2 -2

3. Good and Bad Heuristics • Better heuristic must take this into account! • H(board) = • if my move next and I have 2 in row  MAXINT • if opponent move next and they have 2 in row  -MAXINT • if my move and opponent has > 1 instance of 2 in row  -MAXINT • if opponent move and I have > 1 instance of 2 in row  MAXINT • else (above function) • Will work better, but more complex to compute! Accuracy ofheuristic value Speed to computeheuristic value tradeoff

4. Linear Heuristic Functions • Heuristic is some function of individual pieces on board • Usually weighted in some way • Overall board value = ΣH(piecei)i • Very fast to compute • Example: Chess • Pieces have “standard values” • Heuristic = sum of AI pieces – sum of player pieces

5. Linear Heuristic Functions • Often based on position of pieces on board • Examples: • Games where purpose is to move all pieces to some goal • Backgammon • Sorry • Can often send other player back to “start” • Heuristic value = total distance of AI pieces from goal – total distance of player pieces from goal

6. Linear Heuristic Functions • Reversi (Othello) • Board value of based entirely on piece positions • Corners very valuable (can’t be flipped) • Sides somewhat valuable (very difficult to flip) • Middle little value (will easily be flipped) • H(board) = C1 * number of pieces in corner +C2 * number of pieces on side + C3 * number of pieces in middle +C1 >> C2 >> C3 • Reversi easiest type of game for AI • Low branching factor (5 to 15 possible moves) • Good heuristics • Single move can greatly change board • Hard for human player to see • Easy for MinMax lookahead to see

7. Nonlinear Heuristics • Based on relationships between pieces on board • Simple example: • Prefer chess pieces to protect one another • Piece value = piece value * 1.5 if protected by another • Piece value = piece value * 0.5 if not protected and threatened by opponent piece

8. Nonlinear Heuristics • Drawback: Usually much more expensive to compute • n pieces  O (n2) relationships between them • May be able to explore more moves with simpler (linear) heuristic • Example: • Simple linear heuristic takes k ms to evaluate per board • Complex nonlinear heuristic takes 400kms to evaluate per board • Average of 20 possible next moves per board • 400 possible next two moves • On average, could explore two additional moves if use linear heuristic

9. Linear vs. Nonlinear Heuristics • Nonlinear heuristics often detect future changes in board • Pieces threatened by others might be captured • Pieces protected by others less likely to be captured • Often see effect of nonlinear heuristic with additional levels of linear heuristic • Example: “knight fork” in chess • Can look for this using nonlinearheuristic

10. Linear vs. Nonlinear Heuristics • Searching 2 more moves in game tree will show result of knight fork • Shows state where rook now gone • Higher value for linear heuristic based on piece values • MinMax then gives “knight fork” state a high value Board with knight fork High value h Black moves king Min level Board with knight threatening rook Other moves give checkmate (value = MAXINT) High value h Max level Knight takes rook Board with blackrook gone High value h

11. Horizon Effect • Major weakness of purely linear heuristics based on piece values • “Throwing good money after bad” • May need to recognize when move does not improve board position • Chess example: Queen pinned by bishops Could move rook in way But rook captured and will still lose queen

12. Horizon Effect Board with queen pinned Move rook in front Queen takes bishop Bishop takes queen Bishop takes rook Board with queen lost Board with rook lost White down 6 points White down 5 points This looks worse at cutoff level, but is actually best in long run! Queen takes bishop Bishop takes queen Board with queen and rook lost White now down 11 points!

13. Trappy Minmax • Idea: Use minmax to set traps for player • Branch appears to have short-term benefit for player • In long run, branch has benefit for AI Good chance player might choose this branch AI move player move This state looks ok to player This state looks good to player So worth considering making this move Ok result for AI Great result for AI

14. Trappy Minmax Takes advantage of player horizon effect Queen cancapture rook Some other move Queen takes rook Board with rook ignored Board with rook captured White unchanged White up 5 points Knight captures queen Board with queen lost Player did not lookahead far enough to see this White now down 4 points!

15. Factors in Trappy Minmax • Trappiness: Estimate of how likely player will choose branch corresponding to trap • Usually computed based on median or maximum of player heuristics down best branch Possible player move 5 9 Path player thinks game will follow if they make this move Median = 7 6 8 2 -7

16. Factors in Trappy Minmax • Profitability: Score if player follows “trap” branch – score if player follows their best branch • Trap Quality: Trappiness * Profitability • Use trap quality to adjust heuristic measure of a state:Trappy_heuristic(state) = normal_heuristic(state) + f(trap_quality(state))

17. Data-Driven Approaches Basing actions on known strategies rather than tree search • Opening books of initial moves (Chess) • Often 20-30 moves at start of Grandmaster match • Each book consists of: • List of moves • Evaluation of final outcome • Should we follow this strategy • Allows faster processing • No need to search game tree until end of sequence • Can just use evaluation as heuristic Currentboard Boards in sequence End of sequence Now start branching

18. Opening Books • Choose as own opening strategy • Must have good final evaluation! • Make moves according to script • If opponent follows script, keep following • If opponent leaves script, start MinMax • Will probably be in same way that benefits us • Must also recognize when opponent uses an opening book • Keep database of moves in opening books • Match current board to those in database to find whether it is part of a sequence • If so, make decision about whether following script is good idea

19. Other Set Plays • End Games • Many games have different strategies when few pieces left • Forcing checkmate in chess • Kings vs. kings in checkers • Getting last pieces home in backgammon • Recognize based on pieces left • Follow set strategies • Set Evaluation Values • No heuristic evaluation of board – instead, match board to database to get evaluation • Works best if can match subboards • Example: Edge configurations in Othello • This edge has a known value

20. Alternative Approaches • Go • 19  19 board  branching factor of 361 • Impossible for MinMax • Only known approaches based on template matching • Look in local area for configurationsthat match known strategies • Still very open problem • Best AI for Go only plays atamateur level