Course: Engineering Artificial Intelligence

1. Course:Engineering Artificial Intelligence Dr. Radu Marinescu Lecture 5

2. Today’s class • Game-playing

3. Overview • Computer programs which play 2-player games • game-playing as search • with the complication of an opponent • General principles of game-playing and search • evaluation functions • mini-max principle • alpha-beta-pruning • heuristic techniques • Status of Game-Playing Systems • in chess, checkers, backgammon, Othello, etc, computers routinely defeat leading world players • Applications? • think of “nature” as an opponent • economics, war-gaming, medical drug treatment

4. History 1949 – Shannon paper 1951 – Turing paper 1958 – Bernstein program 55-60 – Simon-Newell program (α – β McCarthy?) 1961 – Soviet program 66-67 – MacHack 6 (MIT AI Lab) 70’s – NW Chess 4.5 80’s – Crazy Blitz 90’s – Belle, Hitech, Deep Thought, Deep Blue

5. Solving 2-player games • Two players, perfect information • Examples: • e.g., chess, checkers, tic-tac-toe • Configuration of the board = unique arrangement of “pieces” • Games as a Search Problem • States = board configurations • Operators = legal moves • Initial State = current configuration • Goal = winning configuration • payoff function = gives numerical value of outcome of the game

6. Game tree search • Game tree: encodes all possible games • We are not looking for a path, only the next move to make (that hopefully leads to a winning position) • Our best move depends on what the other player does

7. Move generation

8. Example: partial game tree for Tic-Tac-Toe MAX (O) MIN (X) MAX (O) MIN (X)

9. Scoring function We will use the same scoring function for both players, simply negating the values to represent the opponent's scores.

10. Example: scoring function for Tic-Tac-Toe Leaves are either win (+1), loss (-1) or draw (0)

11. Min-Max: an optimal procedure • Designed to find the optimal strategy for MAX player and find best move: • 1. Generate the whole game tree to leaves • 2. Apply scoring (payoff) function to leaves • 3. Back-up values from leaves toward the root: • a MAX node computes the max of its child values • a MIN node computes the min of its child values • 4. When value reaches the root: choose max value and the corresponding move.

12. Min-Max

13. Properties of Min-Max • Complete? Yes (if tree is finite) • Optimal? Yes (against an optimal opponent) • Time complexity? O(bd) • Space complexity? O(bd) (depth-first exploration) • For chess, b ≈ 35, m ≈100 for "reasonable" games exact solution completely infeasible • Chess: • b ~ 35 (average branching factor) • d ~ 100 (depth of game tree for typical game) • bd ~ 35100 ~10154 nodes!! • Tic-Tac-Toe • ~5 legal moves, total of 9 moves • 59 = 1,953,125 • 9! = 362,880 (Computer goes first) • 8! = 40,320 (Computer goes second)

14. Min-Max • Naïve Min-Max assumes that it is possible to search the full tree, where the scoring function of the leaves is either win (+1), loss (-1) or draw (0) • However: For most games, it is impossible to develop the whole search tree • Instead develop part of the tree (up to a certain depth or a number of plys) and evaluate promise of leaves using a static evaluation function.

15. Static evaluation function • The static evaluation function: • Estimates how good the current board configuration is for a player. • Typically, one figures how good it is for the player, and how good it is for the opponent, and subtracts the opponents score from the players • Othello: Number of white pieces - Number of black pieces • Chess: Value of all white pieces - Value of all black pieces • Typical values from -infinity (loss) to +infinity (win) or [-1, +1]. • If the board evaluation is X for a player, it’s -X for the opponent • Example: • Evaluating chess boards, Checkers, Tic-tac-toe

16. Evaluation functions: chess For Chess, typically a linear weighted sum of features weight of the piece (Q = 9, P=1, K=3, B=3.5, R=5) e.g.,

17. Evaluation functions: Tic-Tac-Toe Another example E(n) =

18. Backup values Two-ply minimax applied to the opening move of tic-tac-toe

19. Backup values Two-ply minimax applied to the X’s second move

20. Backup values Two-ply minimax applied to X’s move near end game

21. Impact of tree depth

22. Deep Blue • 32 SP2 processors • Each with 8 dedicated chess processors = 256 CP • 50 – 100 billion moves in 3 min • 13-30 ply search

23. α – β pruning • In Min-Max there is a separation between node generation and evaluation. Backup Values

24. α – β pruning • Idea: • Do depth-first search to generate partial game tree • Give static evaluation function to leaves • Compute bound on internal nodes • Alpha (α), Beta (β) bounds: • α value for MAX node means that max real value is at least alpha • β for MIN node means that MIN can guarantee a value below beta • Computation: • Alpha of a MAX node is the maximum value of its seen children • Beta of a MIN node is the lowest value seen of its child node

25. When to prune • Pruning • Below a MIN node whose beta value is lower than or equal to the alpha value of its ancestors. • Below a MAX node having an alpha value greater than or equal to the beta value of any of its MIN nodes ancestors.

26. α – β pruning

27. Example

28. Example

29. Example

30. Example

31. Example

32. Example

33. Example

34. Example

35. Example

36. Example

37. Example

38. Example

39. Example

40. α – β properties • Guaranteed same value as Min-Max • In a perfectly ordered tree, expected work is O(bd/2), vs O(bd) for Min-Max, so can search twice as deep with the same effort • With good move orderings, the actual running time is close to the optimistic estimate

41. Game program • Move generator (ordered moves) 50% • Static evaluation 40% • Search control 10% Openings End games databases [all in place by late 60’s]

42. Move generator • Legal moves • Ordered by • Most valuable victim • Least valuable aggressor • Killer heuristics

43. Static evaluation • Initially - very complex • 70’s - very simple (material) • Now - Deep searchers: moderately complex (hardware) PC programs: elaborate, hand tuned

44. Other games • Backgammon • Involves randomness – dice rolls • Machine-learning based player was able to draw the world champion human player • Bridge • Involves hidden information – other players’ cards – and communication during bidding • Computer players play well but do not bid well • Go • No new elements but huge branching factor • No good computer players exist

45. Observations • Computers excel in well-defined activities where rules are clear • Chess • Mathematics • Success comes after a long period of gradual refinement

46. Summary • Game playing is best modeled as a search problem • Game trees represent alternate computer/opponent moves • Evaluation functions estimate the quality of a given board configuration for the MAX player. • Min-Max is a procedure which chooses moves by assuming that the opponent will always choose the move which is best for them • Alpha-Beta is a procedure which can prune large parts of the search tree and allow search to go deeper • For many well-known games, computer algorithms based on heuristic search match or out-perform human world experts • Reading: R&N Chapter 6