1 / 44

CS 416 Artificial Intelligence

CS 416 Artificial Intelligence. Lecture 8 Adversarial Search Chapter 6. Chess Match – Spring 2003. Ends in a 3-3 Draw. Adversarial Search. Problems involving Multiple agents Competitive environments Agents have conflicting goals Also called games. Since the dawn of time?.

afram
Download Presentation

CS 416 Artificial Intelligence

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 416Artificial Intelligence Lecture 8 Adversarial Search Chapter 6

  2. Chess Match – Spring 2003 • Ends in a 3-3 Draw

  3. Adversarial Search • Problems involving • Multiple agents • Competitive environments • Agents have conflicting goals • Also called games

  4. Since the dawn of time? • Oldest known written fair-division problem Talmud – Jewish Oral Law dating to first century • A Bankruptcy Case • A man married three wives and in each marriage contract he promised each of them different amounts of money upon his death: • one of them gets $100 • another gets $200 • the third gets $300 • When he died, he had fewer than $600 units of money • What do you do?

  5. Bankruptcy law • Modern bankruptcy provides shares of the estate proportional to their individual claims, no matter what size of the estate • A receives 100/600 * estate_holdings • B receives 200/600 * estate_holdings • C receives 300/600 * estate_holdings

  6. Bankruptcy law • Rabbi Nathan in Mishnah section of Talmud • This allocation not understood until recently

  7. Unexplained until 1984 • Aumann and Maschler (Israeli Mathematicians) • Realistically, when you die, people could come out of the woodwork saying you owe them money. Some could coalesce into deceptive groups. How can we reduce the incentives (rewards) of forming such groups? • Minimize largest dissatisfaction among all possible coalitions • A common fair-division problem • http://www.math.gatech.edu/~hill/publications/cv.dir/madevice.pdf

  8. Garment Principle • Two people claim a garment worth $100 • One claims the entire garment belongs to him • The other claims half the garment is his • The one claiming the full garment gets $75The one claiming half gets $25 • Why?

  9. Minimizing maximum dissatisfaction • The one who wants the entire garment cedes nothing to the other and thus wants $100. • The one who wants half the garment would be perfectly happy to cede $50 to the other. • But a split of 50/50 would make one person unhappy and the other perfectly happy • How to make them equally unhappy?

  10. Game Theory • Studied by mathematicians, economists, finance • In AI we limit games to: • deterministic • turn-taking • two-player • zero-sum • perfect information

  11. Games • “Shall we play a game?” • Let’s play tic-tac-toe

  12. Tic-Tac-Toe game tree MAX’s first move MIN’s first move Each layer is aply

  13. What data do we need to play? • Initial State • How does the game start? • Successor Function • A list of legal (move, state) pairs for each state • Terminal Test • Determines when game is over • Utility Function • Provides numeric value for all terminal states

  14. Minimax strategy • Optimal Strategy • Leads to outcomes at least as good as any other strategy when playing an infallible opponent • Pick the option that minimizes the maximum damage your opponent can do • minimize the worst-case outcome • because your skillful opponent will certainly find the most damaging move

  15. Minimax • Algorithm • MinimaxValue(n) = Utility (n) if n is a terminal state max MinimaxValue(s) of all successors, sif n is a MAX node min MinimaxValue(s) of all successors, sif n is a MIN node • This is optimal strategy assuming both players play optimally from there until end of game

  16. A two-ply example • MIN considers minimizing how much it loses…

  17. A two-ply example • MAX considers minimizing how much it loses…

  18. Minimax Algorithm • We wish to identify minimax decision at the root • Recursive evaluation of all nodes in game tree • Time complexity = O (bm)

  19. Feasibility of minimax? • How about a nice game of chess? • Avg branching = 35 and avg # moves = 50 for each player • O(35100) time complexity = 10154 nodes • 1040 distinct nodes • Minimax is impractical if directly applied to chess

  20. Pruning minimax tree • Are there times when you know you need not explore a particular move? • When the move is poor? • Poor compared to what? • Poor compared to what you have explored so far

  21. Alpha-beta pruning • a • the value of the best (highest) choice so far in search of MAX • b • the value of the best (lowest) choice so far in search of MIN • Order of considering successors matters • If possible, consider best successors first

  22. Alpha-beta pruning MIN knows it will at least score a 3.MAX worries that –inf is still possible MIN knows player MAX has an option of going to node B with a min payoff of 3. MAX will never take action C and culling is possible. MAX knows that 3 is worst case for this node. MAX knows that it can accomplish a score of at least 3. Discovery could find a higher value

  23. Alpha-beta pruning • Without pruning • O(bd) nodes to explore • With a good heuristic pruner (consider part (f) of figure) • O(bd/2) • Chess can drop from O(35100) to O(6100) • With a random heuristic • O(b3d/4)

  24. Real-time decisions • What if you don’t have enough time to explore entire search tree? • We cannot search all the way down to terminal state for all decision sequences • Use a heuristic to approximate (guess) eventual terminal state

  25. Evaluation Function (Estimator) • The heuristic that estimates expected utility • Cannot take too long (otherwise recurse to get answer) • It should preserve the ordering among terminal states • otherwise it can cause bad decision making • Define features of game state that assist in evaluation • what are features of chess?

  26. Truncating minimax search • When do you recurse or use evaluation function? • Cutoff-Test (state, depth) returns 1 or 0 • When 1 is returned, use evaluation function

  27. When do you cut off? • When exploring beyond a certain depth • The horizon effect

  28. When do you cut off? • Cutoff if state is stable or quiescient (more predictable)

  29. When do you cut off? • Cutoff moves you know are bad (forward pruning)

  30. Benefits of truncation • Comparing Chess Number of plys that can considered per unit time • Using minimax 5 ply • Average Human 6-8 ply • Using alpha-beta 10 ply • Intelligent pruning 14 ply

  31. Games with chance • How to include chance in game tree? • Add chance nodes

  32. Expectiminimax • Expectiminimax (n) = • utility(n) if n is a terminal state • if n is a MAX node • if n is a MIN node • if n is a chance node

  33. Pruning • Can we prune search in games of chance? • Think about alpha-beta pruning • With alpha-beat, we don’t explore nodes that we know are worse than what we know we can accomplish • With randomness, we never really what we will accomplish • chance node values are average of successors • Thus it is hard to use alpha-beta

  34. History of Games • Chess, Deep Blue • IBM: 30 RS/6000 comps with 480 custom VLSI chess chips • Deep Thought design came from Campbell and Hsu at CMU • 126 mil nodes / s • 30 bil positions per move • routine reaching depth of 14 • iterative deepening alpha-beta search

  35. Deep Blue • evaluation function had 8000 features • 4000 opening moves in memory • 700,000 grandmaster games from which recommendations extracted • many endgames solved for all five piece combos

  36. Checkers • Arthur Samuel of IBM, 1952 • program learned by playing against itself • beat a human in 1962 (but human clearly made error) • 19 KB of memory • 0.000001 Ghz processor

  37. Checkers • Chinook, Jonathan Schaeffer, 1990 • Alpha-beta search on regular PCs • database of all 444 billion endgame positions with 8 pieces • Played against Marion Tinsley • world champion for over 40 years • lost only 3 games in 40 years • Chinook won two games, but lost match • Rematch with Tinsley was incomplete for health reasons • Chinook became world champion

  38. Othello • Smaller search space (5 to 15 legal moves) • Humans are no match for computers

  39. Backgammon • Garry Tesauro, TD-Gammon, 1992 • Reliably ranked in top-three players of world • Learned to play through playing against itself • Reinforcement Learning

  40. Discussion • How reasonable is minimax? • perfectly performing opponent • perfect knowledge of leaf node evaluations • strong assumptions

  41. Building alpha-beta tree • Can we restrict the size of game tree? • alpha-beta will blindly explore tree in depth-first fashion even if only one move is possible from root • even if multiple moves are possible, can we use a quick search to eliminate some entirely? • utility vs. time tradeoff to decide when to explore new branches or to stay with what you have

  42. Metareasoning • Reasoning about reasoning • alpha-beta is one example • think before you think • think about utility of thinking about something before you think about it • don’t think about choices you don’t have to think about

  43. Goal-directed reasoning / planning • Minimax starts from root and moves forward using combinatorial search • What about starting at goal and working backward • We talked about difficulty of identifying goal states in bidirectional search • We do not know how to combine the two in practical way

More Related