310 likes | 484 Views
Game and Tree Searching . Learning Objectives. Introduction to games Optimal decisions in games Alpha-Beta pruning Real-time decisions. Introduction to Games. Games as Search Problems Frameworks: two-player, multi-player; zero-sum; perfect information Minimax algorithm Perfect decisions
E N D
Game and Tree Searching TCSS435A Isabelle Bichindaritz
Learning Objectives • Introduction to games • Optimal decisions in games • Alpha-Beta pruning • Real-time decisions TCSS435A Isabelle Bichindaritz
Introduction to Games • Games as Search Problems • Frameworks: two-player, multi-player; zero-sum; perfect information • Minimax algorithm • Perfect decisions • Imperfect decisions (based upon static evaluation function) • Issues • Quiescence • Horizon effect • Need for pruning (alpha-beta pruning) TCSS435A Isabelle Bichindaritz
Introduction to Games • Perfect Play • General framework(s) • What could agent do with perfect info? • Resource Limits • Search ply • Static evaluation: from heuristic search to heuristic game tree search • Examples • Tic-tac-toe, connect four, checkers, connect-five • Chess, go • Games with Uncertainty • Explicit: games of chance (e.g., backgammon, Monopoly, blackjack) • Implicit TCSS435A Isabelle Bichindaritz
Games versus Search Problems • Unpredictable Opponent • Games are adversarial search problems • Solution is strategy, contingency plan • Time limits • Unlikely to find goal • Must approximate • Plan of Attack • Algorithm for perfect play (J. von Neumann, 1944) • Finite horizon, approximate evaluation (C. Zuse, 1945; C. Shannon, 1950, A. Samuel, 1952-1957) • First chess program (Turing, 1951) • Pruning to allow deeper search (J. McCarthy, 1956) TCSS435A Isabelle Bichindaritz
Types of Games • Information: Can Know (Observe) • … outcomes of actions / moves? • … moves committed by opponent? • Uncertainty • Deterministic vs. nondeterministic outcomes • Thought exercise: sources of nondeterminism? TCSS435A Isabelle Bichindaritz
Minimax • Games with two players MIN and MAX are a search problem with: • Initial state • Successor function • Terminal test / state • Utility function (objective / payoff) • Win / loss / draw (chess) • +1 / -1 / 0 (chess) • -192 … +192 (backgammon) TCSS435A Isabelle Bichindaritz
Minimax TCSS435A Isabelle Bichindaritz
Minimax • Perfect play for deterministic, perfect-information games • Choose move to position with highest minimax value = best achievable payoff against best play • Simple example: 2-ply game, a ply being a half-move • Game ends after one move each by MAX and MIN (one move deep) TCSS435A Isabelle Bichindaritz
Minimax TCSS435A Isabelle Bichindaritz
Minimax Algorithm:Decision and Evaluation what’s this? what’s this? TCSS435A Isabelle Bichindaritz
Properties of Minimax • Complete? TCSS435A Isabelle Bichindaritz
Properties of Minimax • Complete? • … yes, provided following are finite: • Number of possible legal moves (generative breadth of tree) • “Length of game” (depth of tree) – more specifically? • A finite strategy can exist even in an infinite tree! • Perfect vs. imperfect information? • Q: What search is perfect minimax analogous to? • A: Bottom-up breadth-first TCSS435A Isabelle Bichindaritz
Properties of Minimax • Complete? • … yes, provided the tree is finite: • Number of possible legal moves (generative breadth of tree) • “Length of game” (depth of tree) – more specifically? • Optimal? TCSS435A Isabelle Bichindaritz
Properties of Minimax • Complete? • … yes, provided the tree is finite • Optimal? • … yes, provided perfect info (evaluation function) and opponent is optimal! • … otherwise, guaranteed if evaluation function is correct • Time Complexity? TCSS435A Isabelle Bichindaritz
Properties of Minimax • Complete? • … yes, provided the tree is finite: • Optimal? • … yes, provided perfect info (evaluation function) and opponent is optimal! • Time Complexity? • Depth of tree: m • Legal moves at each point: b • O(bm) – NB, m 100, b 35 for chess! • Space Complexity? TCSS435A Isabelle Bichindaritz
Properties of Minimax • Complete? • … yes, provided the tree is finite: • Optimal? • … yes, provided perfect info (evaluation function) and opponent is optimal! • Time Complexity? • Depth of tree: m • Legal moves at each point: b • O(bm) – NB, m 100, b 35 for chess! • Space Complexity? O(bm) – why? TCSS435A Isabelle Bichindaritz
Resource Limits TCSS435A Isabelle Bichindaritz
Static Evaluation Function Example: Chess TCSS435A Isabelle Bichindaritz
Do Exact Values Matter? TCSS435A Isabelle Bichindaritz
Cutting Off Search TCSS435A Isabelle Bichindaritz
Cutting Off Search • Issues • Quiescence • Play has “settled down” • Evaluation function unlikely to exhibit wild swings in value in near future • Horizon effect • “Stalling for time” • Postpones inevitable win or damaging move by opponent • See: Figure 6.9 R&N • Solutions? • Quiescence search: expand non-quiescent positions further • No general solution to horizon problem at present TCSS435A Isabelle Bichindaritz
Why Prune? TCSS435A Isabelle Bichindaritz
Game Tree Search • Perfect Play (TBD from Perfect Information) • Common framework: two-player (or multi-player), zero-sum • Deterministic or known probability distribution function, totally observable • Minimax algorithm given ability to search for arbitrary ply • Assumptions about • … game? • … opponent? • Specifications and Ramifications • Combinatorics: finite depth (m), breadth (b) of search • Resource limits: need for static evaluation function • Practical Issues • Effects of depth cutoff • Need for pruning TCSS435A Isabelle Bichindaritz
- Pruning • Properties of - pruning • Pruning does not affect final result • Good move ordering improves effectiveness of pruning • With “perfect ordering,” time complexity = O(bm/2) • Depth of tree: m • Legal moves at each point: b • Effective branching factor becomes b1/2 • Can easily reach depth 8 and play good chess • A simple example of the value of reasoning about which computations are relevant ( a form of metareasoning) TCSS435A Isabelle Bichindaritz
- Pruning • is the value of the best (i.e. highest value) choice we have found so far at any choice point along the path for MAX • is the value of the best (i.e. lowest value) choice we have found so far at any choice point along the path for MIN. TCSS435A Isabelle Bichindaritz
Alpha-Beta (-) Pruning:Definitions TCSS435A Isabelle Bichindaritz
MAX MIN ≤5 2 MAX 2 3 12 8 14 5 2 Alpha-Beta (-) Pruning Example ≥ 3 3 ≤ 2 ≤ 14 TCSS435A Isabelle Bichindaritz
≥ 3 MAX MIN 3 ≤ 2 ≤ 14 ≤5 2 MAX 2 3 12 8 14 5 2 Alpha-Beta (-) Pruning Example What are , values here? TCSS435A Isabelle Bichindaritz
Alpha-Beta (-) Pruning:Modified Minimax Algorithm TCSS435A Isabelle Bichindaritz
Alpha-Beta (-) Pruning Properties • Perfect Play (TBD from Perfect Information) • Common framework: two-player (or multi-player), zero-sum • Deterministic or known probability distribution function, totally observable • Minimax algorithm given ability to search to arbitrary ply • Assumptions about • … game? • … opponent? • Specifications and Ramifications • Combinatorics: finite depth (m), breadth (b) of search • Resource limits: need for static evaluation function • Practical Issues • Effects of depth cutoff • Need for pruning TCSS435A Isabelle Bichindaritz