Artificial Intelligence

Artificial Intelligence CS 165A Tuesday, October 23, 2007 • Adversarial search – games (Ch 6) • Knowledge and reasoning (Ch 7) Today 1 1

Notes • HW#2 posted tomorrow, due Tuesday • Multiplayer games and minimax search… 2

Minimax search • The minimax decision maximizes the utility under the assumption that the opponent seeks to minimize it (and uses the same evaluation function) • Generate the tree of minimax values • Then choose best (maximum) move • Don’t need to keep all values around • Good memory property • Depth-first search is used to implement minimax • Expand all the way down to leaf nodes • Recursive implementation (Fig. 6.3) 3

Note: Your opponent is not always optimal • What’s your best move? 4 1 2 4 3 A B C D 1 7 2 5 2 8 9 4 6 3 49 57 Minimax says C but what if opponent isn’t perfectly rational?

Optimal? Complete? Time complexity? Space complexity? Yes, against an optimal opponent Yes, if the tree is finite Exponential: O( bm ) Exponential: O( bm) Minimax properties 5

But this could take forever… • For chess, b  35 and m  100 for “reasonable” games • So optimality, completeness are kind of irrelevant! • Rather, cut the search off early and apply a heuristic evaluation function to the leaves • h(n) estimates the expected utility of the game from a given position (node) n • The performance of a game-playing program depends on the quality (and speed!) of its evaluation function 6

Heuristics (Evaluation function) • Typical evaluation function for game: weighted linear function • h(s) = w1 f1(s) + w2 f2(s) + … + wn fn(s) • weights ·features [dot product] • For example, in chess • W = { 1, 3, 3, 5, 8} • F = { # pawns advantage, # bishops advantage, # knights advantage, # rooks advantage, # queensadvantage } • Is this what Deep Blue used? • What are some problems with this? • More complex evaluation functions may involve learning • Adjusting weights based on outcomes • Perhaps non-linear functions • How to choose the features? 7

Cutting off search • Suppose we have 100 seconds per move (e.g., in chess) and we can explore 104 nodes per second • Options: • Stop search after 106 nodes • Choose a depth d that will typically take < 100 sec • Do iterative deepening search until allocated time runs out • E.g., in chess: • bm = 106 , b = 35  m = 4 (can look 4 ply ahead) • 4-ply: Human novice level (1.5M) • 8-ply: Typical PC, good human (2.2T) • 12-ply: Deep Blue, Kasparov (?) (3x1018) 8

Cutting off search (cont.) • This strategy of cutting off search and applying a heuristic evaluation function does not always work well in practice • What’s more, a blind application of the evaluation function can be disastrous • E.g., queen to be captured at ply N+1 • One solution to the cutoff problem: use quiescentsearch • Evaluation function is only valid for quiescent states (ones that are likely to be reasonably steady in the near future) • Have to search further to evaluate non-quiescent states (e.g., right before a capture) • Watch out for the horizon effect 9

Pruning • What’s really needed is “smarter,” more efficient search • Don’t expand “dead-end” nodes! • Pruning – eliminating a branch of the search tree from consideration • Alpha-beta pruning, applied to a minimax tree, returns the same “best” move, while pruning away unnecessary branches • Many fewer nodes might be expanded • Hence, smaller effective branching factor • …and deeper search • …and better performance • Remember, minimax is depth-first search 10

Alpha-beta pruning • Consider traversing a minimax game tree in a depth-first manner • General principle: Prune nodes (drop them and their descendants from consideration) if they cannot do better than previously generated nodes • Alpha-beta search does a minimax search with pruning, using these parameters to describe bounds: •  - best choice so far for MAX •  - best choice so far for MIN • Update  and  as search progresses • Prune remaining branches at a node if current value of node is less than current  value for MAX, or more than current  value for MIN 11

Alpha-beta pruning (cont.) If m is better than n for Player, neither it not its descendents will ever be a candidate solution (so they can be pruned.) 12

Alpha-beta example Worst I could do at this node Best I could do at this node A [ ??, ?? ] [ ??, ?? ] [ ??, ?? ] [ ??, ?? ] B C D 3 12 8 2 1 13 14 5 2 13

Alpha-beta example A [3, 3] [3, 3] [-inf, 2] [2, 2] B C D 3 12 8 2 1 13 14 5 2 14

Non-deterministic games • Games that include an element of chance • E.g., Backgammon, Monopoly, Risk, Scrabble • In backgammon, roll s P(s) of dice determines legal moves • Human (i.e., imperfect!) opponent • A deterministic minimax value is not possible • You do not know what a “perfect” opponent will do! • Instead of an exact minimax value, calculate an expected value of a node (“Expectiminimax”) • Weighted average of utility over all possible rolls of dice • Concept of expected utility • s P(s) u(s), where P(s) is probability of state s u(s) is utility of state s. • Start with actual utilities of terminal nodes

Non-deterministic games • How to represent • Add “chance” nodes to tree with MAX and MIN nodes • Represent possible paths (e.g., roll of the dice) • Calculate the average utility over all the possible chance events • Numerical scale of utility function is important • Aside: Complex deterministic games can be modeled as deterministic games with chance nodes

Schematic diagram of backgammon MAX CHANCE Possible moves MIN Possible dice rolls Possible opponent moves CHANCE Possible dice rolls MAX

Expectiminimax calculation P(s1) = 0.3 P(s2) = 0.2 P(s3) = 0.5 A B 0.3 0.2 0.5 0.3 0.2 0.5 6 4 5 2 2 2 Expectiminimax(A) = 0.3*6 + 0.2*4 + 0.5*5 = 5.1 Expectiminimax(B) = 0.3*2 + 0.2*2 + 0.5*2 = 2.0 Choose move A

Expectiminimax calculation P(s1) = 0.3 P(s2) = 0.2 P(s3) = 0.5 A B 0.3 0.2 0.5 0.3 0.2 0.5 6 4 5 2 9 9 Expectiminimax(A) = 0.3*6 + 0.2*4 + 0.5*5 = 5.1 Expectiminimax(B) = 0.3*2 + 0.2*9 + 0.5*9 = 6.9 Choose move B

Examples Tomorrow’s discussion sessions will include examples of minimax, alpha-beta pruning, and expectiminimax 20

Knowledge and Reasoning Logic

Reminder: Three parts of an AI Program? • AI programs can generally be thought of as comprising three separated parts • Data / knowledge (“knowledge base”) • Operations / rules (“production rules”) • Control • What are these three parts in M&C, TTT, vacuum world? • We need to consider how to represent knowledge and how to reason about what we know (with respect to what we want)\ • More high-level and flexible representations and reasoning abilities

Knowledge and reasoning • We want more powerful methods for • Representing Knowledge – more general methods for representing facts about the world and how to act in world • Carrying outReasoning – more general methods for deducing additional information and a course of action to achieve goals • Focus on knowledge and reasoning rather than states and search • Note that search is still critical • This brings us to the idea of logic

Logic – the basis of reasoning • Logics are formal languages for representing information such that conclusions can be drawn • I.e., to support reasoning • Syntax defines the sentences in the language • Allowable symbols • Rules for constructing grammatically correct sentences • Semantics defines the meaning of sentences • Correspondence (isomorphism) between sentences and facts in the world • Gives an interpretation to the sentence • Sentences can be true or false with respect to the semantics • Proof theory specifies the reasoning steps that are sound (the inference procedure) 24

Knowledge-based agents • A knowledge-based agent uses reasoning based on prior and acquired knowledge in order to achieve its goals • Two important components: • Knowledge Base (KB) • Represents facts about the world (the agent’s environment) • Fact = “sentence” in a particular knowledge representation language (KRL) • KB = set of sentences in the KRL • Inference Engine – determines what follows from the knowledge base (what the knowledge base entails) • Inference / deduction • Process for deriving new sentences from old ones • Sound reasoning from facts to conclusions

Knowledge base (KB) for agents • The KB agent must be able to • Represent states, actions, etc. • Incorporate new percepts • Update the internal representation of the world • Deduce hidden properties of the world • Deduce appropriate actions, given the goals • Can be viewed at different levels of explanation • Knowledge level • Abstract. What does the agent know? What might you TELL or ASK the agent? How does the agent reason? • Logical level • Sentences in the KRL, inference rules • Implementation level • Data structures, implementation efficiency

Basic approach to representing KB agents • Basic activities that must be supported: • KB is consulted and updated when making percepts about world • KB is consulted and updated when making choices of actions in world (use of reasoning mechanisms) • KB is consulted and updated in response to perceived changes following actions • All represented in logical terms

TELL ASK KB Agent Knowledge Base Domain specific content; facts Domain independent algorithms; can deduce new facts from the KB Inference engine

Artificial Intelligence