Artificial Intelligence

1. Artificial Intelligence CS 165A Tuesday, October 16, 2007 • Informed (heuristic) search methods (Ch 4) Today 1

2. Notes • HW#1 • Discussion sessions Wednesday • Office hours • What’s an example of a problem where you don’t need to know the path to a goal? (Finding the goal is enough.) • Does this starting position in the 8-puzzle have a solution? • Is this theorem provable? • Traveling salesman problem (“iterative improvement” approach) • Will this move lead me to checkmate, or be checkmated, within N moves?

3. Informed (“Heuristic”) Search • If AI is “the attempt to solve NP-complete problems in polynomial time,” blind search doesn’t help • Then we typically have two choices • Design algorithms that give good average-case behavior (even if the worst-case behavior is terrible) • Design algorithms that are approximate – give acceptable results (even if not optimal) in acceptable time/memory/etc. • Informed search generally tries to do both of these

4. Informed Search (cont.) • The path cost g(n) only measures the “past,” it tells nothing about the “future” • I.e., it measures cost from initial state to current state only • We wish to minimize the overall cost, so we need an estimate of the “future” cost • I.e., from current state to goal state • Overall cost = past + future costs • This estimate is a heuristic – a “rule of thumb” • A function that estimates the solution cost • A rule expressing informal belief

5. How to choose a heuristic? • One method: Derive a heuristic from the exact solution cost of a relaxed (less restricted) version of the problem • A standard distance metric • E.g., Euclidian distance, Hamming distance, … • Use common sense? • A good heuristic function must be efficient as well as accurate

6. Heuristics • What’s a heuristic for • Driving distance (or time) from city A to city B ? • 8-puzzle problem ? • M&C ? • Time to complete homework assignment ? • Robot navigation ? • Reaching the summit ? • Medical diagnosis ? • Admissible heuristic • Does not overestimate the cost to reach the goal • “Optimistic” • Are the above heuristics admissible?

7. Not covering in detail Informed Search Methods • Best-first search • Greedy best-first search • A* search • Memory bounded search • IDA* • SMA* • Iterated improvement algorithms • Hill-climbing • Simulated annealing As with blind search, the strategy is defined by choosing the order of node expansion

8. Best-First Search • Defines a family of search algorithms • Uses an evaluation function at each node to estimate the desirability of expanding the node • A queuing function sorts the unexpanded nodes in decreasing order of desirability, according to the evaluation function • Therefore, the one with the “highest desirability” is expanded first • We’ve already seen this with Uniform Cost Search • Expand the node n with the lowest g(n)

9. Best-First Search • What’s the difference between QUEUING-FN and EVAL-FN? • EVAL-FN takes one argument (the node) • Returns a scalar value (e.g., approximate distance to goal) • QUEUING-FN takes two arguments: a queue of nodes and a function (e.g., less-than) • Returns a queue of nodes functionBEST-FIRST-SEARCH(problem, EVAL-FN) returns a solution or failure QUEUING-FN a function that orders nodes by EVAL-FN return GENERAL-SEARCH(problem, QUEUING-FN) But what evaluation function to use?

10. Greedy Best-First Search • Uses a heuristic function, h(n), as the EVAL-FN • h(n) estimates of the cost of the best path from state n to a goal state • h(goal) = 0 • Greedy search – always expand the node that appears to be closest to the goal (i.e., with the smallest h) • Instant gratification, hence “greedy” functionGREEDY-SEARCH(problem, h) returns a solution or failure return BEST-FIRST-SEARCH(problem, h) • Greedy search often performs well • It doesn’t always find the best solution • It may get stuck • It depends on the particular h function

11. GBFS Example Use hSLD(n) – straight-line distance to goal (admissible?)

12. Arad h = 366 Zerind Sibiu Timisoara h = 374 h = 253 h = 329 Oradea Arad Fagaras Rimnicu Vilcea h = 380 h = 366 h = 178 h = 193 Sibiu Bucharest h = 253 h = 0 d = 140+99+211 = 450 km Is this the optimal solution? No: Arad  Sibiu  Rimnicu Vilcea  Pitesti  Bucharest 418 km

13. Don’t be greedy? • Greedy methods get stuck in local minima (maxima) Robot GOAL

14. Optimal? Complete? Time complexity? Space complexity? No No Exponential: O( bm ) (worst case) Exponential: O( bm ) – keeps all nodes in memory Greedy Best-First Search A good heuristic function reduces the (practical) complexity substantially!

15. “A” Search • Uniform-cost search minimizes g(n)(“past” cost) • Greedy search minimizes h(n) (“expected” or “future” cost) • “A Search” combines the two: • Minimize f(n) = g(n) + h(n) • Accounts for the “past” and the “future” • Estimates the cheapest solution (complete path) through node n

16. A* Search • “A* Search” is A Search with an admissible h • h is optimistic – it never overestimates the cost to the goal • h(n)  true cost to reach the goal • h is a “Pollyanna” • So f(n) never overestimates the actual cost of the best solution passing through node n functionA*-SEARCH(problem, h) returns a solution or failure return BEST-FIRST-SEARCH(problem, f )

17. A* Example f(n) = g(n) + h(n)

18. Arad f = 0 + 366 = 366 Zerind Sibiu Timisoara f = 75 + 374 = 449 140 + 253 = 393 118+329=447 Oradea Arad Fagaras Rimnicu Vilcea 291+380=671 140+366=506 239+178=417 220+193=413 A* Example

19. Optimal? Complete? Time complexity? Space complexity? Yes Yes Exponential; better under some conditions Exponential; keeps all nodes in memory A* Search • Good news: • A* is optimally efficient for any particular h(n) • That is, no other optimal algorithm is guaranteed to expand fewer nodes

20. A* Search • What if g(n) 0 ? • Greedy best-first search • What if h(n) 0 ? • Uniform cost search • What if h(n) 0 and g(n)  depth(n) ? • Breadth first search • How would you make depth-first search? g(n)  – depth(n)

21. Memory Bounded Search • Memory, not computation, is usually the limiting factor in search problems • Certainly true for A* search • Why? What takes up memory in A* search? • IDA* and SMA* are designed to conserve memory

22. Iterative Deepening A* (IDA*) • IDA* is an optimal, memory-bounded, heuristic search algorithm • Requires space proportional to the longest path that it explores • Space estimate: O(bd) • Like Iterative Deepening Search • Uses f-cost limit rather than depth-limit • In IDS, depth-limit is incremented after each round • In IDA*, f-cost limit is updated after each round

23. Simplified Memory-Bounded A* (SMA*) • IDA* only keeps around the current f-cost limit • Can check the current path for repeated states, but future paths may repeat states already expanded • SMA* uses more memory to keep track of repeated states • Up to the limit of allocated memory • Nodes with high f-cost are dropped from the queue when memory is filled (“forgotten nodes”) • Optimality and completeness depends on how much memory is available with respect to the optimal solution • Produces the best solution that can be reached given the available memory

24. The cost of being informed • Typical performance of informed search methods is much better than uninformed methods • Assuming reasonable heuristics exist • However, there is a tradeoff involved • Evaluating the desirability of a node (h)can be a non-trivial problem • E.g., theorem proving • How much closer to the theorem are we if we apply this rule? • Cost of evaluation (heuristic) can be high • It can be a difficult search problem itself

25. Overall cost Cost Cost of evaluating nodes (control strategy) Cost of expanding nodes (rule application) “Informedness” Cost tradeoff There may be different optima for computation and memory

26. S0 S1 S2 Sn Iterative Improvement Algorithms • An iterative improvement algorithm starts with a (possibly random) proposed solution, and then makes modifications to improve its quality • Each state is a (proposed) solution • Usually keeps information on current state only • Generally for problems in which non-optimal solutions are known, or easily generated • Task: Find the solution that best satisfies the goal test • State space for an IIA = set of all (proposed) solutions • Examples: VLSI layout, TSP, n-queens • Not appropriate for all problems!

27. Start -5 1 iteration -3 Goal 0 Example • n-Queens problem: Put n queens on an n x n chess board with no two queens on the same row, column, or diagonal

28. Example • Traveling Salesman Problem (TSP) • Start with any path through the cities • Change two links at a time

29. Search vs. iterative improvement Search Iterative improvement

30. Iterative Improvement Algorithms • Two classes of iterative improvement algorithms • Hill-climbing • Simulated annealing • Analogy: • You are placed at a random point in some unfamiliar terrain (the solution space) and told to reach the highest peak. It is dark, and you have only a very weak flashlight (no map, compass, etc.). • What should you do?

31. Hill Climbing, a.k.a. Gradient Descent • Strategy: Move in the direction of increasing value (decreasing cost) • Assumes a reasonable evaluation method!!! • n-queens? TSP? VLSI layout?

32. Hill climbing example Measure of value “Solution space”

33. Hill climbing issues “Solution space” • Strategy: • Climb until goal or stuck • If stuck, restart in random location

34. Hill climbing issues • Does not maintain a search tree • Evaluates the successor states, and keeps only the best one • Greedy strategy • Drawbacks • Local maxima • Plateaus and ridges • Can randomize (re-)starting locations and local strategies when stuck • “Random restart hill-climbing” • But how to know when you’re stuck?

35. Simulated Annealing • Similar to hill-climbing • But includes a random element • Sometimes take “bad” steps to escape local maxima • Motivated by the roughly analogous physical process of annealing: heating and then slowly cooling a substance to obtain a strong crystalline structure • Analogy with physical annealing: • T is temperature, E is energy • A schedule determines the rate at which T is lowered • Value(state) measures the state “goodness” • E measures increase in “goodness” resulting from new state