Informed Search Methods

1. How can we make use of other knowledge about the problem to improve searching strategy? Map example: Heuristic: Expand those nodes closest in “as the crow flies” distance to goal 8-puzzle: Heuristic: Expand those nodes with the most tiles in place Informed Search Methods

2. Best-First Search • Create evaluation function which returns estimated “value” of expanding node • Greedy search • Estimate cost of cheapest path from node n to goal • h(n) = “as the crow flies distance”

4. GreedySearch h(n)=366 Arad h(n)=374 h(n)=253 h(n)=329 Timisoara Zerind Sibiu Arad Oradea Fagaras Riminicu h(n)=178 h(n)=193 h(n)=366 h(n)=380 Sibiu Bucharest h(n)=253 h(n)=0

5. Greedy Search • Expand the node with smallest h • Why is it called greedy? • Expands node that appears closest to goal • Similar to depth-first search • Follows single path all the way to goal, backs up when dead end • Worst case time: • O(bm), m = depth of search space • Worst case memory: • O(bm), needs to store all nodes in memory to see which one to expand next

6. Greedy Search • Complete and/or optimal? • No – same problems as depth first search • Can get lost down an incorrect path • How can you (help) to prevent it from getting lost? • Look at shortest total path, not just path to goal

7. A* search (another Best-First Search) • Greedy search minimizes • h(n) = estimated cost to goal • Uniform cost search mininizes • g(n) = cost to node n • Example of each on map • A* search minimizes • f(n) = g(n) + h(n) • f(n) = best estimate of cost for complete solution through n

8. A* search • Under certain conditions: • Complete • Terminates to produce best solution • Conditions: • h(n) must never overestimate cost to goal • admissible heuristic • “optimistic” • “Crow flies” heuristic is admissible • Heuristic is monotone • Never decreases along any path from root • Can always generate a monotone heuristic from a non-monotone one

9. A*Search f(n) = 366 Arad f(n) = 449 f(n) = 393 f(n) = 447 Timisoara Zerind Sibiu Arad Oradea Fagaras Riminicu f(n) = 417 f(n) = 413 f(n) = 646 f(n) = 526 Craiova Pitesti Sibiu f(n) = 526 f(n) = 415 f(n) = 553

10. A*Search Arad Oradea Fagaras Riminicu f(n) = 417 f(n) = 413 f(n) = 646 f(n) = 526 Craiova Riminicu Pitesti Craiova Bucharest Sibiu f(n) = 526 f(n) = 607 f(n) = 415 f(n) = 615 f(n) = 553 f(n) = 418

11. A*Search f(n) = 366 Arad f(n) = 449 f(n) = 393 f(n) = 447 Timisoara Zerind Sibiu Arad Oradea Fagaras Riminicu f(n) = 417 f(n) = 413 f(n) = 646 f(n) = 526 Sibiu Bucharest f(n) = 591 f(n) = 450

12. A* terminates with optimal solution • A* stops when you try to expand a goal state • ... and declares this the best solution • How does it know this is the best? • Suppose you try to expand a non-optimal goal state • A* always expands node with smallest f • Since heuristic is admissible, f is an underestimate • If there is a better goal state available, with a smaller f, there must be a node on graph with smaller f than current – so you would be expanding that instead!

13. More about A* • Completeness • A* expands nodes in order of increasing f • Must find goal state unless • infinitely many nodes with f(n) < f* • infinite branching factor OR • finite path cost with infinite nodes on it • Complexity • Time: Depends on h, can be exponential • Memory: O(bm), stores all nodes

14. Valuing heuristics • Example: 8-puzzle • h1 = # of tiles in wrong position • h2 = sum of distances of tiles from goal position (1-norm, or Manhattan distance) • Which heuristic is better for A*? • Obvious that h2(n) >= h1(n) for any n • h2 dominates h1 • A* will expand less nodes with h2 than with h1 • Since h2 >= h1, any node that A* expands with h2 it will certainly expand with h1 • But A* may be able to avoid expanding some nodes with h2 (larger than goal state f) • Better to use larger heuristic (if not overestimate)

15. Inventing heuristics • h1 and h2 are exact path lengths for simpler problems • h1 = path length if you could transport each tile to right position • h2 = path length if you could just move each tile to right position, irrelevant of blank space • Relaxed problem: less restrictive problem than original • Can generate heuristics as exact cost estimates to relaxed problems

16. Today • Remainder of informed search • Assignment 3: 8-puzzle • On Friday: Chapter 5 (Game playing) • Question of the day: • Is chess episodic?

17. Heuristics in Constraint Satisfaction Problems • CSP: Find values for a set of variables, subject to a set of constraints • Example: Place as many knights as possible on a chessboard so that none attack another • Easier example: Map coloring

18. Color this map... • ... using only red, green, and blue: • no adjacent regions have same color

19. Color this map... • ... using only red, green, and blue: • no adjacent regions have same color

20. Strategies • Variable: • Choose variable with fewest possible choices • most-constrained-variable heuristic • Choose variable that limits other choices the most • most-constraining-variable heuristic • Values: • Choose value that leaves most options available • least-constraining-value heuristic

21. Memory Bounded Search • Can A* be improved to use less memory? • Iterative deepening A* search (IDA*) • Each iteration is a depth-first search, just like regular iterative deepening • Each iteration is not an A* iteration: otherwise, still O(bm) memory • Use limit on cost (f), instead of depth limit as in regular iterative deepening

22. IDA*Search f-Cost limit = 366 f(n) = 366 Arad f(n) = 449 f(n) = 393 f(n) = 447 Timisoara Zerind Sibiu

23. IDA*Search f-Cost limit = 393 f(n) = 366 Arad f(n) = 449 f(n) = 393 f(n) = 447 Timisoara Zerind Sibiu Arad Oradea Fagaras Riminicu f(n) = 417 f(n) = 415 f(n) = 646 f(n) = 526

24. IDA* Analysis • Time complexity • If cost value for each node is distinct, only adds one state per iteration • BAD! • Can improve by increasing cost limit by a fixed amount each time • If only a few choices (like 8-puzzle) for cost, works really well • Memory complexity • Approximately O(bd) (like depth-first) • Completeness and optimality same as A*

25. Simplified Memory-Bounded A* (SMA*) • Uses all available memory • Basic idea: • Do A* until you run out of memory • Throw away node with highest f cost • Store f-cost in ancestor node • Expand node again if all other nodes in memory are worse

26. SMA* Example: Memory of size 3 A f = 12

27. SMA* Example: Memory of size 3 A f = 12 B f = 15 Expand to the left

28. SMA* Example: Memory of size 3 A f = 12 B f = 15 C f = 13 Expand node A, since f smaller

29. SMA* Example: Memory of size 3 A f = 12 forgotten f = 15 C f = 13 D f = 18 Expand node C, since f smaller

30. SMA* Example: Memory of size 3 A f = 12 forgotten f = 15 C f = 13 forgotten f = infinity E f = 24 Node D not a solution, no more memory: so expand C again

31. SMA* Example: Memory of size 3 A f = 12 B f = 15 C f = 13 Forgottenf = 24 (right) Re-expand A; record new f for C

32. SMA* Example: Memory of size 3 A f = 12 forgotten = 24 B f = 15 F f = 25 Expand left B: not a solution, so useless

33. SMA* Example: Memory of size 3 A f = 12 Forgotten f = 24 B f = 15 forgotten f = inf G f = 20 Expand right B: find solution

34. SMA* Properties • Complete if can store at least one solution path in memory • Finds best solution (and recognizes it) if path can be stored in memory • Otherwise, finds best that can fit in memory

35. Iterative Improvement Algorithms • For some problems, path to solution is irrelevant: just want solution itself • Start with an initial state, and change it iteratively to improve it • Examples: • Placing queens on a chessboard • How many airline flights to have to where • If you know the function, can take derivative: solve derivative = 0

36. Hill-climbing search(or gradient descent) • Example: Allocating professors to departments • M&CS: 10 • Physics: 5 • English: 7 • Goal: Maximize majors in 3 departments • For a given configuration of profs, I know (by simulation or something) how many money college will make (tuition for majors – salary) • Assume profs make $10k / year • Start adding profs, end up in local maximum • Show on blackboard

37. Hill-climbing in general • Move in direction of increasing value • Useful when path to solution is irrelevant • Drawbacks: • Local maxima • Plateaux • Ridges • Can get around this some with random-restart hill climbing

38. Simulated Annealing • Technique inspired by engineering practice of cooling liquid • At each iteration make a random move • If position is better than current, do it • Over time, slowly drop “temperature” T • If position is worse, do it with probability P • P becomes smaller as T drops • P = exp(change in value / T) • Eventually, algorithm reverts to hill climbing • Popular in VLSI layout

39. Iterative Improvement for CSPs • Start with an initial configuration (like queens on a chessboard) • Min-conflicts heuristic • Choose a new value that results in minimum number of conflicts with other variables • Queens example