Automated Planning and Decision Making - Search Methods

1. Search Automated Planning and Decision Making 2007

2. Introduction • In the past, most problems you encountered were such that the way to solve them was very “clear” and straight forward: • Solving quadratic equation. • Solving linear equation. • Matching input to a regular expression. • … • Most of the problems in AI are of different kind. • NP-Hard problems. • Require searching in a large solution space. Automated Planning and Decision Making

3. Searching Problems • A Search Problem is defined as: • A set S of possible states – the search space. • Initial State: sI in S. • Operators - which take us from one state to another (S→S functions) • Goal Conditions. • The Search Space defined by a Search Problem is a graph G(V,E) where: • V={v : v in S} • E={<v,u> : exists o in Operators such that o(v)=u} • The goal of search is to find a path in the search space from sI to a state s where the Goal Conditions are valid. • Sometimes we only care about s, and not the path • Sometimes we only care whether s exists. • Sometimes value is associated with states, and we want the best s. • Sometimes cost is associated with operators, and we want the cheapest path Automated Planning and Decision Making

4. Search Problem Example • Finding a solution for Rubik's Cube. • SI: Some state of the cube. • Operators: 90o rotation of any of the 9 plates. • Goal Conditions: Same color for each of the cube's sides. • The Search Space will include: • A node for any possible state of the cube. • An edge between two nodes if you can reach from one to the other by a 90o rotation of some plate. Automated Planning and Decision Making

5. Another Example • States: Locations of Tiles. • Operators: Move blank right/left/down/up. • Goal Conditions: As in the picture. • Note: optimal solution of n-Puzzle family isNP-hard… Initial State Goal State Automated Planning and Decision Making

6. Solving Search Problems • Apply the Operators on States in order to expand (produce) new States, utill we find one that satisfies the Goal Conditions. • At any moment we may have different options to proceed (multiple Operators may apply) • Q: In what order should we expand the States? • Our choice affects: • Computation time • Space used • Whether we reach an optimal solution • Whether we are guaranteed to find a solution if one exists (completeness) Automated Planning and Decision Making

7. Search Tree • You can think of the search as spanning a tree: • Root: The initial state. • Children of node v are all states reachable from v by applying an operator. • We can reach the same state via different paths. • In such a case, we can ignore this duplicate state, and treat it as a leaf node • Important parameters affecting performance: • b – Average branching factor. • d – Depth of the closest solution. • m – Maximal depth of the tree. • Fringe: set of current leaf (unexpanded) nodes Automated Planning and Decision Making

8. Search Methods • Different search methods differ in the order in which they visit/expand the nodes. • It is important to distinguish between: • The search space. • The order by which we scan that space. • Blind Search • No additional information about search states is used. • Informed Search • Additional information used to improve search efficiency Automated Planning and Decision Making

9. Relation to Planning • Simplest way to solve planning problem is to search from the initial state to a goal • Search states are states of the world • Operators are possible actions • There are other formulations! • For instance: search states = plans • Search is a very general technique – very important for many applications Automated Planning and Decision Making

10. Blind Search Automated Planning and Decision Making

11. Blind Search • Main algorithms: • DFS – Depth First Search • Expand the deepest unexpanded node. • Fringe is a LIFO. • BFS – Breath First Search • Expand the shallowest unexpanded node. • Fringe is a FIFO. • IDS – Iterative Deepening Search • Combines the advantages of both methods. • Avoids the disadvantages of each method. • There are other variants… Automated Planning and Decision Making

12. Breadth First Search 1 2 3 4 Automated Planning and Decision Making

13. Properties of BFS • Complete?Yes (if b is finite). • Time?1+b+b2+b3+…+b(bd-1)=O(bd+1( • Space? O(bd( (keeps every node on the fringe). • Optimal?Yes (if cost is 1 per step). • Space is a BIG problem… Automated Planning and Decision Making

14. Depth First Search 1 2 3 4 5 7 8 6 9 10 11 12 Automated Planning and Decision Making

15. Properties of DFS • Complete? • No if given infinite branches (e.g., when we have loops) • Easy to modify: check for repeat states along path • Complete in finite spaces! • May require large space to maintain list of visited states • Time? • Worst case: O(bm(. • Terrible if m is much larger then d. • But if solutions occur often or in the “left” part of the tree, may be much faster then BFS. • Space? • O(m( Linear Space! • Optimal? • No. Automated Planning and Decision Making

16. Depth Limited Search • DFS with a depth limit ℓ, i.e., nodes at depth ℓ have no successors. Automated Planning and Decision Making

17. Iterative Deepening Search • Increase the depth limit of the DLS with each Iteration: Automated Planning and Decision Making

18. Iterative Deepening Search Automated Planning and Decision Making

19. Complexity of IDS • Number of nodes generated in a depth-limited search to depth d with branching factor b: NDLS = b0 + b1 + b2 + … + bd-2 + bd-1 + bd • Number of nodes generated in an iterative deepening search to depth d with branching factor b: NIDS = (d+1)b0 + db1 + (d-1)b2 + … + 3bd-2 +2bd-1 + 1bd • For b = 10, d = 5: • NDLS = 1 + 10 + 100 + 1,000 + 10,000 + 100,000 = 111,111 • NIDS = 6 + 50 + 400 + 3,000 + 20,000 + 100,000 = 123,456 • Overhead = (123,456 - 111,111)/111,111 = 11% Automated Planning and Decision Making

20. Properties of IDS • Complete?Yes. • Time?(d+1)b0 + db1 + (d-1)b2 + … + bd = O(bd) • Space?O(d) • Optimal?Yes. If step cost is 1. Automated Planning and Decision Making

21. Comparison of the Algorithms Automated Planning and Decision Making

22. Bidirectional Search • Assumptions: • There is a small number of states satisfying the goal conditions. • It is possible to reverse the operators. • Simultaneously run BFS from both directions. • Stop when reaching a state from both directions. • Under the assumption that validating this overlap takes constant time, we get: • Time: O(bd/2) • Space: O(bd/2) Automated Planning and Decision Making

23. Informed Search Automated Planning and Decision Making

24. Informed/Heuristic Search • Heuristic Function h:S→R • For every state s, h(s) is an estimation of the minimal distance/cost from s to a solution. • Distance is only one way to set a price. • How to produce h? later on… • Cost Function g:S→R • For every state s, g(s) is the minimal cost to s from the initial state. • f=g+h, is an estimation of the cost from the initial state to a solution. Automated Planning and Decision Making

25. Best First Search • Greedy on h values. • Fringe stored in a queue ordered by h values. • In every step, expand the “best” node so far, i.e., the one with the best h value. Automated Planning and Decision Making

26. Romania with step cost in KMs • How to reach from Arad to Bucharest? Automated Planning and Decision Making

27. 1 2 3 4 Solution by Best First Automated Planning and Decision Making

28. Properties of Best First • Complete? • No, can get into loops • Time? • O(bm). But a good heuristic can give dramatic improvement. • Space? • O(bm). Keeps all nodes in memory. • Optimal • No. Automated Planning and Decision Making

29. A* Search • Idea: Avoid expanding paths that are already expensive. • Greedy on f values. • Fringe is stored in a queue ordered by f values. • Recall, f(n)=g(n)+h(n), where: • g(n): cost so far to reach n. • h(n): estimated cost from n to goal. • f(n): estimated total cost of path through n to goal. Automated Planning and Decision Making

30. Solution by A* 1 2 3 4 Automated Planning and Decision Making

31. Solution by A* 4 5 6 Automated Planning and Decision Making

32. Admissible Heuristics • A heuristic h(n) is admissible if for every node n, h(n) ≤ h*(n), where h*(n) is the realcost to reach the goal state from n. • An admissible heuristic never overestimates the cost to reach the goal, i.e., it is optimistic • Theorem:If h(n)is admissible, A* using Tree-Search is optimal. • Tree search – nodes encountered more than once are not treated as leaf nodes Automated Planning and Decision Making

33. Optimality of A* (Proof) • Suppose some suboptimal goal G2has been generated and is in the fringe. Let n be an unexpanded node in the fringe such that n is on a shortest path to an optimal goal G. Automated Planning and Decision Making

34. Optimality of A* (Proof) • f(G2) = g(G2) since h(G2) = 0. • f(G) = g(G) since h(G) = 0. • g(G2) > g(G) since G2 is suboptimal. • f(G2) > f(G) by 1,2,3. • h(n) ≤ h*(n) since h is admissible. • g(n)+h(n) ≤ g(n)+h*(n) by 5. • f(n) ≤ f(G) by definition of f. • f(G2) > f(n) by 4,7. Hence A* will never select G2 for expansion. Automated Planning and Decision Making

35. Consistent Heuristics • A heuristic is consistent (monotonic) if for every node n, every successor n' of n generated by any action a, h(n) ≤ c(n,a,n') + h(n') • If h is consistent, we have:f(n') = g(n') + h(n') = g(n) + c(n,a,n') + h(n') ≥ g(n) + h(n) = f(n) i.e., f(n) is non-decreasing along any path. • TheoremIf h(n) is consistent, A* using Graph-Search is optimal. • Graph-search: nodes encountered a second time can be treated as leaf nodes. Automated Planning and Decision Making

36. Optimality of A* • A* expands nodes in order of increasing f value. • Gradually adds "f-contours" of nodes . • Contour i has allnodes with f=fi, where fi < fi+1. Automated Planning and Decision Making

37. Properties of A* • Complete?Yes, unless there are infinitely many nodes with f ≤ f(G). • Time?Exponential. • Space?Keeps all nodes in memory. • Optimal?Yes. Automated Planning and Decision Making

38. Iterative Deepening A* (IDA*) • Use Iterative Deepening with A* in order to overcome the space problem. • A simple idea: • Define an upper bound U for f values. • If f(n)>U, n is not enqueued. • If no solution was found for some U, increase it. Automated Planning and Decision Making

39. Branch & Bound • Used when searching for optimal node (i.e., in problems where there are multiple possible solutions, but some are better than others) • Attempts to prune entire sub-trees • Applicable in the context of different search methods (BFS, DFS, …). • Assume an upper and lower bound U on every node n • They bound the value of the best descendent of n • If for some node n there exists a node n’ such thatU(n)<I(n’), we can prune n. Automated Planning and Decision Making

40. Local search algorithms • In many problems, the path to the goal is irrelevant; the goal state itself is all we care about. • Exampe: n-queens problems • More generally: constraint-satisfaction problems • Local search algorithms keep a single "current" state, and try to improve it. Automated Planning and Decision Making

41. Hill-climbing search • “Like climbing mount Everest in thick fog with amnesia…” • Improve while you can, i.e. stop when reaching a maxima (minima). Automated Planning and Decision Making

42. Hill-climbing search • Problem: depending on initial state, can get stuck in local maxima. Automated Planning and Decision Making

43. N-Queens Problem • Situate n queens on a n×n board such that no two queens threat one another. i.e., no two queens share row, column or a diagonal. Automated Planning and Decision Making

44. Solution by Hill-climbing A • h = number of pairs of queens that are attacking each other, either directly or indirectly. • Moves: change position of a single queen • On A, h=17. • On B, h=1. local minimum. B Automated Planning and Decision Making

45. Simulated annealing search • Idea: escape local maxima by allowing some "bad" moves, but gradually decrease their frequency. Automated Planning and Decision Making

46. Properties of SAS • One can prove: If T decreases slowly enough, then SAS will find a global optimum with probability approaching 1. • Widely used in VLSI layout, airline scheduling, etc. Automated Planning and Decision Making

47. Local beam search • Keep track of k states rather than just one. • Start with k randomly generated states. • At each iteration, all the successors of all k states are generated. • If any one is a goal state, stop; else select the k best successors from the complete list and repeat. Automated Planning and Decision Making

48. Heuristic Functions • Heuristic functions are usually obtained by finding a simple approximation to the problem • Simple heuristics often work well (but not very well) • Examples: • N-puzzle: Manhattan distance • Simplification: assumes that we can move each tile independently. Ignores interactions between tiles • A very general idea – ignore certain interactions Automated Planning and Decision Making