More advanced aspects of search

1. More advanced aspects of search Optimality proof of A* Further properties of A* Extensions of A* Concluding comments

2. Optimality of A*:a formal proof Terminology Basic properties of A* Termination of A* Finds a path Finds the best path

3. A* algorithm: 1. QUEUE <-- path only containing the root; 2. WHILE QUEUE is not empty ANDfirst path does not reach goal remove the first path from the QUEUE; create new paths (to all children); reject the new paths with loops; add the new paths and sort the entire QUEUE; IFQUEUE contains path P terminating in I, with cost cost_P, and path Q containing I, with cost cost_Q AND cost_Pcost_Q THEN delete P 3. IF goal reached THEN success; ELSE failure; (by f = cost + h)

4. S p D N C B A M M E G Terminology and notation: • Paths: p, q, … • Accumulated cost of a path p: cost(p) • End node of a path: end(p) • The best path: B = S,N,M,…,G • Subpaths: see drawing: • p = SN is a subpath of B • assumes that it is an initial part of B

5. For each subpath p of B: f(p)  cost(B) S p D N C E B A M M f(p) = cost(p) + h(end(p) ) cost(B) Underestimate of remaining cost G Property 1:

6. At each WHILE-test during A* QUEUE contains a subpath of B S p Initially: S  QUEUE S is subpath of B D N C E B A M M G Note: Subpaths of B can not be redundant. IfB QUEUE, then it never becomes removed. Property 2: During each WHILE-loop: If a subpath of B is selected and removed: - all its extensions are computed, - at least 1 of these is a subpath of B and has no loop (B has no loop), - at least 1 subpath of B is added

7. There is always some q subpath of B in QUEUE, with f(q)cost(B) f(q)cost(B)<f(p) p can never be selected Corollary: For any path p, such that f(p) > cost(B) p is never selected during A* (and never becomes first in QUEUE). Property 1 and Property 2:

8. f(p) = cost(p) + h(end(p))cost(p)d * >cost(B) p is never selected S A B C D F G d H I J there are only a finite number of nodes in the tree that can be selected ! finite branching factor Since: A* removes 1 path from the QUEUE at each step and that same path can never be generated again: TERMINATION Termination of A*: Let d be any natural number >cost(B) /  Let p be any path of length d

9. property 2: goal found corollary B’is never becomes first A* returns a path to a goal On termination, either: goal found empty QUEUE A* returns the best path Assume it returns B’, with cost(B’) > cost(B): f(B’) = cost(B’) + h(B’)  cost(B’) > cost(B)

10. Other properties of A* Classification of which nodes are selected. Heuristic power. Finding the best path to each node: Monotonicity restriction

11. Which nodes get selected?

12. For any path p : Iff(p)>cost(B) , thenp is never selected Ifp QUEUE andp is not removed as redundant andf(p)<cost(B) , thenp is selected For the second: For B we have: f(B) = cost(B) + h( end(B) ) = cost(B) h underestimates remaining cost p is selected! A better characterization of the selected nodes: First implication: corollary A* does not terminate before B becomes the first in the QUEUE and selects lower f-value paths first

13. f(p)cost(B) S selected if generated and not removed A B C f(p) = cost(B) D E F G H I don’t know f(p)cost(B) J K L M N O not selected General picture: • Search tree is partitioned in 3 layers:

14. S 3 4 10 4 (7) B B (0) A A f(SBC) = 6  cost(B) = 8, but SBC never enters the QUEUE! 3 4 (0) C 4 8 G (0) The f(p) cost(B) layer: • Is the top layer completely selected? • Can it happen that some path p with f(p)cost(B) never turns up in the QUEUE? • YES ! Termination

15. Heuristic power To what extend is A* improved by better heuristic functions? Can we compare them?

16. h2(T) = number or incorrectly placed tiles on board: h2 h3 = 4 1 1 3 3 2 2 8 8 4 4 • h3(T) = manhattan distance: 5 5 6 6 7 7 = 1 + 1 + 2 + 2 = 6 Some underestimating heuristics for 8-puzzle: • h1(T)  0 Note: in every state T: h3(T)  h2(T)  h1(T)

17. 0 0+4 h1 = 0 0+5 2 8 3 1 6 4 7  5 h2 = misplaced h3 = manhattan 1+5 1+3 1+5 1 1 1 2 8 3 1 6 4  7 5 2 8 3 1  4 7 6 5 2 8 3 1 6 4 7 5  1+6 1+4 1+6 2+3 2+3 2+4 2 2 2 2 2 2 8 3 1 6  7 5 4 2 8 3  6 4 1 7 5 2 8 3  1 4 7 6 5 2  3 1 8 4 7 6 5 2 8 3 1 4  7 6 5 Not selected 2+5 2+3 2+5 3+3 3+4 3+2 3 3 3 3 3 3 3 3 3 3  8 3 2 6 4 1 7 5 2 8 3 6  4 1 7 5  8 3 2 1 4 7 6 5 2 8 3 7 1 4 6 5  2 3 1 8 4 7 6 5 2 3  1 8 4 7 6 5 2 8 3 1 4 5 7 6  2 8  1 4 3 7 6 5 2 8 3 1  6 7 5 4 2 8  1 6 3 7 5 4 Heuristic power in 8-puzzle:

18. Definition: A*( h1) is more informed than A*( h2) ifh1(T) h2(T) , for all T  G IFA*( h1) is more informed than A*( h2) ANDthere exists a path to a goal, THEN A*( h2) selects every node that A*( h1) does. Heuristic power of h:

19. Minimal cost paths&Monotonicity How to avoid redundant path deletion.

20. S 2 2 6 2 (0) B B (4) A A 2 3 4 Optimality of this path to C not detected C (0) 5 (0) C C 2 7 (0) D Minimal-cost paths to nodes. • A same node may be found multiple times (possibly with different costs ) • dealt with by redundant path deletion in A* • Do we detect all redundant paths ?? NO !

21. for each arc h(A) h(B) + cost(A,B) , and • h(G) = 0 A B Monotonicity: • Definition: A heuristic function h satisfies the monotonicity restriction if:

22.  5 h2 h2 = 4 1 1 3 3 2 2 8 8 4 4 • 8-puzzle:h3 : 5 5 6 6 7 7  7 h3 h3 = 6 Monotonicity: examples: • for each arc: h(A) h(B) + cost(A,B) • 8-puzzle:h2 : • 8-puzzle:h1 : h1  0

23. Monotonicity: the result • So: • under monotonicity: each node is reached through the best path first ! IF h satisfies the monotonicity restriction AND A*( h) selects path p : THENcost(p) is the lowest cost with which we can ever reach end(p)

24. Redundant Path Deletion can be done MUCH more efficiently WhenA*( h) selects path p : Rememberend(p) has been selected (using hash table) Practical Relevance: IFA*( h) later selects a path q with end(q) = end(p) (check via table) Do NOTextend q, simply remove it from the QUEUE

25. (4) 5 + 4 = 9 A A 1 (2) 6 + 2 = 8 B Monotonicity: further motivation • It is unreasonable that f decreases along a path: • This throws away information !! • We already knew that total cost on this path to the goal is at least 9 (knowledge in node A) • So how can this estimate be lower in node B ? • Cannot happen under Monotonicity !

26. IFwe have an arc , and f(S…AB)f(S...A),THENcorrect the value of f(S…AB) to be at least f(S...A) ! A B MONOTONICITY ! Pathmax • == a variant of A* to enforce monotonicity. • Redefine f: • f ’(S…AB) = max( f(S…AB) , f ’(S...A) ) • Is as if we are working with an h’ that satisfies monotonicity: • h’(A) = f ’(S...A) - cost (S…A) • f ’(S…AB) - cost(S…A) • = f ’(S…AB) - cost(S…AB) + cost(A,B) • = h’(B) + cost(A,B)

27. Extensions of A* Iterated deepening A* Simplified Memory-bounded A*

28. Iterative-deepening A*

29. f1 A* d = 1 Breadth- first f2 d = 2 d = 3 f3 d = 4 f4 Expand by depth-layers Expands by f-contours Memory problems with A* • A* is similar to breadth-first: • Here: 2 extensions of A* that improve memory usage.

30. Depth-first in each f- contour • Perform depth-first search LIMITED to some f-bound. • If goal found: ok. • Else: increase de f-bound and restart. f1 f2 f3 f4 Iterative deepening A* How to establish the f-bounds? - initially: f(S) generate all successors record the minimalf(succ)>f(S) Continue with minimalf(succ) instead of f(S)

31. f-new = 120 S f=100 A f=120 B f=130 C f=120 D f=140 G f=125 E f=140 F f=125 Example: f-limited, f-bound = 100

32. f-new = 125 S f=100 A f=120 B f=130 C f=120 D f=140 G f=125 E f=140 F f=125 Example: f-limited, f-bound = 120

33. S f=100 A f=120 B f=130 C f=120 D f=140 G f=125 E f=140 F f=125 SUCCESS Example: f-limited, f-bound = 125

34. f-limited search: 1. QUEUE <-- path only containing the root; f-bound <-- <some natural number>; f-new <--  2. WHILEQUEUE is not empty AND goal is not reached DO remove the first path from the QUEUE; create new paths (to all children); reject the new paths with loops; add the new paths with f(path)f-bound to front of QUEUE; f-new <-- minimum of current f-new and of the minimum of new f-values which are larger than f-bound 3. IF goal reached THEN success; ELSE report f-new ;

35. 1. f-bound <-- f(S) 2. WHILE goal is not reached DO perform f-limited search; f-bound <-- f-new Iterative deepening A*:

36. Properties of IDA* • Complete and optimal: • under the same conditions as for A* • Memory: • Let be the minimal cost of an arc: • == O( b* (cost(B) /)) • Speed: • depends very strongly on the number of f-contours there are !! • In the worst case: f(p)  f(q) for every 2 paths: • 1 + 2 + ….+ N = O(N2)

37. In absence of Monotonicity: • we can have search spaces like: 100 S 120 120 A B C D E F 140 150 90 60 • If f can decrease, • how can we be sure that the first goal reached is the optimal one ??? HOMEWORK Why is this optimal,even without monotonicity ??

38. IDA* is one of the very best optimal search techniques ! • Example: the 8-puzzle • But: also for MANY other practical problems • increase f-bound by a fixed number at each iteration: • effects: less re-computations, BUT: optimality is lost: obtained solution can deviate up to  Properties: practical • Ifthere are only a reduced number of different contours: • Else, the gain of the extended f-contour is not sufficient to compensate recalculating the previous • In such cases:

39. Simplified Memory-bounded A*

40. If memory is full and we need to generate an extra node (C): • Remove thehighest f-value leaffrom QUEUE(A). • Remember thef-value of the best ‘forgotten’ childin each parent node(15 in S). 13 S 13 15 A B B 18 C memory of 3 nodes only Simplified Memory-bounded A* • Fairly complex algorithm. • Optimizes A* to work within reduced memory. • Key idea: (15)

41. When expanding a node (S), only add its children 1 at a time to QUEUE. • we use left-to-right • Avoids memory overflow and allows monitoring of whether we need to delete another node 13 S A A B B First add A, later B Generate children 1 by 1

42. If extending a node would produce a path longer than memory: give up on this path (C). • Set the f-value of the node (C) to  • (to remember that we can’t find a path here) 13 S 13 B B  18 C C D memory of 3 nodes only Too long path: give up

43. If all children M of a node N have been explored and for all M: • f(S...M)f(S...N) • then reset: • f(S…N)= min { f(S…M) | M child of N} • A path through N needs to go through 1 of its children ! 15 13 S 24 15 A B Adjust f-values better estimate for f(S)

44. 0+12=12 10 S 8 8+5=13 10+5=15 A B 8 16 10 10 20+0=20 16+2=18 20+5=25 24+0=24 C G1 D G2 8 10 10 8 30+5=35 24+5=29 E G3 30+0=30 G4 F 24+0=24 12 12 12 13 S S S S A A A B A B 13 13 15 15 15 15  D 18 SMA*: an example: 12 13 (15)

45. 0+12=12 10 S 8 8+5=13 B 10+5=15 A B 13 8 16 10 10 20+0=20 16+2=18 20+5=25 24+0=24 C G1 D G2  8 10 10 8 D 24+0=24 30+5=35 24+5=29 E G3 30+0=30 G4 F (15) 13 (15) 15 15 15 (24) 13 (15) S S S S S 13 B B B B A A A () 15 24 24 24 15 15 G2 G2 G2      C C C G1 D D 24 24 24 25 20 Example: continued 15 (15) (24) 20 13 15 () 13 20 () () 24 15

46. SMA*: properties: • Complete: If available memory allows to store the shortest path. • Optimal: If available memory allows to store the best path. • Otherwise: returns the best path that fits in memory. • Memory: Uses whatever memory available. • Speed: If enough memory to store entire tree: same as A*

47. Concluding comments More on non-optimal methods The optimality Trade-off

48. 8 4 7 8 1 better than 5 5 3 4 3 6 1 7 2 6 2 Non-optimal variants • Sometimes ‘non-admissible’ heuristics desirable: • Example: symmetry • but cannot be captured with underestimating h • = Use non-admissible A*.

49. f(S…N) =  * cost(S…N) + h(N) ,0   1 = 0 : pure heuristic best first (Greedy search) = 1 : A* Non-optimal variants (2) • Reduce the weight of the cost in f:

50. Polynomial parallel algorithms exist, but ALL KNOWN sequential algorithms are exponential • The trade-off: • either use algorithms that; • ALWAYS give the optimal path Approaching the complexity • Optimal path finding is by nature NP complete ! • in the worst case (depending on the actual search space !) , behave exponential • in the average case are polynomial