Artificial Intelligence I nformed search and exploration

1. Artificial IntelligenceInformed search and exploration Fall 2008 professor: Luigi Ceccaroni

2. Heuristic search • Heuristic function, h’(n): it estimates the lowest cost from the n state to a goal state. • At each moment, most promising states are selected. • Finding a solution (if it exist) is not guaranteed. • Types of heuristic search: • BB (Branch & Bound), Best-first search • A, A* • IDA* • Local search: • hill climbing • simulated annealing • genetic algorithms 2

3. Branch & Bound • For each state, the cost is kept of getting from the initial state to that state (g). • The global, minimum cost is kept, too, and guides the search. • A branch is abandoned if its cost is greater than the current minimum. 3

4. Best-first search • Priority is given by the heuristic function (estimation of the cost from a given state to a solution). • At each iteration the node with minimum heuristic function is chosen. • The optimal solution is not guaranteed.

5. Greedy best-first search

6. B G C H A A D G D C B H E E F F Importance of the estimator Operations: - put a free block on the table - put a free block on another free block Estimator H1: - add 1 for each block which is on the right block - subtract 1 otherwise Initial state H1 = 4 H2 = -28 Estimator H2: - for each block, if the underlying structure is correct, add 1 for each block of that structure - otherwise, subtract 1 for each block of the structure Final state H1 = 8 H2 = 28 (= 7+6+5+4+3+2+1)

7. B G H G A E B C D E D H H C B A A G D C H A G D C B E E F F F F Initial state H1 = 4 H2 = -28 H1 = ? H2 = ? H1 = ? H2 = ? H1 = ? H2 = ?

8. B G H G A E B C D E D H H C B A A G D C H A G D C B E E F F F F Initial state H1 = 4 H2 = -28 H1 = 6 H2 = -21 H1 = 4 H2 = -15 H1 = 4 H2 = -16

9. Heuristic functions Initial state Possible heuristic functions: h(n) = w(n) = #misplaced h(n) = p(n) = sum of distances to the final position h(n) = p(n) + 3 ·s(n) where s(n) is obtained cycling over non-central positions and adding 2 if a tile is not followed by the right one and adding 1 if there is a tile in the center 2 8 3 1 6 4 7 5 1 2 3 8 4 7 6 5 Final state

10. The estimation function (h’) has two components: Minimum cost to get from an initial state to the current state Minimum (estimated) cost to get from the current state to a solution f’(n) = g(n) + h’(n) f’ is an estimation of the total cost. h’ is an estimation of the real cost to get to a solution (h). g is a real cost of the minimum path to the current state. Priority is given to nodes with lower f’. In case of same h, priority is given to the node with lower h’. If h’(n) never overestimates the real cost, that is ∀n h’(n) ≤ h(n), it can be demonstrated that the algorithm can find the optimal path. A and A* algorithms A A*

11. A* algorithm

12. A* algorithm • The priority is given by the estimation function f’(n)=g(n)+h’(n). • At each iteration, the best estimated path is chosen (the first element in the queue). • A* is an instance of best-first search algorithms. • A* is complete when the branching factor is finished and each operator has a constant, positive cost.

13. A*: treatment of repeated states • If the repeated state is in the structure of open nodes: • If its new cost (g) is lower, the new cost is used; possibly changing its position in the structure of open nodes. • If its new cost (g) is equal or greater, the node is forgotten. • If the repeated state is in the structure of closed nodes: • If its new cost (g) is lower, the node is reopened and inserted in the structure of open nodes with the new cost. Nothing is done with its successors; they will be reopened if necessary. • If its new cost (g) is equal or greater, the node is forgotten. 13

14. Example: Romania with step costs in km

15. A* search example

16. A* search example

17. A* search example

18. A* search example

19. A* search example

20. A* search example

21. A*: admissibility • The A algorithm, depending on the heuristic function, finds or not an optimal solution. • If the heuristic function is admissible, the optimization is granted. • A heuristic function is admissible if it satisfies the following property: ∀n 0 ≤ h’(n) ≤ h(n) • h’(n) has to be an optimistic estimator; it never has to overestimate h(n). • Using an admissible heuristic function guarantees that a node on the optimal path never seems too bad and that it is considered at some point in time.

22. Example: no admissibility h=3 /\ / \ h=2 h=4 | | | | h=1 h=1 | | | | h=1 goal | | h=1 | | goal

23. Optimality of A* • A* expands nodes in order of increasing f value. • Gradually adds "f-contours" of nodes. • Contour i has all nodes with f=fi, where fi < fi+1.

24. Given a problem, there exist as many A* to solve it as heuristic functions we can define. More and less informed algorithms A1*, A2* admissible ∀n≠final: 0 ≤ h’2(n) < h’1(n) ≤ h(n) ⇒ A1* more informed than A2*

25. More informed algorithms A1* more informed than A2* ⇓ n expanded by A1* ⇒ n expanded by A2* ⇐ ⇓ A1* expands less nodes than A2* ⇓ The more informed, the better

26. More informed algorithms • Compromise between: • Calculation time in h’ • h’1(n) could require more calculation time than h’2(n) ! • Re-expansions • A1* may re-expand more nodes than A2* ! • Not if A1* is consistent • Not if trees (instead of graphs) are considered • Loss of admissibility • Working with non admissible heuristic functions can be interesting to gain speed.

27. 8-puzzle (operation’s cost = 1) h’0(n)=0 breadth first, A0* admissible, many generations and expansions h’1(n)= # wrongly placed tiles A1* admissible, A1* more informed than A0* 1 3 2 1 2 3 8 4 8 4 7 6 5 7 6 5

28. h’2(n)=Σi∈[1,8]di di ≡ distance between current and final position of tile i distance ≡ minimum number of moves between two positions A2* admissible. Statistically, A2* is better than A1*, but it can’t be formally said that it is more informed.

29. h’3(n)=Σi∈[1,8]di + 3 S(n) S(n)=Σi∈[1,8]si si = 0 if tile i is not in the center and its successor is correct 1 if tile i is in the center 2 if tile i is not in the center and its successor is incorrect h’3(n)=1+3(2+1)=10 h(n)=1 A3* cannot be compared to A1* o A2*, but it is faster (even if the h’ to be calculated requires more time). } 1 3 A3* no admissible 8 2 4 7 6 5

30. Memory bounded search • The A* algorithm solves problems in which it is necessary to find the best solution. • Its cost in space and time, in average and if the heuristic function is adequate, is better than that of blind-search algorithms. • There exist problems in which the size of the search space does not allow a solution with A*. • There exist algorithms which allow to solve problems limiting the memory used: • Iterative deepening A* (IDA*) • Recursive best-first • Simplified memory-bounded A* (SMA*)

31. g + h’ Iterative deepening A* (IDA*) • Iterative deepening A* is similar to the iterative deepening (ID) technique. • In ID the limit is given by a maximum depth. • In IDA* the limit is given by a maximum value of the f-cost. • Important: The search is a standard depth-first; the f-cost is used only to limit the expansion. • Starting limit = f (initial) (deepening expansion limit) 0+2 1+1 1+2 2+1 2+1 3+1 3+1 4+1 4+0 goal 5+0 goal

32. g + h’ Iterative deepening A* (IDA*) • Iterative deepening A* is similar to the iterative deepening (ID) technique. • In ID the limit is given by a maximum depth. • In IDA* the limit is given by a maximum value of the f-cost. • Important: The search is a standard depth-first; the f-cost is used only to limit the expansion. • Starting limit = f (initial) (deepening expansion limit) 0+2 (1,3,8) 1+1 (2,4,9) (5,10) (11) (6,12) (7,13) (14) (15) 1+2 2+1 2+1 3+1 3+1 4+1 4+0 goal 5+0 goal

33. IDA* algorithm

34. IDA* algorithm Algorithm IDA* depth=f(Initial_state) While not is_final?(Current) do Open_states.insert(Initial_state) Current= Open_states.first() While not is_final?(Current) and not Open_states.empty?() do Open_states.delete_first() Closed_states.insert(Current) Successors= generate_successors(Current, depth) Successors= process_repeated(Successors, Closed_states, Open_states) Open_states.insert(Successors) Current= Open_states.first() eWhile depth=depth+1 Open_states.initialize() eWhile eAlgorithm • The function generate_successors only generate those with an f-cost less or equal to the cutoff limit of the iteration. • The OPEN structure is a stack (depth-first search). • If repeated nodes are processed there is no space saving. • Only the current path (tree branch) is saved in memory.

35. Other memory-bounded algorithms • IDA*’s re-expansions can represent a high temporal cost. • There are algorithms which, in general, expand less nodes. • Their functioning is based on eliminating less promising nodes and saving information which allows to re-expand them (in necessary). • Examples: • Recursive best-first • Memory bound A* (MA*) 35

36. Recursive best-first • It is a recursive implementation of best-first, with lineal spatial cost. • It forgets a branch when its cost is more than the best alternative. • The cost of the forgotten branch is stored in the parent node as its new cost. • The forgotten branch is re-expanded if its cost becomes the best one again. 36

37. Recursive best-first: example

38. Recursive best-first: example

39. Recursive best-first • In general, it expands less nodes than IDA*. • Not being able to control repeated states, its cost in time can be high if there are loops. • Sometimes, memory restrictions can be relaxed. 39

40. Memory bound A* (MA*) • It imposes a memory limit: number of nodes which can be stored. • A* is used for exploration and nodes are stored while there is memory space. • When there is no more space, the worst nodes are eliminated, keeping the best cost of forgotten descendants. • The forgotten branches are re-expanded if their cost becomes the best one again. • MA* is complete if the solution path fits in memory. 40

41. Local search algorithms and optimization problems • In local search (LS), from a (generally random) initial configuration, via iterative improvements (by operators application), a state is reached from which no better state can be attained. • LS algorithms (or meta-heuristics or local optimization methods) are prone to find local optima, which are not the best possible solution. The global optimum is generally impossible to be reached in limited time. • In LS, there is a function to evaluate the quality of the states, but this is not necessarily related to a cost.

42. Local search algorithms

43. Local search algorithms • These algorithms do not systematically explore all the state space. • The heuristic (or evaluation) function is used to reduce the search space (not considering states which are not worth being explored). • Algorithms do not usually keep track of the path traveled. The memory cost is minimal. • This total lack of memory can be a problem (i.e., cycles).

44. Hill-climbing search • Standard hill-climbing search algorithm • It is a simple loop which search for and select any operation that improves the current state. • Steepest-ascent hill climbing or gradient search • The best move (not just any one) that improves the current state is selected.

45. Steepest-ascent hill-climbing algorithm

46. Steepest-ascent hill-climbing algorithm Algorithm Hill Climbing Current= Initial_state end= false While¬end do Children= generate_successors(Current) Children= order_and_eliminate_worse_ones(Children, Current) if ¬empty?(Children) then Current = Select_best(Children) else end=true eWhile eAlgorithm

47. Hill climbing • Children are considered only if their evaluation function is better than the one of the parent (reduction of the search space). • A stack could be used to save children which are better than the parent, to be able to backtrack; but in general the cost of this is prohibitive. • The characteristics of the heuristic function determine the success and the rapidity of the search. • It is possible that the algorithm does not find the best solution: • Local optima: no successor has a better evaluation • Plateaux: all successors has the same evaluation

48. Simulated annealing • It is a stochastic hill-climbing algorithm (stochastic local search, SLS): • A successor is selected among all possible successors according to a probability distribution. • The successor can be worse than the current state. • Random steps are taken in the state space. • It is inspired by the physical process of controlled cooling (crystallization, metal annealing): • A metal is heated up to a high temperature and then is progressively cooled in a controlled way. • If the cooling is adequate, the minimum-energy structure (a global minimum) is obtained.

49. Simulated annealing • Aim: to avoid local optima, which represent a problem in hill climbing.

50. Simulated annealing • Solution: to take, occasionally, steps in a different direction from the one in which the increase (or decrease) of energy is maximum.