Notes for CS3310 Artificial Intelligence Part 8: Heuristic search

1. Notes for CS3310Artificial IntelligencePart 8: Heuristic search Prof. Neil C. Rowe Naval Postgraduate School Version of January 2006

2. Search • Search = reasoning about "states". • A state is a set of facts true at some instant of time. That is, a state is a snapshot of facts. (Or to save space, just the relevant facts in some problem.) • Search and states permit reasoning about time. • In artificial intelligence, states are discrete, and transitions between states ("branches") are discrete (all at once) too. • Search among states permits hypothetical reasoning about effects of actions. Each possible action can result in a different state. • A state can be represented by a linked list. Possible state transitions can be represented by a directed graph, a "search graph". • Search is the oldest area of artificial intelligence.

3. More search terms • State: a set of facts (whose predicate names are usually past participles, e.g. "opened", and abstract nouns, e.g. "location"). • Successor state: an immediate next state. • Operator: a label on a state transition (usually a verb, e.g. "open") • Starting state: the first state of a search. • Goal states: states in which objectives have been achieved, so search can stop. • Level: the number of state transitions from the starting state to a given state. • Preconditions of an operator: facts that must occur in a state before you can use the operator to it. • Postconditions of an operator: facts that become true after you use the operator from a state.

4. Search examples • Intercity route planning: given a road map, find the best route between two cities for a car. • City route planning: Given intersections A and B, find a route between them for a car. • Car repair: Given part to repair, access it, fix it, and put the car back together. • Smarter forward chaining: Forward chaining but making an intelligent decision at each step about which fact to pursue next. (The goal is to infer interesting facts sooner and save time.) • Scheduling classes: At a university, assign classes to times so every student can take the classes they want. • Mission planning: Given a model of targets and costs of attacking them, and a model of enemy responses, figure the best sequence of attacking actions.

5. Search example I-80 (20) The state is in brackets; air-mileage to San Francisco in curly brackets; mileages in parentheses; road names are the operators. Prefer westmost branch. Compute order of visit and solution path: Use abbreviations: M=Monterey, SC=Santa Cruz, G=Gilroy, SJ=San Jose, SF=San Francisco, Oak=Oakland. [location(Oakland)] {15} [location(San Francisco)] {0} I-280 (50) Cal-17 (50) Cal-17 (30) Cal-1 (85) [location(San Jose)] {45} US-101 (35) [location(Santa Cruz)] {65} [location(Gilroy)] {74} [location(Monterey)] {100} Cal-1 (45) US-101 (35)

6. Work page for comparison of search algorithms

7. A search graph for car repair Complete the search graph to the goal state [ok(battery), in(battery,car), cabled(battery), bolted(battery)]. Possible operators: "attach cables", "attach bolts", and "replace battery". Note: not(cabled(battery)) is a precondition of the action "remove battery"; ok(battery) is a postcondition of the action "replace battery"; facts not mentioned are assumed false in a state. [dead(battery), in(battery,car), bolted(battery), cabled(battery)] remove cables remove bolts [dead(battery), in(battery,car), cabled(battery)] [dead(battery), in(battery,car), bolted(battery)] remove cables [dead(battery), in(battery,car)] remove bolts

8. Work page for car battery problem

9. Classic search methods • These differ in the order in which states are "visited". "Visit” means "find the successors of." • Depth-first: visit an unvisited successor of current state, else back up and try a new successor of previous state. • Breadth-first: visit next-found unvisited state at same level, else the first-found state at next level. • Hill climbing: visit the unvisited successor of current state which has lowest evaluation-function value, else back up. • Best-first: visit the known unvisited state with lowest evaluation-function value, which can be anywhere in search graph. • Branch-and-bound: like best-first but use cost from start to state. • A*: like best-first but use sum of evaluation-function value and cost from start to state.

10. Notes on the search methods • Search stops when you visit a goal state (that is, when you try to find its successors). • Heuristics can give criteria for choosing the next state with all the search methods (and break ties in best-first and A*). • Never visit the same state twice (or add a duplicate state to the agenda) -- but for A*, consider new routes to the same state. • Depth-first can be implemented with a stack; breadth-first with a queue; and the other methods with a sorted "agenda" of states known but not yet visited. • When you find successors of a state, set the "backpointer" for each successor to point to the state. When done, follow backpointers from the goal state to the start state to get the solution path. • Agendas should store for each state its description (list of facts true for it), its backpointer, and any numbers associated with the state. • With A* search, if the evaluation function is a lower bound on the subsequent cost, the first path found to a state is guaranteed to be the best path to that state.

11. Heuristics = Any piece of nonnumeric advice for choosing among possible branches in a search. Heuristics are a form of meta-rules. Examples: For city route planning, prefer not to turn twice in succession. For car repair, do work from top of the engine before work from below the engine. For buying decisions, don't buy anything advertised on television. In mission planning, withdraw from an engagement if you do not have at least 2-to-1 superiority in assets.

12. Search terms that are easy to confuse • evaluation function: a "distance-to-goal" measure, a number. A function of the state. Makes search more intelligent, if you pick direction to search with lowest evaluation function. • heuristic: a non-numeric way for choosing a direction for search at a state. • cost function: a "cost-accumulated" measure, a number. A function of the states in a path. Note: Evaluation function concerns future; cost function concerns past.

13. An example route-planning program, not so good Microsoft Automap Road Atlas Route Planning Demo The quickest route from Monterey, California, to San Francisco, California (going the speed limit), is as follows: Total Time: 2 hours 31 minutes Total Distance: 121 miles Time Road For Dir Towards 00:00 DEPART Monterey (California) 2 miles E on the Fremont St 00:04 Go onto S1 17 miles E Santa Cruz 00:23 At Castroville stay on the S1 14 miles N Santa Cruz 00:39 At Freedom stay on the S1 15 miles NW Santa Cruz 00:55 At Santa Cruz turn S9 23 miles N Saratoga right onto 01:23 Turn left onto S35 14 miles N Woodside 01:40 Turn right onto S84 5 miles E Redwood City 01:46 At Woodside stay on the S84 1/2 mile NE Redwood City 01:47 Turn left onto I280 16 miles W San Mateo 02:07 At Burlingame stay on the I280 1 mile N Brisbane 02:09 Turn off onto I380 1 mile E San Bruno 02:10 At San Bruno stay on the I380 1 mile E 02:12 Turn off onto U101 3 miles N South San Francisco 02:17 At South San Francisco U101 3 miles NE Brisbane stay on the 02:22 At Brisbane stay on the U101 3 miles N 02:26 Turn off onto I80 3 miles E San Francisco 02:31 ARRIVE San Francisco(California) If you are impressed with this demo, you'll be wowed by our full product, Microsoft Automap Road Atlas 4.0. We invite you to join our mailing list, read more information about Automap Road Atlas, or give us feedback on this demo.

14. More about evaluation functions What if states aren’t locations? Then add "amount-of-work-necessary" numbers for certain facts to get the evaluation-function value. For instance for the car repair, start at 0 and then: Add 10 if a dead(battery) fact is in the state; Add 5 if no bolted(battery) fact is in the state; Add 3 if no cabled(battery) fact is in the state. So for instance: [dead(battery), in(battery,car), bolted(battery), cabled(battery)] has evaluation 10 [dead(battery), in(battery,car), cabled(battery)] has evaluation 15 [dead(battery), in(battery,car)] has evaluation 18 [ok(battery), in(battery,car)] has evaluation 8 [ok(battery), in(battery(car), cabled(battery)] has evaluation 5 [ok(battery), in(battery(car), bolted(battery), cabled(battery)] has 0

15. Problems with evaluation functions Evaluation functions don't always work well. Some classic problems: • "pond" problem: you may find a local (instead of global) minimum • "valley" problem: most directions may be worse, and it may be hard to find the few directions that are better • "plateau" problem: you may have lots of ties in the evaluation function value • "pit" problem: evaluation function may not be useful unless you're extremely close to the goal In these case a set of heuristics may be better.

16. Another kind of cost function: The probability of an interpretation of a situation Overall probability can be product of near-independent factors. So to find the best interpretation, minimize its logarithm: Do this by an A* search where branches cost: This is important in language understanding. For instance, interpret "navy fighter wing" with probabilities: of each word sense of "navy” ; of each word sense of "fighter”; of each word sense of "wing”; that "navy" is a kind of "fighter”; that "navy" is part of "fighter”; that "fighter" is part of "navy”; that "navy" is owner of "fighter”; that "fighter" is a kind of "wing”; that "wing" is a part of "fighter” that "wing" is a kind of "fighter”. In a goal state each word has a sense and all senses are linked by relationships.

17. 167 research vulnerabilities overflow buffer become admin close port 5 ping scan open port 180 96 0 97 0.9 0.8 22 1 2 3 4 6 99 0.1 0.2 ping 107 33 research vulnerabilities 101 100 close port 35 become admin close ftp 0.83 10 decompress rootkit ftp 65 9 100 75 login download secureport 31 11 download rootkit 61 0.2 70 74 close ftp 0.17 25 ftp 8 7 62 install rootkit 26 0.4 decompress secureport download secureport download secureport download rootkit 85 24 86 0.8 decompress rootkit 0.8 74 55 decompress rootkit 122 29 0.2 48 32 Numbers in boxes are state numbers. Integers outside boxes are duration from goal state. Decimals are transition probabilities in 10 runs. 50 0.8 27 30 decompress secureport 0.75 0.2 0.25 install rootkit install rootkit test rootkit 0.67 0.9 0.17 34 13 21 22 23 28 download secureport close ftp install secureport decompress secureport 11 47 39 36 30 40 0.33 ftp test rootkit test rootkit test rootkit 0.83 test rootkit test rootkit 14 15 16 17 18 19 36 24 1 0 25 download secureport close ftp install secureport logout decompress secureport Stochastic state graph for rootkit-installation example 0.6 51 55 0.9 0.1 0.1 20

18. Definitions needed to solve a search problem (1) successor: Defines all possible state transitions (with successor(State,Newstate) in Prolog; in Java, successor(State) which returns the array Newstates). (2) goalreached: Defines all possible goal states. (3) eval (for best-first and A*): Returns the evaluation function value for any state. (4) piece_cost (for A*): Returns the cost between any pair of states. (5) To start search, call depthsearch, breadthsearch, bestsearch, or astarsearch with argument the starting state; it should return the solution path.

19. Example Prolog definition of a search problem: finding a route to San Francisco successor(R1,R2) :- successor2(R1,R2). successor(R1,R2) :- successor2(R2,R1). successor2(monterey,gilroy). successor2(monterey,santa_cruz). successor2(gilroy,san_jose). successor2(santa_cruz,san_jose). successor2(santa_cruz,san_francisco). successor2(san_jose,oakland). successor2(san_jose,san_francisco). successor2(oakland,san_francisco). goalreached(san_francisco). eval(monterey,100). eval(gilroy,74). eval(santa_cruz,65). eval(san_jose,45). eval(oakland,15). eval(san_francisco,0).

20. Example Prolog search definition, cont. /* This totals up the pair_cost2 numbers along a path */ piece_cost(X,Y,C) :- piece_cost2(X,Y,C). piece_cost(X,Y,C) :- piece_cost2(Y,X,C). piece_cost2(monterey,gilroy,35). piece_cost2(monterey,santa_cruz,45). piece_cost2(gilroy,san_jose,35). piece_cost2(santa_cruz,san_jose,30). piece_cost2(santa_cruz,san_francisco,85). piece_cost2(san_jose,oakland,50). piece_cost2(san_jose,san_francisco,50). piece_cost2(oakland,san_francisco,20). Then give the starting state, and search returns to sequence of states to the goal, like: ?- depthsearch(monterey,A). A=[san_francisco,san_jose,gilroy,monterey]

21. Another example: the robot housekeeper problem Define behavior of a housekeeping robot. 1. In the starting state: • The robot is at the trash chute (use predicate at); • Offices 1 and 2 need dusting (use dusty); • Office 1 has trash in its trash basket, but Office 2 doesn't (use trashy); • The carpet in Office 1 does not need vacuuming, while the carpet in Office 2 does (use vacuumable). Write the starting state using predicate expressions. 2. A goal state is one in which all these are true: Every office is dusted; every carpet is vacuumed; every trash basket is emptied; the robot is not holding any basket (use holdingbasketfrom); and the robot is at office 1. Give the conditions defining a goal state in logical terms.

22. The cost function for the robot housekeeper 3. Assume these energy costs: • 10 units to vacuum a single room (the robot has a built-in vacuum); • 6 units to dust a room (the robot has a built-in duster); • 3 units to pick up a trash basket (the robot can hold several at once); • 1 unit to put down a trash basket; • 3 units to travel between offices; • 8 units to travel between an office and the trash chute; • 5 units to dispose of the contents of a trash basket down the trash chute. Assume the last action taken by the robot is the argument to a lastact fact in each state. Define a piece_cost function.

23. The eval function for the robot housekeeper 4. For A* search, we prefer an evaluation function that is a lower bound on the subsequent cost. So we'll add up numbers ("piece_eval"s) for each expression in the state. The default number will be zero, but "dusty(X)" for instance will have a piece_eval of 6. Compute the other piece_evals.

24. Successor definition for the robot housekeeper 5. The successor definition must follow these requirements: a. The robot can vacuum the floors, dust, and empty the trash baskets. b. Vacuuming the floor generates dust that goes into the trash basket of the room it is in. c. Emptying the trash baskets in each room requires a trip to the trash chute. d. It doesn't make sense to vacuum or dust if the room isn't vacuumable or dusty respectively. e. Dusting puts dust on the floor, requiring vacuuming. f. It doesn't make sense to vacuum if a room is dusty. g. It doesn't make sense to pick up a trash basket if a room is vacuumable or dusty. h. A lastact fact in each state should hold what the last action was.

25. Write "successor" definition for the "putdown", "dust", and "go" actions Example: For "dispose(Basket)” (meaning to empty the trash basket whose name is Basket). Preconditions: you are at the chute, have a basket, and the basket contains trash. Postconditions: Delete the fact the basket has trash, delete the old “lastact” fact, and add a new “lastact” fact with argument “dispose(Basket)”.

26. successor(S,NS):-member(at(chute),S), member(fullbasket(X),S), member(holdingbasketfrom(X),S), delete(S, fullbasket(X),S2), delete(S2, lastact(A),S3), NS=[lastact(dispose(X)) | S3]. successor(S,NS) :- member(at(X),S), \+ member(dusty(X),S), member(vacuumable(X),S), delete(S, vacuumable(X), S2), delete(S2, fullbasket(X), S3), delete(S3, lastact(A), S4), NS=[lastact(vacuum(X)), fullbasket(X) | S4]. successor(S,NS) :- member(at(X),S), \+ member(dusty(X),S), \+ member(holdingbasketfrom(X),S), \+ member(vacuumable(X),S), member(fullbasket(X),S), delete(S, lastact(A), S2), NS=[lastact(pickup(X)), holdingbasketfrom(X) |S2]. successor(S,NS) :- member(at(X),S), member(dusty(X),S), delete(S, dusty(X), S2), delete(S2, vacuumable(X), S3), delete(S3, lastact(A), S4), NS=[lastact(dust(X)), vacuumable(X) | S4]. successor(S,NS) :- member(at(X),S), member(holdingbasketfrom(X),S), delete(S, holdingbasketfrom(X), S2), delete(S2, lastact(A), S3), NS=[lastact(putdown(X)) | S3]. successor(S,NS) :- member(at(X),S), delete(S, at(X), S2), places(PL), member(Y,PL),\+ X=Y, delete(S2, lastact(A), S3), NS=[lastact(go(X,Y)), at(Y) | S3]. Successor definitions in Prolog

27. Eval function in Prolog eval([],0). eval([X|L],N) :- piece_eval(X,N2), eval(L,N3), N is N2+N3. piece_eval(dusty(X),6). piece_eval(vacuumable(X),10). piece_eval(fullbasket(X),8). piece_eval(holdingbasketfrom(X),1). piece_eval(P,0)

28. Cost function in Prolog cost([_],0). cost([S1,S2|L],C) :- S1=[FS1|_], S2=[FS2|_], piece_cost(FS1,FS2,C2), cost([S2|L],C3), C is C2+C3, !. piece_cost(lastact(vacuum(X)),_,10). piece_cost(lastact(dust(X)),_,6). piece_cost(lastact(pickup(X)),_,3). piece_cost(lastact(dispose(X)),_,5). piece_cost(lastact(putdown(X)),_,1). piece_cost(lastact(go(chute,Y)),_,8). piece_cost(lastact(go(X,chute)),_,8). piece_cost(lastact(go(X,Y)),_,3). piece_cost(lastact(none),_,0).

29. Statistics on the housekeeping problem These programs all eliminate states whose fact lists are permutations of another. "Cells" means the number of cons cells created. Breadth-first search ran extremely slowly. All runs were on SunOS machine ai9 in 1990.

30. The depth-first search algorithm • Initialize S to current state; initialize stack to hold starting state. • Until S is a goal state: • Put (“push”) S onto the stack; • Find the successors of S; • Set S to its best successor (as per heuristics) that is not in the stack; • If no such successor, remove (“pop”) the last state P from stack and go to a different successor of P. • The solution path is the stack items when goal is found.

31. The breadth-first search algorithm • Erase the oldstate array; initialize queue to the starting state with a null backpointer. • Until the first state on the queue is a goal state: • Remove the first item on the queue and call it S. • If S is a goal state, stop. • Otherwise, add successors of S not already on the agenda or in the oldstate array to the end of the queue, using heuristics to determine the order you add them; give each a backpointer to S. • Add S to the oldstate array. • The solution path is found by creating a list of the states encountered in following the backpointers from the goal state (using the oldstate array), then reversing the list.

32. The A* search algorithm • Initialize agenda to just the starting state with null backpointer and starting-state evaluation-function value. • Loop until you pick a goal state: • Remove the first (best) item on the agenda and call it S. • If S is a goal state, stop. • Otherwise, insert successors of S into the agenda if they are not already in the agenda or in the oldstate array, maintaining sorted order of total evaluation. Give each successor state T a backpointer to S and a total evaluation U(T) = U(S) + C(S,T) + E(T) - E(S) where C is the cost from S to T and E is the evaluation function. If T is already on the agenda and this U is better, replace the state on the agenda, else ignore. • Add S to the oldstate array. • Create the solution path by following backpointers from the goal state, then reversing the list.