CSM6120 Introduction to Intelligent Systems

1. CSM6120Introduction to Intelligent Systems Informed search

2. Quick review • Problem definition • Initial state, goal state, state space, actions, goal function, path cost function • Factors • Branching factor, depth of shallowest solution, maximum depth of any path in search state, complexity, etc. • Uninformed techniques • BFS, DFS, Depth-limited, UCS, IDS

3. Informed search • What we’ll look at: • Heuristics • Hill-climbing • Best-first search • Greedy search • A* search

4. Heuristics • A heuristic is a rule or principle used to guide a search • It provides a way of giving additional knowledge of the problem to the search algorithm • Must provide a reasonably reliable estimate of how far a state is from a goal, or the cost of reaching the goal via that state • A heuristic evaluation function is a way of calculating or estimating such distances/cost

5. Heuristics and algorithms • A correct algorithm will find you the best solution given good data and enough time • It is precisely specified • A heuristic gives you a workable solution in a reasonable time • It gives a guided or directed solution

6. Evaluation function • There are an infinite number of possible heuristics • Criteria is that it returns an assessment of the point in the search • If an evaluation function is accurate, it will lead directly to the goal • More realistically, this usually ends up as “seemingly-best-search” • Traditionally, the lowest value after evaluation is chosen as we usually want the lowest cost or nearest

7. Heuristic evaluation functions • Estimate of expected utility value from a current position • E.g. value for pieces left in chess • Way of judging the value of a position • Humans have to do this as we do not evaluate all possible alternatives • These heuristics usually come from years of human experience • Performance of a game playing program is very dependent on the quality of the function

8. Heuristics?

9. Heuristics?

10. Heuristic evaluation functions • Must agree with the ordering a utility function would give at terminal states (leaf nodes) • Computation must not take long • For non-terminal states, the evaluation function must strongly correlate with the actual chance of ‘winning’ • The values can be learned using machine learning techniques

11. Heuristics for the 8-puzzle • Number of tiles out of place (h1) • Manhattan distance (h2) • Sum of the distance of each tile from its goal position • Tiles can only move up or down  city blocks

12. The 8-puzzle • Using a heuristic evaluation function: • h2(n) = sum of the distance each tile is from its goal position

13. Goal state Current state Current state h1=5 h2=1+1+1+2+2=7 h1=1 h2=1

14. Search algorithms • Hill climbing • Best-first search • Greedy best-first search • A*

15. Iterative improvement • Consider all states laid out on the surface of a landscape • The height at any point corresponds to the result of the evaluation function

16. Iterative improvement • Paths typically not retained - very little memory needed • Move around the landscape trying to find the lowest valleys - optimal solutions (or the highest peaks if trying to maximise) • Useful for hard, practical problems where the state description itself holds all the information needed for a solution • Find reasonable solutions in a large or infinite state space

17. Hill-climbing (greedy local) • Start with current-state = initial-state • Until current-state = goal-state OR there is no change in current-state do: • a) Get the children of current-state and apply evaluation function to each child • b) If one of the children has a better score, then set current-state to the child with the best score • Loop that moves in the direction of decreasing (increasing) value • Terminates when a “dip” (or “peak”) is reached • If more than one best direction, the algorithm can choose at random

18. Hill-climbing (gradient ascent)

19. Hill-climbing drawbacks • Local minima (maxima) • Local, rather than global minima (maxima) • Plateau • Area of state space where the evaluation function is essentially flat • The search will conduct a random walk • Ridges • Causes problems when states along the ridge are not directly connected - the only choice at each point on the ridge requires uphill (downhill) movement

20. Best-first search • Like hill climbing, but eventually tries all paths as it uses list of nodes yet to be explored • Start with priority-queue = initial-state • While priority-queue not empty do: • a) Remove best node from the priority-queue • b) If it is the goal node, return success. Otherwise find its successors • c) Apply evaluation function to successors and add to priority-queue

21. Best-first example

22. Best-first search • Different best-first strategies have different evaluation functions • Some use heuristics only, others also use cost functions: f(n) = g(n) + h(n) • For Greedy and A*, our heuristic is: • Heuristic function h(n) = estimated cost of the cheapest path from node n to a goal node • For now, we will introduce the constraint that if n is a goal node, then h(n) = 0

23. Greedy best-first search • Greedy BFS tries to expand the node that is ‘closest’ to the goal assuming it will lead to a solution quickly • f(n) = h(n) • aka “greedy search” • Differs from hill-climbing – allows backtracking • Implementation • Expand the “most desirable” node into the frontier queue • Sort the queue in decreasing order

24. Route planning: heuristic??

25. Route planning - GBFS

26. Greedy best-first search

27. Route planning

28. Greedy best-first search

29. Greedy best-first search This happens to be the same search path that hill-climbing would produce, as there’s no backtracking involved (a solution is found by expanding the first choice node only, each time).

30. Greedy best-first search • Complete • No, GBFS can get stuck in loops (e.g. bouncing back and forth between cities) • Time complexity • O(bm) but a good heuristic can have dramatic improvement • Space complexity • O(bm) – keeps all the nodes in memory • Optimal • No! (A – S – F – B = 450, shorter journey is possible)

31. Practical 2 • Implement greedy best-first search for pathfinding • Look at code for AStarPathFinder.java

32. A* search • A* (A star) is the most widely known form of Best-First search • It evaluates nodes by combining g(n) and h(n) • f(n) = g(n) + h(n) • where • g(n) = cost so far to reach n • h(n) = estimated cost to goal from n • f(n) = estimated total cost of path through n start g(n) n h(n) goal

33. A* search • When h(n) = h*(n) (h*(n) is actual cost to goal) • Only nodes in the correct path are expanded • Optimal solution is found • When h(n) < h*(n) • Additional nodes are expanded • Optimal solution is found • When h(n) > h*(n) • Optimal solution can be overlooked

34. Route planning - A*

35. A* search

36. A* search

37. A* search

38. A* search

39. A* search

40. A* search • Complete and optimal if h(n) does not overestimate the true cost of a solution through n • Time complexity • Exponential in [relative error of h x length of solution] • The better the heuristic, the better the time • Best case h is perfect, O(d) • Worst case h = 0, O(bd) same as BFS, UCS • Space complexity • Keeps all nodes in memory and save in case of repetition • This is O(bd) or worse • A* usually runs out of space before it runs out of time

41. A* exercise NodeCoordinatesSL Distance to K A (5,9) 8.0 B (3,8) 7.3 C (8,8) 7.6 D (5,7) 6.0 E (7,6) 5.4 F (4,5) 4.1 G (6,5) 4.1 H (3,3) 2.8 I (5,3) 2.0 J (7,2) 2.2 K (5,1) 0.0

42. Solution to A* exercise

43. GBFS exercise NodeCoordinatesDistance A (5,9) 8.0 B (3,8) 7.3 C (8,8) 7.6 D (5,7) 6.0 E (7,6) 5.4 F (4,5) 4.1 G (6,5) 4.1 H (3,3) 2.8 I (5,3) 2.0 J (7,2) 2.2 K (5,1) 0.0

44. Solution

45. To think about... f(n) = g(n) + h(n) • What algorithm does A* emulate if we set • h(n) = -g(n) - depth(n) • h(n) = 0 • Can you make A* behave like Breadth-First Search?

46. A* search - Mario • http://aigamedev.com/open/interviews/mario-ai/ • Control of Super Mario by an A* search • Source code available • Various videos and explanations • Written in Java

47. Admissible heuristics • A heuristic h(n) is admissible if for every node n, h(n) ≤ h*(n), where h*(n) is the true cost to reach the goal state from n • An admissible heuristic never overestimates the cost to reach the goal • Example: hSLD(n) (never overestimates the actual road distance) • Theorem: If h(n) is admissible, A* is optimal (for tree-search)

48. Optimality of A* (proof) • Suppose some suboptimal goal G2has been generated and is in the frontier. Let n be an unexpanded node in the frontier such that n is on a shortest path to an optimal goal G • f(G2) = g(G2) since h(G2) = 0 (true for any goal state) • g(G2) > g(G) since G2 is suboptimal • f(G) = g(G) since h(G) = 0 • f(G2) > f(G) from above

49. Optimality of A* (proof) • f(G2) > f(G) • h(n) ≤ h*(n) since h is admissible • g(n) + h(n) ≤ g(n) + h*(n) • f(n) ≤ f(G) (f(G) = g(G) = g(n) + h*(n)) • Hence f(G2) > f(n), and A* will never select G2 for expansion

50. Heuristic functions • Admissible heuristic example: for the 8-puzzleh1(n) = number of misplaced tilesh2(n) = total Manhattan distance i.e. no of squares from desired location of each tileh1(S) = ??h2(S) = ??