GPS Route Planning and AI Search Algorithms

1. Do you drive? Have you thought about how the route plan is created for you in the GPS system? • How would you implement a cross-and-nought computer program? AI Search Algorithms – Tree Searches

2. Artificial Intelligence Search Algorithms Dr Rong Qu School of Computer Science University of Nottingham Nottingham, NG8 1BB, UK rxq@cs.nott.ac.uk Course Introduction & Tree Search

3. Problem Space • Many problems exhibit no detectable regular structure to be exploited, they appear “chaotic”, and do not yield to efficient algorithms The concept of search plays an important role in science and engineering In one way, any problem whatsoever can be seen as a search for “the right answer” AI Search Algorithms – Tree Searches

4. Problem Space AI Search Algorithms – Tree Searches

5. Problem Space • Search space • Set of all possible solutions to a problem • Search algorithms • Take a problem as input • Return a solution to the problem AI Search Algorithms – Tree Searches

6. Problem Space • Search algorithms • Exact (exhaustive) tree search • Uninformed search algorithms • Depth First, Breadth First • Informed search algorithms • A*, Minimax • Meta-heuristic search • Modern AI search algorithms • Genetic algorithm, Simulated Annealing AI Search Algorithms – Tree Searches

7. References • Artificial intelligence: a guide to intelligent systems. Addison-Wesley, 2002. Negnevitsky • Good AI textbook • Easy to read while in depth • Available from library AI Search Algorithms – Tree Searches

8. References • Metaheuristics – From Design to Implementation, Talbi, 2009 Quite recent Seems quite complete, from theory to design, and implementation AI Search Algorithms – Tree Searches

9. References • Artificial Intelligence – A Modern Approach (AIMA) (Russell/Norvig), 1995 & 2003 “Artificial Intelligence (AI) is a big field and this is a big book” (Preface to AIMA) Most comprehensive textbook in AI Web site: http://aima.cs.berkeley.edu/ Textbook in many courses Better to be used as reference book (especially for tree search) You don’t have to learn and read the whole book AI Search Algorithms – Tree Searches

10. Other Teaching Resources • Introduction to AI http://www.cs.nott.ac.uk/~rxq/g51iai.htm • Uniform cost search • Depth limited search • Iterative deepening search • alpha-beta pruning: game playing • Other basics of AI incl. tree search • AI Methods http://www.cs.nott.ac.uk/~rxq/g52aim.htm • Search algorithms AI Search Algorithms – Tree Searches

11. Brief intro to search space AI Search Algorithms – Tree Searches

12. Problem Space • Often we can't simply write down and solve the equations for a problem • Exhaustive search of large state spaces appears to be the only viable approach How? AI Search Algorithms – Tree Searches

13. A C D B E The Travelling Salesman Problem • TSP: • A salesperson has to visit a number of cities • (S)He can start at any city and must finish at that same city • The salesperson must visit each city only once • The cost of a solution is the total distance traveled • Given a set of cities and distances between them • Find the optimal route, i.e. the shortest possible route • The number of possible routes is : AI Search Algorithms – Tree Searches

14. Combinatorial Explosion A 10 city TSP has 181,000 possible solutions A 20 city TSP has 10,000,000,000,000,000 possible solutions A 50 City TSP has 100,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000 possible solutions There are 1,000,000,000,000,000,000,000 litres of water on the planet* *Mchalewicz, Z, Evolutionary Algorithms for Constrained Optimization Problems, CEC 2000 (Tutorial) AI Search Algorithms – Tree Searches

15. Combinatorial Explosion A 50 City TSP has 1.52 * 1064 possible solutions A 10GHz computer might do 109 tours per second Running since start of universe, it would still only have done 1026 tours Not even close to evaluating all tours! One of the major unsolved theoretical problems in Computer Science AI Search Algorithms – Tree Searches

16. Combinatorial Explosion AI Search Algorithms – Tree Searches

17. Tree searches AI Search Algorithms – Tree Searches

18. Search Trees • Does the tree under the following root contain a node “I”? • All you get to see at first is the root • and a guarantee that it is a tree • The rest is up to you to discover during the process of search F AI Search Algorithms – Tree Searches

19. F B J C H G E I A D Search Trees • A tree is a graph that • is connected but becomes disconnected by removing any edge (branch) • has precisely one path between any two nodes • Unique path • makes them much easier to search • so we will start with search on trees AI Search Algorithms – Tree Searches

20. F B J C H G E I A D Search Trees • Depth of a node • Depth of a tree • Examples: TSP vs. game • Tree size • Branching factor b = 2 (binary tree) • Depth d Exponentially - Combinatorial explosion AI Search Algorithms – Tree Searches

21. Search Trees • Heart of search techniques • Nodes: states of problem • Root node: initial state of problem • Branches: moves by operator • Branching factor: number of neighbourhoods AI Search Algorithms – Tree Searches

22. Search Trees – Example I AI Search Algorithms – Tree Searches

23. Search Trees – Example II • 1st level: 1 root node (empty board) • 2nd level: 8 nodes • 3rd level: 6 nodes for each of the node on the 2nd level (?) • … AI Search Algorithms – Tree Searches

24. Implementing a Search • State • The state in the state space to which this node corresponds • Parent-Node • the node that generated the current node • Operator • the operator that was applied to generate this node • Path-Cost • the path cost from the initial state to this node AI Search Algorithms – Tree Searches

25. Breadth First Search Blind searches AI Search Algorithms – Tree Searches

26. Breadth First Search - Method • Expand Root Node First • Expand all nodes at level 1 before expanding level 2 OR • Expand all nodes at level d before expanding nodes at level d+1 • Queuing function • Adds nodes to the end of the queue AI Search Algorithms – Tree Searches

27. C B C E D G D F E A B C D E F D G E Breadth First Search - Implementation A • node = REMOVE-FRONT(nodes) • If GOAL-TEST[problem] applied to STATE(node) succeeds • then return node • nodes = QUEUING-FN(nodes, EXPAND(node, OPERATORS[problem])) AI Search Algorithms – Tree Searches

28. The example node set Initial state A A B C D E F Goal state G H I J K L L M N O P Q R S T U V W X Y Z Press space to see a BFS of the example node set AI Search Algorithms – Tree Searches

29. Node A is removed from the queue. Each revealed node is added to the END of the queue. Press space to continue the search. This node is then expanded to reveal further (unexpanded) nodes. Press space We begin with our initial state: the node labeled A. Press space to continue We then backtrack to expand node C, and the process continues. Press space Node B is expanded then removed from the queue. The revealed nodes are added to the END of the queue. Press space. The search then moves to the first node in the queue. Press space to continue. A A B B C C D D E E F F G G H H I I J J K K L L L L L L M N O P Node L is located and the search returns a solution. Press space to end. Q R S T U Press space to begin the search Press space to continue the search Press space to continue the search Press space to continue the search Press space to continue the search Press space to continue the search Press space to continue the search Press space to continue the search Press space to continue the search Press space to continue the search Size of Queue: 5 Size of Queue: 7 Size of Queue: 0 Size of Queue: 8 Size of Queue: 10 Size of Queue: 0 Size of Queue: 9 Size of Queue: 6 Size of Queue: 1 Queue: Empty Queue: C, D, E, F, G, H Queue: B, C, D, E, F Queue: J, K, L, M, N, O, P, Q, R, S Queue: A Queue: Empty Queue: E, F, G, H, I, J, K, L Queue: D, E, F, G, H, I, J Queue: K, L, M, N, O, P, Q, R, S, T Queue: L, M, N, O, P, Q, R, S, T, U Queue: H, I, J, K, L, M, N, O, P, Q Queue: G, H, I, J, K, L, M, N, O, P Queue: I, J, K, L, M, N, O, P, Q, R Queue: F, G, H, I, J, K, L, M, N Nodes expanded: 2 Nodes expanded: 11 Nodes expanded: 4 Nodes expanded: 6 Nodes expanded: 3 Nodes expanded: 10 Nodes expanded: 8 Nodes expanded: 1 Nodes expanded: 0 Nodes expanded: 9 Nodes expanded: 5 Nodes expanded: 7 Current Action: Expanding Current Action: Expanding Current Action: Expanding Current Action: Expanding Current Action: Backtracking Current Action: Current Action: Backtracking Current Action: Backtracking Current Action: Expanding Current Action: Backtracking Current Action: Backtracking Current Action: Backtracking Current Action: Backtracking Current Action: Expanding Current Action: Expanding Current Action: Backtracking FINISHED SEARCH Current Action: Backtracking Current Action: Expanding Current Action: Expanding Current Action: Backtracking Current Action: Expanding Current level: 1 Current level: 2 Current level: 1 Current level: 2 Current level: 1 Current level: 0 Current level: 0 Current level: 1 Current level: 2 Current level: 2 Current level: 1 Current level: 1 Current level: 1 Current level: 0 Current level: 1 Current level: 2 Current level: 0 Current level: n/a Current level: 0 Current level: 1 Current level: 0 Current level: 2 Current level: 1 Current level: 1 Current level: 1 Current level: 0 Current level: 0 Current level: 1 BREADTH-FIRST SEARCH PATTERN AI Search Algorithms – Tree Searches

30. Evaluating Search Algorithms • Evaluating against four criteria • Optimal • Complete • Time complexity • Space complexity AI Search Algorithms – Tree Searches

31. Evaluating Breadth First Search • Evaluating against four criteria • Complete? • Yes • Optimal? • Yes AI Search Algorithms – Tree Searches

32. Evaluating Breadth First Search • Evaluating against four criteria • Space Complexity • O(bd), i.e. number of leaves • Time Complexity • 1 + b + b2 + b3 + ... + bd-1 i.e. O(bd) • b: the branching factor • d: is the depth of the search tree Note : The space/time complexity could be less as the solution could be found anywhere before the dth level. AI Search Algorithms – Tree Searches

33. Evaluating Breadth First Search Time and memory requirements for breadth-first search, assuming a branching factor of 10, 100 bytes per node and searching 1000 nodes/second AI Search Algorithms – Tree Searches

34. Breadth First Search - Observations • Very systematic • If there is a solution, BFS is guaranteed to find it • If there are several solutions, then BFS • will always find the shallowest goal state first and • if the cost of a solution is a non-decreasing function of the depth then it will always find the cheapest solution AI Search Algorithms – Tree Searches

35. Breadth First Search - Observations • Space is more of a factor to breadth first search than time • Time is still an issue • Who has 35 years to wait for an answer to a level 12 problem (or even 128 days to a level 10 problem) • It could be argued that as technology gets faster then exponential growth will not be a problem • But even if technology is 100 times faster • we would still have to wait 35 years for a level 14 problem and what if we hit a level 15 problem! AI Search Algorithms – Tree Searches

36. Depth First Search Blind searches AI Search Algorithms – Tree Searches

37. Depth First Search - Method • Expand Root Node First • Explore one branch of the tree before exploring another branch • Queuing function • Adds nodes to the front of the queue AI Search Algorithms – Tree Searches

38. Depth First Search - Observations • Space complexity • Only needs to store the path from the root to the leaf node as well as the unexpanded nodes • For a state space with a branching factor of b and a maximum depth of m, DFS requires storage of bm nodes • Time complexity • is bm in the worst case AI Search Algorithms – Tree Searches

39. Depth First Search - Observations • If DFS goes down a infinite branch it will not terminate if it does not find a goal state • If it does find a solution there may be a better solution at a lower level in the tree • Therefore, depth first search is neither complete nor optimal AI Search Algorithms – Tree Searches

40. The example node set Exercise: - Show the queue status of BFS and DFS Initial state A A B C D E F Goal state G H I J K L L M N O P Q R S T U V W X Y Z AI Search Algorithms – Tree Searches

41. Heuristic searches AI Search Algorithms – Tree Searches

42. Blind Search vs. Heuristic Searches • Blind search • Randomly choose where to search in the search tree • When problem get large, not practical any more • Heuristic search • Explore the node which is more likely lead to goal state AI Search Algorithms – Tree Searches

43. Heuristic Searches - Characteristics • Heuristic searches work by deciding which is the next best node to expand • Has some domain knowledge • Use a function to tell us how close the node is to the goal state • Usually more efficient than blind searches • Sometimes called an informed search • There is no guarantee that it is the best node • Heuristic searches estimate the cost to the goal from its current position. It is usual to denote the heuristic evaluation function by h(n) AI Search Algorithms – Tree Searches

44. Heuristic Searches – Example Go to the city which is nearest to the goal city Hsld(n) = straight line distance between n and the goal location AI Search Algorithms – Tree Searches

45. Heuristic Searches - Greedy Search • So named as it takes the biggest “bite” it can out of the problem • That is, it seeks to minimise the estimated cost to the goal by expanding the node estimated to be closest to the goal state • Implementation is achieved by sorting the nodes based on the evaluation function: f(h) = h(n) AI Search Algorithms – Tree Searches

46. Heuristic Searches - Greedy Search • It is only concerned with short term aims • It is possible to get stuck in an infinite loop • It is not optimal • It is not complete Time and space complexity is O(bm); where m is the depth of the search tree AI Search Algorithms – Tree Searches

47. Greedy Search AI Search Algorithms – Tree Searches Performed well, but not optimal

48. outline of graph outline of graph goal goal start start increasing cost increasing cost This region is basically wasted effort Heuristic Searches vs. Blind Searches AI Search Algorithms – Tree Searches

49. outline of graph goal start increasing cost Heuristic Searches vs. Blind Searches • Want to achieve this but stay • complete • optimal • If bias the search “too much” then could miss goals or miss shorter paths AI Search Algorithms – Tree Searches

50. A* Search Heuristic searches AI Search Algorithms – Tree Searches