Lecture 3-Search Algorithms

1. Lecture 3-Search Algorithms Breadth-First Search Algorithm And Depth-First Search Algorithm

2. BFS and DFS • Both are methods to traverse graphs along edges • For finding vertices • People use them to discover some properties of graphs, e.g., cyclic, strongly connected, etc. • They almost have the same ability, but when and where to choose which one. That depends.

3. Breadth-first search(outline) • The single source shortest-paths problem for un-weighted graph • The Breadth-first search algorithm • The running time of BFS algorithm • The correctness proof • Note: We only introduce BFS for undirected graphs. But it also works for directed graphs.

4. d e b c s a Shortest paths Example: 3 simple pathsfromsource s to b: <s, b>, <s, a, b>, <s, e, d, b> of length1, 2, 3, respectively. So the shortest path from s to b is <s, b>. The shortest paths from s to other vertices a, e, d, c are: <s, a>, <s, e>, <s, e, d>, <s, b, c>. There are two shortest paths from s to d.

5. d e b c s a The shortest-paths problem • Distance d[v]: The length of the shortest path from s to v. For example d[c]=2. Define d[s]=0. • The problem: • Input: A graph G = (V, E) and a source vertex sV • Output: A shortest path (if exists) from s to each vertex v V andthe distanced[v]. • Note：If a vertex v is not reachable from s, then d[v]=.

6. What does the BFS do? • It starts from a vertex s, and then searches from all the nearest vertices uniformly. • It computes a breadth-first tree. It first adds s to the tree as the root. Once a new vertex v is found by examining vertex u’s adjacent list, v and (u, v) are added into the tree. u is an ancestor of v and v is a descendant of u if u is on the path from root s to v. • Given a graph G = (V, E), and a specified vertex s, the BFS returns: • The distanced[v] from s to v for each vertex v. • A shortest path from s to vertex v for each vertex v. • BFS actually returns a shortest path tree in which the unique simple path from s to node v is a shortest path from s to v in the original graph. • In addition to the two arrays d[v] and [v], BFS also uses another array color[v], which has three possible values: • WHITE: Refers to “undiscovered” vertices; • GRAY: Refers to “discovered” but not “processed” vertices • BLACK: Refers to “processed” vertices.

7. … The Breadth-First Search • The idea of the BFS: • Expands the frontier between discovered and undiscovered vertices uniformly across the breadth of the frontier. • Visit the vertices as follows: • Visit all vertices at distance 1 • Visit all vertices at distance 2 • Visit all vertices at distance 3 • … … • Initially, s is made GRAY, others are colored WHITE. • When a gray vertex is processed, its color is changed to black, and the color of all white neighbors is changed to gray. • Gray vertices are kept in a queue Q. • Note: Gray vertices form the frontier between Found and Unfound vertices.

8. The Breadth-First Search(more details) • G is given by its adjacency-lists. • Initialization: • First Part: lines 1 – 4 • Second Part: lines 5 - 9 • Main Part: Lines 10 – 18 • Enqueue(Q, v): Add a vertex v to the end of the queue Q • Dequeue(Q): Extract the first vertex in the queue Q

9. b c a s f d e Example of Breadth-First Search • Problem: • Given the undirected graph below and the source vertex s, find the distance d[v] from s to each vertex vV, and the predecessor [v] along a shortest path by the algorithm described earlier.

10. vertex u s a b c d e f color[u] d[u] [u] G W W W W W W 0       NIL NIL NIL NIL NIL NIL NIL Example(Continued) • Initialization Q = <s> (put s into Q (discovered), mark s gray (unprocessed)) b c 0 a s f d e

11. vertex u s a b c d e f color[u] d[u] [u] B W G W W G W 0  1   1  NIL NIL s NIL NIL s NIL Example(continued) b 1 c • While loop, first iteration • Dequeue s from Q. Find Adj[s]=<b, e> • Mark b,e as “G” • Update d[b], d[e], [b], [e] • Put b, e into Q • Mark s as “B” • Q=<b, e> a 0 s f d e 1

12. vertex u s a b c d e f color[u] d[u] [u] B G B G W G G 0 2 1 2  1 2 NIL b s b NIL s b Example(continued) 2 b 1 c • While loop, second iteration • Dequeque b from Q, find Adj[b]=<s, a, c, f> • Mark a, c, f as “G”, • Update d[a], d[c], d[f], [a], [c],[ f] • Put a, c, f into Q • Mark b as “B” • Q=<e, a, c, f> 2 a 0 2 s f d e 1

13. 2 b 1 c vertex u S a b c d e f color[u] d[u] [u] 2 B G B G G B G 0 2 1 2 2 1 2 NIL b s b e s b a 0 2 s f 2 d e 1 Example(continued) • While loop, third iteration • Dequeque e from Q, find Adj[e]=<s, a, d, f> • Mark d as “G”, mark e as “B” • Update d[d], [d], • Put d into Q • Q=<a,c,f,d>

14. vertex u s a b c d e f color[u] d[u] [u] B B B G G B G 0 2 1 2 2 1 2 NIL b s b e s b Example(continued) 2 • While loop, fourth iteration • Dequeque a from Q, find Adj[a]=<b, e> • mark a as “B” • Q=<c, f, d> b 1 c 2 a 0 2 s f 2 d e 1

15. 2 b 1 c vertex u s a b c d e f color[u] d[u] [u] 2 B B B B G B G 0 2 1 2 2 1 2 NIL b s b e s b a 0 2 s f 2 d e 1 Example(continued) • While loop, fifth iteration • Dequeque c from Q, find Adj[c]=<b, d> • mark c as “B” • Q=<f, d>

16. 2 2 b 1 c vertex u S a b c d e f color[u] d[u] [u] 2 B B B B G B B 0 2 1 2 2 1 2 NIL b s b e s b a 0 2 s f 2 2 d e 1 Example(continued) • While loop, sixth iteration • Dequeque f from Q, find Adj[f]=<b,e> • mark f as “B” • Q=<d>

17. 2 2 b 1 c vertex u S a b c d e f color[u] d[u] [u] 2 B B B B B B B 0 2 1 2 2 1 2 NIL b s b e s b a 0 2 s f 2 2 d e 1 Example(continued) • While loop, seventh iteration • Dequeque d from Q, find Adj[d]=<c,e> • mark d as “B” • Q is empty

18. 2 2 b 1 c vertex u s a b c d e f color[u] d[u] [u] 2 B B B B B B B 0 2 1 2 2 1 2 NIL b s b e s b a 0 2 s f 2 2 d e 1 Example(continued) • While loop, eighth iteration • Since Q is empty, stop

19. 2 2 b 1 c vertex u s a b c d e f color[u] d[u] [u] 2 B B B B B B B 0 2 1 2 2 1 2 NIL b s b e s b a 0 2 s f 2 2 d e 1 Example(continued) Question: What happens if we change the order of vertices in each Adj[] How do you construct a shortest path from s to any vertex by using the following table?

20. The Answer

21. Analysis of running time of BFS Constant times C1 n C2 n-1 C3 n-1 C4 n-1 C5 1 C6 1 C7 1 C8 1 C9 1 C10 t(n) C11 t(n)-1 C12 C13 C14 C15 C16 C17 C18 t(n)-1 Where t(n) stands for the # of condition tests of while loop, and w(u) stands for the # of vertices first discovered by u

22. Analysis of the Breadth-First Search Algorithm • For simplification, we use one constant c instead of the ci’s (or just 1). • The following analysis is valid for allgraphs. (How about connected graphs?) • The initialization requires c(4V + 2) time units. • Each vertex u is en-queued and thus de-queued at mostonce, thus t(n) V, and • What is the total amount of time for ? • Since each vertex is exactly first discovered at most once, so, • Hence the total amount of time needed for processing the whole graph is

23. Graphs that are not connected • For such graphs, only the vertices v that are in the same component as s will get a value d[v]. • In particular, we can use the array d[ ] at the end of the computation to decide whether the graph is connected. How? • Alternatively, we can use the array color[ ] or the array [ ]. How?

24. Continued • We can modify BFS so that it returns a forest. • More specifically, if the original graph is composed of connected components C1, C2, …,Ck, then BFS will return a tree corresponding to each Ci.

25. For each vertex uV color[u]=WHITE; d[u]= ; [u]=NIL; For each vertex u V If d[u] =  then BFSR(G, u); Note, BFSR is a revised version of BFS, what is the difference? BFS for disconnected graph

26. BFS algorithm computes the shortest paths • To prove this conclusion, we need to prove the following two parts: • Prove that the BFS algorithm outputs the correct distance d[v] • Prove that the paths obtained by using the array [ ] are shortest. • We need to prove the predecessor sub-graphG in the following is a breadth-first tree (if it contains all the vertices that are reachable from s, and there is a unique simple path from s to each vertex v in the G that is also a shortest path from s to v). • G=(V, E), where V = {v | [v]  Nil }  {s} E = {([v], v), v  V-{S}}

27. Correctness proof • We will prove the correctness of the Dijkstra’s algorithm, which implies the correctness of the BFS algorithm. • Here is a simple architecture of the proof: • The vertices in the queue have at most two consecutive d[] values, and the vertices with smaller d[] values are in the head (by induction on # of vertices processed). • The earlier a vertex is processed, the smaller its d[] value is, vise versa (by induction on the order of vertices processed). • When a vertex is first discovered, its d[] value is equal to its distance from s (by induction on the order of vertices discovered).

28. Depth-first Search Algorithm(outline) • The idea of DFS • The DFS algorithm • The time complexity of DFS algorithm • Properties of the DFS algorithm

29. Depth-First Search Algorithm(ideas) • In DFS, edges are explored out of the most recently discovered vertex v. Only edges to unexplored vertices are explored. • When all of v’s incident edges have been explored, search will backtrack to explore edges leaving the vertex from which v is discovered. • The process continues until all the vertices reachable from the original source vertex are discovered. • If any undiscovered vertex remain, then one of them is selected as a new source vertex, and the search repeat from this new source vertex. • The process is repeated until all the vertices are discovered. • The strategy of DFS is to search “deeper” whenever it is possible.

30. Four Arrays for DFS Algorithm • color[u]: the color of each vertex visited • WHITEundiscovered • GRAY discovered but not finished processing • BLACKfinished processing. • [u]: The predecessor of u, indicating the vertex from which u is first discovered. • d[u]: Discovery time, a counter indicating when vertex u is first discovered. • f[u]: Finishing time, a counter indicating when the processing of vertex u (and the processing of all its descendants ) is finished.

31. DFS Algorithm • Input: A graph G = (V, E). • Outputs: Four arrays: Color[], [], d[], and f[]. • Note: From [], we can derive the predecessor sub-graph G=(V, E), where E = {([v], v), v  V and [v]Nil } The predecessor sub-graph forms a depth-first forest that is composed of depth-first trees.

32. DFS Algorithm

33. DFS Algorithm (Continued)

34. DFS Algorithm (Example) 1/ a d b c g e f

35. Example(continued) 1/ a d 2/ b c g e f

36. Example(continued) 1/ a d 2/ b 3/ c g e f

37. Example(Continued) 1/ a d 2/ b 3/ c g e f 4/

38. Example(Continued) 1/ a d 2/ b 3/ c g e f 4/5

39. Example(Continued) 1/ a d 2/ b 3/6 c g e f 4/5

40. Example(Continued) 1/ a d 2/7 b 3/6 c g e f 4/5

41. Example(Continued) 1/8 a d 2/7 b 3/6 c g e f 4/5

42. Example(Continued) 1/8 9/ a d 2/7 b 3/6 c g e f 4/5

43. Example(Continued) 1/8 a 9/ d 2/7 b 3/6 c g e 10/ f 4/5

44. Example(Continued) 1/8 a 9/ d 2/7 b 3/6 c 10/11 g e f 4/5

45. Example(continued) 1/8 a 9/ d 2/7 b 3/6 12/13 c g e 10/11 f 4/5

46. 1/8 a 9/14 d 2/7 b 12/13 3/6 c 10/11 g e f 4/5 Example(Continued) What happens if we change the order vertices of in each Adj[] or the order of vertices in line 5 of DFS(G)? In a directed graph?

47. What does DFS do? Given a graph G, it traverses all vertices of G and • Constructed a collection of rooted trees together with a set of source vertices (roots) • Outputs two arrays d[ ]/f[ ] Note: Forest is stored in array [ ], the [ ] value of a root is null.

48. Running time analysis of DFS DFS-VISIT(u) color[u]=gray; //1 time=time+1; //1 d[u]=time; //1 for each vadj[u] //d(u) do if color[v]=white //d(u) then [v]=u; //w(u) DFS-VISIT(v); //w(u) color[u]=black; //1 time=time+1; //1 f[u]=time; //1 sum: T(u) =6+2d(u)+2w(u) DFS(G) for each u in V//n do color[u]=white; //n [u]=null; //n time=0; //1 for each u in V//n do if color[u]= white //n then DFS-VISIT(u); //t(n) • Sum:5n+1+t(n)=(n) Where t(n) stands for # of DFS-VISIT(u) calls,w(u) stands for # of vertices first discovered by u, and T(u) stands for the time of DFS-VISIT(u) procedure.

49. Running time analysis of DFS Total running time is And since, So, total running time is: (V+E) Note: Combine t(n) and w(u) together.