1 / 46

CSCE 411 Design and Analysis of Algorithms

CSCE 411 Design and Analysis of Algorithms. Set 9: More Graph Algorithms Prof. Jennifer Welch Spring 2012. Generic MST Algorithm. input: weighted undirected graph G = (V,E,w) T := empty set while T is not yet a spanning tree of G

lisbet
Download Presentation

CSCE 411 Design and Analysis of Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CSCE 411Design and Analysis of Algorithms Set 9: More Graph Algorithms Prof. Jennifer Welch Spring 2012 CSCE 411, Spring 2012: Set 9

  2. Generic MST Algorithm • input: weighted undirected graph G = (V,E,w) • T := empty set • while T is not yet a spanning tree of G • find an edge e in E s.t. T U {e} is a subgraph of some MST of G • add e to T • return T (as MST of G) CSCE 411, Spring 2012: Set 9

  3. Kruskal's Algorithm as a Special Case of Generic Algorithm • Consider edges in increasing order of weight • Add the next edge iff it doesn't cause a cycle • At any point, T is a forest (set of trees); eventually T is a single tree CSCE 411, Spring 2012: Set 9

  4. Idea of Prim's Algorithm • Instead of growing the MST as possibly multiple trees that eventually all merge, grow the MST from a single vertex, so that there is only one tree at any point. • Also a special case of the generic algorithm: at each step, add the minimum weight edge that goes out from the tree constructed so far. CSCE 411, Spring 2012: Set 9

  5. Prim's Algorithm • input: weighted undirected graph G = (V,E,w) • T := empty set • S := {any vertex in V} • while |T| < |V| - 1 do • let (u,v) be a min wt. outgoing edge (u in S, v not in S) • add (u,v) to T • add v to S • return (S,T) (as MST of G) CSCE 411, Spring 2012: Set 9

  6. Prim's Algorithm Example 8 7 b c d 4 9 2 11 a i 4 e 14 7 6 8 10 h g f 1 2 CSCE 411, Spring 2012: Set 9

  7. Correctness of Prim's Algorithm • Let Ti be the tree represented by (S,T) at the end of iteration i. • Show by induction on i that Ti is a subtree of some MST of G. • Basis: i = 0 (before first iteration). T0 contains just a single vertex, and thus is a subtree of every MST of G. CSCE 411, Spring 2012: Set 9

  8. u v Correctness of Prim's Algorithm • Induction:Assume Ti is a subtree of some MST M. We must show Ti+1 is a subtree of some MST. • Let (u,v) be the edge added in iteration i+1. Case 1:(u,v) is in M. Then Ti+1 is also a subtree of M. Ti Ti+1 CSCE 411, Spring 2012: Set 9

  9. Correctness of Prim's Algorithm Case 2:(u,v) is not in M. • There is a path P in M from u to v, since M spans G. • Let (x,y) be the first edge in P with one endpoint in Ti and the other not in Ti. y P x Ti u v CSCE 411, Spring 2012: Set 9

  10. Correctness of Prim's Algorithm • Let M' = M - {(x,y)} U {(u,v)} • M' is also a spanning tree of G. • w(M') = w(M) - w(x,y) + w(u,v) ≤ w(M) since (u,v) is min wt outgoing edge • So M' is also an MST and Ti+1 is a subtree of M' y x Ti u v Ti+1 CSCE 411, Spring 2012: Set 9

  11. Implementing Prim's Algorithm • How do we find minimum weight outgoing edge? • First cut: scan all adjacency lists at each iteration. • Results in O(VE) time. • Try to do better. CSCE 411, Spring 2012: Set 9

  12. Implementing Prim's Algorithm • Idea: have each vertex not yet in the tree keep track of its best (cheapest) edge to the tree constructed so far. • To find min wt. outgoing edge, find minimum among these values • use a priority queue to store the best edge info (insert and extract-min operations) CSCE 411, Spring 2012: Set 9

  13. u v w x Implementing Prim's Algorithm • When a vertex v is added to T, some other vertices might have their best edges affected, but only neighbors of v • add decrease-key operation to the priority queue v's best edge to Ti check if this edge is cheaper for w Ti Ti+1 x's best edge to Ti w's best edge to Ti CSCE 411, Spring 2012: Set 9

  14. Details on Prim's Algorithm Associate with each vertex v two fields: • best-wt[v] : if v is not yet in the tree, then it holds the min. wt. of all edges from v to a vertex in the tree. Initially infinity. • best-node[v] : if v is not yet in the tree, then it holds the name of the vertex (node) u in the tree s.t. w(v,u) is v's best-wt. Initially nil. CSCE 411, Spring 2012: Set 9

  15. Details on Prim's Algorithm • input: G = (V,E,w) // initialization • initialize priority queue Q to contain all vertices, using best-wt values as keys • let v0 be any vertex in V • decrease-key(Q,v0,0) // last line means change best-wt[v0] to 0 and adjust Q accordingly CSCE 411, Spring 2012: Set 9

  16. Details on Prim's Algorithm • while Q is not empty do • u := extract-min(Q) // vertex w/ smallest best-wt • if u is not v0 then add (u,best-node[u]) to T • for each neighbor v of u do • if v is in Q and w(u,v) < best-wt[v] then • best-node[v] := u • decrease-key(Q,v,w(u,v)) • return (V,T) // as MST of G CSCE 411, Spring 2012: Set 9

  17. Running Time of Prim's Algorithm Depends on priority queue implementation. Let • Tins be time for insert • Tdec be time for decrease-key • Tex be time for extract-min Then we have • |V| inserts and one decrease-key in the initialization: O(VTins+Tdec) • |V| iterations of while • one extract-min per iteration: O(VTex) total CSCE 411, Spring 2012: Set 9

  18. Running Time of Prim's Algorithm • Each iteration of while includes a for loop. • Number of iterations of for loop varies, depending on how many neighbors the current vertex has • Total number of iterations of for loop is O(E). • Each iteration of for loop: • one decrease key, so O(ETdec) total CSCE 411, Spring 2012: Set 9

  19. Running Time of Prim's Algorithm • O(V(Tins + Tex) + ETdec) • If priority queue is implemented with a binary heap, then • Tins = Tex = Tdec = O(log V) • total time is O(E log V) • (Think about how to implement decrease-key in O(log V) time.) CSCE 411, Spring 2012: Set 9

  20. Shortest Paths in a Graph • Let's review two important single-source shortest path algorithms: • Dijkstra's algorithm • Bellman-Ford algorithm CSCE 411, Spring 2012: Set 9

  21. s t Single Source Shortest Path Problem • Given: directed or undirected graph G = (V,E,w) and source vertex s in V • Find: For each t in V, a path in G from s to t with minimum weight • Warning! Negative weights are a problem: 4 5 CSCE 411, Spring 2012: Set 9

  22. Shortest Path Tree • Result of a SSSP algorithm can be viewed as a tree rooted at the source • Why not use breadth-first search? • Works fine if all weights are the same: • weight of each path is (a multiple of) the number of edges in the path • Doesn't work when weights are different CSCE 411, Spring 2012: Set 9

  23. Dijkstra's SSSP Algorithm • Assumes all edge weights are nonnegative • Similar to Prim's MST algorithm • Start with source vertex s and iteratively construct a tree rooted at s • Each vertex keeps track of tree vertex that provides cheapest path from s (not just cheapest path from any tree vertex) • At each iteration, include the vertex whose cheapest path from s is the overall cheapest CSCE 411, Spring 2012: Set 9

  24. 4 5 1 s 6 Prim's MST Prim's vs. Dijkstra's 4 5 1 s 6 Dijkstra's SSSP CSCE 411, Spring 2012: Set 9

  25. Implementing Dijkstra's Alg. • How can each vertex u keep track of its best path from s? • Keep an estimate, d[u], of shortest path distance from s to u • Use d as a key in a priority queue • When u is added to the tree, check each of u's neighbors v to see if u provides v with a cheaper path from s: • compare d[v] to d[u] + w(u,v) CSCE 411, Spring 2012: Set 9

  26. Dijkstra's Algorithm • input: G = (V,E,w) and source vertex s // initialization • d[s] := 0 • d[v] := infinity for all other vertices v • initialize priority queue Q to contain all vertices using d values as keys CSCE 411, Spring 2012: Set 9

  27. Dijkstra's Algorithm • while Q is not empty do • u := extract-min(Q) • for each neighbor v of u do • if d[u] + w(u,v) < d[v] then • d[v] := d[u] + w(u,v) • decrease-key(Q,v,d[v]) • parent(v) := u CSCE 411, Spring 2012: Set 9

  28. Dijkstra's Algorithm Example iteration 2 a b 8 12 4 c 10 9 6 3 2 d e 4 source is vertex a CSCE 411, Spring 2012: Set 9

  29. Correctness of Dijkstra's Alg. • Let Ti be the tree constructed after i-th iteration of while loop: • vertices not in Q • edges indicated by parent variables • Show by induction on i that the path in Ti from s to u is a shortest path and has distance d[u], for all u in Ti (i.e., show that Ti is a correct shortest path tree). • Basis: i = 1. s is the only vertex in T1 and d[s] = 0. CSCE 411, Spring 2012: Set 9

  30. Ti+1 Ti s x u Correctness of Dijkstra's Alg. • Induction: Assume Ti is a correct shortest path tree. Show that Ti+1 is a correct shortest path tree. • Let u be the vertex added in iteration i. • Let x = parent(u). Need to show path in Ti+1 from s to u is a shortest path, and has distance d[u] CSCE 411, Spring 2012: Set 9

  31. P', another path from s to u a s x u b Correctness of Dijkstra's Alg P, path in Ti+1 from s to u Ti Ti+1 (a,b) is first edge in P' that leaves Ti CSCE 411, Spring 2012: Set 9

  32. Correctness of Dijkstra's Alg Let P1 be part of P' before (a,b). Let P2 be part of P' after (a,b). w(P') = w(P1) + w(a,b) + w(P2) ≥ w(P1) + w(a,b) (nonneg wts) ≥ w(s->a path in Ti) + w(a,b) (inductive hypothesis) ≥ w(s->x path in Ti) + w(x,u) (alg chose u in iteration i and d-values are accurate, by inductive hypothesis) = w(P). So P is a shortest path, and d[u] is accurate after iteration i+1. Ti P Ti+1 s u x a b P' CSCE 411, Spring 2012: Set 9

  33. Running Time of Dijkstra's Alg. • initialization: insert each vertex once • O(V Tins) • O(V) iterations of while loop • one extract-min per iteration => O(V Tex) • for loop inside while loop has variable number of iterations… • For loop has O(E) iterations total • one decrease-key per iteration => O(E Tdec) • Total is O(V (Tins + Tex) + E Tdec) CSCE 411, Spring 2012: Set 9

  34. Using Different Heap Implementations • O(V(Tins + Tex) + ETdec) • If priority queue is implemented with a binary heap, then • Tins = Tex = Tdec = O(log V) • total time is O(E log V) • There are fancier implementations of the priority queue, such as Fibonacci heap: • Tins = O(1), Tex = O(log V), Tdec = O(1) (amortized) • total time is O(V log V + E) CSCE 411, Spring 2012: Set 9

  35. Using Simpler Heap Implementations • O(V(Tins + Tex) + ETdec) • If graph is dense, so that |E| = (V2), then it doesn't help to make Tins and Tex to be at most O(V). • Instead, focus on making Tdec be small, say constant. • Implement priority queue with an unsorted array: • Tins = O(1), Tex = O(V), Tdec = O(1) • total is O(V2) CSCE 411, Spring 2012: Set 9

  36. What About Negative Edge Weights? • Dijkstra's SSSP algorithm requires all edge weights to be nonnegative • even more restrictive than outlawing negative weight cycles • Bellman-Ford SSSP algorithm can handle negative edge weights • even "handles" negative weight cycles by reporting they exist CSCE 411, Spring 2012: Set 9

  37. Bellman-Ford Idea • Consider each edge (u,v) and see if u offers v a cheaper path from s • compare d[v] to d[u] + w(u,v) • Repeat this process |V| - 1 times to ensure that accurate information propgates from s, no matter what order the edges are considered in CSCE 411, Spring 2012: Set 9

  38. Bellman-Ford SSSP Algorithm • input: directed or undirected graph G = (V,E,w) //initialization • initialize d[v] to infinity and parent[v] to nil for all v in V other than the source • initialize d[s] to 0 and parent[s] to s // main body • for i := 1 to |V| - 1 do • for each (u,v) in E do // consider in arbitrary order • if d[u] + w(u,v) < d[v] then • d[v] := d[u] + w(u,v) • parent[v] := u CSCE 411, Spring 2012: Set 9

  39. Bellman-Ford SSSP Algorithm // check for negative weight cycles • for each (u,v) in E do • if d[u] + w(u,v) < d[v] then • output "negative weight cycle exists" CSCE 411, Spring 2012: Set 9

  40. Running Time of Bellman-Ford • O(V) iterations of outer for loop • O(E) iterations of inner for loop • O(VE) time total CSCE 411, Spring 2012: Set 9

  41. s c a b Bellman-Ford Example process edges in order (c,b) (a,b) (c,a) (s,a) (s,c) 3 2 —4 4 1 <board work> CSCE 411, Spring 2012: Set 9

  42. Correctness of Bellman-Ford Assume no negative-weight cycles. Lemma:d[v] is never an underestimate of the actual shortest path distance from s to v. Lemma:If there is a shortest s-to-v path containing at most i edges, then after iteration i of the outer for loop, d[v] is at most the actual shortest path distance from s to v. Theorem:Bellman-Ford is correct. Proof:Follows from these 2 lemmas and fact that every shortest path has at most |V| - 1 edges. CSCE 411, Spring 2012: Set 9

  43. Correctness of Bellman-Ford • Suppose there is a negative weight cycle. • Then the distance will decrease even after iteration |V| - 1 • shortest path distance is negative infinity • This is what the last part of the code checks for. CSCE 411, Spring 2012: Set 9

  44. Speeding Up Bellman-Ford • The previous example would have converged faster if we had considered the edges in a different order in the for loop • move outward from s • If the graph is a DAG (no cycles), we can fully exploit this idea to speed up the running time CSCE 411, Spring 2012: Set 9

  45. DAG Shortest Path Algorithm • input: directed graph G = (V,E,w) and source vertex s in V • topologically sort G • d[v] := infinity for all v in V • d[s] := 0 • for each u in V in topological sort order do • for each neighbor v of u do • d[v] := min{d[v],d[u] + w(u,v)} CSCE 411, Spring 2012: Set 9

  46. DAG Shortest Path Algorithm • Running Time is O(V + E). • Example: <board work> CSCE 411, Spring 2012: Set 9

More Related