1 / 37

Algorithm Design and Analysis

L ECTURE 7 Greedy Graph Algorithms Topological sort Shortest paths. Algorithm Design and Analysis. CSE 565. Adam Smith. The (Algorithm) Design Process. Work out the answer for some examples Look for a general principle Does it work on *all* your examples? Write pseudocode

axl
Download Presentation

Algorithm Design and Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LECTURE 7 • Greedy Graph Algorithms • Topological sort • Shortest paths Algorithm Design and Analysis CSE 565 Adam Smith A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  2. The (Algorithm) Design Process • Work out the answer for some examples • Look for a general principle • Does it work on *all* your examples? • Write pseudocode • Test your algorithm by hand or computer • Does it work on *all* your examples? • Prove your algorithm is always correct • Check running time Be prepared to go back to step 1! A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  3. Writing algorithms • Clear and unambiguous • Test: You should be able to hand it to any student in the class, and have them convert it into working code. • Homework pitfalls: • remember to specify data structures • writing recursive algorithms: don’t confuse the recursive subroutine with the first call • label global variables clearly A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  4. Writing proofs • Purpose • Determine for yourself that algorithm is correct • Convince reader • Who is your audience? • Yourself • Your classmate • Not the TA/grader • Main goal: Find your own mistakes A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  5. Exploring a graph Classic problem: Given vertices s,t ∈ V, is there a path from s to t? Idea: explore all vertices reachable from s Two basic techniques: • Breadth-first search (BFS) • Explore children in order of distance to start node • Depth-first search (DFS) • Recursively explore vertex’s children before exploring siblings How to convert these descriptions to precise algorithms? A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  6. 1 if (i, j) ∈ E, 0 if (i, j) E. A[i, j] = A 1 2 3 4 1 0 1 1 0 2 1 • Storage: ϴ(V 2) • Good for dense graphs. • Lookup: O(1) time • List all neighbors: O(|V|) 2 0 0 1 0 3 0 0 0 0 3 4 4 0 0 1 0 Adjacency-matrix representation The adjacency matrix of a graph G = (V, E), where V = {1, 2, …, n}, is the matrix A[1 . . n, 1 . . n] given by A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  7. 2 1 3 4 Adjacency list representation • An adjacency list of a vertex v ∈ V is the list Adj[v] of vertices adjacent tov. Typical notation:n = |V| = # verticesm = |E| = # edges Adj[1] = {2, 3} Adj[2] = {3} Adj[3] = {} Adj[4] = {3} For undirected graphs,|Adj[v] | = degree(v). For digraphs,|Adj[v] | = out-degree(v). How many entries in lists? 2|E| (“handshaking lemma”) Total ϴ(V + E) storage — good for sparse graphs. A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  8. DFS pseudocode • Maintain a global counter time • Maintain for each vertex v • Two timestamps: • v.d = time first discovered • v.f= time when finished • “color”: v.color • white = unexplored • gray = in process • black = finished • Parent v.π in DFS tree A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  9. DFS pseudocode • Note: recursive function different from first call… • Exercise: change code to set “parent” pointer? A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  10. DFS example • T = tree edge (drawn in red) • F = forward edge (to a descendant in DFS forest) • B= back edge (to an ancestor in DFS forest) • C = cross edge (goes to a vertex that is neither ancestor nor descendant) A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  11. DFS with adjacency lists • Outer code runs once, takes time O(n) (not counting time to execute recursive calls) • Recursive calls: • Run once per vertex • time = O(degree(v)) • Total: O(m+n) A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  12. Toplogical Sort A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  13. Directed Acyclic Graphs • Def. An DAG is a directed graph that contains no directed cycles. • Ex. Precedence constraints: edge (vi, vj) means vi must precede vj. • Def. A topological order of a directed graph G = (V, E) is an ordering of its nodes as v1, v2, …, vn so that for every edge (vi, vj) we have i < j. v2 v3 v6 v5 v4 v1 v2 v3 v4 v5 v6 v7 v7 v1 a DAG a topological ordering

  14. Precedence Constraints • Precedence constraints. Edge (vi, vj) means task vi must occur before vj. • Applications. • Course prerequisite graph: course vi must be taken before vj. • Compilation: module vi must be compiled before vj. Pipeline of computing jobs: output of job vi needed to determine input of job vj. • Getting dressed shirt mittens socks boots jacket underwear pants

  15. Recall from book • Every DAG has a topological order • If G graph has a topological order, then G is a DAG. A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  16. Review • Suppose your run DFS on a DAG G=(V,E) • True or false? • Sorting by discovery time gives a topological order • Sorting by finish time gives a topological order A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  17. Shortest Paths A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  18. Shortest Path Problem • Input: • Directed graph G = (V, E). • Source node s, destination node t. • for each edge e, length (e) = length of e. • length path = sum of edge lengths • Find: shortest directed path from s to t. 23 2 3 9 s 18 14 6 2 6 4 19 Length of path (s,2,3,5,t)is 9 + 23 + 2 + 16 = 50. 30 11 5 15 5 6 16 20 t 7 44 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  19. Rough algorithm (Dijkstra) • Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known. • Initialize S = {s}, d(s) = 0. • Repeatedly choose unexplored node v which minimizes • add v to S, and set d(v) = (v). shortest path to some u in explored part, followed by a single edge (u, v) (e) v d(u) u S s A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  20. Rough algorithm (Dijkstra) • Maintain a set of explored nodes S whose shortest path distance d(u) from s to u is known. • Initialize S = {s}, d(s) = 0. • Repeatedly choose unexplored node v which minimizes • add v to S, and set d(v) = (v). Invariant: d(u) is known for all vertices in S shortest path to some u in explored part, followed by a single edge (u, v) BFS withweighted edges (e) v d(u) u S s A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  21. Demo of Dijkstra’s Algorithm 2 Graph with nonnegative edge lengths: B D 10 8 A 1 4 7 9 3 C E 2 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  22. Demo of Dijkstra’s Algorithm ¥ ¥ Initialize: 2 B D 10 8 A 0 1 4 7 9 3 C E Q: A B C D E 2 ¥ ¥ ¥ ¥ 0 ¥ ¥ S: {} A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  23. Demo of Dijkstra’s Algorithm ¥ ¥ “A”EXTRACT-MIN(Q): 2 B D 10 8 A 0 1 4 7 9 3 C E Q: A B C D E 2 ¥ ¥ ¥ ¥ 0 ¥ ¥ S: { A } A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  24. Demo of Dijkstra’s Algorithm ¥ 10 Explore edges leaving A: 2 B D 10 8 A 0 1 4 7 9 3 C E Q: A B C D E 2 ¥ ¥ ¥ ¥ 0 ¥ 3 3 ¥ ¥ 10 S: { A } A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  25. Demo of Dijkstra’s Algorithm ¥ 10 “C”EXTRACT-MIN(Q): 2 B D 10 8 A 0 1 4 7 9 3 C E Q: A B C D E 2 ¥ ¥ ¥ ¥ 0 ¥ 3 3 ¥ ¥ 10 S: { A, C } A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  26. Demo of Dijkstra’s Algorithm 7 11 Explore edges leaving C: 2 B D 10 8 A 0 1 4 7 9 3 C E Q: A B C D E 2 ¥ ¥ ¥ ¥ 0 3 5 3 ¥ ¥ 10 7 11 5 S: { A, C } A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  27. Demo of Dijkstra’s Algorithm 7 11 “E”EXTRACT-MIN(Q): 2 B D 10 8 A 0 1 4 7 9 3 C E Q: A B C D E 2 ¥ ¥ ¥ ¥ 0 3 5 3 ¥ ¥ 10 7 11 5 S: { A, C, E } A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  28. Demo of Dijkstra’s Algorithm 7 11 Explore edges leaving E: 2 B D 10 8 A 0 1 4 7 9 3 C E Q: A B C D E 2 ¥ ¥ ¥ ¥ 0 3 5 3 ¥ ¥ 10 7 11 5 S: { A, C, E } 7 11 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  29. Demo of Dijkstra’s Algorithm 7 11 “B”EXTRACT-MIN(Q): 2 B D 10 8 A 0 1 4 7 9 3 C E Q: A B C D E 2 ¥ ¥ ¥ ¥ 0 3 5 3 ¥ ¥ 10 7 11 5 S: { A, C, E, B } 7 11 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  30. Demo of Dijkstra’s Algorithm 7 9 Explore edges leaving B: 2 B D 10 8 A 0 1 4 7 9 3 C E Q: A B C D E 2 ¥ ¥ ¥ ¥ 0 3 5 3 ¥ ¥ 10 7 11 5 S: { A, C, E, B } 7 11 9 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  31. Demo of Dijkstra’s Algorithm 7 9 “D”EXTRACT-MIN(Q): 2 B D 10 8 A 0 1 4 7 9 3 C E Q: A B C D E 2 ¥ ¥ ¥ ¥ 0 3 5 3 ¥ ¥ 10 7 11 5 S: { A, C, E, B, D } 7 11 9 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  32. Proof of Correctness (Greedy Stays Ahead) Invariant. For each node u S, d(u) is the length of the shortest path from s to u. Proof:(by induction on |S|) • Base case: |S| = 1 is trivial. • Inductive hypothesis: Assume for |S| = k 1. • Let v be next node added to S, and let (u,v) be the chosen edge. • The shortest s-u path plus (u,v) is an s-v path of length (v). • Consider any s-v path P. We'll see that it's no shorter than (v). • Let (x,y) be the first edge in P that leaves S,and let P' be the subpath to x. • P + (x,y) has length ·d(x)+ (x,y)·(y)·(v) P y x P' s S u v A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  33. Review Question • Is Dijsktra’s algorithm correct with negative edge weights? A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  34. Implementation • For unexplored nodes, maintain • Next node to explore = node with minimum (v). • When exploring v, for each edge e = (v,w), update • Efficient implementation: Maintain a priority queue of unexplored nodes, prioritized by (v). Priority Queue A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  35. Priority queues • Maintain a set of items with priorities (= “keys”) • Example: jobs to be performed • Operations: • Insert • Increase key • Decrease key • Extract-min: find and remove item with least key • Common data structure: heap • Time: O(logn) per operation A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  36. Pseudocode for Dijkstra(G, ) • d[s]  0 • for each vÎV – {s} • dod[v]  ¥; p[v]  ¥ • S   • Q  V ⊳Q is a priority queue maintainingV – S,keyed on [v] • whileQ ¹ • dou  EXTRACT-MIN(Q) • S  SÈ {u}; d[u]  p[u] • for each vÎAdjacency-list[u] • doif[v] > [u] + (u, v) • then [v]  d[u] + (u, v) explore edges leaving v Implicit DECREASE-KEY A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

  37. n times degree(u) times Handshaking Lemma  ·mimplicit DECREASE-KEY’s. Analysis of Dijkstra • whileQ ¹ • dou  EXTRACT-MIN(Q) • S  SÈ {u} • for each vÎAdj[u] • doif d[v] > d[u] + w(u, v) • then d[v]  d[u] + w(u, v) PQ Operation Dijkstra Array Binary heap d-way Heap Fib heap † ExtractMin n n log n HW3 log n DecreaseKey m 1 log n HW3 1 Total n2 m log n m log m/n n m + n log n † Individual ops are amortized bounds A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

More Related