1 / 64

GRAPH ALGORITHMS BASICS

GRAPH ALGORITHMS BASICS. Graph : G = {V, E} V ={v 1 , v 2 , …, v n }, E = {e 1 , e 2 , … e m }, e ij =(v i , v j , l ij ) A set of n nodes/vertices, and m edges/arcs as pairs of nodes: Problem sizes: n, m Where l ij , if present, is a label or the weight of an edge.

rhiannon-alexander
Download Presentation

GRAPH ALGORITHMS BASICS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GRAPH ALGORITHMSBASICS Graph: G = {V, E} V ={v1, v2, …, vn}, E = {e1, e2, … em}, eij=(vi, vj, lij) A set of n nodes/vertices, and m edges/arcs as pairs of nodes: Problem sizes: n, m Where lij, if present, is a label or the weight of an edge. V ={v1, v2, v3, v4, v5} E = {e1=(v1, v2), (v3, v1), (v3, v4), (v5, v2), e5=(v4, v5)} degree(v1)=2, degree(v2)=2, … :number of arcs For directed graph: indegree, and outdegree v1 v3 v2 v5 v4 (C) Debasis Mitra

  2. GRAPH ALGORITHMSBASICS Adjacency List Representation: v1: (v2,10), (v3, 20) v2: (v1,10), (v5, 25) v3: (v1,20), (v4, 30) v4: (v3,30), (v5, 50) v5: (v2,25), (v4, 50) Weighted graph G: V ={v1, v2, v3, v4, v5} E = {(v1, v2, 10), (v3, v1, 20), (v3, v4 , 30), (v5, v2 , 25), (v4, v5 , 50)} v1 10 20 Matrix representation: v3 Matrix representation for directed Graph? v2 30 25 Matrix representation for unweighted Graph? v5 v4 50 (C) Debasis Mitra

  3. Directed graph: edges are ordered pairs of nodes. Weighted graph: each edge (directed/undirected) has a weight. Path between a pair of nodes vi, vk: sequence of edges with vi, vk at the two ends. Simple path: covers no node in it twice. Loop: a path with the same start and end node. Path length: number of edges in it. Path weight: total wt of all edges in it. Connected graph: there exists a path between every pair of nodes, no node is disconnected. Complete graph: edge between every pair of nodes [NUMBER OF EDGES?]. Acyclic graph: a graph with no cycles. Etc.  Graphs are one of the most used models of real-life problems for computer-solutions. GRAPH ALGORITHMSBASICS (C) Debasis Mitra

  4. Algorithm 1: For each node v in V do -Steps- // (|V|) Algorithm 2: For each node v in V do For each edge in E do -Steps- // (|V|*|E|) Algorithm 3: For each node v in V do For each edge e adjacent to v do -Steps- // ( |E| ) with adjacency list, // but (|V|* |V| ) for matrix representation Algorithm 4: For each node v in V do -steps- For each edge e of v do -Steps- // ( |V|+|E| ) or ( max{|V|, |E| }) GRAPH ALGORITHMSBASICS v1 10 20 v3 v2 30 25 v5 v4 50 (C) Debasis Mitra

  5. TOPOLOGICAL SORTProblem 1 Input: directed acyclic graph Output: sequentially order the nodes without violating any arc ordering. Note: you may have to check for cycles - depending on the problem definition (input: directed graph). An important data: indegree of each node - number of arcs coming in (#courses pre-requisite to "this" course). (C) Debasis Mitra

  6. TOPOLOGICAL SORTProblem 1 Input: directed acyclic graph Output: sequentially order the nodes without violating any arc ordering. A first-pass Algorithm: Assign indegree values to each node In each iteration: Eliminate one of the vertices with indegree=0 and its associated (outgoing) arcs (thus, reducing indegrees of the adjacent nodes) - after assigning an ordering-number (as output of topological-sort ordering) to these nodes, keep doing the last step for each of the n number-of-nodes. If at any stage before finishing the loop, there does not exist any node with indegree=0, then a cycle exists! [WHY?] A naïve way of finding a vertex with indegree0 is to scan the nodes that are still left in the graph. As the loop runs for ntimes this leads to (n2) algorithm. (C) Debasis Mitra

  7. TOPOLOGICAL SORTProblem 1 Input: A directed graph Output: A sorting on the node without violating directions, or Failure Algorithm naïve-topological-sort 1 For each node v ϵ V calculate indegree(v); // (N) 2 counter = 0; 3 While V≠ empty do // (n) 4 find a node v ϵ V such that indegree(v)= =0; // have to scan all nodes here: (n) // total: (n) x (while loop’s n) = O(n2) 5 if there is no such v then return(Failure) else 6 ordering-id(v) = ++counter; 7 V = V – v; 8 For each edge (v, w) ϵ E do //nodes adjacent to v //adjacent nodes: (|E|), total for all nodes O(n or |E|) 9 decrement indegree(w); 10 E = E – (v, w); end if; end while; End algorithm. Complexity: dominating term O(n2) (C) Debasis Mitra

  8. TOPOLOGICAL SORTProblem 1 v1 Indegree(v1)=0 Indegree(v2)=1 Indegree(v3)=1 Indegree(v4)=2 Indegree(v5)=1 Ordering-id(v1) =1 Eliminate node v1, and edges (v1,v2), (v1,v3) from G Update indegrees of nodes v2 and v3 v3 v2 v5 v4 (C) Debasis Mitra

  9. TOPOLOGICAL SORTProblem 1 Indegree(v2)=0 Indegree(v3)=0 Indegree(v4)=2 Indegree(v5)=1 Ordering-id(v1) =1, Ordering-id(v2)=2 Eliminate node v2, and edges (v2,v5) from G Update indegrees of nodes v5 v3 v2 v5 v4 (C) Debasis Mitra

  10. TOPOLOGICAL SORTProblem 1 Indegree(v3)=0 Indegree(v4)=2 Indegree(v5)=0 Ordering-id(v1) =1, Ordering-id(v2)=2, O-id(v3)=3 Eliminate node v3, and edges (v3,v4) from G Update indegrees of nodes v4 v3 v5 v4 (C) Debasis Mitra

  11. TOPOLOGICAL SORTProblem 1 Indegree(v4)=1 Indegree(v5)=0 Ordering-id(v1) =1, Ordering-id(v2)=2, O-id(v3)=3, (v5)=4 Eliminate node v5, and edges (v5,v4) from G Update indegrees of nodes v4 v5 v4 (C) Debasis Mitra

  12. TOPOLOGICAL SORTProblem 1 Indegree(v4)=0 Ordering-id(v1) =1, Ordering-id(v2)=2, O-id(v3)=3, (v5)=4, (v4)=5 Eliminate node v4 from G, and V is empty now Resulting sort: v1, v2, v3, v5, v4 v4 (C) Debasis Mitra

  13. TOPOLOGICAL SORTProblem 1 v1 However, now, Indegree(v1)=0 Indegree(v2)=2 Indegree(v3)=1 Indegree(v4)=2 Indegree(v5)=1 Ordering-id(v1) =1 Eliminate node v1, and edges (v1,v2), (v1,v3) from G Update indegrees of nodes v2 and v3 v3 v2 v5 v4 (C) Debasis Mitra

  14. TOPOLOGICAL SORTProblem 1 Indegree(v2)=1 Indegree(v3)=0 Indegree(v4)=2 Indegree(v5)=1 Ordering-id(v1) =1, Ordering-id(v3)=2 Eliminate node v3, and edges (v3,v4) from G Update indegrees of nodes v4 v3 v2 v5 v4 (C) Debasis Mitra

  15. TOPOLOGICAL SORTProblem 1 Indegree(v2)=1 Indegree(v4)=1 Indegree(v5)=1 No node to choose before all nodes are ordered: Fail v2 v5 v4 (C) Debasis Mitra

  16. TOPOLOGICAL SORTProblem 1 • A smarter way: notice indegrees become zero for only a subset of nodes in a pass, • but all nodes are being checked in the above naïve algorithm. • Store those nodes (who have become “roots” of the truncated graph) in a box • and pass the box to the next iteration. • An implementation of this idea could be done by using a queue: • Push those nodes whose indegrees have become 0’s (when you update those values), • at the back of a queue; • Algorithm terminates on empty queue, • but having empty Q before covering all nodes: • we have got a cycle! (C) Debasis Mitra

  17. TOPOLOGICAL SORTProblem 1 Algorithm Q-based-topological-sort 1 For each vertex v in G do // initialization 2 calculate indegree(v); // (|E|)with adj. list 3 if indegree(v) = =0 then push(Q, v); end for; 4 counter = 0; 5 While Q is not empty do // indegree becomes 0 once & only once for a node // so, each node goes to Q only once: (N) 6 v = pop(Q); 7 ordering-id(v) = ++counter; 8 V = V – v; 9 For each edge (v, w) ϵ E do //nodes adjacent to v // two loops together (max{N, |E|}) // each edge scanned once and only once 10 E = E – (v, w); 11 decrement indegree(w); 12 if now indegree(w) = =0 then push(Q, w); end for; end while; 13 if counter != N then return(Failure); // not all nodes are numbered yet, but Q is empty // cycle detected; End algorithm. Complexity: the body of the inner for-loop is executed at most once per edge, even considering the outer while loop, if adjacency list is used. The maximum queue length is n. Complexity is (|E| + n). (C) Debasis Mitra

  18. SOURCE TO ALL NODES SHORTEST PATH-LENGTHProblem 2 Path length = number of edges on a path. Compute shortest path-length from a given source node to all nodes on the graph. Strategy: starting with the source as the "current-nodes," in each of the iteration expand children (adjacent nodes) of "current-nodes." Assign the iteration# as the shortest path length to each expanded node, if the value is not already assigned. Also, assign to each child, its parent node-id in this expansion tree (to retract the shortest path if necessary). Input: Undirected graph, and source node Output: Shortest path-length to each node from source This is a breadth-first traversal, nodes are expanded as increasing distances from the source: 0, then 1, then 2, etc. (C) Debasis Mitra

  19. SOURCE TO ALL NODES SHORTEST PATH-LENGTHProblem 3 Once again, a better idea is to have the children being pushed at the back of a queue. Input: Undirected graph, and source node Output: Shortest path-length to each node from source Algorithm q-based-shortest-path 1 d(s) = 0; // source 2 enqueue only s in Q; // (1), no loop, constant-time 3 while (Q is not empty) do // each node goes to Q only once: (N) 4 v = dequeueQ; 5 for each vertex w adjacent to v do // (|E|) 6 if d(w) is yet unassigned then 7 d(w) = d(v) + 1; 8 last_on_path(w)= v; 9 enqueuew in Q; end if; end for; end while; End algorithm. Complexity: (|E| + n), by a similar analysis as that of the previous queue-based algorithm. (C) Debasis Mitra

  20. SOURCE TO ALL NODES SHORTEST PATH-LENGTHProblem 3 Source: v1 : v1, length=0 Queue: v1 v3 v2, length=1 v3, length=1 v2 v5, length=1 Queue: v2,v3 , v5 Q: v3 ,v5 v1, no v1, no v5, already assigned v1, no v5 v1, no v4 v4, length=2 Q: v4 v3, no Q: empty (C) Debasis Mitra

  21. SOURCE TO ALL NODES SHORTEST PATH-WEIGHT Input: Positive-weighted graph, a “source” node s Output: Shortest distance to all nodes from the source s v1 10 20 70 v3 v2 Breadth-first search does not work any more! Say, source is v1 Shortest-path-length(v5) = 35, not 70 However, the concept works: expanding distance front from s 25 30 v5 v4 50 (C) Debasis Mitra

  22. SINGLE SOURCE SHORTEST PATH Input: Weighted graph Dsw , a “source” node s [can not handle negative cycle] Output: Shortest distance to all nodes from the source s Algorithm Dijkstra 1 s.distance = 0; // shortest distance from s 2 mark s as “finished”; //shortest distance to s from s found 3 For each node w do 4 if (s, w) ϵ E then w.distance= Dsw else w.distance = inf; // O(n) 5 While there exists any node not marked as finished do // (n), each node is picked once & only once 6 v = node from unfinished list with the smallest v.distance; // loop on unfinished nodes O(n): total O(n2) 7 mark v as “finished”; // WHY? 8 For each edge (v, w) ϵ E do //adjacent to v:(|E|) 9 if (w is not “finished” &(v.distance + Dvw < w.distance) ) then 10 w.distance = v.distance + Dvw; 11 w.previous = v; // v is parent of w on the shortest path end if; end for; end while; End algorithm (C) Debasis Mitra

  23. DJIKSTRA’S ALG: SINGLE SOURCE SHORTEST PATH v1 (0) v1 (0) v1 (0) 10 20 10 20 10 20 70 70 v2 ( ) v3 ( ) 70 v3 (20) v3 (20) v2 (10) v2 (10) 25 25 25 30 30 30 v5 ( ) v4 ( ) 50 v5 (35) v5 (70) v4 ( ) v4 ( ) 50 50 v1 (0) v1 (0) 10 20 10 20 70 v3 (20) v2 (10) 70 v3 (30) v2 (10) 25 25 30 30 v5 (35) v4 (50) 50 v5 (35) v4 (50) 50 (C) Debasis Mitra

  24. DJIKSTRA’S ALG: COMPLEXITY • Lines 2-4, initialization: O(n); • Line 5: while loop runs n-1 times: [(n)], • Line 6: find-minimum-distance-node v runs another (n) within it, • thus the complexity is (N2); • Can you reduce this complexity by a Queue? • Line 8: for-loop within while, for adjacency list graph data structure, • runs for |E| times including the outside while loop. • Grand total: (|E| + n2) = (n2), as |E| is always  n2. (C) Debasis Mitra

  25. DJIKSTRA’S ALG: COMPLEXITY • Djikstra complexity: (|E| + n2) = (n2), as |E| is always  n2. • Using a heap data-structurefind-minimum-distance-node (line 6) may be log_n, • inside while loop (line 5): total O(n log_n) • but as the distance values get changed the heap needs to be kept reorganized, • thus, increasing the for-loop's complexity to (|E|log N). • Grand total: (|E|log_n + n log_n) = (|E|log_n), as |E| is always  n in a connected graph. (C) Debasis Mitra

  26. DJIKSTRA’S ALG: PROOF SKETCH Induction base: Shortest distance for the source s itself must have been found Induction Hypothesis: Suppose, it works up to the k-th iteration. This means, for k nodes the corresponding shortest paths from s have been already identified (say, set F, and V-F is U, the set of unfinished nodes). Induction Step: Let, node p has shortest distance in U, in the k+1 –th iteration. (1) Each node in F has its chance to update p. (2) Each other node in U has a distance >= that of p, so, cannot improve p any more (assuming edge weights are positive or non-negative) (3) Hence, p is ‘finished” and so, moved from U to F (4) Each node in F, except p, has its chance to update nodes in U. (5) Each other node in U has a distance >= that of p, so, cannot improve distances of nodes in U optimally (assuming edge weights are positive or non-negative) (6) Hence, p is the only candidate to improve distances of nodes in U and so, updates only nodes in U in the k+1 -th iteration (C) Debasis Mitra

  27. DJIKSTRA’S ALG:WITH NEGATIVE EDGE • Cost may reduce by adding a new arc • Hence, the ‘finished’ marker does not work • The proof of above Djikstra depends on positive edge weights • Assumption: a path’s cost is non-decreasing as new arc is added to it • This assumption is not true for negative-weighted edges (C) Debasis Mitra

  28. DJIKSTRA’S ALG:WITH NEGATIVE EDGE Algorithm Dijkstra-negative Input: Weighted (undirected) graph G, and a node s in V Output: For each node v in V, shortest distance w.distance from s 1 Initialize a queue Q with the s; 2 While Q not empty do 3 v = pop(Q); 4 For each w such that (v, w) ϵ E do 5 If (v.distance + Dvw < w.distance) then 6 w.distance = v.distance +Dvw; 7 w.parent = v; 8 If w is not already in Q then push(Q, w); // if it is already there in Qdo not duplicate end if end for end while End Algorithm. (C) Debasis Mitra

  29. DJIKSTRA’S ALG:WITH NEGATIVE EDGE • Negative cost on an edge or a path is all right, but negative cycle is a problem • A path may repeat itself on a loop, thus reducing negative cost on it • That will go on infinitely • No limit to minimizing the cost between a source to any node on a negative cycle • Shortest-path is not defined when there is a negative cycle. (C) Debasis Mitra

  30. DJIKSTRA’S ALG:WITH NEGATIVE EDGE • Cost reduces on looping around negative cycles • Q-Djikstra does not prevent that: it may not terminate • But how many looping should we allow? • Can we predict that we are on an infinite loop here? • How many times a node may appear in the queue? • What is the “diameter” of a graph: longest path length? (C) Debasis Mitra

  31. DJIKSTRA’S ALG:WITH NEGATIVE CYCLE • “Diameter” of a graph = n • No node should get a chance to update others more than n+1 times! • Because, there cannot be a simple (loop free) path of length more than n • Line 8 of Q-Djikstra is guaranteed to happen once for any node, • or, any node will be pushed to Q at least once • But not more than n times, unless it is on a negative loop • So, Q-Dijkstra should be terminated if a node v gets n+1 times in to the queue • If that happens: the graph has a negative-cost cycle with the node v on it • Otherwise, Q-Djikstra may go into an infinite loop • (over the cycle with negative path-weight) • Complexity of Q-Djikstra: O(n*|E|), because each node may be updated by • each edge at most one time (if there is no negative cycle). (C) Debasis Mitra

  32. Djikstra’s Algorithm is Greedy Algorithm: pick up the best node from U in each cycle Find all pairs shortest distance (Problem 4): For Each node v as source Run Djikstra(v, G) Complexity: O(n.n2), or O(n.n.|E|) for negative edges in G A dynamic programming algorithm (Floyd’s alg) does better (C) Debasis Mitra

  33. Problem4: All Pairs Shortest Path A variation of Djikstra’s algorithm. Called Floyd-Warshal’s algorithm. Good for dense graph. Algorithm Floyd Copy the distance matrix in d[1..n][1..n]; for k=1 through n do //consider each vertex as updating candidate for i=1 through n do for j=1 through n do if (d[i][k] + d[k][j] < d[i][j]) then d[i][j] = d[i][k] + d[k][j]; path[i][j] = k; // last updated via k End algorithm. O(n3), for 3 loops.

  34. D1(4,5) = min{D0(4,5) , D0(4,1) + D0(1,5)} = min{inf, 2+3} (C) Debasis Mitra Source: Cormen et al.

  35. Problem 5: Max-flow / Min-cut (Optimization) Input: Weighted directed graph (weight = maximum permitted flow on an edge), a source node s without any incoming arc, a sink node n without any outgoing arc. Objective function: schedule a flow graph s.t. no node holds any amount (influx=outflux), except for s and n, and no arc violets its allowed amount Output: maximize the objective function (maximize the amount flowing from s to n) <= output max-Gflow G =>

  36. Max-flow / Min-cut Greedy Strategy Input: Weighted directed graph (weight = maximum permitted flow on an edge), a source node s without any incoming arc, a sink node n without any outgoing arc. Output: maximize the objective function (maximize the amount flowing from s to n) Greedy Strategy: Initialize a flow graph Gf (each edge 0) Gr, and a residual graph as G In each iteration, find a “max path” p from s to n in Gr Max path is a path allowing maximum flow from s->n, subject to the constraints Add p to Gf, and subtract p from Gr, any edge having 0 is removed from Gr Continue iterations until there exists no such path in Gr Return Gf

  37. Greedy Alg, initialization Input G Gflow Gresidual Source: Weiss (C) Debasis Mitra

  38. G => <= optimum output Gflow Gresidual Greedy Alg, iteration 1 Terminates (C) Debasis Mitra

  39. Max-flow / Min-cut Greedy Strategy Complexity: How many paths may exist in a graph? |E| ! How many times any edge may be updated until it gets to 0? f.|E| for max-flow f Each iteration, looks for max-path: |E| log |V| Total: O(f. |E|2 log |V|)

  40. Returned Result, Flow = 3 Optimum Result, Flow = 5 1 Capacities Unutilized => 1 (C) Debasis Mitra

  41. Correct Alg, with oracle, not-greedy, iteration 1 Start with not-max-path s->b->d->t (2 each edge) Iteration 2 (C) Debasis Mitra

  42. Iteration 2 Iteration 3 Terminates with optimum flow=5 (C) Debasis Mitra

  43. Max-flow Backtracking StrategyFord-Fulkerson’s Algorithm Greedy Strategy is sub-optimal A Backtracking optimal algorithm: Initialize a flow graph Gf (each edge 0) Gr, and a residual graph as G In each iteration, find a path p from s to n in Gr Add p to Gf, and Gr is updated with reverse flow opposite to p, and leaving residual amounts in forward direction => allows backtracking if necessary Continue iterations until either all edges from s are incoming, or all edges to n are outgoing Return Gf Guaranteed to terminate for rational numbers input Complexity: If f is the max flow in returned Gf, Then, number of times any edge may get updated is f , even though each arc never gets to 0 O(f. |E|) Random choice for a path p may run into bad input

  44. Ford-Fulkerson, using backflow residual, Iteration 1 Iteration 2 Terminates, no outflow left from source, or inflow to sink (C) Debasis Mitra

  45. Problem input for Ford-Fulkerson: May keep backtracking if starts with random path, say, s->a->b->t Complexity: If f is the max flow in returned Gf, O(f. |E|), or O(100000*|E|) Why bother? Use greedy! (C) Debasis Mitra

  46. Max-flow Backtracking with DjikstraEdmonds-Karp Algorithm Another Backtracking optimal algorithm (may avoid problem of Ford-Fulkerson): Initialize a flow graph Gf (each edge 0) Gr, and a residual graph as G In each iteration, find a “max path” p from s to n in Gr using Djikstra Add p to Gf, and Gr is updated with reverse flow opposite to p, and leaving residual amounts in forward direction => allows backtracking if necessary Continue iterations until either all edges from s are incoming, or all edges to n are outgoing Return Gf Complexity:O(cmax|E|2 log |V|), first term is max capacity of any edge [actually logCmax in complexity!] Min-cut = Minimum total capacity from one partition (of G) S ϵ source to another T ϵ sink Which edges are to cut from G to isolate source from sink? Min-cut = Max-flow

  47. Problem 6: Minimum Spanning Tree (Optimization) Prim’s Algorithm • Input: Weighted un-directed graph • Objective function: Total weight of a spanning tree over G • Output: minimize the objective function (minimum-wt spanning tree - MST) • Prim’s Algorithm: Djikstra’s strategy • Separate two sets of nodes F (in MST now) and U (not in MST yet), starting with any arbitrary node s ϵ F • In each iteration find a ‘closest’ node v ϵ U from any node pϵF; move v from U to F; add Epv to MST: {For all xϵU, For all yϵF, |Evy| ≤ |Exy| } • Continue until U is empty • Return MST • How many nodes are on a spanning tree? • How many arcs? • How many spanning trees exist on a graph? • Complexity of Prim’s Alg:? (C) Debasis Mitra

  48. Prim’s Algorithm for MST: Complexity • How many nodes are on a spanning tree? • How many arcs? • How many spanning trees exist on a graph? • Complexity of Prim’s Alg:? Exactly same as Djikstra: O(|E| log n) (C) Debasis Mitra

  49. Prim’s Algorithm for MST Prim’s Alg starting with a in F Closest to a is b with 1 on Eab a a a 3 2 3 2 3 2 7 c b 7 9 c b 7 9 c b 9 1 10 1 10 8 1 10 8 8 d e d e d e Closest to a or b is e with Ebe=1 Closest to a, b or e is c with Eac=3 Closest to a, b, e or c is d with Ebd=8 a a a 3 2 3 3 2 2 7 7 7 c b c b c b 9 9 9 1 10 1 1 10 10 8 8 8 d e d d e e Done! All nodes in F. Eab+ Ebc + Ebd+ Eac= 14, is MST weight (C) Debasis Mitra

  50. Proof sketch: Prims’s Algorithm Inductionbase: ??? Inductionhypothesis: on k-th iteration (1<=k<n), say, MST is correct for the subgraph of G with k nodes in F Induction Step: Epv is a shortest arc between all arcs from F to U  After adding Epv to MST, it is still correct for the new subgraph with k+1 nodes (C) Debasis Mitra

More Related