UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall , 2008

1. UMass Lowell Computer Science 91.503Analysis of AlgorithmsProf. Karen DanielsFall, 2008 Lecture 4 Tuesday, 9/30/08 Graph Algorithms: Part 1 Shortest Paths Chapters 24-25

2. 91.404 Graph Review Elementary Graph Algorithms Minimum Spanning Trees Single-Source Shortest Paths

3. A A B B D F F E E D C C Introductory Graph Concepts • G= (V,E) • Vertex Degree • Self-Loops • Undirected Graph • No Self-Loops • Adjacency is symmetric • Directed Graph (digraph) • Degree: in/out • Self-Loops allowed This treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.

4. A A B B A B C D E F A B C D E F D A B C D E F A BC B CEF C D D E BD F E A BC B ACEF C AB D E E BDF F BE A B C D E F F F E E D C C Introductory Graph Concepts:Representations • Undirected Graph • Directed Graph (digraph) Adjacency Matrix Adjacency List Adjacency List Adjacency Matrix This treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.

5. A A B B F A B E E F E C D D D C C F Introductory Graph Concepts:Paths, Cycles path <A,B,F> • Path: • length: number of edges • simple: all vertices distinct • Cycle: • Directed Graph: • <v0,v1,...,vk > forms cycle if v0=vk and k>=1 • simple cycle: v1,v2..,vk also distinct • self-loop is cycle of length 1 • Undirected Graph: • <v0,v1,...,vk > forms (simple) cycle if v0=vk and k>=3 • simple cycle: v1,v2..,vk also distinct simple cycle <E,B,F,E> most of our cycle work will be for directed graphs simple cycle <A,B,C,A>= <B,C,A,B> This treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.

6. A A A A B B B D D F F F E E E D D C C C C B strongly connected component F E Introductory Graph Concepts:Connectivity connected • Undirected Graph: connected • every pair of vertices is connected by a path • one connected component • connected components: • equivalence classes under “is reachable from” relation • Directed Graph: strongly connected • every pair of vertices is reachable from each other • one stronglyconnected component • strongly connected components: • equivalence classes under “mutually reachable” relation 2 connected components not strongly connected This treatment follows 91.503 textbook Cormen et al. Some definitions differ slightly from other graph literature.

7. Vertex color shows status: not yet encountered encountered, but not yet finished finished Elementary Graph Algorithms:SEARCHING: DFS, BFS • for unweighted directed or undirected graph G=(V,E) • Breadth-First-Search (BFS): • Shortest Path Distance • From source to each reachable vertex • Record during traversal • Foundation of many “shortest path” algorithms Time: O(|V| + |E|) adj list O(|V|2) adj matrix • predecessor subgraph = forest of spanning trees • Depth-First-Search (DFS): • Encountering, finishing times • “well-formed” nested (( )( ) ) structure • Every edge of undirected G is either a tree edge or a back edge • EdgeColor of vertex when first tested determines edge type See 91.404 DFS/BFS slide show See DFS, BFS Handout for PseudoCode

8. A Tree Edge Back Edge C A B Tree Edge B Tree Edge F Tree Edge E F E Cross Edge Tree Edge D C D Elementary Graph Algorithms:DFS, BFS • Review problem: TRUE or FALSE? • The tree shown below on the right can be a DFS tree for some adjacency list representation of the graph shown below on the left.

9. Elementary Graph Algorithms:Topological Sort • forDirected, Acyclic Graph (DAG) • G=(V,E) TOPOLOGICAL-SORT(G) 1 DFS(G) computes “finishing times” for each vertex 2 as each vertex is finished, insert it onto front of list 3 return list • Produces linear ordering of vertices. • For edge (u,v), u is ordered before v. See also 91.404 DFS/BFS slide show source: 91.503 textbook Cormen et al.

10. Invariant: Minimum weight spanning forest • Produces minimum weight tree of edges that includes every vertex. 2 4 3 Becomes single tree at end 6 5 1 1 7 Invariant: Minimum weight tree A 6 B 8 4 G 2 Spans all vertices at end E F D C Minimum Spanning Tree:Greedy Algorithms Time: O(|E|lg|E|) given fast FIND-SET, UNION Time: O(|E|lg|V|) = O(|E|lg|E|) slightly faster with fast priority queue • forUndirected, Connected, Weighted Graph • G=(V,E) source: 91.503 textbook Cormen et al.

11. 2 A B 4 3 G 6 5 1 1 E 7 6 F 8 4 D C 2 Minimum Spanning Trees • Review problem: • For the undirected, weighted graph below, show 2 different Minimum Spanning Trees. Draw each using one of the 2 graph copies below. Thicken an edge to make it part of a spanning tree. What is the sum of the edge weights for each of your Minimum Spanning Trees?

12. Shortest Paths Chapters 24 & 25

13. for unweighted,undirected graph G=(V,E) Time: O(|V| + |E|) adj list O(|V|2) adj matrix BFS Solution: BFS starting at u. Stop at v. Source/Sink Shortest Path Problem: Given 2 vertices u, v, find the shortest path in G from u to v. Solution: BFS starting at u. Full BFS tree. Problem: Given a vertex u, find the shortest path in G from u to each vertex. Single-Source Shortest Paths Solution: For each u: BFS starting at u; full BFS tree. Problem: Find the shortest path in G from each vertex u to each vertex v. All-Pairs Shortest Paths BFS as a Basis for Some Shortest Path Algorithms Time: O(|V|(|V| + |E|)) adj list O(|V|3) adj matrix but for weighted, directed graphs… source: based on Sedgewick, Graph Algorithms

14. Shortest Path Applications for weighted,directed graph G=(V,E) • Road maps • Airline routes • Telecommunications network routing • VLSI design routing Weight ~ Cost ~ Distance source: based on Sedgewick, Graph Algorithms

15. .45 .45 .83 .83 .45 .83 .38 .1 .1 .1 .1 .1 .1 .51 .51 .41 .41 Shortest Path Trees Shortest Path Tree gives shortest path from root to each other vertex shortest path need not be unique .99 .83 .1 .45 .21 .1 .51 .38 .36 .41 .5 source: Sedgewick, Graph Algorithms

16. Shortest Path Trees 3 as root for reverse graph .41 .29 .41 .29 .29 .45 .21 .21 .32 .51 .36 .32 .38 .32 .36 .50 st = Spanning Tree reverse edges have same weight as forward ones predecessor vertex in tree Shortest Path Tree is a spanning tree. prelude to compact representation, except that uses next vertex, not predecessor source: Sedgewick, Graph Algorithms

17. pux pyv pxy u x y v u y v x Shortest Path Principles: Opotimal Substructure • Lemma: Any subpath of a shortest path is a shortest path. • Proof: Cut-and-paste argument. pux pyv p’xy source: 91.503 textbook Cormen et al.

18. 2 A B 4 3 G 6 5 1 1 E 7 6 F 8 4 D C 2 Shortest Path Principles: Relaxation • “Relax” a constraint to try to improve solution • Relaxation of an Edge (u,v): • test if shortest path to v [found so far] can be improved by going through u source: 91.503 textbook Cormen et al.

19. 2 A 4 3 B 6 5 1 1 G 7 6 F E 8 4 C 2 D Shortest Path Principles (cont.) for weighted,directed graph G=(V,E) with no negative-weight cycles shortest path weight shortest path weight estimate source: 91.503 textbook Cormen et al.

20. Single-Source Shortest Paths Chapter 24

21. Single Source Shortest Paths Bellman-Ford for (negative)weighted,directed graph G=(V,E) with no negative-weight cycles weights source why this upper bound? Time is in O(|V||E|) update d(v) if d(u)+w(u,v) < d(v) detect negative-weight cycle source: 91.503 textbook Cormen et al.

22. Bellman-Ford for (negative)weighted,directed graph G=(V,E) with no negative-weight cycles source: 91.503 textbook Cormen et al.

23. 1 5 10 1 8 3 2 4 3 1 1 3 1 4 6 6 5 2 Single Source Shortest Paths: Dijkstra’s Algorithm Dijkstra’s algorithm solves problem efficiently for the case in which all weights are nonnegative (as in the example graph). Dijkstra’s algorithm maintains a set S of vertices whose final shortest path weights have already been determined. It also maintains, for each vertex v not in S, an upper bound d[v] on the weight of a shortest path from source s to v. The algorithm repeatedly selects the vertex ueV – S with minimum bound d[u], inserts u into S, and relaxes all edges leaving u (determines if passing through u makes it “faster” to get to a vertex adjacent to u).

24. Single Source Shortest Paths: Dijkstra’s Algorithm for (nonnegative)weighted,directed graph G=(V,E) implicit DECREASE-KEY source: 91.503 textbook Cormen et al.

25. 2 A 4 3 B 6 5 G 1 1 7 6 F E 8 4 C 2 D Single Source Shortest Paths Dijkstra’s Algorithm • Review problem: • For the directed, weighted graph below, find the shortest path that begins at vertex A and ends at vertex F. List the vertices in the order that they appear on that path. What is the sum of the edge weights of that path? Why can’t Dijkstra’s algorithm handle negative-weight edges?

26. Single Source Shortest Paths Dijkstra’s Algorithm for (nonnegative)weighted,directed graph G=(V,E) Fibonacci Heap (Ch. 20) O(VlgV + E) amortized analysis PFS = Priority-First Search = generalize graph search with priority queue to determine next step sources: Sedgewick, Graph Algorithms & 91.503 textbook Cormen et al.

27. All-Pairs Shortest Paths Chapter 25

28. Transitive Closure (Matrix):Unweighted, Directed Graph “self-loops” added for algorithmic purposes Transitive Closure concepts will be useful for All-Pairs Shortest Path calculation in directed, weighted graphs G Transitive Closure Graph contains edge (u,v) if there exists a directed path in G from u to v. source: Sedgewick, Graph Algorithms

29. Transitive Closure (Matrix) G2 G “self-loops” added for algorithmic purposes G G2 Boolean Matrix Product: and, or replace *,+ source: Sedgewick, Graph Algorithms

30. Algorithm 3: [Warshall] for i 0 to |V|-1 for s 0 to |V|-1 for t 0 to |V|-1 if G[s][i] and G[i][t] then G[s][t] 1 Transitive Closure (Matrix) G why this upper limit? Algorithm 1: Find G, G2 , G3 ,..., G|V-1| Time: O(|V|4) G2 Algorithm 2: Find G, G2 , G4 ,..., G|V| Time: O(|V|3lg|V|) Time: O(|V|3) G3 G4 source: Sedgewick, Graph Algorithms

31. Transitive Closure (Matrix) Warshall good for dense graphs source: Sedgewick, Graph Algorithms

32. Transitive Closure (Matrix) Warshall • Correctness by Induction on i: • Inductive Hypothesis: ith iteration of loop sets • G[s][t] to 1 iff there’s a directed path from s to t • with (internal) indices at most i. • Inductive Step for i+1 (sketch):2 cases for path <s…t> • internal indices at most i • - covered by inductive hypothesis in prior iteration so • G[s][t] already set to 1 • an internal index exceeds i (= i+1) • - G[s][i+1], G[i+1][t] set in a prior iteration so • G[s][t] set to 1 in current iteration source: Sedgewick, Graph Algorithms

33. All Shortest Paths .41 .29 .29 .45 .21 .51 .32 .38 .32 .36 .50 source: Sedgewick, Graph Algorithms

34. All Shortest Paths (Compact) Total distance of shortest path Shortest Paths .41 .29 .29 .45 .21 .32 .51 .38 .32 .36 .50 Entry s,t gives next vertex on shortest path from s to t. source: Sedgewick, Graph Algorithms

35. All Shortest Paths In a Network Shortest Path Trees for reverse graph source: Sedgewick, Graph Algorithms

36. All-Pairs Shortest Paths for (negative)weighted,directed graph G=(V,E) with no negative-weight cycles similar to Transitive Closure Algorithm 1 Time: O(|V|4) Note: D here is replaced by L in new edition source: 91.503 textbook Cormen et al.

37. All-Pairs Shortest Paths similar to Transitive Closure Algorithm 2 Time: O(|V|3lg|V|) Note: D here is replaced by L in new edition source: 91.503 textbook Cormen et al.

38. All-Pairs Shortest Paths similar to Transitive Closure Algorithm 3 [Warshall] Can have negative-weight edges Time: O(|V|3) How can output be used to detect a negative-weight cycle? Note: D here is replaced by L in new edition source: 91.503 textbook Cormen et al.

39. Food for thought… • What does the following matrix (the nxn form of it) used in shortest-path algorithms correspond to in regular matrix multiplication? Note: D here is replaced by L in new edition

40. Shortest Path Algorithms source: Sedgewick, Graph Algorithms