Algorithms (2IL15) – Lecture 6 ALL-PAIRS SHORTEST PATHS

1. Algorithms (2IL15) – Lecture 6 ALL-PAIRS SHORTEST PATHS w1,k … ai,1 … ai,n = bi,k wn,k

2. Previous lecture: Single-Source Shortest Paths Dijkstra’s algorithm • works only for non-negative edge weights • running time Θ ( |V | log |V | + |E| ) Bellman-Ford algorithm • can handle negative edge weights, works even for negative-weight cycles • running time Θ ( |V | ∙ |E| ) This week: the All-Pairs Shortest-Path Problem

3. All-Pairs Shortest Paths If we only have non-negative edge-weights: Run Dijkstra’s algorithm |V| times, once with each vertex as source • running time Θ ( |V2| log |V | + |V| ∙ |E| ) What to do if we also have negative edge weights?

4. adjacency-matrix representation weighted, directed graph 1 2 3 4 5 6 7 1 2 3 4 5 6 7 v4 − 2 v3 1 v1 1 2 v5 2 v6 4 2.3 v2 v7 M [ i,j ] = 0 if i = j w(vi,vj)if i ≠ j and edge (vi,vj) exists; w(vi,vj) is weight of (vi,vj) if edge (vi,vj) does not exist 8

5. Warm-up/Recap: Bellman-Ford via dynamic programming

6. 5 steps in designing dynamic-programming algorithms • define subproblems [#subproblems] • guess first choice [#choices] • give recurrence for the value of an optimal solution [time/subproblem treating recursive calls as (1))] • define subproblem in terms of a few parameters • define variable m[..] = value of optimal solution for subproblem • relate subproblems by giving recurrence for m[..] • algorithm: fill in table for m [..] in suitable order (or recurse & memoize) • solve original problem Running time: #subproblems * time/subproblem Correctness: (i) correctness of recurrence: relate OPT to recurrence (ii) correctness of algorithm: induction using (i)

7. shortest path from s to vi with m edges Single-Source Shortest Paths:structure of optimal solution V = { v1, v2, … , vn } s = v1 Subproblem: For each vertex vicompute shortest path from s to vi with at most m edges Define variable: L(i,m) = length of shortest path from s to vi with at most m edges Recursive formula: shortest path from s to vj with m−1 edges vj vi v1 if i =1 and m = 0 L(i,m) = if 1 < i ≤ n −1 and m = 0 if 0 < m ≤ n −1 0 Note: here adjacency matrix, but Bellman-Ford uses adjacency list 8 min { L(j,m−1) + w(vj, vi) } 1≤j≤n

8. 0 8 8 8 8 8 8 8 8 8 0 if i =1 and m = 0 L(i,m) = if 1 < i ≤ n −1 and m = 0 min { L(j,m−1) + w(vj, vi) } if 0 < m ≤ n −1 8 Single-Source Shortest Paths:dynamic-programming solution Recursive formula: 1≤j≤n m 0 n-1 1 solution to original problem is in last column i n

9. 0 if i =1 and m = 0 L(i,m) = if 1 < i ≤ n −1 and m = 0 min { L(j,m−1) + w(vj, vi) } if 0 < m ≤ n −1 8 Single-Source Shortest Paths:dynamic-programming solution Recursive formula: 1≤j≤n SSSP-DynamicProgramming ( G,s ) // G = (V,E)withV = { v1, v2, … , vn } and s = v1 • L [1,0 ] ← 0 • for i ← 2 to n do L[i,0] ← • for m ← 1 to n −1 • do for i ← 1 to n • do L[i,m ] ← min { L(j,m−1) + w(vj, vi) } after algorithm: L[i,n-1] = δ (v1,vi ) for all i running time: storage: Note: Bellman-Ford actually runs in O(|V||E|) time using O(|V|) space. O(n3) O(n2) 8 1≤j≤n How to check if a negative-weight cycle is reachable?

10. shortest path from vi to vk with m edges Now adapt to all-pairs shortest paths V = { v1, v2, … , vn } Subproblem: For each pair vi ,vkcompute shortest path from vito vkwith at most m edges Define variable: L(i,k,m) = length of shortest path from vito vkwith at most m edges Recursive formula: shortest path from vi to vj with m−1 edges vj vk vi 0 if i = k and m = 0 L(i,k,m) = if i ≠ k and m = 0 min { L(i,j,m−1) + w(vj, vk) } if 0 < m ≤ n −1 8 1≤j≤n

11. 0 if i = k and m = 0 L(i,k,m) = if i ≠ k and m = 0 min { L(i,j,m−1) + w(vj, vk) } if 0 < m ≤ n −1 8 All-Pairs Shortest Paths:dynamic-programming solution Recursive formula: 1≤j≤n Slow-All-Pairs-Shortest-Paths ( G,s ) • for i ← 1 to n • do for k ← 1 to n • do L [i,k,0 ] ← • for i ← 1 to n do L [i,i,0 ] ← 0 • for m ← 1 to n −1 • do for i ← 1 to n • do for k ← 1 to n • doL[i,k,m ] ← min { L(i,j,m−1) + w(vj, vk) } running time: storage: O(n4) O(n3) 8 NB: still need to check for negative-weight cycles 1≤j≤n

12. Extend-Shortest-Paths ( L,W ) • Let B be a new n x n matrix • for i ← 1 to n • do for k ← 1 to n • doB [i,k ] ← min { L(i,j) + w(vj, vk) } • returnB 1≤j≤n L(i,k,m) = w(vj, vk) if m = 1 min { L(i,j,m−1) + w(vj, vk) } if 1 < m ≤ n −1 1≤j≤n All-Pairs Shortest Paths:slightly different formulation New recursive formula: Slow-All-Pairs-Shortest-Paths ( G,s ) • L ← W // W = matrix of edge weights: W [i,j ] = w(vj, vj) • for m ← 2 to n −1 • do L ← Extend-Shortest-Paths ( L, W)

13. bi,k = min ( ai,k + wi,k ) 1≤j≤n L(i,k,m) = w(vj, vk) if m = 1 min { L(i,j,m−1) + w(vj, vk) } if 1 < m ≤ n −1 1≤j≤n All-Pairs Shortest Paths:relation to matrix multiplication Recursive formula: A = matrix with ai,k = L[i,k,m −1] = min length of path with m−1 edges from vi to vk W = matrix such that wi,k = w(vi, vk) = weight of edge w(vi, vk) B = matrix with bi,k = L[i,k,m] = min length of path with m edges from vi to vk w1,k … ai,1 … ai,n = bi,k wn,k replacing “min” by “∑” and “+” by “∙” gives normal matrix multiplication

14. is associative Let’s change notation for the matrices: D(m) = matrix with di,k = L[i,k,m] = min length of path with m edges from vi to vk W = matrix such that wi,k = w(vi, vk) = weight of edge w(vi, vk) Then D(m) = D(m-1) W = ( D(m-2) W ) W = D(1) W W … W = W m m−1 times

15. Conclusion • Solving the All-Pairs Shortest Paths Problem is equivalent to computing the matrix product Wn-1 with the -multiplication operator. • Extend-Shortest-Paths (L,W) computes matrix “product” LW We can use this insight to speed up the algorithm: • repeated squaring: W → W2 = W W → W4 = W2 W2 → … Extend-Shortest-Paths • W m = W n-1 for all m ≥ n−1 if there are no negative-weight cycles

16. New algorithm Faster-All-Pairs-Shortest-Paths ( W ) \\ W is matrix with edge weights • L ← W; n ← number of rows of W // n = |V | • m ← 1 • whilem < n-1 • do // Invariant: L= Wm • L ← Extend-Shortest-Paths (L,L) // L ← L L • m ← 2m • returnL running time = O ( n3 ) x number of times line 5 is executed = O ( n3 log n ) … still need to check for negative-weight cycles.

17. shortest path from vi to vk with m edges shortest path from vi to vj with m−1 edges vj vk vi Theorem: The All-Pairs Shortest-Paths problem for a graph G=(V,E) with (possibly negative) edge weights can be solved in O( |V |3 log |V |) time. PS There is a different algorithm – the Floyd-Warshall algorithm – that is also based on dynamic programming, but that runs in O( |V |3 ) time. shortest path from vi to vk only using v1 to vm on the way blackboard shortest path from vi to vj only using v1 to vm-1 vm vk vi shortest path from vj to vk only using v1 to vm-1

18. An alternative approach: Johnson’s algorithm Suppose we want to solve a certain graph problem • we have an algorithm ALG that only works for certain types of graphs • we want to develop an algorithm for another type of graph Possible approaches • try to adapt ALG • try to modify the input graph G into a different graph Gsuch that • ALG can be applied to G • we can easily compute the solution for G from the solution for G

19. Recall: If we only have non-negative edge-weights, we can run Dijkstra’s algorithm |V| times, once with each vertex as source • running time Θ ( |V2| log |V | + |V| ∙ |E| ) This is O( |V|3) in the worst case, so faster than previous approach. Idea: modify G so that all weights become non-negative, then use approach above • need to add something to the edge weights to make them non-negative • but shortest paths should stay the same

20. R G=(V,E) weighted graph h: V → function assigning a real number to each vertex v in V G is same graph as G, but with edge weights w(u,v) = w(u,v) + h(u) – h(v) Lemma (i) any shortest path in G is a shortest path inG, and vice versa (ii) G has negative-weight cycle iff G has negative-weight cycle Proof. Consider any path v1,v2,…,vk in G length of path in G = (length of path in G) + h(v1) – h(vk). w(v2,v3) + h(v2) – h(v3) … v1 v2 v3 vk-1 vk w(vk-1,vk) + h(vk-1) – h(vk) w(v1,v2) + h(v1) – h(v2)

21. R G=(V,E) weighted graph h: V → function assigning a real number to each vertex v in V G is same graph as G, but with edge weights w(u,v) = w(u,v) + h(u) – h(v) Lemma (i) any shortest path in G is a shortest path inG, and vice versa (ii) G has negative-weight cycle iff G has negative-weight cycle need to find function h that makes all edge weights non-negative Observation • w(u,v) non-negative h(v) – h(u) ≤ w(u,v) • Let s be any vertex that can reach u. Then δ(s,v) ≤ δ(s,u) + w(u,v)

22. v4 − 2 v3 1 v1 1 2 v5 2 v6 s 4 2.3 v2 v7 need to find function h that makes all edge weights non-negative Observation • w(u,v) non-negative h(u) – h(v) ≥ w(u,v) • Let s be any vertex that can reach u. Then δ(s,v) ≤ δ(s,u) + w(u,v) Idea: compute δ(s,u) for all u, for suitable source s; set h(u) = δ(s,u) 0 0 add extra vertex s, with zero-weight edge to all other vertices 0 0 0 0 0

23. Johnson (G,w) // Modify G into graph G* by adding vertex s with edges to all other vertices • V* ← V U {s} • E* ← E U { (s,u) : u in V }; all new edges (s,u) get weight 0 // Use G* to transform G to graph G that has no negative edge weights • Run Bellman-Ford(G*,s) to compute distances δ(s,u) for all u in V • if Bellman-Ford reports that G* has negative-weight cycle • then report that G has negative-weight cycle • else Let G=(V,E) be the same graph as G, but with edge weights w(u,v) = w(u,v) + δ(s,u) − δ(s,v) \\ and use G to solve the problem • for each vertex u in V \\ compute distances to all other vertices • do Run Dijkstra(G,u) to compute distances δ(u,v) • For all v, set δ(u,v) ← δ(u,v) − δ(s,u) + δ(s,v)

24. Theorem: The All-Pairs Shortest-Paths problem for a graph G=(V,E) with (possibly negative) edge weights can be solved in O( |V |2 log |V | + |V|∙|E | ) time.

25. All-Pairs Shortest Paths: Summary Dynamic-programming algorithm • running time Θ ( |V |4 ) • connection to matrix-multiplication • improved version (repeated squaring) runs in Θ ( |V |3 log |V | ) time • Floyd-Warshall : different dynamic-programming algorithm running in Θ ( |V |3 ) and (also) very simple to implement Johnson’s algorithm: reweighting • modify graph to make all edge-weights non-negative • then run Dijkstra’s algorithm |V | times • running time Θ ( |V |2 log |V | + |V | ∙ |E| )