Shortest Paths

1. Shortest Paths • Definitions • Single Source Algorithms • Bellman Ford • DAG shortest path algorithm • Dijkstra • All Pairs Algorithms • Using Single Source Algorithms • Matrix multiplication • Floyd-Warshall • Both of above use adjacency matrix representation and dynamic programming • Johnson’s algorithm • Uses adjacency list representation

2. Single Source Definition • Input • Weighted, connected directed graph G=(V,E) • Weight (length) function w on each edge e in E • Source node s in V • Task • Compute a shortest path from s to all nodes in V

3. All Pairs Definition • Input • Weighted, connected directed graph G=(V,E) • Weight (length) function w on each edge e in E • We will typically assume w is represented as a matrix • Task • Compute a shortest path from all nodes in V to all nodes in V

4. Comments • If edges are not weighted, then BFS works for single source problem • Optimal substructure • A shortest path between s and t contains other shortest paths within it • No known algorithm is better at finding a shortest path from s to a specific destination node t in G than finding the shortest path from s to all nodes in V

5. Negative weight edges • Negative weight edges can be allowed as long as there are no negative weight cycles • If there are negative weight cycles, then there cannot be a shortest path from s to any node t (why?) • If we disallow negative weight cycles, then there always is a shortest path that contains no cycles

6. Relaxation technique • For each vertex v, we maintain an upper bound d[v] on the weight of shortest path from s to v • d[v] initialized to infinity • Relaxing an edge (u,v) • Can we improve the shortest path to v by going through u? • If d[v] > d[u] + w(u,v), d[v] = d[u] + w(u,v) • This can be done in O(1) time

7. Bellman-Ford (G, w, s) Initialize-Single-Source(G,s) for (i=1 to V-1) for each edge (u,v) in E relax(u,v); for each edge (u,v) in E if d[v] > d[u] + w(u,v) return NEGATIVE WEIGHT CYCLE -5 -5 3 3 5 5 -1 7 s s 1 1 3 3 2 2 Bellman-Ford Algorithm

8. Running Time • for (i=1 to V-1) • for each edge (u,v) in E • relax(u,v); • The above takes (V-1)O(E) = O(VE) time • for each edge (u,v) in E • if d[v] > d[u] + w(u,v) • return NEGATIVE WEIGHT CYCLE • The above takes O(E) time

9. Proof of Correctness • If there is a shortest path from s to any node v, then d[v] will have this weight at end • Let p = (e1, e2, …, ek) = (v1, v2, v3, …, v,+1) be a shortest path from s to v • s = v1, v = vk+1, ei = (vi, vi+1) • At beginning of ith iteration, during which we relax edge ei, d[vi] must be correct, so at end of ith iteration, d[vi+1] will e correct • Iteration is the full sweep of relaxing all edges • Proof by induction

10. Negative weight cycle • for each edge (u,v) in E • if d[v] > d[u] + w(u,v) • return NEGATIVE WEIGHT CYCLE • If no neg weight cycle, d[v] ≤d[u] + w(u,v) for all (u,v) • If there is a negative weight cycle C, for some edge (u,v) on C, it must be the case that d[v] > d[u] + w(u,v). • Suppose this is not true for some neg. weight cycle C • sum these (d[v] ≤ d[u] + w(u,v)) all the way around C • We end up with sum(d[v]) ≤ sum(d[u]) + weight(C) • This is impossible unless weight(C) = 0 • But weight(C) is negative, so this cannot happen • Thus for some (u,v) on C, d[v] > d[u] + w(u,v)

11. DAG-SP (G, w, s) Initialize-Single-Source(G,s) Topologically sort vertices in G for each vertex u, taken in sorted order for each edge (u,v) in E relax(u,v); DAG shortest path algorithm 5 3 4 -4 s 5 1 3 8

12. Running time Improvement • O(V+E) for the topological sorting • We only do 1 relaxation for each edge: O(E) time • for each vertex u, taken in sorted order • for each edge (u,v) in E • relax(u,v); • Overall running time: O(V+E)

13. Proof of Correctness • If there is a shortest path from s to any node v, then d[v] will have this weight at end • Let p = (e1, e2, …, ek) = (v1, v2, v3, …, v,+1) be a shortest path from s to v • s = v1, v = vk+1, ei = (vi, vi+1) • Since we sort edges in topological order, we will process node vi (and edge ei) before processing later edges in the path.

14. Dijkstra (G, w, s) /* Assumption: all edge weights non-negative */ Initialize-Single-Source(G,s) Completed = {}; ToBeCompleted = V; While ToBeCompleted is not empty u =EXTRACT-MIN(ToBeCompleted); Completed += {u}; for each edge (u,v) relax(u,v); 4 15 5 7 s 1 3 2 Dijkstra’s Algorithm

15. Running Time Analysis • While ToBeCompleted is not empty • u =EXTRACT-MIN(ToBeCompleted); • Completed += {u}; • for each edge (u,v) relax(u,v); • Each edge relaxed at most once: O(E) • Need to decrease-key potentially once per edge • Need to extract-min once per node

16. Running Time Analysis cont’d • O(E) edge relaxations • Priority Queue operations • O(E) decrease key operations • O(V) extract-min operations • Three implementations of priority queues • Array: O(V2) time • decrease-key is O(1) and extract-min is O(V) • Binary heap: O(E log V) time assuming E >= V • decrease-key and extract-min are O(log V) • Fibonacci heap: O(V log V + E) time • decrease-key is O(1) amortized time and extract-min is O(log V)

17. Compare Dijkstra’s algorithm to Prim’s algorithm for MST • Dijsktra • Priority Queue operations • O(E) decrease key operations • O(V) extract-min operations • Prim • Priority Queue operations • O(E) decrease-key operations • O(V) extract-min operations • Is this a coincidence or is there something more here?

18. s u v Only nodes in S Proof of Correctness • Assume that Dijkstra’s algorithm fails to compute length of all shortest paths from s • Let v be the first node whose shortest path length is computed incorrectly • Let S be the set of nodes whose shortest paths were computed correctly by Dijkstra prior to adding v to the processed set of nodes. • Dijkstra’s algorithm has used the shortest path from s to v using only nodes in S when it added v to S. • The shortest path to v must include one node not in S • Let u be the first such node.

19. s u v Only nodes in S Proof of Correctness • The length of the shortest path to u must be at least that of the length of the path computed to v. • Why? • The length of the path from u to v must be < 0. • Why? • No path can have negative length since all edge weights are non-negative, and thus we have a contradiction.

20. Computing paths (not just distance) • Maintain for each node v a predecessor node p(v) • p(v) is initialized to be null • Whenever an edge (u,v) is relaxed such that d(v) improves, then p(v) can be set to be u • Paths can be generated from this data structure

21. All pairs algorithms using single source algorithms • Call a single source algorithm from each vertex s in V • O(V X) where X is the running time of the given algorithm • Dijkstra linear array: O(V3) • Dijkstra binary heap: O(VE log V) • Dijkstra Fibonacci heap: O(V2 log V + VE) • Bellman-Ford: O(V2 E) (negative weight edges)

22. Two adjacency matrix based algorithms • Matrix-multiplication based algorithm • Let Lm(i,j) denote the length of the shortest path from node i to node j using at most m edges • What is our desired result in terms of Lm(i,j)? • What is a recurrence relation for Lm(i,j)? • Floyd-Warshall algorithm • Let Lk(i,j) denote the length of the shortest path from node i to node j using only nodes within {1, …, k} as internal nodes. • What is our desired result in terms of Lk(i,j)? • What is a recurrence relation for Lk(i,j)?

23. Conceptual pictures Shortest path using at most 2m edges Try all possible nodes k i k j Shortest path using at most m edges Shortest path using at most m edges Shortest path using nodes 1 through k Shortest path using nodes 1 through k-1 i k j Shortest path using nodes 1 through k-1 OR Shortest path using nodes 1 through k-1

24. Running Times • Matrix-multiplication based algorithm • O(V3 log V) • log V executions of “matrix-matrix” multiplication • Not quite matrix-matrix multiplication but same running time • Floyd-Warshall algorithm • O(V3) • V iterations of an O(V2) update loop • The constant is very small, so this is a “fast” O(V3)

25. Johnson’s Algorithm • Key ideas • Reweight edge weights to eliminate negative weight edges AND preserve shortest paths • Use Bellman-Ford and Dijkstra’s algorithms as subroutines • Running time: O(V2 log V + VE) • Better than earlier algorithms for sparse graphs

26. Reweighting • Original edge weight is w(u,v) • New edge weight: • w’(u,v) = w(u,v) + h(u) – h(v) • h(v) is a function mapping vertices to real numbers • Key observation: • Let p be any path from node u to node v • w’(p) = w(p) + h(u) – h(v)

27. -5 3 4 7 1 3 -7 Computing vertex weights h(v) • Create a new graph G’ = (V’, E’) by • adding a new vertex s to V • adding edges (s,v) for all v in V with w(s,v) = 0 • Set h(v) to be the length of the shortest path from this new node s to node v • This is well-defined if G’ does not contain negative weight cycles • Note that h(v) ≤ h(u) + w(u,v) for all (u,v) in E’ • Thus, w’(u,v) = w(u,v) + h(u) – h(v) ≥ 0

28. Algorithm implementation • Run Bellman-Ford on G’ from new node s • If no negative weight cycle, then use h(v) values from Bellman-Ford • Now compute w’(u,v) for each edge (u,v) in E • Now run Dijkstra’s algorithm using w’ • Use each node as source node • Modify d[u,v] at end by adding h(v) and subtracting h(u) to get true path weight • Running time: • O(VE) [from one run of Bellman-Ford] + • O(V2 log V + VE) [from V runs of Dijkstra]