Exploring Shortest Path Algorithms in Weighted Graphs

1. CS 361 – Chapter 14 • Weighted graph • How would you represent? • Shortest path algorithms from 1 vertex to any other: • General • Handling negative weight • Acyclic algorithm • All pairs shortest paths • Minimum spanning tree

2. Dijkstra’s algorithm • How do you find the shortest path in a weighted graph? • A greedy algorithm found by Edsger Dijkstra in 1959. • Can you find the shortest path in this example? 4 7 9 6 8 7 4 3 2 3 1 6

3. A • Let’s say we want to go from “A” to “Z”. • The idea is to label each vertex with a number – its best known distance from A. As we work, we may find a cheaper distance, until we “mark” or finalize the vertex. • Label A with 0, and mark A. • Label A’s neighbors with their distances from A. • Find the lowest unmarked vertex, and mark it. Let’s call this vertex “B”. • Recalculate distances for B’s neighbors via B. Some of these neighbors may now have a shorter known distance. • Repeat 3 and 4 until you mark Z. 4 7 2 B C 3 4 Z

4. A • First, we label A with 0. Mark A. • The neighbors of A are B and C. Label B = 4 and C = 7. • Now, the unmarked vertices are B=4 and C=7. The lowest is B. • Mark B, and recalculate B’s neighbors via B. The neighbors are C and Z. • If we go to C via B, the total distance is 4 + 2 = 6. This is better than the old distance of 7. So, re-label C = 6. • If we go to Z via B, the total distance is 4 + 3 = 7. • Now, the unmarked vertices are C=6 and Z = 7. C is lowest. • Recalculate C’s neighbors… • … Finally, Z becomes marked. 4 7 2 B C 3 4 Z

5. A • For clarification: The idea is to label each vertex with a number: its best known distance from A. As we work, we may find a cheaper distance, until we “mark” or finalize the vertex. • When you mark a vertex and look to recalculate distances to its neighbors: • We don’t need to recalculate a distance for a vertex already marked. So, only consider unmarked neighbors. • We only update (“relax”) a vertex’s distance if it is an improvement: if it’s shorter than what the distance already was. • Practice with the 9 vertex example. 4 7 2 B C 3 4 Z

6. Variants of problem • Dijkstra’s algorithm works if graph also directed (one-way streets). • But not if there are negative weights! • Greedy algorithm can’t predict future negative number to add. • Can’t just pad all edge values, because # of edges traversed should not factor into total distance. • In this problem, need to make sure there is no “negative cycle”. This would mean there is no solution. • 2nd algorithm: Bellman-Ford. • Finds shortest path in directed weighted graph, allowing for negative weights. • Tedious: for n-1 iterations, need to reconsider all edges in order to “relax” the vertices.

7. Bellman-Ford algorithm // One vertex identified s = source or origin. Label s as 0 and other vertices as  // representing "distance" for i = 1 to n-1: for each edge (u,v): see if this edge can improve label(v) // i.e. "relax" if necessary for each edge (u,v): if (label(v) > label(u) + weight(u,v)) return false! return true

8. Example • Practice Bellman-Ford algorithm on this simple example. Assume origin is A. We’re especially interested in distance to B. • What is the label on each vertex after each iteration? • Is there a negative cycle?

9. DAG shortest paths • Behind door #3 we have an elegant algorithm that finds shortest path as long as there is no directed cycle. • Can handle negative weights. • Relies on a topological ordering of vertices. • Spreads the disease according to this ordering. // Given s is the source or origin topologically sort the graph label(s) = 0 and label(all other vertices) =  for u = each vertex in graph: for v = each neighbor of u: see if (u,v) can improve label(v)

11. Weighted graphs (2) Weighted graphs, continued: • Single-source shortest paths √ • Dijkstra • Bellman-Ford (for negative weights) • DAG shortest path (relies on topological sort) • All-pairs shortest paths • (Distance table on a map/atlas) • Can be done in O(n3) time. • Works for directed/undirected, negative weights too. • Minimum spanning trees

12. Idea • A variant of Floyd-Warshall we saw earlier for transitive closure • For each vertex in graph, we make successive updates to a “distance” matrix • D0 matrix uses initial distances info: • D0 [ v, v ] = 0 • D0 [ u, v ] = adj [ u, v ] if there is an edge from u to v • D0 [ u, v ] =  if not • The adjacency matrix values might already have these  • Computing Dk from Dk – 1 • We’re considering effect of vertex # k. • Dk [ u, v ] = Dk – 1 [ u, v ] or D k – 1 [ u, k ] + D k – 1 [ k, v ] whichever is less.

13. Algorithm // Given adj matrix of weighted graph with n vertices // Assume that 0 entries are for vertex to itself, and // infinity means not adjacent Arbitrarily number the vertices 1 to n. for i = 1 to n: for j = 1 to n: D[0] [i][j] = adj[i][j] for k = 1 to n: for i = 1 to n: for j = 1 to n: D[k][i][j] = min(D[k-1][i][j], D[k-1][i][k]+D[k-1][k][j])

14. Example • Let’s observe what happens. It turns out we don’t need to consider all 125 cases. • k = 1, i = 1, j = 1 • To go from i=1 to j=1, do we save time going thru k=1? No! • k = 1, i = 1, j = 2 • To go from i=1 to j=2, do we save time going thru k=1? • If i = k, this isn’t going to help because “going thru k” is same as “going thru i” which adds 0 to our time.

15. Example (2) • k = 1, i = 2, j = 1 • To go from i=2 to j=1, do we save time going thru k=1? • If j = k, “going thru k” means “going thru j”. But we’re already at j, so adding 0 won’t help. • Another observation: • If i = j, then this distance is already 0. If we improve upon this, then we’d have a negative cycle! • Bottom line is to consider cases where i,j,k distinct. • k = 1, i = 2, j = 3 • Do we save time going from 2  1  3 ? No. • k = 1, i = 2, j = 4 • Do we save time going from 2  1  4 ? No.

16. Example (3) • Another observation: • If k is not reachable from i, then skip on to next value of i. • In this case, for k = 1, we don’t need to consider i = 2 or i = 3. • k = 1, i = 4, j = 5 • Do we save time going from 4  1  5? Yes! • So, D[1][4][5] equals this improved distance, rather than merely copying D[0][4][5]. • Etc. thru k = 5. • Note: Be careful with representing . We don’t want overflow with e.g.  + 2 !

17. Final answer • When algorithm terminates, we have 2-d array D[k], which gives shortest “distance” between any 2 vertices in graph. • For example, D[5][3][5] = 3. • It does not tell us how to get there! • What more should we compute in the algorithm?

18. Weighted graphs (3) Weighted graphs, continued: • Minimum spanning tree algorithms (greedy) • Boruvka • Kruskal • Prim • Minimum Steiner tree

19. Background • MST also known as the shortest network problem • Want to connect all vertices with minimum total length of edges. • Applications • Sources of oil need to be connected to pipelines. Want to minimize total mileage. • Private telecom networks are billed according to total mileage of the network. Client should not have to pay for phone company’s inefficiency. • Some Algorithms • O. Boruvka (1926) – published Slovak paper in obscure journal (first known solution) • J. B. Kruskal (1956) – AT&T Bell Labs • R. C. Prim (1957) – also from Bell Labs

20. Boruvka Create a set (connected component) for each vertex in the graph. repeat: for each set S: find the lightest edge (u,v) connecting a vertex u in S with a vertex v not in S. Add (u,v) to the spanning tree. Union v’s set into u’s set. until we have chosen n-1 edges.

21. Kruskal • Repeatedly add edges from low to high, until you have added n – 1 edges. • How do we avoid a cycle? • Consider the graph to be a forest of trees, initially with 1 vertex per tree. As we pick an edge, we union 2 trees together. When we’re done, there is just 1 tree. • When examining an edge (u, v): check to see that u and v are in different vertex sets. Create a set for each vertex in graph. Sort edges in ascending order of weight. for each edge (u, v): if u and v are in different sets add (u, v) to spanning tree Union v’s set into u’s set.

22. Prim • During algorithm, we have just 1 tree, with vertices either in or “not yet” in the tree. • Spread disease from current vertices of tree: add lightest edge adjacent to some vertex not yet in tree. // Let T = set of vertices accounted for Select one vertex to initially add to T. for i = 2 to n: (u,v) = shortest edge connecting a vertex u in T with a vertex v not in T. add (v,w) to spanning tree add w to T

23. Complexity • The algorithms are conceptually simple, but best performance requires attention to potential bottlenecks. Optimizations can be quite sophisticated! • Kruskal says • for each edge, see if they are in different sets •  algorithm to determine disjoint sets in log time. (?) • Optimizing Prim’s algorithm • Need to speed up the way of finding lightest edge connected to any vertex in the growing tree. • Can assign labels to vertices, and search for vertex with smallest label. • Priority queue  quick way to find smallest element • “Fibonacci” heap has better performance than binomial heap

24. Steiner tree • The “minimum Steiner tree” problem is similar to the minimum spanning tree. • Problem posed by Swiss mathematician Jacob Steiner. • We can find an even shorter spanning tree if we are permitted to add more vertices! • In particular, a vertex in the middle or between existing ones. • Ex. Consider 4 vertices of a square. • Minimum spanning tree has total length 3. • Minimum Steiner tree would put 2 vertices inside the square to reduce total length by about 9%. • No known general solution other than brute force! • Please listen/watch this video lecture by Ron Graham: http://www.archive.org/download/RonaldLG1988/RonaldLG1988.mpeg

25. Background info • Big numbers & infinity • Limits discovered by Gödel, Heisenberg, and Turing • Some problems are: • Impossible (i.e. undecidable). • Infeasible (NP-complete). • There are classifications even among these types of problems! • P vs. NP question