1 / 40

Notes

Notes. This set of slides (Handout #7) is missing the material on Reductions and Greed that was presented in class. Those slides are still under construction, and will be posted as Handout #6. The quiz on Thursday of 8 th week will cover Greed and Dynamic Programming, and through HW #4.

fontaine-callum
Download Presentation

Notes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Notes • This set of slides (Handout #7) is missing the material on Reductions and Greed that was presented in class. Those slides are still under construction, and will be posted as Handout #6. • The quiz on Thursday of 8th week will cover Greed and Dynamic Programming, and through HW #4

  2. Topological Sorting • Prelude to shortest paths • Generic scheduling problem • Input: • Set of tasks {T1, T2, T3, …, Tn} • Example: getting dressed in the morning: put on shoes, socks, shirt, pants, belt, … • Set of dependencies {T1T2, T3 T4, T5 T1, …} • Example: must put on socks before shoes, pants before belt, … • Want: • ordering of tasks which is consistent with dependencies • Problem representation: Directed Acyclic Graph • Vertices = tasks; Directed Edges = dependencies • Acyclic: if $ cycle of dependencies, no solution possible

  3. Topological Sorting • TOP_SORT PROBLEM: Given a DAG G=(V,E) with |V|=n, assign labels 1,...,n to viÎ V s.t. if v has label k, all vertices reachable from v have labels > k • “Induction idea”: • Know how to label DAG’s with < n vertices • Claim: A DAG G always has some vertex with indegree = 0 • Take an arbitrary vertex v. If v doesn’t have indegree = 0, traverse any incoming edge to reach a predecessor of v. If this vertex doesn’t have indegree = 0, traverse any incoming edge to reach a predecessor, etc. • Eventually, this process will either identify a vertex with indegree = 0, or else reach a vertex that has been reached previously (a contradiction, given that G is acyclic). • “Inductive approach”: Find v with indegree(v) = 0, give it lowest available label, then delete v (and incident edges), update degrees of remaining vertices, and repeat

  4. Dynamic Programming Key Phrases • As we discuss shortest paths, and then the dynamic programming approach, keep in mind the following “key phrases” • “Principle of Optimality”: Any subsolution of an optimal solution is itself an optimal solution • “Overlapping Subproblems”: Can exploit a polynomially bounded number of possible subproblems • “Bottom-Up D/Q” / “Table (cache) of Subproblem Solutions”: avoid recomputation by tabulating subproblem solutions • “Relaxation / Successive Approximation”: Often, the DP approach makes multiple passes, each time solving a less restricted version of the original problem instance • Eventually, solve completely unrestricted = original problem instance

  5. Single-Source Shortest Paths • Given G=(V,E) a directed graph, L : E  Â+ a length function, and a distinguished source s Î V. Find the shortest path in G from s to each vi Î V, vi ¹ s. • Let L*(i,j) = length of SP from vi to vj • Lemma: Suppose S ÌV contains s = vi and L*(1,w) is known "w Î S. For viÏ S: • Let D(vi) = minw Î SL*(1,w) + L(w,i) (*) L*(1,w) = path length, with path restricted to vertices of S L(w,i) =edge length • Let vv minimize D(vi) over all nodes viÏ S (**) Then L*(1,v) = D(vv). • Notation: D(v) = length of the 1-to-v SP that uses only vertices of S (except for v). (D(v) is not necessarily the same as L*(1,v), because the path is restricted!)

  6. Single-Source Shortest Paths • Proof of Lemma: To prove equality, prove £ and ³ • L*(1,v) £ D(v) is obvious because D(v) is the minimum length of a restricted path, while L*(1,v) is unrestricted • Let L*(1,v) be the shortest path s=v1, v2, …, vr = v from the source v1 to v. Let vj be the first vertex in this SP that is not in S • Then: L*(1,v) = L*(1, vj-1) + L(vj-1, vj) + L*(vj,v) // else, not a shortest path ³ D(vj) + L*(vj,v) // D(vj) minimizes over all w in S, including vj-1 ³ D(v) + L*(vj,v) // because both vj and vvÏ S, but vv chosen first ³ D(v) // since L*(vj,v) ³0. Lemma  Dijkstra’s Algorithm

  7. A Fact About Shortest Paths • Triangle Inequality: (u,v)  (u,x) + (x,v) (shortest path distances induce a metric) u v x

  8. Shortest Path Formulations • Given a graph G=(V,E) and w: E   • (1 to 2) “s-t”: Find a shortest path from s to t • (1 to all) “single-source”: Find a shortest path from a source s to every other vertex v V • (All to all) “all-pairs”: Find a shortest path from every vertex to every other vertex • Weight of path <v[1],...,v[k]> =  w(v[i],v[i+1]) • Sometimes: no negative edges • Examples of “negative edges”: travel inducements, exothermic reactions in chemistry, unprofitable transactions in arbitrage, … • Always: no negative cycles • Makes the shortest-path problem well-defined <0

  9. Shortest Paths • First case: All edges have positive length • Length of (vi,vj ) edge = dij • Condition 1: dij > 0 • Condition 2: di + djk£ dik for some i,j,k • else shortest-path problem would be trivial • Observation 1: Length of a path > length of any of its subpaths • Observation 2: Any subpath of a shortest path is itself a shortest path Principle of Optimality • Observation 3: Any shortest path contains £ n-1 edges pigeonhole principle; assumes no negative cycles; n nodes total

  10. Shortest Paths • Scenario: All shortest paths from v0 = source to other nodes are ordered by increasing length: • |P1| £ |P2| £ … £ |Pn-1| • Index nodes accordingly • Algorithm: Find P1, then find P2, etc. • Q: How many edges are there in P1 ? • Exactly 1 edge, else can find a subpath that is shorter • Q: How many edges are there in Pk ? • At most k edges, else can find k (shorter) subpaths, which would contradict the definition of Pk • Observation 4: Pk contains £ k edges • To find P1 : only look at one-edge paths (min = P1) • To find P2 : only look at one- and two-edge paths • But, need only consider two-edge paths of form d01 + d1i • Else would have ³1 paths shorter than P2, a contradiction

  11. Another Presentation of Dijkstra’s Algorithm • Terminology • Permanent label: true SP distance from v0 to vi • Temporary label: restricted SP distance from v0 to vi (going through only existing permanently-labeled nodes) Permanently labeled nodes = set S in previous development • Dijkstra’s Algorithm 0. All vertices vi, i = 1,…, n-1, receive temporary labels li with value d0i LOOP: 1. Among all temporary labels, pick lk = minI li and change lk to lk* (i.e., make lk ‘s label permanent) // stop if no temporary labels left 2. Replace all temporary labels of vk ‘s neighbors, using li¬ min (li , lk* + dki)

  12. Prim’s Algorithm vs. Dijkstra’s Algorithm • Prim: Iteratively add edge eij to T, such that viÎT, vjÏ T, and dij is minimum • Dijkstra: Iteratively add edge eij to T, such that viÎT, vjÏ T, and li + dij is minimum • Both are building trees, in very similar ways! • Prim: Minimum Spanning Tree • Dijkstra: Shortest Path Tree • What kind of tree does the following algorithm build? • Prim-Dijkstra: Iteratively add edge eij to T, such that viÎT, vjÏ T, and c × li + dij is minimum // 0 £ c £ 1

  13. Bellman-Ford Algorithm • Idea: Successive Approximation / Relaxation • Find SP using £ 1 edges • Find SP using £ 2 edges • … • Find SP using £ n-1 edges  have true shortest paths • Let lj(k) denote shortest v0 – vj pathlength using £k edges • Then, li(1) = d0j" j = 1, …, n-1 // dij = ¥ if no i-j edge • In general, lj(k+1) = min { lj(k) , mini (li(k) + dij) } • lj(k) : don’t need k+1 arcs • mini (li(k) + dij) : view as length-k SP plus a single edge

  14. Bellman-Ford vs. Dijkstra 4 B A 8 3 1 S C 2 2 D Pass1 2 3 4 Label A 8 min(8, 3+4,  +1, 2+ ) = 7 min(7, 3+4, 4+1, 2+) = 5 min(5, 3+4, 4+1, 2+ )= 5 B 3 min(3, 8+4, +, 2+) = 3 min(3, 7+4, 4+, 2+) = 3 min(3, 5+4, 4+, 2+) = 3 C min(, 8+1, 3+, 2+2) = 4 min(4, 7+1, 3+, 2+2) = 4 min(4, 5+1, 3+, 2+2) = 4 D 2 min(2, 8+, 3+, +2) = 2 min(2, 7+, 3+, 4+2) = 2 min(2, 5+, 3+, 4+2) = 2

  15. Bellman-Ford vs. Dijkstra 4 B A 8 3 1 S C 2 2 D Pass1 2 3 4 Label A 8 min([8], 2+ ) = 8 min([8], 3+4) = 7 min([7], 4+1) = 5* B 3 min([3], 2+ ) = 3* C min([], 2+2) = 4 min(4, 3+) = 4* D2*

  16. Special Case: DAGs (24.2) • Longest-Path Problem: well-defined only when there are no cycles • DAG: topologically sort the vertices  labels v1, …, vn s.t. all edges directed from vi to vj, i < j • Let li denote longest v0 – vj pathlength • l0 = 0 • l1 = d01// dij = -¥ if no i-j edge • l2 = max(d01 + d12 , d02) • In general, lk = maxj<k (lj + djk) • Shortest pathlength in DAG: replace max by min, use dij = +¥ if no i-j edge

  17. DAG Shortest Paths Complexity (24.2) • Bellman-Ford = O(VE) • Topological sort O(V+E) (DFS) • Will never relax edges outof vertex v until have done all edges in to v • Runtime O(V+E) • Application: PERT (program evaluation and review technique) – critical path is the longest path through the DAG

  18. All-Pairs Shortest Paths (25.1-25.2) • Directed graph G = (V,E), weight E   • Goal: Create n  n matrix of SP distances (u,v) • Running Bellman-Ford once from each vertex O( ) = O( ) on dense graphs • Adjacency-matrix representation of graph: • n  n matrix W = (wij) of edge weights • assume wii = 0 i, SP to self has no edges, as long as there are no negative cycles

  19. Simple APSP Dynamic Programming (25.1) • dij(m) = weight of s-p from i to j with  m edges dij(0) = 0 if i = j and dij(0) =  if i  j dij(m) = mink{dik(m-1) + wkj} • Runtime = O( n4) n-1 passes, each computing n2 d’s in O(n) time  m-1 j i  m-1

  20. Matrix Multiplication (25.1) • Similar: C = A  B, two n  n matrices cij = k aik  bkj O(n3) operations • replacing: ‘‘ + ’’  ‘‘ min ’’ ‘‘  ’’  ‘‘ + ’’ • gives cij= mink {aik + bkj} • D(m) = D(m-1) ‘‘’’ W • identity matrix is D(0) • Cannot use Strassen’s because no subtraction • Time is still O(n  n3 ) = O(n4 ) • Repeated squaring: W2n = Wn Wn (addition chains) Compute W, W2 , W4 ,..., W2k , k = log n O(n3 log n)

  21. Floyd-Warshall Algorithm (26.2/25.2) • Also DP, but even faster (by another log n actor  O(n3)) • cij(m) = weight of SP from i to j with intermediate vertices in the set {1, 2, ..., m}  (i, j)= cij(n) • DP: compute cij(n) in terms of smaller cij(n-1) • cij(0) = wij • cij(m) = min {cij(m) , cim(m-1) + cmj(m-1) } intermediate nodes in {1, 2, ..., m} cmj(m-1) m cim(m-1) j i cij(m-1)

  22. Floyd-Warshall Algorithm (26.2/25.2) • Difference from previous: we do not check all possible intermediate vertices. • for m=1..n do for i=1..n do for j = 1..n do cij(m) = min {cij(m-1) , cim(m-1) + cmj(m-1) } • Runtime O(n3 ) • Transitive Closure G* of graph G: • (i,j)  G* iff  path from i to j in G • Adjacency matrix, elements on {0,1} • Floyd-Warshall with ‘‘ min ’’  ‘‘OR’’ , ‘‘+’’  ‘‘ AND ’’ • Runtime O(n3 ) • Useful in many problems

  23. BEGIN DIGRESSION (alternative presentation, can be skipped)

  24. CLRS Notation: Bellman-Ford (24.1) SSSP in General Graphs • Essentially a BFS based algorithm • Shortest paths (tree) easy to reconstruct for each v V do d[v]  ; d[s]  0 // initialization for i =1,...,|V|-1 do for each edge (u,v)  E do d[v]  min{d[v], d[u]+w(u,v)} // relaxation for each v V do if d[v]> d[u] + w(u,v) then no solution // negative cycle checking for each v V, d[v]= (s,v) // have true shortest paths

  25. Bellman-Ford Analysis (CLRS 24.1) • Runtime = O(VE) • Correctness • Lemma: d[v]  (s,v) • Initially true • Let d[v] = d[u] +w(u,v) • by triangle inequality, for first violation d[v] < (s,v)  (s,u)+w(u,v)  d(u)+w(u,v) • After |V|-1 passes all d values are ’s if there are no negative cycles • s  v[1]  v[2]  ...  v (some shortest path) • After i-th iteration d[s,v[i]] is correct and final

  26. Dijkstra Yet Again (CLRS 24.3) • Faster than Bellman-Ford because can exploit having only non-negative weights • Like BFS, but uses priority queue for each v V do d[v]  ; d[s]  0 S  ; Q  V While Q   do u  Extract-Min(Q) S  S + u for v adjacent to u do d[v]  min{d[v], d[u]+w(u,v)} (relaxation = Decrease-Key)

  27. Dijkstra Runtime (24.3) • Extract-Min executed |V| times • Decrease-Key executed |E| times • Time = |V|T(Extract-Min) // find+delete = O(log V)) + |E| T(Decrease-Key) // delete+add =O(log V)) Binary Heap = E  log V (30 years ago) Fibonacci Heap = E + V  log V (10 years ago) • Optimal time algorithm found 1 year ago; runs in time O(E) (Mikel Thorup)

  28. Dijkstra Correctness (24.3) • Same Lemma as for Bellman-Ford: d[v]  (s,v) • Thm: Whenever u is added to S, d[u] = (s,u) Proof: • Assume that u is the first vertex s.t. d[u] > (s,u) • Let y be first vertex  V-S that is on actual shortest s-u path  d[y] = (s,y) • For y’s predecessor x, d[x] = (s,x) • At the moment that we put x in S, d[y] gets value (s,y) • d[u] >(s,u) = (s,y) + (y,u) = d[y] + (y,u)  d[y] S u s y Q x

  29. END DIGRESSION (alternative presentation, can be skipped)

  30. Dynamic Programming (CLRS 15) • Third Major Paradigm so far • DQ, Greed are previous paradigms • Dynamic Programming = metatechnique (not a particular algorithm) • “Programming” refers to use of a “tableau” in the method (cf. “mathematical programming”), not to writing of code

  31. Longest Common Subsequence (CLRS 15.4) • Problem: Given x[1..m] and y[1..n], find LCS x: A B C B D A B  B C B A y: B D C A B A • Brute-force algorithm: • for every subsequence of x, check if it is in y • O(n2m ) time • 2m subsequences of x (each element is either in or out of the subsequence) • O(n) for scanning y with x-subsequence (m  n)

  32. Recurrent Formula for LCS (15.4) • Let c[i,j] = length of LCS of X[i]=x[1..i],Y[j]= y[1..j] • Then c[m,n] = length of LCS of x and y • Theorem: if x[i] = y[j] otherwise • Proof: x[i] = y[j]  LCS([X[i],Y[j]) = LCS(X[i-1],Y[j-1]) + x[i]

  33. DP Properties(15.3) • Any part of the optimal answer is also optimal • A subsequence of LCS(X,Y) is the LCS for some subsequences of X and Y. • Subproblems overlap • LCS(X[m],Y[n-1]) and LCS(X[m-1],Y[n]) have common subproblem LCS(X[m-1],Y[n-1]) • There are polynomially few subproblems in total = mn for LCS • Unlike divide and conquer

  34. DP (CLRS 15.3) • After computing solution of a subproblem, store in table • Time = O(mn) • When computing c[i,j] we need O(1) time if we have: • x[i], y[j] • c[i,j-1] • c[i-1,j] • c[i-1,j-1]

  35. DP Table for LCS y B D C A B A x A B C B D A B

  36. DP Table for LCS y B D C A B A x A B C B D A B 0 0 0 0 0 0 0 1 1 0 1 0 1 0 1 0 1 0 1

  37. DP Table for LCS y B D C A B A x A B C B D A B 0 0 0 0 0 0 0 1 1 0 1 1 0 1 1 0 1 2 0 1 0 1

  38. DP Table for LCS y B D C A B A x A B C B D A B 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 1 1 1 1 2 2 0 1 1 2 2 2 2 0 1 1 2 2 3 3 0 1 2 2 2 3 3 0 1 2 2 3 3 4 0 1 2 2 3 4 4

  39. Optimal Polygon Triangulation • Polygon has sides and vertices • Polygon is simple = not self-intersecting. • Polygon P is convex if any line segment with ends in P lies entirely in P • Triangulation of P is partition of P with chords into triangles. • Problem: Given a convexpolygon and weight function defined on triangles (e.g. the perimeter). Find triangulation of minimum weight (of minimum total length). • # of triangles: Always have n-2 triangles with n-3 chords

  40. Optimal Polygon Triangulation • Optimal sub-triangulation of optimal triangulation • Recurrent formula: t[i,j] = the weight of the optimal triangulation of the polygon <v[i-1],v[i],...,v[j]>; If i=j, then t[i,j]=0; else • Runtime O(n3) and space is O(n2) v[i] v[i] v[i-1] v[i-1] v[j] v[j] v[k] v[k]

More Related