1 / 70

Discrete Optimization Lecture 2 – Part 1

Discrete Optimization Lecture 2 – Part 1. M. Pawan Kumar pawan.kumar@ecp.fr. Slides available online http:// cvn.ecp.fr /personnel/ pawan /. Time Complexity of Djikstra’s. Set d(v) = ∞ for all v  V. Set U = V. Set d(s) = 0. Set u = s. While u is not equal to t.

chace
Download Presentation

Discrete Optimization Lecture 2 – Part 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discrete OptimizationLecture 2 – Part 1 M. Pawan Kumar pawan.kumar@ecp.fr Slides available online http://cvn.ecp.fr/personnel/pawan/

  2. Time Complexity of Djikstra’s Set d(v) = ∞ for all v  V. Set U = V. Set d(s) = 0. Set u = s. While u is not equal to t Choose u = argminvU d(v) For each v such that (u,v)  A d(v) = min { d(v), d(u) + l(u,v)} End Set U = U \ {u} End

  3. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  4. Distribution Estimate a joint probability P(x) from the data P(x) = P(x0,x1,…,xn-1) = P(x0) P(x1|x0) … P(xn-1|x1,…,xn-2) Images courtesy of Pedro Felzenszwalb

  5. Known Tree Structure v0 • Distribution PT(x) vp(a) = “parent” of va v1 v2 v3 v4 v5 PT(x5|x3) PT(x4|x1) PT(x3|x0) PT(x2|x0) PT(x1|x0) PT(x0) Estimate PT(xa|xp(a)) = P(xa|xp(a))

  6. Known Tree Structure v0 • Distribution PT(x) vp(a) = “parent” of va v1 v2 v3 v4 v5 PT(x5|x2) PT(x4|x2) PT(x3|x2) PT(x2|x0) PT(x1|x2) PT(x0) Which tree? Estimate PT(xa|xp(a)) = P(xa|xp(a))

  7. Kullback-Leibler Divergence KL(P1 || P2) = -Σx P1(x) log P2(x) +ΣxP1(x) log P1(x) Constant KL(P1 || P2) ≥ 0 KL(P1 || P1) = 0 Substitute P1 = P and P2 = PT. Minimize KL(P || PT)

  8. Estimating the Tree Structure min -Σx P(x) log PT(x)

  9. Estimating the Tree Structure min -Σx P(x) log ΠaPT(xa|xp(a))

  10. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa|xp(a))

  11. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a)) • PT(xp(a))

  12. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a))P(xa) • PT(xp(a))P(xa)

  13. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a)) • PT(xp(a))P(xa) • -Σx P(x) Σalog P(xa)

  14. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a)) • PT(xp(a))P(xa) • -Σa ΣxaP(xa) log P(xa) Independent of the tree structure

  15. Estimating the Tree Structure min -Σa ΣxaΣxp(a)P(xa,xp(a)) log PT(xa,xp(a)) • PT(xp(a))P(xa) min -Σa l(a,p(a)) Mutual Information

  16. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  17. Undirected Connected Simple Graph v0 G = (V, E) 2 1 v1 v4 l: E R 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4

  18. Spanning Tree v0 G = (V, E) 2 1 v1 v4 T = (V, ET) 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 ET is a subset of E Graph T is a tree

  19. Weight of a Spanning Tree v0 G = (V, E) 2 1 v1 v4 T = (V, ET) 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 w(T) = Sum of the length of all edges in ET w(T) = 2+1-1-5+1-2 = -4

  20. Minimum Spanning Tree Problem v0 G = (V, E) 2 1 v1 v4 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 T* = argminT w(T) Find a tree with the minimum weight

  21. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  22. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Start with a forest where every vertex is a tree

  23. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  24. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  25. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 YES Add the edge to the forest Does this edge connect two different trees?

  26. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  27. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  28. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  29. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  30. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  31. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  32. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  33. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  34. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 NO Discard the edge Does this edge connect two different trees?

  35. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  36. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  37. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  38. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  39. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  40. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  41. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  42. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  43. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 1 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  44. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 1 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Spanning Tree !!! Algorithm terminates.

  45. Summary Given G = (V, E). Define T = (V, {}) and S = E. While T is not a spanning tree Select the minimum length edge e from S If e connects two different trees Add e to T End Remove e from S End

  46. Time Complexity O(mlog(m)) where m = |E|

  47. Proof Sketch Let ET(t) be the edges added by iteration t There exists an MST T containing all edges in ET(t). Trivial for t = 0. Assume it is true for t Edge e is added at iteration t+1. ET(t+1) = ET(t) U e If T does not contain e, adding e to T creates a cycle Remove the edge from the cycle that is not in ET(t+1) Resulting spanning tree T’ must have minimum weight

  48. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  49. Prim’s Method v0 v0 2 1 v1 v4 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 Initialize T = (VT, {}) where VT = {v0}

  50. Prim’s Method v0 v0 2 1 v1 v4 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 Choose (u*,v*) = min l(u,v), u  VT, v  V \ VT

More Related