1 / 77

Discrete Optimization Lecture 3 – Part 1

Discrete Optimization Lecture 3 – Part 1. M. Pawan Kumar pawan.kumar@ecp.fr. Outline. Chow-Liu Tree Minimum Spanning Tree Problem Kruskal’s Method Prim’s Method. Object Detection. Images courtesy of Pedro Felzenszwalb. Graph-based Representation. H. T. L1. L2. L3. L4. H. A1. T.

arne
Download Presentation

Discrete Optimization Lecture 3 – Part 1

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discrete OptimizationLecture 3 – Part 1 M. Pawan Kumar pawan.kumar@ecp.fr

  2. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  3. Object Detection Images courtesy of Pedro Felzenszwalb

  4. Graph-based Representation H T L1 L2 L3 L4 H A1 T A2 L1 L2

  5. Unary Potentials Depend on the current image. An important problem in computer vision. 2 3 6 8 9 2 3 4 3 5 1 2 6 8 9 8 2 5 H

  6. Pairwise Potentials Head Torso Torso Head 1000 1 H T

  7. Pairwise Potentials How do we learn pairwise potentials from this data? How do we learn the structure of the tree? Images courtesy of Pedro Felzenszwalb

  8. Notation • Random variables V = {v0,v2,…,vn-1} • Labels L = {l0,l2,…,lh-1} • Labeling f: V L • Label for va = lf(a) = xa • Labeling is a vector x

  9. Distribution Estimate a joint probability P(x) from the data P(x) = P(x0,x2,…,xn-1) = P(x1) P(x2|x1) … P(xn-1|x1,…,xn-2) Images courtesy of Pedro Felzenszwalb

  10. Known Tree Structure v0 • Distribution PT(x) vp(a) = “parent” of va v1 v2 v3 v4 v5 PT(x5|x3) PT(x4|x1) PT(x3|x0) PT(x2|x0) PT(x1|x0) PT(x0) Estimate PT(xa|xp(a)) = P(xa|xp(a)) θap(a);xaxp(a)= - log PT(xa|xp(a))

  11. Known Tree Structure v0 • Distribution PT(x) vp(a) = “parent” of va v1 v2 v3 v4 v5 PT(x5|x2) PT(x4|x2) PT(x3|x2) PT(x2|x0) PT(x1|x2) PT(x0) Which tree? Estimate PT(xa|xp(a)) = P(xa|xp(a)) θap(a);xaxp(a)= - log PT(xa|xp(a))

  12. Kullback-Leibler Divergence KL(P1 || P2) = -Σx P1(x) log P2(x) +ΣxP1(x) log P1(x) Constant KL(P1 || P2) ≥ 0 KL(P1 || P1) = 0 Substitute P1 = P and P2 = PT. Minimize KL(P || PT)

  13. Estimating the Tree Structure min -Σx P(x) log PT(x)

  14. Estimating the Tree Structure min -Σx P(x) log ΠaPT(xa|xp(a))

  15. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa|xp(a))

  16. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a)) • PT(xp(a))

  17. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a))P(xa) • PT(xp(a))P(xa)

  18. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a)) • PT(xp(a))P(xa) • -Σx P(x) Σalog P(xa)

  19. Estimating the Tree Structure min -Σx P(x) Σalog PT(xa,xp(a)) • PT(xp(a))P(xa) • -Σa ΣxaP(xa) log P(xa) Independent of the tree structure

  20. Estimating the Tree Structure min -Σa ΣxaΣxp(a)P(xa,xp(a)) log PT(xa,xp(a)) • PT(xp(a))P(xa) min -Σa l(a,p(a)) Mutual Information

  21. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  22. Undirected Connected Simple Graph v0 G = (V, E) 2 1 v1 v4 l: E R 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4

  23. Spanning Tree v0 G = (V, E) 2 1 v1 v4 T = (V, ET) 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 ET is a subset of E Graph T is a tree

  24. Weight of a Spanning Tree v0 G = (V, E) 2 1 v1 v4 T = (V, ET) 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 w(T) = Sum of the length of all edges in ET w(T) = 2+1-1-5+1-2 = -4

  25. Minimum Spanning Tree Problem v0 G = (V, E) 2 1 v1 v4 2 1 1 -5 v2 v5 -1 -2 v3 v6 -4 T* = argminT w(T) Find a tree with the minimum weight

  26. Outline • Chow-Liu Tree • Minimum Spanning Tree Problem • Kruskal’s Method • Prim’s Method

  27. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Start with a forest where every vertex is a tree

  28. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  29. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  30. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 YES Add the edge to the forest Does this edge connect two different trees?

  31. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  32. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 Select the edge with the minimum length

  33. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  34. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  35. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  36. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  37. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  38. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  39. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 NO Discard the edge Does this edge connect two different trees?

  40. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  41. Kruskal’s Method v0 v0 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  42. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  43. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  44. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  45. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  46. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  47. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Select the edge with the minimum length

  48. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 1 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 YES Add the edge to the forest Does this edge connect two different trees?

  49. Kruskal’s Method v0 v0 1 2 1 v1 v4 v1 v4 1 2 1 1 1 -5 -5 v2 v5 v2 v5 -1 -2 -2 v3 v6 v3 v6 -4 -4 Spanning Tree !!! Algorithm terminates.

  50. Summary Given G = (V, E). Define T = (V, {}) and S = E. While T is not a spanning tree Select the minimum length edge e from S If e connects two different trees Add e to T End Remove e from S End

More Related