1 / 68

Discrete Optimization Lecture 9

Discrete Optimization Lecture 9. M. Pawan Kumar pawan.kumar@ecp.fr. Slides available online http:// cvn.ecp.fr /personnel/ pawan /. Questions?. Topics. Belief Propagation (tight for trees). LP Relaxation. Tight for trees. Minimum Cut (tight for submodular ). Tight for submodular.

linh
Download Presentation

Discrete Optimization Lecture 9

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discrete OptimizationLecture 9 M. Pawan Kumar pawan.kumar@ecp.fr Slides available online http://cvn.ecp.fr/personnel/pawan/

  2. Questions?

  3. Topics Belief Propagation (tight for trees) LP Relaxation Tight for trees Minimum Cut (tight for submodular) Tight for submodular Move-Making (iterative, approximate) ??

  4. Outline • Metric Labeling • Rounding-based Move-Making • Dual Decomposition

  5. Metric Labeling Variables V= { V1, V2, …, Vn}

  6. Metric Labeling Variables V= { V1, V2, …, Vn}

  7. Metric Labeling wabd(f(a),f(b)) θb(f(b)) wab ≥ 0 θa(f(a)) d is metric Va Vb minf E(f) + Σ(a,b)wabd(f(a),f(b)) = Σaθa(f(a)) Labels L= { l1, l2, …, lh} Variables V= { V1, V2, …, Vn} Labeling f: { 1, 2, …, n}  {1, 2, …, h}

  8. Metric Labeling Va Vb minf E(f) + Σ(a,b)wabd(f(a),f(b)) = Σaθa(f(a)) NP hard Low-level vision applications

  9. Outline • Metric Labeling • Move-Making Algorithms • Linear Programming Relaxation • UGC-Hardness • Rounding-based Move-Making • Dual Decomposition

  10. Move-Making Algorithms Space of All Labelings f

  11. Expansion Algorithm Variables take label lα or retain current label Boykov, Veksler and Zabih, 2001 Slide courtesy PushmeetKohli

  12. Expansion Algorithm Variables take label lα or retain current label Tree Ground House Status: Initialize with Tree Expand Ground Expand House Expand Sky Sky Boykov, Veksler and Zabih, 2001 Slide courtesy PushmeetKohli

  13. Multiplicative Bounds f*: Optimal Labeling f: Estimated Labeling Σaθa(f(a)) + Σ(a,b)wabd(f(a),f(b)) ≥ Σaθa(f*(a)) + Σ(a,b)wabd(f*(a),f*(b))

  14. Multiplicative Bounds f*: Optimal Labeling f: Estimated Labeling Σaθa(f(a)) + Σ(a,b)wabd(f(a),f(b)) ≤ B Σaθa(f*(a)) + Σ(a,b)wabd(f*(a),f*(b))

  15. Outline • Metric Labeling • Move-Making Algorithms • Linear Programming Relaxation • UGC-Hardness • Rounding-based Move-Making • Dual Decomposition

  16. Integer Linear Program Minimize a linear function over a set of feasible solutions Indicator xa(i)  {0,1} for each variable Va and label li Indicator xab(i,k)  {0,1} for each neighbor (Va,Vb) and labels li, lk Number of facets grows exponentially in problem size

  17. Linear Programming Relaxation Indicator xa(i)  {0,1} for each variable Va and label li Indicator xab(i,k)  {0,1} for each neighbor (Va,Vb) and labels li, lk

  18. Linear Programming Relaxation Indicator xa(i)  [0,1] for each variable Va and label li Indicator xab(i,k)  [0,1] for each neighbor (Va,Vb) and labels li, lk

  19. Approximation Factor x*: LP Optimal Solution x: Estimated Integral Solution ΣaΣiθa(i)xa(i) + Σ(a,b)Σ(i,k) wabd(i,k)xab(i,k) ≥ ΣaΣiθa(i)x*a(i) + Σ(a,b)Σ(i,k) wabd(i,k)x*ab(i,k)

  20. Approximation Factor x*: LP Optimal Solution x: Estimated Integral Solution ΣaΣiθa(i)xa(i) + Σ(a,b)Σ(i,k) wabd(i,k)xab(i,k) ≤ F ΣaΣiθa(i)x*a(i) + Σ(a,b)Σ(i,k) wabd(i,k)x*ab(i,k)

  21. Outline • Metric Labeling • Move-Making Algorithms • Linear Programming Relaxation • UGC-Hardness • Rounding-based Move-Making • Dual Decomposition Manokaran et al., 2008

  22. Integrality Gap x*: LP Optimal Solution x: ILP Optimal Solution ΣaΣiθa(i)xa(i) + Σ(a,b)Σ(i,k) wabd(i,k)xab(i,k) ≤ G ΣaΣiθa(i)x*a(i) + Σ(a,b)Σ(i,k) wabd(i,k)x*ab(i,k) Assuming UGC, G-ε is not possible in polynomial time

  23. Outline • Metric Labeling • Rounding-based Move-Making • Dual Decomposition Kumar, 2014

  24. History

  25. Theoretical Guarantees M = ratio of maximum and minimum non-zero distance

  26. Key Observation If d is submodular d(i,k) + d(i+1,k+1) ≤ d(i,k+1) + d(i+1,k), for all i, k energy can be minimized via minimum cut Schlesinger and Flach, 2003

  27. Outline • Metric Labeling • Rounding-based Move-Making • Complete Rounding and Move • Interval Rounding and Moves • Hierarchical Rounding and Moves • Dual Decomposition

  28. Complete Rounding Treat xa(i)  [0,1] as probability that f(a) = i Cumulative probability ya(i) = Σj≤ixa(j) r ya(2) ya(i) ya(k) 0 ya(1) ya(h) = 1 Generate a random number r  (0,1] Assign the label next to r

  29. Complete Move Va Vb θab(i,k) = wabd(i,k) NP-hard

  30. Complete Move Va Vb d’(i,k) ≥ d(i,k) d’ is submodular θab(i,k) = wabd’(i,k)

  31. Complete Move Va Vb d’(i,k) ≥ d(i,k) d’ is submodular θab(i,k) = wabd’(i,k)

  32. Outline • Metric Labeling • Rounding-based Move-Making • Complete Rounding and Move • Interval Rounding and Moves • Hierarchical Rounding and Moves • Dual Decomposition

  33. Interval Rounding Treat xa(i)  [0,1] as probability that f(a) = i Cumulative probability ya(i) = Σj≤ixa(j) ya(2) ya(i) ya(k) 0 ya(1) ya(h) = 1 Choose an interval of length h’

  34. Interval Rounding Treat xa(i)  [0,1] as probability that f(a) = i Cumulative probability ya(i) = Σj≤ixa(j) r ya(i) ya(k) REPEAT Choose an interval of length h’ Generate a random number r  (0,1] Assign the label next to r if it is within the interval

  35. Interval Move Choose an interval of length h’ Va Vb θab(i,k) = wabd(i,k)

  36. Interval Move Choose an interval of length h’ Add the current labels Va Vb θab(i,k) = wabd(i,k)

  37. Interval Move Choose an interval of length h’ Add the current labels d’(i,k) ≥ d(i,k) d’ is submodular Solve to update labels Va Vb Repeat until convergence θab(i,k) = wabd’(i,k)

  38. Interval Move Each problem can be solved using minimum cut Same multiplicative bound as interval rounding Multiplicative bound is tight Kumar and Torr, NIPS 2008

  39. Outline • Metric Labeling • Rounding-based Move-Making • Complete Rounding and Move • Interval Rounding and Moves • Hierarchical Rounding and Moves • Dual Decomposition

  40. Hierarchical Rounding L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Hierarchical clustering of labels (e.g. r-HST metrics)

  41. Hierarchical Rounding L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Assign variables to labels L1, L2 or L3 Move down the hierarchy until the leaf level

  42. Hierarchical Rounding L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Assign variables to labels l1, l2 or l3

  43. Hierarchical Rounding L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Assign variables to labels l4, l5 or l6

  44. Hierarchical Rounding L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Assign variables to labels l7, l8 or l9

  45. Hierarchical Move L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Hierarchical clustering of labels (e.g. r-HST metrics)

  46. Hierarchical Move L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Obtain labeling f1 restricted to labels {l1,l2,l3}

  47. Hierarchical Move L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Obtain labeling f2 restricted to labels {l4,l5,l6}

  48. Hierarchical Move L1 L2 L3 l1 l2 l3 l4 l5 l6 l7 l8 l9 Obtain labeling f3 restricted to labels {l7,l8,l9}

  49. Hierarchical Move L1 L2 L3 f3(a) f3(b) f2(a) f2(b) f1(a) f1(b) Va Vb Move up the hierarchy until we reach the root

  50. Hierarchical Move Each problem can be solved using minimum cut Same multiplicative bound as hierarchical rounding Multiplicative bound is tight Kumar and Koller, UAI 2009

More Related