1 / 33

Submodularization for Binary Pairwise Energies

Lena Gorelick joint work with. Submodularization for Binary Pairwise Energies. I. Ben Ayed. O. Veksler. Y. Boykov. A. Delong. Example of Simple Binary Energy. Potts Model. Binary Pairwise Energy Quadratic Form. Potts Model. Submodular Energy

nat
Download Presentation

Submodularization for Binary Pairwise Energies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lena Gorelick joint work with Submodularizationfor Binary Pairwise Energies I. Ben Ayed O. Veksler Y. Boykov A. Delong

  2. Example of Simple Binary Energy Potts Model

  3. Binary Pairwise EnergyQuadratic Form Potts Model Submodular Energy global optimum with graphcut(Boros & Hammer, 2002)

  4. Binary Pairwise Energy Quadratic Form Middlebury Non-Submodular Energy NP-hard Image credit: Carlos Hernandes

  5. Standard Optimization Methods General energy - NP-hard • Approximate methods: • Global Linearization:QPBO, TRWS, SRMP (Kolmogorov et al. 2006, 2014) • Local Linearization: parallel ICM, IPFP (Leordeanu, 2009) • Message Passing: BP (Pearl 1989)

  6. Related WorkGlobal Linearization Linearize introducing large number of variables and constraints • QPBO, TRWS, SRMP (Kolmogorov et al. 2006, 2014) Solve relaxed LPor its dual Rounding  Integrality Gap

  7. Related WorkIterative Local Linearization parallel ICM (Leordeanu, 2009) large steps  weak min IPFP(Leordeanu, 2009) controls step size by relaxation  Integrality Gap ~ Et(x) Bounded domain of discrete configurations

  8. Local Submodular Approximation LSA • Local SubmodularApproximationmodel • Non-linear • Two ways to control step size ~ Et(x) Bounded domain of discrete configurations

  9. Iterative Optimization Framework • Trust Region • Local submodular approximation • Auxiliary Functions = Surrogate Functions = Upper Bounds = Majorize-Minimize • Local submodular upper bound • Never leave the discrete domain LSA-TR LSA-AUX

  10. Iterative Optimization Framework • Trust Region: • Discrete High Order Energies • Relaxed Quadratic Binary Energies • Levenberg Marquardt • Auxiliary Functions=Surrogate Functions =Upper Bounds = Majorize-Minimize • Discrete High Order Energies Gorelick et al. 2012,2013 Olsson et al. 2008 Hartley & Zisserman 2004 Narasimhan & Bilmes 2005 Rother et al. 2006 Ben Ayed et al. 2013

  11. Local Submodular ApproximationLSA - +

  12. Local Submodular ApproximationLSA

  13. Local Submodular ApproximationLSA Approximate around ~ Et(x)

  14. Local Submodular ApproximationLSA Approximate around ~ Et(x) Linear Approximation Submodular function LSA

  15. Linear Approximation of the Supermodular Term

  16. Linear Approximation of the Supermodular Term

  17. Linear Approximation of the Supermodular Term

  18. Linear Approximation of the Supermodular Term 0,1 0,0 1 0 1,1 1,0 1

  19. Linear Approximation of the Supermodular Term 1 0 1,0 1

  20. Linear Approximation of the Supermodular Term Linear (Unary) approximation 0,0 1 0 1,1 1,0 1

  21. Linear Approximation of the Supermodular Term 1 0 1

  22. LSA-TR:Trust Region Overview

  23. LSA-TR:Trust Region Overview ~ Et(x) Newton Step

  24. LSA-TRTrust Region Sub-Problem ~ Et(x) Trust Region Sub-Problem NP-hard! Constrained Submodular Optimization Trust Region 24

  25. LSA-TR: Approximate TR sub-problem fixed in each iteration inversely related to trust region size adjusted based on quality of approximation Gorelick et al. 2013 Submodular Unary Terms Boykov et al. 2006

  26. Experiments & Results

  27. Experiments & Results:Deconvolution • Binary De-convolution • All pairwise terms supermodular ? Original Img Convolved Convolved+Noise

  28. Experiments & Results:Deconvolution Noise: N(0,0.05) QPBO(0.1 sec.) QPBO-I (0.2 sec.) E=66.44 LBP 5000 iter. E=40.15 IPFP (0.4 sec.) E=32.90 TRWS: 5000 iter. E=65.07 SRMP: 5000 iter. E=39.06 LSA-AUX (0.04 sec) E=34.70 LSA-TR (0.3 sec.) E=21.13

  29. Experiments & Results:Segmentation of Thin Structures QPBO QPBO-I E= -77.08 LBP E= -84.54 IPFP E= 163.25 Image Repulsion = Reward different labels across high contrast edges TRWS E= -67.21 SRMP E= -101.61 LSA-AUX E= -120.03 LSA-TR E= -175.05 Potts, v<0 (submodular) with edge repulsion, v>0 (non-submodular)

  30. Experiments & Results:Inpainting • dtf-chinesechar database Kappes et al., 2013 Ground Truth Input Img LSA-TR

  31. Experiments & Results:In-painting Chinese Characters

  32. Curvature Regularization • Efficient Squared Curvature model – (Nieuwenhuis et al. 2014, poster on Friday) Potts Model Elastica Heber et al. 2012 Our curvature Using LSA-TR 90-degree curvature El-Zehiry&Grady, 2010

  33. Summary • Two novel discrete optimization methods • Simple, efficient, state-of-art results • The code is available online - http://vision.csd.uwo.ca/code/ • Extensions: • Find new applications • Convexity Shape Prior (in ECCV14) • Alternative optimization framework with LSA • Pseudo-Bounds (in ECCV14) • Please come by our poster

More Related