1 / 34

Higher-Order Clique Reduction in Binary Graph Cut

Higher-Order Clique Reduction in Binary Graph Cut. Hiroshi Ishikawa. Nagoya City University Department of Information and Biological Sciences. Contribution of this work. Reduce any higher-order binary MRF into first order Adds variables

koen
Download Presentation

Higher-Order Clique Reduction in Binary Graph Cut

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Higher-Order Clique Reduction in Binary Graph Cut Hiroshi Ishikawa Nagoya City University Department of Information and Biological Sciences CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  2. Contribution of this work Reduce any higher-order binary MRF into first order Adds variables Can also be used for multi-label energy, with the Fusion Move technique CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  3. FindX Given Y Close toY Smooth All pixels Neighboring pixels AssignsXv (= 0 or 1) to each pixel v Energy Minimization CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  4. Better (Lower Energy) Worse (Higher Energy) A B C D Energy Minimization Good (Low Energy) Bad (High Energy) CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  5. Energy Minimization Good (Low Energy) Bad (High Energy) 12 Bad 12 Bad 40 Good 40 Good CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  6. Better (Lower Energy) Worse (Higher Energy) Energy Minimization A B C D 10 As 10 As 4 Bs 8 Bs 7 Cs 3 Cs 0 Ds 0 Ds CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  7. Clique Clique Higher-Order Energy Third Order (Clique up to 4 pixels) First Order (Clique up to 2 pixels) Third Order (Clique up to 4 pixels) Clique Clique General Order C: a set of cliques CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  8. First-Order MRF Minimization Graph cuts Greig et al. ’89 Boykov et al. CVPR’98, PAMI2001(-exp.) Kolmogorov & Zabih. PAMI2004 Belief propagation Felzenszwalb & Huttenlocher. IJCV2006 Meltzer et al. ICCV2005 Tree-reweighted message passing Kolmogorov. PAMI2006 CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  9. Higher-Order MRF Minimization Graph cuts Kolmogorov & Zabih. PAMI2004 Freedman & Drineas. CVPR2005 Woodford et al. CVPR2008 Kohli et al. PAMI’08, Cremers&Grady ECCV’06 Rother et al. CVPR2009 Komodakis & Paragios. CVPR2009 Belief propagation Lan et al. ECCV2006 Potetz. CVPR2008 CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  10. Higher-Order MRF Minimization Graph cuts Kolmogorov & Zabih. PAMI2004 Freedman & Drineas. CVPR2005 Woodford et al. CVPR2008 Kohli et al. PAMI’08, Cremers&Grady ECCV’06 Rother et al. CVPR2009 Komodakis & Paragios. CVPR2009 Belief propagation Lan et al. ECCV2006 Potetz. CVPR2008 CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  11. Functions of Binary Variables Pseudo-Boolean function (PBF) Function of binary (0 or 1) variables Can always write uniquely as a polynomial One variable x : E0 (1x) + E1 x Two variables x, y : E00(1x) (1y) + E01(1x) y + E10x (1y) + E11x y Three variables x, y, z : E000(1x) (1y) (1z) + E001(1x) (1y) z +…+ E111x y z nth order binary MRF = (n+1)th degree PBF CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  12. 2nd-Order (Cubic) Case Kolmogorov & Zabih. PAMI2004 Freedman & Drineas. CVPR2005 Reduce cubic PBF into quadratic one using B={0,1} xy z 0 0 0 0 0 1 0 1 1 1 1 1 CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  13. 2nd-Order (Cubic) Case If a < 0 Thus So, in a minimization problem, we can substitute by CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  14. Higher-Order Case ifa < 0 CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  15. Higher-Order Case For a > 0 and d > 3, nothing similar is known → our contribution Imagine such a formula: CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  16. Higher-Order Case For a > 0 and d > 3, nothing similar is known → our contribution Imagine such a formula: Notice LHS is symmetric i.e., if we swap the value of two variables, LHS is unchanged So RHS must be symmetric, too. CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  17. Symmetric Polynomial Fact Any symmetric polynomial can be written as a polynomial in terms of elementary symmetric polynomials. If f(x, y,z,t) is quadratic symmetric, it can be written with a polynomial P(u,v) : CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  18. Quartic (Degree 4) Case CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  19. Quartic (Degree 4) Case CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  20. Quartic (Degree 4) Case An exhaustive search fora, b, c, d, e yields CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  21. Quintic (Degree 5) Case Similarly, and so on, until one can guess… CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  22. General Case where CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  23. General Case For each monomial, the number of new variable is: For instance, general quintic looks like: So the number is exponential in degree CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  24. Multiple labels: Fusion Move Assume labels Labeling Y assigns a label Yv to each v Fusion Move Iteratively update Y : 1. Generate a proposed labeling P Lempitsky et al. ICCV2007 CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  25. Multiple labels: Fusion Move Assume labels Labeling Y assigns a label Yv to each v Fusion Move Iteratively update Y : 1. Generate a proposed labeling P 2. MergeY and P The merge defines a binary problem: “For each v, changeYv to Pvor not” Lempitsky et al. ICCV2007 CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  26. 0 1 0 0 1 0 1 0 0 1 0 1 1 1 0 0 Multiple labels: Fusion Move Fusion Move Iteratively update Y : 1. Generate a proposed labeling P 2. MergeY and P The merge defines a binary problem: “For each v, changeYv to Pvor not” Y P X CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  27. 0 0 0 1 0 0 0 1 1 0 0 0 0 1 0 0 1 0 0 1 0 0 1 1 1 1 1 1 1 0 1 0 Multiple labels: Fusion Move Fusion Move Iteratively update Y : 1. Generate a proposed labeling P 2. MergeY and P The merge defines a binary problem: “For each v, changeYv to Pvor not” Y P X CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  28. Fusion Move with QPBO QPBO (Roof duality) Minimizes submodular E globally. For non-submodular E, assigns each pixel 0, 1, or unlabeled With fusion move, by not changing unlabeled pixels to P, E doesn’t increase Hammer et al. 1984, Boros et al. 1991, 2006 Kolmogorov & Rother PAMI2007, Rother et al. CVPR2007 CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  29. Experiment: Denoising by FoE FoE (Fields of Experts) Roth & Black CVPR2005 A higher-order prior for natural images C: a set of cliques C : C : CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  30. Experiment: Denoising by FoE Original Noise-added 3rd order 1st order CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  31. Experiment: Denoising by FoE Energy (smaller the better) PSNR (larger the better) 32 45000 40000 31 35000 30 30000 29 25000 28 20000 27 15000 26 10000 25 5000 24 0 Lan et al. Potetz This work Lan et al. Potetz This work • Lan et al. ECCV2006 ~8 hours • Potetz. CVPR2008 ~30 mins • This work ~10 mins  = 10  = 20 CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  32. 12 0 10 0 8 0 6 0 4 0 2 0 0 5 0 10 0 15 0 20 0 25 0 Experiment: Denoising by FoE Energy & PSNR Two proposal generation strategies E (×1000) PSNR 2 7 blur & random 2 6 expansion 2 5 expansion 2 4 blur & random 2 3 2 2 0 5 0 10 0 15 0 20 0 25 0 time (sec.) time (sec.) CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  33. Summary Reduce any higher-order binary MRF into first order Adds variables Number exponential in order For multi-label, can be used with Fusion Move with QPBO CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

  34. Thank you! Code available at http://www.nsc.nagoya-cu.ac.jp/~hi/ Acknowledgements Stefan Roth,Brian Potetz, and Vladimir Kolmogorov CVPR2009: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Miami Beach, Florida. June 20-25.

More Related