1 / 58

P 3 & Beyond Solving Energies with Higher Order Cliques

P 3 & Beyond Solving Energies with Higher Order Cliques. Pushmeet Kohli Pawan Kumar Philip H. S. Torr Oxford Brookes University CVPR 2007. Energy Functions. MAP Inference. Observed Variables. Hidden Variables. Energy Functions. MAP Inference. Energy Minimization.

ray
Download Presentation

P 3 & Beyond Solving Energies with Higher Order Cliques

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. P3 & BeyondSolving Energies with Higher Order Cliques Pushmeet Kohli Pawan Kumar Philip H. S. Torr Oxford Brookes University CVPR 2007

  2. Energy Functions MAP Inference Observed Variables Hidden Variables

  3. Energy Functions MAP Inference Energy Minimization

  4. Energy Functions • Pairwise Energy Functions Unary Pairwise

  5. Energy Functions • Pairwise Energy Functions Unary Pairwise

  6. Energy Functions • Pairwise Energy Functions Unary Pairwise • Efficient Algorithms for Minimization • Message Passing • (BP, TRW) • Move Making • (Expansion/Swap)

  7. Energy Functions • Pairwise Energy Functions Unary Pairwise • Efficient Algorithms for Minimization • Message Passing • (BP, TRW) • Move Making • (Expansion/Swap) Restricted Expressive Power!

  8. Energy Functions • Higher Order Energy Functions Unary Pairwise Higher order More expressive than pairwise FOE: Field of Experts (Roth & Black CVPR05)

  9. Energy Functions • Higher Order Energy Functions Unary Pairwise Higher order • Computationally expensive to minimize! • Exponential Complexity O(LN) • L = Number of Labels • N = Size of Clique

  10. Minimizing Higher Order Energies Efficient BP in Higher Order MRFs (Lan, Roth, Huttenlocher & Black, ECCV 06) • 2x2 clique potentials for Image Denoising • Searched a restricted state space • 16 minutes per iteration Pairwise MRF Noisy Image Higher order MRF

  11. Energy Functions • Higher Order Energy Functions Unary Pairwise Higher order • Our Method • Move making algorithm • Can handle cliques of thousand of variables • Extremely Efficient ( works in seconds)

  12. Talk Outline • Move making Algorithms • Solvable Higher Order Potentials • Moves for the PN Potts Model • Application: Texture Segmentation

  13. Move Making Algorithms Energy Solution Space

  14. Move Making Algorithms Current Solution Search Neighbourhood Optimal Move Energy Solution Space

  15. Computing the Optimal Move Current Solution Search Neighbourhood Optimal Move xc T Transformation function (t) E(x) Em Move Energy x T(xc, t) =xn = xc + t Em(t) = E(T(xc, t))

  16. Computing the Optimal Move Current Solution Search Neighbourhood Optimal Move xc T Transformation function (t) E(x) Em Move Energy x T(xc, t) =xn = xc + t Em(t) = E(T(xc, t))

  17. Computing the Optimal Move Current Solution Search Neighbourhood Optimal Move xc T Transformation function (t) E(x) Em Move Energy x T(xc, t) =xn = xc + t minimize t* Em(t) = E(T(xc, t)) Optimal Move

  18. Computing the Optimal Move Current Solution Search Neighbourhood Optimal Move xc T Transformation function (t) E(x) Em Move Energy x Key Characteristic: Search Neighbourhood Bigger the better!

  19. Moves using Graph Cuts Expansion and Swap Move Algorithm [Boykov, Veksler, Zabih] • Exponential Move Search Space (Good ) • Move encoded by binary vector t • Move Energy

  20. Moves using Graph Cuts Expansion and Swap Move Algorithm [Boykov, Veksler, Zabih] • Exponential Move Search Space (Good ) • Move encoded by binary vector t • Move Energy • Optimal move t* in polynomial time Submodular

  21. Expansion Move • a-Expansion Transformation • Variables take label aor retain current label • Optimal move can be computed for pairwise potentials which are metric. [Boykov, Veksler, Zabih]

  22. Expansion Move Status: Tree Ground Initialize with Tree Expand Ground Expand House Expand Sky House Sky [Boykov, Veksler, Zabih]

  23. Swap Move • Optimal move can be computed for pairwise potentials which are semi-metric. • ab - Swap Transformation • Variables labeled a, bcan swap their labels [Boykov, Veksler, Zabih]

  24. Swap Move Tree Ground Swap Sky, House House Sky [Boykov, Veksler, Zabih]

  25. Moves for Higher Order Potentials • Question you should be asking: Can my higher order potential be solved using α-expansions?

  26. Moves for Higher Order Potentials • Question you should be asking: • Show that move energy is submodularfor all xc Can my higher order potential be solved using α-expansions?

  27. Moves for Higher Order Potentials • Question you should be asking: • Show that move energy is submodularfor all xc Can my higher order potential be solved using α-expansions? Not an easy thing to do!

  28. Moves for Higher Order Potentials • Form of the Higher Order Potentials Clique Inconsistency function: Pairwise potential: xj xi xk Sum Form c xm xl Max Form

  29. Theoretical Results: Swap • Move energy is always submodular if non-decreasing concave. See paper for proofs

  30. Theoretical Results: Expansion • Move energy is always submodular if increasing linear See paper for proofs

  31. PN Potts Model

  32. PN Potts Model c

  33. PN Potts Model c Cost : gred

  34. PN Potts Model c Cost : gmax

  35. Optimal moves for PN Potts • Computing the optimal swap move Label 1(a) Label 2 (b) Label 3 Label 4 c

  36. Optimal moves for PN Potts • Computing the optimal swap move Label 1(a) Case 1 Not all variables assigned label 1 or 2 Label 2 (b) Label 3 Label 4 c

  37. Optimal moves for PN Potts • Computing the optimal swap move Label 1(a) Case 1 Not all variables assigned label 1 or 2 Label 2 (b) Label 3 Label 4 c

  38. Optimal moves for PN Potts • Computing the optimal swap move Label 1(a) Case 1 Not all variables assigned label 1 or 2 Label 2 (b) Label 3 Label 4 c

  39. Optimal moves for PN Potts • Computing the optimal swap move Label 1(a) Case 1 Not all variables assigned label 1 or 2 Label 2 (b) Label 3 Label 4 Move Energy is independent of tc and can be ignored. c

  40. Optimal moves for PN Potts • Computing the optimal swap move Label 1(a) Case 2 All variables assigned label 1 or 2 Label 2 (b) Label 3 Label 4 c

  41. Optimal moves for PN Potts • Computing the optimal swap move Label 1(a) Case 2 All variables assigned label 1 or 2 Label 2 (b) Label 3 Label 4 Can be minimized by solving a st-mincut problem c

  42. Solving the Move Energy Add a constant This transformation does not effect the solution

  43. Solving the Move Energy • Computing the optimal swap move Source Ms v1 v2 vn Mt vi Source Set ti= 0 vj Sink Set tj= 1 Sink

  44. Solving the Move Energy • Computing the optimal swap move Source Ms v1 v2 vn Case 1: all xi= a(vi Source) Mt Cost: Sink

  45. Solving the Move Energy • Computing the optimal swap move Source Ms v1 v2 vn Case 2: all xi= b(vi Sink) Mt Cost: Sink

  46. Solving the Move Energy • Computing the optimal swap move Source Ms v1 v2 vn Case 3: all xi= a,b(vi Source, Sink) Mt Cost: Sink

  47. Optimal moves for PN Potts • The expansion move energy • Similar graph construction. See paper for details

  48. Experimental Results • Texture Segmentation Unary (Colour) Pairwise (Smoothness) Higher Order (Texture) Original Image

  49. Experimental Results • Texture Segmentation Unary (Colour) Pairwise (Smoothness) Higher Order (Texture) Colour Histogram Unary Cost: Tree

  50. Experimental Results • Texture Segmentation Unary (Colour) Pairwise (Smoothness) Higher Order (Texture) Edge Sensitive Smoothness Cost

More Related