1 / 36

Distributed Cosegmentation via Submodular Optimization on Anisotropic Diffusion

Distributed Cosegmentation via Submodular Optimization on Anisotropic Diffusion. Gunhee Kim 1 Eric P. Xing 1 Li Fei-Fei 2 Takeo Kanade 1. 1 : School of Computer Science, Carnegie Mellon University 2 : Computer Science Department, Stanford University. November 9, 2011. Outline.

kelii
Download Presentation

Distributed Cosegmentation via Submodular Optimization on Anisotropic Diffusion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Cosegmentation via Submodular Optimization on Anisotropic Diffusion Gunhee Kim1 Eric P. Xing1 Li Fei-Fei2 Takeo Kanade1 1: School of Computer Science, Carnegie Mellon University 2: Computer Science Department, Stanford University November 9, 2011

  2. Outline • Problem Statement • Submodular Optimization on Diffusion • Applications • Diversity Ranking • Single Image Segmentation • Cosegmentation • Experiments • Conclusion

  3. Outline • Problem Statement • Submodular Optimization on Diffusion • Applications • Diversity Ranking • Single Image Segmentation • Cosegmentation • Experiments • Conclusion

  4. Image Cosegmentation Remove the ambiguity of what should be segmented out? Jointly segment M images into K regions ! (M = 3, K = 2) • Rother et al. 2006 • Hochbaum and Singh, 2009 • Joulin et al, 2010 • Batra et al, 2010 • Mukherjee et al, 2011 • Vincente et al, 2010, 2011

  5. Why is Cosegmentation Interesting? Wide potential in Web applications Photo-taking patterns of general users My son joined baseball club. I saw dolphins in aquarium.

  6. Our Approach Major challenges for web photos (2) Highly-variable (1) Large-scale Jointly segment M images into K regions • [R06] Rother et al. 2006 • [H09] Hochbaum and Singh, 2009 • [B10] Batra et al, 2010 • [J10] Joulin et al, 2010 • [V10] Vincente et al, 2010 • [M11] Mukherjee et al, 2011

  7. Contributions A New optimization framework • Constant-factor approximation of optimal • Easily parallelizable • Automatic selection of K • Robust against wrong K • [R06] Rother et al. 2006 • [H09] Hochbaum and Singh, 2009 • [B10] Batra et al, 2010 • [J10] Joulin et al, 2010 • [V10] Vincente et al, 2010 • [M11] Mukherjee et al, 2011

  8. Outline • Problem Statement • Submodular Optimization on Diffusion • Applications • Diversity Ranking • Single Image Segmentation • Cosegmentation • Experiments • Conclusion

  9. Diffusion Diffusion in physics Spread of particles (or energy) through random motion from high concentration to low concentration [Wikipedia] Examples Heat Equation (Partial Differential Equation) • Heat diffusion • Electric current Temperature Diffusivity (conductance) tensor

  10. Optimization Maximize the sum of temperature of the system max s.t K heat sources Maximize the sum of temperature Environment temperature

  11. Correspondences Temperature maximization and Image Segmentation max s.t

  12. Optimization How can we solve this? max s.t [Theorem] (Neuhauser, Wolsey, Fisher 1978) Let u be , nondecreasing, and submodular. Then, the greedy algorithm finds a set such that 0.632.

  13. Submodularity on Anisotropic Diffusion [Theorem] Suppose that system is under linear anisotropic diffusion Let be temperature at time t point xwhen heat sources are attached to Then, the following holds for (T2) is nondecreasing (T1) (T3) is submodular (Proof)

  14. Submodularity on Anisotropic Diffusion [Theorem] Suppose that system is under linear anisotropic diffusion Let be temperature at time t point xwhen heat sources are attached to Then, the following holds for (T2) is nondecreasing (T1) (T3) is submodular Induction on distance (Proof) (Diminishing Return) x x

  15. Greedy Algorithm Sketch of the greedy algorithm max s.t Find the point with maximum marginal gain in every round. 1. 2. Iterate until 2.1. Marginal gain 2.2.

  16. Outline • Problem Statement • Submodular Optimization on Diffusion • Applications • Diversity Ranking • Single Image Segmentation • Cosegmentation • Experiments • Conclusion

  17. Diversity Ranking Ranking items according to both centrality and diversity A B C Ranking values Items Centrality only: A > B > C Centrality + Diversity: A > C > B

  18. Optimization for Diversity Ranking max s.t Simplification (1) System is a graph (2) Steady-state (3) Diffusivity is defined by Gaussian similarity (4) Every v is connected to the ground with z

  19. Optimization for Diversity Ranking max s.t Simplification (1) System is a graph (2) Steady-state (3) Diffusivity is defined by Gaussian similarity max s.t (4) Every v is connected to the ground with z

  20. Examples of Diversity Ranking (1) vertices (2) features (3) Conductance Input data

  21. Examples of Diversity Ranking Marginal gain Input data 1st item 2nd item 3rd item Clustering

  22. Outline • Problem Statement • Submodular Optimization on Diffusion • Applications • Diversity Ranking • Single Image Segmentation • Cosegmentation • Experiments • Conclusion

  23. Segmenting a Single Image Construct image graph Optimization formulation is similar to that of diversity ranking G = (V, E, W) Input image 1. Superpixels (SP) 2. Connect adjacent SPs G = (V, E, W) G = (V, E, W) Color g(v) = Texture 3. Features on SP 4. Conductance G = (V, E, W)

  24. Basic Behavior of Our Segmentation Greedily select the largest and most coherent regions ! Input image K=2: sky K=3: tree K=4: wall K=5: roof K=6: window K=7: building K=8: trash can Source code is available ! Automatic selection of K

  25. Outline • Problem Statement • Submodular Optimization on Diffusion • Applications • Diversity Ranking • Single Image Segmentation • Cosegmentation • Experiments • Conclusion

  26. Cosegmentation Segment selection should be coupled! Single image segmentation Cosegmentation Objective 1: Segment should be large and coherent. + Objective 2: Segment should be similar to its corresponding ones in other images

  27. Cosegmentation Control source temperatures Cosegmentation A A B B A is better than B to maximize the temperature

  28. An Toy Example of Cosegmentation MSRC cow images (M=3, K=4) Source code is available ! Segments Input images Cosegmentation Likelihood

  29. Outline • Problem Statement • Submodular Optimization on Diffusion • Applications • Diversity Ranking • Single Image Segmentation • Cosegmentation • Experiments • Conclusion

  30. Two Experiments Figure-ground cosegmentation with a pair of images • Goal: Compare with other state-of-the-art techniques • Dataset: MSRC • Dataset: ImageNet ex. cat Scalable cosegmentation • Goal: Feasibility for Web photos ex. green lizard

  31. Exp1.Figure-GroundCosegmentation Segmentation accuracies for 100 random pairs of MSRC [6] ICCV 2009 [7] CVPR 2010 [6, 7] Use their implementation without modification

  32. Cosegmentation on MSRC Cosegmentation Examples (K=8) Ours (K = 8) Normalized cuts (K = 8) (1) Multiple instances (2) Robust against wrong choice of K

  33. Outline • Problem Statement • Submodular Optimization on Diffusion • Applications • Diversity Ranking • Single Image Segmentation • Cosegmentation • Experiments • Conclusion

  34. Conclusion What’s done Prove the temperature in anisotropic diffusion is submodular. Diversity ranking Single-image segmentation Cosegmentation Source code is available ! Next step smoothing Optical flow (1) A large-scale edge-preserving image smoothing (2) Layered motion segmentation

  35. Conclusion What’s done Cosegmentation for Web photos was proposed • Arbitrary K and a larger M by order of magnitude • Easily Parallelizable • Automatic selection of K • Robust against wrong K (Ours) (Ncuts)

  36. Thank you !Stop by our poster at 80!

More Related