1 / 65

Fast, Multiscale Image Segmentation: From Pixels to Semantics

Fast, Multiscale Image Segmentation: From Pixels to Semantics. Ronen Basri The Weizmann Institute of Science Joint work with Achi Brandt, Meirav Galun, Eitan Sharon. Camouflage. Camouflage. Malik et al.’s “Normalized cuts”. Our Results. S egmentation by W eighted A ggregation.

wauna
Download Presentation

Fast, Multiscale Image Segmentation: From Pixels to Semantics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Fast, Multiscale Image Segmentation: From Pixels to Semantics Ronen Basri The Weizmann Institute of Science Joint work with Achi Brandt, Meirav Galun, Eitan Sharon

  2. Camouflage

  3. Camouflage Malik et al.’s “Normalized cuts”

  4. Our Results

  5. Segmentation by Weighted Aggregation A multiscale algorithm: • Optimizes a global measure • Returns a full hierarchy of segments • Linear complexity • Combines multiscale measurements: • Texture • Boundary integrity

  6. The Pixel Graph Couplings (weights) reflect intensity similarity Low contrast – strong coupling High contrast – weak coupling

  7. Normalized-cut Measure Minimize:

  8. Saliency Measure Minimize:

  9. Multiscale Computation of Ncuts • Our objective is to rapidly find the segments (0-1 partitions) that optimize E. • For single-node cuts we simply evaluate E. • For multiple-node cuts we perform “soft contraction” using coarsening procedures from algebraic multigrid solvers of PDEs.

  10. Coarsening the Graph • Suppose we can define a sparse mapping such that for all minimal states

  11. Coarse Energy • Then • PTWP, PTLP define a new (smaller) graph

  12. Recursive Coarsening

  13. Recursive Coarsening Representative subset

  14. , sparse interpolation matrix Recursive Coarsening For a salient segment :

  15. Weighted Aggregation aggregate aggregate

  16. Hierarchical Graph • Pyramid of graphs • Soft relations between levels • Segments emerge as salient nodes at some level of the pyramid

  17. Importance of Soft Relations

  18. Physical Motivation • Our algorithm is motivated by algebraic multigrid solutions to heat or electric networks • u- temperature/potential • a(x, y) – conductivity • At steady state largest temperature differences are along the cuts • AMG coarsening is independent of f

  19. Determine the Boundaries 0 0 1,0,0,…,0 1 P

  20. Hierarchyin SWA

  21. Texture Examples

  22. Filters (From Malik and Perona) Center- surround Oriented filters

  23. A Chicken and Egg Problem Problem: Coarse measurements mix neighboring statistics Solution: Support of measurements is determined as the segmentation process proceeds Hey, I was here first

  24. Texture Aggregation • Aggregates assumed to capture texture elements • Compare neighboring aggregates according to the following statistics: • Multiscale brightness measures • Multiscale shape measures • Filter responses • Use statistics to modify couplings

  25. Recursive Computation of Measures • Given some measure of aggregates at a certain level (e.g., orientation) • At every coarser level we take a weighted sum of this measure from previous level • The result can be used to compute the average, variance or histogram of the measure • Complexity is linear

  26. Use Averages to Modify the Graph

  27. Adaptive vs. Rigid Measurements Original Averaging Geometric Our algorithm - SWA

  28. Adaptive vs. Rigid Measurements Original Interpolation Geometric Our algorithm - SWA

  29. Adaptive vs. Rigid Measurements

  30. Adaptive vs. Rigid Measurements

  31. Adaptive vs. Rigid Measurements

  32. Adaptive vs. Rigid Measurements

  33. Adaptive vs. Rigid Measurements

  34. Texture Aggregation Fine (homogeneous) Coarse (heterogeneous)

  35. Multiscale Variance Vector

  36. Multiscale Variance Vector

  37. Variance: Avoid Mixing aggregation Sliding window

  38. Leopard

  39. More Leopards…

  40. And More…

  41. Birds

  42. More Animals

  43. Boat

More Related