1 / 33

A Database of Human Segmented Natural Images and Two Applications

A Database of Human Segmented Natural Images and Two Applications. David Martin, Charless Fowlkes, Doron Tal, Jitendra Malik UC Berkeley {dmartin,fowlkes,doron,malik}@eecs.berkeley.edu. Motivation. Berkeley Segmentation Dataset  Groundtruth for image segmentation of natural images

kinsey
Download Presentation

A Database of Human Segmented Natural Images and Two Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Database of Human Segmented Natural Images and Two Applications David Martin, Charless Fowlkes, Doron Tal, Jitendra Malik UC Berkeley {dmartin,fowlkes,doron,malik}@eecs.berkeley.edu

  2. Motivation • Berkeley Segmentation Dataset  Groundtruth for image segmentation of natural images • App#1: A segmentation benchmark • App#2: Ecological statistics David Martin - UC Berkeley - ICCV 2001

  3. Benchmark Example for Recognition MNIST handwritten digit dataset [LeCun, AT&T] http://www.research.att.com/~yann/exdb/mnist/index.html Training set, test set, evaluation methodology, algorithm ranking David Martin - UC Berkeley - ICCV 2001

  4. The Image Dataset • 1000 Corel images • Photographs of outdoor scenes • Texture is common • Large variety of subject matter • 481 x 321 x 24b David Martin - UC Berkeley - ICCV 2001

  5. David Martin - UC Berkeley - ICCV 2001

  6. David Martin - UC Berkeley - ICCV 2001

  7. David Martin - UC Berkeley - ICCV 2001

  8. David Martin - UC Berkeley - ICCV 2001

  9. Establishing Groundtruth • Def: Segmentation = Partition of image pixels into exclusive sets • Manual segmentation by human subjects • Custom Java tool to facilitate task • Currently: 1000 images, 5500 segmentations, 20 subjects • Naïve subjects (UCB undergrads) given simple, non-technical instructions David Martin - UC Berkeley - ICCV 2001

  10. Directions to Image Segmentors • You will be presented a photographic image • Divide the image into some number of segments, where the segments represent “things” or “parts of things” in the scene • The number of segments is up to you, as it depends on the image. Something between 2 and 30 is likely to be appropriate. • It is important that all of the segments have approximately equal importance. David Martin - UC Berkeley - ICCV 2001

  11. David Martin - UC Berkeley - ICCV 2001

  12. David Martin - UC Berkeley - ICCV 2001

  13. The segmentations are not identical. • But are they consistent?? David Martin - UC Berkeley - ICCV 2001

  14. image background left bird right bird beak grass bush far beak eye head body eye head body Perceptual organization forms a hierarchy Each subject picks a slice through this hierarchy. David Martin - UC Berkeley - ICCV 2001

  15. Quantifying inconsistency S2 S1 How much is S1 a refinement of S2 at pixel ? David Martin - UC Berkeley - ICCV 2001

  16. Segmentation Error Measure • One-way Local Refinement Error: • Segmentation Error allows refinement in either direction at each pixel: David Martin - UC Berkeley - ICCV 2001

  17. Human segmentations are consistent Distribution of segmentation error over the dataset. David Martin - UC Berkeley - ICCV 2001

  18. Color Gray InvNeg David Martin - UC Berkeley - ICCV 2001

  19. InvNeg David Martin - UC Berkeley - ICCV 2001

  20. Color Gray InvNeg David Martin - UC Berkeley - ICCV 2001

  21. Gray vs. Color vs. InvNeg Segmentations SE (gray, gray) = 0.047 SE (gray, color) = 0.047 Color may affect attention, but doesn’t seem to affect perceptual organization SE (gray, gray) = 0.047 SE (gray, invneg) = 0.059 InvNeg interferes with high-level cues (2500 gray, 2500 color,200 invneg segmentations) David Martin - UC Berkeley - ICCV 2001

  22. Benchmark Methodology • Separate training and test datasets with no images in common • Generate computer segmentation(s) of each image in test set • Determine error of each computer segmentation using SE measure • Algorithm scored by mean SE • Example: • SE (human, human) = 0.05 • SE (NCuts, human) = 0.22 • SE (different images) = 0.30 David Martin - UC Berkeley - ICCV 2001

  23. Ecological Statistics of Image Segmentations • Validating and quantifying Gestalt grouping factors [Brunswik 1953] • Priors on region properties • Recent work on natural image statistics: • Filter outputs [Ruderman 1994, Olshausen & Field 1996, Yuille et. al. 1999] • Object sizes [Alvarez, Gousseau, Morel 1999] • Shape [Zhu 1999] • Contours [August & Zucker 2000, Geisler et al. 2001] David Martin - UC Berkeley - ICCV 2001

  24. Relative power of cues • Pairwise grouping cues • Proximity • Luminance similarity • Color similarity • Intervening contour • Texture similarity David Martin - UC Berkeley - ICCV 2001

  25. P (Same Segment | Proximity) David Martin - UC Berkeley - ICCV 2001

  26. P (Same Segment | Luminance) David Martin - UC Berkeley - ICCV 2001

  27. Bayes Risk for Proximity Cue David Martin - UC Berkeley - ICCV 2001

  28. Bayes Risk for Various Cues Conditioned on Proximity David Martin - UC Berkeley - ICCV 2001

  29. Mutual Information for Various Cues Conditioned on Proximity David Martin - UC Berkeley - ICCV 2001

  30. Priors on Region Properties • Area • Convexity David Martin - UC Berkeley - ICCV 2001

  31. Empirical Distribution of Region Area y = Kx-  = 0.913 Compare with Alvarez, Gousseau, Morel 1999. David Martin - UC Berkeley - ICCV 2001

  32. Empirical Distribution of Region Convexity David Martin - UC Berkeley - ICCV 2001

  33. Conclusion • Large new database of segmentations of natural images by humans • A segmentation benchmark • Ecological statistics • Relative power of grouping cues • Priors on region properties http://www.cs.berkeley.edu/~dmartin/segbench David Martin - UC Berkeley - ICCV 2001

More Related