Object Segmentation

Object Segmentation Presented by Sherin Aly

What is a ‘Good Segmentation’?

http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/resources.htmlhttp://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/resources.html

Learning a classification model for segmentation Xiaofeng Ren and Jitendra Malik

methodology • Two-class classification model • Over segmentation as preprocessing • They use classical Gestalt cues • Contour, texture, brightness and continuation • A linear classifier is used for training

Good Vs Bad segmentation a) Image from Corel Imagebase b) superimposed with a human marked segmentation c) Same image with Bad segmentation

How do we distinguish good segmentations from badsegmentations?

How? • Use “Classical Gestalt cues” • proximity, similarity and good continuation • Instead of Ad-hoc decision about features combination

Gestalt Principles of Grouping In order to interpret what we receive through our senses,we attempt to organize this information into certain groups. http://allpsych.com/psychology101/perception.html

Methodology • Preprocessing • Feature extraction • Feature evaluation • Training • Optimization • Find good segmentaion

Preprocessing • Local • Coherent • Preserve structure • Contour • texture Superpixel map K=200 Reconstruction of human segmentation from Superpixels a contour-based measure is used to quantify this approximation

Tolerance 1,2,and 3 The percentage of human marked boundaries covered by the superpixel maps

Feature Extraction 1. inter-region texture similarity 2. intra-region texture similarity 3. inter-region brightness similarity 4. intra-region brightness similarity 5. inter-region contour energy 6. intra-region contour energy 7. curvilinear continuity

Power of Gestalt cues =

Training the classifier • simple logistic regression classifier, Empirical distribution of pairs of features

Precision is the fraction of detections which are true positives. Recall is the fraction of true positives which are detected

Conclusion • There simple linear classifier had promising results on a variety of natural images. • boundary contour is the most informative grouping cue, and it is in essence discriminative.

Pros & Cons • Cons • The larger spatial support that superpixels provide, allowing more global features to be computed than on pixels alone. • The use of superpixels improves the computational efficiency • SuperPixels technique is very applicable • Pros • Might fall in Local Minima

Combining Top-down and Bottom-up Segmentation Eran Borenstein Eitan Sharon Shimon Ullman

Motivation • Bottom-Up segmentation • Rely on continuity principle • Capture image properties “texture, grey level uniformity and contour continuity” • Segmentation based on similarities between image regions • How can we capture prior knowledge of a specific object (class)? • Answer: Top-Down Segmentation • use prior knowledge about an object Credit: Joseph Djugash

Bottom-Up Segmentation Credit: Joseph Djugash Slides from Eitan Sharon, “Segmentation and Boundary Detection Using Multiscale Intensity Measurements”.

Normalized-Cut Measure Credit: Joseph Djugash Slides from Eitan Sharon, “Segmentation and Boundary Detection Using Multiscale Intensity Measurements”.

Top-Down approach Fragments Input Matching Cover Credit: Joseph Djugash

Another step towards the middle Top-Down Bottom-Up Credit: Joseph Djugash

Some Definitions & Constraints • Measure of saliency h(Γi), hiє [0,1) • A configuration vector s contains labels si (1/-1) of all the segments (Si) in the tree • The label si can be different from its parent’s label s i – • Cost function for a given s Defines the weighted edge between Si & Si– Top-down term Bottom-up term

Classification Costs • The terminal segments of the tree determine the final classification • The top-down term is defined as: • The saliency of a segment should restrict its label (based on its parent’s label) • The bottom-up term is defined as:

Confidence Map • Evaluating the confidence of a region: • Causes of Uncertainty of Classification • Bottom-up uncertainty – regions where there is no salient bottom-up segment matching the top-down classification • Top-down uncertainty – regions where the top-down classification is ambiguous (highly variable shape regions) • The type of uncertainty and the confidence values can be used to select appropriate additional processing to improve segmentation

Results • Calculate average distance between a given segmentation contour and a benchmark contour. • Removing from the average all contour points having a confidence measure less than 0.1. • The resulting confidence map efficiently separated regions of high and low consistency. • The combined scheme improved the top-down contour by over 67% on average. • This improvement was even larger in object parts with highly variable shape.

The initial classification map T(x, y) Results (cont.) Buttom up • top-down process may produce a figure-ground approximation that does not follow the image discontinuities. • Salient bottom-up segments can correct these errors and delineate precise region boundaries

Results III (cont.)

Results III (cont.) the top-down completely misses a part of the object . The confidence map may be helpful in identifying such cases,

Results III (cont.) bottom-up segmentation may be insufficient in detecting the figure-ground contour, and the top-down process completes the missing information

Results III (cont.)

Results III (cont.) Salient bottom-up segments can correct these errors and delineate precise region boundaries

Conclusion • Buttom-up and top-down merits • Provide reliable confidence map • It take into account all discontinuities at all scales But: • If the object is assigned a given category, the specific features cannot be adopted for other categories

Constrained Parametric Min-Cuts for Automatic Object Segmentation Joao Carreira Cristian Sminchisescu

Traditional Segmentation: Finding Homogeneous Regions gPb-owt-ucm: P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik. PAMI 2010.

Bottom-up Object Segmentation Conventional Bottom-up Segmentation Proposed approach • Split multiple times • Retain object-like segmentations High redundancy Credit: J. Carreira

Bottom-up Object Segmentation A single multi-region segmentation or a hierarchy Credit: J. Carreira

Proposed Bottom-up Object Segmentation robust set of overlapping figure-ground segmentations Credit: J. Carreira single-shot multi-region segmentation superpixels Segments with object-like regularities

Constrained Parametric Min-Cuts for Automatic Object Segmentation Figure ground segmentation by growing regions around seeds parametric max-flow solver Ranking Credit: J. Carreira

Constrained Parametric Min-Cuts for Automatic Object Segmentation Credit: J. Carreira

Initialization • Foreground • Regular 5x5 grid geometry • Centroids of large N-Cuts regions • Centroids of superpixels closest to grid positions • Background • Full image boundary • Horizontal boundaries • Vertical boundaries • All boundaries excluding the bottom one Performance broadly invariant to different initializations

Generating a segment pool:constrained min-cut object min cut hard constraint hard constraint background Credit: J. Carreira

Generating a Segment Pool:Constrained Parametric Min-Cuts Credit: J. Carreira

Generating a Segment Pool:Constrained Parametric Min-Cuts Can solve for all values of object bias in the same time complexity of solving a single min-cut using a parametric max-flow solver Credit: J. Carreira

Object Segmentation