220 likes | 477 Views
From Interactive to Semantic Image Segmentation. Varun Gulshan Supervisors: Prof. Andrew Blake Prof. Andrew Zisserman. 20 Jan 2012. Two segmentation tasks. sky. background. building. tree. tree. object. person. car. car. road. bench. Interactive segmentation.
E N D
From Interactive to SemanticImage Segmentation VarunGulshan Supervisors: Prof. Andrew Blake Prof. Andrew Zisserman 20 Jan 2012
Two segmentation tasks sky background building tree tree object person car car road bench Interactive segmentation Semantic segmentation
Thesis Flow Chapter 4: Star convexity Chapter 3: Texture Features Chapter 6: Superpixel based classification Chapter 5: Segmenting humans Low level cues Mid level cues Bounding box interaction + Top down cues Fully automatic segmentation Interactive segmentation Semantic segmentation
Chapter 3: Features for interactive segmentation Low level texture features for improving interactive segmentation methods.
Texture features Pure Texture Feature (L-shape): Texture + Gray Feature (L-shape):
Texture features Pure Texture Feature (Plus-shape): Texture + Gray Feature (Plus-shape):
Camouflage Image Dataset Introduced a dataset of 50 Camouflage images, to demonstrate the power of texture features.
Quantitative evaluation Gray +21% +14% RGB +4% +7% Huge gain in accuracy obtained using texture features on top of gray scale images. Significant improvement on top of RGB images.
Chapter 4: Star Convexity and Extensions Mid level shape constraints for reducing user effort in interactive segmentation systems.
Chapter 4: Star convexity Single Star Multiple Stars Geodesic Star
Robot user evaluation False negative False positive Initial brush strokes Error segmentation Segmentation output with current interaction New Brush Stroke New Brush Stroke New brush stroke placed Centre of connected component Biggest connected component Process is repeated upto 20 strokes Segmentation after 20 strokes Updated segmentation
Robot user evaluation Our method takes least effort
Chapter 5: Learning to segment humans Using top down cues to segment specific object categories.
Segmenting humans Bounding box (given/detected) Top down HOG prediction Bottom up refinement
Kinect Data Acquisition RGB image Kinect scene labels Cleaned up Ground truth Dataset of roughly 3500 images acquired using the Kinect
Top down learning Local Image Local HOG Local mask Classifier trained to predict segmentation masks for local windows based on their HOG descriptor.
Bottom up refinement ….. Top down segmentation Local Color model window Local color model unaries Final segmentation
Chapter 6: Semantic segmentation Fully automatic segmentation based upon learning from multiple superpixelisations.
Combing multiple superpixelisations GlobalPb Veksler QuickShift Various methods to learn from multiple superpixelisations explored: 1. Avg-Indep 2. Avg-Union 3. LPβ-Indep 4. IofR-Joint
Quantitative evaluation Single superpixelisation +7% +5% +6% +3% Multiple superpixelisations Combining multiple superpixelisations improves performance.
Novel pairwise features .... .... CRF trained jointly for appearance and novel pairwise features.