230 likes | 458 Views
GrabCut Interactive Image (and Stereo) Segmentation Joon Jae Lee Keimyung University. Characteristics. Improved from graph cut – Use Gaussian Mixture Model (GMM) for clustering in color space – Iterative optimization User interaction greatly reduced compared
E N D
GrabCutInteractive Image(and Stereo) Segmentation Joon Jae LeeKeimyung University
Characteristics Improved from graph cut – Use Gaussian Mixture Model (GMM) for clustering in color space – Iterative optimization User interaction greatly reduced compared with other methods (Magic Wand, Intelligent Scissors)
GrabCut – Interactive Foreground Extraction1 Photomontage
GrabCut – Interactive Foreground Extraction3 Problem Fast & Accurate ?
GrabCut – Interactive Foreground Extraction4 What GrabCut does Magic Wand(198?) Intelligent ScissorsMortensen and Barrett (1995) GrabCut User Input Result Regions Regions & Boundary Boundary
Gaussian Mixture Model The probability of a pixel vector is represented as:
Gaussian Mixture Model Two sets of GMM, one for background, one for unknown region
Algorithm For each pixel assign a value α α = 0 for background and α = 1 for foreground Use graph cut to minimize a Gibbs energy: α: background / foreground label θ: GMM parameters k: GMM labeling z: pixel value
Iterative minimization Initialize background pixel to α = 0 and unknown region (draw box) to α = 1. Initialize two sets of GMMs. 1. Assign GMM labels to each pixel. (Which set of GMM is determined by α) 2. Graph cut minimize (α optimized, GMM label changed to corresponding set of GMM) 3. Update GMM parameters 4. Repeat from step 1 until convergence
Foreground (source) Min Cut Background(sink) Cut: separating source and sink; Energy: collection of edges Min Cut: Global minimal enegry in polynomial time GrabCut – Interactive Foreground Extraction7 Graph Cuts - Boykov and Jolly (2001) Image
GrabCut – Interactive Foreground Extraction9 Iterated Graph Cuts Guaranteed toconverge 1 3 4 2 Result Energy after each Iteration
GrabCut – Interactive Foreground Extraction10 Colour Model Gaussian Mixture Model (typically 5-8 components) R R Iterated graph cut Foreground &Background Foreground G Background G Background
Coherence Model An object is a coherent set of pixels: 25 Error (%) over training set: How do we choose ? 25
GrabCut – Interactive Foreground Extraction13 Parameter Learning - Problems A Gaussian MRF is not a realistic texture model syntheticGMRF Gaussian? Real Image Gaussian!
GrabCut – Interactive Foreground Extraction14 Moderately simple examples … GrabCut completes automatically
GrabCut – Interactive Foreground Extraction15 Difficult Examples Camouflage & Low Contrast Fine structure No telepathy Initial Rectangle InitialResult
GrabCut – Interactive Foreground Extraction16 Evaluation – Labelled Database Available online: http://research.microsoft.com/vision/cambridge/segmentation/
Error Rate: 0.72% GrabCut – Interactive Foreground Extraction17 Comparison Boykov and Jolly (2001) GrabCut User Input Result Error Rate: 1.87% Error Rate: 1.81% Error Rate: 1.32% Error Rate: 1.25% Error Rate: 0.72%
GrabCut – Interactive Foreground Extraction18 Comparison BimapGrabCut Error Rate: 2.13% Input Image Ground Truth Trimap Boykov and Jolly Error Rate: 1.36% Error rate - modestly increase User Interactions - considerable reduced
GrabCut – Interactive Foreground Extraction19 Results Parameter Learning
GrabCut – Interactive Foreground Extraction20 Comparison Intelligent Scissors Mortensen and Barrett (1995) LazySnappingLi et al. (2004) Graph Cuts Boykov and Jolly (2001) Magic Wand (198?) GrabCutRother et al. (2004)