350 likes | 501 Views
Figure/Ground Assignment in Natural Images. Xiaofeng Ren, Charless Fowlkes and Jitendra Malik University of California, Berkeley. Computer Vision Group. ECCV 2006 Graz. Figure/Ground Organization. Ground (shapeless). Figure (face). Figure (Goblet). Ground (Shapeless).
E N D
Figure/Ground AssignmentinNatural Images Xiaofeng Ren, Charless Fowlkes and Jitendra Malik University of California, Berkeley Computer Vision Group ECCV 2006 Graz
Ground (shapeless) Figure (face) Figure (Goblet) Ground (Shapeless) Figure/Ground Organization • A contour belongs to one of the two (but not both) abutting regions. Important for the perception of shape
A Classical View of Perception Object and Scene Recognition Figure/Ground Organization Grouping / Segmentation Image
A Modern View Object and Scene Recognition Grouping / Segmentation Figure/Ground Organization
Focus of This Work Object and Scene Recognition Grouping / Segmentation Figure/Ground Organization
Overview • Introduction • Groundtruth for Figure/Ground • Local Figure/Ground Cues • Shapemes • Global Consistency • Conditional Random Field • Results and Quantitative Evaluation • Conclusion
G F Figure/Ground: Groundtruth
Figure/Ground Dataset [Martin, Fowlkes & Malik; ECVP 2003]
Cues for Figure/Ground • Local Cues • Gestalt Principles of Figure/Ground • Global Cues • Label Consistency at T-junctions [Kienker, Sejnowski, Hinton & Schumacher 1986] [Heitger & von der Heydt 1993] [Geiger, Kumaran & Parida 1996] [Saund 1999] [Yu, Lee and Kanade 2001] … …
local shapes …… Shapemes: Prototypical Local Shapes collect Use Geometric Blur [Berg & Malik 2001] cluster
parallelism straight line convexity line ending corner
G F G F G Gestalt Principles for Figure/Ground • Convexity • Parallelism • Surroundedness • Symmetry • Lower Region • Common Fate ……
L R L:93.8% L:89.6% L:66.5% L:49.8% L:11.7% L: 5.0% Shapemes for F/G Discrimination Which side is Figure? Train a logistic classifier to linearly combine the shapeme cues
F F G G G F G G F G F F common uncommon Global Consistency
Edge potentialsexp(ii) Junction potentialsexp(jj) where Conditional Random Fields (CRF) X={X1,X2,…,Xm} Xi{Left,Right}
Building a CRF Model • What are the features? • edge features: • Shapemes • junction features: • Junction type • How to make inference? • Loopy Belief Propagation • How to learn the parameters? • Gradient Descent on Max. Likelihood X={X1,X2,…,Xm} Estimate P(Xi|)
F G G G G G F F F F { (G,F),(F,G) } F G F G G F { (F,G),(F,G),(F,G) } Junction Features • One feature for each junction type { (F,G),(G,F),(F,G) } Junction potentials:
G F G G G F F F = -0.611 = 0.185 F F G G F G G G F G F F = 0.428 = -0.857 Learning Junction Weights
CSP > : contour direction + : convex edge - : concave edge possible junctions (constraints) Line Labeling • Probabilistic constraints instead of logical constraints • Training and testing on large datasets of natural images [Clowes 1971, Huffman 1971; Waltz 1972; Malik 1986]
Experiments • Using human-marked segmentations • Using edges computed by an edge detector
50% 50% 88% 88% Results 55.6% ------ 64.8% 64.9% 72.0% 66.5% 78.3% 68.9%
Using human segmentations Image Groundtruth Local Global
Using edge maps computed from an edge detector Image Edge Map Local Global
Summary • Human subjects label groundtruth figure/ground assignments in natural images. • Shapemes encode Gestalt structures and capture local figure/ground cues. • A conditional random field enforces global consistency at T-junctions and improves figure/ground performance. • Quantitative evaluations show that figure/ground organization has a promising solution. Object and Scene Recognition Grouping / Segmentation Figure/Ground Organization
Future Work Object and Scene Recognition Grouping / Segmentation Figure/Ground Organization
Future Work Object and Scene Recognition Grouping / Segmentation Figure/Ground Organization
Baseline: Size/Convexity Size(A) < Size(B): A is figure; B is ground Size(A) > Size(B): B is figure; A is ground B A
F F G G G G F F G F F G Continuity in Figure/Ground If a pair of edges belong to the same foreground, they should have a smooth connection.