Discussion of Pictorial Structures

1. Discussion ofPictorial Structures Pedro FelzenszwalbDaniel HuttenlocherSicily WorkshopSeptember, 2006

2. What are Pictorial Structures? • Local appearance • Part models • Parts  feature detection • Global geometry • Not necessarily fully connected graph • Joint optimization • Combine appearanceand geometry withouthardconstraints • “Stretch and fit” • Qualitative

3. Pictorial Structure Models • Parts have match quality at each location • Location in a configuration space • No feature detection • Maps for parts combined together into overall quality map • According to underlying graph structure

4. A History of Pictorial Structures • Fischler and Elschlager original 1973 paper • Burl, Weber and Perona ECCV 1998 • Probabilistic formulation • Full joint Gaussian spatial model • Computational challenges led to feature-based • Felzenszwalb and Huttenlocher CVPR 2000 • Explicit revisiting of FE73 for trees, probabilistic • Efficient algorithms using distance transforms • Crandall et al CVPR 2005, ECCV 2006 • Low tree-width graph structures, unsupervised

5. Matching Pictorial Structures • Cost map for each part • Distance transform (soft max) using spatial model • Shift and combine • Localize root then recursively other parts

6. Learning Models • Automatically determine which spatial relationships to represent [FH03] • Weakly supervised learning [CH06] • Learn part appearance and geometric relations simultaneously • No labeling of part locations • Use large number of patches, similar to Ullman • Better detection accuracy than strongly supervised Car (rear) star topology

7. Parts as Context • No part detected without using context provided by other parts • Detect overall configuration composed of parts in a spatial arrangement • Allows for weak evidence for a part • Unlike feature detection • Combination of matches can constrain pose • In contrast to scene-level context • More spatial regularity

8. Factored Models • For n parts in fixed arrangement with k templates per part • Exponential number of possibilities, O(kn) • For variable arrangement, another exponential factor • Important both for representation and algorithmic efficiency • Pictorial structures takes particular advantage of this factoring

9. Closely Related Work • Ioffe and Forsyth, Ramanan and Forsyth human body pose • Part detection but very “dense” part locations • Constellation models • Fergus, Perona, Zisserman and others • Hard feature detection in contrast with BWP98 soft feature matching • Amit’s patch models • No assumption of independent part appearance • Fergus and Zisserman star models

10. What’s Important • No decisions until the end • No feature detection • Quality maps or likelihoods • No hard geometric constraints • Deformation costs or priors • Efficient algorithms • Dynamic programming critical or can’t get away without making intermediate decisions • Not applicable to all problems, need good factorizations of geometry and appearance

11. Some Pros • Good for categorical object recognition • Qualitative descriptions of appearance • Factoring variability in appearance and geometry • Deals well with occlusion • In contrast to hard feature detection • Weakly supervised learning algorithms • Sampling as way of dealing with models that don’t factor – more Saturday

12. Some Cons/Limitations • Most applicable to 2D objects defined by relatively small number of parts • Unclear how to extend to large number of transformation parameters per part • Explicit representation grows exponentially • No known way of using to index into model databases

13. Role of Spatial Constraints • For k-fans, spatial information substantially improves detection accuracy • However, limited by relatively small number of parts compared to features in a bag • General question