Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories

1. Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories Svetlana Lazebnik, Cordelia Schmid, Jean Ponce Presented by: Lubomir Bourdev Many of the slides by: Svetlana Lazebnik

2. Key Idea • Pyramid Match Kernel (Grauman & Darrell) Pyramid in feature space, ignore location • Spatial Pyramid (this work) Pyramid in image space, quantize features

3. Algorithm • Extract interest point descriptors (dense scan) • Construct visual word dictionary • Build spatial histograms • Create intersection kernels • Train an SVM

4. Algorithm • Extract interest point descriptors (dense scan) • Construct visual word dictionary • Build spatial histograms • Create intersection kernels • Train an SVM OR Weak (edge orientations) Strong (SIFT)

5. Algorithm • Extract interest point descriptors (dense scan) • Construct visual word dictionary • Build spatial histograms • Create intersection kernels • Train an SVM • Vector quantization • Usually K-means clustering • Vocabulary size (16 to 400)

6. Algorithm • Extract interest point descriptors (dense scan) • Construct visual word dictionary • Build spatial histograms • Create intersection kernels • Train an SVM

7. Algorithm • Extract interest point descriptors (dense scan) • Construct visual word dictionary • Build spatial histograms • Create intersection kernels • Train an SVM

8. Algorithm • Extract interest point descriptors (dense scan) • Construct visual word dictionary • Build spatial histograms • Create intersection kernels • Train an SVM

16. My experiment: Butterfly Classification Peacock Zebra

17. Butterflies • Dataset from Lazebnik / Schmid / Ponce 70 train / 64 test 50 train / 41 test • Images centered on the butterfly • Significant background clutter • Large pose/viewpoint variations • Scale variations: up to x4

18. Butterfly Results Spatial pyramid levels: 1 (No pyramid) Spatial pyramid levels: 4