1 / 34

Matching Shapes Serge Belongie, Jitendra Malik and Jan Puzicha U.C. Berkeley

Learn about a matching framework for object recognition using shape contexts and TPS transformation, robust to outliers & noise, forming a basis for versatile recognition techniques.

akeely
Download Presentation

Matching Shapes Serge Belongie, Jitendra Malik and Jan Puzicha U.C. Berkeley

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Matching ShapesSerge Belongie, Jitendra Malikand Jan Puzicha U.C. Berkeley

  2. Silhouettes • Single, closed bounding contour • No holes • Dual of Medial Axis Transform

  3. Point Sets • Sampled points from edges or other feature detector • sparse or dense • Hausdorff distance, distance transform, shape space, graph matching

  4. Biological Shape • D’Arcy Thompson: On Growth and Form, 1917 • studied transformations between shapes of organisms

  5. Deformable Templates: Related Work • Fischler & Elschlager (1973) • Grenander et al. (1991) • Yuille (1991) • von der Malsburg (1993)

  6. Matching Framework ... model target • Find correspondences between points on shape • Estimate transformation • Measure similarity

  7. Comparing Pointsets

  8. Shape Context Count the number of points inside each bin, e.g.: Count = 4 ... Count = 10 • Compact representation of distribution of points relative to each point

  9. Shape Context

  10. Comparing Shape Contexts Compute matching costs using Chi Squared distance: Recover correspondences by solving linear assignment problem with costs Cij [Jonker & Volgenant 1987]

  11. Matching with Outliers “real” nodes “dummy” nodes : outlier cost

  12. Matching Framework ... model target • Find correspondences between points on shape • Estimate transformation • Measure similarity

  13. Thin Plate Spline Model • 2D counterpart to cubic spline: • Minimizes bending energy: • Solve by inverting linear system • Can be regularized when data is inexact Duchon (1977), Meinguet (1979), Wahba (1991)

  14. Coordinate Transformation using TPS

  15. MatchingExample model target

  16. Matching Example

  17. Synthetic Distortion Test I original deformation noise outlier

  18. Outlier Test Example

  19. Synthetic Test Results Fish - deformation + noise Fish - deformation + outliers ICP Shape Context Chui & Rangarajan

  20. Matching Framework ... model target • Find correspondences between points on shape • Estimate transformation • Measure similarity

  21. Terms in Similarity Score • Shape Context difference • Local Image appearance difference • orientation • gray-level correlation in Gaussian window • … (many more possible) • Bending energy We used the weighted sumShape context + 1.6*image appearance + 0.3*bending

  22. Object Recognition Experiments • COIL 3D Objects • Handwritten Digits • Trademarks • Exactly the same algorithm used on all experiments

  23. COIL Object Database

  24. Error vs. Number of Views

  25. Prototypes Selected for 2 Categories Details in Belongie, Malik & Puzicha (NIPS2000)

  26. Editing: K-medoids • Input: similarity matrix • Select: K prototypes • Minimize: mean distance to nearest prototype • Algorithm: • iterative • split cluster with most errors • Result: Adaptive distribution of resources

  27. Error vs. Number of Views

  28. Handwritten Digit Recognition • MNIST 600 000 (distortions): • LeNet 5: 0.8% • SVM: 0.8% • Boosted LeNet 4: 0.7% • MNIST 60 000: • linear: 12.0% • 40 PCA+ quad: 3.3% • 1000 RBF +linear: 3.6% • K-NN: 5% • K-NN (deskewed): 2.4% • K-NN (tangent dist.): 1.1% • SVM: 1.1% • LeNet 5: 0.95% • MNIST 20 000: • K-NN, Shape Context matching: 0.63%

  29. Results: Digit Recognition 1-NN classifier using:Shape context + 0.3 * bending + 1.6 * image appearance

  30. Trademark Similarity

  31. Shape Similarity: Kimia

  32. Quantitative Comparison Number correct rank

  33. Conclusion • Introduced new matching algorithm matching based on shape contexts and TPS • Robust to outliers & noise • Forms basis of object recognition technique that performs well in a variety of domains using exactly the same algorithm

More Related