1 / 38

Shape Matching and Object Recognition using Shape Contexts Jitendra Malik U.C. Berkeley (joint work with S. Belongie, J.

Shape Matching and Object Recognition using Shape Contexts Jitendra Malik U.C. Berkeley (joint work with S. Belongie, J. Puzicha, G. Mori). Outline. Shape matching and isolated object recognition Scaling up to general object recognition. Biological Shape.

kali
Download Presentation

Shape Matching and Object Recognition using Shape Contexts Jitendra Malik U.C. Berkeley (joint work with S. Belongie, J.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Shape Matching and Object Recognition using Shape ContextsJitendra MalikU.C. Berkeley(joint work with S. Belongie, J. Puzicha, G. Mori)

  2. Outline • Shape matching and isolated object recognition • Scaling up to general object recognition

  3. Biological Shape • D’Arcy Thompson: On Growth and Form, 1917 • studied transformations between shapes of organisms

  4. Deformable Templates: Related Work • Fischler & Elschlager (1973) • Grenander et al. (1991) • Yuille (1991) • von der Malsburg (1993)

  5. Matching Framework ... model target • Find correspondences between points on shape • Estimate transformation • Measure similarity

  6. Comparing Pointsets

  7. Shape Context Count the number of points inside each bin, e.g.: Count = 4 ... Count = 10 • Compact representation of distribution of points relative to each point

  8. Shape Context

  9. Shape Contexts • Invariant under translation and scale • Can be made invariant to rotation by using local tangent orientation frame • Tolerant to small affine distortion • Log-polar bins make spatial blur proportional to r Cf. Spin Images (Johnson & Hebert) - range image registration

  10. Comparing Shape Contexts Compute matching costs using Chi Squared distance: Recover correspondences by solving linear assignment problem with costs Cij [Jonker & Volgenant 1987]

  11. Matching Framework ... model target • Find correspondences between points on shape • Estimate transformation • Measure similarity

  12. Thin Plate Spline Model • 2D counterpart to cubic spline: • Minimizes bending energy: • Solve by inverting linear system • Can be regularized when data is inexact Duchon (1977), Meinguet (1979), Wahba (1991)

  13. MatchingExample model target

  14. Outlier Test Example

  15. Synthetic Test Results Fish - deformation + noise Fish - deformation + outliers ICP Shape Context Chui & Rangarajan

  16. Matching Framework ... model target • Find correspondences between points on shape • Estimate transformation • Measure similarity

  17. Terms in Similarity Score • Shape Context difference • Local Image appearance difference • orientation • gray-level correlation in Gaussian window • … (many more possible) • Bending energy

  18. Object Recognition Experiments • Kimia silhouette dataset • Handwritten digits • COIL 3D objects (Nayar-Murase) • Human body configurations • Trademarks

  19. Shape Similarity: Kimia dataset

  20. Quantitative Comparison Number correct rank

  21. Handwritten Digit Recognition • MNIST 600 000 (distortions): • LeNet 5: 0.8% • SVM: 0.8% • Boosted LeNet 4: 0.7% • MNIST 60 000: • linear: 12.0% • 40 PCA+ quad: 3.3% • 1000 RBF +linear: 3.6% • K-NN: 5% • K-NN (deskewed): 2.4% • K-NN (tangent dist.): 1.1% • SVM: 1.1% • LeNet 5: 0.95% • MNIST 20 000: • K-NN, Shape Context matching: 0.63%

  22. Results: Digit Recognition 1-NN classifier using:Shape context + 0.3 * bending + 1.6 * image appearance

  23. COIL Object Database

  24. Error vs. Number of Views

  25. Prototypes Selected for 2 Categories Details in Belongie, Malik & Puzicha (NIPS2000)

  26. Editing: K-medoids • Input: similarity matrix • Select: K prototypes • Minimize: mean distance to nearest prototype • Algorithm: • iterative • split cluster with most errors • Result: Adaptive distribution of resources (cfr. aspect graphs)

  27. Error vs. Number of Views

  28. Human body configurations

  29. Automatically Locating Keypoints • User marks keypoints on exemplars • Find correspondence with test shape • Transfer keypoint position from exemplar to the test shape.

  30. Results

  31. Trademark Similarity

  32. Outline • Shape matching and isolated object recognition • Scaling up to general object recognition • Many objects (Mori, Belongie & Malik, CVPR 01) • Gray scale matching (Berg & Malik, CVPR 01) • Objects in scenes (scanning or segmentation)

  33. Mori, Belongie, Malik (CVPR 01) • Fast Pruning • Given a query shape, quickly return a shortlist of candidate matches • Database of known objects will be large: ~30000 • Detailed Matching • Perform computationally expensive comparisons on only the few shapes in the shortlist

  34. Representative Shape Contexts • Match using only a few shape contexts • Don’t need to compare every one

  35. Snodgrass Results

  36. Results

  37. Conclusion • Introduced new matching algorithm matching based on shape contexts and TPS • Robust to outliers & noise • Forms basis of object recognition technique that performs well in a variety of domains using exactly the same algorithm

More Related