1 / 25

Learning Decompositional Shape Models from Examples

Learning Decompositional Shape Models from Examples. Alex Levinshtein Cristian Sminchisescu Sven Dickinson University of Toronto. Hierarchical Models. Manually built hierarchical model proposed by Marr And Nishihara

walda
Download Presentation

Learning Decompositional Shape Models from Examples

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Learning Decompositional Shape Models from Examples Alex Levinshtein Cristian Sminchisescu Sven Dickinson University of Toronto

  2. Hierarchical Models Manually built hierarchical model proposed by Marr And Nishihara (“Representation and recognition of the spatial organization of three dimensional shapes”, Proc. of Royal Soc. of London, 1978)

  3. Our goal Automatically construct a generic hierarchical shape model from exemplars • Challenges: • Cannot assume similar appearance among different exemplars • Generic features are highly ambiguous • Generic features may not be in one-to-one correspondence

  4. Automatically constructed Hierarchical Models Input: Question: What is it? Output:

  5. Stages of the system Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Model parts Extract Decomposition Relations Extract Attachment Relations Model decomposition relations Model attachment relations Assemble Final Model

  6. Blob Graph Construction Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Model parts Extract Decomposition Relations Extract Attachment Relations Model decomposition relations Model attachment relations Assemble Final Model

  7. Blob Graph Construction • On the Representation and Matching of Qualitative Shape at Multiple Scales • A. Shokoufandeh, S. Dickinson, C. Jonsson, L. Bretzner, and T. Lindeberg,ECCV 2002 • Edges are invariant to articulation • Choose the largest connected component.

  8. Feature matching Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Model parts Extract Decomposition Relations Extract Attachment Relations Model decomposition relations Model attachment relations Assemble Final Model

  9. Feature matching One-to-one matching. Rely on shape and context, not appearance! Many-to-many matching ?

  10. Feature embedding and EMD Spectral embedding

  11. Returning to our set of inputs • Many-to-many matching of every pair of exemplars.

  12. Part Extraction Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Model parts Extract Decomposition Relations Extract Attachment Relations Model decomposition relations Model attachment relations Assemble Final Model

  13. Results of the part extraction stage

  14. What is next?

  15. Extracting attachment relations Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Model parts Extract Decomposition Relations Extract Attachment Relations Model decomposition relations Model attachment relations Assemble Final Model

  16. Extracting attachment relations Right arm is typically connected to torso in exemplar images !

  17. Extracting decomposition relations Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Model parts Extract Decomposition Relations Extract Attachment Relations Model decomposition relations Model attachment relations Assemble Final Model

  18. Extracting decomposition relations

  19. Model construction stage summary Model Construction: • Clustering blobs into parts based on one-to-one matching results. • Recovering relations between parts based on individual matching and attachment results.

  20. Assemble Final Model Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Model parts Extract Decomposition Relations Extract Attachment Relations Model decomposition relations Model attachment relations Assemble Final Model

  21. Conclusions • General framework for constructing a generic decompositional model from different exemplars with dissimilar appearance. • Recovering decompositional relations requires solving the difficult many-to-many graph matching problem. • Preliminary results indicate good model recovery from noisy features.

  22. Future work • Construct models for objects other than humans. • Provide scale invariance during matching. • Automatically learn perceptual grouping relations from labeled examples. • Develop indexing and matching framework for decompositional models.

More Related