Shape

1. Shape Optimal invariant metrics for shape retrieval Michael Bronstein Department of Computer Science Technion – Israel Institute of Technology TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A

3. 3D warehouse Text search Person Man, person, human Tagged shapes Content-based search Shapes without metadata

4. Outline ? Feature descriptor Geometric words Bag of words

5. Invariance Rigid Scale Inelastic Topology Local geodesic distance histogram Gaussian curvature Heat kernel signature (HKS) Scale-invariant HKS (SI-HKS) Wang, B 2010

6. Heat kernels Heat equation governs heat propagation on a surface Initial conditions: heat distribution at time Solution : heat distribution at time Heat kernel is a fundamental solution of the heat equation with point heat source at (heat value at point after time )

7. Heat kernel signature can be interpreted as probability of Brownian motion to return to the same point after time (represents “stability” of the point) Multiscale local shape descriptor Time (scale) Sun, Ovsjanikov & GuibasSGP 2009

8. Heat kernel signature Heat kernel signatures represented in RGB space Sun, Ovsjanikov, Guibas SGP 2009 Ovsjanikov, BB & Guibas NORDIA 2009

9. Scale invariance Original shape Scaled by HKS= HKS= Not scale invariant! B, Kokkinos CVPR 2010

10. 0 -5 4 0 -10 -0.01 3 -0.02 2 -15 0 100 200 300 -0.03 t 1 -0.04 0 0 100 200 300 0 2 4 6 8 10 12 14 16 18 20 t • =2k/T Scale-invariant heat kernel signature Log scale-space log + d/d Fourier transform magnitude Scaling = shift and multiplicative constant in HKS Undo scaling Undo shift B, Kokkinos CVPR 2010

11. Scale invariance Heat Kernel Signature Scale-invariant Heat Kernel Signature B, Kokkinos CVPR 2010

12. Scale invariance Heat Kernel Signature Scale-invariant Heat Kernel Signature B, Kokkinos CVPR 2010

13. Modeling vs learning Wang, B 2010

14. Learning invariance T Positives P Negatives N

15. Similarity learning positive false positive negative false negative with high probability

16. Similarity-preserving hashing -1 -1 +1 -1 -1 -1 -1 +1 -1 -1 +1 +1 = # of distinct bits +1 +1 -1 +1 +1 +1 +1 -1 +1 +1 +1 +1 with high probability Collision: with low probability Gionis, Indik, Motwani 1999 Shakhnarovich 2005

17. Boosting • Construct 1D embedding -1 +1 • Similarity is approximated by • Downweight pairs with • Upweight pairs with BBK 2010; BB Ovsjanikov, Guibas 2010 Shakhnarovich 2005

18. Boosting • Construct 1D embedding -1 -1 -1 +1 +1 -1 -1 +1 • Similarity is approximated by +1 +1 • Downweight pairs with • Upweight pairs with BBK 2010; BB Ovsjanikov, Guibas 2010 Shakhnarovich 2005

19. SHREC 2010 dataset

20. Total dataset size: 1K shapes (715 queries) • Positives: 10K • Negatives: 100K BB et al, 3DOR 2010 SHREC 2010 dataset

21. BB et al, 3DOR 2010 ShapeGoogle with HKS descriptor

22. BB et al, 3DOR 2010 ShapeGoogle with SI-HKS descriptor

23. BB et al, 3DOR 2010 Similarity sensitive hashing (96 bit)

24. WaldHash • Construct embeddingby maximizing positive • Early decision negative • Remove pairs with and sample in new pairs into the training set • Downweight pairs with • Upweight pairs with B2, Ovsjanikov, Guibas 2010

25. 30% B2, Ovsjanikov, Guibas 2010

26. Cross-modal similarity Modality 1 Modality 2 Incommensurable spaces! Triangular meshes Point clouds How to compare apples to oranges? BB, Michel, Paragios CVPR 2010

27. Cross-modality embedding Modality 1 Modality 2 BB, Michel, ParagiosCVPR 2010 with high probability

28. Cross-modality hashing Modality 1 Modality 2 -1 -1 +1 -1 -1 -1 -1 +1 -1 -1 +1 +1 +1 +1 -1 +1 +1 +1 +1 -1 +1 -1 -1 +1 with high probability Collision: with low probability BB, Michel, ParagiosCVPR 2010

29. Cross-representation 3D shape retrieval Database Query 1052 shapes 8x8 dimensional bag of expressions 32-dimensional bag of words BB, Michel, ParagiosCVPR 2010

30. Retrieval performance Mean average precision BB, Michel, ParagiosCVPR 2010 Number of bits