Shape-Based Retrieval of Articulated 3D Models Using Spectral Embedding

1. Shape-Based Retrieval of Articulated3D Models Using Spectral Embedding Varun Jain and Hao Zhang {vjain,haoz}@cs.sfu.ca GrUVi Lab, School of Computing Science Simon Fraser University, Burnaby, BC Canada

2. Problem Overview

3. … … Problem Overview Database • Outline • Problem Overview • Retrieval Problem • Methods • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements Shape Retrieval Applications • Computer aided design • Game design • Shape recognition • Face recognition Query Interface User Output

4. Shape Retrieval • Outline • Problem Overview • Retrieval Problem • Methods • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements How? • Using Correspondence Efficiency?? • Using Global Descriptors For 3D shapes • Fourier Descriptors • Light Field • Spherical Harmonics • Skeletal Graph Matching • Non-rigid transformations?? • Stretching • Articulation (bending)

5. Shape Retrieval • Outline • Problem Overview • Retrieval Problem • Methods • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements Our Method • Normalize non-rigid transformations • Construct affinity matrix • Spectral embedding • Use global shape descriptors • Light Field Descriptor (LFD) • Spherical Harmonics Descriptor (SHD) • Eigenvalues??

6. Shape Retrieval • Outline • Problem Overview • Retrieval Problem • Methods • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements Advantages of Our Method • Handles shape articulation (best performance for articulated shapes). • Flexibility of affinity matrices • Robustness of affinity matrices

7. Spectral Embeddings

8. 2 ( ) d i j n g e o ; 2 a e 2 = ¾ i j a a a 1 1 1 2 1 ¢ ¢ ¢ n z } | { a 8 2 3 2 1 ¢ ¢ ¢ > . . . > . . . > > 6 7 . . . A = > n > 6 7 > a a a > 6 7 n n < i i i 1 2 6 7 ¢ ¢ ¢ n £ . . . 6 7 . . . 6 7 . . . > > 6 7 > a a a > 6 7 > > 4 5 1 2 ¢ ¢ ¢ n n n n > > : Spectral Embeddings Affinity matrix • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements

9. ¯ ¯ ¯ T A E ¤ E ¯ ¯ ¯ = ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ k ¯ ¯ ¯ ¯ ¯ ¯ k-dimensional spectral embedding coordinates of ith point of P z } | { ~ ~ ~ ¯ ¯ ¯ u u ¢ ¢ ¢ u k 1 2 ¯ ¯ ¯ 8 2 3 ¯ ¯ ¯ ¯ ¯ ¯ > > > > 6 7 > > 6 7 > . . . > 6 7 > . . . > 6 7 . . . 1 < 6 7 U E ¤ 2 ¢ ¢ ¢ u u u n = = k k k i i i 1 2 6 7 k 6 7 > > 6 . . . 7 > . . . > 6 7 . . . > > 6 7 > > 4 5 > > : Spectral Embeddings Eigenvalue decomposition: Scaled eigenvectors: • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements Jain, V., Zhang, H.: Robust 3D Shape Correspondence in the Spectral Domain. Proc. Shape Modeling International 2006.

10. ~ ~ ~ u u u 2 3 4 8 2 3 ¯ ¯ > ¯ ¯ > > > 6 7 ¯ ¯ > > 6 7 > ¯ ¯ . . . > 6 7 > . . . ¯ ¯ > 6 7 . . . < 6 7 U ¯ ¯ u u u n = i i i 2 3 4 3 6 7 ¯ ¯ 6 7 > > 6 7 . . . ¯ ¯ > . . . > 6 7 . . . ¯ ¯ > > 6 7 > ¯ ¯ > 4 5 > > ¯ ¯ : ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ Spectral Embeddings • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements Examples of 3D embeddings

11. f g ¾ ¾ ¾ k 1 2 ; ; : : : ; 1 ³ ´ 2 k k 2 u i ¾ = i n p ¸ i = r P ¸ i i Spectral Embeddings EigenValue Descriptor (EVD): • Use deviation in projected data as descriptor: • Our shape descriptor: • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements

12. 2 3 b b b 2 3 ¢ ¢ ¢ 1 1 1 2 ¢ ¢ ¢ 1 a a a 1 1 1 2 1 n n 2 R A b ¢ ¢ ¢ = 2 1 ¢ ¢ ¢ a 6 7 2 1 6 7 6 7 T 6 7 R A A . . . 6 7 = . . . 6 7 . . . . . . 6 7 . . . B 6 7 . . . A = 6 7 = b b b 6 7 ¢ ¢ ¢ i i i 1 2 6 7 ¢ ¢ ¢ a a a n i i i 1 2 6 7 n 6 7 R E E ¤ 6 7 = 6 . . . 7 6 . . . 7 . . . 4 5 . . . . . . 4 5 . . . b b b ¢ ¢ ¢ 1 2 n n n n ¢ ¢ ¢ a a a 1 2 n n n n Spectral Embeddings Why use eigenvalues?? • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements

13. 2 2 ( ) ( ) l k O O + n o g n n Spectral Embeddings • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements Problems: • Geodesic distance computation • Efficiency of geodesic distance computation & eigendecomposition:

14. Spectral Embeddings • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements Geodesics using Structural Graph: • Add edges to make mesh connected • Geodesic distance ≈ Shortest graph distance Problem: Unwanted (topology modifying) edges! Solution: Add shortest possible edges. Choice of graph to take edges from: • p-nearest neighbor (may not return connected graph) • p-edge connected [Yang 2004]

15. 2 3 i i e g e n . . m a x - m n l i t t e x r a p o a o n . . 8 2 3 . . d i i t ¯ ¯ ¯ ¯ b l i e c o m p o s o n 6 7 s u s a m p n g ( ) ~ ~ l n O E u ¢ ¢ ¢ u n > = ¯ ¯ ¯ ¯ l l l 1 £ 6 7 3 ( ) l O > ( ) l l O z } | { > 4 5 n o g n ¯ ¯ ¯ ¯ . . . > 6 7 8 2 3 . . > . . . ¯ ¯ ¯ ¯ ¢ ¢ ¢ a a a . . 1 1 1 2 1 > 6 7 n . . . . . < > ¯ ¯ ¯ ¯ 6 7 > ~ ~ ~ E > ¢ ¢ ¢ a u u u n 2 1 = ¯ ¯ ¯ ¯ 2 3 4 3 £ > 6 7 6 7 n > > 6 7 ¯ ¯ ¯ ¯ 6 7 > > . . . > 6 7 ¯ ¯ ¯ ¯ > 6 . . . 7 < . . . > . . . 6 7 . . . A > 4 5 ¯ ¯ ¯ ¯ n . . . = > £ 6 7 n n > ¯ ¯ ¯ ¯ ¢ ¢ ¢ a a a i i i 1 2 6 : 7 n > ¯ ¯ ¯ ¯ > 6 7 > ¯ ¯ ¯ ¯ > 6 . . . 7 > . . . > 4 5 ¯ ¯ ¯ ¯ . . . > > ¯ ¯ ¯ ¯ : ¢ ¢ ¢ a a a 1 2 n n n n ¯ ¯ ¯ ¯ ¯ ¯ ¯ ¯ n z } | { 8 2 3 ¢ ¢ ¢ a a a 1 1 1 2 1 n > > ¢ ¢ ¢ a a a 2 1 2 2 2 < n 6 7 l A = l l £ 6 7 . . . 4 5 . . . > . . . > : ¢ ¢ ¢ a a a l l l 1 2 n Spectral Embeddings • Outline • Problem Overview • Spectral Embeddings • Basics • Problems • Solutions • Shape Descriptors • Results • Future Work • Acknowledgements Efficiency with Nyström approximation

16. Shape Descriptor

17. 2 k £ ¤ p ¸ p 1 ¡ ° i i X ( ) S C P Q i t m o s = ; p 2 ¸ p + ° i i i 1 = Shape Descriptor • Outline • Problem Overview • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements • Global Shape Descriptors • Light Field Descriptor (LFD) • Spherical Harmonics Descriptor (SHD) • Our Similarity Measure (EVD):

18. Results

19. Experimental Database • Outline • Problem Overview • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements McGill 3D Articulated Shapes Database http://www.cim.mcgill.ca/~shape/benchMark/

20. Precision-Recall plot for McGill database Results • Outline • Problem Overview • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements

21. McGill articulated shape database Results • Outline • Problem Overview • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements

22. Limitations & Future Work • Outline • Problem Overview • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements • Non-robustness of geodesic distances • Non-robustness to outliers

23. Acknowledgements • Outline • Problem Overview • Spectral Embeddings • Shape Descriptors • Results • Future Work • Acknowledgements • McGill 3D Shape Benchmark. • Phil Shilane (LFD & SHD implementations).

24. Thank You! for you attention