Rock, paper, and scissors Joint extrinsic and intrinsic similarity of non-rigid shapes

1. Rock, paper, and scissors Joint extrinsic and intrinsic similarity of non-rigid shapes Alex Bronstein, Michael Bronstein, Ron Kimmel Department of Computer Science Technion – Israel Institute of Technology

2. Extrinsic vs intrinsic similarity • Rock, paper, and scissors: is the hand similar to a rock? Is it similar to another posture of a hand? • The answer depends on the definition of similarity. Extrinsic similarity Intrinsic similarity • Are the shapes congruent? • Do the shapes have the same metric structure?

3. Extrinsic similarity • Can be expressed as a distance between two shapes and • Find a rigid motion bringing the shapes into best alignment • Misalignment is quantified using the Hausdorff distance or some of its variants • Computed using ICP algorithms

4. Extrinsic similarity – limitations Extrinsically similar Extrinsically dissimilar • Suitable for nearly rigid shapes • Unsuitable for non-rigid shapes

5. Intrinsic similarity • Compare the intrinsic geometries of two shapes • Intrinsic geometry is expressed in terms of geodesic distances • Geodesic distances are computed using Dijkstra’s shortest path algorithm or fast marching Geodesic distance Euclidean distance

6. Intrinsic similarity – canonical forms • Embed intrinsic geometries of and into a common metric space • Minimum-distortion embeddings and computed using multidimensional scaling (MDS) algorithms • Compare the images and as rigid shapes A. Elad, R. Kimmel, CVPR 2001

7. Intrinsic similarity – GMDS • Find the minimum distortion embedding of one shape into the other • The minimum distortion is the measure of intrinsic dissimilarity • Computed using the generalized MDS BBK, PNAS 2006

8. Intrinsic similarity – limitations Intrinsically similar Intrinsically dissimilar • Suitable for near-isometric shape deformations • Unsuitable for deformations modifying shape topology

9. Extrinsically similar Intrinsically dissimilar Extrinsically dissimilar Intrinsically similar Extrinsically dissimilar Intrinsically dissimilar Desired result: THIS IS THE SAME SHAPE!

10. Joint extrinsic and intrinsic similarity • Combine intrinsic and extrinsic similarities into a single criterion • Find a deformation of whose intrinsic geometry is similar to and extrinsic geometry is more similar to • defines the relative importance of intrinsic and extrinsic criteria • is a collection of optimal tradeoffs between intrinsic and extrinsic criteria • Can be formalized using the notion of Pareto optimality

11. Extrinsic similarity Intrinsic similarity

12. Computation of joint similarity • Hybridization of ICP and GMDS in L2 formulation for robustness • Fix correspondence between and for intrinsic similarity where is precomputed and are computed at each iteration • Closest-point distance for extrinsic similarity where are the closest points to in • More details in the paper

13. Numerical example – dataset = topology change Data: tosca.cs.technion.ac.il

14. Numerical example – tradeoff curves Dissimilar Similar

15. Numerical example – intrinsic similarity = topology-preserving no topology changes

16. Numerical example – intrinsic similarity = topology-preserving = topology change

17. Numerical example – extrinsic similarity = topology-preserving = topology change

18. Numerical example – joint similarity = topology-preserving = topology change

19. Numerical example – ROC curves

20. Numerical example – shape morphing Stronger intrinsic similarity (smaller λ) Stronger extrinsic similarity (larger λ)

21. Conclusion • Extrinsic similarity is insensitive to topology changes, but sensitive to non-rigid deformations • Intrinsic similarity is insensitive to nearly-isometric non-rigid deformations, but sensitive to topology changes • Joint similarity is insensitive to both non-rigid deformations and topology changes • Can be used to produce near-isometric morphs

22. References • A. M. Bronstein, M. M. Bronstein, A. M. Bruckstein, R. Kimmel, Analysis of two-dimensional non-rigid shapes, IJCV, to appear. • A. M. Bronstein, M. M. Bronstein, R. Kimmel, Rock, Paper, and Scissors: extrinsic vs. intrinsic similarity of non-rigid shapes, Proc. ICCV, (2007). • I. Eckstein, J. P. Pons, Y. Tong, C. C. J. Kuo, and M. Desbrun, Generalized surface flows for mesh processing, Proc. SGP, (2007). • M. Kilian, N. J. Mitra, and H. Pottmann, Geometric modeling in shape space, Proc. SIGGRAPH, vol. 26, (2007). • A. M. Bronstein, M. M. Bronstein, A. M. Bruckstein, R. Kimmel, Paretian similarity for partial comparison of non-rigid objects, Proc. SSVM, pp. 264-275, 2007. • A. M. Bronstein, M. M. Bronstein, R. Kimmel, Calculus of non-rigid surfaces for geometry and texture manipulation, IEEE TVCG, Vol. 13/5, pp. 902-913, (2007). • A. M. Bronstein, M. M. Bronstein, R. Kimmel, Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching, PNAS, Vol. 103/5, pp. 1168-1172, (2006).

23. References • F. Mémoli and G. Sapiro, A theoretical and computational framework for isometry invariant recognition of point cloud data, Foundations of Computational Mathematics 5 (2005), 313-346. • N. J. Mitra, N. Gelfand, H. Pottmann, and L. Guibas, Registration of point cloud data from a geometric optimization perspective, Proc. SGP, (2004), pp. 23-32. • A. Elad, R. Kimmel, On bending invariant signatures for surfaces, Trans. PAMI 25 (2003), no. 10, 1285-1295. • P. J. Besl and N. D. McKay, A method for registration of 3D shapes, Trans. PAMI 14 (1992), 239-256. • Y. Chen and G. Medioni, Object modeling by registration of multiple range images, Proc. Conf. Robotics and Automation, (1991). • E. L. Schwartz, A. Shaw, and E. Wolfson, A numerical solution to the generalized mapmaker's problem: flattening nonconvex polyhedral surfaces, Trans. PAMI 11 (1989), 1005-1008.

24. Shameless advertisement COMING SOON… • Published by Springer Verlag • To appear in early 2008 • Approximately 320 pages • Over 50 illustrations • Color figures • Book website tosca.cs.technion.ac.il