Shape Matching for Model Alignment in 3D Scans

1. Shape Matching for Model Alignment3D Scan Matching and Registration, Part IICCV 2005 Short Course Michael Kazhdan Johns Hopkins University

2. Goal Given two partially overlapping scans, compute the transformation that merges the two. Aligned Scans Partially Overlapping Scans

3. Approach • (Find feature points on the two scans) Partially Overlapping Scans

4. Approach • (Find feature points on the two scans) • Establish correspondences Partially Overlapping Scans

5. Approach • (Find feature points on the two scans) • Establish correspondences • Compute the aligning transformation Aligned Scans Partially Overlapping Scans

6. Outline Global-Shape Correspondence • Shape Descriptors • Alignment Partial-Shape/Point Correspondence • From Global to Local • Pose Normalization • Partial Shape Descriptors Registration • Closed Form Solutions • Branch and Bound • Random Sample Consensus

7. Correspondence Goal: Identify when two points on different scans represent the same feature ?

8. Correspondence Goal: Identify when two points on different scans represent the same feature: Are the surrounding regions similar? ?

9. Global Correspondence More Generally: Given two models, determine if they represent the same/similar shapes. ? ≈

10. Global Correspondence More Generally: Given two models, determine if they represent the same/similar shapes. ? ≈ Models can have different: representations, tessellations, topologies, etc.

11. Global Correspondence Approach: • Represent each model by a shape descriptor: • A structured abstraction of a 3D model • That captures salient shape information

12. Global Correspondence Approach: • Represent each model by a shape descriptor. • Compare shapesby comparing theirshape descriptors. ? ≈

13. Shape Descriptors: Examples Shape Histograms Shape descriptor stores a histogram of how much surface area resides within different concentric shells in space. [Ankerst et al. 1999] Represents a 3D model by a 1D (radial) array of values Shape Histogram (Sectors + Shells) Model

14. Shape Descriptors: Examples Shape Histograms Shape descriptor stores a histogram of how much surface area resides within different sectors in space. [Ankerst et al. 1999] Represents a 3D model by a 2D (spherical) array of values Shape Histogram (Sectors + Shells) Model

15. Shape Descriptors: Examples Shape Histograms Shape descriptor stores a histogram of how much surface area resides within different shells and sectors in space. [Ankerst et al. 1999] Represents a 3D model by a 3D (spherical x radial) array of values Shape Histogram (Sectors + Shells) Model

16. Shape Descriptors: Challenge The shape of a model does not change when a rigid body transformation is applied to the model. Translation ≈ Rotation

17. Shape Descriptors: Challenge In order to compare two models,we need to compare themat their optimal alignment. -

18. Shape Descriptors: Challenge - In order to compare two models,we need to compare themat their optimal alignment. - - - = min -

19. Shape Descriptors: Alignment Three general methods: • Exhaustive Search • Normalization • Invariance

20. Shape Descriptors: Alignment Exhaustive Search: • Compare at all alignments Exhaustive search for optimal rotation

21. Shape Descriptors: Alignment Exhaustive Search: • Compare at all alignments • Correspondence is determined by the alignment at which models are closest Exhaustive search for optimal rotation

22. Shape Descriptors: Alignment Exhaustive Search: • Compare at all alignments • Correspondence is determined by the alignment at which models are closest Properties: • Gives the correct answer • Even with fast signal processing tools, it is hard to do efficiently

23. Shape Descriptors: Alignment Normalization: Put each model into a canonical frame • Translation: • Rotation: Translation Rotation

24. Shape Descriptors: Alignment Normalization: Put each model into a canonical frame • Translation: Center of Mass • Rotation: Initial Models Translation-Aligned Models

25. PCA Alignment Shape Descriptors: Alignment Normalization: Put each model into a canonical frame • Translation: Center of Mass • Rotation: PCA Alignment Translation-Aligned Models Fully Aligned Models

26. Shape Descriptors: Alignment Normalization: Put each model into a canonical frame • Translation: Center of Mass • Rotation: PCA Alignment Properties: • Efficient • Not always robust • Cannot be used for feature matching

27. Shape Descriptors: Alignment Invariance: Represent a model by a shape descriptor that is independent of the pose. -

28. Shape Descriptors: Alignment Example: Ankerst’s Shells A histogram of the radial distribution of surface area. [Ankerst et al. 1999] Area Radius Shells Histogram

29. Shape Descriptors: Alignment Invariance: Power spectrum representation • Fourier transform for translations • Spherical harmonic transform for rotations Energy Energy Frequency Frequency Circular Power Spectrum Spherical Power Spectrum

30. Translation Invariance 1D Function

31. Translation Invariance … = + + + + 1D Function Cosine/Sine Decomposition

32. Translation Invariance … = + + + + 1D Function = Constant Frequency Decomposition

33. Translation Invariance … = + + + + + 1D Function + = Constant 1st Order Frequency Decomposition

34. Translation Invariance … = + + + + + 1D Function + + = Constant 1st Order 2nd Order Frequency Decomposition

35. Translation Invariance … = + + + + + 1D Function … + + + + = Constant 1st Order 2nd Order 3rd Order Frequency Decomposition

36. Translation Invariance Frequency subspaces are fixed by rotations:

37. Translation Invariance Frequency subspaces are fixed by rotations:

38. Translation Invariance Frequency subspaces are fixed by rotations:

39. Translation Invariance Frequency subspaces are fixed by rotations:

40. Translation Invariance Amplitudes invariantto translation … = + + + + + 1D Function … + + + + = Constant 1st Order 2nd Order 3rd Order Frequency Decomposition

41. = + + + 3rd Order Constant 1st Order 2nd Order + + + Rotation Invariance Represent each spherical function as a sum of harmonic frequencies (orders)

42. Rotation Invariance Frequency subspaces are fixed by rotations:

43. Rotation Invariance Frequency subspaces are fixed by rotations:

44. Rotation Invariance Frequency subspaces are fixed by rotations:

45. Rotation Invariance Frequency subspaces are fixed by rotations:

46. + + + 3rd Order Constant 1st Order 2nd Order + + + Rotation Invariance Store “how much” (L2-norm) of the shape resides in each frequency to get a rotation invariant representation =

47. Shape Descriptors: Alignment Invariance: Represent a model by a shape descriptor that is independent of the pose. Properties: • Compact representation • Not always discriminating

48. Outline Global-Shape Correspondence • Shape Descriptors • Alignment Partial-Shape/Point Correspondence • From Global to Local • Pose Normalization • Partial Shape Descriptors Registration • Closed Form Solutions • Branch and Bound • Random Sample Consensus

49. From Global to Local To characterize the surface about a point p, take a global descriptor and: • center it about p (instead of the COM), and • restrict the extent to a small region about p. p Shape histograms as local shape descriptors

50. From Global to Local Given scans of a model: Scan 2 Scan 1