Template based shape descriptor

1. Template based shape descriptor Raif Rustamov Department of Mathematics and Computer Science Drew University, Madison, NJ, USA

2. Components of descriptors in general • Selection of surface feature • Mapping • Signal Processing • Need this discussion to set up the context for our approach

3. Selection of surface feature • A function on the surface that captures a property relevant to shape description: • constant function (restriction of the surface's characteristic function to the surface itself) • distance to the center of mass • curvature • components of the normal vector • We refer to the selected function as the feature function.

4. Mapping • The feature function is used to construct a new function defined on some predetermined domain • The new domain called the “mapping domain” • the new function the “mapped feature function” • Common mapping domains: • Spheres • Planes • the 3D space (surface's bounding volume) • surface itself • Mapping procedures: • projection • Identity, if mapping domain = surface itself

5. Signal Processing • Extract concise noise-robust numerical descriptor from the mapped feature function. • Depends on the mapping domain: • Sphere – Spherical Harmonic Transform • Plane or box volume – 2D or 3D Fourier transform • Ball volume – 3D Zernike Transform. • Mapped feature function is expanded in a series in terms of the relevant basis • Expansion coefficients are used as the shape descriptor

6. Example I: Saupe, Vranic 2001 • Shoot rays from the origin (center of mass), determine the distance to the farthest intersection point with the bounding mesh • Parameterize the rays by the unit sphere to obtain a function on the sphere • Use spherical harmonic transform on this function to extract the numerical shape descriptor.

7. Example I: Saupe, Vranic 2001 • Surface feature • Distance to the origin • Mapping • Mapping domain: the unit sphere • Mapping procedure: project onto the sphere, resolve collisions by selecting the larger function value • Signal processing • Spherical harmonic transform

8. Example II: Depth Buffer • Heczko, Keim, Saupe, Vranic 2002 • Place a normalized mesh into a unit cube • Generate six gray-scale images on each face of the cube by parallel projection • The grayness value is the distance from the cube face to the model • Apply 2D Fourier transform to each of the six gray-scale images

9. Example II: Depth Buffer • For each cube face: • Surface feature • distance from mesh point to the face • Mapping • Mapping domain: cube face • Mapping procedure: project onto the face, resolve collisions by selecting the smaller function value • Signal processing • 2D Fourier transform

10. Generality • More examples easily generated • Compare to classification in Bustos et al. survey • Mapping ≈ object abstraction • Signal processing ≈ numerical transformation

11. Observations • Mapping: • Mapping domain: • ≠ original surface • a primitive geometry: sphere, plane etc • Mapping procedure: • Projection • Signal processing • well established: Fourier, Zernike, Spherical Harmonics • limits possible mapping domains

12. Contributions • Mapping: • Mapping domain: • any fixed surface – template • Mapping procedure: • interpolation: mean-value coordinates, Shepard • Signal processing • via manifold harmonics – eigenfunctions of Laplace-Beltrami operator

13. Why templates? • Mademlis, Daras, Tzovaras, Strintzis 2008: • Since ellipses approximate elongated shapes better than spheres: • Mapping domain: ellipsoid • Signal processing: ellipsoidal harmonics • Showed experimentally better retrieval results than sphere + spherical harmonics • We take this idea further: • Mapping domain: any fixed surface

14. Why template ≠surface itself? • Expand the feature function in terms of the manifold harmonics of the original surface? • Problem: notoriously difficult to match the harmonics coming from different surfaces • Sign flipping • Eigenfunction switching • Linear combinations • Fixed template: extracted expansion coefficients are in direct correspondence

15. Why interpolation? • Projection • Mapped feature function can be discontinuous at overlaps • Gibbs effect may render low-frequency expansion coefficients used as the shape descriptor inadequate for representing the function Feature function is distance to the origin Jump discontinuity

16. Why interpolation? • Projection • Redundancy • the value sets of the mapped feature functions on various templates will be almost the same • limits the gains of concatenating descriptors obtained from different templates

17. Why interpolation? • Interpolation • No Gibbs effect • mapped feature function is smooth • Less redundancy • the value sets of the mapped feature functions on various templates depend on relative positions • mean-value coordinates can inject more shape information into the mapped feature function • a mesh can be reconstructed given the mean-value coordinates

18. Construction of the descriptor • Selection of surface feature • Mapping • Signal Processing • Now discuss details

19. Selection of surface feature • All models are normalized using shift, continuous PCA, isotropic scaling • Many possibilities, but not: • the characteristic function • nor linear function of coordinates • To focus discussion • f = distance from a mesh point to the origin • Similar to Saupe, Vranic 2001

20. Mapping • Model surface S, Template surface T • Given • Construct • Shepard interpolation • Mean-value interpolation

21. Mapping: Shepard • Model surface S, Template surface T • Given • Construct • 0th order precision: constant functions reproduced

22. Mapping: Barycentric • Model surface S, Template surface T • Given • Construct • are barycentric coordinates of point p with respect to vertex • 1st order precision: linear functions reproduced

23. Mapping: Barycentric • A few different kinds of barycentric coordinates • Mean-value, positive mean-value • Harmonic • Maximum Entropy • Green coordinates, Complex in 2D • We use mean-value coordinates • Closed formula • Fastest to evaluate

24. Signal processing • We have a function • Need a compact representation • Expand the function into series • Use low-frequency coefficients • Need a function basis on template surface T • Manifold harmonics = Laplace-Beltrami eigenfunctions

25. Signal Processing • Manifold harmonics generalize Fourier basis to Riemannian manifolds • Spherical harmonics = manifold harmonics on the sphere • Have similar properties • Orthogonal • Concept of frequency • low-frequency coefficients are noise-robust • convey essential information about function

26. to store point areas Signal Processing • Egenvalues, eigenfunctions solve • Evaluation procedures well known • Solve symmetric eigenvalue problem for a matrix • We use cotangent Laplacian with voronoi point-areas • Pre-compute for the given templates and store • Our templates have about 500 vertices, the process takes less than 3 seconds • The storage for each template = 10,500 floats= =20*500+500 = #eigs * #vertices + #vertices to store eigenvectors

27. Resulting shape descriptor • Feature function • Mapped feature function • The template’s feature function • Quotient function • Expand into series

28. Resulting shape descriptor • Feature vector, N=20 • For template surface T, • Normalization: scale to get a = 1 • Use L2 distance

29. Experiments: Benchmark • Models: watertight benchmark • 400 closed surface models • 20 equal object classes

30. Experiments: Implementation • Implemented in MATLAB • Use C++ for mean-value coordinate computation • Timing • About 1 minute per model when mean-value coordinates used • Could make faster if simplified the models

31. Experiments: Templates • Templates: randomly chosen models • Simplified using Qslim • Makes mean-value computation faster

32. Compare Mapping Methods • Template = sphere • Projection vs. Shepard (a=1,2,3) vs. Mean-value • At long distance behavior of mean-value interpolant is similar to that of Shepard with a=2

33. Compare templates • Mapping via mean-value interpolation • M

34. Compare templates • Beneficial to combine – relative independence • All templates are normalized as objects in the benchmark – span similar spatial regions • The descriptors could have been made even more independent if the templates were differently posed.

35. Future work • Investigate dependency between the nature of the template and the produced retrieval results • No “ideal" template for all kinds of shapes • Flexibility of our approach – choose optimal templates based on the shape database at hand • How to choose? • Can we design a rotationally invariant descriptor?