1 / 42

Non-Rigid Registration of 3D Surface by Deformable 2D Triangular Meshes

Non-Rigid Registration of 3D Surface by Deformable 2D Triangular Meshes. Speaker:James Chang. Introduction. Non-rigid surface registration, particularly registration of human face, finds a wide variety of application in computer vision and graphics.

sirvat
Download Presentation

Non-Rigid Registration of 3D Surface by Deformable 2D Triangular Meshes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Non-Rigid Registration of 3D Surface by Deformable 2D Triangular Meshes Speaker:JamesChang

  2. Introduction • Non-rigid surface registration, particularly registration of human face, finds a wide variety of application in computer vision and graphics. • This study present a new automatic surface registration method utilized both attraction forces originating from geometrical and textural similarities, and stresses due to non-linear elasticity of the surface.

  3. Project flow chart 3D Reference Surface 2D Reference Plane Registration via Triangular Mesh Deformation 3D Target Surface 2D Target Plane Mapping Surfaces onto Image Planes Image Matching with Triangular Meshes Deformed Surface Nonlinear Elastic Deformation Multiresolution Registration Algorithm result

  4. Mapping Surfaces onto Image Planes

  5. Mapping Surfaces onto Image Planes • Blanz and Vetter: A morphable model for the synthesis of 3-D faces.Proc. SIGGRAPH’99, Los Angeles, USA, 187–194, 1999. • Cylindrical projection: Project their vertices onto planes. • Large distortions in the regions where the angle between the surface normal and the projection direction is large. • Worse still, a bijective mapping is not guaranteed.

  6. Bijection

  7. Mapping Surfaces onto Image Planes • Briefly the algorithms are categorized according to mapping distortion criterion that they minimize. • Length-preserving, angle-preserving, are-preserving parameterization approaches. • Least squares conformal mapping technique.(LSCM),angle-preserving. B. Levy, S. Petitjean, N. Ray and J. Maillot. Least squares conformal maps for automatic texture atlas generation. ACM Transactions on Graphics, 21(3):362–371, 2002. Proceedings of SIGGRAPH 2002.

  8. Mapping Surfaces onto Image Planes

  9. Registration via Triangular Mesh Deformation in 2D Domain

  10. Registration via Triangular Mesh Deformation in 2D Domain • Lets denote the reference surface as A which is to be mapped to a target surface B. • Their planar parameterizations yield 2D domains • If we can find a mapping from to ,this leads to the mapping between the surfaces in 3D space.

  11. Registration via Triangular Mesh Deformation in 2D Domain • Image Matching with Triangular Meshes. • Nonlinear Elastic Deformation. • Multiresolution Registration Algorithm.

  12. Image Matching with Triangular Meshes A mapping from to can be expressed via a vector field. B A Where denotes the 2D image coordinates and is a constant displacement

  13. Image Matching with Triangular Meshes • For each triangle of the reference mesh, we have a mapping function that maps a point in a triangle of the reference to the point in the target. • This function interpolates the mapped coordinates of the triangle vertices by

  14. Image Matching with Triangular Meshes Where barycentric coordinates are obtained by P1 P is the area of triangle P2 P3

  15. Image Matching with Triangular Meshes • An image matching energy , ,which accounts for the total square matching error over domain becomes Where is the set of triangles, , is the domain of triangle , and represent multi-modal image values, and is a weighting matrix.

  16. Image Matching with Triangular Meshes • To exploit different surface attributes during matching , weighted sums of image attributes are used by means of diagonal weighting. • The weights are determined according to both variance and importance of the channel.

  17. Image Matching with Triangular Meshes • The driving forces that deform a reference mesh to a target come from minimization of the image matching energy. • This energy can be minimized by using the gradients at each mesh node with respect to its mapped coordinate s

  18. Image Matching with Triangular Meshes • The gradient at vertex is obtained through the chain rule as is the set of triangles connected to the node , is the .vertex of triangle that corresponds to node , and is thus the .barycenteric coordinate for the point .

  19. Image Matching with Triangular Meshes • The gradients are evaluated at each node of the mesh to update the mapping vector, , in a gradient decent scheme. • The image gradients are evaluated using 3 X 3Scharr mask. J. Weickert and H. Scharr. A scheme for coherenceenhancing diffusion filtering with optimized rotation invariance.Jornal of Vis. Comm. Im. Rep., 13:103–118, 2002. • We approximate the integrals in (4) and (5) by sampling at the recursively subdivided triangle centers.

  20. Nonlinear Elastic Deformation

  21. Nonlinear Elastic Deformation • St. VenantKirchoff material model. M. Bro-Nielsen. Medical image registration and surgery simulation.IMM-DTU Ph.D. Dissertation, 1996 . • The potential energy for St. VenantKirchoof material is given by Where and are the Lame material constants.

  22. Nonlinear Elastic Deformation • Green-Lagrange strain tensor

  23. Nonlinear Elastic Deformation • We use minimization of the Green-Lagrange strain tensor to attain elasticity. The deformation energy is defined as the Froebenius norm of this strain,

  24. Nonlinear Elastic Deformation • Analytic expressions for gradients of with respect to vertex coordinated mappings must be obtained for the energy minimization task. • With triangular mesh elements nonlinearities are not too complex and can be solved directly with a gradient descent scheme. • During this minimization, stresses at each node of the mesh deform its neighboring triangles to regularize the displacements due to forces generated by image mating errors.

  25. Multiresolution Registration Algorithm.

  26. Multiresolution Registration Algorithm. • The total energy minimized over the reference triangulation T is • We minimize the total energy via gradient descent methods in a coarse-to-fine approach. In this equation controls the amount of elasticity and thus the regularization during the registration

  27. Multiresolution Registration Algorithm. • To solve the problem in different scales: • First, Gaussian image pyramids are constructed one at the reference, the other at the target. • Second, at each resolution, meshes are generated adapted to the scale of the pyramid level.

  28. Multiresolution Registration Algorithm. • Registration starts from the coarsest level, and the estimated deformations are subsequently transferred to a finer level. • The deformation transfer from one mesh to another is realized by barycentric mapping. • For each node of a finer level, its barycentric coordinates at the coarser mesh are calculated, and thus mapping of that node is obtained by a weighted sum of the previous mesh node value.

  29. Multiresolution Registration Algorithm. • To produce meshes: • First, constrained Delaunay triangulation is performed . The boundary vertices, are used as constraints in this triangulation. Feature points can also be used as additional constraints. J. D. Boissonnat, O. Devillers, S. Pion, M. Teillaud, and M. Yvinec. Triangulations in CGAL. Comput. Geom. Theory Appl., 22:5–19, 2002.

  30. Multiresolution Registration Algorithm. • Second, a mesh generation algorithm which limits the maximum allowed triangle edge length is applied to the triangulation. J. R. Shewchuk. Mesh generation for domains with small angles. Proc. 16th Annu., AACM Sympos. Comput. Geom.,1-10, 2000.

  31. Multiresolution Registration Algorithm. • Energy minimization restarts in one higher resolution level initialized with the values of the previous step. • This procedure continues until the finest and final deformation estimates are found.

  32. Multiresolution Registration Algorithm. Finest level Mesh deformation Barycentric mapping Coarsest level Reference target

  33. Multiresolution Registration Algorithm.

  34. Result

  35. Identities

  36. Expressions

  37. Result • This algorithm is quite efficient. • The number of iterations till convergence were in between 50 and 200, and total registration duration changed from 3 to 7 seconds using 3.0GHz CoreDuo processor.

  38. Conclusion • A new automatic algorithm to solve the non-rigid registration problem of two surfaces is presented. • Registration is realized by deforming one surface onto the other one after mapping the onto a plane.

  39. Conclusion • This is achieved by first using a bijectivemapping, and then utilizing finite element method to obtain the matching deformation. • The differences of geometric and texture related surface attributes generates deformation forces, while elastic stresses regularize the resulting deformations.

  40. Conclusion • Nunlinear elasticity is used to cope with the large deformations by minimizing Green-Lagrange strain tensor. • Also, following a multiresolution approach, good results can be obtained in a short time.

  41. Future work • Currently the algorithm is not designed to handle expression with open mouth. • This team also plan to experiment using different surface attributes and investigate better strategies to combine these modalities.

More Related