Mesh Quilting For Geometric Texture Synthesis

1. Mesh Quilting For Geometric Texture Synthesis Kun Zhou et al. In SIGGRAPH 2006 발표 이성호 2009년 4월 15일

2. Abstract • Mesh quilting • Geometric texture synthesis algorithm • 3D texture sample given in the form of a triangle • inside a thin shell around an arbitrary surface • Allow interactive and versatile editing and animation • Based on stitching together 3D geometry elements • On curved surfaces • Reduce distortion of geometry elements • inside the 3D space of the thin shell • Low-distortion parameterization

3. Introduction • Today’s commodity video cards • Exquisite details can be purely geometrically modeled • Modeling such complex geometric details • A tedious process • Creating mesh-based 3D geometric textures • Remains challenging

4. Mesh quilting • To synthesize geometric details by stitching together small patches • of an input geometric texture sample • Tools to further edit and animate • these geometric details

6. Related work • Modeling of Geometric Detail on Surfaces • Fur • [Kajiya and Kay 1989] • Rendering fur with three dimensional textures. • In Proceedings of SIGGRAPH 89 • Volume textures • [Neyret 1998]

7. More versatile representations • Geometric textures • [Elber2005] • Shell map • [Porumbescu et al. 2005]

8. Limitations • Periodic textures • Mesh-based creation • Of geometric textures on arbitrary meshes • [Fleischer et al. 1995] • Mostly restricted to the dissemination • Of simple texture elements over the surface

9. Example-based Texture Synthesis • Synthesis based on per-pixel non-parametric sampling • [Turk 2001;Wei and Levoy 2001; Ying et al. 2001; Tong et al. 2002; Zelinka and Garland 2003] • Based on the L2-norm • a relatively poor measure • of perceptual similarity, • such algorithms are not applicable • to a large spectrum of textures.

10. Textures by directly copying small parts • of an input texture sample • alpha-blending [Praun et al. 2000] • quilting • [Efros and Freeman 2001; Liang et al. 2001; Soler et al. 2002; Magda and riegman 2003; Kwatra et al. 2003; Wu and Yu 2004; Zhou et al. 2005] • Searching for the “min-cut” seams • further enhance the smoothness across the seams • [Efros and Freeman 2001; Kwatra et al. 2003; Zhou et al. 2005]

11. Feature matching • [Wu and Yu 2004] • Human visual system is so sensitive • to edges, corners and other high-level features in textures • Parallel controllable texture synthesis on GPU • [Lefebvre and Hoppe 2005] • Texture synthesis using Expectation Maximization optimization • [Kwatraet al. 2005]

12. Graphcut textures

15. Shell map

18. Challenges • Little work to provide tool • for 3D geometric texture synthesis. • [Bhat et al. 2004; Lagae et al. 2005] [Lagae et al. 2005]

19. [Bhat et al. 2004] • voxel-based approach • [Lagae et al. 2005] • used distance fields [Bhat et al. 2004]

20. Challenges • Irregular mesh • The input texture sample • is not a regular array of pixel values • Geometry elements • Each being truly a small 3D object identified • As a connected component in 3D • Quilting is performed on curved surfaces • Severe distortion in the 3D space

21. Contributions • Mesh-based geometric texture synthesis • Synthesized over the base mesh. • Triangle meshes • Both the input geometry and output geometry • Integrity • Maintains the integrity of geometry elements • In the synthesized texture • Texture editing and texture animation • can be easily performed

22. Aligns elements • through local deformation • And merges elements to connect texture patches • Mesh quilting on curved surfaces • Low-distortion parameterization • of the shell space

23. Mesh Quilting Synthesis • Setup & Nomenclature

24. Algorithm Overview • Seed Finding • Find a seed region R from which to grow the output mesh texture further out • Geometry Matching • Find the best patch placement around region R using geometry matching to minimize mismatch between the new and the old patch • Element Correspondences • Find correspondences between elements in the new patch and those in the old patch • Element Deformation • Align the corresponding elements through local deformation • Element Merging • Expand the output texture by merging the new patch into the output texture space

26. Seed Finding • Grid-based approach • The bounding boxes of both Moutand Min are subdivided in finer regular grids • These grids are only two-dimensional • Initially, the cells of Moutare tagged unprocessed • Each time we wish to grow out the current mesh Mout, • we look for an unprocessed cell with the largest number of adjacent cells that are already processed • this will be the seed cell that we will try to process next.

27. Geometry Matching • Find how to complete the mesh texture in the seed cell • and possibly add to its surroundings too. Using the nearby existing mesh texture available near the seed cell • Find a portion of the original swatch Min best matching this surrounding to extend Mout .

30. Restrict the translation t to be in • grid unit • Element deformation described in Section 2.6 • will compensate for an imperfect element alignment • Octreedata structure for the input texture • Significant speed-up

31. Element Correspondences • the overlapping region is usually larger than the small sub-patch Pout • since the input mesh texture covers Poutcompletely.

33. Element Deformation

36. Element Merging • Every element (either from Cout or Cin) without correspondence • directly added to Mout . • For every established correspondence (Cout ,Cin) • If Cout is entirely within the overlapping region, Cout is ignored • and Cinis instead added to the final results • if Cin is entirely within the overlapping region, • Cinis ignored and Cout is added to Mout .

37. In all other cases • stitch parts of Cin and Cout • to get a singly-connected, combined element • seek a cut path in each element • the graph cut algorithm • [Boykov et al. 2001]

41. Mesh Quilting Over Curved Surfaces • Setup • Let Mbase be the base mesh that we wish to enhance with added geometric details. • Minthe geometric texture mesh • used as a swatch • seamlessly tile the base mesh • S • the scale of the geometric details

42. From Planar to Curved • 2D grid -> base mesh • Quilting process will stop • Only when there are no more unprocessed triangles • Define a local surface patch • By starting from the chosen triangle • Growing the region • Using breadth-first traversal • Until we reach a certain depth • Or when the total area of the patch exceeds a user-defined threshold

43. Position of vertices located with respect to the base mesh • Location of a vertex v • over a triangle Tbase • is defined by the barycentric coordinates • of its orthogonal projection • on Tbase • along with the orthogonal distance (i.e., height) • from the triangle to v

44. discrete conformal mapping • surface patch is flattened over the 2D plane • using a discrete conformal mapping • DCM [Desbrun et al. 2002]

45. Local operations • Described for planar mesh quilting • Can be performed • Over this parameterization plane • Position of the newly synthesized vertices • Will be reprojected • Onto the local mesh-based coordinate system

46. in very curved regions • If the area distortion induced • by the local parameterization is too large • Reduce the area of the surface patch • This will decrease • the size of the output-sub-patch Pout

47. Final Mesh Embedding • Convert the vertex positions • Stored in local coordinates for now • Into a stand-alone, common embedding • Self-intersections can be created • Build a texture atlas for Mbase • Convert the above local representation of vertex positions to locations • in a geometry texture space • Then, construct a shell space around Mbase • Mapping the vertices • from the geometry texture space to the shell space • will fix the location of the vertices in 3D space

50. Shell Mapping • Porumbescu et al. [2005] • Creates large distortion in curved regions • We alleviate this! • By optimizing a stretch metric on this tetrahedral mesh • A natural extension of • low-distortion parameterization • of triangle meshes • [Sander et al. 2001]