Texture Synthesis on [Arbitrary Manifold] Surfaces

1. Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by:Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation

2. Topics Investigated Thus Far • 2D Texture Mapping (CS318) • 3D Textures (Hypertexture) • 2D Texture Synthesis

3. Today’s Goal Synthesize a texture on a surface by coloring mesh vertices + Input Texture Input Mesh Result

4. Advantages Speed No additional transformation required Simplicity Uniformly distributed across geometry Disadvantages Texture resolution dependent on mesh complexity Vertex Painting

5. 2 SIGGRAPH Papers: • Texture Synthesis on Surfaces, by Greg Turk • Texture Synthesis over Arbitrary Manifold Surfaces, by Li-Yi Wei and Marc Levoy

7. Desirable Properties • Share advantages of 2D algorithm • Quality • Efficient • General • Easy to use • Minimum distortion • Minimum discontinuity

8. image mesh grid ? Pixels/Vertices v normal Local Orientation (Vector field) u ? ? tangent bitangent Synthesis Order scanline ? Neighborhood Differences betweenImages and Meshes

9. image mesh grid ? Pixels/Vertices v normal Local Orientation (Vector field) u ? ? tangent bitangent Synthesis Order scanline ? Neighborhood Wei/Levoy Solution Wei and Levoy mesh re-tiling [Turk’92] user-specified relaxation random random flattening/resampling

10. image mesh grid ? Pixels/Vertices v normal Local Orientation (Vector field) u ? ? tangent bitangent Synthesis Order scanline ? Neighborhood Turk Solution Turk repulsion [Turk’91] User specified/ interpolated sweeping Surface marching

11. Both Papers ExtendFast Texture Synthesis using Tree-structured Vector Quantization Specifically by generalizing their definition of “search neighborhoods” to apply to meshes

12. Copy Search noise Input pyramid Output pyramid Texture Synthesis byNeighborhood Search noise

13. Search Surface Texture Synthesis byNeighborhood Search (Wei/Levoy) Input pyramid Output pyramid

14. Process 1. Build image/mesh pyramids 2. Assign texture orientation/Computation order 3. Generate texture

15. What aspects of image pyramids must we maintain in mesh pyramids? • Uniform density • Power-of-two complexity differences between levels

16. Image & Mesh Pyramids Mesh Retiling [Turk’92]

17. Turk ’92, used by Wei/Levoy Uniformly distributes mesh vertices Requires “shooting normals” to move between levels Turk ’91, used by Turk Uniformly distributes mesh vertices Maintains parent/child relationship between levels 2 Turk-ish Methods for Mesh Retiling

18. Retiling Density 24576 vertices 73728 vertices

19. Turk ‘91 • Create a mesh hierarchy in which mesh Mk = (Vk, Tk) is defined by its Vertices and Triangles • For the lowest mesh in the hierarchy, place n points on the surface • Use repulsion to distribute points easily • Add 3n points to make the next level • Repeat • Connect points by projecting nearby points onto a tangent plane • Perform Delaunay triangulation to reduce triangles

20. Now, we need to determine orientation… In Turk’s method, we use this orientation to determine the computation order (Surface Sweeping) In Wei/Levoy’s method, orientation is needed to “flatten” the neighborhoods.

21. Texture Orientation • Generate a coordinate frame • Three orthogonal axes • s (texture right) • t (texture up) • n (surface normal)

22. Texture Orientation • Methods for orienting textures • user-specified (Turk) • random (Wei/Levoy) • smooth or symmetric (Wei/Levoy)

23. Texture Orientation (User-specified)

24. 4-way symmetric texture 4-way symmetric vector field Texture Orientation (Symmetry)

25. Texture Orientation random 2-way symmetry 4-way symmetry

26. Texture Orientation (Relaxation) • Minimize an error function

27. Results (Wei/Levoy):Random Orientation

28. Results (Wei/Levoy):Other Orientations Random User-specified Relaxation 2-way symmetry 4-way symmetry

29. Texture Synthesis on Surfaces:Wei/Levoy Style

30. Synthesis

31. Synthesis : 2 Lowest Levels

32. Random copy Synthesis : Lowest Level

33. Copy Shooting normal Search Synthesis Pass1 : Extrapolation

34. Resampled Grid Compare Resample Flatten (Maillot’93) 2D Patch Mesh Neighborhood 3D Patch

35. Neighborhood Flattening • Project the triangles adjacent to p onto p’s local texture coordinate system • Add triangles one-at-a-time until neighborhood template is covered

36. Neighborhood Comparison compare ?

37. Copy Search Synthesis Pass 2 :Full Neighborhood

38. Multiresolution Synthesis

39. Texture Synthesis on Surfaces:According to Turk

40. Turk’s Surface Sweeping • Select an anchor vertex A • Assign s(v) = sweep distance to vertex v along the vector field from A (for all v) • Consensus orientation b/w 2 vertices v and w: Ovw = (O(v) + O(w))/2 • Calculate a new s(v) as a weighted average that its neighboring values dictate it should have. • Visit vertices in order by sweep distance

41. Turk’s Texture Synthesis (Pseudocode) I = Input texture N(v)= Neighborhood around v M(a,b) = Neighborhood around (a,b) D(M,N) = Match Value (sum of squared differences)

42. v normal u P O Turk’s Mesh Neighborhoods • r = average distance between mesh vertices • O(v) = Surface tangent vector • P(v) = O(v) rotated 90o about surface normal • Together, O and P make a coordinate frame ! • Now, we can traverse the surface by point repelling

43. Point Repelling • Use color interpolation to determine color at current surface point • Move r in the direction of O or P • When an edge is reached, fold the path over the next polygon

44. Results smooth 4-way symmetry random

45. Results Surface displacement smooth 2-way symmetry 2-way symmetry

46. Results

47. Summary of Differences Turk’s approach Our approach Vector field smooth random, symmetric Traversal order sweeping random Neighborhood surface marching flattening/resampling Mesh hierarchy explicit parent/child shooting normal