Mesostructure Rendering Techniques

1. Mesostructure Rendering Techniques Presented by Keith Yerex

2. Mesostructure • Mesostructure is used to refer to details that are too large to be modeled by reflectance functions, and too small to be modeled efficiently by polygons • Commonly rendered with bump-mapping, horizon mapping, and/or displacement mapping.

3. Bump-mapping [Blinn 78] • Store the normals of the true surface in a texture map, and use these normals in lighting equations. (dot3) • Normals can be stored in object space, or tangent space. In either case, the light must be transformed into that coordinate system.

4. Tangent space • Repeating bump/horizon/displacement maps must be represented in tangent space • Tangent space is a coordinate system made of the normal N, the tangent T, and the binormal B • Where T is in the direction of increasing texture coordinate u and B=NxT

5. Horizon mapping [Max 88] • The horizon angle is precomputed for a discrete number of directions • An incoming light angle must be above the horizon in order to illuminate the surface

6. Displacement Mapping • A displacement map specifies displacement in the direction of the surface normal, for each point on a surface from Klautz & Seidel

7. Displacement Mapping • Generally expressed as a function of u,v texture coordinates (or parametric surface parameters) • Stored as a 2d texture, and/or computed procedurally

8. The Problem • How can we render a model given as a set of polygons, and a displacement map?

9. Current solutions • Geometric • Subdivide and displace • Volume slice rendering • Ray tracing • Image Space • Parallax mapping • Relief textures • View dependent texturing / BDTF • View dependent displacement mapping

10. Subdivide and Displace [Cook 84] • Subdivide each polygon into many tiny polygons, and displace the new vertices along their normals using the displacement map • Produces many new vertices and triangles, which all need to be transformed and rendered. • Improvements include adaptive subdivision, and hardware implementation.

11. Volume Slicing [Kautz & Seidel] • Generate polygons that slice through the displacement volume • Render each slice, but draw only pixels where the displacement map value is greater than height of the slice at that position • Slices alignment varies (bump-out shells, orthogonal, screen aligned)

12. Volume Slicing • Alpha test can be used for efficient hardware implementation • Still generates a lot of geometry and high overdraw/fill-rate from Klautz & Seidel

13. Reference Plane Max Displacement Ray tracing [Smits et al] • In ray tracing, rays are simply intersected with the displacement map, rather than the polygon

14. Image Space Methods For a given viewing direction and texture coordinates P find the displacement in texture space l from Wang et al

15. Parallax mapping [Kaneko et al] • Makes the approximation that l=V*H(P)*scale+bias • Where H(P) is the height at the point of incidence. • Very efficient, and pretty convincing, but not accurate • Movie

16. Parallax mapping

17. Relief Textures [Oliveira, Bishop, McAllistor] • Instead of inverse mapping, use a forward mapping (from texture to image) • Warping is implemented efficiently by a 1d pre-warp, followed by standard texture mapping.

18. View Dependent Texturing [Debevec] • Store textures from all possible views, and then just pick the best one from a list when rendering a new view (or blend the best few) • BDTF also stores different lighting conditions for all views • Very large memory requirements [Dana et al]

19. View Dependent Displacement Mapping [Wang et al] • Store the depth maps along all possible viewing angles. • Computing the texture coordinate offset then becomes as simple as

20. VDM - curvature • For curved surfaces, the situation is slightly more complicated • So depths are pre-computed for variations in curvature as well as viewing direction

21. VDM – function • The vdm function dVDM(x,y,,,c), represents the depth to the surface along any ray to a surface with any curvature. • Note that if texture coordinates are unique, the curvature dimension doesn’t need to be stored.

22. VDM - shadows • Self shadowing is easily computed given the vdm function • This is used for curved surfaces also, even though it isn’t correct.

23. VDM - silhouette • A maximum view polar angle function MVM(x,y,,c) is used to store silhouette information • Any rays where  < MVM(x,y,,c) are not rendered

24. VDM - compression • The 5D vdm function is sampled in this paper at 128x128 x 32x8 x 16 • Plus another 4MB for the mvm function. Taking 68MB of memory!

25. VDM - compression • First, flatten dVDM and MVM into one 2d matrix A=[AVDM,AMVM] • Rows of A are indexed by x,y, and columns are indexed by ,c (or just c for ,AMVM) • SVD is applied to A: A=UET=WET Where E= [EVDM,EMVM]

26. VDM – compression • The two functions can now be reconstructed as: • Only a the first few eigenfunctions Ei need to be used since the eigenvalues (and Wi) decrease rapidly with i

27. VDM – compression • Using 8 eigenfunctions EVDM and 4 eigenfunctions EMVM reduces the data size to 4MB

28. VDM – results • Movie

29. References James F. Blinn, Simulation of wrinkled surfaces, Proceedings of the 5th annual conference on Computer graphics and interactive techniques, p.286-292, August 23-25, 1978 Robert L. Cook, Shade trees, Proceedings of the 11th annual conference on Computer graphics and interactive techniques, p.223-231, January 1984 Kristin J. Dana , Bram van Ginneken , Shree K. Nayar , Jan J. Koenderink, Reflectance and texture of real-world surfaces, ACM Transactions on Graphics (TOG), v.18 n.1, p.1-34, Jan. 1999 Jan Kautz , Hans-Peter Seidel, Hardware accelerated displacement mapping for image based rendering, No description on Graphics interface 2001, p.61-70, June 07-09, 2001, Ottawa, Ontario, Canada Max, N. 1988. Horizon mapping: shadows for bump-mapped surfaces. The Visual Computer 4, 2, 109--117. Manuel M. Oliveira , Gary Bishop , David McAllister, Relief texture mapping, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.359-368, July 2000 Brian E. Smits , Peter Shirley , Michael M. Stark, Direct Ray Tracing of Displacement Mapped Triangles, Proceedings of the Eurographics Workshop on Rendering Techniques 2000, p.307-318, June 26-28, 2000 Lifeng Wang, Xi Wang, Xin Tong, Stephen Lin, Shimin Hu, Baining Guoand Heung-Yeung Shum, View-dependent displacement mapping, ACM Trans. Graph.22 (3), 2002

30. Questions?