Texture Sprites: Texture Elements Splatted on Surfaces

1. Texture Sprites: Texture Elements Splatted on Surfaces Sylvain LefebvreSamuel HornusFabrice Neyret GRAVIR/IMAG-INRIAACM SIGGRAPH 2005 Symposium on Interactive 3D Graphics and Games 2005

2. Abstract

3. Introduction • Texturing 3D models with very high resolutions create two major difficulties • Storage cost of high resolution texture maps can easily exceed the available texture memory • Creating high resolution textures proves to be a difficult and tedious task • Lack of representation for textures composed of sparse elements • Wastes a lot of memory • Discontinuities • Flickering due to Z-fighting

4. Introduction • Texture sprites • The texture element live on the object surface and can be updated dynamically • Provides a general and convenient scheme for sprite-based texturing • Homogeneous appearances obtained by overlapping many sprites • Interactive addition of local details • Geometry and composite texture are independent • Not require: 1) mesh to conform to any constraint 2) global planar parameterization 3) modify the initial mesh

5. Contributions • A parameterization-free texture representation allowing efficient storage and filtered rendering of composite textures • An efficient solution to dynamic decal management and animated texture elements • New appealing texturing effects • Complete GPU-implementation

6. Sprite-Based Textures • Key idea • Store the sprite attributes (position, orientation, etc) in a 3D hierarchical structure surrounding the mesh • Follow the idea of octree textures • The sprites attributes in the leaves of the hierarchical grid • The entire data-structure is compact • Many sprites share a same texture pattern • Information is stored only around the surface

7. Sprite-Based Textures • Easy to author • add, drag, drop, change size, and orientation • The system is very flexible • Sprite ordering is controllable • Sprite overlapping can be mixed • Each sprite can be independently animated • For instance: react to surface deformations

8. Difficulties • How to manage the hierarchical 3D grid on the GPU (fragment program) • How to use the hierarchical 3D grid to store and retrieve sprites attributes • How to filter the composite texture and blend overlapping sprites together

9. 3D Hierarchical Grid (in GPU) • Problem • Store the hierarchical grid in texture memory • Retrieve data from the grid at a given 3D location • (This is implemented in a fragment program) • We call this structure N3-tree • N in each dimension • Octree → N = 2 • Implementation: N = 4

10. N3-tree Memory Layout • Tree structure • Node: indirection grid consisting of N3 cells • Cell content: (cell type is determined by a flag stored in the A-channel) • A = 1, data (leaf) • A = ½ , a pointer • A = 0, empty • Store our structure in a 3D texture called the indirection pool • Data values and pointer values are stored as RGB-triples

11. Notations • N*N*N: resolution of the indirection grid corresponding to each tree node • Nl*Nl*Nl: resolution of virtual 3D grid at level l • At level l a point M in space ( M [0, 1)3 ) lies in a cell located atand has a local coordinate within this cell. We have

12. Notations • Node are packed in the indirection pool • S = (Su, Sv, Sw): size of indirection pool • NSu*NSv*NSw: resolution of the texture hosting the indirection pool • A texture value in the indirection pool is indexed by a real valued P [0, 1)3 • P = Pi+Pf

13. Notations • Pi identifies the node which is stored in the area • Pi is called the node coordinates • Pfis the local coordinate of P within the node’s indirection grid • (!! Packing the cells does not change the local coordinates.) Equation(1)

14. Tree Lookup • For rendering, we need to retrieve the data corresponding to a given location M in space (more precisely, on the object surface) • This is done per-pixel in a fragment program • Start from level 0 in the tree • At level l let Pl,i be the coordinate of the current node in the indirection pool • The RGBA value corresponding to M is found in Pl,i at the location Pl,f , obtained by Equation(1) • A: in a leaf (return), an empty node (discard) or an internal node (continue to the next level (l+1) )

15. Implementation Details • To avoid precision issues, and minimize storage requirements we store S‧Pl,i instead of Pl,i in the indirection cells • S‧Pl,i is an integer • Optimized the encoding: choose the number of bits to stored in the RGB channels to exactly represent at most S indices • The values of the pointers do not depend on the size of the indirection pool

16. Precision Limitations • The limited precision of 32 bits floating point: • Limit on the depth, resolution, number of nodes of the hierarchical grid

17. Managing the Texture Sprites • Each sprite is associated with • various parameters • a bounding volumeV defining the spatial limit of the sprite influence on the composite texture • Bounding volumeV is used to • determine which cells are covered by sprite • detect whether two sprites overlap

18. Storing Sprite Parameters • Sprite parameter table is a texture containing all the parameters for all the sprites • Each sprite corresponds to an index into this texture; each texel encodes a parameter set • Store the index of the sprite parameters in the tree leaf data field • The RGB value encodes a pointer to the sprite parameter table

19. Dealing with Overlapping Sprites • Leaf vectors table • This 3D texture implements a table of vectors storing the sprite parameters associated with a leaf • 2 dimensions: the vector • 1 dimension: to index thevector of sprite parameters • Instead of directly storing the index of one sprite in the leaves, we now store an index to the leaf vector

20. Adding a Sprite to the Tree • Leaf splitting occurs only in full leaves (Omax) • Cmax < Omax • Insertion failures • The maximum depth level is reached only when more than Omax sprites are crowded in the same very small region • Ordering of sprites • Order the sprites in the leaf vectors

21. Positioning the Sprites on the Surface • Associate a frame to each sprite and use a parallel projection to surface • The positioning parameters • A center point c (sprite handle) • Two vectors l, h • defining the plane, scale, orientation of the pattern • A normal vector n (direction of projection) • M: (l, h, n) • U = M-1‧(P-c) • U = (u, v, w): texture coordinates within the pattern

22. Positioning the Sprites on the Surface • Deformation-proof texturing • Store a 3D parameterization (u, v, w) at each vertex and interpolated as usual for any fragment • Blending sprites • When multiple sprite overlap, the resulting color is computed by blending together their contributions • Non standard blending modes • Blobby-painting cellular textures

23. Filtering • Linear interpolation (for close viewpoint) • standard texture unites of the GPU • MIP-mapping of sprites • standard texture unites of the GPU

24. Applications and Result • Texture authoring • Interactively scaled, rotated, or moved above or below the already existing sprites • Lapped texture approximation • The sample is stored only once • N3-tree minimizes the memory space required for positioning information • Rendering does not suffer from filtering issues created by atlases

25. Applications and Result • Animated sprites • Modify the positioning parameters (position, orientation, scaling) • A sprite can cycle over a set of texture patterns, simulating an animation in a cartoon-like fashion • Snake scales • Estimate the local geometric distortion and scale the sprites accordingly to compensate for the deformation of the animated mesh

26. Performance

27. Storage Requirements

28. Conclusions • High texturing resolution at low memory cost • Without need for a global planar parameterization • The sprite’s attributes are efficiently stored in a hierarchical grid surrounding the object’s surface • Flexible • Each sprite can be independently animated, moved, scaled, rotated • Create new texturing effects, such as animated texture, textures reacting to mesh deformations • Complete GPU implementation, real-time performance

29. Future Work • Painterly rendering • Sprite projector functions • For point-based and volumetric representation