Illumination Dependent Refinement of Multiresolution Meshes

1. Illumination Dependent Refinement of Multiresolution Meshes By Reinhard Klein, Andreas Schilling, and Wolfgang Straβer Presented by Ben Gregorski and Greg Streletz ECS289L – Multiresolution Methods – Winter 2003

2. MRMs and Selective Refinement • A mesh simplification algorithm (such as vertex removal, edge collapse, or triangle collapse) leads to a multiresolution model (MRM) • Continuous level-of-detail (LOD) MRMs consist of a base mesh and a sequence of local simplification operations • Selective refinement involves changing the sequence of local simplification operations in order to refine only a particular area of the mesh • Arbitrary reordering of simplification operations is not possible; certain mesh elements must be present in the current mesh for a given operation to be possible • Leads to a hierarchy of modification operations (represented as a DAG) ECS289L – Multiresolution Methods – Winter 2003

3. Illumination Dependent Refinement • Selective refinement can proceed based on geometry alone; however, incorporating illumination information can lead to improved results • For lighting, the surface normals are essential • Normals from the original mesh, not the coarsified mesh • Previous work: the variation of the normals in a local area of the mesh is represented by a cone of normals ECS289L – Multiresolution Methods – Winter 2003

4. Goals of Current Paper • Observation: the cone of normals for a given area of the mesh represents only a bound on the actual normals for that area; it does not represent in any way the actual distribution of the normals • Moreover, the cone of normals in previous methods is larger than necessary • Reason: the simplification hierarchy is used to determine the region of the original mesh to be used for normal calculations, rather than the actual geometric correspondence with the region of interest in the coarse mesh • Both of these factors result in more refinement than is needed • Goal: detect cases in which refinement can be avoided, in order to obtain smaller approximating meshes • Phong interpolation – if normals over a triangle can be approximated by a linear interpolation of the vertex normals • Goraud interpolation – if, in addition, a linear brightness distribution can be assumed ECS289L – Multiresolution Methods – Winter 2003

5. Overview of Algorithms • Generating Information about the Normals • Filtering of normals – to avoid aliasing problems • Takes place during mesh simplification • Also happens during selective refinement • Determination of deviation angles • Creation of an error estimate for surface normals • Takes place only during mesh simplification (i.e., off-line) • Results are stored as part of the MRM • Extraction Algorithm • Used during the extraction of selectively refined meshes from the multiresolution model • Decides when to coarsen of refine the current triangulation • Goal is to minimize number of polygons sent to the rendering pipeline without causing a significant decrease in image quality • Decisions on refinement can be reduced to the problem of deciding whether or not a given triangle in the current mesh is acceptable • Evaluating a given triangle amounts to a series of tests; if the triangle fails any test, its surrounding area must be refined ECS289L – Multiresolution Methods – Winter 2003

6. Filtering of Normals • During simplification (as well as during selective refinement), subsampling of normals can result in aliasing problems • To prevent this, the relevant normals are recalculated after each simplification step or selective refinement extraction • Equation for filtering of normals: ni = normal at the vertex vi, for viincident to the modified area j = index over all triangles incident to vertex vi ECS289L – Multiresolution Methods – Winter 2003

7. Determination of Deviation Angles • Calculate normals based on data from the original (fine) mesh • Map vertices (and triangles) from the original mesh onto the given triangle in the coarse mesh • Associate normals at vertices of the original mesh with the corresponding projected vertices in the coarse mesh • Calculate intersections of edges of projected triangles with edges of the given triangle • Calculate normals for these intersection points by interpolation of normals from the original mesh • Calculate normals based on data from the current (coarse) mesh level • Compute barycentric coordinates of vertex projection and intersection points with respect to the given triangle of the coarse mesh • Using these coordinates, linearly interpolate the normal values at the vertices of the coarse triangle ECS289L – Multiresolution Methods – Winter 2003

8. Determination of Deviation Angles • Can now compare the normals obtained from the current mesh level to those obtained from the original mesh level • αi = deviation angle between normals from the two levels • Construct a linear deviation function fΔ such that for all vi’ (the projected and interpolated points) • This linear deviation function can be represented by its values at the vertices of the coarse level triangle • It is this information that is stored along with the MRM ECS289L – Multiresolution Methods – Winter 2003

9. Determination of Deviation Angles • The algorithm just described can be used with any mesh simplification strategy, including continuous LOD simplification methods • However, if a continuous LOD MRM is to be used for selective refinement, the calculation performed during simplification will reflect only one of several “constellations of adjacent triangles” that a given vertex can encounter • Therefore, the deviations αiare computed by maximizing over all such constellations using a recursive algorithm operating on the DAG of the simplification hierarchy ECS289L – Multiresolution Methods – Winter 2003

10. Extraction Algorithm • Given a multiresolution model (MRM) defining a continuous LOD representation of an initial mesh, selectively refined models can be extracted from it by coarsening or refining the current mesh • The DAG of the hierarchy of modification operations constrains what operations can be performed at a given time • Aside from this consideration, the main issue for an extraction algorithm is to decide whether or not a given region requires refinement • For a given triangle, this decision can be reduced to a series of tests, which can be expressed as a flow diagram ECS289L – Multiresolution Methods – Winter 2003

11. Handling Backfacing Surface Parts • If the triangle in question is on the backside of the surface with respect to the current viewing position, it cannot be seen and therefore does not need to be refined • To determine whether a triangle is a backfacing surface part, the normal cones (normals ni1, ni2, ni3 and deviation angles ai1, ai2, ai3) at the triangle’s vertices are used, together with the vectors ei1, ei2, ei3 from the vertices to the viewing position • The condition for a backfacing triangle is given by: ECS289L – Multiresolution Methods – Winter 2003

12. Check for Irrelevant Geometry • If the projected triangle is smaller than the size corresponding to a single pixel in the image space, elaborating its geometry is meaningless • Therefore, do not refine further. ECS289L – Multiresolution Methods – Winter 2003

13. Controlling the Geometric Error • Convert the Hausdorff distance between the original triangulation and the simplified triangulation into an image space error • Use this image space error to determine whether refinement is necessary strictly for geometric reasons • Can also consider avoiding refinement outside the viewing frustum, except for applications in which the viewing direction changes rapidly ECS289L – Multiresolution Methods – Winter 2003

14. Controlling the Shading Errors • Assume the Blinn-Phong shading model • For a single light source: ECS289L – Multiresolution Methods – Winter 2003

15. Only Ambient Lighting Needed? • If the corresponding surface (in the original mesh) of the triangle being considered faces away from the light source, only ambient lighting is needed • If this is the case, the triangle need not be refined further • To determine whether the surface faces away from the light source, the normal cones (normals ni1, ni2, ni3 and deviation angles ai1, ai2, ai3) at the triangle’s vertices are used, together with the vectors li1, li2, li3 from the vertices to the light source • The relevent criterion is: ECS289L – Multiresolution Methods – Winter 2003

16. Linear Phong Interpolation Sufficient? • First, determine whether linear approximation of the normals of the coarse triangle (Phong interpolation) is sufficient to capture the essence of the correct shading of the region • Criterion: deviation of linearly approximated normal from the correct normal (represented by an α bound computed from the triangle’s linear deviation function in the MRM) should be less than the minimum difference between the linearly approximated normal and the linearly approximated normals of points within a distance of one pixel in screen space ECS289L – Multiresolution Methods – Winter 2003

17. Linear Phong Interpolation Sufficient? • The criterion for sufficiency of the current triangle under Phong interpolation can be interpreted in terms of projections in the space of the normal vectors ECS289L – Multiresolution Methods – Winter 2003

18. Linear Phong Interpolation Sufficient? • Based on the projection interpretation, the following condition can be derived: • nx and ny are the partial derivatives of the normal n with respect to screen-space directions x and y • This formula is valid for ; For , need to replace nx with -nx ECS289L – Multiresolution Methods – Winter 2003

19. Linear Phong Interpolation Sufficient? • Even if linear approximation to the normals is not sufficient from a “deviation of normals” point of view, it may not matter • Specifically, if the gradient of intensities on the triangle (under Phong interpolation) is small enough in spite of high deviation in the normals, the triangle can be retained anyway, and refinement can be avoided • This case is detected by a separate test • Essentially, need to find an upper bound on brightness changes caused by changes of normal bounded by α • Depends on lighting parameters ECS289L – Multiresolution Methods – Winter 2003

20. Linear Phong Interpolation Sufficient? • Consider the derivatives of the diffuse term and the ambient term with respect to the angles between the normal and the light source vector, and between the normal and the halfway vector, respectively • Maximize each of these over its dependent variable, with the ranges for the these optimization parameter angles determined by the deviation α • Multiply the resulting derivative by the deviation α to obtain the bound on intensity changes • Changes caused by angle α are bounded by ECS289L – Multiresolution Methods – Winter 2003

21. Phong Interpolation Supported by Renderer? • If either the normals of the surface represented by the original mesh are sufficiently approximated by the linearly interpolated normals, or the intensity gradient is small enough for it not to matter, then Phong interpolation can accurately represent the desired illumination • Thus, if the graphics hardware supports Phong interpolation, there is no need for more refinement • Otherwise (i.e., if only Gouraud shading is supported), more work is necessary … ECS289L – Multiresolution Methods – Winter 2003

22. Can Gouraud Interpolation Suffice? • If Phong interpolation is not supported by the renderer, we need to determine whether Phong interpolation can be replaced with Gouraud shading • This amounts to ascertaining whether the brightness distribution is sufficiently linear for the Gouraud shading to approximate the Phong shading • Several methods are cited, but a fully robust method is listed as future work • If Gouraud interpolation is deemed sufficient, do not refine further; otherwise, more refinement is necessary ECS289L – Multiresolution Methods – Winter 2003

23. Conclusions • An algorithm was presented that provides a way to calculate error information regarding deviations of normal vectors from one LOD to another • This error information can be stored as part of the MRM • Using the normal deviation error information, selective refinement can be linked to illumination considerations as well as considerations of the mesh geometry • The particular selective refinement extraction algorithm presented consists of a series of tests, and is designed to ensure that the illumination dependent selective refinement uses as few triangles as necessary • This approach is does not depend on any specific mesh simplification strategy; it can be used with MRMs generated using any of the common simplification methods • For best results, it seems that this illumination dependent refinement procedure should be combined with a MRM constructed using a simplification method that considers scalar attributes associated with the mesh, such as color, normals, and texture coordinates, as well as mesh geometry (as proposed by Hoppe in his PM paper, for example) ECS289L – Multiresolution Methods – Winter 2003