K. H. Ko School of Mechatronics Gwangju Institute of Science and Technology

1. Photo-realistic Rendering and Global Illumination in Computer Graphics Spring 2012Ultimate Realism and Speed K. H. Ko School of Mechatronics Gwangju Institute of Science and Technology

2. Fast Global Illumination • Dynamic Scenes • One major challenge with caching shading values is dealing with dynamic scenes. • When objects or lights move in a scene, the cached shading values could become invalid and must be updated accordingly. • There are two reasons this update should be done incrementally. • First, all these algorithms, to varying extents, assume the cached values accumulate over time. Recomputing all cached values from scratch would be too slow for interactive use. • The second reason is that in typical interactive applications, the effect of an update can be quite localized. In such cases updating all cached points unnecessarily is inefficient. • Some of the interactive rendering techniques age samples and eliminate outdated samples to ensure that all samples are recomputed from time to time. • This ensures that images will eventually be correct once the moved objects are stationary for some number of frames. • Since these approaches do not explicitly find the effect of a scene change and invalidate samples, they do not directly handle fully dynamic scenes.

3. Fast Global Illumination • Dynamic Scenes • Most of the techniques try to address the important problem of efficiently updating cached shading values in dynamic scenes. • These approaches must satisfy two conflicting goals: correctness and efficiency. • On he one hand, they must find all shading values that are affected by a change in the scene. • On the other hand, they should avoid unnecessarily invalidating samples that remain accurate in spite of the change to the scene. • While these approaches are promising, this is still an open area of research.

4. Fast Global Illumination • Fast Ray Tracing • The availability of increasingly fast processors and programmable GPUs is fueling research in systems for interactive ray tracing. • These systems explore ray tracing as an alternative to hardware rendering because of ray tracing’s asymptotically superior performance in rendering complex scenes. • When a ray-tracer is supported by an acceleration structure, the cost of determining visibility for a single ray is typically logarithmic in scene complexity. • In contrast, hardware, z-buffer algorithms render the entire scene each frame, achieving linear performance. • Furthermore, ray tracers are extremely flexible and can support with full generality high-quality rendering effects. • Thus, ray tracers are being considered as plausible alternatives for rendering complex scenes.

5. Fast Global Illumination • Graphics Hardware and Precomputed Radiance Transfer • Modern graphics processing units (GPUs) are programmable parallel processors that provide flexibility by supporting vertex and pixel programmability. • Apart from exploiting parallelism, GPUs also derive their power from their support for texturing, which requires very high memory bandwidth. • There has been increased interest in using the graphics hardware to integrate richer shading models and global illumination effects in rendered images. • One deterrent to the practical application of global illumination in interactive applications is the significant time required to compute a full global illumination solution. • While the advent of GPUs has decreased the cost of local pixel shading, the cost of determining visibility between two arbitrary points remains expensive. • This is because this computation requires nonlocal access of data that is not cheap in the GPU model.

6. Fast Global Illumination • Ambient Occlusion. • Ambient occlusion is a popular technique to “simulate” global effects in interactive applications. • This approach precomputes and approximates the visibility between a point and the incoming hemisphere so that shading can be computed on the fly using this precomputed visibility term. • While this technique is not accurate, it is fast and adds realism at a relatively low price. • Ambient occlusion assumes rigid geometry and diffuse materials, though recent work attempts to relax the assumption of rigid geometry.

7. Fast Global Illumination • Ambient Occlusion. • Ambient occlusion precomputes the visibility of the hemisphere at each point of the scene. • This precomputation can be per-vertex or per-pixel, where the per-pixel information is encoded as textures. • At each point, the algorithm samples the hemisphere to approximate visibility. • This is where the assumption of view-independent diffuse materials comes into play. • Using either uniform sampling or cosine-weighted sampling, the following integral is estimated as the ambient occlusion factor: • This precomputation is quite expensive, though.

8. Fast Global Illumination • Ambient Occlusion. • Once the ambient occlusion values are computed, they are then used to render each point as follows: • The simplest use is to modulate ambient lighting, just as its name suggests. • In this case the precomputed ambient occlusion term is multiplied with a constant ambient term to achieve greater realism in the ambient shading term. • Ambient occlusion is also used, less accurately, to modulate more complex lighting environments, such as prefiltered environment maps. • This is done by computing an additional value in the precomputation, the bent normal. • The bent normal is the average unoccluded normal; i.e., it is the average normal of all unoccluded samples over the hemisphere. • This bent normal approximates the main direction of illumination and is used when rendering the point instead of the surface normal. • There are clear cases where the bent normal is inaccurate, but his approximation produces reasonable values for applications, such as movies and games, where accuracy is of less importance.

9. Fast Global Illumination • Precomputed Radiance Transfer • PRT is a family of techniques that supports complex illumination effects by precomputing light transport and computing shading on the fly using the precomputed transport. • This ability to support interactive performance with expensive shading has hada big impact on both research and practical applications such as games. • PRT uses precomputation to support expensive illumination effects such as interreflections, caustics, and subsurface scattering. • However, to achieve interactive performance, some restrictions are imposed. • PRT algorithms typically assume that the scene is static, and that lighting of the scene is from light sources that are infinitely far away, i.e., environment maps.

10. Fast Global Illumination • Diffuse PRT • Illumination of diffuse and glossy objects lit by infinitely far away environment maps • Consider direct illumination. The direct illumination at a point x, denoted as L0, is • Since lighting is from a distant environment map, Lenv does not depend on position and only depends on direction. • We further restrict ourselves to diffuse surfaces. Thus the BRDF is a constant and can be moved out of the integral:

11. Fast Global Illumination • Diffuse PRT • The visibility V and cosine term are often combined together into a cosine-weighted visibility term. • The main insight of PRT is that this integral can be split into a precomputed transfer function that includes the effects of self-shadowing and interreflections, and a runtime computation that efficiently combines the transfer function with the dynamically changing environment map. • To achieve this goal, PRT projects the lighting onto a set of basis functions over the sphere. • The original formulation used spherical harmonics (SH). • Thus • Where yi are the spherical harmonic basis functions and li are the lighting coefficients.

12. Fast Global Illumination • Diffuse PRT • Since PRT aims at supporting shadows from relatively large area lights, the original PRT system used 25 basis functions. • Substituting the SH lighting coefficients, we get

13. Fast Global Illumination • Diffuse PRT • The same derivation can be generalized to handle multiple reflections. • We derive the generalization for the first bounce of illumination; the other bounces can straightforwardly be derived in a similar manner. • We first simplify some of the notation: we denote the diffuse reflectivity at point x as ρx and point y as ρy and the point y = r(x,Ψ) is the surface visible from x in direction Ψ. The first-bounce radiance L1 arriving at x through y is given as

14. Fast Global Illumination • Diffuse PRT • This approach can be generalized to more bounces. • The corresponding terms for all bounces are added together to compute the total transfer function for each spherical harmonic basis function for the point x: tx,i = Σbtbx,i. • A frame is rendered as follows: • The environment map is projected into the spherical harmonic basis functions taking into account the object’s rotation with respect to the environment map. • Rendering a vertex or pixel x then involves computing the dot product of the two vectors: the environment map coefficient vector l and the transfer function vector t for the point x. • This dot product is easily supported on the GPU achieving interactive performance.

15. Fast Global Illumination • All-Frequency PRT • High-frequency illumination effects, such as 노게 shadows, cannot be represented with great fidelity using a small number of coefficients in the spherical harmonics basis functions. • The use of wavelets for high-frequency illumination is proposed. • Wavelet coefficients are selected on-the-fly to achieve a nonlinear approximation of high quality. • This method demonstrates interactive performance for fixed viewing of nondiffuse scenes and arbitrary viewing of diffuse scenes. • Arbitrary, nondiffuse BRDFs can be approximated by factoring them into components that are dependent on the viewing direction only and lighting direction only, respectively. • Essentially, the 4D BRDF is split into two 2D functions that are separately approximated. • Each separable function is represented using Haar wavelets, which are further clustered using PCA. • In this approach, the BRDF are factored as two functions G and F, which are vector functions that depend entirely on the view direction and the light direction. • More approaches have been proposed.

16. ConclusionUnresolved Issues in Photorealistic Rendering • Acquisition and modeling of BRDFs • There has been quite some effort to measure the BRDF of real materials and to design usable models for use in computer graphics. • But this whole field still needs a lot of research to provide us with reliable, accurate, and cheap ways to evaluate BRDF models. • Acquisition of geometry and surface appearance • Computer vision has developed several techniques for acquiring the geometry of real objects from camera images. • But it is still a major problem when the surface of the object is nondiffuse or when the nature of the illumination on the object is unknown. • Self-adaptive light transport • Little effort has been made so far to try to make an overall global illumination algorithm that behaves in an adaptive way in various situations. • Such an algorithm would pick the right mode of simulating the light transport, depending on the nature of the surfaces, the frequency of the geometry, the influence on the final image, etc.

17. ConclusionUnresolved Issues in Photorealistic Rendering • Scalable and robust rendering • Scenes that include very high complexity in illumination, materials and geometry remain challenging. • Currently, a user has to manually pick approximations, rendering algorithms, and levels of detail to achieve reasonable quality and performance of such scenes. -> This manual approach is clearly not desirable. • Geometry-independent rendering • Current light transport algorithms assume that the geometry of the scene is known and explicitly compute a huge number of ray-object intersections in order to know where light gets reflected off surfaces. • Psychoperceptual rendering • Radiometric accuracy has been the main driving force for global illumination algorithms. • But since most images are to be viewed by human observers, it is usually not necessary to compute up to this level of accuracy. • New rendering paradigms should be focused around rendering perceptually correct images.

18. ConclusionUnresolved Issues in Photorealistic Rendering • Integration with real elements • It is likely that more integration between real and virtual environments will become an integral part of many applications. • Putting real objects in virtual scenes. • Putting virtual elements in real scenes. • A perfect blend between the real and virtual elements becomes a major concern. • Some requirements of future rendering algorithms • Interactivity • Any material, any geometry • Many different input models • Realism slider

19. ConclusionUnresolved Issues in Photorealistic Rendering • Interactivity • Any rendering algorithm of the future should be able to render scenes at interactive speeds, irrespective of scene or illumination complexity. • Any material, any geometry • All possible materials, from pure diffuse to pure specular, should be handled efficiently and accurately. • Any type of geometry should be handled as well. • Many different input models • It should be possible to take any form of input, whether it is a virtual model or a model based on acquisition from the real world. • Realism slider • Depending on the application, one might settle for different styles of realism without necessarily altering the scene input or configuration of the light sources. • Realistic lighting as one would experience in unwanted shadows • Lighting designed for optimally presented products and prototypes.

20. Conclusion • Computer graphics is a very exciting field in which to work and is probably one of the most challenging research areas in computer science because it has links with many other disciplines, many of them outside the traditional computer science community. • It is exactly this mix with disciplines such as art, psychology, filmmaking, biology, etc. that makes computer graphics very attractive to many people.