K. H. Ko School of Mechatronics Gwangju Institute of Science and Technology

1. Photo-realistic Rendering and Global Illumination in Computer Graphics Spring 2012Hybrid Algorithms K. H. Ko School of Mechatronics Gwangju Institute of Science and Technology

2. Metropolis Light Transport • It aims for robust global illumination that can handle light transport paths that are difficult to capture. • It demonstrates the application of Metropolis sampling to image generation by sampling the extremely high-dimensional space of all possible paths. • Basic Idea of Metropolis Sampling Technique • It can generate a sequence of samples from a non-negative function f such that the samples are distributed according to f. • There are more samples where f is large and vice-versa. • This important property of Metropolis sampling is achieved without any knowledge of f or its PDF. • The only requirement is that it should be possible to evaluate the function f at each generated sample.

3. Metropolis Light Transport • MLT applies the Metropolis sampling technique to the infinite-dimensional space of paths. • The key idea of MLT is that paths are sampled according to the contribution they make to the final image. • The algorithm generates a sequence of light transport paths by applying random mutations to the previous path. • Example mutations are adding a vertex in the path, deleting an existing vertex, etc. • Each proposed mutation could be accepted or rejected. • The probabilities that determine acceptance/rejection are chosen so that paths are sampled according to their contribution to the image plane. • The image is computed by sampling many paths and recording their contributions to the image plane.

4. Metropolis Light Transport • The main advantage of the MLT algorithm is that it is an unbiased algorithm that can handle hard-to-compute illumination situations. • MLT is efficient in computing images for scenes with strong indirect illumination that only arises through a small set of paths. • Once the algorithm finds an important, but hard to find, light transport path, it explores other paths “near” that path through mutations. • It is assumed that exploring that part of the path space will find other important light transport paths. • This local exploration of the space of paths can result in faster convergence in scenes as compared to other approaches such as bidirectional path tracing. • The fundamental framework of Metropolis sampling ensures that this faster convergence is achieved while still maintaining an unbiased technique.

5. Metropolis Light Transport • Another benefit of this approach is that the contribution of a new path can be computed relatively inexpensively. • Only a small part of the entire path is changed by the mutation. • The visibility information for the unchanged segments of the path do not need to be recomputed.

6. Metropolis Light Transport • Detailed Balance • Given a state space Ω, a non-negative function f: Ω -> R+, and an initial seed x0 ∈ Ω, the Metropolis sampling algorithm generates a random walk x0, x1, …, such that the xi are eventually distributed according to f, irrespective of the choice of x0. • To achieve this steady-state distribution of samples, mutations have to be accepted or rejected carefully. • The acceptance probability a(x->y) gives the probability that a mutation from x to y is accepted. • A transition function T(x->y) gives the probability density that a mutation technique would propose a mutation from state x to y.

7. Metropolis Light Transport • Detailed Balance • For a random walk in steady-state, the transition density between two states must be equal: • f(x)T(x->y)a(x->y) = f(y)T(y->x)a(y->x). • This condition is known as detailed balance. • Since f and T are given, the following choice of a results in equilibrium being achieved the fastest.

8. Metropolis Light Transport • Algorithm • The MLT algorithm starts with a set of n random paths, constructed using bidirectional path tracing, from the lights to the image plane. • These paths are then mutated using mutation strategies. • When a path x is mutated to produce a path y, the mutation is accepted based on the probability a given. • In particular, if the new path y does not contribute to the image, then the acceptance probability will be zero and the mutation will be rejected.

9. Metropolis Light Transport • Mutation Strategies • Bidirectional mutation • This mutation deletes a subpath of the path, extends the ends of the two remaining subpaths with one or more vertices, and then connects these ends together. • The paths are accepted based on the acceptance probability a. • Perturbations • Perturbations try to make small changes to a path, say by moving one or more vertices of the path, while leaving most of the path the same. • Caustic perturbations, lens perturbations, and multichain perturbations to capture different light paths more efficiently.

10. Metropolis Light Transport • Pseudo code of the basic MLT algorithm.

11. Metropolis Light Transport • MLT is an unbiased technique to compute light transport that is efficient at computing images of scenes that include hard-to-find light transport paths. • Once a hard-to-find path is found, mutations explore that part of the path space thoroughly before going to another part of the space.

12. Metropolis Light Transport • However, the implementation of MLT is quite complicated. • Care must be taken to get several important details of the algorithm right for it to work. • It is unclear how MLT performs for scenes that include multiple important paths. • It is possible that the mutations will result in slower convergence by exploring only a few of the many important paths thoroughly.

13. Metropolis Light Transport Ref: www-graphics.stanford.edu/papers/metro

14. Irradiance Caching • Monte Carlo rendering can take a long time to converge to images of reasonable quality. • Using pure Monte Carlo sampling to compute irradiance (incoming radiosity) at a point could require hundreds of ray-tracing operations. • Each of these operations, in turn, could result in more rays being traced in the scene. • Thus, this computation could be extremely slow. • Irradiance caching is an effective technique for accelerating the computation of indirect illumination in diffuse scenes. • It exploits the insight that the irradiance at diffuse surfaces, while expensive to compute, varies smoothly in most scenes. • Irradiance is cached in a data structure, and when possible, these cached values are interpolated to approximate irradiance at nearby surfaces.

15. Irradiance Caching • Interpolation • Irradiance gradients are used to determine when cached values can be interpolated to produce reasonably accurate results. • The translation and rotation gradient estimate how irradiance changes with position and direction. • An error estimate, based on the split-sphere model is used to determine which samples can be used for interpolation without introducing visible artifacts.

16. Irradiance Caching • Interpolation • Using this model, the error at a point P due to a cached sample i at location Pi is given as • Ri is the mean harmonic distance of objects visible from the cached sample i. • NP and Npi are the normals at P and the sample at Pi. • Note that this error term penalizes samples whose normals differ significantly from the normal of the point whose irradiance is being approximated.

17. Irradiance Caching • Interpolation • Similarly, samples that are far away are penalized. • Also, samples that are close to other surfaces, i.e., their mean harmonic distance is small, are penalized. • Irradiance at the point P is interpolated using the cached irradiance values of nearby samples using the weight ωi for the ith sample: ωi(P)=1/Єi(P).

18. Irradiance Caching • Interpolation • If a point has a large error, its weight is small, and vice-versa. • A user-specified parameter a is further used to eliminate samples whose weights are too small.

19. Irradiance Caching • Interpolation • The interpolated irradiance at point P is then • Ei(P) is the computed illuminance at Pi extrapolated to P. • The extrapolation is computed using the rotation and translation gradients of the cached values.

20. Irradiance Caching • The cached samples are stored in an octree constructed over the scene. • This data structure permits the decoupling of geometry from illumination values. • When the irradiance at a point must be computed, the octree is searched to find “nearby” cached samples that are accurate enough to be used to approximate irradiance. • The user-specified weight cutoff a specifies a radius over which the samples are searched. • If such samples are found, they are used to interpolate irradiance using the weighting algorithm. • If such samples do not exist, a sample is computed for the current point. This sample is then stored in the irradiance cache to be reused for interpolation later.

21. Photon Mapping • Photon mapping is a practical two-pass algorithm that traces illumination paths both from the lights and from the viewpoint. • However, unlike bidirectional path tracing, this approach caches and reuses illumination values in a scene for efficiency. • Itis a two-pass algorithm • In the first pass, “photons” are traced from the light sources into the scene. • These photons, which carry flux information, are cached in a data structure, called the photon map. • In the second pass, an image is rendered using the information stored in the photon map.

22. Photon Mapping • Photon mapping decouples photon storage from surface parameterization. • This representation enables it to handle arbitrary geometry, including procedural geometry, thus increasing the practical utility of the algorithm. • It is also not prone to meshing artifacts. • By tracing or storing only particular types of photons, it is possible to make specialized photon maps, just for that purpose. • The caustic map: It is designed to capture photons that interact with one or more specular surfaces before reaching a diffuse surface. These light paths cause caustics. • Traditional Monte Carlo sampling can be very slow at correctly producing good caustics. • By explicitly capturing caustic paths in a caustic map, the photon mapping technique can find caustics efficiently.

23. Photon Mapping • Photon mapping is a biased technique. • The bias is the potentially nonzero difference between the expected value of the estimator and the actual value of the integral being computed. • Since photon maps are typically not used directly, but are used to compute indirect illumination, increasing the photons eliminates most artifacts.

24. Photon Mapping • Tracing Photons: Pass 1 • The use of compact, point-based “photons” to propagate flux through the scene is key in making photon mapping efficient. • Photons are traced from the light sources and propagated through the scene just as rays are in ray tracing. • They are reflected, transmitted, or absorbed. • Russian roulette and the standard Monte Carlo sampling techniques are used to propagate photons. • When the photons hit nonspecular surfaces, they are stored in a global data structure called the photon map. • To facilitate efficient searches for photons, a balanced kd-tree is used to implement this data structure.

25. Photon Mapping • Tracing Photons: Pass 1

26. Photon Mapping • Tracing Photons: Pass 1 • Photon mapping can be efficient for computing caustics. • A caustic is formed when light is reflected or transmitted through one or more specular surface before reaching a diffuse surface. • To improve the rendering of scenes that include caustics, the algorithm separates out the computation of caustics from global illumination. • Thus two photon maps, a caustic photon map and a global photon map, are computed for each scene. • The caustic map includes photons that traverse the paths LS+D. • The global photon map represents all paths L(S|D)*D.

27. Photon Mapping • Tracing Photons: Pass 1

28. Photon Mapping • Tracing Photons: Pass 1 • Caustic photon maps can be computed efficiently because caustics occur when light is focused. • Not too many photons are needed to get a good estimate of caustics. • Additionally, the number of surfaces resulting in caustics in typical scenes is often very small. • Efficiency is achieved by shooting photons only towards this small set of specular surfaces.

29. Photon Mapping • Tracing Photons: Pass 1 • The reflected radiance at each point in the scene can be computed from the photon map. • The photon map represents incoming flux at each point in the scene. Therefore, the photon density at a point estimates the irradiance at that point. • The reflected radiance at a point can then be computed by multiplying the irradiance by the surface BRDF.

30. Photon Mapping • Tracing Photons: Pass 1 • Photon Storing • Photons are stored only when they hit diffuse surfaces (or, more precisely, non-specular surfaces). • Storing photons on specular surface does not give any useful information: the probability of having a matching incoming photon from the specular direction is very small (and zero for perfect specular materials). • For all photon-surface interactions, data is stored in a global data structure, the photon map. • Each emitted photon can be stored several times along its path. • Information about a photon is stored at the surface, where it is absorbed if that surface is diffuse.

31. Photon Mapping • Tracing Photons: Pass 1 • Photon Storing • It is important to realize that photons represent incoming illumination (flux) at a surface. • This is a valuable optimization that enables us to use a photon to approximate the reflected illumination at several points on a surface..

32. Photon Mapping • Photon Storing

33. Photon Mapping • Tracing Photons: Pass 1 • To compute the photon density at a point the n closest photons to that point are found in the photon map. • This search is efficiently done using the balanced kd-tree storing the photons. • The photon density is then computed by adding the flux of these n photons and dividing by the projected area of the sphere containing these n photons. • Thus, the reflected radiance at the point x in the direction ω is

34. Photon Mapping • Tracing Photons: Pass 1

35. Photon Mapping • Computing Images: Pass 2 • The simplest use of the photon map would be to display the reflected radiance values computed in pass 1 for each visible point in an image. • However, unless the number of photons used is extremely large, this display approach can cause significant blurring of radiance, thus resulting in poor image quality. • Instead, photon maps are more effective when integrated with a ray tracer that computes direct illumination and queries the photon map only after one diffuse or glossy bounce from the view point is traced through the scene.

36. Photon Mapping • Computing Images: Pass 2 • The final rendering of images could be done as follows: • Rays are traced through each pixel to find the closest visible surface. • The radiance for a visible point is split into direct illumination, specular or glossy illumination, illumination due to caustics, and the remaining indirect illumination. • Direct illumination for visible surfaces is computed using regular Monte Carlo sampling. • Specular reflections and transmissions are ray traced. • Caustics are computed using the caustic photon map. Since caustics occur only in a few parts of the scene, they are computed at a higher resolution to permit direct high-quality display.

37. Photon Mapping • Computing Images: Pass 2 • The final rendering of images could be done as follows: • The remaining indirect illumination is computed by sampling the hemisphere; the global photon map is used to compute radiance at the surfaces that are not directly visible. This extra level of indirection decreases visual artifacts.

38. Photon Mapping • A visualization of both passes of the photon-mapping algorithm.

39. Photon Mapping

40. Photon Mapping

41. Photon Mapping

42. Photon Mapping • The use of the global photon map for indirect illumination is reminiscent of the final gathering approaches in radiosity algorithms. • However, by storing caustic maps that can be visualized directly, this algorithm is able to capture challenging caustic paths.