Interactive Global Illumination in Complex and Highly Occluded Environments

1. Interactive Global Illumination in Complex and Highly Occluded Environments Ingo Wald Carsten Benthin Philipp Slusallek Computer Graphics Group Saarland University http://graphics.cs.uni-sb.de/

2. Agenda • Instant Global Illumination • Summary and “State of the Art” • Complex and Highly Occluded Scenes • Issues and problems in such scenes • Light Source Importance Sampling • Estimating importance • Importance sampling • Application to Instant Global Illumination • Handling temporal artifacts • Results and Discussion EGSR'03 - Leuven, Belgium

3. Global Illumination in Complex and Highly Occluded Scenes • Important: Proposed Method is General • Also works for BiDirectional Path Tracing, Photon Mapping, etc... • In this talk: Only concentrate on Application to“Instant Global Illumination” ! EGSR'03 - Leuven, Belgium

4. Instant Global Illumination (IGI)Summary • Builds on Instant Radiosity [Keller Sig97] • Generate “Virtual Point Lights” (VPLs) • ... by tracing particles from light sources • VPLs represent both direct and indirect light • Illuminate scene from those point lights (with shadows) • The more VPLs, the better the image quality • But: On top of an interactive distributed ray tracing engine • Saarland RTRT/OpenRT engine [Wald et al. EG01+] • 1 - 48 CPUs (Athlon MP 1800+) • Additional techniques for higher performance (~9x) • Interleaved Sampling [Keller, Heidrich] • Discontinuity Buffering [Keller, Kollig] EGSR'03 - Leuven, Belgium

5. Instant Global Illumination (IGI)“State of the Art” • Now: Much faster and better than RW02 [Benthin EG03] • For simple scenes: up to ~25 fps @ 640x480 • With programmable shading, antialiasing, reflections, etc. • Can handle millions of triangles • As long as rays are coherent ... • ... and if there is few occlusion Shirley-6: 800tris, 25fps “PowerPlant”: 37 million tris, 1 light, 2 fps EGSR'03 - Leuven, Belgium

6. Realistically Complex Scenes • Want to apply IGI to “realistically” complex scenes • Whole buildings, construction sites, etc... • Common features: • High geometric complexity • Millions of triangles • Many lights • Thousands of lights • High occlusion EGSR'03 - Leuven, Belgium

7. Complex and Highly Occluded Scenes • Example: “Soda Hall” • 7 storey building, dozens of rooms per storey • 2.5 million triangles • 36,000 light sources • High occlusion • E.g. different storeys hardly connected at all... EGSR'03 - Leuven, Belgium

8. IGI Problems in Complexand Highly Occluded Scenes IGI Issues in Complex Scenes: • Geometric complexity • Would be bad for radiosity-style algorithms • But: Not a problem for IGI • Underlying Ray tracer logarithmic in scene complexity • If rays are coherent... • Number of light sources: Subsampling • At least of there is no occlusion • High occlusion: Bad... • Strongly varying view importance • Waste VPLs for “illuminating” unimportant regions EGSR'03 - Leuven, Belgium

9. IGI: Problems with High Occlusion • Interactive: Small, fixed set of VPLs • Only ~100-200 affordable (10-20 per interleaving pattern) • VPLs don’t consider view importance • Most VPLs will be in distant, occluded rooms • Many costly shadow rays, but no actual contribution to the image • More rooms  Fewer VPLs per room: • Typical “single room” scene: 100-200 VPLs • In 100-room scene: Only 1-2 VPLs per room on average  Bad image quality • In 1,000-room scene: One VPL every 10 rooms ! • Many rooms completely “black”... Important side note: Similar for other GI algorithms !!! • Light paths in BDPT, photons in photon mapping, ... EGSR'03 - Leuven, Belgium

10. IGI - Problems with high occlusionPractical Examples Example Scenes (Shirley10, Ellipse, Soda Hall) With “standard” Instant Global Illumination (~300VPLs)  Rendering quality simply not tolerable ! EGSR'03 - Leuven, Belgium

11. Light Source Importance Sampling • Observation: Each room influenced by few lights • Much more efficient if all others could be “turned off” • In practice and industry: Often done manually... • Not possible for interactive walkthroughs • Idea: Concentrate VPLs in important rooms  2-step algorithm (every frame) • 1st step: Determine important light sources • I.e. lights that are not occluded • Estimate/Compute contribution of each light to the image • 2nd step: Importance sampling during VPL generation • Generate more VPLs on important lights • Completely ignore (probably) occluded lights EGSR'03 - Leuven, Belgium

12. 1st Step: Determine Light Source Importances Importance estimation: Simple eye path tracer • Simple and correct • Parallelizes trivially • Simple cost control (samples per pixel) • Most importantly: Completely view-importance driven • Can sample all lights, but never touch occluded regions • If shot from the surface to the light • Rays never reach geometry around occluded lights... • Never start rays, paths, particles from the light source • Never touch occluded geometry • Does not destroy ray tracing performance • Problem: Pure path tracing is noisy... EGSR'03 - Leuven, Belgium

13. Path Tracing is noisy.True, but: • Only need rough estimate anyway • Never visualize estimate directly • Only for importance sampling  Does not have to be perfect • Only need light contribution to whole frame • Each pixel value itself noisy, but 640x480=0.3M pixels • Have relatively many samples • 3 Samples per pixel at 640x480 = 1M samples • 1M samples / 1000 lights = 1000 samples per light • Estimate is reasonably correct • Except for very unimportant lights • ... which we want to turn off anyway... EGSR'03 - Leuven, Belgium

14. Estimate Quality Example: “Shirley10”, One path per pixel • Path traced image hardly recognizable... • But: Estimate correct up to a few percent Estimate (1 sample/pix) Real scene EGSR'03 - Leuven, Belgium

15. Constructing an Importance Sampling PDF • In theory: Sampling PDF relative to absolute contribution • Leads to oversampling of very important lights • Often spend 80% of all samples on one small, bright source.... • Leads to undersampling of lights influencing few pixels • I.e. a room seen through a door • Few covered pixels  small total image contribution • In practice • Clamp absolute contribution (user parameter) • Optional: Threshold with minimum contribution (user parameter) • Effectively turn off lights with very small contributions • Bias... EGSR'03 - Leuven, Belgium

16. Importance Sampling using the Estimated Contribution “Clamp and Threshold”: Example • Example light sources contributions from estimate: 80%, 10%, 5%, 3%,0.5%, 0.3%, ... • Clamp (e.g. with 5% of total contribution): 5, 5, 5, 3, 0.5, 0.3 ... • Threshold (e.g. with 1% of total contribution): 5, 5, 5, 3, 0, 0, .... • Probabilities: 5/18th, 5/18th, 5/18th, 3/18th, rest turned off.... EGSR'03 - Leuven, Belgium

17. Integration into IGI • Integration into IGI framework: Quite easy • Only have to modify “eye path” generation But: Temporal artifacts  Flickering • Only few VPLs affordable (~100-200)  Always slight error in the image • Smooth error : Not visible in static images • Changing PDF in-between frames changes VPL positions • “Jumping” of VPLs • Error in image changes from frame to frame • And: PDF always changes (interaction, noise in estimate, etc) • Flickering, due to “jumping” of VPLs • Highly disturbing EGSR'03 - Leuven, Belgium

18. Handling Temporal Artifacts • Main goal: Avoid jumping of VPLs • In “old IGI” : Use same random numbers in successive frames • Here: Won’t work, as the PDF still changes • First measure: Temporal smoothing of PDF • Weighted average of old and new PDF • Higher Latencies until drastic changes take effects • More important: Modify VPL generation algorithm • Don’t generate VPLs independently • First determine total #VPLs for each light source • Give each light its own, unique random number sequence • Use same sequence in successive frames • If #VPLs changes from N to M, first min(N,M) samples will always remain the same  Minimizes VPL jumping ! EGSR'03 - Leuven, Belgium

19. Results IGI Video EGSR'03 - Leuven, Belgium

20. Results IGI Results: • Estimate step quite costly (30-50% of total cost) • Path tracer : Incoherent, single rays, ...  Much slower than “normal” rays in IGI • But: Much less lights to consider • Shirley10: 6-8 vs. 100 (Savings: > 10x) • Soda Hall: 100-300 vs. 36,000 (Savings: > 100x) • Some artifacts still visible • Proposed techniques help reduce flickering • But: Some temporal noise still visible • Additionally: Sometimes undersampling artifacts • Much higher quality at same frame rate • Even when considering 50%-overhead ! EGSR'03 - Leuven, Belgium

21. Results IGI • Side-by-side Comparison of old and new system • ... at same #CPUs and same frame rate: old new EGSR'03 - Leuven, Belgium

22. Results IGI • Side-by-side comparison in “Soda Hall” • ... at same frame rate: EGSR'03 - Leuven, Belgium

23. Future Work • Most important: Further reduce temporal artifacts • Clustering of nearby lights • Cheaper and faster estimate • Improve on PDF construction • Can probably do better than just clamping and thresholding... • Consider more advanced aspects of VPL generation • Direction of emission • Number of bounces • Localization of VPLs ??? • I.e. different VPLs for different parts of image/scene ? • Generate VPLs from the eye ? • I.e. a la Bekaert’s RW02 “Reusing Paths” ... EGSR'03 - Leuven, Belgium

24. Thanks for your attention Questions ? also visit http://www.openrt.de EGSR'03 - Leuven, Belgium

25. ---------------------Cut-out stuff EGSR'03 - Leuven, Belgium

26. Offline Results • Proof-of-concept: Simple Bidirectional Path Tracer • Fixed path length (3 light nodes, 3 eye nodes) • Only have to modify eye path generation • Trivial to implement • Should work similarly for other GI algorithms • Results: • Estimate rather cheap • 1-2 paths per pixel for estimate, 128-1024 during rendering • Much more coherent rays (fewer triangles touched) • Much higher image quality at same number of rays • But: May get undersampling artefacts • I.e. might miss room covering only few pixels... • In practice: Much better than without EGSR'03 - Leuven, Belgium

27. Offline Results Results of integration into Offline Renderer Soda Hall, 64 bidirectional paths per pixel old new EGSR'03 - Leuven, Belgium

28. Complex and Highly Occluded Scenes Complex and Highly Occluded Scenes • Example: Pete Shirleys “ERW10” • 10x10 times “ERW6”-type room • 100 rooms, 100 lights • Almost the same image, but: 100 times as costly ! EGSR'03 - Leuven, Belgium

29. Complex and Highly Occluded Scenes – Issues for GI Such scenes: Highly “non-trivial” for Global Illumination ! • Geometric Complexity • Up to millions of triangles • To much for many algorithms (i.e. radiosity, etc) • Many Lights • Often scattered all over the model • High Occlusion • Lights highly localized • Most areas only influenced by few lights • Strongly varying view importance • Often waste light samples where they will not contribute ! • Photons in a photon map, light paths in BDPT, VPLs in IGI, etc EGSR'03 - Leuven, Belgium