1 / 39

Direct-to-Indirect Transfer for Cinematic Relighting

Direct-to-Indirect Transfer for Cinematic Relighting. Milos Hasan (Cornell University) Fabio Pellacini (Dartmouth College) Kavita Bala (Cornell University). Introduction. Cinematic Relighting [Gershbein 2000, Pellacini 2005] Interactive local light movement, fixed camera

jeffreywaters
Download Presentation

Direct-to-Indirect Transfer for Cinematic Relighting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Direct-to-Indirect Transfer for Cinematic Relighting Milos Hasan (Cornell University) Fabio Pellacini (Dartmouth College) Kavita Bala (Cornell University)

  2. Introduction • Cinematic Relighting [Gershbein 2000, Pellacini 2005] • Interactive local light movement, fixed camera • Procedural shaders, various materials • High-complexity scenes • Only direct illumination • The goal of our system: • Add multiple bouncesof indirect illumination

  3. Transfer matrix Final Direct Indirect Direct-to-Indirect Transfer • Direct illumination from local lights • Use known techniques • Deep frame-buffer, shadow maps • Indirect illumination – key idea: • Precompute direct-to-indirect transfer matrix • Compute indirect from direct on the fly

  4. Interactive Demo

  5. Related Work • Cinematic relighting engines • [Gershbein 00; Pellacini 05, Tabellion 04] • Precomputed radiance transfer • [Sloan 02, 03, 05; Kautz 02; Ng 03, 04; Liu 04; Wang 04, 05; Annen 04, etc.] • Linear combinations of full solutions • [Kristensen 05; Dorsey 91; Debevec 00; etc.]

  6. Related Work • Sparse sampling • [Walter 99, 02; Bala 99, 03; Ward 99; Tole 02; Nijasure 03; Gautron 05; Dayal 05; Simmons 01 etc.] • Hierarchical / instant radiosity methods • [Hanrahan 91; Keller 97; Drettakis 97; Christensen 97; Dachsbacher 05, 06; etc.] • Wavelet radiance transport • [Kontkanen 06] (concurrent work)

  7. Key concepts Transfer matrix Efficient precomputation Efficient compression Efficient evaluation Results Conclusions and Future work Outline

  8. i = T d Indirect on view Transfer matrix Direct on gather Key Concepts • View samples • Gather samples • Transfer: A large matrix • Contributions of gather samples to view samples

  9. View Samples Camera A scene View samples

  10. Gather Samples Gather samples A scene

  11. Transfer Matrix Elements Tij = ?? Gather sample j View sample i

  12. Direct Illumination • Standard techniques: GPU, shaders, shadow maps Direct on gather samples Direct on view samples

  13. Conceptual Overview Direct on view Transfer matrix Final Direct on gather Indirect on view

  14. Key concepts Transfer matrix Efficient precomputation Efficient compression Efficient evaluation Results Conclusions and future work Outline

  15. Why Is Precomputation Hard • Transfer matrix is huge! • View samples:640 x 480 = 300k rows • Gather samples: 64k columns • 19 billion elements • Columns: • Images with 1 light and full global illumination 64k gather 300k view Full global illumination images

  16. Multi-bounce & Final Gather • Split full transfer matrix: • Multiple-bounce matrix (lower accuracy) • Final gather matrix (higher accuracy) Full transfer:Many gather-to-view Multi-bounce:Many gather-to-gather Final gather:One gather-to-view

  17. Final Gather Matrix Elements Gather sample j Fij = ?? Single-bounce contribution View sample i

  18. Multi-bounce Matrix Elements Gather sample i Mij = ?? Gather sample j

  19. Multi-bounce Matrix:Sparse Approximation • Photon mapping [Jensen 96] analogy • But, don’t know light positions • Shoot photons from gather samples instead! • Photons carry gather sample ID • Like standard photon mapping with 64k lights!

  20. Multi-bounce Matrix:Computing by Photon Mapping A gather sample i-th row of M = ?? K nearest photons Gather sample i

  21. Conceptual Overview, Updated Multi-bounce matrix Add direct Direct on gather Indirect on gather Dir + Ind on gather Final gather matrix Add direct Indirect on view Final

  22. Key concepts Transfer matrix Efficient precomputation Efficient compression Efficient evaluation Results Conclusions and future work Outline

  23. Defining a Wavelet Basis • Want to use wavelets • How? There is no obvious domain to apply them… • Unstructured cloud of gather samples • Still possible to define wavelets on the cloud! • Flatten gather cloud to a 2D array • Maintain coherence • Use 2D Haar wavelets on the 2D array

  24. Defining a Wavelet Basis 64k 256 x 256 256 x 256 Gather samples Flattened in a 2D array Wavelet transform

  25. Hierarchical Partitioning

  26. Tree Arrangement

  27. 64k 64k 64k 64k 64k 64k 300k 300k Sparse Even more sparse Dense Sparse Wavelet Compression • Transform rows into Haar wavelets • Remove small coefficients Final gather matrix Multi-bounce matrix

  28. Columns of the Wavelet-Transformed Matrix Final gather matrix in wavelets Images lit by wavelet lights

  29. Direct on gather in wavelet space Direct on gather Dir + Ind on gather Full System Overview Multi-bounce matrix in waveletspace Wavelet Xform Wavelet Xform Final gather matrix in waveletspace Add direct Dir + Ind on gather in wavelet space Indirect on view Final Image

  30. Key concepts Transfer matrix Efficient precomputation Efficient compression Efficient evaluation Results Conclusions and future work Outline

  31. Sparse Matrix-Vector Multiplication Sparse matrix Linear combination of columns Sparse vector

  32. Sparse Matrix-Vector Multiplication on the GPU • Problem: • Columns are sparse themselves • How to represent on the GPU? • Solution: • Non-zero elements tend to cluster… • Cut out rectangular blocks • Pack blocks into texture atlases • Converted problem to blending!

  33. Scene: Still Life Precomputation: 1.6 hours 11.4 – 18.7 fps Polygon count: 107k

  34. Scene: Temple Precomputation: 2.5 hours 8.5 – 25.8 fps Polygon count: 2 million

  35. Scene: Hair Ball Precomputation: 2.9 hours 9.7 – 24.7 fps Polygon count: 320k

  36. Scene: Sponza Atrium Precomputation: 1.5 hours Frame-rate: 13.7 – 24.9 Polygon count: 66k

  37. Comparison Our system: 8-25 fps (2.5 hr precomputation) Monte Carlo path tracer: 32 hours

  38. Conclusions and Future Work • Conclusion • Extend cinematic relighting with indirect illumination • Interactive performance (GPU), complex scenes • Arbitrary light shaders, efficient precomputation • Future work • Environment mapping • Subsurface scattering • Moving camera

  39. Questions? • Acknowledgements • NSF grant CCF-0539996 • Bruce Walter (code, support) • Veronica Sunstedt, Patrick Ledda (Temple) • Marko Dabrovic (Sponza atrium) • Cornell animation TA’s (Still life) • Pixar, NVidia • E-mail: mhasan@cs.cornell.edu

More Related