Ray Tracing with Existing Graphics Systems

1. Ray Tracing with Existing Graphics Systems Jeremy Sugerman, FLASHG 31 January 2006

2. Why Consider Tracing Rays? • Some techniques are hard to mimic with rasterization. • Shadows (arbitrary view visibility) • Reflection, Refraction effects • Global, Indirect Illumination

3. Why Now? • Raytracing is getting fast enough to use interactively. • New architectures seem potentially suited or not too far from being suited.

4. “Interactive”, Really? • Intel MLTRA (SIGGRAPH 2005) • 20-36 fps (20-36 MRays/sec) on one 3.2 GHz • Without MLTRA, still 7-12 fps / MRays/sec • My toy SSE raytracer • About 2 MRays/sec (primary rays) • ‘Quick Hack’ Cell system at SIGGRAPH • Claims 20-25 fps / MRays/sec (unreviewed)

5. What about Rasterization? • 60+ fps over 1024x768 on $99 GPUs • Ray tracing would need 50+ MRays/sec casting primary rays alone. • GPUs are getting faster at a phenomenal rate. • Ray tracing is DOOMED!

6. It’s Not About Rasterization • The vast majority of render time is actually shading in most 3D apps. • True for offline rendering too. • Whether fragments are rasterized or ray traced, shading is the same… • To the extent the fragments are the same

7. And Who Traces Primary Rays? • We want ray tracing for shadows, reflection/refraction, and indirect illumination. • Those are all applications of secondary rays. • The rasterizer already produces (x, y, z), normals, etc. from primary hits. • Might as well rasterize them if you have a fast existing mechanism. It doesn’t matter.

8. Synthesis (Thesis + Antithesis) • Bolt a ray tracer onto a conventional rasterization based system. • Add a bit of ray tracer friendliness to a GPU. • Or, wire together a Cell and a GPU (don’t tell Sony). • Window systems (2D), text are rasterization tasks fundamentally (plus optional compositing)

9. “Ray Tracer Friendliness”? • Ray tracing strongly favours threaded architectures over SIMD. • Packet tracing leverages bandwidth at the cost of very simple horizontal communication. • Being able to use queues / write buffers seems critical.

10. How Would It Work? • Operate on ‘hits’ • Analogous to fragments, plus weights • Each hit feeds any of three (independent) shading choices • Gather rays • Shadowing and local lighting contribution • Secondary rays • Each produces a colour and a weight which are accumulated into a final pixel colour

11. Shadowing And Local Lighting • Effectively runs: • Interpolate shading data (BRDF, Normal, etc.) • Generate as many shadow rays as are desired • Foreach shadow ray { If (shadow ray hits light) { Compute local light contribution from the light } • Fits in the same per-fragment storage • Shadow computations are indepedent

12. Gather and Secondary Rays • Gather rays just perform a weighted lookup in a data structure (e.g. photon map) • Secondary rays are generated based on the surface shading information (BRDF, Normal, …) • Hits are fed back into the same pipeline • Once generated, independent of parent

13. Kinda Like a GPU (If you squint) • GPU shading just does the weighted local lighting calculation. • But a fragment program can generate and trace shadow rays to mask local lighting • And a fragment program can generate rays for final gathers. • All this formulation does is expose parallelism and offer natural places to optimize hardware

14. I Saw You Palm That Card! • Secondary rays are a bit harder to cram into a fragment program. • No variable output buffers • No recursion • Once generated, secondary rays are completely independent. • Only need an unordered write buffer • No state, so recursion becomes iteration • So start with only shadow and gather rays • Exact same system supports them, though

15. What About Coherence? • Rasterization lie: Rasterization systems exploit coherence vastly better than ray tracing. • Really means shading coherence • Coherence between fragments in texture lookups • Can bind a single material shader and rasterize only relevant geometry • Can perform (expensive!) shading math at a different resolution than visibility math.

16. Ray Tracing Is Coherent Too • Packet tracing is useful because of visibility coherence. • The same coherence is also somewhat relevant for local shading and lighting. • Even, render only objects with the same material in each pass and z-cull. • Just as coherent as rasterization

17. Ray Tracing Is Coherent Too • Packet tracing literature demonstrates ray coherence helps visibility most, but also shadow / secondary rays and shading. • Rasterization systems save overdraw by rendering a depth-only pass before material shading with z-culling. • The same technique works with rays!

18. Global Effects Are Incoherent • Nearly by definition, indirect and inter-object shading effects are incoherent. • In rasterization or ray-traced systems! • Packet tracing literature indicates secondary rays retain some coherence. • Geometry level of detail can regain coherence.

19. Coherence: Bottom Line • Without secondary rays, ray tracing shares the coherence characteristics of rasterization (modulo implementation). • With secondary rays, ray tracing offers natural mechanisms for effects that are only awkwardly kludged via rasterization. • Let the developer / director pick the tradeoff.

20. What Needs To Be Done? • Persuade Pat to get a PS3 dev kit? • Get a good ray tracer running on Cell and/or a GPU. • Simulate various extensions • Cram code in fragment programs • Readback to the CPU for now • Readback to a Cell • Talk to the Smart Memories folks? • Talk to GPU vendors (unlikely, they’re busy folks)

21. What’s Behind the Curtain? • K-d tree building and updates • But, Rasterization systems do this today • K-d tree storage and bandwidth needs • Hardware changes for traversing k-d trees or intersecting triangles • Unclear if it’s even necessary initially • Level of detail with secondary rays • Decoupling visibility and shading resolutions

22. Contributions and Inspirations • Tim, Pat • Gordon Stoll • Bill Mark • Phil Slusallek and the Saarland crew • Tim Purcell, John Nickolls (NVIDIA)

23. Rasterization Lies • Rasterization has coherence Ray tracing lacks. • The rasterization argument goes: rasterize all the geometry of a given material and there’s excellent coherence of shading samples. • Without secondary rays, ray tracing is the same. With secondary rays, you get better pictures. The programmer gets to choose.