1 / 45

Optimizing Performance 2

Optimizing Performance 2. David Luebke University of Virginia. But first…. NV40 demos. Project 2. Any questions/problems/issues?. Interruptible Rendering. David Luebke Department of Computer Science University of Virginia. Cliff Woolley Ben Watson Abhinav Dayal.

jennis
Download Presentation

Optimizing Performance 2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Optimizing Performance 2 David Luebke University of Virginia

  2. But first… • NV40 demos Interruptible Rendering

  3. Project 2 • Any questions/problems/issues? Interruptible Rendering

  4. Interruptible Rendering David Luebke Department of Computer Science University of Virginia Cliff WoolleyBen Watson Abhinav Dayal Presented at ACM SIGGRAPH 2003 Symposium on Interactive 3D Graphics

  5. Problem: fidelity vs. performance • Interactive graphics requires trading off detail vs. frame rate • Conventional wisdom dictates: • Maintain a high and constant frame rate • Thus, use a level of detail that can always be rendered in time • This is simplistic! Can we do better? Interruptible Rendering David Luebke

  6. Improve on traditional LOD • Is a high, constant frame rate good enough? • Is fidelity ever more important than frame rate? • How can this be decided at runtime while still guaranteeing interactivity? Interruptible Rendering David Luebke

  7. Key observation • The fidelity/performance tradeoff can be seen in terms of spatial versus temporal error… Interruptible Rendering David Luebke

  8. Key observation • The fidelity/performance tradeoff can be seen in terms of spatial versus temporal error… …and these should be measured and compared directly! Interruptible Rendering David Luebke

  9. Unified spatial/temporal error • We combine temporal error (“lateness”) and spatial error (“coarseness”) into a unified dynamic visual errormetric • We use this to drive progressive refinement Interruptible Rendering David Luebke

  10. Progressive refinement • The big question: At what point does further refinement of the current frame become pointless? Interruptible Rendering David Luebke

  11. Progressive refinement • The big question: • Answer: When temporal error exceeds spatial error At what point does further refinement of the current frame become pointless? Interruptible Rendering David Luebke

  12. Methodology • Refine a stream of continuous LODs • Monitor input frequently • Minimize dynamic visual error Interruptible Rendering David Luebke

  13. Refine a stream of continuous LODs • Render progressive refinements on top of each other until “out of time” • Ensure that we can stop refining at any time and move on to the next frame Interruptible Rendering David Luebke

  14. Monitor input frequently • Ideally: input monitored continuously • Realistically: check every x ms • Allows quick reaction when sudden changes in input occur • Allows system to be self-tuning Interruptible Rendering David Luebke

  15. Minimize dynamic visual error • Always display image with least error • Sometimes: • Further refinement is pointless • Temporal error (lateness) exceeds spatial error (coarseness) for the current frame • The front buffer is “out of date” • Dynamic visual error (spatial + temporal) in front is greater than in the back Interruptible Rendering David Luebke

  16. start Clear the back buffer Clear the front buffer Refine the back buffer Refine the front buffer time tb > sb? N Y Compute dynamic visual error Swap buffers tf > sf? N Y ef >= eb? N Y Swap buffers Rendering to Front Buffer Rendering to Back Buffer

  17. Refinement • Need a progressive rendering scheme • Subtlety: In depth-buffered rendering, you can’t “unrender” pixels from a coarse model • Thus for depth-buffered rendering rendering scheme should satisfy containment Interruptible Rendering David Luebke

  18. Refinement • Three refinement schemes implemented: • Splatting (Tech sketch, SIGGRAPH 2002) • Progressive polygonal hulls • Progressive ray casting Interruptible Rendering David Luebke

  19. V3 V2 Vn V1 V4 Progressive hulls • Simplification method by [Sander et al. 2000] • Record a sequence of constrained edge collapses and play it back in reverse • Guarantees containment Interruptible Rendering David Luebke

  20. Calculating dynamic visual error • We use a simple screen-space metric • Spatial error: • Maximum projected error of visible geometry • Temporal error: • Project points on object and/or bounding volume • Find the maximum screen-space distance any point has moved since frame began • More elaborate/correct metrics possible! Interruptible Rendering David Luebke

  21. Progressive ray casting • Progressive refinement • Coarse to fine sampling/reconstruction • Sampling • Where to shoot the ray • Adaptive • Non-adaptive • Reconstruction • Producing imagery from samples Interruptible Rendering David Luebke

  22. Sampling • Coarse to fine - quadtree approach • Sample every quadtree node's center • Breadth first traversal • (Predetermined) random traversal per level • Ray casting – using OpenRT • Per ray API ~ 600,000 rays per sec • Simple shading, shadows and specular highlights Interruptible Rendering David Luebke

  23. (Simplistic) reconstruction • Placement • Place splat at center of each quadtree node • Shading • Flat shaded quad covering the node's screen space • Alpha-textured quads (smooth reconstruction) • Quad size = twice of node's screen space • Texture: Gaussian blob • Transparent at edges • Opaque at center Interruptible Rendering David Luebke

  24. Alpha textured Flat shaded

  25. Spatial error Calculating dynamic visual error • Temporal error • Computed as in case of polygonal hulls • Spatial error • Diagonal length of the largest quadtree node displayed on the screen Interruptible Rendering David Luebke

  26. Demo Interruptible Rendering David Luebke

  27. Evaluation: the “Gold Standard” • Compares an ideal rendering to interactive approximations • Ideal rendering: full detail, zero delay • Interactive approximations are • Unmanaged • Constant fidelity (in pixels) • Constant frame rate (fixed Hz) • Interruptible rendering Interruptible Rendering David Luebke

  28. Frame generation • Record what the user sees and when • Generate each approximation offline • Record actual frames displayed over time • Account for: • Render time • Double buffering • Frame locking • Rendering into front buffer (interruptible) Interruptible Rendering David Luebke

  29. Comparing frames • Error measures • RMS – Root Mean Square error • Lindstrom's perceptually based error Difft = Idealt - Renderedt Interruptible Rendering David Luebke

  30. Two test input streams • Rotation • Model-centered • Fixed angular velocity • User interaction • Includes both view translation and rotation • Includes both static and dynamic segments • Both input streams recorded to files, with timestamps Interruptible Rendering David Luebke

  31. Interaction sequence: ray caster 42 45 54 57 Frames: 33 36 39 48 51 60 Ideal Inter-ruptible Constant Fidelity Un-managed

  32. Video Interruptible Rendering David Luebke

  33. Rotation input stream Interactive input stream Progressive hulls error(rms) error(rms) time(secs) time(secs) Ray casting error(rms) error(rms) time(secs) time(secs) constant frame rate constant fidelity interruptible unmanaged

  34. Benefits • Accuracy • Balances spatial and temporal error • Interactivity • Even w/ slow renderers like our ray caster • Or large models • Self-tuning system • Adapts to hardware Interruptible Rendering David Luebke

  35. Limitations • Overdraw in progressive renderer • Progressive ray caster better here • Cost of monitoring temporal error • Rendering immersive models • Requires reversing the containment criteria for polygon rendering • Not a problem for ray caster Interruptible Rendering David Luebke

  36. Ongoing and future work • Improve implementation: • Textures, normal maps, etc. • Reducing overdraw • Parallelization of ray tracing system (Ben & Joe) • Adaptive (view-dependent) refinement (Ben & Bob) • Ray tracing seems like the way to go… • Better estimates of temporal & spatial error • Use color comparisons, e.g. to an up-to-date but low-resolution ray-traced image • Use contrast sensitivity function or related ideas Interruptible Rendering David Luebke

  37. To think about • Ultimately this work is about: • A principled approach to the fidelity vs. performance tradeoff • New ideas on temporal sampling & error • Can we take these ideas further? • Frames and framebuffers are overrated • Can we decouple temporal & spatial sampling? • When and where to sample? • How to reconstruct? Interruptible Rendering David Luebke

  38. Next steps • Motivation: Interactive ray tracing • From bad joke to old hat in a couple of years • Lots of work on “how” • Supercomputer [Parker et al. 99] • PCs and PC clusters [Wald & Slussalek 2000] • GPUs [Purcell et al. 2002, Carr & Hart 2002] • We are interested in “why” Interruptible Rendering David Luebke

  39. Interactive ray tracing • The big question: What can you do with an interactive ray-based renderer that you can’t do with a rasterizer? Interruptible Rendering David Luebke

  40. Interactive ray tracing • The big question: • Can vary sampling rate across the image • Focus on sampling edges, high-frequency regions • Exploit eccentricity, velocity, etc. What can you do with an interactive ray-based renderer that you can’t do with a rasterizer? Interruptible Rendering David Luebke

  41. Interactive ray tracing • The big question: • Can vary sampling rate across time • Sample more frequently when things are changing • Sample more frequently where things are changing What can you do with an interactive ray-based renderer that you can’t do with a rasterizer? Interruptible Rendering David Luebke

  42. Rethinking rendering • Goal: rethink spatial & temporal strategies for interactive sampling & reconstruction • Related work: • Frameless rendering • Just-in-time pixels • Sample reuse • Sparse sample reconstruction Interruptible Rendering David Luebke

  43. Nuts and bolts • What we’re doing: spatio-temporally adaptive sampling • Update samples with higher priority in regions of higher variance • Spatial variance: edges • Temporal variance: motion • Reconstruction of resulting samples • The “deep buffer” stores samples in time & space • Reconstruct image at front edge of time: apply filter kernel with varying width in space and time Interruptible Rendering David Luebke

  44. Questions going forward • How best to vary sampling rate to respond to variance? • What exactly do we mean by “variance”? • How best to generate an image from non-uniform samples when some are more “stale” than others? • What shape should kernel be, especially in time? • Can we simulate frameless display hardware? Interruptible Rendering David Luebke

  45. Acknowledgements • Peter Lindstrom for ltdiff • OpenRT Interactive Raytracing Project • Stanford 3D Scanning Repository • National Science Foundation • Awards 0092973, 0093172, and 0112937 Interruptible Rendering David Luebke

More Related