Perceptually Guided Interactive Rendering

1. Perceptually Guided Interactive Rendering David Luebke University of Virginia

2. Always start with a demo…

3. Motivation:Preaching To The Choir • Interactive rendering of large-scale geometric datasets is important • Scientific and medical visualization • Architectural and industrial CAD • Training (military and otherwise) • Entertainment

4. Motivation:Model Size • Incredibly, models are getting bigger as fast as hardware is getting faster…

5. Big Models:Submarine Torpedo Room • 700,000 polygons Courtesy General Dynamics, Electric Boat Div.

6. Big Models:Coal-fired Power Plant • 13 million polygons (Anonymous)

7. Big Models:Plant Ecosystem Simulation • 16.7 million polygons (sort of) Deussen et al: Realistic Modeling of Plant Ecosystems

8. Big Models:Double Eagle Container Ship • 82 million polygons Courtesy Newport News Shipbuilding

9. Big Models:The Digital Michelangelo Project • David:56,230,343 polygons • St. Matthew: 372,422,615 polygons Courtesy Digital Michelangelo Project

10. Motivation: Level of Detail • Clearly, much of this geometry is redundant for a given view • The basic idea: simplify the model, reducing the level of detail used for: • Distant portions • Small portions • Otherwise unimportant portions

11. Traditional Level of DetailIn A Nutshell… • Create levels of detail (LODs) of objects: 249,924 polys 62,480 polys 7,809 polys 975 polys Courtesy Jon Cohen

12. Traditional Level of DetailIn A Nutshell… • Distant objects use coarser LODs:

13. The Big Question How should we evaluate and regulate the visual fidelity of our simplifications?

14. d1 d2 Regulating LOD • LOD is often controlled by distance… Courtesy Martin Reddy

15. Regulating LOD • …or by size

16. Measuring Fidelity • Fidelity of a simplification to the original model is often measured geometrically: METRO by Visual Computing Group, CNR-Pisa

17. Measuring Visual Fidelity • However… • The most important measure of fidelity is usually not geometric but perceptual: does the simplification look like the original? • Therefore: • We are developing a principled framework for LOD in interactive rendering, based on perceptual measures of visual fidelity

18. Perceptually Guided LOD • Several interesting offshoots: • Imperceptible simplification • How to guarantee simplification is undetectable? • Best-effort simplification • How best to spend a limited time/polygon budget? • Silhouette preservation • Silhouettes are important. How important? • Gaze-directed rendering • When can we exploit reduced visual acuity?

19. Related Work • Lots of excellent research on perceptually guided rendering • Bolin & Meyer (SIGGRAPH 98) • Ramasubramanian et al (SIGGRAPH 99) • But all this work has focused on realisticrendering algorithms (e.g., path tracing) • Different time frame! • Seconds or minutes versus milliseconds

20. Related Work • As a result, prior work has incorporated quite sophisticated perceptual metrics • Our goal: a simple, conservative perceptual metric fast enough to run thousands of times per frame

21. The Approach • The contrast sensitivity function or CSFmeasures perceptibility of visual stimuli • We test local simplification operations against a model of the CSF to determine whether they would be perceptible

22. Perception 101:The Contrast Sensitivity Function • Perceptual scientists have long used contrast gratings to measure limits of vision: • Bars of sinusoidally varying intensity • Can vary: • Contrast • Spatial frequency • Eccentricity • Velocity • Etc…

23. Perception 101: The Contrast Sensitivity Function • Contrast grating tests produce a contrast sensitivity function • Threshold contrastvs. spatial frequency • CSF predicts the minimum detectablestatic stimuli

24. Your Personal CSF Campbell-Robson Chart by Izumi Ohzawa

25. Contrast Sensitivity Function:An Empirical Model • The CSF is affected by many factors • Background illumination, adaptation, age, etc • Attentive focus • We chose to sidestep these issues by building an empirical model (lookup table) • User foveates on target, grating fades in • Measuers threshold contrast across different spatial frequencies, eccentricities

26. Contrast Sensitivity Function:Complex Waveforms • The perceptibility of a complex signal is determined by its harmonic components • If no frequency component of an image feature is visible, the feature is imperceptible and may be removed without visible effect • This is the key idea that will allow us to simplify the model • Next: need a framework for simplification

27. Framework: View-Dependent Simplification • We use view-dependent simplification for LOD management • Traditional LOD: create several discrete LODs in a preprocess, pick one at run time • Continuous LOD: create data structure in preprocess, extract desired LOD at run time • View-dependent LOD: extract most appropriate LOD for the given view

28. View-Dependent LOD: Examples • Show nearby portions of object at higher resolution than distant portions View from eyepoint Birds-eye view

29. View-Dependent LOD: Examples • Show silhouette regions of object at higher resolution than interior regions

30. View-Dependent LOD: Examples • Show more detail where the user is looking than in their peripheral vision: 34,321 triangles

31. View-Dependent LOD: Examples • Show more detail where the user is looking than in their peripheral vision: 11,726 triangles

32. View-Dependent LOD:Implementation • We use VDSlib, our public-domain library for view-dependent simplification • Briefly, VDSlib uses a big data structure called the vertex tree • Hierarchical clustering of model vertices • Updated each frame for current simplification

33. The Vertex Tree • Each vertex tree node represents: • A subset of model vertices • A representative vertex or proxy • Folding a node collapses its vertices to the proxy • Unfolding a node splits the proxy back into vertices

34. Vertex Tree Example 8 7 R 2 I II 10 6 9 3 10 A B C 3 1 1 2 7 4 5 6 8 9 4 5 Vertex tree Triangles in active list

35. Vertex Tree Example 8 7 R A 2 I II 10 6 9 3 10 A B C 3 1 1 2 7 4 5 6 8 9 4 5 Vertex tree Triangles in active list

36. Vertex Tree Example 8 R A I II 10 6 9 3 10 A B C 3 1 2 7 4 5 6 8 9 4 5 Vertex tree Triangles in active list

37. Vertex Tree Example 8 R A I II 10 6 9 3 10 A B C 3 B 1 2 7 4 5 6 8 9 4 5 Vertex tree Triangles in active list

38. Vertex Tree Example 8 R A I II 10 9 3 10 A B C 3 B 1 2 7 4 5 6 8 9 Vertex tree Triangles in active list

39. Vertex Tree Example 8 R A C I II 10 9 3 10 A B C 3 B 1 2 7 4 5 6 8 9 Vertex tree Triangles in active list

40. Vertex Tree Example R A C I II 10 3 10 A B C 3 B 1 2 7 4 5 6 8 9 Vertex tree Triangles in active list

41. Vertex Tree Example R A C II I II 10 3 10 A B C 3 B 1 2 7 4 5 6 8 9 Vertex tree Triangles in active list

42. Vertex Tree Example R A II I II 10 10 A B C 3 B 1 2 7 4 5 6 8 9 Vertex tree Triangles in active list

43. Vertex Tree Example R A I II I II 10 10 A B C 3 B 1 2 7 4 5 6 8 9 Vertex tree Triangles in active list

44. Vertex Tree Example R I II I II 10 A B C 3 B 1 2 7 4 5 6 8 9 Vertex tree Triangles in active list

45. Vertex Tree Example R I II I II R 10 A B C 3 B 1 2 7 4 5 6 8 9 Vertex tree Triangles in active list

46. Vertex Tree Example R I II R 10 A B C 3 1 2 7 4 5 6 8 9 Vertex tree Triangles in active list

47. Node folding is the fundamental simplification operation: Some triangles change shape upon folding Some triangles disappear completely The Vertex Tree:Tris and SubTris 8 7 8 Fold Node A A 2 10 10 6 6 9 9 3 3 UnfoldNode A 1 4 5 4 5

48. Perceptually Guided LOD: Key Contribution • Our key contribution: a way to evaluate the perceptibility of a fold operation • Equate the effect of the fold to a worst-case contrast grating • Find the worst-case contrast induced in the image • Find the worst-case spatial frequency

49. Perceptually Guided LOD: Key Contribution • Our key contribution: a way to evaluate the perceptibility of a fold operation • Equate the effect of the fold to a worst-case contrast grating • Find the worst-case contrast induced in the image Bounded by the maximum change in luminance! • Find the worst-case spatial frequency

50. Perceptually Guided LOD: Key Contribution • Our key contribution: a way to evaluate the perceptibility of a fold operation • Equate the effect of the fold to a worst-case contrast grating • Find the worst-case contrast induced in the image Bounded by the maximum change in luminance! • Find the worst-case spatial frequency Bounded by the minimum spatial frequency (in our case)