A Perceptual Heuristic for Shadow Computation in Photo-Realistic Images

2. A Perceptual Heuristic for Shadow Computation in Photo-Realistic Images Wednesday, 2 August 2006 Peter Vangorp Olivier Dumont Toon Lenaerts Philip Dutré Katholieke Universiteit Leuven {peter.vangorp,olivier.dumont,toon.lenaerts,philip.dutre}@cs.kuleuven.be

3. Introduction 3 types of realistic rendering

4. Introduction • 3 types of realistic rendering • radiometric accuracy“render everything a photometer can detect”

5. Introduction • 3 types of realistic rendering • radiometric accuracy“render everything a photometer can detect” • physiological perception“render only what the eye can see”

6. Introduction • 3 types of realistic rendering • radiometric accuracy“render everything a photometer can detect” • physiological perception“render only what the eye can see” • psychological perception“render only what the brain can see”

7. Radiometric accuracy measure shapes, light sources, materials, ... photometer Cornell box [Meyer et al. 1986]

8. Physiological perception use low-level limitations of human visual systemthreshold vs intensity, contrast sensitivity, ... reference threshold map visibly indistinguishable adaptive rendering [Ramasubramanian et al. 1999]

9. Physiological perception use low-level limitations of human visual systemthreshold vs intensity, contrast sensitivity, ... reference threshold map visibly indistinguishable adaptive rendering [Ramasubramanian et al. 1999]

10. Psychological perception use higher-level heuristics “Is a shadow necessary for therealism of a scene?” [Thompson et al. 1998]

11. Psychological perception • use higher-level heuristics “Is a shadow necessary for therealism of a scene?” “Do we need highlights to convey material properties?” [Thompson et al. 1998] [Fleming et al. 2004]

12. Psychological perception • use higher-level heuristics “Is a shadow necessary for therealism of a scene?” “Do we need highlights to convey material properties?” “How detailed should the geometry be?” [Thompson et al. 1998] [Fleming et al. 2004] [Luebke 2001]

13. Motivation & Goals Motivation shadows are important for perception of realism Goal detect perceptually important shadows in the scene render important shadows accurately approximate unimportant shadows [Kersten et al. 1997]

14. Methodology Psycho-physical experiments Derive a heuristic predicting shadow importance Design a perceptually driven algorithm Experimental validation

15. Methodology • Psycho-physical experiments • Derive a heuristic predicting shadow importance • Design a perceptually driven algorithm • Experimental validation

16. Methodology • Psycho-physical experiments • Derive a heuristic predicting shadow importance • Design a perceptually driven algorithm • Experimental validation

17. Methodology • Psycho-physical experiments • Derive a heuristic predicting shadow importance • Design a perceptually driven algorithm • Experimental validation

18. 1. Perceptual experiments Test setup: 162 images, varying sphere over 9 radii and 9 heights correct shadow: no shadow: (avg. illumination)

19. 1. Perceptual experiments • “Does the lighting in this image look realistic?” • single stimulus • 5000+ decisions • avg. 2 sec / decision

20. 2. Heuristic large difference in realism, e.g. difference in realism small difference in realism, e.g.  sphere height  sphere radius

21. 3. A perceptually driven algorithm • Ray tracing

22. 3. A perceptually driven algorithm • Ray tracing • shoot viewing ray

23. distance solid angle 3. A perceptually driven algorithm • Ray tracing • shoot viewing ray • evaluate heuristic in hit point to be shaded in function of distance and solid angle

24. 3. A perceptually driven algorithm • Preprocessing step • shadow photon map [Jensen and Christensen 1995]

25. 3. A perceptually driven algorithm • Preprocessing step • shadow photon map • shadow photons augmented with heuristic [Jensen and Christensen 1995] shadow photon map

26. 3. A perceptually driven algorithm For each viewing ray Gather nearest shadow photons Calculate average perceptual value yes no avg < threshold approximate: photon map render accurately: shadow rays

27. 3. A perceptually driven algorithm For each viewing ray Gather nearest shadow photons Calculate average perceptual value yes no avg < threshold approximate: photon map render accurately: shadow rays

28. 3. A perceptually driven algorithm For each viewing ray Gather nearest shadow photons Calculate average perceptual value yes no avg < threshold approximate: photon map render accurately: shadow rays

29. 3. A perceptually driven algorithm For each viewing ray Gather nearest shadow photons user-defined threshold Calculate average perceptual value yes no avg < threshold approximate: photon map render accurately: shadow rays

30. 3. A perceptually driven algorithm For each viewing ray Gather nearest shadow photons user-defined threshold Calculate average perceptual value yes no avg < threshold approximate: photon map render accurately: shadow rays

31. 3. A perceptually driven algorithm For each viewing ray Gather nearest shadow photons user-defined threshold Calculate average perceptual value yes no avg < threshold approximate: photon map render accurately: shadow rays

32. 3. A perceptually driven algorithm

33. 4. Validation Similar perceptual experiment “Do the lighting and the shadows look realistic?”

34. 4. Validation • Similar perceptual experiment • “Do the lighting and the shadows look realistic?” • Stimuli: 6 scenes • threshold 25%, 50%, 75% • reference rendering (threshold 0%)

35. 4. Validation • Similar perceptual experiment • “Do the lighting and the shadows look realistic?” • Stimuli: 6 scenes • threshold 25%, 50%, 75% • reference rendering (threshold 0%) • 15 subjects, almost 6000 decisions, avg. 5 seconds

36. 4. Validation Up to 50% of the pixels can be approximated, without loss of perceptual realism Threshold 25% Approx px 13% Threshold 50% Approx px 24% Threshold 75% Approx px 48%

37. Results threshold 90% reference

38. Results threshold 90% reference

39. Results threshold 90% reference

40. Results threshold 90% reference

41. Results threshold 80% reference

42. Results threshold 80% reference

43. Results threshold 80% reference

44. Results threshold 80% reference

45. Results threshold 90% reference

46. Results threshold 90% reference

47. Results threshold 90% reference

48. Conclusions & Future Work Intuitions confirmed by statistical data Rendering algorithm driven by perceptual information

49. Conclusions & Future Work • Intuitions confirmed by statistical data • Rendering algorithm driven by perceptual information • Extend methodology to other phenomena • Different questions than “Does this look realistic?” • Better ways to incorporate perceptual information • currently no significant acceleration yet

50. Questions?