Horizon Map Capture

1. Horizon Map Capture H. Rushmeier, L. Balmelli, F. Bernardini IBM TJ Watson Research Center

2. OVERVIEW I. Why Capture Horizon Maps? II. Capture & Processing Under Ideal Conditions III. Dealing with Real World Data IV. Some Example Maps

3. I. Why Capture Horizon Maps? General Problem: How do we capture and represent existing objects ?

4. I. Why Capture Horizon Maps? Image maps used to represent object detail: Textures: colors View Dependent Textures: changing colors Normals: relit details Surface Light Fields: all light from object geometry + map

5. I. Why Capture Horizon Maps?

6. I. Why Capture Horizon Maps? Goal: Add captured cast shadows to normals maps to represent more lighting effects Retain ability to edit Store data in a form that is fast to render

7. I. Why Capture Horizon Maps? Attached versus Cast Shadows:

8. I. Why Capture Horizon Maps? Efficient representation of cast shadows: Horizon Maps (Max ’88) hardware rendering (Sloan & Cohen, ’00, Kautz et al. ’00)

9. I. Why Capture Horizon Maps? Simple Hardware Set Up

10. I. Why Capture Horizon Maps? Use Photometric Stereo to Compute Normals Discard lightest and darkest values at each pixel, solve: Li dot N,p = Gi,p L = Light source direction, light i N = Surface normal at p Gi,p = Gray scale, image I, pixel p

11. I. Why Capture Horizon Maps? Why not reconstruct surface from normals? Discontinuities: Outliers: Results are by integration: Effect of one bad normal spreads across image

12. I. Why Capture Horizon Maps? Why not reconstruct surface from normals? Error in L dot N smaller than error in height.

13. I. Why Capture Horizon Maps? Why not reconstruct surface from normals?

14. I. Why Capture Horizon Maps? shadows from reconstructed heights captured

15. II. Capture and Processing Under Ideal Conditions 1. Identify regions of cast shadow

16. II. Capture and Processing Under Ideal Conditions 2. Identify height of ridge casting the shadow

17. II. Capture and Processing Under Ideal Conditions Ridge height exact for sharp bumps only

18. II. Capture and Processing Under Ideal Conditions Differentiate between bumps and grooves

19. II. Capture and Processing Under Ideal Conditions 3. Compute horizon map for each pixel by marching in light direction for each pixel until ridge encountered Compute angle q ridge height h known find distance from pixel to ridge in

20. III. Dealing With Real World Data Correcting for Finite Light Locations

21. III. Dealing With Real World Data Shadows aren’t really black – analyze histograms No spike at zero “shadow”

22. III. Dealing With Real World Data Computing ridge locations: Identifying types of shadow edges Jaggy edge can cause pixel parallel to be misclassified as the “end” of the shadow

23. III. Dealing With Real World Data Computing Ridge locations:Combining, smoothing data

24. IV. Some Example Maps

25. IV. Some Example Maps captured corrected for finite light

26. IV. Some Example Maps cast shadows relit normals

27. IV. Some Example Maps horizon map reconstructed image

28. IV. Some Example Maps

29. IV. Some Example Maps

30. IV. Some Example Maps

31. IV. Some Example Maps

32. IV. Some Example Maps Editing layers horizon map colors normals ridges Except for horizon maps, no pixel contains lighting information from other pixels. New horizon maps can be generated from edited ridges

33. IV. Some Example Maps copy delete cyan blue yellow

34. IV. Some Example Maps copy delete cyan blue yellow

35. IV. Some Example Maps