1 / 37

Understanding the effect of lighting in images

Understanding the effect of lighting in images. Ronen Basri. Light field. Rays travel from sources to objects T here they are either absorbed or reflected Energy decreases with distance and number of bounces Camera captures the set of rays that travel through the focal center.

lorne
Download Presentation

Understanding the effect of lighting in images

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Understanding the effect of lighting in images Ronen Basri

  2. Light field • Rays travel from sources to objects • There they are either absorbed or reflected • Energy decreases with distance and number of bounces • Camera captures the set of rays that travel through the focal center

  3. Specular reflectance (mirror) • When a surface is smooth light reflects in the opposite direction of the surface normal

  4. Specular reflectance • When a surface is slightly rough the reflected light will fall off around the specular direction

  5. Diffuse reflectance • When the surface is very rough light may be reflected equally in all directions

  6. Diffuse reflectance • When the surface is very rough light may be reflected equally in all directions

  7. BRDF • Bidirectional Reflectance Distribution Function • Specifies for a unit of incominglight in a direction how much light will be reflectedin a direction

  8. Lambertian reflectance

  9. Lambert’s law

  10. Preliminaries • A surface is denoted • A point on is • The tangent plane is spanned by • The surface normal is given by

  11. Photometric stereo • Given several images of the a lambertianobject under varying lighting • Assuming single directional source

  12. Photometric stereo • We can solve for S if L is known (Woodham) • If L is unknown we can use SVD factorization (Hayakawa)

  13. Factorization • Use SVD to find a rank 3 approximation • Define • Factorization is not unique, since invertible To reduce ambiguity we impose integrability –up to generalized bas relief transformation (Belhumeur et al.)

  14. Integrability • Recall that • Given , we set • And solve for

  15. Shape from shading (SFS) • What if we only have one image? • Assuming that lighting is knownand uniform albedo • Every intensity determinesa circle of possible normals • There is only one unknown ()if uniform albedo is assume

  16. Shape from shading • We write • Therefore • We obtain • This is a first order, non-linear PDE (Horn)

  17. Shape from shading • Suppose , then • This is called an Eikonal equation

  18. Distance transform • The distance of each point to the boundary • Posed as an Eikonal equation

  19. Solution • Right hand side in SFSdetermines “speed” • Eikonal equation can be solved by a continuous analog of “shortest path” algorithm, called “fast marching” • The case is handled by change of variables (Kimmel & Sethian)

  20. Example

  21. Illumination cone • What is the set of images of an object under different lighting, with any number of sources? • Due to additivity, this set forms a convex cone in number of pixels (Belhumeur & Kriegman) = 0.5* +0.2* +0.3*

  22. Illumination cone • Cone characterization is generic, holds also with specularities, shadows and inter-reflections • Unfortunately, representing the cone is complicated (infinite degrees of freedom) • Cone is “thin” for Lambertian objects;indeed the illumination cone of many objects can be represented with few PCA vectors (Yuille et al.)

  23. Lambertian reflectance is smooth (Basri & Jacobs; Ramamoorthi & Hanrahan) lighting reflectance

  24. + + + Reflectance obtained with convolution

  25. + + + Reflectance obtained with convolution

  26. Positive values Negative values Spherical harmonics 1 Z X Y XY XZ YZ

  27. Harmonic approximation • Lighting, in terms of harmonics • Reflectance • Approximation accuracy, 99%(Basri & Jacobs; Ramamoorthi & Hanrahan)

  28. Harmonic transform of kernel

  29. Positive values Negative values ρ Albedo n Surface normal “Harmonic faces” r

  30. Image 1 Light 1 : : Image n Light n Photometric stereo (Basri, Jacobs &Kemelmacher) S L M SVD recovers LandS up to an ambiguity

  31. Photometric stereo

  32. Motion + lighting

  33. Motion + lighting (Basri & Frolova) • Given 2 images • Take ratio to eliminate albedo • If motion is small we can represent using a Taylor expansion around

  34. Small motion • We obtain a PDE that is quasi linear in • Wherewith • Can be solved with continuation (characteristics)

  35. Reconstruction

  36. More reconstructions

  37. Conclusion • Understanding the effect of lighting on images is challenging, but can lead to better interpretation of images • We surveyed several problems: • Photometric stereo • Shape from shading • Modeling multiple sources with spherical harmonics • Motion and lighting • We only looked at Lambertian objects…

More Related