1 / 45

MAN-522 Computer Vision

MAN-522 Computer Vision. Spring 2016-17. Given a single image of an object with known surface reflectance taken under a known light source, can we recover the shape of the object?. Shape from shading (Photometric stereo). Definition 1:

donnette
Download Presentation

MAN-522 Computer Vision

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MAN-522Computer Vision Spring 2016-17

  2. Given a single image of an object with known surface reflectance taken under a known light source, can we recover the shape of the object? Shape from shading (Photometric stereo)

  3. Definition 1: Irradiance is the radiant flux (power) received by a surface per unit area. The SI unit of irradiance is the watt per square metre (W/m2). Irradiance is often called "intensity" in branches of physics other than radiometry, but in radiometry this usage leads to confusion with radiant intensity. Definition 2: In radiometry, radiance is the radiant flux emitted, reflected, transmitted or received by a surface, per unit solid angle per unit projected area,

  4. Mechanisms of Reflection source incident direction surface reflection body reflection surface • Surface Reflection: • Specular Reflection • Glossy Appearance • Highlights • Dominant for Metals • Body Reflection: • Diffuse Reflection • Matte Appearance • Non-Homogeneous Medium • Clay, paper, etc Image Intensity = Body Reflection + Surface Reflection

  5. Diffuse Reflection and Lambertian BRDF source intensity I incident direction normal viewing direction surface element • Surface appears equally bright from ALL directions! (independent of ) albedo • Lambertian BRDF is simply a constant : • Surface Radiance : source intensity • Commonly used in Vision and Graphics!

  6. Diffuse and Specular Reflection diffuse specular diffuse+specular

  7. Image Intensity and 3D Geometry • Shading as a cue for shape reconstruction • What is the relation between intensity and shape? • Reflectance Map

  8. Equation of plane or Let Surface normal Surface Normal surface normal

  9. Normal vector Source vector plane is called the Gradient Space (pq plane) • Every point on it corresponds to a particular surface orientation Gradient Space

  10. : source brightness : surface albedo (reflectance) : constant (optical system) Image irradiance: Let then Reflectance Map • Relates image irradiance I(x,y) to surface orientation (p,q) for given source direction and surface reflectance • Lambertian case:

  11. Reflectance Map (Lambertian) Iso-brightness contour cone of constant Reflectance Map • Lambertian case

  12. Reflectance Map iso-brightness contour • Lambertian case Note: is maximum when

  13. Diffuse peak Specular peak Reflectance Map • Glossy surfaces (Torrance-Sparrow reflectance model) diffuse term specular term

  14. Shape from a Single Image? • Given a single image of an object with known surface reflectance taken under a known light source, can we recover the shape of the object? • Given R(p,q) ( (pS,qS) and surface reflectance) can we determine (p,q) uniquely for each image point? NO

  15. Solution • Take more images • Photometric stereo • Add more constraints • Shape-from-shading

  16. Photometric Stereo

  17. Photometric Stereo Lambertian case: Image irradiance: • We can write this in matrix form:

  18. inverse Solving the Equations

  19. Least squares solution: Moore-Penrose pseudo inverse • Solve for as before More than Three Light Sources • Get better results by using more lights

  20. Color Images • The case of RGB images • get three sets of equations, one per color channel: • Simple solution: first solve for using one channel • Then substitute known into above equations to get • Or combine three channels and solve for

  21. Computing light source directions • Trick: place a chrome sphere in the scene • the location of the highlight tells you the source direction

  22. Specular Reflection - Recap • For a perfect mirror, light is reflected about N • We see a highlight when • Then is given as follows:

  23. Computing the Light Source Direction Chrome sphere that has a highlight at position h in the image • Can compute N by studying this figure • Hints: • use this equation: • can measure c, h, and r in the image N h H rN c C sphere in 3D image plane

  24. V2 V1 N Depth from Normals • Get a similar equation for V2 • Each normal gives us two linear constraints on z • compute z values by solving a matrix equation

  25. Limitations • Big problems • Doesn’t work for shiny things, semi-translucent things • Shadows, inter-reflections • Smaller problems • Camera and lights have to be distant • Calibration requirements • measure light source directions, intensities • camera response function

  26. Trick for Handling Shadows • Weight each equation by the pixel brightness: • Gives weighted least-squares matrix equation: • Solve for as before

  27. Original Images

  28. Results - Shape Shallow reconstruction (effect of interreflections) Accurate reconstruction (after removing interreflections)

  29. Results - Albedo No Shading Information

  30. Original Images

  31. Results - Shape

  32. Results - Albedo

  33. Results • Estimate light source directions • Compute surface normals • Compute albedo values • Estimate depth from surface normals • Relight the object (with original texture and uniform albedo)

  34. Shape from a Single Image? • Given a single image of an object with known surface reflectance taken under a known light source, can we recover the shape of the object? • Given R(p,q) ( (pS,qS) and surface reflectance) can we determine (p,q) uniquely for each image point? NO

  35. (f,g)-space Problem (p,q) can be infinite when Redefine reflectance map as Stereographic Projection (p,q)-space (gradient space)

  36. and are known The values on the occluding boundary can be used as the boundary condition for shape-from-shading Occluding Boundaries

  37. Minimize Image Irradiance Constraint • Image irradiance should match the reflectance map (minimize errors in image irradiance in the image)

  38. : surface orientation under stereographic projection Smoothness Constraint • Used to constrain shape-from-shading • Relates orientations (f,g) of neighboring surface points Minimize (penalize rapid changes in surface orientation f and g over the image)

  39. Minimize Shape-from-Shading • Find surface orientations (f,g) at all image points that minimize weight smoothness constraint image irradiance error

  40. Of course you can consider more neighbors (smoother results) Find and that minimize Numerical Shape-from-Shading • Smoothness error at image point (i,j) • Image irradiance error at image point (i,j) (Ikeuchi & Horn 89)

  41. Find and that minimize If and minimize , then where and are 4-neighbors average around image point (k,l) Numerical Shape-from-Shading (Ikeuchi & Horn 89)

  42. Update rule Numerical Shape-from-Shading • Use known values on the occluding boundary to constrain the solution (boundary conditions) • Compare with for convergence test • As the solution converges, increase to remove the smoothness constraint (Ikeuchi & Horn 89)

  43. Euler equations for F Euler equations for shape-from-shading Solve this coupled pair of second-order partial differential equations with the occluding boundary conditions! Calculus of Variations Minimize (read Horn A.6)

  44. Results

  45. Results

More Related