1 / 42

Shape-from-X Class 11

Shape-from-X Class 11. Some slides from Shree Nayar . others. 3D photography course schedule (tentative). Shape-from- X. X =Shading X =Multiple Light Sources (photometric stereo) X =Texture X =Focus/Defocus X = Specularities X =Shadows X =…. Shape from shading.

tasha-richard
Download Presentation

Shape-from-X Class 11

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Shape-from-XClass 11 Some slides from Shree Nayar. others

  2. 3D photography course schedule(tentative)

  3. Shape-from-X • X =Shading • X =Multiple Light Sources (photometric stereo) • X =Texture • X =Focus/Defocus • X =Specularities • X =Shadows • X =…

  4. Shape from shading • Shading as a cue for shape reconstruction • What is the relation between intensity and shape? • Reflectance Map

  5. : source brightness : surface albedo (reflectance) : constant (optical system) Image irradiance: Let then Reflectance Map • Relates image irradiance I(x,y) to surface orientation (p,q) for given source direction and surface reflectance • Lambertian case:

  6. Reflectance Map (Lambertian) Iso-brightness contour cone of constant Reflectance Map • Lambertian case

  7. Reflectance Map iso-brightness contour • Lambertian case Note: is maximum when

  8. Shape from a Single Image? • Given a single image of an object with known surface reflectance taken under a known light source, can we recover the shape of the object? • Given R(p,q) ( (pS,qS) and surface reflectance) can we determine (p,q) uniquely for each image point? NO

  9. Solution • Add more constraints • Shape-from-shading • Take more images • Photometric stereo

  10. (f,g)-space Problem (p,q) can be infinite when Redefine reflectance map as Stereographic Projection (p,q)-space (gradient space)

  11. and are known The values on the occluding boundary can be used as the boundary condition for shape-from-shading Occluding Boundaries

  12. Minimize Image Irradiance Constraint • Image irradiance should match the reflectance map (minimize errors in image irradiance in the image)

  13. : surface orientation under stereographic projection Smoothness Constraint • Used to constrain shape-from-shading • Relates orientations (f,g) of neighboring surface points Minimize (penalize rapid changes in surface orientation f and g over the image)

  14. Minimize Shape-from-Shading weight • Find surface orientations (f,g) at all image points that minimize smoothness constraint image irradiance error

  15. Results

  16. Results

  17. Solution • Add more constraints • Shape-from-shading • Take more images • Photometric stereo

  18. Photometric Stereo

  19. Photometric Stereo : source brightness : surface albedo (reflectance) : constant (optical system) Lambertian case: Image irradiance: • We can write this in matrix form:

  20. inverse Solving the Equations

  21. Least squares solution: Moore-Penrose pseudo inverse • Solve for as before More than Three Light Sources • Get better results by using more lights

  22. Color Images • The case of RGB images • get three sets of equations, one per color channel: • Simple solution: first solve for using one channel • Then substitute known into above equations to get • Or combine three channels and solve for

  23. Computing light source directions • Trick: place a chrome sphere in the scene • the location of the highlight tells you the source direction

  24. Specular Reflection - Recap • For a perfect mirror, light is reflected about N • We see a highlight when • Then is given as follows:

  25. Computing the Light Source Direction Chrome sphere that has a highlight at position h in the image N • Can compute N by studying this figure • Hints: • use this equation: • can measure c, h, and r in the image h H rN c C sphere in 3D image plane

  26. V2 V1 N Depth from Normals • Get a similar equation for V2 • Each normal gives us two linear constraints on z • compute z values by solving a matrix equation

  27. Limitations • Big problems • Doesn’t work for shiny things, semi-translucent things • Shadows, inter-reflections • Smaller problems • Camera and lights have to be distant • Calibration requirements • measure light source directions, intensities • camera response function

  28. Trick for Handling Shadows • Weight each equation by the pixel brightness: • Gives weighted least-squares matrix equation: • Solve for as before

  29. Original Images

  30. Results - Shape Shallow reconstruction (effect of interreflections) Accurate reconstruction (after removing interreflections)

  31. Results - Albedo No Shading Information

  32. Original Images

  33. Results - Shape

  34. Results - Albedo

  35. Results • Estimate light source directions • Compute surface normals • Compute albedo values • Estimate depth from surface normals • Relight the object (with original texture and uniform albedo)

  36. Shape from texture • Obtain normals from texture element (or statistics) deformations, Examples from Angie Loh

  37. Shape from texture • Obtain normals from texture element (or statistics) deformations, Examples from Angie Loh

  38. Depth from focus • Sweep through focus settings • “most sharp” pixels correspond to depth • (most high frequencies)

  39. Depth from Defocus • More complicates, but needs less images • Compare relative sharpness between images

  40. Shape from shadows S. Savarese, M. Andreetto, H. Rusmeier, F. Bernardini, P. Perona, “3D Reconstruction by Shadow Carving: Theory and Practical Evaluation”, International Journal of Computer Vision (IJCV) , vol 71, no. 3, pp. 305-336, March 2007.

  41. Shape from specularities Toward a Theory of Shape from Specular Flow Y. Adato, Y. Vasilyev, O. Ben-Shahar, T. Zickler Proc. ICCV’07

  42. Next class: 3d modeling and registration

More Related