1 / 17

3D Vision

Explore how to extract 3D information from 2D images using various cues, labeling image edges, pixel contents, and shape-from methods. Learn about reflections, geometry, photometric stereo, and shape reconstructions.

garyclark
Download Presentation

3D Vision

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3D Vision • Why? • The world is 3D • Not all useful information is readily available in 2D • Why so hard? • “Inverse problem”: one image = many scenes • Complex relationship between objects & pixels • Noise, occlusion, etc. • What can we do? • Use "hints" from our knowledge of the world • Add more information to the problem!

  2. Labeling Image Edges • Many edge labels carry 3D information • Occluding blade (>) • Occluding surface to right along arrow • Convex crease (+) • Edge is closer than both surfaces • Concave crease (–) • Edge is further than both surfaces • Limb (>>) • Edge is "horizon" of curving-away surface • Others are about reflectance or illumination changes • Mark (M) • Change due to paint or material boundary • Illumination Boundary (S) • Shadow edge

  3. Intrinsic Image Pixel Contents • Depth (range) • Orientation (surface normal) • Illumination • Albedo (reflectance) • Given this information, the picture (intensity values) can be completely reconstructed

  4. 3D Cues in Single 2D Images • Occlusion • Occluding objects are closer to the camera • The crossbar of a T-junction belongs to the occluding object • Perspective scaling and foreshortening • If two copies of the same object appear in a picture, the smaller one is further away. • Scaling is parallel to the image plane • Foreshortening is perpendicular to the image plane • Texture Gradient • Since texture is composed of repeated patterns, changes in size and density of texture convey depth cues

  5. "Shape-From" Methods • Use a cue (e.g. texture, shading) from a small region of an image • Cues generally give partial surface orientation information • E.g. degree of tilt • Related cues can give "boundary conditions" to start from • Solve for continuous surfaces that satisfy both the general and boundary constraints

  6. Example: Shape From Shading Figure 12.2

  7. Gradient Space Represents Surface Normal

  8. Reflectance Geometry • Three directions are important: normal to the surface, surface to light source, and surface to camera

  9. Types of Reflection (Review) • Specular reflection (mirror) • Color depends on light source color • Limited scattering: angle of incidence = angle of reflection • Camera sees light if it’s pointed in the right direction • Nearly all light is reflected • Lambertian reflection (matte) • Color depends on material properties of object • Light evenly scattered throughout half-space • Camera sees light if surface is visible • Amount of light reflected is proportional to angle between surface and light source

  10. Shape From Shading • Lambertian surface, known light source direction • Reflective component (highlights) can be subtracted out in preprocessing • Relative brightness of surface patches constrain their directions • Darker patches are more tilted away • A given brightness value represents a circle in gradient space • Boundary pixels indicate surface at 90 degrees from normal (if smooth surface) • Solve an optimization problem: brightness term and smoothness term

  11. Shape From Shading (cont’d) • Brightness term • Intensity is a function of reflectance, and reflectance is a function of surface normals (p,q) and light source direction (vx, vy, vz) • Smoothness term • Try to minimize integral of partial derivatives of p and q in x and y direction

  12. Photometric Stereo • Extension to multiple "images" • Lambertian surface, several light sources • Each image has one light source, constrains surfaces • Solve an overdetermined linear (matrix) system - like camera calibration with extra points • Implementation • Surround your object with a frame containing inward-pointing lights • Take an image with each light in turn • Use images and known light directions to solve the equations.

  13. More Variations • SHINY • Photometric stereo system for highly reflective materials • Used to accurately characterize welds • Accurate color determination (plastic objects) • Separate highlights from matte portion • Determine illumination color from highlight • Determine object color • Create “matte object” for photometric stereo or shape from shading

  14. Shape From Texture Figure 12.3

  15. Shape from Texture • Transformation of original texture related to surface normal • Solve for affine transformation between original texel and viewed texel • Transformation depends on surface normal & distance • (Assume camera is far enough to avoid worst perspective distortion) • If the original texel is known, transformations can be computed directly • If the original texel is unknown, assume the largest visible texel is directly facing the camera • Use smoothness or shape constraints to eliminate alternatives

  16. Shape from Focus • Vary the focal length of the camera (i.e. zoom lens) • Objects at different distances will become clear at different focal lengths • Can use comparisons between pairs of images to get relative distances

  17. Choice Depends on Object • Solid, matte object (or matte separated from specular) • Shape from shading, photometric stereo • Highly reflective object • Shape from specular reflection • Regular textured object • Shape from texture, stereo, focus • Irregular textured object • Stereo, focus

More Related