1 / 66

Stereopsis

Stereopsis. Thanks to Richard Szeliski and George Bebis for the use of some slides. Mark Twain at Pool Table", no date, UCR Museum of Photography. Woman getting eye exam during immigration procedure at Ellis Island, c. 1905 - 1920 , UCR Museum of Phography. P. Q. P’=Q’. O.

kpantoja
Download Presentation

Stereopsis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Stereopsis Thanks to Richard Szeliski and George Bebis for the use of some slides

  2. Mark Twain at Pool Table", no date, UCR Museum of Photography

  3. Woman getting eye exam during immigration procedure at Ellis Island, c. 1905 - 1920, UCR Museum of Phography

  4. P Q P’=Q’ O Why Stereo Vision? • 2D images project 3D points into 2D: • 3D Points on the same viewing line have the same 2D image: • 2D imaging results in depth information loss

  5. Stereo • Assumes (two) cameras. • Known positions. • Recover depth.

  6. Recovering Depth Information: P Q P’1 P’2=Q’2 Q’1 O2 O1 Depth can be recovered with two images and triangulation.

  7. Finding Correspondences: Q P P’1 P’2 Q’2 Q’1 O2 O1

  8. Finding Correspondences:

  9. Correspondence problem • Multiple hypotheses satisfy the epipolar constraint. Which one is correct?

  10. 3D Reconstruction P P’1 P’2 O2 O1 We must solve the correspondence problem first!

  11. epipolar line epipolar line epipolar plane Stereo correspondence • Determine Pixel Correspondence • Pairs of points that correspond to same scene point • Epipolar Constraint • Reduces correspondence problem to 1D search along conjugateepipolar lines (Seitz)

  12. Simplest Case • Image planes of cameras are parallel. • Focal points are at same height. • Focal lengths same. • Then, epipolar lines are horizontal scan lines.

  13. T Epipolar Geometryfor Parallel Cameras f f Or Ol el er P Epipoles are at infinite Epipolar lines are parallel to the baseline

  14. We can always achieve this geometry with image rectification • Image Reprojection • reproject image planes onto common plane parallel to line between optical centers • Notice, only focal point of camera really matters (Seitz)

  15. Stereo: epipolar geometry • for two images (or images with collinear camera centers), can find epipolar lines • epipolar lines are the projection of the pencil of planes passing through the centers • Rectification: warping the input images (perspective transformation) so that epipolar lines are horizontal

  16. Rectification • Project each image onto same plane, which is parallel to the epipole • Resample lines (and shear/stretch) to place lines in correspondence, and minimize distortion • [Zhang and Loop, MSR-TR-99-21]

  17. Example courtesy of Marc Pollefeys

  18. Epipolar Line Example courtesy of Marc Pollefeys

  19. Rectification

  20. Rectification

  21. Rectification Epipolar lines are colinear and parallel to baseline Epipolar lines are at arbitrary locations and orientations

  22. ul ur ul rectification ur Rectification (cont’d) • Rectification is a transformation which makes pairs of conjugate epipolar lines become collinear and parallel to the horizontal axis (i.e., baseline) • Searching for corresponding points becomes much simpler for the case of rectified images

  23. Rectification (cont’d) • Disparities between the images are in the x-direction only (i.e., no y disparity)

  24. Rectification: Example before rectification after rectification

  25. Rectification (cont’d) • Main steps (assuming knowledge of the extrinsic/intrinsic stereo parameters): (1) Rotate the left camera so that the epipolar lines become parallel to the horizontal axis (i.e., epipole is mapped to infinity).

  26. Rectification (cont’d) (2) Apply the same rotation to the right camera to recover the original geometry. (3) Align right camera with left camera using rotation R. (4)Adjust scale in both camera frames.

  27. Rectification (cont’d) • Consider step (1) only (i.e., other steps are easy): • Construct a coordinate system (e1, e2, e3) centered at Ol . • Aligning it with image plane coordinate system.

  28. Rectification (cont’d) (1.1) e1 is a unit vector along the vector T (baseline)

  29. Rectification (cont’d) (1.2) e2 must be perpendicular to e1 (i.e., cross product of e1 with the opticalaxis) z=[0,0,1]T

  30. Rectification (cont’d) (1.3) choose e3 as the cross product of e1 and e2

  31. Rectification (cont’d) The rotation matrix that maps the left epipole to infinity is the transformation that aligns (e1, e2, e3) with (i ,j, k):

  32. Let’s discuss reconstruction with this geometry before correspondence, because it’s much easier. blackboard P Z Disparity: xl xr f pl pr Ol Or T Then given Z, we can compute X and Y. T is the stereo baseline d measures the difference in retinal position between corresponding points

  33. Correspondence: What should we match? • Objects? • Edges? • Pixels? • Collections of pixels?

  34. Julesz: had huge impact because it showed that recognition not needed for stereo.

  35. Correspondence: Epipolar constraint.

  36. Correspondence Problem • Two classes of algorithms: • Correlation-based algorithms • Produce a DENSE set of correspondences • Feature-based algorithms • Produce a SPARSE set of correspondences

  37. Correspondence: Photometric constraint • Same world point has same intensity in both images. • Lambertian fronto-parallel • Issues: • Noise • Specularity • Foreshortening

  38. For each epipolar line For each pixel in the left image Improvement: match windows Using these constraints we can use matching for stereo • compare with every pixel on same epipolar line in right image • pick pixel with minimum match cost • This will never work, so:

  39. ? = g f Most popular Comparing Windows: For each window, match to closest window on epipolar line in other image.

  40. Minimize Sum of Squared Differences Maximize Cross correlation It is closely related to the SSD:

  41. Similarity Constraint Left Right scanline • Slide a window along the right scanline and compare its contents with the reference window in the left image • Matching cost: SSD or normalized correlation Matching cost disparity

  42. Similarity Constraint Left Right scanline SSD

  43. Similarity Constraint Left Right scanline Norm. corr

  44. W = 3 W = 20 Window size • Effect of window size Better results with adaptive window • T. Kanade and M. Okutomi,A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment,, Proc. International Conference on Robotics and Automation, 1991. • D. Scharstein and R. Szeliski. Stereo matching with nonlinear diffusion. International Journal of Computer Vision, 28(2):155-174, July 1998 (Seitz)

  45. Stereo results • Data from University of Tsukuba Scene Ground truth (Seitz)

  46. Results with window correlation Window-based matching (best window size) Ground truth (Seitz)

  47. Results with better method State of the art method Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, International Conference on Computer Vision, September 1999. Ground truth (Seitz)

  48. Ordering constraint • Usually, order of points in two images is same. • Is this always true?

More Related