1 / 63

CREDITS

CREDITS. Rasmussen, UBC (Jim Little), Seitz (U. of Wash.), Camps (Penn. State), UC, UMD (Jacobs), UNC, CUNY. 3D World Points. Camera Centers. Camera Orientations. Multi-View Geometry. Relates. 3D World Points. Camera Centers. Camera Intrinsic Parameters. Image Points.

kenyon-green
Download Presentation

CREDITS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CREDITS Rasmussen, UBC (Jim Little), Seitz (U. of Wash.), Camps (Penn. State), UC, UMD (Jacobs), UNC, CUNY Computer Vision : CISC 4/689

  2. 3D World Points • Camera Centers • Camera Orientations Multi-View Geometry Relates Computer Vision : CISC 4/689

  3. 3D World Points • Camera Centers • Camera Intrinsic Parameters • Image Points Multi-View Geometry Relates • Camera Orientations Computer Vision : CISC 4/689

  4. Stereo scene point image plane optical center Computer Vision : CISC 4/689

  5. Stereo • Basic Principle: Triangulation • Gives reconstruction as intersection of two rays • Requires • calibration • point correspondence Computer Vision : CISC 4/689

  6. Stereo Constraints p’ ? p Given p in left image, where can the corresponding point p’in right image be? Computer Vision : CISC 4/689

  7. Epipolar Line p’ Y2 X2 Z2 O2 Epipole Stereo Constraints M Image plane Y1 p O1 Z1 X1 Focal plane Computer Vision : CISC 4/689

  8. Stereo • The geometric information that relates two different viewpoints of the same scene is entirely contained in a mathematical construct known as fundamental matrix. • The geometry of two different images of the same scene is called the epipolar geometry. Computer Vision : CISC 4/689

  9. Stereo/Two-View Geometry • The relationship of two views of a scene taken from different camera positions to one another • Interpretations • “Stereo vision” generally means two synchronized cameras or eyes capturing images • Could also be two sequential views from the same camera in motion • Assuming a static scene http://www-sop.inria.fr/robotvis/personnel/sbougnou/Meta3DViewer/EpipolarGeo Computer Vision : CISC 4/689

  10. 3D from two-views There are two ways of extracting 3D from a pair of images. • Classical method, called Calibrated route, we need to calibrate both cameras (or viewpoints) w.r.t some world coordinate system. i.e, calculate the so-called epipolar geometry by extracting the essential matrix of the system. • Second method, called uncalibrated route, a quantity known as fundamental matrix is calculated from image correspondences, and this is then used to determine the 3D. Either way, principle of binocular vision is triangulation. Given a single image, the 3D location of any visible object point must lie on the straight line that passes through COP and image point (see fig.). Intersection of two such lines from two views is triangulation. Computer Vision : CISC 4/689

  11. Mapping Points between Images • What is the relationship between the images x, x’ of the scene point X in two views? • Intuitively, it depends on: • The rigid transformation between cameras (derivable from the camera matrices P, P’) • The scene structure (i.e., the depth of X) • Parallax: Closer points appear to move more Computer Vision : CISC 4/689

  12. x3 x’3 x’2 x2 x1 x’1 Example: Two-View Geometry courtesy of F. Dellaert Is there a transformation relating the points xi tox’i ? Computer Vision : CISC 4/689

  13. Epipolar Geometry • Baseline: Line joining camera centers C, C’ • Epipolar plane ¦: Defined by baseline and scene point X Computer Vision : CISC 4/689 baseline from Hartley & Zisserman

  14. Epipolar Lines • Epipolar lines l, l’: Intersection of epipolar plane ¦ with image planes • Epipoles e, e’: Where baseline intersects image planes • Equivalently, the image in one view of the other camera center. C’ C Computer Vision : CISC 4/689 from Hartley & Zisserman

  15. Epipolar Pencil • As position of X varies, epipolar planes “rotate” about the baseline (like a book with pages) • This set of planes is called the epipolar pencil • Epipolar lines “radiate” from epipole—this is the pencil of epipolar lines Computer Vision : CISC 4/689 from Hartley & Zisserman

  16. Epipolar Constraint • Camera center C and image point define ray in 3-D space that projects to epipolar line l’ in other view (since it’s on the epipolar plane) • 3-D point X is on this ray, so image of X in other view x’ must be on l’ • In other words, the epipolar geometry defines a mapping x !l’, of points in one image to lines in the other x’ C’ C Computer Vision : CISC 4/689 from Hartley & Zisserman

  17. Example: Epipolar Lines for Converging Cameras Left view Right view from Hartley & Zisserman Intersection of epipolar lines = Epipole ! Indicates direction of other camera Computer Vision : CISC 4/689

  18. Special Case: Translation Parallel to Image Plane Note that epipolar lines are parallel and corresponding points lie on correspond- ing epipolar lines (the latter is true for all kinds of camera motions) Computer Vision : CISC 4/689

  19. P p p’ O’ O From Geometry to Algebra Computer Vision : CISC 4/689

  20. P p p’ O’ O From Geometry to Algebra Computer Vision : CISC 4/689

  21. Rotation arrow should be at the other end, to get p’ in p coordinates Linear Constraint:Should be able to express as matrix multiplication. Computer Vision : CISC 4/689

  22. Review: Matrix Form of Cross Product Computer Vision : CISC 4/689

  23. Review: Matrix Form of Cross Product Computer Vision : CISC 4/689

  24. Matrix Form Computer Vision : CISC 4/689

  25. The Essential Matrix If un-calibrated, p gets multiplied by Intrisic matrix, K Computer Vision : CISC 4/689

  26. line point The Fundamental Matrix F • Mapping of point in one image to epipolar line in other image x !l’ is expressed algebraically by the fundamental matrixF • Write this as l’ = Fx • Since x’ is on l’, by the point-on-line definition we know that x’T l’ = 0 • Substitute l’ = Fx, we can thus relate corresponding points in the camera pair (P, P’) to each other with the following: x’T Fx = 0 Computer Vision : CISC 4/689

  27. The Fundamental Matrix F • F is 3 x 3, rank 2 (not invertible, in contrast to homographies) • 7 DOF (homogeneity and rank constraint take away 2 DOF) • The fundamental matrix of (P’, P) is the transpose FT NOW, can get implicit equation for any x, which is epipolar line) x’ Computer Vision : CISC 4/689 from Hartley & Zisserman

  28. Computing Fundamental Matrix (u’ is same as x in the prev. slide, u’ is same as x) Fundamental Matrix is singular with rank 2 In principal F has 7 parameters up to scale and can be estimatedfrom 7 point correspondences Direct Simpler Method requires 8 correspondences Computer Vision : CISC 4/689

  29. Estimating Fundamental Matrix The 8-point algorithm Each point correspondence can be expressed as a linear equation Computer Vision : CISC 4/689

  30. The 8-point Algorithm Lot of squares, so numbers have varied range, from say 1000 to 1. So pre-normalize. And RANSaC! Computer Vision : CISC 4/689

  31. Computing F: The Eight-point Algorithm • Input: n point correspondences ( n >= 8) • Construct homogeneous system Ax= 0 from • x = (f11,f12, ,f13, f21,f22,f23 f31,f32, f33) : entries in F • Each correspondence gives one equation • A is a nx9 matrix (in homogenous format) • Obtain estimate F^ by SVD of A • x (up to a scale) is column of V corresponding to the least singular value • Enforce singularity constraint: since Rank (F) = 2 • Compute SVD of F^ • Set the smallest singular value to 0: D -> D’ • Correct estimate of F : • Output: the estimate of the fundamental matrix, F’ • Similarly we can compute E given intrinsic parameters Computer Vision : CISC 4/689

  32. el lies on all the epipolar lines of the left image P Pl Pr True For every pr Epipolar Plane F is not identically zero Epipolar Lines p p r l Ol el er Or Epipoles Locating the Epipoles from F • Input: Fundamental Matrix F • Find the SVD of F • The epipole el is the column of V corresponding to the null singular value (as shown above) • The epipole er is the column of U corresponding to the null singular value • Output: Epipole el and er Computer Vision : CISC 4/689

  33. Special Case: Translation along Optical Axis • Epipoles coincide at focus of expansion • Not the same (in general) as vanishing point of scene lines Computer Vision : CISC 4/689 from Hartley & Zisserman

  34. Finding Correspondences • Epipolar geometry limits where feature in one image can be in the other image • Only have to search along a line Computer Vision : CISC 4/689

  35. Simplest Case • Image planes of cameras are parallel. • Focal points are at same height. • Focal lengths same. • Then, epipolar lines are horizontal scan lines. Computer Vision : CISC 4/689

  36. We can always achieve this geometry with image rectification • Image Reprojection • reproject image planes onto common plane parallel to line between optical centers • Notice, only focal point of camera really matters Computer Vision : CISC 4/689 (Seitz)

  37. P Pl Pr p’ r p’ l Y’l Y’r Z’r Z’l X’l T X’r Ol Or Stereo Rectification • Stereo System with Parallel Optical Axes • Epipoles are at infinity • Horizontal epipolar lines • Rectification • Given a stereo pair, the intrinsic and extrinsic parameters, find the image transformation to achieve a stereo system of horizontal epipolar lines • A simple algorithm: Assuming calibrated stereo cameras Computer Vision : CISC 4/689

  38. P Pl Pr Yr p p r l Yl Xl Zl Zr X’l T Ol Or R, T Xr Xl’ = T_axis, Yl’ = Xl’xZl, Z’l = Xl’xYl’ Stereo Rectification • Algorithm • Rotate both left and right camera so that they share the same X axis : Or-Ol = T • Define a rotation matrix Rrect for the left camera • Rotation Matrix for the right camera is RrectRT • Rotation can be implemented by image transformation Computer Vision : CISC 4/689

  39. P Pl Pr Yr p p r l Yl Xl Zl Zr X’l T Ol Or R, T Xr Xl’ = T_axis, Yl’ = Xl’xZl, Z’l = Xl’xYl’ Stereo Rectification • Algorithm • Rotate both left and right camera so that they share the same X axis : Or-Ol = T • Define a rotation matrix Rrect for the left camera • Rotation Matrix for the right camera is RrectRT • Rotation can be implemented by image transformation Computer Vision : CISC 4/689

  40. P Pl Pr p’ r p’ l Y’l Y’r Zr Z’l X’l T X’r Ol Or R, T T’ = (B, 0, 0), Stereo Rectification • Algorithm • Rotate both left and right camera so that they share the same X axis : Or-Ol = T • Define a rotation matrix Rrect for the left camera • Rotation Matrix for the right camera is RrectRT • Rotation can be implemented by image transformation Computer Vision : CISC 4/689

  41. Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923 Computer Vision : CISC 4/689

  42. Teesta suspension bridge-Darjeeling, India Computer Vision : CISC 4/689

  43. Mark Twain at Pool Table", no date, UCR Museum of Photography Computer Vision : CISC 4/689

  44. Woman getting eye exam during immigration procedure at Ellis Island, c. 1905 - 1920, UCR Museum of Phography Computer Vision : CISC 4/689

  45. Stereo matching • attempt to match every pixel • use additional constraints Computer Vision : CISC 4/689

  46. disparity Depth Z Elevation Zw A Simple Stereo System LEFT CAMERA RIGHT CAMERA baseline Right image: target Left image: reference Zw=0 Computer Vision : CISC 4/689

  47. Let’s discuss reconstruction with this geometry before correspondence, because it’s much easier. ( -ve,  +ve, refer previous slide fig.) P xl,yl=(f X/Z, f Y/Z) Xr,yr=(f (X-T)/Z, f Y/Z) d=xl-xr=f X/Z – f (X-T)/Z Z Disparity: xl xr f pl pr Ol Or T Then given Z, we can compute X and Y. T is the stereo baseline d measures the difference in retinal position between corresponding points (Camps) Computer Vision : CISC 4/689

  48. Correspondence: What should we match? • Objects? • Edges? • Pixels? • Collections of pixels? Computer Vision : CISC 4/689

  49. Extracting Structure • The key aspect of epipolar geometry is its linear constraint on where a point in one image can be in the other • By correlation-matching pixels (or features) along epipolar lines and measuring the disparity between them, we can construct a depth map (scene point depth is inversely proportional to disparity) courtesy of P. Debevec View 1 View 2 Computed depth map Computer Vision : CISC 4/689

  50. Correspondence: Photometric constraint • Same world point has same intensity in both images. • Lambertian fronto-parallel • Issues: • Noise • Specularity • Foreshortening Computer Vision : CISC 4/689

More Related