1 / 39

Multiple View Geometry

Multiple View Geometry. Marc Pollefeys University of North Carolina at Chapel Hill. Modified by Philippos Mordohai. Outline. 3-D Reconstruction Fundamental matrix estimation Chapters 10 and 11 of “Multiple View Geometry in Computer Vision” by Hartley and Zisserman. Three questions:.

Download Presentation

Multiple View Geometry

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai

  2. Outline • 3-D Reconstruction • Fundamental matrix estimation • Chapters 10 and 11 of “Multiple View Geometry in Computer Vision” by Hartley and Zisserman

  3. Three questions: • Correspondence geometry: Given an image point x in the first image, how does this constrain the position of the corresponding point x’ in the second image? • (ii) Camera geometry (motion): Given a set of corresponding image points {xi ↔x’i}, i=1,…,n, what are the cameras P and P’ for the two views? • (iii) Scene geometry (structure): Given corresponding image points xi ↔x’i and cameras P, P’, what is the position of (their pre-image) X in space?

  4. p p L2 L2 m1 m1 m1 C1 C1 C1 M M L1 L1 l1 l1 e1 e1 lT1 l2 e2 e2 Canonical representation: l2 m2 m2 m2 l2 l2 Fundamental matrix (3x3 rank 2 matrix) C2 C2 C2 Epipolar Geometry Underlying structure in set of matches for rigid scenes • Computable from corresponding points • Simplifies matching • Allows to detect wrong matches • Related to calibration

  5. 3D reconstruction of cameras and structure reconstruction problem: given xi↔x‘i , compute P,P‘ and Xi for all i without additional information possible up to projective ambiguity

  6. Outline of reconstruction • Compute F from correspondences • Compute camera matrices from F • Compute 3D point for each pair of corresponding points computation of F use x‘iFxi=0 equations, linear in coeff. F 8 points (linear), 7 points (non-linear), 8+ (least-squares) (more on this next class) computation of camera matrices use triangulation compute intersection of two backprojected rays

  7. Reconstruction ambiguity: similarity

  8. Reconstruction ambiguity: projective

  9. Terminology xi↔x‘i Original scene Xi Projective, affine, similarity reconstruction = reconstruction that is identical to original up to projective, affine, similarity transformation Literature: Metric and Euclidean reconstruction = similarity reconstruction

  10. The projective reconstruction theorem If a set of point correspondences in two views determine thefundamental matrix uniquely, then the scene and cameras may be reconstructed from these correspondences alone, and any two such reconstructions from these correspondences are projectively equivalent • along same ray ofP2, idem for P‘2 two possibilities: X2i=HX1i, or points along baseline key result: allows reconstruction from pair of uncalibrated images

  11. Stratified reconstruction • Projective reconstruction • Affine reconstruction • Metric reconstruction

  12. Projective to affine (if D≠0) can be sufficient depending on application, e.g. mid-point, centroid, parallellism

  13. Translational motion points at infinity (not necessarily visible) are fixed for a pure translation  reconstruction of xi↔xi is on p∞

  14. Scene constraints Parallel lines parallel lines intersect at infinity reconstruction of corresponding vanishing point yields point on plane at infinity 3 sets of parallel lines allow to uniquely determine p∞ remark: in presence of noise determining the intersection of parallel lines is a delicate problem remark: obtaining vanishing point in one image can be sufficient

  15. Scene constraints

  16. * * projection constraints Affine to metric identify absolute conic transform so that then projective transformation relating original and reconstruction is a similarity transformation in practice, find image of W∞ image w∞back-projects to cone that intersects p∞ in W∞ note that image is independent of particular reconstruction

  17. The absolute conic (reminder) The absolute conic Ω∞ is a (point) conic on π. In a metric frame: or conic for directions: (with no real points) The absolute conic Ω∞ is a fixed conic under the projective transformation H iff H is a similarity • Ω∞is only fixed as a set • Circles intersect Ω∞ in two points • Spheres intersect π∞ in Ω∞

  18. The image of the absolute conic • It can be shown that the image of the absolute conic is: • ω is a 3×3 symmetric matrix representing the conic: xTωx=0 • Even though the absolute conic contains only imaginary points, its image may include real points

  19. Affine to metric given possible transformation from affine to metric is proof: (Cholesky factorization to obtain A)

  20. Orthogonality vanishing points corresponding to orthogonal directions vanishing line and vanishing point corresponding to plane and normal direction

  21. rectangular pixels square pixels Known internal parameters

  22. Same camera for all images same intrinsics  same image of the absolute conic e.g. moving camera given sufficient images there is in general only one conic that projects to the same image in all images, i.e. the absolute conic This approach is called self-calibration transfer of IAC: provides 4 constraints, one more needed

  23. (in general two solutions) Direct metric reconstruction using ω approach 1 calibrated reconstruction approach 2 compute projective reconstruction back-project w from both images intersection defines W∞ and its support plane p∞

  24. (2 lin. eq. in H-1per view, 3 for two views) Direct reconstruction using ground truth use control points XEi with know coordinates to go from projective to metric (3 lin. eq. in H per point, H has 15 d.o.f.)

  25. Reconstruction summary • Given two uncalibrated images compute (PM,P‘M,{XMi}) • (i.e. within similarity of original scene and cameras) • Algorithm • Compute projective reconstruction (P,P‘,{Xi}) • Compute F from xi↔x‘i • Compute P,P‘ from F • Triangulate Xi from xi↔x‘i • Rectify reconstruction from projective to metric • Direct method: compute H from control points • Stratified method: • Affine reconstruction: compute p∞ • Metric reconstruction: compute IAC w

  26. Outline • 3-D Reconstruction • Fundamental matrix estimation • Chapters 10 and 11 of “Multiple View Geometry in Computer Vision” by Hartley and Zisserman

  27. Epipolar geometry: basic equation separate known from unknown (data) (unknowns) (linear)

  28. The singularity constraint SVD from linearly computed F matrix (rank 3) Compute closest rank-2 approximation

  29. The minimum case – 7 point correspondences one parameter family of solutions but F1+lF2 not automatically rank 2

  30. 3 F7pts F F2 F1 The minimum case – impose rank 2 (obtain 1 or 3 solutions) (cubic equation) Compute possible l as eigenvalues of (only real solutions are potential solutions)

  31. ~10000 ~100 ~10000 ~100 ~10000 ~10000 ~100 ~100 1 Orders of magnitude difference between column of data matrix  least-squares yields poor results ! The NOT normalized 8-point algorithm

  32. Transform image to ~[-1,1]x[-1,1] (-1,1) (1,1) (0,0) (-1,-1) (1,-1) normalized least squares yields good results(Hartley, PAMI´97) The normalized 8-point algorithm (0,500) (700,500) (0,0) (700,0)

  33. Geometric distance Gold standard Sampson error Symmetric epipolar distance

  34. Gold standard Maximum Likelihood Estimation (= least-squares for Gaussian noise) Initialize: normalized 8-point, (P,P‘) from F, reconstruct Xi Parameterize: (overparametrized F=[t]xM) Minimize cost using Levenberg-Marquardt (preferably sparse LM, see book)

  35. Gold standard Alternative, minimal parametrization (with a=1) (note (x,y,1) and (x‘,y‘,1) are epipoles) • problems: • a=0  pick largest of a,b,c,d to fix to 1 • epipole at infinity  pick largest of x,y,w and of x’,y’,w’ 4x3x3=36 parametrizations! reparametrize at every iteration, to be sure

More Related