1 / 41

Computing F class 13

Computing F class 13. Multiple View Geometry Comp 290-089 Marc Pollefeys. Two-view geometry. Epipolar geometry 3D reconstruction F-matrix comp. Structure comp. p. p. L 2. L 2. m 1. m 1. m 1. C 1. C 1. C 1. M. M. L 1. L 1. l 1. l 1. e 1. e 1. l T 1. l 2. e 2. e 2.

baruch
Download Presentation

Computing F class 13

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computing Fclass 13 Multiple View Geometry Comp 290-089 Marc Pollefeys

  2. Two-view geometry Epipolar geometry 3D reconstruction F-matrix comp. Structure comp.

  3. p p L2 L2 m1 m1 m1 C1 C1 C1 M M L1 L1 l1 l1 e1 e1 lT1 l2 e2 e2 Canonical representation: l2 m2 m2 m2 l2 l2 Fundamental matrix (3x3 rank 2 matrix) C2 C2 C2 Epipolar geometry Underlying structure in set of matches for rigid scenes • Computable from corresponding points • Simplifies matching • Allows to detect wrong matches • Related to calibration

  4. The projective reconstruction theorem If a set of point correspondences in two views determine thefundamental matrix uniquely, then the scene and cameras may be reconstructed from these correspondences alone, and any two such reconstructions from these correspondences are projectively equivalent allows reconstruction from pair of uncalibrated images!

  5. Objective • Given two uncalibrated images compute (PM,P‘M,{XMi}) • (i.e. within similarity of original scene and cameras) • Algorithm • Compute projective reconstruction (P,P‘,{Xi}) • Compute F from xi↔x‘i • Compute P,P‘ from F • Triangulate Xi from xi↔x‘i • Rectify reconstruction from projective to metric • Direct method: compute H from control points • Stratified method: • Affine reconstruction: compute p∞ • Metric reconstruction: compute IAC w

  6. Epipolar geometry: basic equation separate known from unknown (data) (unknowns) (linear)

  7. the singularity constraint SVD from linearly computed F matrix (rank 3) Compute closest rank-2 approximation Frobenius distance

  8. the minimum case – 7 point correspondences one parameter family of solutions but F1+lF2 not automatically rank 2

  9. 3 F7pts F F2 F1 the minimum case – impose rank 2 (obtain 1 or 3 solutions) (cubic equation) Compute possible l as eigenvalues of (only real solutions are potential solutions)

  10. ~10000 ~100 ~10000 ~100 ~10000 ~10000 ~100 ~100 1 Orders of magnitude difference Between column of data matrix  least-squares yields poor results ! the NOT normalized 8-point algorithm

  11. (0,500) (700,500) (-1,1) (1,1) (0,0) (0,0) (700,0) (-1,-1) (1,-1) the normalized 8-point algorithm Transform image to ~[-1,1]x[-1,1] Least squares yields good results(Hartley, PAMI´97)

  12. algebraic minimization possible to iteratively minimize algebraic distance subject to det F=0 (see book if interested)

  13. Geometric distance Gold standard Sampson error Symmetric epipolar distance

  14. Gold standard Maximum Likelihood Estimation (= least-squares for Gaussian noise) Initialize: normalized 8-point, (P,P‘) from F, reconstruct Xi Parameterize: (overparametrized) Minimize cost using Levenberg-Marquardt (preferably sparse LM, see book)

  15. Parameterization of F Alternative, minimal parametrization (with a=1) (note (x,y,1) and (x‘,y‘,1) are epipoles) • problems: • a=0  pick largest of a,b,c,d to fix • epipole at infinity  pick largest of x,y,w and of x’,y’,w’ 4x3x3=36 parametrizations! reparametrize at every iteration, to be sure

  16. First-order geometric error (Sampson error) (one eq./point JJT scalar) (problem if some x is located at epipole) advantage: no subsidiary variables required

  17. Symmetric epipolar error

  18. Some experiments:

  19. Some experiments:

  20. Some experiments:

  21. Some experiments: Residual error: (for all points!)

  22. Recommendations: • Do not use unnormalized algorithms • Quick and easy to implement: 8-point normalized • Better: enforce rank-2 constraint during minimization • Best: Maximum Likelihood Estimation (minimal parameterization, sparse implementation)

  23. Special case: Enforce constraints for optimal results: Pure translation (2dof), Planar motion (6dof), Calibrated case (5dof)

  24. The envelope of epipolar lines What happens to an epipolar line if there is noise? Monte Carlo n=50 n=25 n=15 n=10

  25. Other entities? Lines give no constraint for two view geometry (but will for three and more views) Curves and surfaces yield some constraints related to tangency

  26. Automatic computation of F • Interest points • Putative correspondences • RANSAC • (iv) Non-linear re-estimation of F • Guided matching • (repeat (iv) and (v) until stable)

  27. Feature points • Extract feature points to relate images • Required properties: • Well-defined (i.e. neigboring points should all be different) • Stable across views (i.e. same 3D point should be extracted as feature for neighboring viewpoints)

  28. Feature points (e.g.Harris&Stephens´88; Shi&Tomasi´94) Find points that differ as much as possible from all neighboring points homogeneous edge corner Mshould have large eigenvalues Feature = local maxima (subpixel) of F(1,2)

  29. Feature points Select strongest features (e.g. 1000/image)

  30. ? Feature matching Evaluate Normalized Cross Correlation for all features with similar coordinates Keep mutual best matches Still many wrong matches!

  31. 3 3 2 2 4 4 1 5 1 5 Feature example Gives satisfying results for small image motions

  32. Wide-baseline matching… • Requirement to cope with larger variations between images • Translation, rotation, scaling • Foreshortening • Non-diffuse reflections • Illumination geometric transformations photometric changes

  33. Wide-baseline matching… (Tuytelaars and Van Gool BMVC 2000) Wide baseline matching for two different region types

  34. (generate hypothesis) (verify hypothesis) RANSAC Step 1. Extract features Step 2. Compute a set of potential matches Step 3. do Step 3.1 select minimal sample (i.e. 7 matches) Step 3.2 compute solution(s) for F Step 3.3 determine inliers until (#inliers,#samples)<95% Step 4. Compute F based on all inliers Step 5. Look for additional matches Step 6. Refine F based on all correct matches

  35. Finding more matches restrict search range to neighborhood of epipolar line (1.5 pixels) relax disparity restriction (along epipolar line)

  36. Degenerate cases: • Degenerate cases • Planar scene • Pure rotation • No unique solution • Remaining DOF filled by noise • Use simpler model (e.g. homography) • Model selection (Torr et al., ICCV´98, Kanatani, Akaike) • Compare H and F according to expected residual error (compensate for model complexity)

  37. More problems: • Absence of sufficient features (no texture) • Repeated structure ambiguity • Robust matcher also finds • support for wrong hypothesis • solution: detect repetition (Schaffalitzky and Zisserman, BMVC‘98)

  38. geometric relations between two views is fully described by recovered 3x3 matrix F two-view geometry

  39. Next class: image pair rectificationreconstructing points and lines

More Related