Epipolar Geometry from Two Correspondences

1. Epipolar Geometry from Two Correspondences Michal Perďoch, Jiří Matas, Ondřej Chum Center for Machine Perception Czech Technical University, Prague http://cmp.felk.cvut.cz

2. Problem formulation Problem: Wide-baseline stereo with severe viewpoint and scale change. Message of this paper: Two correspondences of Local Affine Frames suffice to estimate epipolar geometry. Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague2/16

3. Local Affine Frame correspondences Local Affine Frames (LAFs), MatasICPR2002: • Affine-covariantly detected local coordinate systems. • Onematching LAF pair providesthree point-to-point correspondences. Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague3/16

4. The correspondence problem Wide-baseline stereo framework (Matas ICPR2002): • Detect regions. MSERs are used here. • Build Local Affine Frames,define Measurement Region. • Geometric and photometric normalisation of MRs. • Establish correspondencesby nearest neighbour search. • Verify correspondences, • estimate model using • RANSAC. Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague4/16

5. RANSAC Algorithm • How to find correct model of EG in presence of outliers? • RANSAC - widely used robust estimator proposed by Fishler and Bolles 1988. • Hypothesise & verify search: Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague5/16

6. RANSAC Algorithm • Average number of RANSAC samples k: • Example (h0 = 0.01): • ChumDAGM2003, has shown that the theoretical number of samples can be achieved by introducing Local Optimisationstep. – probability of finding a better model – proportion of inliers – size of the sample 2 LAFs 3 LAFs 7 points Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague6/16

7. Proposed by ChumDAGM2003, used with 3LAFs and seven point-to-point correspondences. Input: set of data points T, confidence h0. Output: model M* of EG with the largest support S*. Repeat Draw a random sample of minimal size m from datapoints. Compute model parameters Mi and its support Si. If new maximum was detected (i.e., |Si| > |Sj| for (j <i): Apply local optimisation. Store the best model with support S*. Until the probability of finding model better than S*falls under h0. LO-RANSAC Algorithm Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague7/16

8. Six Point EG estimator • Stewénius CVPR2005has shown that EG can be estimated from six points under following assumptions on the cameras: • Unit aspect ratio, zero skew, principal point in the middle of the image. • Unknown but the same focal length. • We take Stewénius’ assumptions. • Is it possible to estimate EG from two Local Affine Frame correspondences? Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague8/16

9. Planar Degeneracy • ChumCVPR2005 has that planar degeneracy causes suboptimal EG estimates. • Six points on a plane do not provide enough constraints for estimation of the fundamental matrix. • Planar degeneracy test - take sample points and check if they lie on a plane. • When a plane is detected do plane and parallax search: • Local optimisation that takes into account plane found. • Sample another pair not consistent with the plane and compute fundamental matrix Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague9/16

10. Input: set of data points T, confidence h0, sample size m = 2. Output: model M*of EG with largest support S*. Repeat Draw a random sample of minimal size m from data points. Compute model parametersMi and its support Si. If new maximum has occurred (i.e. |Si| > |Sj| for (j < i): If a degenerated sample configuration is detected: Perform plane-and-parallax search for EG, otherwise Apply local optimisation. Store the best model with support S*. Until the probability of finding model better than S*falls under h0. 2LAF-LO-RANSAC Algorithm Performed at most log(k) times. Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague10/16

11. Experiments: Scenes Corner e= 0.26 The China Wall e = 0.28 Wash e = 0.23 Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague11/16

12. Experiment 1: Efficiency • What is the “efficiency” of six point estimator? Is the planar degeneracy test necessary? • Comparison of 2LAF, 3LAF with and without planar degeneracy test and PTS7 algorithm against reference EG. • Number of good EG models generated from 1000 all-inlier samples(quality is measured by number of inliers, good – at least 90%). ND - without degeneracy test Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague12/16

13. Experiment 2: Performance • Average number of inliers and samples measured in hundred runs of 7PTS, 3LAF, 2LAF-LO-RANSAC algorithms. • Small drop (two to four percent) in average number of inliers on difficult and noisy scenes for both 3LAF and 2LAF algorithm. • Note the significant speedup(measured by number of samples) of 2LAF algorithm. Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague13/16

14. Experiment 3: Assumptions violation • The six point solver assumes equal focal length in both images. • The focal length ratio on the scene Corner is 1:3 andoff-plane rotation about 30 degrees. • No performance drop was observed. Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague14/16

15. Experiment 3: Assumptions violation • Robustness to change of pixel aspect ratio was measured on scene China Wall (#inliers in 100 runs). • Width of pixel scaled: 0.5, 1.0, 2.0 times from top to bottom. Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague15/16

16. Conclusions • Contribution • Novel wide-baseline stereo algorithm using two LAF correspondences was proposed. • 6 point (= 2 LAFs) EG estimator can be used in LO-RANSAC framework with the planar degeneracy test. • Performance was experimentally tested and compared to 7PTS and 3LAF algorithms in wide-baseline stereo setup. • Future work • Detect other degenerated configurations. • Compare the number of verifications instead of the number of samples. Thank you for your attention. Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague16/16