Correspondence-Free Determination of the Affine Fundamental Matrix

1. 2007. 2. 6 (Tue) Young Ki Baik, Computer Vision Lab. Correspondence-Free Determinationof the Affine Fundamental Matrix

2. Correspondence-Free Determination of the Affine fundamental Matrix • References • Correspondence-Free Determination of the Affine Fundamental Matrix • Stefan Lehmann et. al. PAMI 2007 • Radon-based Structure from Motion Without Correspondences • Ameesh Makadia et. al. CVPR 2005 • Robust Fundamental Matrix Determination without Correspondences • Stefan Lehmann et. al. APRS 2005

3. Correspondence-Free Determination of the Affine fundamental Matrix • Contents • The conventional method of SfM • Features of the proposed method • Theory of the proposed algorithm • Experimental results • Discussion

4. Correspondence-Free Determination of the Affine fundamental Matrix • Conventional SfM Image Sequence Feature Extraction/ Matching Relating Image Projective Reconstruction Auto-Calibration Dense Matching 3D Model Building

5. Correspondence-Free Determination of the Affine fundamental Matrix • The Problem of conventional SfM • The high sensitivity of fundamental matrix • Noise and outlier correspondences in feature data severely affect the precision of the fundamental matrix • Incomplete 3D reconstruction

6. Correspondence-Free Determination of the Affine fundamental Matrix • The Key Feature • Correspondence-free • Finding Correspondence (X) • Illumination changes-free (?) • Intensity value (X) • Position of features (O) • Limitation • Occlusion ? (X) • Affine camera only!!

7. Correspondence-Free Determination of the Affine fundamental Matrix • Parallel projection • Orthographic projection

8. Correspondence-Free Determination of the Affine fundamental Matrix • Mathematical Model • Assumption • We have 3-dimensional N features. • The 3D feature space is represented by,

9. Correspondence-Free Determination of the Affine fundamental Matrix • Mathematical Model • Assumption • Parallel projection model determines the 2D feature projections along the lines that are running parallel to the view axis (z-axis) of the camera. • The model considers a continuous projection plane with infinite extent. • The corresponding 2D projection data is…

10. Correspondence-Free Determination of the Affine fundamental Matrix • Mathematical Model • Fourier spectra • The Fourier spectra of and can be denoted as

11. Correspondence-Free Determination of the Affine fundamental Matrix • Mathematical Model • 2-view case • The 3D correspondence feature point • Relation between images • The 3D frequency vector

12. Correspondence-Free Determination of the Affine fundamental Matrix • Mathematical Model • 2-view case • Relation between 3D spectrums The equation shows that rotation R also establishes the transformation between corresponding frequency indices in the 3D Fourier spaces of the original and the transformed spectrum or scene.

13. Correspondence-Free Determination of the Affine fundamental Matrix • Mathematical Model • Matching line • The magnitudes of two spectra along these lines will be identical, while the phases will show a linear offset dependent upon the translational component of transformation. • The proposed method is to detect these matching lines.

14. Correspondence-Free Determination of the Affine fundamental Matrix • Mathematical Model • Matching line angle pair • Angle pair of the matching lines with respect to the axes of the frequency spectra F and F’, respectively.

15. Correspondence-Free Determination of the Affine fundamental Matrix • Mathematical Model • Analysis of the transformation parameters • as the corresponding frequency locations along the matching lines of the spectrum F of the first and the spectrum F’ of the second set of 2D features, respectively. • It follows that,

16. Correspondence-Free Determination of the Affine fundamental Matrix • Mathematical Model • Analysis of the transformation parameters

17. Correspondence-Free Determination of the Affine fundamental Matrix • Mathematical Model • Derivation of a 3D rotation matrix

18. Correspondence-Free Determination of the Affine fundamental Matrix • Estimation of the fundamental matrix • By using 3D rotation matrix, we can obtain the relation between 2D projection point (x’,y’) of a 3D feature (x,y,z) with translation.

19. Correspondence-Free Determination of the Affine fundamental Matrix • Estimation of the fundamental matrix • In the orthographic projection case, all epipolar lines are parallel. • Then we can denote the epipolar line of 2D feature point (x,y) as

20. Correspondence-Free Determination of the Affine fundamental Matrix • Estimation of the fundamental matrix

21. Correspondence-Free Determination of the Affine fundamental Matrix • Estimation of the fundamental matrix

22. Correspondence-Free Determination of the Affine fundamental Matrix • Estimation of matching line angle • For the practical purpose, corresponding discrete spectra should be defined as follows.

23. Correspondence-Free Determination of the Affine fundamental Matrix • Estimation of matching line angle • The final object function • Discrete Fourier-Mellin transformation method • To find out the matching line (According to the well known shift theorem of the FT, a shift in the space domain corresponds to a phase shift in the frequency domain.)

24. Correspondence-Free Determination of the Affine fundamental Matrix • Overall flow

25. Correspondence-Free Determination of the Affine fundamental Matrix • Experimental result • test images : telephoto lens • Feature points : Harris corner detection method • Ideal epipolar lines are the horizontal lines. • The proposed method shows us good result relative to conventional methods.

26. Camera Calibration Methods for Wide Angle view • Discussion • Key feature • Correspondence-free method for obtaining the fundamental matrix is presented. • Matching line exists between the Fourier transformed data. • Limitation • Proposed method • Considers only affine projection model • Does not treat occlusion problem • Future work • Applying projective projection model