450 likes | 572 Views
Stanford CS223B Computer Vision, Winter 2005 Lecture 11: Structure From Motion 2. Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford. Overall Distribution. Question 1: Calibration. Question 1: Calibration. Calibration with planar unknown target
E N D
Stanford CS223B Computer Vision, Winter 2005Lecture 11: Structure From Motion 2 Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford
Question 1: Calibration • Calibration with planar unknown target • Unknown parameters • 4 intrinsics • 6K extrinsics (K = #images) • 2M calibration target parameters (but can’t recover 3) • 2KM constraints
Question 2: Perspective Geometry • Collinearity in 3D 2D (but not converse) • Order in 3D 2D (but not converse) • Equidistance: Not preserved! • Proof (collinearity in 2D):
Question 3: Stereopsis • How does DZ scale with Z? – in approximation!!!
Question 5: Build A System! • Range: stereo or laser • Classification : template, optical flow?, SIFT? • Alternatively: segmentation, range discontinuities • Prediction: person and car • Robustness: normalize image, bring light source • (many other possibilities)
Stanford CS223B Computer Vision, Winter 2005Lecture 11: Structure From Motion 2 Sebastian Thrun, Stanford Rick Szeliski, Microsoft Hendrik Dahlkamp and Dan Morris, Stanford
Structure From Motion (1) [Tomasi & Kanade 92]
Structure From Motion (2) [Tomasi & Kanade 92]
Structure From Motion (3) [Tomasi & Kanade 92]
Structure From Motion • Problem 1: • Given n points pij =(xij, yij) in m images • Reconstruct structure: 3-D locations Pj =(xj, yj, zj) • Reconstruct camera positions (extrinsics) Mi=(Aj, bj) • Problem 2: • Establish correspondence: c(pij)
The “Trick Of The Day” • Replace Euclidean Geometry by Affine Geometry • Solve SFM linearly (“closed” form) • Post-Process to make Euclidean • By Tomasi and Kanade, 1992
Orthographic Camera Model Extrinsic Parameters Rotation Orthographic Projection Limit of Pinhole Model:
Orthographic Projection Limit of Pinhole Model: Orthographic Projection
Count # Constraints vs #Unknowns • m camera poses • n points • 2mn point constraints • 8m+3n unknowns • Suggests: need 2mn 8m + 3n • But: Can we really recover all parameters???
How Many Parameters Can’t We Recover? We can recover all but… Place Your Bet!
Points for Solving Affine SFM Problem • m camera poses • n points • Need to have: 2mn 8m + 3n-12
Affine SFM Fix coordinate system by making p0=origin Rank Theorem: Q has rank 3 Proof:
The Rank Theorem 2m elements n elements
Tomasi/Kanade 1992 Singular Value Decomposition
Tomasi/Kanade 1992 Gives also the optimal affine reconstruction under noise
Back To Orthographic Projection Find C and d for which constraints are met
Back To Projective Geometry Orthographic (in the limit) Projective
The “Trick Of The Day” • Replace Euclidean Geometry by Affine Geometry • Solve SFM linearly (“closed” form) • Post-Process to make Euclidean • By Tomasi and Kanade, 1992
SFM With Projective Camera: See Rick Szeliski’s Lecture! Non-Linear Optimization Problem: Bundle Adjustment!
Structure From Motion • Problem 1: • Given n points pij =(xij, yij) in m images • Reconstruct structure: 3-D locations Pj =(xj, yj, zj) • Reconstruct camera positions (extrinsics) Mi=(Aj, bj) • Problem 2: • Establish correspondence: c(pij)
The Correspondence Problem View 1 View 2 View 3
Correspondence: Solution 1 • Track features (e.g., optical flow) • …but fails when images taken from widely different poses
Correspondence: Solution 2 • Start with random solution A, b, P • Compute soft correspondence: p(c|A,b,P) • Plug soft correspondence into SFM • Reiterate • See Dellaert et al 2003, Machine Learning Journal
Summary SFM • Problem • Determine feature locations (=structure) • Determine camera extrinsic (=motion) • The name SFM is somewhat of a misdemeanor • Two Principal Solutions • Nonlinear optimization (local minima) • Linear (affine geometry) • Correspondence • RANSAC • Expectation Maximization