290 likes | 495 Views
Structure From Motion. Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision http://robots.stanford.edu/cs223b. Structure From Motion (1). [Tomasi & Kanade 92]. Structure From Motion (2). [Tomasi & Kanade 92]. Structure From Motion (3). [Tomasi & Kanade 92].
E N D
Structure From Motion Sebastian Thrun, Gary Bradski, Daniel Russakoff Stanford CS223B Computer Vision http://robots.stanford.edu/cs223b
Structure From Motion (1) [Tomasi & Kanade 92]
Structure From Motion (2) [Tomasi & Kanade 92]
Structure From Motion (3) [Tomasi & Kanade 92]
Structure From Motion • Problem 1: • Given n points pij =(xij, yij) in m images • Reconstruct structure: 3-D locations Pj =(xj, yj, zj) • Reconstruct camera positions (extrinsics) Mi=(Aj, bj) • Problem 2: • Establish correspondence: c(pij)
Orthographic Camera Model Extrinsic Parameters Rotation Orthographic Projection Limit of Pinhole Model:
Orthographic Projection Limit of Pinhole Model: Orthographic Projection
Count # Constraints vs #Unknowns • m camera poses • n points • 2mn point constraints • 8m+3n unknowns • Suggests: need 2mn 8m + 3n • But: Can we really recover all parameters???
How Many Parameters Can’t We Recover? We can recover all but… Place Your Bet!
Points for Solving Affine SFM Problem • m camera poses • n points • Need to have: 2mn 8m + 3n-12
Affine SFM Fix coordinate system by making p0=origin Rank Theorem: Q has rank 3 Proof:
The Rank Theorem 2m elements n elements
Tomasi/Kanade 1992 Singular Value Decomposition
Tomasi/Kanade 1992 Gives also the optimal affine reconstruction under noise
Back To Orthographic Projection Find C and d for which constraints are met
Back To Projective Geometry Orthographic (in the limit) Projective
Projective Camera: Non-Linear Optimization Problem: Bundle Adjustment!
Structure From Motion • Problem 1: • Given n points pij =(xij, yij) in m images • Reconstruct structure: 3-D locations Pj =(xj, yj, zj) • Reconstruct camera positions (extrinsics) Mi=(Aj, bj) • Problem 2: • Establish correspondence: c(pij)
The Correspondence Problem View 1 View 2 View 3
Correspondence: Solution 1 • Track features (e.g., optical flow) • …but fails when images taken from widely different poses
Correspondence: Solution 2 • Start with random solution A, b, P • Compute soft correspondence: p(c|A,b,P) • Plug soft correspondence into SFM • Reiterate • See Dellaert/Seitz/Thorpe/Thrun 2003