1 / 44

Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion

Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion. Professor Sebastian Thrun CAs: Dan Maynes-Aminzade, Mitul Saha, Greg Corrado Slides by: Gary Bradski, Intel Research and Stanford SAIL. Structure From Motion. features. camera.

brone
Download Presentation

Stanford CS223B Computer Vision, Winter 2006 Lecture 8 Structure From Motion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Stanford CS223B Computer Vision, Winter 2006Lecture 8 Structure From Motion Professor Sebastian Thrun CAs: Dan Maynes-Aminzade, Mitul Saha, Greg Corrado Slides by: Gary Bradski, Intel Research and Stanford SAIL

  2. Structure From Motion features camera Recover: structure (feature locations), motion (camera extrinsics)

  3. Structure From Motion (1) [Tomasi & Kanade 92]

  4. Structure From Motion (2) [Tomasi & Kanade 92]

  5. Structure From Motion (3) [Tomasi & Kanade 92]

  6. Structure From Motion (4a): Images Marc Pollefeys

  7. Structure From Motion (4b) Marc Pollefeys

  8. Structure From Motion • Problem 1: • Given n points pij =(xij, yij) in m images • Reconstruct structure: 3-D locations Pj =(xj, yj, zj) • Reconstruct camera positions (extrinsics) Mi=(Aj, bj) • Problem 2: • Establish correspondence: c(pij)

  9. SFM: General Formulation O X -x Z f

  10. SFM: Bundle Adjustment O X -x Z f

  11. Bundle Adjustment • SFM = Nonlinear Least Squares problem • Minimize through • Gradient Descent • Conjugate Gradient • Gauss-Newton • Levenberg Marquardt (!) • Prone to local minima

  12. Count # Constraints vs #Unknowns • m camera poses • n points • 2mn point constraints • 6m+3n unknowns • Suggests: need 2mn  6m + 3n • But: Can we really recover all parameters???

  13. How Many Parameters Can’t We Recover? We can recover all but… Place Your Bet!

  14. Count # Constraints vs #Unknowns • m camera poses • n points • 2mn point constraints • 6m+3n unknowns • Suggests: need 2mn  6m + 3n • But: Can we really recover all parameters??? • Can’t recover origin, orientation (6 params) • Can’t recover scale (1 param) • Thus, we need 2mn  6m + 3n -7

  15. Are done? • No, bundle adjustment has many local minima.

  16. The “Trick Of The Day” • Replace Perspective by Orthographic Geometry • Replace Euclidean Geometry by Affine Geometry • Solve SFM linearly (“closed” form, globally optimal) • Post-Process to make solution Euclidean • Post-Process to make solution perspective By Tomasi and Kanade, 1992

  17. Orthographic Camera Model Extrinsic Parameters Rotation Orthographic Projection Limit of Pinhole Model:

  18. Orthographic Projection Limit of Pinhole Model: Orthographic Projection

  19. The Orthographic SFM Problem subject to

  20. The Affine SFM Problem drop the constraints subject to

  21. Count # Constraints vs #Unknowns • m camera poses • n points • 2mn point constraints • 8m+3n unknowns • Suggests: need 2mn  8m + 3n • But: Can we really recover all parameters???

  22. How Many Parameters Can’t We Recover? We can recover all but… Place Your Bet!

  23. The Answer is (at least): 12

  24. Points for Solving Affine SFM Problem • m camera poses • n points • Need to have: 2mn  8m + 3n-12

  25. Affine SFM Fix coordinate system by making p0=origin Rank Theorem: Q has rank 3 Proof:

  26. The Rank Theorem 2m elements n elements

  27. Singular Value Decomposition

  28. Affine Solution to Orthographic SFM Gives also the optimal affine reconstruction under noise

  29. Back To Orthographic Projection Find C and d for which constraints are met Search in 12-dim space (instead of 8m + 3n-12)

  30. Back To Projective Geometry Orthographic (in the limit) Projective

  31. Back To Projective Geometry O X -x Z f Optimize Using orthographic solution as starting point

  32. The “Trick Of The Day” • Replace Perspective by Orthographic Geometry • Replace Euclidean Geometry by Affine Geometry • Solve SFM linearly (“closed” form, globally optimal) • Post-Process to make solution Euclidean • Post-Process to make solution perspective By Tomasi and Kanade, 1992

  33. Structure From Motion • Problem 1: • Given n points pij =(xij, yij) in m images • Reconstruct structure: 3-D locations Pj =(xj, yj, zj) • Reconstruct camera positions (extrinsics) Mi=(Aj, bj) • Problem 2: • Establish correspondence: c(pij)

  34. The Correspondence Problem View 1 View 2 View 3

  35. Correspondence: Solution 1 • Track features (e.g., optical flow) • …but fails when images taken from widely different poses

  36. Correspondence: Solution 2 • Start with random solution A, b, P • Compute soft correspondence: p(c|A,b,P) • Plug soft correspondence into SFM • Reiterate See Dellaert/Seitz/Thorpe/Thrun, Machine Learning Journal, 2003

  37. Example

  38. Results: Cube

  39. Animation

  40. Tomasi’s Benchmark Problem

  41. Reconstruction with EM

  42. 3-D Structure

  43. Correspondence: Alternative Approach • Ransac [Fisher/Bolles] = Random sampling and consensus

  44. Summary SFM • Problem • Determine feature locations (=structure) • Determine camera extrinsic (=motion) • Two Principal Solutions • Bundle adjustment (nonlinear least squares, local minima) • SVD (through orthographic approximation, affine geometry) • Correspondence • (RANSAC) • Expectation Maximization

More Related