1 / 36

Step-by-Step Camera Model Building with Euclidean Reconstruction

This review discusses feature selection, camera calibration, and sparse structure modeling in computer vision for augmented reality applications. The process includes feature tracking, epipolar rectification, dense correspondence, and wide-baseline feature matching. Techniques like RANSAC and two-view initialization are used for robust multi-view reconstruction and Euclidean structure estimation. Examples and step-by-step instructions are provided for upgrading from projective to Euclidean reconstruction. Nonlinear refinement methods such as bundle adjustment are also covered.

msailer
Download Presentation

Step-by-Step Camera Model Building with Euclidean Reconstruction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Step-by-Step model building Jana Kosecka Department of Computer Science George Mason University http://www.cs.gmu.edu/~kosecka

  2. Review Feature selection Feature selection Feature correspondence Camera Calibration Landing Augmented Reality Euclidean Reconstruction Vision Based Control Sparse Structure and camera motion SIGGRAPH, August 8, 2004

  3. Review Feature selection Feature selection Feature correspondence Camera Calibration Epipolar Rectification Dense Correspondence Texture mapping Euclidean Reconstruction Sparse Structure and motion 3-D Model SIGGRAPH, August 8, 2004

  4. Partial Scene Knowledge Partial Motion Knowledge Partial Calibration Knowledge Review Feature selection Feature selection Feature correspondence Projective Reconstruction Epipolar Rectification Camera Self-Calibration Dense Correspondence Texture mapping Euclidean Reconstruction 3-D Model SIGGRAPH, August 8, 2004

  5. Examples SIGGRAPH, August 8, 2004

  6. Feature Selection • Compute Image Gradient • Compute Feature Quality measure for each pixel • Search for local maxima Feature Quality Function Local maxima of feature quality function SIGGRAPH, August 8, 2004

  7. Feature Tracking • Translational motion model • Closed form solution • Build an image pyramid • Start from coarsest level • Estimate the displacement at the coarsest level • Iterate until finest level SIGGRAPH, August 8, 2004

  8. Coarse to fine feature tracking 0 1 2 • compute • warp the window in the second image by • update the displacement • go to finer level • At the finest level repeat for several iterations SIGGRAPH, August 8, 2004

  9. Optical Flow • Integrate around over image patch • Solve SIGGRAPH, August 8, 2004

  10. Affine feature tracking Intensity offset Contrast change SIGGRAPH, August 8, 2004

  11. Tracked Features SIGGRAPH, August 8, 2004

  12. Wide baseline matching Point features detected by Harris Corner detector SIGGRAPH, August 8, 2004

  13. Wide baseline Feature Matching • Select the features in two views • For each feature in the first view • Find the feature in the second view that maximizes • Normalized cross-correlation measure Select the candidate with the similarity above selected threshold SIGGRAPH, August 8, 2004

  14. More correspondences and Robust matching • Select set of putative correspondences • Repeat 1. Select at random a set of 8 successful matches 2. Compute fundamental matrix 3. Determine the subset of inliers, compute distance to epipolar line 4. Count the number of points in the consensus set SIGGRAPH, August 8, 2004

  15. RANSAC in action Inliers Outliers SIGGRAPH, August 8, 2004

  16. Epipolar Geometry • Epipolar geometry in two views • Refined epipolar geometry using nonlinear estimation of F SIGGRAPH, August 8, 2004

  17. Two view initialization • Recover epipolar geometry • Compute (Euclidean) projection matrices and 3-D struct. • Compute (Projective) projection matrices and 3-D struct. calibrated uncalibrated unknown SIGGRAPH, August 8, 2004

  18. 3-D reconstruction from two views Projective Reconstruction SIGGRAPH, August 8, 2004

  19. Repeat until convergence 2. Refine structure estimate given all the motions Multi-view reconstruction • Two view - initialized motion and structure estimates (scales) • Multi-view factorization - recover the remaining camera positions and refine the 3-D structure by iteratively computing 1. Compute i-th motion given the known structure iteration SIGGRAPH, August 8, 2004

  20. Projective Ambiguity • calibrated case – projection matrices - and Euclidean structure • Uncalibrated case • for i-th frame • for 1st frame • Transform projection matrices and 3-D structure by H SIGGRAPH, August 8, 2004

  21. Upgrade to Euclidean Reconstruction • From Projective to Euclidean Reconstruction • Remove the ambiguity characterized by • How to estimate H ? - Absolute quadric constraint • be recovered by a nonlinear minimization unknowns SIGGRAPH, August 8, 2004

  22. Upgrade to Euclidean Reconstruction • Special case unknown focal length • other internal parameters are known (proj. center, aspect ratio) • Q can be parametrized in the following way • Zero entries in lead to linear constraints on Q • Initial estimate of Q can be obtained using linear techniques SIGGRAPH, August 8, 2004

  23. Example of multi-view reconstruction Projective Reconstruction Euclidean reconstruction SIGGRAPH, August 8, 2004

  24. Nonlinear Refinement • Euclidean Bundle adjustment • Initial estimates of are available • Final refinement, nonlinear minimization with respect to all unknowns SIGGRAPH, August 8, 2004

  25. Example - Euclidean multi-view reconstruction SIGGRAPH, August 8, 2004

  26. Example Original sequence Tracked Features SIGGRAPH, August 8, 2004

  27. Recovered model SIGGRAPH, August 8, 2004

  28. Euclidean Reconstruction SIGGRAPH, August 8, 2004

  29. 1. Map the epipole to infinity Translate the image center to the origin Rotate around z-axis for the epipole lie on the x-axis Transform the epipole from x-axis to infinity 2. Find a matching transformation is compatible with the epipolar geometry is chosen to minimize overall disparity Epipolar rectification • Make the epipolar lines parallel • Dense correspondences along image scanlines • Computation of warping homographies SIGGRAPH, August 8, 2004

  30. Epipolar rectification Rectified Image Pair SIGGRAPH, August 8, 2004

  31. Epipolar rectification Rectified Image Pair SIGGRAPH, August 8, 2004

  32. Dense Matching • Establish dense correspondences along scan-lines • Standard stereo configuration • Constraints to guide the search 1. ordering constraint 2. disparity constraint – limit on disparity 3. uniqueness constraint – each point has a unique match in the second view SIGGRAPH, August 8, 2004

  33. Dense Matching SIGGRAPH, August 8, 2004

  34. Dense Reconstruction SIGGRAPH, August 8, 2004

  35. Texture mapping, hole filling SIGGRAPH, August 8, 2004

  36. Texture mapping SIGGRAPH, August 8, 2004

More Related