1 / 12

Structure from Motion

Structure from Motion. A moving camera/computer computes the 3D structure of the scene and its own motion. Sensing 3D scene structure via a moving camera. We now have two views over time/space compared to stereo which has multiple views at the same time. Assumptions for now.

arsenio-dunn
Download Presentation

Structure from Motion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Structure from Motion A moving camera/computer computes the 3D structure of the scene and its own motion CSE 803 Fall 2008 Stockman

  2. Sensing 3D scene structure via a moving camera We now have two views over time/space compared to stereo which has multiple views at the same time. CSE 803 Fall 2008 Stockman

  3. Assumptions for now • The scene is rigid. • The scene may move or the camera may move giving a sequence of 2 or more 2D images • Corresponding 2D image points (Pi, Pj) are available across the images CSE 803 Fall 2008 Stockman

  4. What can be computed • The 3D coordinates of the scene points • The motion of the camera Camera sees many frames of 2D points Rigid scene with many 3D interest points From Jabara, Azarbayejani, Pentland CSE 803 Fall 2008 Stockman

  5. From 2D point correspondences, compute 3D points and TR CSE 803 Fall 2008 Stockman

  6. applications • We can compute a 3D model of a landmark from a video • We can compute the trajectory of the sensor relative to the 3D object points CSE 803 Fall 2008 Stockman

  7. Use only 2D correspondences, SfM can compute 3D jig pts … up to one scale factor. CSE 803 Fall 2008 Stockman

  8. http://www1.cs.columbia.edu/~jebara/htmlpapers/SFM/sfm.html Jabara, Azarbayejani, Pentland • Two video frames with corresponding 2D interest points. 3D points can be computed from SfM method. • Some edges detected from 2D gradients. • Texture mapping from 2D frames onto 3D polyhedral model. • 3D model can be viewed arbitrarily! CSE 803 Fall 2008 Stockman

  9. Virtual museums • Much work, and software, from about 10 years ago. • 3D models, including shape and texture can be made of famous places (Notre Dame, Taj Mahal, Titanic, etc.) and made available to those who cannot travel to see the real landmark. • Theoretically, only quality video is required. • Usually, some handwork is needed. CSE 803 Fall 2008 Stockman

  10. SfM methods • Typically require careful mathematics • EX: from 5 matched points, get 10 equations to estimate 10 unknowns; also a more popular 8 pt linear method • Methods must consider effects of noise • See Faugeras et al and ******** • Methods can run in real time CSE 803 Fall 2008 Stockman

  11. Special mathematics • Epipolar geometry • Fundamental matrix: computed from a pair of cameras • Essential matrix: specialization of fundamental matrix when calibration is available CSE 803 Fall 2008 Stockman

  12. HW7 on stereo computation (Sat 6 Dec) (60%/7) Final Exam (25%): Monday 8 Dec 12:45-2:45 SfM report (6 Dec) and demonstration. Find existing code and get it to work on CSE 803 images (60%/7 + 10%) Final Exam (15%) limited problems Finishing the course: 2 options CSE 803 Fall 2008 Stockman

More Related