1 / 31

Multiple View Reconstruction Class 25

Multiple View Reconstruction Class 25. Multiple View Geometry Comp 290-089 Marc Pollefeys. Content. Background : Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation. Single View : Camera model, Calibration, Single View Geometry.

iman
Download Presentation

Multiple View Reconstruction Class 25

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multiple View ReconstructionClass 25 Multiple View Geometry Comp 290-089 Marc Pollefeys

  2. Content • Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation. • Single View: Camera model, Calibration, Single View Geometry. • Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies. • Three Views: Trifocal Tensor, Computing T. • More Views: N-Linearities, Self-Calibration, Multi View Reconstruction, Bundle adjustment, Cheirality, Duality, Dynamic SfM

  3. Multi-view computation

  4. practical structure and motion recovery from images • Obtain reliable matches using matching or tracking and 2/3-view relations • Compute initial structure and motion • sequential structure and motion recovery • hierarchical structure and motion recovery • Refine structure and motion • bundle adjustment • Auto-calibrate • Refine metric structure and motion

  5. X x F Sequential SaM recovery • Select two initial views • Extract features • Compute two-view geometry and matches • Initialize projective pose for two-views • Initialize new structure • For every additional view i • Extract features • Compute two view geometry i-1/i and matches • Compute pose using robust algorithm • Refine existing structure • Initialize new structure • Refine proj. SaM estimation • Self-calibrate • Refine metr.SaM estimation

  6. Hierarchical structure and motion recovery • Compute 2-view • Compute 3-view • Stitch 3-view reconstructions • Merge and refine reconstruction F T H PM

  7. Refining structure and motion • Minimize reprojection error • Maximum Likelyhood Estimation (if error zero-mean Gaussian noise) • Huge problem but can be solved efficiently (Bundle adjustment)

  8. P1 P2 P3 M U1 U2 W U3 WT V 3xn (in general much larger) 12xm Sparse bundle adjustment LM iteration: Jacobian of has sparse block structure im.pts. view 1 Needed for non-linear minimization

  9. U-WV-1WT WT V 3xn 11xm Sparse bundle adjustment Eliminate dependence of camera/motion parameters on structure parameters Note in general 3n >> 11m Allows much more efficient computations e.g. 100 views,10000 points, solve 1000x1000, not 30000x30000 Often still band diagonal use sparse linear algebra algorithms

  10. Degenerate configurations (H&Z Ch.21) • Camera resectioning • Two views • More views

  11. Camera resectioning • Cameras as points • 2D case – Chasles’ theorem

  12. Ambiguity for 3D cameras Twisted cubic (or less) meeting lin. subspace(s) (degree+dimension<3)

  13. Ambiguous two-view reconstructions Ruled quadric containing both scene points and camera centers  alternative reconstructions exist for which the reconstruction of points located off the quadric are not projectively equivalent • hyperboloid 1s • cone • pair of planes • single plane + 2 points • single line + 2 points

  14. Multiple view reconstructions • Single plane is still a problem • Hartley and others looked at 3 and more view critical configurations, but those are rather exotic and are not a problem in practice.

  15. Carlsson-Weinshall duality (H&Z Ch.19) • Exchange role of points and cameras • Dualize algorithm for n views and m+4 points to algorithm for m views and n+4 points e.g. (2im,7+pts)↔(3+im,6pts)

  16. Reduced camera duality Reduced camera: Carlsson-Weinshall duality

  17. Reduced camera reconstruction  N M N M

  18. Obtain reduced cameras Pick 4 reference points to form projective basis in P2

  19. Dual algorithm outline: transpose input transform Solve dual problem Dualize Transform to reduced cameras Reverse transform extend

  20. Applications • 6 points in 3 views minimal, useful for 3-view RANSAC reduced F-matrix (eiTFei=0,i=1…4) …in N views useful for reconstruction from tracks • 7 points in 4 or more views reduced trifocal tensor

  21. 6 points in N views (Hartley and Dano CVPR00) use Sampson error in stead of algebraic (important because of projective warping!)

  22. Oriented projective geometry (~H&Z Ch.20) • two geometric entities are equivalent if they are equal up to a strictly positive scale factor projective geometry oriented projective geometry (from PhD Stephane Laveau)

  23. Oriented projective geometry back front

  24. Oriented line oriented line pxq goes from p to q over shortest distance

  25. Oriented plane • Front • Camera focal plane • In front of camera or

  26. oriented plane through camera center and 3D point M C

  27. Oriented epipolar plane 1 point correspondence allows orientation Eliminates zone -+ and +-, But not -- (P∞ not known)

  28. Multi camera orientation constraint Hartley’s Cheirality Nister’s QUARC

  29. Application to view synthesis Laveau’96 which point is in front?

  30. Epipole orientation

  31. Next class: Dynamic structure from motion

More Related