1 / 21

Projective Factorization of Multiple Rigid-Body Motions

Projective Factorization of Multiple Rigid-Body Motions. Ting Li, Vinutha Kallem, Dheeraj Singaraju, and Ren é Vidal Johns Hopkins University. 3-D motion segmentation problem. Mathematics of the problem depends on Number of frames (2, 3, multiple) Projection model (affine, perspective)

noahj
Download Presentation

Projective Factorization of Multiple Rigid-Body Motions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Projective Factorization of Multiple Rigid-Body Motions Ting Li, Vinutha Kallem, Dheeraj Singaraju, and René Vidal Johns Hopkins University

  2. 3-D motion segmentation problem • Mathematics of the problem depends on • Number of frames (2, 3, multiple) • Projection model (affine, perspective) • Motion model (affine, translational, homography, fundamental matrix, etc.) • 3-D structure (planar or not) Given a set of point correspondences in multiple views, determine the model to which each correspondence belongs.

  3. Motion estimation: multiple affine views Structure = 3D surface • Affine camera model • p = point • f = frame • Motion of one rigid-body lives in a 4-D subspace(Boult and Brown ’91, Tomasi and Kanade ‘92) • P = #points • F = #frames Motion = camera position and orientation Linear relationship between and

  4. Motion segmentation: multiple affine views • Multibody grouping: Gear ’98, Costeira and Kanade ’98, Kanatani ’01, Kanatani and Matsunaga ’02 • Multiframe segmentation algorithms that use all the frames simultaneously • Cannot deal with partially dependent motions • Statistical methods: Sugaya and Kanatani ’04, RANSAC • Robust to noise and outliers • Computationally intense • Algebraic approaches: Vidal and Hartley ’04, Yan and Pollefeys ’06 • Errors comparable to the statistical methods • Significantly reduce computation time Segmentation of multiple motions is equivalent to clustering subspaces of dim 4

  5. Motion estimation: multiple perspective views Estimate depths Compute M and S • Set all depths to 1 • Use epipolar geometry to estimate depths Rk 4 projection factorization • Perspective camera model UNKNOWN depths are unknown depths are known ITERATE Sturm and Triggs ’96 Iterative extensions of the Sturm and Triggs algorithm Triggs ‘96, Mahmud et al ‘01, Oliensis and Hartley ‘06

  6. Motion segmentation: multiple perspective views • Algebraic approaches:Vidal, Ma, Soatto and Sastry ’06, Hartley and Vidal ‘04 • Fit global models using two-view and three-view geometry to segment sequences with two and three frames respectively • Statistical approaches:Torr ‘1998, Schindler, U and Wang ‘06 • Use two-view geometry along with model selection and RANSAC to segment sequences with two frames only • Generate candidate models for pairs of 2 views and combine the results across all frames using model selection Segmentation of multiple motions for multiple motions is a non-linear problem One would prefer to use all frames simultaneously ?

  7. Paper contributions • A 3-D motion segmentation algorithm from multiple perspective views that uses all frames simultaneously • Generalizes the subspace separation methods from the affine to the perspective case • Can deal with partially dependent and transparent motions • Can achieve good trade off between speed and accuracy • Generalizes the Sturm & Triggs algorithm to the case of multiple motions • Use the Sturm & Triggs algorithm to generalize the subspace separation techniques to the case of perspective views

  8. Schematic of our proposed algorithm Dimension is now 3F, unlike 2F earlier Given depths Do subspace separation Given segmentation Estimate depths Estimate depths Subspace separation

  9. Generalized Principal Component Analysis (GPCA) Apply spectral clustering to Estimate normals Segment Dimensionality reduction Fit global model

  10. Local Subspace Affinity (LSA) Project each point onto the unit sphere Dimensionality reduction Spectral clustering using subspace angles as similarity Locally fit a subspace through each point Segment

  11. Estimation of depths for each group Iterate until convergence of Rank 4 projection Make initial guess for depths Re-estimate the depths

  12. Implementation details • Normalization of the image co-ordinates such that they have 0 mean and a standard deviation of (Hartley ’97) • Balancing by enforcing its columns and rows to have norm 1(Sturm and Triggs ’96) • Initialization of the iterative algorithm for estimating depths, by setting all depths to 1

  13. Summary of algorithm Trajectories in multiple perspective views Given initial value of depths Given initial segmentation Estimation of depths Subspace separation ITERATE GPCA 4 variations of our algorithm

  14. Results Testing on the Hopkins155 database Description of database • Comparison with algorithms for segmenting affine views • Multi Stage Learning (MSL): Sugaya & Kanatani ’04 • Generalized Principal Component Analysis (GPCA): Vidal & Hartley ’04 • Local Subspace Affinity (LSA): Yan & Pollefeys ’06

  15. Results: Statistics for affine segmentation Tron and Vidal ’07 Statistics for three motions Statistics for two motions • LSA gives errors comparable to those of MSL • GPCA performs well with 2 motions, but not 3 motions • LSA and GPCA are significantly faster than MSL

  16. Results: Statistics for 2 motions • The errors of the affine GPCA algorithm are reduced by our iterative algorithm • The errors of the affine LSA algorithm are reduced by our iterative algorithm, when given an initial segmentation

  17. Results: Statistics for 3 motions • The errors of GPCA are reduced by our iterative algorithm • The errors of LSA are reduced by our iterative algorithm • Our best method improves best existing algorithm by 2%

  18. Conclusions • We propose the first multibody multiframe 3D motion segmentation algorithm for perspective views, that uses all the frames simultaneously • The algorithm generalizes subspace separation techniques such as LSA and GPCA to segment perspective views • Comprehensive testing on an extensive database shows that our method improves the results of existing algorithms for affine segmentation

  19. Open Issues • Rigorous convergence analysis of the algorithm • Extension of the algorithm to deal with missing data and outliers

  20. Acknowledgements • This work was supported by startup funds from Johns Hopkins University, and by grants NSF CAREER IIS-04-47739, NSF EHS-05-09101 and ONR N00014-05-1083. • We would like to thank the following people for providing us with data and code • Dr. K. Kanatani • Dr. M. Pollefeys • R. Tron

  21. QUESTIONS ?

More Related