1 / 16

Segmentation of Dynamic Scenes from Image Intensities

This talk discusses the problem of segmenting dynamic scenes from image intensities using motion models, specifically affine and Euclidean models. It explores the use of probabilistic approaches, generative models, and spectral clustering, and proposes an algebraic-geometric approach to affine motion segmentation. The talk also presents experimental results and discusses future work.

mlake
Download Presentation

Segmentation of Dynamic Scenes from Image Intensities

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Segmentation of Dynamic Scenes from Image Intensities René Vidal Shankar Sastry Department of EECS, UC Berkeley

  2. A static scene: multiple 2D motion models A dynamic scene: multiple 3D motion models Motivation and problem statement • Given an image sequence, determine • Number of motion models (affine, Euclidean, etc.) • Motion model: affine (2D) or Euclidean (3D) • Segmentation: model to which each pixel belongs • 3-D Motion Segmentation (Vidal-Soatto-Ma-Sastry, ECCV’02) • Generalization of the 8-point algorithm • Multibody fundamental matrix • This Talk: 2-D Motion Segmentation

  3. Previous work • Probabilistic approaches (Jepson-Black’93, Ayer-Sawhney ’95, Darrel-Pentland’95, Weiss-Adelson’96, Weiss’97, Torr-Szeliski-Anandan ’99) • Generative model: data membership + motion model • Obtain motion models using Expectation Maximization • E-step: Given motion models, segment image data • M-step: Given data segmentation, estimate motion models • How to initialize iterative algorithms? • Spectral clustering: normalized cuts (Shi-Malik ‘98) • Similarity matrix based on motion profile • Local methods (Wang-Adelson ’94) • Estimate one model per pixel using a data in a window • Global methods (Irani-Peleg ‘92) • Dominant motion: fit one motion model to all pixels • Look for misaligned pixels & fit a new model to them

  4. Our Approach to Motion Segmentation • Towards an analytic solution to motion segmentation • Can we estimate ALL motion models simultaneously using ALL the image measurements? • When can we do so analytically? In closed form? • Is there a formula for the number of models? • We propose an algebraic geometric approach to affine motion segmentation • Number of models =degree of a polynomial • Groups ≈roots of a polynomial =polynomial factorization • In the absence of noise • Derive a constraint that is independent on the segmentation • There exists a unique solution which is closed form iff n<5 • The exact solution can be computed using linear algebra • In the presence of noise • Derive a maximum likelihood algorithm for zero-mean Gaussian noise in which the E-step is algebraically eliminated

  5. Number of models? with a. ames One-dimensional Segmentation

  6. with a. ames One-dimensional Segmentation • For n groups • Number of groups • Groups

  7. Motion segmentation: the affine model • Constant brightness constraint • Affine motion model for the optical flow • Bilinear affine constraint • Mixture of n affine motion models

  8. 1-dimensional case Multibody affine constraint Veronese map Affine segmentation case The multibody affine constraint

  9. The multibody affine matrix Multibody affine constraint Multibody affine matrix Embedding Lifting Embedding

  10. 1-dimensional case Affine segmentation case Affine motion segmentation algorithm Estimate all models: coefficients of a polynomial Estimate number models: rank of a matrix Estimate individual models: roots/factors of the polynomial

  11. Estimation of individual affine models Can be reduced to scalar case!! Factorization of affine motion models • Factorization of bilinear forms can be reduced to factorization of linear forms • Factorization of linear forms corresponds to segmentation of mixtures of subspaces Generalized PCA: mixture of subspaces • Find roots of polynomial of degree n in one variable • Solve one linear systems in n variables

  12. Optimal affine motion segmentation • Zero-mean Gaussian noise • Minimize distance error in image intensities subject to multibody affine constraints • Using Langrange optimization • After some algebra

  13. Experimental results: flower sequence

  14. Experimental results • Two motions • Camera panning to the right • Car translating to the right http://www.cs.otago.ac.nz/research/vision/Research/

  15. Experimental results • Transparent motion • Outline of a hand behind a lace curtain

  16. Conclusions • There is an analytic solution to affine motion segmentation based on • Multibody affine constraint: segmentation independent • Polynomial factorization: linear algebra • Solution is closed form iff n<5 • A similar technique also applies to • Eigenvector segmentation: from similarity matrices • Generalized PCA: mixtures of subspaces • 3-D motion segmentation: of fundamental matrices • Future work • Reduce data complexity, sensitivity analysis, robutness

More Related