Dynamical Statistical Shape Priors for Level Set Based Tracking

1. Dynamical Statistical Shape Priors for Level Set Based Tracking Dheeraj Singaraju Reading group: 07/27/06

2. Segmentation using shape priors • In the past, level set based segmentation methods have incorporated statistical shape knowledge • Statistically learned shape information can deal with missing information or misleading information due to noise, clutter or occlusion • Such segmentation frameworks can be applied to tracking objects across frames

3. The need for dynamic shape priors • All the previous approaches in tracking assume statistical shape priors that are static in time • In silhouette data corresponding to a walking person, certain poses become less / more likely with time • We need to exploit the temporal coherence in the poses of a human during the course of an entire action

4. Aim of the paper • This paper deals with learning dynamical statistical shape models for implicitly represented shapes to track human motion • Such an approach segments an image based on its intensities and the segmentations obtained in the previous image frames • The resulting approach is expected to perform better in the presence of noise and clutter than the previous mentioned approaches, due to the temporal shape consistency being enforced.

5. Features of the proposed algorithm • Implicit shape representation: The method can deal with shapes of varying topology • Intensity based segmentation: The tracking scheme is region based as opposed to being edge based • Optimization using gradient descent: Facilitates an extension to data in higher dimension

6. Implicit representation of hyperspaces • The level set method helps propagate hyperspaces in a domain by evolving an appropriate embedding function , where • Advantages of such a representation • The boundary representation does not depend on any parameterization • Topological changes such as merging and splitting can be dealt with • One can generalize the framework easily to a higher dimensional data

7. Defining terms to work with • Shape( ): A set of closed 2D contours modulo a certain transformation • Transformations( ): They are essentially problem dependent and can be rigid body transformations, similarity or affine transforms, etc • The object of interest is thus • We want to model the temporal evolution of the shapes separately from that of the transformations

8. The Bayesian inference problem • Assume we are given consecutive images from an image sequence where denotes the set of images • We then want to maximize the following conditional probability Does not depend on the estimated quantities temporal consistency goodness of segmentation

9. Assumptions to make life easier • The images are assumed to be mutually independent • The intensities of the shape and the background are modeled as independent samples from two Gaussian distributions with unknown means and unknown variances

10. Evaluating the Gaussian models • We use the Heaviside step function to denote the areas where is positive or negative

11. Simplification of distribution • To avoid computational burden, it is assumed that the distribution of previous states of and to be strongly peaked around the maxima of the respective distribution • If we assume that the tracking system does not stored the previous images but only the past estimates of shape and transformation, then the inference problem reduces to

12. Distributions for temporal evolutions • We now need to evaluate • We break down the analysis into two cases • We assume that the shape and transformation are mutually independent and assume a uniform prior on the transformation parameters • We consider the joint distribution of the shape and transformation parameters

13. Shapes and eigenmodes • It is known that statistical models can be estimated more reliably if the dimensionality of the model and data are low • The Bayesian inference problem is now formulated in low dimensions within the subspace spanned by the largest principal eigenmodes of a set of a set of sample shapes • The training sequence is therefore used for the following • Extraction of eigenmodes of the sample shapes • Learning of dynamical models for the low dimensional representation of the implicit shapes

14. Projection into the space of shapes • If is a temporal sequence of training shapes. We denote the mean shape as and the n most significant eigenmodes as • Given the mean and n most significant eigenmodes, we can represent any arbitrary shape by a shape vector as follows

15. How effective is the projection ? Actual walking sequence Approximated walking sequence using 6 eigenmodes

16. Projections instead of the actual shape • Now we work with the parameters which are believed to be synonymous with the segmentations • We therefore need to maximize the following conditional probability Contribution of this paper

17. Temporal evolution of shapes • The paper proposes to learn the temporal dynamics by modeling the shape vectors by a Markov chain of order k • The probability of a shape conditioned on the shapes at the previous times is then modeled as DOUBT Dependent on time ?

18. Effectiveness of predicting evolution • The paper considers 151 frames of a training sequence and estimates the parameters of a second order autoregressive model • These model parameters can then be used to synthesize walking sequences

19. Synthesis of walking sequence • The synthesized sequences captures most of the characteristic motion of the walking person • Discrepancies in the shapes are due to a great reduction in the number of parameters used to describe the motion

20. Coupling the shape and transformation parameters • In general, one can expect the deformation parameters and the transformation parameters to be coupled • We want to learn dynamical models that are invariant to rotation, translation or any other desired transformations • Instead of the absolute transformation, we consider the incremental transformation between frames • We deal with a new vector as rather than a shape vector

21. Optimization using gradient descent • Given an image and a set of previously segmented shape parameters and transformation parameters the goal is to maximize the conditional probability with respect to • This goal can be achieved by minimizing the following energy function Relative weighting between the shape prior and the data term

22. A close look at the energy function • The data term can be written as • The shape term can be written as

23. Updating the shape • Update equation with respect to the shape parameters separation of image intensities draws the shape towards a prior

24. Updating the transformation parms • Update equation with respect to the transformation parms draws the shape towards the most likely transformation draws the shape towards a prior

25. Results Noisy images of a man walking Segmentation using static shape priors, 25%noise Segmentation using static shape priors, 50%noise

26. Results (contd.) Segmentation using dynamic shape priors, 50% noise Segmentation using dynamic shape priors, 75% noise

27. Results (contd.) Segmentation using dynamic shape priors, 75% noise

28. Results (contd.)

29. Results (contd.) • Comparison of algorithm with ground truth

30. Results (contd.) • Invariance to speed • To increase speed – removed frames • To decrease speed – replicated and inserted frames