1 / 16

Spatio-temporal Segmentation of Video by Hierarchical Mean Shift Analysis

Spatio-temporal Segmentation of Video by Hierarchical Mean Shift Analysis. Daniel DeMenthon SMVP 2002. Motivation. Semantic understanding of video Object segmentation Video compression Event detection Video surveillance. Related Work. Jojic et al. Flexible sprites Layer extraction

karen-lucas
Download Presentation

Spatio-temporal Segmentation of Video by Hierarchical Mean Shift Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Spatio-temporal Segmentation of Video by Hierarchical Mean Shift Analysis Daniel DeMenthon SMVP 2002

  2. Motivation • Semantic understanding of video • Object segmentation • Video compression • Event detection • Video surveillance

  3. Related Work • Jojic et al. Flexible sprites • Layer extraction • Shi and Malik normalized cuts • Irani et al. Event detection

  4. Space – Time Volume Segmentation • Frame to Frame • Video Stack segmentation Patch motion (1,u,v)

  5. Feature Space • 7 D feature vector, three color features in CIE L*u*v*, 2 motion angles, 2 motion distances.

  6. Mapping Pixels in Feature Space

  7. Mean Shift Clustering • Introduced by Fukunaga (1990) and applied to image analysis by Yizhong Cheng and Comaniciu and Meer (1997) • Natural borders (Leung et al.)

  8. Range Search • ATRIA tree • O(N log N) for small radii • O(N) for large radii

  9. Hierarchical Mean Shift • First standard mean shift is run until competition with very small radius • Weights are assigned to cluster centers equal to the sum of the weights of the member points • Clusters are now treated as the points, and radius is multiplied with factor of 1.25 or 1.50 • Repeat until desired radius or the desired number of regions is reached

  10. “Flower Garden” Video Sequence 88 x 60, 12 frames

  11. Video Strands

  12. Color Segmentation

  13. Motion Segmentation Faster lateral motion corresponds to lighter color

  14. Comparison of Two Segmentation Algorithms

  15. Comments on this Approach • Spatially distant color patches can be clustered together • Experiment was with small number of frames • It is not clear if it can handle the case when video object changes the direction of motion, or when video object stops • All features (color features and motion features) are scaled using heuristics, and that might not work for different video sequences

  16. Conclusions and Further Work • Hierarchical mean shift analysis is of lower empirical complexity then standard mean shift analysis • Segmentation can be improved, the bounding areas of moving areas are jagged, usually by post-processing • Leung et al. suggested non parametric segmentation that can be applied here

More Related