1 / 21

A Progressive Tri-level Segmentation Approach for Topology-Change-Aware Video Matting

A Progressive Tri-level Segmentation Approach for Topology-Change-Aware Video Matting. Jinlong Ju 1 , Jue Wang 3 , Yebin Liu 1 , Haoqian Wang 2 , Qionghai Dai 1 Department of Automation, Tsinghua University, China Graduate School at Shenzhen, Tsinghua University, China

race
Download Presentation

A Progressive Tri-level Segmentation Approach for Topology-Change-Aware Video Matting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Progressive Tri-level SegmentationApproach for Topology-Change-Aware Video Matting Jinlong Ju1, Jue Wang3, Yebin Liu1, Haoqian Wang2, Qionghai Dai1 Department of Automation, Tsinghua University, China Graduate School at Shenzhen, Tsinghua University, China Adobe Research, USA

  2. The goal • Interactive video object segmentation and matting Local correction Initial keyframe segmentation

  3. Previous work • 3D volume segmentation (global optimization) • Hard to converge Video object cut & paste [Li et al., SIGGRAPH’05] Interactive video cutout [Wang et al., SIGGRAPH’05]

  4. Previous work • Tracking & segmentation • Easy and intuitive workflow Frame t Frame t+1 Unbiased Directional Classifiers [Zhong et al., SIGGRAPH Asia’12] Video Object Segmentation by TrackingRegions. ICCV’09. Object Tracking and Segmentationin a Closed Loop. ISVC'10. Video SnapCut [Bai et al., SIGGRAPH’09]

  5. Previous work • Video matting Video Matting of Complex Scenes [Chuang et al., SIGGRAPH’02] Towards Temporally-Coherent Video Matting [Bai et al., Mirage’11]

  6. Main ideas Previous methods Our approach Global, one-time optimization Progressive Binary segmentation -> trimap -> matting Tri-level segmentation -> matting

  7. Framework • Frame-to-frame propagation Frame t Frame t+1

  8. Tri-level initial segmentation • Step 1: coarse object alignment • SIFT feature tracking and alignment • Optical flow Frame t-1 Frame t

  9. Initial Tri-level segmentation • Step 2: color models Frame t-1 Frame t Foreground probability: Background probability: Normalized foreground probability:

  10. Initial Tri-level segmentation • Step 3: initial labeling • If is very high, label as ; • If is very low, label as ; • If both and are very low, label as Unmatch; • Otherwise, label as Uncertain. Frame t-1 Frame t , ,

  11. Initial Tri-level segmentation • Comparison Frame t-1 Frame t Our initial labeling Color probability maps of Gaussian Mixtures

  12. Initial Tri-level segmentation • Step 4: dealing with Unmatched pixels • both and are very low Unmatched pixels Frame t Frame t+1

  13. Initial Tri-level segmentation • Step 5: local smoothing • Mean shift, same color models for each MS region Initial map Frame t Frame t+1

  14. Tri-level segmentation Refinement • Goal: remove large unknown regions Before refinement After refinement

  15. Tri-level segmentation Refinement • Cross-frame window matching + shape prior Frame t+1 Frame t

  16. Tri-level segmentation Refinement • Cross-frame window matching + shape prior Frame t+1 Frame t

  17. Final matting • Directly use tri-level segmentation as trimap (ideally) Before matting After matting

  18. Results Tri-level segmentation Final result Adobe After Effects RotoBrush

  19. Results Tri-level segmentation Final result Adobe After Effects RotoBrush

  20. Quantitative comparison

  21. Conclusion • Tri-label labeling • Progressive segmentation • Handles topology-change, fast-moving objects well • Limitations: • Segmentation is a little bit too aggressive (not ideal for matting) • A few thresholds to tune

More Related