190 likes | 350 Views
3D Scene Reconstruction by Stereo Methods for Analysis and Visualization of Sports Scenes. Margrit Gelautz, Michael Bleyer, Danijela Markovic and Christoph Rhemann Email: gelautz@ims.tuwien.ac.at Institute for Software Technology and Interactive Systems Vienna University of Technology, Austria.
E N D
3D Scene Reconstruction by Stereo Methods for Analysis and Visualization of Sports Scenes Margrit Gelautz, Michael Bleyer, Danijela Markovic and Christoph Rhemann Email: gelautz@ims.tuwien.ac.at Institute for Software Technology and Interactive Systems Vienna University of Technology, Austria
Dagstuhl, Sept 7-10, 2008 Stereo Analysis • Principles of stereo analysis - Scene is observed by two cameras. - The key task in stereo analysis is the automated identification of corresponding points between the two input images (stereo matching problem). - The result of stereo analysis is a depth map.
Dagstuhl, Sept 7-10, 2008 Stereo Matching Algorithm • Bleyer, M. and M. Gelautz, A Layered Stereo Matching Algorithm Using Image Segmentation and Global Visibility Constraints, ISPRS Journal of Photogrammetry and Remote Sensing, vol. 59, no. 3, pp. 128-150, 2005. (Helava Award – best paper of the year). Left image Computed depth map Ground truth Right image Computed layers Error map
Dagstuhl, Sept 7-10, 2008 Stereo Applications • Reconstruction of 3D shape of human body and posture • Example: Breakdance • Robotics and autonomous navigation • Novel view generation • Augmented and mixed reality (e.g. merging real with synthetic video content) • 3D (autostereoscopic) displays • …
Dagstuhl, Sept 7-10, 2008 Soccer Scene – Stereo Analysis • Original images taken by stereo camera Left image Right image
Dagstuhl, Sept 7-10, 2008 Soccer Scene – Novel View Synthesis • The stereo-derived depth map is computed and used to generate novel virtual views of the scene. Depth map Synthesized view
Dagstuhl, Sept 7-10, 2008 Video Object Segmentation (1) • Automated extraction of video objects based on motion analysis and clustering of motion vectors. MPEG test sequence “Mobile & Calendar”
Dagstuhl, Sept 7-10, 2008 Video Object Segmentation (2) • Automated segmentation of video object „Ball“. Overlay of segment borders. Extracted video object.
Dagstuhl, Sept 7-10, 2008 Video Object Segmentation (3) • Video Compositing
Dagstuhl, Sept 7-10, 2008 Stereo-based Visualization (1) • Stereo pair, depth and contour extraction Left image Right image Stereo depth map Significant contours
Dagstuhl, Sept 7-10, 2008 Stereo-based Visualization (2) • Frame sequence
Dagstuhl, Sept 7-10, 2008 Motion Tracking Tracking points Tracking path Curve fitting Multiple motion lines
Dagstuhl, Sept 7-10, 2008 Comic-like Motion Style (1)
Dagstuhl, Sept 7-10, 2008 Comic-like Motion Style (2)
Dagstuhl, Sept 7-10, 2008 Comic-like Motion Style (3)
References • Bleyer, M. and M. Gelautz, Graph-cut Based Stereo Matching Using Image Segmentation with Symmetrical Treatment of Occlusions, Signal Processing: Image Communication(Special Issue on Three-dimensional Video and Television), vol. 22, issue 2, pp. 127-143, 2007. • Markovic, D. and M. Gelautz, Comics-like Motion Depiction from Stereo, Proceedings of WSCG 2006, Plzen, Czech Republic, pp. 155-160, 2006. • Bleyer, M., C. Rhemann, and M. Gelautz, Segmentation-based Motion with Occlusions using Graph-cut Optimization, Proceedings of 28th Annual Symposium of the German Association for Pattern Recognition (DAGM 2006), Lecture Notes in Computer Science (LNCS) 4174, pp. 465–474, 2006.