1 / 37

Using Strong Shape Priors for Multiview Reconstruction

Using Strong Shape Priors for Multiview Reconstruction. Yunda Sun Pushmeet Kohli Mathieu Bray Philip HS Torr. Department of Computing Oxford Brookes University. Objective. Images Silhouettes. Parametric Model. +. Pose Estimate Reconstruction. [Images Courtesy: M. Black, L. Sigal].

issac
Download Presentation

Using Strong Shape Priors for Multiview Reconstruction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Strong Shape Priors for Multiview Reconstruction Yunda Sun Pushmeet Kohli Mathieu Bray Philip HS Torr Department of Computing Oxford Brookes University

  2. Objective Images Silhouettes Parametric Model + Pose Estimate Reconstruction [Images Courtesy: M. Black, L. Sigal]

  3. Outline • Multi-view Reconstruction • Shape Models as Strong Priors • Object Specific MRF • Pose Estimation • Results

  4. Outline • Multi-view Reconstruction • Shape Models as Strong Priors • Object Specific MRF • Pose Estimation • Results

  5. Multiview Reconstruction Need for Shape Priors

  6. Multiview Reconstruction • No Priors • Silhouette Intersection • Space Carving • Weak Priors • Surface smoothness • Snow et al. CVPR ’00 • Photo consistency and smoothness • Kolmogorov and Zabih [ECCV ’02] • Vogiatzis et al. [CVPR ’05] [Image Courtesy: Vogiatzis et al.]

  7. Outline • Multi-view Reconstruction • Shape Models as Strong Priors • Object Specific MRF • Pose Estimation • Results

  8. Shape-Priors for Segmentation • OBJ-CUT [Kumar et al., CVPR ’05] • Integrate Shape Priors in a MRF • POSE-CUT [Bray et al., ECCV ’06] • Efficient Inference of Model Parameters

  9. Parametric Object Models as Strong Priors • Layered Pictorial Structures • Articulated Models • Deformable Models

  10. Outline • Multi-view Reconstruction • Shape Models as Strong Priors • Object Specific MRF • Pose Estimation and Reconstruction • Results

  11. Object-Specific MRF

  12. Object-Specific MRF Energy Function Shape Prior Unary Likelihood Smoothness Prior x:Voxel label θ: Model Shape

  13. Object-Specific MRF Shape Prior : shortest distance of voxel i from the rendered model x:Voxel label θ: Model Shape

  14. Object-Specific MRF Smoothness Prior Potts Model x:Voxel label θ: Model Shape

  15. Object-Specific MRF Unary Likelihood For a soft constraint we use a large constant K instead of infinity x:Voxel label θ: Model Shape : Visual Hull

  16. Object-Specific MRF Energy Function Shape Prior Unary Likelihood Smoothness Prior Can be solved using Graph cuts [Kolmogorov and Zabih, ECCV02 ]

  17. Object-Specific MRF Energy Function Shape Prior Unary Likelihood Smoothness Prior How to find the optimal Pose?

  18. Outline • Multi-view Reconstruction • Shape Models as Strong Priors • Object Specific MRF • Pose Estimation • Results

  19. Inference of Pose Parameters Rotation and Translation of Torso in X axes Rotation of left shoulder in X and Z axes

  20. Inference of Pose Parameters Let F(ө) = Minimize F(ө) using Powell Minimization Computational Problem: Each evaluation of F(ө) requires a graph cut to be computed. (computationally expensive!!) BUT.. Solution: Usethe dynamic graph cut algorithm [Kohli&Torr, ICCV 2005]

  21. Outline • Multi-view Reconstruction • Shape Models as Strong Priors • Object Specific MRF • Pose Estimation • Results

  22. Experiments • Deformable Models • Articulated Models • Reconstruction Results • Human Pose Estimation

  23. Deformable Models Visual Hull • Four Cameras • 1.5 x 105 voxels • DOF of Model: 5 Our Reconstruction Shape Model

  24. Articulated Models

  25. Articulated Models • Four Cameras • 106 voxels • DOF of Model: 26 Camera Setup Shape Model

  26. Articulated Models • 500 function evaluations of F(θ) required • Time per evaluation: 0.15 sec • Total time: 75 sec Let F(ө) =

  27. Articulated Models Visual Hull Our Reconstruction

  28. Pose Estimation Results Visual Hull Reconstruction Pose Estimate

  29. Pose Estimation Results • Quantitative Results • 6 uniformly distributed cameras • 12 degree (RMS) error over 21 joint angles

  30. Pose Estimation Results • Qualitative Results

  31. Pose Estimation Results Video 1, Camera 1

  32. Pose Estimation Results Video 1, Camera 2

  33. Pose Estimation Results Video 2, Camera 1

  34. Pose Estimation Results Video 2, Camera 2

  35. Future Work • Use dimensionality reduction to reduce the number of pose parameters. • results in less number of pose parameters to optimize • would speed up inference • High resolution reconstruction by a coarse to fine strategy • Parameter Learning in Object Specific MRF

  36. Thank You

  37. Object-Specific MRF Energy Function Shape Prior Unary Likelihood Smoothness Prior +

More Related