Temporally Coherent Completion of Dynamic Shapes

1. Temporally Coherent Completion of Dynamic Shapes  Hao Li, LinjieLuo, Daniel Vlasic, Pieter Peers, Jovan Popović, Mark Pauly, SzymonRusinkiewicz ACM Transactions on Graphics 31(1) , ( to be presented at SIGGRAPH 2012) Presenter: Kangxue Yin Apr. 20, 2012

2. outline • Introduction and Related Work • The proposed Method: • Pairwise correspondence • Hole filling • Temporal filtering • Detail resynthesis • Results and Conclusion

3. Introduction and Related Work The problem: How to obtain temporally coherent and watertight 3D meshes from scan sequences of dynamic performances? Related topics： Space-time reconstruction Non-rigid registration

4. Introduction and Related Work • hole filling techniques JU, T. 2009. Fixing geometric errors on polygonal models: A survey. Journal of Computer Science and Technology 24, 1, 19–29. • Non-rigid registration: • template-based method • articulation-based method • consensus skeleton fails with drastic topology changes

5. The proposed Method: overview Shape completion

6. The proposed Method: Pairwise Correspondence A two-scale registration: Coarse-scale and fine-scale. Coarse-scale Registration Based on Deformation Graphs [Li et al. 2009]: a non-rigid ICP algorithm

7. The proposed Method: Pairwise Correspondence Fine-scale alignment a non-rigid locally weighted ICP algorithm based on [Brown and Rusinkiewicz 2007]

8. The proposed Method: Pairwise Correspondence

9. The proposed Method: Hole filling visual Hull prior [Vlasic et al. 2009] + weighted Poisson surface reconstruction [Kazhdan et al. 2006] Fairing: Minimizing bending energy of the patch’s vertices using bi-Laplacian

10. The proposed Method : Temporal filtering Warp two neighboring frames to current frame based on the pairwise correspondences Combine them using Poisson reconstruction with different weight for different region Poisson reconstruction warp to warp to

11. The proposed Method : Detail resynthesis 1. Acquire normal maps: Daniel Vlasic, Pieter Peers, IlyaBaran, Paul Debevec, Jovan Popović, SzymonRusinkiewicz, and WojciechMatusik. Dynamic Shape Capture using Multi-View Photometric Stereo. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 28(5), December 2009. 2. Resynthesize high frequency detail from normal maps: Diego Nehab, SzymonRusinkiewicz, James Davis, and Ravi Ramamoorthi. Efficiently Combining Positions and Normals for Precise 3D Geometry. ACM Transactions on Graphics (Proc. of ACM SIGGRAPH 2005) 24(3), August 2005.

12. The proposed Method: summary Shape completion

13. The proposed Method: Results and Conclusion

14. The proposed Method: Results and Conclusion Contribution: the framework and some little improvements on previous algorithms ( too detailed ) Limitation: each frame should cover most part of the object surface ( limited to multi-view scans) Topology change problem was not solved fundamentally Future work: To take physical properties into account

15. My Plan • registration of dynamic shape with drastic topology changed • Reconstruction of interacting objects. interacting objects topology change