1 / 41

Example Based 3D Shape Completion

Example Based 3D Shape Completion. Mark Pauly 1,2 , Niloy J. Mitra 1 , Joachim Giesen 2 , Markus Gross 2 , Leonidas J. Guibas 1. 1 Stanford University. 2 ETH, Zurich. Incomplete raw scans Imperfect range scanned data Complex objects with occluded regions

kanoa
Download Presentation

Example Based 3D Shape Completion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Example Based 3D Shape Completion Mark Pauly1,2, Niloy J. Mitra1, Joachim Giesen2, Markus Gross2, Leonidas J. Guibas1 1 Stanford University 2 ETH, Zurich

  2. Incomplete raw scans Imperfect range scanned data Complex objects with occluded regions Misalignment of multiple-views depth image scans Specular highlights Shape Completion Ill-posed problem Use prior knowledge !!

  3. Diffusion (Davis, Marschner, Garr, Levoy, 2002)

  4. Context Based Solution (Sharf, Alexa, Cohen-or, 2004)

  5. Template based Solution (Allen, Curless, Popovic, 2003; Kraevoy and Sheffer, 2005)

  6. Our Solution • Use 3D model database to provide geometric priors for shape completion • Apply non-rigid transforms on the models • More deformation  less likely completion • Consistently combine geometric information from multiple context models • Final result comes with confidence values

  7. Shape Completion Pipeline

  8. Shape Completion Pipeline

  9. Data Classification • Local analysis • quality of fit • uniformity of sample distribution High • Scored Point Cloud • confidence value assigned to each point Low

  10. Shape Completion Pipeline

  11. 1.93 1.71 1.46 1.27 1.0 Database Retrieval

  12. Shape Completion Pipeline

  13. Non-rigid Alignment • Similar to the approaches proposed by: • Allen, Curless and Popovic, 2003. • Sumner and Popovic, 2004.

  14. Non-rigid Alignment • Deformation Model • Piecewise linear.Each vertex of the mesh assigned an independent displacement vector. • Optimize for smallest Shape Matching Penalty • Distortion Measure • Geometric Error • Feature Correspondence Derived in the continuous setting to allow consistent comparison between different context models.

  15. Warped Models Low Context Model Warped Model Matching Penalty High

  16. Non-rigid Transform

  17. Shape Completion Pipeline

  18. Input Data Warped Context Model Initial Segmentation

  19. Initial Segmentation Final Segmentation Patch Growing

  20. Video

  21. Shape Completion Pipeline

  22. Blending Final Segments

  23. Result

  24. Shape Completion Pipeline

  25. Shape Completion Pipeline

  26. Context Models Deformed Models Segmentation Giraffe Example

  27. Context Models Deformed Models Final Model Giraffe Example

  28. Evaluation Input Data Context Model Final Model Evaluation

  29. Enriching the Database

  30. Physical Model Acquired Data Context Model Symmetry Constraints No Constraints Additional Constraints

  31. Galleria dell’ Accademia

  32. Galleria dell’ Accademia

  33. Galleria dell’ Accademia

  34. Galleria dell’ Accademia

  35. Future Directions • Improve the retrieval stage. • Automatic feature point detection. • Use of more semantic information. • Apply learning techniques to shape completion. • Completion of additional attributes like surface texture, color.

  36. Acknowledgements • NSF grants CARGO-0138456, ITR-0205671, FRG-0454543, ARO grant DAAD19-03-1-033. • Stanford Graduate fellowship. • Mario Botsch, David Koller, Doo Young Kwon, Marc Levoy, Filip Sadlo, Vin da Silva, and Bob Sumner.

  37. Questions

  38. High Warped Model Input Data Low Correspondence Invalid Valid

  39. Video

  40. Example-based image completion [Drori et al. 2003; Jia and tang 2003; Sun et al. 2003] Texture synthesis [Efros and Leung 1999; Efros and Freeman 2001; Wei and Levoy 2000; Ying et al. 2001] Smooth surface completion [Curless and Levoy 1996; Davis et al. 2002; Ilic and Fua 2003; Verdera et al. 2003; Liepa 2003] Model-based surface reconstruction [Savchenko et al. 2002] Curve analogies [Hertzmann et al. 2002] Related Works Drori et al. 03 Wei and Levoy 00 Liepa 03 Hertzmann et al. 02

More Related