1 / 21

Combining Approximate Geometry with VDTM – A Hybrid Approach to 3D Video Teleconferencing

SIBGRAPI 2002. Combining Approximate Geometry with VDTM – A Hybrid Approach to 3D Video Teleconferencing Celso Kurashima 2 , Ruigang Yang 1 , Anselmo Lastra 1 1 Department of Computer Science University of North Carolina at Chapel Hill 2 Laboratório de Sistemas Integráveis - LSI

chuck
Download Presentation

Combining Approximate Geometry with VDTM – A Hybrid Approach to 3D Video Teleconferencing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SIBGRAPI 2002 Combining Approximate Geometry with VDTM – A Hybrid Approach to 3D Video Teleconferencing Celso Kurashima2, Ruigang Yang1, Anselmo Lastra1 1Department of Computer Science University of North Carolina at Chapel Hill 2Laboratório de Sistemas Integráveis - LSI Escola Politécnica da Universidade de São Paulo Fortaleza, October 8th, 2002

  2. Introduction

  3. Introduction Video Conference 2D vs. 3D

  4. Outline • 3D Video Teleconference System • Geometry Extraction • Rendering Images with VDTM • Results • Conclusion

  5. 3D Video Teleconference System (1/3) • Top View (A) (B)

  6. 3D-Video Teleconference System (2/3)

  7. 3D-Video Teleconference System (3/3) • How does it work? • 1st: Create a geometry proxy of the person • 2nd: Texture map images onto the proxy with VDTM : an IBR method developed by Debevec et al.(’96, ’98)

  8. Geometry Proxy Extraction (1/5) • The proxy : a simple geometric representation of the person • Mesh of triangles • Map textures onto the triangles faces • Extraction: a pair of cameras • May be two of texture cameras

  9. Geometry Proxy Extraction (2/5) • Algorithm- Plane + Parallax (Kumar, 94) method: • Robust Plane Fitting • Stereo Feature tracking

  10. Geometry Proxy Extraction (3/5) • Robust Plane Fitting • Segmentation • Points on the Silhouette • Matching • Fit Plane • Distance • Std. Dev. • Remove distant points • Repeat

  11. Geometry Proxy Extraction (4/5) Stereo Feature TrackingKLT tracker (Kanade-Lucas-Tomasi, ‘91, ‘94)

  12. Geometry Proxy Extraction (5/5) Triangulation & Proxy formation + =

  13. Rendering Images with VDTM – View Dependent Texture Mapping (1/2) • VDTM requires a good spatial geometry of the objects (Debevec98) • Our system [ Buehler et al (2001) and Heigl et al (1999) ] Spatial geometry == proxy Textures == live images (from cameras) • Textures are mapped onto the proxy according to the viewpoint

  14. Rendering Images with VDTM – View Dependent Texture Mapping (2/2) • Virtual Camera at Viewpoint D • Texture from cameras Ci mapped onto the triangles faces • Blending weights in vertex V • Angles qi, used to compute the weights values wi = exp(-qi2/2.s2)

  15. Results (1/3) • Geometry Proxy & Image rendering with VDTM • Geometry Proxy • Image rendered with 2 texture cameras • Image rendered with 4 texture cameras

  16. Results (2/3) • Cameras: Firewire IEEE 1394 SONY • Frame size: 320x240 pixels • Geometry/Renderer PC • Processor: Intel Pentium4, 2.2 GHz • Graphics card: nVidia GeForce3 • Video Frame Rate: 3-4 fps

  17. Results (3/3) • 3D Video Teleconferencing Movie

  18. Conclusions • A hybrid system for 3D Video Teleconferencing • Fast geometry proxy extraction, using a robust plane fitting method and stereo feature tracking, combined with view-dependent texture mapping • Real-time demonstration with personal computers and commodity graphics card

  19. Future Work • Vision-based head-tracking for viewpoint control • Increase of the frame rate with faster segmentation

  20. Acknowledgements • Herman Towles • Office of the Future (OOTF) group at UNC-CH • U.S. Dept. of Energy and Sandia National Labs. • U.S. National Science Foundation • NEC/CPDIA

  21. Thank you! http://www.cs.unc.edu/Research/ootf/ http://www.lsi.usp.br/~kurash/

More Related