1 / 26

3D Tele-Collaboration over Internet2 Herman Towles, UNC-CH representing members of the National Tele-Immersion Initiati

3D Tele-Collaboration over Internet2 Herman Towles, UNC-CH representing members of the National Tele-Immersion Initiative (NTII) ITP 2002 Juan-les-Pins, France 06 December 2002. NTII Collaborators & Co-authors. University of North Carolina at Chapel Hill

arleen
Download Presentation

3D Tele-Collaboration over Internet2 Herman Towles, UNC-CH representing members of the National Tele-Immersion Initiati

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3D Tele-Collaboration over Internet2 Herman Towles, UNC-CH representing members of the National Tele-Immersion Initiative (NTII) ITP 2002 Juan-les-Pins, France 06 December 2002

  2. NTII Collaborators & Co-authors • University of North Carolina at Chapel Hill • Wei-Chao Chen, Ruigang Yang, Sang-Uok Kum, and Henry Fuchs • University of Pennsylvania • Nikhil Kelshikar, Jane Mulligan, and Kostas Daniilidis • Brown University • Loring Holden, Bob Zeleznik, and Andy Van Dam • Advanced Network & Services • Amela Sadagic and Jaron Lanier

  3. Clear Motivation to Provide • Higher Resolution • Larger, more immersive Field-of-View • Participants at Accurate Geometric Scale • Eye Contact • Spatialized Audio (Group settings) • More Natural Human-Computer Interfaces

  4. Related Work • Improved Resolution & FOV • Access Grid – Childers et al., 2000 • Commerical, multi-channel extensions of ‘1-camera to 1-display’ • Gaze-Awareness • MONJUnoCHIE System – Aoki et al., 1998 • Blue-C Project - Kunz and Spagno, 2001-2002 • VIRTUE Project – Cooke, Kauff, Schreer et al., 2000-2002 • 3D Reconstruction/New Novel Views • CMU’s Virtualized Reality Project – Narayanan, Kanade, 1998 • Visual Hull Methods – Matusik, McMillan et al, 2000 • VIRTUE Project – Cooke, Kauff, Schreer et al., 2000-2002 • Human Computer Interfaces • T-I Data Exploration (TIDE) – Leigh, DeFanti et al., 1999 • VisualGlove Project - Constanzo, Iannizzotto, 2002

  5. XTP: ‘Xtreme Tele-Presence UNC ‘Office of the Future’ Andrei State 1998

  6. Research Snapshots

  7. Presentation Outline • Motivation and Related Work • NTII Tele-Collaboration Testbed • Acquisition and 3D Reconstruction • Collaborative Graphics & User Interfaces • Rendering & Display • Network • Results • Future Challenges

  8. Scene Acquisition & Reconstruction • Foreground: Real-Time Stereo Algorithm • Frame Rate: 2-3 fps (550MHz Quad-CPU) - REAL-TIME! • Volume: 1 cubic meter • Resolution: 320x240 (15K-25K foreground points) • Background: Scanning Laser Rangefinder • Frame Rate: 1 frame in 20-30 minutes - OFFLINE! • Volume: Room-size • Resolution: More data than you can handle! Composite Live Foreground & Static Background

  9. + Real-Time Foreground Acquisition • Trinocular Stereo Reconstruction Algorithm • After background segmentation, find corresponding pixels in each image using MNCC method • 3D ray intersection yields pixel depth • Median filter the disparity map to reduce outliers • Produce 320x240 Depth Maps (1/z, R,G,B) = Images courtesy of UPenn GRASP Lab

  10. UNC Acquisition Array Five Dell 6350 Quad-Processor Servers Seven Sony Digital 1394 Cameras – Five Trinocular Views

  11. Stereo Processing Sequence Camera Views Disparity Maps 3 Views of Combined Point Clouds Images courtesy of UPenn GRASP Lab

  12. Collaborative Graphics & User I/F

  13. Shared 3D Objects • Scene Graph Sharing • Distributed, Common Scene Graph Dataset • Local Changes, Shared Automatically with Remote Nodes • Object Manipulation with 2D & 3D Pointers • 3D Virtual Laser Pointing Device • Embedded magnetic tracker • Laser beam rendered as part of Scene Graph • One event/behavior button

  14. Rendering System Overview

  15. 3D Stereo Display • Passive Stereo & Circular Polarization • Custom Filters on Projectors • Lightweight Glasses • Silvered Display Surface • Front Projection • Usable in any office/room • Ceiling-mounted Configurations • Two Projector Stereo • 100% Duty Cycle • Brighter & No flicker • Permits multi-PC Rendering

  16. View-Dependent Rendering • HiBall 6DOF Tracker • 3D Position & Orientation • Accurate, Low latency & noise • Headband-mounted Sensor • HiBall to Eyeball Calibration • PC Network Server

  17. Rendering Configurations • One PC Configuration (Linux) • Dual-channel NVIDIA graphics • Three PC Configuration (Linux) • Separate left & right-eye rendering PCs w/NVIDIA graphics • One PC used as network interface, multicasts depth map stream to rendering PCs • Performance – 933MHz PCs & GeForce2 • Interactive Display Rates of 25-100fps • Asynchronous updates of 3D Reconstruction (2-3Hz) & Scene Graph (20Hz) • Newest Rendering Configuration 10-20X • 2.4GHz, GeForce4, Multi-Threaded, VAR Arrays

  18. Network Considerations • All Tests over Internet2 • Data Rates of ~20-75 Mbps from Armonk, NY and Philadelphia into Chapel Hill • 320 x 240 Resolution • Up to 5 Reconstruction Views per site • Frame Rates 2-3 fps • TCP/IP • Latency of 2-3 seconds typical

  19. Presentation Outline • Motivation and Related Work • NTII Tele-Collaboration Testbed • Acquisition and 3D Reconstruction • Collaborative Graphics & User Interfaces • Rendering & Display • Network • Results • Future Challenges

  20. Results ‘Roll the Tape’

  21. Summary • ‘One-on-One’ 3D Tele-Immersion Testbed • Life-size, view-dependent, passive stereo display • Interact with shared 3D Objects using a virtual laser pointer • Half-Duplex Operation today • Operation over Internet2 between Chapel Hill, Philadelphia and Armonk • Audio over H.323 or POTS

  22. Future Challenges • Improved 3D Reconstruction Quality • Larger Working Volume, Faster Frame Rates – 60 cameras • Fewer Reconstruction Errors (using structured light and adaptive correlation kernels) • Reduce System Latency and Susceptibility to Network Congestion • Pipelined architecture • Shunt Protocol (between TCP/UDP and IP layers) that allows multiple flows to do coordinated congestion control • Full Duplex Operation • Unobtrusive Operation • No headmounts, No eyeglasses!

  23. Thank You Research funded by Advanced Network and Services, Inc. and National Science Foundation (USA)

  24. UPenn Acquisition Array Fifteen Sony Digital 1394 Cameras – Five Trinocular Views

  25. System Overview

  26. 3D Data + 2D Images UPenn Philadelphia 2D Video + Audio UNC Chapel Hill UNC Chapel Hill Advanced Armonk, NY Scene Bus Data 2D Video + Audio Advanced Armonk, NY 3D Data + 2D Images 3D Data + 2D Images Past Experiments With Collaboration w/o Collaboration

More Related