1 / 69

3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes

3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes Wojciech Matusik, MERL Hanspeter Pfister, MERL Just like a window! 3D TV – Our Vision Immersive Unobtrusive Multi-user 3D TV – Our Goals

andrew
Download Presentation

3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3D TV: A Scalable System for Real-time Acquisition, Transmission, and Autostereoscopic Display of Dynamic Scenes Wojciech Matusik, MERL Hanspeter Pfister, MERL

  2. Just like a window! 3D TV – Our Vision • Immersive • Unobtrusive • Multi-user

  3. 3D TV – Our Goals • Capture light flowing through a “window” • Real-time transmission • Multiview autostereoscopic light field display

  4. Challenges • Bandwidth • Processing • Real-time • Multiview autostereoscopic display • End-to-end system • Automatic setup and calibration

  5. Contributions • Real-time end-to-end 3D TV system • Distributed, scalable architecture • Multiview video rendering • Computational alignment for 3D displays

  6. Emerging Technologies Exhibit

  7. Outline • Previous Work • 3D Display • System Architecture • Display Calibration • Rendering • Future Work and Conclusions

  8. Previous Work – Early Beginnings • Stereoscope [Wheatstone 1838] • Parallax stereogram [Ives 1903] • Integral display [Lippman 1908] • Parallax panoramagram [Ives 1928] • Lenticular sheets, 1930s • Hologram [Gabor 1948]

  9. Previous Work - Acquisition • Model-based systems [Kanade 97], [Gross 03], [Carranza 03] • Light-field systems [Levoy 96], [Gortler 96], [Schirmacher 01], [Yang 02], [Wilburn 02], [Naemura 02] • Multiview video compression and transmission [Fehn 02], [Magnor 03], [Ramanathan 03], [Yang 02], [Tanimoto 03], [Zitnick 04], [Smolic 03]

  10. Previous Work – Displays • Holographic displays [St.-Hillaire 95], [Maeno 96], [Kajiki 96], [Stanley 00], [Huebschman 03] • Volumetric displays [McKay 00], [Favalora 01] • Parallax displays [Nakajima 01], [Liao 02], [Moore 96], [Perlin 00] • Multi-projector displays [Raskar 98], [Li 02], [Humphreys 02]

  11. Outline • Previous Work • 3D Display • System Architecture • Display Calibration • Rendering • Future Work and Conclusions

  12. Regular Pixels • Ideally emit the same light in all directions Emitted Light

  13. View-dependent Pixels • Emit different amounts of light/color in different directions Emitted Light

  14. View-dependent Pixels Lens or Pinhole = Pixel High Resolution Screen

  15. View-dependent Pixels Emitted Light Lens or Pinhole = Pixel High Resolution Screen

  16. Rear Projection Design Lens = Pixel Semi-transparent Material Lens

  17. Rear Projection Design Lens = Pixel Semi-transparent Material Lens

  18. Rear Projection Design Emitted Light Lens = Pixel Semi-transparent Material Lens

  19. Our Rear Projection Display

  20. Front Projection Design Lens Reflective Material

  21. Front Projection Design Lens Reflective Material

  22. Front Projection Design Emitted Light Lens Reflective Material

  23. Our Front Projection Display

  24. Limitations – Field of View

  25. Limitations – Field of View

  26. Limitations – Field of View

  27. Limitations – Field of View

  28. Limitations – Discretization & Cross-talk

  29. Display Trade-offs • Horizontal and vertical parallax 3D TV • requires O(n2) bandwidth, computation, & cost • Horizontal parallax only 3D TV • requires O(n) bandwidth, computation, & cost • still produces immersive and convincing 3D experience

  30. Display Trade-offs • Horizontal and vertical parallax 3D TV • requires O(n2) bandwidth, computation, & cost • Horizontal parallax only 3D TV • requires O(n) bandwidth, computation, & cost • still produces immersive and convincing 3D experience

  31. Outline • Previous Work • 3D Display • System Architecture • Display Calibration • Rendering • Future Work and Conclusions

  32. Acquisition Compression Transmission 3D Display System Architecture

  33. Acquisition • Array (16) of hardware synchronized, calibrated, cameras • Distributed acquisition

  34. Compression & Transmission • Temporal encoding • Each stream encoded separately • Uses existing video standards (MPEG-2) • Scalable • Spatial (multiview) encoding • Not scalable • Not real-time • Both temporal & spatial encoding • Best compression

  35. Compression & Transmission • Temporal encoding • Each stream encoded separately • Uses existing video standards (MPEG-2) • Scalable • Spatial (multiview) encoding • Not scalable • Not real-time • Both temporal & spatial encoding • Best compression

  36. Compression & Transmission

  37. Distributed Display • Decoders • decode video streams • send pixel streams to consumers • Consumers • render video streams

  38. Distributed Display • Controller • decides where to send pixels • ensures data flow to each consumer is at most kx video stream (k = 3) • allows interactively changing display parameters

  39. Our System

  40. Outline • Previous Work • 3D Display • System Architecture • Display Calibration • Rendering • Future Work and Conclusions

  41. Display Calibration • Geometric calibration • Project checkerboard pattern Display Plane Calibration Camera Projector Array

  42. Display Calibration • Geometric calibration • Project checkerboard pattern • Compute homographies Display Plane Calibration Camera Projector Array

  43. Display Calibration • Geometric calibration • Project checkerboard pattern • Compute homographies • Compute intersection

  44. Display Calibration • Geometric calibration • Project checkerboard pattern • Compute homographies • Compute intersection • Compute maximum rectangle

  45. Display Calibration • Photometric Calibration • Compute minimum intensity

  46. Display Calibration • Photometric Calibration • Compute minimum intensity • Equalize intensities

  47. Outline • Previous Work • 3D Display • System Architecture • Display Calibration • Rendering • Future Work and Conclusions

  48. No Rendering

  49. Simple System Limitations • Physical alignment is impossible • No flexibility

  50. Lightfield Rendering • Unstructured Lumigraph Rendering [Buehler 01] – Blend 3 rays / pixel • Closest ray • 1 ray / pixel

More Related