Stereo 3D Technologies in Computing, Entertainment and Art

1. Stereo 3D TechnologiesinComputing, Entertainment and Art A Reminder of Today and a Look at Tomorrow by Peter Calvache petercalvache@gmail.com University of Applied Arts Vienna

2. The 3D Experience You attempt to test the 3D monitor capabilities and search the net for appropriate images with little success. Everything seems to be geared for conventional displays. Even general-purpose software products don't use these facilities. You finally locate suitable image files but have misplaced the 3D glasses. At last you find them. Stunning 3D scenes appear to reach out towards you. You want to interact with them but interaction using keyboard and mouse is far from easy. They are unsuitable for navigation in 3D space. And now you notice the image quality of the new display in 2D mode is not as good as your old monitor either. You consign it to the attic and your conventional display is again your window into the digital world.Dr. Barry BlundellBritish Computer Society

3. 3D for Computing • Initial target audience is used to interpreting3D spatial relationships from 2D images • PC Enthusiasts, Technicians, Engineers, ... • Common preconceptions about 3D displays • At best, perceptive augumentation • At worst, distraction • Scarce compatibility • More trouble than it‘s worth • Overpriced • Not necessary • Greatest competition from high quality 3D images on 2D displays • „It looks real enough to me!“

4. 3D for Computing • Computer interfaces designed around 2D paradigms • Approximation of paper • Easiest to comprehend • Experiments with 3D user interfaces • Gimmicky? • No increase in productivity? • 3D not useful for user interface technology?

5. 3D for Computing „This 3D monitor is only good if you can stand wearing the glasses every time you work in 3D. Thanks to the doubled-up internal LCDs, the regular Windows desktop looks fuzzy when compared to my Dell 19-inch LCD. Good for 3D, so-so for 2D.“gizmodo.com iZ3D 17“ 3D LCD

6. 3D for Computing • Banal but important point • 3D is only useful for displaying natively 3D content „mapping“ natively 2D content into 3D space for visual attractionis unnecessary, distracting and very consciously seen as such • Most content produced for the screen will always be rooted in 2D • Web Browsing • Word Processing • Text Data Editing • Tabular Data Editing • Graphical Data Editing (excl. 3D Graphics) • Multimedia Production (excl. 3D Media) • The customer expects to be able to display 2D data in perfect quality A good product can have no drawbacks while in 2D mode

7. The ideal 3D Monitor for the PC • Customers are less and less interestedas these requirements are not met, with good reason! • High-Resolution capability on par with native 2D displays • 2D display quality on par with native 2D displays • Hassle-free installation (not more than a driver) • Compatibility with a huge range of existing 3D media and applications • Intuitive compatibility (this program is 3D, so it will display as 3D!) • No glasses required (Vanity!) • Affordable (???) • All of these points, save for the last, are nearly achievable ☺

8. The Compatibility Hurdle • Very few programs, if any, are compatiblewith 3D monitor libraries such as Philips WOWvx • Very hard to overcome, virtually impossible in the short run No market share? No incentive to develop for WOWvx! Viscious Cycle No programs for WOWvx? No customer interest!

9. The Compatibility Hurdle „Although 3D technology poses a myriad of benefits and potential, the infrastructure to support this industry is extremely fragmented and embryonic. While there are numerous companies working on bringing this technology to the mainstream, there is no central platform coordinating this sector.“ U.S. Display Consortium “What is remarkable is that each application has developed its own solutions, culture and technology with little awareness of other applications with similar needs and technology.” Insight Media

10. The Compatibility Hurdle • Solution • Make programs compatible automatically (no developer intervention!) • OpenGL or DirectX libraries used for most 3D applications. Take advantage of that! • nVidia Stereo Driver • Works on nearly all OpenGL or DirectX 3D applications and gamesincluding legacy applications and games! • Works on nearly all nVidia Graphics Cardsincluding legacy cards! • Compatible with select 3D equipmentfrom Sharp, Asus, SeeReal, Dimension Techn. (requires 3D Glasses) • Inconsistent support from nVidia (Requires year-old video drivers)

11. The 3D Monitor Killer App • Example: Next-Gen Philips WOWvx Monitor

12. The 3D Monitor Killer App • Next-Gen Philips WOWvx Monitor • Customer expectations satisfied: • High-Resolution capability on par with native 2D displays • 2D display quality on par with native 2D displays • Hassle-free installation (not more than a driver) • Compatibility with a huge range of existing 3D media and applications • Intuitive compatibility (this program is 3D, so it will display as 3D!) • No glasses required (Vanity!) • Affordable (Eventually!)

13. To Capture the Customer Imagination • 3D media must not be hard to come by • 3D media should be producible by the customer • Consider a stereoscopic camera phone (!)

14. Groups and Consortia • „Meant to be Seen“ (www.mtbs3d.com) • Stereoscopic 3D certification and advocacy group • Stereoscopic 3D programming guide for video game developers • Program to certify 3D quality and compatibility • “Interactive Digital Center” (www.eonreality.com/idc) • Consortium made up of nVidia, Christie Digital, HP and EON • Raising awareness of interactive 3D visualization • Establishing a de facto standard for real time visualization solutions • „Imaging and Display Research Group“ • Developing laser-based 3D rear projection display systems

15. Stereoscopic 3D in Entertainment and Art

16. History of 3D Cinema • First known 3D movie in 1903 • 3D „fad“ in 1953 (61 3D films shown!) • Episodic resurgence • In the 70s due to 3D adult films • In the 90s due to 3D theme park attractions [Insight Media]

17. 3D Cinema • 3D screens attracted ~2.5x more customers on opening weekendswhile 2D screens were also available [Screen Digest] • Cinemas who do not invest may be losing out to rival theaters • Long term survival of 3D Cinema depends on • Quality of the content attracts audiences • Wide enough screenbase to justify production of 3D features attracts producers

18. Stereoscopic Filming • „Native“ process • Depth of Field Problem • Multiple focal planes create „blurry“ regions • Age-old artistic device of photographers • Distraction during stereoscopic viewing • Looks unnatural during 3D viewing • May cause eye strain and headaches • Ideal stereoscopic camera: Infinite depth of field • Small frame or sensor (diminished quality) • Narrow aperture • Current film equipment • Large 35mm frame (for narrow D-O-F), even larger on IMAX! (70mm) • Wide aperture (more flexible during lighting) • Ideal can be achieved for CG animated movies!

19. Stereoscopic Filming • Artistic limitations impossible to enforce? • No shallow depth of field? • No telescopic lenses? • No close-up shots? “We learned that you can't use the same filmmaking process. The shots can't be too close up, or the shots have to be longer sometimes.” Tim CheungHead of character animationDreamWorks Animation

20. 3D Conversion of 2D Content • Recent examples (partial conversions) • Superman Returns • Harry Potter and the Order of the Phoenix • For shots with heavy movement very time consuming • Can only achieve basic effect • For feature films it‘s heavily criticized! “It looked blurrier than Polar Express 3D and about as good as the old colored-glasses films. The scenes weren't designed for 3D so it becomes a distraction. It gets to the big action scene and you have put your glasses on. Suddenly you can't see Superman's face clearly and the screen looks smaller and faded. I was actually wishing it wasn't in 3D.” www.geocities.com/outlawvern (on Superman Returns)

21. Digital 3D Cinema • More advancedthan earlier renditions of 3D cinema • Noreported side effects (eye strain, headaches, ...) • Can be installed at anytheater on top of basic d-cinema equipment • Various methods exist for displaying digital 3D content • Once converted into digital 3D, a film can be shown in any 3D formats guaranteed one-time premium during production • Real D (leading provider) • Cooperation with Doremi, Kodak, QuVis, GDC, Sony, XDC, Dolby, ...

22. 3D Cinema Adoption (US) • 34 cinema chains have 3D equipment (H1 2007) • 639 digital 3D screens (H1 2007) [Screen Digest] • 10 biggest chains own ~1900 cinema sites ~20000 screens most large cinema sites have a digital 3D screen • Wide cinema releases total ~4000 screens • 3D screens in H1 2007 capable of handling ~10% of a wide release

23. 3D Cinema Adoption (International) • About ~150 Digital 3D Screens • International territories severely stalling behind the US • Leading countries: • Germany, South Korea, Australia

24. 3D Cinema Adoption (Future) „ 5000 digital 3D screenswill be installed in the U.S.by May 2009 “ Joshua GreerPresident of Real D • ~1200 digital 3D screens by November 2007 [Hollywood Reporter] • for release of „Beowulf“ (dir. Robert Zemeckis) • enough 3D screens by 2009 to lauch Avatar in 3D only[Insight Media] • (dir. James Cameron)

25. Imax 3D • First large-scale commercial 3D cinema technology • Analogoue 70mm Projector • Difficult for feature-length presentations(~60 minute film roll buffer) • Slow switch to digital 3D based technology • Dual Sony 4k projectors

26. Imax 3D (Box Office Analysis)

27. Imax 3D (Box Office Analysis)

28. Imax 3D (Box Office Analysis)

29. Times Change • Roger EbertChicago Sun Times „As a movie device, 3D sucks, always has, maybe always will. The problems are: (1) It‘s pointless except when sticking things in the audience's eyes; (2) It is distracting when not pointless; and (3) It dims the colors and makes the image indistinct.“Spy-Kids 3D: Game Over (Reality Camera System 3D, 2003) „In general, I haven't been a fan of 3D, with its murky images and pathetic little cardboard glasses. But IMAX does it right, and I was astonished by how effective it is on the big screen. That made it the best 3D viewing experience I had ever had.“Polar Express (Imax Digital 3D, 2005)

30. Influence of 3D Cinema • Non-cinematic entertainment venues have relied on 3D imaging • Theme Parks, Pop Concerts, ... (More innovation in these areas) • „4D“ experiences become necessary • Physical Effects (water splashes, wind, smell, ...) • Not applicable in a cinema or home cinema context  competitive advantage • Expansion of 3D movie paradigm? • 3D trailers in front of 2D films?  impractical with 3D glasses • 2D trailers in front of 3D films?  bad marketing

31. 3D releases to watch out for • Harry Potter and the Order of the Phoenix (dir. David Yates) • Partial 2D to 3D conversion (now showing) • Beowulf (dir. Robert Zemeckis) • Native 3D, CG Animated (2007) • Star Wars Series 3D Re-Release (dir. George Lucas, et al.) • Full 2D to 3D conversion (2007 to 2008) • Battle Angel (dir. James Cameron) • Native 3D Production with the Fusion Camera System (2009) • Avatar (dir. James Cameron) • Native 3D Production with the Fusion Camera System (2009)

32. Volumetric Displays Not stereoscopic, but related!

33. Volumetric Displays • Plasma Flashpoints • National Institute of Advanced Industrial Science and Technology • Keio University • Burton Inc.

34. Volumetric Displays • Plasma Flashpoints • Emission time on the order of 1 nano-second (1 pulse for each dot) • Human recognition due to after-image effect enables 100 dots/sec

35. Volumetric Displays • Perspecta • Actuality Systems

36. Volumetric Displays • Perspecta • OpenGL Compatible!

37. Volumetric Displays • Perspecta • Bandwidth bottleneck • 1600x1200x32 bit colors at 85Hz requires 652MB/s • 3rd Dimension: 851GB/s

38. Here‘s to a good future for 3D