1 / 74

Development of immersive real-time applications

Explore the development of immersive real-time applications, including hardware, software design, clustering, and rendering techniques for virtual environments. Learn about GPU optimization, stereo synchronization, and calibration processes to create impactful VR experiences.

gravesa
Download Presentation

Development of immersive real-time applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Development of immersive real-time applications Miroslav Andel miroslav.andel@liu.se

  2. Lectures • Development of immersive real-time applications • Online/direct rendering • Hardware • Programming • Video production • Offline rendering • Hybrid solutions Miroslav Andel - Development of immersive real-time applications

  3. Outline • Hardware • Understanding the technology • Clusters and GPUs • Warping, color matching & blending • Immersion in a virtual environment (VE) • Software design • Desktop application vs. immersive cluster application • Scalability & configuration of VEs • Frameworks • Scene graph Miroslav Andel - Development of immersive real-time applications

  4. Hardware Miroslav Andel - Development of immersive real-time applications

  5. Projectors 6x Digital Projection Titan 1920 x 1200 @ 120 Hz Frame doubling Miroslav Andel - Development of immersive real-time applications

  6. Image Generators (IGs) • Workstation high-end graphics cards • Frame sync • External house sync • Genlock (video signal sync) • Swaplock (swap buffer sync) • 3D sync (for 3D glasses) • Supports active stereoscopy using OpenGL • NvidiaQuadro or AMD/ATI FirePro • No gaming cards Miroslav Andel - Development of immersive real-time applications

  7. Gaming vs. Workstation GPU • GPU is pretty much the same but different driver software and firmware • Gaming cards are optimized for speed and DirectX • Hi frame-rate in games, highly optimized • Workstation cards are optimized for precision, OpenGL & OpenCL • Designed for 3D modeling, CAD, video editing and computation • Higher floating point and sub-pixel precision • Better quality and larger memory (12 GB in QuadroK6000, 16 GB AMD FirePro 9100) • Ray tracing on the GPU • Our problem: We use gaming techniques but need workstation features like synchronization and stereoscopy (less performance) Miroslav Andel - Development of immersive real-time applications

  8. Cluster of IGs Often consists of workstations with the same specification Master – Slave In our case the master has an additional surround sound card (live encoding to Dolby Digital or DTS) Miroslav Andel - Development of immersive real-time applications

  9. Clusters(at the visualization center) • Dome • Alpha (primary real-time cluster used for shows) • Beta (secondary development cluster) • Delta (movie and presentations playback cluster) • Gamma (single node eyefinity machine) • VR-Arena • Theta • Videowall (just a single node with 6 outputs)used for movie and presentations playback Miroslav Andel - Development of immersive real-time applications

  10. Color/gammutmatching What is white? Miroslav Andel - Development of immersive real-time applications

  11. Blending • Digital blending – good bright levels • Not a linear gradient (gamma) • Done after color and brightness matching Miroslav Andel - Development of immersive real-time applications

  12. Blending • Optical blending for better dark levels • Projected black is not black, still projected light Miroslav Andel - Development of immersive real-time applications

  13. Blending • Optical blending for better dark levels • Projected black is not black, still projected light Miroslav Andel - Development of immersive real-time applications

  14. Blending • Blending in the dome • Blue = Projection borders • Red = Blend zones Miroslav Andel - Development of immersive real-time applications

  15. Warping/geometric correction Correct the rendered frame to match the geometry of the screen, lens and projector’s position Correct overlaps between adjacent channels Spatial 2D transform Miroslav Andel - Development of immersive real-time applications

  16. Warping and blending hardware Processing unit with input & outputs cards (FPGA) with minimal latency Miroslav Andel - Development of immersive real-time applications

  17. Warping and blending using OpenGL No latency but uses a bit of the GPU Miroslav Andel - Development of immersive real-time applications

  18. Calibration process • Manual warping and blending calibration takes many hours • Few reference points in the dome • Moving mesh points by hand causes linearity problems • Semi-automated calibration • Automated camera-based systems • Current systems don’t handle optical blends Miroslav Andel - Development of immersive real-time applications

  19. Hardware (end) Miroslav Andel - Development of immersive real-time applications

  20. Immersion To give the audience the impression that they have stepped into a synthetic world Miroslav Andel - Development of immersive real-time applications

  21. Immersion • To give the audience the impression that they have stepped into a synthetic world • Enhance immersion by: • Large field of view • High resolution • Stereoscopy (can also reduce immersion) • Surround sound • Smooth navigation (based on a physical model) • Realistic rendering (shadows etc., to improve depth cue) Miroslav Andel - Development of immersive real-time applications

  22. Immersion • Stereoscopy in the dome? • The paradox • Field of view wearing 3D glasses • Use stereoscopy wisely Miroslav Andel - Development of immersive real-time applications

  23. Software design Miroslav Andel - Development of immersive real-time applications

  24. The difference(from normal desktop application development) • Several projectors/screens • A lot of pixels • Alignment, blending & warping • Color & Brightness matching • Several computers (cluster) • Synchronization • Frame • Buffers swap • Data • Video signal (genlock) • Multi-threading Miroslav Andel - Development of immersive real-time applications

  25. Stereoscopy How does it work? Miroslav Andel - Development of immersive real-time applications

  26. Stereoscopy Miroslav Andel - Development of immersive real-time applications

  27. Stereoscopy Miroslav Andel - Development of immersive real-time applications

  28. Stereoscopy Miroslav Andel - Development of immersive real-time applications

  29. Stereoscopy • Co-location • Camera position = eye position • Projection plane = real physical screen Miroslav Andel - Development of immersive real-time applications

  30. Stereoscopy Don’t change eye separation, change the scale of your scene ~15 meters limit Miroslav Andel - Development of immersive real-time applications

  31. Miroslav Andel - Development of immersive real-time applications

  32. Miroslav Andel - Development of immersive real-time applications

  33. Miroslav Andel - Development of immersive real-time applications

  34. Stereoscopy Object in focus in negative parallax Miroslav Andel - Development of immersive real-time applications

  35. Stereoscopy Object in focus in positive parallax Miroslav Andel - Development of immersive real-time applications

  36. Stereoscopy Problems with a fixed head setup (no head tracking) Miroslav Andel - Development of immersive real-time applications

  37. Scalability • One application can run in several virtual environments without the need of changing the code • In the dome • In the VR-Arena • In the VR-Lab • At home using your computer • At the bus using a laptop • If the platform is the same then there is no need for recompiling the application (just copy & paste) Miroslav Andel - Development of immersive real-time applications

  38. Scalability The application uses a configuration file that describes the hardware in terms like: Number of IGs Number of displays Number of viewports Physical location of displays/projection planes Physical location of the user Eye separation of the user Type if Stereoscopy Etc… Miroslav Andel - Development of immersive real-time applications

  39. Configuration “3d drawing”

  40. Configuration Monoscopicprojection Immersive applications Often uses non-symmetric frustums/projections, while normal OpenGL applications don’t (left FOV = right FOV, top FOV = bottom FOV)

  41. Configuration Stereoscopic projection

  42. Configuration

  43. Configuration All users except the target user will perceive the rendered object incorrectly.

  44. Configuration All users except the target user will perceive the rendered object incorrectly. One solution would be using HMDs or a multi-view screen.

  45. Navigation • In a desktop application a user navigates by moving the camera Miroslav Andel - Development of immersive real-time applications

  46. Navigation • In a desktop application a user navigates by moving the camera • In a scalable application the number of cameras/frustums is not known Miroslav Andel - Development of immersive real-time applications

  47. Navigation • In a desktop application a user navigates by moving the camera • In a scalable application the number of cameras/frustums is not known • Instead of moving the user and all “cameras” in the same direction the whole scene is moved in the opposite direction Miroslav Andel - Development of immersive real-time applications

  48. Frameworks(the good stuff) • Will help you with • Cluster setup & synchronization • Frustums & viewports • Input • Tracking Miroslav Andel - Development of immersive real-time applications

  49. Frameworks VR Juggler CAVElib (commercial) Equalizer Miroslav Andel - Development of immersive real-time applications

  50. Frameworks(the bad stuff) • Massive and have a lot of dependencies • Big initial effort to get started • Often in legacy OpenGL • Takes usually several days to get started • Getting all dependencies • Compile all dependencies • Compile the whole framework • Read the documentation Miroslav Andel - Development of immersive real-time applications

More Related