1 / 70

Development of immersive real-time applications

Development of immersive real-time applications. Miroslav Andel miroslav.andel@liu.se. Outline. Immersion in a virtual environment (VE) Difference between development of a desktop application and an immersive application Understanding the ( complexity ) technology Hardware

airlia
Download Presentation

Development of immersive real-time applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Development of immersive real-time applications Miroslav Andel miroslav.andel@liu.se

  2. Outline • Immersion in a virtual environment (VE) • Difference between development of a desktop application and an immersive application • Understanding the (complexity) technology • Hardware • Warping, color matching & blending • Scalability & configuration of VEs • Frameworks (simplicity) • Scene graph Miroslav Andel - Development of immersive real-time applications

  3. Immersion (VE) • To give the audience the impression that they have stepped into a synthetic world • Enhance immersion by: • Large field of view • High resolution • Stereoscopy (can also reduce immersion) • Surround sound • Smooth navigation (based on a physical model) • Realistic rendering Miroslav Andel - Development of immersive real-time applications

  4. Immersion (VE) • Stereoscopy in the dome? • The paradox • Field of view wearing 3D glasses • Use stereoscopy wisely Miroslav Andel - Development of immersive real-time applications

  5. Immersion (VE) • Stereoscopy in the dome? • The paradox • Field of view wearing 3D glasses • Use stereoscopy wisely • Cybersickness in immersive environments • Nausea • Eye strain • Headache • Vomiting Miroslav Andel - Development of immersive real-time applications

  6. The difference(from normal desktop application development) • Several projectors/screens • A lot of pixels • Alignment, blending & warping • Color & Brightness matching • Several computers (cluster) • Synchronization • Frame • Data • Video signal Miroslav Andel - Development of immersive real-time applications

  7. Hardware • Image Generators (IGs) • Professional high-end graphics cards supports various types of synchronization with an additional synchronization card • Frame sync • External house sync • Genlock (video signal sync) • Swaplock (swap buffer sync) • 3D sync (for 3D glasses) • Professional high-end graphics cardssupports active stereoscopy using OpenGL • NvidiaQuadro and AMD/ATI FirePro MiroslavAndel - Development of immersive real-time applications

  8. Hardware • AMD Eyefinity – multi-display technology • Massive pixel output • Used in the VR-Arena • 3 projectors • Passive stereoscopy input toprojectors (6 outputs@60 Hz each) • Active 120 Hz stereoscopyfrom each projector MiroslavAndel - Development of immersive real-time applications

  9. Cluster of IGs Often consists of workstations with the same specification Master – Slave In our case the master has an additional surround sound card (live encoding to Dolby Digital or DTS) Miroslav Andel - Development of immersive real-time applications

  10. Clusters(at the visualization center) • Dome • Alpha (primary real-time cluster used for shows) • Beta (secondary development cluster) • Delta (movie and presentations playback cluster) • VR-Arena • Theta • Videowall (just a single node with 6 outputs)used for movie and presentations playback Miroslav Andel - Development of immersive real-time applications

  11. OpenGL projections One viewplane per projector Curved surface? Miroslav Andel - Development of immersive real-time applications

  12. Warping/geometric correction Correct the rendered frame to match the geometry of the screen, lens and projector’s position Correct overlaps between adjacent channels Spatial 2D transform Miroslav Andel - Development of immersive real-time applications

  13. Warping/geometric correction Original Warped Miroslav Andel - Development of immersive real-time applications

  14. Color matching What is white? Miroslav Andel - Development of immersive real-time applications

  15. Color matching D65 = 6500 K = (0.3127, 0.3290) D75 = 7500 K = (0.2990, 0.3149) Other white points: DCI = (0.3140, 0.3510) DICOM clear base = (0.280, 0.305) DICOM blue base = (0.250, 0.304) White points (CIE standard illuminant) D65 and D75 Miroslav Andel - Development of immersive real-time applications

  16. Color matching All display devices and projectors have a different gamut Projector lamp color temperature changes over time Must be done before blending Can be done on each IG but with less precision (8 or 10 bits per color) Better to do that in the projector (12-16 bit precision) Miroslav Andel - Development of immersive real-time applications

  17. Color matching Miroslav Andel - Development of immersive real-time applications

  18. Blending • Digital blending – good bright levels • Not a linear gradient (gamma) • Done after color and brightness matching Miroslav Andel - Development of immersive real-time applications

  19. Blending • Optical blending – good dark levels • Projected black is not black, still projected light Miroslav Andel - Development of immersive real-time applications

  20. Blending • Optical blending – good dark levels • Projected black is not black, still projected light Miroslav Andel - Development of immersive real-time applications

  21. Blending • Blending in the dome • Blue = Projection borders • Red = Blend zones Miroslav Andel - Development of immersive real-time applications

  22. Warping and blending hardware Processing unit with input & outputs cards (minimal latency) Miroslav Andel - Development of immersive real-time applications

  23. Warping and blending using OpenGL No latency but uses a bit of the GPU Miroslav Andel - Development of immersive real-time applications

  24. Warping and blending using OpenGL No latency but uses a bit of the GPU Miroslav Andel - Development of immersive real-time applications

  25. Calibration process • Manual warping and blending calibration takes many hours • Few reference points in the dome • Moving mesh points by hand causes linearity problems • Automated camera-based systems • Current systems don’t handle optical blends Miroslav Andel - Development of immersive real-time applications

  26. Scalability • One application can run in several virtual environments without the need of changing the code • In the dome • In the VR-Arena • In the VR-Lab • At home using your computer • At the bus using a laptop • If the platform is the same then there is also no need for recompiling the application Miroslav Andel - Development of immersive real-time applications

  27. Scalability The application uses a configuration file that describes the hardware in terms like: Number of IGs Number of displays Number of viewports Physical location of displays/projection planes Physical location of the user Eye separation of the user Type if Stereoscopy Etc… Miroslav Andel - Development of immersive real-time applications

  28. Configuration “3d drawing” Miroslav Andel - Development of immersive real-time applications

  29. Configuration Monoscopicprojection Immersive applications Often uses non-symmetric frustums/projections, while normal OpenGL applications don’t (left FOV = right FOV, top FOV = bottom FOV) Miroslav Andel - Development of immersive real-time applications

  30. Configuration Stereoscopic projection Miroslav Andel - Development of immersive real-time applications

  31. Stereoscopy Don’t change eye separation, change the scale of your scene ~15 meters limit Miroslav Andel - Development of immersive real-time applications

  32. Stereoscopy Miroslav Andel - Development of immersive real-time applications

  33. Stereoscopy Miroslav Andel - Development of immersive real-time applications

  34. Configuration Miroslav Andel - Development of immersive real-time applications

  35. Configuration All users except the target user will perceive the rendered object incorrectly. One solution would be using HMDs or a multi-view screen. Miroslav Andel - Development of immersive real-time applications

  36. Configuration Miroslav Andel - Development of immersive real-time applications

  37. Configuration Miroslav Andel - Development of immersive real-time applications

  38. Navigation In a desktop application a user navigates by moving the camera Miroslav Andel - Development of immersive real-time applications

  39. Navigation In a desktop application a user navigates by moving the camera In a scalable application the number of cameras/frustums is not known Miroslav Andel - Development of immersive real-time applications

  40. Navigation In a desktop application a user navigates by moving the camera In a scalable application the number of cameras/frustums is not known Instead of moving the user and all “cameras” in the same direction the whole scene is moved in the opposite direction Miroslav Andel - Development of immersive real-time applications

  41. Frameworks(the good stuff) • Will help you with • Cluster setup & synchronization • Frustums & viewports • Input • Tracking Miroslav Andel - Development of immersive real-time applications

  42. Frameworks VR Juggler CAVElib (commercial) Equalizer Miroslav Andel - Development of immersive real-time applications

  43. Frameworks(the bad stuff) • Massive and have a lot of dependencies • Big initial effort to get started • Takes usually several days to get started • Getting all dependencies • Compile all dependencies • Compile the whole framework • Read the documentation Miroslav Andel - Development of immersive real-time applications

  44. SGCTSimple Graphics Cluster Toolkit Miroslav Andel - Development of immersive real-time applications

  45. SGCTSimple Graphics Cluster Toolkit • Based on: • GLFW • GLEW • GLM • Freetype • TinyXML • PNG • Z • VRPN Miroslav Andel - Development of immersive real-time applications

  46. SGCTSimple Graphics Cluster Toolkit • Developed here and is free for everybody to use • Designed to get started immediately (rapid development) • Cross-platform C++ • Single static library • Using callbacks rather than inheritance/polymorphism • Easy porting application from GLUT, GLFW or similar Miroslav Andel - Development of immersive real-time applications

  47. SGCTSimple Graphics Cluster Toolkit • Wiki at: c-student.itn.liu.se • Documentation • Tutorials • Configuration files • Downloads • Support Miroslav Andel - Development of immersive real-time applications

  48. SGCT – How it works • Has an engine which handles • Initiation of OpenGL • Rendering • Synchronization • Network communication • Configuration handling • Input • The user can bind callbacks (function pointers) to the engine to customize the functionality MiroslavAndel - Development of immersive real-time applications

  49. SGCT – How it works Miroslav Andel - Development of immersive real-time applications

  50. SGCT – How it works Miroslav Andel - Development of immersive real-time applications

More Related