740 likes | 873 Views
Development of immersive real-time applications. Miroslav Andel miroslav.andel@liu.se. Lectures. Development of immersive real-time applications Online/direct rendering Hardware Programming Video production Offline rendering Hybrid solutions. Outline. Hardware
E N D
Development of immersive real-time applications Miroslav Andel miroslav.andel@liu.se
Lectures • Development of immersive real-time applications • Online/direct rendering • Hardware • Programming • Video production • Offline rendering • Hybrid solutions Miroslav Andel - Development of immersive real-time applications
Outline • Hardware • Understanding the technology • Clusters and GPUs • Warping, color matching & blending • Immersion in a virtual environment (VE) • Software design • Desktop application vs. immersive cluster application • Scalability & configuration of VEs • Frameworks • Scene graph Miroslav Andel - Development of immersive real-time applications
Hardware Miroslav Andel - Development of immersive real-time applications
Projectors 6x Digital Projection Titan 1920 x 1200 @ 120 Hz Frame doubling Miroslav Andel - Development of immersive real-time applications
Image Generators (IGs) • Workstation high-end graphics cards • Frame sync • External house sync • Genlock (video signal sync) • Swaplock (swap buffer sync) • 3D sync (for 3D glasses) • Supports active stereoscopy using OpenGL • NvidiaQuadro or AMD/ATI FirePro • No gaming cards Miroslav Andel - Development of immersive real-time applications
Gaming vs. Workstation GPU • GPU is pretty much the same but different driver software and firmware • Gaming cards are optimized for speed and DirectX • Hi frame-rate in games, highly optimized • Workstation cards are optimized for precision, OpenGL & OpenCL • Designed for 3D modeling, CAD, video editing and computation • Higher floating point and sub-pixel precision • Better quality and larger memory (12 GB in QuadroK6000, 16 GB AMD FirePro 9100) • Ray tracing on the GPU • Our problem: We use gaming techniques but need workstation features like synchronization and stereoscopy (less performance) Miroslav Andel - Development of immersive real-time applications
Cluster of IGs Often consists of workstations with the same specification Master – Slave In our case the master has an additional surround sound card (live encoding to Dolby Digital or DTS) Miroslav Andel - Development of immersive real-time applications
Clusters(at the visualization center) • Dome • Alpha (primary real-time cluster used for shows) • Beta (secondary development cluster) • Delta (movie and presentations playback cluster) • Gamma (single node eyefinity machine) • VR-Arena • Theta • Videowall (just a single node with 6 outputs)used for movie and presentations playback Miroslav Andel - Development of immersive real-time applications
Color/gammutmatching What is white? Miroslav Andel - Development of immersive real-time applications
Blending • Digital blending – good bright levels • Not a linear gradient (gamma) • Done after color and brightness matching Miroslav Andel - Development of immersive real-time applications
Blending • Optical blending for better dark levels • Projected black is not black, still projected light Miroslav Andel - Development of immersive real-time applications
Blending • Optical blending for better dark levels • Projected black is not black, still projected light Miroslav Andel - Development of immersive real-time applications
Blending • Blending in the dome • Blue = Projection borders • Red = Blend zones Miroslav Andel - Development of immersive real-time applications
Warping/geometric correction Correct the rendered frame to match the geometry of the screen, lens and projector’s position Correct overlaps between adjacent channels Spatial 2D transform Miroslav Andel - Development of immersive real-time applications
Warping and blending hardware Processing unit with input & outputs cards (FPGA) with minimal latency Miroslav Andel - Development of immersive real-time applications
Warping and blending using OpenGL No latency but uses a bit of the GPU Miroslav Andel - Development of immersive real-time applications
Calibration process • Manual warping and blending calibration takes many hours • Few reference points in the dome • Moving mesh points by hand causes linearity problems • Semi-automated calibration • Automated camera-based systems • Current systems don’t handle optical blends Miroslav Andel - Development of immersive real-time applications
Hardware (end) Miroslav Andel - Development of immersive real-time applications
Immersion To give the audience the impression that they have stepped into a synthetic world Miroslav Andel - Development of immersive real-time applications
Immersion • To give the audience the impression that they have stepped into a synthetic world • Enhance immersion by: • Large field of view • High resolution • Stereoscopy (can also reduce immersion) • Surround sound • Smooth navigation (based on a physical model) • Realistic rendering (shadows etc., to improve depth cue) Miroslav Andel - Development of immersive real-time applications
Immersion • Stereoscopy in the dome? • The paradox • Field of view wearing 3D glasses • Use stereoscopy wisely Miroslav Andel - Development of immersive real-time applications
Software design Miroslav Andel - Development of immersive real-time applications
The difference(from normal desktop application development) • Several projectors/screens • A lot of pixels • Alignment, blending & warping • Color & Brightness matching • Several computers (cluster) • Synchronization • Frame • Buffers swap • Data • Video signal (genlock) • Multi-threading Miroslav Andel - Development of immersive real-time applications
Stereoscopy How does it work? Miroslav Andel - Development of immersive real-time applications
Stereoscopy Miroslav Andel - Development of immersive real-time applications
Stereoscopy Miroslav Andel - Development of immersive real-time applications
Stereoscopy Miroslav Andel - Development of immersive real-time applications
Stereoscopy • Co-location • Camera position = eye position • Projection plane = real physical screen Miroslav Andel - Development of immersive real-time applications
Stereoscopy Don’t change eye separation, change the scale of your scene ~15 meters limit Miroslav Andel - Development of immersive real-time applications
Miroslav Andel - Development of immersive real-time applications
Miroslav Andel - Development of immersive real-time applications
Miroslav Andel - Development of immersive real-time applications
Stereoscopy Object in focus in negative parallax Miroslav Andel - Development of immersive real-time applications
Stereoscopy Object in focus in positive parallax Miroslav Andel - Development of immersive real-time applications
Stereoscopy Problems with a fixed head setup (no head tracking) Miroslav Andel - Development of immersive real-time applications
Scalability • One application can run in several virtual environments without the need of changing the code • In the dome • In the VR-Arena • In the VR-Lab • At home using your computer • At the bus using a laptop • If the platform is the same then there is no need for recompiling the application (just copy & paste) Miroslav Andel - Development of immersive real-time applications
Scalability The application uses a configuration file that describes the hardware in terms like: Number of IGs Number of displays Number of viewports Physical location of displays/projection planes Physical location of the user Eye separation of the user Type if Stereoscopy Etc… Miroslav Andel - Development of immersive real-time applications
Configuration “3d drawing”
Configuration Monoscopicprojection Immersive applications Often uses non-symmetric frustums/projections, while normal OpenGL applications don’t (left FOV = right FOV, top FOV = bottom FOV)
Configuration Stereoscopic projection
Configuration All users except the target user will perceive the rendered object incorrectly.
Configuration All users except the target user will perceive the rendered object incorrectly. One solution would be using HMDs or a multi-view screen.
Navigation • In a desktop application a user navigates by moving the camera Miroslav Andel - Development of immersive real-time applications
Navigation • In a desktop application a user navigates by moving the camera • In a scalable application the number of cameras/frustums is not known Miroslav Andel - Development of immersive real-time applications
Navigation • In a desktop application a user navigates by moving the camera • In a scalable application the number of cameras/frustums is not known • Instead of moving the user and all “cameras” in the same direction the whole scene is moved in the opposite direction Miroslav Andel - Development of immersive real-time applications
Frameworks(the good stuff) • Will help you with • Cluster setup & synchronization • Frustums & viewports • Input • Tracking Miroslav Andel - Development of immersive real-time applications
Frameworks VR Juggler CAVElib (commercial) Equalizer Miroslav Andel - Development of immersive real-time applications
Frameworks(the bad stuff) • Massive and have a lot of dependencies • Big initial effort to get started • Often in legacy OpenGL • Takes usually several days to get started • Getting all dependencies • Compile all dependencies • Compile the whole framework • Read the documentation Miroslav Andel - Development of immersive real-time applications