1 / 12

Immersive Media Vision

Immersive Media Vision. Create seamless immersive environment for distributed, interactive real and virtual events Reproduce audio, video and other senses with fidelity approaching the limits of human perception Relevant to any type of human interaction: entertainment, education, communications.

flanagand
Download Presentation

Immersive Media Vision

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Immersive Media Vision • Create seamless immersive environment for distributed, interactive real and virtual events • Reproduce audio, video and other senses with fidelity approaching the limits of human perception • Relevant to any type of human interaction: entertainment, education, communications

  2. IMSC Entertainment Vision:RMI Applications • A fusion of internet browsing with a theater-like immersive experience • HD Video at up to 45 Mbits/sec • 10.2 ch Immersive Audio (16 Mbits/sec) • Streaming over the Internet on-demand to a mouse click (20 min prog) • RMI demonstrates IMSC innovations in several critical technical areas: Immersed in a college football game Doctors assisting in a remote procedure Immersive audio capture and rendering Business people negotiating like they are in the same room Network protocols for error correction Streaming media servers Synchronization Students visiting an aquarium a thousand miles away

  3. Premier Demonstration May 9, 2002 • Cross-country streaming from ISI-East in Virginia • NY Times coverage in “Circuits” section • NBC-TV and KTLA • http://imsc.usc.edu/rmi/ • http://www.east.isi.edu/NGI-S/

  4. Internet2 Fall 2002 Member Meeting New World Symphony, Miami Beach Audio: 10.2 channel, immersive sound system Video: HDTV 1920x1080i 550 seat USC Bing Theater

  5. RMI Experimental Setup • Synchronized immersive audio and HDTV streamed playback from Yima server over Internet2 • Control of end-to-end process: capturing, network interface, transmission, rendering • CENIC 2003 “Gigabit or Bust” Award USC Client University of Maryland Server Georgia Tech Server USC New World Symphony Server

  6. Remote Master Class • LA Philharmonic cellist located at our USC lab • Student at New World Symphony in Miami Beach • Three hour session with MPEG2 video and 10.2 immersive audio • Teacher reports that student "was really there" with immersive audio • Many psychophysical, perceptual and artistic tests to be done

  7. Recent Developments... • Two-way audio communication to New World Symphony in Miami Beach, Florida • 8-channel audio – low latency • Live video capture/transmission (DVCam) • Near term prospects • RMI theater at Inha University (Korea) • 2-way live HD-video/immersive audio streaming for Internet collaboration

  8. Distributed Immersive Performance • Create seamless immersive interactive environment • Distributed musicians, conductor (active) and audience (passive) • Q: Why a musical performance?A: A distributed orchestra is one of the most challenging immersive, interactive environments • Scenario: • Multiple soloists scheduled in different cities • Under served areas can be reached

  9. IMSC Communication VisionInterAct! PDA provides spatial array of participants and sonic translation of details UWB console captures 3D model and problem details Immersive workstation coordinates video, avatars, graphics, audio, and haptics to facilitate problem resolution

  10. Research Highlights IMSC has produced ground breaking fundamental research in: • immersive audio • multichannel and HRTF approaches - holistic DSP approach • streaming servers • distributed and scalable architecture • computer vision • computational framework for grouping based on tensor voting, tracking for augmented realities and SFX • graphics & animation • 3D DSP mesh processing, compression, mesh operations, hair modeling and animation • multimodal emotive interfaces • Speech and dialog, vision sensing of body and hands, facial expressions analysis and expressive avatars • virtual reality -- chair of VR2003 in LA • applications to psychology (ADD diagnosis), haptics applications, and user studies • perception & cognitive modeling • theoretical foundations for understanding benefits of media immersion • UWB wireless • leadership in technology, FCC regulation, and commercialization

  11. Sensory Interfaces (SI) • Robust Vision Systems • Segmentation, motion, 3D body and hand tracking, tensor voting, activity analysis, recognition and tracking • Speech Recognition and Synthesis • Emotive dialog, translation, emotion analysis • Immersive Audio • Sonic visualization, spatialization, autocalibration • Facial Analysis -- Gestures and Animation • Expression signatures for emotion and recognition • Video fusion with 3D models for Surveillance • Structure extraction from LiDAR and texture projection • Digital Geometry (3D) Processing • Compression, surface and topology smoothing, mesh simplification

More Related