1 / 29

Interactive Video Tours

Interactive Video Tours. MSR Interactive Visual Media Group //msrweb/vision/IBR Rick Szeliski, Sing Bing Kang, Matt Uyttendaele, Simon Winder, Antonio Criminisi. Being There: Video-Based Reality Project. A new computer-mediated interactive medium

bat
Download Presentation

Interactive Video Tours

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interactive Video Tours MSR Interactive Visual Media Group //msrweb/vision/IBR Rick Szeliski, Sing Bing Kang, Matt Uyttendaele, Simon Winder, Antonio Criminisi

  2. Being There:Video-Based Reality Project A new computer-mediated interactive medium Goal:Combine theinteractivity of 3D games and virtual worlds with therealism and richness of film and video

  3. Being There:Video-Based Reality Project How? Capture real-world video footage with an omni-directional camera and add interactive video/graphic elements Application: Video-Based Walkthroughs[IEEE CG&A May/June 2004]

  4. Virtual Tours • Move camera along a rail (“dolly track”) and play back a 360 video • Applications: • Homes and architecture • Outdoor locations(tourist destinations)

  5. Surround video acquisition system OmniCam (six-camera head)

  6. OmniCam • Built by Point Grey Research (Ladybug) • Six camera head • Portable hard drives, fiber-optic link • Resolution per image: 1024 x 768 • FOV: ~100o x ~80o • Acquisition speed: 15 fps uncompressed

  7. Acquisition platforms • Robotic cart • Wearable

  8. Calibration: Lens distortion Original (distorted) image Output (corrected, perspective) image

  9. Calibration: Vignetting Before correction After correction

  10. Stitching (Only 4 of 6 images shown here)

  11. Problem with simple feathering

  12. Multiperspective plane sweep Without MPPS With MPPS

  13. Feature tracking for calibration and ego-motion Points tracked in all frames in all 6 cameras Edges tracked in all frames in all 6 cameras

  14. Stabilization Align frame- to-frame and distribute Dheading Before motion stabilization After motion stabilization

  15. Augmented surround video A movie clip from the original surround data. The augmented movie clip. Surround augmented with video

  16. DemoA Virtual Home Tour

  17. A second scenario: Bellevue Botanical Gardens • Head-mounted outdoor scene acquisition • Map control • Localized audio for a richer experience • Complex path navigation (greater freedom of motion)

  18. Data acquisition • Walk through the garden with the Ladybug camera mounted on helmet • Acquisition done in about 15 minutes • 44 Gb worth of video and audio data

  19. Map control • The garden map • Drawing the • acquisition path • Mapping video frames • positions on map • Placing sound sources • (background and dynamic) • Output is a descriptive • XML file

  20. Some audio sources • Current ego-position Audio processing and streaming • All audio sources are: • Attenuated (based on • distance from current • position), • Mixed together, • Streamed to sound card • (DirectX Audio). • Two types of sound: • background and dynamic

  21. Bifurcation handling • Hypothesize, align, choose best pair as bifurcation • Choice of path depends on current heading and bifurcation point orientation Current heading

  22. Bifurcation handling • Hypothesize, align, choose best pair as bifurcation • Choice of path depends on current heading and bifurcation point orientation Current heading

  23. Demo 2A walk in Bellevue Botanical Garden

  24. 3rd scenario: High-End Home(architectural walkthrough) • High-Dynamic range (HDR) imaging • Complex indoor path navigation

  25. High dynamic range imaging • Combine multiple exposures to obtain a high dynamic range (HDR) image. Underexposed Overexposed Combined

  26. High dynamic range imaging Underexposed frames Overexposed frames All frames • HDR imaging is challenging for videos. • Image registration is necessary.

  27. Demo 3A High-End Home

  28. Features and statistics • Rapid scene acquisition • Map control • Localized audio for a richer experience • Complex path navigation • High data rates: • 44 Gb worth of video and audio data in 15 minutes

  29. Current/future work • Scene Modeling • Motion estimation and map generation • More interactive 3D elements • Compression & selective decoding • Tiled video representation • Rendering • Better use of GPU • Interactive DVD deployment

More Related