1 / 29

Tyler Johnson, Florian Gyarfas, Rick Skarbez, Herman Towles and Henry Fuchs

A Personal Surround Environment: Projective Display with Correction for Display Surface Geometry and Extreme Lens Distortion. Tyler Johnson, Florian Gyarfas, Rick Skarbez, Herman Towles and Henry Fuchs Department of Computer Science University of North Carolina at Chapel Hill.

michaelsturner
Download Presentation

Tyler Johnson, Florian Gyarfas, Rick Skarbez, Herman Towles and Henry Fuchs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Personal Surround Environment:Projective Display with Correction forDisplay Surface Geometry and Extreme Lens Distortion Tyler Johnson, Florian Gyarfas, Rick Skarbez, Herman Towles and Henry Fuchs Department of Computer Science University of North Carolina at Chapel Hill IEEE VR– March 14, 2007

  2. Multi-Projector Display

  3. Wide-FOV Projectors • Conventional vs. fisheye lens projector Conventional Projector Fisheye-Lens Projector

  4. Outline • Distortion due to Non-Planar Surfaces • Lens Distortion Correction for Projectors • Results • Contributions and Future Work

  5. P Non-planar Display Surfaces

  6. Projector Image Geometry Compensationvia Ray Tracing • Required Information • Viewer location • Display surface model • Projector calibration Display Surface Projector Viewer

  7. Pass 1: Render application scene Pass 2: Remap Pass-1 Image Classic Remapping Algorithm2-Pass Rendering (Raskar ’98) Projector Viewer

  8. Display Surface Geometry Pinhole Projection Pinhole Projection P Pass-1 Pass-2 2-Pass Rendering Assumes no lens distortion!

  9. Distortion with Wide-FOV Lenses Input Image Projected Result

  10. Previous Applications ofWide-FOV Projection • Elumens VisionStation • Hemispherical display screen with fisheye-lens projector for immersive display. • Static projector/display surface relationship. • Distortion correction based on optical/mechanical design specifications.

  11. Previous Applications ofWide-FOV Projection • Konieczny ’05, “A Handheld Flexible Display System” • Allows exploration of volume data using a deformable hand-held screen. • Dynamic display surface.

  12. Our Goals • Our goals were to provide • camera-based calibration for wide-FOV projectors • geometric correction for arbitrary display surfaces • support for tracked viewers

  13. Display Surface Geometry Lens Distortion Correction Pinhole Projection Pinhole Projection Pass-1 Pass-2 Pass-3 3-Pass Rendering

  14. 3-Pass Rendering Issuesfor Wide-FOV Lenses • Fisheye lenses are highly non-linear • Pinhole approximation in Pass-2 leads to sampling issues during lens distortion correction in Pass-3.

  15. Sampling Artifacts Display Surface P 3-Pass Rendering All Passes at Projector Resolution (1024 x 768)

  16. Under-sampled! Over-sampled! Over-sampled! Pass-3 Sampling Issues • Fisheye and pinhole lenses have very different spatial sampling characteristics • Leads to sampling issues during lens distortion correction. Pinhole lens (input) Fisheye lens (output)

  17. Sampling Artifact Reduction Display Surface P Pass 1-2: 4x Projector Resolution (4096x3072) Pass 3: (1024x768)

  18. A New Approach • Remove the pinhole approximation of the wide-FOV lens. • Use the actual lens model of the projector. • Allows simultaneous compensation for lens distortion and non-planar surface geometry.

  19. Projector Lens Model Display Surface Geometry Lens Distortion Correction Pinhole Projection Pinhole Projection Pass-1 Pass-2 Pass-3 Changes to Correction Pipeline Use actual projector lens model to avoid resampling artifacts!

  20. Display Surface Projector Image Projector A New Approach Pre-compute where projector rays intersect the display surface! Viewer

  21. Implementation • Pre-compute the projector to display surface (ray) mapping • Store this projector-resolution, 2D to 3D map on GPU as a floating-point texture. • Static map - independent of viewer location. • New Pixel Shader • At each projector pixel: • Look up <x,y,z> vertex on the display surface. • Project the vertex into ideal image to obtain output color.

  22. Results Display Surface P All Passes at Projector Resolution (1024x768)

  23. Calibration • Capture structured light patterns with camera-pair • Calibrate projector • Linear calibration followed by non-linear optimization incorporating projector lens model • Reconstruct surface geometry • RANSAC-based plane fitting technique [Quirk, EDT 06] • General reconstruction techniques can also be used

  24. Pass-1 Viewer Tracking • Could apply simple 2D-2D mapping • Applies only to stationary viewer! • Our approach is independent of the viewer location • Viewer location can be updated each frame 2D - Mapping Only applicable for Stationary Viewer Combined Pass-2 & -3 Remapping

  25. Heterogeneous 2-Projector Display • Also applicable to conventional projectors Display Surface Fisheye-lens Projector Hybrid Display with Conventional and Fisheye-lens Projectors Conventional Projector

  26. Results • Video

  27. Contributions • Single-pass geometric remapping algorithm • Correcting for both extreme lens distortion and display surface geometry • Minimizes sampling and FOV issues with fisheye lenses • Simple GPU shader program implementation • Static 2D:3D mapping stored as 3D floating point texture • Demonstrated applicability • Personal surround environment with a single fisheye lens projector • Conventional or Wide-FOV fisheye AND Heterogeneous displays • Applicable to real-time VR applications with user-head tracking

  28. Future Work • Methods to further reduce sampling artifacts • Mip-mapped Pass-1 textures for geometry optimized sampling. • Application of more general lens models • [Kannala, IEEE TPAMI, Aug. 06] • Real-time, continuous calibration • Accommodate projector or display surface changes while the application running. • Dynamic vs. static Pass-2 re-map function.

  29. Thank You • Funding support: ONR N00014-03-1-0589 • DARWARS Training Superiority program • VIRTE – Virtual Technologies and Environments program • Elumenati LLC for loan of fisheye-lens projector

More Related