1 / 84

Multi-Projector Displays using Camera-based Registration

Multi-Projector Displays using Camera-based Registration. Ramesh Raskar, Michael S. Brown, Ruigang Yang, Wei-Chao Chen, Greg Welch, Herman Towles, Brent Seales, Henry Fuchs University of North Carolina at Chapel Hill. Multi-Projector Panoramic Displays. Traditional Display Setups.

lyndon
Download Presentation

Multi-Projector Displays using Camera-based Registration

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multi-Projector Displays usingCamera-based Registration Ramesh Raskar, Michael S. Brown, Ruigang Yang, Wei-Chao Chen, Greg Welch, Herman Towles, Brent Seales, Henry Fuchs University of North Carolina at Chapel Hill

  2. Multi-Projector Panoramic Displays Traditional Display Setups

  3. Multi-Projector Panoramic Displays Traditional Display SetupsPrecise Geometry, Well-defined overlaps

  4. Generalized Panoramic Display Flexible Display SetupsCasually aligned projectors and screens

  5. Traditionally rigid design Painstaking construction Room must be modified Projector alignment is tedious High maintenance cost Constant adjustments Projectors become “unaligned” over time Problem :

  6. Generalized display environment Setup display surface approximately Align projectors casually Render with what you have Problem : Instant Panoramic Display

  7. Seamless imagery Immersive 3D images, moving user Casually setup projectors and surfaces Low maintenance Auto-Calibration Perceptually single logical projector Multi-Projector Goal

  8. (a) Projector Registration Geometric alignment (b) Intensity blending Smooth transition Goal : Seamless Display Projection 2 Projection 1

  9. (a) Projector Registration Geometric alignment (b) Intensity blending Smooth transition Goal : Seamless Display

  10. (a) Projector Registration Geometric alignment (b) Intensity blending Smooth transition Goal : Seamless Display

  11. Goal : Seamless Display (a) Projector Registration • Geometric alignment (b) Intensity blending • Smooth transition

  12. (a) Projector Registration Geometric alignment (b) Intensity blending Smooth transition Goal : Seamless Display

  13. Sweet spot Static user Moving User Head-tracked user Previous approachesMulti-projector setups

  14. Multiple projectors (well-defined overlaps) Flight sims, Panoram, Trimensions [Raskar98] Single wide-field-of-view projector OmniMax Alternate Realities Previous approaches: Sweet spot (static user) User

  15. CAVE Rigid setup Well-defined geometry Large support structure Previous approaches:Moving user Projector User

  16. Office of the Future [Raskar98] Relaxed construction constraints Assumes precise display geometry recovery Demonstrated for a single projector Previous approaches:Moving user

  17. QuickTime VR Pair-wise mosaicing of photographs Generalized mosaicing Related work:Image Mosaicing

  18. Seamless image Immersive 3D images, moving user Arbitrary projectors and surfaces Eliminate maintenance Self-calibration Perceptually single logical projector Goal for this paper

  19. Position projectors approximately Use video cameras to recover display geometry projector configuration Track user Render images of 3D scenes Warp and blend projected images to achieve registration Basic Approach: Panoramic Display

  20. Single projector Calibration, Display recovery, Rendering Multiple projector Registration, Blending New techniques Post rendering warp, mesh unification Outline

  21. Video

  22. Single projector Rendering, Calibration, Display recovery Multiple projector Registration, Blending New techniques Post rendering warp, mesh unification Outline

  23. (2) Render Model (1) Desired Image 2-pass Rendering 3D Display Surface Model Projector User

  24. 2-pass Rendering Project this “warped” image, the viewer will see it correctly 3D Display Surface Projector (2) Render Model (1) Desired Image User

  25. Arbitrary display surface Arbitrary projector location Need to determine these for 2 pass rendering Display Surface ? Stereo camera pair Projector ? Single projector display2-pass unknowns -How? User

  26. Display Surface? Projector ? Single projector displayCamera calibration 1. Calibrate stereo pair Stereo camera pair

  27. Display Surface? Projector ? Single projector displayDisplay surface estimation 1. Calibrate stereo pair 2. Determine display surface Stereo camera pair

  28. Display Surface Projector ? Single projector displayProjector estimation 1. Calibrate stereo pair 2. Determine display surface 3. Determine projector Stereo camera pair

  29. Single projector displayProjector estimation 1. Calibrate stereo pair 2. Determine display surface 3. Determine projector Display Surface Stereo camera pair Projector

  30. Display surface is recovered by cameras (via projector) Surface mesh is created from the 3D points Allows us to calibrate the projector Approximate speed 720 surface point samples (~1 minute) 14,000 surfaces point samples (~15 minutes) Single projector display Display Surface Estimation

  31. Display surface and projector are in the coordinate frame of the “Calibration Pattern” x y z Single projector displayCoordinate Frame

  32. A single stereo-pair cannot see entire display Must use several stereo-pairs Each in its own coordinate frame Must register data into common coordinate frame Advantage We control the light Makes registration easier Multiple ProjectorsPanoramic display environments

  33. Multiple ProjectorsSurface mesh registration Display Surface

  34. x x x y y y z z z Multiple ProjectorsDisplay surfaces are in different coordinate frames 3 Display Surfaces 2 1

  35. x x x y y y z z z Multiple ProjectorsMust register into common coordinate frame 3 Display Surfaces 2 1

  36. x y z Multiple ProjectorsSurface meshes are registered into the same coordinate frame 3 Display Surfaces 2 1

  37. x y z Multiple ProjectorsRe-calibrate projectors so they are registered 3 Display Surfaces 2 Re-calibrate the projectors so they are registered into a common coordinate frame 1

  38. x y z Multiple ProjectorsRe-calibrate projectors so they are registered 3 Display Surfaces 2 With display surfaces and projectors registered we can achieve geometric correctness . . . 1

  39. Multiple ProjectorsIntensity Blending Need to attenuate intensities in the overlapped region

  40. Intensity BlendingUse camera to find overlaps Camera observes overlapped region

  41. Intensity BlendingAssigning intensity weights (alpha) Camera image plane Assign intensity weights based on distance between projector boundaries

  42. Intensity BlendingAssigning intensity weights (alpha) Camera image plane Assign intensity weights based on distance between projector boundaries

  43. 2 1 3 3 1 2 Intensity Blending:Alpha-masks Projector alpha-masks

  44. Display surface and projector registration Geometrically alignment imagery Intensity blending Smooth intensity transition between projectors Achieving our goal Seamless display

  45. Errors in estimation user location 3D display surface projector parameters Registration Issues

  46. Errors in estimation user location 3D display surface projector parameters Single projector, errors not visible Multi-projector overlap Geometric mis-registrations Gaps, Discontinuities Registration Issues

  47. Head-tracker for user location Display surface recovery Camera Calibration Radial Distortion Pattern Imperfection Feature detection Projector calibration Relies on display surface recovery Sources of errors

  48. Re-projection error M Projector framebuffer m' m m' m P’ P P

  49. Minimization residual Projector not pin-hole device Inaccuracies in estimated 3D pts on display surface Sources of re-projection error

  50. User location Display surface meshes difficult to register incorrect surface representation Projector parameters re-projection error incorrect projection matrix Effect of errors

More Related