190 likes | 205 Views
Auto-Calibration of Multi-Projector Display Walls. Marc Pollefeys ( marc@cs.unc.edu ) UNC-Chapel Hill. Andrew Raij ( raij@cise.ufl.edu ) UNC-Chapel Hill University of Florida. Multi-Projector Display. Projector misalignment The tedious solution -- align by hand
E N D
Auto-Calibration of Multi-Projector Display Walls Marc Pollefeys (marc@cs.unc.edu) UNC-Chapel Hill Andrew Raij (raij@cise.ufl.edu) UNC-Chapel Hill University of Florida ICPR 2004
Multi-Projector Display • Projector misalignment • The tedious solution -- align by hand • Many automatic camera-based methods [Raskar Viz1999, Chen Vis2000, Yang Viz2001, Raij Procams2003, …] ICPR 2004
Multi-Projector Display cont’d • Undefined display area ICPR 2004
Related Work – Display Area • Rectangle on Screen • Require manual placement and / or interaction to achieve desired display area properties • Yang Vis2001, Rehg CARV2002, Raij Procams2003, Ashdown CVPR2004 • Raskar CVPR2001, Raskar SIGGRAPH2003 – Requires tilt sensors and rigidly attached projector-camera pairs • Okatani ICCV2003 – Requires calibrated projectors ICPR 2004
Main Contribution • We propose a fully automatic method for defining the display area of a planar multi-projector display • Single camera • No fiducials, interaction or additional sensors • Uncalibrated projectors • Note planar screen makes auto-calibration harder ICPR 2004
Our Testbed: PixelFlex2 • Planar, casually-aligned, multi-projector display system developed at UNC-Chapel Hill • 8 projectors in 4x2 configuration, roughly 12' x 6' image • Calibration camera sees all projectors • Original system uses 4 fiducials to define display area Note: without photometric correction ICPR 2004
Outline of our Approach • Find camera-projector homographies • Projector calibration with planar auto-calibration (TriggsECCV98) • Reconstruct calibration camera, projectors and display plane. • Define world-aligned frame in plane and choose display area. ICPR 2004
Step 1: Camera-Projector Homographies Projector 1 2 4 5 6 7 8 3 Features Hcp Camera ICPR 2004
Step 2: Projector Auto-Calibration xc • Projector ≡ Camera • The image of the circular points must lie on the image of the absolute conic (IAC) in all views Ω∞ X π∞ πd ωc ωp Hcpxc Hcp C P ICPR 2004
Projector Auto-Calibration cont’d (0, yp) • n projectors (n ≥ 3) • 2n+2 constraints • n+5 unknowns • Camera is calibrated - ωc is known • Camera image of circ. points xc unknown • Projector - fpi, yp unknown • pixels are square • principal point same in all projectors, offset vertically • Note: Separate offset yields 2n+4 unknowns > 2n+2 • Levenberg-Marquardt least squares minimization of n ||xcTωcxc ||2 + ∑||(Hcpxc)Tωp(Hcpxc)||2 p=1 ICPR 2004
Initialization to Minimization C • For each hypothesis on Kp and for each projector π1 π2 SVD-based Pose Estimation for Planar Scenes (TriggsECCV98) Projector Kp-1HcpKc Camera xc1 xc2 ||xcTωcxc ||2 + ∑||(Hcpxc)Tωp(Hcpxc)||2 n p=1 ICPR 2004
Initialization (cont’d) • Clear minima produced by initialization algorithm ICPR 2004
Intrinsics are consistent for different geometric configurations Estimated Projector Intrinsics ICPR 2004
Step 3: Reconstruction π1 π2 • Camera-projector baseline is normalized to 1 so n reconstructions are in different frames, up to scale • Merge by normalizing by distance from camera to plane K1-1Hc1Kc P1 C SVD π1 π2 Kn-1HcnKc Pn C ICPR 2004
Step 4: Display Area Selection • Display area is a world-aligned rectangle in the plane of desired aspect ratio • Vertical = Horizontal V Plane Normal • Assume camera x-axis is horizontal (i.e. allows tilting) and perform orthogonal projection on plane • Can choose rectangle of desired aspect ratio since reconstruction up to scale ICPR 2004
Results Camera, projector array, and plane reconstruction. 16:9 max inscribed area viewport shown in yellow. Camera image with dotted lines extending towards vanishing points. Vanishing points found by projection of world direction basis into camera. ICPR 2004
Conclusions • Contributions • Automatic estimation of intrinsics of array of projectors projecting on a plane • Automatic estimation of projector extrinsics and display plane • Automatic selection of world-aligned viewport of proper aspect ratio • In sum, fully automatic calibration of a planar multi-projector /camera system without physical calibration objects and/or interaction. ICPR 2004
Conclusions cont’d • Discussion • Possible even though only scene observed by the camera and projectors is a plane! • “Metric" calibration is not highly accurate (intrinsics, etc.), but this is ok for the application. Perceptually, display area will appear to be a properly-aligned rectangle. • Future Work • Precise evaluation of calibration accuracy • Improve calibration with max likelihood estimation based on bundle-adjustment. • Calibrate for camera intrinsics, radial distortion in both camera and projectors. ICPR 2004
Acknowledgements • This work was partially supported by: • NSF Career award IIS 0237533 • Department of Energy ASC VIEWS Program B519834 • DARPA DARWARS program ONR N00014-03-1-0589. • The UNC PixelFlex Display Research Team Henry Fuchs Herman Towles Chris Ashworth Sundeep Tirumalareddy ICPR 2004