Shamsuddin Ladha, Sharat Chandran, Kate Smith-Miles

1. Projection Defocus Correction using Adaptive Kernel Sampling and GeometricCorrection in Dual-planar Environments Shamsuddin Ladha, Sharat Chandran, Kate Smith-Miles

2. Outline • Goal • Features and Contributions • Solution Framework • Experiments and Results

3. Goal Create visually appealing projection on dual-planar surfaces • Artifacts to correct • Defocus blur • Geometric distortion

4. Why? Can we replace the desktop monitor(s) by a projector? • Humans crave for large displays • Real estate is seldom available • The ubiquitous office cubicle is dual-planar! R. Raskar, G. Welch, M. Cutts, A. Lake, L. Stesin, and H. Fuchs. The office of the future: A unified approach to image-based modeling and spatially immersive displays. ACM SIGGRAPH:179-188, 1998.

5. Features and Contributions • Non-parametric blur estimation • No assumption of exemplar (in-focus) region • No explicit depth computation • Sparse and adaptive sampling for defocus kernel computation • Correction for both luminance and chrominance channels • Geometry correction

6. Solution Framework Lambertian Dual-planar Display Surface f(x; z) – Defocus kernel Γ – Ambient light radiance Not in-focus points Q1 Q2 Iop = αf(x;z)P(x) + Γ In-focus point P1 z Iop Projector P(x) Camera L. Zhang and S. Nayar. Projection defocus analysis for scene capture and image display. ACM Transactions on Graphics, 25(3):907-915, 2006.

7. Solution Framework Not in-focus points Q1 Q2 P1 In-focus point z Camera (Iop) Projector P(x)

8. Solution Framework • Project P(x) = Iip then Iop≠ Iip • Project P’ = (αf )-1 * (Iop - Γ)thenIop = Iip P’ = arg min{d(αfP + Γ; Iip) P(x) є [0, 255]} Unknowns: αf and Γ

9. Kernel Estimation Project point patterns and measure Point Spread Function (PSF) PSFs as columns of F Defocus kernel • Incorrect kernel estimates near the edge • Can we do sparse and geometry aware sampling? L. Zhang and S. Nayar. Projection defocus analysis for scene capture and image display. ACM Transactions on Graphics, 25(3):907-915, 2006.

10. Sparse Sampling • With increasing distance blur radius can be linearly approximated • Sample few points and interpolate S. Chaudhuri and A. Rajagopalan. Depth from defocus: a real aperture imaging approach. Springer, 1999.

11. Adaptive Sparse Sampling • Measure PSF at few points on each plane • Measure PSF at the edge (adaptive) Compute geometry in image and projector spaces!

12. Geometry in Image Space Detect the edge between two planar surfaces in image space • Project structured light • Detect bend points • Fit a line K. Paidimarri and S. Chandran. Computer Vision, Graphics and Image Processing, pages 289-298. Lecture Notes in Computer Science, Springer 2006.

13. Geometry in Projector Space • Find corresponding points in image and projector space for each planar surface • Compute forward and inverse homography matrices • Find the edge in projector space • Sparse adaptive sampling can be done!

14. Geometric Correction Inv(HR) Original input Display scene surface HL Inv(HL) HR Corrected input Corrected image (camera)

15. Summary We have seen • Compute defocus kernels and measure Γ • Perform geometry correction Next • Improve color fidelity of the corrected image

16. Better Distance Metric P’ = arg min{d(αfP + Γ; Iip), P(x) є [0, 255]} • Sum of squared pixel difference • R, G, B channels treated independently • Humans differently sensitive to luminance and chrominance • Do not treat a pixel in isolation – consider its neighborhood Neighborhood based distance metric in YCbCr space! L. Zhang and S. Nayar. Projection defocus analysis for scene capture and image display. ACM Transactions on Graphics, 25(3):907-915, 2006.

17. Better Distance Metric (absolute luminance error) (absolute chrominance error) ‒ (spatial luminance discontinuities error) ‒ ‒ (spatial chrominance discontinuities error) non-negative weighting parameters Y. Sheng, T. Yapo, and B. Cutler. Global illumination compensation for spatially augmented reality. In Computer Graphics Forum:387-396,2010

18. Experiments • One time offline processing • Run-time correction • Homography matrices • Defocus kernel matrix • Measurement of Γ • Pre-warp and stitch image • Optimal image using steepest descent

19. Results

20. Results

21. Results

22. Results (Quantitative)

23. Summary • A better method • Projection defocus correction • Geometric distortion correction • Future work • Indirect illumination compensation • Composite framework for all corrections

24. References • K. Paidimarri and S. Chandran. Computer Vision, Graphics and Image Processing, pages 289-298. Lecture Notes in Computer Science, Springer 2006. • L. Zhang and S. Nayar. Projection defocus analysis for scene capture and image display. ACM Transactions on Graphics, 25(3):907-915, 2006. • Y. Sheng, T. Yapo, and B. Cutler. Global illumination compensation for spatially augmented reality. In Computer Graphics Forum:387-396,2010. • R. Raskar, G. Welch, M. Cutts, A. Lake, L. Stesin, and H. Fuchs. The office of the future: A unified approach to image-based modeling and spatially immersive displays. ACM SIGGRAPH, 32(Annual Conference Series):179-188, 1998. • S. Chaudhuri and A. Rajagopalan. Depth from defocus: a real aperture imaging approach. Springer, 1999.

25. Thank You