470 likes | 480 Views
Spectral Focal Sweep (SFS): Extended Depth of Field from Chromatic Aberrations ICCP 2010 Ollie Cossairt, Shree Nayar Columbia University. Conventional Camera (F/4). BLURRY. Conventional Camera (F/16). NOISY. EDOF Camera (F/4). CAPTURED. EDOF Camera (F/4). DEBLURRED. Related Work.
E N D
Spectral Focal Sweep (SFS):Extended Depth of Field from Chromatic AberrationsICCP 2010Ollie Cossairt, Shree NayarColumbia University
Conventional Camera (F/4) BLURRY Spectral Focal Sweep: ICCP 2010
Conventional Camera (F/16) NOISY Spectral Focal Sweep: ICCP 2010
EDOF Camera (F/4) CAPTURED Spectral Focal Sweep: ICCP 2010
EDOF Camera (F/4) DEBLURRED Spectral Focal Sweep: ICCP 2010
Related Work Coded Aperture [Welford ’60] [Levin et al. ‘07] [Veeraraghavan et al. ’07] [Zhou and Nayar ’09] Phase Plates [Dowski and Cathey ’95] [Chi and George ’01] [Levin et al. ’09] Focus Sweep [Nagahara et al. ’08] [Hausler ’72] Sensor Lens Focal Plane Color [Guichard et al. ’09] Blue Green Red RGB Sensor Lens Focal Planes Spectral Focal Sweep: ICCP 2010
Spectral Focal Sweep Idea Broadband reflectance spectra Poorly Corrected Lens B&W Sensor Focus Sweep = + + Sensor Lens Focal Plane No moving parts or custom elements Spectral Focal Sweep: ICCP 2010
Spectral Focal Sweep Idea Broadband reflectance spectra • Works exactly for black and white scenes • Many naturally occurring reflectance spectra are also broadband [Parkkinen ‘89] Poorly corrected lens • Simpler lens design optimization Black & White Sensor • Can also approximate with luminance channel from color sensor Spectral Focal Sweep: ICCP 2010
Conventional Focus Sweep PSF u = object depth (fixed) v= sensor distance (varying) Instantaneous PSF Focal Plane Lens EFL = f Sensor u v1 Scene Depth Range Spectral Focal Sweep: ICCP 2010
Conventional Focus Sweep PSF u = object depth (fixed) v= sensor distance (varying) Instantaneous PSF Focal Plane Lens EFL = f Sensor d2 u v2 Scene Depth Range Spectral Focal Sweep: ICCP 2010
Conventional Focus Sweep PSF u = object depth (fixed) v= sensor distance (varying) Instantaneous PSF Focal Plane Lens EFL = f Sensor d3 u v3 Scene Depth Range Spectral Focal Sweep: ICCP 2010
Conventional Focus Sweep PSF Integrate over time to get final PSF Circ/Top-hat function Defocus disk diameter (A = aperture diameter) Varying either object/sensor distance, or focal length will produce the same PSF Spectral Focal Sweep: ICCP 2010
Refractive Dispersion (prism) Snell’s Law of refraction Refraction depends on wavelength Spectral Focal Sweep: ICCP 2010
Refractive Dispersion (lens) For an uncorrected lens, Focal length depends on wavelength R1, R2 =lens radii of curvature Focal Planes PSF Sensor v u Scene Depth Range Spectral Focal Sweep: ICCP 2010
Spectral Focal Sweep PSF Each wavelength produces a different defocus disk A B&W Sensor integrates over wavelength, weighted by the spectral reflectance of the scene For white objects, a SFS PSF is identical to a mechanical focal sweep PSF Spectral Focal Sweep: ICCP 2010
SFS Lens Design • Design Criteria • Wavelength range 400-700nm • 75mm EFL F/4 • 1/3” Sensor, 10um pixel size • Optimization Criteria • Zemax Optical Design Software • Maximize chromatic defocus across wavelength range • Minimize spot size at 550nm • Fit to off-the-shelf lenses Maximum defocus diameter 100um at 400nm and 700nm Spectral Focal Sweep: ICCP 2010
Zemax SFS PSFs(white spectrum) field Corrected lens SFS Lens 15x depth Spectral Focal Sweep: ICCP 2010
Zemax SFS PSFs(white spectrum) 1 x 15 field SFS Lens Corrected lens depth Spectral Focal Sweep: ICCP 2010
Simulating Real Spectra • Munsell database of 1250 measured reflectance spectra • Zemax simulation of SFS PSF at 12 depth locations (1.5m-3m, 100um max defocus dia.) • For each Munsell spectrum • Calculate average deblurring error when a white spectrum is used for deblurring Spectral Focal Sweep: ICCP 2010
Average Deblurring Error Input Image: Munsell color PSFs: White PSF at center depth: PSF Distance Metric: [Zhou and Nayar ‘09] Average Deblurring Error: Averaged over many images Spectral Focal Sweep: ICCP 2010
Munsell Spectra PSFs 1 x 15 Corrected lens depth Most Munsell spectra show little PSF variation Spectral Focal Sweep: ICCP 2010
Average Deblurring Error(%95 of Munsell spectra) Most Munsell PSFs introduce negligible deblurring artifacts Spectral Focal Sweep: ICCP 2010
Doublet SFS Implementation Implementation Zemax Design Edmund Optics #48184 1” Dia. 100mm FL Plano-convex asphere Thorlabs #LE1929 1” Dia. 300mm FL positive meniscus Spectral Focal Sweep: ICCP 2010
Experimental Setup Corrected Lens SFS Doublet Lens B&W Examples (Basler 1/3” sensor) F/4 SFS 75mm EFL Doublet F/1.4 75mm EFL Cosmicar Color Examples (Canon 450D sensor) F/4 SFS 75mm EFL Doublet F/2.8 100mm EFL Cannon Spectral Focal Sweep: ICCP 2010
Measured SFS PSFs(white spectrum) depth Cosmicar SFS lens Measured SFS PSF approximately depth-invariant Spectral Focal Sweep: ICCP 2010
Deblurring Deblur with measured white PSF B&W Images • Wiener deconvolution Color Images • Transform from RGB to YUV space • Wiener deconvolution of Y image • Transform back to RGB space Spectral Focal Sweep: ICCP 2010
EDOF Examples Spectral Focal Sweep: ICCP 2010
F/4 Corrected Lens 8ms exposure Spectral Focal Sweep: ICCP 2010
F/4 SFS Camera (Captured) 8ms exposure Spectral Focal Sweep: ICCP 2010
F/4 SFS Camera (Deblurred) 8ms exposure Spectral Focal Sweep: ICCP 2010
F/4 Corrected Lens Foreground Background 8ms exposure Spectral Focal Sweep: ICCP 2010
F/4 SFS Lens (Captured) 8ms exposure Spectral Focal Sweep: ICCP 2010
F/4 SFS Lens (Deblurred) 8ms exposure Spectral Focal Sweep: ICCP 2010
F/4 Corrected Lens Foreground Background 16ms exposure Spectral Focal Sweep: ICCP 2010
F/4 SFS Lens (Captured) 16ms exposure Spectral Focal Sweep: ICCP 2010
F/4 SFS Lens (Deblurred) 16ms exposure Spectral Focal Sweep: ICCP 2010
F/4 Corrected Lens Background Foreground 16ms exposure Spectral Focal Sweep: ICCP 2010
F/4 SFS Lens (Captured) 16ms exposure Spectral Focal Sweep: ICCP 2010
F/4 SFS Lens (Deblurred) 16ms exposure Spectral Focal Sweep: ICCP 2010
F/4 Corrected Lens Background Foreground 16ms exposure Spectral Focal Sweep: ICCP 2010
F/4 SFS Lens (Captured) 16ms exposure Spectral Focal Sweep: ICCP 2010
F/4 SFS Lens (Deblurred) 16ms exposure Spectral Focal Sweep: ICCP 2010
F/4 Corrected Lens Background Foreground 16ms exposure Spectral Focal Sweep: ICCP 2010
F/4 SFS Lens (Captured) 16ms exposure Spectral Focal Sweep: ICCP 2010
F/4 SFS Lens (Deblurred) 16ms exposure Spectral Focal Sweep: ICCP 2010
Limitations • Broadband assumption does not always hold (e.g. %5 of Munsell Spectra) • deblurring artifacts introduced • Color deblurring method does not correct blur in chrominance channels Spectral Focal Sweep: ICCP 2010
Conclusion Broadband spectra Chromatic aberrations B&W sensor Focus sweep + + = = • Works exactly for B&W scenes • Works well for many naturally occurring spectra • Results for color images “look” good • Easy to build EDOF camera Spectral Focal Sweep: ICCP 2010