Radiometric Self Calibration

1. Radiometric Self Calibration Tomoo Mitsunaga Shree K. Nayar Hashimoto Signal Processing Lab. Dept. of Computer Science Sony Corporation Columbia University CVPR Conference Ft. Collins, Colorado June 1999

2. Image M2 (Low exposure) Image M1 (High exposure) Usual imaging systems have : • Limited dynamic range • Non-linear response Problem Statement • How well does the image represent the real world? CVPR99

3. E L Scene Radiance and Image Irradiance Radiance Irradiance Image irradiance : Ideal camera response : Aperture area Exposure : CVPR99

4. Scene Radiance and Measured Brightness Video Image Formation Image Exposure Camera Electronics Digitization CCD Measured brightness M Scaled radiance I Scene radiance L linear Photo Image Formation Image Exposure Film Development Scanning Film f (M) : The radiometric response function CVPR99

5. Calibration with Reference Objects • The scene must be controlled • The reflectance of the objects must be known • The illumination must be controlled CVPR99

6. Input Images Response function High dynamic range radiance image Calibration without Reference Objects • Differently exposed images from an arbitrary scene • Recover the response function from the images • Calibrate the images with the response function CVPR99

7. Previous Works • Mann and Picard (95) : • Take two images with known exposure ratio R • Restrictive model for f : • Find parameters a, b, g by regression • Debevec and Malik (97) : • General model for f : only smoothness constraint • Take several (say, 10) high quality images • At precisely measured exposures (shutter speed) CVPR99

8. Obtaining Exposure Information • We have only rough estimates • Mechanical error • Reading error (ex. F-stop number) CVPR99

9. Radiometric Self-Calibration • Works with roughly estimated exposures • Inputs : • Differently exposed images • Rough estimates of exposure values • ex. F-stop reading • Outputs : • Estimated response function • Corrected exposure values CVPR99

10. video posi nega A Flexible Parametric Model High order polynomial model : f (M) • Parameters to be recovered : • Coefficients cn • Order N M f(M) of some popular imaging products CVPR99

11. Thus, we obtain ... Objective function : Response Function and Exposure Ratio Images: q = 1,2,….Q , Pixels: p = 1, 2, …..P Exposure ratio: Using polynomial model : CVPR99

12. An Iterative Scheme for Optimization Rough estimates Rq,q+1(0) Rq,q+1(i) Optimize for Rq,q+1 Optimize for f f (i) Optimized f and Rq,q+1 CVPR99

13. Evaluation : Noisy Synthetic Images f (M) M Solid : Computed response function Dots : Actual response function CVPR99

14. Evaluation : Noisy Synthetic Images (cont’d) Percentage Error in Computed Response Function Trial Number Maximum Error : 2.7 % CVPR99

15. Computing a High Dynamic Range Image • Calibrating by the response function • Normalizing by corrected exposure values • Averaging with SNR-based weighting CVPR99

16. Results : Low Library (video) Captured images I Calibration chart M Computed responsefunction CVPR99

17. Results : Low Library (video) Captured images Computed radiance image CVPR99

18. Results : Adobe Room (photograph) Captured images I M Computed radiance image Computed response function CVPR99

19. Results : Taos Clay Oven (photograph) Captured images I M Computed radiance image Computed response function CVPR99

20. Conclusions • A Practical Radiometric Self-calibration Method • Works with • Arbitrary still scene • Rough estimates of exposure • Recovers • Response function of the imaging system • High dynamic range image of the scene Software and Demo http://www.cs.columbia.edu/CAVE/ CVPR99