Color constancy beyond bags of pixels

1. Color constancy beyond bags of pixels Ayan Chakrabarti Keigo Hirakawa Todd Zickler Harvard School of Engineering and Applied Sciences CVPR 2008 2009/01/012 Reporter : 品芝

2. Outline • Same object * different illuminants = different colors • Color constancy : map scene to a canonical illuminance • Application: white balance • Introduction • Beyond “Bags of pixels ” -> bags of patches • Image formation • Spatio-spectral correlation and color constancy • Distributions of pixels vs. patches • Frequency-domain perspective • Spatio-spectral modeling • Estimation • Experiment results • Conclusion

3. Introduction • Estimating the color of scene illuminant plays a central role in computational color constancy • Compute a mapping from an input color image to an illuminant-invariant representation • Existing methods – bags of pixels • Individual pixels, ignore spatial contexts • Difficult to infer the illuminant direction • Break from bags of pixels paradigm by building an explicit statistical model of the spatial dependencies between nearby pixels Patches encode more useful information about illuminant

4. Illuminant spectrum Observercolor Color filters reflectance • Image formation • Spatio-spectral analysis (frequency-domain perspective) 1. Spatial correlations provide important cues about illuminant 2. Should encode and exploit these correlations Correlation between color channels frequency dependent Most of the high-frequency components in luminance Luminance Chrominance

5. Spetio-spectual modeling • Statistical model for a color image patch X • Spatial decorrelation basis: K DCT filters • DC component is modeled as uniform • Other components modeled as “multivariate Gaussian distribution” • estimate the training set • The probability of the entire reflectance image patch

6. Insights from • The eigenvector of compute from the natural image, and represent directions in tri-stimulus space • Visualizes these direction for three choices • Large eigenvector for all K>0 is achromatic • Corresponding eigenvalue is much bigger than those of chromatic components • This is consistent with a highly eccentric elliptical shape that is aligned with the illuminant direction

7. Estimation • Observed image -> overlapping patches • Estimate w by model fitting • Trivially w=0 (P zero mean) • Constraint • Closed form solution

8. Taking eigen-decomposition of • eigenvalues eigenvectors • Smallest eigenvetor of A • Need to eigen decomposition 3*3 matrix A • The smallest ellipsoid touches the sphere is along the major axis • The eigen vector e of A that corresponds to the minimum eigen-value

9. Experiment results