Light, Surface and Feature in Color Images

1. Light, Surface and Featurein Color Images Lilong Shi Postdoc at Caltech Computational Vision Lab, Simon Fraser University

2. Topics Color Constancy Surface Reflectance Model Feature Analysis

3. Light Surface Feature Color Formation sensor sensitivity camera response Illum. power distribution reflectance spectral

4. Light Surface Feature Color Constancy

5. Light Surface Feature Automatic White Balance AWB Canonical

6. Light Surface Feature Color Constancy Methods = • Retinex Theory (McCann64) • MaxRGB/White-Patch (Land77): max(R) • Gray-World (Buchsbaum80): mean(R) • Shades-of-Gray (Finlayson04): [mean(Rp)]1/p • Gray-Edge Hypothesis (Weijer07): mean(edge(R)) • Non-Negative Matrix Factorization (Shi07)

7. Light Surface Feature Color Constancy Methods Gamut Mapping (Forsyth90) Color by Correlation (Finlayson01) Neural Network (Cardei02) Support Vector Regression (Xiong06) Thin Plate Spline (Shi11)

8. Light Surface Feature Color Constancy Methods Classification-based (Bianco09) Scene-based (Gijsenij11)

9. Light Surface Feature Color Constancy Evaluation

10. Light Surface Feature Blackbody Radiator Lights Tungsten lamps, sunrise/sunset, sky light Planckianlocus Narrowband sensors

11. Light Surface Feature Surface Reflectance Model LIS Coordinate (Finlayson 01)

12. Light Surface Feature Achromatic Surface Gray Surface Detection in LIS

13. Light Surface Feature Skin Color Model length in epidermis/dermis absorbance of other material pigment density absorbance Skin: melanin + hemoglobin Skin Reflectance (Hiraoka et al 93) Under blackbody illumination

14. Light Surface Feature Skin Color Locus • Linear model • m is melanin basis, • h is hemoglobin basis, • is blackbody radiator basis, • cis a constant vector

15. Light Surface Feature Skin Tone Correction 16 different illum + camera calibrations Tone correction Preserve melanin Even simpler model:

16. Light Surface Feature Features Textures, edges, corner, blobs, etc.. Colors Integrated by Quaternion

17. Light Surface Feature Quaternion Real, complex, quaternion (q = a + bi + cj + dk) Non-commutative (pq ≠ qp) Convolution, Correlation, Fourier, Wavelet, etc SVD, EVD, PCA

18. QPCA Image-specific quaternion texture basis Light Surface Feature Texture Feature Extraction Sampled sub-windows

19. 1st QPCA Basis Light Surface Feature Texture Feature Extraction T Single quaternion  A texture patch

20. Light Surface Feature Texture Feature 1st Feature

21. Light Surface Feature Segmentation Quaternion Hoang(05)

22. Light Surface Feature Segmentation

23. Light Surface Feature Color Curvature ?

24. Light Surface Feature Iso-luminance Color -> Gray Cancellation in combining +/- derivatives

25. 1 1 λ2 λ2 e2 e2 e1 e1 Light Surface Feature Hessian Descriptor eigenvectors: (e1, e2 ) eigenvalues: |1|<|2| 2nd order local shape Principle Curvature

26. Light Surface Feature Curvature • Tubular, vessel-like structures [Frangi98] • With eigen-values • blobness: • backgroundness: • vesselness: R and S • Gray image, 2 λ’s; RGB image, 6λ’s

27. Light Surface Feature Color Curvature Quaternion-valued Hessian QSVD on H2 real singular values

28. Light Surface Feature Curvature Detection Frangi Quaternion

29. Light Surface Feature Future Works Content-based color constancy Color blob/points detection Possibilities …