Multispectral Image Invariant to Illumination Colour, Strength, and Shading

Multispectral Image Invariant to Illumination Colour, Strength, and Shading Mark S. Drew and Amin Yazdani Salekdeh School of Computing Science, Simon Fraser University, Vancouver, BC, Canada {mark/ayazdani}@cs.sfu.ca

Table of Contents • Introduction • RGB Illumination Invariant • Multispectral Image Formation • Synthetic Multispectral Images • Measured Multispectral Images • Conclusion

Introduction • Invariant Images – RGB: • Information from one pixel, with calibration • Information from all pixels – use entropy New  • Multispectral data: • Information from one pixel without calibration, but knowledge of narrowband sensors peak wavelengths

RGB Illumination Invariant Removing Shadows from Images, ECCV 2002 Graham Finlayson, Steven Hordley, and Mark Drew 4

An example, with delta function sensitivities B P R W G Y RGB… Narrow-band (delta-function sensitivities) Log-opponent chromaticities for 6 surfaces under 9 lights

Deriving the Illuminant Invariant RGB… Log-opponent chromaticities for 6 surfaces under 9 lights Rotate chromaticities This axis is invariant to illuminant colour

An example with real camera data RGB… Normalized sensitivities of a SONY DXC-930 video camera Log-opponent chromaticities for 6 surfaces under 9 different lights

Deriving the invariant RGB… Log-opponent chromaticities Rotate chromaticities The invariant axis is now only approximately illuminant invariant (but hopefully good enough)

Image Formation Multispectral • Illumination : motivate using theoretical assumptions, then test in practice • Planck’s Law in Wien’s approximation: • Lambertian surface S(), shading is , intensity is I • Narrowband sensors qk(), k=1..31, qk()=(-k) • Specular: colour is same as colour of light (dielectric):

Multispectral Image Formation … • To equalize confidence in 31 channels, use a geometric-mean chromaticity: • Geometric Mean Chromaticity:  with

Multispectral Image Formation … surface-dependent sensor-dependent illumination-dependent So take a log to linearize in(1/T)! 11

Multispectral Image Formation … •  Only sensor-unknown is ! ( spectral-channel gains) • Logarithm: known because, in special case of multispectral, *know* k!

Multispectral Image Formation … • If we could identify at least one specularity, we could recover log k ?? • Nope, no pixel is free enough of surface colour . • So (without a calibration) we won’t get log k, but instead it will be the origin in the invariant space. • Note: Effect of light intensity and shading removed: 31D  30-D • Now let’s remove lighting colour too: we know 31-vector (ek – eM)  (-c2/k - c2/M) • Projection  to (ek – eM) removes effect of light, 1/T : 30D  29-D

Form 31-D chromaticity k • Take log • Project  to (ek – eM) using projector Pe Algorithm:

Algorithm: • What’s different from RGB? • For RGB have to get “lighting-change direction” • (ek – eM) either from • calibration, or • internal evidence (entropy) in the image. • For multispectral, we know (ek – eM) !

Carry out all in 31-D, but show as camera would see it. First, consider synthetic images, for understanding: Surfaces: 3 spheres, reflectances from Macbeth ColorChecker Camera: Kodak DSC 420 31 sensor gains qk()

shading, for light 1, for light 2 Synthetic Images Under blue light, P10500 Under red light, P2800

Synthetic Images Original: not invariant Spectral invariant

Measured Multispectral Images Under D75 Under D48 Invt. #1 Invt. #2

After invt. processing Measured Multispectral Images In-shadow, In-light

Measured Multispectral Images

Next: removing shadows from remote-sensing data. Conclusion • A novel method for producing illumination invariant, multispectral image • Successful in removing effects of • Illuminant strength, colour, and shading

Thanks! Funding: Natural Sciences and Engineering Research Council of Canada Multispectral Images Invariant to Illumination Colour, Strength and Shading

Multispectral Image Invariant to Illumination Colour, Strength, and Shading