Advisor ：林惠勇老師 Reporter ：林正祐

Illumination and Camera Invariant Stereo Matching Yong SeokHeo, Kyoung Mu Lee, Sang Uk Lee School of EECS, ASRI, Seoul National University, 151-742, Seoul, Korea CVPR’08 Advisor：林惠勇老師 Reporter：林正祐 9/28/2009

Outline • Introduction • Stereo Energy Formulation • Color Normalization Representation • Color Image Formation Model • Color Image Normalization • Stereo Matching using ANCC • Chromaticity Normalization • Adaptive Normalized Cross Correlation • Experimental Results

Introduction (1/2) • In generally, two corresponding pixels have different values (in a real scene) [9,14]: • radiometric changes including lighting geometry and illuminant color. • camera device change. • Past works [3,13,14,17,20,22,23] using only intensity information is not appropriate. • Color information is necessary for handling various radiometric factors.

Introduction (2/2) • This paper present a new algorithm which explicitly modeled the color formation process and propose the invariant measure called Adaptive Normalized Cross Correlation (ANCC)

Stereo Energy Formulation

Color Normalization Representation • Two approaches for finding illuminant invariant representation [6]: • Color constancy algorithm • Attempt to separate the illumination and the reflectance components (like human visual system did). • Example: Retinex algorithm • Color constancy problem is ill-posed, the estimation of the illuminant is generally not an easy task.

Color Normalization Representation • Two approaches for finding illuminant invariant representation [6]: • Color invariant approach • Finds the function which is independent from lighting conditions and imaging devices, two methods are commonly used: • Chromaticity normalization (remove lighting geometry effects) • Gray-world assumption (remove illuminant color effects) • Only a comprehensive normalization method [7] can remove both of them.

Color Image Formation Model (1/2) • A image taken by a linear imaging device can be described by this equation[8]:

Color Image Formation Model (1/2) • A image taken by a linear imaging device can be described by this equation[8]: Assume Lambertian reflectance model and

Color Image Formation Model (2/2) • After consider all factors, representing the color image formation model at pixel p as follows[7]:

Color Image Formation Model (2/2) • After consider all factors, representing the color image formation model at pixel p as follows[7]: Gamma correction parameter Individual brightness factor Illumination color scale factor

Color Image Normalization (1/2) • Use chromaticity normalization to eliminate the effect of lighting geometry.

Color Image Normalization (1/2) • Use gray-world assumption to eliminate the effect of illumination color. Number of pixels

Color Image Normalization (2/2) • Use comprehensive normalization method(mentioned previously) combines two normalization methods. • However, true corresponding pixel values are still not the same after comprehensive normalization. • Because the left and right values are generally not the same.

Stereo Matching using ANCC window mean • NCC • This NCC has invariant property to following affine transformation. • Two critical problems: • Does not work well because of various radiometric changes by ρ, a, b, c and γ • Usually produces fattening effects near the object boundary.

Chromaticity Normalization (1/2) • Representation when two images have different lighting geometries, illuminant colors and camera gamma functions:

Chromaticity Normalization (1/2) • Representation when two images have different lighting geometries, illuminant colors and camera gamma functions: Non-linear

Chromaticity Normalization (1/2) • Representation when two images have different lighting geometries, illuminant colors and camera gamma functions: Apply logarithm, transform to linear

Chromaticity Normalization (2/2) • Subtracting the average of the transformed color values to eliminate ρ(．) term. (chromaticity normalization)

Chromaticity Normalization (2/2) • Subtracting the average of the transformed color values to eliminate ρ(．) term. (chromaticity normalization) Independent with ρ, a, b, c and γ. If the corresponding pixels are correct, this two must be the same

Chromaticity Normalization (2/2) • Subtracting the average of the transformed color values to eliminate ρ(．) term. (chromaticity normalization) Affine transformation. Matching of them is also invariant to radiometric changes by NCC

Adaptive Normalized Cross Correlation (1/2) • Use weight distribution information to reduce the fattening effect. • Weight is computed using bilateral filter [21]. Pixel t in window Normalizing constant

Adaptive Normalized Cross Correlation (1/2) • Use weight distribution information to reduce the fattening effect. • Weight is computed using bilateral filter [21]. • Use this sum value instead of subtracting the mean value in NCC. Remove α term

Adaptive Normalized Cross Correlation (2/2) • ANCC • ANCC does not vary with ρ, illuminant color(a, b, c) and camera gamma correction γ. • Fattening effect can also be reduced since the spatial weight information is incorporated adaptively.

Global Energy Modeling • Data cost • Pairwise cost

Global Energy Modeling • Data cost • Pairwise cost Total energy is defined.

Experimental Results (1/3) • Fixed all parameters: • Vmax= 5, M = (31x31), σd= 14, σs= 3.8. • Graph-cuts(GC) implementing by source code in [2] • test image come from [1,14], • each data set have three light sources (indexed as 1,2,3) and three exposures(indexed as 0,1,2)

Experimental Results (2/3)

Experimental Results (2/3) Compare

Experimental Results (3/3)

Advisor ：林惠勇老師 Reporter ：林正祐