A Rapid S tereo M atching A lgorithm B ased on Disparity I nterpolation

1. A Rapid Stereo Matching Algorithm Based on Disparity Interpolation Gang Yao Yong Liu Bangjun Lei Dong Ren Institute of Intelligent Vision and Image Information China Three Gorges University

2. Outline • Introduction • Method • Disparity Interpolation • Experiments • Conclusion

3. Introduction • The disparity and the matching cost are calculated based on stereo matching • left image as the base image. • right image as the base image. • The disparity in this two kind of situations are also optimized.

4. Introduction • Matching area and un-matching area can be further marked. • The disparity interpolation process is done • started from the edge of un-matching area. • stop until all pixels in un-matching area are finished.

5. Introduction • The proposed algorithm in this paper on the basis of the adaptive cost function in [13]. • Just to interpolation the results of local matching algorithm • maintain the disparity continuity. • improve the stereo matching speed. [13]Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure

6. Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure Andreas Klaus Mario Sormann Konrad Karner VRVis Research Center, Graz, Austria

7. Introduction • Color segmentation and Self-adapting matching • maximize the number of reliable correspondences. • Self-adapting conception : • The scene structure is modeled by a set of planar surface patches. • Instead of assigning a disparity value to each pixel, a disparity plane is assigned to each segment. • The optimal disparity plane labeling is approximated by applying belief propagation.

8. Work-flow Diagram

9. Local Matching in Pixel Domain • . • . • .

10. Disparity Plane Estimation • . • step 1 : calculate matching cost. • step 2 : the disparity plane with the minimum matching cost is assigned to each segment. • step 3 : segments that are assigned to the disparity plane are grouped. • Repeating the plane fitting for grouped regions. • increase the accuracy

11. Disparity Plane Assignment • . • . • .

12. Experiments

13. A Rapid Stereo Matching Algorithm Based on Disparity Interpolation Back to …

14. The Proposed Algorithm

15. Outline • Introduction • Method • Cost function • Robust matching • Disparity optimization • Disparity Interpolation • Experiments • Conclusion

17. Cost Function • The cost function includes two parts : • C(x,y,d) = (1-ω) × Cdata(x,y,d) + ω × Csmooth(x,y,d) • Data term • . • Smooth term • .

18. Cost Function • Can ensure the gray consistency between the correspondent pixels. • Can guarantee the smooth consistency in the pixel neighbourhood • find matching pixel accurately. • improve the smoothness of disparity.

20. Robust Matching • The similarity cost function including • the gray consistency. • the smoothness consistency in pixel neighbourhood. • The cost function is actually a quadratic function • like p(x) = x2, • its derivatives is p'(x) = 2x.

21. Robust matching • The noise affects is linear increase in stereo matching. • The noise is not control by the function, so the quantizing error and noise is very sensitive in both the data item and smooth item in cost function.

22. Robust Matching • In order to reduce the affection of noise • robustness matching function Lorentzian [16]. • the noise is control by Lorentzian function, and the influence is nearly zero

23. Robust Matching • New cost function • . • .

26. Disparity Optimization • No matter what kind of method, the corresponding points can not be found for a part of the pixels in the image. • Left occluded and right occluded cannot appear at the same time.

27. Disparity Optimization • The process of optimization as follows : • Step 1 : • After robust matching method • right-to-left disparity dR-L(x,y) and left-to-right disparity dL-R(x,y). • value of cost function is calculated respectively as CR-L(x,y) and CL-R(x,y).

28. Disparity Optimization • Step 2 : • dR-L(x,y)、dL-R(x,y)、CR-L(x,y)、CL-R will be optimized. occluded

30. Outline • Introduction • Method • Disparity Interpolation • Experiments • Conclusion

32. Disparity Interpolation • Many pixels can not find the corresponding points because of occluded in the image. • The corresponding disparity can not be calculated in the disparity after optimized. • the disparity can be divided into • matching area • un-matching area

33. Disparity Interpolation • The disparity in the un-matching area should be continuity with the disparity in matching area surrounding the un-matching area. • Disparity interpolation for un-matching area to get the smooth disparity on basis of the optimized disparity.

34. Disparity Interpolation • Step 1 : Mark the matching area and un-matching • 1 stands for un-matching area. • 0 stands for matching area. • Step 2 : Find the edge points in the un-matching area • scanning the 8 neighbourhood of pixels. • calculate the disparity of edge pixel. • mark the point as 0. • Step 3 : Stop scanning until all the points in un-matching area have been matched, or then return to Step2.

35. Disparity Interpolation

37. Outline • Introduction • Method • Disparity Interpolation • Experiments • Conclusion

38. Experiments • The results of each steps

40. Experiments • The results of other methods

41. Experiments • Compare the efficiency

42. Outline • Introduction • Method • Disparity Interpolation • Proposed Algorithm • Experiments • Conclusion

43. Conclusion • The dense and smooth disparity can be required. • The effect of the new algorithm is better than traditional stereo matching algorithm. • Time complexity can be reduced. • Efficiency is closer to segment based algorithm. • The main problem of this algorithm : • un-matching area is very big.

44. Conclusion • The disparity error of pixels in centre of un-matching area will be bigger because of interpolating operation is start from the edge of un-matching area.