A Region Based Stereo Matching Algorithm Using Cooperative Optimization

1. A Region BasedStereoMatching Algorithm Using Cooperative Optimization Zeng-Fu Wang , Zhi-Gang Zheng University of Science and Technology of China Computer Vision and Pattern Recognition, 2008

2. Outline • Introduction • Algorithmn • Experimental Results • Conclusion

3. Introduction • The stereo correspondence problem is a key point in computer vision. • Goal : Find a more reasonable disparity map that closes to the ground truth data.

4. Outline • Introduction • Algorithmn • Experimental Results • Conclusion

5. Algorithmn

6. Algorithmn

7. Mean-shift algorithm [19] • No assumptions about probability distributions. • Find local maxima • clusters close in space and range correspond to classes. [19] D. Comanicu, P. Meer: “Mean shift: A robust approach toward feature space analysis”, IEEE Trans. Pattern Anal. Machine Intell., May 2002.

8. Mean-shift algorithm [19] [19] D. Comanicu, P. Meer: “Mean shift: A robust approach toward feature space analysis”, IEEE Trans. Pattern Anal. Machine Intell., May 2002.

9. Algorithmn

10. Window stereo matching

11. Adaptive correlation window stereo matching algorithm[16] • Assumption : depth discontinuities occur at colour boundaries • Reduce the outliers wieght • A variation window sizes on the recurrsive moving average implementation [16] Mark Gerrits and Philippe Bekaert. “Local Stereo Matching with Segmentation-based Outlier Rejection . ” Third Canadian Conference on Computer and Robot Vision, June 2006.

12. Adaptive correlation window stereo matching algorithm[16] [16] Mark Gerrits and Philippe Bekaert. “Local Stereo Matching with Segmentation-based Outlier Rejection . ” Third Canadian Conference on Computer and Robot Vision, June 2006.

13. Algorithmn

14. Disparity plane fitting • The disparity plane corresponding to a segmented region function : • : image coordinates • : plane parameters • RANSC (RANdom Sample Consensus) • Voting

15. Disparity plane fitting algorithmbased on voting • Disparity plane: • For a pair of points on a line along 𝑥-axis belonging to the region, we can obtain an estimation of the plane parameter 𝑎 by calculating .

16. Disparity plane fitting algorithmbased on voting (parameter a)

17. Disparity plane fitting algorithm comparison The comparison of the plane fitting results based on the RANSAC algorithm(blue)and the voting algorithm(red).

18. Disparity plane fitting algorithmbased on voting The disparities obtained by the plane fitting algorithm based on voting

19. Algorithmn

20. The cooperative optimization • Goal :To optimize the disparity plane parameters of each region such that the disparity plane parameters of the adjacent regions keep consistent. • The total energy function E(x) of all regions is defined as E(x) = E1(x)+ E2(x) + ... + En(x) • Ei(x) : the energy function of the ith region

21. The cooperative optimization The sketch map for optimization of sub-targets

22. Energy functional of each region

23. Energy functional of each region • = • : the visible pixel set on the current region of the left image • : the visible pixel set on the current region of the right image • : matching pixels ( is a projection pixel of p)

24. Energy functional of each region

25. Energy functional of each region • : the number of pixels of the left occlusion area • : the number of pixels of the right occlusion area • : the penalty constant for occlusion

26. Energy functional of each region • : the set of the border pixels on the current region • : the smooth penalty constant • : the neighborhood of p • : the corresponding disparities , if , otherwise

27. The cooperative optimization • Where 0 ≤ ≤ 1 , 0 ≤ ≤ 1 are the corresponding weights. • Ej(x) is the energy functional of the jth region Rj • For each region i, use the Powell’s method to minimize the at each iteration. • The disparity plane parameters of each region could be optimized. • The optimized parameters will be regarded as the initial estimation in the next iteration.

28. The cooperative optimization • The optimization is carried out until the algorithm converges or the number of iteration is reached.

29. The cooperative optimization

30. The cooperative optimization

31. Algorithmn

32. Outline • Introduction • Algorithmn • Experimental Results • Conclusion

33. Experimental Results • Device:A notebook with CPU of PM1.6G • Settingsparameters: 𝜆𝑖 = 0.5, 𝜆𝑠 = 0.5, 𝜆𝑜𝑐𝑐 = 0.5 𝑤𝑖𝑗are set according to [17] • Source : Middlebury http://vision.middlebury.edu/stereo/ • Time:20s ( 4 iterations ) • Segmentation:8s [17] Xiaofei Huang. “Cooperative Optimization for Energy Minimization: A Case Study of Stereo Matching”, cs.CV/0701057, Jan 2007. http://front.math.ucdavis.edu/author/X.Huang,

34. Experimental Results

35. Experimental Results Black : occluded border regions  White : discontinuities

36. Experimental Results

38. Experimental Results

39. Outline • Introduction • Algorithmn • Experimental Results • Conclusion

40. Conclusion • Contributions • Combine some known techniques to obtain the high quality disparity map. • The algorithm only requests the initial estimation of disparities is roughly correct. • Futureworks • Improve the plane fitting by introducing B-spline fitting technique. • Develop a more efficient segmentation algorithm.