Fast Census Transform-based Stereo Algorithm using SSE2

1. Fast Census Transform-based Stereo Algorithm using SSE2 Young Ki Baik* Kyoung Mu Lee Computer Vision Lab. School of Electrical Engineering and Computer Science Seoul National University

2. Contents • Stereo Vision • Census Transform Stereo Vision • Fast approaches • Experimental result • Conclusion and Future work

3. Introduction • What is the stereo vision? • The stereo vision is the method to extract 3D information using image from different view points. • Topographical survey • Obstacle detection • Object tracking • Face recognition

4. Introduction • Trade off of algorithms • Algorithm for accurate results • Complex computation and iteration • Slow processing time • Unable to realize real-time system • Algorithm for fast processing time • Simple computation and no iteration • Fast processing time • Unable to realize accurate system

5. Introduction • Fast stereo vision algorithm • Window size invariant method • Box filtering method • “Box-filtering techniques”, M.J.McDonnell (CGIP-81’) • “Real time correlation-based stereo : algorithm, implementations and applications”, Olivier Faugeras , Zhengyou Zhang , … (Tech.Rep.RR-2013, INRIA,1993) • Disparity range invariant method • Rectangular subregioning method • “Rectangular Subregioning and 3-D Maximum-Surface Techniques for Fast Stereo Matching”, Changming Sun (CVPR-2001) • Parallel processing technique

6. Introduction • Problem • Real images from grabbers can not assure of brightness consistency in corresponding region. • Intensity correlation method is not proper for real images. • Census transform • Census transform has been evaluated as the method robust to radiometric distortion. • J. Banks and P. Corke, "Quantitative evaluation of matching methods and validity measures for stereo vision," Int. J. Robotics Research, vol. 20, pp. 512-532, July 2001. • Heiko Hirschmller, "Improvements in Real-Time Correlation Based Stereo Vision", Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision, pp. 141-148, Kauai, Hawaii, December 2001.

7. 121 130 26 31 39 1 1 0 0 0 109 115 33 40 30 1 1 0 0 0 98 102 78 67 45 1 1 X 0 0 47 67 32 170 198 0 0 0 1 1 39 86 99 159 210 1 1 1 1 1 Census transform window (CTW) 1 1 0 0 0 1 1 0 0 0 1 1 0 0 0 0 0 1 1 1 1 1 1 1 Census Transform Stereo Vision • Census transform • Census transform converts relative intensity difference to 0 or 1 and deforms 1 dimensional vector as much as window size of census transform.

8. Height Width Census Transform Stereo Vision • Result of census transform • Census transform makes data of (image size * vector size). (Square size of CTW)-1 Height Width

9. Census Transform Stereo Vision • Sum of Hamming distance • The Hamming distance of two transformed vectors with correlation windows is used to find corresponding region. Sum of Hamming distance Disparity range Right census transformed vector 3D disparity space Left census transformed vector

10. Census Transform Stereo Vision • Complexity of algorithm • Census transform-based stereo vision (CTSV) has high complexity. N : Searching window size D : Disparity range C : Census transform window size

11. Fast Census Transform Stereo Vision • Fast Census Transform Stereo Vision Census transform Hamming distance Parallel processing - SSE2 8bit look-up table Parallel processing - SSE2 Correlation Moving window technique Parallel processing - SSE2

12. Fast Census Transform Stereo Vision • SSE2 (Streaming SIMD Extension 2) • 128-bit SIMD packed integer & floating point arithmetic operation • Cache and memory management operation • Continuous memory structure is required • No advantages in separate data

13. Fast Census Transform Stereo Vision • Fast approaches (census transform) • Usage of parallel processing (SSE2) • 16 pixels are loaded to XMM(SSE2 memory) and computed at once.

14. Fast Census Transform Stereo Vision • Fast approaches (sum of Hamming distance) • Usage of 8bit look-up table (LUT) • Parallel processing : SSE2 • Parallel processing is faster than 8 bit LUT

15. Fast Census Transform Stereo Vision • Fast approaches (correlation) • Moving window technique

16. Fast Census Transform Stereo Vision • Fast approaches (correlation) • Combination of moving window and SIMD • Moving window technique for x, y-axis • SSE2 for d-axis

17. Experimental result • Environment • System : Pentium-IV 2.4GHz • Cache memory : 512Kbyte • Camera : Stereo Mega-D (Videre design) • Condition • Image size : 320 x 240 gray stereo images • Census transform window size :5x5, 7x7, 9x9 • Disparity searching range : 32 • Correlation window size : 11x11

18. Experimental result • Detail processing time • 32 disparity searching range

19. Experimental result • Performance of census transform stereo vision

20. Conclusion and Future work • Conclusion • Moving window technique reduces processing time to constant except in transforming stage • SSE2 instructions reduces running time by 2.5 to 3 times • Possibility for faster result • Specialized Instruction • 16 bit look-up table • Fixed window size of census transform • Future work • Applying real-time approach to another stereo algorithm • Combine stereo system to other applications