EECS 274 Computer Vision

1. EECS 274 Computer Vision Stereopsis

2. Stereopsis • Stereopsis: Fusion and Reconstruction • Correlation-Based Fusion • Multi-Scale Edge Matching • Dynamic Programming • Using Three or More Cameras • Reading: FP Chapter 11, S Chapter 11

3. An application: mobile robot navigation The INRIA Mobile Robot, 1990. The Stanford Cart, H. Moravec, 1979. Courtesy O. Faugeras and H. Moravec.

4. Stereo vision • Fusion: match points observed by two or more cameras • Reconstruction: find the pre-image of the matching points in 3D world • Assume calibrated camera and essential matrix/trifocal tensors associated with 3 cameras are known

5. Reconstruction / triangulation

6. (Binocular) Fusion

7. Reconstruction Geometric approach Algebraic approach • Linear Method: • find P such that • Non-Linear Method: find Q minimizing

8. Rectification All epipolar lines are parallel in the rectified image plane

9. Reconstruction from rectified images Disparity: d=u’-u Depth: z = -B/d × f

10. 2 Minimize |w-w’| Correlation methods (1970--) Slide the window along the epipolar line until w.w’ is maximized. Normalized Correlation: minimize q instead.

11. Correlation methods: foreshortening problems Solution: add a second pass using disparity estimates to warp the correlation windows, e.g. Devernay and Faugeras (1994)

12. Multi-scale edge matching (Marr, Poggio and Grimson, 1979-81) • Edges are found by repeatedly smoothing the image and detecting • the zero crossings of the second derivative (Laplacian) • Matches at coarse scales are used to offset the search for matches • at fine scales (equivalent to eye movements)

13. Multi-scale edge matching (Marr, Poggio and Grimson, 1979-81) One of the two input images Image Laplacian Zeros of the Laplacian

14. Multi-scale edge matching (Marr, Poggio and Grimson, 1979-81)

15. The ordering constraint In general the points are in the same order on both epipolar lines. But it is not always the case..

16. Ordering constraints points are not necessarily in order d-b-a c’-b’-d’

17. Dynamic programming (Baker and Binford, 1981) • Find the minimum-cost path going monotonically • down and right from the top-left corner of the • graph to its bottom-right corner. • Nodes = matched feature points (e.g., edge points). • Arcs = matched intervals along the epipolar lines. • Arc cost = discrepancy between intervals.

18. Dynamic programming (Ohta and Kanade, 1985) use inter-scanline f or point correspondence on vertical edges use DP for both intra-scanline and inter-scanline search

19. Three views The third eye can be used for verification.. b1-a2 match is wrong as thee is no corresponding point in camera 3

20. More views (Okutami and Kanade, 1993) Pick a reference image, and slide the corresponding window along the corresponding epipolar lines of all other images, using inverse depth relative to the first image as the search parameter. Use the sum of correlation scores to rank matches.

21. I1 I2 I10

22. Correspondence • Extensive literature • Region based • Graph cut • Mutual information • Belief propagation • Conditional random field • Smooth surfaces

23. Middlebury stereo dataset • De facto data set with ground truth, code, and comprehensive performance evaluation • http://vision.middlebury.edu/stereo/data/

24. Performance evaluation