Stereo Matching Using Dynamic Programming

1. Stereo Matching Using Dynamic Programming Jim Rehg CS 4495/7495 Computer Vision Lecture 4 Mon Sept 2, 2002

2. Correspondence • It is fundamentally ambiguous, even with stereo constraints Ordering constraint… …and its failure

3. Occluded Pixels Dis-occluded Pixels Search Over Correspondences Three cases: • Sequential – cost of match • Occluded – cost of no match • Disoccluded – cost of no match Left scanline Right scanline

4. Occluded Pixels Stereo Matching with Dynamic Programming Dynamic programming yields the optimal path through grid. This is the best set of matches that satisfy the ordering constraint Left scanline Start Dis-occluded Pixels Right scanline End

5. Dynamic Programming 1 1 1 2 2 2 3 3 3 Principle of Optimality for an n-stage assignment problem:

6. Dynamic Programming 1 1 1 2 2 2 3 3 3 Principle of Optimality for an n-stage assignment problem:

7. Dynamic Programming 1 1 1 2 2 2 3 3 3 Principle of Optimality for an n-stage assignment problem:

8. Dynamic Programming 1 1 1 2 2 2 3 3 3 Principle of Optimality for an n-stage assignment problem:

9. Dynamic Programming 1 1 1 2 2 2 3 3 3 Principle of Optimality for an n-stage assignment problem:

10. Dynamic Programming 1 1 1 2 2 2 3 3 3 Back-chaining recovers the optimal path and its cost:

11. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal

12. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal

13. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal

14. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal

15. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal

16. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal

17. Occluded Pixels Stereo Matching with Dynamic Programming Scan across grid computing optimal cost for each node given its upper-left neighbors.Backtrack from the terminal to get the optimal path. Left scanline Dis-occluded Pixels Right scanline Terminal

18. Computing Correspondence • Another approach is to match edges rather than windows of pixels: • Which method is better? • Edges tend to fail in dense texture (outdoors) • Correlation tends to fail in smooth featureless areas

19. Computing Correspondences • Both methods fail for smooth surfaces • There is currently no good solution to the correspondence problem