1 / 90

Vision Sensors for Stereo and Motion

Explore the use of mirrors in stereo vision systems for improved matching. Learn calibration parameters, rectification methods, and real-time depth map generation. Discover optimal rectifying transformations to minimize resampling effects.

jmichaels
Download Presentation

Vision Sensors for Stereo and Motion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Vision Sensors for Stereo and Motion Joshua Gluckman Polytechnic University

  2. Stereo Vision depth map

  3. Stereo With Mirrors [ Gluckman and Nayar (CVPR 99)]

  4. Why Use Mirrors? • Identical system response • Better stereo matching • Faster stereo matching

  5. Why Use Mirrors? • Identical system response • Better stereo matching • Faster stereo matching • Data acquisition • No synchronization • Data Storage

  6. Stereo Systems Using Mirrors Mitsumoto `92 Inaba `93 Goshtasby and Gruver `93 Teoh and Zhang `84 Mathieu and Devernay `95 Zhang and Tsui `98

  7. Geometry and Calibration

  8. Background – Relative Orientation p p` C C` R,t – 6 parameters

  9. Background – Epipolar Geometry p p` C e C` e`

  10. Background – Epipolar Geometry 3 p p` C e C` e` 4 Epipolar geometry – 7 parameters

  11. Background – Epipolar Geometry 3 p p` C e C` e` 4 Epipolar geometry – 7 parameters

  12. One Mirror – Relative Orientation mirror virtual camera camera

  13. One Mirror – Relative Orientation virtual camera camera 3 parameters

  14. One Mirror – Relative Orientation virtual camera camera 3 parameters

  15. One Mirror – Epipolar Geometry 2 parameters – location of epipole

  16. Two Mirrors – Relative Orientation D virtual camera virtual camera camera

  17. - 1 D 1 Two Mirrors – Relative Orientation - = = 1 D D D D D 1 2 1 2 virtual camera virtual camera D 2 camera

  18. Two Mirrors – Relative Orientation virtual camera virtual camera q 5 parameters camera

  19. 4 Two Mirrors – Epipolar Geometry 6 parameters 2 p p` V e V` e`

  20. image of the axis m Two Mirrors – Epipolar Geometry p` p epipole e` epipole e

  21. image of the axis m Two Mirrors – Epipolar Geometry p1` p1 p2 p2` epipole e` epipole e p3` p3 p4 p4`

  22. (b) (a) (c) (d)

  23. Calibration Parameters Relative orientation Epipolar geometry

  24. Mirror Stereo Systems

  25. Get Images Rectify Matching Depth Map Real Time Stereo System Calibrate

  26. Rectification of Stereo Images Perspective transformations

  27. Why Rectify Stereo Images? • Fast stereo matching • O(hw2s)  O(hw2) • Removes differences in rotation and scale

  28. Not All Rectification Transforms Are the Same

  29. Rectification – Previous Methods Ayache and Hansen `88 3D methods – need calibration Faugeras `93 Robert et al. `93 2D methods – rectify from epipolar geometry Hartley `98 Loop and Zhang `99 Roy et al. `97 Non-perspective transformations Pollefeys et al. `99

  30. The Bad Effects of Resampling the Images • Creation of new pixels causes • Blurs the texture • Additional computation • Loss of pixels • Loss of information • Aliasing [Gluckman and Nayar CVPR ’01]

  31. Measuring the Effects of Resampling determinant of the Jacobian change in local area

  32. Measuring the Effects of Resampling determinant of the Jacobian change in local area

  33. Measuring the Effects of Resampling determinant of the Jacobian change in local area

  34. Change In Aspect Ratio Preserves Local Area pixels created pixels lost

  35. Skew Preserves Local Area aliasing

  36. Minimizing the Effects of Resampling change in local area • P and P’ must be rectifying transformation • No change in aspect ratio and skew

  37. e ¢ e e ¢ e The Class of Rectifying Transformations Fundamental matrix Rotation and translation

  38. e ¢ e The Class of Rectifying Transformations

  39. e ¢ e ¢ e e The Class of Rectifying Transformations

  40. e ¢ e ¢ e e The Class of Rectifying Transformations 6 parameters

  41. e ¢ e ¢ e e The Class of Rectifying Transformations no skew maintain aspect ratio 2 parameters

  42. The Class of Rectifying Transformations scale perspective distortion 2 parameters

  43. Finding the Best Rectifying Transform change in local area Find p1 and p8 that minimize e

  44. Finding the Best Rectifying Transform change in local area Find p1 and p8 that minimize e • e is quadratic in p1 so the optimal scale can be found as a function of p8 • e is a 16th degree rational polynomial in p8

  45. e e 1 2 Finding the Best Rectifying Transform • e1 and e2 are symmetric convex polynomials • e1 has a minimum at p8 = 0 • e2 has a minimum at p8 = f5 The minimum of e is between 0 and f5

  46. e e 1 2 Finding the Best Rectifying Transform e1 and e2 depend on the location of epipoles epipoles at the same distance

  47. e e 1 2 Finding the Best Rectifying Transform e1 and e2 depend on the location of epipoles epipoles at a distance of 3 and 10

  48. Rectifying While Minimizing Resampling Effects Step 1: Rotate and translate the epipolar geometry

More Related