1 / 49

Phase Correlation

Phase Correlation. Phase Correlation. Take cross correlation. Take inverse Fourier transform.  Location of the impulse function gives the translation amount between the images. Phase Correlation. Computer Vision . Stereo Vision. Coordinate Systems.

bat
Download Presentation

Phase Correlation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Phase Correlation

  2. Phase Correlation Take cross correlation Take inverse Fourier transform  Location of the impulse function gives the translation amount between the images

  3. Phase Correlation

  4. Computer Vision Stereo Vision

  5. Coordinate Systems • Let O be the origin of a 3D coordinate system spanned by the unit vectors i, j, and k orthogonal to each other. i P O k j Coordinate vector

  6. Homogeneous Coordinates n H P O Homogeneous coordinates

  7. Coordinate System Changes • Translation

  8. Coordinate System Changes • Rotation where Exercise: Write the rotation matrix for a 2D coordinate system.

  9. Coordinate System Changes • Rotation + Translation

  10. Perspective Projection • Perspective projection equations

  11. Review: Pinhole Camera

  12. Review: Perspective Projection

  13. 3D World Points • Camera Centers • Image Points Multi-View Geometry Relates • Camera Orientations • Camera Parameters

  14. Stereo scene point p p’ image plane optical center

  15. Finding Correspondences p p’

  16. Three Questions • Correspondence geometry: Given an image point p in the first view, how does this constrain the position of the corresponding point p’ in the second? • Camera geometry (motion): Given a set of corresponding image points {pi ↔ p’i}, i=1,…,n, what are the cameras C and C’ for the two views? Or what is the geometric transformation between the views? • Scene geometry (structure): Given corresponding image points pi ↔ p’i and cameras C, C’, what is the position of the point X in space?

  17. Epipolar Line p’ Y2 X2 Z2 O2 Epipole Stereo Constraints M Image plane Y1 p O1 Z1 X1 Focal plane

  18. Epipolar Constraint

  19. P p p’ O’ O From Geometry to Algebra All vectors shown lie on the same plane.

  20. P p p’ O’ O From Geometry to Algebra

  21. Matrix form of cross product a=axi+ayj+azk a×b=|a||b|sin(η)u b=bxi+byj+bzk

  22. The Essential Matrix Essential matrix

  23. Stereo Vision • Two cameras. • Known camera positions. • Recover depth.

  24. Recovering Depth Information P Q P’1 P’2=Q’2 Q’1 O2 O1 Depth can be recovered with two images and triangulation.

  25. disparity Depth Z Elevation Zw A Simple Stereo System LEFT CAMERA RIGHT CAMERA baseline Right image: target Left image: reference Zw=0

  26. Stereo View Right View Left View Disparity

  27. Stereo Disparity • The separation between two matching objects is called the stereo disparity.

  28. Parallel Cameras P Z xl xr f pl pr Ol Or Disparity: T T is the stereo baseline

  29. Finding Correspondences

  30. (xl, yl) Correlation Approach LEFT IMAGE • For Each point (xl, yl) in the left image, define a window centered at the point

  31. Correlation Approach RIGHT IMAGE (xl, yl) • … search its corresponding point within a search region in the right image

  32. Correlation Approach RIGHT IMAGE (xr, yr) dx (xl, yl) • … the disparity (dx, dy) is the displacement when the correlation is maximum

  33. epipolar line epipolar line epipolar plane Stereo correspondence • Epipolar Constraint • Reduces correspondence problem to 1D search along epipolar lines

  34. For each epipolar line For each pixel in the left image Of course, matching single pixels won’t work; so, we match regions around pixels. Stereo correspondence • Compare with every pixel on same epipolar line in right image • Pick pixel with the minimum matching error

  35. ? = g f Most popular Comparing Windows For each window, match to closest window on epipolar line in other image.

  36. Comparing Windows Minimize Sum of Squared Differences Maximize Cross correlation

  37. Feature-based correspondence • Features most commonly used: • Corners • Similarity measured in terms of: • surrounding gray values (SSD, Cross-correlation) • location • Edges, Lines • Similarity measured in terms of: • orientation • contrast • coordinates of edge or line’s midpoint • length of line

  38. line corner structure Feature-based Approach LEFT IMAGE • For each feature in the left image…

  39. line corner structure Feature-based Approach RIGHT IMAGE • Search in the right image… the disparity (dx, dy) is the displacement when the similarity measure is maximum

  40. Correspondence Difficulties • Why is the correspondence problem difficult? • Some points in each image will have no corresponding points in the other image. (1) the cameras might have different fields of view. (2) due to occlusion. • A stereo system must be able to determine the image parts that should not be matched.

  41. Structured Light • Structured lighting • Feature-based methods are not applicable when the objects have smooth surfaces (i.e., sparse disparity maps make surface reconstruction difficult). • Patterns of light are projected onto the surface of objects, creating interesting points even in regions which would be otherwise smooth. • Finding and matching such points is simplified by knowing the geometry of the projected patterns.

  42. Stereo results • Data from University of Tsukuba Scene Ground truth (Seitz)

  43. Results with window correlation Estimated depth of field (a fixed-size window) Ground truth (Seitz)

  44. Results with better method • A state of the art method • Boykov et al., Fast Approximate Energy Minimization via Graph Cuts, • International Conference on Computer Vision, September 1999. Ground truth (Seitz)

  45. W = 3 W = 20 Window size • Effect of window size • Better results with adaptive window • T. Kanade and M. Okutomi,A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment,, Proc. International Conference on Robotics and Automation, 1991. • D. Scharstein and R. Szeliski. Stereo matching with nonlinear diffusion. International Journal of Computer Vision, 28(2):155-174, July 1998 (Seitz)

  46. Other constraints • It is possible to put some constraints. • For example: smoothness. (Disparity usually doesn’t change too quickly.)

  47. P Pl Pr Yr p p r l Yl Xl Zl Zr fl fr Ol Or R, T Xr Parameters of a Stereo System • Intrinsic Parameters • Characterize the transformation from camera to pixel coordinate systems of each camera • Focal length, image center, aspect ratio • Extrinsic parameters • Describe the relative position and orientation of the two cameras • Rotation matrix R and translation vector T

  48. Applications First-down line courtesy of Sportvision

  49. Applications Virtual advertising courtesy of Princeton Video Image

More Related