Lecture 04

1. Lecture 04 22/11/2011 ShaiAvidan הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.

2. Epipolar Geometry • Essential Matrix (Longuet-Higgins 1981) • Fundamental Matrix (Faugeras 1992) • F and Homographies

3. Essential Matrix

4. epipolar line epipolar line epipolar plane Stereo correspondence constraints • Geometry of two views allows us to constrain where the corresponding pixel for some image point in the first view must occur in the second view. • Epipolar constraint: Why is this useful? • Reduces correspondence problem to 1D search along conjugateepipolar lines Adapted from Steve Seitz

5. Epipolar geometry: terms • Baseline: line joining the camera centers • Epipole: point of intersection of baseline with the image plane • Epipolar plane: plane containing baseline and world point • Epipolar line: intersection of epipolar plane with the image plane • All epipolar lines intersect at the epipole • An epipolar plane intersects the left and right image planes in epipolar lines

6. Epipolar constraint • Potential matches for p have to lie on the corresponding • epipolar line l’. • Potential matches for p’ have to lie on the corresponding • epipolar line l. http://www.ai.sri.com/~luong/research/Meta3DViewer/EpipolarGeo.html Source: M. Pollefeys

7. Example

8. Example: converging cameras As position of 3d point varies, epipolar lines “rotate” about the baseline Figure from Hartley & Zisserman

9. Example: motion parallel with image plane Figure from Hartley & Zisserman

10. Example: forward motion e’ e Epipole has same coordinates in both images. Points move along lines radiating from e: “Focus of expansion” Figure from Hartley & Zisserman

11. Stereo geometry, with calibrated cameras • If the rig is calibrated, we know : • how to rotate and translate camera reference frame 1 to get to camera reference frame 2. • Rotation: 3 x 3 matrix; translation: 3 vector.

12. 3d rigid transformation ‘ ‘ ‘

13. Stereo geometry, with calibrated cameras Camera-centered coordinate systems are related by known rotation R and translation T:

14. Cross product Vector cross product takes two vectors and returns a third vector that’s perpendicular to both inputs. So here, c is perpendicular to both a and b, which means the dot product = 0.

15. From geometry to algebra Normal to the plane

16. Matrix form of cross product Can be expressed as a matrix multiplication.

17. From geometry to algebra Normal to the plane

18. Essential matrix Let This holds for the rays p and p’ that are parallel to the camera-centered position vectors X and X’, so we have: E is called the essential matrix, which relates corresponding image points [Longuet-Higgins 1981]

19. Essential matrix and epipolar lines Epipolar constraint: if we observe point p in one image, then its position p’ in second image must satisfy this equation. is the coordinate vector representing the epipolar line associated with point p is the coordinate vector representing the epipolar line associated with point p’

20. Essential matrix: properties • Relates image of corresponding points in both cameras, given rotation and translation • Assuming intrinsic parameters are known • E is a rank 2 matrix with two equal eigvenvalues • Given E, the camera projection matrix can be computed using SVD factorization. There are four possible solutions

21. From E to Motion

22. Fundamental Matrix

23. Uncalibrated case Camera coordinates Image pixel coordinates Internal calibration matrices, one per camera For a given camera: Camera coordinates So, for two cameras (left and right):

24. Uncalibrated case: fundamental matrix From before, the essential matrix E. Fundamental matrix

25. Fundamental matrix • Relates pixel coordinates in the two views • More general form than essential matrix: we remove need to know intrinsic parameters • If we estimate fundamental matrix from correspondences in pixel coordinates, can reconstruct epipolar geometry without intrinsic or extrinsic parameters

26. Computing F from correspondences Cameras are uncalibrated: we don’t know E or left or right K matrices Estimate F from 8+ point correspondences.

27. Computing F from correspondences Each point correspondence generates one constraint on F Collect n of these constraints Solve for f , vector of parameters.

28. F and Homographies

29. F decomposition • Given F, we want to decompose to left and right projection matrices. Because reconstruction is up to global projective transformation we can set the left camera projection matrix to [I;0]. • Since p’^tFp=0 for every pair of points, there is one point e’ s.t. e’^tFp=0 for every p -> e’ is in the null space of F. • e’ is the epipole and all epipolar lines go through it. • From the epipolar constraint (shown for the essential matrix) we have that • So, we have M_1=[I;0] and M_2 = [H,e’] • Where H is a homography

30. The rank-4 of homographies

31. The canonical homography • Choose some line l’ going through p’ then: • l’^T p’ = 0 • And from the epipolar constraint we have: • p’^TFp = 0 • Combining the two we have that p’ is on the intersection of the two lines: • P’ =~ [l’]_x Fp • And we have a homography equation: p’ =~ [l’]_x Fp =Hp • In particular, lets choose l’ =~ e’, which is guaranteed not to coincide with the epipolar lines (e’Te’ !=0) and have that H = [e’]_xF is a homography that is called the canonical homography matrix

32. Relationship between F and H What is the relationship between homography and fundamental matrix? Let’s choose some homography, then we have: (Hp)^TFp = 0 and this is true for every point p. So we have that Fp =~ [e’]_x Hp, because they both describe the epipolar line s.t. p’^TFp and p’^T [e’]_x Hp=0 Geometric interpretation: The points Hp, p’ and e’ are all on the same line (the epipolar line) And this can be written as: p’^T[e]_xHp=0