Multiple View Geometry

1. Multiple View Geometry Marc Pollefeys University of North Carolina at Chapel Hill Modified by Philippos Mordohai

2. Outline • 2-D Projective geometry • 3-D Projective geometry • Chapters 1,2 and 5 of “Multiple View Geometry in Computer Vision” by Hartley and Zisserman

3. Projective 2D Geometry • Points, lines & conics (last week) • Transformations & invariants

4. Theorem: A mapping h:P2P2is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P2 reprented by a vector x it is true that h(x)=Hx Definition: Projective transformation or 8DOF Projective transformations Definition: A projectivity is an invertible mapping h from P2 to itself such that three points x1,x2,x3lie on the same line if and only if h(x1),h(x2),h(x3) do. projectivity=collineation=projective transformation=homography

5. Mapping between planes central projection may be expressed by x’=Hx (application of theorem)

6. Removing projective distortion select four points in a plane with known coordinates (linear in hij) (2 constraints/point, 8DOF  4 points needed) Remark: no calibration at all necessary, better ways to compute (see later)

7. More examples

8. Transformation for conics Transformation for dual conics Transformation of lines and conics For a point transformation Transformation for lines

9. A hierarchy of transformations Projective linear group Affine group (last row (0,0,1)) Euclidean group (upper left 2x2 orthogonal) Oriented Euclidean group (upper left 2x2 det 1) Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant e.g. Euclidean transformations leave distances unchanged

10. orientation preserving: orientation reversing: Class I: Isometries (iso=same, metric=measure) 3DOF (1 rotation, 2 translation) special cases: pure rotation, pure translation Invariants: length, angle, area

11. Class II: Similarities (isometry + scale) 4DOF (1 scale, 1 rotation, 2 translation) also know as equi-form (shape preserving) metric structure = structure up to similarity (in literature) Invariants: ratios of length, angle, ratios of areas, parallel lines

12. Class III: Affine transformations 6DOF (2 scale, 2 rotation, 2 translation) non-isotropic scaling! (2DOF: scale ratio and orientation) Invariants: parallel lines, ratios of parallel lengths, ratios of areas

13. Class VI: Projective transformations 8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity) Action non-homogeneous over the plane Invariants: cross-ratio of four points on a line (ratio of ratios)

14. Action of affinities and projectivitieson line at infinity Line at infinity stays at infinity, but points move along line Line at infinity becomes finite, allows to observe vanishing points, horizon

15. Decomposition of projective transformations decomposition unique (if chosen s>0) upper-triangular, Example:

16. Overview of transformations Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio Projective 8dof Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l∞ Affine 6dof Ratios of lengths, angles. The circular points I,J Similarity 4dof Euclidean 3dof lengths, areas.

17. Projective 3D Geometry • Points, lines, planes and quadrics • Transformations • П∞, ω∞and Ω ∞

18. Hierarchy of transformations Projective 15dof Intersection and tangency Parallellism of planes, Volume ratios, centroids, The plane at infinity π∞ Affine 12dof Similarity 7dof The absolute conic Ω∞ Euclidean 6dof Volume

19. The plane at infinity The plane at infinity π is a fixed plane under a projective transformation H iff H is an affinity • canical position • contains all vanishing points • two planes are parallel  line of intersection in π∞ • line // line (or plane)  point of intersection in π∞ • fixed as set under affinities • Other planes may be fixed under some affinities, but π∞is fixed under all affinities

20. The absolute conic The absolute conic Ω∞ is a (point) conic on π. In a metric frame: or conic for directions: (with no real points) The absolute conic Ω∞ is a fixed conic under the projective transformation H iff H is a similarity • Ω∞is only fixed as a set • Circles intersect Ω∞ in two points • Spheres intersect π∞ in Ω∞

21. The absolute dual quadric The absolute conic Ω*∞ is a fixed conic under the projective transformation H iff H is a similarity • 8 dof • plane at infinity π∞ is the nullvector of Ω∞

22. Outline • 2-D Projective geometry • 3-D Projective geometry • Camera model re-visited

23. Pinhole camera model linear projection in homogeneous coordinates!

24. Pinhole camera model

25. Pinhole point offset principal point

26. Pinhole point offset calibration matrix

27. Camera rotation and translation

28. CCD camera

29. When is skew non-zero? arctan(1/s) g 1 for CCD/CMOS, always s=0 Image from image, s≠0 possible (non coinciding principal axis)

30. non-singular Finite projective camera 11 dof (5+3+3) decompose P in K,R,C? {finite cameras}={P4x3 | det M≠0} If rank P=3, but rank M<3, then cam at infinity

31. Camera anatomy Camera center Column points Principal plane Axis plane Principal point Principal ray

32. Camera center null-space camera projection matrix For all A all points on AC project on image of A, therefore C is camera center Image of camera center is (0,0,0)T, i.e. undefined Finite cameras: Infinite cameras:

33. Column vectors Image points corresponding to X,Y,Z directions and origin

34. Row vectors note: p1,p2 dependent on image reparametrization

35. principal point The principal point

36. (pseudo-inverse) Action of projective camera on point Forward projection Back-projection

37. =( )-1= -1 -1 R R Q Q Camera matrix decomposition Finding the camera center (use SVD to find null-space) Finding the camera orientation and internal parameters (use RQ decomposition ~QR) (if only QR, invert)

38. Euclidean vs. projective general projective interpretation Meaningfull decomposition in K,R,t requires Euclidean image and space Camera center is still valid in projective space Principal plane requires affine image and space Principal ray requires affine image and Euclidean space