Content

1. ROBOT VISIONLesson 3: Projective GeometryMatthias RütherSlides courtesy of Marc Pollefeys Department of Computer ScienceUniversity of North Carolina, Chapel Hill

2. Content • Projective geometry in 3D • Points • Planes • Lines • Points from planes, planes from points • Hierarchy of transformations / invariants • Parameter Estimation • Problem: intersecting lines -> SVD • Problem: rectification • Normalization • Robust Estimation • Algorithm Evaluation • Error measures

3. Projective 3D Geometry • Points, lines, planes and quadrics • Transformations

4. 3D points 3D point in R3 in P3 projective transformation (4x4-1=15 dof)

5. Transformation Euclidean representation Planes 3D plane Dual: points ↔ planes, lines ↔ lines

6. (solve as right nullspace of ) Planes from points Or implicitly from coplanarity condition

7. (solve as right nullspace of ) Points from planes Representing a plane by its span

8. Lines (4dof) Example: X-axis

9. Points, lines and planes

10. Plücker matrices Plücker matrix (4x4 skew-symmetric homogeneous matrix) • L has rank 2 • 4dof • generalization of • L independent of choice A and B • Transformation Example: x-axis

11. Plücker matrices Dual Plücker matrix L* Correspondence Join and incidence (plane through point and line) (point on line) (intersection point of plane and line) (line in plane) (coplanar lines)

12. on Klein quadric Plücker line coordinates

13. Plücker line coordinates (Plücker internal constraint) (two lines intersect) (two lines intersect) (two lines intersect)

14. Estimation • 2D Problem: Lines l1, l2, l3 intersect in a point l1 ?? Point of intersection x x l2 l3 • Solve equation system: How many solutions exist? When does a solution exist? Geometric interpretation?

15. Estimation • Practical 2D Problem: measured lines l1, l2, l3, … lmapproximately intersect in a point x l1 ?? Point of intersection x x l2 l3 lm • Find an approximate solution to the system: Find a solution that minimizes |Lx|.

16. Singular Value Decomposition (SVD) • SVD: algorithm that decomposes Amxn (mn) to • Properties: • U has orthogonal columns: UTU = I • U is norm-preserving: |Ux| = |x| • D is diagonal, diagonal elements are the singular values • V is orthogonal: VTV = I

17. Singular Value Decomposition (SVD) • Solve Ax=0  minimize |Ax| subject to |x|=1: • minimize |DVTx| subject to |VTx|=1 1 is in the same row as the smallest singular value in D. X is the column of V corresponding to the smallest Singular Value.

18. Robust Estimation • Practical 2D Problem: SVD obtains an optimal least squares solution: l1 x l2 l3 • Problem with outliers: l4 l1 x l2 l3

19. RANdom SAmple Consensus • RANSAC algorithm: given a set of lines l1,…,lm, find an intersection point x such that |lkx| is minimal for k=1..m, subject to the condition that none of the valid lines deviates from x by more than t. lm l4 l1 t x l2 l3

20. RANSAC • Objective • Robust fit of model to data set S which contains outliers • Algorithm • Randomly select a sample of s data points from S and instantiate the model from this subset. • Determine the set of data points Si which are within a distance threshold t of the model. The set Si is the consensus set of samples and defines the inliers of S. • If the subset of Si is greater than some threshold T, re-estimate the model using all the points in Si and terminate • If the size of Si is less than T, select a new subset and repeat the above. • After N trials the largest consensus set Si is selected, and the model is re-estimated using all the points in the subset Si

21. Parameter estimation • 2D homography Given a set of (xi,xi’), compute H (xi’=Hxi) • 3D to 2D camera projection Given a set of (Xi,xi), compute P (xi=PXi) • Fundamental matrix Given a set of (xi,xi’), compute F (xi’TFxi=0 x3 x3‘ x4‘ x4 ?H? x2 x1 x1‘ x2‘

22. Number of measurements required • At least as many independent equations as degrees of freedom required • Example: 2 independent equations / point 8 degrees of freedom 4x2≥8

23. Approximate solutions • Minimal solution 4 points yield an exact solution for H • More points • No exact solution, because measurements are inexact (“noise”) • Search for “best” according to some cost function • Algebraic or geometric/statistical cost

24. Gold Standard algorithm • Cost function that is optimal for some assumptions • Computational algorithm that minimizes it is called “Gold Standard” algorithm • Other algorithms can then be compared to it

25. Direct Linear Transformation(DLT)

26. Direct Linear Transformation(DLT) • Equations are linear in h • Only 2 out of 3 are linearly independent (indeed, 2 eq/pt) (only drop third row if wi’≠0) • Holds for any homogeneous representation, e.g. (xi’,yi’,1)

27. Direct Linear Transformation(DLT) • Solving for H size A is 8x9 or 12x9, but rank 8 Trivial solution is h=09T is not interesting 1-D null-space yields solution of interest pick for example the one with

28. Direct Linear Transformation(DLT) • Over-determined solution No exact solution because of inexact measurement i.e. “noise” • Find approximate solution • Additional constraint needed to avoid 0, e.g. • not possible, so minimize

29. DLT algorithm • Objective • Given n≥4 2D to 2D point correspondences {xi↔xi’}, determine the 2D homography matrix H such that xi’=Hxi • Algorithm • For each correspondence xi ↔xi’ compute Ai. Usually only two first rows needed. • Assemble n 2x9 matrices Ai into a single 2nx9 matrix A • Obtain SVD of A. Solution for h is last column of V • Determine H from h

30. Inhomogeneous solution Since h can only be computed up to scale, pick hj=1, e.g. h9=1, and solve for 8-vector Solve using Gaussian elimination (4 points) or using linear least-squares (more than 4 points) However, if h9=0 this approach fails also poor results if h9 close to zero Therefore, not recommended Note h9=H33=0 if origin is mapped to infinity