Camera parameters

1. Camera parameters

2. Extrinisic parameters define location and orientation of camera reference frame with respect to world frame • Intrinsic parameters define pixel coordinates of image point with respect to coordinates in camera reference frame

3. Homogenous coordinates Add an extra coordinate and define equivalence Relationship (x,y) -> (kx, ky, k) (X,Y,Z) ->(wX, wY, wZ, w) Makes it possible to write the Perspective projection as a linear Transformation (matrix) (from projective space to projective plane)

4. Central projection HC NHC

5. Scaled orthographic projection HC NHC

10. In a simpler notation T describes the position of the origin of camera frame with respect to world frame R describes the rotation which aligns the camera frame with the world frame Pc = R(Pw – T) (here –RT = BOA)

11. Translation and Rotation

12. Intrinsic parameters y ypix x xpix Scaling

13. Intrinsic parameters y ypix x xpix

14. Intrinsic parameters y ypix x xpix

15. The internal calibration parameters

16. with

17. Properties of matrix M • M has 11 degrees of freedom (5 internal 3 rotation, 3 translation parameters) , 3x4 matrix defined up to scale • The 3x3 submatrix M’=MintR is non-singular (Mint is upper triangular, R is orthogonal -> essential QR decomposition)

18. Radial distortion from lens distortion (pin cushioning effect) (significant error for cheap optics and short focal length) Straight lines are not imaged straight x and xd measured from image center

19. Radial calibration Using lines to be straight (x’,y’) is radial projection of (xd, yd) on straight line

21. Calibration Procedure • Calibration target : 2 planes at right angle with checkerboard (Tsai grid) • We know positions of corners of grid with respect to a coordinate system of the target • Obtain from images the corners • Using the equations (relating pixel coordinates to world coordinates) we obtain the camera parameters (the internal parameters and the external (pose) as a side effect)

22. Image Processing • Canny edge detection • Straight line fitting to detect long edges • Intersection of lines to detect image corners • Matching image corners and 3D checkerboard corners

23. Estimation procedure • First estimate M from corresponding image points and scene points (solving homogeneous equation) • Second decompose M into internal and external parameters • Use estimated parameters as starting point to solve calibration parameters non-linearly.

24. (homogeneous equation)

26. Solving A m = 0 Linear homogeneous system Have at least 5 times as many equations as unknowns (28 points) Minimize ||Am||2 with the constraint ||m||2=1 M is the unit singular value of A corresponding to the smallest singular value (the last column of V, where A = UDVT is the SVD of A), or the eigenvector (corresponding to smallest eigenvalue ) of ATA

27. Finding camera translation (position of camera center) Let be the homogeneous representation of T is the null vector of M: Null vector is found using SVD ( is the unit singular vector corresponding to the smallest singular value of M)

28. Finding camera orientation and internal parameters • Left 3x3 submatrix of M is of the form M’= • Mint R • Mint upper triangular • R orthogonal • Any nonsingular matrix can be decomposed • into the product of an upper triangular and • an orthogonal matrix (RQ factorization—here • R refers to upper triangular and Q to orthogonal) • (Similar to QR factorization)

29. RQ factorization of M’ • Givens rotations To set M’32 to zero, solve equation Thus: Multiply M’ by Rx ( such that term (3,2) is 0), then by, Ry (choosing c’, s’ such that term (3,1) is zero), then by Rz (with c’’, s’’ such that term (2,1) is zero)

30. Improving solution with nonlinear optimization Find m using the linear constraint Use as initialization for nonlinear optimization (Levenberg-Marquardt iterative minimization)

31. Algorithm described in Multiple View Geometry in Computer Vision (Hartley, Zisserman)