EECS 274 Computer Vision

1. EECS 274 Computer Vision Geometric Camera Calibration

2. Geometric camera calibration • Camera calibration problem • Least-squares techniques • Linear calibration from points • Analytical photogrammetry • Reading: Chapter 1 and 22 of FP, Chapters 2, 6 of S

3. Calibration • Determine the intrinsic and extrinsic parameters • Assume that the camera observes a set of features (points, or lines) with known positions • Calibration: modeled as an optimization to minimize the discrepancy between the observed image features and their theoretical projections (using the perspective projection equations)

4. Calibration problem Given n points, P1, …, Pn with known positions and their images points, p1, …, pn, find ξ

5. Linear systems Square system: A x b • unique solution • Gaussian elimination = Rectangular system ?? • underconstrained: • infinity of solutions A x b = • overconstrained: • no solution 2 Minimize ||Ax-b||

6. Overconstrained problems

7. In matrix form Can be derived from the perspective of projection matrix

8. Homogenous linear systems Square system: A x 0 • unique solution: 0 • unless Det(A)=0 = Rectangular system ?? • 0 is always a solution A x 0 = 2 Minimize |Ax| under the constraint |x| =1 2

9. Overconstrained homogenous linear systems The solution is e . 1

10. Example: linear fitting Problem: minimize with respect to (a,b,d). • Minimize E with respect to d: • Minimize E with respect to a,b: where • Solution is the unit eigenvector with minimum eigenvalue

11. Note • Matrix of second moments of inertia • Axis of least inertia in mechanics

12. Linear camera calibration

13. When M is known Once M is known, need to recover the intrinsic and extrinsic parameters This is a decomposition problem, not an estimation problem r ρ: scale factor • Intrinsic parameters • Extrinsic parameters

14. Decomposition of M As the recovered Orthonormal basis vector θ is close to π/2 and has positive sine

15. Degenerate point configuration Are there other solutions besides M ? • One solution: (l,m,n)=(m1, m2, m3) • Consider the points Pi all lie in some plane, s.t., P∙Pi=0 for some P • Coplanar points: choose (l,m,n)=(P,0,0) or (0,P,0) or (0,0,P ), or • any linear combination of these vectors yields a solution Does not (usually) happen for 6 or more random points!

16. Radial distortion • Depends on the distance between the image center and an image point, d Barrel distortion Corners are detected by fitting lines in each square Using estimated distortion parameters

17. Correct radial distortion • Tsai’s algorithm (1987) exploits radial alignment constraints for estimating extrinsic parameters 11+q parameters

18. Analytic photogrammetry Given n points, P1, …, Pn with known positions and their images situations, p1, …, pn, find ξ Non-Linear Least-Squares Methods • Newton • Gauss-Newton • Levenberg-Marquardt Iterative, quadratically convergent in favorable situations

19. Application Mobile Robot Localization (Devy et al., 1997)

20. Calibration • Numerous ways that exploits properties of projective geometry • e.g. calibration using lines, calibration circular controlled points

21. Camera calibration toolbox • Excellent MATLAB toolbox by Jean-Yves Bouguet http://www.vision.caltech.edu/bouguetj/calib_doc/ • Steps: • Generate calibration board • Collect images under different views • Select extreme points • Find corner points • Solve optimization problem

22. Calibration images

23. Extreme points

24. Guessed grid corners

25. Corner extraction

26. Repeat for all other images

27. Solving optimization problem

28. Reprojected corners

29. Camera centered view

30. World centered view

31. Applications • Augmented reality • Image registration • Image stitching • Panoramic image

32. Panoramic image

33. Notes • Camera pose estimation • Multi-camera calibration • Auto/self calibration • Multi-camera self calibration • Projective geometry • Multi-view geometry • RANSAC (RANdom Sample Consensus)