EECS 274 Computer Vision

1. EECS 274 Computer Vision Geometric Camera Calibration

2. GEOMETRIC CAMERA CALIBRATION • Camera calibration problem • Least-squares techniques • Linear calibration from points • Analytical photogrammetry • Reading: Chapter 3 of FP, Chapters 2,6 of S

3. Calibration • Determine the intrinsic and extrinsic parameters • Assume that the camera observe a set of features (points, or lines) with known positions • Calibration: modeled as an optimization to minimize the discrepancy between the observed image features and their theoretical projections (using the perspective projection equations)

4. Calibration Problem Given n points, P1, …, Pn with known positions and their images points, p1, …, pn, find ξ

5. Linear Systems Square system: A x b • unique solution • Gaussian elimination = Rectangular system ?? • underconstrained: • infinity of solutions A x b = • overconstrained: • no solution Minimize |Ax-b| 2

6. How do you solve overconstrained linear equations ?

7. In matrix form Can be derived from the perspective projection matrix

8. Homogeneous Linear Systems Square system: A x 0 • unique solution: 0 • unless Det(A)=0 = Rectangular system ?? • 0 is always a solution A x 0 = 2 Minimize |Ax| under the constraint |x| =1 2

9. How do you solve overconstrained homogeneous linear equations ? The solution is e . 1

10. Example: Line Fitting Problem: minimize with respect to (a,b,d). • Minimize E with respect to d: • Minimize E with respect to a,b: where • Solution is the unit eigenvector with minimum eigenvalue

11. Note: • Matrix of second moments of inertia • Axis of least inertia in mechanics

12. Linear Camera Calibration min |Pm|2, |m|=1

13. Once M is known, you still got to recover the intrinsic and extrinsic parameters ! This is a decomposition problem, not an estimation problem. r • Intrinsic parameters • Extrinsic parameters

14. Decomposition of M As the recovered Orthonormal basis vector θ is close to π/2 and has positive sine

15. Degenerate Point Configurations Are there other solutions besides M ? • One solution: (l,m,n)=(m1, m2, m3) • Consider the points Pi all lie in some plane, s.t., P∙Pi=0 for some P • Coplanar points: choose (l,m,n)=(P,0,0) or (0,P,0) or (0,0,P ), or • any linear combination of these vectors yields a solution Does not (usually) happen for 6 or more random points!

16. Radial distortion • Depends on the distance separating the optical axis from the point of interest, d Barrel distortion Corners are detected by fitting lines in each square Using estimated distortion parameters

17. Correct radial distortion • Tsai’s algorithm (1987) exploits radial alignment constraints for estimating extrinsic parameters

18. Analytical Photogrammetry Given n points, P1, …, Pn with known positions and their images situations, p1, …, pn, find ξ Non-Linear Least-Squares Methods • Newton • Gauss-Newton • Levenberg-Marquardt Iterative, quadratically convergent in favorable situations

19. Mobile Robot Localization (Devy et al., 1997)

20. Calibration • Numerous ways that exploits properties of projective geometry • E.g. calibration using lines, calibration circular controlled points

21. Camera calibration toolbox • Excellent MATLAB toolbox by Jean-Yves Bouguet http://www.vision.caltech.edu/bouguetj/calib_doc/ • Steps: • Generate calibration board • Collect images under different views • Select extreme points • Find corner points • Solve optimization problem

22. Calibration images

23. Extreme points

24. Guessed grid corners

25. Corner extraction

26. Repeat for all other images

27. Solving optimization problem

28. Reprojected corners

29. Camera centered view

30. World centered view

31. Applications • Augmented reality • Image registration • Image stitching • Panoramic image

32. Panoramic image

33. Notes • Camera pose estimation • Multi-camera calibration • Auto/self calibration • Multi-camera self calibration • Projective geometry • Multi-view geometry • RANSAC (RANdom Sample Consensus)