1 / 35

Wide Angle Camera Calibration: Modeling and Correcting Distortion

Wide Angle Camera Calibration: Modeling and Correcting Distortion. Vitaliy Orekhov Imaging, Robotics, & Intelligent Systems Laboratory The University of Tennessee November 29, 2005. Tasks for this Semester. ECE503 – Modern Transforms ECE572 – Digital Image Processing

jessicathomas
Download Presentation

Wide Angle Camera Calibration: Modeling and Correcting Distortion

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Wide Angle Camera Calibration:Modeling and Correcting Distortion Vitaliy Orekhov Imaging, Robotics, & Intelligent Systems Laboratory The University of Tennessee November 29, 2005

  2. Tasks for this Semester • ECE503 – Modern Transforms • ECE572 – Digital Image Processing • ECE573 –Test, Evaluate, and Transfer Camera Calibration Code to C++ • ECE672 – Write Survey on Wide Angle Camera Calibration and Image Correction

  3. Outline • Wide angle lenses and omnivision • Camera Calibration • Lens Projections • Distortion Models • Calibrating Distortion • Line based approach • Point correspondence • Converting Calibration Code to C++

  4. Chris Broaddus, “Universal Geometric Camera Calibration with Statistical Model Selection” Wide-Angle Lenses • Normal lenses Field-Of-View (FOV) is about 50 degrees • Wide angle lenses FOV greater than 50 degrees • Ultra wide angle lenses FOV greater than 70 (140) degrees • Fish-eye lenses have FOV of around 180 degrees • Rectilinear lens • Pin-hole camera model • Straight lines are projected straight • For large FOV details at edges are stretched • Fish-eye lens • If lines do not run through the center of frame then they appear as curves

  5. NIKON 14MM f/2.8D ED Ultra Wide Angle AF Nikkor Lens www.amazon.com Wide-Angle Lenses • Easier to map local information for visual search, navigation, or detection • M.M. Fleck, “ Perspective projection: The wrong imaging model,” Technical Report TR 95-01, Computer Science Dept., University of Iowa, 1995. • Mapping the local environment for visual search, planning actions, navigation, and detection of hazards • Obtaining a representative sample of colors for color constancy or a large set of features for identifying one’s current location • Imaging large objects, nearby objects, and objects in a confined space • Robust analysis of egomotion (estimation of the observers motion)

  6. www.0-360.com Omni-directional Vision • Desired for tracking/observing multiple moving targets or objects • Camera network • Panning a camera takes time and is not sufficient for real time applications • Single omni-directional camera (catadioptric) • Applications: Robot navigation, stereo reconstruction, surveillance

  7. Shimamura, J.; Yokoya, N.; Takemura, H.; Yamazawa, K., "Construction of an immersive mixed environment using an omnidirectional stereo image sensor," Omnidirectional Vision, 2000. Proceedings. IEEE Workshop on , vol., no.pp.62-69, 2000 Twelve individual images generated from omnidirectional sensor. Pair of reconstructed panoramic stereo images.

  8. Calibration for Computer Vision • It is a necessary step in 3D computer vision to extract metric information from 2D images • Before the camera can be used for precise computer vision applications, the camera needs to be characterized • How does a point in 3D world coordinates get projected onto the camera imaging plane • Initially research in camera calibration was done by photogrammetry community

  9. Camera Calibration • Camera calibration is the process of calculating the intrinsic and extrinsic parameters of a camera • The intrinsic parameters describe the internal characteristics of the camera • focal length, camera pixel size, and principal point coordinates • The extrinsic parameters describe the position of the camera in the world • six parameters: three are for position of the center of projection, and three are for orientation of the image plane coordinate frame

  10. Camera Calibration • Most common calibration method is the Direct Linear Transformation (DLT). • Das, G.B. "A Mathematical Approach to Problems in Photogrammetry," Empire Survey Review, Vol X, No. 73, July 1949. • Abdel-Aziz Y. I., Karara H. M. “Direct Linear Transformation from Comparator Coordinates into Object Space Coordinates,” American Society of Photogrammetry Symposium on Close-Range Photogrammetry, Falls Church, Virginia, U.S.A., pp. 1-18, 1971 • Set of control points are used whose 3D coordinates are knows.

  11. Camera Calibration • Given 3D coordinates of control points and extracted 2D coordinates from an image homography matrix is calculated. • At least four points are required. • Using the homography matrix from each image camera matrix , rotation matrix, and translation matrix are calculated. • Variations of DLT method: • Normalized DLT and Gold Standard Algorithm.

  12. Modeling Lens Distortion • Two approaches to model wide-angle lenses • Lens projections • Radial distortion Deviation from the ideal pin-hole camera model • Christopher Broaddus, “Universal Geometric Camera Calibration with Statistical Model Selection,” Master’s Thesis, University of Tennessee, 2005. • Perspective cameras are modeled with radial distortion. • Lens projections allow for complex non-linear perspective projections. • Wide-angle and fisheye lenses are modeled better using lens projections rather than radial distortion.

  13. Lens Projection • (1) Perspective projection • (2) Stereographic projection • (3) Equidistance projection • (4) Equisolid angle projection • (5) Orthogonal projection Image from Universal Geometric Camera Calibration with Statistical Model Selection by Chris Broaddus Equidistance projection Perspective projection

  14. Lens Projection • J. Kannala and S. Brandt, “A Generic Camera Calibration Method for Fish-Eye Lenses,” In Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol. 1, pp. 10-13, 23-16 August 2004. • Real lenses do not exactly follow a particular projection model. • To have an automatic calibration procedure it would be beneficial to have a single model for different lenses. • Modeling the different projections can also be achieved with a single polynomial approximation. • The number of terms is fixed to include only k1 and k2.

  15. Lens Projection • H. Bakstein and T. Pajdla, “Panoramic Mosaicing with a 180/spl deg/ Field of View Lens,” In Proceedings Third Workshop on Omnidirectianl Vision, pp. 60-67, 2002. • To find the best model calibration pattern was wrapped around a cylinder. • Stereographic projection produced the most accurate results. • Model was extended to combine stereographic projection and equisolid projection. Nikon FC-8 fish-eye converter all images and figures from “Panoramic Mosaicing with a 180/spl deg/ Field of View Lens” 183° FOV

  16. Modeling Distortion In general, radial distortion is approximated by: Distortion function is mostly dominated by the first terms. (a) barrel distortion (blue) (b) pincushion distortion (red)

  17. Polynomial model Division model Approximated error from “Simultaneous linear estimation of multiple view geometry and lens distortion” Radial Distortion Models • A. W. Fitzgibbon, “Simultaneous linear estimation of multiple view geometry and lens distortion,” in Proceedings of the 2001 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2001, CVPR-2001, vol. 1, pp. I-125 – I-132. • Model which provides a multi-view relationship with lens distortion incorporated into the model. • Model which does not complicate the computations. • Division Model (DM) • RMS(PM) = 0.77 pixels • RMS(DM) = 0.65 pixels • Allowed Fitzgibbon to simultaneously estimate the fundamental matrix and radial distortion from multiple views.

  18. Radial Distortion Models • D. Claus and A. W. Fitzgibbon, “A rational function lens distortion model for general cameras,” In CVPR 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 213-219, 20-25 June 2005. • Rational function model is another model whith simultaneous linear estimation of distortion and lens parameters from two uncalibrated views of a 3D scene. • Polynomial and Division Models combined. • Model is not specific to any particular lens geometry. • F. Devernay and O. Faugeras, “Straight lines have to be straight. Automatic calibration and removal of distortion from scenes of structured environments,” Machine Vision and Applications, 13(1), pp. 14--24, 2001. • Field of View model is based on the way fish-eye lenses are designed. • Has only one parameter which is the field of view (w).

  19. Classifying Calibration Techniques • Calibration objects • One-dimensional line • Two-dimensional plane • Three-dimensional object • Self-calibration • Assumptions • Camera parameters • Distortion characteristics • Distortion center • Calibrating Distortion • Line based calibration • Point correspondence based calibration

  20. Wide Angle Calibration Techniques • Traditional method • Calibrate camera parameters and distortion together • Difficult to calculate camera parameters and distortion simultaneously • Attempt to separate the estimation of distortion and other camera parameters • Calibrate distortion without knowing internal camera parameters • S. Graf, T. Hanning, “Analytically solving radial distortion parameters”, in CVPR 2005 IEEE computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 20-26, June 2005. • Analytical method to solve for radial distortion • Projection matrix used to calculate distortion parameters • Internal and external camera parameters kept fixed • Once distortion parameters are found the optimization has better initial parameters • Smaller number of iterations required • Smaller error

  21. Line Based Distortion Calibration • Straight lines have to be straight • Early work by Brown using the Plumb line method • D. C. Brown, “Close-range camera calibration”, Photogrammetric Engineering, vol. 37, no. 8, pp. 323-344, 1971 • Single image can be used to find distortion parameters • Scene must contain lines • Manual and semi-automatic methods exist to find distorted lines. • Methods to extract distortion parameters • Iterative variation of distortion parameters • Sum of square distance • Slope of points

  22. Line Based Distortion Calibration • Extracting lines from images • User manually selecting lines • R. Swaminathan and S. K. Nayar, “Nonmetric calibration of wide-angle lenses and polycameras,” In IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 22, Iss. 10, pp. 172-1178, October 2000. • Semi-automatic extracting of lines • S.B. Kang, “Radial distortion snakes,” IEICE Trans. Inf. & Syst., vol. E84-D, No. 12, Dec. 2001, pp. 1603-1611. • User first draws lines at approximate line location • Deformable contours are fitted to the edges representing lines • Eliminate outliers in extracted data • F. Devernay and O. Faugeras. Straight lines have to be straight. Automatic calibration and removal of distortion from scenes of structured environments. Machine Vision and Applications, 13(1), pp. 14--24, 2001. • M, Ahmed and A. Farag, “Nonmetric Calibration of Camera Lens Distortion: Differential Methods and Robust Estimation,” IEEE Transactions on Image Processing, vol. 14, Iss. 8, pp. 1215-1230, August 2005. • Least-median-of-squares

  23. Line Based Distortion Calibration • Solving for Radial Distortion Parameters • R. Cucchiara, C. Grana, R. Vezzani, “A Hough transform-based method for radial lens distortion correction,” In 2003 Proceedings of 12th International Conference on Image Analysis and Processing, pp. 182-187, 17-19 September 2003. • First semi-automatic extraction of lines • Perform iterative variation of distortion parameters until the straightness of the found line(s) is maximized. • Hough Transform is used to detect and measure straightness of the lines. • F. Devernay and O. Faugeras, “Straight lines have to be straight. Automatic calibration and removal of distortion from scenes of structured environments,” Machine Vision and Applications, 13(1), pp. 14--24, 2001. • Sum of squares of the distances from a least square fit line are taken to measure amount of distortion. • Error is minimized by optimizing the distortion parameters with a nonlinear least-square minimization method.

  24. Line Based Distortion Calibration • Solving for Radial Distortion Parameters • M, Ahmed and A. Farag, “Nonmetric Calibration of Camera Lens Distortion: Differential Methods and Robust Estimation,” IEEE Transactions on Image Processing, vol. 14, Iss. 8, pp. 1215-1230, August 2005. • New distortion measure is derived which is minimized by a non-linear optimization algorithm and which provides a closed form solution to solve for the distortion parameters. • Minimizes the error of a function which relates the slopes of points that belong to the same distorted line. • Good estimation of distortion center must be known. • Equations representing the slopes of lines become rational functions if the distortion center is known.

  25. Point Correspondence Calibration • Point correspondence described by homography matrix. • Y. Meng and H. Zhuang, “What you see is what you get: self-calibration camera lens distortion,” IEEE Robotics and Automation Magazine, Vol. 11, Iss. 4, pp. 123-127, December 2004. • Uses point correspondence between two images. • Epipolar geometry • Fundamental matrix (F) can be found from eight matching points in the two images. • Epipolar distance is zero without noise or distortion. • With distortion Epipolar distance is minimized by an iterative method. • G. P. Stein, “Lens distortion calibration using point correspondences,” in IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 1997. pp. 602-608. 17-19 June 1997.

  26. Plane projection from lines 1D Radial Camera. Point Correspondence Calibration • S. Thirthala and M. Pollefeys, “Multi-view geometry of 1D radial cameras and its application to omnidirectional camera calibration.” UNC-CS-TR-05-007 • S. Thirthala and M. Pollefeys, “The radial trifocal tensor: a tool for calibrating the radial distortion of wide angle cameras,” in IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2005, vol. 1, pp. 321-328, 20-25 June 2005. • Metric reconstruction of features is performed using trifocal and quadrifocal constraints independently of radial distortion. • Then the reconstructed features are used to recover the distortion parameters for each image.

  27. Point Correspondence Calibration • Calibration matrix is calculated once the distortion is temporarily ignored by using the radial camera. • The reconstructed features are then used with any distortion model to find the distortion parameters. • Devision model is used because it provides a linear solution. • Method can be used to calibrate multiple cameras since distortion parameters can be approximated for each individual image after the reconstructed features have been obtained.

  28. Task 3 –Test, Evaluate, and Transfer Camera Calibration Code to C++

  29. Test, Evaluate, and Transfer Camera Calibration Code to C++ • Calibration code was originally produced by Christopher Broaddus • Written in MATLAB • Main sections/algorithms • Corner Detection and subpixel refinement • Homography estimation • Solving for intrinsic parameters • Solving for extrinsic parameters • Solving for the lens projection estimation • Decentering distortion • Complete model • Bundle adjustment • Model selection

  30. Test, Evaluate, and Transfer Camera Calibration Code to C++ Visual Studio C++ Intel’s Open Source Computer Vision (OpenCV) Library • OpenCV is cross-platform middle-to-high level API that consists of a few hundred C functions • Put together so that efforts of the vision community can be consolidated and performance optimized • Open source library is mainly aimed at real-time computer vision

  31. Test, Evaluate, and Transfer Camera Calibration Code to C++ • Corner detection and subpixel refinement • Harris corner detector. • Fitting lines to the chess pattern and finding the intersection to locate the control points.

  32. Test, Evaluate, and Transfer Camera Calibration Code to C++ • Homography estimation • Homography matrix is estimated for each image. • Known 3D coordinates of control points are used to obtain projection matrix for each image. • Direct linear transformation • Goldstandard algorithm • Intrinsic and Extrinsic Camera Parameter • Projection matrices are combined to find the calibration matrix. • Rotation matrix and translation vectors are also computed for each image.

  33. Results using C++ • Calibration Matrix 321.8031      31.72945      559.9948  0.000000      331.4059      234.10390.000000      0.000000      1.000000  • Rotation Matrix (1 of 8) 0.99775      0.02773      0.06101  0.03604      -0.98953     -0.139760.05649      0.14164      -0.98831  • Translation Matrix (1 of 8) -167.9748111.3087151.2151 Results using MATLAB • Calibration Matrix 321.8031      31.7295        559.9949  0.000000      331.4059      234.10390.000000      0.000000      1.000000  • Rotation Matrix (1 of 8) 0.9978       0.0277       0.0610  0.0360       -0.9895      -0.13980.0565       0.1416       -0.9883  • Translation Matrix (1 of 8) -167.9748111.3087151.2151

  34. Conclusions • General calibration method are beneficial since they can calibrate a wide range of lenses and cameras. • Both lens projections and distortion models should be able to model more than one lens. • Reliable model selecting methods should be used. • Separating calibration of distortion and other camera parameters provides advantages. Future work • Finish converting the calibration code. • Compare our calibration performance with other methods. • Apply calibration to our wide angle and fish-eye lenses.

  35. Thank you Suggestions/Comments/Questions

More Related