Camera calibration based on arbitrary parallelograms

1. Camera calibration based on arbitrary parallelograms 授課教授：連震杰 學生：鄭光位

2. Outline • Introduction • Camera of warped images • Camera calibration by transferring scene constrain • Experiments result • Conclusion

3. Introduction(1) • The reconstruction of the three-dimensional (3D) objects from images is one of the most important topics in computer vision • Estimating the camera’s intrinsic parameters or, equivalently, to recover metric invariants such as the absolute conic or the dual absolute quadric is a key issue of 3D metric reconstruction . • There’re three kinds of approaches : using only scene constrains of known targets called calibration objects、using only camera assumption without calibration object and combination of two above.

4. Introduction(2) • However, all of the above methods, which are based on the camera constraints, are not applicable to image sets that have geometrically transformed images such as pictures of pictures, scanned images of printed pictures, and images modified with image-processing tools. • In order to use geometrically transformed images in calibrating cameras or metric reconstruction of scenes with the previous methods, it is needed to find a way to transform the scene constraints in the image into another image without camera constraints and this requires additional processes such as estimating epipolar geometry.

5. Introduction(3) • In this paper, we propose a unified framework to deal with geometrically transformed images basing on warping images containing parallelograms. • The image warping converts the problems with scene constraints of parallelograms into the well-known autocalibration problems of intrinsically constrained cameras. • In addition, we propose a method to estimate infinite homographies between views using the parallelograms and how to reconstruct the physical cameras in affine space by using the newly introduced constraints of the warped images without knowledge about the cameras or projective reconstruction.

6. Outline • Introduction • Camera of warped images • Camera calibration by transferring scene constrain • Experiments result • Conclusion

7. Camera of warped images(1) Assume that there is a plane–plane homography H and a pinhole projection model is given as By warping the image with the homography H ※Warping an image does not change the apparent camera center because This means that the camera of the warped image is a rotating camera of the original one

8. Camera of warped images(2) Assume that there is a rectangle whose aspect ratio is Rm in 3D space ,and we set the reference plane as Z = 0,which is the plane containing the rectangle. homography that transforms the projected rectangles to canonical coordinates is defined as: v1,v2, and v3 are the vanishing points that correspond to the three orthogonal directions in 3D xc is the projection of the pre-defined origin Xc on the reference plane After transforming an image with HFP, the reference plane becomes fronto-parallel one whose aspect ratio is not identical to the actual 3D plane.

9. Camera of warped images(3) Theorem: In the warped image by HFP, the ICDCP of the reference plane is given as diage(R2m; R2FP; 0), where Rm is an aspect ratio of the real rectangle and RFP is an aspect ratio of the warped rectangle. Proof: Assume that a projection process of a plane is described with HP. The CDCP is a dual circle whose radius is infinity and we derive it from regular circles. Assume that there is a circle whose radius is r in the model plane (Euclidean world). The conic dual to the projected circle in the projected image is

10. Camera of warped images(4) a point (X,Y,1)T on the reference plane is warped to the point at

11. Camera of warped images(5) ◎Intrinsic parameters of fronto-parallel (FP) cameras We can imagine that there is a camera, called a fronto-parallel (FP) camera, that may capture the warped image. v1, v2 and v3 are three vanishing points that are in orthogonal directions In warped image , setting vanishing points as gives the IAC wFP of the FP camera

12. Camera of warped images(6) Because the ICPs of the plane are on the IAC By decomposing (*),the FP camera matrix is

13. Camera of warped images(7) ◎Motion constraints of FP cameras Constraint 1. Assume that there is a parallelogram in 3D space and two views i and j that see the parallelogram. The FP cameras PFPi and PFPj , which are derived from an identical parallelogram in 3D, are pure translating cameras.

14. Camera of warped images(8) Constraint 2. Assume that there are two static parallelograms I and J in general position in 3D space that are seen in two view i and j. The relative rotation between two FP cameras of the view i is identical to that between two FP cameras of the view j.

15. Outline Introduction Camera of warped images Camera calibration by transferring scene constrain Experiments result Conclusion

16. Camera calibration by transferring scene constraints camera calibration using geometric constraints of rectangles is equivalent to the autocalibration of rotating cameras whose intrinsic parameters are varying. A group of cameras DOF : 5 but if we get HijFPI， then it will remains at 5 ; and that H will be infinite homography between two cameras

17. Camera calibration by transferring scene constraints there are two views i and j that see an unknown parallelogram where vi3 and vj3 represent scaled orthogonal vanishing points in the warped images of view I and j, respectively.

18. Camera calibration by transferring scene constraints

19. Outline Introduction Camera of warped images Camera calibration by transferring scene constrain Experiments result Conclusion

20. Experiments result(1) ※Camera calibration using static cameras captured with a SONY DSC-F717 camera in 640 * 480 resolution

21. Experiments result(2) ※Reconstruction using image sets including pictures of pictures

22. Experiments result(3) The angle between major walls was estimated to be 90.69 degree； the true angle measured 90 degree using SICK laser range finder as blue lines

23. Experiments result(4)

24. Outline Introduction Camera of warped images Camera calibration by transferring scene constrain Experiments result Conclusion

25. Conclusion • Conventional camera calibration methods based on the camera constraints are not applicable to images containing transformed images, such as pictures of pictures or scanned images from printed pictures. • we propose a method to convert scene constraints in the form of parallelograms into camera constraints by warping images. • A linear method is also proposed to estimate an infinite homography linearly using fronto-parallel cameras without any assumptions about cameras. • we show that the affine reconstruction of cameras is possible directly from the infinite homography derived from images of parallelograms.