Zhaoxiang Zhang, Min Li, Kaiqi Huang and Tieniu Tan National Laboratory of Pattern Recognition,

1. Practical Camera Auto-Calibration Based on Object Appearance and Motion for Traffic Scene Visual Surveillance Zhaoxiang Zhang, Min Li, Kaiqi Huang and Tieniu Tan National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences Reporter: Chia-Hao Hsieh Date: 2009/11/3 CVPR 2008

2. Outline • Introduction • Methods • Extract foreground • Estimate vanishing points • Auto-calibration • Experimental results

3. Introduction Practical Camera Auto-Calibration Based on Object Appearance and Motion for Traffic Scene Visual Surveillance • Recover intrinsic and extrinsic parameters of cameras • Based on appearance and motion of objects • Measure the camera height only

4. Motion Detection • Disadvantages of Gaussian Mixture Model • Fast illumination changes • Shadows • Methods deal with the disadvantages • Model each pixel as the product of irradiance component and reflectance component • Model each reflectance component as a mixture of Gaussian

5. Outline • Introduction • Methods • Extract foreground • Estimate vanishing points • Auto-calibration • Experimental results

6. Vanishing Points Estimation • Helpful general properties • Moving objects move on the ground plane • Vehicles run along the straight roadway • Vehicles are rich in line segments along two orientations • Pedestrians walk with their trunks perpendicular to the ground plane

7. Vanishing Points Estimation • Coarse Moving Object Classification • Two directions • Velocity direction • Main axis direction • Difference of direction • K-Mean clustering • Thresholding θ < 5° θ > 20° from moment analysis of silhouette

8. Vanishing Points Estimation • Line Equations Estimation Vehicles are rich in line segments along two orientations • Histogram of Orientated Gradient

9. Vanishing Points Estimation • Intersection Estimation • Least square • Levenberg-Marquardt • RANSAC • But… more and more frames • Voting strategy • Each point lying on every line generates a Gaussian impulse in the voting space Vehicles: 2 vanishing points Pedestrians: 1 vanishing point

10. Outline • Introduction • Methods • Extract foreground • Estimate vanishing points • Auto-calibration • Experimental results

11. Camera Calibration • Recovery of K and R • 3 vanishing points  3 orthogonal directions of world coordinate system Assume αu= αv=f s=0 Assume (u0, v0) is on the middle of image plane 3 DOF 1 DOF 3 DOF 3 DOF

12. Camera Calibration K and λi solved Solve R

13. Camera Calibration • Recovery of T • Choose one arbitrary reference point (u4,v4) from image plane to correspond to the origin of the world coordinate system Camera height H is measured The optical center of the camera lies on the z = H plane Propose a method of complete calibration of surveillance scenes with three estimated orthogonal vanishing points and the measured camera height H

14. Experimental Results 720 × 576 αu= αv = 884, (u0, v0) =(336, 226) (u1, v1) = (−217, 70) (u2, v2) = (1806, 31) (u3, v3) = (427, 4906) vary in a small range less than 2%

15. Conclusions • Practical camera auto-calibration method for traffic scene surveillance • Completely recover both intrinsic and extrinsic parameters of cameras • Only the camera height H measured • Based on appearance and motion of moving objects in videos • Accuracy and practicability Thank you!