Course 23: Multiple-View Geometry For Image-Based Modeling

1. Course 23: Multiple-View Geometry For Image-Based Modeling Jana Kosecka (CS, GMU) Yi Ma (ECE, UIUC) Stefano Soatto (CS, UCLA) Rene Vidal (Berkeley, John Hopkins) SIGGRAPH’04, Los Angeles

2. PRIMARY REFERENCE SIGGRAPH’04, Los Angeles

3. Multiple-View Geometry for Image-Based Modeling Lecture A: Overview and Introduction Stefano Soatto Computer Science Department, ULCA http://www.cs.ucla.edu/~soatto SIGGRAPH’04, Los Angeles

4. COURSE LECTURES • A. Overview and introduction (Soatto) • B. Preliminaries: geometry & image formation (Soatto) • C. Image primitives & correspondence (Soatto) • D. Two-view geometry (Kosecka) • E. Uncalibrated geometry and stratification (Ma) • F. Multiview recon. from points and lines (Vidal) • G. Reconstruction from scene knowledge (Ma) • H. Step-by-step building of 3-D model (Kosecka) • I. Multiple motion estimation & segmentation (Vidal) SIGGRAPH’04, Los Angeles

5. COURSE LOGICAL FLOW IMAGING and VISION:From 3D to 2D and then back again GEOMETRY FOR TWO VIEWS GEOMETRY FOR MULTIPLE VIEWS MULTIVIEW GEOMETRY WITH SYMMETRY APPLICATIONS: System Implementation, motion segmentation. SIGGRAPH’04, Los Angeles

6. Reconstruction from images – The Fundamental Problem Input: Corresponding “features” in multiple perspective images. Output: Camera pose, calibration, scene structure representation. SIGGRAPH’04, Los Angeles

7. surface curve line point practice projective algorithm affine theory Euclidean 2 views algebra 3 views geometry 4 views optimization m views Fundamental Problem: An Anatomy of Cases SIGGRAPH’04, Los Angeles

8. VISION AND GEOMETRY “The rise of projective geometry made such an overwhelming impression on the geometers of the first half of the nineteenth century that they tried to fit all geometric considerations into the projective scheme. ... The dictatorial regime of the projective idea in geometry was first successfully broken by the German astronomer and geometer Mobius, but the classical document of the democratic platform in geometry establishing the group of transformations as the ruling principle in any kind of geometry and yielding equal rights to independent consideration to each and any such group, is F. Klein's Erlangen program.” -- Herman Weyl, Classic Groups, 1952 Synonyms: Group = Symmetry SIGGRAPH’04, Los Angeles

9. APPLICATIONS – Autonomous Highway Vehicles SIGGRAPH’04, Los Angeles Image courtesy of California PATH

10. Rate: 10Hz; Accuracy: 5cm, 4o APPLICATIONS – Unmanned Aerial Vehicles (UAVs) SIGGRAPH’04, Los Angeles Courtesy of Berkeley Robotics Lab

11. APPLICATIONS – Real-Time Virtual Object Insertion SIGGRAPH’04, Los Angeles UCLA Vision Lab

12. APPLICATIONS – Real-Time Sports Coverage First-down line and virtual advertising SIGGRAPH’04, Los Angeles Princeton Video Image, Inc.

13. APPLICATIONS – Image Based Modeling and Rendering SIGGRAPH’04, Los Angeles Image courtesy of Paul Debevec

14. APPLICATIONS – Image Alignment, Mosaicing, and Morphing SIGGRAPH’04, Los Angeles

15. GENERAL STEPS – Feature Selection and Correspondence • Small baselines versus large baselines • Point features versus line features SIGGRAPH’04, Los Angeles

16. GENERAL STEPS – Structure and Motion Recovery • Two views versus multiple views • Discrete versus continuous motion • General versus planar scene • Calibrated versus uncalibrated camera • One motion versus multiple motions SIGGRAPH’04, Los Angeles

17. GENERAL STEPS – Image Stratification and Dense Matching Left Right SIGGRAPH’04, Los Angeles

18. GENERAL STEPS – 3-D Surface Model and Rendering • Point clouds versus surfaces (level sets) • Random shapes versus regular structures SIGGRAPH’04, Los Angeles

19. Multiple-View Geometry for Image-Based Modeling Lecture B: Rigid-Body Motion and Imaging Geometry Stefano Soatto Computer Science Department, ULCA http://www.cs.ucla.edu/~soatto SIGGRAPH’04, Los Angeles

20. OUTLINE • 3-D EUCLIDEAN SPACE & RIGID-BODY MOTION • Coordinates and coordinate frames • Rigid-body motion and homogeneous • coordinates • GEOMETRIC MODELS OF IMAGE FORMATION • Pinhole camera model • CAMERA INTRINSIC PARAMETERS • From metric to pixel coordinates SUMMARY OF NOTATION SIGGRAPH’04, Los Angeles

21. Coordinates of a point in space: Standard base vectors: 3-D EUCLIDEAN SPACE - Cartesian Coordinate Frame SIGGRAPH’04, Los Angeles

22. Coordinates of the vector : 3-D EUCLIDEAN SPACE - Vectors A “free” vector is defined by a pair of points : SIGGRAPH’04, Los Angeles

23. Cross product between two vectors: 3-D EUCLIDEAN SPACE – Inner Product and Cross Product Inner product between two vectors: SIGGRAPH’04, Los Angeles

24. Coordinates are related by: RIGID-BODY MOTION – Rotation Rotation matrix: SIGGRAPH’04, Los Angeles

25. Coordinates are related by: Velocities are related by: RIGID-BODY MOTION – Rotation and Translation SIGGRAPH’04, Los Angeles

26. Homogeneous coordinates: Homogeneous coordinates/velocities are related by: RIGID-BODY MOTION – Homogeneous Coordinates 3-D coordinates are related by: SIGGRAPH’04, Los Angeles

27. IMAGE FORMATION – Perspective Imaging “The Scholar of Athens,” Raphael, 1518 SIGGRAPH’04, Los Angeles Image courtesy of C. Taylor

28. Frontal pinhole IMAGE FORMATION – Pinhole Camera Model Pinhole SIGGRAPH’04, Los Angeles

29. Homogeneous coordinates IMAGE FORMATION – Pinhole Camera Model 2-D coordinates SIGGRAPH’04, Los Angeles

30. metric coordinates Linear transformation pixel coordinates CAMERA PARAMETERS – Pixel Coordinates SIGGRAPH’04, Los Angeles

31. Calibration matrix (intrinsic parameters) Projection matrix Camera model CAMERA PARAMETERS – Calibration Matrix and Camera Model Pinhole camera Pixel coordinates SIGGRAPH’04, Los Angeles

32. Projection of a 3-D point to an image plane IMAGE FORMATION – Image of a Point Homogeneous coordinates of a 3-D point Homogeneous coordinates of its 2-D image SIGGRAPH’04, Los Angeles

33. Homogeneous representation of its 2-D image Projection of a 3-D line to an image plane IMAGE FORMATION – Image of a Line Homogeneous representation of a 3-D line SIGGRAPH’04, Los Angeles

34. SUMMARY OF NOTATION – Multiple Images . . . • Images are all “incident” at the corresponding features in space; • Features in space have many types of incidence relationships; • Features in space have many types of metric relationships. SIGGRAPH’04, Los Angeles