875: Recent A dvances in Geometric C omputer V ision & Recognition

1. 875: Recent Advances in Geometric Computer Vision & Recognition Jan-Michael Frahm Fall 2011

2. Introductions

3. Class • Recent methods in computer vision • Broadness of the class depends on YOU! • Initial papers in class are the best papers of CVPR(2010,2011), ICCV(2011), ECCV(2010)

4. Grade Requirements • Presentation of 3 papers in class • 30 min talk, • 10 min questions • Papers for selection must come from: • top journals: IJCV, PAMI, CVIU, IVCJ • top conferences: CVPR (2010,2011), ICCV (2011), ECCV (2010), MICCAI (2010, 2011) • approval for all other venues is needed • Final project • evaluation, extension of a recent method from the above

5. Schedule • Aug. 29th, Introduction • Aug. 31st, Geometric computer vision, first paper selection • Sept. 5th, Labor day • Sept. 7th, Geometric computer vision, Robust estimation • Sept. 12th, Optimization • Sept. 19th, Classification • Sept. 21st, 1. round of presentations starts • Oct 17th, 2. round of presentations starts • Oct 31st, definition of final projects due • Nov 7th, 3. round of presentations starts • Dec 5th and 7th, final project presentation

6. How to give a great presentation • Structure of the talk: • Motivation (motivate and explain the problem) • Overview • Related work (short concise discussion) • Approach • Experiments • Conclusion and future work

7. How to give a great presentation • Use large enough fonts • 5-6 one line bullet items on a slide max • Keep it simple • No complex formulas in your talk • Bad Powerpoint slides • How to for presentations

8. How to give a great presentation • Abstract the material of the talk • provide understanding beyond details • Use pictures to illustrate • find pictures on the internet • create a graphic (in ppt, graph tool) • animate complex pictures

9. How to give a good presentation • Avoid bad color schemes • no red on blue looks awful • Avoid using laser pointer (especially if you are nervous) • Add pointing elements in your presentation • Practice to stay within your time! • Don’t rush through the talk!

10. Projective and homogeneous points • Given: Plane  in R2 embedded in P2 at coordinates w=1 • viewing ray g intersects plane at v (homogeneous coordinates) • all points on ray g project onto the same homogeneous point v • projection of g onto  is defined by scaling v=g/l = g/w R3 w w=1  (R2) y O x

11. Affine and projective transformations • Projective transformations move infinite points into finite affine space Example: Parallel lines intersect at the horizon (line of infinite points). We can see this intersection due to perspective projection! • Affine transformation leaves infinite points at infinity

12. Homogeneous coordinates Homogeneous representation of points on if and only if Homogeneous coordinates but only 2DOF Inhomogeneous coordinates Homogeneous representation of lines equivalence class of vectors, any vector is representative Set of all equivalence classes in R3(0,0,0)T forms P2 The point x lies on the line l if and only if xTl=lTx=0

13. Ideal points and the line at infinity normal direction Example Ideal points Line at infinity Intersections of parallel lines Note that in P2 there is no distinction between ideal points and others

14. Pinhole Camera Model Camera obscura (Frankreich, 1830)

15. Pinhole Camera Model • Skew s • focal length, aspect ratio • (f, af) image plane object principal point (u,v) optical axis aperture

16. Pinhole Camera Model Selbstkalibrierung bestimmt die intrinsischen Kameraparameter

17. Projective Transformation Y O X • Projective Transformation maps M onto Mp inP3 space • Projective Transformation linearizes projection Introduction to Computer Vision for Robotics

18. Projection in general pose Projection: Rotation [R] mp Projection center C M World coordinates Introduction to Computer Vision for Robotics

19. Projection matrix P • Camera projection matrix P combines: • inverse affine transformation Tcam-1 from general pose to origin • Perspective projection P0 to image plane at Z0 =1 • affine mapping K from image to sensor coordinates Introduction to Computer Vision for Robotics

20. Homography • Homography • plane to plane warping • purely rotating camera Y 0 X

21. Self-calibrationforRotatingCameras Agapito et al. • Rotation invariant formulation Projectionofthe dual absolute conicintoimagej Projectionofthedual absolute conicintoimagei • calibrationthroughCholeskidecomposition

22. Removing projective distortion select four points in a plane with know coordinates (linear in hij) (2 constraints/point, 8DOF  4 points needed) Remark: no calibration at all necessary

23. FreelyMovingCamera EpipolarLinie: Ci • Computablefromimagecorrespondences Z Y Cj X

24. Example: motion parallel with image plane

25. Example: forward motion e’ e

26. The Essential Matrix E • F is the most general constraint on an image pair. If the camera calibration matrix K is known, then more constraints are available • Essential Matrix E • E holds the relative orientation of a calibrated camera pair. It has 5 degrees of freedom: 3 from rotation matrix Rik, 2 from direction of translation e, the epipole. Introduction to Computer Vision for Robotics

27. Estimation of P from E • From E we can obtain a camera projection matrix pair: E=Udiag(0,0,1)VT • P0=[I3x3 | 03x1] and there are four choices for P1: P1=[UWVT | +u3] or P1=[UWVT | -u3] or P1=[UWTVT | +u3] or P1=[UWTVT | -u3] only one with 3D point in front of both cameras four possible configurations:

28. KruppaEquations forconstantcameracalibration Dual absolute conic limited toepipolar-geometrie Kruppa-equation(Faugeras et al.`92)