Announcements

1. Announcements • Email list – let me know if you have NOT been getting mail • Assignment hand in procedure • web page • at least 3 panoramas: • (1) sample sequence, (2) w/Kaidan, (3) handheld • hand procedure will be posted online • Readings for Monday (via web site) • Seminar on Thursday, Jan 18 Ramin Zabih: “Energy Minimization for Computer Vision via Graph Cuts “ • Correction to radial distortion term

2. Today • Single View Modeling • Projective Geometry for Vision/Graphics • on Monday (1/22) • camera pose estimation and calibration • bundle adjustment • nonlinear optimization

3. 3D Modeling from a Photograph

4. Shape from X • Shape from X • Shape from shading • Shape from texture • Shape from focus • Others… • Make strong assumptions – limited applicability • Lee & Kau 91 (from survey by Zhang et al., 1999)

5. Model-Based Techniques Blanz & Vetter, Siggraph 1999 • Model-based reconstruction • Acquire a prototype face, or database of prototypes • Find best fit of prototype(s) to new image

6. User Input + Projective Geometry • Horry et al., “Tour Into the Picture”, SIGGRAPH 96 • Criminisi et al., “Single View Metrology”, ICCV 1999 • Shum et al., CVPR 98 • Images above by Jing Xiao, CMU

7. Projective Geometry for Vision & Graphics • Uses of projective geometry • Drawing • Measurements • Mathematics for projection • Undistorting images • Focus of expansion • Camera pose estimation, match move • Object recognition via invariants • Today: single-view projective geometry • Projective representation • Point-line duality • Vanishing points/lines • Homographies • The Cross-Ratio • Later: multi-view geometry

8. 4 3 2 1 1 2 3 4 Measurements on Planes Approach: unwarp then measure

9. Planar Projective Transformations • Perspective projection of a plane • lots of names for this: • homography, texture-map, colineation, planar projective map • Easily modeled using homogeneous coordinates p’ H p • To apply a homography H • compute p’ = Hp • p’’ = p’/s normalize by dividing by third component

10. Image Rectification • To unwarp (rectify) an image • solve for H given p’’ and p • solve equations of the form: sp’’ = Hp • linear in unknowns: s and coefficients of H • need at least 4 points

11. (x,y,1) image plane The Projective Plane • Why do we need homogeneous coordinates? • represent points at infinity, homographies, perspective projection, multi-view relationships • What is the geometric intuition? • a point in the image is a ray in projective space y (sx,sy,s) (0,0,0) x z • Each point(x,y) on the plane is represented by a ray(sx,sy,s) • all points on the ray are equivalent: (x, y, 1)  (sx, sy, s)

12. A line is a plane of rays through origin • all rays (x,y,z) satisfying: ax + by + cz = 0 l p • A line is also represented as a homogeneous 3-vector l Projective Lines • What is a line in projective space?

13. l1 p l l2 Point and Line Duality • A line l is a homogeneous 3-vector (a ray) • It is  to every point (ray) p on the line: lp=0 p2 p1 • What is the line l spanned by rays p1 and p2 ? • l is  to p1 and p2  l = p1p2 • l is the plane normal • What is the intersection of two lines l1 and l2 ? • p is  to l1 and l2  p = l1l2 • Points and lines are dual in projective space • every property of points also applies to lines

14. (sx,sy,0) y y (sx,sy,0) z z image plane image plane x x • Ideal line • l  (a, b, 0) – parallel to image plane • Corresponds to a line in the image (finite coordinates) Ideal points and lines • Ideal point (“point at infinity”) • p  (x, y, 0) – parallel to image plane • It has infinite image coordinates

15. Homographies of Points and Lines • Computed by 3x3 matrix multiplication • To transform a point: p’ = Hp • To transform a line: lp=0  l’p’=0 • 0 = lp = lH-1Hp = lH-1p’  l’ = lH-1 • lines are transformed bypremultiplication of H-1

16. 3D Projective Geometry • These concepts generalize naturally to 3D • Homogeneous coordinates • Projective 3D points have four coords: P = (X,Y,Z,W) • Duality • A plane L is also represented by a 4-vector • Points and planes are dual in 3D: L P=0 • Projective transformations • Represented by 4x4 matrices T: P’ = TP, L’ = LT-1 • However • Can’t use cross-products in 4D. We need new tools • Grassman-Cayley Algebra • generalization of cross product, allows interactions between points, lines, and planes via “meet” and “join” operators • Won’t get into this stuff today

17. It’s useful to decompose  into T R  project  A intrinsics projection orientation position 3D to 2D: “Perspective” Projection • Matrix Projection:

18. Projection Models • Orthographic • Weak Perspective • Affine • Perspective • Projective

19. Properties of Projection • Preserves • Lines and conics • Incidence • Invariants (cross-ratio) • Does not preserve • Lengths • Angles • Parallelism

20. vanishing point Vanishing Points • Vanishing point • projection of a point at infinity image plane camera center ground plane

21. vanishing point Vanishing Points (2D) image plane camera center line on ground plane

22. line on ground plane Vanishing Points • Properties • Any two parallel lines have the same vanishing point • The ray from C through v point is parallel to the lines • An image may have more than one vanishing point image plane vanishing point V camera center C line on ground plane

23. v1 v2 Vanishing Lines • Multiple Vanishing Points • Any set of parallel lines on the plane define a vanishing point • The union of all of these vanishing points is the horizon line • called vanishing line in the Criminisi paper • Note that different planes define different vanishing lines

24. Vanishing Lines • Multiple Vanishing Points • Any set of parallel lines on the plane define a vanishing point • The union of all of these vanishing points is the horizon line • called vanishing line in the Criminisi paper • Note that different planes define different vanishing lines

25. Computing Vanishing Points • Properties • Pis a point at infinity, v is its projection • They depend only on line direction • Parallel lines P0 + tD, P1 + tD intersect at X V P0 D

26. Computing Vanishing Lines • Properties • l is intersection of horizontal plane through C with image plane • Compute l from two sets of parallel lines on ground plane • All points at same height as C project to l • Provides way of comparing height of objects in the scene C l ground plane

27. Fun With Vanishing Points

28. Perspective Cues

29. Perspective Cues

30. Perspective Cues

31. Comparing Heights Vanishing Point

32. Measuring Height 5.4 5 Camera height 4 3.3 3 2.8 2 1

33. Measuring Height Without a Ruler Y C ground plane • Compute Y from image measurements • Need more than vanishing points to do this

34. The Cross-Ratio of 4 Collinear Points P4 P3 P2 P1 The Cross Ratio • A Projective Invariant • Something that does not change under projective transformations (including perspective projection) • Can permute the point ordering • 4! = 24 different orders (but only 6 distinct values) • This is the fundamental invariant of projective geometry • likely that all other invariants derived from cross-ratio

35. Scene Cross Ratio t vL C Image Cross Ratio b vZ Measuring Height  T (top of object) L (height of camera on the this line) Y B (bottom of object) ground plane

36. Algebraic Derivation • Eliminating  and  yields • Can calculate  given a known height in scene Measuring Height t Y C b reference plane

37. So Far • Can measure • Positions within a plane • Height • More generally—distance between any two parallel planes • These capabilities provide sufficient tools for single-view modeling • To do: • Compute camera projection matrix

38. Vanishing Points and Projection Matrix = vx (X vanishing point) • Not So Fast! We only know v’s up to a scale factor • Can fully specify by providing 3 reference lengths

39. 3D Modeling from a Photograph

40. 3D Modeling from a Photograph

41. Conics • Conic (= quadratic curve) • Defined by matrix equation • Can also be defined using the cross-ratio of lines • “Preserved” under projective transformations • Homography of a circle is a… • 3D version: quadric surfaces are preserved • Other interesting properties (Sec. 23.6 in Mundy/Zisserman) • Tangency • Bipolar lines • Silhouettes