CS 395/495-26: Spring 2003

CS 395/495-26: Spring 2003 IBMR: Week 2 B2-D Projective Geometry Jack Tumblin jet@cs.northwestern.edu

Admin; Starter Code • Website Ready: Plans evolving... • Starter Code Ready: also evolves... • About Your Projects: • Simple directory-based version control: • Save & Number everything that works:NEVER edit an unsaved working program • Always, Always: One Little Step at a Time! • Explain it as you write it (comments)“Writing IS thinking”

Admin; Starter Code • Website Ready: Plans evolving... • Starter Code Ready: also evolves... • About Your Projects: • Simple directory-based version control: • What works well for me: • One master directory for the project • All files of latest version in one directory • Numbered previous versions: ‘snapshot’ copies • Copy master directory EVERY time something works. • Program is ALWAYS working; one step at a time... • > 1 hour without compile/run? too big a step!

2D Homogeneous Coordinates ‘Central Image Plane’ (x,y,1) x2 An ‘Ideal Point’ (0,0,0) at Center of Projection Ray -x3 y x1 x • 3 coords, but only 2 degrees of freedom (2DOF)(x1 / x3), (x2 / x3) • Ideal points == (x,y,0) • Line Intersections = point: (a 3D cross-product) • Point ‘Intersections’ = line: • Duality: P2 theorem for lines  P2 theorem for points! l1 l2 = x x1 x2 = l

2D Homogeneous Coordinates ‘Central Image Plane’ (x,y,1) x2 An ‘Ideal Point’ (0,0,0) at Center of Projection Ray -x3 y x1 x Left-Handed: use –x3 or (x,y,-1) image ctr, or... • 3 coords, but only 2 degrees of freedom (2DOF)(x1 / x3), (x2 / x3) • Ideal points == (x,y,0) • Line Intersections = point: (a 3D cross-product) • Point ‘Intersections’ = line: • Duality: P2 theorem for lines  P2 theorem for points! l1 l2 = x x1 x2 = l

Useful 3D Graphics Ideas • Every xy point describes a 3D ray in (x1,x2,x3) • ‘Phantom’ dimension x3 is the ‘z-buffer’ value • Homogeneous coords allows translation matrix: • (BUT a 3x3 transform in R3Euclidean Space (x,y,z) can’t translate—only rotate,scale, skew!) ‘Central Image Plane’ (x,y,1) 1 0 3 0 1 5 0 0 1 x y = 1 x+3 y+5 1 x2 An ‘Ideal Point’ (0,0,0) at Center of Projection Ray -x3 y x1 x 1 0 3 0 1 5 0 0 1 x y = z x+z y+z z

Projective Transform: H • Matrix H transforms Points: • Given a point x: (x1,x2,x3) (or in P2 : (x1/x3, x2/x3)) x’ = Hx • And H-T can also transform Lines: • Recall point x is on line l iff xTl =0 = • Lines transform ‘covariantly’ (points: ‘contravariantly’: x’ = Hx) l’ = H-T l ‘Line Homographies’ of many kinds are often denoted by a star: H*

Projective Transform H x y Apply the 3x3 matrix HHx = x’ 2D image (x,y)Homog. coords [x,y,1]T = x Homog. coordsx’ =[x’,y’,1]T 2D image (x’,y’) x x x2 (x,y) y x y y’ y x3 x x’ x’ y (x’,y’) x2 x

Projective Transform: Hx = x’ h11 x + h12 y + h13 h31 x + h32 y + h33 h21 x + h22 y + h21 h31 x + h32 y + h33 x’2 x’3 x’1 x’3 Finding H from point pairs (correspondences) h11 h12 h13 h21 h22 h21 h31 h32 h33 x1 x2 x3 x’1 x’2 x’3 x1 x2 x3 x y 1 Input (or output) image is on central plane = = x’ = = y’ = = Write R2 expressions;(for each x,x’ pair) x’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13) Rearrange, solve as a matrix problem… y’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

Projective Transform: Hx = x’ h11 x + h12 y + h13 h31 x + h32 y + h33 h21 x + h22 y + h21 h31 x + h32 y + h33 x’2 x’3 x’1 x’3 Finding H from point pairs (correspondences) h11 h12 h13 h21 h22 h21 h31 h32 h33 x1 x2 x3 x’1 x’2 x’3 x1 x2 x3 x y 1 Input (or output) image is on central plane = = —!STOP! — You get Easier, Robust, but Partial results using l or conics methods! Let’s try them first. Later we’ll return to this method (called ‘DLT’: direct linear xfm.) x’ = = y’ = = Write R2 expressions; (for each x,x’ pair) x’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13) Rearrange, solve as a matrix problem… y’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

Projective Transform: H • Comp. Graphics ‘View interpolation’ notion • Fixed, rigid 2D viewing point, viewing plane • Rigid 2D world plane positioned in 3D (x1 x2 x3) • only 6 DOF: world plane rotate & position. • Comp. Vision ‘Projective Transform’ notion • Fixed, rigid 2D viewing point, viewing plane • ‘Stretchy’ 2D world plane: allow affine changes • result: (up to) 8 DOF

The bits and pieces of H H has 8 independent variables (DOF) • Computer Graphics method (3x3 matrix): 2D Translation(tx, ty)3D Scale (sx,sy,sz)3D Rotation (x,y,z) • Computer Vision method(2D projective): Isometry --3DOF (2D translate tx,ty; 2Drotate z; ) Similarity --4DOF (add uniform scale s;) Affine --6DOF (add orientable scale s,/s, s/s) Projective--8DOF (changes x3; 3D-rotation-like)

The bits and pieces of H H has 8 independent variables (DOF) • Computer Graphics method (3x3 matrix): 2D Translation(tx, ty)3D Scale (sx,sy,sz)3D Rotation (x,y,z) • Computer Vision method(2D projective): Isometry --3DOF (2D translate tx,ty; 2Drotate z; ) Similarity --4DOF (add uniform scale s;) Affine --6DOF (add orientable scale s,/s, s/s) Projective--8DOF (changes x3; 3D-rotation-like) Affects only x1,x2

The bits and pieces of H H has 8 independent variables (DOF), not 9! • Decomposable into 3 useful parts: (pg 22) H = HS HA HP = • Similarity HS: • 2D translate, rotate, uniform scale only (4DOF) • Affine HA: • non-uniform scale (2DOF) • Projective HP: • Projective coupling for x3 (2DOF)    sR t 0T 1 K 0 0T 1 I 0 vT v            

The bits and pieces of H H has 8 independent variables (DOF), not 9! • Decomposable into 3 useful parts: (pg 22) H = HS HA HP = • Similarity HS: • 2D translate, rotate, uniform scale only (4DOF) • Affine HA: • non-uniform scale (2DOF) • Projective HP: • Projective coupling for x3 (2DOF) s  cos -s sintx s  sin s cos ty 0 0 1 1 0 0 0 1 0 v1 v2vS 1 k2 0 k3 1 0 0 01

1-D Projective Geometry (?Why? we use it later) • A ‘side’ view of 2D projective geometry • Convert R1 scalar b to a 2-vector inP1 • As with P2, we can transform points: • (use various H2x2 s to change cyan lines below) b 1 H2b H2a

1-D Projective Geometry a1 a2 b1 b2 What is invariant here? all the positions, lengths and ratios change with each transform… Answer: ‘Cross Ratio’: cross(a,b,c,d) • let a= , b= … • THEN: cross = |ab||cd| |ac||bd| a’ b’ c’ d’ a b c d

Image Rectifying: Undo parts of H x’ = H x where H = HS HA HP GOAL: Put world plane x’ into view plane • Affine Rect.; (find only HP(2DOF)); • Similarity Rect.; (find only HA HP (6DOF)); • Full Rect.; (find all of H (8DOF)); METHODS:(mix & match?) • Full: 4-point correspondence • ‘Vanishing Point’, Infinity line methods • Conics and circular points

Image Rectifying: Undo parts of H x’ = H x where H = HS HA HP GOAL: Put world plane x’ into view plane • Affine Rect.; (find only HP(2DOF)); • Similarity Rect.; (find only HA HP (6DOF)); • Full Rect.; (find all of H (8DOF)); METHODS:(mix & match?) • Full: 4-point correspondence • ‘Vanishing Point’, Infinity line methods • Conics and circular points Other DOF? make assumptions, or ignore

Vanishing-Point Methods: 1 • In 2D image, find two pairs of lines that are parallel in ‘world’ space. H world space 2D image

Vanishing-Point Methods: 1 • Find intersection (vanishing points) p1, p2 (compute line intersections with 3D cross-products (see last lecture) • Horizon line Ih connects p1, p2. Ih p1 p2 H world space 2D image

Vanishing-Point Methods: 1 • H-T transforms world infinity linel (0,0,1)Tto Ih • Answer (see book)find, copy the lh coordinates! Hp= 1 0 0 0 1 0 lh1 lh2 lh3 Ih p1 p2 H world space 2D image

Vanishing-Point Methods: 1 Limitations: • Accurate point, line-finding can be tricky • Can have high error sensitivity: ‘twitchy’ • Only rectifies Hp-- but what if HA, HS needed? Ih p1 p2 H world space 2D image

Vanishing-Point Methods: 2 No parallel lines? • find 2D image line with a known distance ratio • Use Cross-Ratio (in P1along that line) to • Find vanishing point distance at point d’ Ih p1 p2 d’ c’ H b’ a b c d(at infinity) a’ 2D image world space

END --Please download the starter code, compile and run. --Then you can get started on Project 1 right away.

Conic Methods • Better-behaved, easier to use (? maybe...) • DeterminesHA HP (4DOF) (all but 2D trans, rot, scale) Must go back and review conics first (pg.8) • ‘Conics’ == intersection of cone & plane: • Many possible shapes:circles, ellipses, parabola, hyperbola, degenerates (lines & points

Conic Methods • Equation of any/all conics solve a 2D quadratic: ax2 + bxy + cy2 +dx +ey +f = 0 • Write in homogeneous coordinates: xTCx = 0 • C is symmetric, 5DOF (not 6, because x3 scaling) • Find any C from 5 homogeneous points(solve for null space—see book pg 9) x1 x2 x3 a b/2 d/2 b/2 c e/2 d/2 e/2 f x1 x2 x3 = 0 a ‘Point Conic’

Conic Methods • Matrix C makes conics from points: xTCx = 0 C is a ‘point conic’ • Given a point x on a conic curve, the homog. tangent line l is given by l = C x • Matrix C* makes conics from lines: lTC*l = 0C* is a ‘Dual Conic’ defined by tangent lineslinstead of points.

Conic Methods • If C is non-singular (rank 3), then C* = C-1 • If C (or C*) has… Rank 3: it is an ellipse, circle, parab., hyperb. Rank 2: it is a pair of lines (forms an ‘x’) THE BIG IDEA---MOST USEFUL PROPERTYProjective transform of a conic C is conic C’: C’ = H-T C H-1

CS 395/495-26: Spring 2003