CS 395/495-26: Spring 2004

CS 395/495-26: Spring 2004 IBMR: 2-D Projective Geometry --Introduction-- Jack Tumblin jet@cs.northwestern.edu

Recall: Scene & Image Light + 3D Scene: Illumination, shape, movement, surface BRDF,… 2D Image: Collection of rays through a point Image Plane I(x,y) Position(x,y) Angle(,) GOAL: a Reversible Mapping Scene angles image positions

View Interpolation: How? • LATER: 3D Projection(Hartley,Zisserman Chapter 2) Find new views of that scene From a 3D scene view1 view2

View Interpolation: How? • BUT FIRST, the simpler caseHartley, Zisserman Chapter 1: 2D Projection Find new views of that image From a flat 2D image, view1 view2

Answer: 2D Homogeneous Coords Chapter 1: 2D Projection From a flat 2D image, Find new views of that image view1 view2 2D Homogeneous (x1 x2 x3) coordinates in P2 Cartesian (x,y) coordinates in R2

Overview • Chapter 1: any 2D plane, as seen byany 3D camera view (texture mapping generalized) • Homogeneous Coordinates are Wonderful • Everything is a Matrix: • Conics (you can skip for now) • Aside: interpolation of pixels and points… • Transform your Point of View: H matrix • Parts of H: useful kinds of Transforms

2-D Homogeneous Coordinates y x • WHY? makes MUCH cleaner math! • Unifies lines and points • Puts perspective projection into matrix form • No divide-by-zero; defines horizon, infinities … in R2, write point x as But in P2, write same point x as where: x1 x2 x3 x y x = x1 / x3, y = x2 / x3, x3 = anything non-zero!(but usually defaults to 1) (x,y)

Useful 3D Graphics Ideas • Every Homog. point (x1,x2,x3) describes a 3D ray • ‘Phantom’ dimension x3 is the ‘z-buffer’ value • 3D2D ‘hard-wired’: (Fig 1.1, pg 8) • “Homogeneous coords allows translation matrix:” (true. But there is more!) ‘Image Plane’ (x,y,-1) (xmax, ymax, -1) (0,0,-1) An ‘Ideal Point’ at (0,0,0) Center of Projection 1 0 3 0 1 5 0 0 1 x y = 1 x+3 y+5 1

2-D Homogeneous Coordinates WHAT?!Why x3? Why ‘default’ value of 1? • Look at lines in R2 : • ‘line’ == all (x,y) points where • scale by ‘k’  no change: • Using ‘x3’ for points UNIFIES notation: • line is a 3-vector named l • now point (x,y) is a 3-vector too, named x ax + by + c = 0 kax + kby + kc = 0 ax + by + c = 0 a b c x1 x2 x3 = 0 xT.l = 0

2D Homogeneous Coordinates Important Properties 1(see book for details) • 3 coordinates, but only 2 degrees of freedom(only 2 ratios (x1 / x3), (x2 / x3) can change) • DUALITY: points, lines are interchangeable • Line Intersections = point: (a 3D cross-product) • Point ‘Intersections’ = line: • Projective theorem for lines  theorem for points! l1 l2 = x x1 x2 = l

Homogeneous Coordinates Important Properties 2(see book for details) • Neatly Sidesteps ‘divide-by-zero’: • in (x,y) space (R2) • Remember, we store (x1 ,x2, x3)T, then compute (x1 / x3), (x2 / x3)only if OK. • “OK, yeah, sure, but so what?” ...

Homogeneous Coordinates Why is deferring divide-by-zero so important? • Cleanly Defines all ‘Points at Infinity’ or ‘Ideal Points’ (x,y,0)T • ==outermost pixels (+/-, +/-) of an infinite image plane ((x,y), or R2) • (x,y,0) is an entire plane of points in the (semi-bogus) 3D space (x1 ,x2, x3)T space • but it is a ‘circle around infinity’ in the (legitimate) P2 projective space (x1 /x3, x2 /x3)T • Note! ‘Center of Projection’ is an ideal point. (Do outer limits of image plane ‘wrap around’ to that point?!?!)

Homogeneous Coordinates Why is deferring divide-by-zero so important? Cleanly defines one ‘Line at infinity’ l (0,0,1)T, or “0x +0y +1=0” • All ideal points are on l: proof? l•(x1,x2,0)T = 0 • Any line l intersects with l line at an ideal point • Two parallel lines l and l’ always meet at an ideal point Let l = (a,b,c)T and l’ = (a,b,c’)T …(page 7)

Homogeneous Coordinates x1 x2 x3 x’1 x’2 x’3 Important Properties 3(see book for details) • View Interpolation (‘Projective Transform’) • ‘Central’ image plane (x,y,-1)T • Choose a known point x’ • Apply 3x3 Matrix H to (x,y,-1)Tto make point on OTHER plane • Ray through known point x’ pierces unknown point x 3D world plane x’ x H x = x’ ‘Image Plane’ (x,y,-1) (0,0,-1) h11 h12 h13 h21 h22 h21 h31 h32 h33 = (0,0,0)

END

Useful 3D Graphics Ideas • Every xy point describes a 3D ray in (x1,x2,x3) • ‘Phantom’ dimension x3 is the (lost) distance-to-camera • Homogeneous coords allows matrix for 2-D translation: • (BUT a 3x3 transform in R3Euclidean Space (x,y,z) can’t translate—only rotate,scale, skew!) ‘Central Image Plane’ (x,y,1) 1 0 3 0 1 5 0 0 1 x y = 1 x+3 y+5 1 x2 An ‘Ideal Point’ (0,0,0) at Center of Projection Ray -x3 y x1 x 1 0 3 0 1 5 0 0 1 x y = z x+z y+z z

2D Homogeneous Coordinates ‘Central Image Plane’ (x,y,1) x2 An ‘Ideal Point’ (0,0,0) at Center of Projection Ray -x3 y x1 x • 3 coords, but only 2 degrees of freedom (2DOF)(x1 / x3), (x2 / x3) • Ideal points == (x,y,0) (every line hits the horizon at an ideal point) • Line Intersections = point: (a 3D cross-product) • Point ‘Intersections’ = line: • Duality: P2 theorem for lines  P2 theorem for points! l1 l2 = x x1 x2 = l

2D Homogeneous Coordinates ‘Central Image Plane’ (x,y,1) x2 An ‘Ideal Point’ (0,0,0) at Center of Projection Ray -x3 y x1 x Left-Handed: use –x3 or (x,y,-1) image ctr, or... • 3 coords, but only 2 degrees of freedom (2DOF)(x1 / x3), (x2 / x3) • Ideal points == (x,y,0) • Line Intersections = point: (a 3D cross-product) • Point ‘Intersections’ = line: • Duality: P2 theorem for lines  P2 theorem for points! l1 l2 = x x1 x2 = l

Projective Transform H x y Apply the 3x3 matrix Hx’ = Hx 2D image (x,y)Homog. coords [x,y,1]T = x Homog. coordsx’ =[x’,y’,1]T 2D image (x’,y’) x x x2 (x,y) y x y y’ y x3 x x’ x’ y (x’,y’) x1 x

Recall: Projective Transform H x y Goal: Find H for ‘Image Rectification’ Apply the 3x3 matrix Hx’ = Hx 2D image (x,y)Homog. coords [x,y,1]T = x Homog. coordsx’ =[x’,y’,1]T 2D image (x’,y’) x x x2 (x,y) y x y y’ y x3 x x’ x’ y (x’,y’) x1 x ‘World Plane’ ‘View Plane’

Projective Transform H H matrix: Plane-to-Plane mapping What if H is unknown, but we have four or more pairs of x,x’ points? (see pg 15, Zisserman) x1, x2, x3, x4, x’1, x’2, x’3, x’4 … x1 x2 x3 h11 h12 h13 h21 h22 h21 h31 h32 h33 x’1 x’2 x’3 H x’ = x = x’ x ‘Image Plane’ (x,y,-1) (0,0,-1) (0,0,0)

Projective Transform: H • Matrix H transforms Points: • Given a point x: (x1,x2,x3) (or in R2 : (x1/x3, x2/x3)) x’ = Hx • And matrix H-T transform Lines: • Given a line l: (a,b,c) in R2 : (a·x+ b·y + c = 0) in P2 : (a·x1 + b·x2 + c·x3 = 0) l’ = H-T l Recall point x is on line l iff xTl =0 Jargon: points transform ‘covariantly’; lines transform ‘contravariantly’

The bits and pieces of H H has 8 independent variables (DOF) • Cartesian (‘3D’) decomposition: 2D Translation(tx, ty)3D Scale (sx,sy,sz)3D Rotation (x,y,z) • Computer Vision (Projective 2D) decomp. Isometry --3DOF (2D translate tx,ty; 2D rotate z; ) Similarity --4DOF (add uniform scale s;) Affine --6DOF (add orientable scale s,/s, s/s) Projective--8DOF (changes x3; 3D-rotation-like)

The bits and pieces of H H has 8 independent variables (DOF) • Computer Graphics method (3x3 matrix): 2D Translation(tx, ty)3D Scale (sx,sy,sz)3D Rotation (x,y,z) • Computer Vision method(2D projective): Isometry --3DOF (2D translate tx,ty; 2Drotate z; ) Similarity --4DOF (add uniform scale s;) Affine --6DOF (add orientable scale s,/s, s/s) Projective--8DOF (changes x3; 3D-rotation-like) Affects only x1,x2

The bits and pieces of H H has 8 independent variables (DOF), not 9! • Decomposable into 3 useful parts: (pg 22) H = HS HA HP = • Similarity HS: • 2D translate, rotate, uniform scale only (4DOF) • Affine HA: • non-uniform scale (2DOF) • Projective HP: • Projective coupling for x3 (2DOF)    sR t 0T 1 K 0 0T 1 I 0 vT v            

The bits and pieces of H H has 8 independent variables (DOF), not 9! • Decomposable into 3 useful parts: (pg 22) H = HS HA HP = • Similarity HS: • 2D translate, rotate, uniform scale only (4DOF) s=scale, =translate details… • Affine HA: • non-uniform scale (2DOF) • Projective HP: • Projective coupling for x3 (2DOF) s  cos  -s sin tx s sin  s  cos ty 0 0 1 1 0 0 0 1 0 v1 v2vS 1 k2 0 k3 1 0 0 01

Compare HA and HP sR  t    0T 1 K  0    0T 1 I 0  vT v L M (recall)H = HS HA HP = L (parallelogram) HA m (K matrix, 2DOF) 2D image space (square) L m 2D world space HP (quadrilateral) ( v vector) 2D image space

1-D Projective Geometry (?Why? We’ll use it soon…) • A ‘side’ view of 2D projective geometry • Convert R1 scalar b to a 2-vector inP1 • As with P2, we can transform points: • (use various H2x2 s to change cyan lines below) b 1 H2b H2a

1-D Projective Geometry a1 a2 b1 b2 What is invariant here? all the positions, lengths and ratios change with each transform… Answer: ‘Cross Ratio’: cross(a,b,c,d) • let a= , b= … cross = |a-b||c-d| |a-c||b-d| cross(a,’b,’c,d’) = cross(a’,b’,c’,d’) a’ b’ c’ d’ a b c d

Image Rectifying: Undo parts of H x’ = H x where H = HS HA HP GOAL: Put world plane x’ into view plane • Affine Rect.; (find only HP(2DOF)); • Similarity Rect.; (find only HA HP (6DOF)); • Full Rect.; (find all of H (8DOF)); METHODS:(mix & match?) • Full: 4-point correspondence • ‘Vanishing Point’, Infinity line methods • Other feature points, conics, …

Image Rectifying: Undo parts of H x’ = H x where H = HS HA HP GOAL: Put world plane x’ into view plane • Affine Rect.; (find only HP(2DOF)); • Similarity Rect.; (find only HA HP (6DOF)); • Full Rect.; (find all of H (8DOF)); METHODS:(mix & match?) • Full: 4-point correspondence • ‘Vanishing Point’, Infinity line methods • Conics and circular points Other DOF? make assumptions, or ignore

Vanishing-Point Methods: 1 • In 2D image, find two pairs of lines that are parallel in ‘world’ space. H world space 2D image

Vanishing-Point Methods: 1 • Find intersection (vanishing points) p1, p2 (compute line intersections with 3D cross-products (see last lecture) • Horizon line Ih connects p1, p2. Ih p1 p2 H world space 2D image

Vanishing-Point Methods: 1 • H-T transforms world infinity linel (0,0,1)Tto Ih • Answer (see old book)find, copy the lh coordinates! Hp= 1 0 0 0 1 0 lh1 lh2 lh3 Ih p1 p2 H world space 2D image

Vanishing-Point Methods: 1 Limitations: • Accurate point, line-finding can be tricky • Can have high error sensitivity: ‘twitchy’ • Only rectifies Hp-- but what if HA, HS needed? Ih p1 p2 H world space 2D image

Vanishing-Point Methods: 2 No parallel lines? • find 2D image line with a known distance ratio • Use Cross-Ratio (in P1along that line) to • Find vanishing point distance at point d’ Ih p1 p2 d’ c’ H b’ a b c d(at infinity) a’ 2D image world space

Projective Transform: Hx = x’ h11 x + h12 y + h13 h31 x + h32 y + h33 h21 x + h22 y + h21 h31 x + h32 y + h33 x’2 x’3 x’1 x’3 Finding H from point pairs (correspondences) h11 h12 h13 h21 h22 h21 h31 h32 h33 x1 x2 x3 x’1 x’2 x’3 x1 x2 x3 x y 1 Input (or output) image is on central plane = = x’ = = y’ = = Write R2 expressions;(for each x,x’ pair) x’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13) Rearrange, solve as a matrix problem… y’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

Projective Transform: Hx = x’ h11 x + h12 y + h13 h31 x + h32 y + h33 h21 x + h22 y + h21 h31 x + h32 y + h33 x’2 x’3 x’1 x’3 Finding H from point pairs (correspondences) h11 h12 h13 h21 h22 h21 h31 h32 h33 x1 x2 x3 x’1 x’2 x’3 x1 x2 x3 x y 1 Input (or output) image is on central plane = = —!STOP! — Trucco Book Does This… Zisserman Book calls it “naïve”, gives better method: ‘DLT’: direct linear xfm.) x’ = = y’ = = Write R2 expressions; (for each x,x’ pair) x’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13) Rearrange, solve as a matrix problem… y’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

Projective Transform H Finding H from point pairs (correspondences) • We know that Hx’ = x, and • we know at least 4 point pairs x’ and x that satisfy it: • ATTEMPT 1:‘plug & chug’: make a matrix of x and x’ values… H x’ = x x1 x2 x3 h11 h12 h13 h21 h22 h21 h31 h32 h33 x’1 x’2 x’3 = H x’1 x1 x’2 x’3 x’4 x2 x3 x4 =

END --Please download the starter code, compile and run. --Then you can get started on Project 1 right away.

CS 395/495-26: Spring 2004