1 / 23

CS 395/495-26: Spring 2004

Dive into the concept of Projective Transformation using the Direct Linear Transform (DLT) method. Learn to solve for point correspondences, manipulate matrix expressions, and explore the flexibility and compatibility of DLT. Discover the subtleties and robustness of this approach, including the significance of SVD in matrix mathematics.

esherry
Download Presentation

CS 395/495-26: Spring 2004

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 395/495-26: Spring 2004 IBMR:Projective 2D DLT Jack Tumblin jet@cs.northwestern.edu

  2. Recall: Projective Transform H x y Apply the 3x3 matrix Hx’ = Hx 2D image (x,y)Homog. coords [x,y,w]T = x w=1 Homog. coordsx’ =[x’,y’,w’]Tw’=1 2D image (x’,y’) x x x2 (x/w,y/w) y x y y’ y x3 x x’ x’ y x1 (x’/w’,y’/w’) x

  3. Recall: Projective Transform: Hx = x’ h11 x + h12 y + h13 h31 x + h32 y + h33 h21 x + h22 y + h21 h31 x + h32 y + h33 x’2 x’3 x’1 x’3 H defines point pairs (x,x’) (correspondences) h11 h12 h13 h21 h22 h21 h31 h32 h33 x1 x2 x3 x’1 x’2 x’3 x1 x2 x3 x y w Input (or output) image is on central plane = = x = [x1, x2, x3] notation is confusing for multiple points! Instead, let’s use [x,y,w]: (so we can use subscripts to number the point pairs x,x’…) x’ = = y’ = = Write R2 expressions;(for each x,x’ pair) x’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13) Rearrange, solve as a matrix problem… y’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

  4. Recall: Projective Transform: Hx = x’ h11 x + h12 y + h13 h31 x + h32 y + h33 h21 x + h22 y + h21 h31 x + h32 y + h33 x’2 x’3 x’1 x’3 H defines point pairs (x,x’) (correspondences) h11 h12 h13 h21 h22 h21 h31 h32 h33 x1 x2 x3 x’1 x’2 x’3 x1 x2 x3 x y w Input (or output) image is on central plane = = let w=1 x’ = = y’ = = Write R2 expressions;(for each x,x’ pair) x’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13) Rearrange, solve as a matrix problem… y’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

  5. Recall: Projective Transform: Hx = x’ h11 x + h12 y + h13 h31 x + h32 y + h33 h21 x + h22 y + h21 h31 x + h32 y + h33 x’2 x’3 x’1 x’3 Finding H from point pairs (correspondences) h11 h12 h13 h21 h22 h21 h31 h32 h33 x1 x2 x3 x’1 x’2 x’3 x1 x2 x3 x y 1 Input (or output) image is on central plane = = x’ = = y’ = = Write R2 expressions;(for each x,x’ pair) x’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13) Rearrange, solve as a matrix problem… y’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13)

  6. Naïve Method: h11 h12 h13 h21 h22 h23 h31 h32 h33 4 point correspondence: Book shows: Rearrange: known vector (dot) unknown vector stack, solve for null space… But this assumes x1=x, x2=y, x3=w=1, There is a better way (DLT)… x’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13) y’ (h31 x + h32 y + h33) = (h11 x + h12 y + h13) [x y 1 0 0 0 -x’x -x’y -x’] = 0 [0 0 0 x y 1 -y’x -y’y -y’]

  7. Direct Linear Tranform (DLT) • Naive method: Solve Hx – x’ = 0 • Requires constant scale (x3=1 always) • compare: • A BETTER WAY:DLT method: Solve Hx  x’ = 0 • Accepts any scale (x3=w1 OK…) h11 h12 h13 h21 h22 h23 h31 h32 h33 [ 0 0 0 x y 1 -y’x -y’y -y’ ] [ x y 1 0 0 0 -x’x -x’y -x’ ] [ 0 0 0 -w’x -w’y -w’w y’x y’y y’w] = 0 [w’x w’y w’w 0 0 0 -x’x -x’y -x’w]

  8. Direct Linear Tranform (DLT) • Naive method: Solve Hx – x’ = 0 • Requires constant scale (x3=1 always) • Compare to DLT: • DLT method: Solve Hx  x’ = 0 • Accepts any scale (x3=w1, x3’=w’1) h1 h2 h3 [ 0 0 0 x y 1 -y’x -y’y -y’ ] [ x y 1 0 0 0 -x’x -x’y -x’ ] [ 0 0 0 -w’x -w’y -w’w y’x y’y y’w] = 0 [w’x w’y w’w 0 0 0 -x’x -x’y -x’w] [ 0T -w’i xiT y’I xiT ] [w’i xiT 0T -x’i xiT ]

  9. Direct Linear Tranform (DLT) • DLT method: Solve H  x = 0 • Accepts any scale, any point (x3=w1 is OK) • ‘Pure’, Compatible –uses only P2terms. • Much better suited to error measurements. • Subtlety: • Won’t divide-by-zero if w=0 or h33=0(it happens!) • has a 3rd row; it is not degenerate if/when w’=0 • OK to use it…(Solve 8x12) 0T -w’i xiTy’I xiT w’i xiT0T -x’i xiT =0 y’i xiT -x’i xiT 0T h1 h2 h3

  10. STOP: Learn about SVDs …

  11. Matrix Math & SVDs • Matrix multiply is a change of coordinates • Rows of A = coordinate axis vectors • Ax = a dot product for each axis • input spaceoutput space • SVD ‘factors’ matrix A into 3 parts: • U columns: orthonormal axes for OUTPUT • V columns: orthonormal axes for INPUT • S: scale factors VU svd(A) = U S VT

  12. Actual ‘Robustness’ • Vectorizing (‘Flatten, Stack, Null Space’ method) works for almost ANY input! (Points, lines, planes, ||, , conics, quadrics, cross-ratios, vanishing points, twisted cubics… • Key to DLT formulation: (Hx  x’) = 0 • Rearrange as dot product: (known)·(unknown) = 0 • Be careful to have ENOUGH constraints tricky when you mix types: points, lines,… (pg 75) = 0 measured output measured inputtransformed 

  13. Deceptions: false ‘Robustness’… • Suppose we have 4 pt-correspondences: • Use DLT to write 8x9 (or 12x9) matrix A: Ah=0 • Solve for h null space. ALWAYS gives H matrix • But what if points are bad / fictional? • 3 collinear input pts, crooked out: IMPOSSIBLE! • Yet we get an H solution! Why? • A matrix rank is 7 or 6 rank 1 H result(s) • ‘Null Space’ of A may contain >1 H solution! • ‘Degenerate’ H solution of form aH1 + bH2… • Answer: SVD ranks A; reject bad point sets.

  14. Most Important SVD Ideas SVD solves Null-Space problems Ax = 0(‘flatten, stack, find null space’ to solve) • MANY problems can be written this way • Expandable! --more data points tall, ‘overconstrained’ A matrix --SVD finds optimal ‘least squares’ solution. Robustness: bane of all inverse methodsAccuracy vs. # of measurements (more isn’t always better! Outliers...) Data-rich images: quantity easier than qualitymore is easier than better) ?What links measurement errorsH errors? ?How can more measurements reduce error?

  15. How else can we use DLT? • I know this building consists of MANY parallel and perpendicular lines, corners, regularly spaced points. • How many different ways can we find to ‘rectify’ this image to get this one?

  16. Direct Linear Tranform (DLT) • Naïve method: Solves Hx – x’ = 0 • Uses constant scale (x3=w=1 in measurement) • Compare to DLT: • DLT method: Solve Hx  x’ = 0 • (a different identity: x’  x’ = 0) • Accepts any scale (x3=w1 OK, xx3) h11 h12 h13 h21 h22 h23 h31 h32 h33 [ 0 0 0 x y 1 -y’x -y’y -y’ ] [ x y 1 0 0 0 -x’x -x’y -x’ ] [ 0 0 0 -w’x -w’y -w’w y’x y’y y’w] = 0 [w’x w’y w’w 0 0 0 -x’x -x’y -x’w]

  17. Direct Linear Tranform (DLT) • Naïve method: Solve Hx – x’ = 0 • Requires constant scale (x3=1 always) • compare: • DLT method: Solve Hx  x’ = 0 • Accepts any scale (x3=w1, x3’=w’1) h1 h2 h3 [ 0 0 0 x y 1 -y’x -y’y -y’ ] [ x y 1 0 0 0 -x’x -x’y -x’ ] [ 0 0 0 -w’x -w’y -w’w y’x y’y y’w] = 0 [w’x w’y w’w 0 0 0 -x’x -x’y -x’w] [ 0T -w’i xiT y’I xiT ] [w’i xiT 0T -x’i xiT ]

  18. Direct Linear Tranform (DLT) • DLT method: Solve H  x = 0 • Accepts any scale, any point (x3=w1 is OK) • ‘Pure’ and Compatible -- P2terms only, no R2 • Much better suited to error measurements. • Subtlety: • Won’t divide-by-zero if w=0 or h33=0(it happens!) • has a 3rd row; it is not degenerate if w’=0 • OK to use it…(Solve 8x12) 0T -w’i xiTy’I xiT w’i xiT0T -x’i xiT =0 -y’i xiT -x’i xiT 0T h1 h2 h3

  19. Adding More Measurements How can we use >4 point correspondences? • Easy: • Add more rows to our 8x9 matrix A: A·h = 0 • Use SVD to find Null space (Always gives an answer!) • Result: ‘Least squares’ solution; error=A·h =  • minimizes || A h ||2 = i 2i where,iis error for i-th pt. correspondence: • i = || Hxi xi’ ||2 = || (2 rows of A)*h ||2 = ‘algebraic distance’ • ‘Algebraic Distance’ ? No geometric meaning!

  20. Adding More Measurements ( - )2+ ( - )2 d(a,b)2 = b2 b3 b1 b3 a1 a3 a2 a3 • 2D‘Algebraic Distance’ ?No geometric meaning! • 2D ‘Geometric Distance’ d(a,b)2is Better: measurable length in input or output space if a = (a1 a2 a3) and b = (b1 b2 b3), then define Turns out that: d(a,b)2 = dalgebraic(a,b) a3·b3 (Not very surprising)

  21. Adding More Measurements Iterative Strategy: • Overconstrain the answer H • Collect extra measurements (>4 point pairs, etc. …) • expect errors; call them ‘estimates’ x • Compute a 1st solution (probably by SVD) • Compute error d(Hx, x’)2, and use this to… • ‘Tweak’ answer Hand estimates x • Compute new answer • Stop when error < useful threshold ^ ^ ^ ^

  22. Using Estimates • Simplest: ‘one image’ transfer method: • Assume inputs are a perfect test pattern: only output pts are in error • Adjust output estimates x’ until d(Hx, x’)20 (note we re-compute H as x’ changes) • Better: ‘Symmetric’ transfer method: • Assume BOTH input x and output x’ has error. • Adjust BOTH input and output estimates x x’(note we also re-compute H as x, x’ change) • Stop when d(Hx,x’)2 + d(H-1x’, x)2is nearly 0 ^ ^ ^ ^ ^ ^ ^ ^

  23. END

More Related