250 likes | 377 Views
CS 395/495-25: Spring 2003. IBMR: Week 6B R 3 R 2 and P 3 P 2 The Camera Matrix Jack Tumblin jet@cs.northwestern.edu. Reminders. ProjA graded: Good Job! ProjB being graded... MidTerm Due today, Thurs May 8: Solutions!
E N D
CS 395/495-25: Spring 2003 IBMR: Week 6B R3R2 and P3P2 The Camera Matrix Jack Tumblin jet@cs.northwestern.edu
Reminders • ProjA graded: Good Job! • ProjB being graded... • MidTerm Due today, Thurs May 8: Solutions! • Watson’s Late Policy: Grade -(3n) points; n=# of class meetings late • ProjC posted tonight; Due next Thurs, May 15 • ProjD coming Thurs May 15, due Thurs May 29 • Take-Home Final Exam coming June 5, due
Cameras Revisited • Goal: Formalize projective 3D2D mapping • Homogeneous coords handles infinities well • Projective cameras ( convergent ‘eye’ rays) • Affine cameras (parallel ‘eye’ rays) • Composed, controlled as matrix product • But First: Cameras as Euclidean R3R2: x’ = fx / zy’ = fy / z y y’ C z f
Cameras Revisited Plenty of Terminology: • Image Plane or Focal Plane • Focal Distance f • Camera Center C • Principal Point • Principal Axis • Principal Plane (?!?! DOESN’T touch principle point!?!?) • Camera Coords (xc,yc,zc) • Image Coords (x’,y’) yc zc xc
Cameras Revisited Plenty of Terminology: • Image Plane or Focal Plane • Focal Distance f • Camera Center C • Principal Point • Principal Axis • Principal Plane (?!?! DOESN’T touch principle point!?!?) • Camera Coords (xc,yc,zc) • Image Coords (x’,y’) yc zc f xc
Cameras Revisited Plenty of Terminology: • Image Plane or Focal Plane • Focal Distance f • Camera Center C • Principal Point • Principal Axis • Principal Plane (?!?! DOESN’T touch principle point!?!?) • Camera Coords (xc,yc,zc) • Image Coords (x’,y’) yc C f zc xc
Cameras Revisited Plenty of Terminology: • Image Plane or Focal Plane • Focal Distance f • Camera Center C • Principal Point p • Principal Axis • Principal Plane (?!?! DOESN’T touch principle point!?!?) • Camera Coords (xc,yc,zc) • Image Coords (x’,y’) yc C p f zc xc
Cameras Revisited Plenty of Terminology: • Image Plane or Focal Plane • Focal Distance f • Camera Center C • Principal Pointp • Principal Axis • Principal Plane (?!?! DOESN’T touch principle point!?!?) • Camera Coords (xc,yc,zc) • Image Coords (x’,y’) yc C p f zc xc
Cameras Revisited Plenty of Terminology: • Image Plane or Focal Plane • Focal Distance f • Camera Center C • Principal Pointp • Principal Axis • Principal Plane (?!?! DOESN’T touch principle point P !?!?) • Camera Coords (xc,yc,zc) • Image Coords (x’,y’) yc C p f zc xc
Cameras Revisited Plenty of Terminology: • Image Plane or Focal Plane • Focal Distance f • Camera Center C • Principal Pointp • Principal Axis • Principal Plane(?!?! DOESN’T touch principle point!?!?) • Camera Coords (xc,yc,zc) • Image Coords (x’,y’) yc C p zc f xc
Cameras Revisited Plenty of Terminology: • Image Plane or Focal Plane • Focal Distance f • Camera Center C • Principal Pointp • Principal Axis • Principal Plane(?!?! DOESN’T touch principle point!?!?) • Camera Coords (xc,yc,zc) • Image Coords (x’,y’) yc C p f zc y' x' xc
Homogeneous Coords: R3P2 • Basic Camera as a 3x4 matrix: • Tricky!X is in augmented R3, not P3 (yet) x is in P2 space • As shown: origin of (x’,y’) is principal point p, but pixel counting starts at corners… yc xc yc zc 1 f 0 0 0 0 f 0 00 0 1 0 x’ y’ z’ = y y’ p C z zc f y' P0 X = x x' xc
Homogeneous Coords: R3P2 • Basic Camera as a 3x4 matrix: • Translate image coords (x’,y’):Shifts principal point p from (0,0) to (px,py) • is NOT obvious: scales zc • does NOT use ‘homogeneous’ 1 term in X yc xc yc zc 1 f 0 px0 0 fpy00 0 1 0 x’ y’ z’ = y y’ p C z zc f y' P0 X = x x' xc
Homogeneous Coords: R3P2 • Basic Camera as a 3x4 matrix: Less common adjustments: • Non-square pixels? change scaling (x, y) • Parallelogram pixels? set nonzero skew s K matrix: “(internal) camera calib. matrix” yc xc yc zc 1 xfspx0 0 yfpy00 0 1 0 x y z = y y’ p C z zc f K (3x3 submatrix) y' P0 X = x x' xc
Matrix Form: Euclidean R3R2 y x X (world space) z yc C zc xc y x • K matrix: “internal camera calib. matrix” • R·T matrix: “external camera calib. matrix” • T matrix: Translate world to cam. origin • R matrix: 3D rotate world to fit cam. axes Combine: write (P0·R·T)·X = x as P·X = x Input:X 3D World Space Output: x 2D Camera Image x(camera space)
Uses for Camera Matrix P xw yw zw tw • P’s Null Space: camera center C in world space!P C = 0 (solve for C. SVD works, but answer matches these:) • Finite Camera: C = Affine Camera: C = • Surprise!(minor) works projectively too: P3P2 (see book pg. 152-3 for restrictions—almost none!) • • • •• • • •• • • • • • • •• • • •• • • • xc yc zc P p4 M P·X = x, or = P = ~ ~ ~ ~ • • • 1 • • • 0 d -M-1·p4 (where Md=0)
Uses for Camera Matrix P • • ••• • ••• • • • xw yw zw tw Columns of P matrix: Points in image-space: • P1,P2,P3 == image of x,y,z axis vanishing points • Proof: let D = [1 0 0 0]T = point on x axis, at inifinity • PD = 1st column of P. Repeat for y and z axes • P4 == image of the world-space origin pt. • Proof: let D = [0 0 0 1]T = world origin • PD = 4th column of P = image of origin pt. • • • •• • • •• • • • xc yc zc P1 P2 P3 P4 P P = P·X = x, or = yc p C f zc xc
Uses for Camera Matrix P yc p C f zc xc xw yw zw tw Rows of P matrix: planes in world space • row 1 = p1T = image x-axis plane • row 2 = p2T = image y-axis plane • row 1 = p3T = camera’s principal plane p1T • • • •• • • •• • • • • • ••• • ••• • • • xc yc zc P P·X = x, or = P = p2T p3T
Uses for Camera Matrix P xw yw zw tw Rows of P matrix: planes in world space • row 1 = p1 = image x-axis plane • row 2 = p2 = image y-axis plane • row 1 = p3 = camera’s principal plane p1T • • • •• • • •• • • • • • ••• • ••• • • • xc yc zc P P·X = x, or = P = p2T p3T yc p C f zc xc
Uses for Camera Matrix P xw yw zw tw Rows of P matrix: planes in world space • row 1 = p1 = image x-axis plane • row 2 = p2 = image y-axis plane • row 1 = p3 = camera’s principal plane p1T • • • •• • • •• • • • • • ••• • ••• • • • xc yc zc P P·X = x, or = P = p2T p3T yc p C f zc xc
Uses for Camera Matrix P y' x' xw yw zw tw Rows of P matrix: planes in world space • row 1 = p1 = image x-axis plane • row 2 = p2 = image y-axis plane • Careful! Shifting image origin shifts the x,y axis planes! p1T • • • •• • • •• • • • • • ••• • ••• • • • xc yc zc P P·X = x, or = P = p2T p3T yc zc p C f xc
Uses for Camera Matrix P xw yw zw tw Rows of P matrix: planes in world space • row 1 = p1 = image x-axis plane • row 2 = p2 = image y-axis plane • row 3 = p3 = camera’s principal plane p1T • • • •• • • •• • • • • • ••• • ••• • • • xc yc zc P P·X = x, or = P = p2T p3T yc p C f zc xc
Uses for Camera Matrix P xw yw zw tw Rows of P matrix: planes in world space • row 1 = p1 = image x-axis plane • row 2 = p2 = image y-axis plane • row 3 = p3 = camera’s principal plane • princip. plane p3= [p31 p32 p33 p34]T • its normal direction: [p31 p32 p33 0 ]T • Why is it normal? It’s the world-space endpoint of zc axis (at infinity ) p1T • • • •• • • •• • • • • • ••• • ••• • • • xc yc zc P P·X = x, or = P = p2T p3T Principal plane p3 yc zc p C f xc
Chapter 6 In Just One Slide: Given point correspondence sets (xi Xi), How do you find camera matrix P ? (full 11 DOF) Surprise! You already know how ! • DLT method: -rewrite H x = x’ as Hx x’ = 0 -rewrite P X = x as PX x = 0 -vectorize, stack, solve Ah = 0 for h vector -vectorize, stack, solve Ap = 0 for p vector -Normalizing step removes origin dependence • More data better results (at least 28 point pairs) • Algebraic & Geometric Error, Sampson Error…