Multi-linear Systems and Invariant Theory in the Context of Computer Vision and Graphics

Multi-linear Systems and Invariant Theory in the Context of Computer Vision and Graphics Class 5: Self Calibration CS329 Stanford University Amnon Shashua

Material We Will Cover Today • The basic equations and counting arguments • The “absolute conic” and its image. • Kruppa’s equations • Recovering internal parameters.

The Basic Equations and Counting Arguments Recall, 3D->2D from Euclidean world frame to image world frame to first camera frame Let K,K’ be the internal parameters of camera 1,2 and choose canonical frame in which R=I and T=0 for first camera.

The Basic Equations and Counting Arguments where (8 unknown parameters) maps from the projective frame to Euclidean

The Basic Equations and Counting Arguments are the points on the plane at infinity (in Euc frame) is the plane at infinity is the plane at infinity in Proj frame (recall: if W maps points to points (Euc -> Proj), then the dual maps planes to planes)

The Basic Equations and Counting Arguments

The Basic Equations and Counting Arguments Projective frame

The Basic Equations and Counting Arguments since then, but provides 5 (non-linear) constraints!

The Basic Equations and Counting Arguments Since the right-hand side is symmetric and up to scale, we have 5 constraints.

The Basic Equations and Counting Arguments Lets do some counting: Let be the number of internal parameters be the number of views

The Basic Equations and Counting Arguments not enough measurements (!) (fixed internal params)

The remainder of this lecture is about a geometric insight of

The Absolute Conic where represents a conic in 2D are the points on the plane at infinity (in Euc frame) is the plane at infinity is conic on the plane at infinity when is the “absolute” conic (imaginary circle)

The Absolute Conic Plane at infinity is preserved under affine transformations: because is preserved under similarity transformation (R,t up to scale) and if then but so in order that we must have: is orthogonal

The Image of the Absolute Conic Image of points at infinity: let if is a conic on the plane at infinity then is the projected conic onto the image then since the image of is

The Image of the Dual Absolute Conic is tangent to the conic at p is the image of the dual absolute conic The basic equation: Becomes: Why 8 parameters? 5 for the conic, 3 for the plane

Geometric Interpretation of p direction of optical ray The angle between two optical rays given one can measure angles

Kruppa’s Equations General idea: eliminate n from the basic equation. are degenerate (rank 2) conics

Kruppa’s Equations Note: is a degenerate conic iff or Let be the homography induced by the plane of the conic (slide 14)

Kruppa’s Equations Recall: and the conic is In our case Likewise:

Determining K given the location of the plane at inifinity in the projective coordinate frame. Recall: We wish to represent the homography induced by be a point on the plane at infinity. Let

Determining K given Recall: (slide 16) Note: this could be derived from “first principles” as well: tangents lines to the image of the absolute conic

Determining K given Assume fixed internal parameters Note: Provides 4 independent linear constraints on Why 4 and not 5? we need 3 views (since has 5 unknowns)

Why 4 Constraints? and are “similar” matrices, i.e., have the same eigenvalues be the axis of rotation, i.e., Let has an eigenvalue = 1, with eigenvector

Why 4 Constraints? if is a solution to then is also a solution We need one more camera motion (with a different axis of rotation).

Kruppa’s Equations (revisited) Kruppa’s equations: Start with the basic equation: Multiply the terms by on both sides:

Multi-linear Systems and Invariant Theory in the Context of Computer Vision and Graphics