Affine Registration in R m

1. Higher Dimensional Affine Registration and Applications • Affine Registration and Vision Applications • Stereo Correspondences under Motion Assumption: Non-rigid motion that can be modeled using linear shape bases. Setup: 1. Two (affine) cameras observing an object undergoing non-rigid motion. 2. Feature points on object are tracked consistently in each camera. 3. Problem: Compute the correspondences between the observed feature points in two cameras. The affine connection comes from the factorization: where Xit is the 2D tracked points in images, B is the matrix of shape basis. The tensor product is between the shape coefficients and camera matrices. In the paper, we show that the points X1t X2t … Xkt belong to a subspace S of dimension 3m, where m is the number of basis elements. The correspondences, between {X1t, X2t … Xkt } and {Y1t, Y2t, … Ykt}, are given by an affine transformation A on S. • Image Set Matching • Assumption: Two collections of images are related by some affine transformation of the image space. • Example: • 1. Images related by a 2D orthogonal transform. • 2. Images from two different cameras. • Method: • 1. { X1t, X2t, …, Xkt } and { Y1t, Y2t, …, Ykt } are two collections of images. • 2. Compute PCA subspaces for the image collections and projects. • 3. Estimate the affine transformation between the projected points. • Affine Registration in Rm • Let P ={ p1, p2, …, p k} and Q = { q1, q2, …, qk } be two point sets in Rm related by • where A is a nonsingular matrix in GL(m) and • gives the correspondences. • Affine Registration in Rm • Algorithm - Reduce the problem to orthogonal case. Minimizing the error function • Coordinate transforms by the squared-root of the covariance matrix: • The transformed points are related by an unknown orthogonal transformation. • - Spectral Approach • 1. Pair-wise distances are preserved under orthogonal transform. • 2. Use the spectral information of a symmetric matrix to estimate the correspondences. • One example for the function • and this gives the (un-normalized) discrete Laplacian. • 3. The eigenvector of the symmetric matrix L can be considered as functions on the point set. If two point-sets are related by an orthogonal transform, there will be correspondences between the eigenvalues of their matrices. • 4. Using eigenvectors with corresponding eigenvalues as the features for computing correspondences. • Let • be the two sets of corresponding eigenvectors. • A matching function • Affine Registration in Rm • 5. The matching function allows to define tentative correspondences and a RANSAC-like algorithm can be used to estimate the full correpsondences. • - Affine Iterative Closest Point (Affine ICP) • 1. Once the correspondences are given, the optimal affine transform A can be computed quickly by minimizing the error function above. • 2. A can be solved via a linear system of equations. • 3. Iteratively estimate the correspondences and the affine transform A. Experiments • Experiment with Synthetic Data Points - Experiments with point sets in different dimensions. - Added noises range from 0% to 10%. - Exact affine transformation is recovered in noiseless setting. • 2D point matching. Small view differences can be accounted for by affine transformation of the images. 1. No image features (i.e., intensities) are required. 2. Matching results have small RMS error. • Image set matching. The algorithm correctly estimates all the correspondences. Convergence is observed in this experiment while a direct application of the affine ICP does not converge. • Abstract • Affine Registration has a long and venerable history in computer vision literature, and extensive work have been done for affine registration in R2 and R3. In this paper, we study affine registrations in Rm for m > 3. To justify breaking this dimension barrier, we show two interesting types of matching problems that can be formulated and solved as affine registration problems in dimensions higher than three: stereo correspondence under motion and image set matching. More specifically, for an object undergoing non-rigid motion that can be linearly modeled using a small number of shape basis vectors, the stereo correspondence problem can be solved by affine registering points in R3n. Given two collections of images related by an unknown linear transformation of the image space, the correspondences between images in the two collections can be recovered by solving an affine registration problem in Rm, where m is the dimension of a PCA subspace. The algorithm proposed in this paper estimates the affine transformation between two points in Rm. It does not require continuous optimization, and our analysis shows that in the absence of data noise, the algorithm will recover the exact affine transformation for almost all point sets. We validate the proposed algorithm on a variety of synthetic point sets in different dimensions with varying degrees of deformation and noise, and we also show experimentally that the two types of matching problems can indeed be solved satisfactorily using the proposed affine registration algorithm. • Overview • Affine registration in dimension higher than three. • Why do we need to study it? • Stereo Correspondences under Motion • Non-rigid motion modeled with linear shape basis. • Establishing correspondences across different views. • Image Set Matching • Two collections of images, related by some affine transformation of the image space. • A global approach for matching that does not require matching individual images.