Motion Analysis

1. Motion Analysis Slides are from RPI Registration Class.

2. Registration • Images are described as discrete sets of point locations associated with a geometric measurement • Locations may have additional properties such as intensities and orientations • Registration problem involves two parts: • Finding correspondences between features • Estimating the transformation parameters based on these correspondences

3. Notation • Set of moving image features • Set of fixed image features • Each feature must include a point location in the coordinate system of its image. Set of correspondences:

4. Mathematical Formulation • Error objective function depends on unknown transformation parameters and unknown feature correspondences • Each may depend on the other! • Transformation may include mapping of more than just locations • Distance function, D, could be as simple as the Euclidean distance between location vectors.

5. Correspondence Problem • Determine correspondences before estimating transformation parameters • Based on rich description of features • Error prone • Determine correspondences at the same time as estimation of parameters • “Chicken-and-egg” problem • For the next few minutes we will assume a set of correspondences is given and proceed to the estimation of parameters • We will return to the correspondence problem later

6. Example: Estimating Parameters • 2d point locations: • Similarity transformation: • Euclidean distance:

7. Putting This Together

8. What Do We Have? • Least-squares objective function • Quadratic function of each parameter • We can • Take the derivative with respect to each parameter • Set the resulting gradient to 0 (vector) • Solve for the parameters through matrix inversion • We’ll do this in two forms: component and matrix/vector

9. Component Derivative (a)

10. Component Derivative (b) At this point, we’ve dropped the leading factor of 2. It will be eliminated when this is set to 0.

11. Component Derivatives tx and ty

12. Gathering • Setting each of these equal to 0 we obtain a set of 4 linear equations in 4 unknowns. Gathering into a matrix we have:

13. Solving • This is a simple equation of the form • Provided the 4x4 matrix X is full-rank (evaluate SVD) we easily solve as

14. Matrix Version • We can do this in a less painful way by rewriting the following intermediate expression in terms of vectors and matrices:

15. Matrix Version (continued) Equal to zero. Take transpose on either side..

16. Matrix Version (continued) • Taking the derivative of this wrt the transformation parameters (we didn’t cover vector derivatives, but this is fairly straightforward): • Setting this equal to 0 and solving yields:

17. Comparing the Two Versions • Final equations are identical (if you expand the symbols) • Matrix version is easier (once you have practice) and less error prone • Sometimes efficiency requires hand-calculation and coding of individual terms

18. Registration (rigid+scale) • Given two point sets • Our goal is to find the similarity transformation that best aligns the moments of the two point sets. • In particular we want to find • The rotation matrix R, t • The scale factor s, and • The translation vector t

19. The first moment is the center of mass

20. Second Moments • Scatter matrix • Eigenvalue / eigenvector decomposition (Sp) • where li are the eigenvalues and the columns of Vp hold the corresponding eigenvectors:

21. Rotation, Translation and Scale • Rotated, uniformly scaled points: • Center of the rotated data is the same as the rotated, scaled and translated center before rotation:

22. Scatter Matrix • Rotated, scaled scatter matrix:

23. Eigenvectors and Eigenvalues • An eigenvector is a favorite direction for a matrix. • Any square matrix M has at least one nonzero vector v which is mapped in a particularly simple way M v =  vv is eigenvector of A and  is corresponding eigenvalue • If  > 0, M v is parallel to v. If  < 0, they are antiparallel. If zero, v is in the nullspace. • In all cases, M v is just a multiple of v.

24. Eigenvalues and Eigenvectors • Given an eigenvector vj of Sp and its rotation vj’ = Rvj: R is rotation matrix • Multiply both sides by s2R, and manipulate: • So, • Eigenvectors are rotated eigenvectors • Eigenvalues are multiplied by the square of the scale j

25. The process • compute similarity transformation that best aligns the first and second moments • Assume the 2nd moments (eigenvalues) are distinct • Procedure: • Center the data in each coordinate system • Compute the scatter matrices and their eigenvalues and eigenvectors • Using this, compute • Scaling and rotation • Then, translation

26. Rotation • The rotation matrix should align the eigenvectors in order: • Manipulating: • As a result:

27. Scale • Ratios of corresponding eigenvalues should determine the scale: • Taking the derivative with respect to s2, which we are treating at • Setting the result to 0 and solving yields

28. Translation • Once we have the rotation and scale we can compute the translation based on the centers of mass:

29. Summary • Calculations are straightforward, non-iterative and do not require correspondences: • Moments, first • Rotation and scale separately • Translation • Assumes the viewpoints of the two data sets coincide approximately • Can fail miserably for significantly differing viewpoints