CSci 6971: Image Registration Lecture 5: Feature-Base Regisration January 27, 2004

1. CSci 6971: Image Registration Lecture 5: Feature-Base RegisrationJanuary 27, 2004 Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware

2. Overview • What is feature-based (point-based) registration? • Feature points • The correspondence problem • Solving for the transformation estimate • Putting it all together: ICP • Discussion and conclusion Lecture 5

3. What is Feature-Based Registration? • Images are described as discrete sets of point locations associated with a geometric measurement • Locations may have additional properties such as intensities and orientations • Registration problem involves two parts: • Finding correspondences between features • Estimating the transformation parameters based on these correspondences Lecture 5

4. Feature Examples: Range Data • Range image points: • (x,y,z) values • Triangulated mesh • Surface normals are sometimes computed • Notice: • Some information (locations) is determined directly by the sensor (“raw data”) • Some information is inferred from the data Lecture 5

5. Feature Examples: Vascular Landmarks • Branching points pulmonary images: • Lung vessels • Airway branches • Retinal image branches and cross-over points • Typically augmented (at least) with orientations of vessels meeting to form landmarks Lecture 5

6. Points Along Centers of Vessels and Airways • Airways and vessels modeled as tubular structures • Sample points spaced along center of tubes • Note that the entire tube is rarely used as a unit • Augmented descriptions: • Orientation • Radius Lecture 5

7. “Interest” Points • Locations of strong intensity variation in all directions • Augmented with summary descriptions (moments) of surrounding intensity structures • Recent work in making these invariant to viewpoint and illumination. • We’ll discuss interest points during Lectures 16 and 17 Brown and Lowe, Int. Conf. On Computer Vision, 2003 Lecture 5

8. Feature Points: Discussion • Many different possible features • Problem is reliably extracting features in all images • This is why more sophisticated features are not used • Feature extraction methods do not use all intensity values • Use of features dominates range-image registration techniques where “features” are provided by the sensor Lecture 5

9. Preamble to Feature-Based Registration: Notation • Set of moving image features • Set of fixed image features • Each feature must include a point location in the coordinate system of its image. It may include more • Set of correspondences Lecture 5

10. Mathematical Formulation • Error objective function depends on unknown transformation parameters and unknown feature correspondences • Each may depend on the other! • Transformation may include mapping of more than just locations • Distance function, D, could be as simple as the Euclidean distance between location vectors. • We are using the forward transformation model. Lecture 5

11. Correspondence Problem • Determine correspondences before estimating transformation parameters • Based on rich description of features • Error prone • Determine correspondences at the same time as estimation of parameters • “Chicken-and-egg” problem • For the next few minutes we will assume a set of correspondences is given and proceed to the estimation of parameters • Then we will return to the correspondence problem Lecture 5

12. Example: Estimating Parameters • 2d point locations: • Similarity transformation: • Euclidean distance: Lecture 5

13. Putting This Together Lecture 5

14. What Do We Have? • Least-squares objective function • Quadratic function of each parameter • We can • Take the derivative with respect to each parameter • Set the resulting gradient to 0 (vector) • Solve for the parameters through matrix inversion • We’ll do this in two forms: component and matrix/vector Lecture 5

15. Component Derivative (a) Lecture 5

16. Component Derivative (b) At this point, we’ve dropped the leading factor of 2. It will be eliminated when this is set to 0. Lecture 5

17. Component Derivatives tx and ty Lecture 5

18. Gathering • Setting each of these equal to 0 we obtain a set of 4 linear equations in 4 unknowns. Gathering into a matrix we have: Lecture 5

19. Solving • This is a simple equation of the form • Provided the 4x4 matrix X is full-rank (evaluate SVD) we easily solve as Lecture 5

20. Matrix Version • We can do this in a less painful way by rewriting the following intermediate expression in terms of vectors and matrices: Lecture 5

21. Matrix Version (continued) • This becomes • Manipulating: Lecture 5

22. Matrix Version (continued) • Taking the derivative of this wrt the transformation parameters (we didn’t cover vector derivatives, but this is fairly straightforward): • Setting this equal to 0 and solving yields: Lecture 5

23. Comparing the Two Versions • Final equations are identical (if you expand the symbols) • Matrix version is easier (once you have practice) and less error prone • Sometimes efficiency requires hand-calculation and coding of individual terms Lecture 5

24. Resetting the Stage • What we have done: • Features • Error function of transformation parameters and correspondences • Least-squares estimate of transformation parameters for fixed set of correspondences • Next: • ICP: joint estimation of correspondences and parameters Lecture 5

25. Iterative Closest Points (ICP) Algorithm • Given an initial transformation estimate 0 • t = 0 • Iterate until convergence: • Establish correspondences: • For fixed transformation parameter estimate, t, apply the transformation to each moving image feature and find the closest fixed image feature • Estimate the new transformation parameters, • For the resulting correspondences, estimate t+1 ICP algorithm was developed almost simultaneous by at least 5 research groups in the early 1990’s. Lecture 5

26. Finding Correspondences • Map feature into coordinate system of If • Find closest point Lecture 5

27. Finding Correspondences (continued) • Enforce unique correspondences • Avoid trivial minima of objective function due to having no correspondences • Spatial data structures needed to make search for correspondences efficient • K-d trees • Digital distance maps • More during lectures 11-15… Lecture 5

28. Initialization and Convergence • Initial estimate of transformation is again crucial because this is a minimization technique • Determining correspondences and estimating the transformation parameters are two separate processes • With Euclidean distance metrics you can show they are working toward the same minimum • In general this is not true • Convergence in practice is sometimes problematic and the correspondences oscillate between points. Lecture 5

29. 2d Retinal Example • White = vessel centerline points from one image • Black = vessel centerline points from second image • Yellow line segments drawn between corresponding points • Because of the complexity of the structure, initialization must be fairly accurate Lecture 5

30. For a given transformation estimate, we can only find a new, better estimate, not the best estimate, based on the gradient step. We then need to update the constraints and re-estimate For given set of correspondences, we can directly (least-squares) estimate the best transformation BUT, the transformation depends on the correspondences, so we generally need to re-establish the correspondences. Comparison Intensity-Based Feature-Based Lecture 5

31. Summary • Feature-based registration • Feature types and properties • Correspondences • Least-squares estimate of parameters based on correspondences • ICP • Comparison Lecture 5

32. Looking Ahead to Lecture 6 • Introduction to ITK and the ITK registration framework. Lecture 5