Retinal Image Registration: Mosaics, Multimodal Integration, and Change Visualization

1. CSci 6971: Image Registration Lecture 16: View-Based RegistrationMarch 16, 2004 Prof. Chuck Stewart, RPI Dr. Luis Ibanez, Kitware

2. Overview • Retinal image registration • The Dual-Bootstrap ICP algorithm • Covariance matrix • Covariance propagation • Model selection • View-based registration • Software design • Warning: mathematically, this lecture is a little “rich.” You will not be responsible for knowing the details Lecture 16

3. Retinal Image Registration: Applications • Mosaics • Multimodal integration • Blood flow animation • Change detection Lecture 16

4. Mosaics Lecture 16

5. Multimodal Integration Lecture 16

6. Fluorescein Angiogram Animation Lecture 16

7. Change Visualization Lecture 16

8. Retinal Image Registration - Preliminaries • Features • Transformation models • Initialization Lecture 16

9. Features • Vascular centerline points • Discrete locations along the vessel contours • Described in terms of pixel locations, orientations, and widths • Vascular landmarks • Pixel locations, orientations and width of vessels that meet to form landmarks landmarks vascular centerlines Lecture 16

10. Transformation Models Lecture 16

11. Initializing Registration • Form list of landmarks in each image • Form matches of one landmark from each image • The selection of these matches will be discussed in Lectures 18 and 19 • Choose matches, one at a time • For each match: • Compute an initial similarity transformation in the small image region surrounding the landmarks • Apply Dual-Bootstrap ICP procedure to see if the initial alignment can be successfully grown into an accurate, image-wide alignment • End when one match leads to success, or all matches are exhausted Lecture 16

12. Dual-Bootstrap - Overview Iterate until convergence • Match and refine estimate in each region • Bootstrap the model: • Low-order for small regions; • High-order for large • Automatic selection • Bootstrap the region: • Covariance propagation gives uncertainty Lecture 16

13. Matching and Estimation in Each Region • Matching - standard stuff: • Vascular centerline points from within current region of moving image • Mapped using current transform estimate • Find closest point using Borgefors digital distance map • Estimation: • Fix scale estimate • Run IRLS Lecture 16

14. Covariance Matrix of Estimate • Measures uncertainty in estimate of transformation parameters • Basis for region growth and model selection • The next few slides will give an overview of computing an approximate covariance matrix • We’ll start with linear regression Lecture 16

15. Problem Formulation in Linear Regression • Independent (non-random) variable values: • Dependent (random) variable values • Linear relationship based on k+1 dimensional parameter vector a: Lecture 16

16. Least-Squares Formulation • Least-squares error term: • Here: Lecture 16

17. Estimate and Covariance Matrix • Estimate: • Residual error variance (square of “scale”): • Parameter estimate covariance Lecture 16

18. Aside: Line Fitting in 2D • Form of the equation: • If the xi values are centered: • Then the parameters are independent with variances for the linear and constant terms, respectively Lecture 16

19. Hessians and Covariances • Back to k dimensions, re-consider the objective function: • Compute the Hessian matrix: • Observe the relationship Lecture 16

20. Hessians and Covariances • This is exact for linear regression, but serves as a good approximation for non-linear least-squares • In general, the Hessian will depend on the estimate (in regression it doesn’t because the problem is quadratic), so the approximate relationship is Lecture 16

21. Hessian in Registration • Recall the weighted least-squares objective function: • Keeping the correspondences and the weights fixed, where Dk gives the error of the k-th correspondence • Inverting this gives the covariance approximation. • This approximation is only good when the estimate is fairly accurate Lecture 16

22. Back to Dual-Bootstrap ICP • Covariance is used in two ways in each DB-ICP iteration: • Determining the region incorporates enough constraints to switch to a more complex model • Similarity => Affine => Reduced Quadratic => Quadratic • Determining the growth of the dual-bootstrap region: • More stable transformation estimates lead to faster growth Lecture 16

23. Model Selection • What model should be used to describe a given set of data? • Classic problem in statistics, and many methods have been proposed • Most trade-off the fitting accuracy of higher-order models with the stability (or lower complexity) of lower-order models Lecture 16

24. Model Selection in DB-ICP • Use correspondence set • Estimate the IRLS parameters and covariance matrices for each model in current set • For each model (with dmparameters) this generates a set of weights and errors and a covariance matrix: • Choose the model maximizing the model selection equation (derived from Bayesian modeling): • The first two terms increase with increasingly complex models; the last term decreases Lecture 16

25. Region Growth in DB-ICP • Grow each side independently • Grow is inversely proportional to uncertainty in mapping of boundary point on the center of each side • New rectangular region found from the new positions of each of the boundary points Lecture 16

26. Aside: Covariance Propagation and Transfer Error • Given mapping function: • We will treat Q as a random variable, but not gk • Uncertainty in Qmakes gk’ a random variable. • What then is the covariance of gk’? • We solve this using standard covariance propagation techniques: • Compute the Jacobian of the transformation, evaluated at gk: • Pre- and post-multiply to obtain the covariance of gk’ • In computer vision, this is called the “transfer error” Lecture 16

27. Outward Growth of a Side • Let hk be the outward normal of the side, and let rk be the distance of the side from the center of the region • Project the transfer error covariance onto hk to obtain a scalar variance sk • The outward growth (along normal hk) is • where b controls the maximum growth rate, which occurs when sk < 1 Lecture 16

28. Putting It All Together - The Example, Revisited Lecture 16

29. Turning to the Software • A “view” is a definition or snapshot of the registration problem. • A “view” contains: • An image region (current region, plus goal region) • A current transformation estimate and estimator • A current stage (resolution) of registration • Views work in conjunction with multistage registration Lecture 16

30. View-Based Registration - Procedural • The following is repeated for each initial estimate • For each stage: • Do • Match • Compute weights • Estimate scale • For each model • Run IRLS to estimate parameters and covariances • Re-estimate scale • Generate next view • For DB-ICP view generator this choose the best model and grows the region • Until region has converged and highest order model used • Prepare for next stage Lecture 16

31. Implementation • rgrl_view • Store the information about the view • rgrl_view_generator • Generate the next view • rgrl_view_based_registration • Mirrors rgrl_feature_based_registration with modifications based on the outline on previous slide • Example • rgrl/example/registration_retina.cxx Lecture 16

32. Summary • Retina registration: • Models, features and initialization • DB-ICP: • Matching, estimation and covariances • Model selection • Region growing • Generalization to view-based registration and its implementation in the toolkit. Lecture 16

33. Looking Ahead to Lecture 17 • Discussion of toolkits: • What’s easy, what’s hard • What’s missing • Project discussion • Requirements • Topics and initial steps Lecture 16