Automated Medical Image Registration using Global and Conditional Mutual Information

1. Automated Medical Image Registration using Global and Conditional Mutual Information Dirk Loeckx Frederik Maes, Dirk Vandermeulen, Paul Suetens

2. Medical Imaging Research Center Medical Image Computing Group of BiomedicalSciences Group of Science, Engineeringand Technology Radiology, Nuclear Medicine, Cardiology, Radiotherapy Processing of Speech & Images

3. Image Registration Find geometrical relationship between images

4. Why? • Wealth of images • Registration • Integrate information from different images • Different modality, time, patient, pose, contrast • Automatic ↔ implicit • Clinical workflow • Fully digital • Acquisition, transmission, storage, retrieval, reading • Processing time, visualisation • Validation

5. Input data Similarity Measure Transformation Penalties R Smoothness Volume Rigidity F Optimizer Deformed Image Visualization Multiresolution Analytic Derivatives LBFGSB (quasi Newton) F’ Building Blocks

6. Similarity Measure Sum of Squared Differences (SSD) Constant Relation Cross Correlation(CC) Linear Relation Mutual Information(MI) Statistical Relation

7. Mutual Information • ‘Information’ Rcarries about F and vice versa • Based on joint histogram • Combine intensity R(x) with F(g(x;m))in discrete bins • If I know p(r), how good can I predict p(f)? • Joint and marginal probabilityp(r;m), p(f;m), p(r,f;m) • F(g(x;m))unknown (digital images, partial volume effect) • No spatial information!

8. Mutual Information + – = : Reference image (histogram) : Floating image (histogram) : Entropy

9. Mutual Information • Excellent results for rigid registration • Only translation/rotation (6 parameters) • Parameters influence whole image • Sometimes works for nonrigid registration • 100’s to 1000’s of parameters • Local influence

10. Some Examples

11. Rigid CT/MR F. Maes, A. Collignon, D. Vandermeulen, G. Marchal, and P. Suetens, “Multimodality image registration by maximization of mutual information,” IEEE Trans. Med. Imag., vol. 16, no. 2, pp. 187–198, Apr 1997.

12. Nonrigid Cardiac US • Left ventricle • Manual delineation • Triangulation • Propagate delineation • Mutual information image registration • Backward and forward • Constant connectivity • 4D path of each vector • Visual validation: ok

13. Subtraction CT angiography • 3D acquisitions of region of interest • Without / with contrast • Registration • Nonrigid mutual information • Compensate for movement artefacts • Subtraction • Only contrast remains • Clear view of vasculature

14. sCTA: Examples

15. MI for nonrigid registration • Ongoing field of research • Global Histogram • Multimodal registration • Minimise minor in favour of major peaks • Reduce smaller image details • Bias field • Register bias fields, not image features (also rigid) • Local Histogram (image subdivision) • Limited ‘hits’: statistical power? • Solution: overlapping subregions

16. - Reference position Floating position Tensor-product B-splines Rectangular mesh Displacement vectors Weighted sum B-splines Multiresolution, limited span, analytical derivative Transformation model

17. Local Mutual Information • Overlapping subregions • : Spatial label, spatial bins • Multidimensional mutual information? • Total correlation, 3 channel regional MI • ‘Amount of redundancy in a set of variables’ • Conditional MI • ‘MI between R and F when X is known’ • …

18. Conditional MI (cMI) – = + • Locally, if I know p(r), • can I better predict p(f)?

19. Validation

20. In theory R F Global MI: whole image local optimum Conditional MI: central region global optimum

21. 200 2D image pairs ‘CT’, ‘MR’ 256x256 pixels I = 0, 200, 400, noise s =50 Mesh spacing 32 voxels 32 bins, PW, PV Initial transformation m uniform, < 30 pixels Validation Intensity difference Warping index ROI: 10% outside polygon Multimodal registration CT (original) MR (warped) PV, global MI PW, global MI PV, conditional MI PW,conditional MI

22. Multimodal registration CT (original) MR (warped) PV, global MI PW, global MI PV, conditional MI PW,conditional MI

23. 200 2D image pairs Lena image 8 bit, 256x256 pixels Floating: Bias field 2nd degree multiplicative Initial transformation m uniform, < 30 pixels Validation Intensity difference Warping index Bias field registration CT (original) MR (warped) PV, global MI PW, global MI PV, conditional MI PW,conditional MI

24. Bias field registration CT (original) MR (warped) PV, global MI PW, global MI PV, conditional MI PW,conditional MI

25. Conclusion • Small structures • Global joint entropy has two local optima • Bias towards minimal floating entropy • Local joint entropy has single optimum • Combination of joint and marginal entropy • Bias fields • Global MI • Combines contributions from all over the image • Bias field: widening of tissue histogram peaks • Local MI • Local histograms only • Less widening

26. Conclusion • Mesh size? • Smaller mesh = fewer voxels/bin = less statistical power • Compensated by overlapping B-spline windows (?) • 2D: 9216 voxels/bin (~ 3D: 7x7x7 voxels) • Calculation time • 10x for 2D (200s), 20x for 3D (days) • Other conditional similarity measures • Conditional Cross Correlation

27. Questions?