Constructing Image Graphs for Segmenting Lesions in Brain MRI

1. Constructing Image Graphs for Segmenting Lesions in Brain MRI May 29, 2007 Marcel Prastawa, Guido Gerig Department of Computer Science UNC Chapel Hill

2. Goal • Segmentation of “lesions”: • Abnormal tissue associated with neurodegeneration • Small patches • Clinical applications: lupus (NAMIC), MS, aging, depression, NF1 • Different appearances, locations, and shapes • Method needs to be adaptable • Example: Constructing Image Graphs for Segmenting Lesions in Brain MRI 2

3. Outline • Background • Goal • Image Graph • Previous Work • Overview • Methodology • Results • Conclusions and Future Work Constructing Image Graphs for Segmenting Lesions in Brain MRI 3

4. Challenges • Lesions are relatively small • Wide variety of shape • Partial voluming can be confused as lesions • Requires knowledge of neighboring structures • Voxel classification typically fails • Common MRF scheme oversmooths segmentation, hard to balance Proposed solution: Use a hierarchical graph representation Constructing Image Graphs for Segmenting Lesions in Brain MRI 4

5. Image Graph Manage hierarchical information Object WM GM CSF Lesion Atom / Supervoxel / Neighborhood A1 A2 A3 Voxel v1 v2 v3 Segmentation = determining info at nodes and edges Constructing Image Graphs for Segmenting Lesions in Brain MRI 5

6. Previous Work [1/3] [Barbu et al, PAMI 2005] • Image segmentation by graph clustering • Group similar regions using Swendsen-Wang cuts • For natural images, no anatomical prior Constructing Image Graphs for Segmenting Lesions in Brain MRI 6

7. Previous Work [2/3] [Corso, Zhuowen Tu*, et al, IPMI 2007 (UCLA Loni)] • Segmentation of subcortical structures through graph shifts • Training using boosting • No pathological class *DDMCMC discriminative model guided generative model computing Constructing Image Graphs for Segmenting Lesions in Brain MRI 7

8. Previous Work [3/3] • Marcel Prastawa PhD: “An MRI Segmentation Framework for Brains with Anatomical Deviations” • EMS modulated by probabilistic brain atlas: • Nonparametric statistics • Robust clustering • Separation of pathology from healthy (tumor, edema, myelination, ..) • ITK implementation: GUI and XML scripts for large throughput • Rigorous validation (repeatability, validity, traveling phantom etc.) • Tested on over 1500 brain MRI • Marcel Prastawa, John H. Gilmore, Weili Lin, Guido Gerig, Automatic Segmentation of MR Images of the Developing Newborn Brain, Medical Image Analysis Vol 9, October 2005, pages 457-466 • John H. Gilmore, Weili Lin, Marcel W. Prastawa, Christopher B. Looney, Y. Sampath K. Vetsa, Rebecca C. Knickmeyer, Dianne Evans, J. Keith Smith, Robert M. Hamer, Jeffrey A. Lieberman, Guido Gerig, Cerebral Asymmetry, Sexual Dimorphism, and Regional Gray Matter Growth in the Neonatal Brain, Accepted by J of Neuroscience, Oct 2006 • Bénédicte Mortamet, Donglin Zeng, Guido Gerig, Marcel Prastawa, and Elizabeth Bullitt. Effects of Healthy Aging Measured By Intracranial Compartment Volumes Using a Designed MR Brain Database. Lecture Notes in Computer Science LNCS 3749, Oct. 2005, pp. 383 -- 391 • Marcel Prastawa, Elizabeth Bullitt, Sean Ho, Guido Gerig, A Brain Tumor Segmentation Framework Based On Outlier Detection, Medical Image Analysis Vol. 8, Issue 3, Sept. 2004, pages 275-283 • Marcel Prastawa, John Gilmore, Weili Lin, and Guido Gerig, Automatic Segmentation of Neonatal Brain MRI, Lecture Notes in Computer Science LNCS 3216, Springer Verlag, pp. 10-17, 2004 • Marcel Prastawa, Elizabeth Bullitt, Nathan Moon, Koen van Leemput, and Guido Gerig, Automatic Brain Tumor Segmentation by Subject Specific Modification of Atlas Priors. Academic Radiology, Vol. 10 pp. 1341-1348 Dec. 2003 • Marcel Prastawa, Elizabeth Bullitt, Sean Ho, Guido Gerig, Robust Estimation for Brain Tumor Segmentation, Lecture Notes in Computer Science LNCS 2879, pp. 530-537, Nov. 2003 Constructing Image Graphs for Segmenting Lesions in Brain MRI 8

9. Method Overview Atlas based training WM GM CSF Lesion Top-down Bottom-up interface Data driven clustering + anatomy A1 A2 A3 v1 v2 v3 Bayesian classification Constructing Image Graphs for Segmenting Lesions in Brain MRI 9

10. Outline • Background • Methodology • Results • Conclusions and Future Work Constructing Image Graphs for Segmenting Lesions in Brain MRI 10

11. Object Level • Training based on prior knowledge: brain atlas • Use for sampling and as priors • No lesion model • Lesion prior = fraction of wm or gm priors WM GM CSF Lesion ICBM/MNI atlas, average of 152 healthy adult subjects Constructing Image Graphs for Segmenting Lesions in Brain MRI 11

12. after before T2 T1 Outlier Detection • Lesion training data obtained via outlier detection • Robust estimation using MCD (minimum covariance determ.) • WM example: • Use outlier samples that fit user defined rule for lesion Constructing Image Graphs for Segmenting Lesions in Brain MRI 12

13. Lesion Rules • User defined rule for different lesion [van Leemput, TMI 2001] • Embedded Python interpreter, any function with variables for voxel data (i1, … in) and training data (mu1_1 … mu#d_#c) • Example rules: • MS lesion for [T1, T2, FLAIR]: (i2 > mu2_2) and (i3 > mu3_1) and (i3 > mu3_2) Radiology terms: Lesion is brighter than gm in T2, brighter than wm in Flair, and lesion is brighter than gm in Flair • NF1 lesion for [T1, T2, PD]: i2 > mu2_2 • Can use arithmetic: (i2/i3 > mu2_2/mu3_2) • Adaptable: input parameter, can have user def. functions, etc Constructing Image Graphs for Segmenting Lesions in Brain MRI 13

14. Atom Assignments • Atom: group of voxels that are perceptually similar • Group neighboring voxels that: • Look similar • Located close to each other • Belong to the same category • Combining 1, 2 leads to data-driven schemes CSF A1 A2 A3 v1 v2 v3 Constructing Image Graphs for Segmenting Lesions in Brain MRI 14

15. Initial Voxel Grouping • Group voxels that are similar and close to each other • Use watershed algorithm: • Input for watershed transform is gradient magnitude image (pictures from Matlab tutorial manual) Constructing Image Graphs for Segmenting Lesions in Brain MRI 15

16. Multimodal Image Gradient [Lee & Cok, IEEE TSP 1991] on gradients of vector field • Use largest singular value of Jacobian matrix (DTI analogy: use λ1 vs MD) • Example gradient image: Constructing Image Graphs for Segmenting Lesions in Brain MRI 16

17. Information Flow Communication between different levels in the hierarchy: CSF Appearance parameter adjustments • Appearance parameters (mean, covar) • Atlas priors A2 • Boundary adjustments • Split / merge atoms v2 Constructing Image Graphs for Segmenting Lesions in Brain MRI 17

18. Object-Atom and Object-Voxel Interface CSF • Object passes down intensity parameters and atlas priors • In atom level, image represented as flat patches • Compute class posterior probabilities of each voxel and atom A2 v2 Constructing Image Graphs for Segmenting Lesions in Brain MRI 18

19. Voxel-Atom Interface • Change voxel grouping based on anatomy • Possible adjustments: • Split/merge voxel groups • Boundary shift • Split / merge not yet implemented (clustering) • Boundary shift: • Every voxel in boundary between atoms get assigned to atom with nearest Kullback-Leibler (KL) distance • Simulate region competition (SNAP) CSF S2 v2 Constructing Image Graphs for Segmenting Lesions in Brain MRI 19

20. Atom-Object Interface • Atom posteriors determine classification of every child voxel • May have conflict between voxel and atom classication • Resolve by adjusting global parameters • Rationale: similar voxels must have the same classification, if not then need to accommodate violating voxels • Currently implemented by adjusting covariance CSF S2 v2 Constructing Image Graphs for Segmenting Lesions in Brain MRI 20

21. Bias Correction • MR images present intensity inhomogeneities or bias fields (“vignetting”) • Bias corrected using polynomial fit Polynomial Fit Constructing Image Graphs for Segmenting Lesions in Brain MRI 21

22. Method Summary Bias Correction CSF A2 v2 Constructing Image Graphs for Segmenting Lesions in Brain MRI 22

23. Outline • Background • Methodology • Results • Conclusions and Future Work Constructing Image Graphs for Segmenting Lesions in Brain MRI 23

24. Duke C1011A3 Depression Study Low contrast MRI Constructing Image Graphs for Segmenting Lesions in Brain MRI 24

25. Voxel Only vs Hierarchical Classification Low tissue contast Duke C1011A3 data: FLAIR Voxel-only Hierarchical Constructing Image Graphs for Segmenting Lesions in Brain MRI 25

26. hierarchical voxel only Duke C1011 Constructing Image Graphs for Segmenting Lesions in Brain MRI 26

27. Duke CRC-Oct04 (Aging/Depression) Constructing Image Graphs for Segmenting Lesions in Brain MRI 27

28. Duke CRC Constructing Image Graphs for Segmenting Lesions in Brain MRI 28

29. Multi-channel Segmentation T1w T2w Flair Labels Segmentation uses signature of all channels combined, using user-specified rules. Constructing Image Graphs for Segmenting Lesions in Brain MRI 29

30. Outline • Background • Methodology • Results • Conclusions and Future Work Constructing Image Graphs for Segmenting Lesions in Brain MRI 30

31. Conclusions • Segmentation using hierarchical scheme • Integrate top-down atlas-based approach and bottom-up data driven approach • Segments small abnormal regions • OK results on obvious high contrast lesions Constructing Image Graphs for Segmenting Lesions in Brain MRI 31

32. Future Work • Splitting / merging of atoms • Improve classification scheme using non-parametric kernel densities • Improve global parameter adjustment scheme • Partial voluming? • Tests/Adapt to lesions in NAMIC MIND DBP (lupus) • Validation Constructing Image Graphs for Segmenting Lesions in Brain MRI 32

33. Example NPSLE Lesion Hypointense on T1 Hyperintense T2 Hyperintense on FLAIR H Jeremy Bockholt , Charles Gasparovic The MIND Institute / UNM Albuquerque, NM Constructing Image Graphs for Segmenting Lesions in Brain MRI 33