The Use of Geometrical and Physical Models in Quantitative Medical Image Analysis

1. The Use of Geometrical and Physical Models in Quantitative Medical Image Analysis James S. Duncan, Ph.D. Departments of Diagnostic Radiology and Electrical Engineering Yale University

2. Segmentation: -Larry Staib -Amit Chakraborty -Isil Bozma -Xiaolan Zeng Collaborators • Cardiac Deformation: • - Xenios Papademetris • - Pengcheng Shi • -Albert Sinusas, M.D. • -Todd Constable • Brain Shift Compensation: • - Oskar Skrinjar - Dennis Spencer, M.D.

3. Outline I. Overview of Problem Areas - will focus on 3 heart/brain problems - types of 3D/4D Image Datasets II. Using Geometrically- Based Models - segmentation - motion tracking - neuroanatomical measurement III. Using Biomechanical Models - cardiac deformation analysis - brain shift compensation in neurosurgery

4. Outline I. Overview of Problem Areas - will focus on 3 heart/brain problems - types of 3D/4D Image Datasets II. Using Geometrically- Based Models - segmentation - motion tracking - neuroanatomical measurement III. Using Biomechanical Models - cardiac deformation analysis - brain shift compensation in neurosurgery

5. Active Research Areas in Medical Image Analysis • Image Distortion Correction • Quantitative Measurement of Anatomy/ Physiology • Quantitative Characterization of Disease • Data/ Information Handling/Fusion • Modeling/ prediction of Function/Disease • Improved Visualization of All of Above • Image- Guided Intervention/Surgery

6. We’ll Look at Several of These Issues from the Viewpoint of Three Examples: a.) Characterization of Cardiac Function from Noninvasive 4D Imaging b.) Study of Neuroanatomical Structure from MRI c.) Brain Shift Compensation in Image Guided Neurosurgery

7. a.) . Recovery of Cardiac Deformation(Problem Statement)

8. The Left Ventricle ED ES 2D Motion Analysis X-ray Ventriculography

9. 15 Minutes 40 Minutes RV LV LAD 3 Hours  6 Hours Occluded Vascular Bed (area at risk) Nonischemic Infarct

10. 4D Cardiac Image Datasets

11. Transmyocardial Revascularization (TMR) Canine Experiment

12. b.) Measurement of Neuroanatomical Structure(Problem Statement)

13. Example Neuroanatomical Structures of Interest 3D MRI data Parcellated cortex

15. c.) Brain Shift Compensation in Image-Guided Neurosurgery(Problem Statement)

16. Neurosurgical Issues Related to Brain Shift • Navigationvia Image-Guided Stereotactic System is Difficult/ Inaccurate: - perceived vs. actualposition of structures(especially below the surface) off by many mm. - especially problematic forplacing depth electrodesduring epilepsy surgery • in Epilepsy Surgery: Implanted Electrode Arrays(used to gather confirmatory functional information) further deform brain: - nowdifficult to relate to preop estimatesof seizure location (from SPECT/PET) and/or cognitive activity (fMRI) • Tumor localization(especially below the surface) difficult in Image-Guided system both pre- and post- resection

17. Forms of Brain Deformation • TypicalPost-Craniotomy Shiftfor different gray/CSF surface positions:0.75 mm to 7.5mm(per D. Hill, et al, CVRMed97) • AverageBrain Shift Post- (Tumor/ Hematoma) Resection, in Surrounding Region:9.5mm/7.9 mm(per Bucholz, et al., CVRMed97)

18. All 3 Examples Involve the Same Basic Algorithmic Steps • Segment Structural Boundaries from 3D/4D image datasets: - Invoke Model for Context • Recover Quantitative Information (size/ motion/deformation) about Heart or Brain: - Models Guide this Process • Visualize Results (most helpful to relate to raw image data) Each Step Utilizes Geometrical or Biomechanical Models !

19. Outline I. Overview of Problem Areas - will focus on 3 heart/brain problems - types of 3D/4D Image Datasets II. Using Geometrically- Based Models - segmentation - motion tracking - neuroanatomical measurement III. Using Biomechanical Models - cardiac deformation analysis - brain shift compensation in neurosurgery

20. Geometrical Models:Segmentation

21. Segmentation: Why is it useful in Medical Image analysis ? • identify anatomical structure/ landmarks/ROI’s for : • measurement of anatomy and function • to help register and then compare images • visualization of structure and function • group spatially-related functional activity

22. Segmentation:What’s typically used ? • Manual outlining of structure(problems: reproducibility, labor intense) • thresholding gray level histograms(no sense of spatial connectivity) • seed simple region growing algorithms that fill within isocontours (poor accuracy on real data) • feature clustering of multiecho MR images to get tissue classes(no spatial connectivity)

23. Our Take on Deformable Model- based approaches to Segmentation • Have looked generally at one structure at a time • Desire to utilize BOTH bounding structure and region homogeneity constraints • Feel that there are two general classes of problems in the brain: • 1.) “find something that roughly looks like this global shape”(e.g. the LV, many subcortical structures) • 2.) “locate the most locally-coherent structure according to these constraints”(e.g. the outer cortex)

24. Related Work from Other Groups • Shape Priors: Cootes and Taylor (IPMI93) ; Grenander/ Miller (atlases/templates- 1991) ; Vemuri, et al. (MedIA97) • Integrated Methods:1.) Region grow w/ edges- Pavlidis and Liow (PAMI91); Zhu and Yuille (ICCV95) ; Ahuja (PAMI96) 2.) level sets - Tek and Kimia (ICCV95) • Segment/ Measure Cortical Gray Matter:Macdonald and Evans (SPIE95) ; Teo and Sapiro (TMI97) ; Xu and Prince (MICCAI98) ; Davatzikos (TMI95 - ribbons) • Also seeReview of Deformable ModelsbyMcInerney and Terzopolous, Medical Image Analysis, Vol. 1 No. 2, 1996.

25. Parametrically Deformable Boundary Finding(posed as MAP Estimation)

26. The Influence of Prior Shape Orig image Region segment. Intialization/ Shape prior Integrated (region-segment. - influenced) boundary finder- NO prior Integrated boundary finder WITH shape prior (Left Thalamus) Gradient -based boundary finder- NO prior Gradient-based boundary finder WITH shape prior Objective: locate Left Thalamus

27. 3D LV Segmentation Using Prior Shape Model Short Axis Long Axes

28. Integrated Segmentation via Game Theory Region-Based segmentation P1* = X (classified pixels) Image P1 P2 Boundary Finding P2* = p (boundary parameters)

29. Game-Theoretic Integration- Results Test image: SNR =0.67 Black= intialization White = result with game-theoretic integration Black= initialization White = gradient- based result Black = gradient-based White = game-theoretic

30. Subcortical Structure Segmentation Integrated Algorithm results Manual Expert Tracing Left Caudate Nucleus Right Thalamus

31. Interactive Seeding and Correction

32. Coupled Surfaces: Image Feature Extraction • MR bias field correction performed using simple nonlinear map • local operator is used to derived the likelihood of a voxel lying on the boundary of tissues A and B Orig image gradient gray/white CSF/gray

33. g(p(q*)) 1 p(q*) 0 h(d) 0 min max |d| Coupled Surfaces Propagation A=white matter, B= gray matter, C= CSF d: distance between the two bounding surfaces

34. Coupled Surface Segmentation Results Initialization Gray/CSF surface Gray/White surface

35. Coupled Surfaces vs. Single Surface Segmentation 2 single 3D surfaces 3D coupled surfaces Note: coupled surfaces approach prevents inner surface from collapsing into CSF Note: the coupled surfaces approach prevents the outer surface from penetrating non-brain tissue (Sagittal cuts through 3D result)

36. Geometrical Models:Cardiac Motion Tracking

37. Cardiac Motion Tracking: Related Work from Other Groups • MR Tagging: Young and Axel, 1993 ; Park, Metaxas, and Axel, 1995 ; Denney and Prince, 1994. • MR Phase Contrast Velocities: Pelc, Herfkens and L. Pelc, 1992; Meyer, Constable, Sinusas and Duncan, 1995. • Biomechanical Model- based Tracking efforts: Guccione, McCulloch and Waldman, 1991; Park and Metaxas, 1996.

38. Finding Principal Curvatures Principal directions along LV endocardial surface (from DSR) • for any surface patch: curves C of max and min curvature = principal curvatures • directions on plane tangent to normal plane are principal directions

39. Curvatures are Relatively Stable From Time Frame to Time Frame e.g., note Bending Energies: (White= less bending away from flat plane,Green= more bending) From cine-MRI from DSR

40. u u v v t2 t1 Shape Based Matching t2 t1 d0d Smoothed Displacement Vectors

41. ED to ES Motion Trajectories MRI CT

42. MRI Validation w/ Markers

43. CT Validation

44. Initial Evaluation of Shape-Based Displacement: Comparison to Post Mortem/ SPECT Shape- Tracked Displacements classified by path length white = normal, blue = short, yellow/red =long Post- mortem TTC staining blue = injury zone, red = normal SPECT -derived perfusion white = normal, blue = short, yellow/red =long (n=5 canine studies)MRI/shape(in vivo) Post mortem correlation Estimated infarct area 796 +/- 124 783 +/- 131 r=.968

45. Geometrical Models:Measurement of Neuroanatomical Structure

46. Cortical Thickness Measurements on N = 30 normal male controls: Region (lobe) Left Frontal Right Frontal Left Post. Right Post Thickness 3.40(.43) 3.25 (.42) 3.06(.41) 3.00(.40) (mm) (+/- SD) Two examples: brain 1: (a and b), brain 2: (c) Note marked thinning in postcentral gyrus and prim/sec visual cortices 1 3 5 (thickness in mm)

47. Shape Index Map cup saddle cap

48. Sulcal Surface Finding (Cont.) Surface deformation to medial axis (using distance function) Brain Parcellation Initialize surface: piecewise linear mesh Extract sulcal bottom and top curves

49. Detected Sulcal Ribbons Right Central Sulcus Left Central Sulcus Right Frontal Superior Left Frontal Superior

50. Robust Point Matching of Sulcal Patterns(5 patient datasets) interhemispheric fissure;central sulcus;sylvian fissure;superior temporal sulcus