1 / 51

The Uses of Object Shape from Images in Medicine

This article explores the various uses of object shape from medical images in fields such as radiology, surgery, and radiotherapy. It covers topics like object extraction, shape and volume measurement, and image registration.

richardelee
Download Presentation

The Uses of Object Shape from Images in Medicine

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Uses of Object Shape from Images in Medicine Stephen M. Pizer Kenan Professor Medical Image Display & Analysis Group University of North Carolina Credits: Many on MIDAG, especially Daniel Fritsch, Guido Gerig, Edward Chaney, Elizabeth Bullitt, Stephen Aylward, George Stetten, Gregg Tracton, Tom Fletcher, Andrew Thall, Paul Yushkevich, Nikki Levine, Greg Clary, David Chen

  2. Object Representation in Medical Image Analysis • Extract an object from image(s) [segmentation] • Radiotherapy • Tumor; plan to hit it • Radiosensitive normal anatomy; plan to miss it • Surgery • Plan to remove it • Plan to miss it • During surgery, view where it is & effect of treatment • Radiology • View it to judge its pathology PD MRA T2 T1 Contrast

  3. Image Guided Planning of Radiotherapy • Planning in 3D • Extracting normal anatomy • Extracting tumor • Planning beam poses

  4. Object Representation in Medical Image Analysis • Registration (find geometric transformation that brings two images into alignment) • Radiotherapy • Fuse multimodality images (3D/3D) for planning • Verify patient placement (3D/2D) • Surgery • Fuse multimodality images (3D/3D or 2D) for planning • Fuse preoperative (3D) & intraoperative (2D) images • Radiology • Fuse multimodality images (3D/3D) for diagnosis

  5. Object Representation in Medical Image Analysis • Shape & Volume Measurement • Make physical measurement • Radiotherapy • Measure effect of therapy on tumor • Radiology, Neurosciences • Use measurement in science of object development • Find how probable an object is • Radiology, Neurosciences • Use measurement as quantitative input to diagnosis • Use measurement in science of object development • Use as prior in object extraction • E.g., extract the kidney shaped object

  6. Object Shape & Volume Measurement: Neurofibromatosis (Gerig, Greenwood) Infant Ventricle from 3D U/S (Gerig, Gilmore)

  7. Object Extraction (Segmentation) • Approach 1: preanalyze, then fit to model • Neurosurgery (MR Angiogram), Radiology (CT) • Vessels, ribs, bronchi, bowel via tube skeletons • Cardiology (3D Ultrasound) • Geometry via clouds of medial atoms • Fit appropriately labeled clouds to 3D LV model • Cardiac Nuclear Medicine (2D Gated Blood Pool Cine) • Extract LV, with previous frame providing model • Extraction via deformable m-rep model • Shape from extracted LV; analyze shape series • Surgery, Radiation Oncology (Multimodality MRI) • Extract tumor, using local shape characteristics

  8. Extracting Trees of Vessels via Skeletons (Aylward, Bullitt)

  9. Presenting Ribs via Tube Skeletons (Aylward)

  10. Presenting Bronchi and Lung Vessels via Tube Skeletons (Aylward)

  11. Presenting Small Bowel via Tube Skeletons (Aylward)

  12. Presenting Blood Vessels Supplying a Tumor for Embolization (Bullitt) Full tree, 2D Subtree, 2D 3D, from 2 poses

  13. Heart Model (G. Stetten)

  14. Statistical Analysis of Medial Atom Clouds (G. Stetten)

  15. LV Tube Identified by Medial Atom Statistical Analysis (G. Stetten) sphere slab cylinder

  16. Mitral Valve Slab Identified by Medial Atom Statistical Analysis (G. Stetten) sphere slab cylinder

  17. Automatic LV Extraction via Mitral Valve/LV Tube Axis (G. Stetten)

  18. Gated Blood Pool Cardiac LV Cine Shape Analysis (G. Clary) Example sequence 4-sided medial elliptical analysis

  19. Object Extraction (Segmentation) • Approach 2: deform model to optimize reward for image match + reward for shape normality • Radiation Oncology (CT or MRI) • Abdominal, pelvic organs • Deform m-reps model • Neurosciences (MRI or 3D Ultrasound) • Internal brain structures • Spherical harmonics boundary model • Deformable m-reps model • Neurosurgery (CT) • Vertebrae

  20. M- Reps for Medical Image Object Extraction and Presentation (Chen, Thall)

  21. Displacements from Figurally Implied Boundary Boundary implied by figural model Boundary after displacements

  22. Vertebral M-reps Model

  23. Vertebral M-reps Model: Spinous Process Figure

  24. Cerebral Ventricle M-reps Model

  25. Extraction with Object Shape as a Prior Brain structures (Gerig)

  26. Registration • Registration (find geometric transformation that brings two images into alignment) • Radiotherapy • Fuse multimodality images (3D/3D) for planning • Verify patient placement (3D/2D) • Surgery • Fuse multimodality images (3D/3D or 2D) for planning • Fuse preoperative (3D) & intraoperative (2D) images • Radiology • Fuse multimodality images (3D/3D) for diagnosis

  27. Image Guided Delivery of Radiotherapy • Patient placement • Verification of plan via portal image • Calculation of new treatment pose

  28. Finding Treatment Pose from Portal Radiograph and Planning DRR

  29. Medial Net Shape Models Medial net Medial nets, positions only

  30. Image Match Measurment of M-rep

  31. Registration Using Lung Medial Object Model : Reference Radiograph (Levine) Medial net Medial nets, positions only

  32. Radiograph/Portal Image Registration (Levine) Intensity Matching Relative to Medial Model Medial net

  33. Shape & Volume Measurement • Find how probable an object is • Training images; Principal components • Global vs. global and local • Correspondence Hippocampi (Gerig)

  34. Modes of Global Deformation Training set: Mode 1: x=xmean+ b1p1 Mode 2: x=xmean+ b2p2 Mode 3: x=xmean+ b3p3

  35. Shape & Volume Measurement • Shape Measurement • Modes of shape variation across patients • Measurement = amount of each mode Hippocampi (Gerig)

  36. Multiscale Medial Model • From larger scale medial net, interpolate smaller scale medial net and represent medial displacements b.

  37. Summary: What shape representation is for in medicine • Analysis from images • Extract the “anatomic object”-shaped object • Register based on the objects • Diagnose based on shape and volume • Medical science via shape • Shape and biology • Shape-based diagnostic approaches • Shape-based therapy planning and delivery approaches

  38. Shape Sciences • Medicine • Biology • Geometry • Statistics • Image Analysis • Computer Graphics

  39. The End

  40. Options for Primitives • Space: xi for grid elements • Landmarks: xi described by local geometry • Boundary: (xi ,normali) spaced along boundary • Figural: nets of diatoms sampling figures

  41. Figural Models • Figures: successive medial involution • Main figure • Protrusions • Indentations • Separate figures • Hierarchy of figures • Relative position • Relative width • Relative orientation

  42. Figural Models with Boundary Deviations • Hypothesis • At a global level, a figural model is the most intuitive • At a local level, boundary deviations are most intuitive

  43. Medial Atoms • Imply boundary segments with tolerance • Similarity transform equivariant • Zoom invariance implies width-proportionality of • tolerance of implied boundary • boundary curvature distribution • spacing along net • interrogation aperture for image

  44. Need for Special End Primitives • Represent • non-blobby objects • angulated edges, corners, creases • still allow rounded edges , corners, creases • allow bent edges • But • Avoid infinitely fine medial sampling • Maintain tangency, symmetry principles

  45. Coarse-to-fine representation • For each of three levels • Figural hierarchy • For each figure, net chain, successively smaller tolerance • For each net tile, boundary displacement chain

  46. Multiscale Medial Model • From larger scale medial net • Coarsely sampled • Smooother figurally implied boundary • Larger tolerance • Interpolate smaller scale medial net • Finer sampled • More detail in figurally implied boundary • Smaller tolerance • Represent medial displacements

  47. Multiscale Medial/Boundary Model • From medial net • Coarsely sampled, smoother implied boundary • Larger tolerance • Represent boundary displacements along implied normals • Finer sampled, more detail in boundary • Smaller tolerance

  48. Shape Repres’n in Image Analysis • Segmentation • Find the most probable deformed mean model, given the image • Probability involves • Probability of the deformed model • Probability of the image, given the deformed model

  49. Medialness: medial strength of a medial primitive in an image • Probability of image | deformed model • Sum of boundariness values • at implied boundary positions • in implied normal directions • with apertures proportional to tolerance • Boundariness value • Intensity profile distance from mean (at scale)

  50. Shape Rep’n in Image Analysis • Segmentation • Find the most probable deformed mean model, given the image • Registration • Find the most probable deformation, given the image • Shape Measurement • Find how probable a deformed model is

More Related