Clustering and Shifting of Regional Appearance for Deformable Model Segmentation

1. Clustering and Shifting of Regional Appearance for Deformable Model Segmentation Joshua V. Stough MIDAG, UNC-Chapel Hill July 25, 2008

2. Radiation Therapy Planning for Prostate Cancer • Goal: Treat prostate while avoiding neighboring tissue. • Requires Segmentation • Problems: • Low contrast • High variability • Result: Segmentation is challenging and high cost for clinicians.

3. Automating Segmentation Using Deformable Models • Automatically deform an object model to fit the image data. • Bayesian DMS • Requires an object model and a measure of image fit – image match.

4. M-rep Deformable Model • M-rep provides: • Boundary rep. with normals • Statistical deformation p(m) • Correspondence • Locality [Pizer et al., IJCV 55 (2) 2003][Fletcher et al., TMI 23 (8) 2004]

5. Image Match p(I | m): Related Work • Profile-based, voxel-scale correspondence • AAM, [Cootes et al., CVIU 61 (1) 1995] • AAM +, [Scott et al., IPMI 2003] • Intensity Profile Clustering, [Stough et al., ISBI 2004] • Region-based • Intensity Ranges, [Zhu et al., PAMI 18 (9) 1996] • Summary Statistics • [Tsai et al., TMI 22 (2) 2003] • [Chan et al., TIP 10 (2) 2001] • Histogram Metrics • [Rubner et al., CVIU 84 2001] • [Freedman et al., TMI 24 (3) 2005] • Statistics on Distributions, [Broadhurst, UNC 2008]

6. Appearance: Regional Intensity Quantile Functions (RIQFs) • Example – global object-relative regions. [Credit: Eli Broadhurst] • Principal component analysis (PCA)

7. Correspondence in Regional Image Matches Key is correspondence, both geometry- and image-based. Two factors: m-rep induced actual differences inorgan relationships • Result: the geometric correspondence given is weakly related to the image correspondence desired. • Clustering to find regions. Shifting to account for their motion.

8. Goal: Improved Segmentation Through Image Match Thesis:Segmentation benefits from the use of object-relative, regional statistical appearance models. Specifically, Clustering provides an effective understanding of the context of the object of interest in the image. Shifting of appearance allows for relaxation of the geometric correspondence.

9. Question: Which Image Match Produces Best Segmentations? Global regions Versus geometrically defined local regions Versus regions defined by RIQF clusters.

10. Determine Region Types • Pool RIQFs over all regions and training images. • Gath-Geva Clustering • Example: C= 4 on bladder exterior. [Gath, Geva PAMI 11 (7) 1989]

11. Characterize the Appearance of the Larger-Scale Regions • For each local region, choose most popular cluster. • Combine local regions belonging to the same cluster. • PCA on the larger-scale regions. • PCA over all training images

12. Experimental Setup Determine local RIQF-types in training data. Construct Gaussian models on the regions represented by the cluster types. Build a template of optimal types. • 5 patient image sets, ~16 images per patient. • UNC RadOnc and William Beaumont, Michigan. • 512  5120.98  0.98  3 mm • Volume Overlap, Average Surface Distance, Maximum Distance

13. Results • Clustered regional image match led to improved segmentations over global. • For this study: approaching expert quality, exceeding agreements between experts

14. Motivation: Shifting • Geometric correspondence is not image correspondence.

15. Allegiance-based Shifting • Each point and its RIQF on the surface is assigned a region. • Larger-scale RIQF is formed by combining the RIQFs of all points with that allegiance. • A point can change allegiance if: • On boundary • Exclusion from current region benefits current region • Addition to prospective region benefits prospective region • Or: change is a net benefit

16. Clustered Shifting • Initialize at modal allegiance • Iterate over points on boundary, change allegiance if appropriate

17. Example of Clustered Shifting Before: After:

18. Local Shifting • Subdivision of local regions • Additional connectivity constraints

19. Results

20. Results • Shifting is better than static in bladders, not in prostates. • All image matches: Overall quality approaching clinical in this study.

21. Conclusions • Clustering leads to effective appearance models for DMS. • Shifting can account for the variability in the external conformation of neighboring regions.

22. Future Directions • Applications • hippocampus, caudate and other subcortical structures in MRI • Reformulation of shifting (warp or contours) • Tissue Mixture Modeling

23. Thank you. • Committee • MIDAG • Department staff • Teaching opportunities • Mia

28. Outline • Medical Image Segmentation • Computed tomography • Bayesian deformable model segmentation • Appearance models and image match • Thesis • Novel appearance models, results • Conclusions

29. Computed Tomography • Diagnosis • Radiation Therapy Planning (RTP) • Image-guided surgery • Shape analysis…

30. M-rep Deformable Model • M-rep provides: • Boundary rep. with normals • Statistical deformation p(m) • Correspondence • Locality [Pizer et al., IJCV 55 (2) 2003][Fletcher et al., TMI 23 (8) 2004]

31. Image Match p(I | m): Related Work • Profile-based, voxel-scale correspondence • AAM, [Cootes et al., CVIU 61 (1) 1995] • AAM +, [Scott et al., IPMI 2003] • Intensity Profile Clustering, [Stough et al., ISBI 2004] • Region-based

32. Profile Appearance • Relative object position affects intensity pattern. • Local profile types. • grey-to-light, grey-to-dark, notch Axial CT slice Coronal CT slice IN OUT

33. Template Best Describes Dominant Intensity Pattern in the Boundary Region Outside Inside Left kidney Right kidney

34. Bladder and Prostate in CT • Profiles are unstable, discount spatial coherence. • Think regional.

35. Image Match p(I | m): Related Work • Profile-based, voxel-scale correspondence • AAM, [Cootes et al., CVIU 61 (1) 1995] • AAM +, [Scott et al., IPMI 2003] • Intensity Profile Clustering, [Stough et al., ISBI 2004] • Region-based • Intensity Ranges, [Zhu et al., PAMI 18 (9) 1996] • Summary Statistics • [Tsai et al., TMI 22 (2) 2003] • [Chan et al., TIP 10 (2) 2001] • Histogram Metrics • [Rubner et al., CVIU 84 2001] • [Freedman et al., TMI 24 (3) 2005] • Statistics on Distributions, [Broadhurst, UNC 2008]

36. Appearance: Regional Intensity Quantile Functions (RIQFs) • Example – global object-relative regions. [Credit: Eli Broadhurst] • Example – PCA.

37. Previous Regional Techniques • Deficiencies • Global discounts context • Summary statistics ignore nuance • Solution: regional, QF-based statistical image match • Region: organ or other volume whose local intensity distributions are distinguishable from those of neighboring volumes.

38. Thesis Segmentation benefits from the use of object-relative, regional statistical appearance models. Specifically, Clustering on object-relative image descriptors Shifting to relax geometric correspondence

39. Outline • Medical Image Segmentation • Thesis • Novel appearance models, results • Clustering and Appearance at Regional Scale • Shifting Regional Appearance • Conclusions

40. Question: Which Image Match Produces Best Segmentations? Global regions Versus geometrically defined local regions Versus regions defined by RIQF clusters.

41. Determine Region Types • Pool RIQFs over all regions and training images. • Gath-Geva Clustering • Example: C= 4 on bladder exterior. [Gath, Geva PAMI 11 (7) 1989]

42. Partition the Boundary by Cluster Type • For each local region, choose most popular cluster. • Combine local regions belonging to the same cluster.

43. PCA on larger-scale RIQFs • Combine local RIQFs through interpolation in the CDF space. • PCA over all training images

44. Experimental Setup Determine local RIQF-types in training data. Construct Gaussian models on the regions represented by the cluster types. Build a template of optimal types. • 5 patient image sets, ~16 images per patient. • UNC RadOnc and William Beaumont, Michigan. • 512  5120.98  0.98  3 mm • Dice Similarity Coefficient, Average Surface Distance

45. Results, Global v Local v Clustered RIQF clustered image match compares favorably with global, not as good as local

46. Results, Global v Local v Clustered RIQF clustered image match compares favorably with global, not as good as local

47. Results • Hausdorff medians • Images • Good improvement • Bad improvement

48. Outline • Medical Image Segmentation • Thesis • Novel appearance models, results • Clustering and Appearance at Regional Scale • Shifting Regional Appearance • Conclusions

49. Motivation: Shifting • Geometric correspondence is not image correspondence.

50. Allegiance-based Shifting • Each point and its RIQF on the surface is assigned a region. • Larger-scale RIQF is formed by combining the RIQFs of all points with that allegiance. • A point can change allegiance if: • On boundary • Exclusion from current region benefits current region • Addition to prospective region benefits prospective region