1 / 57

Spatially Constrained Segmentation of Dermoscopy Images

Spatially Constrained Segmentation of Dermoscopy Images. Howard Zhou 1 , Mei Chen 2 , Le Zou 2 , Richard Gass 2 , Laura Ferris 3 , Laura Drogowski 3 , James M. Rehg 1. 1 School of Interactive Computing, Georgia Tech 2 Intel Research Pittsburgh

yahto
Download Presentation

Spatially Constrained Segmentation of Dermoscopy Images

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Spatially Constrained Segmentation of Dermoscopy Images Howard Zhou1, Mei Chen2, Le Zou2, Richard Gass2, Laura Ferris3, Laura Drogowski3, James M. Rehg1 • 1School of Interactive Computing, Georgia Tech • 2Intel Research Pittsburgh • 3Department of Dermatology, University of Pittsburgh

  2. Skin cancer and melanoma • Skin cancer : most common of all cancers [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

  3. Skin cancer and melanoma • Skin cancer : most common of all cancers • Melanoma : leading cause of mortality (75%) [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

  4. Skin cancer and melanoma • Skin cancer : most common of all cancers • Melanoma : leading cause of mortality (75%) • Early detection significantly reduces mortality [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

  5. Clinical View Dermoscopy view [ Image courtesy of “An Atlas of Surface Microscopy of Pigmented Skin Lesions: Dermoscopy” ]

  6. Dermoscopy view Clinical view Dermoscopy • Improve diagnostic accuracy by 30% in the hands of trained physicians • May require as much as 5 year experience to have the necessary training • Motivation for Computer-aided diagnosis (CAD) in this area

  7. First step of analysis:Segmentation • Separatinglesions from surrounding skin • Resulting border • Gives lesion size and border irregularity • Crucial to the extraction of dermoscopic features for diagnosis • Previous Work : • PDE approach – Erkol et al. 2005, … • Histogram thresholding – Hintz-Madsen et al. 2001, … • Clustering – Schmid 1999, Melli et al. 2006… • Statistical region merging – Celebi et al. 2007, …

  8. Domain specific constraints • Spatial constraints • Four corners are skin (Melli et al.2006, Celebi et al. 2007) • Implicitly enforcing Local neighborhood constraints on image Cartesian coordinates (Meanshift)

  9. Meanshift (c = 32, s = 8) Domain specific constraints • Spatial constraints • Four corners are skin (Melli et al.2006, Celebi et al. 2007) • Implicitly enforcing Local neighborhood constraints on image Cartesian coordinates (Meanshift)

  10. Meanshift (c = 32, s = 8) We explore … • Spatial constraints arise from thegrowth pattern of pigmented skin lesions

  11. Meanshift (c = 32, s = 8) We explore … • Spatial constraints arise from thegrowth pattern of pigmented skin lesions – radiating pattern

  12. Meanshift (c = 32, s = 8) Embedding constraints • Radiating pattern from lesion growth • Embedding constraints as polar coords improves segmentation performance Polar (k = 6)

  13. Embedding constraints • Radiating pattern from lesion growth • Embedding constraints as polar coords improves segmentation performance Meanshift Polar Polar (k = 6)

  14. White: Dr. Ferris Red : Dr. Zhang Blue : computer Comparison to the Doctors • Radiating pattern from lesion growth • Embedding constraints as polar coords improves segmentation performance Meanshift Polar

  15. Dermoscopy images Common radiating appearance

  16. Growth pattern of pigmented skin lesions • lesions grow in both radial and vertical direction • Skin absorbs and scatters light. • Appearance of pigmented cells varies with depth • Dark brown  tan  blue-gray • Common radiating appearance pattern on skin surface [ Image courtesy of “Dermoscopy : An Atlas of Surface Microscopy of Pigmented Skin Lesions]

  17. Radiating growth pattern on skin surface • Difference in appearance: more significant along the radial direction than any other direction.

  18. Radiating growth pattern on skin surface • Difference in appearance: more significant along the radial direction than any other direction.

  19. original r {R, G, B} Embedding spatial constraintsFeature vectors • Each pixel  feature vector in R4 • 3D: R,G,B or L, a, b in the color space • 1D: polar radius measured from the center of the image (normalized by w)

  20. original filtered r {R, G, B} Embedding spatial constraintsGrouping features • Each pixel  feature vector in R4 • Clustering pixels in the feature space • Replace pixels with mean for compact representation

  21. … … Radiating pattern Dermoscopy vs. natural images Derm dataset (216) BSD dataset (300)

  22. Cartesian {Rc, Gc, Bc} original polar {Ro, Go, Bo} {Rp, Gp, Bp} Embedding spatial constraintsGrouping features • Mean per-pixel residue: average per-pixel color difference of each pair

  23. Residue (Cartesian) Residue (Cartesian) Residue (polar) Residue (polar) BSD dataset (300) Derm dataset (216) Dermoscopy vs. natural images Polar vs. Cartesion • Mean per-pixel residue (k-means++, k = 30)

  24. Dermoscopy vs. natural images Polar vs. Cartesion • Mean per-pixel residue (k-means++, k = 30)

  25. Polar vs. Cartesian • The regions appear more blocky in the Cartesian case Polar (k = 30) Cartesian (k = 30)

  26. Six super-regions • 30 clusters  6 super clusters (K-means++) Polar (k = 6) Cartesian (k = 6)

  27. Final segmentation Polar Cartesian

  28. Polar vs. Meanshift • The regions appear more blocky in the Meanshift case Polar (k = 6) Meanshift (c = 32, s = 8)

  29. Final segmentation Polar Meanshift

  30. Algorithm overview • Given a dermoscopy image

  31. original Algorithm overview • Given a dermoscopy image

  32. original 30 clusters Algorithm overview 1. First round clustering: K-means++ (k = 30)

  33. original 30 clusters 6 Super-regions Algorithm overview 2. Second round: clusters(30) super-regions(6)

  34. original 30 clusters 6 Super-regions Texture edge map Algorithm overview 3. Apply texture gradient filter (Martin, et al. 2004)

  35. original 30 clusters 6 Super-regions Final segmentation Texture edge map Algorithm overview 4. Find optimal boundary (color+texture)

  36. original 1. First round clustering • First round clustering: K-means++ (k = 30) • Reduce noise • Groups pixels into homogenous regions – a more compact representation of the image • Artuhur and Vassilvitskii, 2007 • R4 : {L*a*b* (3D), w * polar radius (1D)}

  37. original 30 clusters 1. First round clustering • First round clustering: K-means++ (k = 30) • Reduce noise • Groups pixels into homogenous regions – a more compact representation of the image • Artuhur and Vassilvitskii, 2007 • R4 : {L*a*b* (3D), w * polar radius (1D)}

  38. original 30 clusters 2. Second round clustering • K = 6 : clusters(30) super-regions(6) • Account for intra-skin and intra-lesion variations • Avoid a large k • Super-regions correspond to meaningful regions such as skin, skin-lesion transition, and inner lesion, etc.

  39. original 30 clusters 6 super-regions 2. Second round clustering • K = 6 : clusters(30) super-regions(6) • Account for intra-skin and intra-lesion variations • Avoid a large k • Super-regions correspond to meaningful regions such as skin, skin-lesion transition, and inner lesion, etc.

  40. original 3. Color-texture integration • Incorporating texture information can improve segmentation performance. • Severely sun damaged skin; texture variations at boundaries in addition to color variations

  41. original 3. Color-texture integration • Incorporating texture information can improve segmentation performance. • Severely sun damaged skin; texture variations at boundaries in addition to color variations • Apply texture gradient filter(Martin, et al. 2004)

  42. original Texture edge map 3. Color-texture integration • Incorporating texture information can improve segmentation performance. • Severely sun damaged skin; texture variations at boundaries in addition to color variations • Apply texture gradient filter(Martin, et al. 2004) • Texture edge map: pseudo-likelihood

  43. 6 super-regions 4. Optimal boundary • Optimal skin-lesion boundary • Color: Earth Mover’sDistance (EMD) between every pair of super-regions

  44. Texture edge map 6 super-regions 4. Optimal boundary • Optimal skin-lesion boundary • Color: Earth Mover’sDistance (EMD) between every pair of super-regions • Texture: Texture edge map

  45. Texture edge map 6 super-regions 4. Optimal boundary • Optimal skin-lesion boundary • Color: Earth Mover’sDistance (EMD) between every pair of super-regions • Texture: Texture edge map • Minimizing theintegrated color-texture measure

  46. Validation and results • Our collaborating dermatologist Dr. Ferris manually outline thelesions in 67 dermoscopy images • The border error is givenby • Computer: binary image obtained by filling the automaticdetected border • ground-truth: obtained by fillingin the boundaries outlined by Dr. Ferris

  47. White: Dr. Ferris Red : Dr. Zhang Blue : computer Error = 12.96% Typical segmentation result

  48. Comparison To account for inter-operator variation, we also asked Dr. Alex Zhang to manually outline boundaries on the same dataset

  49. White: Dr. Ferris Red : Dr. Zhang Blue : computer Additional results Error = 5.80%

  50. White: Dr. Ferris Red : Dr. Zhang Blue : computer Additional results Error = 13.61%

More Related