Automatic Grading for HCC in Biopsy Images

1. Automatic Grading for HCC in Biopsy Images

2. Introduction • Liver cancer is one of major health problems in the world. • Hepatocellular Carcinoma (HCC) is the most common histological type of liver cancer. • Accurate grading for HCC is important to prognosis and treatment planning.

3. Introduction • Pathological analysis of tissue is: • Time-consuming • Subjective • Inconsistent • We proposed a novel approach to automatically classifying biopsy images into five grades (0~4). • Grade 0: normal • Grade 1~4: the number increases with increasing malignancy level.

4. Image Acquisition • Every biopsy image was stained with colored dyes. • There are 804 biopsy images acquired from more than 100 patients, and every image has resolution 4080*3072. • These images were analyzed by an experienced pathologist and classified into five grades in advance.

5. Materials (G0~G4) Grade 0(Normal) Grade 1 Grade 2  Irregularity Size N/C Ratio Grade 3 Grade 4

6. Nuclei Segmentation • Step1: • Select red plane as grayscale image to process. Biopy image red plane green plane blue plane

7. Nuclei Segmentation -- Morphological Reconstruction • Step 2: • Use morphological reconstruction to remove small blemishes without affecting the shapes of nuclei. • Starts with eroding the original image, and then applies a series of conditional dilations to the marker image using the original image as the conditional image until stability is reached.

8. Morphological Reconstruction Eroded image First reconstruction Original image Dilated image Second reconstruction (result)

9. Nuclei Segmentation – Thresholding • If using thresholding method, performance of segmentation can be determined by looking at an intensity histogram of the image. • The value of threshold is not easy to choose, because intensities within nuclei and in the background are not uniform. • Set too high: many undesired elements will be generated. • Set too low: will lose some nuclei.

10. Nuclei Segmentation – Watershed • Step 3: • Using Sobel operator to get gradient image • Using marker-controlled watershed method to segment image

11. Snake Method for smoothing • Step 4: • Shape of nucleus is important information for grading. • Snake method is applied to refine the contour of the nucleus.

12. Nuclei Segmentation – Segmentation Result

13. Features Extraction • Six types of characteristics for HCC tumor cells • Nucleus-to-Cytoplasm Ratio • Nuclear Irregularity • Hyperchromatism • Nuclear Size • Anisonucleosis • Nuclear Texture

14. Type 1 - Nucleocytoplasmic Ratio • Biopsy images with high HCC grades have high nuclear density, high N/C ratio, and small cell-size. • Building Minimum Spanning Tree (MST) • Regard nuclei as nodes.

15. Type 1 - Nucleocytoplasmic Ratio • Nuclear density (F1) • Nucleus-to-Cytoplasm ratio (F2) • Cell size (F3)

16. Type 2 - Nuclear Irregularity • Because of serious deformity, the shapes of nuclei are no longer kept round in cancerous tissues. • We exploit both region-based and contour-based shape representation methods to estimate irregularity. • Circularity (F4)

17. Type 2 - Nuclear Irregularity • Area irregularity (F5) Smaller irregularity (normal) Larger irregularity(malignant)

18. Type 2 - Nuclear Irregularity • Contour irregularity (F6)

19. Type 3 - Hyperchromatism • For the case of higher grade HCC, chromatin abnormality will increase the staining capacity by staining colored dyes especially in cell nuclei.

20. Type 3 - Hyperchromatism • Average intensity (F7) • Average intensity reflects the degree of dyeing for nuclear staining and can be easily extracted from gray-level nuclei. • Higher grade HCC appears darker than that of normal nuclei. • B/D spot ratio (F8) • Within a malignant tumor, increasing chromatin will cause more holes to occur. • The bright and dark spots can be found by using top-hat and bottom-hat transforms, respectively

21. Type 4,5 – Nuclear Size, Anisonucleosis • Area (F9) • HCC with higher grade implies a higher probability of larger nuclei. • In high grades, the variance among the areas of nuclei is noticeable. • Standard deviation of nuclear size (F10) • Squared root of the average squared deviation from the mean nuclear size. • Difference of extreme nuclear sizes (F11) • Difference between the maximal and minimal areas of nuclei

22. Type 6 – Nuclear Texture • Using Co-occurrence matrix to measure nuclear texture. • Uniformity Energy (F12) • Contrast (F13) • Homogeneity (F14)

23. Grade Classification • Conventional image classification methods use a fixed number of features for classifying images. • For HCC grading, the features used for identifying various grades are different. • Nuclear texture is the best feature to distinguish grade 0 from grade 1. • N/C ratio is the best to distinguish grade 1 from grade 3. • We proposed a hierarchical classifier by combining SVM (support vector machine), SFFS (sequential floating forward selection) and DT (decision tree) for distinguishing all five grades.

24. Grade Discrimination

25. G0, G1, G2, G3, G4 FS1 G0, G1, G2, G3 G1, G2, G3, G4 FS2 FS3 G0, G1, G2 G1, G2, G3 G2, G3, G4 FS4 FS5 FS6 G1, G2 G3, G4 G0, G1 G2, G3 FS7 FS8 FS9 FS10 G0 G1 G2 G3 G4 Hierarchical Classifier Biopsy image Classifier(SVM) Optimal feature subset (SFFS) Grading result

26. Feature Selection • Sequential Floating Forward Selection (SFFS) • Plus l – take away r concept • Each SFS followed by a number of SBS steps

27. Experiments • 200 biopsy images as training set. • 604 biopsy images for testing. Hierarchical Classifier Conventional SVM

28. Experiments

29. Conclusions • HCC analysis for biopsy images is a very complex task for human. • Time-consuming • Subjective • Inconsistent • We proposed an automated nuclei segmentation, feature extraction, and a hierarchical decision model to classify HCC in biopsy images into five grades. • Automatic • Objective • Consistent • Experimental results demonstrated that 94.5 % of accuracy can be achieved for more than 100 patients with 804 biopsy images.

30. Thank you for your attention !