C4.5 and the K-Means Clustering Algorithms

1. C4.5 and the K-Means Clustering Algorithms By: Clint Tomer MATH 4200 Final Project

2. Outline • Overview • Introduction to the C4.5 Algorithm • Introduction to the K-Means Clustering Algorithm • Dataset Overview • Description of the Experiment

3. Outline cont. • Graph of Experiment • Hypothesized Results of Experiment • Actual Results of the Experiment • Experiment Conclusion • Summary

4. Overview • Use of the C4.5 and K-Means Algorithms • Partitioning the Iris.xls dataset • Run training set through K-Means and C4.5 • Run test set through rules • Compare results

5. Introduction to the C4.5 Algorithm • An upgrade • Basic idea is to ask questions • Choose splitting attributes

6. Introduction to the C4.5 Algorithm cont. • Entropy • Entropy is defined as: The given probabilities p1,p2,…..,ps where pi = 1 and H (p1,p2,….,ps) = ( pi log (1/pi))

7. Introduction to the C4.5 Algorithm cont. • C4.5 improves ID3 in the following ways: • Missing Data • Continuous Data • Pruning • Subtree Replacement • Subtree Raising

8. Introduction to the C4.5 Algorithm cont. • C4.5 improves ID3 in the following ways cont. • Rules • Splitting • GainRatio (D,S) =

9. Introduction to the K-Means Clustering Algorithm • Cluster objects • Determine the K-Means • Objects attributes form a vector space

10. Introduction to the K-Means Clustering Algorithm cont. • The objective K-Means tries to achieve is to minimize total intra-cluster variance, or the function …. where there are k clusters Si, i=1,2,….,k and mi is the mean of all points Xj E Si.

11. Introduction to the K-Means Clustering Algorithm cont. • The K-Means Clustering Algorithm can be broken down into the following steps • Place k points into the space represented by the objects that are being clustered. • Assign each object to the group that has the closest mean. • When all objects have been assigned, recalculate the positions of the k means. • Repeat steps 2 and 3 until the means no longer move.

12. Dataset Overview • Iris.xls dataset • 3 Classes • Setosa • Versicolor • Virginica • 4 Attributes • Sepal Length • Sepal Width • Pedal Length • Pedal Width • 150 Items • 50 of each class

13. Description of Experiment • K-Means • C4.5 • C4.5 Rules • K-Means • Compare • Classification

14. Graph of the Experiment Ts Td K-Means Classes C4.5 Rules Ts Results Td Results Ts Iris K-Means Classes Compare

15. Hypothesized Results • Very accurate • Precise

16. Results of the Experiment • Setosa • Versicolor • Virginica • Classification of data

17. Experiment Conclusion • C4.5 • K-Means • Classification

18. Summary • C4.5 Algorithm • K-Means Clustering Algorithm • Dataset • Experiment