1 / 54

Introduction to Data Mining: Clustering Analysis

Learn about the principles, applications, and requirements of clustering analysis in data mining. Understand similarity measures and analyze different types of data.

cclaude
Download Presentation

Introduction to Data Mining: Clustering Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. 7.11. 24./26.10. 14.11. Home Exam 30.10. 21.11. 28.11. Course on Data Mining (581550-4) Intro/Ass. Rules Clustering Episodes KDD Process Text Mining Appl./Summary Data mining: Clustering

  2. Course on Data Mining (581550-4) Today 14.11.2001 • Today's subject: • Classification, clustering • Next week's program: • Lecture: Data mining process • Exercise: Classification, clustering • Seminar: Classification, clustering Data mining: Clustering

  3. Classification and clustering • Classification and prediction • Clustering and similarity Data mining: Clustering

  4. Cluster analysis • What is cluster analysis? • Similarity and dissimilarity • Types of data in cluster analysis • Major clustering methods • Partitioning methods • Hierarchical methods • Outlier analysis • Summary Overview Data mining: Clustering

  5. What is cluster analysis? • Cluster: a collection of data objects • similar to one another within the same cluster • dissimilar to the objects in the other clusters • Aim of clustering: to group a set of data objects into clusters Data mining: Clustering

  6. Typical uses of clustering • As a stand-alone tool to get insight into data distribution • As a preprocessing step for other algorithms Used as? Data mining: Clustering

  7. Applications of clustering • Marketing: discovering of distinct customer groups in a purchase database • Land use: identifying of areas of similar land use in an earth observation database • Insurance: identifying groups of motor insurance policy holders with a high average claim cost • City-planning: identifying groups of houses according to their house type, value, and geographical location Data mining: Clustering

  8. What is good clustering? • A good clustering method will produce high quality clusters with • high intra-class similarity • low inter-class similarity • The quality of a clustering result depends on • the similarity measure used • implementation of the similarity measure • The quality of a clustering method is also measured by its ability to discover some or all of the hidden patterns Data mining: Clustering

  9. Requirements of clustering in data mining (1) • Scalability • Ability to deal with different types of attributes • Discovery of clusters with arbitrary shape • Minimal requirements for domain knowledge to determine input parameters Data mining: Clustering

  10. Requirements of clustering in data mining (2) • Ability to deal with noise and outliers • Insensitivity to order of input records • High dimensionality • Incorporation of user-specified constraints • Interpretability and usability Data mining: Clustering

  11. Similarity and dissimilarity between objects (1) • There is no single definition of similarity or dissimilarity between data objects • The definition of similarity or dissimilarity between objects depends on • the type of the data considered • what kind of similarity we are looking for Data mining: Clustering

  12. Similarity and dissimilarity between objects (2) • Similarity/dissimilarity between objects is often expressed in terms of a distancemeasure d(x,y) • Ideally, every distance measure should be a metric, i.e., it should satisfy the following conditions: Data mining: Clustering

  13. Type of data in cluster analysis • Interval-scaled variables • Binary variables • Nominal, ordinal, and ratio variables • Variables of mixed types • Complex data types Data mining: Clustering

  14. Interval-scaled variables (1) • Continuous measurements of a roughly linear scale • For example, weight, height and age • The measurement unit can affect the cluster analysis • To avoid dependence on the measurement unit, we should standardize the data Data mining: Clustering

  15. Interval-scaled variables (2) To standardize the measurements: • calculate the mean absolute deviation where and • calculate the standardized measurement (z-score) Data mining: Clustering

  16. Interval-scaled variables (3) • One group of popular distance measures for interval-scaled variables are Minkowski distances where i = (xi1, xi2, …, xip)andj = (xj1, xj2, …, xjp)are twop-dimensional data objects, andqis a positive integer Data mining: Clustering

  17. Interval-scaled variables (4) • If q = 1,the distance measure is Manhattan (or city block) distance • If q = 2,the distance measure is Euclidean distance Data mining: Clustering

  18. Binary variables (1) • A binary variable has only two states: 0 or 1 • A contingency table for binary data Object j Object i Data mining: Clustering

  19. Binary variables (2) • Simple matching coefficient (invariant similarity, if the binary variable is symmetric): • Jaccard coefficient (noninvariant similarity, if the binary variable is asymmetric): Data mining: Clustering

  20. Binary variables (3) Example: dissimilarity between binary variables: • a patient record table • eight attributes, of which • gender is a symmetric attribute, and • the remaining attributes are asymmetric binary Data mining: Clustering

  21. Binary variables (4) • Let the values Y and P be set to 1, and the value N be set to 0 • Compute distances between patients based on the asymmetric variables by using Jaccard coefficient Data mining: Clustering

  22. Nominal variables • A generalization of the binary variablein that it can take more than 2 states, e.g., red, yellow, blue, green • Method 1: simple matching • m: # of matches,p: total # of variables • Method 2: use a large number of binary variables • create a new binary variable for each of the M nominal states Data mining: Clustering

  23. Ordinal variables • An ordinal variable can be discrete or continuous • Order of values is important, e.g., rank • Can be treated like interval-scaled • replacing xif by their rank • map the range of each variable onto [0, 1] by replacing i-th object in the f-th variable by • compute the dissimilarityusing methods for interval-scaled variables Data mining: Clustering

  24. Ratio-scaled variables • A positive measurement on a nonlinear scale, approximately at exponential scale • for example, AeBt or Ae-Bt • Methods: • treat them like interval-scaled variables — not a good choice! (why?) • apply logarithmic transformation yif = log(xif) • treat them as continuous ordinal data and treat their rank as interval-scaled Data mining: Clustering

  25. Variables of mixed types (1) • A database may contain all the six types of variables • One may use a weighted formula to combine their effects: where Data mining: Clustering

  26. Variables of mixed types (2) Contribution of variable f to distance d(i,j): • if f is binary or nominal: • if f is interval-based: use the normalized distance • if f is ordinal or ratio-scaled • compute ranks rif and • and treat zif as interval-scaled Data mining: Clustering

  27. Complex data types • All objects considered in data mining are not relational => complex types of data • examples of such data are spatial data, multimedia data, genetic data, time-series data, text data and data collected from World-Wide Web • Often totally different similarity or dissimilarity measures than above • can, for example, mean using of string and/or sequence matching, or methods of information retrieval Data mining: Clustering

  28. Major clustering methods • Partitioning methods • Hierarchical methods • Density-based methods • Grid-based methods • Model-based methods (conceptual clustering, neural networks) Data mining: Clustering

  29. Partitioning methods • A partitioning method: construct a partition of a database D of n objects into a set of k clusters such that • each cluster contains at least one object • each object belongs to exactly one cluster • Given a k, find a partition of k clustersthat optimizes the chosen partitioning criterion Data mining: Clustering

  30. Criteria for judging the quality of partitions • Global optimal: exhaustively enumerate all partitions • Heuristic methods: • k-means (MacQueen’67): each cluster is represented by the center of the cluster (centroid) • k-medoids (Kaufman & Rousseeuw’87): each cluster is represented by one of the objects in the cluster(medoid) Data mining: Clustering

  31. K-means clustering method (1) • Input to the algorithm: the number of clusters k, and a database of n objects • Algorithm consists of four steps: • partition object into k nonempty subsets/clusters • compute a seed points as the centroid (the mean of the objects in the cluster) for each cluster in the current partition • assign each object to the cluster with the nearest centroid • go back to Step 2, stop when there are no more new assignments Data mining: Clustering

  32. K-means clustering method (2) Alternative algorithm also consists of four steps: • arbitrarily choose k objects as the initial cluster centers (centroids) • (re)assign each object to the cluster with the nearest centroid • update the centroids • go back to Step 2, stop when there are no more new assignments Data mining: Clustering

  33. K-means clustering method - Example Data mining: Clustering

  34. Strengths of K-means clustering method • Relatively scalable in processing large data sets • Relatively efficient: O(tkn), where n is # objects, k is # clusters, and t is # iterations. Normally, k, t << n. • Often terminates at a local optimum; the global optimum may be found using techniques such as genetic algorithms Data mining: Clustering

  35. Weaknesses of K-means clustering method • Applicable only when the mean of objects is defined • Need to specifyk, the number of clusters, in advance • Unable to handle noisy data and outliers • Not suitable to discover clusters with non-convex shapes, or clusters of very different size Data mining: Clustering

  36. Variations of K-means clustering method (1) • A few variants of the k-means which differ in • selection of the initial k centroids • dissimilarity calculations • strategies for calculating cluster centroids Data mining: Clustering

  37. Variations of K-means clustering method (2) • Handling categorical data: k-modes (Huang’98) • replacing means of clusters with modes • using new dissimilarity measures to deal with categorical objects • using a frequency-based method to update modes of clusters • A mixture of categorical and numerical data: k-prototype method Data mining: Clustering

  38. K-medoids clustering method • Input to the algorithm: the number of clusters k, and a database of n objects • Algorithm consists of four steps: • arbitrarily choose k objects as the initial medoids (representative objects) • assign each remaining object to the cluster with the nearest medoid • select a nonmedoid and replace one of the medoids with it if this improves the clustering • go back to Step 2, stop when there are no more new assignments Data mining: Clustering

  39. Hierarchical methods • A hierarchical method: construct a hierarchy of clustering, not just a single partition of objects • The number of clusters k is not required as an input • Use a distance matrix as clustering criteria • A termination conditioncan be used (e.g., a number of clusters) Data mining: Clustering

  40. A tree of clusterings • The hierarchy of clustering is ofter given as a clustering tree, also called a dendrogram • leaves of the tree represent the individual objects • internal nodes of the tree represent the clusters Data mining: Clustering

  41. Two types of hierarchical methods (1) Two main types of hierarchical clustering techniques: • agglomerative (bottom-up): • place each object in its own cluster (a singleton) • merge in each step the two most similar clusters until there is only one cluster left or the termination condition is satisfied • divisive (top-down): • start with one big cluster containing all the objects • divide the most distinctive cluster into smaller clusters and proceed until there are n clusters or the termination condition is satisfied Data mining: Clustering

  42. Step 0 Step 1 Step 2 Step 3 Step 4 agglomerative a a b b a b c d e c c d e d d e e divisive Step 3 Step 2 Step 1 Step 0 Step 4 Two types of hierarchical methods (2) Data mining: Clustering

  43. Inter-cluster distances • Three widely used ways of defining the inter-cluster distance, i.e., the distance between two separate clusters, are • single linkagemethod (nearest neighbor): • complete linkagemethod (furthest neighbor): • average linkagemethod (unweighted pair-group average): Data mining: Clustering

  44. Strengths of hierarchical methods • Conceptually simple • Theoretical properties are well understood • When clusters are merged/split, the decision is permanent => the number of different alternatives that need to beexamined is reduced Data mining: Clustering

  45. Weaknesses of hierarchical methods • Merging/splitting of clusters is permanent => erroneous decisions are impossible to correct later • Divisive methods can be computational hard • Methods are not (necessarily) scalable for large data sets Data mining: Clustering

  46. Outlier analysis (1) • Outliers • are objects that are considerably dissimilar from the remainder of the data • can be caused by a measurement or execution error, or • are the result of inherent data variability • Many data mining algorithms try • to minimizethe influence of outliers • to eliminatethe outliers Data mining: Clustering

  47. Outlier analysis (2) • Minimizing the effect of outliers and/or eliminating the outliers maycause information loss • Outliers themselves may be of interest => outlier mining • Applications of outlier mining • Fraud detection • Customized marketing • Medical treatments Data mining: Clustering

  48. Summary (1) • Cluster analysis groups objects based on their similarity • Cluster analysis has wide applications • Measure of similarity can be computed for various type of data • Selection of similarity measure is dependent on the data used and the type of similarity we are searching for Data mining: Clustering

  49. Summary (2) • Clustering algorithms can be categorized into • partitioning methods, • hierarchical methods, • density-based methods, • grid-based methods, and • model-based methods • There are still lots of research issues on cluster analysis Data mining: Clustering

  50. Seminar Presentations/Groups 7-8 Classification of spatial data K. Koperski, J. Han, N. Stefanovic: “An Efficient Two-Step Method of Classification of Spatial Data", SDH’98 Data mining: Clustering

More Related