1 / 38

Pattern Recognition: Statistical and Neural

Nanjing University of Science & Technology. Pattern Recognition: Statistical and Neural. Lonnie C. Ludeman Lecture 29 Nov 11, 2005. Lecture 29 Topics. 1. Review Fuzzy Sets and Fuzzy Partitions 2. Fuzzy C- Means Clustering Algorithm Preliminaries

lee-hicks
Download Presentation

Pattern Recognition: Statistical and Neural

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Nanjing University of Science & Technology Pattern Recognition:Statistical and Neural Lonnie C. Ludeman Lecture 29 Nov 11, 2005

  2. Lecture 29 Topics 1. Review Fuzzy Sets and Fuzzy Partitions 2. Fuzzy C- Means Clustering Algorithm Preliminaries 3. Fuzzy C-Means Clustering Algorithm Details 4. Fuzzy C-Means Clustering Algorithm Example 5. Comments about Fuzzy Clustering

  3. Review Example of a Fuzzy Set: Define a Fuzzy set A by the following membership function Example Or equivalently

  4. Review S Function defined on S

  5. Membership Function for F1 Review Domain Pattern Vectors EXAMPLE Membership Function for F2 Domain Pattern Vectors Membership Functions for Fuzzy Clusters

  6. Review A Fuzzy Partition F, of a set S, is defined by its membership functions for the fuzzy sets Fk : k =1, 2, ... , K ) ] Fuzzy Partition

  7. where Review Each value bounded by 0 and 1 Sum of each columns values =1 Sum of each row is less than n

  8. Conversion of Fuzzy Clustering into Crisp Clustering 1 1 1 2 2 2 ) ] C C C Fuzzy Clusters Crisp Clustering All membership values are equal to 1 or 0 Only one 1 in each column where membership value for a Fuzzy cluster is a maximum

  9. Example: Given Fuzzy clusters below convert to Crisp Clusters Fuzzy Clusters Cl1 : [ 0.6 0.7 0.3 0.1 0.4 0.2 0.1 ] Cl2 : [ 0.1 0.1 0.3 0.5 0.1 0.7 0.1 ] Cl3 : [ 0.3 0.2 0.4 0.4 0.5 0.1 0.8 ] Crisp Clusters from Fuzzy Clusters Cl1 : [ 1 1 0 0 0 0 0 ] Cl2 : [ 0 0 0 1 0 1 0 ] Cl3 : [ 0 0 1 0 1 0 1 ]

  10. Cl1 : { x1x2} Cl2 : { x4x6 } Cl3 : { x3x5x7} Cl1 : { x1 , x2} Cl2 : { x4 , x6 } Cl3 : { x3 , x5 , x7} Cl1 : [ 1 1 0 0 0 0 0 ] Cl2 : [ 0 0 0 1 0 1 0 ] Cl3 : [ 0 0 1 0 1 0 1 ] Answer: Crisp Clusters

  11. 1 1 1 2 2 2 ) ] C C C Fuzzy C-Means Clustering Preliminary Given a set S composed of pattern vectors which we wish to cluster S = { x1, x2, ... , xN} Define C Cluster Membership Functions ... ...

  12. Define C Cluster Centroids as follows Let Vibe the Cluster Centroid for Fuzzy Cluster Cli , i = 1, 2, …, C Define a Performance Objective Jmas where

  13. Definitions Ais a symmetric positive definite matrix Nsis total number of pattern vectors m = Fuzziness Index (m >1 ) Higher numbers being more fuzzy The Fuzzy C-Means Algorithm minimizes Jmby selecting Viand i,i=1, 2, … , C by an alternating iterative procedure as described in the algorithm’s details

  14. Fuzzy C-Means Clustering Algorithm (a) Flow Diagram Yes No

  15. Fuzzy C-Means Clustering Algorithm (b) Details of Steps in the Algorithm

  16. Step 1: Initialization Select initial membership functions such that This is equivalent to specifying fuzzy clusters F1, F2, … , FC

  17. One Method to accomplish this selection is to chose rkirandomly from the open interval (0, 1) and then Normalize

  18. Step 2: Computation of Fuzzy Centroids Compute the Fuzzy Centroids as

  19. Step 3: Compute New Fuzzy Membership Functions Using theVi, i = 1, 2, … , C from step 2 compute i(k)

  20. Alternative method for computing the Membership Functions where

  21. Step 4: Check for Convergence If membership functions do not change convergence has occurred If algorithm converges then thei represent the fuzzy clusters and we Stop If Convergence has not occurred and the number of iterationsis less than some preassigned maximum value (MAXIT) then return to step 2 If otherwise then Stop with no solution.

  22. Properties of Fuzzy C-Means Algorithm Usually convergences Answer not unique as depends upon initial conditions. Can converge toa local minimum Can be used toproduce hard clusters

  23. Example – Application of Fuzzy Clustering Algorithm Given the following set of data vectors • Perform a Fuzzy Clustering of the data using Fuzzy C-Means Algorithm to obtain two fuzzy clusters. Try several initial conditions. Is the result unique? (use MAXIT=1000) • Using the results of (a) give a crisp clusteringof the data. • Repeat (a) and (b) for three Fuzzy Clusters.

  24. Plot of Data for Fuzzy clustering example

  25. (a) Solution for two clusters Fuzzy cluster membership functions randomly selected Calculation of Fuzzy Centroids

  26. Calculation of New Membership Functions Calculation of Performance Not the same as preceding iteration Membership function (No convergence) Number of iterations not greater than 1000 therefore the iterations continue.

  27. Results Converge at Iteration 17 Cluster membership Functions Performance Measure Cluster Centroids

  28. (a) Final Cluster membership Functions Cl1 : Cl2 :

  29. (b) Solution Crisp Clustering Fuzzy Membership functions Cl1 : Cl2 : Crisp Membership functions Set Assignment Cl1 = { x1, x2, x3 } Cl2 = { x4 , x5 }

  30. (c) Solution for three fuzzy clusters Applying the Fuzzy Clustering Algorithm convergence was obtained in ?? iterations as Jm = 0.9337 Final Cluster membership functions Cl1 : Cl2 : Cl3 :

  31. (c) Solution Crisp Clustering Membership functions F1 : [ 1, 1, 0, 0, 0 ] F2 : [ 0, 0, 0, 1, 1 ] F3 : [ 0, 0, 1, 0, 0 ] Set Assignment Cl1 = { x1 , x2 } Cl2 = { x4 , x5 } Cl3 = { x3 } “Crisp Clusters”

  32. Comments: Fuzzy Clustering Can be used to Produce Hard Clustering The larger the value of m the fuzzier the clusters The Fuzzy algorithm is relatively stable and usually converges in a reasonable number of iterations The Fuzzy algorithm is relatively insensitve to initial conditions

  33. Of the two different fuzzy clusterings given below, which clustering is the Fuzzier ??? Cl1 : [ 0.52 0.51 0.04 0.47 0.46 ] Cl2 : [ 0.48 0.49 0.96 0.5 0.53 ] # 1 or Cl1 : [ 0.89 0.85 0.04 0.26 0.15 ] Cl2 : [ 0.11 0.15 0.96 0.74 0.85 ] # 2

  34. ANSWER:#1 is the fuzzier of the two different clusterings Cl1 : [ 0.52 0.51 0.04 0.47 0.46 ] Cl2 : [ 0.48 0.49 0.96 0.53 0.53 ] # 1 Cl1 : [ 0.89 0.85 0.04 0.26 0.15 ] Cl2 : [ 0.11 0.15 0.96 0.74 0.85 ] # 2

  35. Why is #1 the Fuzzier of the two ???

  36. Why is #1 the Fuzzier of the two ??? ANSWER: Because the cluster membership functions contain many entries close to 0.5 ( for the two class case) as opposed to values close to 0 and 1. * For the M class case values close to 1/M would indicate most fuzziness.. *

  37. Lecture 29 Topics 1. Reviewed Fuzzy Sets and Fuzzy Partitions 2. Presented Fuzzy C- Means Clustering Algorithm Preliminaries 3. Gave Fuzzy C-Means Clustering Algorithm Details 4. Showed an Example of the Fuzzy C-Means Clustering Algorithm 5. Made a few comments about Fuzzy Clustering in General.

  38. End of Lecture 29

More Related