1 / 196

Hisao Ishibuchi Osaka Prefecture University, Japan

Multiobjective Genetic Fuzzy Systems - Accurate and Interpretable Fuzzy Rule-Based Classifier Design -. Hisao Ishibuchi Osaka Prefecture University, Japan. Contents of This Presentation Accurate and Interpretable Fuzzy Rule-Based Classifier Design.

kovit
Download Presentation

Hisao Ishibuchi Osaka Prefecture University, Japan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multiobjective Genetic Fuzzy Systems- Accurate and Interpretable Fuzzy Rule-Based Classifier Design - Hisao Ishibuchi Osaka Prefecture University, Japan

  2. Contents of This PresentationAccurate and Interpretable Fuzzy Rule-Based Classifier Design 1. Introduction to Fuzzy Rule-Based Classification - Is Fuzzy Rule-Based Classification a Popular Research Area? 2. Fuzzy Rule-Based Classifier Design - Accuracy Improvement - Scalability to High-Dimensional Problems - Complexity Minimization 3. Multiobjective Fuzzy Rule-Based Classifier Design - Formulation of Multi-objective Problems - Accuracy-Complexity Tradeoff Analysis - Maximization of Generalization Ability 4. Current Hot Issues and Future Research Directions - Search Ability of EMO for Fuzzy System Design - Definition of Interpretability of Fuzzy Systems - Explanation Ability of Fuzzy Rule-Based Systems - Various Classification Problems: Imbalanced, Online, ...

  3. Application Areas of Fuzzy Systems Application Areas of Fuzzy Systems - Fuzzy Control - Fuzzy Clustering - Fuzzy Classification Control and clustering are well-known application areas ! Question: Is “fuzzy classification” popular ? Web of Science

  4. Application Areas of Fuzzy Systems Application Areas of Fuzzy Systems - Fuzzy Control - Fuzzy Clustering - Fuzzy Classification Control and clustering are well-known application areas ! Question: Is “fuzzy classification” popular ? Control, Clustering, Classification Web of Science

  5. Application Areas of Fuzzy SystemsFuzzy Control: Well-Known Application Area 9421 Fuzzy Control Papers Web of Science

  6. Application Areas of Fuzzy SystemsFuzzy Control: Well-Known Application Area Takagi-Sugeno Model (1985) 3801 ANFIS (1993) 1983 CC Lee: Fuzzy Logic Controller (1990) 1896 Web of Science

  7. Application Areas of Fuzzy SystemsFuzzy Clustering: Well-Known Application Area 2968 Fuzzy Clustering Papers Web of Science

  8. Application Areas of Fuzzy SystemsFuzzy Clustering: Well-Known Application Area Pal NR, Pal SK 819 Bezdek JC Pal NR, Bezdek JC Web of Science

  9. Application Areas of Fuzzy SystemsFuzzy Classification: Well-Known? 4144 Fuzzy Classification Papers Web of Science

  10. Application Areas of Fuzzy SystemsFuzzy Classification: Well-Known? 389 Dubois D, Prade H Dubois D, Prade H Fuzzy Min-Max NN Fuzzy If-Then Rules Web of Science

  11. Application Areas of Fuzzy SystemsControl, Clustering, and Classification 9421 (Fuzzy Control) 2968 (Fuzzy Clustering) 4144 (Fuzzy Classification) Web of Science

  12. Application Areas of Fuzzy SystemsControl, Clustering, and Classification IEEE Trans FS 1993 FUZZ-IEEE 1992 9421 (Fuzzy Control) IEEE Trans FS 1993 FUZZ-IEEE 1992 2968 (Fuzzy Clustering) IEEE Trans FS 1993 FUZZ-IEEE 1992 4144 (Fuzzy Classification) Web of Science

  13. Contents of This Presentation Accurate and Interpretable Fuzzy Rule-Based Classifier Design 1. Introduction to Fuzzy Rule-Based Classification - Is Fuzzy Rule-Based Classification a Popular Research Area? 2. Fuzzy Rule-Based Classifier Design - Accuracy Maximization - Scalability to High-Dimensional Problems - Complexity Minimization 3. Multiobjective Fuzzy Rule-Based Classifier Design - Formulation of Multi-objective Problems - Accuracy-Complexity Tradeoff Analysis - Maximization of Generalization Ability 4. Current Hot Issues and Future Research Directions - Search Ability of EMO for Fuzzy System Design - Definition of Interpretability of Fuzzy Systems - Explanation Ability of Fuzzy Rule-Based Systems - Various Classification Problems: Imbalanced, Online, ...

  14. This Presentation: Accurate and Interpretable Fuzzy Rule-Based Classifier Design Web of Science

  15. This Presentation: Accurate and Interpretable Fuzzy Rule-Based Classifier Design Fitness Function w1 Accuracy(S) -w2 Complexity(S) Accuracy Maximization and Complexity Minimization

  16. This Presentation: Accurate and Interpretable Fuzzy Rule-Based Classifier Design The number of selected fuzzy rules w1 Accuracy(S) -w2 Complexity(S) The number of correctly classified training patterns

  17. Fuzzy Rules for Classification Accurate and Interpretable Fuzzy Rule-Based Classifier Design Class 1 Class 2 Class 3 Basic Form If x1 is smalland x2 is small then Class 2 1.0 x2 0.0 large small medium x1 large medium small 0.0 1.0

  18. Fuzzy Rules for Classification Accurate and Interpretable Fuzzy Rule-Based Classifier Design Class 1 Class 2 Class 3 Basic Form If x1 is smalland x2 is small then Class 2 If x1 is smalland x2 is medium then Class 2 1.0 large x2 medium small 0.0 large small medium x1 0.0 1.0

  19. Fuzzy Rules for Classification Accurate and Interpretable Fuzzy Rule-Based Classifier Design Class 1 Class 2 Class 3 Basic Form If x1 is smalland x2 is small then Class 2 If x1 is smalland x2 is medium then Class 2 If x1 is smalland x2 is large then Class 1 1.0 large x2 medium small 0.0 large small medium x1 0.0 1.0

  20. Fuzzy Rules for Classification Accurate and Interpretable Fuzzy Rule-Based Classifier Design Class 1 Class 2 Class 3 1.0 Basic Form If x1 is smalland x2 is small then Class 2 If x1 is smalland x2 is medium then Class 2 If x1 is smalland x2 is large then Class 1 . . . If x1 is largeand x2 is large then Class 3 High Interpretability Easy to Understand ! large small medium x2 0.0 large medium small x1 0.0 1.0

  21. Classification Boundary Accurate and Interpretable Fuzzy Rule-Based Classifier Design Class 1 Class 2 Class 3 1.0 Basic Form If x1 is smalland x2 is small then Class 2 If x1 is smalland x2 is medium then Class 2 If x1 is smalland x2 is large then Class 1 . . . If x1 is largeand x2 is large then Class 3 High Interpretability Easy to Understand ! large small medium x2 0.0 large medium small x1 0.0 1.0

  22. Fuzzy Rules for ClassificationBasic form does not always have high accuracy Class 1 Class 2 Class 3 1.0 Basic Form If x1 is smalland x2 is small then Class 2 If x1 is smalland x2 is medium then Class 2 If x1 is smalland x2 is large then Class 1 . . . If x1 is largeand x2 is large then Class 3 High Interpretability Low Accuracy large small medium x2 0.0 large medium small x1 0.0 1.0

  23. Fuzzy Rules for ClassificationAnother form has a rule weight (certainty) Class 1 Class 2 Class 3 Basic Form If x1 is small and x2 is medium then Class 2 Rule Weight Version If x1 is smalland x2 is medium then Class 2 with 0.158 1.0 large x2 medium small 0.0 large small medium x1 0.0 1.0

  24. Classification BoundaryAccurate and Interpretable Fuzzy Rule-Based Classifier Design Class 1 Class 2 Class 3 Basic Form If x1 is small and x2 is medium then Class 2 Rule Weight Version If x1 is smalland x2 is medium then Class 2 with 0.158 1.0 large x2 medium small 0.0 large small medium x1 0.0 1.0

  25. Fuzzy Rules with Rule Weights Accurate and Interpretable Fuzzy Rule-Based Classifier Design Class 1 Class 2 Class 3 Basic Form If x1 is small and x2 is medium then Class 2 Rule Weight Version If x1 is smalland x2 is medium then Class 2 with 0.158 1.0 large x2 medium small 0.0 large small medium x1 0.0 1.0 H. Ishibuchi et al., Fuzzy Sets and Systems (1992) Web of Science

  26. Fuzzy Rules in This PresentationFuzzy Rules with Rule Weights Class 1 Class 2 Class 3 Basic Form If x1 is small and x2 is medium then Class 2 Rule Weight Version If x1 is smalland x2 is medium then Class 2 with 0.158 1.0 large x2 medium Use of Rule Weights: Controversial Issue (1) Rule weight adjustment can be replaced with membership learning. small 0.0 large small medium x1 0.0 1.0 Google Scholar D. Nauck and R. Kruse, Proc. of FUZZ-IEEE 1998 (1998).

  27. Fuzzy Rules in This PresentationFuzzy Rules with Rule Weights Class 1 Class 2 Class 3 Basic Form If x1 is small and x2 is medium then Class 2 Rule Weight Version If x1 is smalland x2 is medium then Class 2 with 0.158 1.0 large x2 medium Use of Rule Weights: Controversial Issue (2) Membership learning can be partially replaced with weight adjustment. small 0.0 large small medium x1 0.0 1.0 Google Scholar H. Ishibuchi and T. Nakashima, IEEE Trans. FS (2001)

  28. Fuzzy Rules for ClassificationAnother Form: Multiple Consequents Class 1 Class 2 Class 3 Basic Form If x1 is small and x2 is medium then Class 2 Rule Weight Version If x1 is small and x2 is medium then Class 2 with 0.158 Multiple Consequents If x1 is small and x2 is medium then Class 1 with 0.579, Class 2 with 0.421, Class 3 with 0.000 1.0 large x2 medium small 0.0 large small medium x1 0.0 1.0

  29. Fuzzy Rules for ClassificationAnother Form: Multiple Consequents Class 1 Class 2 Class 3 Basic Form If x1 is small and x2 is medium then Class 2 Rule Weight Version If x1 is small and x2 is medium then Class 2 with 0.158 Multiple Consequents If x1 is small and x2 is medium then Class 1 with 0.579, Class 2 with 0.421, Class 3 with 0.000 1.0 large x2 medium small 0.0 large small medium x1 0.0 1.0 O. Cordon et al., IJAR (2001) Google Scholar

  30. Other Forms of Fuzzy RulesHandling of Classification as Function Approximation Class 1 Class 2 Class 3 Integer Consequent If x1 is small and x1 is large then y = 1 If x1 is large and x2 is large then y = 3 1.0 large x2 medium Binary Consequent If x1 is small and x1 is large then (y1, y2, y3) = (1, 0, 0) If x1 is large and x2 is large then (y1, y2, y3) = (0, 0, 1) small 0.0 large small medium x1 0.0 1.0

  31. Contents of This Presentation Accurate and Interpretable Fuzzy Rule-Based Classifier Design 1. Introduction to Fuzzy Rule-Based Classification - Is Fuzzy Rule-Based Classification a Popular Research Area? 2. Fuzzy Rule-Based Classifier Design - Accuracy Improvement - Scalability to High-Dimensional Problems - Complexity Minimization 3. Multiobjective Fuzzy Rule-Based Classifier Design - Formulation of Multi-objective Problems - Accuracy-Complexity Tradeoff Analysis - Maximization of Generalization Ability 4. Current Hot Issues and Future Research Directions - Search Ability of EMO for Fuzzy System Design - Definition of Interpretability of Fuzzy Systems - Explanation Ability of Fuzzy Rule-Based Systems - Various Classification Problems: Imbalanced, Online, ...

  32. Accuracy ImprovementUse of Fine Fuzzy Partition Class 1 Class 2 Class 3 1.0 L ML M x2 MS S 0.0 S MS M ML L x1 0.0 1.0

  33. Accuracy ImprovementHow to choose an appropriate partition ? Class 1 Class 2 Class 3 1.0 L ML M x2 MS S 0.0 S MS M ML L x1 0.0 1.0 Too Fine Fuzzy Partition ==> Over-Fitting (Poor Generalization Ability)

  34. Accuracy ImprovementHow to choose an appropriate partition ? Class 1 Class 2 Class 3 1.0 L ML M x2 MS S 0.0 S MS M ML L x1 0.0 1.0 One Idea: Use of All Partitions (Multiple Fuzzy Grid Approach) Web of Science Ishibuchi et al., Fuzzy Sets and Systems (1992)

  35. Accuracy ImprovementLearning of Membership Functions Class 1 Class 2 Class 3 1.0 x2 0.0 x1 0.0 1.0

  36. Accuracy ImprovementLearning of Membership Functions Class 1 Class 2 Class 3 1.0 x2 0.0 x1 0.0 1.0 Various learning methods such as neuro-fuzzy and genetic-fuzzy methods are available. Web of Science D. Nauck and R. Kruse, Fuzzy Sets and Systems (1997).

  37. Accuracy ImprovementLearning of Membership Functions Class 1 Class 2 Class 3 1.0 x2 0.0 x1 0.0 1.0 Various learning methods such as neuro-fuzzy and genetic-fuzzy methods are available. Interpretability is degraded. Web of Science D. Nauck and R. Kruse, Fuzzy Sets and Systems (1997).

  38. Accuracy ImprovementUse of Independent Membership Functions Class 1 Class 2 Class 3 1.0 Each fuzzy rule can be generated and adjusted independently from other rules. ==> High Accuracy Membership functions can be heavily overlapping. ==> Poor Interpretability x2 0.0 x1 0.0 1.0

  39. Accuracy ImprovementUse of Independent Membership Functions Class 1 Class 2 Class 3 1.0 Each fuzzy rule can be generated and adjusted independently from other rules. ==> High Accuracy Membership functions can be heavily overlapping. ==> Poor Interpretability x2 0.0 Web of Science x1 0.0 1.0 S. Abe and M. S. Lan, IEEE Tras. on FS (1995)

  40. Accuracy ImprovementUse of Multi-Dimensional Membership Functions Class 1 Class 2 Class 3 If x is A then Class 2 A: Multi-dimensional Fuzzy Set (Membership Function) A

  41. Accuracy ImprovementUse of Multi-Dimensional Membership Functions Class 1 Class 2 Class 3 If x is A then Class 2 A: Multi-dimensional Fuzzy Set (Membership Function) A Fuzzy rules are flexibility. ==> High Accuracy Each membership function is multi-dimensional. ==> Poor Interpretability

  42. Accuracy ImprovementUse of Multi-Dimensional Membership Functions Class 1 Class 2 Class 3 If x is A then Class 2 A: Multi-dimensional Fuzzy Set (Membership Function) A S. Abe et al., IEEE TSMC-C (1998) Web of Science S. Abe et al., IEEE TSMC-C (1999)

  43. Accuracy ImprovementUse of Tree-Type Fuzzy Partitions Class 1 Class 2 Class 3 1.0 x2 0.0 large small x1 0.0 1.0 If x1 is smallthen Class 2. If x1 is large and x2 is small then Class 3. If x1 is large and x2 is large then Class 1. small large

  44. Accuracy ImprovementUse of Tree-Type Fuzzy Partitions x1 is small x1 is large x2 is large x2 is small Web of Science

  45. Contents of This Presentation Accurate and Interpretable Fuzzy Rule-Based Classifier Design 1. Introduction to Fuzzy Rule-Based Classification - Is Fuzzy Rule-Based Classification a Popular Research Area? 2. Fuzzy Rule-Based Classifier Design - Accuracy Improvement - Scalability to High-Dimensional Problems - Complexity Minimization 3. Multiobjective Fuzzy Rule-Based Classifier Design - Formulation of Multi-objective Problems - Accuracy-Complexity Tradeoff Analysis - Maximization of Generalization Ability 4. Current Hot Issues and Future Research Directions - Search Ability of EMO for Fuzzy System Design - Definition of Interpretability of Fuzzy Systems - Explanation Ability of Fuzzy Rule-Based Systems - Various Classification Problems: Imbalanced, Online, ...

  46. Difficulty of High-Dimensional ProblemsExponential Increase of Fuzzy Rules Class 1 Class 2 Class 3 1.0 Basic Form If x1 is smalland x2 is small then Class 2 If x1 is smalland x2 is medium then Class 2 . . . If x1 is largeand x2 is large then Class 3 Number of Fuzzy Rules: 2-D Problem: 3x3 3-D Problem: 3x3x3 4-D Problem: 3x3x3x3 5-D Problem: 3x3x3x3x3 large small medium x2 0.0 large medium small x1 0.0 1.0

  47. Scalability ImprovementUse of Independent Membership Functions Class 1 Class 2 Class 3 1.0 Fuzzy rules are generated in the multi-dimensional space. => No Exponential Increase x2 0.0 x1 0.0 1.0

  48. Scalability ImprovementUse of Multi-Dimensional Membership Functions Class 1 Class 2 Class 3 If x is A then Class 2 A: Multi-dimensional Fuzzy Set (Membership Function) A Fuzzy rules are generated in the multi-dimensional space. => No Exponential Increase

  49. Scalability ImprovementUse of Tree-Type Fuzzy Partitions Web of Science An appropriate stopping condition prevents the exponential increase in the number of fuzzy rules. x1 is large x1 is small x2 is large x2 is small

  50. Scalability ImprovementHierarchical Fuzzy Systems subsystem subsystem subsystem subsystem subsystem x1 x2 x3 x4 x5

More Related