Fuzzy Pattern Trees for Regression and Fuzzy Systems Modeling

1. Fuzzy Pattern Trees for Regression and Fuzzy Systems Modeling Robin Senge & Eyke Hüllermeier WCCI 2010, Barcelona

2. Outline • Problem Setting • IntroductiontoFuzzy Pattern Trees (FPT) • Learning Fuzzy Pattern Treesfrom Data • Experiments • Relation toFuzzyRule-based Systems • UsingFuzzy Pattern TreesforFuzzy System Modeling

3. Problem Setting Standard setting of supervised learning: • attribute-value representation of instances • let be input domains and be the output domain • input attribute domains discretized by fuzzy sets, e.g., low, medium and high • rescale to by model functional relationship, i.e.

4. Example: Wine Quality • aim: predicting quality of wine based on its ingredients (UCI) • input attributes: acidity, alcohol, sulfates, sulfur, ... • target (output) attribute is quality

5. Example Fuzzy Pattern Tree (FPT) wine quality 0.5 0.8 0.2 alcohol high MIN MAX • AVG 0.8 0.2 acidity low 0.8 0.3 acidity med sulfates med 10.2

6. Operators

7. Features of Fuzzy Pattern Trees high wine quality • interpretabilityofthe model class • modularity: recursivepartitioningofcritriainto sub-criteria • flexibilitywithoutthetendencytooverfitthedata • monotonicity in singleattributes • built-in featureselection alcohol high • AVG MIN MAX acidity low acidity med sulfates med

8. iteratively refining = growing up trees start with primitive pattern tree growing tree in a top-down manner selection based on tree performance measure check relative performance improvement Learning Fuzzy Pattern Trees from Examples A A B E D A E D B B A C B C MIN • AVG • AVG MIN MIN • AVG MAX MIN MAX • AVG MIN MAX • AVG • AVG • AVG A A B D D greedy beam search (details in thepaper) B C

9. Experiments Are Fuzzy Pattern Treescompetitive in termsofpredictiveaccuracy? • 12 data sets from UCI and STATLIB • 10-fold-cross validation • root mean squared error (RMSE) baseline algorithms • Linear Regression (LR) • Multi Layer Perceptron (MLP) • Support Vector Machine with linear kernel (SMO-lin) • Support Vector Machine with RBF kernel (SMO-rbf) • Fast decision tree learner with reduced error pruning (REPtree) • Fuzzy RuleLearnerby Wang & Mendel (FR)

10. Results Ranks accordingto RMSE PT-reg appearstobe (at least) competitive tobaseline algorithms.

11. Fuzzy Pattern Trees vs. Rule-basedFuzzy Systems • Fuzzy Pattern TreesarecloselyrelatedtoFuzzyRule-based Systems • fuzzy rules for property: low qualityIFhigh(acidity) ANDlow(alcohol) THENquality is lowIFlow(acidity) ANDmedium(sulfates) THENquality is lowIFhigh(alcohol) ANDmedium(sulfur)THENquality is low fuzzy rules for property: low quality Score(quality is low) = MAX { MIN {high(acidity), low(alcohol)},MIN {low(acidity), medium(sulfates)},MIN {high(alcohol), medium(sulfur)}} • fuzzy rules for property: low qualityIF MIN {high(acidity), low(alcohol)}THENquality is low IF MIN {low(acidity), medium(sulfates)} THENquality is low IF MIN {high(alcohol), medium(sulfur)}THENquality is low MIN MIN MIN low quality MAX aciditylow sulfatesmed alcoholhigh sulfurmed acidityhigh alcohollow

12. Fuzzy Systems Modeling • usually, not onlyonefuzzyseton but completefuzzypartition • let be the fuzzy sets on model functional relationships, i.e. high quality medium quality low quality F-AND AVG-OP AVG-OP F-AND F-OR sulfur low sulfate low acid high sulfate med alcohol med acid high acid low alcohol high

13. Fuzzy Systems Modeling contd. low quality high quality medium quality low quality medium quality F-AND AVG-OP AVG-OP F-AND F-OR high quality sulfur low sulfate low acid high sulfate med alcohol med acid high acid low alcohol high

14. Conclusions • Fuzzy Pattern Trees have been introduced as a new model class for regression and fuzzy systems design. • They do have several interesting features (interpretability , monotonicity, flexibility, feature selection). • Data-driven model construction: WecanlearnFuzzy Pattern Trees from data. • Regression withFuzzy Pattern Treesiscompetitive to state-of-the-artalgorithms in termsofpredictiveaccuracy. Formoreinformationsearchthe web for„kebimarburg“.