GATree Genetically Evolved Decision Trees

1. GATree Genetically Evolved Decision Trees Papagelis Athanasios - Kalles DimitriosComputer Technology Institute

2. Introduction • We use GA’s to evolve simple and accurate binary decision trees • Simple genetic operators over tree structures • Experiments with UCI datasets • very good size • competitive accuracy results

3. Why it should work ? • GA’s are not • Hill climbers • Blind on complex search spaces • Exhaustive searchers • Extremely expensive • They are … • Beam searchers • They balance between time needed and space searched

4. The question… • Are there datasets where hill-climbing techniques are really inadequate ? • e.g unnecessary big – misguiding output • Yes there are… • Conditionally dependent attributes • e.g XOR • Irrelevant attributes • Many solutions that use GAs as a preprocessor so as to select adequate attributes • Direct genetic search can be proven more efficient for those datasets

5. The proposed solution • Select the desired decision tree characteristics (e.g small size) • Create an appropriate fitness function • Adopt a decision tree representation with appropriate genetic operators • Evolve for as long as you wish!

6. Genetic operators

7. Payoff function • Balance between accuracy and size • set x depending on the desired output characteristics. • Small Trees ?  x near one • Emphasis on accuracy ?  x grows big

8. Results

9. Future work • Minimize evolution time • Improved node statistics • Choose the output class using a majority vote over the produced tree forest • Dynamic tuning of initial parameters • Experiments with synthetic datasets • Specific characteristics