Presented by Lindsay Stetson

1. Intelligible Models for Classification and Regression Yin Lou, Rich Caruana, Johannes Gerhke2012 ACM Conference on Knowledge Discovery and Data Mining Presented by Lindsay Stetson

2. Outline • Background and Motivation • Generalized Additive Models • Experimental Overview • Results • Conclusion

3. Outline • Background and Motivation • Generalized Additive Models • Experimental Overview • Results • Conclusion

4. Background • Linear Model • Regression: y = β0+ β1x1 + … + βnxn • Classification: y = logit(β0 + β1x1 + … + βnxn) • Easy to interpret, intelligible, but less accurate • Complex Model (SVM, Random Forest, Neural Networks) • y = (x1, …, xn) • More accurate, but usually unintelligble

5. Goals of Work “…construct accurate models that are interpretable.” Intelligibility is important! In applied fields like biology, physics, and medicine we need to understand the individual contributions of the features in the model.

6. Outline • Background and Motivation • Generalized Additive Models • Experimental Overview • Results • Conclusion

7. Generalized Additive Model • Regression: y = f1(x1) + … + fn(xn) • Classification: y = logit(f1(x1) + … + fn(xn)) • Each feature gets shaped by a function fi • Goal: Accurate and intelligble

8. Example

9. Fitting Generalized Additive Models • Splines (SP) • Single Tree (TR) • Bagged Trees (bagTR) • Boosted Trees (bstTR) • Boosted Bagged Trees (bbTR)

10. Learning Methods • Least Squares (P-LS/P-IRLS) • Backfitting (BF) • Gradient Boosting (BST)

11. Outline • Background and Motivation • Generalized Additive Models • Experimental Overview • Results • Conclusion

12. Experimental Design

13. Datasets

14. Outline • Background and Motivation • Generalized Additive Models • Experimental Overview • Results • Conclusion

15. Results

16. Results

17. Bias Variance Analysis

18. Outline • Background and Motivation • Generalized Additive Models • Experimental Overview • Results • Conclusion

19. Conclusion • Generalized additive models are accurate and intelligible • Trees have a low bias but a high variance • Bagging reduces the variance, making the trees methods high performers • Bagged trees, with a low number of leaves, that are gradient boosted are the most accurate

20. ?