1 / 5

Evidence Contrary to the Statistical View of Boosting

Evidence Contrary to the Statistical View of Boosting. David Mease & Abraham Wyner. What is the Statistical View? . The idea presented in J . Friedman, T. Hastie, and R. Tibshirani . Additive logistic regression: A statistical view of boosting. Annals of Statistics, 28:337–374, 2000a

edan
Download Presentation

Evidence Contrary to the Statistical View of Boosting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evidence Contrary to the Statistical View of Boosting David Mease & Abraham Wyner

  2. What is the Statistical View? • The idea presented in • J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: A statistical view of boosting. Annals of Statistics, 28:337–374, 2000a • Ada Boost is similar to LogitBoost Regression

  3. The Challenge • Centers on areas where the paper’s view fails to explain important characteristics of Ada Boost • The statistical view of boosting focuses only on one aspect of the algorithm - the optimization. • Does not explain why the statistical view doesn’t work, merely presents evidence to the contrary

  4. Areas of Deficiency for Statistical View • Stagewisenature of the algorithm • Empirical variance reduction that can be observed on hold out samples • variance reduction seems to happen accidently. • Strong resistance to over fitting of Ada Boost which is lost in regression model

  5. Practical Advice • AdaBoost is one of the most successful boosting algorithms • Do not assume that newer, regularized and modified versions of boosting are necessarily better • Try standard AdaBoost along with these newer algorithms • If classification is the goal, monitor the misclassification error on hold out (or cross-validation) samples • Much of the evidence presented is counter-intuitive • keep an open mind when experimenting with AdaBoost • If stumps are causing overfitting, be willing to try larger trees • Intuition may suggest the larger trees will overfit, but we have seen that is not necessarily true

More Related