1 / 12

Robust Fisher Discriminant Analysis

Robust Fisher Discriminant Analysis. Article presented at NIPS 2005 By Seung-Jean Kim, Alessandro Magnani, Stephen P. Boyd Presenter: Erick Delage February 14, 2006. Outline. Background on Fisher Linear Discriminant Analysis

shina
Download Presentation

Robust Fisher Discriminant Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Robust Fisher Discriminant Analysis Article presented at NIPS 2005 By Seung-Jean Kim, Alessandro Magnani, Stephen P. Boyd Presenter: Erick Delage February 14, 2006

  2. Outline • Background on Fisher Linear Discriminant Analysis • Making the approach robust to small sample sets while maintaining computation efficiency • Experimental results

  3. Fisher Discriminant Analysis Given two Random Variables X,Yn, find the linear discriminant  n that maximizes Fisher’s discriminant ratio: • Unique solution : • Easy to compute • Probabilistic interpretation • Kernelizable • Naturally extends to k-class problems

  4. The Fisher discriminant is the Bayes optimal classifier for two normal distributions with equal covariance. Fisher discriminant analysis can be shown to: Probabilistic Interpretation

  5. Using Kernels When discriminating in feature space (x). We can use kernels: And show that  is of the form: A projection along  given by: And find  by solving:

  6. Robust Fisher Discriminant Analysis • Uncertainty in (x, x) & (y, y) • FDA is sensitive to estimation errors of these parameters. • Can we make it more robust using general convex uncertainty models on the problem data? • Is it still a computationally feasible technique?

  7. Assuming ,where U is a convex compact subset. We can try optimizing: From basic min-max theory, we know (1)  (2) Max Worst-case Fisher Discriminant ratio

  8. Max Worst-case Fisher Discriminant ratio

  9. Because , (1) is equivalent to (2) which is convex and can be solved efficiently using a tractable general methods (e.g. interior point methods). Max Worst-case Fisher Discriminant ratio Given ,

  10. Experimental Results Two benchmark problems from the machine learning repository • Sonar: 208 points, n = 60 • Ionosphere: 351 points, n = 34 Uncertainty models:

  11. Experimental Results

  12. References • S.-J. Kim, A. Magnani, and S. P. Boyd: Robust Fisher Discrminant Analysis. In T. Leen, T. Dietterich and V. Tresp, editors, Advances in Neural Information Processing Systems, 18, pp 659-666 ,MIT Press, 2006. • S. Mika, G. Rätsch, and K.-R. Müller: A Mathematical Programming Approach to the Kernel Fisher Algorithm. In T. Leen, T. Dietterich and V. Tresp, editors, Advances in Neural Information Processing Systems, pp 591-597,MIT Press, 2000.

More Related