1 / 14

Minimax Probability Machine (MPM)

Minimax Probability Machine (MPM). Jay Silver. Very High Level Diagram of Training a Pattern Classifier. Augmented. Testing a New Data Point. , choose class w x. If. , choose class w y. If. Finding a Function that Decides. Decision. Assume Binary. Non Parametric. Parametric.

dianabell
Download Presentation

Minimax Probability Machine (MPM)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Minimax Probability Machine (MPM) Jay Silver

  2. Very High Level Diagram of Training a Pattern Classifier Augmented Testinga New Data Point

  3. , choose class wx If , choose class wy If Finding a Function that Decides Decision Assume Binary Non Parametric Parametric Support Vector Machine (SVM) Minimax Probability Machine (MPM) Gaussian

  4. Non-Parametric Linear Decision Boundaries MPM SVM Maximal Margin Classifier Minimize Worst Future Error An SVM and MPM toolbox were used for implementation [1,4]. MPM figure borrowed from [2].

  5. MPM Upper bound of misclassifying future point with Mahalanobis Distance Equal Problem Statement s.t. Lower bound on test accuracy An SVM and MPM toolbox were used for implementation [1,4]. MPM figure borrowed from [2].

  6. Expanding the Feature Space with Kernels Original Feature Space Expanded Feature Space XOR: {x1, x2} XOR: {x1, x2, x1x2} Not Linearly Separable Linearly Separable Kernel Examples Gaussian Kernel: Polynomial Kernel:

  7. Take a Look at Some Linear Decision Boundaries Key

  8. Results for the Distribution We Just Saw SVM Performs Best MPM Performs Well SVM Homogeneous Polynomial Fails to Converge

  9. Alpha as an Underbound to Test Accuracy Compare Alpha to Test Accuracy Just Note Correlation Between Alpha and Test Accuracy Key

  10. Testing on a Real Speech Task Deterding Data – 11 vowel sounds with 10 features Multiple classes – Use 1 vs. 1 voting to generalize binary classifiers Test Accuracy for the Gaussian Kernel MPM Peaks At 67.3% Key SVM Peaks At 68.4%

  11. Summary of Deterding Results Distill Results Further Linear Nonlinear

  12. Conclusions Alpha is an accurate lower bound for all cases but one. Alpha was reasonably well correlated with test accuracy. SVM homogeneous polynomial kernel outperformed MPM But MPM homo. poly. kernel was more consistent MPM Gaussian kernel performed 1% below SVM on Deterding MPM: Competitive, including realistic speech tasks Mathematically pleasing Room to grow Not quite as accurate as SVMs

  13. References

  14. Questions? The Rainbow Linear Discriminant Between CSTIT Students

More Related