1 / 24

Naïve Bayesian on CDF Pair Scores

Naïve Bayesian on CDF Pair Scores. 29 August 2013 Venkat. Outline. Naïve Bayesian Overview Adapting Naïve Bayesian to CDF Pairscores Comparisons with Logistic Regression Comparisons with the Voting Scheme. Bayesian Framework. Bayesian Framework.

rusti
Download Presentation

Naïve Bayesian on CDF Pair Scores

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Naïve BayesianonCDF Pair Scores 29 August 2013 Venkat

  2. Outline • Naïve Bayesian Overview • Adapting Naïve Bayesian to CDF Pairscores • Comparisons with Logistic Regression • Comparisons with the Voting Scheme

  3. Bayesian Framework

  4. Bayesian Framework • Unbiased Learning requires O(NKd) samples for reasonable parameter estimation • Impractical for most values of d

  5. Naïve Bayes Assumption • Let • The Naïve Bayes Assumption implies class conditional independence • Requires O(NK) samples

  6. Gaussian Naïve Bayes • What are the parameters to be estimated ? • N Priors • Nd Likelihood functions

  7. Naïve Bayes on CDF Pairscores • Direct application of GNB on CDF Pairscores guaranteed to give poor results. • Must make use of which features are irrelevant conditioned on a class. • For instance, conditioned on class 7, the score for the say class-36-vs-class-9 model is irrelevant.

  8. Naïve Bayes on CDF Pairscores • We have N=50 classes and d = 1225 pairscores

  9. Naïve Bayes on CDF Pairscores

  10. Naïve Bayes on CDF Pairscores

  11. Naïve Bayes on CDF Pairscores

  12. Naïve Bayes on CDF Pairscores

  13. Naïve Bayes on CDF Pairscores

  14. Naïve Bayes on CDF Pairscores

  15. Likelihood Distributions

  16. Likelihood Distributions P(s(c,c’)|y=c) P(s(c’,c)|y=c)

  17. Results (2nd level) • Naïve Bayesian : 57.18% • Voting : 59.01% • Logistic Regression : 57.51% • So, which is the overall best scheme ??

  18. Naïve Bayesvs Logistic Regression • GNB (generative) and LR (discriminative) essentially model the same classifier when Naïve Bayesian Assumptions hold. • However, LR converges to asymptotic accuracies slower than GNB • This is due to LR requiring exponentially higher number of samples compared to GNB for good parameter estimates

  19. Naïve Bayesvs Logistic Regression

  20. Naïve Bayesvs Logistic Regression LOGISTIC REGRESSION

  21. Naïve Bayesvs Logistic Regression NAÏVE BAYESIAN

  22. Naïve Bayesvs Logistic Regression NAÏVE BAYESIAN

  23. Naïve Bayesvs Logistic Regression • When training data is scarce, GNB theoretically outperforms LR • Moreover, if LR only marginally outperforms GNB, then GNB should still be chosen due to its low variance property.

  24. Naïve Bayesvs Voting Scheme • Naïve Bayes is equivalent to a weighted voting scheme. • Unweighted voting scheme takes unbiased votes from pairwise models, ignoring scores and scales. • The binary structure of the unweighted scheme has ill-defined bias-variance properties. • One can argue that it just happens to work well in this case.

More Related