1 / 34

Information Geometry of Self-organizing maximum likelihood

Bernoulli 2000 Conference at Riken on 27 October, 2000. Information Geometry of Self-organizing maximum likelihood. Shinto Eguchi ISM, GUAS. This talk is based on joint research with Dr Yutaka Kano, Osaka Univ. Consider a statistical model:.

duaa
Download Presentation

Information Geometry of Self-organizing maximum likelihood

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Bernoulli 2000 Conference at Riken on 27 October, 2000 Information Geometry of Self-organizing maximum likelihood Shinto Eguchi ISM, GUAS This talk is based on joint research with Dr Yutaka Kano, Osaka Univ

  2. Consider a statistical model: Maximum Likelihood Estimation (MLE) ( Fisher, 1922), Consistency, efficiency sufficiency, unbiasedness invariance, information Take an increasing function . -MLE

  3. Normal density -MLE given data -MLE MLE

  4. 0.4 0.3 0.2 0.1 3 -3 -2 -1 1 2 Normal density MLE outlier -MLE

  5. Examples KL-divergence (1) (2) -divergence -divergence (3)

  6. g h f Pythagorian theorem (0,1) (1,1) . ( t, s ) (0,0) (1,0)

  7. (Pf)

  8. Differential geometry of Riemann metric Affine connection Conjugate affine connection Ciszsar’s divergence

  9. -divergence Amari’s -divergence

  10. -likelihood function Kullback-Leibler and maximum likelihood M-estimation ( Huber, 1964, 1983)

  11. Another definition of Y-likelihood Take a positive function k(x, q) and define Y-likelihood equation is a weighted score with integrabity.

  12. Consistency of Y-MLE

  13. Fisher consistency e -contamination model of Influence function Asymptotic efficiency Robustness or Efficiency

  14. Generalized linear model Regression model Estimating equation

  15. Bernoulli regression Logistic regression

  16. Misclassification model MLE MLE

  17. Logistic Discrimination Group I = from Group II from Mislabel 5 Group I Group II 35 Group I Group II

  18. Misclassification 5 data Group II Group I 35 data

  19. Poisson regression -likelihood function -contamination model Canonical link

  20. Neural network

  21. Input Output

  22. Maximum likelihood -maximum likelihood

  23. Classical procedure for PCA Let off-line data. Self-organizing procedure

  24. Classic procedure Self-organizing procedure

  25. Independent Component Analysis (Minami & Eguchi, 2000) F F

  26. Theorem (Semiparametric consistency) S F S (Pf)

  27. -likelihood satisfies the semiparametric consistency

  28. Usual method self-organizing method Blue dots Blue & red dots

  29. 150 the exponential power http://www.ai.mit.edu/people/fisher/ica_data/ 50

  30. Concluding remark Bias potential function Y-sufficiency Y-factoriziable Y-exponential family Y-EM algorithm Y-Regression analysis Y-Discriminant analysis Y-PCA Y-ICA ? !

More Related