Information Geometry of Self-organizing maximum likelihood

1. Bernoulli 2000 Conference at Riken on 27 October, 2000 Information Geometry of Self-organizing maximum likelihood Shinto Eguchi ISM, GUAS This talk is based on joint research with Dr Yutaka Kano, Osaka Univ

2. Consider a statistical model: Maximum Likelihood Estimation (MLE) ( Fisher, 1922), Consistency, efficiency sufficiency, unbiasedness invariance, information Take an increasing function . -MLE

3. Normal density -MLE given data -MLE MLE

4. 0.4 0.3 0.2 0.1 3 -3 -2 -1 1 2 Normal density MLE outlier -MLE

6. Examples KL-divergence (1) (2) -divergence -divergence (3)

8. g h f Pythagorian theorem (0,1) (1,1) . ( t, s ) (0,0) (1,0)

9. (Pf)

10. Differential geometry of Riemann metric Affine connection Conjugate affine connection Ciszsar’s divergence

11. -divergence Amari’s -divergence

12. -likelihood function Kullback-Leibler and maximum likelihood M-estimation ( Huber, 1964, 1983)

13. Another definition of Y-likelihood Take a positive function k(x, q) and define Y-likelihood equation is a weighted score with integrabity.

14. Consistency of Y-MLE

15. Fisher consistency e -contamination model of Influence function Asymptotic efficiency Robustness or Efficiency

16. Generalized linear model Regression model Estimating equation

17. Bernoulli regression Logistic regression

18. Misclassification model MLE MLE

19. Logistic Discrimination Group I = from Group II from Mislabel 5 Group I Group II 35 Group I Group II

20. Misclassification 5 data Group II Group I 35 data

21. Poisson regression -likelihood function -contamination model Canonical link

22. Neural network

23. Input Output

24. Maximum likelihood -maximum likelihood

25. Classical procedure for PCA Let off-line data. Self-organizing procedure

27. Classic procedure Self-organizing procedure

28. Independent Component Analysis (Minami & Eguchi, 2000) F F

29. Theorem (Semiparametric consistency) S F S (Pf)

30. -likelihood satisfies the semiparametric consistency

32. Usual method self-organizing method Blue dots Blue & red dots

33. 150 the exponential power http://www.ai.mit.edu/people/fisher/ica_data/ 50

34. Concluding remark Bias potential function Y-sufficiency Y-factoriziable Y-exponential family Y-EM algorithm Y-Regression analysis Y-Discriminant analysis Y-PCA Y-ICA ? !