Generative Models for Image Analysis

1. Generative Models for Image Analysis Stuart Geman (with E. Borenstein, L.-B. Chang, W. Zhang)

2. Bayesian (generative) image models • Feature distributions and data distributions • Conditional modeling • Sampling and the choice of null distribution • Other applications of conditional modeling

3. I. Bayesian (generative) image models Prior Conditionallikelihood Posterior focus here on

4. II. Feature distributions and data distributions image patch Model patch through a feature model:

5. e.g. detection and recognition of eyes image patch actually:

6. Use maximum likelihood…but what is the likelihood? ? The first is fine for estimating λ but not fine for estimating T

7. III. Conditional modeling

8. Conditional modeling: a perturbation of the null distribution

9. Estimation Much Easier!

10. Example: learning eye templates image patch

11. Example: learning eye templates

12. Example: learning eye templates

13. Example: learning eye templates Maximize the data likelihood for the mixing probabilities, the feature parameters, and the templates themselves…

14. Example: learning (right) eye templates

15. Example: learning (right) eye templates

16. How good are the templates? A classification experiment…

17. How good are the templates? A classification experiment… Classify East Asian and South Asian * mixing over 4 scales, and 8 templates East Asian: (L) examples of training images (M) progression of EM (R) trained templates South Asian: (L) examples of training images (M) progression of EM (R) trained templates Classification Rate: 97%

18. Other examples: noses 16 templates multiple scales, shifts, and rotations samples from training set learned templates

19. Other examples: mixture of noses and mouths samples from training set (1/2 noses, 1/2 mouths) 32 learned templates

20. Other examples: train on 58 faces …half with glasses…half without samples from training set 32 learned templates 8 learned templates

21. Other examples: train on 58 faces …half with glasses…half without 8 learned templates random eight of the 58 faces row 2 to 4, top to bottom: templates ordered by posterior likelihood

22. Other examples: train random patches (“sparse representation”) 500 random 15x15 training patches from random internet images 24 10x10 templates

23. Other examples: coarse representation sample from training set (down-converted images) training of 8 low-res (10x10) templates

24. IV. Sampling and the choice of null distribution

25. (approximate) sampling…

26. (approximate) sampling…

27. (approximate) sampling…

28. (approximate) sampling…

29. (approximate) sampling…

30. V. Other applications of conditional modeling

32. Markov property… Markov model Estimation Computation Representation

33. Markov model

34. Hierarchical models and the Markov dilemma license plates license numbers (3 digits + 3 letters, 4 digits + 2 letters) plate boundaries, strings (2 letters, 3 digits, 3 letters, 4 digits) generic letter, generic number, L-junctions of sides characters, plate sides parts of characters, parts of plate sides

35. Hierarchical models and the Markov dilemma Original image Zoomed license region Top object: Markov distribution Top object: perturbed (“content-sensitive”) distribution

36. PATTERN SYNTHESIS = PATTERN ANALYSIS Ulf Grenander