a factor model for learning higher-order features in natural images yan karklin ( nyu + hhmi)

1. a factor model for learning higher-order features in natural images yan karklin (nyu + hhmi) joint work with mike lewicki (case western reserve u) icml workshop on learning feature hierarchies - june 18, 2009

2. retina LGN V2 V1

3. non-linear processing in the early visual system contrast/size response function estimated linear receptive field cat LGN (DeAngelis, Ohzawa, Freeman 1995) cat LGN (Carandini 2004)

4. normalized response ÷ 2 ( ) + ... cell (V1) model (Schwartz and Simoncelli, 2001)

5. non-linear statistical dependencies in natural images 0

10. a generative model for covariance matrices multivariate Gaussian model latent variables specify the covariance

11. a distributed code if only we could find a basis for covariance matrices... = y1 + y2 + y3 + y4 + . . .

12. a distributed code if only we could find a basis for covariance matrices... = y1 + y2 + y3 + y4 + . . . use the matrix-logarithm transform = y1 + y2 + y3 + y4 + . . .

13. a compact parameterization for cov-components find common directions of change in variation/correlation bark foliage

14. a compact parameterization for cov-components find common directions of change in variation/correlation bark foliage vectors bk allow a more compact description of the basis functions bk’s are unit norm (wjk can absorb scaling)

15. the full model . . . . . .

16. the full model . . . . . .

17. the full model . . . . . .

18. the full model . . . . . .

19. normalization by the inferred covariance matrix normalized response ÷

20. training the model on natural images . . . . . . • learning details: • gradient ascent on data likelihood • 20x20 image patches from ~100 natural images • 1000 bk’s • 150 yj’s

21. learned vectors . . . . . .

22. learned vectors . . . . . . black – 25 example features grey – all 1000 features in model

23. one unit encodes global image contrast w105,: + . . . . . . 0 -

24. original patches

25. normalization by inferred contrast

26. w105,: w85,: . . . . . . w57,: w11,: 0

27. w12,: w125,: . . . . . . w90,: w144,:

28. w27,: w37,: . . . . . . w78,: w66,:

29. original patches

30. normalization by inferred contrast

31. normalization by inverse covariance

32. Restricted Boltzmann Machines h v

33. gated Restricted Boltzmann Machines (Memisevic and Hinton, CVPR 07) h x y - Gaussian (pixels) - Gaussian (pixels) - binary trained on image sequences image sequence (t  )

34. gated Restricted Boltzmann Machines (Memisevic and Hinton, CVPR 07) h x y - Gaussian (pixels) - Gaussian (pixels) - binary trained on image sequences image sequence (t  ) log-Covariance factor model

35. ÷ • summary • modeling non-linear dependencies motivated by statistical patterns • higher-order models can capture context • normalization by local “whitening” removes most structure • questions • joint model for normalized response and normalization (context) signal? • how to extend to entire image? • where are these computations found in the brain?