Nonlinear models for Natural Images

1. Nonlinear models for Natural Images Urs Köster & Aapo Hyvärinen University of Helsinki 1

2. Overview • Limitations of linear models • A hierarchical model learns Complex Cell pooling • A Horizonal product model for Contrast Gain Control • A Markov Random Field generalizes ICA to large images 2

3. 1. Limitations of ICA image models • Natural images have complex structure, cannot be modeled as superpositions of basis functions • Linear models ignore much of the rich interactions between units • Modeling the dependencies leads to more abstract representations • Variance dependencies are particularly obvious structure not captured by ICA • Model with (complex cell) pooling of filter outputs - hierarchical models • Alternative: Model dependencies by gain control on the pixel level Schwartz & Simoncelli 2001

4. A hierarchical model estimated with Score Matching learns Complex Cell receptive fields

5. 2. Two Layer Model estimated with Score Matching • Define an energy based model of the form • Squaring the outputs of linear filters • Second layer linear transform v • Nonlinearity that leads to a super-gaussian pdf. • Cannot be normalized in closed form. Estimation with Score Matching makes learning possible without need for Monte Carlo methods or approximations 6

6. Results • The second layer learns to pool over units with similar location and orientation, but different spatial phase • Following the energy model of Complex Cells without any assumptions on the pooling • Estimating W and V simultaneously leads to a better optimum and more phase invariance of the higher order units Some pooling patterns

7. Learning to perform Gain Control with a Horizontal Product model

8. Multiplicative interactions A horizontal network model: Horizontal layers • Two parallel streams or layers on one level of the hierarchy • Unrelated aspects of the stimulus are generated separately • Observed data is generated by combining all the sub-models • Data is described by element-wise multiplying outputs of sub-models • Can implement highly nonlinear (discontinuous) functions • Combine aspects of a stimulus generated by separate mechanisms

9. The model • Definition of the model: • Likelihood: • Constraints: B and t are non-negative, W invertible • g(.) is a log-cosh nonlinearity (logistic distribution) • t has a Laplacian sparseness prior 10

10. Results First layer W 4 units in B First layer W 16 units in B

11. Second Layer: Contrast Gain Control • Emergence of a contrast map in the second layer • It performs Contrast Gain Control on the LGN level (rather than on filter outputs) • Similar effect to performing divisive normalization as preprocessing • The model can be written as • Something impossible to do with hierarchical models True image patches Reconstruction from As only Modulation from Bt

12. The “big” picture: A Markov Random Field generalizes ICA to arbitrary size images

13. 4. Markov Random Field • Goal: Define probabilities for whole images rather than small patches • A MRF uses a convolution to analyze large images with small filters • Estimating the optimal filters in an ICA framework is difficult, the model cannot be normalized • Energy based optimization using Score Matching

14. Model estimation • The energy (neg. log pdf) is • We can rewrite the convolution • where xi are all possible patches from the image, wk are the different filters • We can use score matching just like in an overcomplete ICA model • The MRF is equivalent to overcomplete ICA with filters that are smaller than the patch and copied in all possible locations.

15. Results • We can estimate MRF filters of size 12x12 pixels (much larger than previous work, e.g. 5x5) • This is possible from 23x23 pixel ‘images’, but the filters generalize to images of arbitrary size • This is possible because all possible overlaps are accounted for in the (2 n -1) size image • Filters similar to ICA, but less localized (since they need to explain more of the surrounding patch) • Possible applications in denoising and filling-in 16