Cortical circuits for vision

1. Jamie Mazer Neurobiology of Cortical Systems Lecture 7 March 12, 2012 Cortical circuits for vision

2. Readings for Thursday

3. How much of cortex is visual? (in primates) Primates are likely an extreme example or an upper bound.. Van Essen flat map of macaque cortex

4. How much of cortex is visual? “simplified” Felleman & Van Essen hierarchy Van Essen flat map of macaque cortex

5. Key concepts • phenomenon vs. implementation vs. function • “centrally synthesized maps” • everything we perceive must be encoded by the retina • if so, what’s all that visual cortex doing? • generating explicit sensory representations • “emergent” properties seem to be a key feature of high-level sensory cortical function • Question: Is cortex required to generate explicit or abstract properties? • Answer: What’s emergent in the retina? What about animals with not cortex, like birds and fish? • are there common “motifs” across sensory modalities? • computational maps in other modalites? • what about other species? are they unique to cortex?

6. Retinal bipolar cells receptive fields

7. Retinal ganglion cell RFs (only retinal output)

8. Receptive fields and center-surround opponency

9. Receptive fields and center-surround opponency • Center-surround organization • Observed phenomenon? • Implementation? • Function?

10. Receptive fields and center-surround opponency • Center-surround organization • Observed phenomenon?  Characteristic RF structure • Implementation?  Lateral inhibition • Function?  Spatial derivative; contrast enhancement

11. Behavioral consequences of center surround organization herring grid mach bands

12. Behavioral consequences of center surround organization herring grid mach bands

13. Thalamus: dLGN

14. What changes between the photoreceptors and LGN? • transition from receptor potentials to spiking • center-surround spatial receptive fields • “color opponency” (B-Y/R-G) instead of simple cone-based wavelength tuning • segregation into parallel processing streams • sustained and transient • fast and slow • on and off channels • color and luminance

15. Which brings us to primary visual cortex (BA 17; V1) visual association m primary visual

16. Topographic organization of V1 • retinotopy • orientation columns • occular dominance columns • non-oriented blobs (L2) • orientation topography

17. Thalamocortical projections and the canonical microcircuit

18. Primary visual cortex: simple cell orientation tuning orientation tuned V1 neuron MOVIE hubel & wiesel 1968

19. Primary visual cortex: simple cell orientation tuning orientation tuned V1 neuron hubel & wiesel model hubel & wiesel 1968

20. Primary visual cortex: simple cell orientation tuning orientation tuned V1 neuron hubel & wiesel model Key failures for the feedforward model? - contrast invariant orientation tuning hubel & wiesel 1968

21. Primary visual cortex: simple cell orientation tuning orientation tuned V1 neuron hubel & wiesel model • Hubel & Wiesel Interpretation • Observed phenomenon? • preferred orientation • Implementation? • linear summation of LGN cells • Function? • feature detectors for edges hubel & wiesel 1968

22. Primary visual cortex: simple cell orientation tuning orientation tuned V1 neuron hubel & wiesel model • Spatial Vision Interpretation • Observed phenomenon? • preferred orientation • Implementation? • quasi-linear combination of LGN cells • Function? • spatiotemporal filtering hubel & wiesel 1968

23. Feature detector model “spatial vision” model • cells prefer light increments or decrements • cells have orientation tuning • cells have spatial frequency tuning • cells have temporal frequency tuning •  cells are half-wave rectified spatiotemporal filters (Gabors) • requires some math chops to understand, but has predictive power • cells prefer light increments or decrements • cells have orientation tuning • cells have a width tuning • cells have length tuning • cells have speed tuning •  cells are feature detectors where the feature is a bar of a particular orientation, size and speed • intuitively obvious, simple to understand, seems to imply obvious behavioral function

24. Primary visual cortex: spatial frequency tuning Robson, DeValois, Maffei etc..

25. Feature detector model “spatial vision” model • cells prefer light increments or decrements • cells have orientation tuning • cells have a width tuning • cells have length tuning • cells have speed tuning •  cells are feature detectors where the feature is a bar of a particular orientation, size and speed • intuitively obvious, simple to understand, seems to imply obvious behavioral function • cells prefer light increments or decrements • cells have orientation tuning • cells have spatial frequency tuning • cells have temporal frequency tuning •  cells are half-wave rectified spatiotemporal filters • requires some math chops to understand, but has predictive power

26. Primary visual cortex: simple  complex simple complex hubel & wiesel 1968

27. Primary visual cortex: simple  complex simple complex MOVIE hubel & wiesel 1968

28. Primary visual cortex: simple  complex simple complex hypercomplex +length tuning +length tuning +length tuning hubel & wiesel 1968

29. Primary visual cortex: simple  complex “simple cells” pool center-surround neurons to form orientation selectivity “complex cells” pool simple cells to become position or phase invariant. and turtles all the way down… hubel & wiesel 1968

30. Complex cells and the F1/F0 ratio cats monkeys Skottun et al, 1991

31. What’s the spatial vision model got to say?

32. Complex cells and the F1/F0 ratio is this all an artifact? cats monkeys Skottun et al, 1991 Mechler & Ringach, 2002

33. Reverse correlation and the spike triggered average Jones & Palmer, 1987

34. Reverse correlation and the spike triggered average Jones & Palmer, 1987

35. Reverse correlation and the spike triggered average Jones & Palmer, 1987

36. V1 neurons are Gabor’s and Gabor’s are optimal… Daugman, 1985

37. V1 neurons are Gabor’s and Gabor’s are optimal… Daugman, 1985

38. Where do Gabor’s come from and the efficient coding hypothesis Barlow, 1972

39. Where do Gabor’s come from and the efficient coding hypothesis

40. Where do Gabor’s come from and the efficient coding hypothesis Vinje & Gallant, 2000

41. Where do Gabor’s come from and the efficient coding hypothesis Haider et al, 2010

42. What have we established? • simple cells • simple cells are partially assembled from LGN afferents • one basic flavor: Gabor • they are bar-detectors as well (glass half empty), but • the Gabor-model seems like a more compact framework • complex cells • complex cells are assembled from simple cells • strict dichotomy not likely, more likely is, • thalamocortical direct recipient simple cells, and, • cells that are a combination of simple and non-simple innputs • coding in V1 • sparseness is a hallmark of an efficient code • simple cells can be learned by maximizing sparseness • sparseness in V1 is based on center-surround (intracortical) inhibitory interations • the neural representation is awful close to what the computer vision people call a wavelet or multiscale pyramid and is the basis for things like MPG and JPG compression… • perhaps we need more data from more complex stimuli?

43. Reverse correlation, complex cells and natural scenes

44. Reverse correlation, complex cells and natural scenes Problems: STA doesn’t really work fornatural (non-white) stimuli the STA is just plain “wrong” for complex cells

45. Linear receptive field maps in early vision still orientation tuned! where’s it coming from? DeAngelis et al, 1995

46. Reverse correlation, complex cells and natural scenes Problems: STA doesn’t really work fornatural (non-white) stimuli the STA is just plain “wrong” for complex cells

47. Reverse correlation, complex cells and natural scenes Problems: STA doesn’t really work fornatural (non-white) stimuli the STA is just plain “wrong” for complex cells Spike Triggered Covariance (STC)

48. What have we established? • simple cells • simple cells are partially assembled from LGN afferents • one basic flavor: Gabor • they are bar-detectors as well (glass half empty), but • the Gabor-model seems like a more compact framework • complex cells • complex cells are assembled from simple cells • strict dichotomy not likely, more likely is, • thalamocortical direct recipient simple cells, and, • cells that are a combination of simple and non-simple innputs • coding in V1 • sparseness is a hallmark of an efficient code • simple cells can be learned by maximizing sparseness • sparseness in V1 is based on center-surround (intracortical) inhibitory interations • the neural representation is awful close to what the computer vision people call a wavelet or multiscale pyramid and is the basis for things like MPG and JPG compression… • perhaps we need more data from more complex stimuli? • STRFs, STC, regression analysis, MII etc all provide new tools could complex cells and complex stimuli… • what did I not talk about??

49. Direction selectivity MOVIE hubel&wiesel 1968

50. Direction selectivity is Gabor-ish too (vs. Reichardt Detector) DeAngeliset al, 1993. 1995