Perceptual systems: Central visual pathways

1. Perceptual systems:Central visual pathways Kandell et. al., Princ. of Neural Sci.

2. Outline • Subcortical pathways (retina, LGN) • Magno- & Parvo- streams • Primary visual cortex (V1) • Partial integration of M and P • Gateway to Ventral/Dorsal • Ventral pathway (V4, IT) • Object processing • Dorsal pathway (MT, IP) • Motion processing • Summary

3. Retinal ganglion cells (RGCs):Quick Summary • RGC’s • Output of the retina (“optic nerve”)

4. Retinal ganglion cells (RGCs):Quick Summary • RGC’s • Output of the retina (“optic nerve”) • Center-surround receptive fields Kandell et. al., Princ. of Neural Sci.

5. RGCs: Two principal cell-types Parasol RGCs receptive field tiling Midget RGC receptive field tiling Send axons to LGN to form Magnocellular (‘M’) pathway Send axons to LGN to form Parvocellular (‘P’) pathway

6. RGCs: Two principal cell-types Parasol RGCs receptive field tiling Midget RGC receptive field tiling Send axons to LGN to form Magnocellular (‘M’) pathway Send axons to LGN to form Parvocellular (‘P’) pathway

7. RGCs: Two principal cell-types Parasol RGCs receptive field tiling Midget RGC receptive field tiling P pathway characteristics: -Smaller RFs, so better spatial resolution -Selective for color (red-green opponency) M pathway characteristics: -Better temporal resolution -More sensitive to low light levels -Monochromatic (black-white opponency)

8. Lateral geniculate nucleus (LGN) • Thalamus is the meeting point of sensory organs • LGN is the visual portion: • Receives direct input from RGC cells

9. Lateral geniculate nucleus (LGN) • LGN receives direct input from RGC cells • Receptive fields have very similar RFs to their RGC inputs: • Circular • Center-surround • Reflected in anatomy… Kandell et. al., Princ. of Neural Sci.

10. Optic chiasm RGCs in left hemifield send axons to right LGN, and vice-versa Kandell et. al., Princ. of Neural Sci.

11. LGN lamina: 3 layers for each eye -Contralateral= the eye at the opposite side of brain -Ipsilateral= the eye at the same side of the brain -Contra and ipsi inputs are segregated by LGN layers Parvocellular(P channel) Magnocellular (M channel) Kandell et. al., Princ. of Neural Sci.

12. LGN lamina: 4 Parvo & 2 Magno layers Inputs from RGC Midget cells: -Smaller RFs -Selective for Color Inputs from RGC Parasol cells: -Larger RFs -Monochromatic -Respond to faster changes Parvocellular(P-channel) -Similar RFs to Midget inputs Magnocellular (M-channel) -Similar RFs to Parasol inputs Kandell et. al., Princ. of Neural Sci.

13. LGN lamina: 4 Parvo & 2 Magno layers All combinations transmitted to cortex: M-Contra, M-Ipsi, P-Contra, P-Ipsi Kandell et. al., Princ. of Neural Sci.

14. Subcortical pathways summary: • Unique cell-types in the retina form the first stages of 2 major parallel pathways: Magno- and Parvo- • M and P pathways carry unique spatio-temporal-chromatic information • LGN RFs are very similar to those in the retina (one reliable synapse from RGC) • 6 layers in the LGN to maintain segregation of eye input and M/P input. • Subcortical pathways are trying to take basic features about the visual scene and reliably transmit this information to cortex.

15. Outline • Subcortical pathways (retina, LGN) • Magno- & Parvo- streams • Primary visual cortex (V1) • Partial integration of M and P • Gateway to Ventral/Dorsal • Ventral pathway (V4, IT) • Object processing • Dorsal pathway (MT, IP) • Motion processing • Summary Kandell et. al., Princ. of Neural Sci.

16. Laminar architecture of cortex • The primate cortex is ~2mm thick • 6 layers within this thickness

17. Laminar architecture of cortex Outputs from layers II/III • The primate cortex is ~2mm thick • 6 layers within this thickness • Feedforward circuit: • Layer IV, then layer II/III, then next cortical area Inputs to layer IV

18. Laminar architecture of V1 Outputs from layers II/III V1 Inputs to layer IV Kandell et. al., Princ. of Neural Sci.

19. Laminar architecture of V1 V1 Spatial-temporal-chromatic information of visual world is still segregated anatomically within the input layers of cortex: i.e. Parallel pathways are still segregated Kandell et. al., Princ. of Neural Sci.

20. Laminar architecture of V1 V1 Finally, this information is partially combined once it hits the output layers of V1. Spatial-temporal-chromatic information of visual world is still segregated anatomically within the input layers of cortex: i.e. Parallel pathways are still segregated Kandell et. al., Princ. of Neural Sci.

21. V1 ocular dominance • Eye inputs are also segregated at the input layer of V1 • A neuron’s response preference for one eye over another is called “ocular dominance” • In V1, there are alternating bands of ocular dominance across the cortical surface… “map” 1mm LGN V1 Kandell et. al., Princ. of Neural Sci.

22. Primary visual cortex (V1):Orientation tuning arises • LGN inputs are not tuned for edges • V1 neurons, one synapse from LGN, have tuning for edges of a particular orientation: “orientation tuning”

23. V1 orientation tuning Response Stimulus

24. V1 orientation tuning • Classic Hubel & Weisel model: Orientation tuning arises from an alignment of LGN inputs LGN V1 Kandell et. al., Princ. of Neural Sci.

25. V1 orientation maps Similar tuning within column • Columnar organization: similar response properties at all depths of a vertical penetration

26. V1 orientation maps • Tuning progresses smoothly across horizontal penetration

27. V1 orientation maps 1mm Kandell et. al., Princ. of Neural Sci.

28. V1 orientation maps • Continuous maps of orientation preference: “pinwheels” • Consistent preference though the depth of cortex: columnar architecture Kandell et. al., Princ. of Neural Sci.

29. V1 retinotopic maps • Each point of the visual field maps on to a local group of neurons in V1. • Retinotopy = Remapping of retinal image onto cortical surface • Foveal region uses more of V1 (greater magnification factor) (V1)

30. Classic V1 maps: summary Retinotopic map Ocular dominance map Orientation map 1mm Why have maps? How are they created? Kandell et. al., Princ. of Neural Sci.

31. Classic V1 maps: summary Retinotopic map Ocular dominance map Orientation map 1mm Why have maps? How are they created? Nobody knows for certain Kandell et. al., Princ. of Neural Sci.

32. Primary visual cortex (V1): Summary • Pathways segregated in LGN are still segregated at input layers of V1 • M & P separated by sublayers in IVC • Eye inputs segregated into columns • Tuning for edges arises at first synapse: orientation selectivity. • Maps: orientation, ocular dominance, retinotopy. Preview to higher areas • V1 is the “trunk” of the visual cortex hierarchy:V1 output forms dorsal and ventral streams. • M-to-Dorsal (mostly) • P-to-Ventral (mostly)

33. Outline • Subcortical pathways (retina, LGN) • Magno- & Parvo- streams • Primary visual cortex (V1) • Partial integration of M and P • Gateway to Ventral/Dorsal • Ventral pathway (V4, IT) • Object processing • Dorsal pathway (MT, IP) • Motion processing • Summary Kandell et. al., Princ. of Neural Sci.

34. Ventral pathway: Preview • Two principal paths: • V1-V2-V4-TEO-IT • V1-V4-TEO-IT • Involved in object identification. Called the “what” pathway. • Late stages of inferior temporal (IT) cortex contain the celebrated “face cells” Kandell et. al., Princ. of Neural Sci.

35. Ventral pathway: Receptive field hierarchy • Subcortical pathways compute local differences • V1 identifies edges • V1 projects to V2 & V4: What might V4 neurons represent? ?

36. Ventral pathway: Receptive field hierarchy • Subcortical pathways compute local differences • V1 identifies edges • V1 projects to V2 & V4: What might V4 neurons represent? • Curvature, contour integration, foreground/background.

37. Ventral pathway IT represents complex objects, such as faces Desimoneet. al. 1984

38. Ventral pathway: Convergence of inputs at each stage • We don’t know how this selectivity is built from the circuitry, but we have very general (i.e. crude) ideas. • Each region receives convergent input from the previous region (e.g. V1-to-V4), thus progressively increasing the size and complexity of receptive fields V4 V1

39. Ventral pathway: Convergence of inputs at each stage

40. Ventral pathway: Convergence of inputs at each stage What happens when we scale or rotate the image?

41. Ventral pathway: Convergence of inputs at each stage Simple models predict that the IT neurons will no longer respond, but instead IT neurons demonstrate scale, shift, and rotational invariance. i.e. IT responds to particular objects regardless of simple transformations.

42. Ventral pathway: Convergence of inputs at each stage -Lots of models attempting to explain object recognition in the ventral pathway, particular invariance (many useful engineering applications): -None of them come close to the performance of the primate visual system. 200 ms to IT spikes; Also human response time ?

43. Ventral Pathway: Summary • Receives input from both M and P subcortical pathways, but mostly P inputs. • Progressive increase in the complexity of spatial selectivity. Neurons are ultimately responsive to holistic objects in IT. • IT neurons also exhibit well-known scale, shift, rotation invariance. How does this happen? Kandell et. al., Princ. of Neural Sci.

44. Outline • Subcortical pathways (retina, LGN) • Magno- & Parvo- streams • Primary visual cortex (V1) • Partial integration of M and P • Gateway to Ventral/Dorsal • Ventral pathway (V4, IT) • Object processing • Dorsal pathway (MT, IP) • Motion processing • Summary Kandell et. al., Princ. of Neural Sci.

45. Dorsal pathway (preview) • The “where” pathway • Main input from M-pathway circuits within V1. • V1-to-MT-to-IP • MT (aka V5) neurons are highly selective to direction of motion. • IP neurons tell the motor system how to respond to the environment. Kandell et. al., Princ. of Neural Sci.

46. Medial Temporal Area (MT):Motion Selectivity Motion selectivity seen at 2 stages of dorsal hierarchy: 1) Neurons in layer IVB of V1 2) Neurons in MT (IVB of V1 is major MT input) So what is MT doing?? IVB of V1 MT Both stages are direction selective

47. Medial Temporal Area (MT)computes global motion You could easily identify the motion of this object. Born & Bradley., Annu Rev Neurosci

48. Medial Temporal Area (MT)computes global motion Apparent motion Actual motion -Now, it would appear that movement is in the direction perpendicular to the unmasked edge. -This is what a V1 neuron “sees”. V1 neurons only detect local motion. Born & Bradley., Annu Rev Neurosci

49. Medial Temporal Area (MT)computes global motion Apparent motion Actual motion -MT neurons have much larger receptive fields than V1, so they can integrate multiple cues in the image. MT neurons can detect global motion of the object.

50. Parietal cortex: A ridiculously brief summary • MT passes its output to the intraparietal cortical areas. Each IP area is responsible for a different “class” of movement. • In addition to unparalleled object recognition (ventral pathway), primates have great hand-eye coordination so that we can use tools, or hit a baseball (dorsal pathway). • IP uses visual input to tell the motor system how to interact with the environment.