Visual Neuron Responses

1. Visual Neuron Responses • This conceptualization of the visual system was “static” - it did not take into account the possibility that visual cells might change their response selectivity over time and it was firmly based in the classical notion of a receptive field • Logic went like this: if the cell is firing, its preferred line/edge must be present and… • if the preferred line/edge is present, the cell must be firing • We will encounter examples in which these don’t apply! • Representing boundaries and surfaces must be more complicated than simple edge detection! WHY??

2. What can a visual neuron “know” about the image? • If a neuron is only an edge detector and/or only has a small receptive field, it can’t represent information about the relationship between the contents of its receptive field and other receptive fields elsewhere in the display.

3. What can a visual neuron “know” about the image? • If a neuron is only an edge detector and/or only has a small receptive field, it can’t represent information about the relationship between the contents of its receptive field and other receptive fields elsewhere in the display. • For example the famous 1961 Rosebowl hoax…no single person could know what the big picture showed

4. What can a visual neuron “know” about the image? • If a neuron is only an edge detector and/or only has a small receptive field, it can’t represent information about the relationship between the contents of its receptive field and other receptive fields elsewhere in the display. • Also the 2004 Harvard – Yale Game:

5. Visual Neuron Responses • Edges are important because they are the boundaries between objects and the background or objects and other objects

6. Visual Neuron Responses • Boundaries between objects can be defined by color rather than brightness

7. Visual Neuron Responses • Boundaries between objects can be defined by texture

8. Visual Neuron Responses • Boundaries between objects can be defined by motion and depth cues

9. Visual Neuron Responses • Boundaries between objects can be defined by motion and depth cues

10. Feed-Forward and Feed-Back Processing in the Visual System

11. The Feed-Forward Sweep • What is the feed-forward sweep?

12. The Feed-Forward Sweep • The feed-forward sweep is the initial response of each visual area “in turn” as information is passed to it from a “lower” area • Characteristics: • a single spike per synapse • no time for lateral connections • no time for feedback connections

13. The Feed-Forward Sweep • The feed-forward sweep is the initial response of each visual area “in turn” as information is passed to it from a “lower” area • What does it mean for an area to be “lower” or “higher”

14. The Feed-Forward Sweep • Hierarchy of visual cortical areas defined anatomically Notice the direct connection from SC to MT/V5 Dorsal “where”/”how” Ventral “what”

15. The Feed-Forward Sweep • Hierarchy can be defined more functionaly • The feed-forward sweep is the initial response of each visual area “in turn” as information is passed to it from a “lower” area • Consider the latencies of the first responses in various areas

16. The Feed-Forward Sweep • Thus the “hierarchy” of visual areas differs depending on temporal or anatomical features • aspects of the visual system account for this fact: • multiple feed-forward sweeps progressing at different rates (I.e. magno and parvo pathways) in parallel • M pathway is myelinated • P pathway is not • signals arrive at cortex via routes other than the Geniculo-striate pathway (LGN to V1) • Will be important in understanding blindsight

17. The Feed-Forward Sweep • The feed-forward sweep gives rise to the “classical” receptive field properties • tuning properties exhibited in very first spikes • Orientation tuning in V1 • Optic flow tuning in MST • think of cortical neurons as “detectors” only during feed-forward sweep

18. After the Forward Sweep • By 150 ms, virtually every visual brain area has responded to the onset of a visual stimulus • But visual cortex neurons continue to fire for hundreds of milliseconds!

19. After the Forward Sweep • By 150 ms, virtually every visual brain area has responded to the onset of a visual stimulus • But visual cortex neurons continue to fire for hundreds of milliseconds! • What are they doing?

20. After the Forward Sweep • By 150 ms, virtually every visual brain area has responded to the onset of a visual stimulus • But visual cortex neurons continue to fire for hundreds of milliseconds! • What are they doing? • with sufficient time (a few tens of ms) neurons begin to reflect aspects of cognition other than “detection”

21. Extra-RF Influences • One thing they seem to be doing is helping each other figure out what aspects of the entire scene are contained within a given receptive field • That is, the responses of visual neurons begin to change to reflect global rather than local features of the scene • recurrent signals sent via feedback projections are thought to mediate these later properties

22. Extra-RF Influences Note that these are responses to the same stimulus!

23. Extra-RF Influences • consider texture-defined boundaries • classical RF tuning properties do not allow neuron to know if RF contains figure or background • At progressively later latencies, the neuron responds differently depending on whether it is encoding boundaries, surfaces, the background, etc.

24. Extra-RF Influences • Consider this analogy: • Imagine when each fan puts up a card he or she is told to shake it – so that the entire scene is full of shaking cards • After some delay, the fans holding up the red cards are told to keep shaking but the fans holding white cards are told to stop…the words will be enhanced • But the fans can’t each figure that out on their own because they don’t actually know the color of the card they are holding

25. Extra-RF Influences • How do these data contradict the notion of a “classical” receptive field?

26. Extra-RF Influences • How do these data contradict the notion of a “classical” receptive field? • Remember that for a classical receptive field (i.e. feature detector): • If the neuron’s preferred stimulus is present in the receptive field, the neuron should fire a stereotypical burst of APs • If the neuron is firing a burst of APs, its preferred stimulus must be present in the receptive field

27. Extra-RF Influences • How do these data contradict the notion of a “classical” receptive field? • Remember that for a classical receptive field (i.e. feature detector): • If the neuron’s preferred stimulus is present in the receptive field, the neuron should fire a stereotypical burst of APs • If the neuron is firing a burst of APs, its preferred stimulus must be present in the receptive field

28. Recurrent Signals in Object Perception • Can a neuron represent whether or not its receptive field is on part of an attended object? • What if attention is initially directed to a different part of the object?

29. Recurrent Signals in Object Perception • Can a neuron represent whether or not its receptive field is on part of an attended object? • What if attention is initially directed to a different part of the object? Yes, but not during the feed-forward sweep

30. Recurrent Signals in Object Perception • curve tracing • monkey indicates whether a particular segment is on a particular curve • requires attention to scan the curve and “select” all segments that belong together • that is: make a representation of the entire curve • takes time

31. Recurrent Signals in Object Perception • curve tracing • neuron begins to respond differently at about 200 ms • enhanced firing rate if neuron is on the attended curve

32. Feedback Signals and the binding problem • What is the binding problem?

33. Feedback Signals and the binding problem • What is the binding problem? • curve tracing and the binding problem: • if all neurons with RFs over the attended curve spike faster/at a specific frequency/in synchrony, this might be the binding signal

34. Feedback Signals and the binding problem • So what’s the connection between Attention and Recurrent Signals?

35. Feedback Signals and Attention • One theory is that attention (attentive processing) entails the establishing of recurrent “loops” • This explains why attentive processing takes time - feed-forward sweep is insufficient

36. Feedback Signals and Attention • Instruction cues (for example in the Posner Cue-Target paradigm) may cause feedback signal prior to stimulus onset (thus prior to feed-forward sweep) • think of this as pre-setting the system for the upcoming stimulus • What does this accomplish?

37. Feedback Signals and Attention • What does this accomplish? • Preface to attention: Two ways to think about attention • Attention improves perception, acts as a gateway to memory and consciousness • Attention is a mechanism that routes information through the brain • It is the brain actively reconfiguring itself by changing the way signals propagate through networks • It is a form of very fast, very transient plasticity

38. Feedback Signals and Attention • Put another way: • It may strike you as remarkable that a single visual stimulus should “activate” so many brain areas so rapidly • In fact it should be puzzling that a visual input doesn’t create a runaway “chain reaction” • The brain is massively interconnected • Why shouldn’t every neuron respond to a visual stimulus

39. Feedback Signals and Attention • We’ll consider the role of feedback signals in attention in more detail as we discuss the neuroscience of attention