Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. G

1. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press Chapter 6 The Vision “The question is not what you look at, but what you see.” Henry David Thoreau

2. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press Chapter Outline 1.0 Introduction 2.0 Functional organization of the visual system 3.0 Theories of visual consciousness: where does it happen? 4.0 Brain areas necessary for visual awareness: lesion studies 5.0 Linking brain activity and visual experience 6.0 Manipulations of visual awareness 7.0 Summary

3. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 1.0 Introduction The mystery of visual experience Most humans intuitively think that human vision works much like a camera. However, it turns out this is far from the case. Unlike the camera, you actually experience the image, you know what it is you are looking at.

4. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 1.0 Introduction The mystery of visual experience How does the brain perceive what something is? Studies of human visual perception suggest that there are many levels of perception. At the most basic level, the human brain processes visual features such as color, orientation, motion, texture, and stereoscopic depth.

5. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 1.0 Introduction The purpose of vision: knowing what is where Knowing what:perceiving features, groups, and objects Gestalt Laws of Perceptual Grouping How is the brain able to combine individual features in order to recognize an object? In the early 1900’s, Gestalt psychologists proposed that perception could not be understood by simply studying the basic elements of perception and proposed a series of perceptual grouping principles. Three core laws are grouping by similarity, proximity, and good continuation.

6. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 1.0 Introduction The purpose of vision: knowing what is where Knowing where:perceiving where things are How do we know where objects are located in the world? When we look at the world, the image that strikes the back of our eye is essentially two-dimensional, similar to the image that would be taken by a camera.

7. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 1.0 Introduction The purpose of vision: knowing what is where Knowing where:perceiving where things are This two-dimensional map of the world projected onto the eye is preserved in the early visual areas of the cortex, which provides a map of where objects are located relatively to the center of gaze. The brain is also able to figure out the missing third dimension and estimate how far away objects are in space.

8. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system The retina Objects in the environment are physically projected to the back of the eye -- the retina.

9. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system The retina There are two types of photoreceptors: cones and rods. Cones are color-selective, less sensitive to dim light, and important for detailed color vision in daylight. Rods are are important for night vision -- we rely on seeing with them once our eyes have adapted to darkness. The cross section above shows where the photoreceptors are located in the retina.

10. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system The retina Signals from the photoreceptors are processed by intermediate neurons -- bipolar, horizontal, and amacrine cells -- before they reach the ganglion cells, the final processing stage before the signals leave the eye. The actual bodies of the ganglion cells are in the retina but they have long axons that leave the retina at the blind spot and form the optic nerve. Each ganglion receives excitatory inputs from a collection of rods and cones.

11. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system The retina Center-surround receptive fields A neuron’s receptive field refers to the portion of the visual field that can activate or inhibit response. Retinal ganglion cells receive both excitatory and inhibitory inputs from bipolar cells. The spatial pattern of these inputs determines the cell’s receptive field. For example, a cell with an on-center off-surround receptive field will response strongly if a spot of light is presented at the center of the receptive field.

12. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system Lateral geniculate nucleus (LGN) From the eye, retinal ganglion cells send their axons to a structure in the thalamus called the lateral geniculate nucleus (LGN). The left half of each retina projects to the left LGN, the right half to the right LGN, with the inputs crossing at the optic chiasm. The result is that the left LGN receives input from the right visual field and the right LGN receives input from the left visual field. Inputs from each eye go to separate monocular layers of the LGN, so signals from the two eyes remain separate until they reach the primary visual cortex (V1) where signals are combined. left hemisphere right hemisphere

13. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system Primary visual cortex (V1) From the LGN, neurons send their signals to the primary visual cortex, called V1 because it is the first cortical visual area. The left LGN projects to V1 in the left hemisphere, the right LGN projects to the right hemisphere. In V1, the spatial layout of the inputs from the retina is preserved. Left V1 contains a retinotopic map of the entire right visual field, while right VI contains a map of the left visual field. left hemisphere right hemisphere

14. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system Primary visual cortex (V1) An example of how a collection of center-surround receptive fields could lead to orientation selectivity in V1 neurons. The overlapping circles on the left show center-surround receptive fields. When the bar of light lays vertically it triggers all of the on-centers of each receptive field, when its orientation changes fewer centers and more surrounds are activated, resulting in smaller neural response.

15. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system Extrastriate visual areas -- outside of V1 V1 sends feedforward to many higher visual areas including areas such as V2, V3, V4, and motion-sensitive area MT. Area V4 is especially important for color perception, and some neurons in V4 respond well to more complex features or combination of features. For example, V4 neurons are sensitive to curvature or to two lines that meet at a specific angle.

16. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system Area MT The middle-temporal -- MT -- area is important for motion perception. Almost all neurons in area MT are direction-selective, meaning that they respond selectively to a certain range of motion directions and not to others.

17. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system The ventral and dorsal pathways: knowing what and where The projections from V1 to higher areas in the cortex can be roughly divided into two major parallel pathways: a ventral pathway leading from V1 to the temporal lobe that is important for representing ‘what‘ objects are and a dorsal pathway leading from V1 to the parietal lobe that is important for representing ’where’ things are

18. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system Areas involved in object recognition - a hierarchy of visual processing Hierarchical response properties of the visual system to simple and complex stimuli. The leftmost column shows a house stimulus and what receptive fields of each visual area would see is shown in the balloons. Not only do the receptive fields increase in size in each visual area, but also in the complexity of the shapes they respond to.

19. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system Lateral occipital complex (LOC) The lateral occipital complex (LOC) has a role in object recognition and responds strongly to a variety of shapes and objects.

20. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system Fusiform face area (FFA) Human neuroimaging studies have shown that there is a region in the fusiform gyrus, called the fusiform face area (FFA), that responds more strongly to faces than to just about any other category of objects.

21. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 2.0 Functional organization of the visual system Parahippocampal place area (PPA) The parahippocampal place area (PPA)is another strongly category-selective region that responds best to houses, landmarks, and indoor and outdoor scenes. This area responds more weakly to other types of stimuli such as faces, bodies, or inanimate objects.

22. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 3.0 Theories of visual consciousness: where does it happen? Hierarchical and interactive theories of vision In the hierarchical model, with each step further in visual processing, awareness is more likely to result from that processing. In the interactive model, feedback signals from later processing areas to earlier ones are needed to attain awareness. At present, it is not clear which theory best describes the way brain activity results in visual awareness.

23. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 4.0 Brain areas necessary for visual awareness: Lesion studies Brain lesion studies are important for understanding what brain areas are necessary for certain kinds of visual awareness -- awareness of color, motion, faces, objects, or the capacity to be aware of seeing anything at all. Brain lesions may be performed experimentally, in animal studies, or may be investigated in humans who have suffered from unfortunate injury to certain parts of the brain due to strokes, tumors, trauma, or neurodegenerative disease. By studying these patients, it may be possible to understand the neural causes of their impairment. This may inform scientists about brain function and eventually lead to new ways to help treat such impairments.

24. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 4.0 Brain areas necessary for visual awareness: Lesion studies Consequences of damage to early visual areas Damage to V1 in one hemisphere may result in blindness in the visual hemifield contralateral to the hemisphere that is damaged. For example, if a patient suffers from a lesion in V1 in the left hemisphere, he may have blindness in the right hemifield in both eyes. This damage may result in a condition called blindsight, where the patient is not aware of seeing objects in the blind hemifield, however under careful testing, it may be the case that the patient’s visual cortex is ‘seeing’ the object but the patient is not aware of this. Thus, damage to V1 can eliminate almost all visual awareness, even though sufficient visual information seems to be reaching visual areas.

25. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 4.0 Brain areas necessary for visual awareness: Lesion studies Extrastriate lesion Damage to areas outside of V1 Damage to area MT in both hemispheres can lead to motion blindness Damage to color areas (such as V4) in only one hemisphere can result in a loss of color perception to one side of visual space.

26. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 4.0 Brain areas necessary for visual awareness: Lesion studies Damage to ventral object areas Patients with visual agnosia have difficulties with recognizing objects because of impairments in basic perceptual processing or higher-level recognition processes. Such patients can still recognize objects by using other senses, such as touch, hearing or smell: the loss of recognition is strictly in the visual domain.

27. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 4.0 Brain areas necessary for visual awareness: Lesion studies Three types of visual agnosia: Apperceptive agnosia: patients can detect the appearance of a visually presented item but they have difficulty perceiving their shape and cannot recognize or name them. This is a perceptual deficit. Associative agnosia: patients are unable to recognize an object despite intact perception of the object. They can draw a copy of the object but cannot name it. This is an associative deficit. Prosopagnosia: patients have difficulty recognizing faces.

28. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 4.0 Brain areas necessary for visual awareness: Lesion studies Damage to dorsal parietal areas Damage to the posterior parietal lobe (or superior temporal gyrus) can lead to a global modulation of visual awareness called neglect. Patients with right parietal damage may ignore the left half of the visual field: eat just half the food on the plate, or apply makeup to just one half of their face. This syndrome can resemble a disorder of visual perception, however neglect happens in the absence of damage to the visual system and can involve multimodal deficits, including tactile and motor.

29. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 5.0 Linking brain activity and visual experience Multistable perception Bistable figures: (a) after you look at the figure for a while, you will notice that there are two interpretations -- two faces in silhouette or one vase. It is bistable, your vision will alternate between the two interpretations. (b) the Necker cube has two equally likely spatial interpretations. Perception tends to alternate between the two. In both of these figures, the physical pattern does not change, but your awareness of it does.

30. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 5.0 Linking brain activity and visual experience Binocular rivalry One of the most powerful and best-studied examples of bistable perception is a phenomenon called binocular rivalry. When two very different patterns are shown, one to each eye, because they are so different the brain cannot fuse them together like normally does. What then happens is quite striking: awareness of one pattern lasts for a few seconds, then the other pattern seems to magically appear and wipe away the previously visible pattern. It is like the two patterns are fighting it out in the brain for your perceptual awareness!

31. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 5.0 Linking brain activity and visual experience Binocular rivalry Brain areas active when a face and a house are presented simultaneously in order to evoke binocular rivalry. Note on the top right of the figure, the activation fluctuates in the fusiform face area and the parietal place area during binocular rivalry (Tong et al., 1998).

32. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 5.0 Linking brain activity and visual experience Constructive perception: more to vision than meets the eye … Perceptual filling-in Although we are not aware of it, we have a blind spot at the back of the retina where the axons of the ganglion cells meet to form the optic nerve as it exits the eye. Why aren’t we aware of it? The brain fills in perception of the blind spot: the brain uses visual information from around the blind spot and constructs awareness of what it ‘thinks’ should be there. Try the blind spot demonstration on page 174 of the book!

33. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 5.0 Linking brain activity and visual experience Constructive perception: more to vision than meets the eye … The blind spot is not the only case of the brain perceptually filling-in: notice how the area between the colored lines appears to be colored.

34. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 5.0 Linking brain activity and visual experience Constructive perception: more to vision than meets the eye … Careful inspection reveals that the background is actually white. This illusion is called color spreading and the experience of color here seems to come from constructive filling-in mechanisms at work in visual cortex.

35. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 6.0 Manipulations of visual awareness Transcranial magnetic stimulation (TMS) When TMS is applied over early visual areas, there are two primary perceptual consequences. One, when people have their eyes closed they tend to experience a weak flash of light, a phosphene. This is attributed to the activation of visual neurons due to the electrical stimulation. The second consequence is that you can experience a visual hole or momentary blind spot -- a transient scotoma.

36. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 6.0 Manipulations of visual awareness Transcranial magnetic stimulation (TMS) What is interesting is that the type of phosphene people experience corresponds to the area of cortical tissue stimulated with TMS. For example, when V1 is stimulated, people report smallish static (not moving) phosphenes. When area MT (the motion area) is stimulated, people report moving phosphenes. TMS has been used to investigate neural transmission and feedback across regions in visual cortex.

37. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 6.0 Manipulations of visual awareness Unconscious perception Investigations of unconscious perception use situations where a subject reports not seeing a given stimulus but their behavior or brain activity suggests that specific information about the unperceived stimulus was indeed processed by the brain. Perceiving emotions in faces unperceived -- here is an example of an investigation of unconscious perception using fMRI: the red building presented to the left eye suppresses the face in the right eye out of awareness, as in binocular rivalry.

38. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 6.0 Manipulations of visual awareness Perceiving emotions in faces unperceived -- the graph shows the activity in the amygdala (an emotional response area). The red plot shows the activity in the amygdala increasing when emotional faces are presented, even though they are out of awareness.

39. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 7.0 Summary Over the last decade or so, scientists have learned a great deal about the neural correlates of conscious and unconscious perception. Using new techniques such as transcranial magnetic stimulation, they have shown how the disruption of different brain areas can disrupt specific aspects of visual consciousness. Some key concepts: Primary visual cortex -- V1 -- is important for the ability to perceive any visual feature at all, whereas higher brain areas are important for perceiving particular visual features or objects. Progressing up the visual pathway, receptive fields gradually become larger and respond to more complex stimuli, following the hierarchical organization of the visual system.

40. Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press 7.0 Summary More key concepts: V1 is selective for many visual features, including orientation, motion, and binocular disparity. Damage to V1 can severely impair or eliminate conscious vision, although remaining activity in extrastriate areas may support the ability to detect visual events even without being visually conscious -- the condition called blindsight. Extrastriate areas are important for perceiving specific visual features: area V4 is important for color perception and area MT for motion perception. Damage to the central temporal cortex can lead to impairments (agnosias) in visual perception, object recognition, or face recognition. Damage to the dorsal (parietal) pathway can lead to neglect or impairments in visually guided actions.