Perception and A ttention

1. Cognitive Architectures Perception and Attention Based on book Cognition, Brain and Consciousness ed. Bernard J. Baars courses taught byProf. Randall O'Reilly, University of Colorado, and Prof. Włodzisław Duch, Uniwersytet Mikołaja Kopernika and http://wikipedia.org/ http://grey.colorado.edu/CompCogNeuro/index.php/CECN_CU_Boulder_OReilly http://grey.colorado.edu/CompCogNeuro/index.php/Main_Page Janusz A. Starzyk

2. Where does invariance come from? A 3D image based on 2D projections, what's remembered is just one 3D representation (Marr 1982). Syntactic approach: form a whole from pieces of a model. Recognition Variant (Hinton 1981): look for transformations (displacement, scaling, rotation), conform to the canonical representation in the memory. Problem: many 2D objects can form different 3D objects; it's difficult to match the objects because the search space to connect fragments into a whole is too large – do we really remember 3D objects?

3. In the brain, rotational invariance is strongly limited – eg. recognizing rotated faces. Limited invariant object recognition can be achieved thanks to gradual hierarchical parallel transformations, increasing invariance and creating increasingly complex features of distributed representations. Gradual transformations • Goal: not 3D, but to retain enough details to be able to recognize objects in an invariant manner after transformation. • Map seeking circuits in visual cognition (D. W. Arathorn, 2002 )

4. Recognition is a function of the ventral pathway, now let's turn to the dorsal pathway. Functions: motion detection, localization, "where” and how to act, but also on what to focus attention and how to shift attention from one object to another. Attention allows us to tie different properties of an object into one whole, to solve the problem of cohesion of sensations in spite of distributed processing; distributed activation => features related to each other, referring to one object. Mainly an attention model, an emergent process resulting from the structure and dynamic of neural networks, mainly inhibition. The effects of attention are universal, visible in different situations. Dorsal pathway What to pay attention to? Is this a well posed question? Dogs bite, but not only Spot, not only mongrels, not only black ones...

5. Lesion studies Consequences of damage to early visual areas • Different visual deficits can result from neural damage at different levels of the visual processing hierarchy. • Damage to the retina can result in monocular blindness. • Damage to the LGN can lead to loss of vision. • Patients with damage to V1 area may still perform better than chance forced choice discrimination of objects (blindsight), although they claim they see nothing. • Although the pathway from retina to LGN to V1 provides most of visual inputs to cortex, several alternative subcortical pathways project to extrastriate areas (MT, V3, V4), bypassing V1. This may explain forced choice results.

6. Lesion studies Extrastriate lesions – damage outside area V1 • Motion blindness caused by a lesion to area MT: the world appears to be a series of still snapshots. • Crossing street is dangerous since the patient cannot tell how fast the cars are approaching. • Pouring a cap of coffee becomes a challenge since she cannot tell how fast the liquid was rising.

7. Lesion studies • Cortical color blindness may be caused by a lesion to area V4: • The world appears to be drained of color, just shades of gray. • Patients can perceive the boundaries of colors but cannot name them.

8. Lesion studies Damage to ventral object areas Visual Agnosia:Patients with visual agnosia have difficulties with recognizing objects because of impairments in basic perceptual processing or higher-level recognition processes Three types of agnosia: apperceptive agnosia, associative agnosia, and prosopagnosia Agnosia=to lack knowledge of

9. Lesion studies • Patients with apperceptive agnosia can detect the appearance of visually presented items, but they have difficulty perceiving their shape and cannot recognize or name them. • Associative agnosia refers to the inability to recognize objects, despite apparently intact perception of the object. • Patient can copy a picture of the object but does not recognize it. • A patient mistook his wife for a hat. • Associative agnosia results from damage to ventral temporal cortex.

10. Lesion studies • Patients with optic ataxia can perceive visual orientation and recognize objects but cannot perform visually guided actions. • Optic ataxia results from damage to parietal lobe in dorsal pathway. • Patients with prosopagnosia are still able to recognize objects well, but have great difficulty recognizing faces. • All faces look the same • Patients can recognize animals but not people • Brodman area no. 37 is responsible for face recognition • over 90% of cells in area 37 responds to faces only.

11. Lesion studies • fMRI analysis of the face recognition process. • Visible is activity in right hemisphere in lower temporal area • Face recognition is important from evolutionary perspective.

12. Lesion studies • Patients with achromatopsia are unable to recognize colors. • This is often a result of damage to area V4 or thalamus.

13. Lesion studies • Daltonism refers to dichromacy characterized by a lowered sensitivity to green light resulting in an inability to distinguish green and purplish-red. • It is an inherited defect in perception of red and green, or in other words, red-green colorblindness.

14. Lesions in the parietal cortex strongly affect mechanisms of attention and spatial orientation, extensive lesions in one hemisphere lead to the inability to focus attention to the opposite half of the visual space. Dorsal pathway lesions For small unilateral lesions, we can see a noticeable slowing of attention switching to the damaged side. For more severe cases, switching attention is not possible. Bilateral lesions lead to difficulties with the coordination of hand and eye movement.

15. Lesion studies Self-portrait • Damage to the posterior parietal lobe can lead to a unilateralneglect, in which a patient completely ignores or does not respond to objects in the contralateral hemifield. • Patients with damaged spatial-temporal recognition forget about half the space even though they see it • Patients with right parietal damage may ignore the left half of the visual field, eat half of the food from the plate, or apply make-up to half of the face.

16. Unilateral Neglect Horizontal line bisection task Copying drawings

17. Lesion studies • Bilateral lesions to parietal areas can lead to a much more profound deficit called Balint’s syndrome, which is primarily a disruption of spatial attention. • It can be characterized by three main deficits: • Optic ataxia – inability to point into a target • Ocular apraxia – inability to shift the gaze • Simultanagnosia – inability to perceive more than one object in the visual field • People with Balint’s syndrome appearblind since they only focus on one object and cannot shift attention to anything else.

18. Linking brain activity and visual experience • Imagine you are sitting in a dark room and looking at a jacket on a chair. • Since you cannot see well, your perception is driven by your imagination – you may perceive a strange animal, a person, or a statue sitting there. • When vision is ambiguous, perception falters or alternates between different things. This is known as multistable perception. • There are many examples of multistable patterns or ambiguous figures that scientists use to investigate these neural correlates of consciousness.

19. Linking brain activity and visual experience Binocular rivalry: what you see is what you get activated • When two very different pattern are shown, one to each eye, the brain cannot fuse them together like it would normally do. • What happens is striking: awareness of one pattern last few seconds, then the other pattern appears You can cause binocular rivalry here using a pair of red-green glasses

20. Linking brain activity and visual experience Another way to separate physical stimulation and perceptual awareness is a visual detection task. • A subject has to detect a particular pattern. • The researcher makes the pattern harder and harder to see. • Sometimes there is no pattern at all in the picture. • Because this task gets difficult, people will get it wrong sometimes. • What is interesting, that when there is ‘false positive’ (people see pattern even when it is not there), there is strong activity in areas V1, V2, and V3. • When the faint stimulus is not detected activities in these areas are much weaker. • So, it does not matter what was presented, but what does matter is what is happening in the brain.

21. Linking brain activity and visual experience • Close your left eye, look directly at the cross with your right eye and move the page up close to your nose, then move it slowly away from your face, while keeping your eye fixed on the cross. At the right distance, which should be around 12 inches (30 cm) away from the page you should notice the red dot vanish. • Likewise, notice how the black stripes now fill-in; they become joined and the red dot vanishes. • Brain fills-in perception of the blind spot using visual information from around the blind spot – constructive perception or perceptual filling-in.

22. Linking brain activity and visual experience Optical illusions are a result of our mind filling-in patterns based on experience Adelson's motion without movement

23. Linking brain activity and visual experience Zoom in on the color spiral – two colors are the same shade of green. Two color spirals

24. Linking brain activity and visual experience • These pictures illustrate another type of filling-in known as neon color spreading (a) and visual phantoms (b). • Neon color spreading were found in V1 area. • In a similar way apparent motion that we see in a movie theater is another type of filling-in by neural activities in V1 area.

25. Linking brain activity and visual experience Neural correlates of object recognition • In binocular rivalry, activity in the fusiform face area and parahippocampal place area is closely linked to the observer’s awareness of faces and houses. • Other studies deals with visually masked objects which can just barely be recognized. • Mooney face shown in figure can be recognized at right orientation, while it is hard to recognized at different orientations. • If the objects are recognized activity in ventral temporal region is greater, while activity in V1 region shows no difference..

26. Manipulations of visual awareness • To find out causal relations between activities in various brain regions it is useful to directly stimulate the selected brainarea with electrical impulses. • One way is to use implants for instance in V1 area • Another way is to use transcranial magnetic stimulation (TMS) Patients report various experiences including ‘out of body experience’ – seeing its own body from above.

27. Manipulations of visual awareness Unconscious perception • We use the term unconscious perception when subjects report not seeing a stimulus, but their behavior or brain activity suggests that specific information about the unperceived stimulus was indeed processed by the brain. • When two different stimuli are flashed in quick succession, the visual system can no longer separate the two stimuli. • Instead, what people perceive is a mix, or a fused blend of the two images. • They may respond to individual images in various brain areas without being aware of seeing them

28. Manipulations of visual awareness • For instance, a quick presentation of a red square followed by a green square can be perceived as a yellow one. • Presentation of the images of the house or face in complementary colors to different eyes has the same effect of not seeing one. • However, the brain still responds to these unseen patterns – fusiform face area (FFA) to face and parahippocampal place area (PPA) to house.

29. Summary • Vision is our most important sensory modality. • We discussed the functional properties of neurons as visual signals travel up from the retina to the primary visual cortex and onward to higher areas in the dorsal and ventral visual pathways. • Progressing up the visual pathway, receptive fields gradually become larger and respond to more complex stimuli, following the hierarchical organization of the visual system. • V1 supports conscious vision, provides visual features like orientation, motion and binocular disparity. • V4 is important for color perception. • MT is important for motion perception. • Damage to dorsal pathway leads to optic ataxia (neglect). • Damage to ventral temporal cortex leads to impairments in object or face recognition. • In ventral temporal cortex some regions like LOC have general role in object recognition, while others like FFA and PPA are more specialized

30. Why does the primary visual cortex react to oriented edges? Because correlational learning in a natural environment leads to this type of detector. Why does the visual system separate information into the dorsal pathway and the ventral pathway? Because signal transformations extract qualitatively different information, strengthening some contrasts and weakening others. Why does damage to the parietal cortex lead to disorders of spatial orientation and attention (neglect)? Because attention is an emergent property of systems with competition. How do we recognize objects in different locations, orientations, distances, with different images projected on the retina? Thanks to transformations, which create distributed representations based on increasingly complex and spatially invariant features. Some answers