Chapter 10 Visual Awareness

1. Chapter 10Visual Awareness The neural bases of visual awareness as revealed by dissociations between vision and awareness of vision

2. Conscious Product (content) of seeing Text Faces Attended input Activated memory Not Conscious Process of seeing How we read How we see faces Unattended input Non-activated memory Visual Awareness: Conscious of What?

3. Conscious Visual Awareness • Subjective Experience: • Qualia (redness, roundness) • P(henomenal)-consciousness (experiential state) • “What it feels like” to be a bat or a person • Behaviour: • We can act on available visual information intentionally. • We can communicate about available visual information.

4. Why are we conscious? • Consciousness may provide some adaptive advantage. • Frogs. Reflexive responses (Zoombie mode) to: -- small round moving objects -- looming objects - Species with many systems (reflexive and other) that could be in conflict. Combine outputs of systems into a higher order complex representation and make it available to the relevant portions of the brain for a sufficient amount of time to support decision making and action.

5. Some Potential Functions of Consciousness • Coping with novelty and unpredictability through learning, adapting, and problem solving • Prioritizing information • Controlling mental and physical actions • Making decisions

6. Neural Correlates of Visual Awareness • Six (five) syndromes with evidence of preserved visual function and impaired visual awareness: • Blind sight • Agnosia: Apperceptive and Associative • Prosopagnosia • Neglect & Extinction • Reading in Pure Alexia (awareness not impaired) • Identifying the mechanism of awareness “Brain function of people with normal awareness” – “Brain function of people with impaired awareness” “Brain mechanism of awareness”

7. Neural Correlates: Three Different Accounts • Privileged Role • Schacter: Dissociable Interactions and Conscious Experience Model (DICE) • Gazzaniga: Left-Hemisphere Interpretive Mechanisms • Integration • Kinsbourne: Integrated Field Theory • Crick & Koch: Binding Model • Damasio: Convergence Zones • Quality of Representation • Farah et al: Degree of Impairment in Visual Perception

8. Consciousness: The Privileged Role of Particular Brain Systems • Descartes (1600s): Patterns of brain activity impinging upon pineal gland were consciously experienced. • Schacter (late 1980s): Dissociable Interactions and Conscious Experience (DICE) model • Conscious Awareness System (CAS): A brain system separate from perception and cognition that gives rise to conscious awareness. • Loss of awareness reflects a disconnection between the visual system and CAS. • Gazzaniga (1980s): Left-Hemisphere Interpretation Mechanism.

9. Schacter, 1989

10. Roser & Gazzaniga (2005, p. 58) “The left-hemisphere interpreter may be responsible for our feeling that our conscious experience is unified. Generation of explanations about our perceptions, memories, and actions, and the relationships among them, leads to the construction of a personal narrative that ties together elements of our conscious experience into a coherent whole. The constructive nature of our consciousness is not apparent to us. The action of an interpretive system becomes observable only when the system can be tricked into making obvious errors by forcing it to work with an impoverished set of inputs, such as in the split brain or in lesion patients. But even in the damaged brain, this system lets us feel like ‘us’.” “It is becoming increasingly clear that consciousness involves disunited processes that are integrated in a dynamic context. It is assembled on the fly, as our brains respond to constantly changing inputs, calculate potential courses of action, and execute responses. “

11. Consciousness: A State of Integration among Distinct Brain Systems Dynamic states of brain systems • Kinsbourne (1988): Integrated Field Theory. • Awareness as a state of the brain wherein all modality-specific perceptions, memories, current actions and action plans are mutually consistent. • Vision without awareness: “disconnection of or damage to the visual system” prevents vision from participating in integrated patterns of activity across the brain. • Crick & Koch (1990s): Binding Model. • Visual awareness arises when visual properties (e.g., color) of stimulus are bound together via synchronized oscillations • Damasio (1990s): Convergence Zones. • Binding operates across, as well as within, modality-specific representations of an object.

12. A Neural Correlate of Consciousness • Crick & Koch (1990) • “Our basic idea is that consciousness depends crucially on some form of rather short-term memory and also on some form of serial attentional mechanism. This attentional mechanism helps sets of relevant neurons to fire in a coherent semi-oscillatory way, probably at a frequency in the 40-70 Hz range, so that a temporary global unity is imposed on neurons in many different parts of the brain. These oscillations then activate short-term (working) memory.” [p. 277 of Block et al (Eds., 1997]

13. Consciousness: A Graded Property (Quality) of Neural Information Processing • Representation of information in neural systems is graded, not all-or-none. • Information may be partially represented: • Normal subjects: Using impoverished input (e.g., Marcel’s studies of subliminal perception and masking studies) • Patients: Damage to the visual network. • Farah, O’Reilly & Vecera (1993). Impairment in visual perception model. • Quality of perceptual representation correlated with probability of conscious awareness • Consciousness may be associated only with high quality representations.

14. Accuracy in Experiment 1 (1983a). Semantic similarity > Graphic Similarity > Detection. (1983a). Five experiments challenge the view that “representations yielded by perceptual analysis are identical to and directly reflected by phenomenal percepts.” (p. 197) (1983b). In other words, conscious perception is constructed from perceptual representations; it is not “identical to or a direct reflection of representations yielded by perceptual processes.” (p. 238) A typical trial (1983a) Present fixation point Present word briefly (too short to be aware of seeing it) Present pattern mask Participant makes decision about: (a) presence/absence of word (b) graphic shape or meaning of the presented word. For these decisions, the subject selected the item that was most similar to the target word from a pair of test words. Conscious and unconscious perception (Marcel, 1983a, b)

15. The Six Syndromes… • Covered by Rita Anderson • Prosopagnosia • Pure Alexia (in passing – not relevant) • Covered by Carolyn Harley • Blindsight • Agnosias • Neglect & Extinction

16. Tranel & Damasio (1985). Task: Rate familiarity of faces of family (herself, family, friends), famous people (actors, politicians) interspersed with unfamiliar faces. Despite inability to recognize familiar faces, two females with prosopagnoia generated larger and more frequent skin conductance responses (SCR) to familiar than to unfamiliar faces. Demonstrates total dissociation between overt and covert face recognition, despite normal visual perception and associated memories. S1 Mean Rated familiarity (1=very familiar, 6 = very unfamiliar) Family 6.0 Famous 6.0 Unfamiliar 6.0 SCR (mean amplitude) Family .934 Nonfamily .048 Famous .731 Nonfamous .012 Prosopagnosia: Evidence for implicit (covert) face recognition

17. Prosopagnosia: A case study of face recognition without awareness de Haan, Young, & Newcombe (1987) • PH: Unable to recognize familiar faces visually, but can recognize them from their names. Knows that he is looking at a face, can comment on age, sex, hairstyle. Performance on the following three tasks similar to normal people, but he feels like he is guessing. • Matching Task(Identity): See two photos, Same/Different person?). PH is slower than normal and makes more errors, but he matched familiar faces faster than unfamiliar faces (same pattern for controls) Error Rate Mean RT (ms) Familiar Unfamiliar Familiar Unfamiliar • PH 18.7% 16.4% 2550 2762 • Controls 1.6% 3.9% 977 1045

18. PH case study cont. • Interference Task. Classify name as pop star or politician when name presented alone or accompanied by a photo. • RT slowest when name and photo are unrelated with respect to occupation than in other conditions. PH must be processing facial identity to show this occupational interference effect. • Learning task. Learn correct/incorrect pairings of names/faces from before and after the accident. • PH learned correct name/face pairings faster than incorrect pairings, suggesting that he can make relatively fine perceptual face discriminations. Same person 1502 ms Unrelated 1714 ms Related 1560 ms From deHaan et al (1987)

19. Three types of priming conditions (using faces or names) for target names Related: John Lennon (prime) for Paul McCartney (target name) Neutral: Unfamiliar person Unrelated: Two familiar people not related to one another. Task: Classify name as familiar or not; measure RT Typical trial Fixation See Prime (450 ms for PH, 250ms for controls) See Printed Name Mean RT (ms) to familiar and unfamiliar target names preceded by face or name primes E4 E1 Familiar PH Control Related 981 695 Neutral 1056 776 Unrelated 1083 815 Unfamiliar 1373 883 Semantic priming: PH and Controls Young, Hellawell & de Haan (1988) PH slower, but shows facilitation from related face or word primes

20. What type of model can best account for the basic results? • Basic Result • Dissociation between overt and covert recognition of faces in some (but not all) prosopagnosics • Privileged role and integration models assume that face processing is normal and that conscious access is post-perceptual. Privileged role • de Haan, Bauer & Greve (1992): Face recognition units (FRUs) disconnected from Conscious Awareness System (CAS) (p. 308). • Bauer (1983). Ventral system (mediates identity and conscious awareness) damaged; dorsal system (mediates affective response to faces) supports covert recognition (pp. 307-308). Integration • Burton et al (1991): Partial disconnection of face recognition units (FRUs) from personal identity units, the major pathway to the rest of the system (p. 309).

21. Privileged Role account: de Haan, Bauer & Greve (1992) • Face processing module operates normally and hence, supports covert recognition of faces. • Overt recognition of faces fails because the output of the face processing module cannot access awareness due to a lesion at Location 1. From Farah (2000)

22. Integration Account. Burton et al (1991) Interactive Activation and Competition (IAC) model of face recognition • Pools of units correspond to: • FRU: Face recognition units • NRU: Name recognition units • SIU: Semantic information (about the person) units • PIN: Person identity node (amodal interface between input and semantic information) • If reduce connection strength between FRUs and PINs sufficiently, activating a FRU will not activate the PIN beyond threshold and overt face recognition will not occur. But, the PIN is activated above resting level and can hence, support implicit effects (relearning, priming, etc). From Farah (2000)

23. Quality of Representation: Farah, O’Reilly & Vecera (1993). A simulation to test whether dissociation effects in prosopagnosia could be due to an impairment in visual recognition • Both Privileged Role and Integration accounts assume that face recognition proceeds normally. Does it? • PH makes more errors and is much slower than control subjects. No empirical data exist to determine whether the face module in prosopagnosia patients is normal. • Is it necessary to postulate a separate mechanism for conscious awareness? • If covert recognition tests are more sensitive than overt recognition tests, a person with a degraded representation that is below the threshold of awareness might show some residual recognition when tested implicitly. • No empirical data are available. Can test the viability of the Quality of Representation hypothesis by running a simulation.

24. Farah et al (1993) The Neural Network Model • Input Unit Pools; Hidden Unit Pools • 16 Face units; 16 Name units • Semantic Pool • 18 units, one either politician or artist Distributed representation Faces (Names): random pattern of 5 units activated Semantic: random activation of 6 units (one occupational) Network trained to associate an individual’s face and name (via semantic units) using a Hebbian rule (i.e., units that fire together wire together). Learned 40 distinct individuals of which 10 were actors and 10 politicians

25. Farah et al (1993): Simulation 1a Overt Recognition • Lesion network by eliminating randomly chosen units from face input (or hidden) unit pools. Seven levels of lesion created by removing 2, 4, 8, 10, 12, or 14 units from the 16 unit pools. • Overt Recognition: Percent correct name identification of faces in a 10-afc test (Chance = 10%) • Overt recognition significantly above chance until more than 50% of units removed.

26. Farah et al (1993): Simulation 1b Savings in Relearning Savings in Relearning Face-Name Associations • The lesioned network was retrained on the names and faces of 10 familiar actors and politicians paired correctly or incorrectly. • Despite chance performance on overt test (at 75% lesion), the system relearned correct pairings faster than incorrect pairings. This is similar to PH results. • Why? System could build on residual knowledge to relearn the familiar pairings.

27. Farah et al (1993)Simulation 2: Perceptual Speed • Presented 10 familiar and 10 unfamiliar faces (half actors and half politicians) to the lesioned network. • Measured time for the network to settle (assesses speed of processing). • Settling time faster for familiar than unfamiliar faces, even at lesion levels that yield chance overt recognition. This is similar to pattern seen for PH when matching faces.

28. Farah et al (1993): Simulation 3Interference Effects • Primed the lesioned network with faces of 5 familiar actors and 5 politicians. The prime face had the same occupation as the target name, a different occupation, or the name was presented alone. • Task: Identify the occupation of the target name. • Number of cycles for the occupation unit to become active was higher when the face and name were from a different occupational category at all but the most extreme levels of damage. • As in PH, the effect is largely interference.

29. Farah et al (1993)Conclusions • Impaired visual representations can lead to failure in tests of overt recognition; residual knowledge can support covert (implicit) recognition. • Simulations demonstrate patterns of dissociations between overt and covert recognition similar to those seen in studies of prosopagnosia patients. • Prosopagnosia patients who show a dissociation and those who do not may differ in the severity of visual impairment

30. Dissociations in Pure Alexia • Clinical tests: Pure alexics only understand words they have read letter-by-letter. • Yet, pure alexics show evidence of implicit reading. They can make some types of decisions about words (especially concrete, high frequency words) presented too briefly to be read using a letter-by-letter strategy: • lexical decisions (is it a word?); semantic judgments (is it living or nonliving?l) But not: • morphological (“appluadly) or rhyme/nonrhyme (does it rhyme with x?) decisions • Understanding a presented word is dissociated from knowing about its lexical status and semantic category [similar to students in the Marcel (1983) studies]. • Evidence is consistent with right-hemisphere mediation of implicit reading. • This type of dissociation does not have direct implications for the relationship between word perception and visual awareness. • Hence, evidence from pure alexia is not likely to be relevant to finding the neural correlates of visual awareness.

31. Absence of visual awareness is revealed differently within and across the different syndromes • Blindsight patients: • Claim to be guessing, have no subjective experience • May report a non-visual experience • May report a visual experience • Apperceptive patient: • Has no awareness of size & orientation of objects that she can grasp • Prosopagnosia patient: • Has no sense of familiarity when viewing a face, low confidence in face identifications • Neglect & Extinction patients: • Are unaware of having seen anything at all • Alexia patients • Have a dissociation between ability to report a specific word and the ability to make judgments about its lexical status or semantic category, but no dissociation between word perception and awareness of that perception.

32. Tallying up the evidence • Privileged Role • Blindsight (disconnection of cortical visual system) • Apperceptive Agnosia (ventral stream disconnection) • Integration • Blindsight, Associative Agnosia, Prosopagnosia, Neglect & Extinction (perceptual impairments interfere with integration) • Quality of Representation • Blindsight (limited input from LGN to extrastriate visual cortex) • Associative Agnosia (degraded object representations) • Prosopagnosia (perceptual impairments) • Neglect & Extinction (perceptual impairments)

33. General conclusions regarding the vision-awareness dissociation • A hybrid account of awareness. • Quality of representation is critical. • Poor quality representations cannot play a privileged role in awareness, nor can they be integrated into global awareness. • There is no empirical support for a dedicated conscious awareness system. • The mind and brain are complex. It is unrealistic to expect that there is one correct (simple) solution.

34. Some final comments • Course Objective: To provide a foundation for the third year psychology courses showing how knowledge of cognition and neuroscience interacts to help us understand mind and behavior. • Cognition serves to delineate the mental and physical dimensions of the problem of interest. • Neuroscience probes the brain in the search for neural correlates (mechanisms) that will guide future theoretical development. • Today, virtually all substantive areas in psychology involve some cognitive exploration of the mental and physical behavior in question and most are using or exploring the utility of functional neuroimaging or other brain studies.