Visual Recognition of Objects, Faces, and Printed Words

1. Visual Recognition of Objects, Faces, and Printed Words The best of Farah, Ch. 5 & 6

2. Are Faces Special? Are these two faces the same or different?

3. WELL!!! Well! • Inversion effect. Face recognition is more orientation sensitive than other types of stimuli • Inversion decreases face recognition accuracy by ~20%; object recognition accuracy decreases by ~2%.

4. Infant face processing • Infants 2 – 3 weeks old can imitate adult mouth movements • Newborn infants (mean age = 43 min) follow face-like patterns longer than non-face like patterns (Johnson et al, 1991) From Bruce & Young (1998)

5. How are faces processed? • By a face processing module? • By a general purpose visual recognition system in which • faces are more sensitive to orientation than other objects and/or • faces are the first type of stimulus that a general purpose system is programmed to process.

6. Criteria for identifying separate processing systems (modules) • Anatomical-Neural Systems: Are objects, faces, and words recognized in different parts of the brain? • Functional independence: Is our ability to recognize objects independent of our ability to recognize faces or words? • Information processing: Do different modules process information in different ways?

7. Anatomical-Neural Systems:The Evidence • “Face” cells in monkeys • Prosopagnosia • Brain damage • Functional neuroimaging

8. Neural systems. “Face” cells • Single unit recording in monkey IT cortex. Up to 20% of cells near superior temporal sulcus (STS) are “face” cells. • Seem to be responsive to configural properties (Configuration: the relative position of elements of a thing, such as a face). From Farah (2000)

9. Neural systems: ProsopagnosiaDifficulty recognizing facesA case study: LH • At 18, severe head injury in car accident and subsequent surgery damaged temporal-occipital (and other) regions. • Great recovery, eventually earned two Masters degrees, works, etc. • Can’t recognize his wife or children, nor can he recognize himself in a group photo or mirror • Rarely has problems recognizing objects • Is not dyslexic

10. A case study: LH (Farah et al, 1995)Is LH’s ability to recognize faces disproportionately impaired? • Study photos of objects and faces • 2-choice recognition test • Recognition accuracy Faces Objects LH 62% < 92% Normals 86% = 85% From Farah ( 2000)

11. LH and 8 normal subjects tested in a sequential matching task; 300 trials: 150 upright, 150 inverted. See unfamiliar face 1.5 s Brief ISI See unfamiliar face Make same/different judgment Percent Correct Up Invert LH 58% < 72% Controls 94% > 82% LH showed an inverted inversion effect. Demonstrates obligatory use of his damaged face processing system LH and the inversion effect Farah, Levinson, & Klein (1995)

12. Neural Systems: Brain damage • Had thought that prosopagonsia required bilateral damage. But, recent studies reveal that unilateral right hemisphere lesions lead to prosopagnosia. • Wide range of areas in occipital and ventral temporal areas involved in face processing, with the right hemisphere playing a dominant role. • Impaired object processing, but spared face recognition, more likely to be observed with left hemisphere damage.

13. Neural systems. Functional neuroimaging Kanwisher et al (1997): fMRI study • Subtraction procedure: • Passive viewing of: • intact faces vs. objects • intact faces vs. scrambled faces • front-view faces vs. front-view houses • ¾ view faces vs. human hands • Consecutive matching of ¾ view faces vs. hands Both objects and faces activated inferior temporo-occipital regions, with face activation confined to the right fusiform gyrus. • FFA: Fusiform Face Area (just behind the ear)

14. Kanwisher et al: Sample data. • 1st column: Sample stimuli • 2nd column: Activation in ROI (green box) S1 • 3rd column: Mean percent signal change in ROI (5 subjects).

15. Humans: fMRI (“BloodOxygenLevelDependent”) imaging Fusiform gyrus face area (Haxby et al. 2000 Trends Cog Sci)

16. Functional independence: Dissociating face and object perception in patients Patient CK: Is alexic and agnosic for everything but faces. Although he could identify the Archimbaldo painting as a face, he could not identify objects that made up the face. Patient WJ: Sheep farmer who had prosopagnosia for human faces could still recognize sheep (e.g., his problem was restricted to human faces)

17. Functional properties: How are face and object recognition systems related? • (a) All stimuli are processed first by one general system, and faces receive further processing. • (b) Faces and objects are processed by independent systems operating in parallel.

18. Functional differences between face and object recognition • Humans are experts at human face recognition. • Holistic face representation? • Diamond & Carey (1986): The inversion effect in dog experts From Diamond & Carey (1986)

19. Inversion effect • First-order relational information: How the parts of an object or face relate to one another. • Second-order relational information: How the spatial relations of the parts (i.e., first-order relations) compare to the prototypical arrangement of those parts. • Inversion disrupts the processing of second-order relational information, but not first-order relational information. • Inversion disrupts processing of faces more than processing of other objects. • Faces have strong configural properties and are processed holistically.

20. Are faces processed holistically or are they decomposed like other objects? Tanaka & Farah (1993) • Participants trained to identify faces and houses • 2-choice recognition test on parts in isolation and in whole object. • Faces: Parts recognized more poorly in isolation than in whole (.66 < .77). • Houses: Equal recognition of parts in isolation and in whole (.80 = .78). From Farah (2000)

21. Holistic processing: A function of expertise? • People are experts at face recognition. They have been making these within-category (subordinate) identifications since birth. • People with expertise in other domains (e.g., birds, cars, dogs) process exemplars from those domains holistically and the FFA is activated when processing these stimuli. • All evidence presented to date is correlational. A study that compares the performance of participants randomly assigned to become experts in some domain and novices is needed to establish a causal relationship between expertise and holistic processing.

22. Experimental studies of expertise: Becoming a Greeble expert From Farah (2000)

23. Gauthier & Tarr (1997, 2002) • Students trained to categorize 30 Greebles at the family (5 families) and individual level until they could do both equally fast. • Tested recognition of parts in isolation and in the whole Greeble using same/different task: A typical trial: Fixation cross (500 ms), study Greeble (1500 ms), pattern mask (200 ms), cue (name of target part, 100 ms), test Greeble or isolated part, same-different response.

24. Gauthier & Tarr (1997, 2002) • Parts recognized better in the context of the whole Greeble than in isolation for both novices and experts. • (2002). Activation in FFA correlated with obligatory processing of all features (holistic-inclusive processing).

25. Gauthier & Tarr (1997, 2002) • They suggest: “An emerging hypothesis is that the neural specificity seen for face processing is due to the particular constraints of humans’ extensive experience with faces… A correlate is that given similar constraints, nonface stimuli will recruit the same neural substrate. In other words, the FFA may not be specific for faces per se, but rather only for the operations we typically, and by default, perform when perceiving faces” (p. 431, 2002).

26. Is there a module dedicated to the processing of faces? • Currently, a hotly debated question. The evidence is not clear. Much more research is needed. • Farah and colleagues suggest that faces are processed in a more holistic fashion than objects. • Gauthier and colleagues suggest that this type of holistic processing applies to all expert subordinate categorization of homogeneous stimuli. • Whoever is right, faces are important. Faces convey information about personal identity, age, sex, gaze direction, and emotion, information that is critical to social beings.

27. What about printed words? • We cannot argue for built-in text processing module: • Reading is evolutionarily recent. • In contrast to the acquisition of spoken language, reading is taught. • However, we can ask “How could the brain develop a localized, dedicated system for such an arbitrary and evolutionary recent category of stimuli?” (p. 148). From Gazzaniga, Irvy, & Mangun (1998)

28. Focus on Pure Alexia • Impaired visual word recognition, despite intact auditory word comprehension, and the ability to “write” and to recognize visual patterns other than words. • Slow letter-by-letter reading From Farah (2000)

29. Explanations of Pure Alexia.H1: Disconnection Hypothesis • Reading involves associating visual information in the visual cortex with language representations in the posterior language areas. Alexia reflects damage to that connection. • Although the neuropathology is consistent with this hypothesis, it cannot explain letter-by-letter reading.

30. Explanations of Pure Alexia.H2: Visual Impairment Hypothesis • Simultanagnosia: Disorder of simultaneous form perception. • Alexia due to impairment in the rapid visual perception of multiple objects or multipart objects (most noticeable during reading). • Evidence: • Some impairments in visual processing of line drawings, simple letter matching, naming letters in isolation • Increased difficulty when required to process multiple letters (even in a simple matching task) • Word length effect exacerbated by visual noise, suggesting a perceptual effect

31. The visual impairment hypothesis: A computational interpretation • Multishape recognition problems due to interference among representations of separate shapes. • The more distributed the representation, the greater the interference when two or more shapes must be represented simultaneously. • Alexics have lost cortex that supports localist shape representations (minimizes interference), which are important for recognition of multishape objects, especially words.

32. Explanations of Pure Alexia. H3: Orthography-specific (Word form) Impairment Hypothesis • Alexia due to loss of high-level orthographic representations of word forms (morphemes and words). • Alternatively, input from the letter recognition system to the word form system is limited to one letter at a time. From Gazzaniga, Ivry & Mangun (1998)

33. Word Form Hypothesis cont. • Positive evidence Region of extrastriate visual cortex specialized for processing of words and orthographically legal pseudo-words in contrast to nonletter strings of shapes. • Negative Evidence Some alexics show a word superiority effect (WSE). From Gazzaniga, Irvy & Mangun (1998)

34. Word Superiority Effect (WSE). Letters recognized better when seen in a word than when seen alone or in a nonword string. Standard trial See letter string: W O R D _ _ _ D O R W D See pattern mask: See 2-afc test: _ _ _ D _ _ _ K What is the Word Superiority Effect?

35. Explaining the Word Superiority Effect

36. Goal: To simulate word processing using a single structure and set of parameters • Interactive Activation Model: • Three processing levels (feature, letter, word) • Connections within levels are inhibitory • Connections between levels are excitatory and inhibitory • Basic Assumptions: • Perception occurs in a multilevel system • Deeper levels of processing are accessed via intermediate levels • Processing is interactive • Information flow is continuous McClelland & Rumelhart (1981)

37. IA: The unit for the letter T in the first position of a four-letter array and some of its neighbors McClelland & Rumelhart (1981)

38. IA Model, WSE, Pure Alexia and H3: The Word Form Hypothesis • Pure alexics who abandon letter-by-letter reading show WSE. Suggests word form system intact for at least some pure alexics; consistent with prelexical visual impairment. • Independent evidence from pure alexics who do not show a WSE also suggests general visual impairment.

39. Explanations for Pure Alexia. H4: General visual impairment most severe for orthography • A hybrid hypothesis stating that pure alexia results from a general visual impairment which affects the perception of multiple shapes, especially for letters. • Evidence: • Neuroimaging data showing cortical areas specialized for processing word-like stimuli • Patient data showing intact number recognition • Artificial neural network research: self organization of letter area and abstract letter identities.

40. Self-organizing artificial neural networks • Strength of connections among units changes to reflect the pattern of activity levels of the units according to Hebb rule: “Neurons that fire together wire together” • Creates “letter area” • Self “abstract letter identities”

41. LETTER-DIGIT FUNCTIONAL LOCALIZATION Polk, T.A. & Farah, M.J. (1995). Brain localization for arbitrary stimulus categories: A simple account based on Hebbian learning. Brain damaged patients can show selective impairments in the processing of music relative to other sounds, in writing relative to other sensory motor functions of the hand, and so on. This simulation examines the spatial segregation of letter and digits.

42. Polk & Farah (1995): Neural Network Simulation (Letter-digit functional localization) State of the neural net after training under varying initial conditions. For each pair: activation when all 8 letters are present (left) and all 8 digits are present (right).

43. How does it work? • Architecture designed to provide: • COOPERATIVE MECHANISM to produce clusters of activity • COMPETITIVE MECHANISM to inhibit multiple clusters of activity • Initial presentation of a stimulus “A” • Pattern of activity in output layer random, reflecting random weight of connections between input and output layers. Over time excitatory connections will produce clusters around active units and these will drive down activity elsewhere via inhibitory connections, leading to one (or very few) clusters of activity. • Hebb rule strengthens connection between “A” and the active output units and weakens connections from other inactive input units. “A” will subsequently be biased toward activating the same cluster, while other input units will activate other output units. • Initial presentation of a digit “6” • Same process as above, except that the units activated by “A” will be less active to the digit. The beginning of segregation/localization.

44. Letter “Pop out”: Some behavioral data on the alphanumeric category effect • Tested: • Canadian postal workers • US postal workers • US college graduates • A visual search task: • finding a target letter among other letters • Finding a target letter among digits Polk & Farah (1998)

45. Find the Letter “T” L L L L L L L L L L L L L L L L L L L L L L L L L L L L L L L L L L L L L L T L L L L L L L L L L L L L L L Find the Letter “T” 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 T 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 7 Visual Search: Pop-out

46. How do dedicated neural systems arise? • Evolution. Only abilities that evolved can be carried out by genetically specialized brain regions. • Learning. Learning modifies the brain (alters connections between individual neurons). Can learning create functionally specialized brain areas? • Pure alexia: “clearest evidence so far available for localization of a category of knowledge that is fundamentally arbitrary, lacks an evolutionary history, and is learned relatively late in life” (p. 173).

47. Current “Best Story”: Two classes of processing apply differentially to faces, objects, and printed words. • Objects processed by parts and holistically • Faces processed holistically • Printed words processed in parts From Gazzaniga, Ivry, & Mangun (1998)