The Development and Neural Bases of Face Recognition

1. The Development and Neural Bases of Face Recognition Charles A. Nelson, Ph.D.1,2,3 University of Minnesota 1 Institute of Child Development 2 Department of Pediatrics 3 Center for Neurobehavioral Development

2. Adult Models of Face Processing • Kanwisher • Suggests that the ventral temporal cortex contains a limited number of separable areas that are specialized for representing specific categories of stimuli (faces or objects) • Haxby • Neural representations of faces and different categories of objects are widely distributed and overlapping (not one area devoted to processing faces and objects) • Gauthier • Activation of the fusiform area is not specific to faces but to any category of stimulus in which expertise has been obtained. • Can studying development help us resolve this debate? Changeable aspects Identity

3. Why is the study of the development of face processing interesting? • Prior to the onset of language (0-2 years), most communication is non-verbal; thus accurately decoding facial signals is important • e.g., facial identity (e.g., who is this?); facial expression (e.g., what is this person feeling?) How does this develop? • Even with language, attending to the face occurs naturally (thought experiment: carry on a conversation with someone without looking at the face). Why? • Speculation that it may be adaptive (from reproductive fitness perspective) for young infant to recognize caregiver

4. Why is the study of the development of face processing interesting? (Con’t) • Processing some faces (e.g., familiar ones) can be selectively knocked out or perturbed in various disorders (e.g., prosopagnosia, autism), suggesting to some face recognition is “hard wired.” Only studies of development can address this hypothesis. Similarly… • May provide a model system for teasing apart experiential from nonexperiential effects. • For example… Debate continues over whether face recognition represents • a domain specific “module” that is innate and does not develop or • an experience-expectant and activity-dependent ability that does develop • Former would argue that neural architecture for face recognition would be present very early in life (which I will demonstrate is true); • Later would suggest remodeling of this architecture over months and years (which I will also demonstrate is true).

5. My charge • Review the available evidence, both behavioral and neural, which describes the development of face processing. • Will start with critique of newborn work, then move to behavioral work with older infants and children • Then turn to developmental cognitive neuroscience work • Will then provide examples of three prominent models of development of face processing • Johnson • deSchonen • Nelson

6. Behavioral Studies - Newborns • History: Bowlby (e.g., 1969) argued that would be adaptive (from reproductive fitness perspective) for newborns to recognize caregiver; thus, newborns should show visual preference for face vs. non-face (mother in particular) • Archival Data: Some studies (e.g., Goren, Sarty, & Wu, 1975) supported Bowlby’s prediction - newborns would look longer at moving schematic face vs. moving schematic non-face. Others did not (e.g., Hershenson, 1965; Thomas, 1965). All were plagued by methodological problems. • Modern Data: Johnson, Slater, Simion, etc. have demonstrated that newborns show visual preference for face-like stimuli vs. non faces.

7. Newborn Studies, Con’t • BUT, what are infants really preferring (i.e., is it “facedness” per se or something else)? • Simion, Macchi-Cassia and colleagues (2002, 2003):a stimulus with an oval perimeter and internal features situated in the upper half of the oval are preferred over the same oval with features situated in the lower half; moreover, these features do not need to resemble faces, they can also simply be dot patterns. > -Johnson & Morton, 1991 -Valenxa, Simion, Macchi Cassia, & Umilta, 1996

8. > > > Simion, Valenza, Macchi Cassia, Turati,& Umilta, in press • Newborns prefer geometrical, non-facelike stimuli with more elements in the upper part over stimuli in which more elements are in the lower part.

9. …and of course, these effects also obtain with “real” faces...

10. How Robust is this effect? • Caldara et al. (2003) presented these same stimuli to adults; reported that the head-shaped patterns with a greater number of elements in the upper visual field showed greater fusiform activation than other stimuli • Conclusion: newborn preference for face-like stimuli probably not driven by “facedness” per se, as much as arrangement of perceptual elements, which in turn influenced by tuning properties and limitations of newborn visual system.

11. Behavioral Studies – Older Infants • Archival Data: Fagan (1972) demonstrated that around 4 months, infants begin to show facial inversion effect, pointing to development of face “schema.” • Modern Data: is rapid improvement in face processing skills >4-6 months. Examples: Infants <1 year can categorize facial gender, some facial expressions. However, no evidence of categorizing negative facial expressions till 1-2 years; furthermore, recognition of negative facial expressions still lags behind adult levels through (at least) middle childhood.

12. Neural Bases of Face Recognition • Relatively new field of inquiry. Why? • Most developmental psychologists less interested in the neurobiological bases of behavioral development than in describing changes in behavior across time. • Tools to study development of neural bases more limited than what is available to study adults • Can’t use PET (see French [Mazoyer] exception) • Can’t easily use fMRI <5 (or so) years (see French [Dehaene] exception) • Infants and young children typically do not suffer from the kinds of infarcts/injuries that afflict adults and that selectively knock out face circuits; thus, use of classic “lesion” approach limited.

13. Neuropsychological Studies • de Schonen and her colleagues have demonstrated that 4- to 9-month-old infants show a right hemisphere (left visual field; LVF) bias towards processing faces, similar to what is observed in the adult (i.e., is long history of research using divided field presentations, and more recent fMRI studies, point to right>left hemisphere specialization)

14. Neuroimaging Studies: PET • Neurologically compromised 2-month-old infants were studied using PET* • Infants were presented with faces or flashing red and green diodes. • A broad array of areas were activated to faces that seemingly had little to do with face processing per se (e.g., left superior temporal and inferior frontal gyri). • However, activity was also observed in the fusiform gyrus, similar to what has been observed in adults (see next slide) *Mazoyer, N., de Schonen, S., Quinton, O., Crivello, F., Reutter, B., & Mazoyer, B. (1999). NeuroImage, 9, S346.

16. Neuroimaging Studies: Event-Related Potentials • de Haan & Nelson* used event-related potentials (ERPs) to examine discrimination of familiar and novel faces and objects in 6-month-old infants: • Clear ERP differences when mother contrasted to stranger, specifically • NC (see next slide) larger to mother vs. stranger • Greater ERP activity was observed at the right vs. left temporal scalp, consistent with de Schonen observation (and adult data). • Observed a component over occipital scalp that had a shorter latency to faces than to objects, leading to speculation that this component (P400) may be a) a precursor to the adult N170 (see next slide) and b) specific to faces *de Haan, M., & Nelson, C.A. (1997). Child Development, 68, 187-210. de Haan, M. & Nelson, C.A. (1999). Developmental Psychology, 35, 1113-1121

18. N170 and P400 “Face” Components

19. What about differences in face vs. object processing? • Difficult studies to do with infants, as all the relevant control conditions are hard to implement (see Gauthier & Nelson, 2001* for discussion). • Caveat notwithstanding…. * Gauthier, I., & Nelson, C.A. (2001). The development of face expertise. Current Opinions in Neurobiology,11, 219-224.

20. Neural correlates of preferential-looking in 6-month olds(Kelly Snyder, in preparation)

21. Experimental Design Familiar Novel Test Phase (Preferential-looking) • Encoding Phase (ERPs) Familiar Novel Snyder (in preparation).

22. NP FP FP FP NP NP Memory Effects Effect: Novelty preferences associated with a more negative Nc (but only for face stimuli), possibly reflecting greater attention to the stimulus during encoding. * NP < FP 10 5 0 -5 -10 250 500 750 1000 1250 1500 Snyder (in preparation).

23. Memory Effects FP NP FP NP Effect: Novelty preferences associated with a reduction in the amplitude of the slow wave (but only for face stimuli), possibly reflecting a more fully encoded stimulus (one that requires less updating) * NP < FP 10 5 0 -5 -10 250 500 750 1000 1250 1500 Snyder (in preparation).

24. FP FP NP FP NP NP Memory Effects Effect: Novelty Preferences associated with a reduction in amplitude of several components over occipital scalp regions (but only for object stimuli), possibly reflecting the habituation of visual perceptual responses. * NP < FP 10 5 0 -5 -10 250 500 750 1000 1250 * NP < FP 10 5 0 -5 -10 750 1000 1250 1500 500 250

25. Summary & Conclusions • Is clear association between stimulus encoding (as inferred by ERPs) and recognition memory (as inferred from looking time) • Are clear differences in face vs. object processing by 6 months, both behaviorally (e.g., infants look longer at faces) and electrophysiologically. • Points to early specialization of this ability • …but how developed is this function and how specific is it?

26. The Other-Species Effect • Pascalis & Bachevalier (1998) report that both non-human primates and humans are better at recognizing faces of their own species. • Monkeys look longer at novel monkey faces but not human faces and humans look longer at novel human faces but not monkey faces.

27. The Other-Species Effect • Pascalis, de Haan & Nelson (2002)* directly tested theory of perceptual narrowing** using the Visual Paired Comparison procedure in 6 and 9 month old infants and adults. • Results indicate that adults and 9 month olds looked longer at novel human faces, but looked equally long at monkey novel and familiar faces. • In contrast, 6-month-olds showed novelty preferences in both the human and the monkey conditions. • Thus, perceptual window through which faces are viewed initially tuned broadly, then narrows with experience (accounting for why 6 month olds are “better” at discriminating monkey faces than are 9 month olds and adults) • *Pascalis, O., de Haan, M., & Nelson, C.A. (2002). Science, 296, 1321-1323. • ** Nelson, C.A. (2001). Infant and Child Development, 10, 3-18

28. The Other-Species Effect • deHaan and Johnson* studied adults and 6-month-olds in a human and monkey face recognition task while they recorded ERPs. • Would adults and infants show the same cortical specificity during face processing? • Adults and infants were shown both monkey and human faces in upright and inverted orientations. * de Haan, M., Pascalis, O., & Johnson, M.J. (2002). Journal of Cognitive Neuroscience,14, 199-209.

29. Other-Species Effect • In adults, all stimuli evoked an N170 over temporal and occipital leads. • This N170 was larger in amplitude and longer in latency for upright monkey compared to upright human faces. • Inversion effects were apparent in the human but not the monkey conditions (increased amplitude and latency to inverted faces).

30. The Other-Species Effect • No component of the infant ERP showed the same specificity as the adult N170. • 6-month-olds did show sensitivity to both inversion and species but it was distributed across two components. • Nc (species), P400 (orientation)

31. 3-month-olds did not show the same specificity for human faces that 12 month olds and adults exhibit. • 12 month olds exhibited a negative component (N290) that was larger in amplitude and longer in latency for human than monkey faces • This same component showed an inversion effect (larger amplitude to inverted human faces only). • The P400 showed longer latency to human and inverted faces. • 3-month olds did not show the same specificity for human faces. Halit et al., in press

32. Summary • Both the N290 and the P400 seem like reasonable candidates for the adult face-specific N170….although much more work remains to be done.

33. Scott, Shannon & Nelson, in preparation Infant ERP Task Adult ERP and Behavioral Study

34. Scott, Shannon, & Nelson, in preparation • Adult Study: • Adults performed significantly better on human compared to monkey task. • P2 amplitude differentiated species; P2 latency differentiated orientation; N300/400 differentiated familiarity. • Infant Study (9-month-olds):

35. T6 O2 T6 O2 LEFT HEMISPHERE RIGHT HEMISPHERE T5 O1 T5 O1 Frontal Profile Human Monkey Familiar Unfamiliar T6 O1 O2 T5 P400 N290

36. Summary • Infants’ ERPs reflect processing differences along all three dimensions – species, orientation, and familiarity. • Infants’ ERPs are more defined in the right hemisphere compared to the left hemisphere, indicating cortical specialization by 9 months.

37. Summary of Other Species Effect • Advantage in processing faces from same species develops over the first year of life • Typical development characterized by the loss of ability • Presumably experience drives perceptual expertise that is acquired towards processing faces from same species. • In theory, exposure to faces from other species would keep perceptual window open (work ongoing by Pascalis/Scott, Shannon & Nelson)

38. Summary • The neural substrate for face recognition is clearly not fully developed within the first year of life. • It appears that this development is activity-dependent and experience-driven.

39. What Develops Beyond Infancy? • Carver et al. (2003)*: • Presented familiar (mother) and novel faces and familiar and novel objects to 18-54 month old typically developing children • 18-24 months • 24-45 months • 45-54 months • Recorded ERPs from 64 channels • * Carver, L.J., Dawson, G., Panagiotides, H., Meltzoff, A.N., McPartland, J., Gray, J., & Munson, J. (2002). Developmental Psychobiology, 42, 148-159.

40. Face Processing

41. Object Processing

42. Carver et al. • Conclusions: • Consistent with deHaan/Nelson/Johnson, is segregation of face vs. object processing • Responses to familiar face but NOT object varied as a function of age. • More specifically, all age groups showed a larger response for the Nc and P400 to unfamiliar versus familiar objects, but only children between 18-24 months of age showed greater ERP response to the mother’s face compared to a strangers face. • Ability to recognize faces continues to be refined through 5 years • Since in previous work demonstrated that children with autism fail to show ERP differentiation to mother/stranger face (but do show typical profile to familiar/novel objects), suggests that face recognition “system” can be corrupted by something other than lack of experience.

43. What Develops Beyond the Preschool Years? • Passarotti et al. (2003)*. • Presented 10-12 year old children and adults with face matching vs. location matching task • fMRI performed at 1.5Tesla • RESULTS: Behavioral Data: accuracy and RT improve with age fMRI Data: Relative to adults, children showed more distributed pattern of activation in both right and left temporal regions (i.e., increased activation in areas lateral to fusiform gyrus and anterior to fusiform in middle temporal gyrus) (see next slide) * Passarotti, A.M., Brianna, M.P., Bussiere, J.R., Buxton, R.B., Wong, E.C., & Stiles, J. (2003). Developmental Science, 6, 100-117.

44. Me A. Medio-lateral fusiform regions B. Middle Temporal region RH LH RH LH RH LH RH LH 5 5 4 4 3 3 2 2 1 1 0 0 children N=12 Adults N=16 Children N=12 Adults N=16 Children N=12 Z=-17 Z=-9 Face-matching Task Figure 2

45. Passarotti et al. • Conclusions: • Children have a more diffuse and distributed area of activation in both face and location processing compared to adults. • Areas involved in face processing are still undergoing refinement as “late” as 10-12 years.

46. General Conclusions About Face Recognition • The bulk of the evidence supports a rapid improvement in face recognition abilities through infancy, with more subtle changes occurring well beyond infancy. • Neurophysiological (ERP) and metabolic (fMRI) data, not surprisingly, also show changes in neural substrate across first decade of life. • Presumably these changes are experience-dependent, and do not reflect any sort of “innate” module.

47. What about Disorders of Face Recognition? • The study of typical development can often be informed by the study of children who deviate from a normal trajectory. • Prosopagnosia • Autism

48. Prosopagnosia • Special nature of face processing originates from reports of patients who are reportedly unable to recognize familiar faces while maintaining the ability to recognize objects. • This impairment is often accompanied by focal damage to the ventral occipitotemporal and temporal cortices. • Reported to process faces similar to objects (sometimes absence of inversion effects).

49. Prosopagnosia • Marotta, Genovese, & Behrmann (2001) • fMRI of prosopagnosic patient did not show normal activation of fusiform. • Did show left hemisphere posterior fusiform activation, suggesting faces are being processed featurally.

50. Developmental Prosopagnosia • There are a small number of developmental prosopagnosic cases. • Most have more general visual processing deficits • Farah et al (2000) report a case study of a boy who sustained brain damage at 1 day of age. They report that this boy (now late teens) has damage to occipitotemporal pathway and is severely impaired in face compared to object recognition. • Evidence that face processing specified in genome? No experience necessary?