What made you respond face (or word)?

13. What made you respond face (or word)? • Something in your brain made you decide face or word. • Can we determine where this decision is made? • Related domain: Motion Direction Discrimination and area MT

14. What determines the percept and the response? • Observe a correlation between motion direction judgements and activity of cells in area MT. (Britten et al. 1996, Gold & Shadlen, 2000) • If neurons that responded to leftward motion were highly active, the monkey chose 'left' as the decision. From Schall, 1999

16. Evoked Response Potential (ERP) and Face Stimuli • N170: negative-going potential at 170 ms • Largest over the right parietal lobe, also on the left parietal lobe. From Tanaka and Curran (2001)

17. N170 Properties: • Faces produce the largest amplitude. • Strong evidence of expertise: Bird experts have larger N170's to pictures of birds than pictures of dogs. Dog experts show the reverse. (Tanaka & Curran 2001). • Mainly perceptually based: prior exposure of a face does not produce large changes in the N170 for subsequent presentations (Rik Henson, AIC 2003). • Scalp distribution and latency suggest that the N170 component reflects the perceptual processing of complex visual stimuli.

18. A Thought Question: • What was going on in your perceptual regions when you thought you saw a face or a word? • Could we capture the current state (at least indirectly) with the N170 component? • Would the N170 be larger when you thought you saw a face?

19. Central Question: • Can we relate the size of the N170 to the response in the noise-alone condition? • Will it be larger when subjects think they see a face?

20. An Experiment • Show Faces and Words Embedded in Noise: High Contrast Faces Low Contrast Faces Low Contrast Words High Contrast Words Noise Alone

21. Methods • Face, word and noise-alone trials were presented in random order. • Ten naïve participants • 120 trials per condition per subject • On each trial: Did you see a face or a word? • Subjects were told that a stimulus appeared on each trial, and that they should guess if they were unsure.

22. EEG Recording Sites

23. Methods continued • Analyze the data according to the subject’s responses on the noise-alone trials. • Incredibly important point: The noise was the same across ALL trials and stimuli. Physically the same. Not just identically distributed, but identical. There was only one noise field for the entire experiment. Together, these procedures hold the physical stimulus constant on noise-alone trials.

24. Amplitude (V)

25. Central Question: Will we see a larger N170 to the noise-alone stimulus when subjects think they see a face as opposed to a word?

26. * t(9) = 2.74, p = .023, two-tailed * Amplitude (V)

27. Main Result: • On noise-alone trials: Larger N170 when observers report seeing a face than when report seeing a word. • Occurs in 9 of the 10 subjects. • No other differences in any other channel at the P100, N170 or P300 components. • Unlikely to just reflect activity for an already-made decision. • Relates activity in the perceptual processing areas to the behavioral response. Greater activity in the N170 neurons is associated with ‘face’ responses to the noise-alone stimulus. One interpretation: Greater activity in the face processing region biases the response towards a ‘face’ response.

28. Alternative Explanations for this Greater Activity • Attention to different spatial frequencies or face-like features in the noise • Unlikely to see rapid changes in spatial frequency tuning in a mixed design. • No P100 differences that might be associated with changes in activity in different spatial frequency channels. • One obvious exception: Prior trial priming • Seeing a face on the previous trial may leave residual activity in the face neurons or make subjects look for face-like features in the noise.

29. Prior Trial Priming Amplitude (V)

30. Third Possibility:Stochastic Activity • In the domain of binocular rivalry, Blake and Logothetis introduced the idea of a process that involved stochastic activity in perceptual regions, which could bias the response toward one percept or the other at different points in time. (Blake & Logothetis 2002). • A similar process could be at work in the face processing neurons: When activity is high in the N170 neurons due to random stochastic fluctuations, the observer may be biased to respond ‘face’ on that trial.

31. Implications for Internal Noise • Internal noise does not just limit performance or decrease calculation efficiency, but also operates in feature space to bias the response toward one alternative or another. • Manipulations such as varying the power of the noise or the stimulus pairs being compared may help constrain models of internal noise. • Supports trial-by-trial variability in parameters: • Starting point and drift-rate variability in Ratcliff’s Diffusion model. • These sources of variability take on a perceptually-based interpretation. Link with perceptual brain areas.

32. High Contrast Faces Low Contrast Faces Low Contrast Words High Contrast Words Noise Alone The Important Stuff: For Noise-Alone Trials: * t(9) = 2.74, p = .023 Amplitude (V)

33. ERP and Faces • Intracranial recordings reveal N200 at sites in IT/fusiform gyrus From Allison, Puce, Spencer, and McCarthy (1999)

34. Which Stimuli Evoke an N170/N200? Any face or face-like visual stimulus

35. N170 Properties: Top-down Influences • Could be just a face-detection system (Bentin, et. al. 1996) • No effect of task demands such as selective attention to faces vs. objects. (Caquil, Edmonds, & Taylor, 2000) • No effect of familiarity of the face (Bentin & Deouell, 2000) • fMRI - IT active anytime there’s a face. (Gauthier et al. 1999) • However, IT also active while imagining a face. (O’Craven & Kanwisher 2000)

36. Contextual Influences On N170 Bentin et al (2002) • Block 1: • Stimulus Set A • Block 2: • Set B (experimental) • Set C (control) • Set D (all; targets) • Block 3: • Set A again (all)

37. Contextual Influences On N170 Bentin et al (2002) • Face context elicits N170 to schematic eyes • Once the stimulus has been interpreted as containing face-like features, a stronger N170 is produced.

38. N170 Summary • Represents activity of face-selective neurons most likely in area IT. • Magnitude varies with the degree to which physiognomic information is perceived in the image. • Is the N170 related to the response in the noise-alone condition? • If so, tie the activity in face-selective cells to the percept and the response.

39. Would we get an N170 to the noise-alone trials at all? • Previous work showed only a weak N170 to random noise stimuli. • Pilot study: Blocked presentations of faces and words. • Result: when looking for faces, get a larger N170 to noise-alone trials than when looking for words. • Replicates Bentin et al., extends to images that contain no facial features. We see top-down effects on the N170 even with no face-like features.

40. Face Condition : Male or female?

41. Word condition:Honesty or Trust?

42. Experiment 1 Results - T5 Grand Average for T5 8 6 4 High-Contrast Face Low-Contrast Face Zero-Contrast Face High-Contrast Word Low-Contrast Word Zero-Contrast Word 2 0 Voltage (microvolts) -2 -4 -6 -8 -10 -12 -100 0 100 200 300 400 500 600 Time (ms)

43. Experiment 1 Results - T5 Grand Average for T5 Zero-Contrast Word Zero-Contrast Face 6 * 5 4 *t(8) = 2.62, p = .031 3 Voltage (microvolts) 2 1 0 -1 -2 -3 -100 0 100 200 300 400 500 600 Time (ms)

44. Neural Processing of Face Stimuli • Cells in inferotemporal cortex (IT) are known to respond selectively to faces • (single cell recording, fMRI) • Slight right hemisphere dominance

45. Experiment 1 Results - T6 Grand Average for T6 Zero-Contrast Word Zero-Contrast Face 5 * 4 3 * t(8) = 2.35, p = .047 2 Voltage (microvolts) 1 0 -1 -2 -3 -100 0 100 200 300 400 500 600 Time (ms)

46. Experiment 1 Conclusion When an observer expects a face (versus a word) there is a greater N170