Attention and emotion: From data to conceptual issues

1. Attention and emotion: From data to conceptual issues Luiz Pessoa Department of Psychology University of Maryland, College Park

2. Background • 1990s: work showing limitations of visual processing and the need for attention • Change blindness • Attentional blink

3. L R Whalen et al. (1998) Morris et al. (1998) Background • Processing of emotion-laden information is prioritized • Independent of awareness

4. “Automatic” Processing Amygdala LeDoux

5. Research goal • Understand the role of attention and awareness during the processing of emotional visual items • Employ strong attentional manipulations • Evaluate awareness with Signal Detection Theory

6. Role of spatial attention • Is activity evoked by emotional faces automatic? OR • Does activity evoked by emotional faces require attention?

7. Spatial attention Unattended Faces Attended Faces Not drawn to scale 200 ms 200 ms Male/female Same/different Easy: 91% correct Difficult: 64% correct

8. Right Amygdala Fear ATT 0.25 Fear UNATT 0.2 Happy ATT Response Amplitude 0.15 Happy UNATT Neutral ATT 0.1 Neutral UNATT 0.05 0 -0.05 -0.1 0 2 4 6 8 10 12 Seconds Attention is required for the expression of valence (N = 21) • Strong valence X attention interaction: Effect of valence depends on attention L R X Pessoa et al. (2002): PNAS

9. Emotional perception requires attention • Attention parametrically manipulated within the same task • Affective significance enhanced via conditioning Easy Hard Pessoa et al. (2005): Neuroimage Task: find X Hsu and Pessoa (2007): Neuropsychologia Lim et al. (2008): Neuropsychologia

10. Confidence? Role of visual awareness Target Mask Yes/No 1-3 scale Pessoa et al. (2005): Emotion

11. 67 ms 33 ms Visual awareness: Signal Detection

12. 33 ms UNAWARE L L R R 67 ms Y = -4 AWARE Y = -4 Amygdala responses VS. Pessoa et al. (2006): Cerebral Cortex

13. Behavioral results: Individual differences • Many participants can detect fearful faces even at 17 ms 17 ms Szczepanowski and Pessoa et al. (2007): Journal of Vision

14. 67 ms 33 ms AWARE UNAWARE “Normals” N = 19 L L L L R R R R Y = -4 Y = -4 AWARE AWARE “Detecters” N = 8 Y = -6 Y = -6 Fear stimulus > Neutral stimulusAmygdala x . Pessoa et al. (2006): Cerebral Cortex

15. Role of temporal attention/awareness CS+ vs. CS– T1 T2 . . . . . . 100 ms 100 ms 2 s Lim, Padmala, and Pessoa (2009): PNAS

16. Attentional blink: Behavior (N = 30) • Enhanced perception of CS+: Reduced blink CS+ Building or House? CS– T1 T2

17. Role of attention/awareness T2 . . . . . . Parahippocampal gyrus

18. Miss trials T2 . . . . . . Parahippocampal gyrus

19. Role of attention/awareness Visual ctx % signal change CS+ • Miss trials: no differences observed between CS+ and CS- trials CS- Amygdala % signal change Time

20. Conceptual issues

21. Impasse • While a great deal has been learned about the extent and limits of affective visual processing, two camps have opposing and entrenched views

22. Impasse • While a great deal has been learned about the extent and limits of affective visual processing, two camps have opposing and entrenched views Capacity-limited Capacity-Unlimited

23. Impasse • While a great deal has been learned about the extent and limits of affective visual processing, two camps have opposing and entrenched views Capacity-limited Capacity-Unlimited

24. Not too surprising… • Emotional stimuli are sufficiently potent that they exhibit a host of properties that do not appear to occur with neutral items • They are processed when unattended • Affective processing is subject to capacity limitations, as revealed by several experimental manipulations • Attentional blink

25. Impasse: will it go away? • Advocates of limited processing can claim that processing resources have not been consumed • “If the manipulation were stronger, the impact of affective items would go away…”

26. Impasse • Showing that the emotional effect has disappeared is always subject to the “null problem” • Arguing for the absence of an effect

27. Fear unatt Happy unatt Neutral unatt Power vs. strength of manipulation Left Amygdala 0.25 Right Amygdala 0.2 Fear att 0.15 Happy att 0.1 Neutral att 0.05 Response Amplitude 0 -0.05 X -0.1 0 2 4 6 8 10 12 Seconds 0 2 4 6 8 10 12 Seconds Pessoa et al. (2002): PNAS

28. Processing resources • Given the limited capacity of mental processes, performance is impaired if demands are greater than available resources Easy/efficient performance Hard/inefficient processing resources Norman and Bobrow (1975)

29. Processing resources performance processing resources performance “automatic” Nakayama and Joseph (1998) processing resources

30. Processing resources performance processing resources performance Capacity limitation Dual-tasks Nakayama and Joseph (1998) processing resources

31. Processing resources • Moors and De Houwer (2006): Every process is uncontrolled, efficient, unconcious, and fast

32. Processing resources • Moors and De Houwer (2006): Every process is uncontrolled, efficient, unconcious, and fast, to some degree… • Relative to what? • Affective processing: relative to neutral not enough • Fine comparisons needed (e.g., abrupt onsets, search, etc.) • Broad set of comparison tasks

33. Two camps Capacity-limited Capacity-Unlimited

34. Preattentive-attentive model Stage 2: Attentive t2 t1 boundary Stage 1: Preattentive

35. Preattentive-attentive model • Some features are processed pre-attentively in virtue of the fact that they are optimally matched to properties of the early visual system (e.g., orientation) • Affective processing: Sub-cortical pathway • Superior colliculus  pulvinar  amygdala

36. Dynamic model Multiple interactive “stages”

37. Dynamic model • Processing is not pre-attentive or attentive, but a gradient of processing efficiency is hypothesized to exist • Gradient based on the properties of early visual areas • But critically, gradient is dynamically configured based on task demands • Configuring is suggested to depend on parietal and frontal cortex

38. Dynamic model More susceptible to capacity limitations • Multiple “gates” • Variable permeability “bottlenecks” Less susceptible to capacity limitations

39. Dynamic model • Hierarchical and “short-cut” connections

40. Multiple waves • Initial processing of visual information proceeds simultaneously along parallel channels • “Multiple waves” of activation across visual cortex and beyond • The multiple waves are engaged dynamically based on task requirements

41. Subcortical processing

42. Subcortical processing “passive” “integrative” Pessoa and Adolphs, Nat. Rev. Neurosci (2010)

43. Subcortical processing Pessoa and Adolphs, Nat. Rev. Neurosci (2010)

44. Processing architecture and attention Task 2 Task 1

45. Ralph Adolphs Jan Engelmann Shruti Japee Shen-Mou Hsu Seung-Lark Lim Srikanth Padmala Remik Szczepanowski Leslie Ungerleider Collaborators National Institute of Mental Health emotioncognition.org