Neural mechanisms of feature-based attention

1. Taosheng Liu Neural mechanisms of feature-based attention

2. What is attention? • “Everyone knows what attention is. It is the taking possession by the mind in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought.” • -William James (1890)‏ • Types of visual attention • Overt attention • Covert attention • Spatial • Feature-based • Object-based

3. Attention and the brain • Effects vs. control

4. Outline • The effect of feature-based attention on visual cortex • How does attention modulate sensory representations? • The control of feature-based attention • What is the source of control and how is control implemented? • Attention and object recognition

5. MT Response Attend ‘up’ Attend ‘down’ The effect of FB attention to motion Treue & Martinez-Trujillo, 1999, Nature • Questions: • Does feature-based attention modulate neuronal subpopulations in the attended location? • If so, how does it correlate with behavior?

6. upwardpreferring units Response Attend ‘up’ Attend ‘down’ downward preferring units Response Use adaptation to assess feature selectivity More adaptation for a upward test stimulus when attending ‘up’ vs. ‘down’

7. fMRI adaptation • A voxel contains many neurons. • fMRI adaptation can assess feature selectivity within a voxel.

8. -20° +20° 3 y c n e 2 u )‏ q d e p r f c 1 l ( a i t a p 0 S 0 1 2 3 4 T i m e ( s )‏ Adapting stimulus play demo

12. … 0° Adapter (4 s)‏ 1 s Pre-adaptation (40 s)‏ Test (0.5 s)‏ -20° +20° . . . . . Behavior: tilt aftereffect (n=8)‏ Attend -20 Attend -20 Attend +20 Attend +20

13. Adapter (4 s)‏ Test (1 s)‏ Pre-adaptation (40 s)‏ … 1 s 1.2 s Attended Unattended . . . . . … Blank fMRI adaptation protocol Task inside the scanner: report the orientation of the test stimulus.

14. fMRI details • Siemens 3T Allegra • Surface coil • 21 coronal/oblique slices • 3 mm isotropic voxels • TE = 30 ms, FA = 75º • TR = 1.2 s • Bite bar to minimize head motion

15. Surface reconstruction and retinotopic mapping

16. Retinotopic mapping and localizer real data (TL)‏

17. Unattended Attended V1 V2 0.8 . fMRI response (%)‏ 0.6 0.4 0.2 0 -0.2 -0.4 0 0 10 5 10 5 15 15 Time (s)‏ Time (s)‏ fMRI response to the test stimulus adapter test

18. V 3 h V 4 0 . 8 0 . 6 0 . 4 0 . 2 0 - 0 . 2 - 0 . 4 L O 1 L O 2 0 . 8 0 . 6 0 . 4 0 . 2 0 - 0 . 2 - 0 . 4 V 3 A / B V 7 fMRI response (%)‏ 0 . 8 U n a t t e n d e d 0 . 6 A t t e n d e d 0 . 4 0 . 2 0 - 0 . 2 - 0 . 4 0 5 1 0 1 5 0 5 1 0 1 5 Time (s)‏

19. Rattn – Runattn Rattn + Runattn Attention modulation index

20. Correlation between behavioral and imaging results

21. neutral attended 1 Neural response 1 0 . 8 0 . 8 0 . 6 0 . 6 0 . 4 0 . 4 0 . 2 0 0 . 2 - 9 0 - 4 5 0 4 5 9 0 0 Preferred orientation (deg)‏ - 9 0 - 4 5 0 4 5 9 0 Preferred orientation (deg)‏ Shift in preferred orientation 1 1 0 5 0 . 8 0 0 . 6 - 5 0 . 4 - 1 0 - 9 0 - 4 5 0 4 5 9 0 0 . 2 Preferred orientation (deg)‏ 0 - 9 0 - 4 5 0 4 5 9 0 Dragoi et al, 2000, 2001 Preferred orientation (deg)‏ A model relating psychophysical and imaging data Psychophysics Neural response fMRI Neural response

22. Summary & conclusion • Feature-based attention enhances activity of neuronal subpopulations when the attended and unattended features are processed in the sameretinotopic region. • Attentional modulation of orientation-selective fMRI response adaptation. • Attentional modulation constant across visual areas, suggesting a feed-forward mechanism. • Significant correlation between TAE and AMI only in V1. Liu etal, 2007, Neuron

23. The control of feature-based attention • Components of attentional control • Disengage/shift • Engage/maintain • Goal: • Separate different components • Feature-based attention

24. instruction Color Motion response ‘shift’ Red button1 ‘hold’ Green button2 Task and design

25. L ITG (MT+)‏ R SPL/IPL 0.20 0.20 0.15 0.15 % signal change 0.10 0.10 0.05 0.05 0.00 0.00 -0.05 -0.05 -0.10 -0.10 -0.15 -0.15 -0.20 -0.20 -5 0 5 10 15 -5 0 5 10 15 Time (sec)‏ Time (sec)‏ Sustained effect for motion FEF, SPL/IPL: sustained attentional control for motion. MT+: effects of attention for motion. color to motion motion to color hold motion hold color

26. Transient shift activity Precu, IPS, PCG: transient control of attention shift. color to motion motion to color hold motion hold color

27. Summary • Effects of attention: • MT+ (motion) and V4 (color)‏ • Attentional control: • Transient control: disengage/shift (superior parietal lobule, left intra-parietal sulcus, left pre-central gyrus). • Sustained control: engage/maintain (frontal eye fields, superior-inferior parietal lobule for motion; superior frontal gyrus for color). Liu etal, 2003, Cerebral Cortex

28. Current and future plans • Attentional control within feature dimensions • What are the ‘shift’ regions? • What are the ‘hold’ regions?--attentional priority

29. The representation of attentional priority • Spatial attention • Higher areas with a spatiotopic map send feedback signals • Feature-based attention • Are there neurons that encode the attended direction in higher areas? LIP FEF

30. Decoding of brain activity Kamitani & Tong (2007)‏ • Classifier scheme • Classifier can reliably decode orientation information in early visual cortex

31. The effect of temporal coherence on object memory Learning sequence of views of three-dimensional objects:

32. How do we recognize shapes? Temporal association: object views appearing close in time are associated. Wallis & Bulthoff (1999)‏

33. Harman & Humphrey (1999)‏ No accuracy effects Attention? Effort? ??? 7 views x 1 s/view x 3 repeats

34. Exp 1 - replication stimuli

35. Exp 1 - method

36. Exp 1 - results

37. Exp 2: test novel views Test views 1,3,5,7

38. Exp 3 – method

39. Exp 3 - results

40. Exp 4 Encoding task: preference rating “rate how much you like each sequence on a 3-point scale”

41. Exp 4 - results

42. Summary • RR always the worst • temporal association works • SS never exceeds SR • temporal vs. spatiotemporal coherence • SS depends on study time and intention • potential confound