Neural Correlates of Variations in Event Processing during Learning in Basolateral Amygdala

1. (2010) Neural Correlates of Variations in Event Processing during Learning in Basolateral Amygdala Matthew R. Roesch*, Donna J. Calu, Guillem R. Esber, and Geoffrey Schoenbaum * Department of Psychology and Program in Neuroscience and Cognitive Science, University of Maryland College Park

2. Background… • To optimize reward, animals must learn to associate cues withrewards and recognize the differencebetween the reward expected and that which actually occurs to guide their behaior • The prediction error • 2 categories of learning models:

3. Category 1: “Signed error” Models • If a rewardis larger than expected(+), the association between the cue andreward will be strengthened, whereas if the reward is smallerthan expected(-), the association will be weakened. • …predict that the sign of the prediction error (i.e., whetherthe reward is bigger or smaller than expected) will be encodedin neural activity.

4. Category 1 • This correlate has been shown in midbrain dopamineneurons (Rescorla and Wagner, 1972; Suttonand Barto, 1998; Mirenowicz and Schultz, 1994; Montague et al., 1996; Schultz et al., 1997; Hollerman and Schultz, 1998; Waelti et al., 2001; Bayer and Glimcher, 2005; Pan et al., 2005; Bayer et al., 2007; D'Ardenne et al., 2008; Matsumoto and Hikosaka, 2009) . • Firing in these neurons increases in the face of unexpectedreward (+) and is suppressed when reward is unexpectedlyomitted (-). • Evidence also from other brain areas (Hong and Hikosaka, 2008; Matsumoto and Hikosaka, 2009) .

5. Category 2: “Unsigned error’ models • Prediction errorstell an animal that it must learn more about the cue–rewardassociation and therefore serve to drive attention. • A cue should be more thoroughly processed (andlearned about) when it is a poor predictor of reward. When thecue becomes a more reliable predictor, processing (and learning) should decline (Pearce–Hallmodel (1980, 1982)).

6. Category 2 • These modelspredict that neural activity encoding prediction errors willbe similar regardless of the sign of the error(+/-). • …lackof evidence for neural correlates of unsigned prediction errors—e.g.,increased firing when reward is either better or worse thanexpected.

7. What did this paper do? • Basolateral Amygdalar (ABL) Neurons Encode Unsigned Prediction Errors. • This neural signal increasedimmediately after a change in reward, and stronger firing wasevident whether the value of the reward increased or decreased.

8. How did they do it • Recording single unit activity in a behavioraltask in which rewards were unexpectedly delivered or omitted. • Basic paradigm is a choice task Reward well Reward well Odor Port 3 different odor cues: one signaled rewardon the right (forced-choice), a second for left (forced-choice), and a third for either well (free-choice).

9. Trials and Blocks > - < + + > - < + Each Block consists of at least 60 trials; In between blocks, rewarding value shifted (i.e. value of the port for rats changed)

10. Results • Performance and recording sites

11. 70reward-responsive ABL neurons recorded ; • 58/70 exhibited differential firing base on timing (short/long delay) or size of the reward(large/small) after learning,  signed coding theory;outcome-selective • They also exhibited changes in reward-related firing between the beginning and end trials of each block, regardless reward upshift or downshift, unsigned coding theory

13. 2 factor ANOVA analysis ineach neuron across learning (early vs late) and shift type (upshiftvs downshift) • 10 of the 58 neurons (17%) fired significantly more early in a block(after a change inreward), than later(after learning).

14. Indices [(early – late)/(early + late)], representing the difference in firing to reward delivery (within 1 s) during trials 3–10 (early) and during the last 10 trials (late) after shifts .

15. Main contribution of this paper • The activity in the outcome-selective ABL neuronswas higher at the start of a new training block, whether rewardwas better or worse than expected, and declined as the ratslearned to predict the value of reward. • This pattern of firingis generally consistent with the notion of an “unsigned error”models such as that of Pearceand Hall (1980)

16. Another distinctive feature • Theirfiring did not immediately increase at the start ofa new block, in response to a change in reward, but rather appearedto gather momentum and peak a few trials into the block (3rd trial).

17. Given the remarkable fit provided by the amended Pearce–Hallmodel (1982) and the role attributed to unsigned errors withinthis theoretical context, it seems natural to speculate thatthis ABL signal may be related to variations in event processing (title). • Especially in view of the strikingsimilarity between changes in the ABL signal and changes inthe rats' latency to approach the odor portat the start ofeach trial.

18. The close relationship between the ABL signal and a behavioral measure “speed of orienting” Increase in speed of orienting to the odor port Trial by trial analysis

19. Explanation • Faster odor-port approachlatencies may reflect error-driven increases in the processingof trial events (e.g., cues and/or reward), because rats acceleratethe reception of those events when shifted contingencies needto be worked out. • In this sense, approaching the odor port fastercan be looked upon as similar to conditioned orienting. Conditioned orienting responses, also known as investigatoryreflexes, means to recover from habituation when learnedcontingencies are shifted.

20. To further investigate the relationship between the ABL signal and odor-portapproach latency, ABL was inactivated in some rats during performance of the recording task. • Inactivation of ABL disrupted the change in orienting. ns

21. Inactivation of ABL also retarded learning inresponse to changes in reward. Block 1 well 1> well2, rats prefer well1 • Inactivation of ABL with DNQX • Block 2, well 1< well 2, rats continue to approach well 1

22. Choice performance in vehicle versus NBQX sessions, plotted according to whether the well values in a particular trial block were similar to or opposite from those learned at the end of the prior session.

23. Conclusions • Basolateral Amygdalar Neurons Encode Unsigned Prediction Errors ; • This neural signal was correlated withfaster orienting to predictive cues after changes in reward,and abolition of it disrupted this change in orienting and retarded learning inresponse to changes in reward. • These results suggest that basolateralamygdala serves a critical function in attention for learning.