1 / 142

PVLV Model of Phasic Dopamine Learning

PVLV Model of Phasic Dopamine Learning. R. O’Reilly T. Hazy, J. Reynolds, G. Frank. Temporal Difference Dopamine Reward Model. Brain Dopamine Reward Response = Reward Occurred – Reward Predicted. Schultz, Dayan & Montague 1997. DA ↑. Unexpected Reward. DA ↑. DA ↔. Expected Reward.

aldon
Download Presentation

PVLV Model of Phasic Dopamine Learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PVLV Model ofPhasic Dopamine Learning R. O’Reilly T. Hazy, J. Reynolds, G. Frank

  2. Temporal Difference Dopamine Reward Model Brain Dopamine Reward Response = Reward Occurred – Reward Predicted Schultz, Dayan & Montague 1997 DA ↑ Unexpected Reward DA ↑ DA↔ Expected Reward DA ↑ DA ↓ Unexpected No-Reward

  3. Phasic Dopamine Firing • Transition from Reward/US to CS onset (Schultz et al) • TD model says why, but not how (neural mechanisms) • And TD not a good fit for growing set of data..

  4. PVLV Model • Central Nucleus of the Amygdala (CNA) = CS -> DA • Ventral Striatum (NAc, patch) = US -> no DA

  5. PVLV Model • PV = Cancelling US burst • LV = Driving CS burst • No chaining: just simpler delta-rule/Rescorla Wagner

  6. PVLV Predictions • CS – driven DA is dissociable from US – driven: • Conditioned orienting & autoshaping to a CS = CNA, but not to US • CS DA is not subject to blocking effect • CS DA cannot drive further CS learning: 1st order (CNA) is dissociable from 2nd order (BLA), and no 3rd order • Time is just another input; no temporal chaining as in TD

  7. PVLV Drives PBWM

  8. ROI Activation Analysis after Full Brain Statistical Analysis Guido K.W. Frank, M.D. The Children’s Hospital, University of Colorado Denver

  9. Receiving Sucrose (US) Unexpected AN greater than CW p=0.005 5vox uncorr. SVC p<0.02, FEW, FDR, Cluster

  10. Receiving Sucrose (US) Unexpected  Time Activity Curves Anorexia Nervosa VTA CNA-L CNA-R AVS-L AVS-R VTA CNA-L CNA-R AVS-L AVS-R Control Women

  11. Influence of Emotion/Reward & Punishment on EF:Incentive Stroop Task Jeff Spielberg, with Wendy Heller, Gregory A. Miller, Laura Crocker, Stacie Warren, Christina Murdock-Jordan, post-doc Dave Towers, Brad Sutton & Tracey Wzalek at BIC, and Center colleagues Marie Banich & Randy O’Reilly Department of Psychology and Beckman Biomedical Imaging Center, University of Illinois at Urbana-Champaign

  12. Influence of Emotion/Reward & Punishment on EF • Multiple ways to conceptualize emotional processes psychologically • Circumplex models, valence, bipolarity • Valence/arousal, positive affect/negative affect • Valence-based dichotomies prevalent (pos. vs. neg. emotionality/temperament/extraversion) • Fundamental motive systems that underlie behavior • Appetitive/approach/behavioral activation/incentive motivation • Defensive/withdrawal/behavioral inhibition/aversive motivation

  13. Influence of Emotion/Reward & Punishment on EF • No comprehensive model(s) of brain function or structure integrating roles or mechanisms of brain regions implicated in emotion: • Left & Right DLPFC: positive/negative affect • dACC: anxiety • Amygdala: fear • Nucleus accumbens: reward/punishment • Posterior ACC: emotional autobiographical memory • Orbital frontal cortex: reward punishment • Distinct lines of research, most not integrated with each other

  14. Influence of Emotion/Reward & Punishment on EF • Lateralization a predominant feature of models of PFC organization for emotion • Divided according to superior/inferior lines • Davidson motivation model, also Harmon-Jones, Coan, & Allen, others • Approach/withdrawal mapping onto left vs. right DLPFC • Heller, Miller, Banich, and others’ valence/arousal model • Positive/negative valence mapping onto left vs. right DLPFC • Adds emotional arousal & right parietal activity

  15. Prefrontal Lateralization for Emotional Processes Approach Motivation Withdrawal Motivation L R Positive Emotion Negative Emotion

  16. Influence of Emotion/Reward & Punishment on EF Positive valence Approach Motivation Attachment Trait Anger Anger Out Anxious Apprehension Anxious Arousal Anhedonic Depression Distinguishable, lateralized prefrontal areas sensitive to… L R L R L R (From Engels et al., 2007; Herrington et al., 2005; Herrington et al., 2009; Spielberg et al., 2008; Stewart et al., 2008; all regions depicted replicated in at least two studies)

  17. Patterns of lateralization for emotion tend to be reversed for orbital frontal cortex (OFC) Positive/negative valence mapping onto right vs. left OFC Note LDLPFC spot! Influence of Emotion/Reward & Punishment on EF deAraujo, Rolls, Kringelbach, McGlone, & Phillips, 2003

  18. Influence of Emotion/Reward & Punishment on EF • Incentive Stroop Task: • Engages motivational systems, emotional systems, and executive functions simultaneously • TASK • Press a button ASAP when a word appears • AFFECTIVE CONTEXT COMPONENT OF THE TASK • IGNORE the meaning of the word, which can be positive, negative, or neutral • Design allows us to examine the effect of the emotional content of the irrelevant information (affective context) on the ability to ignore that information

  19. Influence of Emotion/Reward & Punishment on EF • MOTIVATIONAL COMPONENT OF THE TASK • Before each word is shown, participants see a cue which tells them whether they will win and/or lose money depending on how fast they push the button • If they push the button fast enough they get the positive outcome on that trial (i.e., win money or avoid losing money) • If they don’t push the button fast enough they get the negative outcome on that trial (i.e., lose money or miss winning money) • On some trials, they neither win nor lose money • Allows us to examine the effect of anticipating rewards and punishments on the ability to ignore irrelevant information

  20. For dollar sign on left: if it’s green, can win money if push button fast enough if it’s grey, can’t win money on that trial For dollar sign on right: if it’s red, can lose money if don’t push button fast enough if it’s grey, can’t lose money on that trial

  21. Fast enough = win money Too slow = do not win money Fast enough = win money Too slow = lose money Fast enough = do not lose money Too slow = lose money Do not win or lose money regardless if they are fast or slow

  22. Incentive Stroop Task Cue ISI Emotional word ISI Feedback Time within trial

  23. Influence of Emotion/Reward & Punishment on EF • Recruiting 3 groups of subjects according to PANAS scores • High positive, low negative affect • High negative, low positive affect • Low negative, low positive affect • Participants are run in counterbalanced EEG/fMRI sessions • SCIDs & assessment of EF components via neuropsychological tests

  24. Influence of Emotion/Reward & Punishment on EF • Preliminary behavioral findings: • Emotional content of the irrelevant information affects ability to ignore that information • Pleasant or unpleasant words elicit slower RTs than do neutral words • Emotional words thus harder to ignore • Motivational context also affects the ability to ignore irrelevant information • RT faster on trials in which reward or punishment is possible • Thus, possible rewards and punishments make it easier to ignore irrelevant information • Findings indicate effects of both affective context and motivation on executive function (as measured by RT)

  25. Influence of Emotion/Reward & Punishment on EF • Activation in left DLPFC (yellow) when viewing cues signaling the potential for reward (associated with faster RTs) • Activation in right OFC (green) when receiving rewards (associated with faster RT) • Activation in left OFC (red) when receiving punishments (also associated with faster RT)

  26. Influence of Emotion/Reward & Punishment on EF • Preliminary results thus: • Confirm effects of both affective and motivational contexts on executive function • Replicate opposing patterns of lateralization for DLPFC and OFC • Allow us to examine timing of regional activity, connectivity, relationships of regional and temporal dynamics to emotional disposition • Can be extended to examine dysfunctional relationships in depression & anxiety

  27. Effects of Anxiety on Selection Among Competing Options Hannah R. Snyder & Yuko Munakata (Project 5) in collaboration with: Marie T. Banich (Project 1) Tim Curran & Erika Nyhus (Imaging Core) With consultation from Project 3

  28. Anxiety and Uncertainty • Anxious apprehension (worry) is linked to intolerance of uncertainty(e.g. Ladouceur, Talbot & Dugas, 1997), decision-making problems, and indecisiveness(e.g. Sachdev & Malhi, 2005). • Prominent symptoms of anxiety disorders including GAD and OCD. • Why? • Approach this question using our framework for understanding one aspect of EF: selection among competing options.

  29. Selection Among Competing Options • We constantly face the need to choose one option from among multiple valid choices. • e.g. grocery shopping, selecting a retirement plan, or choosing a word to express a thought. • Selecting between multiple options is time-consuming and effortful (e.g. Iyenger & Lepper, 2000; Sethi-Iyenger et al., 2004; Snyder & Munakata, 2008). • Left ventrolateral prefrontal cortex (VLPFC) is involved in selection (e.g. Thompson-Schill et al., 1997,1998; Schnur et al., 2009). • Illinois center colleagues Engels, Heller, & Miller have shown that left VLPFC is involved in anxious apprehension. • What specific mechanisms might support selection, and how might they be affected by anxiety?

  30. Selection Among Competing Options • Test neural network predictions about selection using a well-controlled language-production task: verb generation.

  31. Pyramidal Cell Pyramidal Cell - + - + GABAergic Interneurons Neural Network Model • Demonstrates that competitive, inhibitory dynamics among neurons in prefrontal cortical networks support selection among competing alternatives. • Amplify activity of most active representation and suppress activity of competing representations, via inhibitory, GABAergic interneurons.

  32. Neural Network Model • Suggest that prefrontal GABA function plays key role in selection and breakdown of this process. • Makes sense of findings which were previously disconnected from each other, linking anxiety to: • Reduced GABA(e.g. Kalueff & Nutt., 2007) • VLPFC dysfunction(e.g. Engles et al., 2007).

  33. Competitive Inhibition & Selection • Neural network predictions: • Anxiety (reduced neural inhibition) impairs selection and associated VLPFC activity, even in a simple, non-affective language-production task. • The GABA agonist midazolam (increased neural inhibition) improves selection. • Retrieval from semantic memory is unaffected. • These predictions were supported in 3 studies.

  34. Anxiety: Decreased Inhibition Impairs Selection Network Predictions Participants (RTs) • Reduced competitive inhibition in the VLPFC layer impairs selection. • High anxious apprehension participants have impaired selection. • No effect on retrieval.

  35. Anxiety: Decreased Inhibition Impairs VLPFC Function During Selection • Anxious apprehension correlates negatively with VLPFC activity during selection (when retrieval demands are low). • No correlation during retrieval. VLPFC ROI

  36. Midazolam: Increased Inhibition Improves Selection Network Predictions Participants (RTs) • Increased competitive inhibition in the VLPFC layer improves selection when retrieval demands are low. • Midazolam improves selection when retrieval demands are low. • No effect on retrieval.

  37. Conclusions • Neural network model suggests that competitive inhibitory dynamics in prefrontal networks are critical for selection. • As predicted by model, participants high in anxious apprehension (linked to reduced GABAergic function) show impaired selection but not retrieval. • Consistent with clinical evidence for decision-making problems and intolerance of uncertainty in anxiety disorders. • Participants high in anxious apprehension show reduced left VLPFC recruitment during selection. • Could represent failure to activate inhibitory interneurons.

  38. Conclusions (cont.) • As predicted by model, midazolam (GABA agonist) improves selection when retrieval demands are low. • Suggests GABA agonists may be beneficial in treating cognitive, in addition to affective, symptoms of anxiety disorders.

  39. Ongoing and Future Directions • Study with selected high and low anxiety participants across multiple selection tasks. • Comparing underdetermined to prepotent competition (behavioral and fMRI studies). • Effects of depression on controlled retrieval.

  40. Thanks! • Professional research assistants: Paula Villar, Kirsten Orcutt, and Luka Ruzic • Undergraduate honors thesis students: Natalie Hutchison and Teesa Dutta • Clinical collaborators: Rosi Kaiser and Mark Whisman • All DEFD members for helpful input.

  41. Major Component Processes Involved in Executive Function Friedman, Hewitt, Willcutt, Young, Smolen, Miyake, O’Reilly, Hazy, Herd, Brant, Chatham

  42. Three Components of EFs • Inhibition • Stopping prepotent (dominant or automatic) responses (e.g., stop-signal) • Updating • Monitoring and rapid addition/deletion of the contents of working memory (e.g., n-back) • Shifting • Switching flexibly between tasks or mental sets (e.g., number-letter)

  43. Unity and Diversity .59 Plus-Minus .57 Shifting Number-Letter .46 Local-Global .56 .46 Keep Track .45 Updating .42 Tone Monitoring .63 Letter Memory .63 .40 Stroop .33 Inhibition Stop Signal .57 Antisaccade Miyake et al. (2000), Cognitive Psychology

  44. Unity and Diversity of EFs Unity Diversity Common EF = + UpdatingAbility Updating-Specific = + ShiftingAbility Shifting-Specific = + InhibitionAbility Inhibition-Specific

  45. Common EF Updating specific Shifting specific Keep Letter S2ba Num Col Cat Anti Stop Stroop Nested Factor Model .54 .53 .22 .49 .46 .58 .46 .58 .43 .41 .44 .37 .47 .42 .46

More Related