PVLV Model ofPhasic Dopamine Learning R. O’Reilly T. Hazy, J. Reynolds, G. Frank
Temporal Difference Dopamine Reward Model Brain Dopamine Reward Response = Reward Occurred – Reward Predicted Schultz, Dayan & Montague 1997 DA ↑ Unexpected Reward DA ↑ DA↔ Expected Reward DA ↑ DA ↓ Unexpected No-Reward
Phasic Dopamine Firing • Transition from Reward/US to CS onset (Schultz et al) • TD model says why, but not how (neural mechanisms) • And TD not a good fit for growing set of data..
PVLV Model • Central Nucleus of the Amygdala (CNA) = CS -> DA • Ventral Striatum (NAc, patch) = US -> no DA
PVLV Model • PV = Cancelling US burst • LV = Driving CS burst • No chaining: just simpler delta-rule/Rescorla Wagner
PVLV Predictions • CS – driven DA is dissociable from US – driven: • Conditioned orienting & autoshaping to a CS = CNA, but not to US • CS DA is not subject to blocking effect • CS DA cannot drive further CS learning: 1st order (CNA) is dissociable from 2nd order (BLA), and no 3rd order • Time is just another input; no temporal chaining as in TD
ROI Activation Analysis after Full Brain Statistical Analysis Guido K.W. Frank, M.D. The Children’s Hospital, University of Colorado Denver
Receiving Sucrose (US) Unexpected AN greater than CW p=0.005 5vox uncorr. SVC p<0.02, FEW, FDR, Cluster
Receiving Sucrose (US) Unexpected Time Activity Curves Anorexia Nervosa VTA CNA-L CNA-R AVS-L AVS-R VTA CNA-L CNA-R AVS-L AVS-R Control Women
Influence of Emotion/Reward & Punishment on EF:Incentive Stroop Task Jeff Spielberg, with Wendy Heller, Gregory A. Miller, Laura Crocker, Stacie Warren, Christina Murdock-Jordan, post-doc Dave Towers, Brad Sutton & Tracey Wzalek at BIC, and Center colleagues Marie Banich & Randy O’Reilly Department of Psychology and Beckman Biomedical Imaging Center, University of Illinois at Urbana-Champaign
Influence of Emotion/Reward & Punishment on EF • Multiple ways to conceptualize emotional processes psychologically • Circumplex models, valence, bipolarity • Valence/arousal, positive affect/negative affect • Valence-based dichotomies prevalent (pos. vs. neg. emotionality/temperament/extraversion) • Fundamental motive systems that underlie behavior • Appetitive/approach/behavioral activation/incentive motivation • Defensive/withdrawal/behavioral inhibition/aversive motivation
Influence of Emotion/Reward & Punishment on EF • No comprehensive model(s) of brain function or structure integrating roles or mechanisms of brain regions implicated in emotion: • Left & Right DLPFC: positive/negative affect • dACC: anxiety • Amygdala: fear • Nucleus accumbens: reward/punishment • Posterior ACC: emotional autobiographical memory • Orbital frontal cortex: reward punishment • Distinct lines of research, most not integrated with each other
Influence of Emotion/Reward & Punishment on EF • Lateralization a predominant feature of models of PFC organization for emotion • Divided according to superior/inferior lines • Davidson motivation model, also Harmon-Jones, Coan, & Allen, others • Approach/withdrawal mapping onto left vs. right DLPFC • Heller, Miller, Banich, and others’ valence/arousal model • Positive/negative valence mapping onto left vs. right DLPFC • Adds emotional arousal & right parietal activity
Prefrontal Lateralization for Emotional Processes Approach Motivation Withdrawal Motivation L R Positive Emotion Negative Emotion
Influence of Emotion/Reward & Punishment on EF Positive valence Approach Motivation Attachment Trait Anger Anger Out Anxious Apprehension Anxious Arousal Anhedonic Depression Distinguishable, lateralized prefrontal areas sensitive to… L R L R L R (From Engels et al., 2007; Herrington et al., 2005; Herrington et al., 2009; Spielberg et al., 2008; Stewart et al., 2008; all regions depicted replicated in at least two studies)
Patterns of lateralization for emotion tend to be reversed for orbital frontal cortex (OFC) Positive/negative valence mapping onto right vs. left OFC Note LDLPFC spot! Influence of Emotion/Reward & Punishment on EF deAraujo, Rolls, Kringelbach, McGlone, & Phillips, 2003
Influence of Emotion/Reward & Punishment on EF • Incentive Stroop Task: • Engages motivational systems, emotional systems, and executive functions simultaneously • TASK • Press a button ASAP when a word appears • AFFECTIVE CONTEXT COMPONENT OF THE TASK • IGNORE the meaning of the word, which can be positive, negative, or neutral • Design allows us to examine the effect of the emotional content of the irrelevant information (affective context) on the ability to ignore that information
Influence of Emotion/Reward & Punishment on EF • MOTIVATIONAL COMPONENT OF THE TASK • Before each word is shown, participants see a cue which tells them whether they will win and/or lose money depending on how fast they push the button • If they push the button fast enough they get the positive outcome on that trial (i.e., win money or avoid losing money) • If they don’t push the button fast enough they get the negative outcome on that trial (i.e., lose money or miss winning money) • On some trials, they neither win nor lose money • Allows us to examine the effect of anticipating rewards and punishments on the ability to ignore irrelevant information
For dollar sign on left: if it’s green, can win money if push button fast enough if it’s grey, can’t win money on that trial For dollar sign on right: if it’s red, can lose money if don’t push button fast enough if it’s grey, can’t lose money on that trial
Fast enough = win money Too slow = do not win money Fast enough = win money Too slow = lose money Fast enough = do not lose money Too slow = lose money Do not win or lose money regardless if they are fast or slow
Incentive Stroop Task Cue ISI Emotional word ISI Feedback Time within trial
Influence of Emotion/Reward & Punishment on EF • Recruiting 3 groups of subjects according to PANAS scores • High positive, low negative affect • High negative, low positive affect • Low negative, low positive affect • Participants are run in counterbalanced EEG/fMRI sessions • SCIDs & assessment of EF components via neuropsychological tests
Influence of Emotion/Reward & Punishment on EF • Preliminary behavioral findings: • Emotional content of the irrelevant information affects ability to ignore that information • Pleasant or unpleasant words elicit slower RTs than do neutral words • Emotional words thus harder to ignore • Motivational context also affects the ability to ignore irrelevant information • RT faster on trials in which reward or punishment is possible • Thus, possible rewards and punishments make it easier to ignore irrelevant information • Findings indicate effects of both affective context and motivation on executive function (as measured by RT)
Influence of Emotion/Reward & Punishment on EF • Activation in left DLPFC (yellow) when viewing cues signaling the potential for reward (associated with faster RTs) • Activation in right OFC (green) when receiving rewards (associated with faster RT) • Activation in left OFC (red) when receiving punishments (also associated with faster RT)
Influence of Emotion/Reward & Punishment on EF • Preliminary results thus: • Confirm effects of both affective and motivational contexts on executive function • Replicate opposing patterns of lateralization for DLPFC and OFC • Allow us to examine timing of regional activity, connectivity, relationships of regional and temporal dynamics to emotional disposition • Can be extended to examine dysfunctional relationships in depression & anxiety
Effects of Anxiety on Selection Among Competing Options Hannah R. Snyder & Yuko Munakata (Project 5) in collaboration with: Marie T. Banich (Project 1) Tim Curran & Erika Nyhus (Imaging Core) With consultation from Project 3
Anxiety and Uncertainty • Anxious apprehension (worry) is linked to intolerance of uncertainty(e.g. Ladouceur, Talbot & Dugas, 1997), decision-making problems, and indecisiveness(e.g. Sachdev & Malhi, 2005). • Prominent symptoms of anxiety disorders including GAD and OCD. • Why? • Approach this question using our framework for understanding one aspect of EF: selection among competing options.
Selection Among Competing Options • We constantly face the need to choose one option from among multiple valid choices. • e.g. grocery shopping, selecting a retirement plan, or choosing a word to express a thought. • Selecting between multiple options is time-consuming and effortful (e.g. Iyenger & Lepper, 2000; Sethi-Iyenger et al., 2004; Snyder & Munakata, 2008). • Left ventrolateral prefrontal cortex (VLPFC) is involved in selection (e.g. Thompson-Schill et al., 1997,1998; Schnur et al., 2009). • Illinois center colleagues Engels, Heller, & Miller have shown that left VLPFC is involved in anxious apprehension. • What specific mechanisms might support selection, and how might they be affected by anxiety?
Selection Among Competing Options • Test neural network predictions about selection using a well-controlled language-production task: verb generation.
Pyramidal Cell Pyramidal Cell - + - + GABAergic Interneurons Neural Network Model • Demonstrates that competitive, inhibitory dynamics among neurons in prefrontal cortical networks support selection among competing alternatives. • Amplify activity of most active representation and suppress activity of competing representations, via inhibitory, GABAergic interneurons.
Neural Network Model • Suggest that prefrontal GABA function plays key role in selection and breakdown of this process. • Makes sense of findings which were previously disconnected from each other, linking anxiety to: • Reduced GABA(e.g. Kalueff & Nutt., 2007) • VLPFC dysfunction(e.g. Engles et al., 2007).
Competitive Inhibition & Selection • Neural network predictions: • Anxiety (reduced neural inhibition) impairs selection and associated VLPFC activity, even in a simple, non-affective language-production task. • The GABA agonist midazolam (increased neural inhibition) improves selection. • Retrieval from semantic memory is unaffected. • These predictions were supported in 3 studies.
Anxiety: Decreased Inhibition Impairs Selection Network Predictions Participants (RTs) • Reduced competitive inhibition in the VLPFC layer impairs selection. • High anxious apprehension participants have impaired selection. • No effect on retrieval.
Anxiety: Decreased Inhibition Impairs VLPFC Function During Selection • Anxious apprehension correlates negatively with VLPFC activity during selection (when retrieval demands are low). • No correlation during retrieval. VLPFC ROI
Midazolam: Increased Inhibition Improves Selection Network Predictions Participants (RTs) • Increased competitive inhibition in the VLPFC layer improves selection when retrieval demands are low. • Midazolam improves selection when retrieval demands are low. • No effect on retrieval.
Conclusions • Neural network model suggests that competitive inhibitory dynamics in prefrontal networks are critical for selection. • As predicted by model, participants high in anxious apprehension (linked to reduced GABAergic function) show impaired selection but not retrieval. • Consistent with clinical evidence for decision-making problems and intolerance of uncertainty in anxiety disorders. • Participants high in anxious apprehension show reduced left VLPFC recruitment during selection. • Could represent failure to activate inhibitory interneurons.
Conclusions (cont.) • As predicted by model, midazolam (GABA agonist) improves selection when retrieval demands are low. • Suggests GABA agonists may be beneficial in treating cognitive, in addition to affective, symptoms of anxiety disorders.
Ongoing and Future Directions • Study with selected high and low anxiety participants across multiple selection tasks. • Comparing underdetermined to prepotent competition (behavioral and fMRI studies). • Effects of depression on controlled retrieval.
Thanks! • Professional research assistants: Paula Villar, Kirsten Orcutt, and Luka Ruzic • Undergraduate honors thesis students: Natalie Hutchison and Teesa Dutta • Clinical collaborators: Rosi Kaiser and Mark Whisman • All DEFD members for helpful input.
Major Component Processes Involved in Executive Function Friedman, Hewitt, Willcutt, Young, Smolen, Miyake, O’Reilly, Hazy, Herd, Brant, Chatham
Three Components of EFs • Inhibition • Stopping prepotent (dominant or automatic) responses (e.g., stop-signal) • Updating • Monitoring and rapid addition/deletion of the contents of working memory (e.g., n-back) • Shifting • Switching flexibly between tasks or mental sets (e.g., number-letter)
Unity and Diversity .59 Plus-Minus .57 Shifting Number-Letter .46 Local-Global .56 .46 Keep Track .45 Updating .42 Tone Monitoring .63 Letter Memory .63 .40 Stroop .33 Inhibition Stop Signal .57 Antisaccade Miyake et al. (2000), Cognitive Psychology
Unity and Diversity of EFs Unity Diversity Common EF = + UpdatingAbility Updating-Specific = + ShiftingAbility Shifting-Specific = + InhibitionAbility Inhibition-Specific
Common EF Updating specific Shifting specific Keep Letter S2ba Num Col Cat Anti Stop Stroop Nested Factor Model .54 .53 .22 .49 .46 .58 .46 .58 .43 .41 .44 .37 .47 .42 .46