Search

Article
Peer Reviewed

Complementary contributions of basolateral amygdala and orbitofrontal cortex to value learning under uncertainty

UCLA Previously Published Works (2017)

We make choices based on the values of expected outcomes, informed by previous experience in similar settings. When the outcomes of our decisions consistently violate expectations, new learning is needed to maximize rewards. Yet not every surprising event indicates a meaningful change in the environment. Even when conditions are stable overall, outcomes of a single experience can still be unpredictable due to small fluctuations (i.e., expected uncertainty) in reward or costs. In the present work, we investigate causal contributions of the basolateral amygdala (BLA) and orbitofrontal cortex (OFC) in rats to learning under expected outcome uncertainty in a novel delay-based task that incorporates both predictable fluctuations and directional shifts in outcome values. We demonstrate that OFC is required to accurately represent the distribution of wait times to stabilize choice preferences despite trial-by-trial fluctuations in outcomes, whereas BLA is necessary for the facilitation of learning in response to surprising events.

Cover page: Complementary contributions of basolateral amygdala and orbitofrontal cortex to value learning under uncertainty

Creative Commons 'BY' version 4.0 license

Article
Peer Reviewed

Distinct patterns of outcome valuation and amygdala-prefrontal cortex synaptic remodeling in adolescence and adulthood

UCLA Previously Published Works (2015)

Adolescent behavior is typified by increased risk-taking, reward- and novelty-seeking, as well as an augmented need for social and environmental stimulation. This behavioral phenotype may result from alterations in outcome valuation or reward learning. In the present set of experiments, we directly compared adult and adolescent animals on tasks measuring both of these processes. Additionally, we examined developmental differences in dopamine D1-like receptor (D1R), dopamine D2-like receptor (D2R), and polysialylated neural cell adhesion molecule (PSA-NCAM) expression in animals that were trained on an effortful reward valuation task, given that these proteins play an important role in the functional development of the amygdala-prefrontocortical (PFC) circuit and mesocorticolimbic dopamine system. We found that adolescent animals were not different from adults in appetitive associative learning, but exhibited distinct pattern of responses to differences in outcome values, which was paralleled by an enhanced motivation to invest effort to obtain larger rewards. There were no differences in D2 receptor expression, but D1 receptor expression was significantly reduced in the striatum of animals that had experiences with reward learning during adolescence compared to animals that went through the same experiences in adulthood. We observed increased levels of PSA-NCAM expression in both PFC and amygdala of late adolescents compared to adults that were previously trained on an effortful reward valuation task. PSA-NCAM levels in PFC were strongly and positively associated with high effort/reward (HER) choices in adolescents, but not in adult animals. Increased levels of PSA-NCAM expression in adolescents may index increased structural plasticity and represent a neural correlate of a reward sensitive endophenotype.

Cover page: Distinct patterns of outcome valuation and amygdala-prefrontal cortex synaptic remodeling in adolescence and adulthood

Thesis
Peer Reviewed

Benefit, Burdens and Brain Correlates of Heightened Reward Sensitivity in Adolescence

Stolyarova, Alexandra
Advisor(s): Izquierdo Edler, Alicia

UCLA Electronic Theses and Dissertations (2020)

Increased exploration, risk-taking and reward-seeking are the hallmarks of adolescence. These and related behaviors prepare the young to transition from the parent’s nest to independent living. Adolescence is also a period of heightened structural and functional brain reorganization, particularly within the mesocorticolimbic dopamine system, frontal cortex, striatum and amygdala - an interconnected network of brain regions that supports reward-directed behavior. The goal of this dissertation to further our understanding of ways in which the adolescent brain interacts with rewards. In Chapter 2, I present the results of a set of experiments demonstrating that adolescent rats do not differ from adults in a simple form of stimulus-reward learning but are willing to invest greater effort to obtain larger rewards. While previous studies have focused on the role of neurodevelopmental changes in the dopamine system and within the striatum in heightened reward seeking in adolescence, our data suggest that synaptic remodeling within the frontal cortex and amygdala may also contribute to enhanced reward sensitivity. Chapter 3 explores the long-term consequences of prescription drug exposure during the adolescent period of heightened reward sensitivity in rats. The data suggest that adolescent exposure to both fluoxetine and methylphenidate impairs learning and cognitive flexibility in adulthood in male rats long after the drug administration has been terminated. Adolescent methylphenidate exposure has the direst consequences, impairing both the initial learning and reversal of reward contingencies. The data also reveal sex differences: while females take longer to learn the task, they are also less vulnerable to the negative effects of drug exposure. In Chapter 5, I compare adult and adolescent humans in their approaches to solving the credit assignment problem (i.e., the problem of discovering which choices are responsible for rewards, introduced in Chapter 4). The data provide preliminary evidence for enhanced contingent credit assignment in adolescence. While adults integrate information about their previous decisions and past outcomes to guide their subsequent choices, adolescents are more narrowly focused on the most recent choice-reward history. In Chapter 6, I discuss the implications of the present work and offer cautious advice on drug abuse prevention and improvements in pedagogical practice.

Cover page: Benefit, Burdens and Brain Correlates of Heightened Reward Sensitivity in Adolescence

Article
Peer Reviewed

Reductions in Frontocortical Cytokine Levels are Associated with Long-Lasting Alterations in Reward Valuation after Methamphetamine

UCLA Previously Published Works (2015)

Alterations in reward valuation are thought to have a central role at all stages of the addiction process. We previously reported work aversion in an effortful T-maze task following a binge exposure to methamphetamine, and no such changes in effort following escalating doses. Limitations of the T-maze task include its two available options, with an effort requirement, in the form of increasing barrier height, varying incrementally as a function of time, and reward magnitudes held constant. Reward preferences and choices, however, are likely affected by the number of options available and the manner in which alternatives are presented. In the present experiment, we investigated the long-lasting, off-drug effects of methamphetamine on reward choices in a novel effortful maze task with three possible courses of action, each associated with different effort requirements and reward magnitudes. Neuroinflammatory responses associated with drug exposure, proposed as one of the mechanisms contributing to suboptimal choices on effort-based tasks, were also examined. We investigated region-specific changes in pro- and anti-inflammatory markers in the mesocorticolimbic pathway after methamphetamine, and their relationship with animals' reward choices. We observed long-lasting, increased sensitivity to differences in reward magnitude in the methamphetamine group: animals were more likely to overcome greater effort costs to obtain larger rewards on our novel effortful maze task. These behavioral changes were strongly predicted by pronounced decreases in frontocortical cytokines, but not amygdalar or striatal markers. The present results provide the first evidence that neuroinflammatory processes are associated with alterations in reward valuation during protracted drug withdrawal.

Cover page: Reductions in Frontocortical Cytokine Levels are Associated with Long-Lasting Alterations in Reward Valuation after Methamphetamine

Article
Peer Reviewed

Positive and negative feedback learning and associated dopamine and serotonin transporter binding after methamphetamine

UCLA Previously Published Works (2014)

Learning from mistakes and prospectively adjusting behavior in response to reward feedback is an important facet of performance monitoring. Dopamine (DA) pathways play an important role in feedback learning and a growing literature has also emerged on the importance of serotonin (5HT) in reward learning, particularly during punishment or reward omission (negative feedback). Cognitive impairments resulting from psychostimulant exposure may arise from altered patterns in feedback learning, which in turn may be modulated by DA and 5HT transmission. We analyzed long-term, off-drug changes in learning from positive and negative feedback and associated striatal DA transporter (DAT) and frontocortical 5HT transporter (SERT) binding in rats pretreated with methamphetamine (mAMPH). Specifically, we assessed the reversal phase of pairwise visual discrimination learning in rats receiving single dose- (mAMPHsingle) vs. escalating-dose exposure (mAMPHescal). Using fine-grained trial-by-trial analyses, we found increased sensitivity to and reliance on positive feedback in mAMPH-pretreated animals, with the mAMPHsingle group showing more pronounced use of this type of feedback. In contrast, overall negative feedback sensitivity was not altered following any mAMPH treatment. In addition to validating the enduring effects of mAMPH on early reversal learning, we found more consecutive error commissions before the first correct response in mAMPH-pretreated rats. This behavioral rigidity was negatively correlated with subregional frontocortical SERT whereas positive feedback sensitivity negatively correlated with striatal DAT binding. These results provide new evidence for the overlapping, yet dissociable roles of DA and 5HT systems in overcoming perseveration and in learning new reward rules.

Cover page: Positive and negative feedback learning and associated dopamine and serotonin transporter binding after methamphetamine

Article
Peer Reviewed

Post-training depletions of basolateral amygdala serotonin fail to disrupt discrimination, retention, or reversal learning

UCLA Previously Published Works (2015)

In goal-directed pursuits, the basolateral amygdala (BLA) is critical in learning about changes in the value of rewards. BLA-lesioned rats show enhanced reversal learning, a task employed to measure the flexibility of response to changes in reward. Similarly, there is a trend for enhanced discrimination learning, suggesting that BLA may modulate formation of stimulus-reward associations. There is a parallel literature on the importance of serotonin (5HT) in new stimulus-reward and reversal learning. Recent postulations implicate 5HT in learning from punishment. Whereas, dopaminergic involvement is critical in behavioral activation and reinforcement, 5HT may be most critical for aversive processing and behavioral inhibition, complementary cognitive processes. Given these findings, a 5HT-mediated mechanism in BLA may mediate the facilitated learning observed previously. The present study investigated the effects of selective 5HT lesions in BLA using 5,7-dihydroxytryptamine (5,7-DHT) vs. infusions of saline (Sham) on discrimination, retention, and deterministic reversal learning. Rats were required to reach an 85% correct pairwise discrimination and single reversal criterion prior to surgery. Postoperatively, rats were then tested on the (1) retention of the pretreatment discrimination pair, (2) discrimination of a novel pair, and (3) reversal learning performance. We found statistically comparable preoperative learning rates between groups, intact postoperative retention, and unaltered novel discrimination and reversal learning in 5,7-DHT rats. These findings suggest that 5HT in BLA is not required for formation and flexible adjustment of new stimulus-reward associations when the strategy to efficiently solve the task has already been learned. Given the complementary role of orbitofrontal cortex in reward learning and its interconnectivity with BLA, these findings add to the list of dissociable mechanisms for BLA and orbitofrontal cortex in reward learning.

Cover page: Post-training depletions of basolateral amygdala serotonin fail to disrupt discrimination, retention, or reversal learning

Article
Peer Reviewed

Sex differences, learning flexibility, and striatal dopamine D1 and D2 following adolescent drug exposure in rats

UCLA Previously Published Works (2016)

Corticostriatal circuitry supports flexible reward learning and emotional behavior from the critical neurodevelopmental stage of adolescence through adulthood. It is still poorly understood how prescription drug exposure in adolescence may impact these outcomes in the long-term. We studied adolescent methylphenidate (MPH) and fluoxetine (FLX) exposure in rats and their impact on learning and emotion in adulthood. In Experiment 1, male and female rats were administered MPH, FLX, or saline (SAL), and compared with methamphetamine (mAMPH) treatment beginning in postnatal day (PND) 37. The rats were then tested on discrimination and reversal learning in adulthood. In Experiment 2, animals were administered MPH or SAL also beginning in PND 37 and later tested in adulthood for anxiety levels. In Experiment 3, we analyzed striatal dopamine D1 and D2 receptor expression in adulthood following either extensive learning (after Experiment 1) or more brief emotional measures (after Experiment 2). We found sex differences in discrimination learning and attenuated reversal learning after MPH and only sex differences in adulthood anxiety. In learners, there was enhanced striatal D1, but not D2, after either adolescent MPH or mAMPH. Lastly, also in learners, there was a sex x treatment group interaction for D2, but not D1, driven by the MPH-pretreated females, who expressed significantly higher D2 levels compared to SAL. These results show enduring effects of adolescent MPH on reversal learning in rats. Developmental psychostimulant exposure may interact with learning to enhance D1 expression in adulthood, and affect D2 expression in a sex-dependent manner.

Cover page: Sex differences, learning flexibility, and striatal dopamine D1 and D2 following adolescent drug exposure in rats

Article
Peer Reviewed

Contributions of anterior cingulate cortex and basolateral amygdala to decision confidence and learning under uncertainty.

UCLA Previously Published Works (2019)

The subjective sense of certainty, or confidence, in ambiguous sensory cues can alter the interpretation of reward feedback and facilitate learning. We trained rats to report the orientation of ambiguous visual stimuli according to a spatial stimulus-response rule that must be learned. Following choice, rats could wait a self-timed delay for reward or initiate a new trial. Waiting times increase with discrimination accuracy, demonstrating that this measure can be used as a proxy for confidence. Chemogenetic silencing of BLA shortens waiting times overall whereas ACC inhibition renders waiting times insensitive to confidence-modulating attributes of visual stimuli, suggesting contribution of ACC but not BLA to confidence computations. Subsequent reversal learning is enhanced by confidence. Both ACC and BLA inhibition block this enhancement but via differential adjustments in learning strategies and consistent use of learned rules. Altogether, we demonstrate dissociable roles for ACC and BLA in transmitting confidence and learning under uncertainty.

Cover page: Contributions of anterior cingulate cortex and basolateral amygdala to decision confidence and learning under uncertainty.

Article
Peer Reviewed

Methamphetamine blocks exercise effects on Bdnf and Drd2 gene expression in frontal cortex and striatum.

UCLA Previously Published Works (2015)

Exposure to drugs of abuse can produce many neurobiological changes which may lead to increased valuation of rewards and decreased sensitivity to their costs. Many of these behavioral alterations are associated with activity of D2-expressing medium spiny neurons in the striatum. Additionally, Bdnf in the striatum has been shown to play a role in flexible reward-seeking behavior. Given that voluntary aerobic exercise can affect the expression of these proteins in healthy subjects, and that exercise has shown promise as an anti-addictive therapy, we set out to quantify changes in D2 and Bdnf expression in methamphetamine-exposed rats given access to running wheels. Sixty-four rats were treated for two weeks with an escalating dose of methamphetamine or saline, then either sacrificed, housed in standard cages, or given free access to a running wheel for 6 weeks prior to sacrifice. Rats treated with methamphetamine ran significantly greater distances than saline-treated rats, suggesting an augmentation in the reinforcement value of voluntary wheel running. Transcription of Drd2 and Bdnf was assessed via RT-qPCR. Protein expression levels of D2 and phosphorylation of the TrkB receptor were measured via western blot. Drd2 and Bdnf mRNA levels were impacted independently by exercise and methamphetamine, but exposure to methamphetamine prior to the initiation of exercise blocked the exercise-induced changes seen in rats treated with saline. Expression levels of both proteins were elevated immediately after methamphetamine, but returned to baseline after six weeks, regardless of exercise status.

Article
Peer Reviewed

Methamphetamine blocks exercise effects on Bdnf and Drd2 gene expression in frontal cortex and striatum

UCLA Previously Published Works (2015)

Exposure to drugs of abuse can produce many neurobiological changes which may lead to increased valuation of rewards and decreased sensitivity to their costs. Many of these behavioral alterations are associated with activity of D2-expressing medium spiny neurons in the striatum. Additionally, Bdnf in the striatum has been shown to play a role in flexible reward-seeking behavior. Given that voluntary aerobic exercise can affect the expression of these proteins in healthy subjects, and that exercise has shown promise as an anti-addictive therapy, we set out to quantify changes in D2 and Bdnf expression in methamphetamine-exposed rats given access to running wheels. Sixty-four rats were treated for two weeks with an escalating dose of methamphetamine or saline, then either sacrificed, housed in standard cages, or given free access to a running wheel for 6 weeks prior to sacrifice. Rats treated with methamphetamine ran significantly greater distances than saline-treated rats, suggesting an augmentation in the reinforcement value of voluntary wheel running. Transcription of Drd2 and Bdnf was assessed via RT-qPCR. Protein expression levels of D2 and phosphorylation of the TrkB receptor were measured via western blot. Drd2 and Bdnf mRNA levels were impacted independently by exercise and methamphetamine, but exposure to methamphetamine prior to the initiation of exercise blocked the exercise-induced changes seen in rats treated with saline. Expression levels of both proteins were elevated immediately after methamphetamine, but returned to baseline after six weeks, regardless of exercise status.