Developmental Trajectories of Anhedonia in Preclinical Models

Contents 1 The Concept of Anhedonia and Its Operationalization in Preclinical Models 1.1 Novel Tools for the Study of Anhedonia in Preclinical Models 1.2 The Reward Circuit and Its Study in Experimental Models 2 Anhedonia and Early-Life Adversity (ELA) 2.1 Why Study ELA and Anhedonia? The Human Landscape 2.2 What Is ELA and How Do We Model It? 2.3 Anhedonia Following ELA Involves a Developing Reward Circuit 2.4 Anhedonia After ELA: Manifestations and Sex Speci ﬁ city 3 General Conclusions References Abstract This chapter discusses how the complex concept of anhedonia can be operationalized and studied in preclinical models. It provides information about the development of anhedonia in the context of early-life adversity, and the power of preclinical models to tease out the diverse molecular, epigenetic, and network mechanisms that are responsible for anhedonia-like behaviors. Speci ﬁ cally, we ﬁ rst discuss the term anhedonia, reviewing the conceptual components underlying reward-related behaviors and distinguish anhedonia pertaining to de ﬁ cits in motivational versus consummatory behaviors. We then describe the repertoire of experimental approaches employed to study anhedonia-like behaviors in preclinical models, and the progressive re ﬁ nement over the past decade of both experimental instruments (e.g., chemogenetics, optogenetics) and


The Concept of Anhedonia and Its Operationalization in Preclinical Models
Anhedonia denotes a transdiagnostic construct that necessitates understanding the role of the reward circuit and its altered function in the pathophysiology of mental illnesses (Bedwell et al. 2014;Lake et al. 2017).Thus, the Research Domain Criteria (RDoC) framework put forth by the National Institute of Mental Health (NIMH) in the USA suggests that the neurobiological basis of anhedonia, together with empirical behavioral measures in humans and animal models is key to understanding how specific conditions including genetic make-up or early-life adversity (ELA) might lead to mental health vulnerabilities.Indeed, the common presence of anhedonia as a herald or component of many psychiatric illnesses supports the notion that the disruption of reward-circuit function which characterizes anhedonia is a common mechanism for several neuropsychiatric disorders.The concept of anhedonia and the diverse definitions of the term are well addressed in other chapters in this tome.Here we note that, in both humans and experimental models, there are multiple domains of anhedonic behaviors, and these may involve distinct neural mechanisms and processes (Der-Avakian and Markou 2012; Shankman et al. 2014;Zald and Treadway 2017).Preclinical studies have identified several behavioral components that are grouped within the concept of anhedonia (Berridge and Kringelbach 2015;Der-Avakian and Pizzagalli 2018;Levis et al. 2021).Some studies of anhedonia have focused on deficits in motivation or anticipatory reward (Sherdell et al. 2012;Bryant et al. 2017;Szczepanik et al. 2019), whereas others emphasize the importance of assessing consummatory reward (Kringelbach and Berridge 2016;Wright et al. 2020).In addition, the attenuation of reward-seeking behaviors observed in humans or preclinical models can be limited to some reinforcers but not others (e.g., social vs. food rewards), further complicating the definition of anhedonic behaviors.Yet, whereas there is a vibrant ongoing discussion of the definition and boundaries of the term anhedonia, a broad consensus is emerging regarding the brain circuitry that is involved, namely, the reward circuitry.Indeed, human and experimental animal studies conclude that distinct deficits in reward-seeking behaviors which comprise specific aspects of anhedonia all arise from selective and yet overlapping disruption of the operations of specific nodes and connections within the reward circuit.
The facts above suggest that key insights into the nature and mechanisms of anhedonia require interrogation of the reward circuits, yet the ability to do so is limited in humans.Whereas the advent of structural and functional magnetic resonance imaging has proven invaluable to visualizing the human brain and its circuits, establishing causality, teasing apart the distinct roles of genetics and environment, and overcoming other uncontrolled confounders limit the capacity of human studies to fully uncover the issues revolving around anhedonia.Thus, the establishment of animal models with the goals of studying anhedonia and identifying salient projections, nodes, and circuits together with molecules and mechanisms that are disrupted are key to elucidating the origins and trajectories of anhedonia.

Novel Tools for the Study of Anhedonia in Preclinical Models
Across neuropsychiatric diagnoses, anhedonia can manifest as consummatory and/or motivational deficits (Sherdell et al. 2012;Kringelbach and Berridge 2016;Bryant et al. 2017;Szczepanik et al. 2019;Wright et al. 2020).These responses to rewarding stimuli are often relatively easy to study in rodents, yet historically, anhedonia has not been formally distinguished in rodent studies from measures of depression-like behaviors.Thus, often, repeated exposure to aversive conditions has been utilized along with reward specific tasks to model clinical depressive symptoms.Behavioral despair tests, such as the forced swim (FS) tail suspension (TS), and chronic stress (CS) and chronic mild stress (CMS) paradigms have been used to measure anhedonia and antidepressant-like behavior in rodents (Katz 1982;Willner et al. 1992;Cryan et al. 2005;Castagné et al. 2011).However, these tasks often lead to difficulties in interpretation and reproducibility and thus may not be optimal measures of the diverse facets of anhedonia.Therefore, more complex and nuanced behavioral tasks are currently used to measure aspects of anhedonia in preclinical models (Table 1), as described below.
Reward circuit dysfunction, which is thought to underlie aspects of anhedonia, is often studied using tasks involving motivated behavior, such as drug seeking, food seeking, and the seeking of sex-reward cues.Notably, reward consumption at low effort can be distinguished from highly motivated, effortful reward seeking.These two types of tasks have dissociable underlying neural processes (Berridge and Robinson 2003;Vanderschuren et al. 2005;Baldo and Kelley 2007;Bentzley et al. 2013;Berridge and Kringelbach 2015;Salamone et al. 2016;Volkow et al. 2017) that may therefore be differentially susceptible to disruption and the production of anhedonia.These distinct types of reward seeking behaviors can be individually measured using tasks such as taste reactivity assays (Smith et al. 2010), Developmental Trajectories of Anhedonia in Preclinical Models intracranial self-stimulation (Olds and Milner 1954;Carlezon and Chartoff 2007), drug or food self-administration and relapse (Berridge and Aldridge 2009), reinforcement learning (Der-Avakian et al. 2013;Kangas et al. 2021), and others.
In both humans and rodents, an eloquent behavioral tool for simultaneously studying both consummatory and motivational aspects of reward in the setting of anhedonia capitalizes on behavioral economic theory, which stipulates that consumption of any commodity is sensitive to increasing price.Relative sensitivity to increasing price is referred to as "demand elasticity" (Hursh 1980).Inelastic demand, or relative insensitivity to price, is a feature of the excessive reward seeking associated with substance use disorders (Bickel et al. 2014), whereas relatively high sensitivity to increasing price, or a lack of motivation to obtain a reward at high cost, may be a feature of anhedonia.This behavior is distinct from reward intake when required effort is very low.Specifically, while consumption that persists at high cost is more reliant on motivational processes, drug consumption when cost is low corresponds to hedonic value, or "liking" of the drug, governed by a so-called hedonic setpoint (Hursh and Silberberg 2008;Bickel et al. 2014;Strickland et al. 2019).Anhedonia may therefore manifest as reduced "liking," or decreased hedonic setpoint for a given reinforcer, independent of changes to demand elasticity.Thus, behavioral economic tasks allow for the dissociation of anhedonic behaviors resulting from motivational deficits from those resulting from deficits in the hedonic aspects of reward consumption.Indeed, the neural substrates of demand elasticity and hedonic setpoint for drug rewards appear to be distinct (Bentzley and Aston-Jones 2015;Bolton et al. 2018b;Salamone et al. 2018;Levis et al. 2019).
In rodents, demand elasticity and hedonic setpoint for rewards such as palatable food or abused drugs can be modeled by examining intake at different "prices," operationalized as the amount of effort required to receive one unit of the reward (e.g., a single drug infusion or a single food pellet) (Hursh and Silberberg 2008;Oleson andRoberts 2008, 2009;Bentzley et al. 2013Bentzley et al. , 2014;;Newman and Ferrario 2020).For example, using this method, early-life adversity (ELA) leads to reduced hedonic setpoint for cocaine in male rats (Bolton et al. 2018b), suggestive of anhedonia.Strikingly, in ELA-reared female rodents there is a distinct lack of anhedonia.Rather, females have enhanced motivation (reduced demand elasticity) to obtain both opioid drugs and palatable food rewards, and no alteration in hedonic setpoint (Levis et al. 2019).Interestingly, in humans, there are no compelling data identifying sex differences in the prevalence and pathophysiology of anhedonia, although sex-specific effects have not been extensively probed.Yet, these disparate findings in preclinical models demonstrate the power of sophisticated tests to tease out distinct, sex-specific disruptions of reward circuits by ELA or other insults that are associated with vulnerability to anhedonia in humans.

The Reward Circuit and Its Study in Experimental Models
The reward circuitry is complex, encompassing nodes including the nucleus accumbens (NAc), prefrontal cortex (PFC), ventral tegmental area (VTA), ventral pallidum (VP), amygdala (Amyg), hippocampus (HPC), and paraventricular nucleus of the thalamus (PVT) (Fig. 1).These are engaged during reward processing and related choices.The NAc modulates the response to reward-related cues, as well as the value of expected versus actual reward outcomes.Studies to date have largely focused on glutamatergic and dopaminergic input pathways to the NAc and have demonstrated its role in integrating excitatory and inhibitory input to signal the salience of rewarding stimuli.These stimuli in turn are encoded via projections to and from limbic structures (Fig. 1) (Ballard et al. 2011;Tritsch et al. 2012;Britt et al. 2012;Bagot et al. 2015;Ferenczi et al. 2016;Robbins 2016).
Several neurotransmitters and neuromodulators convey information from and to the NAc (Fig. 1).Specifically, glutamatergic projections from cortical, thalamic, hippocampal, and amygdalar regions terminate in the NAc and mediate behavioral effects via AMPA, NMDA, and metabotropic glutamate (mGluR) receptors (Krystal et al. 2003;Cozzoli et al. 2012;Wang et al. 2012).For instance, reward seeking can be restricted via glutamatergic VP neurons increasing the activity of GABA VTA neurons (Tooley et al. 2018).Dopamine plays a role in motivation and reward and has been extensively studied.Dopamine influences incentive salience and instrumental behaviors in cue reward tasks (Peciña et al. 2003), and blocking dopamine receptors in the NAc reduces the effort expended to obtain a reward (Aberman et al. 1998).Serotonin also plays a role: dorsal raphe serotonin transporter (SERT) terminals, which synapse onto VTA dopaminergic neurons, increase rewarding behaviors (Wang et al. 2019).
Beyond the classically known neurotransmitters, several peptides and neuromodulators are expressed in reward circuit nodes to influence behaviors.Opioids and endocannabinoids are well-established major neurochemical mediators of reward responsiveness (Pecina and Berridge 2005;Mahler et al. 2007).Importantly, focusing on consummatory anhedonia, several peptides are expressed and function within the reward circuit.These include orexin and neuropeptide Y which modulate food intake (Van Den Heuvel et al. 2015;Castro et al. 2016), and corticotropin-releasing hormone (CRH), a stress responsive peptide and its receptors (Peciña et al. 2006;Lemos et al. 2012;Walsh et al. 2014;Peng et al. 2017;Bolton et al. 2018a).
Whereas the complexity of the neuroanatomical and molecular interactions described above is daunting, animal models allow the use of novel and evolving instruments and techniques to address these intricacies.Specifically, they enable both mapping and cell-type-specific and projection-specific manipulation of select components of the reward circuit.Anterograde and retrograde tracers have been extensively used to visualize the connectivity between circuit nodes (Nassi et al. 2015;Tervo et al. 2016;Itoga et al. 2019;Engelke et al. 2021).These connections can be interrogated further, with the use of Designer Receptors Exclusively Activated by Designer Drugs (DREADDs) (Mahler et al. 2014(Mahler et al. , 2019) ) and channelrhodopsins (Miyazaki et al. 2014;Cole et al. 2018) to dissect brain region and cell type specific functional control of behavior.For instance, in rats and mice, anterograde and retrograde tracing has identified novel CRH expressing projection sources to the NAc from regions integrating reward and stress circuits (Birnie et al. 2020a), and optogenetic activation of a CRH+ PVT-NAc projection reduced food seeking behavior (Engelke et al. 2021), identifying a novel role for a stress-regulated peptide that influences reward behaviors.

Why Study ELA and Anhedonia? The Human Landscape
In humans, ELA is linked to several mental health disorders that indicate dysfunction of the operation of the reward circuit.Functional magnetic resonance imaging (fMRI) studies have probed the functional activation of components of the reward circuitry in individuals who have experienced early-life adversity and identified several deficits (Boecker et al. 2014;Corral-Frías et al. 2015).ELA and subsequent emotional problems differ in men and women, with women with a history of earlylife trauma being more likely to crave comfort food and opioids (Mason et al. 2013).These differences between the sexes are a result of both intrinsic differences in the operation of the reward circuit and sex dimorphism in the response to ELA and to stress in general (Davis and Pfaff 2014;Walker et al. 2017).

What Is ELA and How Do We Model It?
Early-life adversity is a heterogenous concept.In humans, it typically denotes low socioeconomic level, poverty, parental depression, neglect or abuse and a violent, chaotic environment (Short and Baram 2019;Luby et al. 2020).These complex contexts generate and convey numerous types of signals to the developing infant and child, including significant emotional and sometimes physical anguish.Yet, in humans and especially in experimental models, the rich array of signals from caretakers and the environment, which reach and activate selective brain regions and nodes of the stress and reward circuits are often simply summed up as "stress," and considered interchangeable with activation of the neuroendocrine stress response.Instead, ELA is a multidimensional construct, and simply looking at the activation of the hypothalamic-pituitary-adrenal system may not recapitulate its numerous effects on the different functions of the brain (Molet et al. 2016a;Bolton et al. 2018a).It is also likely that different types of ELA and even the same type of ELA generated at different developmental stages might exert distinct differences to mesolimbic structures which undergo significant growth, maturation, and plasticity throughout specific sensitive periods (Tottenham 2020;Marini et al. 2020;Birnie et al. 2020b;Luby et al. 2020).
The development of preclinical models for ELA provides basic and translational scientists with the opportunity to understand complex neural mechanisms using techniques and approaches that are not possible in humans (Baram et al. 2012).Indeed, several approaches have been used to generate adversity early in life.Maternal separation has been used extensively to study the effects of such adversity/stress, and several variants exist including daily short (3-4 h) separation or a single prolonged deprivation.These models have generally yielded deficits in cognitive abilities as well as anxiety-like, depression-like and addiction-like behaviors, yet the development of anhedonia has not been consistent (Matthews et al. 1996;Leventopoulos et al. 2009;Nelson et al. 2009;Andersen 2019).More recently, with the aim of generating a naturalistic, highly reproducible model for early-life adversity, a paradigm of simulated poverty, using cages with limited bedding and nesting material (LBN) in rodents, has been devised and used extensively around the world.This environment stresses the dams and leads to disruption of maternal caring behaviors (Ivy et al. 2008;Molet et al. 2016a).This translates into unpredictable and fragmented sensory signals received by the developing pups.Whereas the overall duration and quality of maternal care remain unaltered, the pattern of their signals to pups is aberrant.Sensory signals from the environment are required for normal maturation of brain circuits including visual and auditory systems (Espinosa and Stryker 2012;Short and Baram 2019).It is believed that aberrant patterns of tactile signals from the dam may interfere with the maturation of reward-and stress-related circuits in the pups (Short and Baram 2019;Birnie et al. 2020b;Luby et al. 2020).Indeed, aberrant brain circuit maturation is generated in the pups, evidenced with MRI (Molet et al. 2016b;Bolton et al. 2018a) and manifesting as impaired memory as well as deficits in emotional-like and reward-executed behaviors, measured by food consumption, social play, and hedonic setpoint for cocaine.A third model of ELA involves cross-fostering, aiming to model early postnatal instability in humans.Starting during the first 2 days of birth, pups are cross-fostered with a lactating dam.This intervention allows for distinguishing between relatedness and environment, thus allowing researchers to recognize how genes and environment interact to influence developmental processes and later adult behaviors (Hager et al. 2009;Ventura et al. 2013;McCarty 2017).Comprehensive reviews of these preclinical models have been published (Molet et al. 2014;Walker et al. 2017).

Anhedonia Following ELA Involves a Developing
Reward Circuit In the context of the mechanisms by which ELA generates anhedonic behaviors, it is useful to consider that the reward circuity overlaps with nodes of the stress circuit, and that both are in a state of maturation and heightened plasticity during the developmental epochs comprising ELA (Avishai-Eliner et al. 2002;Birnie et al. 2020b).
Robust evidence exists for profound, likely permanent changes in brain reward and stress systems of individuals experiencing ELA, including mesolimbic and extended amygdala circuits, and neurotransmitter/neuromodulator systems including dopamine, endogenous opioids, and CRH.These enduring changes may promote vulnerability to anhedonia.At the circuit level, amygdala-PFC structural connectivity was augmented in adult male rats that had experienced ELA relative to controls (Bolton et al. 2018a).Pre-weaning LBN males, but not females, have reduced BLA-PFC and altered PFC-striatum resting state functional connectivity (Guadagno et al. 2018a, b), a finding that persists into adulthood and is accompanied by reduced sucrose preference and social interaction (Yan et al. 2017).Disruption of the early maturation of BLA-PFC connections has been reported after maternal separation as well (Brenhouse et al. 2013;Honeycutt et al. 2020;Nieves et al. 2020), further implicating this circuit in the effects of ELA.Notably, human studies suggest that ELA's impact on amygdala development is critical for the resulting depression and anxiety (Callaghan and Tottenham 2016;Fareri and Tottenham 2016), and potentially for anhedonia as well.
Changes in neurotransmitter and neuromodulators in both reward and stress circuitries result from ELA.For example, endogenous opioid systems are enduringly altered by ELA, a fact that may impact pleasure or other reward-relevant processes (Peciña et al. 2019).Maternal separation persistently alters endogenous opioid peptides, as well as opioid and dopamine receptor expression in reward and stressrelated areas, including striatum, midbrain, hippocampus, and hypothalamus in both a sex-and ELA timing-dependent manner (Ploj et al. 1999(Ploj et al. , 2001(Ploj et al. , 2003;;Gustafsson et al. 2008;Chang et al. 2019).The development of the mesolimbic dopamine system is also strongly impacted by ELA (Rodrigues et al. 2011;Ventura et al. 2013;Peña et al. 2014;Bonapersona et al. 2018), thereby potentially disrupting dopamine-dependent incentive motivation and learning.
ELA leads to enduring changes in the expression levels of the stress-sensitive neuropeptide CRH in the amygdala central nucleus (CeA) (Dubé et al. 2015) and hippocampus (Ivy et al. 2010), and is associated with major changes in circuit functions (Brunson et al. 2005;Ivy et al. 2010).Such changes in circuit function are apparent from both neuronal activation and from performance in tasks probing the reward circuits and specifically anhedonia.For example, palatable food, social play, and acute cocaine reward induce a stronger Fos response in CeA of ELA males than of control males, an effect accompanied by anhedonia-like behavioral responses to those same rewards (Bolton et al. 2018a, b).These findings suggest that ELA promotes aberrant activation of stress-circuit nodes during tasks that typically engage the reward circuit exclusively.

Anhedonia After ELA: Manifestations and Sex Specificity
The expression of anhedonia following ELA in preclinical models appears to involve interactions among the timing and nature of the ELA paradigm, biological sex, and the nature of the behavioral tests (Matthews and Robbins 2003;Rüedi-Bettschen et al. 2005;Der-Avakian and Markou 2010;Leussis et al. 2012;Molet et al. 2016a;Lukkes et al. 2017;Di Segni et al. 2019;Luby et al. 2020).For example, in male rodents, ELA imposed via rearing for 1 week in cages with limited bedding and nesting materials leads to enduring anhedonia for both natural and drug rewards.This includes blunted sucrose and palatable food preference, and reduced interest in social play (Molet et al. 2016a;Rincón-Cortés and Sullivan 2016;Yan et al. 2017;Bolton et al. 2018a, b).Using a behavioral economic assay of cocaine seeking, ELA does not change motivation to obtain drug at high effort (demand elasticity), but leads to reduced hedonic setpoint for cocaine in male rats when the drug is "free" (Bolton et al. 2018b).This suggests that, in addition to anhedonia for natural rewards (Bolton et al. 2018a), these animals are anhedonic for drug rewards.Such anhedonia is not observed in female rats after LBN (Levis et al. 2019), however others have identified an age-dependent reduction of sucrose preference and depressive-like behaviors in female mice following LBN (Goodwill et al. 2019).Using a maternal separation model of ELA, both male and female rats have reduced sucrose preference later in life (Matthews et al. 1996;Leventopoulos et al. 2009;Coccurello et al. 2014).Anhedonia has also been reported in nonhuman primates exposed to maternal deprivation and maltreatment (Rosenblum and Paully 1987;Paul et al. 2000;Pryce et al. 2004;Kaufman et al. 2007;Glynn and Baram 2019), including reduced sucrose preference (Paul et al. 2000) and lack of interest in social interaction (Coplan et al. 1996).However, others have found increased sucrose drinking in juvenile males following ELA (Nelson et al. 2009).
Together, the findings in rodents and nonhuman primates suggest the manifestations of reward circuit disruption by ELA may vary by the timing, duration, and nature of the ELA, and may be further modulated by sex.Whereas deficits in reward seeking behaviors are observed in males, such deficits are not as commonly found in females.Rather, in females, the prevailing phenotype includes the enhanced consumption palatable food and drugs of abuse.Furthermore, the variable consequences of ELA on distinct assays of reward-seeking behaviors in animal models (e.g., reduced "hedonic setpoint" but unchanged "demand elasticity") demonstrate that reward processing is not a singular entity; rather, individuals may express different and dissociable anhedonia phenotypes that suggest potentially discrete mechanisms of reward circuit disruption.Thus, further investigation into how ELA alters specific aspects of reward processing, and the underlying neural substrates will be critical for understanding the biological processes that contribute to the risk anhedonia, a key harbinger and component of several psychiatric disorders.

General Conclusions
Anhedonia is a transdiagnostic construct which is both a herald and a core component of several mental illnesses, yet the identification and characterization of the neural mechanisms that contribute to its emergence remain challenging.Strides have been made in understanding the origins of anhedonia through both human studies and the use of preclinical animal models.In rodents, researchers can induce anhedonia-like behaviors, enabling the identification and characterization of individual reward-circuit nodes and projections that influence these behaviors, and further characterize them at the cellular and molecular level.The study of the emergence of anhedonia-like behaviors following experimental ELA is an exciting avenue for two reasons.First, it recapitulates the human association, allowing delineation of trajectories and life-long progression.Second, it enables the use of powerful modern technologies including chemo-and optogenetics for the interrogation and functional identification of neural projections and molecules that may mediate anhedonia.Critically, with the use of animal models and asking the right questions, clinical queries can be translated to lab-based mechanistic studies, which in turn can lead to novel discoveries for the prevention of anhedonia or its mitigation.

Fig. 1
Fig. 1 Cross-species nodes and connectivity of the reward circuit

Table 1
Preclinical models of anhedonia