Abstract
As the opioid epidemic presents an ever-expanding public health threat, there is a growing need to identify effective new treatments for opioid use disorder (OUD). OUD is characterized by a behavioral misallocation in choice behavior between opioids and other rewards, as opioid use leads to negative consequences, such as job loss, family neglect, and potential overdose. Preclinical models of addiction that incorporate choice behavior, as opposed to self-administration of a single drug reward, are needed to understand the neural circuits governing opioid choice. These choice models recapitulate scenarios that humans suffering from OUD encounter in their daily lives. Indeed, patients with substance use disorders (SUDs) exhibit a propensity to choose drug under certain conditions. While most preclinical addiction models have focused on relapse as the outcome measure, our data suggest that choice is an independent metric of addiction severity, perhaps relating to loss of cognitive control over choice, as opposed to excessive motivational drive to seek drugs during relapse. In this review, we examine both preclinical and clinical literature on choice behavior for drugs, with a focus on opioids, and the neural circuits that mediate drug choice versus relapse. We argue that preclinical models of opioid choice are needed to identify promising new avenues for OUD therapy that are translationally relevant. Both forward and reverse translation will be necessary to identify novel treatment interventions.
I. Introduction
The United States has seen a growing epidemic of opioid use disorder (OUD) and associated overdose deaths. According to the 2020 National Survey on Drug Use and Health, an estimated 2.7 million people aged 12 or older had OUD and in 2021, the Center for Disease Controls’ National Center for Health Statistics reported 75,673 opioid-related overdose deaths. With the pervasiveness of opioid misuse and the increasing rate of overdose deaths, it is imperative to better understand the neural underpinnings of OUD and re-evaluate existing and potential new therapeutics. It has been suggested that substance use disorders (SUD) such as OUD, are a result of a behavioral misallocation between drugs and alternative, more advantageous rewards . Therefore, preclinical models of addiction are needed that capture this component of the human condition associated with SUDs. Choice models of addiction, in which animals are allowed to choose between a drug and a non-drug reward, are becoming more prevalent in the literature. These models are translationally relevant because they resemble scenarios that humans face when presented with commonly misused drugs in an environment where alternative rewards are constantly available. In this review, we examine the existing preclinical and clinical literature on choice behavior associated with SUDs, focusing on OUD, and the neural circuits that may underlie maladaptive choice for drug reward over natural reward. Treatments aimed at shifting choice behavior away from drug reward and towards natural rewards may provide promising new avenues for the development of OUD therapeutics.
II. Neural Systems Mediating Opioid Use Disorder
The Opioid System
Drug taking begins with the goal of eliciting a pleasant, rewarding effect, the same reason that we seek out non-drug rewards. However, as drug taking and seeking becomes compulsive, it reduces time spent engaged in other life goals and actions that are advantageous . Like other SUDs, OUD is characterized by a behavioral misallocation between opioids and non-drug rewards . Instead of directing behaviors towards actions that are biologically advantageous, energy is primarily expended on drug seeking and taking. The biasing of choice towards opioids and away from natural rewards might be driven, at least in part, by opioid withdrawal – an aversive physical and motivational state characterized to alleviate these symptoms by taking more drug. Together, the opposing processes of opioid-induced euphoria coupled with dysphoria from withdrawal and dependence lead to a profound negative emotional state, termed hyperkatifeia. Under these circumstances, negative reinforcement is occurring because the drug-seeking behavior is increased in order to avoid withdrawal symptoms. This however is separate from seeking out drug because of it rewarding effects and continuing to take drugs to obtain those rewarding effects. Together however, repeated opioid use produces a disruption in homeostasis, which increases the baseline for reward effects, such that more drug use is necessary to achieve the same level of euphoria . To understand the physiological effects of repeated drug use that lead to OUD, it is important to understand the opioid system, brain regions that mediate opioid reward, as well as the cellular mechanism of opioids.
The opioid system generally includes the endogenous receptors: mu-, delta- and kappa-opioid receptors (MOR, DOR, KOR) and nociceptin opioid peptide (NOP) receptors . These receptors play a key role in processing responses to pain and stress, and importantly reward. Though each of these receptors has been the focus of opioid receptor-based therapeutics for OUD, MORs (and sometimes KORs) are the primary target for the therapeutic and adverse effects of most commonly misused opiates. MORs are the primary contributor to the rewarding effects of opioids in part by promoting dopamine release in the mesolimbic pathway. The mesolimbic pathway is comprised of dopaminergic projections originating in the ventral tegmental area (VTA), which project to the nucleus accumbens (NAc), amygdala, and prefrontal cortex (PFC). Drug administration paired with presentation of stimuli, such as cues or environmental contexts, results in dopamine release in the VTA-→NAc pathway, altering synaptic strength and resulting in positive reinforcement. With repeated drug use and drug-cue learning, the drug cues become a trigger for return to use and can engender drug seeking. Dopamine projections from the substantia nigra (SN) to the dorsal striatum may be similarly engaged, particularly after repeated opioid use.
Importantly, dopamine neurons are under tonic inhibition from GABAergic neurons that express MORs, and MOR activation results in the disinhibition of dopamine neurons through reduction of GABAergic neuronal activity. This disinhibition was originally thought to derive from local VTA interneurons, but current evidence points to the rostral medial tegmental nucleus (RMTg) as a MOR-rich region that tonically inhibits VTA dopamine neurons. In the presence of opioids, GABAergic inhibition of VTA dopamine neurons is reduced, resulting in increased dopamine release in the NAc and increased opioid reward. This promotes the formation and strengthening of associative memories between drug-seeking behavioral patterns, environmental cues, and drug reward, thus perpetuating drug seeking and promoting cue-induced drug craving. After chronic heroin self-administration, MOR-activated G-proteins are desensitized in areas mediating analgesia (thalamus) and emotional regulation (amygdala) in the thalamus and amygdala, areas responsible for the effects of analgesia and emotional regulation, respectively, whereas the NAc displayed less desensitization. Based on commonly described symptoms of withdrawal, this pattern aligns well as there is a tolerance developed towards effects such as analgesia, but the reinforcing effects of opioids, mediated by the NAc, are not as broadly affected. Overall, MORs are an important site of consideration, especially since many of the current FDA-approved pharmacotherapies target these receptors.
The Dopamine Hypothesis of Addiction
Previous preclinical research suggests that the pervasive nature of drug seeking results from supranormal levels of dopamine in the NAc after drug rewards compared to non-drug rewards. Neural adaptations, such as neuroplasticity in the mesolimbic dopamine system following chronic drug exposure, may also promote the transition to compulsive drug use. One prevailing theory in the field is the prediction error hypothesis of dopamine signaling. This theory posits that dopamine release is greater when the value of the predicted reward is greater than expected and lower when the rewarding effect is lower than expected, thus acting as a prediction error signal. Opioids are thereby able to “hijack” dopaminergic signaling in reward circuits by increasing this error signal so that when the drug associated cue is presented the neural representation, or estimated value of the predicted reward, is greater than its actual value. Thus, the next time the cue is experienced, the inflated perceived value of opioid reward will drive opioid craving and behavioral patterns of drug seeking. Recent data suggest that the NAc dopamine signal only acts as a reward prediction error under specific learning conditions. It appears that dopamine may instead encode the perceived saliency of a stimulus, a theory that fits observations across a broad range of learning scenarios. The NAc dopamine signal associated with drug taking would thus increase the perceived salience of drug cues, which would be expected to enhance subsequent relapse and potentially other addiction-related outcomes.
There is some evidence, however, to suggest that the rewarding effects of opioids may occur independently of dopamine. For instance, morphine induced locomotion is reduced in dopamine-deficient mice, but these mice can still develop a morphine conditioned place preference. Chemical lesioning of dopamine terminals with 6-OHDA does not alter heroin self-administration. Self-administration of opioids is less subject to disruption by dopamine antagonists, and this is perhaps one of the most apparent distinctions between the neural mechanisms underlying opioid versus psychostimulant reward. And yet, chemogenetic inhibition and response-contingent optogenetic inhibition of VTA dopamine neurons was capable of reducing heroin self-administration. This suggests the dopamine system is at least capable of mediating opioid reward, but there may be opioid-independent systems for opioid reward and reinforcement as well.
Thus, the dopamine hypothesis of addiction as it pertains to opioids is rapidly evolving and may not apply as a blanket theory for all misused substances. For instance, self-administration of intravenous cocaine in rats was found to produce a supranormal dopamine signal, but despite this, cocaine was less preferred from sweet water during a choice assay, due to its delayed pharmacokinetics. It’s been shown that intravenous cocaine has a delayed effect of dopamine in the NAc on the order of tens of seconds, longer than the effects from non-drug reward. Thus, the speed of reward onset after drug use is an important variable when interpreting dopamine signals (See Section V - Factors that Influence Choice). In the context of the dopamine hypothesis of OUD, several brain regions have been implicated in the rewarding properties of opioid use. In addition to the dopamine signal in the NAc, glutamatergic inputs to the NAc from other limbic brain regions can impact NAc output and subsequent addiction-related behaviors. For example, clinically it has been shown that drug use severity and opioid withdrawal symptoms are positively associated with neural responses to drug cues measured by fMRI in the NAc, orbitofrontal cortex (OFC), and amygdala of patients with OUD. Below we expand upon the neural circuitry that has been implicated in OUD.
Neural Circuitry of Relapse and Choice
Preventing relapse and reducing rates of opioid choice over alternative rewards are two distinct points of behavioral intervention for OUD therapeutics. Most preclinical research on the neural circuitry of addiction has focused on relapse, which can be triggered by multiple stimuli, including drug-associated cues, priming doses of the drug itself, or stress. The PFC has been identified as a key component of the neural circuitry controlling relapse and can be subdivided into a dorsal and ventral regions. The dorsomedial PFC (dmPFC) encompasses the prelimbic cortex and neighboring areas, and the ventromedial PFC (vmPFC) is comprised primarily of the infralimbic cortex; both areas have been implicated in relapse circuitry. Glutamatergic projections from the dmPFC to the nucleus accumbens core have been dubbed a “final common pathway” to relapse, driving relapse triggered by multiple types of stimuli across multiple classes of misused drugs, including opioids. The role of the vmPFC, however, is more nuanced, as activity in the vmPFC inhibits cocaine relapse but can either promote or inhibit heroin seeking, possibly due to the existence of different functional neuronal ensembles within the vmPFC, and underscoring the fact that relapse circuitry may differ depending on the type of misused drug. Indeed, if specific subpopulations of vmPFC neurons are targeted, specifically those projecting to the nucleus accumbens shell (NAshell), this pathway appears to inhibit drug seeking for heroin, as well as numerous other drugs, including cocaine and alcohol.
In contrast to relapse circuitry, the neural circuits of choice between drug and non-drug rewards are not well understood. Whereas the OFC in rats has been shown to play a role in relapse after choice leading to voluntary abstinence, the involvement of OFC during the choice to abstain has not been examined. Therefore, while the OFC is implicated in relapse, it is unclear whether it plays any part in drug versus non-drug choice. In contrast, the vmPFC is highly interconnected with the OFC, and the vmPFC→NAshell pathway has indeed been shown to act as a limiter of both heroin relapse and heroin choice. In a recent study, Heinsbroek et al. used a preclinical model of heroin choice in which rats do not voluntarily abstain, rather, a subpopulation of rats that prefer heroin versus food emerges such that there is a population average of 50% choice for each reward (Figure 1). This parsing of subpopulations is unusual in the choice literature especially since no manipulations (See Factors That Influence Choice) were used in this model. This study also demonstrated that choice and relapse are distinct behavioral constructs, as, inactivation of the vmPFC produced opposing outcomes on these behaviors, increasing choice but reducing relapse. However, pathway-specific chemogenetic inhibition of the vmPFC→NAshell shifted choice toward heroin and increased heroin relapse, indicating that endogenous activity in the vmPFC→NAshell pathway limits opioid choice and relapse. Other, competing vmPFC outputs may thus be responsible for the effect of vmPFC inactivation on relapse. The latter may underlie observations that relapse and choice are distinct behavioral constructs that do not correlate. That is, animals that are more likely to be heroin choosers are not necessarily more likely to have higher relapse rates.
Current evidence strongly suggests a role for frontal cortex regions (vmPFC and OFC) in choice behavior between natural rewards and multiple types of drugs. These frontal cortex regions have been implicated in maladaptive choice and risky decision making, characteristics of SUDs that make them behavioral learning disorders. OFC damage or dysfunction in humans has been correlated with a higher likelihood of risky decision making. Similarly, inactivation of the vmPFC in rats promotes choice of a less optimal option and increases risky choice behavior. The vmPFC and OFC may similarly mediate maladaptive choice behavior in SUDs. For instance, repeated exposure to cocaine causes OFC neurons to fail to signal adverse outcomes (receival of a bolus of quinine) in a rat odor discrimination task at the time of choice and fail to exhibit plastic changes in cue responsivity after rule reversal. Additionally, in a study where abstinent persons with a SUD took part in a decision-making task, it was found that risky choices were negatively correlated with activation of the left medial OFC. These findings indicate that the OFC may undergo neuroadaptations with long-term drug use that promote pathological outcomes, suggesting that other circuitry implicated in maladaptive choice may similarly affect behavioral outcomes in SUDs. Overall, the neural circuitry of choice and its functional importance in the transition from recreational drug use to full-blown SUD is an active area of research, and current evidence points to a significant role of frontal cortex regions (vmPFC and OFC) in choice between natural rewards and multiple drugs, including opioids.
III. Cortical Circuits Mediating Choice
There is a large literature focusing on the role of cortical projections to subcortical regions such as the NAc and VTA and their functionality within addiction neural circuitry. Here we will focus on microcircuits within the cortex itself and the functional neuronal ensembles that may be related to different types of rewarded behavior and choice behavior. The human cortex is made up of six layers; however, in the rodent cortex there are only five. The cortex has two distinct types of cells that possess different roles in cortical circuitry: principle pyramidal neurons and GABAergic interneurons. Principle pyramidal neurons are glutamatergic cells that serve as the principal cortical output cell, whereas the GABAergic interneurons primarily inhibit pyramidal neuronal activity and importantly, express MORs. There are two main subclasses of interneurons, parvalbumin (PV) expressing interneurons (PV+) and PV deficient (PV−) interneurons. PV+ interneurons play a vital role in regulating pyramidal firing behavior and richly express MORs. Each layer contains these cell types but differs in their projections and roles. Layer I primarily consists of afferent terminals projecting from other brain regions and several populations of interneurons. Layers II and III contain the cell bodies of pyramidal neurons as well as a minority of GABAergic neurons that project to other cortical regions. Layer IV is a major cortical input location but is missing from the rodent PFC, with cortical inputs instead existing across the other layers. Layer V also contains principle pyramidal neurons and a small number of GABAergic neurons that target subcortical regions such as the NAc, and finally, layer VI is another output layer projecting primarily to thalamic regions. Of course, there are several other differences between cortices across species, but these have been reviewed elsewhere and are not within the scope of this review.
Though originally used to illustrate the mechanisms through which loss of inhibitory control in binge eating occurs, the Baldo hypothesis can be applied to other dysregulated motivated behaviors such as OUD. According to the Baldo hypothesis the vmPFC has been shown to possess two functionally opposed circuits. These circuits consist of two efferents that differentially guide rewarded behaviors with one being the “motivational driver” (projections to the hypothalamus) and the other being the “motivational limiter” (projections to the NAshell) (Figure 2). However, these outputs can be disrupted by excessive MOR signaling on local interneurons within the vmPFC, resulting in a disinhibition of glutamate projections to subcortical regions. This enhanced glutamate release has been found to result in unregulated appetitive motivational behaviors. Infusions of the MOR agonist DAMGO into the vmPFC was found to degrade inhibitory control leading to excess binge eating. Additionally, in a choice procedure between a palatable fat-enriched versus carbohydrate-enriched diet, intra-vmPFC DAMGO selectively increased carbohydrate intake even in fat-diet preferring rats. Together these results indicate the importance of MOR signaling in the vmPFC in the loss of control over food-seeking responses.
Though this theory was used to illustrate how disruption of these pathways may lead to dysregulated appetitive behaviors seen in binge-eating disorders, it can also be applied to other dysregulated motivational behaviors such as OUD, especially since the disruption of these outputs is driven by excessive MOR signaling during opioid use (Figure 2). Continuous exposure to exogenous opioids may degrade inhibitory control of the NAshell “limiter” pathway via neuroadaptations in GABAergic neurons that disinhibit GABAergic signaling in the vmPFC, thereby leading to heightened activity in the driver pathway. With both vmPFC outputs controlled by MOR-expressing interneurons, one output may be preferentially engaged in choice for opioids and the other output in relapse triggered by opioid-related cues. Additionally, continuous exposure to drug cues during drug taking may produce plastic changes in the driver pathway that render it the more dominant pathway controlling behavioral output. The opposite could occur in the limiter pathway, resulting in reduced inhibitory control over drug seeking. It can be extrapolated from this theory that repeated exposure to opioids and the opioid-related cues may lead to neuroadaptations in local inhibitory control over vmPFC pyramidal neuron output. The resulting disinhibition and dysregulation of vmPFC outputs may result in an enhancement or attenuation of the vmPFC driver and limiter pathways, respectively. Either of these adaptations, or both, would be expected to result in excessive drug seeking and taking that characterize OUD. Importantly, this theory restricts its view to vmPFC inputs to the NAshell, but other glutamatergic inputs to the NAc can also regulate drug seeking.
In line with the previous section, prolonged opioid exposure may induce plastic changes in other regions of the PFC resulting in similar hypo- and hyperactive states. More specifically, there may be subregional effects on the neurons in the dmPFC and vmPFC regions. Prolonged self-administration of remifentanil in mice produced a long-lasting hypoactive basal state via a decrease in ex vivo excitability accompanied by an increase in pyramidal neuron firing in the dmPFC but not the vmPFC. Thus, there are subregional plastic changes that result from repeated opioid exposure that correlate with deficits in cognitive flexibility. Repeated exposure to opioids may also induce plasticity within other cortical regions that lead to long-term changes within brain reward circuitry. For example, in a study assessing the effects of opioid exposure on distinct subregions of the OFC, in vitro patch-clamp electrophysiological recordings of mice brain slice containing the OFC showed that washes with the MOR agonist, DAMGO, induced a long-lasting suppression of inhibitory synaptic transmission onto layer II/III pyramidal neurons of the OFC via presynaptic MOR activation of local PV+ interneurons. This effect was specific to the medial OFC and this specificity is due to a reduction in the functional MOR coupling to downstream cAMP/PKA intracellular cascades in inhibitory synapses and an induction of long-term depression following a DAMGO wash. Altogether these results indicate that chronic exposure to MOR agonists can result in subregional specific long-term depression of inhibitory synapses, suggesting continuous taking of opioids causes synaptic changes that could promote a loss of inhibitory control over motivated behaviors. This may perhaps provide the neural foundation for the development of OUD.
Within the PFC and the OFC, subsets of functional neuronal ensembles have been identified that differentially encode drug and non-drug rewards. In one of these preclinical studies, neuronal activity within the OFC was recorded while rats were choosing between heroin and water sweetened with saccharin, or when each reward was available separately. It was found that choice for either reward was encoded by two non-overlapping OFC neuronal ensembles and that the relative size of the heroin-encoding population was correlated with the number of individual choices for heroin. Furthermore, the OFC neurons encoding the action of preferred choice fired more than neurons encoding the non-preferred action in the seconds before the choice for the preferred reward was made, suggesting that OFC neuronal activity in these ensembles may direct choice by creating pre-choice neuronal competition for action selection . These same researchers had also previously recorded OFC neuronal activity while rats instead chose between cocaine and sweetened water and found that these rewarded actions were also encoded by distinct OFC neuronal ensembles with similar pre-choice activity as those in the heroin study. Additionally, manipulation of the pre-choice firing to activate the cocaine population shifted non-drug preferring rats towards cocaine seeking. Interestingly, the number of neurons that non-selectively encoded both the drug and the non-drug reward was higher when the drug was heroin than when the drug choice was cocaine. Neuronal ensembles in the vmPFC also differentially encode alcohol versus saccharin reward, albeit with some overlap in the ensembles. Altogether, the implication of both vmPFC and OFC as well as similar findings between drug types suggest that these highly interconnected brain regions interact to encode reward choice, and that a neuronal ensemble is capable of simultaneously distinguishing between multiple rewards. Thus, these regions and specific ensembles may be important targets for shifting opioid choice, though more research is required to parse out drug differences.
As illustrated here, microcircuits within the cortex play an important role in the control of choice and dysregulated motivational behaviors. More specifically, these subcircuits can be distinguished, at least in part, by distinct anatomical outputs that subserve distinct functions. Additionally, prolonged exposure to opioids leads to long-lasting changes within the PFC, some of which may result in a loss of inhibitory control over motivational behaviors. Choice models in the preclinical addiction field are beneficial for several reasons. They can be used not only to parse out the circuitry of reward processes, but they simultaneously also allow researchers to determine a subject's reward preference during choice itself and the subsequent motivation to seek drug in the presence of drug-related cues (e.g. relapse). Additionally, they provide a means to identify translational therapies (e.g., contingency management therapy) that promote behaviors maintained by nondrug rewards, as opposed to drug reward. Here, we will examine the literature related to choice as it applies to OUD and discuss the translational importance of these studies in the context of other SUDs.
IV. Choice Model Considerations
Animal Models of Choice
Animal models of addiction are crucial for testing the efficacy of putative novel addiction therapies. Self-administration models in rodents and non-human primates are the gold standard in the field. In these models, animals are trained to press a lever or nose poke for an intravenous infusion (or oral reward, e.g., for alcohol) in daily behavioral sessions for several days, weeks, or months (or years for non-human primates). In the standard drug self-administration model, the animal only has access to a single reward at a time. Though this standard, single-drug model captures the chronic nature of drug exposure characteristic of addiction, it is not comparable to the human experience wherein multiple rewards might be available at the same time, and choices must be made, often in a fixed economy. In preclinical studies, this can be complemented by enriching the model with other reward options.
Choice models of addiction have excellent translational potential as they permit simultaneous access to a drug and a non-drug reward, and the animal must make a mutually exclusive choice between two options. Often the non-drug reward is food, but other alternative rewards, such as sucrose, saccharin, or social interaction, can be used. Commonly misused drugs are generally accepted to have supranormal reinforcing effects compared to natural rewards (See Section III - Dopamine Hypothesis of Addiction). This is supported by observations that animals will work harder to obtain a single reward of the drug under progressive ratio schedules of reinforcement, wherein the price (in number of lever presses) required to obtain a single reward increases at an exponential rate over time. Conflicting with this view, numerous studies indicate that rats will undergo “voluntary abstinence” when offered a non-drug reward. For example, several studies have concluded that rodents in an unmanipulated choice paradigm, will consistently choose a food reward over drugs. These experiments present a face validity problem, as in order to test potential treatments and parse out the cellular and molecular basis of SUDs, there must be a sufficient subpopulation of experimental animals with a phenotype characteristic of addiction, e.g. excessive pursuit of drug reward over non-drug rewards.
Most preclinical models yield drug preferring animals only through external influences that would shift natural preference towards choice (e.g., devaluation). If a model does not recapitulate the human condition, it becomes difficult to gauge whether existing or novel therapies can ameliorate behavioral outcomes. Heinsbroek et al. recently established a choice model that yields subpopulations of both food and heroin preferring rats, with a population average of 50% choice between rewards. This model recapitulates the human condition more closely, and thus may have predictive validity when it comes to identifying novel therapeutics for OUD. There are, however, ways of shifting choice toward drug reward in models where animals would otherwise voluntarily abstain. These studies have provided insight into the factors that influence choice behavior. Thus, to understand the advantages of the choice model, we must critically evaluate the current preclinical choice literature and the suitability of these models for both forward and reverse translation. Because choice models can be emulated with other similarly rewarding drugs, we include and address studies involving SUDs other than OUD in order to use them to better inform our understanding of OUD.
Factors That Influence Choice
Not all humans who take drugs become addicted to them. Hogarth proposed that SUDs are primarily driven by excessive goal-directed choice as a consequence of negative affective and/or somatic states rather than by habit or compulsion. Goal-directed choice is the act of making decisions based on the desire for an ideal outcome, and thus the choice to execute an action depends on the value allocated to the outcome and the cost of the action required to achieve it. Thus, when one is experiencing a negative affective or somatic state as a consequence of drug withdrawal, they may be more inclined to choose drugs over another option. Drug choice, because of excessive goal-directed choice under negative emotional state, could also be compounded by the assigned value or type of alternative reward. Additionally, there are a variety of non-drug options available broadly and/or concurrently with drugs. For example, increasing preclinical evidence points to social interaction as a potential non-pharmacological intervention to suppress the drive to seek drugs. Moreover, in humans, social support reduces the likelihood of developing psychiatric disorders, and the lack of social support may be a risk factor for development of a SUD.
Rodent studies typically do not deprive food/liquid before choice, and so there is no biological need pushing them towards a natural satiation. For this reason, the type of alternative reward offered matters based on the value it holds. Value of a reward can be experimentally manipulated, although it is difficult to produce a shift towards an increase in drug taking behavior. Frequently, researchers will attempt to devalue either the drug or non-drug reward to determine the extent a reward needs to be devalued before it is no longer the preferred choice. One method of devaluing a reward is through counterconditioning. Counterconditioning is the process of training a subject to respond to an already conditioned stimulus with a different response that is opposed to the original. This can be done through aversion therapies, where drug-paired cues are re-associated (counterconditioned) with aversive stimuli. For example, aversive counterconditioning conducted shortly after memory retrieval (during reconsolidation) successfully prevented relapse to cocaine seeking long-term.
Devaluation can also be done by satiating needs/wants prior a choice trial in that the magnitude of the reward is then diminished and there is a smaller drive to select. For example, in a study on squirrel monkeys, assessing choice between remifentanil and a milk food reward, devaluation by free access to milk prior to testing significantly increased choice for remifentanil relative to food whereas devaluation of remifentanil by pretreating with the opioid antagonist naltrexone, significantly reduced choice for drug. This was again seen in a study where giving rhesus monkeys food pellets before a cocaine-food choice session resulted in devaluation of food and an increase in cocaine choice. Conversely, in this same study, delivery of non-contingent cocaine failed to devalue cocaine but did result in a dampened response rate and decreased the overall number of cocaine infusions received during that session. Alternatively, in a baboon study, administration of chronic non-contingent morphine (8 days) leading up to a choice between heroin and food resulted in almost a complete abolishment of choice for heroin. This effect was reversed by decreasing the dose of non-contingent morphine and resulted in an increased choice for heroin. A factor for consideration, however, is this study design included a chronic administration of non-contingent heroin leading up to the choice test. This should be considered when assessing the viability of devaluation for shifting choice. Devaluation using satiation, however, has proven to be tricky. In a discrete trial choice between cocaine and water, previously water-restricted rats preferred water even after water satiation. However, following several sessions of water satiation, preference gradually shifted towards cocaine. Similarly, in another study by Vandaele et al., a saccharin reward was devalued using satiation and rats were still found to respond more for saccharin than for cocaine during extinction. It was postulated that the reason for this was because their behaviors were biased toward habitual (as opposed to goal-directed) control and therefore inflexible. These results may also explain why animals will undergo voluntary abstinence and suggest human populations may have more flexible and goal-directed behaviors. Alternatively, the relative reinforcer magnitudes used in studies that see voluntary abstinence could be larger for the alternate reward compared to the dosages of drug reward that is being offered, thus inducing voluntary abstinence. Therefore, it is important that studies assessing choice also measure relative reinforcer magnitudes for both rewards and preemptively adjust dosages to be approximately the same value before testing.
Researchers have also manipulated the drug and non-drug reward to determine how the reward magnitude and availability conditions (e.g., price and timing) may shift choice. Availability of a reward can be manipulated by altering the time-out period between individual choice trials or by prolonging the time of drug availability during the session. For instance, in rodents and rhesus monkeys given extended access to heroin or cocaine self-administration, subsequent choice was shifted towards the drug versus non-drug reward. Additionally, with extended access, the proportion of heroin-preferring rats was higher than those who preferred cocaine over a non-drug reward (51% vs 15%). It has been established that immediate rewards are generally more highly valued than a delayed reward. However, the temporal availability of drugs and the impact of drug pharmacokinetics also impact choice behavior. The delay before the effects of the drug is first felt and peaks dictates whether the relative immediacy of a non-drug reward can compete with the delayed drug reward. Though the delay may be nominal to a human, in rodent models the delayed effects may lead to discounting the drug reward. For instance, intravenous cocaine was found to be less preferred by rats than the non-drug alternative (e.g., sweet water) because of its delayed pharmacokinetics and sensation of reward, outweighing the effects of the drugs pharmacodynamics. Alternatively, having the drug already on board, with the subject feeling its effects while making a choice, could also alter decision-making. Townsend et al. recently established a rodent choice model where during choice training there are two separate phases: a sample component and a response component. During the sample component there was a non-contingent delivery of the drug followed by a time-out and then a delivery of food followed by a time-out. This was followed by 5 response component phases wherein both levers were available, and rewards were delivered based on a fixed ratio 5 response requirement and different drug doses were administered during each response components by changing the duration of the infusion. During the response component, researchers were able to produce stable drug choice, defined as ≥80% choice for drug, where the drug was cocaine or a fentanyl/methamphetamine mixture, even with the lowest drug dose, without any interventions. However, in order to get ≥80% drug choice for fentanyl, methamphetamine, heroin, amphetamine, interventions such as increasing the response requirement for food or changing the alternative reward to water were required. This indicates that non-contingent administration of certain drugs can influence the decision to choose drug but may be insufficient to drive drug choice for all misused substances.
Environment and ambiance in which the drug is offered and/or taken can also influence choice. While it is not possible to completely reproduce such environmental factors that influence human choice in a rodent model, enriched environments that provide non-drug rewards and/or opportunities for social interaction, promote voluntary abstinence from drugs, suggesting that environment influences choice behavior. Furthermore, the impact of environment may be drug dependent. In a study comparing the effect of environment on heroin versus cocaine taking, rats that were perpetually housed in the drug-taking context preferred heroin whereas those that were housed in a distinct context preferred cocaine. Though this study did not directly compare choice for heroin versus cocaine, it does imply that the effect of environment on drug-taking can be drug-dependent. Environment can also be narrowed down to the type of choice setting that a researcher chooses to provide. Researchers can provide a setting where rats can make choices without being under the pharmacological influence of the drug or can provide a setting where there is not a restriction on the ability to make consecutive choices and thus rats may still be under the influence of the drug by the next trial. These results suggest that prior drug use could impact future choice. In sum, numerous factors are capable of impacting choice behavior, but the degree to which these factors can readily model the human condition is arguable. Indeed, as SUD is a human condition, it is necessary to understand choice from a clinical perspective.
V. Lessons From the Clinic
Studying SUDs in human populations is difficult but necessary especially with the ever-increasing rates of OUD. There are several barriers including but not limited to finding voluntary participants with OUD but also maintaining that number throughout the study due to high dropout rates. Furthermore, it is difficult to account for stressors, comorbidities, polydrug use, and other independent variables. Much of the clinical research that has been conducted has focused on the prevention of relapse but studies assessing the viability of potential medication for reducing opioid choice over other rewards is lacking. Like the benefits of preclinical choice models, clinical studies that utilize a choice paradigm are more translationally valid to real-world scenarios since human drug use often occurs in environments that also have availability of alternative rewards. By focusing therapeutic efforts towards reducing drug choice, treatments may promote a reallocation of behavior towards alternative rewards and actions as opposed to drug seeking.
When it comes to clinical choice studies, money is often offered as the alternative reward. Indeed money, of sufficient value, is effective in reducing heroin taking in humans (Figure 1). However, one study found that money was ineffective in reducing cocaine seeking once cocaine had already been taken. In this study, the amount of money offered was between $1 to 16, whereas the aforementioned studies offered between $10 to $40, suggesting that the amount of money offered must be of sufficient value in order to shift choice. Furthermore, in a study that offered $1000 as a delayed monetary reward versus the equivalent amount of heroin that could be obtained for that amount, it was found that opioid-dependent participants discounted delayed heroin more than they discounted delayed money, indicating that larger amounts of money were valued more than larger amounts of heroin.
In both clinical and preclinical experiments, the type of alternative reward, cost to obtain the reward, and assigned value of the reward are all factors that may shift choice. Some factors, however, can only be studied in humans. For example, subjective feelings that result from drug use and societal factors such as socioeconomic status may influence drug choice. It is therefore necessary to assess the contribution of demographics as well as individual comorbidities to drug seeking/use and the influence they might have in affecting treatment or choice toward a non-drug alternative. One study assessing drug craving and choice found that in heroin-dependent individuals, a negative mood (induced by researchers) increases choice of heroin over food pictures and therefore may act as a trigger for heroin seeking. Overall, taking these factors into consideration will be beneficial in determining the proper course of treatment.
Clinical Pharmacotherapies
Several pharmacotherapies have been clinically screened and deemed successful at reducing opioid taking, for instance the opioid antagonist, naltrexone. Additionally, the selective dopamine D2/D3 receptor antagonist, amisulpride, successfully reduced drug-cue responsivity and reward impulsivity (defined by the inability to delay gratification and wait for a larger reward, in the face of a smaller immediate reward). These features were measured using a Pavlovian-instrumental transfer task and a delay discounting task respectively. However, most clinical research in this area has focused on opioid agonist/antagonist therapies, such as methadone and buprenorphine, which are current FDA-approved treatments for OUD. In one study assessing the effectiveness of different methadone doses on choice between an injection of heroin and varying amounts of money, larger amounts of money were required to reduce heroin taking in participants on a 50 mg methadone maintenance dose compared to groups on a 100 or 150 mg methadone maintenance dose. In another study assessing three doses of methadone maintenance, the higher doses of methadone completely suppressed withdrawal and all effects of heroin (subjective ratings of drug effect such as asking, “how high are you? / “do you feel any drug effect?”, observer-rated measures such as nodding and sedation, and physiological measure such as oxygen saturation). In contrast, the lower doses (30 and 60 mg) suppressed withdrawal but did not entirely block subjective and observer-rated measures of the effects of heroin. These results indicate that high doses of methadone paired with substantial offers of money may be most effective at reducing heroin taking. Similarly, maintenance on 16 mg buprenorphine reduced heroin taking relative to 8 mg dose of buprenorphine. Furthermore, individuals with OUDs maintained on 4 mg/day sublingual buprenorphine, when given the choice, money was preferred over all drug doses (Figure 1). Buprenorphine combined with naloxone (Suboxone) is another first-line treatment for OUD, and similarly to buprenorphine, higher dose ratios such as 8/2 and 32/8 mg ratio of buprenorphine/naloxone were found to be more effective treatments compared to lower dose ratios. In addition, a study comparing the effectiveness of money versus food as an alternative reinforcer found that cocaine was chosen to a greater extent than placebo across alternative reinforcer types and values, but the monetary alternative reinforcer suppressed drug choice to a greater degree than the food reinforcer. Overall, these results point to the importance of dose, dose ratio with compound pharmacotherapies, and dose ratio with monetary compensation, as variables influencing treatment efficacy.
The therapeutic potential of serotonin (5-HT) receptor targets (specifically the 2A and 2C receptors) for the treatment of OUD is also receiving growing attention. Serotonin (5-HT2) receptors are expressed in brain regions responsible for feeding, motivation, and reward. The 5-HT2 receptor agonist lorcaserin, a drug previously marketed for weight-loss, has been shown to reduce cued relapse and self-administration for alcohol, cocaine, nicotine, opioids, and methamphetamines in several preclinical studies with both rodents and non-human primates. Furthermore, studies using Ro60-0175, another 5-HT2C receptor agonist, have demonstrated efficacy in reducing cocaine and nicotine self-administration in rats. Despite the success in these single-reward (non-choice) preclinical models, lorcaserin has demonstrated moderate success in clinical trials for some substance use disorders, but in one study actually increased choice for cocaine while simultaneously reducing cocaine craving, and in another study had no effect on oxycodone self-administration, but trended towards increasing oxycodone wanting. Collectively, these results indicate that lorcaserin may have limited efficacy as a treatment option for SUDs in humans, or its efficacy may depend on the primary substance being misused. Notably, some of these studies relied only on self-reported craving as their behavioral metric as opposed to actual reductions in drug intake. This difference in metrics is worth considering when assessing viable candidates for treatment as reductions in craving do not always coincide with decreases in drug intake. Furthermore, lorcaserin was removed from the market in 2020 over concerns of increased risk for developing cancer. Analysis of clinical studies using lorcaserin did not confirm this increased risk but did suggest a trend in that direction. Thus, despite the call for additional research to reconcile the existent clinical and preclinical data with lorcaserin’s treatment efficacy for SUDs, it is important to proceed with caution until lorcaserin’s safety can be confirmed.
Though the literature regarding choice in the development of potential pharmacotherapies humans is limited and most clinical interventions aimed at reducing opioid choice have shown modest success at best, research in this area should be expanded to consider additional factors that may influence choice in humans. Furthermore, many of the clinical trials in which a pharmacotherapy successfully decreased drug self-administration should be tested using a choice protocol to evaluate it as a potential medication and to increase predictive validity. Other future directions should include offering other potent rewards (besides money) during clinical choice assays. Contingency management studies have found that employment offers or vouchers (exchangeable for goods and services; may act as a controlled form of money) can reinforce naltrexone maintenance therapy, leading to reductions in opioid use over time. Implementing contingency management protocols in combination with pharmacotherapy could assist in the reallocation of choice behavior towards healthy rewards (staying employed, maintaining treatment, etc.) and facilitate recovery.
VI. How to Treat Disorders of Choice
Potential Pharmacotherapies
Based on the current literature, treatments that are effective at reducing opioid choice include pharmacotherapies that can be administered before, during, and/or after opioid self-administration. Combination pharmacotherapies that mimic opioid reward (eg. MOR/KOR therapies), but with properties that have “punishing”/negative effects, may also be efficacious. Of course, if the "punishing"/negative effects are too strong, these pharmacotherapies could have the opposite of the intended effect and produce medication non-compliance. Such treatments have demonstrated some success in preclinical and clinical trials. Here, we will primarily discuss novel potential pharmacotherapies based on ongoing preclinical research.
FDA-approved pharmacotherapies for OUD include opioid agonists and antagonists (Table 1). Several studies have compared the effects of opioid drugs in their ability to shift choice. In one, researchers compared the effects of naloxone, buprenorphine, and methadone on choice in heroin dependent and non-dependent rhesus monkeys. Of the two agonists, methadone was found to prevent withdrawal related shifts towards heroin choice more effectively than buprenorphine (naloxone was not tested in dependent animals due to concerns about precipitated withdrawal). These results are interesting in that in non-dependent monkeys, buprenorphine decreased choice for heroin whereas methadone had no effect, inconsistent with data from clinical treatment studies . Overall, these results suggest that there are many other factors that play into shifting choice such as prior dependency on heroin as well as translational differences in the way these drugs have been seen to treat patients with OUDs. Though naloxone was not tested in this specific study and because naloxone is an antagonist, had it produced no effect, it would better support the idea that full agonism is needed to attenuate these withdrawal-dependent increases. Different opioid drugs with distinct mechanisms of action at specific opioid receptors may produce similar results. For example, dosing of rhesus monkeys with the lesser-known MOR antagonist methocinnamox (MCAM) decreases heroin self-administration and remifentanil choice, while shifting choice toward natural reward (i.e. food). Additionally, KOR agonists, specifically MOR/KOR mixed therapeutics, reduce the reinforcing effects, and subsequently the self-administration, of opioids. One study reported that when combined with self-administered oxycodone, the KOR agonists salvinorin A and nalfurafine, reduced taking of oxycodone in male rhesus monkeys. The researchers surmised that reductions in opioid self-administration were caused by the punishing effects of the KOR agonists (dysphoria, psychotomimesis, and thermal antinociception), further suggesting that combinations of KOR agonists with prescription opioids may reduce potential for misuse. Additionally, when the KOR agonists U50,488 and nalfurafine were added to intravenously self-administered fentanyl, fentanyl choice relative to food decreased, demonstrating that KOR agonists also punish fentanyl choice. Despite these successes, we must consider the translational capabilities of these treatments. In non-human animals, the treatments are experimenter administered and thus a drawback for application in a human population is that treatment with “punishers” is likely to result in medication non-compliance. This must be considered as any of these combination therapies must be “punishing” enough to reduce risk for misuse, but not so punishing that it results in the refusal to be treated.
In addition to opioid-based therapies, use of cannabinoids in conjunction with the administration of therapeutic opioids may be a potential treatment option as the cannabinoids are able to enhance the antinociceptive effects of opioids while reducing the dosage of opioids needed to produce analgesic effects, thereby diminishing the positive reinforcing effects (i.e., potential for misuse) of opioids. Clinically, administration of oxycodone and smoked delta-9-tetrahydrocannabinol (Δ9-THC) did not elicit analgesia but was able to increase pain threshold and tolerance during a Cold-Pressor test. However, the combination of the two drugs produced an increase in oxycodone abuse liability based on subjective reports from participants. In contrast to this, one study examining the reinforcing effects of an opioid/cannabinoid mixture in rhesus monkeys reported that the mixture of remifentanil with Δ9-THC failed to alter choice over sucrose pellets, but nonetheless indicated that cannabinoids do not enhance the reinforcing effects of MOR agonists. These differing reports on the effect of cannabinoids to increase abuse potential for opioid warrants consideration and should continue to be investigated. Further research should be conducted into potential alternative pharmacotherapies, including other cannabinoids, in conjunction to those covered here. Table 1 summarizes these current and emerging pharmacotherapies for OUD.
As previously established, preclinical evidence from single-drug models has supported the therapeutic efficacy of lorcaserin for reducing opioid relapse and self-administration in non-human primates. However, as with lorcaserin’s mixed effects in clinical studies, preclinical evidence from choice models using lorcaserin have also produced mixed results. In one study, treatment with 3.0 mg/kg of lorcaserin non-selectively decreased remifentanil and food self-administration in rats when both rewards were offered independent of each other. However, when forced to make a choice between immediate drug and delayed food (food delay was adjusted to manipulate responding for drug to be at 90%), only a dose of 1.7 and 3.0 mg/kg significantly reduced responding for the drug hole compared to vehicle. Additionally, a study in non-human primates found that maintenance on 1.0 mg/kg of lorcaserin selectively reduced heroin self-administration (dose-dependently), without impacting responding for food. Both studies, however, are contradicted by a non-human primate study that showed chronic dosing with up to 0.32 mg/kg of lorcaserin instead increased choice for heroin. These results underscore the mixed findings with lorcaserin and could be explained by the dose variation between studies, as well as variation in experimental design. The results also suggest that there could be other, unknown variables that may mediate (or diminish) lorcaserin’s efficacy. This possibility that there are other variables that mediate lorcaserin’s efficacy can be supported by one study that showed that a buprenorphine/lorcaserin mixture reduced variability and decreased heroin preference in non-human primates, when compared to treatment with buprenorphine alone. This study displays that it may not be feasible for lorcaserin to be a treatment on its own but instead may have potential as a treatment in combination with others like buprenorphine. Altogether, these studies with lorcaserin support our main tenet that the use of preclinical choice models (as opposed to single-drug models) will provide predictive validity for outcomes in clinical trials.
Novel and Potential Treatment Strategies
Alternative strategies to pharmacological approaches are currently under investigation due to the low success rate in relapse prevention long-term with existing therapies (Figure 3). This literature primarily addresses how these strategies reduce self-administration and relapse in preclinical models, and in clinical populations, but nonetheless should be considered for future applications in modifying choice behavior.
Non-invasive techniques that target cortical structures, such as transcranial magnetic stimulation (TMS), and invasive techniques that target subcortical structures, such as deep brain stimulation are forms of neuromodulation that may be useful clinical treatment strategies. TMS, an FDA-approved treatment for clinical depression, uses electromagnets to induce hyperpolarization or depolarization of neurons in specific cortical areas and may be applicable as a treatment for OUD and other SUDs. In several studies determining the effects of repetitive TMS (rTMS) therapy on the left dorsolateral prefrontal cortex (dlPFC), researchers observed a reduction in self-reported cue-induced heroin craving in heroin dependent subjects. Furthermore, in a case study observing a patient with OUD and cocaine use disorder, 7 separate sessions of rTMS targeting the left dlPFC reduced cue-induced cravings by ~60% to ~82%, compared to the ratings given prior to treatment. However, in a pilot study that applied rTMS to the dlPFC in patients with OUD undergoing methadone maintenance therapy, rTMS did not reduce craving, heroin use, or positive urine screens in opioid users compared to patients receiving a sham treatment.
Vagal nerve stimulation (VNS) is a moderately invasive method that is currently FDA-approved for treating epileptic seizures and treatment-resistant depression, and it is currently being investigated for treatment of SUDs. However, there are few reports investigating VNS in patients with OUD specifically, and none regarding its effect on choice. This is an unexplored area that should be considered for future research. Furthermore, as preclinical studies identify the specific neural circuits controlling opioid choice, another alternative treatment strategy may involve adeno-associated virus (AAV) mediated gene therapy. An example of this strategy is its use in chemogenetics where researchers are able to transfect designer receptors exclusively activated by designer drugs (DREADDs) into rodent or non-human primate brains to then directly manipulate choice circuits.
The more invasive deep brain stimulation (DBS) is a candidate treatment but is not widely used to treat SUDs. In a case study of a singular patient with long-term heroin dependence, DBS of the NAc prevented misuse of drug during active stimulation for the first 2.5 years and continued the prevention for 3.5 years even after the stimulation was removed with no relapse. Similarly, in a pilot trial of two patients with therapy-resistant OUD, bilateral application of DBS to the NAc induced complete abstinence from heroin and methadone in both patients for at least a year. More recently, another case study of a patient with long-term OUD reported that DBS of the NAc/ventral anterior internal capsule was able to reduce opioid use and craving throughout the 12-weeks of DBS and throughout the 12-month follow-up. These preliminary case studies suggest that DBS may be a promising treatment for OUD, but more research is needed in additional subjects to verify its safety and efficacy in patients with OUD.
Immunotherapies are another novel strategy that could be used to reduce drug choice. Immunotherapies use antibodies to sequester these drugs in the blood and prevent them from activating reward circuits in the brain. Several preclinical studies have shown that conjugate vaccines effectively reduce the reinforcing effects of opioids, thereby reducing opioid self-administration. Conjugate vaccines have also been shown to affect choice. For example, a fentanyl-tetanus toxoid conjugate vaccine reduced fentanyl self-administration in rats. These results were then translated from rats to monkeys, and vaccination reduced fentanyl choice and increased food choice in four-of-the-five monkeys. These results suggest that conjugate vaccines are promising strategies for clinical application. Despite these successes with conjugate vaccines, there are some caveats to this strategy. In several studies, animals could overcome the effect of the vaccine by taking more drug and, in essence, exceeding the binding capacity of the anti-drug antibodies that the vaccine produced. This effect of course can be seen with therapies that utilize competitive agonists and thus it is not a point against vaccine therapies but instead a factor for consideration.
In addition to the novel treatments discussed here, psychedelics are a drug class that is receiving growing attention as potential treatments for psychiatric disorders. For example, the psychedelic alkaloid ibogaine and some of its analogs have demonstrated efficacy in preclinical models of drug taking and seeking for opioids as well as multiple other substances, including cocaine, nicotine, and alcohol. Most psychedelics act as serotonin receptor agonists, with their hallucinogenic effects appearing to be mediated by the 5HT2A and 5HT2C receptors. Other than the 5HT2C receptor agonist lorcaserin, which produced mixed results, the 5HT2A agonist tabernanthalog (TBG) appears to be more promising. TBG reduced heroin self-administration and relapse, and evidence indicates reduced hallucinogenic and cardiotoxic side-effects compared to its parent compound, noribogaine. TBG was also found to reduce alcohol binge-drinking behavior, suggesting potential for treating polydrug use disorders and other individual SUDs.
In conclusion, there are many future directions to identify novel treatment strategies for OUD (Figure 3), and these strategies should be evaluated for their efficacy using preclinical models of opioid choice behavior, in addition to the standard self-administration and relapse models. By testing these novel therapies on opioid choice, we may better develop a clinically translational treatment that is more effective than current therapeutics at mitigating opioid use in favor of healthy reward alternatives.
VII. Conclusions
OUD is a pervasive public health threat, impacting millions worldwide. Much of the preclinical research into both the neural circuits and potential therapeutics is centered on targeting relapse and reducing self-administration. We have argued here that preclinical models of drug choice have greater face validity than single drug studies because they recapitulate a poly-reward economy resembling the human environment, and choice models are becoming more widely used to study OUD. Here we have reviewed the current preclinical and clinical literature on the role of choice in SUD, with a focus on OUD, and the neural circuits that may underlie maladaptive choice for drug reward over natural reward. Of course, there are many additional factors that may influence choice, many of which have not been fully parsed out but are beyond the scope of this review. The recent identification of the vmPFC→NAshell pathway as a limiter pathway for both heroin choice and relapse indicates at least some overlap in the circuitry for these behaviors, despite the fact they do not correlate with one another. More research is needed to further understand and characterize the neural mechanisms of choice, as well as the myriad of external factors that can influence choice. With relatively little data on a successful pharmacological method of shifting opioid choice, it is important to look at the success of novel potential therapeutics in shifting choice away from drugs and towards natural alternative rewards. We must continue to study choice behavior in the search for new addiction therapeutics and apply both forward and reverse translation to identify these novel treatment interventions.