bs_bs_banner

ORIGINAL ARTICLE

doi:10.1111/adb.12267

Extinction of alcohol seeking is enhanced by compound extinction and the noradrenaline reuptake inhibitor atomoxetine Hiu T. Leung & Laura H. Corbit School of Psychology, University of Sydney, Australia

ABSTRACT Alcohol-related stimuli can trigger relapse of alcohol-seeking behaviors even after extended periods of abstinence. Extinction of such stimuli provides a means for reducing their impact on relapse. However, the expression of extinction can be disrupted by exposure to the previous reinforcer as well as the simple passage of time. We investigated whether augmentation of prediction error or of noradrenaline neurotransmission by the reuptake inhibitor atomoxetine would enhance long-term extinction of alcohol-seeking behavior. Rats received Pavlovian conditioning of multiple stimuli signaling the delivery of an alcohol reward before individual extinction was given to each of these stimuli. Further extinction was then given to a target stimulus presented in compound with another alcoholpredictive stimulus intended to augment prediction error (Experiment 1) or after a systemic injection of atomoxetine (1.0 mg/kg; Experiment 2). Experiment 3 examined whether the compound stimulus effect relied on noradrenergic activity by testing the effects of the β-adrenergic antagonist propranolol, given prior to compound stimulus trials. Long-term retention of extinction learning was assessed a week later. Compound stimulus presentations enhanced long-term extinction as the stimulus extinguished in compound elicited less responding than a stimulus receiving equal extinction alone when tested a week later. This effect was mimicked by atomoxetine and blocked by propranolol given during extinction training. Thus, extinction of alcohol-seeking behavior can be improved by extinguishing multiple alcohol-predictive stimuli or enhancing noradrenaline neurotransmission during extinction training. Both behavioral and neurobiological processes could be exploited to enhance the outcome of extinction-based treatments for alcohol use disorders. Keywords

Ethanol, learning, Pavlovian, prediction error, propranolol, stimulus.

Correspondence to: Laura H. Corbit, School of Psychology, Brennan MacCallum Building (A18), University of Sydney, New South Wales 2006, Australia. E-mail: [email protected]

INTRODUCTION Environmental stimuli signaling the availability of drugs of abuse can gain control over an organism’s behavior through Pavlovian conditioning and later evoke behaviors leading to their procurement (Krank 1989; Le & Shaham 2002; Corbit & Janak 2007; Chaudhri, Sahuque & Janak 2008). The ability of these stimuli to promote drug-seeking behaviors is often long lasting and remains potent even after extended period of abstinence from the drug itself (Marlatt 1990; O’Brien et al. 1990; Sinha et al. 2009). The excitatory effects of drugassociated stimuli can be extinguished when the subject is repeatedly exposed to these stimuli in the absence of the © 2015 Society for the Study of Addiction

drug reinforcer, a process that is thought to involve the formation of a new inhibitory association. However, factors including reexposure to the original reinforcer (Rescorla & Heth 1975; Delamater 1997), stress (Erb, Shaham & Stewart 1996; Shalev, Erb & Shaham 2010), a change in context (Bouton et al. 2006) or the simple passage of time (Rescorla 2004) can each precipitate the return of extinguished behaviors limiting the efficacy of current cue-exposure therapies (Conklin & Tiffany 2002). While increasing the amount of extinction training can lead to a complete loss of the original behavior during this training, such treatment does not completely protect from the recovery phenomena described above (Katner, Magalong & Weiss 1999; Leung & Westbrook Addiction Biology

2

Hiu T. Leung & Laura H. Corbit

2010). Thus, it is of interest to develop ways to enhance the strength or longevity of the learning produced through extinction and recent developments in the animal learning field suggest two ways for approaching this. Contemporary learning theories hold that the inhibitory learning that occurs during extinction relies on a discrepancy between what is expected (availability of drug) and what actually occurs (nothing) which generates a so-called prediction error (Rescorla & Wagner 1972; Sutton & Barto 1981; Wagner 1981; Schultz, Dayan & Montague 1997). Thus, one approach for enhancing extinction has been to increase the prediction error generated during extinction. For example, presenting multiple stimuli that have previously signaled the same outcome together, or in compound, during extinction should lead to greater expectation of that outcome than a single stimulus and generate a larger negative prediction error when this compound is not reinforced. This in turn should produce more learning about those stimuli, in this case deeper extinction, than giving the same amount of extinction of a stimulus alone. Evidence for such an effect has been found using a range of conditioning preparations and test measures (Rescorla 1997, 2006; Janak & Corbit 2011; Leung, Reeks & Westbrook 2012). Long-term memory formation is also influenced by the biological significance or salience of events and the ability to engage neuromodulatory systems including dopamine and noradrenaline (NA). A role for NA in extinction learning has been demonstrated for extinction of conditioned fear (Cain, Blouin & Barad 2004; Morris & Bouton 2007; Chai et al. 2014) and operant responding for food and cocaine (Janak & Corbit 2011; Janak, Bowers & Corbit 2012). Other recent studies have demonstrated that NA is important for consolidation of extinction in both cocaine-conditioned place preference (Bernardi et al. 2009; Bernardi & Lattal 2010) and self-administration procedures (LaLumiere, Niehoff & Kalivas 2010) using post-training administrations of adrenergic drugs. Furthermore, the effects of compound stimulus extinction and manipulations of the noradrenergic system appear to be related. Suppression of noradrenergic neurotransmission by the noradrenergic beta-receptor antagonist propranolol blocked the impact of compound stimulus presentations both on performance during these presentations and on the long-term benefit of this treatment on extinction learning. Further, treatment with the NA reuptake inhibitor atomoxetine before extinction of an individual stimulus enhanced long-term extinction (Janak & Corbit 2011), mimicking the effect of compound extinction in improving the long-term expression of extinction. © 2015 Society for the Study of Addiction

It is unknown whether similar effects can be found for alcohol-seeking behaviors and so the aim of the following experiments was to test whether simultaneous presentation of multiple, non-reinforced alcohol-predictive stimuli would enhance extinction of an alcohol-seeking response and whether this effect relies on noradrenergic transmission. Animals underwent training in which conditional stimuli (CSs) signaled the delivery of alcohol and then the CSs were extinguished. Experiment 1 examined whether presentation of a stimulus compound during additional extinction of alcohol-predictive CSs would enhance long-term extinction. Experiment 2 investigated whether atomoxetine could mimic the long-term benefit of compound extinction. Finally, Experiment 3 evaluated whether the enhanced extinction produced by compound stimulus extinction relied on noradrenergic activity by testing the effects of the β-adrenergic antagonist propranolol.

MATERIALS AND METHODS Experiment 1: effects of compound stimulus presentation on extinction of alcohol-seeking behavior Subjects The subjects were 26 naïve adult male hooded Wistar rats obtained from Laboratory Animal Services at University of Adelaide (Adelaide, SA, Australia), weighing 350– 430 g at the start of the experiment. They were housed in groups of four to six in an air-conditioned colony room maintained on a 12-hour light-dark cycle (lights on at 0700 hours) with continuous access to food and water. Experiments were carried out in the light cycle. All procedures were approved by the Animal Ethics Committee at the University of Sydney and were in accordance with the Guide for the Care and Use of Laboratory Animals.

Apparatus Training and testing took place in eight Med Associates (East Fairfield, VT, USA) operant chambers housed in sound- and light-attenuating shells. Each chamber was equipped with a pump fitted with a syringe that could deliver a 10 percent ethanol (EtOH) solution into a recessed magazine. Photobeam emitters and detectors mounted in the magazines recorded entries. The chambers also contained a white noise generator, a solenoid that, when activated, delivered a 5-Hz clicker stimulus, and round stimulus lights positioned on the same wall and to either side of the magazine. Auditory stimuli were adjusted to 80 dB in the presence of a background noise of 60 dB provided by a ventilation fan. Microcomputers Addiction Biology

Extinction of alcohol seeking

equipped with MED-PC software (Med Associates) controlled all experimental events and recorded responses. Procedures Acclimation. To acclimate the rats to alcohol, a 24-hour access to a bottle of 10 percent EtOH was given in their home cages for 10 days. This was followed by 10 days of 1-hour access to a bottle of 10 percent EtOH in separate drinking cages at the time when training would subsequently occur. Water was always available in a separate bottle. Rats were weighed daily and their EtOH and water consumption were determined by weighing the bottles to the nearest 0.1 g before and after each access period. Acquisition. The design of Experiment 1 is summarized in Fig. 1a. Following alcohol acclimation, rats were placed into the operant chambers and given 3 days of magazine training. In each daily session, 10 deliveries of 0.1 ml of 10 percent EtOH were made to the magazine at random times across 30 minutes. Over the next 25 sessions, rats were given Pavlovian conditioning to three CSs: a white noise, a 5-Hz clicker and a steady light pre-

3

sented during an otherwise dark session. Each stimulus was presented for 30 seconds with variable intertrial intervals (defined as stimulus offset to stimulus onset) around a mean of 2 minutes and a range of 1–3 minutes. Magazine entries during each CS and the 30 seconds immediately preceding each CS presentation were recorded. The unconditional stimulus (US) was 0.1 ml of 10 percent EtOH solution delivered across the first 3 seconds of each CS presentation which the rats collected by entering the magazine. This was done to account for any delay between reward delivery and reward collection, which could be up to 20 seconds based on casual observations. Each session was 30 minutes in duration and each stimulus was presented four times. A pseudorandom order where each stimulus type could not occur on more than two consecutive trials was used throughout training and extinction. At the end of each session, the magazines were checked to determine whether the EtOH solution was consumed. Extinction phase 1. On the first day of extinction, rats received a session similar to acquisition except no alcohol was delivered. On the next day, rats were given four non-

Figure 1 Extinction of a stimulus compound reduces spontaneous recovery of an alcohol-seeking response relative to extinction of a single stimulus. (a) Summary of the design of Experiment 1. All rats underwent Pavlovian conditioning where three discrete stimuli (noise, clicker and light) each predicted the delivery of alcohol. Responding to all stimuli was extinguished before trials in which the stimuli were presented either alone or as part of a compound. Finally, spontaneous recovery of responding to each of the auditory stimuli was tested 1 week later. (b) Mean (±SEM) magazine entries per stimulus for the CS and pre-CS periods in the final Pavlovian conditioning session with 10 percent EtOH reward. (c) Mean (±SEM) magazine entries per stimulus in the initial extinction training phase. Responding between stimuli did not differ in this stage. (d) Mean magazine entries per stimulus during the final extinction session, in which two of the three stimuli were presented in compound and the remaining stimulus continued to be presented alone, and in the test conducted 1 week later. Presentation of a stimulus compound increased responding during extinction; however, when the element of that compound was tested alone 1 week later, animals responded less to the stimulus that was extinguished in compound than to the stimulus extinguished alone. *Indicates a significant difference in number of magazine entries; P < 0.05 © 2015 Society for the Study of Addiction

Addiction Biology

4

Hiu T. Leung & Laura H. Corbit

reinforced presentations of each of the auditory stimuli. This was done to compensate for the rapid loss of responding to the light when the US was withheld; however, responding to the noise and clicker did not differ.

Extinction phase 2. On the third day of extinction, on the basis of performance in the previous extinction sessions, the two auditory stimuli were assigned to either the compound or single condition in a counterbalanced fashion to ensure that responding to the two stimuli up to this point was equated and would not bias responding at test (see Fig. 1). The session began with one nonreinforced presentation of each of the three stimuli alone, before six further presentations of each AX and Y where one of the auditory stimuli (X) was combined with the light (A) to generate the compound (AX) and the other auditory stimulus served as Y (see Fig. 1a).

Test. Rats remained in the colony room for the next 7 days before they were tested for spontaneous recovery. During the test, four non-reinforced presentations of each of the auditory stimuli were given separated by an average 2-minute intertrial interval. The order of trials was counterbalanced across subjects.

Experiment 2: effects of the noradrenergic reuptake inhibitor atomoxetine on extinction of alcohol-seeking behavior Subjects and apparatus Twenty-six experimentally naïve rats served as subjects. The source, housing conditions and training apparatus were identical to those used in Experiment 1. Procedures The design of Experiment 2 is summarized in Fig. 2a. The procedures for alcohol acclimation, acquisition and for the first phase of extinction were identical to Experiment 1. The rats received a saline injection before the final acquisition session in order to habituate them to the injection procedure. Extinction phase 2 On the basis of response rates during training and the previous extinction sessions, rats were divided into two groups with approximately equal response rates (Fig. 2b). Rats were given an i.p. injection of either saline or atomoxetine (Tocris, Ellisville, MO, USA; 1.0 mg/kg) in a volume of 1 ml/kg approximately 45 minutes prior to the extinction session. This dose was chosen as it has previ-

Figure 2 Atomoxetine treatment enhances extinction of alcohol seeking. (a) Summary of the design of Experiment 2. (b) Mean (±SEM) magazine entries per stimulus for the CS and pre-CS periods in the final Pavlovian conditioning session with 10 percent EtOH reward. Responding was greater during the CS than the pre-CS period but there were no group differences in this stage. (c) Mean (±SEM) magazine entries per stimulus in the initial extinction training phase. Responding between groups did not differ in this stage. (d) Mean (+SEM) magazine entries per stimulus during the final extinction session following atomoxetine treatment (1.0 mg/kg) and during the test session conducted drug-free 1 week later. Rats treated with saline showed significant spontaneous recovery of responding from extinction to test.This effect was blocked by atomoxetine. *Indicates a significant difference in magazine entries; P < 0.05 © 2015 Society for the Study of Addiction

Addiction Biology

Extinction of alcohol seeking

ously been shown to effectively enhance extinction of food seeking (Janak & Corbit, 2011) and confirmed in pilot studies conducted with alcohol reward (data not shown). We predicted that atomoxetine would mimic the effects of the compound and enhance extinction and so rats received six non-reinforced presentations of a single auditory CS (clicker or noise balanced within groups and responding for the assigned stimulus in previous extinction sessions was equated across groups). Test Rats were tested 1 week later for spontaneous recovery to the extinguished auditory stimulus. The test consisted of six trials of the previously extinguished CS, each separated by an average 2-minute intertrial interval. Experiment 3: effects of the noradrenergic β-receptor antagonist propranolol on extinction of alcohol seeking Atomoxetine, like compound extinction, was able to augment extinction, suggesting that compound stimulus effect could be explained by increased NA release when these trials resulted in non-reinforcement. However, it is

5

possible that these two effects were produced through different mechanisms. Thus, this experiment sought direct evidence that the compound stimulus effect relies on noradrenergic activity by administering the β-receptor antagonist propranolol prior to the compound stimulus presentations. Further, as the volume of alcohol delivered in the first two experiments was relatively small to ensure that all alcohol was consumed and the correlation between stimulus presentations and alcohol consumption was high, we examined whether similar results would be produced when consumption levels were higher. Thus, the volume of alcohol delivered with each stimulus was doubled (0.2 ml). Subjects and apparatus Eighteen experimentally naïve rats served as subjects. Housing conditions and training apparatus were as described in Experiment 1. Procedure The design of Experiment 3 is summarized in Fig. 3a. The procedures for alcohol acclimation, acquisition and

Figure 3 Propranolol treatment blocks the benefit to extinction produced by extinction of a stimulus compound. (a) Summary of the design of Experiment 3. (b) Mean (±SEM) magazine entries per stimulus for the CS and pre-CS periods in the final Pavlovian conditioning session with 10 percent EtOH reward. Responding was greater during the CS than the pre-CS period but there were no differences between rats in the saline and propranolol groups. (c) Mean (±SEM) magazine entries per stimulus in the initial extinction training phase. Responding between groups did not differ in this stage. (d) Mean lever presses per stimulus during the final extinction session, in which two of the three stimuli were presented in compound and the third stimulus was presented alone, and during the test session, in which stimuli X and Y were presented alone, conducted drug-free 1 week later. In saline-treated animals, compound stimulus presentation produced robust responding in extinction and reduced responding when the element was tested 1 week later relative to the single stimulus. In contrast, the group treated with 5.0 mg/kg of propranolol showed reduced responding to the compound in extinction and greater responding when the element of the compound subsequently tested drug-free; *P < 0.05. There was no effect of propranolol treatment on responding to the single stimulus © 2015 Society for the Study of Addiction

Addiction Biology

6

Hiu T. Leung & Laura H. Corbit

initial extinction are described in Experiment 1 except that 0.2 ml of alcohol was delivered during each stimulus. Magazines were examined at the end of each session to ensure that all alcohol was consumed.

Extinction phase 2 Rats were divided into two groups based on response rates on the final day of acquisition and the first 2 days of extinction. On the third day of extinction when the compound stimulus presentations would occur, rats were injected with either 5 mg/kg of propranolol (propranolol hydrochloride; Sigma, Castle Hill, NSW, Australia; dissolved in sterile water) in a volume of 1 ml/kg or an equivalent volume of saline 30 minutes before the extinction session.This dose was chosen as it has previously been shown to significantly reduce the augmented extinction learning following compound stimulus extinction with both food and cocaine reward (Janak & Corbit 2011; Janak et al. 2012). The session contained six presentations of a stimulus compound [click or noise (X) paired with light (A); balanced within groups and matched for previous response rates]. These trials were intermixed with an equivalent number of presentations of the other auditory stimulus (Y) presented alone (Fig. 3a).

Test Rats were tested 1 week later for spontaneous recovery. The test consisted of four non-reinforced presentations of each of the auditory stimuli separated by an average 2-minute intertrial interval.

Data analyses Magazine entries during the 30-second CS and the 30 seconds immediately preceding the CS were recorded. Data are presented as means (±SEM). Data were analyzed with repeated measures or mixed ANOVA and one-way ANOVA to further examine significant main effects or interactions.

RESULTS Experiment 1: compound stimulus presentation enhances extinction of alcohol-seeking behavior Alcohol acclimation During the first 10 days of alcohol acclimation, the animals’ average intake of 10 percent EtOH over 24 hours was 8.4 ml. Over the next 10 days, their average 10 percent EtOH intake over the 1-hour period was 2.4 ml producing an average alcohol level of 0.51 g/kg © 2015 Society for the Study of Addiction

demonstrating that rats would voluntarily consume unsweetened alcohol.

Pavlovian conditioning Magazine entry responses steadily increased during the CSs over the pre-CS periods across the 25 conditioning sessions. Mean (+/−SEM) magazine entries per trial on the last day of acquisition for the pre-CS, light, noise and clicker were 0.2 (0.13), 3.8 (0.19), 3.6 (0.21) and 3.8 (0.22), respectively. Responding was greater during the CS than the pre-CS periods, F(1, 24) = 139.4, P < .01, but there was no difference in responding to the three stimuli nor between the stimulus that would subsequently be extinguished alone or as part of a compound, Fs(1, 24) < 1; Fig. 1b. Extinction Mean (+/−SEM) magazine entries in the last trial of the first extinction session for the pre-CS, light, noise and clicker were 0 (0), 1.1 (0.3), 2.0 (0.6) and 1.9 (0.5), respectively. While responding to the light was lower, responding to the two auditory stimuli did not differ, F(1, 24) < 1. Figure 1c shows the animals’ responding to the individual stimuli (X and Y) during the initial phase of extinction training (trial data are shown in Supporting Information Fig. S1). There was no significant difference in responding to the two CSs, F(1, 24) < 1. Figure 1d shows the animals’ responding to the compound stimulus (AX) and to the single stimulus (Y) during the final extinction session and to the stimulus extinguished as part of a compound (X) or only alone (Y) at test. Extinguished responding was augmented by compound stimulus presentation relative to the single stimulus for which responding remained low. This pattern reversed at test with rats showing greater spontaneous recovery of responding to the stimulus that had been extinguished alone. Repeated measures ANOVA indicated no effect of time [extinction versus test; F(1, 24) = 3.4, P > .050] or stimulus [X versus Y; F(1, 24) = 0.9, P > .05] but an interaction between these factors, F(1, 24) = 13.9, P < .01. Because rats received a stimulus compound in extinction and single stimuli at test, it is difficult to directly compare responding across these conditions. Thus, to explore the observed interaction, we examined the effect of stimulus in extinction and at test. As expected, during extinction, responding was significantly greater to the AX compound than to the single stimulusY, F(1, 24) = 7.6, P < .01. The opposite was observed at test where responding was less to the stimulus previously extinguished in compound (X) than one extinguished alone throughout (Y), F(1, 24) = 5.7, P < .05. This shows that the extinction of alcohol seeking is enhanced by compound stimulus presentations. Addiction Biology

Extinction of alcohol seeking

7

Experiment 2: systemic atomoxetine enhances extinction of alcohol-seeking behavior

Experiment 3: propranolol blocks the benefit to extinction produced by compound stimulus extinction

Alcohol acclimation

Here we tested the hypothesis that the benefit to extinction produced by compound stimulus presentation relies on noradrenergic activation by testing whether this effect could be blocked by the β-adrenergic receptor antagonist propranolol (Fig. 3a). We predicted that rats receiving saline would show elevated responding to the stimulus compound when it was presented in extinction and would subsequently show enhanced extinction and thus low spontaneous recovery when presented with an element of that compound a week later. In contrast, if β-adrenergic receptor activation contributes to the compound stimulus effect, we predicted that rats pre-treated with propranolol before the last extinction day would not benefit from enhanced extinction and would show greater spontaneous recovery at test despite compound extinction.

During the first 10 days of alcohol acclimation, the animals’ average intake of 10 percent EtOH over 24 hours was 7.0 ml. Over the next 10 days, their average EtOH intake over the 1-hour period was 2.2 ml producing an alcohol level of approximately 0.52 g/kg.

Pavlovian conditioning Two rats were excluded at this stage for failing to consistently consume all of the EtOH solution delivered during conditioning. For the remaining rats, magazine entries to the CSs increased over the course of training, and performance on the final training day is shown in Fig. 2b. In the final session, responding was significantly greater during the CS than in the pre-CS interval, F(1, 22) = 24.6, P < .01. There was no effect of group and no group by interval interaction, Fs < 1. Extinction Figure 2c shows the animals’ responding to the target CS during the first phase of extinction when the CS was presented alone to the animals drug-free (trial data are shown in Supporting Information Fig. S2). There were no apparent differences between the groups at this stage, F(1, 22) < 1. Figure 2d shows the animals’ responding to the CS during the final extinction session where additional extinction was given after an injection of either saline or atomoxetine (1.0 mg/kg) and in the test of spontaneous recovery conducted a week later. Analyses of the effects of atomoxetine during extinction and at test revealed no effect of group, F(1, 22) = 2.1, P > .05, but an overall effect of time, indicating that responding increased from extinction to test, F(1, 22) = 10.7, P < .01, and a significant interaction between these factors, F(1, 22) = 4.5, P < .05. We predicted that atomoxetine should enhance noradrenergic activity and improve extinction learning and thus reduce responding in a subsequent test of spontaneous recovery. To provide such evidence, we tested for spontaneous recovery in each group by comparing responding to a single cue in the final extinction session and a week later which provides an opportunity for spontaneous recovery. As expected, rats treated with saline showed an increase in responding, and thus robust evidence of spontaneous recovery, between extinction and test, F(1, 10) = 7.0, P < .05. No such effect was observed in rats treated with atomoxetine, F(1, 12) = 2.1, P > .05, indicating that atomoxetine prevented subsequent spontaneous recovery. © 2015 Society for the Study of Addiction

Alcohol acclimation During the first 10 days of alcohol acclimation, the average intake of 10 percent EtOH per rat over 24 hours was 8.2 ml. Over the next 10 days, their average EtOH intake in the 1-hour access period was 3.6 ml, resulting in an average 0.84 (+/−0.02) g/kg in the 1-hour period.

Pavlovian conditioning Magazine entries increased during the CS periods across conditioning sessions as in the previous experiments. On the final day of training, mean (+/−SEM) magazine entries for the pre-CS, light, noise and clicker were 0.5 (0.6), 4.0 (0.32), 5.2 (0.35) and 4.8 (0.36), respectively, and responding was significantly greater during the CS than the pre-CS periods (Fig. 3b; F(1, 16) = 135.4, P < .01). There were no differences in responding to the stimuli based on identity (light, noise, clicker) or those assigned to compound and single conditions, no differences between groups, and no interactions involving these factors (all Fs < 1).

Extinction and test Mean magazine entries (+/−SEM) for the pre-CS, light, noise and clicker in the last trial of the first extinction session were 0 (0), 0.3 (.2), 2.2 (.64) and 1.9 (.6), respectively. Figure 3c shows the animals’ responding to the to-be compound and single stimuli (X and Y; noise and clicker, counterbalanced) during the first phase of extinction training (trial data are shown in Supporting Information Fig. S3). Importantly, there was no significant difference in responding to the two auditory CSs Addiction Biology

8

Hiu T. Leung & Laura H. Corbit

during the initial extinction, Fs(1, 16) < 1. Figure 3d shows the animals’ responding to the compound stimulus (AX) and to the single stimulus (Y) during the final extinction session. As indicated in the figure, and consistent with the findings of Experiment 1, for rats treated with saline, rats responded more during presentations of the compound stimulus than to additional presentations of a single stimulus and this pattern reversed at test; rats responded more, thus showing greater recovery and weaker extinction of responding to the single stimulus compared to one that was extinguished as part of a compound. Importantly, and as predicted, this pattern was different in animals that received propranolol prior to the compound stimulus trials. Propranolol treatment reduced the responding to the compound stimulus during extinction and these animals subsequently showed greater spontaneous recovery at test. ANOVA examining the performance of the two groups to the two stimuli in extinction and at test revealed no main effect of stimulus (compound versus single; F(1, 16) = 2.2, P > .05), no interaction between stimulus and group, F(1, 16) = 2.7, P > .05, and no overall effect of time (extinction versus test; F(1, 16) = 1.9, P > .05) or group, F(1, 16) = 1.2, P > .05. There was however an interaction between time and group, F(1, 16) = 7.4, P < .05, between stimulus and time, F(1, 16) = 36.4, P < .01, and a three-way interaction between stimulus, time and group, F(1, 16) = 12.8, P < .01. Because rats received a compound stimulus in extinction and only single stimuli at test, to explore the observed interactions further, we examined the effects of stimulus and drug treatment in extinction and at test independently. Analysis of the final extinction session revealed a significant effect of stimulus, F(1, 16) = 27.0, P < .01, with rats responding more to the compound than to the single stimulus. There was also an effect of group, F(1, 16) = 9.7, P < .01, and an interaction between these factors, F(1, 16) = 7.5, P < .01. While the two groups showed similar responding to the single stimulus, F(1, 16) = 0.5, P > .05, rats receiving propranolol responded less to the stimulus compound, F(1, 16) = 15.1, P < .01, indicating that propranolol treatment specifically affected responding to the compound. Analyses of the test session revealed an effect of stimulus, F(1, 16) = 9.8, P < .01, no effect of group, F(1, 16) = 1.8, P > .05, but an interaction between these factors, F(1, 16) = 4.9, P < .05. Rats that had received saline responded less to the stimulus that had been extinguished in compound than those that received propranolol, indicating that propranolol treatment reduced the benefit of this treatment on extinction learning, thus allowing greater spontaneous recovery, F(1, 16) = 6.5, P < .05. There was no effect of drug treatment on responding to the single stimulus, F(1, 16) < 1. © 2015 Society for the Study of Addiction

DISCUSSION The present experiments demonstrate that extinction of a compound of two extinguished CSs enhances extinction of alcohol-predictive stimuli diminishing alcohol-seeking behavior. This effect relies on noradrenergic signaling as treatment with the noradrenergic β-receptor antagonist, propranolol, blocked the effect of the compound stimulus. Further, the selective NA reuptake inhibitor atomoxetine, given prior to extinction training, mimicked the effects of compound stimulus presentation, also improving extinction. Importantly, both of these effects survived the passage of time which otherwise allows for disruption of the expression of extinction and the return of the original drug-seeking response; alcohol seeking remained low when rats were tested a week later. The effects of compound extinction are consistent with the view that extinction is regulated by the discrepancy or prediction error between what is expected based on conditioning history and what actually occurs. When the US is stronger than what is predicted by the CSs (e.g. during initial acquisition), it generates a positive prediction error which leads to the formation of an excitatory CS–US association and promotes responding. In contrast, when a previously conditioned CS is followed by the absence of an expected US, a negative prediction error is generated, an inhibitory association is formed, and responding decreases. When two previously independent predictors of alcohol were presented in compound during additional extinction, their remaining excitation summated to produce an increase in alcoholseeking behavior. However, when the compound was not reinforced, the enhanced expectation of reward followed by non-reinforcement would increase the negative prediction error providing an opportunity for further extinction learning to occur. Evidence of deepened extinction was found in the form of reduced spontaneous recovery a week later. While we favor the prediction error account, an alternative explanation for the improved extinction of the stimulus given compound extinction is that increased responding and/or attention to the compound stimulus provided more opportunity to learn the new relationship between the stimulus and lack of alcohol reward. While this possibility is difficult to rule entirely as levels of responding and of associative strength are often linked, Rescorla (2006; Experiment 5) demonstrated that presentation of a stimulus trained as a facilitator, that itself had no net excitatory strength but was nonetheless able to augment responding to another trained excitor when presented together, failed to promote extinction. In contrast, presentations of a diffuse excitor, a stimulus that produced a conditional response distinct from that produced by the target stimulus that thus failed to increase Addiction Biology

Extinction of alcohol seeking

responding when the stimuli were presented together, did facilitate extinction. These results suggest that the deepened extinction in these experiments relates to the reintroduction of excitation or error rather than elevated responding alone. Notably, in Experiment 2, treatment with atomoxetine was also found to enhance extinction without affecting performance on the drug treatment day providing further evidence that increased responding is not required to improve extinction. It is also possible that the novelty of the stimulus compound increased attention to the stimuli improving learning without necessarily relying on a prediction error account. Rescorla compared the effects of pairing an extinguished stimulus with an excitatory stimulus or a neutral stimulus and found that only the excitatory stimulus resulted in deepened extinction at test. Thus, the novelty of the compound trials alone, without invoking some expectation about reinforcement, does not account for the improved extinction (Rescorla 2000). Current neurobiological models of reinforcement learning hold that the prediction error signal that supports reward-related learning is encoded by midbrain dopaminergic neurotransmission, originating from the ventral tegmental area and substantia nigra and projecting to the nucleus accumbens and various forebrain targets (Waelti, Dickinson & Schultz 2001; Steinberg et al. 2013; Hart et al. 2014). However, a role for NA in reward-related prediction and learning should not be discounted (Smith & Aston-Jones 2008). For example, measurements of subsecond catecholamine release in the extended amygdala by fast-scan cyclic voltammetry showed that while reward-predictive cues evoke dopaminergic responses (Park et al. 2012, 2013), reward omission in extinction evokes noradrenergic responses occurring specifically at the time when reward is expected but not delivered (Park et al. 2013), suggesting that the noradrenergic system could mediate negative prediction error. This is consistent with electrophysiological recording studies that have demonstrated that noradrenergic neurons fire in response to novel events and changes in stimulus–outcome contingencies (Sara, Vankov & Herve 1994). Our result that noradrenergic activation mimics, and antagonism blocks the impact of increased negative prediction error on long-term extinction learning supports a complementary role for the noradrenergic system in the neural coding of prediction error. Another potential explanation for these results is that the noradrenergic system could mediate the negative aspects of appetitive conditioning, such as when reward terminates unexpectedly in extinction. This is consistent with demonstrations in a variety of aversive conditioning paradigms that the noradrenergic system regulates both the acquisition and the extinction of conditioned emotional responses (Cain et al. 2004; Morris & Bouton © 2015 Society for the Study of Addiction

9

2007; Bernardi & Lattal 2010), potentially through its role in facilitating attention allocation to salient environmental stimuli and/or memory consolidation (Mueller & Cahill 2010). It is possible that reward termination in appetitive extinction also constitutes a frustrating or motivationally aversive event (Amsel 1958), which similarly recruits noradrenergic release. Notably, propranolol was without effect on a CS just given continued individual extinction, indicating a role of noradrenergic activation specifically during compound but not (continued) single stimulus extinction. Exposure to alcohol-associated stimuli is thought to play a critical role in triggering drug desire and relapse (Loeber et al. 2006; Marlatt 1990; O’Brien et al. 1990). Behavioral extinction is one approach for reducing the influence of alcohol-related cues. However, a major limitation of many extinction-based therapies is that the expression of extinction is easily disrupted by reexposure to the reward or associated stimuli, stress, or simply the passage of time (Bouton et al. 2006), and current treatments are not designed to protect against these effects which may explain generally disappointing efficacy (Conklin & Tiffany 2002). While most studies to date have focused on manipulations that affect reinstatement and related phenomena at the time of testing (Le & Shaham 2002), such effects may not survive the passage of time or changes in context and thus may not generalize outside the clinic setting. By instead focusing on treatments that augment extinction learning, the current study demonstrates behavioral and pharmacological methods of improving extinction that persist across time. Further studies that examine whether these treatments similarly make the expression of extinction resistant to other recovery phenomena such as renewal will also be of importance. Further, recent work suggests that extinction of hierarchical (e.g. where a discriminative stimulus signals that a specific instrumental response will be reinforced) rather than simple Pavlovian associations was more effective in preventing future drug seeking (Hogarth et al. 2014). Of note, both compound extinction and atomoxetine enhance extinction of discriminative stimuli in addition to Pavlovian stimuli (Rescorla 2006; Janak et al. 2012). Several factors must be considered in translating the current findings to humans. Human drug use history involves a wide range of cues far more complex than those used with rats. Identification of high-risk situations and individualized stimuli such as guided imagery related to previous alcohol use could be used to ensure that the cues being extinguished are relevant to an individual’s alcohol use history and has been shown to reduce craving and limbic neural activation in related studies (Sinha et al. 2009; Vollstädt-Klein et al. 2011). Our findings also suggest that administration of a drug-like atomoxetine, Addiction Biology

10

Hiu T. Leung & Laura H. Corbit

already proven safe for use in humans, may be a useful adjunct to extinction-based or other cognitive behavioral therapies to enhance the long-term efficacy of these interventions (Ganasen, Ipser & Stein 2010; Mueller & Cahill 2010). Further, treatment with widely prescribed betaantagonist drugs may prove counterproductive if given during extinction-based therapies. Finally, consideration of other factors known to disrupt the expression of extinction (e.g. changes in context, the passage of time, spacing of extinction, extinction of drug-taking behaviors and discriminative stimuli; Conklin & Tiffany 2002) will be essential for improving clinical efficacy. In summary, the present experiments demonstrate that extinction of alcohol seeking can be enhanced by compound extinction and increasing noradrenergic activity during extinction training. Both of these methods have the potential to enhance the outcome of extinction-based treatments of alcohol use disorders. Acknowledgements This research was supported by a grant from the Australian National Health and Medical Research Council (APP1051037 to L.H.C.). Authors Contribution HTL and LHC performed the experiments and prepared the manuscript. All authors have critically reviewed content and approved final version submitted for publication. References Amsel A (1958) The role of frustrative nonreward in noncontinuous reward situations. Psychol Bull 55:102–119. Bernardi RE, Lattal KM (2010) A role for alpha-adrenergic receptors in extinction of conditioned fear and cocaine conditioned place preference. Behav Neurosci 124:204– 210. Bernardi RE, Ryabinin AE, Berger SP, Lattal KM (2009) Postretrieval disruption of a cocaine conditioned place preference by systemic and intrabasolateral amygdala beta2and alpha1- adrenergic antagonists. Learn Mem 16:777– 789. Bouton ME, Westbrook RF, Corcoran KA, Maren S (2006) Contextual and temporal modulation of extinction: behavioral and biological mechanisms. Biol Psychiatry 60:352– 360. Cain CK, Blouin AM, Barad M (2004) Adrenergic transmission facilitates extinction of conditional fear in mice. Learn Mem 11:179–187. Chai N, Liu JF, Xue YX, Yan C, Yan W, Wang HM, Luo YX, Shi HS, Wang JS, Bao YP, Meng SQ, Ding ZB, Wang XY, Lu L. (2014) Delayed noradrenergic activation in the dorsal hippocampus promotes the long-term persistence of extinguished fear. Neuropsychopharmacology 39:1933–1945. Chaudhri N, Sahuque LL, Janak PH (2008) Context-induced relapse of conditioned behavioral responding to ethanol cues in rats. Biol Psychiatry 64:203–210. © 2015 Society for the Study of Addiction

Conklin CA, Tiffany ST (2002) Applying extinction research and theory to cue-exposure addiction treatments. Addiction 97:155–167. Corbit LH, Janak PH (2007) Ethanol-associated cues produce general Pavlovian-instrumental transfer. Alcohol Clin Exp Res 31:766–774. Delamater AR (1997) Selective reinstatement of stimulusoutcome associations. Anim Learn Behav 25:400–412. Erb S, Shaham Y, Stewart J (1996) Stress reinstates cocaineseeking behaviour after prolonged extinction and a drug-free period. Psychopharmacology (Berl) 128:408–412. Ganasen KA, Ipser JC, Stein DJ (2010) Augmentation of cognitive behavioral therapy with pharmacotherapy. Psychiatr Clin North Am 33:687–699. Hart AS, Rutledge RB, Glimcher PW, Phillips PE (2014) Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. J Neurosci 34:698– 704. Hogarth L, Retzler C, Munafò MR, Tran DM, Troisi JR 2nd, Rose AK, Jones A, Field M (2014) Extinction of cue-evoked drugseeking relies on degrading hierarchical instrumental expectancies. Behav Res Ther 59:61–70. Janak PH, Corbit LH (2011) Deepened extinction following compound stimulus presentation: noradrenergic modulation. Learn Mem 18:1–10. Janak PH, Bowers MS, Corbit LH (2012) Compound stimulus presentation and the norepinephrine reuptake inhibitor atomoxetine enhance long-term extinction of cocaine-seeking behavior. Neuropsychopharmacology 37:975–985. Katner SN, Magalong JG, Weiss F (1999) Reinstatement of alcohol-seeking behavior by drug-associated discriminative stimuli after prolonged extinction in the rat. Neuropsychopharmacology 20:471–479. Krank MD (1989) Environmental signals for ethanol enhance free-choice ethanol-consumption. Behav Neurosci 103:365– 372. LaLumiere RT, Niehoff KE, Kalivas PW (2010) The infralimbic cortex regulates the consolidation of extinction after cocaine self-administration. Learn Mem 17:168–175. Le A, Shaham Y (2002) Neurobiology of relapse to alcohol in rats. Pharmacol Ther 94:137–156. Leung HT, Westbrook RF (2010) Increased spontaneous recovery with increases in conditioned stimulus alone exposures. J Exp Psychol Anim Behav Process 36:354–367. Leung HT, Reeks LM, Westbrook RF (2012) Two ways to deepen extinction and the difference between them. J Exp Psychol Anim Behav Process 38:394–406. Loeber S, Croissant B, Heinz A, Mann K, Flor H (2006) Cue exposure in the treatment of alcohol dependence: effects on drinking outcome, craving and self-efficacy. Br J Clin Psychol 45:515–529. Marlatt GA (1990) Cue exposure and relapse prevention in the treatment of addictive behaviors. Addict Behav 15:395–399. Morris RW, Bouton ME (2007) The effect of yohimbine on the extinction of conditioned fear: a role for context. Behav Neurosci 121:501–514. Mueller D, Cahill SP (2010) Noradrenergic modulation of extinction learning and exposure therapy. Behav Brain Res 208:1–11. O’Brien CP, Childress AR, McLellan T, Ehrman R (1990) Integrating systematic cue exposure with standard treatment in recovering drug dependent patients. Addict Behav 15:355–365. Park J, Wheeler RA, Fontillas K, Keithley RB, Carelli RM, Wightman RM (2012) Catecholamines in the bed nucleus of Addiction Biology

Extinction of alcohol seeking

the stria terminalis reciprocally respond to reward and aversion. Biol Psychiatry 71:327–334. Park J, Bucher ES, Fontillas K, Owesson-White C, Ariansen JL, Carelli RM, Wightman RM. (2013) Opposing catecholamine changes in the bed nucleus of the stria terminalis during intracranial self-stimulation and its extinction. Biol Psychiatry 74:69–76. Rescorla RA (1997) Response-inhibition in extinction. Q J Exp Psychol B 50:238–252. Rescorla RA (2000) Extinction can be enhanced by a concurrent excitor. J Exp Psychol Anim Behav Process 26:251– 260. Rescorla RA (2004) Spontaneous recovery. Learn Mem 11:501– 509. Rescorla RA (2006) Deepened extinction from compound stimulus presentation. J Exp Psychol Anim Behav Process 32:135– 144. Rescorla RA, Heth CD (1975) Reinstatement of fear to an extinguished conditioned stimulus. J Exp Psychol Anim Behav Process 104:88–96. Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF, eds. Classical Conditioning II: Current Research and Theory, pp. 64–99. New York: Appleton-Century-Crofts. Sara SJ, Vankov A, Herve A (1994) Locus coeruleus-evoked responses in behaving rats: a clue to the role of noradrenaline in memory. Brain Res Bull 35:457–465. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599. Shalev U, Erb S, Shaham Y (2010) Role of CRF and other neuropeptides in stress-induced reinstatement of drug seeking. Brain Res 1314:15–28. Sinha R, Fox HC, Hong KA, Bergquist K, Bhagwagar Z, Siedlarz KM (2009) Enhanced negative emotion and alcohol craving, and altered physiological responses following stress and cue exposure in alcohol dependent individuals. Neuropsychopharmacology 34:1198–1208. Smith RJ, Aston-Jones G (2008) Noradrenergic transmission in the extended amygdala: role in increased drug-seeking and relapse during protracted drug abstinence. Brain Struct Funct 213:43–61.

© 2015 Society for the Study of Addiction

11

Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH (2013) A causal link between prediction error, dopamine and learning. Nat Neurosci 16:966–973. Sutton RS, Barto AG (1981) Toward a modern theory of adaptive networks: expectation and prediction. Psychol Rev 88:135–170. Vollstädt-Klein S, Loeber S, Kirsch M, Bach P, Richter A, Bühler M, von der Goltz C, Hermann D, Mann K, Kiefer F (2011) Effects of cue-exposure treatment on neural cue reactivity in alcohol dependence: a randomized trial. Biol Psychiatry 69:1060– 1066. Waelti P, Dickinson A, Schultz W (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature 412:43–48. Wagner AR (1981) SOP: a model of automatic memory processing in animal behavior. In: Spear NE, Miller RR, eds. Information Processing in Animals: Memory Mechanisms, pp. 5–47. Hillsdale, NJ: Erlbaum.

SUPPORTING INFORMATION Additional Supporting Information may be found in the online version of this article at the publisher’s web-site: Figure S1 Trial data for extinction of the to-be compound and to-be single stimuli during initial extinction (Ext 1 and Ext 2), during the compound and single stimulus trials (Ext 3), and during the test of spontaneous recovery conducted 1 week later Figure S2 Trial data during initial extinction (Ext 1 and Ext 2), on the drug treatment day (Ext 3), and during the test of spontaneous recovery conducted 1 week later for animals treated with atomoxetine or saline Figure S3 Trial data for extinction of the to-be compound and to-be single stimuli during initial extinction (Ext 1 and Ext 2), during the compound and single stimulus trials under drug (Ext 3), and during the test of spontaneous recovery conducted 1 week later for rats treated with propranolol or saline

Addiction Biology