Behavioural Brain Research 275 (2014) 150–156

Contents lists available at ScienceDirect

Behavioural Brain Research journal homepage: www.elsevier.com/locate/bbr

Review

A new proposal for drug conditioning with implications for drug addiction: The Pavlovian two-step from delay to trace conditioning Robert J. Carey a,b,∗ , Marinette Pinheiro Carrera c , Ernest N. Damianopoulos d a

Research Service and Development (151), VA Medical Center, 800 Irving Avenue, Syracuse, NY 13210, USA Department of Psychiatry, SUNY Upstate Medical University at Syracuse, Syracuse, NY 13210, USA Behavioral Pharmacology Group, Laboratory of Animal Morphology and Pathology, State University of North Fluminense, Darcy Ribeiro, Avenida Alberto Lamego, 2000, Campos dos Goytacazes, 28013-600 RJ, Brazil d Research Service and Development (151), Room 326; VA Medical Center, 800 Irving Avenue, Syracuse, NY 13210, USA b c

a r t i c l e

i n f o

Article history: Received 4 August 2014 Received in revised form 24 August 2014 Accepted 26 August 2014 Available online 9 September 2014 Keywords: Drug conditioning Addiction Pavlovian trace conditioning Pavlovian delay conditioning Psycho-stimulant

a b s t r a c t Pavlovian conditioning of drug effects is generally acknowledged to be a critical factor in the development and persistence of drug addiction. In drug conditioning the focus has essentially been on one type of Pavlovian conditioning, namely, delay conditioning in which the CS and drug UCS overlap and are temporally contiguous. Another type of Pavlovian conditioning is trace-conditioning in which the CS terminates before the onset of the UCS. While trace conditioning has been extensively studied in conditioning studies using a punctual CS and a non-drug UCS, trace conditioning has not been considered as having a role in drug conditioning. In several recent reports we have conducted experiments in which we first established a contextual drug CS using a delay conditioning protocol and subsequently used this same CS in a trace drug conditioning protocol with the same or different drug treatment and showed that the CS could be strongly modified by trace conditioning. These observations take on importance in that it has been well established that delay and trace conditioning are mediated by different CNS systems. Delay conditioning is mediated by cerebellar mechanisms, conforming to the general idea of Pavlovian conditioning as a reflexive type of learning whereas trace conditioning involves the hippocampus and frontal cortex brain structures more commonly associated with voluntary behavior. In this proposal we suggest that the emergence of potent drug associations that motivate drug-seeking behavior and addiction are initiated by delay conditioning and subsequently amplified and linked to higher brain functions by trace conditioning. © 2014 Elsevier B.V. All rights reserved.

Contents 1.

2.

3.

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1. Pavlovian delay and trace conditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1. Differential CNS neural systems mediating delay and trace conditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Apomorphine: a dose dependent dopamine agonist/antagonist interacting differentially with delay vs. trace conditioning paradigms . . . . . . . 2.1. Bidirectional behavioral effects of Apomorphine: as a function of dose level . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1. Differential apomorphine high dose vs. low dose conditioning effects: conditioning vs. no conditioning . . . . . . . . . . . . . . . . . . . . . . 2.2. Absence of conditioning of the apomophine low dose induced hypo-activity effect with a delay conditioning protocol but robust conditioning with a trace conditioning protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1. Design characteristics of the effective apomorphine low dose trace conditioning paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2. Potentiation of conditioned hyperactivity effects with trace conditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Differential CNS mechanisms mediating trace conditioning effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1. Brain systems involved in the mediation of trace conditioning: frontal cortices and hippocampus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

∗ Corresponding author at: Research Service and Development (151), VA Medical Center, 800 Irving Avenue, Syracuse, NY 13210, USA. Tel.: +1 315 6829251. E-mail address: [email protected] (R.J. Carey). http://dx.doi.org/10.1016/j.bbr.2014.08.053 0166-4328/© 2014 Elsevier B.V. All rights reserved.

151 151 151 152 152 152 152 152 153 153 153

4. 5.

R.J. Carey et al. / Behavioural Brain Research 275 (2014) 150–156

151

Overlap of Pavlovian and instrumental conditioning in the trace drug conditioning paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1. Pavlovian conditioning versus instrumental conditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Relevance to drug addiction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1. Two Steps in the efficacy of Pavlovian dopaminergic drug conditioning in promoting chronic drug use: delay conditioning is the first step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2. Two steps in the efficacy of Pavlovian dopaminergic drug conditioning in promoting chronic drug use: trace conditioning is the second step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3. Reversal of psycho-stimulant sensitization and conditioning: a potential application to drug addiction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

153 153 154

1. Introduction It is widely accepted that activation of dopamine systems is a common feature of drugs of abuse. Of major importance for the addictive potency of psycho-stimulant drugs is the transformation of the contiguous contextual stimuli into conditioned stimuli and incentive stimuli that can motivate and maintain addictive behavior. In that the pairing of a drug effect with a specific environment is essentially a Pavlovian drug conditioning protocol, it is not surprising that the contextual cues can acquire conditioned stimulus properties; and, in a post-treatment non-drug test evoke, a conditioned drug response. In this paper we detail how the two Pavlovian conditioning processes, delay and trace conditioning, can interact and contribute to the development of addiction to psycho-stimulant drugs. Psycho-stimulant drugs have a well-documented high abuse liability that is associated with, if not attributed to, its potential to induce sensitization effects with repeated use. The repeated use of psycho-stimulants induces both context specific sensitization and conditioned effects that enhance the drug effects [1–10]. These features of psycho-stimulant drugs are generally ascribed to the pro-dopamine effects of these drugs such that the association of dopamine activation to contextual cues can transform the associated stimuli into conditioned and incentive stimuli that can promote further drug taking and seeking [11]. These associations formed between context and drug are a major obstacle in the treatment of psycho-stimulant drug addiction in that conditioned situational stimuli can act as triggers and initiate drug craving even after prolonged periods of drug abstinence. Drug dependent individuals typically demonstrate enduring cue and contextual reactivity to drug associated stimuli. Indeed, drug cue exposure in cocaine abusing individuals results in real-time drug craving and consequent drug use [12]. Clearly, the reduction of cue reactivity to conditioned drug stimuli is a highly desirable objective in drug abuse treatment. 1.1. Pavlovian delay and trace conditioning In Pavlovian or classical conditioning there are various ways in which the CS and UCS can be paired. Most often the CS and the UCS overlap and Pavlov termed this arrangement delay conditioning. In that the CS and UCS are present in temporal contiguity, the association appears straightforward. Pavlov also showed that if the CS is presented and terminated before the onset of the UCS conditioning can also occur and he labeled it trace conditioning. In order to induce trace conditioning in Pavlov’s model [13,14] the interval or gap between the CS and UCS needed to be brief. Pavlov assumed that a transient trace of the discrete CS persists in the CNS after its termination and therefore can overlap with the UCS so that contiguity is preserved. In the reverse arrangement where the UCS precedes and terminates before the CS Pavlov termed this backward conditioning; and, no conditioning occurs. Seemingly though, a UCS being of substantially greater intensity than a CS, a UCS trace would persist for an even longer duration than a CS so UCS-CS overlap

154 154 154 155

would occur. Pavlov’s studies as well as many subsequent investigations [15–17] have confirmed the forward unidirectional nature of CS/UCS conditioning. In that the CS is a neutral and inconsequential stimulus it is easy to understand the irrelevance and absence of adaptive value in having a biologically significant UCS predict a biologically insignificant CS. In preclinical Pavlovian psycho-stimulant drug conditioning studies the simplest and most common arrangement is to administer the drug prior to placement in a test environment that serves as the contextual CS. This contrasts with conventional Pavlovian conditioning in which the CS is restricted to one sensory system (visual, auditory, tactile) and closely paired temporally to the UCS. In this discrete CS conditioning arrangement, the general testing environment provides the environmental context for the conditioning and can contribute to the conditioning as an occasion setter [18] rather than simply as a CS. In contrast, in many preclinical Pavlovian conditioning studies using a drug as a UCS the experimental test environment in which the drug response occurs serves as the CS. Typically, drug conditioning protocols differ from the standard Pavlovian conditioning paradigm in that the drug treatment (the UCS) precedes rather than follows the CS but critically the UCS and the CS overlap extensively so that drug conditioning can be considered a kind of reverse Pavlovian delay conditioning. While the administration of the drug UCS, procedurally, precedes the contextual CS, this arrangement is unlike backward conditioning in that the onset of the psycho-stimulant UCS drug effect is not followed by an inconsequential stimulus as in conventional conditioning but rather the psycho-stimulant drug effect induces sensory/motor activation so that the environmental context is transformed into a highly salient stimulus complex by the drug UCS. Another deviation of psycho-stimulant drug conditioning from conventional Pavlovian conditioning is that the UCS drug treatment can last for a substantial duration and it occurs in a test environment serving as the CS. Perhaps, the most straightforward drug conditioning entails the use of psycho-stimulant drugs such as cocaine, amphetamine, apomorphine, etc. that increase spontaneous locomotor behavior and the conditioned drug response is hyperactivity relative to the unpaired control group. While this paradigm can have complications that are not completely resolved with the paired/unpaired experimental design [19,20], there is little doubt that this is a reliable drug conditioning effect [21]. The use of a locomotor stimulant drug response to investigate conditioning fulfills a basic element in Pavlovian conditioning in that the conditioned response is similar albeit attenuated in magnitude to the unconditioned drug response [22]. 1.1.1. Differential CNS neural systems mediating delay and trace conditioning Given the limited neurophysiological information and capacity to investigate the neural processes available to the Pavlov group, it is only recently that the neural mechanisms mediating Pavlovian delay and forward trace conditioning have emerged. It is

152

R.J. Carey et al. / Behavioural Brain Research 275 (2014) 150–156

now appears well-established that delay and trace conditioning are mediated by different neural systems with the cerebellum having an important role in delay conditioning [23] but forebrain structures such as the hippocampus and frontal cortex having a more pivotal role in trace conditioning [24,25]. These neural system differences tie in with memory processes in that delay conditioning is consistent with manipulations that affect procedural memory whereas trace conditioning is affected by variables pertinent to declarative memory [26]. 2. Apomorphine: a dose dependent dopamine agonist/antagonist interacting differentially with delay vs. trace conditioning paradigms 2.1. Bidirectional behavioral effects of Apomorphine: as a function of dose level We have found the drug apomorphine is of particular interest in terms of dopamine drug conditioning in that this drug can have potent but opposite effects upon dopamine neurotransmission depending upon dose level. At the low dose range in rats (0.5 mg/kg) apomorphine increasingly stimulates dopamine postsynaptic receptors and is a behavioral stimulant and generates increased levels of locomotor behavior [30–32]. 2.1.1. Differential apomorphine high dose vs. low dose conditioning effects: conditioning vs. no conditioning In line with the presumed role of dopamine in the formation of stimulus–response associations, the high dose locomotor activation induced by a high dose of apomorphine is readily conditioned to the associated environmental cues [33,34]. In contrast, the low dose apomorphine treatment induces a pronounced hypo-activity; and, even after repeated pairings of this apomorphine inhibitory effect to environmental cues, it does not produce a conditioned inhibitory locomotor response [35,36]. This disparity appears consistent with the importance of dopamine in learning and memory in that the behavioral inhibition manifested in response to the low dose apomorphine treatment is reflective of dopamine inactivity wherein both sensory and motor systems are suppressed. This circumstance is seemingly unique to auto-receptor agonist induced hypo-activity such as induced by a low dose of apomorphine in that hypo-activity induced by dopamine postsynaptic antagonists such as haloperidol can be conditioned even though the suppression of locomotion can be severe [37–39]. Unlike the case of low dose apomorphine where the dopamine neurons are inactive, the dopamine antagonism at the postsynaptic location increases the activity of the dopamine neurons as a compensatory mechanism to overcome the postsynaptic receptor block [40]. 2.2. Absence of conditioning of the apomophine low dose induced hypo-activity effect with a delay conditioning protocol but robust conditioning with a trace conditioning protocol We recently reported [41,42] that low dose apomorphine induced hypo-activity could be conditioned. This was a startling and unexpected outcome. What made this finding so surprising and unexpected was the way in which the conditioning was induced. Unlike the conventional drug conditioning protocol, this conditioning was achieved using a trace conditioning protocol. In a trace conditioning protocol the CS is terminated before the UCS is presented. In a conventional trace conditioning procedure such as eye-blink conditioning the gap between the termination of the

CS and the onset of the UCS is brief on the order of 0.5 s and the context remains the same. In the case of psycho-stimulant conditioning, however, there are no reports of trace conditioning; and, all reported conditioning effects entail delay conditioning in which there is an overlap of many seconds between the drug effect and a CS such as an open-field environment. If a trace conditioning procedure were to be attempted, then the gap between the CS (the test environment) and administration of the drug treatment following removal of the animal from the test environment would seemingly be too long for conditioning to occur. In addition, the context in which the drug effect occurred in this protocol would be different from the open-field CS in that the drug is administered after the animal is removed from the open-field test environment. For these reasons there would be little reason to expect conditioning. On the other hand, the drug CS evokes a CR that persists for several minutes and includes strong arousal, emotional and neuro-endocrine components that would not cease immediately in the absence of the CS [34,43]. Guided by this premise we undertook to assess conditioning of a low dose apomorphine response inhibition effect with a trace-conditioning paradigm. 2.2.1. Design characteristics of the effective apomorphine low dose trace conditioning paradigm In conducting this trace drug conditioning protocol, we first had to establish a potent dopamine agonist induced conditioned stimulus by pairing a high dose (2.0 mg/kg) apomorphine treatment with test environment placement [34]. After the conditioned stimulant response was established selectively in the paired group then the trace conditioning protocol was initiated. Briefly, the paired group as well as the saline control and unpaired groups were given a 5 min exposure to the test environment in which the conditioning had been induced and then either immediately or 15 min after removal from the test environment the groups received SC injections of either a low dose of apomorphine (0.05 mg/kg) or vehicle and were removed from the testing room. The use of the 5 min test environment exposure was not entirely arbitrary in that we have shown that the CR induced by a high dose of apomorphine lasts for approximately 10 min [34] so we selected the midpoint as a likely duration to evoke the maximum CR and presumed that the CS activated interoceptive and exteroceptive cues would still have the potential to persist for a sufficient duration after removal from the test environment to become associated with the immediate post-trial drug treatment but not to last long enough to become associated after a 15 min post-trial delay. In contrast, the control and unpaired groups had previously only received several non-drug exposures to the test environment cues so that the cue activation evoked by the brief test environment exposure in these groups would have a salience level insufficient in intensity and duration persist to become associated with the post-trial treatments. In subsequent saline conditioning test sessions, only the group that had previously been conditioned with the high dose of apomorphine and then, received the immediate post-trial low dose of apomorphine treatment, exhibited a conditioned behavioral inhibition; i.e., hypoactivity. In all other groups (unpaired and vehicle control groups) the immediate low dose apomorphine post-trial trace conditioning treatment was without effect and the activity levels were the same as before the trace conditioning treatment. In all the 15 min delay groups the post-session treatments had no effect on behavior in the subsequent test for conditioning. Thus, the trace-conditioning group that had been hyperactive, compared to the control groups before the trace conditioning protocol, became hypoactive relative to the control groups following the trace conditioning protocol. In a related counter-conditioning experiment, groups also conditioned with the high dose of apomorphine were given the same low dose (0.05 mg/kg) apomorphine treatments but in a conventional delayconditioning protocol as a counter-conditioning treatment (either

R.J. Carey et al. / Behavioural Brain Research 275 (2014) 150–156

immediately or 15 min) before a 5 min placement in the test environment. These groups did not develop a conditioned response inhibitory effect even though the low dose apomorphine treatments given before placement in the test environment induced a profound drug induced response inhibition in the test environment as the unconditioned drug response [41]. This is the first instance in which a conditioning protocol is used as a variable, trace vs. delay, and conditioning is induced with the trace conditioning protocol but not with the delay conditioning protocol. The different results obtained in these two experimental protocols are surprising in that diametrically opposite responses were evoked in the presence of the CS in the two conditioning protocols. In the trace conditioning protocol the response in the presence of the contextual CS is hyperactivity whereas in the delay conditioning counter-conditioning protocol the response in the presence of the CS is a profound response inhibition induced by the low dose of apomorphine yet conditioned hypo-activity was only induced with the trace conditioning protocol. The absence of conditioning with the counter-conditioning protocol is consistent with the important role of dopamine in conditioning in that the response inhibition induced by the low dose of apomorphine is a consequence of autoreceptor stimulation of dopamine neurons thereby inactivating the dopamine neurons. Of course, the response inhibition observed is not simply a motoric inhibition in that inactivity in the dopamine system also inactivates sensory as well as motor responses. Consequently, processing of stimuli as well as motoric behavior is suppressed so that the CS is essentially not present and unavailable to be associated with the drug effect. The induction of a conditioned response of hyper-locomotion following high dose apomorphine delay conditioning treatments also induces a context specific sensitization response to the high dose apomorphine treatment. Importantly, We have shown [33] that the subsequent use of the trace conditioning protocol with the low dose apomorphine treatment not only converts the conditioned hyper-locomotion response to a conditioned hypolocomotion response to the test environment cues but, in addition, eliminates the sensitization response to the high dose apomorphine treatment. All other previously sensitized groups retained sensitization after being given the same low dose apomorphine treatments either prior to exposure to the test environment cues in a counter-conditioning protocol; or, if given in an ineffective trace conditioning protocol 15 min after exposure to the test environment cues. It is important to recognize that the apomorphine sensitization is context dependent so that it is controlled by the cues associated with the sensitization treatment. The loss of the sensitization effect is not simply a devaluation of the conditioned response cues in that extinction of the conditioned apomorphine response does not affect the apomorphine sensitization response [44]. The absence of an effect of extinction upon context dependent sensitization is not unique to apomorphine sensitization but is typical for psycho-stimulant drug sensitization [4]. Thus, the surprising efficacy of the trace conditioning protocol using a low inhibitory dose of apomorphine to induce both a conditioned response of hypo-activity and to de-sensitize dopamine receptors points to a profound re-organization of associational linkage of the cues to the output systems (Glutamate, Acetylcholine, etc.) impacted by dopamine receptors. 2.2.2. Potentiation of conditioned hyperactivity effects with trace conditioning Complementary to the effects of trace conditioning of low dose apomorphine response inhibition, we have also shown that following conditioning with a high dose of apomorphine, that, if a high rather than low dose of apomorphine is administered immediately after a 5 min exposure to the test environment in a trace

153

conditioning protocol the conditioned hyper-locomotion response is enhanced [34]. In contrast, if the same high dose of apomorphine is administered to groups previously conditioned with a high dose of apomorphine prior to a 5 min CS exposure in a delay conditioning protocol then, the conditioned hyper-locomotion response is not augmented [41]. Importantly, at asymptotic levels of conditioning the trace conditioning protocol is effective in augmenting the conditioned response while additional delay conditioning is without effect. This lack of efficacy of additional delay conditioning pairings appears understandable when it is considered that the initial induction of the CR entailed five 20 min CS-UCS delay conditioning pairings so that the 1–3 additional 5 min delay conditioning CS-UCS pairings would not be expected to produce a measurable increase in the CR. In fact, our previous findings indicated that asymptotic levels of CR and sensitization are reached after three to four 20 min delay-conditioning pairings of 2.0 mg/kg apomorphine with placement in the test environment [41,42]. Our finding that the conditioned drug response and sensitization response can be increased in a trace conditioning protocol after asymptotic conditioning had been achieved with a delay conditioning protocol indicates that the trace conditioning protocol brings additional mechanisms into the conditioning process. 3. Differential CNS mechanisms mediating trace conditioning effects 3.1. Brain systems involved in the mediation of trace conditioning: frontal cortices and hippocampus In psycho-stimulant drug conditioning generally and apomorphine conditioning specifically, the drug sensitization effects as well as the conditioned drug effects are context specific, indicative of a control by complex sensory/motor processes. Numerous non-drug Pavlovian trace-conditioning reports have recently demonstrated the involvement of higher order brain systems in the mediation of trace conditioning including the frontal cortices and hippocampus [45,46]. In drug conditioning with dopaminergic drugs the importance of dopamine in frontal cortex is well documented [47–51]. Also, we have found that the phosphorylated extracellular signal-regulated kinase (ERK) is selectively increased in the frontal cortex following apomorphine-induced sensitization [52] and increases in ERK have been repeatedly reported in the frontal cortex following cocaine treatment [53]. Moreover, as one considers the phenomenon of trace conditioning, the distinction between Pavlovian trace conditioning and operant or instrumental conditioning becomes blurred. In a recent seminal analysis of Conditioned Place Preference (CPP) the importance of classical and operant conditioned responses to the occurrence of CPP has highlighted this issue and brought into question facile hedonic interpretations of CPP [54]. 4. Overlap of Pavlovian and instrumental conditioning in the trace drug conditioning paradigm 4.1. Pavlovian conditioning versus instrumental conditioning Pavlov [13,14] focused on conditioning of the autonomic nervous system and for some time Pavlovian conditioning was considered an autonomic nervous system form of conditioning in contrast to instrumental conditioning that involved the skeletalmuscle motor system. Subsequently, it has become recognized that the key difference between instrumental and classical or Pavlovian conditioning is not defined by the nature of the elicited/emitted response (i.e., autonomic versus skeletal-muscle motor responses) rather, the difference is in the order and sequence in which the

154

R.J. Carey et al. / Behavioural Brain Research 275 (2014) 150–156

target response is elicited/emitted in relation to the UCS. In instrumental conditioning the response is dependent upon ongoing behavior in that the selected target response first needs to be emitted by the organism in order to have the UCS presented; that is, the response is instrumental to the presentation of the UCS. In classical conditioning the response is directly evoked by the UCS and is not contingent upon the organism’s ongoing behavior. While this basic distinction fits well with Pavlovian delay conditioning, in trace conditioning the gap between the CS and the UCS creates the opportunity for an implicit contingency in that it is the cortical systems mediating the trace that lead to the presentation of the UCS. This cortical activation prior to the presentation of the UCS creates the potential for an inference of a contingency of the cortical CS representation leading to the delivery of the UCS. In instrumental conditioning the cortical systems that initiate the instrumental response are followed by a UCS. Consequently, the organism can infer control of the contingency in both circumstances even though in both instances it is the experimenter that determines the actual presentation of the UCS. 5. Relevance to drug addiction 5.1. Two Steps in the efficacy of Pavlovian dopaminergic drug conditioning in promoting chronic drug use: delay conditioning is the first step Clinical research has shown in a number of studies that the Pavlovian conditioned effects induced by psycho-stimulant drugs such as cocaine can be powerful and intense and make abstinence very difficult to achieve [55]. While numerous preclinical reports have demonstrated Pavlovian conditioning of psycho-stimulant drugs; however, these conditioned drug effects are generally modest in comparison to the drug-induced responses and are readily extinguished [4]. We posit that a new way to conceptualize the conditioning manifested in clinical studies is to consider that the initial conditioning and impetus for further drug use is a product of delay conditioning in which there is the formation of a simple association between the stimulus compound of contextual cues and the drug experience. This association energizes and elevates the salience of the contextual cues and makes trace conditioning possible as a second step. It is important to recognize that the cue activation of the drug state evokes an intense sensory/motor stimulation including both strong internal and external sensory/motor responses. This consideration can account for the persistence of the effects of cue exposure to make it sensitive to trace conditioning as well as the capability of emotional responses to trigger drug craving [56]. 5.2. Two steps in the efficacy of Pavlovian dopaminergic drug conditioning in promoting chronic drug use: trace conditioning is the second step After the initial induction of Pavlovian delay conditioning the conditioned drug cues with elevated salience, subsequently, are followed by additional drug administration so that the cues and the anticipatory responses can be strengthened through trace conditioning and thereby engage cortical and hippocampal brain systems. In this way conditioned drug effects progress from the primitive sub-cortical limbic brain reward processes initiated by delay conditioning mechanisms to become linked by trace conditioning to higher brain goal directed systems. As trace conditioning enhances the addiction process it also subverts these higher brain systems toward drug consumption. In this expanded application of Pavlovian conditioning to drug addiction, the Pavlovian drug conditioning is a two-step process that occurs sequentially with repeated drug use. While this

two-step conditioning process is seemingly built into the development of drug addiction, in any preclinical conditioning experimental study it needs to be explicitly manipulated. In our studies [34] we showed that when trace conditioning was experimentally employed following a conventional delay conditioning procedure that the strength of conditioned response was increased. This approach needs to be greatly expanded both in terms of the drugs used as well as in the number of trace conditioning pairings administered to determine if both magnitude of the CR and resistance to extinction are increased in order to develop a fuller appreciation of the potential efficacy of delay and trace conditioning protocols administered in tandem with psychostimulant drugs. In our studies we used a 5 min exposure to the drug induced CS but the duration of the exposure needs to be systematically manipulated and used in different drug conditioning models. Furthermore, the rate of onset of the post-trial treatment is a function of the drug and route of administration and is of critical importance with regard to the efficacy of the trace conditioning protocol. Certainly, trace conditioning would appear to be readily applicable to CPP. After initially inducing CPP with conventional procedures a trace conditioning protocol could be employed using a brief exposure to the CPP cues followed by drug treatments designed to enhance or reverse the CPP. In this way one of the core models for the preclinical investigation of addictive drugs could be employed in conjunction with trace conditioning to modulate CPP. Finally, in the analysis we have presented we focused on the transformation of “neutral” cues with low or modest salience to a high salience state by delay conditioning with psycho-stimulant drugs. On the other hand, if the cues have an intrinsic initial high salience, for example, a highly novel environment then trace conditioning can occur as the first step and potent psycho-stimulant conditioning can seemingly be induced if a brief exposure to the high salience novel environment stimulus is followed by a psychostimulant drug treatment. The key element is the salience state of the cues and that trace conditioning of psycho-stimulant drugs requires an activated highly salient CS. 5.3. Reversal of psycho-stimulant sensitization and conditioning: a potential application to drug addiction While the Pavlovian two-step conditioning process would appear to be built into the development of strong associations between internal and external cues and drug effects to foster addiction it would seem that it is also the case that trace conditioning can be used to potentially reverse or blunt conditioning to the addictive drug. We have amassed substantial evidence that our protocol can induce a drug trace conditioning substitution effect [34,40]. That is, during a drug conditioned stimulus salience activation state the CS can become linked by trace conditioning to a different drug state; in this instance, a conditioned inhibitory response. Rather than using the post-trial protocol to interfere with the conditioned cue/dopamine association, this strategy entails the use of a trace conditioning protocol to connect the CS to an alternative dopamine inactivation state and have this anti-dopamine drug state substitute for the original conditioned dopamine drug state. Guided by this formulation, it will be important to use this trace conditioning anti-dopamine drug treatment strategy in preclinical experimentation to modify drug induced conditioned effects after conditioned and sensitized drug responses have been induced by addictive drugs such as cocaine and morphine. This approach is novel in that it uses a trace conditioning protocol to link a different drug treatment that is antagonistic and inhibitory to the high salience CS originally induced by the addictive drug. This finding is unlike that found with non-specific memory impairment treatments and is suggestive of a very different process than interference with memory processes such as re-consolidation.

R.J. Carey et al. / Behavioural Brain Research 275 (2014) 150–156

Rather than interfering with re-consolidation of the original drugcue association, a new conditioned response becomes linked to the CS. That is, a conditioned behavioral inhibitory response takes the place of the conditioned excitatory response to the context CS. Thus, the addictive drug induced conditioning is not simply diminished or disrupted but is replaced by one that is incompatible with the drug seeking state. Critically, the drug substitution we have used in our preclinical studies, namely, a low dose apomophine is effective for inducing conditioned anti-dopaminergic effects only when targeted to coincide with the cues that activate the addictive drug state. This approach is unlike other neuropharmacological approaches that are directed at the modification of the substrates of the associational neuronal processes such as protein synthesis mechanisms to disrupt re-consolidation and, therefore, entail the uncertain risks embedded in the manipulation of complex neuro-chemical interactions [57] Thus, the trace conditioning drug substitution approach offers the possibility of the development of new uses of established pharmacological agents such as a low dose of apomorphine in a trace conditioning procedure designed to address critical addiction resistant issues such as craving and relapse.

References [1] Borowski B, Kuhn CM. Chronic cocaine administration sensitizes behavioral but no neuroendocrine responses. Brain Res 1991;543:301–6. [2] Heidbrenner CS, Shippenberg TS. U-69593 prevents cocaine sensitization by normalizing basal accumbens dopamine. NeuroReport 1994 1994;5, 17971780. [3] Mattingly BA, Hart TC, Lim K, Perkins C. Selective antagonism of dopamine D1 and D2 receptors does not block the development of behavioral sensitization to cocaine. Psychopharmacology (Berlin) 1994;114:239–42. [4] Carey RJ, Gui J. Cocaine conditioning and cocaine sensitization: what is the relationship? Behav Brain Res 1998;1998(92):67–76. [5] Bloise E, Carey RJ, Carrera MP. Behavioral sensitization produced by a single administration of apomorphine: implications for the role of Pavlovian conditioning in the mediation of context-specific sensitization. Pharmacol Biochem Behav 2007;86:449–57. [6] Braga P, Galvanho JP, Bloise E, Carey RJ, Pinheiro-Carrera M. The expression of locomotor sensitization to apomorphine is dependent on time interval between injection and testing. Pharmacol Biochem Behav 2009;91:278–82. [7] Braga PQ, Dias FRC, Carey RJ, Pinheiro-Carrera M. Behavioral sensitization to dopaminergic inhibitory and stimulatory effects induced by low vs. high dose apomorphine treatments: an unconventional dose and response reversal sensitization challenge test reveals new sensitization mechanisms. Behav Brain Res 2009;204:169–74. [8] Dias F, Carey RJ, Carrera M. Apomorphine induced context specific behavioural sensitization is prevented by the D1 antagonist SCH-23390 but potentiated and uncoupled from contextual cues by the D2 antagonist sulpiride. Psychopharmacology (Berlin) 2010;209:137–51. [9] Filip M, Alenina N, Bader M, Przegalinski E. Behavioral evidence for the significance of serotonin (5-HT) receptors in cocaine addiction. Addict Biol 2010;15:227–49. [10] de Matos LW, Carey RJ, Carrera MA. Apomorphine conditioning and sensitization: the paired/treatment order as a new major determinant of drug conditioned and sensitization effects. Pharmacol Biochem Behav 2010;96:317–24. [11] Robinson CE, Berridge KC. The neural basis of drug craving: an incentivesensitization theory of addiction. Brain Res Rev 1993;18:247–91. [12] Epstein DH, Willner-Reid J, Vahabzadeh M, Mezghanni M, Lin J, Preston KL. Real time electronic diary reports of cue exposure and mood in the hours before cocaine and heroin craving and use. Arch Gen Psychiatr 2009;66: 88–94. [13] Pavlov IP. Conditioned reflex. London: Oxford University Press; 1927. [14] Pavlov IP. Lectures on conditioned reflexes. London: Oxford University Press; 1928. [15] Rescorla RA, Wagner AR. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In: Black AH, Prokasy WF, editors. Classical conditioning. New York: Appleton-Century-Crofts; 1972. p. 64–9. [16] Estes WK, Suppes P. Foundations of stimulus sampling theory. In: Kranz DH, Luce RD, Atkinson R, Suppes P, editors. Contemporary developments in mathematical psychology, 2. San Francisco: Freeman Press; 1974. p. 26. [17] Prokasy WF, Harsanyi MA. Two phase model for human classical conditioning. J Exp Psychol 1968;78:359–68. [18] Holland PC. Occasion setting in Pavlovian conditioning. In: Medlin DL, editor. The psychology of learning and motivation. San Diego: Academic Press; 1992. p. 28.

155

[19] Carey RJ, DePalma G, Damianopoulos EN. Cocaine conditioned behavioral effects: a role for habituation processes. Pharmacol Biochem Behav 2003;74:701–12. [20] Carey RJ, Damianopoulos EN. Cocaine conditioning and sensitization: the habituation factor. Pharmacol Biochem Behav 2006;84:128–33. [21] Carey RJ, Müller CP. Serotonin and behavioral stimulant effects of addictive drugs. In: Biological research on addiction: comprehensive addictive behaviors and disorders. San Diego: Academic Press; 2013. p. 231–9. [22] Carey R, DePalma G, Damianopoulos E. Evidence for Pavlovian conditioning of cocaine induced responses linked to emotional behavioral effects. Pharmacol Biochem Behav 2005;80:123–34. [23] Thompson RF. An essential memory trace found. Neurobiol Learn Mem 2011;96:181–91. [24] Reis DS, Jarome TJ, Helmstetter FJ. Memory formation for trace fear conditioning requires ubiquitin-proteasome mediated protein degradation in the prefrontal cortex. Front Behav Neurosci 2013 Oct 23;7:150. [25] Weiss C, Disterhoft JF. Exploring prefrontal cortical memory mechanisms with eyeblink conditioning. Behav Neurosci 2011;125:318–26. [26] Clark RE, Manns JR, Squire LR. Trace and delay eyeblink conditioning: contrasting phenomena of declarative and nondeclarative memory. Psychol Sci 2001;12:304–8. [27] Aghajanian GK, Bunney BS. Central dopaminergic drugs: neurophysiological identification and responses to drugs. In: Snyder G, Usdin E, editors. Frontiers in catecholamine research. New York: Pergamon; 1973. p. 643–8. [28] Missale C, Nash SR, Robinson SW, Jabber M, Caron MG. Dopamine receptors: from structure to function. Physiol Rev 1998;78:189–225. [29] DiChiara G, Porceddu ML, Fratta W, Gessa GL. Postsynaptic receptors are not essential for dopaminergic feedback regulation. Nature 1977;267:270–2. [30] Mattingly BA, Gotsick JE, Marin C. Locomotor activity and stereotypy in rats following repeated apomorphine treatments at 1-, 3-, or 7-day intervals. Pharmacol Biochem Behav 1988;31:871–5. [31] Mattingly BA1, Gotsick JE, Salamanca K. Latent sensitization to apomorphine following repeated low doses. Behav Neurosci 1988;102:553–8. [32] Rowlett JK1, Mattingly BA, Bardo MT. Locomotor activity and dopamine synthesis following 1 and 15 days of withdrawal from repeated apomorphine treatments. Pharmacol Biochem Behav 1997;57:13–8. [33] Carrera MP, Carey RJ, Dias FR, de Matos LW. Reversal of apomorphine locomotor sensitization by a single post-conditioning trial treatment with a low autoreceptor dose of apomorphine. Pharmacol Biochem Behav 2011;99:29–34. [34] Carrera MP, Carey RJ, Dias FR, de Matos LW. Memory re-consolidation and drug conditioning: an apomorphine conditioned locomotor stimulant response can be enhanced or reversed by a single high versus low apomorphine post-trial treatment. Psychopharmacology 2012;220:281–91. [35] Braga PQ, Dias FR, Carey RJ, Carrera MP. Low dose apomorphine induces context-specific sensitization of hypolocomotion without conditioning: support for a new state dependent retrieval hypothesis of drug conditioning and sensitization. Pharmacol Biochem Behav 2009;93:128–33. [36] Braga PO, Dias FR, Carey RJ, Carrera MP. Behavioral sensitization to dopaminergic inhibitory and stimulatory effects induced by low vs. high dose apomorphine treatments: an unconventional dose and response reversal sensitization challenge test reveals sensitization mechanisms. Behav Brain Res 2009:204169–74. [37] Banasikowski TJ, Beninger RJ. Haloperidol conditioned catalepsy in rats: a possible role for D1-like receptors. Int J Neuropsychopharmacol 2012;15:1525–34. [38] Banasikowski TJ, Beninger RJ. Reduced expression of haloperidol conditioned catalepsy in rats by the dopamine D3 receptor antagonists nafadotride and NGB 2904. Eur Neuropsychopharmacol 2012;22:761–8. [39] Carey RJ, Kenney S. Operant conditioning and Haloperidol-induced Hypokinetic effects. Neuropsychobiology 1987;18:199–204. [40] Carey RJ, DeVeaugh-Geiss J. Treatment schedule as a determinant of the development of tolerance to haloperidol. Psychopharmacology (Berlin) 1984;82:164–7. [41] Marcos J, Bastos M, Regina F, Dias C, Alves VN, Carey RJ, et al. Drug memory substitution during re-consolidation: a single inhibitory autoreceptor apomorphine treatment given during psychostimulant memory re-consolidation replaces psychostimulant conditioning with conditioned inhibition and reverses psychostimulant sensitization. Behav Brain Res 2014;260: 139–47. [42] Carrera MP, Carey RJ, Cruz Dias FR, dos Santos Sampaio Mde F, de Matos LW. Post-trial apomorphine at an autoreceptor dose level can eliminate apomorphine conditioning and sensitization: support for the critical role of dopamine in re-consolidation. Behav Brain Res 2013;236:244–50. [43] Carey RJ, DePalma G, Damianopoulos E. Acute and chronic cocaine behavioral effects in novel versus familiar environments: open-field familiarity differentiates cocaine locomotor stimulant effects from cocaine emotional behavioral effects. Behav Brain Res 2005;158(2):321–30. [44] de Matos LW, Carey RJ, Carrera MP. Apomorphine conditioning and sensitization: the paired/unpaired treatment order as a new major determinant of drug conditioned and sensitization effects. Pharmacol Biochem Behav 2010;96:317–24. [45] Flesher MM, Butt AE, Kinney-Hurd BL. Differential acetylcholine release in the prefrontal cortex and hippocampus during pavlovian trace and delay conditioning. Neurobiol Learn Mem 2011;96:181–91. [46] McLaughlin J1, Skaggs H, Churchwell J, Powell DA. Medial prefrontal cortex and pavlovian conditioning: trace versus delay conditioning. Behav Neurosci 2002;116:37–47.

156

R.J. Carey et al. / Behavioural Brain Research 275 (2014) 150–156

[47] Weidemenn G, Broderick J, Lovibond PF, Mitchell CJ. Both trace and delay conditioned eyeblink responding can be dissociated from outcome expectancy. J Exp Psychol Anim Behav Process 2012;38:1–10. [48] Devonshire IM, Mayhew JE, Overton PG. Cocaine preferentially enhances sensory processing in the upper layers of the primary sensory cortex. Neuroscience 2007;146:841–51. [49] Rutter JJ, Devilbiss DM, Waterhouse BD. Effects of systemically administered cocaine on sensory responses to peri-threshold vibrissae stimulation: individual cells, ensemble activity, and animal behaviour. Eur J Neurosci 2005;22:3205–16. [50] Happel MF, Deliano M, Handschuh J, Ohl FW. Dopamine-modulated recurrent cortico-efferent feedback in primary sensory cortex promotes detection of behaviorally relevant stimuli. J Neurosci 2014;34: 1234–47. [51] Eshel N, Tian J. Dopamine gates sensory representations in cortex. J Neurophysiol 2014;111:2161–3.

[52] Sanguedo FV, Dias FR, Bloise E, Cespedes IC, Giraldi-Guimarães A, Samuels RI, et al. Increase in medial frontal cortex ERK activation following the induction of apomorphine sensitization. Pharmacol Biochem Behav 2014;118:60–8. ˜ [53] Munoz-Cuevas FJ, Athilingam J, Piscopo D, Wilbrecht L. Cocaine-induced structural plasticity in frontal cortex correlates with conditioned place preference. Nat Neurosci 2013;16:1367–9. [54] Huston JP, Silva MA, Topic B, Müller CP. What’s conditioned in conditioned place preference? Trends Pharmacol Sci 2013;34(3):162–6. [55] O’Brien CP, Childress AR, McLellan AT, Ehrman R. Developing treatments that address classical conditioning. NIDA Res Mono 1993;135:71–91 [Review]. [56] Childress AR, Ehrman R, McLellan AT, MacRae J, Natale M. Can induced moods trigger drug-related responses in opiate abuse patients. J Sub Abuse Treat 1994;11:17–23. [57] Aguilar-Valles A, Vaissière T, Griggs EM, Mikaelsson MA, Takács IF, Young EJ, et al. Methamphetamine-associated memory is regulated by a writer and an eraser of permissive histone methylation. Biol Psychiatr 2014;76:57–65.