Physiology& Behavior, Vol. 47, pp. 691-698. ©Pergamon Press plc, 1990. Printed in the U.S.A.
0031-9384/90 $3.00 + .00
Dynamics of Plasma Catecholamine and Corticosterone Concentrations During Reinforced and Extinguished Operant Behavior in Rats 1 S. F. DE B O E R , R. DE B E U N , J. L. S L A N G E N A N D J. VAN DER G U G T E N
Netherlands Institute for Drugs and Doping Research, Department of Psychopharmacology, University of Utrecht Sorbonnelaan 16, 3584 CA Utrecht, The Netherlands R e c e i v e d 29 S e p t e m b e r 1989
DEBOER, S. F., R. DEBEUN, J. L. SLANGEN AND J. VANDERGUGTEN. Dynamics of plasma catecholamine and corticosterone concentrations during reinforced and extinguished operant behavior in rats. PHYSIOL BEHAV 47(4) 691-698, 1990.--Plasma noradrenaline (NA), adrenaline (A) and corticosterone (CS) concentrations were determined simultaneously in permanently heart-cannulated rats before and during the performance of reinforced and nonreinforced (extinguished) operant behavior. Shortly before the experimental food-reinforced (VI 15-sec schedule) lever-pressing task, anticipatory elevations of plasma NA and CS contents were observed. During reinforced lever responding plasma NA increased, A did not change and CS declined. Extinction was associated with a transient increase in A, decreasing NA and elevated CS concentrations. In addition, a relationship was found between individual lever-pressing rate, neurosympathetic and adrenomedullary reactivity and the degree of schedule-induced polydipsia. The results indicate that presence and absence of expected behavioral consequences (controllability and loss of control, respectively) are attended by selective, but highly dissociated patterns of neurosympathetic, adrenomedullary and adrenocortical output. Collectively, the findings reinforce the concept that distinctive emotional and/or motivational states are associated with different patterns of neuroendocrine responses. The reactivity of the sympathoadrenomedullary component is heavily dependent upon a rat's individual behavioral strategy, Noradrenaline Rat
Adrenaline
Corticosterone
Operant behavior
IT is well known that the main neuroendocrine pathways which are recruited in the response of the organism to environmental and bodily demands are the neuronal and adrenomedullary branches of the sympathoadrenal system, as well as the pituitary-adrenocortical axis. Activation of these systems results in an increased release of noradrenaline (NA), adrenaline (A) and corticosterone (CS), respectively, into the bloodstream (1, 20, 41). Adequate physiological and behavioral adaptations require optimal neuroendocrine states (3,51), which, therefore, necessitate a finely tuned differentiation of neurosympathetic, adrenomedullary and adrenocortical outflow. In addition to a number of organismic characteristics (strain, sex, age, individual response strategy) and various physical stimulus properties (type, intensity, duration and frequency of stimulation) [see (20) for references], the plasma NA, A and CS response patterns seem to depend strongly on the degree of factual or perceived behavioral control over a stimulus situation (31, 32, 45, 53, 54). In animal psychoneuroendocrine research, the influence of this psychological factor has been studied predominantly on adrenocortical reactivity by use of a now-classic research
Extinction
Stress
Frustrative nonreward
paradigm which compares the consequences of controllable versus uncontrollable yoked noxious stimulation (intense noise, electric shock). In general, it has been shown that animals with the ability to alter onset, termination, duration, intensity or pattern of aversive stimulation respond with lower CS levels than yoked control subjects without this ability (2, 19, 21, 28, 48) or losing this ability (11, 24, 28, 37, 40, 43). This latter phenomenon, i.e., loss of behavioral control, can also be observed in appetitive operant-conditioning experiments: Animals trained to lever press for food or water reinforcement show a marked elevation of plasma CS levels when lever pressing is no longer reinforced. Anthropomorphically, this condition has been described as frustrative nonreward (8-10, 14, 18, 29, 42). Furthermore, CS concentrations increase when rats are switched to lower levels of reinforcement and decrease when switched from a lean appetitive reinforcement schedule to a denser one (8, 13, 27). These bidirectional changes in adrenocortical secretion have been interpreted to reflect the emotion producing quality (positive or negative) of a shift in the amount and/or frequency of reinforcement from that expected (30,43), rather than undifferentiated
1These investigations were supported in part by the Foundation for Medical and Health Research MEDIGON (grant No. 900-548-076). 691
692
DE BOER, DE BEUN, SLANGEN AND VAN DER GUGTEN
emotional arousal due to novelty of the new reinforcement schedule (18, 27, 30). At present, no data are available regarding sympathoadrenomedullary responses to reinforcement and extinction. Recently, however, plasma free fatty acid and glucose levels were shown to increase during water-reinforced operant behavior, while CS concentration was concomitantly decreasing (13). Since, under nonbasal conditions (physical activity, emotional stress), the sympathoadrenal system strongly regulates blood levels of FFA and glucose (44,56), it may be reasoned that during performance of operant behavior the sympathoadrenal system is activated. The aim of the present study was to investigate the effect of the psychological factor controllability upon both pituitary-adrenocortical and sympathoadrenomedullary functioning. Therefore, plasma NA, A and CS concentrations were determined simultaneously in individual rats during performance of reinforced (controllable situation) and nonreinforced (extinguished; loss of control) operant behavior.
home cage
experimental lever retracted
box lever available
,k\\\\\\\\\\\\\\\\\\\\\ \ \ \ \ \\ \ \ ~
1 1 1 1 1 1
~\\\\\\\\~
c : - : : ::--T-T-[C~ time
-35
samples
1 1 1 1 1 1
-30
10
-5
0
5
10
15 20
25 30 rain
2
3
4
5
6
7
9
8
(~
In/E-cood©
llljl E.... ,
U _- _- -_-_- _ C _ r
1
R-cond.
©
10
reinforcement period I
]
exlinction period
FIG. 1. Schematic representation of the experimental conditions.
METHOD
Animals and Housing Twelve male Wistar rats (CPB:WU, CPB-TNO, Zeist, The Netherlands), weighing approximately 300 g on their arrival in the laboratory, were housed individually in clear Plexiglas cages (25 x 25 × 30 cm) on woodshavings. Cages were placed in an animal room under conditions of constant ambient temperature ( 2 1 - 2 ° C ) and a fixed 12 hour light/12 hour dark photoperiod (lights on at 7:00 a.m.). Tap water was freely available at all times. Rats were maintained at approximately 85% of their free-feeding body weights by giving a daily amount (5-7% of their individual body weight) of lab chow (Muracon).
Apparatus The subjects were tested in two identical experimental boxes, each consisted of a modified home cage equipped with a retractable lever (4 cm wide) mounted 5 cm to the left of a food tray which was connected with a food pellet dispenser. A graduated water bottle was mounted outside the chamber, and the spout extended 1 cm into the box through a hole located approximately 10 cm from the food tray. The experimental boxes were set inside a testing room which had the same dimensions, temperature and lighting conditions as the animal room. The distance between the animal room and the testing room was approximately 6 m. Programming of reinforcement schedule and recording of responses (lever presses, pushing of the Plexiglas flap covering the food tray) and reinforcements was accomplished by standard electromechanical equipment (Campden Instruments LTD, London, U.K.) located outside the testing room. Water intake was calculated from fluid levels in the bottle before and after each session.
Thereafter, rats were allowed two days of free access to food before being equipped surgically with a silastic catheter (i.d. 0.5 mm; o.d. 1.0 mm) into the entrance of the right atrium (venae cava) via an external jugular venotomy and externalized on the top of the skull according to the techniques described by Steffens (46). Surgery was performed under complete ether anesthesia. After a 5-day recovery period, during which the rats received ad lib food, the food deprivation regimen was reinstated and rats were submitted to 6 supplementary VI 15-sec reinforcement sessions (30 min) preceded by 30-min adaptation periods. Ninety minutes before these experimental sessions, rats were connected to polyethylene blood-sampling tubing, in order to accustom them to the bloodsampling procedure. Blood was sampled from the animals during three different conditions using a balanced repeated measures within-subject design: condition A, 30-min reinforcement (VI 15 sec); condition B, 10-min reinforcement (VI 15 sec) followed by 20-min extinction; condition C, 30-min extinction (see Fig. 1). Between subsequent sampling sessions, rats received 5 daily normal VI 15-sec reinforcement sessions. During extinction, the lever was available and responding on it still activated the pellet dispenser, but no food was delivered in the food tray, thus, secondary cues are maintained. Blood samples (0.35 ml) were taken under baseline resting conditions at t = - 3 5 min in the rat's home cage, at the end (t = - 10 and - 5 min) of the 30-min adaptation period in the experimental box and at 5-min intervals up to 30 min after insertion (at t = 0 min) of the lever. Immediately after each sample, an equal volume of heparinized blood, freshly obtained from a cannulated donor rat, was transfused through the catheter. After experimentation, catheter patency was ensured by filling the cannula with a viscous solution containing 500 IU/ml heparin plus 60% polyvinylpyrrolidone (Merck; Darmstadt, F.R.G.) and closed with a small polyethylene plug until the animal was used again.
Procedure All animals were trained to press the lever for food pellet reinforcers (45-mg Noyes pellets) and given several (5-10) sessions of continuous reinforcement. Thereafter, they received 15-20 half-hour sessions with reinforcements presented according to a variable interval (VI) 15-sec schedule (range 2-32 sec) until cumulative response records revealed stability of responding within and between daily sessions. The daily VI reinforcement sessions were preceded by 30 min of adaptation to the experimental box, with the lever retracted. The training sessions for individual rats were performed at approximately the same time (between 9:00 a.m. and 1:00 p.m.) each day, 5 days a week.
Chemical Determinations Blood samples were immediately transferred to chilled (0°C) centrifuge tubes containing 10 ILl heparin solution (500 IU/ml) as anticoagulant. For the determination of plasma catecholamine (CA) contents, an aliquot of 250 1.1,1 transferred blood was immediately pipetted into chilled tubes containing 10 txl of a solution of 25 mg/ml disodium EDTA and 27.5 mg/ml reduced glutathione in order to prevent CA degradation. The remaining 100 ILl blood was used for the CS assay. After centrifugation (4000× g for 10 min at 4°C), supernatants were removed and stored at - 30°C.
STRESS HORMONES AND OPERANT BEHAVIOR
693
30,
6O0 20
P~
r-
~ 40o
£
,3
0031-9384/90 $3.00 + .00
Dynamics of Plasma Catecholamine and Corticosterone Concentrations During Reinforced and Extinguished Operant Behavior in Rats 1 S. F. DE B O E R , R. DE B E U N , J. L. S L A N G E N A N D J. VAN DER G U G T E N
Netherlands Institute for Drugs and Doping Research, Department of Psychopharmacology, University of Utrecht Sorbonnelaan 16, 3584 CA Utrecht, The Netherlands R e c e i v e d 29 S e p t e m b e r 1989
DEBOER, S. F., R. DEBEUN, J. L. SLANGEN AND J. VANDERGUGTEN. Dynamics of plasma catecholamine and corticosterone concentrations during reinforced and extinguished operant behavior in rats. PHYSIOL BEHAV 47(4) 691-698, 1990.--Plasma noradrenaline (NA), adrenaline (A) and corticosterone (CS) concentrations were determined simultaneously in permanently heart-cannulated rats before and during the performance of reinforced and nonreinforced (extinguished) operant behavior. Shortly before the experimental food-reinforced (VI 15-sec schedule) lever-pressing task, anticipatory elevations of plasma NA and CS contents were observed. During reinforced lever responding plasma NA increased, A did not change and CS declined. Extinction was associated with a transient increase in A, decreasing NA and elevated CS concentrations. In addition, a relationship was found between individual lever-pressing rate, neurosympathetic and adrenomedullary reactivity and the degree of schedule-induced polydipsia. The results indicate that presence and absence of expected behavioral consequences (controllability and loss of control, respectively) are attended by selective, but highly dissociated patterns of neurosympathetic, adrenomedullary and adrenocortical output. Collectively, the findings reinforce the concept that distinctive emotional and/or motivational states are associated with different patterns of neuroendocrine responses. The reactivity of the sympathoadrenomedullary component is heavily dependent upon a rat's individual behavioral strategy, Noradrenaline Rat
Adrenaline
Corticosterone
Operant behavior
IT is well known that the main neuroendocrine pathways which are recruited in the response of the organism to environmental and bodily demands are the neuronal and adrenomedullary branches of the sympathoadrenal system, as well as the pituitary-adrenocortical axis. Activation of these systems results in an increased release of noradrenaline (NA), adrenaline (A) and corticosterone (CS), respectively, into the bloodstream (1, 20, 41). Adequate physiological and behavioral adaptations require optimal neuroendocrine states (3,51), which, therefore, necessitate a finely tuned differentiation of neurosympathetic, adrenomedullary and adrenocortical outflow. In addition to a number of organismic characteristics (strain, sex, age, individual response strategy) and various physical stimulus properties (type, intensity, duration and frequency of stimulation) [see (20) for references], the plasma NA, A and CS response patterns seem to depend strongly on the degree of factual or perceived behavioral control over a stimulus situation (31, 32, 45, 53, 54). In animal psychoneuroendocrine research, the influence of this psychological factor has been studied predominantly on adrenocortical reactivity by use of a now-classic research
Extinction
Stress
Frustrative nonreward
paradigm which compares the consequences of controllable versus uncontrollable yoked noxious stimulation (intense noise, electric shock). In general, it has been shown that animals with the ability to alter onset, termination, duration, intensity or pattern of aversive stimulation respond with lower CS levels than yoked control subjects without this ability (2, 19, 21, 28, 48) or losing this ability (11, 24, 28, 37, 40, 43). This latter phenomenon, i.e., loss of behavioral control, can also be observed in appetitive operant-conditioning experiments: Animals trained to lever press for food or water reinforcement show a marked elevation of plasma CS levels when lever pressing is no longer reinforced. Anthropomorphically, this condition has been described as frustrative nonreward (8-10, 14, 18, 29, 42). Furthermore, CS concentrations increase when rats are switched to lower levels of reinforcement and decrease when switched from a lean appetitive reinforcement schedule to a denser one (8, 13, 27). These bidirectional changes in adrenocortical secretion have been interpreted to reflect the emotion producing quality (positive or negative) of a shift in the amount and/or frequency of reinforcement from that expected (30,43), rather than undifferentiated
1These investigations were supported in part by the Foundation for Medical and Health Research MEDIGON (grant No. 900-548-076). 691
692
DE BOER, DE BEUN, SLANGEN AND VAN DER GUGTEN
emotional arousal due to novelty of the new reinforcement schedule (18, 27, 30). At present, no data are available regarding sympathoadrenomedullary responses to reinforcement and extinction. Recently, however, plasma free fatty acid and glucose levels were shown to increase during water-reinforced operant behavior, while CS concentration was concomitantly decreasing (13). Since, under nonbasal conditions (physical activity, emotional stress), the sympathoadrenal system strongly regulates blood levels of FFA and glucose (44,56), it may be reasoned that during performance of operant behavior the sympathoadrenal system is activated. The aim of the present study was to investigate the effect of the psychological factor controllability upon both pituitary-adrenocortical and sympathoadrenomedullary functioning. Therefore, plasma NA, A and CS concentrations were determined simultaneously in individual rats during performance of reinforced (controllable situation) and nonreinforced (extinguished; loss of control) operant behavior.
home cage
experimental lever retracted
box lever available
,k\\\\\\\\\\\\\\\\\\\\\ \ \ \ \ \\ \ \ ~
1 1 1 1 1 1
~\\\\\\\\~
c : - : : ::--T-T-[C~ time
-35
samples
1 1 1 1 1 1
-30
10
-5
0
5
10
15 20
25 30 rain
2
3
4
5
6
7
9
8
(~
In/E-cood©
llljl E.... ,
U _- _- -_-_- _ C _ r
1
R-cond.
©
10
reinforcement period I
]
exlinction period
FIG. 1. Schematic representation of the experimental conditions.
METHOD
Animals and Housing Twelve male Wistar rats (CPB:WU, CPB-TNO, Zeist, The Netherlands), weighing approximately 300 g on their arrival in the laboratory, were housed individually in clear Plexiglas cages (25 x 25 × 30 cm) on woodshavings. Cages were placed in an animal room under conditions of constant ambient temperature ( 2 1 - 2 ° C ) and a fixed 12 hour light/12 hour dark photoperiod (lights on at 7:00 a.m.). Tap water was freely available at all times. Rats were maintained at approximately 85% of their free-feeding body weights by giving a daily amount (5-7% of their individual body weight) of lab chow (Muracon).
Apparatus The subjects were tested in two identical experimental boxes, each consisted of a modified home cage equipped with a retractable lever (4 cm wide) mounted 5 cm to the left of a food tray which was connected with a food pellet dispenser. A graduated water bottle was mounted outside the chamber, and the spout extended 1 cm into the box through a hole located approximately 10 cm from the food tray. The experimental boxes were set inside a testing room which had the same dimensions, temperature and lighting conditions as the animal room. The distance between the animal room and the testing room was approximately 6 m. Programming of reinforcement schedule and recording of responses (lever presses, pushing of the Plexiglas flap covering the food tray) and reinforcements was accomplished by standard electromechanical equipment (Campden Instruments LTD, London, U.K.) located outside the testing room. Water intake was calculated from fluid levels in the bottle before and after each session.
Thereafter, rats were allowed two days of free access to food before being equipped surgically with a silastic catheter (i.d. 0.5 mm; o.d. 1.0 mm) into the entrance of the right atrium (venae cava) via an external jugular venotomy and externalized on the top of the skull according to the techniques described by Steffens (46). Surgery was performed under complete ether anesthesia. After a 5-day recovery period, during which the rats received ad lib food, the food deprivation regimen was reinstated and rats were submitted to 6 supplementary VI 15-sec reinforcement sessions (30 min) preceded by 30-min adaptation periods. Ninety minutes before these experimental sessions, rats were connected to polyethylene blood-sampling tubing, in order to accustom them to the bloodsampling procedure. Blood was sampled from the animals during three different conditions using a balanced repeated measures within-subject design: condition A, 30-min reinforcement (VI 15 sec); condition B, 10-min reinforcement (VI 15 sec) followed by 20-min extinction; condition C, 30-min extinction (see Fig. 1). Between subsequent sampling sessions, rats received 5 daily normal VI 15-sec reinforcement sessions. During extinction, the lever was available and responding on it still activated the pellet dispenser, but no food was delivered in the food tray, thus, secondary cues are maintained. Blood samples (0.35 ml) were taken under baseline resting conditions at t = - 3 5 min in the rat's home cage, at the end (t = - 10 and - 5 min) of the 30-min adaptation period in the experimental box and at 5-min intervals up to 30 min after insertion (at t = 0 min) of the lever. Immediately after each sample, an equal volume of heparinized blood, freshly obtained from a cannulated donor rat, was transfused through the catheter. After experimentation, catheter patency was ensured by filling the cannula with a viscous solution containing 500 IU/ml heparin plus 60% polyvinylpyrrolidone (Merck; Darmstadt, F.R.G.) and closed with a small polyethylene plug until the animal was used again.
Procedure All animals were trained to press the lever for food pellet reinforcers (45-mg Noyes pellets) and given several (5-10) sessions of continuous reinforcement. Thereafter, they received 15-20 half-hour sessions with reinforcements presented according to a variable interval (VI) 15-sec schedule (range 2-32 sec) until cumulative response records revealed stability of responding within and between daily sessions. The daily VI reinforcement sessions were preceded by 30 min of adaptation to the experimental box, with the lever retracted. The training sessions for individual rats were performed at approximately the same time (between 9:00 a.m. and 1:00 p.m.) each day, 5 days a week.
Chemical Determinations Blood samples were immediately transferred to chilled (0°C) centrifuge tubes containing 10 ILl heparin solution (500 IU/ml) as anticoagulant. For the determination of plasma catecholamine (CA) contents, an aliquot of 250 1.1,1 transferred blood was immediately pipetted into chilled tubes containing 10 txl of a solution of 25 mg/ml disodium EDTA and 27.5 mg/ml reduced glutathione in order to prevent CA degradation. The remaining 100 ILl blood was used for the CS assay. After centrifugation (4000× g for 10 min at 4°C), supernatants were removed and stored at - 30°C.
STRESS HORMONES AND OPERANT BEHAVIOR
693
30,
6O0 20
P~
r-
~ 40o
£
,3
