Brain Research, 111 (1976) 185-188 © Elsevier ScientificPublishing Company, Amsterdam - Printed in The Netherlands
185
Reward related neurons in cat association cortex
HANS J. MARKOWITSCH and MONIKA PRITZEL Department of Psychology, University of Konstanz, Postbox 7733, D-7750 Konstanz ( G.F.R.)
(Accepted April 6th, 1976)
Electrophysiological studies have found several regions within the cat's brain which reveal similar response patterns that are specific to sensory stimulation in different modalities. Such regions have been named polysensory or association areas 1,12. Lesions of these areas lead to deficits in acquisition and/or retention of complex learning tasks2, 6. Within the frame of a learning paradigm this communication reports data disclosing reward related changes of single unit activity in association areas of the cat. Animals were first trained to perform a spatial reversal task modified so as to permit spatial responses while the cat's head was held stationary for the purpose of single-cell recording. Therefore, instead of using WGTAs or mazes, the cats were shaped to sit in a rectangular wooden box and to depress one of two retractable levers which were positioned within reach of the cat's paws and separated laterally by 12 cm. Responses to one side were rewarded (non-correction procedure) until the cat pressed the same lever 10 consecutive times within 50 trials. Then 1-3 reversals of this task were added, whereby the cat was required to press the lever on the other side for 10 consecutive trials. Due to the restriction of the cat's movements to lever pressing, the abundance of kinesthetic cues which can be assumed to be present in more conventional investigations of spatial reversal performance in cats may have been lacking in this situation. Perhaps for this reason the cats required 4-6 months to reach criterion. Three cats of the experimental group have so far been trained and operated. Openings in the skull were made over the prefrontal region and over the middle suprasylvian gyrus (PMSA lz) of the same hemisphere. A control cat was prepared with two trephine holes, one positioned over the prefrontal and one over the caudal portion of the pericruciate region (PCA12). Plastic chambers, insulated copper wires for EOG and E M G recordings, and two horizontal tubes to hold the cat's head rigidly during recording were fixed to the skull with dental acrylics. Though restricted during recording sessions, the cats usually performed the learned task in the same manner and to the same criterion as before the operation, where the cat's head was unrestrained. Fig. 1 shows an example of a trial sequence. The cat initiated a trial by pressing one of the response levers. The arm of a feeder moved to the cat's mouth
186 Lp
Lp
/T/
I IT/
0
1
2
3
4
5
6 7
sec.
Fig. 1. Example of a trial sequence. ITI: variable intertrial interval (depending on the cat's reaction); Lp: lever press (after response to left or right lever both retract for 7 secs); R: reinforcement period (if response was correct); Le: simultaneous extension of both levers (next response is possible). The time scale shows shortest duration of one trial (ITI period is omitted). 0.2 sec after a correct response, reaching it after about 0.5 sec and moved back again after about 2 sec (all times measured from response onset). The two response levers were extended again 7 sec after response onset. Usually the cat responded within 4 sec after lever extension. Trial sequences were automated by an electronic module system (Zak, Simbach) and were masked by white noise. Extra-cellular recordings were performed with tungsten-in-glass electrodes 4, monitored by earphones, and stored on magnetic tape (Hewlett-Packard 3960) together with E M G or E O G potentials and with D C signals of the trial sequences (left or right response, reinforcement and lever extension). The neurobehavioral sequence required a number of controls which were implemented in two ways. First, recordings were made from a control cat which had learned to press only one of the two levers in order to obtain reward. Second, the same units were recorded in the 3 experimental animals both during task performance and while the levers were removed. By recording from regions both anterior and posterior to the cruciate sulcus, it was possible to control for the influence of motor involvement; by omitting or delaying reward after a response, possible neuronal changes (e.g., related to expectation) could be detected. In the experimental group, possible changes in neuronal activity during feeding per se and during presentation of visual, acoustic and olfactory stimuli were checked. A histological analysis showed that all tracks were confined to the association areas. Results of this on-going investigation are based on evaluation of 78 neurons. All of these were held for the duration of at least 11 trials so that it was possible to record activity both during rewarded ( ÷ ) and unrewarded (--) trials. Usually average
~31
~3
-10 1 2 3 4 5 6 7 time (sec )
I ......... -10 1234567 time (sec)
Fig. 2, Frequency-time histograms of averaged changes of activity level in 26 reward related neurons in prefrontal, PMSA and PCA regions. Left: reward related example. Right: expectancy (and reward) related example. Dotted line: baseline activity; solid line: rewarded trial; broken line: unrewarded trial; sd: standard deviation units; binwidth: 0.5 sec.
187 values of firing rates and distributions during left (L) and right (R) trials were compared, as well as averaged activity during all 6 combinations of L-k, L--, R + , R--. Unit activity during trials is expressed in standard deviation units of baseline rates. The method used revealed several relationships between unit firing in the 3 association areas and behavioral activities during performance of the learning task. A notable and consistent result was a change in activity during the reinforcement period in about a third of the units analyzed. Fig. 2 illustrates these results. Changes were seen in 7 (35 ~) of the prefrontal, 9 (32 ~) of the PMSA and 10 (26 ~) units in the PCA region; in each of the last two regions one unit was also found that was inhibited during the reward period. In all 3 areas some neurons showed heightened activity immediately after the animal's response. Their activity was interpreted to be related to reward expectancy, as their discharge rate usually decreased throughout the reward period (2.5 sec) in rewarded trials. At least one neuron in each area increased activity during the reinforcement period in + trials, while in - - trials it was excited beginning 2-3 sec after the response. Our single unit recordings in association areas of the cat have therefore revealed statistically significant differences between correct and incorrect responses, whereas electroencephalographic recordings in subcortical structures have not 7. In addition one (5 ~o) of the prefrontal and 13 (40 ~) of the PCA units were correlated with arm movements and 4 (16 ~) of the PMSA cells changed activity during eye movements. Unit responses observed after sensory stimulation showed long latencies (40-120 msec), were tonic and usually inconsistent (e.g., excitatory to some, inhibitory or unresponsive to other stimuli). About every third neuron seemed to be unresponsive to any task component, and the rest showed complex relationships which we cannot describe here in detail. As the neuronal activity usually either did not change or changed less when feeding the cats in times unrelated to the trial sequence, we interpret the units' firing during the reward period to be reward or reinforcement related. In this respect such units are functionally different from basal ganglia neurons which also change activity during consummatory periods, but which seem to be related to sensory rather than to motivational properties of ingestion~. Changes related to reward expectancy have been reported using cortical steady potential shifts, but not in corresponding multi-unit recordings TM. Our data, however, present evidence for their visibility on the single-neuron level. Nevertheless the specific functions of such a high percentage of reward related neurons in all 3 association areas investigated remains to be determined. The similarity of reward-specific activity in such neurons in all 3 areas does not allow a functional differentiation among them, confirming related evoked potential researcha, 12. Evidence for a possible functional dissociation of neurons within the association areas of the cat cortex must wait further investigations. The authors would like to express their gratitude to Prof. R. B. Freeman, Jr. both for continued support and for many helpful comments on this paper. They also thank Dr. T. Tobias for his valuable criticisms on this manuscript.
188 I Albe-Fessard, D. and Besson, J. M., Convergent thalamic and cortical projections - - the nonspecific system. In A. Iggo (Ed.), Handbook of Sensory Physiology, Vol. 2, Somatosensory Systeln, Springer, Berlin, 1973, pp. 489-560. 2 Iversen, S. D., Brain lesions and memory in animals. In J. A. Deutsch (Ed.), The Physiological Basis of Memory, Academic Press, New York, 1973, pp. 305-364. 3 John, E. R., Bartlett, F., Shimokochi, M. and Kleinman, D., Neural readout from memory, J. Neurophysiol., 36 (1973) 893--924. 4 Levick, W. R., Another tungsten microelectrode. Med. biol. Engng., 10 (1972) 510-515. 5 Lidsky, T. I., Buchwald, N. A., Hull, C. D. and Levine, M. S., Pallidal and entopeduncular single unit activity in cats during drinking, Electroenceph. clin. Neurophysiol., 39 (1975) 79-84. 6 Markowitsch, H. J. and Pritzel, M., Learning and the prefrontal cortex of the cat: anatomicobehavioral interrelations, Physiol. PsychoL, in press. 7 Moise, S. L., Jr. and Costin, A., Hippocampal, hypothalamic and lateral geniculate activity during visual discrimination in the monkey, Physiol. Behav., 12 (1974) 835-841. 8 Noda, H., Freeman, R. B., Jr., Gies, B. and Creutzfeldt, O., Neuronal responses in the visual cortex of awake cats to stationary and moving targets, Exp. Brain Res., 12 (1971) 389-405. 9 Rolls, E. T., Reward and the Brain, Pergamon, Oxford, 1975, pp. 115. 10 Sheafor, P. J. and Rowland, V., Dissociation of cortical steady potential shifts from mass action potentials in cats awaiting food rewards, Physiol. PsychoL, 2 (1974) 471480. 11 Stellar, E. and Corbit, J. D., Neural control of motivated behavior, Neurosc. Res. Prog. Bull., 11 (1973) 296-410. 12 Thompson, R. F., Johnson, R. H. and Hoopes, J. J., Organization of auditory, somatic sensory and visual projection to association fields of cerebral cortex in the cat. J. Neurophysiol., 26 (1963) 343-364.
185
Reward related neurons in cat association cortex
HANS J. MARKOWITSCH and MONIKA PRITZEL Department of Psychology, University of Konstanz, Postbox 7733, D-7750 Konstanz ( G.F.R.)
(Accepted April 6th, 1976)
Electrophysiological studies have found several regions within the cat's brain which reveal similar response patterns that are specific to sensory stimulation in different modalities. Such regions have been named polysensory or association areas 1,12. Lesions of these areas lead to deficits in acquisition and/or retention of complex learning tasks2, 6. Within the frame of a learning paradigm this communication reports data disclosing reward related changes of single unit activity in association areas of the cat. Animals were first trained to perform a spatial reversal task modified so as to permit spatial responses while the cat's head was held stationary for the purpose of single-cell recording. Therefore, instead of using WGTAs or mazes, the cats were shaped to sit in a rectangular wooden box and to depress one of two retractable levers which were positioned within reach of the cat's paws and separated laterally by 12 cm. Responses to one side were rewarded (non-correction procedure) until the cat pressed the same lever 10 consecutive times within 50 trials. Then 1-3 reversals of this task were added, whereby the cat was required to press the lever on the other side for 10 consecutive trials. Due to the restriction of the cat's movements to lever pressing, the abundance of kinesthetic cues which can be assumed to be present in more conventional investigations of spatial reversal performance in cats may have been lacking in this situation. Perhaps for this reason the cats required 4-6 months to reach criterion. Three cats of the experimental group have so far been trained and operated. Openings in the skull were made over the prefrontal region and over the middle suprasylvian gyrus (PMSA lz) of the same hemisphere. A control cat was prepared with two trephine holes, one positioned over the prefrontal and one over the caudal portion of the pericruciate region (PCA12). Plastic chambers, insulated copper wires for EOG and E M G recordings, and two horizontal tubes to hold the cat's head rigidly during recording were fixed to the skull with dental acrylics. Though restricted during recording sessions, the cats usually performed the learned task in the same manner and to the same criterion as before the operation, where the cat's head was unrestrained. Fig. 1 shows an example of a trial sequence. The cat initiated a trial by pressing one of the response levers. The arm of a feeder moved to the cat's mouth
186 Lp
Lp
/T/
I IT/
0
1
2
3
4
5
6 7
sec.
Fig. 1. Example of a trial sequence. ITI: variable intertrial interval (depending on the cat's reaction); Lp: lever press (after response to left or right lever both retract for 7 secs); R: reinforcement period (if response was correct); Le: simultaneous extension of both levers (next response is possible). The time scale shows shortest duration of one trial (ITI period is omitted). 0.2 sec after a correct response, reaching it after about 0.5 sec and moved back again after about 2 sec (all times measured from response onset). The two response levers were extended again 7 sec after response onset. Usually the cat responded within 4 sec after lever extension. Trial sequences were automated by an electronic module system (Zak, Simbach) and were masked by white noise. Extra-cellular recordings were performed with tungsten-in-glass electrodes 4, monitored by earphones, and stored on magnetic tape (Hewlett-Packard 3960) together with E M G or E O G potentials and with D C signals of the trial sequences (left or right response, reinforcement and lever extension). The neurobehavioral sequence required a number of controls which were implemented in two ways. First, recordings were made from a control cat which had learned to press only one of the two levers in order to obtain reward. Second, the same units were recorded in the 3 experimental animals both during task performance and while the levers were removed. By recording from regions both anterior and posterior to the cruciate sulcus, it was possible to control for the influence of motor involvement; by omitting or delaying reward after a response, possible neuronal changes (e.g., related to expectation) could be detected. In the experimental group, possible changes in neuronal activity during feeding per se and during presentation of visual, acoustic and olfactory stimuli were checked. A histological analysis showed that all tracks were confined to the association areas. Results of this on-going investigation are based on evaluation of 78 neurons. All of these were held for the duration of at least 11 trials so that it was possible to record activity both during rewarded ( ÷ ) and unrewarded (--) trials. Usually average
~31
~3
-10 1 2 3 4 5 6 7 time (sec )
I ......... -10 1234567 time (sec)
Fig. 2, Frequency-time histograms of averaged changes of activity level in 26 reward related neurons in prefrontal, PMSA and PCA regions. Left: reward related example. Right: expectancy (and reward) related example. Dotted line: baseline activity; solid line: rewarded trial; broken line: unrewarded trial; sd: standard deviation units; binwidth: 0.5 sec.
187 values of firing rates and distributions during left (L) and right (R) trials were compared, as well as averaged activity during all 6 combinations of L-k, L--, R + , R--. Unit activity during trials is expressed in standard deviation units of baseline rates. The method used revealed several relationships between unit firing in the 3 association areas and behavioral activities during performance of the learning task. A notable and consistent result was a change in activity during the reinforcement period in about a third of the units analyzed. Fig. 2 illustrates these results. Changes were seen in 7 (35 ~) of the prefrontal, 9 (32 ~) of the PMSA and 10 (26 ~) units in the PCA region; in each of the last two regions one unit was also found that was inhibited during the reward period. In all 3 areas some neurons showed heightened activity immediately after the animal's response. Their activity was interpreted to be related to reward expectancy, as their discharge rate usually decreased throughout the reward period (2.5 sec) in rewarded trials. At least one neuron in each area increased activity during the reinforcement period in + trials, while in - - trials it was excited beginning 2-3 sec after the response. Our single unit recordings in association areas of the cat have therefore revealed statistically significant differences between correct and incorrect responses, whereas electroencephalographic recordings in subcortical structures have not 7. In addition one (5 ~o) of the prefrontal and 13 (40 ~) of the PCA units were correlated with arm movements and 4 (16 ~) of the PMSA cells changed activity during eye movements. Unit responses observed after sensory stimulation showed long latencies (40-120 msec), were tonic and usually inconsistent (e.g., excitatory to some, inhibitory or unresponsive to other stimuli). About every third neuron seemed to be unresponsive to any task component, and the rest showed complex relationships which we cannot describe here in detail. As the neuronal activity usually either did not change or changed less when feeding the cats in times unrelated to the trial sequence, we interpret the units' firing during the reward period to be reward or reinforcement related. In this respect such units are functionally different from basal ganglia neurons which also change activity during consummatory periods, but which seem to be related to sensory rather than to motivational properties of ingestion~. Changes related to reward expectancy have been reported using cortical steady potential shifts, but not in corresponding multi-unit recordings TM. Our data, however, present evidence for their visibility on the single-neuron level. Nevertheless the specific functions of such a high percentage of reward related neurons in all 3 association areas investigated remains to be determined. The similarity of reward-specific activity in such neurons in all 3 areas does not allow a functional differentiation among them, confirming related evoked potential researcha, 12. Evidence for a possible functional dissociation of neurons within the association areas of the cat cortex must wait further investigations. The authors would like to express their gratitude to Prof. R. B. Freeman, Jr. both for continued support and for many helpful comments on this paper. They also thank Dr. T. Tobias for his valuable criticisms on this manuscript.
188 I Albe-Fessard, D. and Besson, J. M., Convergent thalamic and cortical projections - - the nonspecific system. In A. Iggo (Ed.), Handbook of Sensory Physiology, Vol. 2, Somatosensory Systeln, Springer, Berlin, 1973, pp. 489-560. 2 Iversen, S. D., Brain lesions and memory in animals. In J. A. Deutsch (Ed.), The Physiological Basis of Memory, Academic Press, New York, 1973, pp. 305-364. 3 John, E. R., Bartlett, F., Shimokochi, M. and Kleinman, D., Neural readout from memory, J. Neurophysiol., 36 (1973) 893--924. 4 Levick, W. R., Another tungsten microelectrode. Med. biol. Engng., 10 (1972) 510-515. 5 Lidsky, T. I., Buchwald, N. A., Hull, C. D. and Levine, M. S., Pallidal and entopeduncular single unit activity in cats during drinking, Electroenceph. clin. Neurophysiol., 39 (1975) 79-84. 6 Markowitsch, H. J. and Pritzel, M., Learning and the prefrontal cortex of the cat: anatomicobehavioral interrelations, Physiol. PsychoL, in press. 7 Moise, S. L., Jr. and Costin, A., Hippocampal, hypothalamic and lateral geniculate activity during visual discrimination in the monkey, Physiol. Behav., 12 (1974) 835-841. 8 Noda, H., Freeman, R. B., Jr., Gies, B. and Creutzfeldt, O., Neuronal responses in the visual cortex of awake cats to stationary and moving targets, Exp. Brain Res., 12 (1971) 389-405. 9 Rolls, E. T., Reward and the Brain, Pergamon, Oxford, 1975, pp. 115. 10 Sheafor, P. J. and Rowland, V., Dissociation of cortical steady potential shifts from mass action potentials in cats awaiting food rewards, Physiol. PsychoL, 2 (1974) 471480. 11 Stellar, E. and Corbit, J. D., Neural control of motivated behavior, Neurosc. Res. Prog. Bull., 11 (1973) 296-410. 12 Thompson, R. F., Johnson, R. H. and Hoopes, J. J., Organization of auditory, somatic sensory and visual projection to association fields of cerebral cortex in the cat. J. Neurophysiol., 26 (1963) 343-364.
