Action, time and the basal ganglia.

Downloaded from rstb.royalsocietypublishing.org on April 13, 2014

Action, time and the basal ganglia Henry H. Yin Phil. Trans. R. Soc. B 2014 369, 20120473, published 20 January 2014

References

This article cites 52 articles, 12 of which can be accessed free

Subject collections

Articles on similar topics can be found in the following collections

http://rstb.royalsocietypublishing.org/content/369/1637/20120473.full.html#ref-list-1

behaviour (519 articles) cognition (332 articles) computational biology (44 articles) neuroscience (451 articles) physiology (62 articles) theoretical biology (61 articles)

Email alerting service

Receive free email alerts when new articles cite this article - sign up in the box at the top right-hand corner of the article or click here

To subscribe to Phil. Trans. R. Soc. B go to: http://rstb.royalsocietypublishing.org/subscriptions


Action, time and the basal ganglia Henry H. Yin1,2 1

Department of Psychology and Neuroscience, and 2Department of Neurobiology, Center for Cognitive Neuroscience, Duke University, PO Box 91050, Durham, NC 27708, USA

rstb.royalsocietypublishing.org

Review Cite this article: Yin HH. 2014 Action, time and the basal ganglia. Phil. Trans. R. Soc. B 369: 20120473. http://dx.doi.org/10.1098/rstb.2012.0473 One contribution of 14 to a Theme Issue ‘Timing in neurobiological processes: from genes to behaviour’. Subject Areas: behaviour, cognition, neuroscience, physiology, computational biology, theoretical biology Keywords: dopamine, basal ganglia, bradykinesia, action, substantia nigra, striatum Author for correspondence: Henry H. Yin e-mail: [email protected]

The ability to control the speed of movement is compromised in neurological disorders involving the basal ganglia, a set of subcortical cerebral nuclei that receive prominent dopaminergic projections from the midbrain. For example, bradykinesia, slowness of movement, is a major symptom of Parkinson’s disease, whereas rapid tics are observed in patients with Tourette syndrome. Recent experimental work has also implicated dopamine (DA) and the basal ganglia in action timing. Here, I advance the hypothesis that the basal ganglia control the rate of change in kinaesthetic perceptual variables. In particular, the sensorimotor cortico-basal ganglia network implements a feedback circuit for the control of movement velocity. By modulating activity in this network, DA can change the gain of velocity reference signals. The lack of DA thus reduces the output of the velocity control system which specifies the rate of change in body configurations, slowing the transition from one body configuration to another.

1. Introduction Although there is no specialized sensory organ for time, our perception of time nevertheless depends on the rate of change in different sensory modalities. In the absence of sensory input, e.g. under anaesthesia, the sense of time is often lost or impaired. That timing depends on the rate of change in perceptual variables may seem trivially true, but the implications of this proposition are seldom acknowledged. Here, I shall explain some of these implications in the kinaesthetic domain by focusing on the timing of actions. When we move, our body changes its posture, the configurations of different body parts. Not only can we maintain specific body configurations, we can also control how quickly they change. Although this aspect of behaviour is often neglected, it becomes more conspicuous in neurological disorders, which commonly feature deficits in action timing. In Parkinson’s disease, for instance, movement is often slowed dramatically, a condition known as bradykinesia. By contrast, Tourette syndrome is associated with symptoms like involuntary and rapid tics. The hypokinetic or hyperkinetic symptoms therefore suggest that a fundamental action timing mechanism is impaired in these disorders. The deficits of action timing in both Parkinson’s disease and Tourette syndrome implicate dopaminergic projections to the basal ganglia, a set of subcortical nuclei in the cerebrum (figure 1a) critical for the learning and initiation of actions [1 –5]. In Parkinson’s disease, the dopamine (DA) neurons die, resulting in reduced DA in the basal ganglia. Effective treatments for bradykinesia include DA replacement therapy with L-DOPA, a DA precursor. By contrast, Tourette syndrome is thought to result from excessive dopaminergic signalling in the dorsal striatum, and treatments include antagonists of DA receptors like haloperidol [6]. Slow movements are therefore associated with reduced dopaminergic signalling; fast movements with excessive dopaminergic signalling.

2. Deficits in timing following dopamine depletion Clinical symptoms can only provide clues to the underlying mechanisms. To assess the role of DA in action timing, we experimentally manipulated dopaminergic signalling in mice and used the duration of lever presses as a measure of action timing. We trained mice on an operant task, in which they

& 2014 The Author(s) Published by the Royal Society. All rights reserved.


(a) cerebral cortex

pallidum diencephalon

nomifensine in normal mice

(b)

(c) L-DOPA/carbidopa rescue 300 mean press duration (ms)

mean press duration (ms)

300 200 100 0

10

depleted saline depleted 10/10 mg kg–1 depleted 50/50 mg kg–1

200 100 0

e

–1

n ali

s

excitatory inhibitory modulatory

tectum/brain stem

g

m

–1

kg

20

g

kg

m

(d )

Figure 1. DA and action timing. (a) Schematic of the basic cortico-basal ganglia circuit. Cortical projection neurons, the pyramidal cells in layer 5, are glutamatergic and excitatory. Striatal and pallidal projection neurons are by contrast GABAergic and usually inhibitory. The modulatory dopaminergic projections target most components of the cortico-basal ganglia network, but by far the largest proportion target the striatum. (b) Intraperitoneal injection of nomifensine, a blocker of DA transporter, dose-dependently reduced duration of lever presses in normal mice (n ¼ 5, p , 0.05). All mice were food deprived and maintained at approximately 85% of normal body weight. They were trained on a continuous reinforcement schedule for 4 days (each lever press earned a food pellet). They were then trained on a FI-60 schedule for at least 5 days before testing (1 h daily sessions). (c) DA depletion in the sensorimotor striatum increased the duration of lever presses, but this increase in press duration can be rescued with intraperitoneal injections of L-DOPA/ carbidopa (n ¼ 5, p , 0.0001). (d ) A coronal section of the mouse brain with tyrosine hydroxylase staining showing selective depletion of DA in the lateral sensorimotor striatum after local 6-OHDA injections (20 mg ml21, 0.5 ml per side to target dopaminergic terminals bilaterally in the striatum). Note that the lesion is selective, showing depletion limited to the sensorimotor striatum. Stereotaxic coordinates relative to bregma in mm: þ0.5, ML + 3.0 and 23 in dorsolateral striatum). must press a lever for food pellets. First, we used nomifensine to inhibit DA uptake and increase DA level. We found that nomifensine dose-dependently reduced press duration (figure 1b). We then selectively induced DA depletion in sensorimotor or lateral striatum using 6-hydroxydopamine (6-OHDA), a toxin that kills DA neurons. DA depletion in the sensorimotor striatum reduced rate of lever pressing and increased press duration (figure 1c), in agreement with previous work [7] as well as clinical observations on

3. Movement as changes in body configuration A body configuration is the geometrical relationship of body parts to each another. It is more commonly called a posture, though there is a persistent misunderstanding of posture as essentially a static phenomenon that does not require the active engagement of the brain. But as shown by centuries of clinical observations, the control of posture or body configuration is an essential function of the nervous system, from the spinal cord to the brain. It is an active control process that requires the production of continuous neural outputs coordinated to counter environmental disturbances. When such control is impaired, a variety of neurological symptoms result [8]. In fact, impairments in posture and body configuration are among the most common and obvious symptoms of neurological disorders. Movement, then, is but the shifting from one body configuration to another [9]. Movement speed, or the rate of change in body configurations, can be controlled by the nervous system. This capacity I shall call ‘movement velocity control’. Some clarifications are needed to avoid any misunderstanding: (i) Movement velocity control refers only to the rate of change in body configurations, though velocity control also applies to perceptions in other modalities, e.g. visual or auditory configurations. (ii) A distinction should also be made between movement velocity and actual velocity of the body in space, as in locomotion. Riding a stationary bike, for example, requires high movement velocity, even though the bike does not move. (iii) Although movement velocity can be directly controlled, it is not necessarily controlled at all times. It can also be used in order to achieve the reference conditions of higher systems, in which case the velocity simply varies as required by the higher level (see §5 for discussion of how this is possible). (iv) The perceptual signal representing velocity cannot be equated with the rate of change in muscle length, as detected by first-order sensors such as muscle spindles. With very simple body geometry, stretch receptors and movement velocity detectors may be equivalent. But in most cases, the rate of change in body configuration requires a higher order representation of a collection of lower order kinaesthetic inputs. However it is implemented, for our purposes it is sufficient to assume only that movement velocity can be detected and controlled by the nervous system.

Phil. Trans. R. Soc. B 369: 20120473

DA cell groups

2


striatum

bradykinesia in Parkinson’s patients. This deficit can also be rescued by DA replacement treatment with L-DOPA (figure 1c). These results suggest that dopaminergic innervation, particularly that of the sensorimotor striatum, is critical for the timing of actions. Action duration reflects the rate of change in kinaesthetic variables. Following DA depletion, it appears that movement velocity, i.e. rate of transition from one body configuration to another, cannot exceed a certain level. These observations raise the question of how DA and the basal ganglia can implement action timing. Before attempting to answer this question, however, it helps to define action timing more explicitly.


4. Closed-loop control

The nervous system is hierarchically organized, implementing multiple levels of control [9–11]. Different negative feedback control systems can be arranged hierarchically, the output of a higher level system serving as a reference signal that specifies the input that the lower control system must obtain. This basic principle distinguishes the hierarchical relationship proposed here from other hierarchical models in neuroscience and psychology which are all open loop and thus incapable of controlling anything [12,13]. Since each control system is defined by its controlled variable, different levels of the hierarchy control different variables. To control the rate of change in body configuration, it is necessary first to control body configuration, which in turn requires the control of muscle length and tension.

6. Bradykinesia Deficits in action timing can be understood as a result of altered signalling in specific parts of the hierarchy just described. In bradykinesia, for example, it is as if the configuration reference signal were low-pass filtered, changing very slowly. The most extreme example is akinesia, in which the configuration is simply fixed. It is important to note that the lower body configuration control system is still functioning, still producing resistance to any disturbance to body configuration, regardless of how slowly the velocity reference signal changes, if at all. Such resistance is observed, for example, in the so-called ‘lead pipe rigidity’, found in Parkinson’s patients.

Phil. Trans. R. Soc. B 369: 20120473

5. Hierarchical control

3


Control systems, as defined here, bring the value of a particular input variable closer to a desired value, despite environmental disturbances. For example, room temperature can be controlled by a thermostat, just as body temperature is controlled by the nervous system, despite fluctuations in the outside temperature. For this to be possible, it is necessary to sense the value of the input variable to be controlled, and to have an internal representation of the desired value of the variable. The sensor can send a signal representing the value of the controlled variable to a comparator, which computes the difference (error signal) between the input and a reference signal representing the desired value of the controlled variable. The error signal is transformed into the appropriate output. A velocity control system, then, can detect how rapidly its input signal is changing, compare the sensed speed with a reference speed, and adjust the output until the sensed speed matches the reference signal. For this to be possible, the output must affect sensed velocity through a feedback function which expresses the input signal as a mathematical function of the output signal. The loop is thus closed in the environment. Negative feedback reduces the error signal so that the desired value of the controlled variable can be reached. This organization, it should be noted, is the only one found to be effective for controlling the value of a given variable despite environmental disturbances. In practice, no engineer would rely on any other method, e.g. feed-forward mechanisms, for effective control. One example of velocity control is cruise control in cars, where a negative feedback mechanism is used. Yet how such control is achieved by the brain is different. In biological control systems, the reference signal is intrinsic to the organism, not accessible to external users as it is in manmade control systems like the thermostat. The organism is self-adaptive, able to tune its own parameters. The userspecified ‘input’ is eliminated in natural control systems. Moreover, for any moderately complex body geometry, first-order sensors cannot adequately sense movement velocity. A simple negative feedback control system is therefore not sufficient. Rather than sending its output to the muscles directly, the velocity control system can only exert indirect effects on the muscles. Instead, a hierarchy of control systems is required.

In other words, it is necessary to use additional lower level control systems to exert effects on the environment. As the controlled variable must be sensed, each level of the hierarchy is defined by its sensory input (figure 2). Movement velocity control is largely independent of the distal senses such as vision and hearing. The primary sensory modality required is kinaesthetic. The relevant sensory inputs are ultimately derived from signals from proximal sensors in the body. Such signals, like all sensory signals, are transformed by successive levels of the hierarchy into representations of higher order variables such as movement velocity. It is this higher order input variable that is fed into the velocity control system and compared with the velocity reference signal. Still higher levels can also use velocity control for their purposes by sending their reference signals that are proportional to their own error signals. For example, when tracking a moving target, the proximity between the hand and the target is an error signal in a higher level that controls the relationship between the two. Such a relationship control system requires visual inputs. Closing the distance reduces the relationship error, which can be transformed into a reference signal for the velocity control system. But this can be achieved in different ways, such as reaching with the arm or walking towards the moving object, which involve transitions in different body configurations. Either way, it is possible to control the proximity between hand and object by varying the reference signal for velocity. Proportional to the velocity error signal is the output of the velocity control system, which must send reference signals to a lower control system for body configuration or posture. A body configuration can be conceived as a collection of joint angles. At each joint, antagonistic muscles can pull in opposite directions, their relative lengths determining the joint angle. The simplest configuration is that of one joint, but for normal movements multiple joints are involved. The movement velocity control system is involved in transitioning from one body configuration to another. It determines how much time it takes to change from one configuration to another and how long a particular configuration is maintained. It can reach the desired velocity by varying the reference signal to the body configuration system. The transition from one configuration to another is normally continuous and smooth as the reference signal changes. I propose that a velocity error signal is transformed into the rate of change in the configuration reference signal. This transformation is similar to the mathematical operation of leaky integration [14].


comparator velocity input

velocity error

input

output (with leaky integration) configuration reference comparator

input

kinaesthetic signals

configuration error output

muscle length control

Figure 2. Hierarchical implementation of velocity control. The hierarchical relationship between velocity control and body configuration control situated immediately below in the hierarchy. The error signal from the velocity control system is converted into a reference signal for the body configuration control system. With integration in the output function of the velocity control system, the velocity error signal is proportional to the rate of change in the body configuration reference. The higher the velocity reference, the faster the change in body configuration. In principle, bradykinesia can be produced by increased sensitivity to velocity input, reduced velocity reference signal or altered velocity output. Exaggerated velocity perception can be produced by an increase in input gain, so that velocity is perceived as faster than it actually is. That is, given a particular velocity, the velocity sensor sends a larger than normal signal to the comparator. With the same reference signal, the error signal in the velocity system will be reduced, and less output (lower rate of change in configuration reference) is produced. Should this be the case, small variations in velocity could produce large effects on behavioural output, which would show high-frequency noise, high frequencies having larger effects on sensed velocity. If, on the other hand, bradykinesia is produced by reduced gain in the function that transforms the error signals from higher levels into the velocity reference signal, then a smaller velocity reference signal is produced. Thus, velocity control is functioning normally, though the reference signal it receives is always low (which is also the case when one deliberately tries to move slowly). The maximum value the reference signal attains during a movement is reduced. With an integrator in its output function, the output of the velocity control system—the reference signal for configuration/position control will change slowly. Finally, the problem may be found in the transformation of a normal velocity error into a reference signal for configuration control, in the generation of the time integral. For example, if the integrator becomes too leaky, the configuration reference signal can also change more slowly. The above possibilities are not mutually exclusive. From clinical observations alone, it is difficult to rule out any one of them [15]. Selective manipulation of the relevant neural circuits will be necessary to determine the specific mechanisms underlying bradykinesia. The outlining of these possibilities above will serve to illustrate the reasoning used in analysing properties of hierarchical control systems.

4

DA in the basal ganglia appears to play a critical role in the control of movement velocity. As many comprehensive reviews on the organization of the basal ganglia are available [16 –18], only the most relevant features are outlined here. The basal ganglia are a group of subcortical nuclei with highly conserved circuitry [19– 21]. As shown in figure 1, unlike the cerebral cortex, which contains excitatory glutamatergic projection neurons, the basal ganglia contain g-aminobutyric acid (GABA) projection neurons [18]. The main input nucleus is the striatum, a large and heterogeneous region including the dorsal striatum (caudate and putamen in primates) and ventral striatum (nucleus accumbens). The lateral septum and parts of the central amygdala are also classified as striatal regions on account of their GABAergic projection neurons. The main output nucleus is the pallidum, which includes entopeduncular nucleus in rodents or globus pallidus internus in primates, and substantia nigra pars reticulata [18]. These ‘pallidal’ structures send projections to the tectum, brainstem and thalamus. The striatum is the major target of dopaminergic projections, with the highest density of DA receptors in the brain. It also receives extensive and topographically organized glutamatergic projections from the cortex and thalamus [22– 24]. The medium spiny striatal projection neurons are the primary targets of the glutamatergic and dopaminergic projections. Rather than depolarizing or hyperpolarizing the target neurons, the activation of G protein-coupled DA receptors modulates the excitatory effect of glutamate, depending on the subtype of DA receptors expressed on the target neuron [25,26]. An important feature of the basal ganglia is the existence of two neuronal populations in the striatum giving rise to the so-called direct and indirect pathways, one projecting to the substantia nigra pars reticulata (SNr) and the internal segment of the globus pallidus (GPi) and the other projecting to the external segment of the globus pallidus (GPe), which in turn projects to the SNr. These two populations differ in the type of DA receptor expressed: the direct or striatonigral pathway expresses D1-type DA receptors whereas the indirect or striatopallidal pathway expresses D2-type receptors [16,27]. These two pathways can thus exert opposite effects on the SNr neurons [28]. The direct pathway has a net inhibitory effect, while the indirect pathway has a net excitatory effect on the SNr outputs [28]. Because the output of the SNr is inhibitory, the activation of the direct pathway is expected to disinhibit downstream structures, whereas that of the indirect pathway produces the opposite effect [28,29].

8. Neural activity related to action duration To study the role of the basal ganglia in action timing, we developed an operant duration differentiation task in mice [30 –34]. On each trial, to earn a piece of food reward, the mouse must produce an action of a minimum duration [32,33]. The lever is transiently retracted after release, and a reward is delivered if the press duration exceeded the criterion duration. No cue tells the animal whether the action duration is long enough. Only after lever retraction is the outcome revealed, and the presentation of the reward is the only way that the animal learns about the efficacy of its lever

Phil. Trans. R. Soc. B 369: 20120473

configuration input

7. Dopamine and the basal ganglia


velocity reference


9. Targets of the basal ganglia The outputs of the nigral GABAergic neurons can reach the tectum (superior colliculus), the ventral lateral and ventral anterior thalamus, the pedunculopontine nucleus and the reticular formation [16,17,41,42]. If basal ganglia output represents a configuration reference signal, then among the major targets of these projections there should be control systems for body configurations. The rate of change in this reference signal will then determine how quickly one body configuration changes to another. It is possible that different targets in the diencephalon and brainstem mediate different aspects of body configuration control. For example, locomotion requires relatively stereotyped transitions like alternation of limb flexors and extensors, whereas changes in posture require mainly transitions in the proximal musculature. The SNr sends strong projections to the mesopontine tegmentum, a brainstem region critical for initiation of locomotion and regulation of posture [43]. The lateral SNr projects to the pedunculopontine nucleus, whereas the medial SNr projects to the mesencephalic locomotor region. They can be the targets for the pair of opponent nigral output reference signals discussed above. Whether these structures implement the body configuration control systems remains to be determined. The nuclei in the mesopontine tegmentum in turn project to the reticulospinal system, the major motor pathway in all vertebrates [19,44]. Axons from the reticular formation in the brainstem reach motor neurons innervating different muscles all over the

5

Phil. Trans. R. Soc. B 369: 20120473

The same reasoning can be applied to the antagonistic body configuration systems, which are hierarchically higher than the spinal reciprocal inhibition circuits. A body configuration control system can simultaneously send signals to multiple muscles, adjusting their lengths via the activation of g-motor neurons. The higher order perceptual signal could be a weighted sum of signals from multiple muscle length detectors as well as other sensors for joint angles. Thus, more global opponent signals can be found at a higher level, controlling the lengths of groups of muscles, instead of the local opponent signals sent to two muscles doing work in opposite directions with respect to a single joint. For example, to pull the door open, a set of muscles (e.g. biceps) does the pulling while a different set of muscles (leg extensors) pushes against the ground. The value of a controlled variable like muscle tone can vary from zero to some maximum, and a viable operating point would be in the middle of this range, to permit variations above and below the average value. If the tone is too low, then there is no room for the signal to decrease. If it is too high, there is no room for any increase. Likewise, the high tonic firing rate of the GABAergic output neurons in the SNr may also allow neural activity in this basal ganglia output nucleus to increase and decrease, generating a pair of opponent signals [39,40]. It is hypothesized that, for any body configuration, opponent reference signals may be needed. In fact, at lower levels, the reference signals should come in pairs. But at higher levels, the velocity reference signal is singular, and it is only in the transformation from velocity reference to position/ configuration that a pair of reference signals is generated.


pressing (figure 3a). In other words, the reward delivery is contingent upon press duration, rather than press rate, as in traditional operant conditioning [32,35,36]. Press duration can be used to tag the neural activity generating the action itself [33,35]. The body configuration reference signal reaches and stays at a steady-state value as long as the lever is held down. With the release of the lever, the output of the velocity control system changes the reference signal for the configuration control system again. To obtain reward, the instrumental contingency requires the velocity reference to be very low for the duration of the action. Using temporal differentiation, we studied the neural activity of GABAergic output neurons in the SNr and of the nearby dopaminergic neurons in the pars compacta [35]. We found two major neural activity patterns in relation to the lever press action in the recorded neurons: in one type of neurons (‘action-on’) there was a sustained increase in firing rate during the action, and in the other population (‘action-off’) there was a sustained pause in firing rate during the press (figure 3). These two types are found in both GABAergic and dopaminergic neurons, though it is not clear how these types relate to previously identified heterogeneous neuronal populations in the nigra [37]. As long as the lever is held down, the reference signal for the velocity control system should be close to zero, while the reference signal for body configuration remains unchanged. Once a body configuration is reached, it is maintained for the duration of the action. With an integrator in the output function, the velocity control system can send a relatively constant reference signal for body configuration. The rate of change is low when a posture is maintained. The observed action-related neural activity can reflect either the velocity or body configuration reference signal. But we found two signals that are similar in amplitude but opposite in polarity. Although the pause of the ‘actionoff’ neurons can represent zero velocity during the action, the increased output of the ‘action-on’ neurons does not appear to represent the velocity reference signal. Nor does it seem plausible to send a high velocity reference signal and a zero velocity reference signal at the same time. Another possibility is that the opponent outputs reflect descending body configuration reference signals for antagonistic lower control systems. But why should there be a pair of signals, rather than a single reference signal? The most wellknown example of opponent signals in the nervous system is reciprocal innervation in the spinal motor neurons. The output of a given a-motor neuron innervates a particular muscle, but at the same time a branch of this output excites the Ia interneuron, which inhibits the a-motor neuron innervating the antagonist muscle. Thus, a pair of signals is sent to the effectors, one with a net excitatory effect on a muscle and the other with a net inhibitory effect on the antagonist muscle. This arrangement is needed because muscles can only pull. Antagonism between muscles is defined with respect to a particular joint: e.g. the biceps will pull in one direction, reducing the joint angle; whereas the triceps will pull in the opposite direction, increasing the joint angle. To reduce the angle of the elbow joint, it is not only important for the biceps to contract but also for the triceps to relax. At rest, there is a balance of activity in agonist and antagonist motor neurons. Simultaneous contraction of these muscles increases net angular stiffness or impedance at a joint [38].


(a)

6

release

press


criterion duration rewarded trial

unrewarded trial

no reward

Phil. Trans. R. Soc. B 369: 20120473

(b)

reward

(c) average waveforms DA GABA

spikes s−1

(d )

80 40 0 –1

0 1 time from press start (s)

20 10 0 2 –1

0 1 time from press start (s)

2

Figure 3. Opponent outputs from the basal ganglia. (a) Illustration of the duration differentiation task [33,35]. Mice must press a lever and hold it down for a minimum amount of time in order to earn a food pellet. Only after the lever is released is the trial outcome revealed (reward or no reward). (b) Illustration of the electrode implant into the substantia nigra. (c) The positioning of the electrode array allows simultaneous recording of activity from dopaminergic and GABAergic neurons, which can be distinguished on the basis of their action potential waveforms. (d ) Raster plots showing typical ‘action-on’ and ‘action-off’ GABAergic neurons. Each row is a single trial from a temporal differentiation session. Yellow markers indicate ‘lever start’; red markers indicate ‘lever end unrewarded’; green markers indicate ‘lever end rewarded’. The trials are sorted according to the duration of the lever press, starting with trials with the shortest action durations on top.

body. Interestingly, opponent activity is also observed in the reticulospinal tract [45,46]. Instead of agonist and antagonist muscles defined with respect to single joints, groups of muscles probably send kinaesthetic signals to the reticulospinal system [44]. Although the mesopontine tegmentum and the reticulospinal system are discussed here, the other targets of the basal ganglia output (including the tectum and ventral thalamus) may also contain body configuration control systems [17,41,47].

10. Lower levels for motor control Below the level of the body configuration control, there are at least two additional levels (figure 4). But, partly because of technical limitations, not much is known about the functional organization of the spinal cord where these lower levels are located. The reticulospinal pathway can send reference

signals representing desired muscle lengths for specific muscles, probably by activating spinal g-motor neurons and interneurons. The g-motor neurons can in turn send reference signals for muscle length, which are compared with the input from muscle spindles. Finally, the a-motor neurons serve as comparators that receive force reference signals from multiple sources. Their output is roughly proportional to the degree of muscle contraction. Whether there is co-activation of g- and a-motor neurons, as is often claimed [48], remains unclear, because in a closed-loop system signals cannot be considered seriatim as in verbal description. The muscle is the output function of the force control system, the lowest level in the motor hierarchy that closes the loop in the environment. Muscle tension is sensed by the Golgi tendon organs, which send Ib afferents representing the negative feedback for the force control system. In this connection, it is worth noting that increased stiffness in skeletal muscles is also observed in Parkinson’s patients, as a result


input function

comparator

output function DA

transition/velocity

striatonigral MSN

body configuration

PPN/MLR

g-motor neuron

muscle length

muscle spindle

muscle tension

a-motor neuron

reticular formation

intrafusal muscle

la afferent

extrafusal muscle

external environment

Figure 4. Possible neural implementation of the control hierarchy. A highly simplified illustration of the hierarchy for movement velocity control. Note that DA neurons also receive strong projections from the body configuration control system. The tectum and parts of the ventral thalamus should also belong to the body configuration control system. They might specialize in configurations of specific body parts, e.g. head and neck. PPN, pedunculopontine nucleus; MLR, mesencephalic locomotor region; MSN, medium spiny neuron. of increased muscle tone in both agonist and antagonist muscles. Such rigidity can also reduce movement velocity, because mechanical impedance is the ratio between velocity and force needed to achieve that velocity, which can be increased by simultaneous activation of agonist and antagonist muscles [38]. This can be achieved by a descending reference signal to both types of motor neurons, a mechanism that overrides reciprocal inhibition. Parkinsonian rigidity, however, is sometimes relieved when the patient is suspended in water, suggesting that it is manifested when posture must be defended against gravity [15]. Thus, rigidity may well be a compensatory reaction to the loss of normal control functions for body configurations.

11. The role of dopamine in the basal ganglia Based on the above outline of basal ganglia anatomy and review of experimental data, I propose that the sensorimotor cortico-basal ganglia network is responsible for controlling transitions between body configurations and plays a critical role in movement speed (figure 4). The role of the basal ganglia in movement speed has long been recognized [49–52], yet the mechanisms for hierarchical control have never been incorporated in a model to explain action timing. The perceptual input signal representing the rate of change in body configuration is the velocity feedback. It may be carried by the projections to the sensorimotor striatum from sensorimotor cortical regions, the intralaminar thalamus and the globus pallidus [53]. On the other hand, reference signals for movement velocity may be sent via the

cortical and thalamic projections to the sensorimotor or lateral striatum. In vivo recording in mice has also shown high correlation between movement velocity on a rotarod and the firing of the sensorimotor striatal and motor cortical neurons [54]. Results from DA depletion experiments discussed in §2 suggest that dopaminergic innervation of the sensorimotor striatum, in particular, is critical for movement velocity control (figure 1). Selective depletion in this region is sufficient to increase the duration of normal actions by reducing gain in the velocity control system. For higher levels in a hierarchy, the output function transforms a given error signal into a reference signal for the level below, i.e. an error in proximity between one’s body and the target in tracking movements is transformed into a reference signal for the desired velocity for the hand. As a neuromodulator, DA cannot produce significant synaptic currents directly to produce firing. But it can change the gain of other signals, in this case the reference signal for the rate of change, which is sent via the glutamatergic projections to the striatum [55]. Reduced DA may therefore result in a reduced velocity reference signal. This proposed mechanism is in agreement with the known function of DA in modulating striatal synaptic transmission [25,56,57]. As a result of DA depletion, the velocity reference signal is reduced. The control system receiving such a signal produces a low output entering the integrator, resulting in a slow rate of change in the body configuration reference signal. When the velocity reference signal is zero, then the configuration reference signal will no longer change, resulting in a lack of voluntary movement. This condition, akinesia, is seen in severe cases of Parkinsonism.

Phil. Trans. R. Soc. B 369: 20120473

joint angle

SNr


relationship reference

7


12. Conclusion and implications

8

Phil. Trans. R. Soc. B 369: 20120473

Observable behaviour consists of transitions between body configurations. It is hypothesized here that movement velocity—rate of change of body configurations—is controlled by the sensorimotor cortico-basal ganglia circuit. It does not follow that velocity control is the sole function of the basal ganglia. Nor does the present hypothesis exclude other roles of DA in behaviour. But it does suggest that there is a common computational function performed by the basal ganglia networks. A striking feature of the cortio-basal ganglia networks is the uniformity in the basic circuitry. All cortical areas have a similar structure, as do all striatal areas and all pallidal areas. Different networks are simply variations of a common motif: e.g. excitatory outputs of the cerebral cortex and thalamus, inhibitory outputs of the basal ganglia, and modulation of synaptic transmission and neuronal excitability by transmitters such as DA. This motif has been conserved in the evolution of vertebrates [19]. By comparison, variations in the expression of receptors and cytoarchitectonic features are minor. On the other hand, there is also much functional heterogeneity in the cortico-basal ganglia networks. Lesions to different regions produce different effects on behaviour [58]. The question is how we can reconcile this functional heterogeneity, the ‘localization of function’ known since antiquity, and the proposed common computational function of the basal ganglia. What is normally called function is related to the behavioural consequences of lesion or stimulation of a brain region, determined by the overall connectivity of the region. For example, visual deficits are a consequence of lesions to the primary visual cortex. But a distinction must be made between ‘function’ in this sense and computation, which is what happens to signals entering a neural circuit. The neural signal is firing rate, an analogue signal, in spite of the misleading all-or-none digital property of the generation of individual action potentials [59]. Computations using these signals can be described with a set of mathematical operations often used in analogue computing, such as addition, subtraction and integration. The difference, say, between the limbic cortico-basal ganglia network and the sensorimotor network lies not in the type of computation implemented by the neural circuits, but in the content of signals, i.e. what they represent. As control is always the control of input, the identity of the controlled variable depends solely on its input to the control system. The sensorimotor striatum, for example, receives higher order kinaesthetic and somesthetic signals from the primary sensorimotor cortices, whereas a limbic striatal region such as the nucleus accumbens shell receives a different set of more poorly defined inputs from areas like the basolateral amygdala, a very different cortical region [60].

Exactly what the controlled variables are for different brain circuits remains unclear. So far only speculations are possible based on the known anatomy and the behavioural deficits following lesions. Given the tremendous heterogeneity in the connectivity of different cortical, striatal and pallidal regions, elucidating the content of the variables will require extensive experimental work designed to test the specific controlled variables. Here, a hypothesis is advanced regarding the sensorimotor cortico-basal ganglia network, based on the DA depletion and in vivo recording data. But the basic circuit for transforming a velocity error signal into a configuration or position reference signal is still the same. A velocity reference signal enters the basal ganglia circuit which then generates opponent configuration reference signals that are sent to the level below. One possible neural implementation is suggested by the anatomy: the excitatory input to the striatum activates both the striatonigral and striatopallidal pathways [61], and the intrinsic organization of these two pathways transforms the uniform excitation into a pair of signals, one increasing and the other decreasing from a common mode signal. And finally, a time integral can be produced in the basal ganglia transforming a given magnitude of velocity error into a rate of change in the reference signal for body configuration control. If negative feedback control of velocity is accomplished by the sensorimotor cortico-basal ganglia circuit, then the other networks (e.g. limbic and associative) must be responsible for controlling different input variables [21,62]. In this light, different cortico-basal ganglia networks can be viewed as distinct higher level control systems for the control of the rate of change in different higher order perceptual variables, of which body configuration in only one example. The number of possible controlled variables is virtually unlimited, as different sensory variables can be combined to form higher order variables representing abstract categories (e.g. reward), especially in the cerebral cortex. In conclusion, control of the rate of change or transitions is perhaps the biological basis for our sense of time, even though time itself may not be a directly controlled variable. Much research has implicated the basal ganglia and DA in timing, but the underlying mechanisms remain unclear [63]. While the timing mechanism can be relatively independent of actual movements, it must still depend on the higher level of control of transitions between perceptual configurations which implicate the basal ganglia and modulation by DA (figure 4). I have attempted to incorporate the known anatomy of the nervous system in a model of the control hierarchy, linking velocity control with higher levels such as relationship control, and lower levels such as body configuration control, muscle length control and muscle force control. Clearly, much of the model remains speculative, as many relevant facts are still unknown. But even at this early stage some predictions can be made. First, glutamatergic input to the sensorimotor striatum and its dopaminergic modulation play a critical role in movement velocity. Manipulations of the glutamatergic signal alone, or of the firing of striatonigral projection neurons, should systematically affect movement velocity. Second, the basal ganglia outputs from the substantia nigra pars reticulata should contain reference signals for body configuration. As it is a general property of successful control systems that inputs match reference signals, activity in basal ganglia output nuclei should be


Increased dopaminergic signalling in the sensorimotor cortico-basal ganglia network produces the opposite effect. Even a small relationship error signal, which is not normally sufficient to produce any movement velocity reference, is now greatly amplified. Consequently, ‘uncontrollable’ rapid movements may be produced. Such symptoms appear to be common in Tourette syndrome.


Acknowledgements. The author thanks Mark Rossi for comments and suggestions on the manuscript. Funding statement. The author is supported by NIH AA021074.

References 1.

3.

4.

5.

6.

7.

8. 9. 10.

11. 12.

13. 14.

15. 16.

17.

18. Swanson LW. 2000 Cerebral hemisphere regulation of motivated behavior. Brain Res. 886, 113 –164. (doi:10.1016/S0006-8993(00)02905-X) 19. Grillner S, Wallen P, Saitoh K, Kozlov A, Robertson B. 2008 Neural bases of goal-directed locomotion in vertebrates—an overview. Brain Res Rev. 57, 2 –12. (doi:10.1016/j.brainresrev.2007.06.027) 20. Luo M, Ding L, Perkel DJ. 2001 An avian basal ganglia pathway essential for vocal learning forms a closed topographic loop. J. Neurosci. 21, 6836 –6845. 21. Yin HH, Knowlton BJ. 2006 The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 7, 464 –476. (doi:10.1038/nrn1919) 22. Berendse HW, Groenewegen HJ. 1990 Organization of the thalamostriatal projections in the rat, with special emphasis on the ventral striatum. J. Comp. Neurol. 299, 187–228. (doi:10.1002/cne.902990206) 23. Ding J, Peterson JD, Surmeier DJ. 2008 Corticostriatal and thalamostriatal synapses have distinctive properties. J. Neurosci. 28, 6483–6492. (doi:10.1523/JNEUROSCI.0435-08.2008) 24. Ragsdale Jr CW, Graybiel AM. 1991 Compartmental organization of the thalamostriatal connection in the cat. J. Comp. Neurol. 311, 134–167. (doi:10. 1002/cne.903110110) 25. Kreitzer AC. 2009 Physiology and pharmacology of striatal neurons. Annu. Rev. Neurosci. 32, 127– 147. (doi:10.1146/annurev.neuro.051508.135422) 26. Surmeier DJ, Ding J, Day M, Wang Z, Shen W. 2007 D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends Neurosci. 30, 228–235. (doi:10.1016/j.tins.2007.03.008) 27. Gerfen CR, Surmeier DJ. 2011 Modulation of striatal projection systems by dopamine. Annu. Rev. Neurosci. 34, 441 –466. (doi:10.1146/annurevneuro-061010-113641) 28. Hikosaka O, Takikawa Y, Kawagoe R. 2000 Role of the basal ganglia in the control of purposive saccadic eye movements. Physiol. Rev. 80, 953–978. 29. Kravitz AV, Freeze BS, Parker PR, Kay K, Thwin MT, Deisseroth K, Kreitzer AC. 2010 Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature 466, 622 –626. (doi:10.1038/nature09159) 30. Platt JR, Kuch DO, Bitgood SC. 1973 Rats’ leverpress durations as psychophysical judgements of time. J. Exp. Anal. Behav. 19, 239 –250. (doi:10. 1901/jeab.1973.19-239) 31. Skinner BF. 1938 The behavior of organisms. New York, NY: Appleton-Century-Crofts. 32. Yin HH. 2009 The role of the murine motor cortex in action duration and order. Front Integr. Neurosci. 3, 23. (doi:10.3389/neuro.07.023.2009)

33. Yu C, Gupta J, Yin HH. 2010 The role of mediodorsal thalamus in temporal differentiation of rewardguided actions. Front. Integr. Neurosci. 4, 14. (doi:10.3389/fnint.2010.00014) 34. Zeiler MD. 1985 Pure timing in temporal differentiation. J. Exp. Anal. Behav. 43, 183–193. (doi:10.1901/jeab.1985.43-183) 35. Fan D, Rossi MA, Yin HH. 2012 Mechanisms of action selection and timing in substantia nigra neurons. J. Neurosci. 32, 5534– 5548. (doi:10.1523/ JNEUROSCI.5924-11.2012) 36. Rossi MA, Sukharnikova T, Hayrapetyan VY, Yang L, Yin HH. 2013 Operant self-stimulation of dopamine neurons in the substantia nigra. PLoS ONE 8, e65799. (doi:10.1371/journal.pone.0065799) 37. Henny P, Brown MT, Northrop A, Faunes M, Ungless MA, Magill PJ, Bolam JP. 2012 Structural correlates of heterogeneous in vivo activity of midbrain dopaminergic neurons. Nat. Neurosci. 15, 613 –619. (doi:10.1038/nn.3048) 38. Hogan N. 1984 Adaptive control of mechanical impedance by coactivation of antagonist muscles. Autom. Control IEEE Trans. 29, 681–690. (doi:10. 1109/TAC.1984.1103644) 39. Rossi MA, Fan D, Barter JW, Yin HH. 2013 Bidirectional modulation of substantia nigra activity by motivational state. PLoS ONE 8, e71598. (doi:10. 1371/journal.pone.0071598) 40. Zhou FM, Lee CR. 2011 Intrinsic and integrative properties of substantia nigra pars reticulata neurons. Neuroscience 198, 69 –94. (doi:10.1016/j. neuroscience.2011.07.061) 41. Garcia-Rill E. 1986 The basal ganglia and the locomotor regions. Brain Res. 396, 47 –63. (doi:10. 1016/0165-0173(86)90009-3) 42. Niijima K, Yoshida M. 1982 Electrophysiological evidence for branching nigral projections to pontine reticular formation, superior colliculus and thalamus. Brain Res. 239, 279–282. (doi:10.1016/ 0006-8993(82)90852-6) 43. Takakusaki K. 2008 Forebrain control of locomotor behaviors. Brain Res. Rev. 57, 192 –198. (doi:10. 1016/j.brainresrev.2007.06.024) 44. Peterson BW. 1979 Reticulospinal projections to spinal motor nuclei. Annu. Rev. Physiol. 41, 127– 140. (doi:10.1146/annurev.ph.41.030179.001015) 45. Deliagina TG, Beloozerova IN, Zelenin PV, Orlovsky GN. 2008 Spinal and supraspinal postural networks. Brain Res. Rev. 57, 212 –221. (doi:10.1016/j. brainresrev.2007.06.017) 46. Orlovsky GN, Deliagina TG, Wallen P. 1992 Vestibular control of swimming in lamprey. I. Responses of reticulospinal neurons to roll and pitch. Exp. Brain Res. 90, 479– 488. (doi:10.1007/BF00230930)

Phil. Trans. R. Soc. B 369: 20120473

2.

Buse J, Schoenefeld K, Munchau A, Roessner V. 2012 Neuromodulation in Tourette syndrome: dopamine and beyond. Neurosci. Biobehav. Rev. 37, 1069–1084. (doi:10.1016/j.neubiorev.2012.10.004) Graybiel AM. 1998 The basal ganglia and chunking of action repertoires. Neurobiol. Learn. Mem. 70, 119–136. (doi:10.1006/nlme.1998.3843) Hallett M, Khoshbin S. 1980 A physiological mechanism of bradykinesia. Brain 103, 301 –314. (doi:10.1093/brain/103.2.301) Yin HH. 2010 The sensorimotor striatum is necessary for serial order learning. J. Neurosci. 30, 14 719 – 14 723. (doi:10.1523/JNEUROSCI.3989-10.2010) Yin HH, Ostlund SB, Knowlton BJ, Balleine BW. 2005 The role of the dorsomedial striatum in instrumental conditioning. Eur. J. Neurosci. 22, 513–523. (doi:10.1111/j.1460-9568.2005.04218.x) Worbe Y, Malherbe C, Hartmann A, Pelegrini-Issac M, Messe A, Vidailhet M, Lehericy S, Benali H. 2012 Functional immaturity of cortico-basal ganglia networks in Gilles de la Tourette syndrome. Brain 135, 1937 –1946. (doi:10.1093/brain/aws056) Cousins MS, Sokolowski JD, Salamone JD. 1993 Different effects of nucleus accumbens and ventrolateral striatal dopamine depletions on instrumental response selection in the rat. Pharmacol. Biochem. Behav. 46, 943–951. (doi:10. 1016/0091-3057(93)90226-J) Martin JP. 1967 Basal ganglia and posture. Philadelphia, PA: Lippincott. Bernstein N. 1996 Dexterity and its development. Mahwah, NJ: Lawrence Erlbaum Associates. Cools A. 1985 Brain and behavior: hierarchy of feedback systems and control of input. In Perspectives in ethology, pp. 109–168. Berlin, Germany: Springer. Powers WT. 1973 Behavior: control of perception. New Canaan, CT: Benchmark. Botvinick MM, Niv Y, Barto AC. 2009 Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition 113, 262–280. (doi:10.1016/j.cognition.2008.08.011) Fuster JM. 1995 Memory in the cerebral cortex. Cambridge, MA: MIT press. Robinson DA. 1989 Integrating with neurons. Annu. Rev. Neurosci. 12, 33 –45. (doi:10.1146/annurev.ne. 12.030189.000341) Sacks O. 1991 Awakenings. London, UK: Pan Macmillan. Gerfen CR, Wilson CJ. 1996 The basal ganglia. In Handbook of chemical neuroanatomy, vol. 12 (eds LW Swanson, A Bjorklund, T Hokfelt), pp. 371–468. Amsterdam, The Netherlands: Elsevier. Hikosaka O. 2007 GABAergic output of the basal ganglia. Progr. Brain Res. 160, 209–226. (doi:10.1016/S0079-6123(06)60012-5)

9


correlated with actual body configuration. It is hoped that predictions such as these may be useful in guiding future experimental endeavours.


59. 60.

61.

62.

63.

accumbens: the integrative functions of corticobasal ganglia networks. Eur. J. Neurosci. 28, 1437– 1448. (doi:10.1111/j.1460-9568.2008.06422.x) Von Neumann J. 2012 The computer and the brain. New Haven, CT: Yale University Press. Swanson LW. 2003 The amygdala and its place in the cerebral hemisphere. Ann. NY Acad. Sci. 985, 174–184. (doi:10.1111/j.1749-6632.2003.tb07081.x) Cui G, Jun SB, Jin X, Pham MD, Vogel SS, Lovinger DM, Costa RM. 2013 Concurrent activation of striatal direct and indirect pathways during action initiation. Nature 494, 238 –242. (doi:10.1038/nature11846) Redgrave P, Prescott TJ, Gurney K. 1999 The basal ganglia: a vertebrate solution to the selection problem? Neuroscience 89, 1009 –1023. (doi:10. 1016/S0306-4522(98)00319-4) Buhusi CV, Meck WH. 2005 What makes us tick? Functional and neural mechanisms of interval timing. Nat. Rev. Neurosci. 6, 755–765. (doi:10. 1038/nrn1764)

10

Phil. Trans. R. Soc. B 369: 20120473

53. Kita H, Kita T. 2001 Number, origins, and chemical types of rat pallidostriatal projection neurons. J. Comp. Neurol. 437, 438–448. (doi:10.1002/cne.1294) 54. Costa RM, Cohen D, Nicolelis MA. 2004 Differential corticostriatal plasticity during fast and slow motor skill learning in mice. Curr. Biol. 14, 1124–1134. (doi:10.1016/j.cub.2004.06.053) 55. Hernandez G, Breton Y-A, Conover K, Shizgal P. 2010 At what stage of neural processing does cocaine act to boost pursuit of rewards? PLoS ONE 5, e15081. (doi:10.1371/journal.pone.0015081) 56. Servan-Schreiber D, Printz H, Cohen JD. 1990 A network model of catecholamine effects: gain, signal-to-noise ratio, and behavior. Science 249, 892 –895. (doi:10.1126/science.2392679) 57. Thurley K, Senn W, Luscher HR. 2008 Dopamine increases the gain of the input-output response of rat prefrontal pyramidal neurons. J. Neurophysiol. 99, 2985–2997. (doi:10.1152/jn.01098.2007) 58. Yin HH, Ostlund SB, Balleine BW. 2008 Rewardguided learning beyond dopamine in the nucleus


47. Hess WR. 1957 The functional organization of the diencephalon. New York, NY: Grune & Stratton. 48. Granit R. 1970 The basis of motor control: integrating the activity of muscles, alpha and gamma motoneurons and their leading control systems. London, UK: Academic Press. 49. Anderson M, Horak F. 1985 Influence of the globus pallidus on arm movements in monkeys. III. Timing of movement-related information. J. Neurophysiol. 54, 433–448. 50. Bullock D, Grossberg S. 1989 VITE and FLETE: Neural modules for trajectory formation and postural control. In Volitional action (ed. W Hershberger), pp. 253–298. Amsterdam, The Netherlands: Elsevier/North Holland. 51. Horak FB, Anderson ME. 1984 Influence of globus pallidus on arm movements in monkeys. I. Effects of kainic acid-induced lesions. J. Neurophysiol. 52, 290–304. 52. Horak FB, Anderson ME. 1984 Influence of globus pallidus on arm movements in monkeys. II. Effects of stimulation. J. Neurophysiol. 52, 305–322.

Time representation in reinforcement learning models of the basal ganglia.

A basal ganglia circuit for evaluating action outcomes.

Basal ganglia subcircuits distinctively encode the parsing and concatenation of action sequences.

Neurotransmitters and neuromodulators in the basal ganglia.

Basal ganglia-thalamus and the "crowning enigma".

The connectome of the basal ganglia.

Bilateral traumatic hemorrhage of the basal ganglia.

Model-based action planning involves cortico-cerebellar and basal ganglia networks.

Learning Reward Uncertainty in the Basal Ganglia.

Rapid postanoxic calcification of the basal ganglia.

Reward functions of the basal ganglia.

Basal ganglia calcification in schizophrenia.

[Basal ganglia calcifications in children].

Basal ganglia mineralization in schizophrenia.

Extrinsic connections of the basal ganglia.

Basal ganglia echogenicity in tauopathies.

Model for basal ganglia disorders.

Obsessive compulsive disorder and basal ganglia dysfunction.

Basal ganglia plus insula damage yields stronger disruption of smoking addiction than basal ganglia damage alone.

Basal ganglia calcification and psychosis in mongolism.

Time-course of coherence in the human basal ganglia during voluntary movements.

Toward sophisticated basal ganglia neuromodulation: Review on basal ganglia deep brain stimulation.

Germ cell tumors of the thalamus and the basal ganglia.

A Biologically Inspired Computational Model of Basal Ganglia in Action Selection.