Available online at www.sciencedirect.com

ScienceDirect Spinal circuits for motor learning Robert M Brownstone1,2, Tuan V Bui3,4 and Nicolas Stifani2 Studies of motor learning have largely focussed on the cerebellum, and have provided key concepts about neural circuits required. However, other parts of the nervous system are involved in learning, as demonstrated by the capacity to ‘train’ spinal circuits to produce locomotion following spinal cord injury. While somatosensory feedback is necessary for spinal motor learning, feed forward circuits within the spinal cord must also contribute. In fact, motoneurons themselves could act as comparators that integrate feed forward and feedback inputs, and thus contribute to motor learning. Application of cerebellarderived principles to spinal circuitry leads to testable predictions of spinal organization required for motor learning. Addresses 1 Department of Surgery (Neurosurgery), Dalhousie University, Halifax, Nova Scotia, Canada B3H 4R2 2 Department of Medical Neuroscience, Dalhousie University, Halifax, Nova Scotia, Canada B3H 4R2 3 Department of Biology, University of Ottawa, Ottawa, Ontario, Canada K1N 6N5 4 Centre for Neural Dynamics, University of Ottawa, Ottawa, Ontario, Canada K1N 6N5 Corresponding author: Brownstone, Robert M ([email protected])

Current Opinion in Neurobiology 2015, 33:166–173 This review comes from a themed issue on Motor circuits and action Edited by Ole Kiehn and Mark Churchland

spinal transections, animals can be trained to walk on a treadmill; the sensory input provided by treadmill activity ‘retrains’ the spinal cord to produce coordinated locomotor activity [6,7,8]. These studies have been translated to humans, in whom treadmill training with body weight support can lead to significant improvement in gait [9–11]. Therefore following transection, latent spinal locomotor circuits are progressively recruited or reconfigured during training to produce locomotor movements, demonstrating that the spinal cord has the capacity to ‘learn’ motor behaviour. Yet the circuits and cellular mechanisms underlying this plasticity remain enigmatic. In this brief review, we outline some key principles of motor learning demonstrated in studies of cerebellar function. We then turn to invertebrate studies to examine how neuronal interactions lead to ion channel expression and circuit homeostasis, key mechanisms for circuit plasticity. We next ask whether the spinal cord contains known modules similar in configuration to those identified as necessary for cerebellar-mediated learning, and whether there is evidence that these modules may be influenced in a manner similar to that shown in invertebrate circuits. And finally, we predict the structure of spinal circuits responsible for recovery of locomotor function following spinal cord injury. Understanding these circuits and the mechanisms governing their plasticity is crucial for the development of strategies to improve motor function following injury to or disease of the CNS.

http://dx.doi.org/10.1016/j.conb.2015.04.007 0959-4388/# 2015 Published by Elsevier Ltd.

Introduction The central nervous system (CNS) is remarkable for its plasticity, which is evident in both normal learning as well as in functional recovery following diseases or injuries. During the lifetime of an organism, motor behaviour continually adjusts to changing environments and new motor tasks are learned. Fundamental principles of motor learning have been gleaned through studies of the cerebellum [1]. Yet given the capacity for motor learning following injuries, including those to the cerebellum [2,3], it is clear that motor learning is not solely the purview of the cerebellum [4,5], and that the circuitry involved in the learning process must be distributed in the CNS. One dramatic example of non-cerebellar motor learning can be seen following spinal cord injuries. After complete Current Opinion in Neurobiology 2015, 33:166–173

Lessons from the cerebellum: modules for motor learning We will not review the role of the cerebellum in motor learning or the complex models that have been developed [1,12–14]. Rather, we will discuss a few key concepts of motor learning that have emerged through decades of studies of the cerebellum [15,16] or cerebellar-like structures such as the electrosensory lobe of the mormyrid electric fish [17]. The starting point for a motor circuit is a controller that translates motor intention (a signal encoding the goal of the movement) into a signal that drives effectors of motor activity (Figure 1a). This translation is mediated by an inverse model — a circuit that computes the motor commands required for the motor action [16]. Sensory-tomotor feedback then provides the CNS with information on what the intended motor command accomplished (Figure 1a). Thus, this feedback is referred to as an ‘instructive’ input to motor circuits [18]. There are many different modalities of feedback originating from sensory receptors in the limbs or other specialised organs. The www.sciencedirect.com

Spinal circuits for motor learning Brownstone, Bui and Stifani 167

Figure 1

(a) Feedback and feed forward concepts

(b)

Cerebellar network Feed forward

Intention 1

Sensory Prediction Error Controller Comparator

Instructive 5

Motor command

Motor command

Efference copy

Feedback

4

Feedback

mf

Predictive

Effector

cf

Efference copy

Feed forward 3

Pc

Gc

6

2

CTX

pf

-

-

PN

RN

Intention

Action

AIN

-

IO

Sensors

Action

Sensor

Current Opinion in Neurobiology

Feedback and feed forward motor circuits. (a) The intention (1) for a given motor action is sent to a controller. In response, the controller generates a motor command (2) that is sent to the effectors. An efference copy of the motor command (3) feeds forward to a comparator as a predictive signal of the sensory consequences of the motor command. In parallel, the effector generates the action (4), and the result is monitored by multiple sensory modalities. These sensors provide instructive feedback (5) to the comparator that compares the feedback with the predicted feed forward signal to compute the sensory prediction error (6). This signal is fed to the controller for potential corrective motor commands. (b) Cerebellar circuits for eye blinking motor learning. Motor intentions (grey arrow) are conveyed via the controller (orange) composed of pontine nuclei (PN) and the deep cerebellar nucleus, anterior interpositus nucleus (AIN, light green). The controller sends the motor command to the effector (red nucleus, RN, blue) that will lead to the action (in this case, eyelid closure). PN also conveys an efference copy via mossy fibers (mf) to Granule cells (Gc) located in the cerebellar cortex (CTX, shaded area). The Gc through paralell fibers (pf) synapse with Purkinje cells (Pc) that act as comparators between this predictive input and the instructive feedback relayed via the inferior olive (IO) through climbing fibers (cf). The AIN also acts as a nested comparator, receiving inputs from PN and IO, and is a hybrid comparator/controller as discussed in text. Synapses are excitatory unless a minus sign is indicated. Adapted with permission from Freeman and Steinmetz [37].

information captured by these receptors is relayed by sensory neurons in the periphery to motor circuits directly or via intermediary relay neurons. The degree to which feedback affects a motor circuit will depend on tuning of that sensory input, which can be gated [19], as well as the reliability of that sensory input in the given task [1,19–21]. Thus feedback is a dynamic process. But reliance on feedback alone is insufficient for motor control and learning, as it is relatively slow and can generate motor instabilities [22]. As such, the nervous system has evolved to predict the sensory consequences of a motor action, as first beautifully demonstrated by Helmholtz in the mid-19th century. That is, a second key requirement for a motor learning circuit is that of a forward model (Figure 1a). A forward model uses a variety of inputs to predict the sensory consequences of the movement [16,23–25]. Feed forward input is not error based and necessitates a prerequisite concept of the action [16,23–25]. For example, if the intention is to grab an object, forward models will take into account the properties of the object and environment to predict the sensory consequences [26,27]. A key source of input to a www.sciencedirect.com

forward model is ‘efference copy’: the output of the controller feeds into the forward model. For example, copies of spinal motor commands are provided to the cerebellum via neurons of the ventral spinocerebellar tract [28] or neurons of the lateral reticular nucleus [29]. Thus the motor command is continually fed into the forward models, minimising the consequences of relying solely on sensory feedback with its inherent delays [30]. Feedback and feed forward circuitry converge on a comparator [16,18,30,31]. The comparator integrates the predictive input encoded in the internal forward model with the instructive feedback from the actual movement to determine the ‘sensory prediction error’ (Figure 1a). The predictive input could be a negative image of the motor command such that the error measurement is a simple subtraction of the efference copy from the instructive feedback [17,32]. This error measurement can then be used to dampen oscillations that would be produced by the inherent correction delay if afferent input were the only source of error correction [16]. The output of this circuitry drives adaptive mechanisms in which ‘learning Current Opinion in Neurobiology 2015, 33:166–173

168 Motor circuits and action

rules’ instigate changes in synaptic strength (long-term potentiation or depression) based on input signals [33,34]. That is, the comparators form the basis of motor learning [16]. Patients with cerebellar ataxia have a ‘mismatch’ between predictive and instructive inputs. They have lower feedback gains than controls, but whether this is a primary problem or results from compensation in response to a reduction in feed forward input is not clear [35]. This demonstrates the importance of internal models for accurate movement. Thus the key modules for motor learning are the controller, the controlled object, the predictive input (forward model), the instructive input (sensory feedback), and the comparator (where prediction and instruction converge) (Figure 1a) [1]. Although there are gaps in relating these concepts to specific circuits for motor learning, a number of correlates can be found. Purkinje neurons play a key role in motor learning [36]. In examination of defined circuits, it can be seen that the precerebellar nuclei provide feedforward input via granule cells to Purkinje cells and the anterior interpositus nucleus, both of which receive feedback via the inferior olivary system [37]. Thus these structures act as comparators (Figure 1b). Here, the anterior interpositus nucleus acts as a hybrid comparator/controller, similar to that seen in feedback control systems that measure motor errors and are crucial for motor learning [38]. Together, these modules (Figure 1a) offer the flexibility for both short-term adjustments, and a circuit in which plasticity can lead to longterm changes [39]. That is, these loops offer a simple yet robust framework for learning.

Neuronal homeostasis Despite adaptive modifications induced during the learning process, neuronal stability or homeostasis must be maintained both at the cellular and circuit levels. In this context, prototypical Hebbian processes, which are positive feedback in nature, are inherently unstable [40]. Thus homeostatic mechanisms (cellular and circuit) that maintain neuronal excitability and/or firing within appropriate ranges are important to maintain stability, and thus essential for learning circuits [41,42]. Several mechanisms can introduce stability to these circuits: synaptic scaling, spike-timing dependent plasticity, and synaptic redistribution [40]. Synaptic scaling may be independent of neuronal firing, resulting instead from alterations in glutamatergic transmission [42]. Whether it is confined to specific synapses or is cell-wide, it is probably important to maintain synaptic strength and would thus be an important homeostatic mechanism involved in learning. Studies of invertebrate nervous systems in which physiological function is conserved despite large variability in biophysical properties between animals have contributed to our understanding of the mechanisms underlying Current Opinion in Neurobiology 2015, 33:166–173

neuronal homeostasis. It has been demonstrated in several systems (for example the crustacean stomatogastric ganglion) that, despite a large degree of variability in neuronal properties (for example, in ion channel expression), the output behaviour of a network remains stereotypical. In other words, there are many different solutions that produce a given behaviour, and different animals within a species may use quite different solutions [43–45]. In fact, the resultant behaviour can be so similar that the inherent variability of its elements cannot be appreciated unless there is a considerable insult to the nervous system. For example, in the mollusc Tritonia, swimming behaviour is stereotypical. But when one of the pedal commissures is severed, the resultant behaviour is variable between animals and depends on the synaptic strength between two neuronal types [46]. These studies indicate that circuits and neuronal properties develop in parallel in order to produce behaviour, and that the connections and properties need not be the same between individuals. A system comprised of variable elements that produces a relatively invariant behaviour would require homeostatic mechanisms within a given neuron [47] as well as homeostatic changes in the circuits [48]. Such mechanisms — perhaps regulated by calcium activity ‘set points’ — are responsible for the expression of ion channels in any given neuronal type [48]. These ‘cell-autonomous regulation rules’ could in turn lead to circuit homeostasis, with calcium signalling operating in a feedback role [48]. In other words, activity within self-assembled circuits leads to changes in ion channel expression and thus neuronal properties. Such changes are the basis for learning. It would seem that instructive input — the feedback relaying what the animal did in response to a command — could drive such changes.

The Ia afferent-motoneuron-Renshaw cell circuit as a fundamental learning module Studies of the cerebellum have outlined that a key feature of motor learning is a module of neurons that acts as a comparator between feed forward predictive commands and feedback instructive data. While there is good evidence that the cerebellum can provide an internal model for a motor program [49], this does not preclude the presence of other models elsewhere in the CNS. In fact, there is evidence that hierarchical control loops are involved in motor control [25,50], so there may be such modules elsewhere. Do such comparators exist in the spinal cord, and are they involved in motor learning? From cerebellar studies, we learned that a comparator is defined by its inputs: there are predictive and instructive inputs of opposite signs. By this definition, a-motoneurons (MNs), which function to produce muscle contraction, are fundamental comparator neurons. a-MNs are the controllers of the effector www.sciencedirect.com

Spinal circuits for motor learning Brownstone, Bui and Stifani 169

muscles (Figure 2a), and receive direct sensory feedback from Ia proprioceptive afferents innervating stretchsensitive muscle spindles. In addition, they provide an efference copy to Renshaw cells, which in turn synapse on the originating and/or homonymous MNs. Thus a-MNs receive instructive (Ia) and predictive (Renshaw cell) inputs of different signs (excitatory and inhibitory respectively), and thus could act as comparators. A dual role as comparators and controllers would be similar to that of the deep cerebellar nuclei (Figure 1b) and to a feedback control system that measures motor errors for learning [38]. This circuitry would confer the ability to tune a-MN activity such that appropriate muscle contraction is produced. In the absence of proprioceptive feedback, a-MNs would have reduced output and would thus produce insufficient muscle contraction (‘servo-assist’ model of motor control) [51]. In the absence of feed forward input (Renshaw cell input), the degree of a-MN activity would be greater than that needed to produce appropriate contraction [52]. But could these feedback and feed forward loops be involved in learning? In other words, are there homeostatic mechanisms in MNs, and would these lead to this basic form of motor learning? In attempting to answer these questions, we ask whether the development of ion channels in a-MNs depends on activity that will be regulated by the balance of (in this simplified example) feedback (Ia) and feed forward (Renshaw cell) input (see, for example [53]). As comparators, MNs would measure discrepancies between motor commands and motor output, and these discrepancies would provide the stimulus to shape motoneuronal biophysical properties such as channel densities and distributions. If one of these inputs is altered, we would predict a change in a-MN ion channel expression. For example, following axotomy beyond the site of recurrent motor axon collaterals to Renshaw cells, we would predict an initial imbalance between feed forward (still present) and feedback (now absent) input to MNs. This imbalance may also be present in spinal muscular atrophy, in which there is a reduction of Ia afferent input to MNs [54]. In both of these instances, changes in MN ion channel expression and excitability have been reported [54–56]. In fact, changes to MN properties can occur fairly rapidly following block of neuromuscular transmission, which would result in a mismatch between feed forward and feedback inputs [57]. Thus there is evidence that MN properties may be governed, in part, by the balance between instructive and predictive inputs. Evolution can also provide insights into the importance of this balance. As distal limbs developed the ability to manipulate objects, the microcircuitry of the controlling MNs also evolved: there are no Renshaw cells, and Ia input is more refined [58]. That is, the balance between www.sciencedirect.com

Figure 2

Motoneuron learning

(a)

Feedback Instructive

Feed forward

Ia

Predictive

Sensors

γ

α RC

Controller Comparator

Effectors

(b) Locomotor learning network

Intention Instruction

Sensors Comparator Prediction Controller Effectors (c)

Spinal locomotor learning

Intention Instruction

Sensors Comparator Prediction Controller Effectors Current Opinion in Neurobiology

Spinal circuitry for motor learning. (a) Model of motoneuron learning. Alpha motoneurons (a) send the excitatory (triangular endings) motor command to muscles in the periphery as well as an efference copy to Renshaw cells (RC). In turn, RCs inhibit (flat endings) both a and gamma (g) MNs as a feed forward predictive circuit. g-MNs regulate the contraction of muscle spindles therefore adjusting the sensory feedback provided by proprioceptive Ia afferents. a-MNs are at the center of this circuitry receiving feed forward and feedback signals, acting both as comparators and as controllers. (b) Hypothesised minimal spinal circuitry for learning locomotor activity. Motor intention signals descending from supraspinal centers excite spinal locomotor

Current Opinion in Neurobiology 2015, 33:166–173

170 Motor circuits and action

feed forward and feedback was, to a degree, maintained by a concomitant reduction in both. Perhaps this reflects the importance of hand function, with more weight being placed on forward models in ‘higher’ circuits.

modules within the spinal cord that compare instructive input, receive feed forward commands, and compute the sensory prediction error? And are these modules a substrate for spinal learning?

Finally, it is interesting to consider the role of g-MNs in this elementary circuit (Figure 2a). g-MNs innervate the contractile elements of muscle spindles, and thus modulate the degree of spindle feedback [59]. In the absence of g-MN activity, spindles will convey little stretch information when the muscle is contracted, whereas in the presence of a high degree of g-MN activity, spindles will be taut and thus convey even minor muscle stretches. This notion is reminiscent of motor-to-sensory control in whisking [60], and could guide a probabilistic weighting of afferent inputs [1]. Thus g-MN activity will impact the comparator, a-MNs, by adjusting their instructive input. In this view, g-MN activity would adjust the ‘weighting’ of spindle afferent feedback, and thus contribute to aMN biophysical properties. With higher g-MN activity, feedback will be accentuated, and learning capacity would therefore be enhanced.

To examine these questions, we will use locomotion as an example (Figure 2b). Instructive (proprioceptive) input is crucial for the development of normal spinal locomotor circuit function [63]. Following spinal cord injury, there are many changes to spinal motor circuits [6] including in MN gene expression such as chloride transporters [64,65], ligand and voltage-gated channels [65], or serotonin receptors [66]. While these responses may be homeostatic changes in response to loss of input, these changes can lead to increased excitability resulting in spasticity [67,68]. Changes in expression of chloride transporters may be particularly sensitive to changes in MN inputs, and can lead to significant alterations in MN physiology [69]. Locomotor training, which provides instructive input to the spinal cord, can lead to improvement in function [67,70,71]. Such training leads to changes in synaptic transmission at specific pathways [72], and in fact has been shown to lead to upregulation of select serotonin receptors and the chloride transporter KCC2 in extensor MNs [68]. Thus instructive input can lead to changes in gene expression in spinal neurons.

Taking these considerations together, we would suggest that MN plasticity resulting from their roles as comparators in spinal circuits ensures that muscle activation meets the motor intention as new tasks are learned.

Spinal motor learning circuits This Ia-MN-RC circuitry in which a-MNs act as comparators can be considered to be the most basic spinal learning module. But while modifications of MN properties may be necessary, such a process would clearly not be sufficient for learning motor tasks such as grasping or locomotion. Learning new motor tasks will also require plasticity of motor circuits, and depend upon error signals from sensory feedback [61]. Computational studies suggest that the spinal cord contains microcircuits necessary to efficiently learn complex motor tasks [62]. Are there ( Figure 2 Legend Continued ) networks and a comparator. The controller produces both a motor command that is sent to the effectors and an efference copy to the comparator. The effectors, that is, muscles, generate the locomotor activity that will be monitored by the sensors and instructive feedback is provided to the comparator. The comparator sends the controller corrective signals based upon the sensory prediction error calculated by the difference between the instructive and the predictive signals. In this model, supraspinal input is crucial for directing spinal locomotor learning, with spinal networks playing an accessory (dotted lines) role. (c) Proposed model for intraspinal locomotor learning. After removal of supraspinal inputs (light grey), such as following spinal cord injury, spinal circuits (plain circles) assume the responsibility for learning to restore the generation of locomotor commands. The spinal controller sends the generated locomotor command to the effectors as well as an efference copy (prediction) to the comparator. Sensory feedback produced by locomotor training is conveyed from the periphery to the comparator by sensors. The comparator feeds the computed sensory prediction error back to the controller thus forming the basis of learning. Current Opinion in Neurobiology 2015, 33:166–173

Electrical stimulation of afferent input can also be used to induce changes to locomotor circuitry and improvement in locomotor function following spinal cord injury in animal models [73] and in humans [74,75]. This conditioning paradigm (in incomplete injury) involves alterations in feed forward input (predictive, see [76] as well as in feedback. Furthermore, although the mechanisms of spinal cord stimulation are not known but may involve stimulation of afferent fibers (either in dorsal roots or antidromically in dorsal columns, or perhaps dorsal grey matter), dorsal epidural stimulation in animals or humans with spinal cord injury can also lead to improvement in locomotor function [77], indicating that the induced neuronal activity can induce plastic changes in circuits. Under normal conditions, locomotor networks are driven by descending inputs from higher centers (Figure 2b). Following spinal cord transection and the resulting absence of descending inputs, training would result in an increase in rhythmic instructive (feedback) input to the comparator, which would activate the locomotor network that in turn provides feed forward input to the comparators and input to the effectors (Figure 2c) [67,70,71,77,78]. We would predict that these inputs would result in cellular plastic changes, leading to homeostatic plasticity and activation of locomotor networks. Understanding spinal circuits for motor learning is also important when considering recovery of function in other neurological diseases that affect motor performance. For www.sciencedirect.com

Spinal circuits for motor learning Brownstone, Bui and Stifani 171

example, infants with cerebral palsy have normal motor tone [79], possibly because the Renshaw cell-Ia-MN circuit described above is intact, with normal feed forward and feedback input. During post-natal development, when many movements are learned, spinal forward modules fail to receive usual descending inputs, and we would propose that the resulting mismatch between feed forward and feedback inputs to the MNs (and other spinal circuits) results in improper muscle length control (and coordination by spinal circuits). But physiotherapy — which would provide an increase in feedback — makes a profound difference to these children, and their movement repertoire can improve dramatically [80]. We would propose that a good deal of this motor learning occurs in the spinal cord.

Conclusion While valuable lessons about motor learning have been gleaned from years of studies of the cerebellum, it is unlikely that the cerebellum is the only structure in the CNS in which forward models are crucial for motor function. We suggest that motor learning is distributed in hierarchical networks that are dependent on feed forward and feedback loops. The ‘deepest’ of these loops relies on motoneurons as comparators — these circuits ‘learn’ how to regulate muscle length and force. These feed into spinal learning circuits, which can learn organizational control such as locomotion. And higher loops include the brain stem, cerebrum, and cerebellum would be needed to learn more complex motor tasks. This hierarchical, nested control structure is necessary to ensure normal motor learning.

Conflict of interest statement Nothing declared.

Acknowledgments

4.

Morton SM, Bastian AJ: Cerebellar contributions to locomotor adaptations during splitbelt treadmill walking. J Neurosci 2006, 26:9107-9116.

5.

Timmann D, Dimitrova A, Hein-Kropp C, Wilhelm H, Dorfler A: Cerebellar agenesis: clinical, neuropsychological and MR findings. Neurocase 2003, 9:402-413.

6.

Rossignol S, Frigon A: Recovery of locomotion after spinal cord injury: some facts and mechanisms. Annu Rev Neurosci 2011, 34:413-440.

7.

Leblond H, L’Esperance M, Orsal D, Rossignol S: Treadmill locomotion in the intact and spinal mouse. J Neurosci 2003, 23:11411-11419.

8. 

Takeoka A, Vollenweider I, Courtine G, Arber S: Muscle spindle feedback directs locomotor recovery and circuit reorganization after spinal cord injury. Cell 2014, 159:1626-1639. When proprioceptive feedback is genetically eliminated, mice with spinal hemisections do not regain locomotor function. This demonstrates the importance of feedback in motor circuit plasticity. 9.

Harkema SJ, Schmidt-Read M, Lorenz DJ, Edgerton VR, Behrman AL: Balance and ambulation improvements in individuals with chronic incomplete spinal cord injury using locomotor training-based rehabilitation. Arch Phys Med Rehabil 2012, 93:1508-1517.

10. Barbeau H, Basso M, Behrman A, Harkema S: Treadmill training after spinal cord injury: good but not better. Neurology 2006, 67:1900-1901 author reply 1901–1902. 11. Yang JF, Livingstone D, Brunton K, Kim D, Lopetinsky B, Roy F, Zewdie E, Patrick SK, Andersen J, Kirton A et al.: Training to enhance walking in children with cerebral palsy: are we missing the window of opportunity? Semin Pediatr Neurol 2013, 20:106-115. 12. Shmuelof L, Krakauer JW: Are we ready for a natural history of motor learning? Neuron 2011, 72:469-476. 13. Koziol LF, Budding D, Andreasen N, D’Arrigo S, Bulgheroni S, Imamizu H, Ito M, Manto M, Marvel C, Parker K et al.: Consensus paper: the cerebellum’s role in movement and cognition. Cerebellum 2014, 13:151-177. 14. Wolpert DM, Ghahramani Z, Jordan MI: An internal model for sensorimotor integration. Science 1995, 269:1880-1882. 15. Ramnani N: The primate cortico-cerebellar system: anatomy and function. Nat Rev Neurosci 2006, 7:511-522. 16. Wolpert DM, Miall RC, Kawato M: Internal models in the cerebellum. Trends Cogn Sci 1998, 2:338-347.

This paper is dedicated to the memory of our friend and colleague, Laurent Vinay. The insightful comments of Larry Jordan and Tom Jessell thankfully prevented submission of a much earlier version of the manuscript. This work has been supported by grants to RMB from the Canadian Institutes of Health Research (FRN 74633, 79413, and 89820), and is undertaken thanks, in part, to funding to RMB from the Canada Research Chairs program.

17. Kennedy A, Wayne G, Kaifosh P, Alvina K, Abbott LF, Sawtell NB:  A temporal basis for predicting the sensory consequences of motor commands in an electric fish. Nat Neurosci 2014, 17:416-422. The authors demonstrate a substrate for how motor commands are translated to sensory predictions and how these negative images are used to cancel expected instructive input.

References and recommended reading

18. Nguyen-Vu TDB, Kimpo RR, Rinaldi JM, Kohli A, Zeng H, Deisseroth K, Raymond JL: Cerebellar Purkinje cell activity drives motor learning. Nat Neurosci 2013, 16:1734-1736.

Papers of particular interest, published within the period of review, have been highlighted as:  of special interest  of outstanding interest 1.

2.

3.

Krakauer JW, Mazzoni P: Human sensorimotor learning: adaptation, skill, and beyond. Curr Opin Neurobiol 2011, 21:636-644. Kelly PJ, Stein J, Shafqat S, Eskey C, Doherty D, Chang Y, Kurina A, Furie KL: Functional recovery after rehabilitation for cerebellar stroke. Stroke 2001, 32:530-534. Konczak J, Pierscianek D, Hirsiger S, Bultmann U, Schoch B, Gizewski ER, Timmann D, Maschke M, Frings M: Recovery of upper limb function after cerebellar stroke: lesion symptom mapping and arm kinematics. Stroke 2010, 41:2191-2200.

www.sciencedirect.com

19. Jordan LM: Comment: gating effects and constraints on the central pattern generators for rhythmic movements. Can J Physiol Pharmacol 1981, 59:727-732. 20. Wolpert DM, Flanagan JR: Motor prediction. Curr Biol 2001, 11:R729-R732. 21. Fink AJP, Croce KR, Huang ZJ, Abbott LF, Jessell TM, Azim E: Presynaptic inhibition of spinal sensory feedback ensures smooth movement. Nature 2014, 509:43-48. 22. Shadmehr R, Smith MA, Krakauer JW: Error correction, sensory prediction, and adaptation in motor control. Annu Rev Neurosci 2010, 33:89-108. 23. Lennerstrand G, Bach-y-Rita P: Basic mechanisms of ocular motility and their clinical implications. In Proceedings of the Current Opinion in Neurobiology 2015, 33:166–173

172 Motor circuits and action

International Symposium; Wenner-Gren Center, Stockholm, June 4–6, 1974: edn 1st. Oxford, New York: Pergamon Press; 1975, .

25. Kawato M, Furukawa K, Suzuki R: A hierarchical neural-network model for control and learning of voluntary movement. Biol Cybern 1987, 57:169-185.

46. Sakurai A, Tamvacakis AN, Katz PS: Hidden synaptic differences  in a neural circuit underlie differential behavioral susceptibility to a neural injury. Elife 2014:3. Inter-animal variability in neuronal properties and synaptic strengths is not evident in the output from a neural circuit in the mollusc, Tritonia diomeda. However, in response to injury, these ‘hidden’ differences lead to variability in behaviour. The authors thus demonstrate that specific synaptic strengths need not be the same to produce normal behaviour but differences may impact recovery in disease or injury.

26. Panek I, Bui T, Wright AT, Brownstone RM: Cutaneous afferent regulation of motor function. Acta Neurobiol Exp (Wars) 2014, 74:158-171.

47. MacLean JN, Zhang Y, Johnson BR, Harris-Warrick RM: Activityindependent homeostasis in rhythmically active neurons. Neuron 2003, 37:109-120.

27. White O: The brain adjusts grip forces differently according to gravity and inertia: a parabolic flight experiment. Front Integr Neurosci 2015, 9:7.

48. O’Leary T, Williams AH, Franci A, Marder E: Cell types, network  homeostasis, and pathological compensation from a biologically plausible ion channel expression model. Neuron 2014, 82:809-821. The authors develop a model in which calcium dynamics are responsible for neuronal homeostasis, thus encoding an activity ‘set point.’ In the model, these rules govern synaptic and intrinsic channel expression, and lead to self-assembly of a motor circuit.

24. Jordan MI, Rumelhart DE: Forward models: supervised learning with a distal teacher. Cogn Sci 1992, 16:307-354.

28. Jankowska E, Hammar I: Interactions between spinal interneurons and ventral spinocerebellar tract neurons. J Physiol 2013, 591:5445-5451. 29. Pivetta C, Esposito MS, Sigrist M, Arber S: Motor-circuit communication matrix from spinal cord to brainstem neurons revealed by developmental origin. Cell 2014, 156:537-548. 30. Miall RC, Weir DJ, Wolpert DM, Stein JF: Is the cerebellum a smith predictor? J Mot Behav 1993, 25:203-216. 31. Marsat G, Maler L: Preparing for the unpredictable: adaptive feedback enhances the response to unexpected communication signals. J Neurophysiol 2012, 107:1241-1246. 32. Bell CC: Duration of plastic change in a modifiable efference copy. Brain Res 1986, 369:29-36. 33. Dean P, Porrill J, Ekerot CF, Jorntell H: The cerebellar microcircuit as an adaptive filter: experimental and computational evidence. Nat Rev Neurosci 2010, 11:30-43. 34. Bell CC, Han VZ, Sugawara Y, Grant K: Synaptic plasticity in a cerebellum-like structure depends on temporal order. Nature 1997, 387:278-281.

49. Cerminara NL, Apps R, Marple-Horvat DE: An internal model of a moving visual target in the lateral cerebellum. J Physiol 2009, 587:429-442. 50. Gordon G, Ahissar E: Hierarchical curiosity loops and active sensing. Neural Netw 2012, 32:119-129. 51. Windhorst U: Muscle proprioceptive feedback and spinal networks. Brain Res Bull 2007, 73:155-202. 52. Noga BR, Shefchyk SJ, Jamal J, Jordan LM: The role of renshaw cells in locomotion — antagonism of their excitation from motor axon collaterals with intravenous mecamylamine. Exp Brain Res 1987, 66:99-105. 53. Krutki P, Haluszka A, Mrowczynski W, Gardiner PF, Celichowski J: Adaptations of motoneuron properties to chronic compensatory muscle overload. J Neurophysiol 2015, 113:2769-2777.

35. Bhanpuri NH, Okamura AM, Bastian AJ: Predicting and correcting ataxia using a model of cerebellar function. Brain 2014, 137:1931-1944.

54. Mentis GZ, Blivis D, Liu W, Drobac E, Crowder ME, Kong L, Alvarez FJ, Sumner CJ, O’Donovan MJ: Early functional impairment of sensory-motor connectivity in a mouse model of spinal muscular atrophy. Neuron 2011, 69:453-467.

36. Lee KH, Mathews PJ, Reeves AM, Choe KY, Jami SA, Serrano RE, Otis TS: Circuit mechanisms underlying motor memory formation in the cerebellum. Neuron 2015, 86:529-540.

55. Pinter MJ, Vanden Noven S: Effects of preventing reinnervation on axotomized spinal motoneurons in the cat. I. Motoneuron electrical properties. J Neurophysiol 1989, 62:311-324.

37. Freeman JH, Steinmetz AB: Neural circuitry and plasticity mechanisms underlying delay eyeblink conditioning. Learn Mem 2011, 18:666-677.

56. Meehan CF, MacDermid VE, Montague SJ, Neuber-Hess M, Rose PK: Dendrite-derived supernumerary axons on adult axotomized motor neurons possess proteins that are essential for the initiation and propagation of action potentials and synaptic vesicle release. J Neurosci 2011, 31:6732-6740.

38. Ito M: Error detection and representation in the olivocerebellar system. Front Neural Circuits 2013, 7:1.  The author reviews cerebellar control systems, and illustrates differences in modules for detecting sensory vs. motor errors in olivocerebellar systems and how these modules relate to adaptive systems for learning. 39. Kuo AD: The relative roles of feedforward and feedback in the control of rhythmic movements. Motor Control 2002, 6:129-145. 40. Abbott LF, Nelson SB: Synaptic plasticity: taming the beast. Nat Neurosci 2000, 3(Suppl.):1178-1183. 41. Fong MF, Newman JP, Potter SM, Wenner P: Upward synaptic scaling is dependent on neurotransmission rather than spiking. Nat Commun 2015, 6:6339. 42. Davis GW: Homeostatic signaling and the stabilization of neural function. Neuron 2013, 80:718-728. 43. Prinz AA, Bucher D, Marder E: Similar network activity from disparate circuit parameters. Nat Neurosci 2004, 7:1345-1352. 44. Bucher D, Prinz AA, Marder E: Animal-to-animal variability in motor pattern production in adults and during growth. J Neurosci 2005, 25:1611-1619. 45. Marder E, Goeritz ML, Otopalik AG: Robust circuit rhythms in small circuits arise from variable circuit components and mechanisms. Curr Opin Neurobiol 2015, 31:156-163. Current Opinion in Neurobiology 2015, 33:166–173

57. Nakanishi ST, Cope TC, Rich MM, Carrasco DI, Pinter MJ: Regulation of motoneuron excitability via motor endplate acetylcholine receptor activation. J Neurosci 2005, 25:2226-2232. 58. Illert M, Kummel H: Reflex pathways from large muscle spindle afferents and recurrent axon collaterals to motoneurones of wrist and digit muscles: a comparison in cats, monkeys and humans. Exp Brain Res 1999, 128:13-19. 59. Eccles JC, Eccles RM, Iggo A, Lundberg A: Electrophysiological studies on gamma motoneurones. Acta Physiol Scand 1960, 50:32-40. 60. Saig A, Gordon G, Assa E, Arieli A, Ahissar E: Motor-sensory confluence in tactile perception. J Neurosci 2012, 32:1402214032. 61. Lundbye-Jensen J, Petersen TH, Rothwell JC, Nielsen JB: Interference in ballistic motor learning: specificity and role of sensory error signals. PLoS One 2011, 6:e17451. 62. Raphael G, Tsianos GA, Loeb GE: Spinal-like regulator facilitates control of a two-degree-of-freedom wrist. J Neurosci 2010, 30:9431-9444. 63. Akay T, Tourtellotte WG, Arber S, Jessell TM: Degradation of  mouse locomotor pattern in the absence of proprioceptive www.sciencedirect.com

Spinal circuits for motor learning Brownstone, Bui and Stifani 173

sensory feedback. Proc Natl Acad Sci U S A 2014, 111:1687716882. This paper demonstrates the importance of proprioceptive feedback for the development of normal locomotor circuitry. When this feedback is genetically eliminated, locomotor circuits produce only simple rhythmic patterns. 64. Boulenguez P, Liabeuf S, Bos R, Bras H, Jean-Xavier C, Brocard C, Stil A, Darbon P, Cattaert D, Delpire E et al.: Down-regulation of the potassium-chloride cotransporter KCC2 contributes to spasticity after spinal cord injury. Nat Med 2010, 16:302-307. 65. Ryge J, Winther O, Wienecke J, Sandelin A, Westerdahl AC, Hultborn H, Kiehn O: Transcriptional regulation of gene expression clusters in motor neurons following spinal cord injury. BMC Genomics 2010, 11:365. 66. Murray KC, Nakae A, Stephens MJ, Rank M, D’Amico J, Harvey PJ, Li X, Harris RL, Ballou EW, Anelli R et al.: Recovery of motoneuron and locomotor function after spinal cord injury depends on constitutive activity in 5-HT2C receptors. Nat Med 2010, 16:694-700. 67. Martinez M, Delivet-Mongrain H, Rossignol S: Treadmill training  promotes spinal changes leading to locomotor recovery after partial spinal cord injury in cats. J Neurophysiol 2013, 109:2909-2922. Following partial spinal cord injury, treadmill training — which produces rhythmic instructive input to the spinal cord — leads to plasticity in spinal circuitry and recovery of locomotor function. 68. Chopek JW, Sheppard PC, Gardiner K, Gardiner PF: Serotonin  receptor and KCC2 gene expression in lumbar flexor and extensor motoneurons posttransection with and without passive cycling. J Neurophysiol 2015, 113:1369-1376. The authors demonstrate that instructive input following complete spinal transection leads to changes in gene expression in spinal motoneurons.

71. Slawinska U, Rossignol S, Bennett DJ, Schmidt BJ, Frigon A, Fouad K, Jordan LM: Comment on ‘‘Restoring voluntary control of locomotion after paralyzing spinal cord injury’’. Science 2012, 338:328 author reply 328. 72. Cote MP, Gossard JP: Step training-dependent plasticity in spinal cutaneous pathways. J Neurosci 2004, 24:11317-11327. 73. Chen Y, Chen L, Liu R, Wang Y, Chen XY, Wolpaw JR: Locomotor impact of beneficial or nonbeneficial H-reflex conditioning after spinal cord injury. J Neurophysiol 2014, 111:1249-1258. 74. Thompson AK, Pomerantz FR, Wolpaw JR: Operant conditioning  of a spinal reflex can improve locomotion after spinal cord injury in humans. J Neurosci 2013, 33:2365-2375. The authors demonstrate that operant conditioning of the H-reflex — a paradigm in which predictive feed-forward input is regulated — leads to improvement in locomotor function. This study is a translation of animal work undertaken by this group. 75. Manella KJ, Roach KE, Field-Fote EC: Operant conditioning to increase ankle control or decrease reflex excitability improves reflex modulation and walking function in chronic spinal cord injury. J Neurophysiol 2013, 109:2666-2679. 76. Chen Y, Chen XY, Jakeman LB, Schalk G, Stokes BT, Wolpaw JR: The interaction of a new motor skill and an old one: H-reflex conditioning and locomotion in rats. J Neurosci 2005, 25:6898-6906. 77. Angeli CA, Edgerton VR, Gerasimenko YP, Harkema SJ: Altering spinal cord excitability enables voluntary movements after chronic complete paralysis in humans. Brain 2014, 137:1394-1409. 78. Shah PK, Garcia-Alias G, Choe J, Gad P, Gerasimenko Y, Tillakaratne N, Zhong H, Roy RR, Edgerton VR: Use of quadrupedal step training to re-engage spinal interneuronal networks and improve locomotor function after spinal cord injury. Brain 2013, 136:3362-3377.

69. Gackiere F, Vinay L: Contribution of the potassium-chloride cotransporter KCC2 to the strength of inhibition in the neonatal rodent spinal cord in vitro. J Neurosci 2015, 35:5307-5316.

79. Cioni G, Di Paco MC, Bertuccelli B, Paolicelli PB, Canapicchi R: MRI findings and sensorimotor development in infants with bilateral spastic cerebral palsy. Brain Dev 1997, 19:245-253.

70. Slawinska U, Majczynski H, Dai Y, Jordan LM: The upright posture improves plantar stepping and alters responses to serotonergic drugs in spinal rats. J Physiol 2012, 590:1721-1736.

80. Chen Y-N, Liao S-F, Su L-F, Huang H-Y, Lin C-C, Wei T-S: The effect of long-term conventional physical therapy and independent predictive factors analysis in children with cerebral palsy. Dev Neurorehabil 2013, 16:357-362.

www.sciencedirect.com

Current Opinion in Neurobiology 2015, 33:166–173

Spinal circuits for motor learning.

Studies of motor learning have largely focussed on the cerebellum, and have provided key concepts about neural circuits required. However, other parts...
488KB Sizes 0 Downloads 16 Views