ORIGINAL RESEARCH
How Reproducible Are Transcranial Magnetic Stimulation–Induced MEPs in Subacute Stroke? Maurits H. W. J. Hoonhorst,* Boudewijn J. Kollen,† Peter S. P. van den Berg,‡ Cornelis H. Emmelot,§ and Gert Kwakkelk
Purpose: Motor evoked potentials (MEPs) and total motor conduction time (TMCT) induced by transcranial magnetic stimulation (TMS) are used to make assumptions about the prognosis of motor outcome after stroke. Understanding the different sources of variability is fundamental to the concept of reliability. Reliability testing of TMS-MEPs and TMCTs within and between two independent examiners in healthy and stroke subjects is still an unexplored field in the clinical neurophysiology. Assessing the reproducibility of TMS measurements requires studies to investigate the test–retest reliability of TMS-induced MEPs and TMCT. The authors set out to test the reliability of these TMS measurements. Methods: Eighteen patients with stroke and 8 healthy volunteers were tested twice within a 1-week period by 2 examiners using TMS to determine MEPs and TMCT for the abductor pollicis brevis muscle of their affected and unaffected hands. Results: The authors found moderate to perfect reliability of TMS-induced MEPs in healthy volunteers, noninfarcted hemispheres (perfect agreement), and infarcted hemispheres (Kappa’s ¼ 0.45–0.87). Reliability of TMCT was good to excellent in the volunteers (intraclass correlation coefficients ¼ 0.77–0.97), excellent in the noninfarcted hemispheres (intraclass correlation coefficients ¼ 0.97–1.00), and poor to excellent in the infarcted hemispheres (intraclass correlation coefficients ¼ 0.44–0.90). Conclusions: The reliability of TMS-induced MEPs and TMCT measurements in healthy volunteers and the noninfarcted hemisphere of patients with stroke with an upper paretic limb was good to excellent. In contrast, TMS measurements in the infarcted hemisphere were less consistent. Based on the lower reproducibility of TMCT measurements in the infarcted hemisphere, we recommend to repeat the TMCT measurements to improve the reliability of tests. Key Words: Stroke, Transcranial magnetic stimulation, Reliability, Reproducibility, Motor evoked potential, TMS. (J Clin Neurophysiol 2014;31: 556–562)
S
troke is a major public health concern, with more than 1.1 million stroke events in Europe in the year 2000, and is the most common cause of acquired disability among adults in developed countries (Truelsen et al., 2006; WHO, 2003). Early prediction of outcome
From the *Department of NeuroRehabilitation, Vogellanden Center for Rehabilitation, Zwolle, the Netherlands; †Department of General Practice, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands; Departments of ‡Neurology and §Rehabilitation Medicine, Isala Klinieken, Zwolle, the Netherlands; and kDepartment of Rehabilitation Medicine, MOVE Research Institute Amsterdam, VU University Medical Center, Amsterdam, the Netherlands. This study was supported by Grant No. 154 from the Zwols Wetenschapsfonds Isala Klinieken, the Netherlands. Address correspondence and reprint requests to Maurits H. W. J. Hoonhorst, MD, Rehabilitation Centre De Vogellanden, Hyacinthstraat 66A, 8013 XZ Zwolle, the Netherlands; e-mail: [email protected]. Copyright Ó 2014 by the American Clinical Neurophysiology Society
ISSN: 0736-0258/14/3106-0556
556
after stroke is paramount because it allows professionals to (1) set realistic, attainable treatment goals, (2) select patients who will benefit most from a particular intervention, (3) increase the efficiency of stroke services and allow adequate triage, (4) anticipate early supported discharge including the need for home adjustments and community support, and (5) give optimal information to patients and their family. Understanding prognosis is also paramount for researchers, enabling them to improve future trials by adequate stratification of patients and so generate prognostically comparable patient groups (Veerbeek et al., 2011). Finger extension and shoulder abduction measured at 72 hours have been found to be essential for the prediction of upper limb recovery 6 months after stroke (Nijland et al., 2010; Stinear, 2010). Recently, it has been suggested that transcranial magnetic stimulation (TMS), a noninvasive method to test the functional integrity of the corticospinal tract system (Rothwell, 2005), may offer added value in predicting the final outcome for the paretic upper limb (Langhorne et al., 2011; Stinear, 2012). With that, the presence or absence of TMS-induced motor evoked potentials (MEPs) may offer clinicians an added valid diagnostic tool to just clinical assessment. Nevertheless, TMS of the ipsilesional motor cortex (M1) to elicit MEPs in the affected upper limb is not yet routinely used. Assessing the reliability of TMS measurements requires studies to investigate the test–retest reliability of TMS-induced MEPs (Guyatt, 1987). However, measurements of MEPs and total motor conduction time (TMCT) induced by TMS suffer from random and systematic error. Understanding these different sources of random and systematic variability is fundamental to the concept of reliability (Streiner and Norman, 2000). Despite its essential role in psychometrics, reliability testing of TMS-MEPs and TMCTs within and between examiners in healthy and stroke subjects is still an unexplored field in the clinical neurophysiology. Agreement between examiners about TMS-induced MEPs in both healthy volunteers and patients with stroke is an essential step in determining the validity of TMS as a method to assess the integrity of the corticospinal tract (Schulz et al., 2012). Replication of MEPs is imperative to demonstrate clinical pathology and exclude abnormalities due to error. Important factors for accurate replication include shape of the TMS coil, orientation of the stimulating coil, subject-related sources of variability and accuracy of observation (Pell, 2011). To date, the issue of the reliability of standard TMS techniques to elicit MEPs in healthy and stroke volunteers has only been addressed by a small number of investigations. These studies found that intraclass correlation coefficients (ICCs) for MEP latencies and motor thresholds within and between observers for healthy volunteers ranged from 0.60 to 0.92, whereas MEP amplitudes proved less consistent (Livingston and Ingersoll, 2008). Similar results were found in an intraobserver reliability study among 20 volunteers assessing motor threshold, map area, and stimulus–response curves (Malcolm et al.,
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
2006), and in reliability studies assessing silent period (Fritz et al., 1997) and MEP size (Carroll et al., 2001). Yet, there is almost no evidence on the test–retest reliability of MEP parameters of the paretic and nonparetic upper limbs after stroke. One reliability study involving 9 patients with stroke showed that ICCs were above 0.70 for parameters such as MEP amplitude and silent period (Koski et al., 2007), whereas a study by Butler et al. (2005) in 10 patients with stroke found that the group means of MEP amplitude, motor threshold, and map sizes did not change when tested 3 times within 1 week (Butler et al., 2005). The objective of this study was to establish the intra- and interobserver reliability for two examiners of TMS-induced MEPs in the hemispheres of patients with stroke and healthy volunteers. Our second aim was to establish the intra- and interobserver reliability for two laboratory examiners of TMS-induced TMCT in the infarcted and noninfarcted hemispheres of patients with stroke and healthy volunteers.
SUBJECTS AND METHODS Subjects This study was approved by the local Medical Ethics Committee of the Isala Klinieken Hospital. Written informed consent was obtained from all participants. The patients with stroke were recruited from the Department of Neurology of the Isala Klinieken and the volunteers from the staff of this department. The following inclusion and exclusion criteria were used: (1) first-ever ischemic hemispheric stroke more than 3 months before testing, as confirmed by computed tomography or MRI scan, (2) unilateral paralysis or paresis of the upper limb, (3) score on the modified Rankin scale ranging from 2 to 4 points, (4) absence of peripheral nerve pathology including diabetes and neuromuscular disease, and (5) no history of head injury, cranial surgery, epilepsy, cervical spine stenosis, or other contraindications for TMS. Finally, none of the participants were allowed to use pharmaceutical agents that could influence MEPs in a neuromodulatory way, such as modulators of NMDA and GABA receptors, epilepsy medications, or dopaminergic agonists.
Laboratory Examiners Two experienced laboratory examiners were selected for this reliability study. Both were instructed to independently perform TMSinduced MEPs in all participants and to analyze TMCT. Accurate electrode placement in the target muscle was checked for by visual inspection of the compound motor action potential of the abductor pollicis brevis muscle (Rossini et al., 1994). The test–retest interval was set at 7 days. Each examiner was fully blinded for the general medical evaluation and also for the other’s data. Clinical evaluation was performed by a neurologist (P.S.P.v.d.B.), who was blinded for the MEP data. Neurologic examination included quantification of the degree of paresis and disability after stroke. No effort was made to control for the amount of motor activity performed between sessions because none of the participants were engaged in a rehabilitation program during the study, and no recovery was expected between test and retest.
Transcranial Magnetic Stimulation
2003; Rossini et al., 1994). Participants were seated in a quiet temperature-controlled room and measurements took place in the afternoon. The infarcted and noninfarcted hemispheres in patients with stroke and both hemispheres in the volunteers were assessed. Motor evoked potentials were recorded from the abductor pollicis brevis muscle by surface electrodes, using a conventional electromyography system. The motor cortex was stimulated with a calibrated Magstim Dantec Maglite (Dantec Dynamics, Bristol, United Kingdom). Cortical (single-pulse) TMS was performed with a figure-ofeight shaped coil and cervical stimulation with a circular coil with an outer diameter of 9 cm. Hand motor activation was optimized by positioning the coil on the scalp at an angle of 458 away from the midsagittal line. First, the spot that produced the highest MEP amplitude of the abductor pollicis brevis muscle (hotspot) was determined by moving the coil over the scalp in the area of M1 of the noninfarcted hemisphere, with the stimulator at submaximal output. Subsequently, the coil positioning at the hotspot was mirrored to the infarcted hemisphere to elicit an MEP. The absence of response to TMS was recorded if no response was obtained after three of the six successive discharges at maximal intensity. When no MEP could be elicited at a given position, the coil was moved slightly to find a hotspot in adjoining sites. The criterion for the presence of an electromyogram response to TMS was a negative slope $ 50 mV, and latencies were determined by visual inspection of 3 successive MEPs (Paulus et al., 2003; Rossini et al., 1994). The latency data were then used to calculate TMCT. Thereafter, MEPs were divided into two groups: present or absent MEP. Present MEP was defined as any delayed or normal MEP. Delayed MEP was defined as a TMCT of $ 23.3 milliseconds (Di Lazzaro et al., 2004; Rossini et al., 1994). Because voluntary muscle contraction reduces the motor threshold and increases fluctuations in spinal excitability, resulting in greater variation in the electromyogram muscle response, all recordings were performed with the arm and hand muscles relaxed, to obtain homogeneous data and to avoid variations of the muscle response due to different levels of preinnervation (Di Lazzaro et al., 2004; Rossini et al., 1994).
Design and Statistical Analysis We investigated both interobserver and intraobserver reliability for TMS-induced MEPs and TMCTs. Cohen k was used to compute the interobserver reliability of inducing normal and delayed MEPs. Kappa scores were categorized with values .0.75 recorded as excellent, 0.40 to 0.75 as fair to good, and ,0.40 as poor (Fleiss, 1981). The limits of agreement (LOA) were calculated for intra- and interobserver reliability of TMCT. Intraobserver reliability was assessed by comparing the mean TMCT values from the initial test with the mean values on the retest, and calculating the LOA. Interobserver reliability was assessed by calculating the mean value for test and retest for each examiner and comparing the results between the two examiners, referred to as A and B. Intra- and interobserver reliability of TMCT were calculated using the ICC model (Fleiss, 1981). Systematic differences between and within examiners were tested using independent and paired Student t-tests. The a-level was set at 0.05 for each test. SPSS software (SPSS Version 15.0, IBM Corporation, Armonk, New York, United States) was used for the statistical analyses.
TMS Protocol The TMS technique, including the hotspot method and measurements of MEP latencies with electromyogram recording (Nihon Kohden Neuropack 8, Nihon Kohden Corporation, Tokyo, Japan), was performed according to the recommendations of the International Federation of Clinical Neurophysiology (Paulus et al., Copyright Ó 2014 by the American Clinical Neurophysiology Society
RESULTS Eighteen patients with stroke and eight volunteers participated in the study. All participants tolerated the TMS well, without any adverse events. Measurable MEPs were obtained from all participants, except for the noninfarcted hemisphere of one patient with 557
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
M. H. W. J. Hoonhorst et al.
stroke, whose MEP could not be obtained due to brachial plexus pathology. Table 1 shows the characteristics of the healthy and stroke participants. The volunteers were significantly younger, with a mean age of 28.3 6 4.9 years, compared with the patients with a mean age of 68.2 6 10.6 years. Table 2 presents the test–retest data on TMCT for all participants. For examiner A, the mean TMCT for the infarcted hemisphere in patients with stroke was 22.31 6 11.01 milliseconds at the first test and 20.53 6 11.82 milliseconds at the retest. For examiner B, the mean TMCT of the infarcted hemisphere was 20.72 6 11.85 milliseconds at the test and 18.61 6 12.16 milliseconds at the retest. Peripheral motor conduction time for all patients with stroke and volunteers showed latency times within the normal range of ,18 milliseconds.
TABLE 2. Comparison of Mean TMCT Form Both Hemispheres in All Subjects at the Test and Retest by Both Examiners Measure Healthy subjects (N ¼ 8) Test Retest Patients with stroke (N ¼ 18) Test Retest
Intra- and Interobserver TMS Data in the Healthy Volunteers Both the examiners identified only normal MEPs in all volunteers, resulting in perfect agreement. The ICC results for test–retest reliability of all MEP latencies are summarized in Table 3. Intraclass correlation coefficients for the interobserver reliability of TMCTs at the first test were 0.87 and 0.97 at the retest. Intraclass correlation coefficients of the intraobserver reliability of TMCTs were 0.77 for examiner A and 0.91 for examiner B. Fig. 1 shows the LOA for the first test and retest measurements in the volunteers obtained by the two examiners. The intraobserver LOA values between test and retest are plotted for each examiner in Figs. 1A and 1B, which show LOAs of 2.18 and 3.46 milliseconds and mean differences of 20.83 and 20.63 milliseconds. The interobserver LOAs for TMCT between the examiners are displayed in Figs. 1C and 1D. The intraobserver reliability shows a LOA of 1.45 milliseconds and a mean difference of 20.25 milliseconds at the retest, compared with a LOA of 2.72 milliseconds and a mean difference of 20.45 milliseconds at the first test.
Intra- and Interobserver TMS Data of the Noninfarcted Hemisphere The interobserver for inducing an MEP by the 2 examiners showed a perfect Cohen k (1.00) at the test and retest. Both
TABLE 1.
Mean (SD) Examiner A
Mean (SD) Examiner B
Right Left Right Left
22.96 22.81 23.96 23.47
(2.44) (2.12) (2.43) (1.73)
23.55 23.12 24.08 23.85
(2.20) (1.64) (2.51) (2.26)
AH UH AH UH
22.31 24.78 20.53 24.65
(11.01) (1.90) (11.82) (1.66)
20.72 24.84 18.61 24.48
(11.85) (1.82) (12.16) (1.87)
TMCT, total motor conduction time in milliseconds; right, right hemisphere; left, left hemisphere; AH, affected hemisphere; UH, unaffected hemisphere.
examiners identified 16 normal or delayed MEPs and 1 absent MEP at the first test and 15 normal and 2 absent MEPs at the retest in all patients with stroke, without conflicting measurements between them. The intraobserver reliability had a Cohen k of 0.638 for both examiners, with 1 conflicting measurement for MEP elicitation each. The ICC for interobserver reliability for TMCT at test and retest was 1.00. The ICCs for TMCT intraobserver reliability for examiners A and B were both 0.79. The intraand interobserver LOAs between test and retest are plotted for each examiner in Fig. 2. As this figure shows, the intraobserver LOA for examiner A was 2.73 milliseconds and that for examiner B 1.00 milliseconds, with mean differences of 0.14 and 0.37 milliseconds, respectively. Measurements were consistent between the 2 examiners, with LOAs of 2.66 and 1.90 milliseconds and mean differences of 20.07 and 20.97 milliseconds.
Intra- and Interobserver TMS Data in the Infarcted Hemisphere The interobserver reliability for the elicitation of an MEP by the 2 examiners at the first test had a Cohen k of 0.47, with 2 conflicting measurements. At the retest, the Cohen k was 0.85,
Baseline Characteristics of the Subjects
Age, mean 6 SD, years Gender: male/female, N Infarct localization Cortical Subcortical Affected hemisphere (right/left), N Dominant side (right/left) Duration of stroke, mean (range), months Modified Rankin score, median (range), months MRC score, median (range)
Healthy Subjects (N ¼ 8)
Patients With Stroke (N ¼ 18)
28.3 6 4.9 4/4
68.2 6 10.6 11/7
NA NA NA NA
18 0 8/10 18/0 3.5 (3–5)
NA
3 (2–4)
5
3 (0–4)
MRC score, muscle strength according to Medical Research Council; NA, non applicable.
558
Side
TABLE 3. Inter and Intraexaminer Reliability of TMCT of Healthy Subjects and Patients With Stroke With Respective ICC Values Measure/Examiner Patients with stroke TMCT A 1 B TMCT A TMCT B Healthy subjects TMCT A 1 B TMCT A TMCT B
Side
ICC
r
CI
P
Left/right Left/right Left/right
Inter Intra Intra
0.772 0.638 0.585
0.562–0.905 0.247–0.853 0.123–0.834
0.000 0.003 0.011
Left/right Left/right Left/right
Inter Intra Intra
0.965 0.896 0.956
0.913–0.992 0.704–0.976 0.871–0.990
0.000 0.000 0.000
Ratings for two examiners averaged together. TMCT, total conduction time in milliseconds; ICC, intraclass correlation coefficient; r (rho), ICC value; CI, confidence interval (lower–upper); P, significance level; inter, interexaminer reliability; intra, intraexaminer reliability.
Copyright Ó 2014 by the American Clinical Neurophysiology Society
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
Transcranial Magnetic Stimulation
FIG. 1. Distribution plots from Bland–Altman test showing mean TMCT (in milliseconds) against differences between measurements for intrarater reliability (A and B) and interrater reliability (C and D) of both hemispheres (N ¼ 16) in healthy subjects. A, Intratester measurements by examiner A. Mean difference: 20.83 milliseconds; LOA: 3.46 milliseconds. B, Intratester TMCT measurements by examiner B. Mean difference: 20.63 milliseconds; LOA: 2.18 milliseconds. C, Intertester measurements between examiner A and B at the test. Mean difference: 20.45 milliseconds; LOA: 2.72 milliseconds. D, Intertester measurements between examiner A and B at the retest. Mean difference: 20.25 milliseconds; LOA: 1.45 milliseconds. TMCT, total motor conduction time; LOA, limits of agreement.
with 1 conflicting measurement between the 2 examiners. Intraobserver reliability had a Cohen k of 0.82 for examiner A and 0.85 for examiner B, both with 1 conflicting measurement. The ICC for interobserver reliability for TMCT was 0.44 at the first test and 0.90 at the retest (ICC ¼ 0.90). The ICC for TMCT intraobserver reliability was 0.90 for both examiners. Intraobserver TMCT measurements of the infarcted hemisphere yielded 95% LOAs of 13.89 and 13.56 milliseconds and mean differences of 2.11 and 1.77 milliseconds for examiners A and B, respectively (Figs. 3A and 3B). The interobserver TMCT measurements yielded 95% LOAs of 27.11 and 13.56 milliseconds and a difference of, respectively, 2.11 and 1.77 milliseconds. Mean differences were 1.59 milliseconds for examiner A and 1.90 milliseconds for examiner B. Copyright Ó 2014 by the American Clinical Neurophysiology Society
DISCUSSION In general, the reliability of measurements between and within observers of TMS-induced MEPs was good to excellent. We also found good to excellent reliability of TMCT assessment in healthy volunteers and in the noninfarcted hemispheres of patients with stroke. In contrast, TMCT measurements in the infarcted hemisphere of patients with stroke were less consistent. Based on the lower reproducibility of TMCT measurements in the infarcted hemisphere, we recommend repeating TMCT measurements to improve intraobserver and interobserver reliability of tests. The LOAs for the measurements in the infarcted hemisphere differed by about 13 milliseconds. This difference means that, in practice, one examiner may measure a TMCT within normal limits, 559
M. H. W. J. Hoonhorst et al.
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
FIG. 2. Distribution plots from Bland–Altman test showing mean TMCT (in milliseconds) against differences between measurements for intrarater reliability (A and B) and interrater reliability (C and D) of the noninfarcted hemispheres (N ¼ 18) in patients with stroke. Dashed vertical line is the limit of normal TMCT of 23.3 milliseconds. A, Intratester measurement of examiner A. Mean difference: 0.14 milliseconds; LOA: 2.73 milliseconds. B, Intratester measurements of examiner B. Mean difference: 0.37 milliseconds; LOA: 1.00 milliseconds. C, Intertester measurements between examiners A and B at the test. Mean difference: 20.07 milliseconds; LOA: 2.66 milliseconds. D, Intertester measurements between examiners A and B at the retest. Mean difference: 20.97 milliseconds; LOA: 1.90 milliseconds. TMCT, total motor conduction time; LOA, limits of agreement. whereas another examiner measures a significantly “delayed” TMCT. In our opinion, this LOA is too wide to enable TMCT to be categorized into “normal” or “delayed” latencies in patients with stroke because of the overlap between the two categories. Instead, dichotomization between normal plus delayed TMCT versus the absence of TMCT probably makes more clinical sense. To the best of our knowledge, this is the most extensive reliability study of TMS in subacute stroke to include TMCT measurements of the noninfarcted hemisphere. The interobserver reliability of MEP induction in the noninfarcted hemisphere proved excellent in comparison with the intraobserver reliability. However, this finding could be caused by statistical coincidence between both observers. Sometimes, the delayed latencies of the noninfarcted hemisphere are remarkable and may be due to 560
dynamic changes in the pattern of brain activity in response to injury (Duque et al., 2005; Liepert et al., 2000a). Further analysis of this phenomenon might provide a valuable parameter for prognosis and follow-up after stroke. Measurements of TMCT demonstrated good to excellent ICCs, with good inter- and intraobserver consistency in all measurements. The outlier we found in the TMCT measurements of the infarcted hemisphere was caused by one examiner measuring a high TMCT, whereas the other was unable to measure any TMCT at all. In agreement with our TMCT results, other reports have also described mostly good ICCs for healthy volunteers (Livingston and Ingersoll, 2008; Malcolm et al., 2006) and for the test–retest reliability of TMCT of the lower limb in patients with chronic stroke (Cacchio et al., 2011; Wheaton et al., 2009). Copyright Ó 2014 by the American Clinical Neurophysiology Society
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
Transcranial Magnetic Stimulation
FIG. 3. Distribution plots from Bland–Altman test showing mean TMCT (in milliseconds) against differences between measurements for intrarater reliability (A and B) and interrater reliability (C and D) of the infarcted hemispheres (N ¼ 18) in patients with stroke. Dashed vertical line is the limit of normal TMCT of 23.3 milliseconds. A, Intratester measurements of examiner A. Mean difference: 2.11 milliseconds; LOA: 13.89 milliseconds. B, Intratester measurements of examiner B. Mean difference: 1.77 milliseconds; LOA: 13.56 milliseconds. C, Intertester measurements between examiners A and B at the test. Mean difference: 1.59 milliseconds; LOA: 27.11 milliseconds. D, Intertester measurements between examiners A and B at the retest. Mean difference: 1.90 milliseconds; LOA: 13.68 milliseconds. TMCT, total motor conduction time; LOA, limits of agreement.
There are a number of systematic and random sources of variability that may have influenced our results. Theoretically, one can distinguish between three sources of error that may have affected our results: errors induced by the examiners, errors induced by the participants, and errors caused by variations in the TMS equipment itself. However, we feel that this third error source is unimportant because we used the same equipment (apparatus and coil) for all experiments and calibrated it just before the start of the study. It should be noted that identification of an MEP by an examiner is more difficult with a figure-of-eight coil when compared with a circular coil because of the larger volleys produced (Thickbroom et al., 1999). The orientation of the coil above the skull to target the hotspot is crucial for inducing MEPs because the structure of the Copyright Ó 2014 by the American Clinical Neurophysiology Society
primary motor cortex consists of clusters of neurons that provide the activation of lower motor neurons, so different orientations may activate different sites of the neuronal structure (Di Lazzaro et al., 2004; Jung et al., 2010). Hence, inaccuracy of handheld orientation by the two examiners in our study may have added to the variability within and between them. In patients with stroke, the difficulty of detecting hotspots in the damaged brain due to altered electrophysiological properties could also contribute to the variability, because of possible temporal dispersion and shifts in the cortical motor representation (Byrnes et al., 1999; Liepert et al., 2000b). The influence of spontaneous fluctuations in electrophysiological excitability on the variability of TMCT is smaller and seems not to have been an issue in our study (Brasil-Neto et al., 1992; Ellaway et al., 561
M. H. W. J. Hoonhorst et al.
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
1998; Jung et al., 2010; Kiers et al., 1993; Livingston and Ingersoll, 2008; Schmidt et al., 2009; Sommer et al., 2002). This study was subjected to some limitations. First, the test– retest interval of 7 days was arbitrarily chosen, as a compromise between minimizing the risk of any possible motor recovery after stroke and minimizing the risk of possible recall by the examiners. The differences in reliability between the first test and the retest suggest a learning curve in both examiners. However, both examiners were experienced and had followed a prestudy training program. Second, our sample size was small and was restricted to healthy volunteers and those who had had their stroke atleast 3 months before this study. In addition, we included only those patients with a first-ever ischemic stroke. Third, we did not correct for age. As a consequence, comparisons with observed reliability data of healthy and stroke subjects should be interpreted with caution. Aging is associated with an altered capacity for processes that are important for synaptic plasticity after stroke (Fathi, 2010; Muller-Dahlhaus et al., 2008) and somatosensory conductive function (Onofrj et al., 2001; Tanosaki et al., 1999). Finally, we did not quantify the exact coil position on the skull, so we cannot be fully certain whether the coil moved during the handheld measurements. In conclusion, the reliability between and within examiners of TMS-induced MEPs measured was generally good to excellent. We also found good to excellent reliability of TMCT assessment in healthy participants and in the noninfarcted hemispheres of patients with stroke. In contrast, TMCT measurements in the infarcted hemisphere of patients with stroke were less consistent. Based on lower reproducibility of TMCT measurements and induced MEPs generated from the infarcted hemisphere, we recommend to repeat the TMS measurements to improve the reliability of tests within and between examiners.
ACKNOWLEDGMENTS The authors are grateful to all participants and especially to Johan Bisschop, Jan Middendorp, and other colleagues at the Department of Clinical Neurophysiology for performing the TMS measurements. REFERENCES Brasil-Neto JP, McShane LM, Fuhr P, et al. Topographic mapping of the human motor cortex with magnetic stimulation: factors affecting accuracy and reproducibility. Electroencephalogr Clin Neurophysiol 1992;85:9–16. Butler AJ, Kahn S, Wolf SL, Weiss P. Finger extensor variability in TMS parameters among chronic stroke patients. J Neuroeng Rehabil 2005;2:10. Byrnes ML, Thickbroom GW, Phillips BA, et al. Physiological studies of the corticomotor projection to the hand after subcortical stroke. Clin Neurophysiol 1999;110:487–498. Cacchio A, Paoloni M, Cimini N, et al. Reliability of TMS-related measures of tibialis anterior muscle in patients with chronic stroke and healthy subjects. J Neurol Sci 2011;303:90–94. Carroll TJ, Riek S, Carson RG. Reliability of the input-output properties of the cortico-spinal pathway obtained from transcranial magnetic and electrical stimulation. J Neurosci Methods 2001;112:193–202. Di Lazzaro V, Oliviero A, Pilato F, et al. The physiological basis of transcranial motor cortex stimulation in conscious humans. Clin Neurophysiol 2004;115:255–266. Duque J, Hummel F, Celnik P, et al. Transcallosal inhibition in chronic subcortical stroke. Neuroimage 2005;28:940–946. Ellaway PH, Davey NJ, Maskill DW, et al. Variability in the amplitude of skeletal muscle responses to magnetic stimulation of the motor cortex in man. Electroencephalogr Clin Neurophysiol 1998;109:104–113. Fathi D, Ueki Y, Mima T, et al. Effects of aging on the human motor cortical plasticity studied by paired associative stimulation. Clin Neurophysiol 2010;121:90–93.
562
Fleiss JL. Statistical Methods for Rates and Proportions. New York: Wiley, 1981:38–46. Fritz C, Braune HJ, Pylatiuk C, Pohl M. Silent period following transcranial magnetic stimulation: a study of intra- and inter-examiner reliability. Electroencephalogr Clin Neurophysiol 1997;105:235–240. Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Dis 1987;40:171–178. Jung NK, Delvendahl I, Kuhnke NG, et al. Navigated transcranial magnetic stimulation does not decrease the variability of motor-evoked potentials. Brain Stimul 2010;3:87–94. Kiers L, Cros D, Chiappa KH, Fang J. Variability of motor potentials evoked by transcranial magnetic stimulation. Electroencephalogr Clin Neurophysiol 1993;89:415–423. Koski L, Lin JC, Wu AD, Winstein JC. Reliability of intracortical and corticomotor excitability estimates obtained from the upper extremities in chronic stroke. Neurosci Res 2007;58:19–31. Langhorne P, Bernhardt J, Kwakkel G. Stroke rehabilitation. Lancet 2011;377:1693–1702. Liepert J, Hamzei F, Weiller C. Motor cortex disinhibition of the unaffected hemisphere after acute stroke. Muscle Nerve 2000a;23:1761–1763. Liepert J, Storch P, Fritsch A, Weiller C. Motor cortex disinhibition in acute stroke. Clin Neurophysiol 2000b;111:671–676. Livingston SC, Ingersoll CD. Intra-rater reliability of a transcranial magnetic stimulation technique to obtain motor evoked potentials. Int J Neurosci 2008;118:239–256. Malcolm MP, Triggs JC, Light KE, et al. Reliability of motor cortex transcranial magnetic stimulation in four muscle representations. Clin Neurophysiol 2006;117:1037–1046. Muller-Dahlhaus JF, Orekhov I, Liu Y, Ziemann U. Interindividual variability and age-dependency of motor cortical plasticity induced by paired associative stimulation. Exp Brain Res 2008;187:467–475. Nijland RH, van Weegen EE, Harmeling-van der Wel BC, Kwakkel G; EPOS Investigators. Presence of finger extension and shoulder abduction within 72 hours after stroke predicts functional recovery: early prediction of functional outcome after stroke: the EPOS cohort study. Stroke 2010;41:745–750. Onofrj M, Thomas A, Iacono D, et al. Age-related changes of evoked potentials. Neurophysiol Clin 2001;31:83–103. Paulus W, Tergau F, Nitsche MA, et al. Proceedings of the 2nd International Transcranial Magnetic Stimulation and Transcranial Direct Current Stimulation Symposium, Göttingen, Germany, 11–14 June, 2003. 2003;56, Clin Neurophysiol Series. New York: Elsevier Science & Technology. Pell GS, Roth Y, Zangen A. Modulation of cortical excitability induced by repetitive transcranial magnetic stimulation: influence of timing and geometrical parameters and underlying mechanisms. Prog Neurobiol 2011;93:59–98. Rossini PM, Barker AT, Berardelli A, et al. Non-invasive electrical and magnetic stimulation of the brain, spinal cord and roots: basic principles and procedures for routine clinical application. Report of an IFCN committee. Electroencephalogr Clin Neurophysiol 1994;91:79–92. Rothwell JC. Transcranial electrical and magnetic stimulation of the brain: basic physiological mechanisms. In: Hallett M, Chokroverty S, eds. Magnetic Stimulation in Clinical Neurophysiology. Philadelphia: Elsevier, 2005:43–60. Schmidt S, Cichy RM, Kraft A, et al. An initial transient-state and reliable measures of corticospinal excitability in TMS studies. Clin Neurophysiol 2009;120:987–993. Schulz R, Park CH, Boudrias MH, et al. Assessing the integrity of corticospinal pathways from primary and secondary cortical motor areas after stroke. Stroke 2012;43:2249–2251. Sommer M, Wu T, Tergau F, Paulus W. Intra- and interindividual variability of motor responses to repetitive transcranial magnetic stimulation. Clin Neurophysiol 2002;113:256–296. Stinear C. Prediction of recovery of motor function after stroke. Lancet Neurol 2010;9:1228–1232. Stinear C. The PREP algorithm predicts potential for upper limb recovery after stroke. Brain 2012;135:2527–2535. Streiner DI, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 2nd ed. Oxford: Oxford University Press, 2000;104. Tanosaki M, Ozaki I, Shimamura H, et al. Effects of aging on central conduction in somatosensory evoked potentials: evaluation of onset versus peak methods. Clin Neurophysiol 1999;110:2094–2103. Thickbroom GW, Byrnes ML, Mastaglia FL. Methodology and application of TMS mapping. Transcranial magnetic stimulation. Electroencephalogr Clin Neurophysiol Suppl 1999;51:4854. Truelsen T, Piechowski-Józwiak B, Bonita R, et al. Stroke incidence and prevalence in Europe: a review of available data. Eur J Neurol 2006;13:581–598. Veerbeek JM, Kwakkel G, van Wegen EE, et al. Early prediction of outcome of activities of daily living after stroke: a systematic review. Stroke 2011;42:1482–1488. Wheaton LA, Villagra F, Hanley DF, et al. Reliability of TMS motor evoked potentials in quadriceps of subjects with chronic hemiparesis after stroke. J Neurol Sci 2009;276:115–117. WHO. World Health Report 2003. Geneva: World Health Organization, 2003.
Copyright Ó 2014 by the American Clinical Neurophysiology Society
How Reproducible Are Transcranial Magnetic Stimulation–Induced MEPs in Subacute Stroke? Maurits H. W. J. Hoonhorst,* Boudewijn J. Kollen,† Peter S. P. van den Berg,‡ Cornelis H. Emmelot,§ and Gert Kwakkelk
Purpose: Motor evoked potentials (MEPs) and total motor conduction time (TMCT) induced by transcranial magnetic stimulation (TMS) are used to make assumptions about the prognosis of motor outcome after stroke. Understanding the different sources of variability is fundamental to the concept of reliability. Reliability testing of TMS-MEPs and TMCTs within and between two independent examiners in healthy and stroke subjects is still an unexplored field in the clinical neurophysiology. Assessing the reproducibility of TMS measurements requires studies to investigate the test–retest reliability of TMS-induced MEPs and TMCT. The authors set out to test the reliability of these TMS measurements. Methods: Eighteen patients with stroke and 8 healthy volunteers were tested twice within a 1-week period by 2 examiners using TMS to determine MEPs and TMCT for the abductor pollicis brevis muscle of their affected and unaffected hands. Results: The authors found moderate to perfect reliability of TMS-induced MEPs in healthy volunteers, noninfarcted hemispheres (perfect agreement), and infarcted hemispheres (Kappa’s ¼ 0.45–0.87). Reliability of TMCT was good to excellent in the volunteers (intraclass correlation coefficients ¼ 0.77–0.97), excellent in the noninfarcted hemispheres (intraclass correlation coefficients ¼ 0.97–1.00), and poor to excellent in the infarcted hemispheres (intraclass correlation coefficients ¼ 0.44–0.90). Conclusions: The reliability of TMS-induced MEPs and TMCT measurements in healthy volunteers and the noninfarcted hemisphere of patients with stroke with an upper paretic limb was good to excellent. In contrast, TMS measurements in the infarcted hemisphere were less consistent. Based on the lower reproducibility of TMCT measurements in the infarcted hemisphere, we recommend to repeat the TMCT measurements to improve the reliability of tests. Key Words: Stroke, Transcranial magnetic stimulation, Reliability, Reproducibility, Motor evoked potential, TMS. (J Clin Neurophysiol 2014;31: 556–562)
S
troke is a major public health concern, with more than 1.1 million stroke events in Europe in the year 2000, and is the most common cause of acquired disability among adults in developed countries (Truelsen et al., 2006; WHO, 2003). Early prediction of outcome
From the *Department of NeuroRehabilitation, Vogellanden Center for Rehabilitation, Zwolle, the Netherlands; †Department of General Practice, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands; Departments of ‡Neurology and §Rehabilitation Medicine, Isala Klinieken, Zwolle, the Netherlands; and kDepartment of Rehabilitation Medicine, MOVE Research Institute Amsterdam, VU University Medical Center, Amsterdam, the Netherlands. This study was supported by Grant No. 154 from the Zwols Wetenschapsfonds Isala Klinieken, the Netherlands. Address correspondence and reprint requests to Maurits H. W. J. Hoonhorst, MD, Rehabilitation Centre De Vogellanden, Hyacinthstraat 66A, 8013 XZ Zwolle, the Netherlands; e-mail: [email protected]. Copyright Ó 2014 by the American Clinical Neurophysiology Society
ISSN: 0736-0258/14/3106-0556
556
after stroke is paramount because it allows professionals to (1) set realistic, attainable treatment goals, (2) select patients who will benefit most from a particular intervention, (3) increase the efficiency of stroke services and allow adequate triage, (4) anticipate early supported discharge including the need for home adjustments and community support, and (5) give optimal information to patients and their family. Understanding prognosis is also paramount for researchers, enabling them to improve future trials by adequate stratification of patients and so generate prognostically comparable patient groups (Veerbeek et al., 2011). Finger extension and shoulder abduction measured at 72 hours have been found to be essential for the prediction of upper limb recovery 6 months after stroke (Nijland et al., 2010; Stinear, 2010). Recently, it has been suggested that transcranial magnetic stimulation (TMS), a noninvasive method to test the functional integrity of the corticospinal tract system (Rothwell, 2005), may offer added value in predicting the final outcome for the paretic upper limb (Langhorne et al., 2011; Stinear, 2012). With that, the presence or absence of TMS-induced motor evoked potentials (MEPs) may offer clinicians an added valid diagnostic tool to just clinical assessment. Nevertheless, TMS of the ipsilesional motor cortex (M1) to elicit MEPs in the affected upper limb is not yet routinely used. Assessing the reliability of TMS measurements requires studies to investigate the test–retest reliability of TMS-induced MEPs (Guyatt, 1987). However, measurements of MEPs and total motor conduction time (TMCT) induced by TMS suffer from random and systematic error. Understanding these different sources of random and systematic variability is fundamental to the concept of reliability (Streiner and Norman, 2000). Despite its essential role in psychometrics, reliability testing of TMS-MEPs and TMCTs within and between examiners in healthy and stroke subjects is still an unexplored field in the clinical neurophysiology. Agreement between examiners about TMS-induced MEPs in both healthy volunteers and patients with stroke is an essential step in determining the validity of TMS as a method to assess the integrity of the corticospinal tract (Schulz et al., 2012). Replication of MEPs is imperative to demonstrate clinical pathology and exclude abnormalities due to error. Important factors for accurate replication include shape of the TMS coil, orientation of the stimulating coil, subject-related sources of variability and accuracy of observation (Pell, 2011). To date, the issue of the reliability of standard TMS techniques to elicit MEPs in healthy and stroke volunteers has only been addressed by a small number of investigations. These studies found that intraclass correlation coefficients (ICCs) for MEP latencies and motor thresholds within and between observers for healthy volunteers ranged from 0.60 to 0.92, whereas MEP amplitudes proved less consistent (Livingston and Ingersoll, 2008). Similar results were found in an intraobserver reliability study among 20 volunteers assessing motor threshold, map area, and stimulus–response curves (Malcolm et al.,
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
2006), and in reliability studies assessing silent period (Fritz et al., 1997) and MEP size (Carroll et al., 2001). Yet, there is almost no evidence on the test–retest reliability of MEP parameters of the paretic and nonparetic upper limbs after stroke. One reliability study involving 9 patients with stroke showed that ICCs were above 0.70 for parameters such as MEP amplitude and silent period (Koski et al., 2007), whereas a study by Butler et al. (2005) in 10 patients with stroke found that the group means of MEP amplitude, motor threshold, and map sizes did not change when tested 3 times within 1 week (Butler et al., 2005). The objective of this study was to establish the intra- and interobserver reliability for two examiners of TMS-induced MEPs in the hemispheres of patients with stroke and healthy volunteers. Our second aim was to establish the intra- and interobserver reliability for two laboratory examiners of TMS-induced TMCT in the infarcted and noninfarcted hemispheres of patients with stroke and healthy volunteers.
SUBJECTS AND METHODS Subjects This study was approved by the local Medical Ethics Committee of the Isala Klinieken Hospital. Written informed consent was obtained from all participants. The patients with stroke were recruited from the Department of Neurology of the Isala Klinieken and the volunteers from the staff of this department. The following inclusion and exclusion criteria were used: (1) first-ever ischemic hemispheric stroke more than 3 months before testing, as confirmed by computed tomography or MRI scan, (2) unilateral paralysis or paresis of the upper limb, (3) score on the modified Rankin scale ranging from 2 to 4 points, (4) absence of peripheral nerve pathology including diabetes and neuromuscular disease, and (5) no history of head injury, cranial surgery, epilepsy, cervical spine stenosis, or other contraindications for TMS. Finally, none of the participants were allowed to use pharmaceutical agents that could influence MEPs in a neuromodulatory way, such as modulators of NMDA and GABA receptors, epilepsy medications, or dopaminergic agonists.
Laboratory Examiners Two experienced laboratory examiners were selected for this reliability study. Both were instructed to independently perform TMSinduced MEPs in all participants and to analyze TMCT. Accurate electrode placement in the target muscle was checked for by visual inspection of the compound motor action potential of the abductor pollicis brevis muscle (Rossini et al., 1994). The test–retest interval was set at 7 days. Each examiner was fully blinded for the general medical evaluation and also for the other’s data. Clinical evaluation was performed by a neurologist (P.S.P.v.d.B.), who was blinded for the MEP data. Neurologic examination included quantification of the degree of paresis and disability after stroke. No effort was made to control for the amount of motor activity performed between sessions because none of the participants were engaged in a rehabilitation program during the study, and no recovery was expected between test and retest.
Transcranial Magnetic Stimulation
2003; Rossini et al., 1994). Participants were seated in a quiet temperature-controlled room and measurements took place in the afternoon. The infarcted and noninfarcted hemispheres in patients with stroke and both hemispheres in the volunteers were assessed. Motor evoked potentials were recorded from the abductor pollicis brevis muscle by surface electrodes, using a conventional electromyography system. The motor cortex was stimulated with a calibrated Magstim Dantec Maglite (Dantec Dynamics, Bristol, United Kingdom). Cortical (single-pulse) TMS was performed with a figure-ofeight shaped coil and cervical stimulation with a circular coil with an outer diameter of 9 cm. Hand motor activation was optimized by positioning the coil on the scalp at an angle of 458 away from the midsagittal line. First, the spot that produced the highest MEP amplitude of the abductor pollicis brevis muscle (hotspot) was determined by moving the coil over the scalp in the area of M1 of the noninfarcted hemisphere, with the stimulator at submaximal output. Subsequently, the coil positioning at the hotspot was mirrored to the infarcted hemisphere to elicit an MEP. The absence of response to TMS was recorded if no response was obtained after three of the six successive discharges at maximal intensity. When no MEP could be elicited at a given position, the coil was moved slightly to find a hotspot in adjoining sites. The criterion for the presence of an electromyogram response to TMS was a negative slope $ 50 mV, and latencies were determined by visual inspection of 3 successive MEPs (Paulus et al., 2003; Rossini et al., 1994). The latency data were then used to calculate TMCT. Thereafter, MEPs were divided into two groups: present or absent MEP. Present MEP was defined as any delayed or normal MEP. Delayed MEP was defined as a TMCT of $ 23.3 milliseconds (Di Lazzaro et al., 2004; Rossini et al., 1994). Because voluntary muscle contraction reduces the motor threshold and increases fluctuations in spinal excitability, resulting in greater variation in the electromyogram muscle response, all recordings were performed with the arm and hand muscles relaxed, to obtain homogeneous data and to avoid variations of the muscle response due to different levels of preinnervation (Di Lazzaro et al., 2004; Rossini et al., 1994).
Design and Statistical Analysis We investigated both interobserver and intraobserver reliability for TMS-induced MEPs and TMCTs. Cohen k was used to compute the interobserver reliability of inducing normal and delayed MEPs. Kappa scores were categorized with values .0.75 recorded as excellent, 0.40 to 0.75 as fair to good, and ,0.40 as poor (Fleiss, 1981). The limits of agreement (LOA) were calculated for intra- and interobserver reliability of TMCT. Intraobserver reliability was assessed by comparing the mean TMCT values from the initial test with the mean values on the retest, and calculating the LOA. Interobserver reliability was assessed by calculating the mean value for test and retest for each examiner and comparing the results between the two examiners, referred to as A and B. Intra- and interobserver reliability of TMCT were calculated using the ICC model (Fleiss, 1981). Systematic differences between and within examiners were tested using independent and paired Student t-tests. The a-level was set at 0.05 for each test. SPSS software (SPSS Version 15.0, IBM Corporation, Armonk, New York, United States) was used for the statistical analyses.
TMS Protocol The TMS technique, including the hotspot method and measurements of MEP latencies with electromyogram recording (Nihon Kohden Neuropack 8, Nihon Kohden Corporation, Tokyo, Japan), was performed according to the recommendations of the International Federation of Clinical Neurophysiology (Paulus et al., Copyright Ó 2014 by the American Clinical Neurophysiology Society
RESULTS Eighteen patients with stroke and eight volunteers participated in the study. All participants tolerated the TMS well, without any adverse events. Measurable MEPs were obtained from all participants, except for the noninfarcted hemisphere of one patient with 557
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
M. H. W. J. Hoonhorst et al.
stroke, whose MEP could not be obtained due to brachial plexus pathology. Table 1 shows the characteristics of the healthy and stroke participants. The volunteers were significantly younger, with a mean age of 28.3 6 4.9 years, compared with the patients with a mean age of 68.2 6 10.6 years. Table 2 presents the test–retest data on TMCT for all participants. For examiner A, the mean TMCT for the infarcted hemisphere in patients with stroke was 22.31 6 11.01 milliseconds at the first test and 20.53 6 11.82 milliseconds at the retest. For examiner B, the mean TMCT of the infarcted hemisphere was 20.72 6 11.85 milliseconds at the test and 18.61 6 12.16 milliseconds at the retest. Peripheral motor conduction time for all patients with stroke and volunteers showed latency times within the normal range of ,18 milliseconds.
TABLE 2. Comparison of Mean TMCT Form Both Hemispheres in All Subjects at the Test and Retest by Both Examiners Measure Healthy subjects (N ¼ 8) Test Retest Patients with stroke (N ¼ 18) Test Retest
Intra- and Interobserver TMS Data in the Healthy Volunteers Both the examiners identified only normal MEPs in all volunteers, resulting in perfect agreement. The ICC results for test–retest reliability of all MEP latencies are summarized in Table 3. Intraclass correlation coefficients for the interobserver reliability of TMCTs at the first test were 0.87 and 0.97 at the retest. Intraclass correlation coefficients of the intraobserver reliability of TMCTs were 0.77 for examiner A and 0.91 for examiner B. Fig. 1 shows the LOA for the first test and retest measurements in the volunteers obtained by the two examiners. The intraobserver LOA values between test and retest are plotted for each examiner in Figs. 1A and 1B, which show LOAs of 2.18 and 3.46 milliseconds and mean differences of 20.83 and 20.63 milliseconds. The interobserver LOAs for TMCT between the examiners are displayed in Figs. 1C and 1D. The intraobserver reliability shows a LOA of 1.45 milliseconds and a mean difference of 20.25 milliseconds at the retest, compared with a LOA of 2.72 milliseconds and a mean difference of 20.45 milliseconds at the first test.
Intra- and Interobserver TMS Data of the Noninfarcted Hemisphere The interobserver for inducing an MEP by the 2 examiners showed a perfect Cohen k (1.00) at the test and retest. Both
TABLE 1.
Mean (SD) Examiner A
Mean (SD) Examiner B
Right Left Right Left
22.96 22.81 23.96 23.47
(2.44) (2.12) (2.43) (1.73)
23.55 23.12 24.08 23.85
(2.20) (1.64) (2.51) (2.26)
AH UH AH UH
22.31 24.78 20.53 24.65
(11.01) (1.90) (11.82) (1.66)
20.72 24.84 18.61 24.48
(11.85) (1.82) (12.16) (1.87)
TMCT, total motor conduction time in milliseconds; right, right hemisphere; left, left hemisphere; AH, affected hemisphere; UH, unaffected hemisphere.
examiners identified 16 normal or delayed MEPs and 1 absent MEP at the first test and 15 normal and 2 absent MEPs at the retest in all patients with stroke, without conflicting measurements between them. The intraobserver reliability had a Cohen k of 0.638 for both examiners, with 1 conflicting measurement for MEP elicitation each. The ICC for interobserver reliability for TMCT at test and retest was 1.00. The ICCs for TMCT intraobserver reliability for examiners A and B were both 0.79. The intraand interobserver LOAs between test and retest are plotted for each examiner in Fig. 2. As this figure shows, the intraobserver LOA for examiner A was 2.73 milliseconds and that for examiner B 1.00 milliseconds, with mean differences of 0.14 and 0.37 milliseconds, respectively. Measurements were consistent between the 2 examiners, with LOAs of 2.66 and 1.90 milliseconds and mean differences of 20.07 and 20.97 milliseconds.
Intra- and Interobserver TMS Data in the Infarcted Hemisphere The interobserver reliability for the elicitation of an MEP by the 2 examiners at the first test had a Cohen k of 0.47, with 2 conflicting measurements. At the retest, the Cohen k was 0.85,
Baseline Characteristics of the Subjects
Age, mean 6 SD, years Gender: male/female, N Infarct localization Cortical Subcortical Affected hemisphere (right/left), N Dominant side (right/left) Duration of stroke, mean (range), months Modified Rankin score, median (range), months MRC score, median (range)
Healthy Subjects (N ¼ 8)
Patients With Stroke (N ¼ 18)
28.3 6 4.9 4/4
68.2 6 10.6 11/7
NA NA NA NA
18 0 8/10 18/0 3.5 (3–5)
NA
3 (2–4)
5
3 (0–4)
MRC score, muscle strength according to Medical Research Council; NA, non applicable.
558
Side
TABLE 3. Inter and Intraexaminer Reliability of TMCT of Healthy Subjects and Patients With Stroke With Respective ICC Values Measure/Examiner Patients with stroke TMCT A 1 B TMCT A TMCT B Healthy subjects TMCT A 1 B TMCT A TMCT B
Side
ICC
r
CI
P
Left/right Left/right Left/right
Inter Intra Intra
0.772 0.638 0.585
0.562–0.905 0.247–0.853 0.123–0.834
0.000 0.003 0.011
Left/right Left/right Left/right
Inter Intra Intra
0.965 0.896 0.956
0.913–0.992 0.704–0.976 0.871–0.990
0.000 0.000 0.000
Ratings for two examiners averaged together. TMCT, total conduction time in milliseconds; ICC, intraclass correlation coefficient; r (rho), ICC value; CI, confidence interval (lower–upper); P, significance level; inter, interexaminer reliability; intra, intraexaminer reliability.
Copyright Ó 2014 by the American Clinical Neurophysiology Society
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
Transcranial Magnetic Stimulation
FIG. 1. Distribution plots from Bland–Altman test showing mean TMCT (in milliseconds) against differences between measurements for intrarater reliability (A and B) and interrater reliability (C and D) of both hemispheres (N ¼ 16) in healthy subjects. A, Intratester measurements by examiner A. Mean difference: 20.83 milliseconds; LOA: 3.46 milliseconds. B, Intratester TMCT measurements by examiner B. Mean difference: 20.63 milliseconds; LOA: 2.18 milliseconds. C, Intertester measurements between examiner A and B at the test. Mean difference: 20.45 milliseconds; LOA: 2.72 milliseconds. D, Intertester measurements between examiner A and B at the retest. Mean difference: 20.25 milliseconds; LOA: 1.45 milliseconds. TMCT, total motor conduction time; LOA, limits of agreement.
with 1 conflicting measurement between the 2 examiners. Intraobserver reliability had a Cohen k of 0.82 for examiner A and 0.85 for examiner B, both with 1 conflicting measurement. The ICC for interobserver reliability for TMCT was 0.44 at the first test and 0.90 at the retest (ICC ¼ 0.90). The ICC for TMCT intraobserver reliability was 0.90 for both examiners. Intraobserver TMCT measurements of the infarcted hemisphere yielded 95% LOAs of 13.89 and 13.56 milliseconds and mean differences of 2.11 and 1.77 milliseconds for examiners A and B, respectively (Figs. 3A and 3B). The interobserver TMCT measurements yielded 95% LOAs of 27.11 and 13.56 milliseconds and a difference of, respectively, 2.11 and 1.77 milliseconds. Mean differences were 1.59 milliseconds for examiner A and 1.90 milliseconds for examiner B. Copyright Ó 2014 by the American Clinical Neurophysiology Society
DISCUSSION In general, the reliability of measurements between and within observers of TMS-induced MEPs was good to excellent. We also found good to excellent reliability of TMCT assessment in healthy volunteers and in the noninfarcted hemispheres of patients with stroke. In contrast, TMCT measurements in the infarcted hemisphere of patients with stroke were less consistent. Based on the lower reproducibility of TMCT measurements in the infarcted hemisphere, we recommend repeating TMCT measurements to improve intraobserver and interobserver reliability of tests. The LOAs for the measurements in the infarcted hemisphere differed by about 13 milliseconds. This difference means that, in practice, one examiner may measure a TMCT within normal limits, 559
M. H. W. J. Hoonhorst et al.
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
FIG. 2. Distribution plots from Bland–Altman test showing mean TMCT (in milliseconds) against differences between measurements for intrarater reliability (A and B) and interrater reliability (C and D) of the noninfarcted hemispheres (N ¼ 18) in patients with stroke. Dashed vertical line is the limit of normal TMCT of 23.3 milliseconds. A, Intratester measurement of examiner A. Mean difference: 0.14 milliseconds; LOA: 2.73 milliseconds. B, Intratester measurements of examiner B. Mean difference: 0.37 milliseconds; LOA: 1.00 milliseconds. C, Intertester measurements between examiners A and B at the test. Mean difference: 20.07 milliseconds; LOA: 2.66 milliseconds. D, Intertester measurements between examiners A and B at the retest. Mean difference: 20.97 milliseconds; LOA: 1.90 milliseconds. TMCT, total motor conduction time; LOA, limits of agreement. whereas another examiner measures a significantly “delayed” TMCT. In our opinion, this LOA is too wide to enable TMCT to be categorized into “normal” or “delayed” latencies in patients with stroke because of the overlap between the two categories. Instead, dichotomization between normal plus delayed TMCT versus the absence of TMCT probably makes more clinical sense. To the best of our knowledge, this is the most extensive reliability study of TMS in subacute stroke to include TMCT measurements of the noninfarcted hemisphere. The interobserver reliability of MEP induction in the noninfarcted hemisphere proved excellent in comparison with the intraobserver reliability. However, this finding could be caused by statistical coincidence between both observers. Sometimes, the delayed latencies of the noninfarcted hemisphere are remarkable and may be due to 560
dynamic changes in the pattern of brain activity in response to injury (Duque et al., 2005; Liepert et al., 2000a). Further analysis of this phenomenon might provide a valuable parameter for prognosis and follow-up after stroke. Measurements of TMCT demonstrated good to excellent ICCs, with good inter- and intraobserver consistency in all measurements. The outlier we found in the TMCT measurements of the infarcted hemisphere was caused by one examiner measuring a high TMCT, whereas the other was unable to measure any TMCT at all. In agreement with our TMCT results, other reports have also described mostly good ICCs for healthy volunteers (Livingston and Ingersoll, 2008; Malcolm et al., 2006) and for the test–retest reliability of TMCT of the lower limb in patients with chronic stroke (Cacchio et al., 2011; Wheaton et al., 2009). Copyright Ó 2014 by the American Clinical Neurophysiology Society
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
Transcranial Magnetic Stimulation
FIG. 3. Distribution plots from Bland–Altman test showing mean TMCT (in milliseconds) against differences between measurements for intrarater reliability (A and B) and interrater reliability (C and D) of the infarcted hemispheres (N ¼ 18) in patients with stroke. Dashed vertical line is the limit of normal TMCT of 23.3 milliseconds. A, Intratester measurements of examiner A. Mean difference: 2.11 milliseconds; LOA: 13.89 milliseconds. B, Intratester measurements of examiner B. Mean difference: 1.77 milliseconds; LOA: 13.56 milliseconds. C, Intertester measurements between examiners A and B at the test. Mean difference: 1.59 milliseconds; LOA: 27.11 milliseconds. D, Intertester measurements between examiners A and B at the retest. Mean difference: 1.90 milliseconds; LOA: 13.68 milliseconds. TMCT, total motor conduction time; LOA, limits of agreement.
There are a number of systematic and random sources of variability that may have influenced our results. Theoretically, one can distinguish between three sources of error that may have affected our results: errors induced by the examiners, errors induced by the participants, and errors caused by variations in the TMS equipment itself. However, we feel that this third error source is unimportant because we used the same equipment (apparatus and coil) for all experiments and calibrated it just before the start of the study. It should be noted that identification of an MEP by an examiner is more difficult with a figure-of-eight coil when compared with a circular coil because of the larger volleys produced (Thickbroom et al., 1999). The orientation of the coil above the skull to target the hotspot is crucial for inducing MEPs because the structure of the Copyright Ó 2014 by the American Clinical Neurophysiology Society
primary motor cortex consists of clusters of neurons that provide the activation of lower motor neurons, so different orientations may activate different sites of the neuronal structure (Di Lazzaro et al., 2004; Jung et al., 2010). Hence, inaccuracy of handheld orientation by the two examiners in our study may have added to the variability within and between them. In patients with stroke, the difficulty of detecting hotspots in the damaged brain due to altered electrophysiological properties could also contribute to the variability, because of possible temporal dispersion and shifts in the cortical motor representation (Byrnes et al., 1999; Liepert et al., 2000b). The influence of spontaneous fluctuations in electrophysiological excitability on the variability of TMCT is smaller and seems not to have been an issue in our study (Brasil-Neto et al., 1992; Ellaway et al., 561
M. H. W. J. Hoonhorst et al.
Journal of Clinical Neurophysiology Volume 31, Number 6, December 2014
1998; Jung et al., 2010; Kiers et al., 1993; Livingston and Ingersoll, 2008; Schmidt et al., 2009; Sommer et al., 2002). This study was subjected to some limitations. First, the test– retest interval of 7 days was arbitrarily chosen, as a compromise between minimizing the risk of any possible motor recovery after stroke and minimizing the risk of possible recall by the examiners. The differences in reliability between the first test and the retest suggest a learning curve in both examiners. However, both examiners were experienced and had followed a prestudy training program. Second, our sample size was small and was restricted to healthy volunteers and those who had had their stroke atleast 3 months before this study. In addition, we included only those patients with a first-ever ischemic stroke. Third, we did not correct for age. As a consequence, comparisons with observed reliability data of healthy and stroke subjects should be interpreted with caution. Aging is associated with an altered capacity for processes that are important for synaptic plasticity after stroke (Fathi, 2010; Muller-Dahlhaus et al., 2008) and somatosensory conductive function (Onofrj et al., 2001; Tanosaki et al., 1999). Finally, we did not quantify the exact coil position on the skull, so we cannot be fully certain whether the coil moved during the handheld measurements. In conclusion, the reliability between and within examiners of TMS-induced MEPs measured was generally good to excellent. We also found good to excellent reliability of TMCT assessment in healthy participants and in the noninfarcted hemispheres of patients with stroke. In contrast, TMCT measurements in the infarcted hemisphere of patients with stroke were less consistent. Based on lower reproducibility of TMCT measurements and induced MEPs generated from the infarcted hemisphere, we recommend to repeat the TMS measurements to improve the reliability of tests within and between examiners.
ACKNOWLEDGMENTS The authors are grateful to all participants and especially to Johan Bisschop, Jan Middendorp, and other colleagues at the Department of Clinical Neurophysiology for performing the TMS measurements. REFERENCES Brasil-Neto JP, McShane LM, Fuhr P, et al. Topographic mapping of the human motor cortex with magnetic stimulation: factors affecting accuracy and reproducibility. Electroencephalogr Clin Neurophysiol 1992;85:9–16. Butler AJ, Kahn S, Wolf SL, Weiss P. Finger extensor variability in TMS parameters among chronic stroke patients. J Neuroeng Rehabil 2005;2:10. Byrnes ML, Thickbroom GW, Phillips BA, et al. Physiological studies of the corticomotor projection to the hand after subcortical stroke. Clin Neurophysiol 1999;110:487–498. Cacchio A, Paoloni M, Cimini N, et al. Reliability of TMS-related measures of tibialis anterior muscle in patients with chronic stroke and healthy subjects. J Neurol Sci 2011;303:90–94. Carroll TJ, Riek S, Carson RG. Reliability of the input-output properties of the cortico-spinal pathway obtained from transcranial magnetic and electrical stimulation. J Neurosci Methods 2001;112:193–202. Di Lazzaro V, Oliviero A, Pilato F, et al. The physiological basis of transcranial motor cortex stimulation in conscious humans. Clin Neurophysiol 2004;115:255–266. Duque J, Hummel F, Celnik P, et al. Transcallosal inhibition in chronic subcortical stroke. Neuroimage 2005;28:940–946. Ellaway PH, Davey NJ, Maskill DW, et al. Variability in the amplitude of skeletal muscle responses to magnetic stimulation of the motor cortex in man. Electroencephalogr Clin Neurophysiol 1998;109:104–113. Fathi D, Ueki Y, Mima T, et al. Effects of aging on the human motor cortical plasticity studied by paired associative stimulation. Clin Neurophysiol 2010;121:90–93.
562
Fleiss JL. Statistical Methods for Rates and Proportions. New York: Wiley, 1981:38–46. Fritz C, Braune HJ, Pylatiuk C, Pohl M. Silent period following transcranial magnetic stimulation: a study of intra- and inter-examiner reliability. Electroencephalogr Clin Neurophysiol 1997;105:235–240. Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Dis 1987;40:171–178. Jung NK, Delvendahl I, Kuhnke NG, et al. Navigated transcranial magnetic stimulation does not decrease the variability of motor-evoked potentials. Brain Stimul 2010;3:87–94. Kiers L, Cros D, Chiappa KH, Fang J. Variability of motor potentials evoked by transcranial magnetic stimulation. Electroencephalogr Clin Neurophysiol 1993;89:415–423. Koski L, Lin JC, Wu AD, Winstein JC. Reliability of intracortical and corticomotor excitability estimates obtained from the upper extremities in chronic stroke. Neurosci Res 2007;58:19–31. Langhorne P, Bernhardt J, Kwakkel G. Stroke rehabilitation. Lancet 2011;377:1693–1702. Liepert J, Hamzei F, Weiller C. Motor cortex disinhibition of the unaffected hemisphere after acute stroke. Muscle Nerve 2000a;23:1761–1763. Liepert J, Storch P, Fritsch A, Weiller C. Motor cortex disinhibition in acute stroke. Clin Neurophysiol 2000b;111:671–676. Livingston SC, Ingersoll CD. Intra-rater reliability of a transcranial magnetic stimulation technique to obtain motor evoked potentials. Int J Neurosci 2008;118:239–256. Malcolm MP, Triggs JC, Light KE, et al. Reliability of motor cortex transcranial magnetic stimulation in four muscle representations. Clin Neurophysiol 2006;117:1037–1046. Muller-Dahlhaus JF, Orekhov I, Liu Y, Ziemann U. Interindividual variability and age-dependency of motor cortical plasticity induced by paired associative stimulation. Exp Brain Res 2008;187:467–475. Nijland RH, van Weegen EE, Harmeling-van der Wel BC, Kwakkel G; EPOS Investigators. Presence of finger extension and shoulder abduction within 72 hours after stroke predicts functional recovery: early prediction of functional outcome after stroke: the EPOS cohort study. Stroke 2010;41:745–750. Onofrj M, Thomas A, Iacono D, et al. Age-related changes of evoked potentials. Neurophysiol Clin 2001;31:83–103. Paulus W, Tergau F, Nitsche MA, et al. Proceedings of the 2nd International Transcranial Magnetic Stimulation and Transcranial Direct Current Stimulation Symposium, Göttingen, Germany, 11–14 June, 2003. 2003;56, Clin Neurophysiol Series. New York: Elsevier Science & Technology. Pell GS, Roth Y, Zangen A. Modulation of cortical excitability induced by repetitive transcranial magnetic stimulation: influence of timing and geometrical parameters and underlying mechanisms. Prog Neurobiol 2011;93:59–98. Rossini PM, Barker AT, Berardelli A, et al. Non-invasive electrical and magnetic stimulation of the brain, spinal cord and roots: basic principles and procedures for routine clinical application. Report of an IFCN committee. Electroencephalogr Clin Neurophysiol 1994;91:79–92. Rothwell JC. Transcranial electrical and magnetic stimulation of the brain: basic physiological mechanisms. In: Hallett M, Chokroverty S, eds. Magnetic Stimulation in Clinical Neurophysiology. Philadelphia: Elsevier, 2005:43–60. Schmidt S, Cichy RM, Kraft A, et al. An initial transient-state and reliable measures of corticospinal excitability in TMS studies. Clin Neurophysiol 2009;120:987–993. Schulz R, Park CH, Boudrias MH, et al. Assessing the integrity of corticospinal pathways from primary and secondary cortical motor areas after stroke. Stroke 2012;43:2249–2251. Sommer M, Wu T, Tergau F, Paulus W. Intra- and interindividual variability of motor responses to repetitive transcranial magnetic stimulation. Clin Neurophysiol 2002;113:256–296. Stinear C. Prediction of recovery of motor function after stroke. Lancet Neurol 2010;9:1228–1232. Stinear C. The PREP algorithm predicts potential for upper limb recovery after stroke. Brain 2012;135:2527–2535. Streiner DI, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 2nd ed. Oxford: Oxford University Press, 2000;104. Tanosaki M, Ozaki I, Shimamura H, et al. Effects of aging on central conduction in somatosensory evoked potentials: evaluation of onset versus peak methods. Clin Neurophysiol 1999;110:2094–2103. Thickbroom GW, Byrnes ML, Mastaglia FL. Methodology and application of TMS mapping. Transcranial magnetic stimulation. Electroencephalogr Clin Neurophysiol Suppl 1999;51:4854. Truelsen T, Piechowski-Józwiak B, Bonita R, et al. Stroke incidence and prevalence in Europe: a review of available data. Eur J Neurol 2006;13:581–598. Veerbeek JM, Kwakkel G, van Wegen EE, et al. Early prediction of outcome of activities of daily living after stroke: a systematic review. Stroke 2011;42:1482–1488. Wheaton LA, Villagra F, Hanley DF, et al. Reliability of TMS motor evoked potentials in quadriceps of subjects with chronic hemiparesis after stroke. J Neurol Sci 2009;276:115–117. WHO. World Health Report 2003. Geneva: World Health Organization, 2003.
Copyright Ó 2014 by the American Clinical Neurophysiology Society
