Visual system 441

Eye–hand coordination in on-line visuomotor adjustments Naotoshi Abekawaa,b, Toshio Inuib and Hiroaki Gomia When we perform a visually guided reaching action, the brain coordinates our hand and eye movements. Eye–hand coordination has been examined widely, but it remains unclear whether the hand and eye motor systems are coordinated during on-line visuomotor adjustments induced by a target jump during a reaching movement. As such quick motor responses are required when we interact with dynamic environments, eye and hand movements could be coordinated even during on-line motor control. Here, we examine the relationship between online hand adjustment and saccadic eye movement. In contrast to the well-known temporal order of eye and hand initiations where the hand follows the eyes, we found that on-line hand adjustment was initiated before the saccade onset. Despite this order reversal, a correlation between hand and saccade latencies was observed, suggesting that the hand motor system is not independent of eye control even when the hand response was induced before the saccade. Moreover, the latency of the hand adjustment with saccadic eye movement was significantly shorter than that with eye fixation. This hand latency modulation cannot be ascribed

to any changes of visual or oculomotor reafferent information as the saccade was not yet initiated when the hand adjustment started. Taken together, the hand motor system would receive preparation signals rather than reafference signals of saccadic eye movements to provide quick manual adjustments of the goal-directed c 2014 eye–hand movements. NeuroReport 25:441–445 Wolters Kluwer Health | Lippincott Williams & Wilkins.

Introduction

such as hitting a ball or catching an insect, there is a need for quick on-line adjustments of the reach with eye movements. However, there has been little investigation of eye–hand coordination during on-line movement control.

Hand movements toward visual targets are typically preceded by eye movements. Numerous lines of evidence show that the hand and eye motor systems interact with each other in visually guided reaching actions (see Bekkering and Sailer [1] for a review). In particular, eye movements toward a goal ahead of the hand reaction can provide the hand motor system with some crucial information. Indeed, saccadic eye movements affect several aspects of concurrent hand movements, such as reaction time, initial acceleration, or final position [2–7]. In addition, eye and hand reaction times are correlated from trial to trial. This temporal coupling implies that both motor systems share a common process during movement initiation [8,9]. Eye–hand coordination has been examined widely as described above, but those studies have focused mainly on motor behavior from a static posture. This means that in the experiments, the participants were asked to wait for a target appearance and then start reaching toward the designated target as soon as possible. After the reaching initiation, the target remained stationary. In our daily life, however, the reach target could move unpredictably during a reach. To interact with dynamic environments, Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website (www.neuroreport.com). c 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins 0959-4965

NeuroReport 2014, 25:441–445 Keywords: automatic pilot, eye–hand coordination, human, on-line motor control, reaching, saccades a NTT Communication Science Laboratories, Nippon Telegraph and Telephone Corporation, Kanagawa and bDepartment of Intelligence Science and Technology, Graduate School of Informatics, Kyoto University, Kyoto, Japan

Correspondence to Naotoshi Abekawa, ME, NTT Communication Science Laboratories, Nippon Telegraph and Telephone Corporation, 3-1 Wakamiya, Morinosato, Atsugi, Kanagawa 243-0198, Japan Tel: + 81 46 240 4659; fax: + 81 46 240 4721; e-mail: [email protected] Received 30 October 2013 accepted 19 November 2013

Recent studies on on-line motor control have shown that we can initiate rapid and automatic manual adjustments in response to a sudden target jump during a reach [10,11]. The underlying mechanisms of these automatic adjustments are considered to differ from those involved in a voluntary reaching movement toward a stationary target (see Gomi [12] and Desmurget and Grafton [13] for a review). In addition, previous studies have found that on-line manual adjustments were frequently accompanied by saccades [14–16]. This suggests that hand and eye motor systems somehow seem to work cooperatively during on-line movement control, but the detailed mechanisms of the coordination are not well understood. Specifically, it is unknown whether the initial component of the rapid manual adjustment is coupled with and dependent on concurrent saccadic eye movements. To address this issue, the present study examined on-line manual adjustments to a sudden target jump under two gaze conditions: saccade or fixation. We found that the quick on-line manual adjustment initiated earlier than the onset of saccade. Despite this temporal order, the DOI: 10.1097/WNR.0000000000000111

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

442

NeuroReport 2014, Vol 25 No 7

manual response latency was modulated by the existence of the following saccade. Our results showed that the online manual system coordinated with the oculomotor system before the actual eye movements.

Materials and methods Participants

Seventeen individuals (six men and 11 women, age range 20–45 years, mean age±SD; 30.6±7.3), including one of the authors, participated in the experiment. All the participants were right-handed and neurologically healthy, and provided written informed consent before participating. None of the participants had professional experience in sports or playing musical instruments. All, except for the author, were unaware of the purpose of the research and had never participated in our reaching experiments. The study was approved by the NTT Communication Science Laboratories Research Ethics Committee.

Apparatus

Participants were seated 48 cm in front of a CRT monitor (CV921PJ; Totoku Electric Co., Tokyo, Japan, size: 22 inch, vertical refresh rate: 160 Hz, mean luminance: 17.5 cd/m2), as shown in Fig. 1a. The head was stabilized by a chin rest on which a video-based eye-tracking system (Eyelink; SR Research Ltd, Kanata, Ontario, Canada, 2 kHz sampling) was mounted. Visual stimuli were controlled by Matlab (MathWorks, Natick, Massachusetts, USA) and Cogent Graphics (University College London, London, UK). The exact onset time of the visual stimulus was detected using photodiodes. The coordinate axes are shown in Fig. 1a; x: horizontal direction parallel to Fig. 1

(a)

(b)

Motion capture system Eye tracking system

Cursor Start box

Fixation cross Reach target

Reach cue

Time

Digitize z r x y

the monitor, y: perpendicular direction to the monitor, z: vertical direction. Participants held a stylus pen and made reaching movements in a forward direction on a digitizer (Intuos2; Wacom Co., Saitama, Japan), which detected the stylus position to provide visual feedback of the hand cursor. The hand was occluded from view, and the instantaneous stylus location was presented visually as a black square (4  4 mm) on the monitor. Forward reaching was presented on the monitor as a cursor motion from the bottom to the top. The actual hand position for data analysis was recorded by an infrared marker (attached to the back of the hand around the base of the ring finger) with an Optotrak system (Optotrak 3020; Northern Digital Incorporated, Waterloo, Ontario, Canada) at 250 Hz.

Experimental paradigm

At the beginning of each trial, a start box (8  8 mm) was presented at the lower part of the monitor with a gray background of 17.5 cd/m2 (Fig. 1b). Participants placed the hand cursor at the start box by locating the stylus pen at about 15 cm in front of their body [(x, y, z) = (0, 5, – 40) cm relative to the center of the eye]. Then, a fixation cross (1.5 cm and 21 cd/m2) and a reach target (circle of 1 cm diameter, 21 cd/m2) were presented at the same location 22 cm above the start box. After participants maintained their gaze on the fixation cross for 700 ms, a reach cue (red circle of 1 cm diameter) was briefly flashed for 100 ms over the cross, and participants started reaching toward the target at a moderate speed (B0.6 s). In randomly selected trials, the target jumped 7.6 cm (91) rightward (32/96 trials) or leftward (32/96 trials) 100 ms after the reach onset. The target was not jumped in the remaining trials (32/96 trials). In target-jump trials, participants were asked to adjust their reaching movements toward the jumped target as fast as possible with a saccadic eye movement (SAC condition) or while maintaining eye fixation (FIX condition). Each gaze condition was run in separate blocks of 48 trials, and the block order was counterbalanced within and between participants (SAC–FIX–FIX–SAC or FIX–SAC–SAC–FIX).

Reach start

Data analysis Target jump On-line adjustment

Experimental paradigm. (a) Experimental apparatus. Participants made reaching movements on the digitizer in a forward direction. (b) Time course of a single trial. Participants made on-line manual adjustments following an abrupt target jump with saccade or fixation (see Materials and methods for details).

The observed eye and hand position data were aligned at the onset time of the target jump. They were low-pass filtered (fourth-order Butterworth, 30 Hz cutoff). We analyzed the acceleration of the arm and eye movements along the x-axis (target-jump directions). The armreaching baseline was defined by the mean acceleration patterns of all the no-jump trials. This baseline was subtracted from the hand acceleration pattern in each target-jump trial. The onset of the manual adjustment was defined by the time from which the acceleration

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Eye–hand coordination Abekawa et al.

continuously exceeded the threshold of 260 m/s2 (B1/10 of peak acceleration) for at least 40 ms. This method was also used for saccade detection with a threshold of 2600 degree/s2 for a duration of 30 ms. The latency was detected for each trial. A paired t-test was used to compare statistical differences in response latencies between two motor responses or between two gaze conditions. In addition, effect size (ES) and statistical power (Power) were calculated using the G*Power 3 program (Heinrich Heine University, Dusseldorf, Germany) [17].

Results Figure 2a and b shows on-line manual adjustments for the SAC condition from a particular participant. Reach trajectories deviated during a reach according to the target-jump directions (Fig. 2a). Figure 2b shows the mean manual responses (x-acceleration) from the target-

Fig. 2

(b) x-hand acceleration (m/s2)

(a)

R L No 3 cm

y x

3 cm

3.0

R L No

2.0 1.0 0 −1.0 −2.0 −3.0 0

50

100 150 200 250 300 Time (ms)

(c)

(d) N = 17

220 Response latencies (ms)

Saccade latency (ms)

400

300

200

∗∗∗

200 180 160

100 200 300 Manual latency (ms)

400

M an u Sa al cc ad e

140 100

Results under the SAC condition. (a) Reach trajectories for the targetjump directions (rightward: black solid curve, leftward: black dashed curve, and no-jump: gray curve) obtained from a particular participant. Circle: hand position at the onset of the target jump. (b) Mean x-hand accelerations against the time from the target jump (same participant as in a). Shaded area: SD across trials. Filled and open triangle: detected onset time of the manual adjustment and saccade. (c) Trial-by-trial latency for manual adjustments and saccades (all participants). Dashed line: diagonal line. Solid line: first principal component. (d) Response latencies of manual adjustment and saccade averaged across participants. Error bar: SE. ***P < 0.001.

443

jump onset. The response latency was computed for each trial and the mean latency for this participant (143 ms) is indicated by the filled triangle. All the participants successfully performed mid-flight reaching adjustments, and the mean response latency across participants was 168 ms (SD = 14.1). The observed latencies were slightly slower than that reported in previous studies [16,18–20] perhaps because of differences in visual stimulus conditions (low target luminance and the presentation of a fixation cross), which strongly affect the response latency [21]. The SAC condition required participants to make a saccade with a manual response to the target jump. The mean saccade latency for the identical participant (187 ms) is indicated by an open triangle in Fig. 2b. As compared between filled and open triangles, the saccade initiation was preceded by the manual response. This temporal order appears to be consistent across all the participants. Figure 2c shows hand and eye latencies for all the trials across all the participants. Most of the data (81.6%) fell above the diagonal line, indicating that the manual adjustment usually started before the saccade onset. As shown in Fig. 2d, the mean latency of saccades across participants (204±24.8 ms) was significantly longer than that of manual responses [paired t-test, t(16) = – 7.17, P < 0.000005, ES = 1.74, Power = 0.99]. Note that our control experiment (six men and six women, age range 20–41 years, mean age±SD; 29.9±6.3), in which participants started to move their hands and eyes toward the jumped target location after remaining stationary, confirmed that the hand start was significantly slower than the saccade onset [hand: 230±29 ms, saccade: 185±19 ms, paired t-test: t(11) = – 5.93, P < 0.0001, ES = 1.71, Power = 0.99]. Thus, latency difference in the two motor responses during on-line control cannot be ascribed to the motor difference or experimental artifacts. In addition, we observed that hand and eye latencies were correlated positively with each other on a trial-totrial basis (correlation coefficient = 0.39, P < 0.001). We used correlation analysis for each participant. Although the degree of correlation varied considerably across the participants (range: 0.23–0.6, mean: 0.41), the correlation was statistically significant (P < 0.05) in 13 and marginally significant (P < 0.1) in one out of 17 participants. We next investigated the effect of the gaze condition on manual adjustments. Here, the manual response was characterized by taking the difference in the mean x-hand accelerations for rightward and leftward target jumps for each gaze condition. Figure 3a shows the mean manual response across participants for the SAC (solid curve) and the FIX (dashed curve) condition. The initial hand acceleration to the target jump was larger for the SAC than for the FIX condition. The statistical difference (paired t-test) was calculated between two conditions at

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

444 NeuroReport 2014, Vol 25 No 7

Fig. 3



(b) 4.0

SAC FIX

Manual latency (ms)

3.0 2.0 1.0 0

190 180 170 160 150

200 250 Time (ms)

300

FI

150

X

140 100

SA C

Hand acceleration difference (m/s2)

(a)

Manual responses for SAC and FIX conditions. (a) Manual responses were calculated by taking the differences in the x-acceleration (rightward – leftward) in each gaze condition. Solid and dashed curves show the average manual responses across participants for the SAC and FIX conditions, respectively. Shaded area: SE. (b) The manual adjustment latency averaged across participants (with SE). *P < 0.05.

each time with a time window of 12 ms, and the horizontal line in this figure shows the time during which a significant difference was detected (P < 0.05). This significant duration was 150–270 ms after the target jump. Furthermore, the latency of the manual adjustments was significantly faster for the SAC than for the FIX condition, as shown in Fig. 3b [paired t-test, t(16) = – 2.89, P = 0.011, ES = 0.70, Power = 0.77]. These results indicated that the manual adjustments were modulated by whether or not they were accompanied by a saccade. Interestingly, this modulation cannot be ascribed to the ocular or visual information derived after saccade execution because the hand start typically preceded the eye start. To support this conclusion, we again compared the manual latencies between SAC and FIX conditions while discarding trials in which the saccade was faster than the manual adjustment, and also found a significant difference [paired t-test, t(16) = – 3.45, P = 0.003, ES = 0.84, Power = 0.90].

Discussion We measured on-line quick manual adjustments triggered by a sudden target jump under the different gaze conditions (saccade or fixation). We found that the manual latency was correlated with the saccade latency on a trial-to-trial basis. Furthermore, the manual response initiation was quicker with than without a saccade. These results showed a distinct manner of interaction between the manual and ocular controller specific in the online control. The temporal order of hand and eye response initiations found in this study differed from that reported for conventional eye–hand tasks (see Bekkering and Sailer [1] for a review). Eye first and hand second in the conventional task indicated that reaching initiations from

static posture can rely on the target information captured by the fovea. In contrast, hand first and eye second in the on-line adjustment task indicated that the manual response needs to be initiated only by peripheral retinal information before the saccade. Recent studies have shown that distinct cortical mechanisms underlie reaching toward the central and peripheral visual fields. Prado et al. [22] showed that, compared with central reaching, peripheral reaching involved a more extensive network that included the posterior parietal cortex (PPC). More interestingly, optic ataxia patients with PPC lesions can make slow mid-flight reaching adjustments only when the new target location is captured by the fovea after a preceding saccade [14,15]. Taken together, it is inferred that PPC uses peripheral retinal information to initiate the rapid on-line manual response. In addition to a significant correlation between the manual and ocular response initiations (Fig. 2c), we also found that manual adjustments are dependent on the gaze conditions (Fig. 3). These results suggest that the on-line manual system interacts with the ocular system before the execution of actual eye movements. Ocular behavior is known to be controlled by two competing internal signals: one leading to the initiation of eye movement (go process) and the other leading to its inhibition (stop process). One possible explanation for our result is that oculomotor go signals affect the on-line manual system, which resulted in induction of more rapid and large manual responses. This facilitation phenomenon seems to be consistent with the previous reports [2,3] that showed that the initial acceleration and amplitude of hand movements increased with the amplitude of accompanying saccade. Together, these results suggested that information derived from saccade planning and/or execution can interact with the hand motor system in a similar manner in reach-planning and in on-line reaching control. An alternative explanation is that the oculomotor stop signals required to maintain eye fixation in the FIX condition might attenuate the manual response. Further study is required to clarify the details of the interaction between the hand and the eye controllers. One could also assume a global motor attenuation by stopping saccade to explain the manual response modulation. A recent study [23] showed that stopping eye movements attenuated the corticospinal excitability of the hand even when the hand was kept stationary. To test whether such global motor attenuation can explain our result, we performed a two-choice reaction time task and examined the effect of the gaze condition (saccade or fixation) on the finger response, which consisted of pressing the right or left button according to the targetjump directions (see Supplementary Fig. S1A, Supplemental digital content 1, http://links.lww.com/WNR/A271 for the detail). In contrast to the main experiment, the result showed that rather than slowing the finger reaction,

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Eye–hand coordination Abekawa et al.

fixation accelerated it slightly (Supplementary Fig. S1B, Supplemental digital content 1, http://links.lww.com/WNR/ A271). This indicates that the global suppression induced by eye fixation [23] could not explain the gazedependent manual modulation observed in the main experiment. Instead, we suggest that the on-line manual controller interacts with oculomotor go or stop signals specifically when both hand and eye movements are directed toward a common target location. Quick on-line motor adjustments are one of the bases supporting our skillful motor actions when we interact with dynamic environments under severe time constraints. Indeed, we showed that on-line manual responses could be rapidly initiated by a peripheral visual stimulus without waiting for the saccade initiation. The most striking feature of the current study is that such rapid motor response initiating before a saccade is dependent on eye control systems. We found that manual response latency was correlated with saccade latency. In addition, manual responses became faster when there were both hand and eye movements than when the hand moved alone. These results imply that the hand and eye motor systems interact tightly even during the on-line movement control. The on-line manual controller would receive oculomotor preparation signals before actual eye movements, which could change the initiation of rapid manual responses.

Acknowledgements The authors thank E. Maeda and T. Yamada for support and encouragement. The experiments used Cogent Graphics, developed by J. Romaya at the Laboratory of Neurobiology at the Wellcome Department of Imaging Neuroscience. Conflicts of interest

2 3 4

5 6 7 8

9

10

11 12 13 14

15

16 17

18

19

20

21

There are no conflicts of interest. 22

References 1

Bekkering H, Sailer U. Commentary: coordination of eye and hand in time and space. Prog Brain Res 2002; 140:365–373.

23

445

Van Donkelaar P. Eye-hand interactions during goal-directed pointing movements. Neuroreport 1997; 8 (9–10):2139–2142. Van Donkelaar P. Saccade amplitude influences pointing movement kinematics. Neuroreport 1998; 9:2015–2018. Binsted G, Elliott D. Ocular perturbations and retinal/extraretinal information: the coordination of saccadic and manual movements. Exp Brain Res 1999; 127:193–206. Soechting JF, Engel KC, Flanders M. The Duncker illusion and eye-hand coordination. J Neurophysiol 2001; 85:843–854. Gonzalez CC, Burke MR. The brain uses efference copy information to optimise spatial memory. Exp Brain Res 2013; 224:189–197. Van Donkelaar P, Staub J. Eye-hand coordination to visual versus remembered targets. Exp Brain Res 2000; 133:414–418. Sailer U, Eggert T, Ditterich J, Straube A. Spatial and temporal aspects of eye-hand coordination across different tasks. Exp Brain Res 2000; 134:163–173. Dean HL, Martı´ D, Tsui E, Rinzel J, Pesaran B. Reaction time correlations during eye-hand coordination: behavior and modeling. J Neurosci 2011; 31:2399–2412. Goodale MA, Pelisson D, Prablanc C. Large adjustments in visually guided reaching do not depend on vision of the hand or perception of target displacement. Nature 1986; 320:748–750. Prablanc C, Martin O. Automatic control during hand reaching at undetected two-dimensional target displacements. J Neurophysiol 1992; 67:455–469. Gomi H. Implicit online corrections of reaching movements. Curr Opin Neurobiol 2008; 18:558–564. Desmurget M, Grafton S. Forward modeling allows feedback control for fast reaching movements. Trends Cogn Sci 2000; 4:423–431. Blangero A, Gaveau V, Luaute´ J, Rode G, Salemme R, Guinard M, et al. A hand and a field effect in on-line motor control in unilateral optic ataxia. Cortex 2008; 44:560–568. Gaveau V, Pe´lisson D, Blangero A, Urquizar C, Prablanc C, Vighetto A, et al. Saccade control and eye-hand coordination in optic ataxia. Neuropsychologia 2008; 46:475–486. Day BL, Brown P. Evidence for subcortical involvement in the visual control of human reaching. Brain 2001; 124:1832–1840. Faul F, Erdfelder E, Lang A-G, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods 2007; 39:175–191. Day BL, Lyon IN. Voluntary modification of automatic arm movements evoked by motion of a visual target. Exp Brain Res 2000; 130: 159–168. Boulinguez P, Blouin J, Nougier V. The gap effect for eye and hand movements in double-step pointing. Exp Brain Res 2001; 138: 352–358. Sarlegna F, Blouin J, Bresciani J-P, Bourdin C, Vercher J-L, Gauthier GM. Target and hand position information in the online control of goal-directed arm movements. Exp Brain Res 2003; 151:524–535. Veerman MM, Brenner E, JBJ Smeets. The latency for correcting a movement depends on the visual attribute that defines the target. Exp Brain Res 2008; 187:219–228. Prado J, Clavagnier S, Otzenberger H, Scheiber C, Kennedy H, Perenin M-T. Two cortical systems for reaching in central and peripheral vision. Neuron 2005; 48:849–858. Wessel JR, Reynoso HS, Aron AR. Saccade suppression exerts global effects on the motor system. J Neurophysiol 2013; 110:883–890.

Copyright © Lippincott Williams & Wilkins. Unauthorized reproduction of this article is prohibited.

Eye-hand coordination in on-line visuomotor adjustments.

When we perform a visually guided reaching action, the brain coordinates our hand and eye movements. Eye-hand coordination has been examined widely, b...
372KB Sizes 0 Downloads 0 Views