This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSRE.2014.2356472, IEEE Transactions on Neural Systems and Rehabilitation Engineering

TNSRE-2013-00345.R1

1

Assessment of Cognitive Engagement in Stroke Patients from Single-Trial EEG During Motor Rehabilitation Wanjoo Park, Student member, IEEE, Gyu Hyun Kwon, Member, IEEE, Da-Hye Kim, Student member, IEEE, Yun-Hee Kim, Sung-Phil Kim, and Laehyun Kim, Member, IEEE

Abstract—We propose a novel method for monitoring cognitive engagement in stroke patients during motor rehabilitation. Active engagement reflects implicit motivation and can enhance motor recovery. In this study, we used EEG to assess cognitive engagement in 11 chronic stroke patients while they executed active and passive motor tasks involving grasping and supination hand movements. We observed that the active motor task induced larger event-related desynchronization (ERD) than the passive task in the bilateral motor cortex and supplementary motor area (SMA). ERD differences between tasks were observed during both initial and post-movement periods (p < 0.01). Additionally, differences in beta band activity were larger than differences in mu band activity (p < 0.01). EEG data was used to help classify each trial as involving the active or passive motor task. Average classification accuracy was 80.7 ± 0.1% for grasping movement and 82.8 ± 0.1% for supination movement. Classification accuracy using a combination of movement and post-movement periods was higher than in other cases (p < 0.05). Our results support

Manuscript received December 29, 2013; revised April 08, 2014; accepted June 16, 2013. Date of publication September, 2014. This work was partly supported by the IT R&D program of MOTIE/MISP/KEIT [10045452, the Development of Multimodal Brain-Machine Interface System Based on User Intent Recognition], the Development of Robot-Assisted Motor Rehabilitation of the Upper Limb Using Bio-Signal Interfaces Project of the Korea Institute of Science and Technology (KIST), the IT R&D Program of MSIP/KEIT, "Development of Cultural Contents Evaluation Technology based on Real-Time Bio-Signal in Human Populations." (10045461), the Mid-career Researcher program (NRF-2012R1A2A2A04047239) through the National Research Foundation of Korea funded by the Ministry of Science, ICT and Future Planning. S. -P. Kim and L. Kim are co-corresponding authors. W. Park and D. -H. Kim are with Center for Bionics, Korea Institute of Science and Technology, Seoul, Korea and Department of Brain and Cognitive Engineering, Korea University, Seoul, Korea (e-mail: [email protected]; [email protected]). G. H. Kwon and L. Kim are with the Center for Bionics, Korea Institute of Science and Technology, Seoul, Korea and the Department of HCI & Robotics, University of Science and Technology, Daejeon, Korea (e-mail: [email protected]; [email protected]). Y. -H. Kim is with the Department of Physical and Rehabilitation Medicine, Center for Prevention and Rehabilitation, Heart Vascular and Stroke Institute, Samsung Medical Center Sungkyunkwan University School of Medicine, Seoul, Korea and Samsung Advanced Institute for Health Science and Technology, Sungkyunkwan University, Seoul, Korea (e-mail: [email protected]). S. -P. Kim is with the Department of Human and Systems Engineering, Ulsan National Institute of Science and Technology, Ulsan, Korea (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier

using EEG to assess cognitive engagement in stroke patients during motor rehabilitation. Index Terms— Brain-computer interface, stroke, EEG, cognitive engagement, rehabilitation

I. INTRODUCTION

S

TROKE is one of the leading causes of death and currently ranked the second most severe disease worldwide [1]. Stroke occurs primarily as a result of blood circulation problems in the brain, which can induce injuries to motor and sensory nerve systems [2]. Rehabilitation training involving repeated movement of the upper and lower limbs can stimulate damaged brain areas and lead to partial or full motor function recovery [3]. Animal studies have shown evidence of neuroplasticity, a functional and structural reorganization of the brain, following rehabilitation training [4]. Human studies have also demonstrated that active and repetitive training foster motor function recovery following stroke [5], [6]. In traditional motor rehabilitation training, patients practice repetitive limb movements aimed at improving motor function with the help of physical therapists [7]. However, this training paradigm requires extensive training periods for patients and intensive labor for therapists [8]. To address these problems, robot-assisted therapy has been developed [9], [10]. This method allows for autonomous and repetitive training without a therapist’s help. Several studies have shown that robot-assisted therapy can be effective [11]. However, there are still limitations. Robot-assisted therapy simply guides movement without knowing a patient’s motor intention, a key factor in effective rehabilitation training [12]. Previous studies highlighted the clinical importance of a sense of accomplishment in limb movement based on a patient’s motor intention during rehabilitation [13], [14]. To incorporate motor intention into rehabilitation, many studies have attempted to detect motor intention by analyzing physiological signals using electrocardiography (ECG), electromyography (EMG), and galvanic skin response (GSR) [15]-[18]. However, it is difficult to relate ECG and GSR findings to motor intention because the autonomic nervous system typically reacts much later. For EMG signals, measuring muscle activity is the proper method of identifying

1534-4320 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSRE.2014.2356472, IEEE Transactions on Neural Systems and Rehabilitation Engineering

TNSRE-2013-00345.R1 either active or passive movement in the case of gross motor control such as arm movement. Ebaught et al. reported EMG differences during active and passive humeral elevation [19]. However, the opposite result was reported when the degree of humeral elevation was small (10-50 degrees) [20]. Less muscle activity is involved in grasping and supination movements than in humeral elevation. The participants of the former EMG study were healthy people; however, in the case of stroke patients, EMG signals may suffer from interference caused by spasticity even if participants are performing a passive task [21], [22]. Also, motor imagery tasks, rather than actual movement, are more beneficial for severe patients who have difficulties performing active movements. Motor imagery can be detected by neuroimaging methods. Thus, we turned our attention to neuroimaging approaches [23]. Neuroimaging studies directly measuring motor intention have been performed using electroencephalography (EEG), functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS) to evaluate brain signals. While fMRI provides excellent spatial resolution for localizing brain activation, the method lends itself to limited training environments and has low temporal resolution. fNIRS can detect blood oxygenation level dependent (BOLD) signals without MRI, but it also has limited temporal resolution [24], [25]. Despite its low spatial resolution and sensitivity to noise, EEG is probably best suited to measuring brain activity during motor rehabilitation. Many studies have shown that imagined, planned, or executed limb movements produce EEG signals with high temporal resolution and that spatial and noise sensitivity are not as important [23], [26]. Additionally, compared to other methods, EEG is relatively easy to use in various clinical settings. There have been efforts to detect motor intention using EEG for motor rehabilitation in stroke patients. Specifically, studies have investigated the effects of rehabilitation training by combining robot-assisted therapy with an EEG-based brain-computer interface (BCI) that can detect motor intention in patients [27], [28]. Kai Keng et al. showed that the Fugl-Meyer Assessment (FMA) score, a measure of motor function, increased as a result of repetitive feedback of motor imagery (MI) to the chronic stroke patient’s affected hand [29]. Moreover, Ramos-Murguialday et al. conducted a double-blind, sham-controlled experiment to study robot-assisted rehabilitation and reported that FMA scores in the BCI-rehabilitation group were higher than the control group [30]. Gomez-Rodriguez et al. proposed a real-time rehabilitation system using motor-sensory feedback and detection of motor intent [31]. Kaiser et al. demonstrated a correlation between motor ability and event-related desynchronization/synchronization (ERD/ERS) during motor execution and MI tasks in stroke patients [32]. Kang et al. compared ERD/ERS patterns of alpha and beta oscillations between different tasks (passive, motor imagery, and passive with motor imagery) involving pronation and supination movements [33]. They showed that alpha ERD in contralateral sensorimotor areas was stronger than that in ipsilateral areas, which is typical in healthy people [34]. However, no significant

2 difference in the pattern of beta oscillations was reported. In the case of stroke patients, however, activation areas in the brain are changed by neuroplasticity [35]. In addition, active and passive movements may elicit different ERD/ERS patterns from those of this previous study. In contrast to motor intention, active engagement during rehabilitation training has rarely been studied in stroke patients. Measuring patient engagement during training is important for both refining training paradigms and tracking recovery. Motivation and feedback have been known to influence the brain’s reward system for rehabilitation training in the basal ganglia, playing an important role in recovering motor functions [36]. Thus, it is important to maintain stroke patients’ motivation throughout rehabilitation therapy and if one can provide feedback of active patient engagement in real time during rehabilitation training, it may help keep patients motivated and improve rehabilitation. Langhorn et al. showed that active patient participation is crucial for enhancing motor rehabilitation, and Takeuchi and Izumi proposed that rehabilitation training depends on the degree of volitional engagement [37], [38]. Still, how to properly monitor patient engagement during training has yet to be addressed. Therefore, development of new tools for tracking active engagement is important for stroke rehabilitation. Measurement of force using a sensor incorporated into a rehabilitation device can extract more direct and clearer signals than EEG in general. In previous studies in our laboratory, we measured force signals using pressure and torque sensors during upper limb rehabilitation tasks [39], [40]. This system used the force to trigger a robotic device to initiate hand movements. However, it was designed to help motor initiation rather than to monitor persistent active engagement during the entire task period, similar to the study by Marchal-Crespo et al. [15]. Force sensor monitoring during motor tasks in stroke patients is of limited use because forces exerted by patients are likely to be unstable and unintentional due to motor impairment. In addition, some stroke patients are unable to exert sufficient force due to severe motor impairment [15]. Hence, active task engagement may be better detected by direct measurement of brain activity. In the present study, we aimed to build an active engagement monitoring system based on brain activity that can be applied to various types of stroke patients. The present study also aimed to gage patient engagement during rehabilitation training using an EEG-based BCI. Information related to motor task engagement can be extracted from distinct neural activity patterns associated with either active or passive movement. Previous research has revealed that neural activity patterns differ when a subject actively moves a limb versus passive limb movement by an external force [41], [42]. Here, EEG patterns, specifically ERD/ERS, are evaluated during active grasping, passive grasping, and supination movements in stroke patients. These hand movements were selected based on their involvement in upper limb rehabilitation. To examine the feasibility of developing an online monitoring tool, we discriminated between EEG patterns associated with active versus passive movements on a single-trial basis. Our hypothesis was that the ERD during an

1534-4320 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSRE.2014.2356472, IEEE Transactions on Neural Systems and Rehabilitation Engineering

TNSRE-2013-00345.R1

3 TABLE I CLINICAL DATA OF PARTICIPANTS

No

Age

Sex

AH

Diagnosis

1

58

M

Rt.

2

56

F

Lt.

3

49

M

Lt.

4

52

F

Rt.

5

53

M

Lt.

6

57

M

Rt.

7

60

M

Lt.

8 9 10

65 46 59

M F M

Rt. Rt.

Lt. medial medullary infarct Rt. CR infarction Cerebral infarction, Rt. medial medulla Lt. MCA infarction, border zone Rt. MCA territory infarction Lt. pons infarction Rt. Thalamus, IC infarction old BG ICH Lt. BG infarction

Lt.

Rt. MCA infarct

11

52

M

Lt.

Rt. Pontine infarction

Mean Std.

55.2 5.2

M (73%)

Lt. (55%)

Grip Strength/Grasp

Purdue Pegboard Test

AH

UH

AH

UH

AH

UH

AH

UH

NT

23.3

6

14

31

66

1

0

0

2.5

4

11

55

65

0

0

6

36

2

15

44

65

0

1+

0

14.6

2

15

52

62

0

0

NT

23.33

NT

13

32

66

0

0

FMA

MAS/Elbow

8

27.6

6

12

40

66

+1

0

8

26.66

10

13

54

64

0

0

5 1 15.3 3 11.3 3 6.07 4.96

21 16

13 7

12 15

53 54

63 66

0 1

0 0

35.33

9

15

48

64

0

0

23.66

5

10

66

45

0

0

22.73 9.04

6.40 3.32

13.18 1.70

48.09 10.05

62.91 5.81

0.27 0.45

0.09 0.30

* AH: Affected Hand, UH: Unaffected Hand * MAS: Modified Ashworth Scale * CR: Corona Radiata * MCA: Middle Cerebral Artery * IC: Internal Capsule * BG: Basal Ganglia * ICH: IntraCerebral Hemorrhage

active task would be greater than that during a passive task. We hypothesized that active task involving a wider range of neural activities would yield greater ERD. Two previously developed robotic devices were used to guide grasping and supination hand movements, respectively [40], [43]. The grasping device performs a 1-degree of freedom (DOF) opening/closing hand movement by controlling the grasping handle, where the hand is fastened to the handle so that it does not slip off due to motor impairment. Two parallel linear motors push or pull the grasping handle. The supination device performs a 1-DOF supination/pronation movement by rotating the supination handle, where an additional arm supporter holds the forearm and the hand is fastened to the handle in a similar manner to the grasping device. II. MATERIALS AND METHODS A. Participants Eleven chronic stroke patients participated in this study (8 males, 3 females; mean age = 55.5 ± 5.6). All participants were first-time stroke patients, exhibited unilateral motor problems in upper extremities/limbs, continued to have issues at least 3 months after stroke, were aged between 45 and 70 years old, and were categorized as moderate or mild by FMA scoring. Participants with other cognitive disorders rendering them unable to understand task instructions and/or with other orthopedic disorders that led to amputation or joint contraction were excluded. Mean FMA scores were 46.4 ± 9.2 for affected and 64.7 ± 1.4 for unaffected sides, lower scores indicating more severe impairment (Table I). Stroke patients did not have

a history of neurological illness and had not previously participated in an EEG experiment. The Institutional Review Boards of the Samsung Medical Center (SMC, Application Number: SMC 2013-02-091) and Korea Institute of Science and Technology (KIST, Application Number: KIST 2013-009) approved this study. Participants were informed about the study’s purpose, experimental procedures involved, and right to withdraw at any time. Written informed consent was obtained from all participants. All research data were collected and analyzed under IRB guidance. B. Experimental protocol We selected the two types of movement, grasping and supination, because these two movements are among the latest recovered movements during upper limb rehabilitation. Thus, in many real motor rehabilitation programs, these movements are intensively trained and were therefore selected as target movements in our study. We expected greater ERD with the supination movement than with grasping because supination is more difficult to perform in stroke patients in general. For each movement, three motor tasks were performed: passive (P) movement of a robotic device, active (A) movement of a robotic device, and motor imagery (MI) of kinetic movement. We also included the MI task in our experimental protocol for the purpose of detecting motor intention by EEG, as performed in many previous BCI-based rehabilitation studies [27], [31]. However, since this paper primarily focuses on the monitoring of active engagement in stroke patients during rehabilitation, we excluded the analysis of EEG patterns

1534-4320 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSRE.2014.2356472, IEEE Transactions on Neural Systems and Rehabilitation Engineering

TNSRE-2013-00345.R1

4

(Translation in English: When ● appears, move yourself.)

(Translation in English: Rest.)

Fig. 1. Experimental Design. The upper panel illustrates our experimental protocol, which is composed of training and main sessions. In the training session (the first box), the participants repeatedly performed passive, active, and motor imagery (MI) tasks until they were familiar with each. In the main session (the second box), a single run began with task instructions followed by 14 subsequent task trials. A 10-s rest period followed each run. Rest instructions are shown in the lower right panel. During this time, participants were able to move around freely and relax. The lower panel illustrates the task paradigm in a single run. The task instruction screen was first shown on the computer screen. A rest period followed with a random duration between 2000 and 3000 ms. A cross mark appeared on the screen during this period. Then, a 2000-ms motor task period started with auditory and visual cues, a 1000-ms stay period followed, and a 2000-ms return period followed this, in which the robotic device was reset to its original position. During the motor task period, a circle was shown on the computer screen as a visual cue for the motor task. During the stay and return periods, a cross mark was shown on the screen to indicate these periods and to guide the participants to focus their eyes on the mark. (a)

(b)

(c)

(d)

Fig. 2. Two robotic devices used in our study for motor rehabilitation with grasping (a) and supination movements (b). Expanded figures (c) and (d) represent zoomed in sections of (a) and (b) respectively. Details of these devices can be found in our previous studies [40], [43].

generated during the MI task. Participants were trained on each motor task before the actual experiment (Fig. 1). The main experiment involved nine runs of motor tasks separated by 10-s breaks. Motor task order was permutated (e.g., P-A-MI-A-MI-P-MI-P-A), such that the first task was chosen randomly for counterbalancing. Each motor task consisted of 14 trials and took approximately 2 min. Participants performed 42 trials of each motor task for both grasping and supination movements.

At the beginning of each run, task instructions were shown on the screen. Then, a fixation cross was displayed, and participants were instructed to stare at the cross without moving their head. In the few seconds that followed, participants waited for a task cue while gazing at the fixation cross. When the fixation cross changed to a circle paired with a beep sound, the participants performed a motor task for 2000 ms. The training device stopped for 1000 ms, and the circle changed back to the fixation cross. The training device then returned to its starting position for 2000 ms. During this return period, participants were instructed not to exert control on the device. In general clinical paradigms, upper limb motor rehabilitation treatments employ continuous action. The length of time for the hand movement task should not be too long to cope with a continuous motor control paradigm, or too short because of the difficulty in performing movements in a short period. In this regard, we empirically determined that the 2000 ms task period was not too short for patients to perform movements and that it was also not too long for the overall training task to be completed in continuous action. The 2000 ms return period was determined to be equal to the motor task period. C. Robotic Devices for Upper Limb Motor Tasks The robotic devices used were designed to guide users to complete tasks that call for active and passive movement [40], [43]. Using these devices, participants performed upper limb motor tasks involving grasping and supination (Fig. 2). During the experiment, participants sat in a chair that either had a board to support their arm during grasping (Fig. 2a) or an arm supporter for supination movement (Fig. 2b). The participant’s affected hand was fastened around the handle of the device with a bandage. A monitor that displayed instructions and visual feedback was located at eye level and 1 m away from participants. Contrast and luminosity were adjusted to optimize concentration and comfort for all participants. During passive motor tasks, participants did not exert control, and instead, a robotic device guided performance. For active motor tasks, participants performed the task by themselves without assistance from the device. The robotic device operated differently according to task type. After completing motor tasks, the device’s handle automatically returned to its neutral position to prepare the next task. Completing the motor task within 2000 ms may be difficult for some stroke patients. Thus we did not ask patients perform a full range of movement. In the grasping movement, the initial posture was not in a fully open position but was instead half closed. The maximum distance of the grasping handle was set to 24 mm to allow a relatively short range of movement. We also asked patients to perform supination movements within a limited range between -18 and 60 degrees. D. EEG Data Analysis EEG signals were acquired using a 64-channel active electrode EEG system (sampling rate: 2048 Hz; Active-two, Biosemi, Amsterdam, Netherlands). EEG signals recorded

1534-4320 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSRE.2014.2356472, IEEE Transactions on Neural Systems and Rehabilitation Engineering

TNSRE-2013-00345.R1

5

Fig. 3. EEG preprocessing procedure. First, the recorded EEG signals were down-sampled from 2048 Hz to 1024 Hz. Signals were then filtered with a 1– 80 Hz pass band and again with a 60 Hz notch filter to remove noise. Filtered signals were divided into sets of epochs, such that each epoch covered the period from 1000 ms before to 5000 ms after motor task onset. Independent component analysis (ICA) was applied to the signals to remove artifacts produced by eletrooculography (EOG) and electromyography (EMG). Furthermore, epochs with signal amplitudes that exceeded a predetermined threshold or visually different waveforms were removed after visual inspection. Finally, EEG signals from all the electrodes were re-referenced using a common average reference (CAR). gr Passive Ipsi

gr Active Contra

2 1 0 -1

0

1000

1 0

-2 3

8 -4 5000 -10001

0

1000

4

3

3

2

2

2

2

2

1

1

1

1

0

0

0

0

-1

-1

-2 32 13 4

-2 4

-3 3

-3 3

8 -10004000

8 -4 2 0 5000 -1000 1000

2000

-1 32 4

-3 2 8 -10004000 32 13

0 5000

8 -4 -1000 10001

-1

8 -1000 32 13

0

1000

2000

1 0 -1

-3 8 -4 5000 -1000

8 -4 2 4000 3000 5000 -1000

2000 4000

su Passive SMA

gr Passive 2000 0 Contra

1000 3000

0

1000-2 32 13 4

su Active Contra 2000

3000

su Passive SMA 4000

32

0

1000

8 -10004000

8 -4 2 0 5000 -1000 1000

1

3 0

0

2 -1 32 4

-1 4

1 -2 13 3

-2 3

0 -3 2

-3 2

32

1000 3000

2000 4000

1

13

-1 1000 13

2000

3000

4000

-2

5000

8 -1000

3000

8 -10004000 32 13

0 5000

8 -4 -1000 1000

1000

2000

2000 4000

0

1000

2000

3000

0 -3 2 3000

4000

su Active Ipsi 2000

4

3000

8 3 -1000 4000

-1 8 -4 5000 -10001 32 -2 4 13 0 3 -3 -1 2 8 -4 0 5000 -1000 1000-2 32 4 13 1

su Passive 2000 Ipsi

3000

1

-3 3 0

0

8 -4 2 5000 -1000 -1

4000

13

-2 -3

2000

-2 3000

4000

3000 5000

gr Passive 2000 SMA

1000

3000 gr Active Contra

32

2000 0

4000

4 1

3

3 0

2

2 -1 4

1

1 -2 3

0

0 -3 2

su Active SMA 2000 4000

1000 3000

8 -10004000 -4

0 5000

13

-1 4000 32 4 -2 13

su Passive 2000 SMA

1000

3000

2000 0

1 -2

4000 -1

13

32 4 -2 13 3 -3

8 -1000

2 8 -4 -10004000 1 3000

1000 3000

2000 4000 0

1000

3000 5000 2000

0 -3 su Passive Ipsi 2000

3000

4000 324 13

-1 8 -4 5000 -1000 32 4 -2 13

3 2

1000

2000

3000

8 -1000 4000 1

1

0

su Active Contra 0

su Passive SMA 2000

1000

su Active 3000 Contra

2 -3 -1 -4 0 5000 1000 -4 4000 5000 1 -2

32

2 8 -4 3000 5000 -10004000 1

0

0

0

-1

13

13 8 -1000

8 -1000

0

1000

2000

3000

4000

-2

0 -3

8 -4 5000 -1000

-1

-3

13 1000

2000

0

3000

1000

4000

2000

5000

3000

-4

8 -10004000

-1

-1

-2

-2 -3

13

13

-2

13 8 -1000

-3 8 -4 0 5000 -1000 1000

2000 0

0

1000 3000

1000

2000 4000

2000

3000 5000

-3 3000 8 -4 -10004000

4000

0 5000

5000

4 -2 3 -3

0

1000

2000

3000

4000

5000

0

-1

3000

4000

2 8 -4 5000 -1000 1

2000

-1

4 -2 3 -3

0

1000

2000

3000

4000

5000

-2

-4 1000

2 8 -4 5000 -1000

13

-3 -4

[dB]

4000

0

-1 4 -2 3 -3

0

1000

2000

3000

4000

5000

Time [ms] Relax Task Stay Return

Fig. 4. EEG spectrogram during active and passive motor tasks in bilateral and supplementary motor areas. Each spectrogram has time (in ms units) along the horizontal axis and frequency (in Hz) along the vertical axis. Power changes (in dB) relative to signal baseline at any given time and frequency are indicated by the color heat map. Spectrograms of active and passive motor tasks during the grasping movement are placed on the upper two rows and the other spectrograms are placed on the lower two rows. The three columns indicate the contralateral and ipsilateral motor cortex, and the SMA.

from each channel were preprocessed through a series of steps (Fig. 3). We used EEGLAB toolbox for EEG signal processing [44]. First, EEG signals were down-sampled to 1024 Hz. And then a zero-phase finite impulse response filter was used for band pass filtering (order: 256; low cut-off frequency: 1 Hz; high cut-off frequency 80Hz). A notch filter was applied with a

3000

4000

2 -4

1

1

0

0

-1

-1

-2

-2

-3

2000

2 -4 1

su Active SMA

-1

3000

2 -4 1

su Active SMA

3 -3

-1

-2

2 8 -4 5000 -1000

32 4 -2 13

0

-1

13

2000

2

0

0

4000

32 4 -2 13 3 -3

3 3 -3

0 5000

3000

0

su Active Ipsi

4 4 -2

1

2000 4000

-1

1

su Active Ipsi

2 -4 1000 1 1

1000 3000

0

0

-1 -4

3 -3

2000 0

2000

0 -3

5000

1

-1

3 -3

3 -2 0 -3

-1 4000 32 4 -2 13

2

gr Active SMA

32 -2 4 13

4 1 -1 -2

0

3 -3 2 8 -4 0 5000 -1000 1000

-3 3 0 1 -4 2 -1

5000

3

0

gr Active Ipsi

2

0

su Active 0 Contra

4

3

1

0 3 -3 -1 4 2 -4 3 1000 -2 4 1

0 5000

gr Active SMA

32 4

2

-1 -4 1 4 -2

5000

3 -3

8 2 -4 3000 5000 -10004000

gr Active Ipsi

-4 2

5000

32 4

1

3 0 -3

2 -1 8 -4 -1000 1000

-3

0

gr Active 0 Contra

32 4 1 -2

-1

Passive -4 0 5000 Contra 1000

2 8 -4 5000 -1000

4000

3000

su Active SMA

2 -1 32 4

3 -3

8 -1000

gr 2000 Passive SMA

1000

3 0

32

su

-1 8 -4 3000 5000 -1000 4000 32 32 4 -2 13

gr 0 Active Contra

32 4 1

32 13

-1 1000

0

0 -3

1000 3000

8 -4 2 5000 -1000

13

13

su Passive Contra 2000 0

gr Passive 2000 Ipsi

-4 2 1000

0

2 -1 8 -4 -1000

-3

2000

4000

-3 3

3 0 -3

0 1 -2

3000

2

1

13

gr 2000 Passive Ipsi

-3 3

1 -2 13 3

1000 -2 4

0 5000

-1 -2 4

-3 3

0

32 4 1 -2

13 0

0

0

0 gr Active SMA

-2 32 13 4

32

-1

8 -4 2 4000 3000 5000 -1000 -1

13

8 -1000

gr Passive Contra

3 -3 -1 2 8 -4 5000 -1000-2 32 4 13 1

1

-1

gr Active Ipsi

su Active Ipsi

-4 1 1000

0 5000

-3 3 0

su Passive 2000 0 Contra

-4 2 1000

0 5000

32 4 1

-1 8 -4 3000 5000 -100014000 32 -2 4 13 0

2000 4000

Passive

4000

3000

-2

Supination

13

1000 3000

su Active Contra

-3 3

Active

4000

gr 2000 0 Passive Contra

0

-3 3 8 -4 2 5000 -1000

gr Passive SMA

-2 13 3

13

3000

gr Active Contra

1 0

su Passive Ipsi

-2 32 13 4

32 4

3

32

0 -1

4

3

-3 3

32

-3 2 4000

3000

gr Active SMA

Supplementary motor area

32 4

3

-2 32 13 4

32 13

su Passive Ipsi

-1 32 4 13

2000

Passive

8 -4 2 5000 -1000

Grasping

-3 3

gr Active Ipsi

Ipsilateral motor cortex

32 4

-1

gr Passive Ipsi

-2 32 13 4

4000

Active Frequency [Hz]

32

3

gr Passive SMA

Contralateral motor cortex

32 4

zero-phase digital filter to remove line noise (order: 1024; start and end of pass band: 56, 64 Hz, first and second stop band: 58, 62 Hz). Second, the filtered EEG signal was divided into epochs corresponding to each motor task trial. A single epoch spanned from 1000 ms before the movement cue until 5000 ms after the cue. Artifacts from eye or body movement were removed after an independent component analysis (ICA) [45]. Third, trials with suspected head movement were excluded. These trials typically showed extremely large fluctuations exceeding a threshold of 100 µV and activity that was more than five standard deviations away from the mean. Finally, EEG signals were re-referenced using the common average reference (CAR) [46]. After preprocessing, a spectrogram of the EEG signal at each channel was computed via short-time Fourier transform (STFT) with a 500-ms Hamming window, sliding by 50 ms. Baseline for each epoch was determined to be 1000 ms prior to movement cues. Spectral power was normalized by subtracting the mean at baseline from every data point within an epoch and dividing by the standard deviation at baseline. This process was repeated for all EEG channels, motor tasks (active and passive), and movement types (grasping and supination). Two frequency bands were selected in our study: the mu band (8–13 Hz) and the beta band (13–32 Hz), both of which reflect sensorimotor rhythms (SMRs) [23], [26], [47]. To discriminate between active and passive movements, C3, C4, and Fz channels, the ipsilateral/contralateral motor cortex, and the supplementary motor area (SMA) were selected based on a previous finding that ERD/ERS of SMRs in these channels reflect imagined, planned, or executed arm/hand movements [34]. We extracted spectral features by dividing the period after the movement cue into six non-overlapping 500-ms segments (four segments in the motor task period and two in the stay period). Spectral power values at each frequency (1-Hz resolution) within mu and beta bands (8–32 Hz, a total of 25 frequencies in this range) were time-averaged within each segment. This resulted in 300 (25 × 6) features for each channel. Next, a one-way ANOVA was conducted to select a subset of features that showed a significant difference between active and passive movement (p < 0.01). The number of selected features differed among participants and movement types. Using the selected features as input, we classified single-trial EEG data as involving either active or passive movement. A naïve Bayes classifier was used and classification was performed using a ten-fold cross-validation method [48], [49]. Classification was performed separately for grasping and supination in all participants. Further analysis was conducted to investigate the effect of the task type on classification using features from the motor task period (0–2000 ms after the cue), the stay period (2000–3000 ms after the cue), or both. 8 -4 5000 -1000

-3

0

1000

2000

3000

4000

5000

-4

III. RESULTS A. Spectral power differences between active and passive motor tasks We investigated temporal changes in mu and beta band power in bilateral sensorimotor areas and the SMA during

1534-4320 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSRE.2014.2356472, IEEE Transactions on Neural Systems and Rehabilitation Engineering

8~13 [Hz] ; Grasp ; Ipsilateral

0 1

0 2000 3000 Time [s] 3000 0 2000 0 13~32 [Hz] ; Time 2000 Grasp ; Contralateral [s] 3000 Time [s] ; 3000 Ipsilateral 0 8~13 [Hz] ; Supination 2000 13~32 [Hz] ; Time Grasp[s]; Contralateral

5000 5000 5000

5000

8~13 [Hz] ; Supination ; Ipsilateral

0 2000 3000 Time [s] 3000 0 2000 Time [s] ;3000 Ipsilateral 013~32 [Hz] ; Supination 2000 8~13 [Hz] ; supination Time [s] ; Contralateral 0 2000 3000 13~32 [Hz] ; Supination ; Ipsilateral Time [s] 8~13 [Hz] ; supination ; Contralateral

5000 5000

0 2000 3000 Time [s] 3000 0 2000 Time [s] ; Contralateral 13~32 [Hz] ; supination 0 2000 3000 Time [s] 3000 0 2000 13~32 [Hz] ; supination ; Contralateral Time [s]

5000 5000

5000

5000

5000

5000

-1 0 -2 -1 -1000

0

2000 Time [s]

3000

5000

-2 -1000

0

2000 Time [s]

3000

5000

0 1 -1 0 -2 -1 -1000 -2 1 -1000 1 01 0 1 -10 -10 -2 -1 -1000 -2 -1 -1000 -2 -1000 -2 1 -1000 1 01

01 -10 -10 -2 -1 -1000 -2 -1 -1000 -2 -1000 1 -2 -1000 1 01 01 -10 -10 -2 -1 -1000 -1 -2 -1000 -2 -1000 1 -2 -1000 0 1

Normalized power

8~13 [Hz] ; Grasp ; Contralateral

1

Normalized power

5000

1

6

0 -1

Active Passive Active 2000 3000 5000 Active 13~32 [Hz] ; Grasp ; Ipsilateral Passive Time [s] 1 Passive 8~13 [Hz] ; Grasp ; Ipsilateral 8~13 [Hz] ; Grasp ; Ipsilateral 8~13 [Hz] ; Grasp ; Contralateral 0 2 1 0 -1 0 0 -1 -2 -2 -1 Active 0-1 2000 3000 5000 0 2 3 5 -1000 0 2000 3000 5000 -2 ; Grasp ; Ipsilateral [s] ; Ipsilateral Time [s] 0 13~32 [Hz] Time Passive 13~32 [Hz] Time ; Grasp -1000 2000[s] 3000 5000 1 8~13 [Hz] ; Grasp ; Contralateral 8~13 [Hz] ; Grasp ; Contralateral -2 8~13 [Hz] ; Grasp ; Contralateral Time [s] 0 2000 3000 5000 -1000 0 2000 3000 5000 2 1 13~32 [Hz] Time ; Grasp 13~32 [Hz] ; Time Grasp[s] ; Contralateral [s] ; Ipsilateral 0 1 8~13 [Hz] ; Supination ; Ipsilateral 0 8~13 [Hz] ; Grasp ; Contralateral 0 1 -1 0 -2 -1 2 -1 0 3 5 0 -2 [s] Time -1 0 2000 3000 5000 0 2000 3000 5000 -1000 -2 8~13 [Hz] ; Supination ; Ipsilateral Time [s] 3000 -1 0 2000 5000 Time [s] 3000 -1000 0 2000 5000 2 -2 13~32 [Hz] ; Time Grasp[s]; Contralateral [s]; Contralateral Grasp 013~32 [Hz] ; Time 2000 3000 5000 -1000 0 2000 3000 5000 1 8~13 [Hz] ; Supination ; Ipsilateral -2 8~13 [Hz] ; Supination ; Ipsilateral Time [s] 3000 Time [s] 3000 0 2000 5000 -1000 0 2000 5000 1 13~32 [Hz] ; Time Grasp[s]; Contralateral 13~32 [Hz] ; Supination Time [s] ; Ipsilateral 0 1 -28~13 [Hz] ; Supination ; Ipsilateral 8~13 [Hz] ; supination ; Contralateral 0 2 -1 0 3 5 1 -1 [s] Time 0 8~13 [Hz] ; supination ; Contralateral -1 0 2 -2 -1 0 2000 3000 5000 0 2000 3000 5000 -1000 -2 Time [s] 3000 Time [s] 3000 -1 00 2000 5000 -1000 0 2000 5000 -2 13~32 [Hz] ; Supination ; Ipsilateral 13~32 [Hz] ; Supination ; Ipsilateral Time [s] 3000 Time 0 2000 5000 -1000 0 2000[s] 3000 5000 1 8~13 [Hz] ; supination ; Contralateral -2 8~13 [Hz] ; supination ; Contralateral -2 Time [s] 0 2000 3000 5000 [s] 3000 -1000 0 2000 5000 -12 0 3 Time 5 1 2 13~32 [Hz] ; Supination Time [s] ; Ipsilateral 13~32 [Hz] ; supination Time [s] ; Contralateral Time 0 [s] 8~13 [Hz] ; supination ; Contralateral 1 0 -1 0 -1 -2 0 2000 3000 5000 -1000 -1 0 2000 3000 5000 -2 Time [s] Time [s] 3000 -1000 0 2000 5000 0 2000 3000 5000 13~32 [Hz] ; supination ; Contralateral 13~32 [Hz] ; supination ; Contralateral -2 Time [s] Time [s] 3000 0 2000 5000 -1000 0 2000 3000 5000 1 Time [s] 3000 0 2000 5000 Time [s] 13~32 [Hz] ; supination Time [s] ; Contralateral 0

Ipsilateral motor cortex 8~13 [Hz] ; Grasp ; Ipsilateral

Normalized power Normalized power Normalized power Normalized power Normalized power Normalized Normalized power Normalized Normalized power Normalized Normalized power Normalized power power Normalized power power Normalized power power

-1 0 -1 0 -2 -1 -1000 -2 -1 -1000 -2 -1000 -21 -1000

0 8~13 [Hz] ; Grasp 2000; Contralateral 3000 13~32 [Hz] Time ; Grasp [s] ; Ipsilateral

5000

Normalized power

10 10

3000

Normalized power

β

-1 -2 -1000 -1-2 -1000 -2 -1000 -21 -1000 1

2000

Normalized power

Supination

μ

-21 -1000 1 10 10 0 -1

0

13~32 [Hz] Time ; Grasp Contralateral motor cortex [s] ; Ipsilateral

Normalized Normalized power Normalized Normalized power Normalized Normalized power Normalized Normalized power Normalized power Normalized power Normalized Normalized power power Normalized Normalized power Normalized power Normalized power power power power power power

β

-1

0-1 -1 -2 -1000 -1-2 -1000 -2 -1000 -21 -1000 1 10 10 0 -1 0-1

Normalized power power Normalized Normalized power Normalized power power power Normalized Normalized power Normalized Normalized power Normalized power Normalized power

Grasping

μ

Normalized power Normalized power Normalized power Normalized power

TNSRE-2013-00345.R1 0 -1-2 -1000

Active Passive

8~13 [Hz] ; Grasp ; Ipsilateral

1 0

Normalized power

Normalized powerpower Normalized

8~13 [Hz] ; Grasp ; Ipsilateral 1

-2 -1000 1

Relax

Supplementary motor area 8~13 [Hz] ; Grasp ; Ipsilateral 0

Time [ms] Task Stay

Return

Fig. 5. Comparisons of event-related changes in the power of sensorimotor rhythms between the active and passive motor tasks in bilateral motor cortical and -1 0 supplementary motor areas. The temporal changes in the power of sensorimotor rhythms at the mu (8–13 Hz)-1and beta (13–32 Hz) bands are shown for each of the three brain areas (bilateral motor cortical areas and supplementary-2-1motor area) and for grasping and supination, respectively. These time courses of spectral power, -2 -1000 0 2000 3000 5000 -1000 0 2000 3000 5000 starting 1000 ms before the motor task onset (time point of 0 s in each graph) and ending Time [s] 5000 ms after the onset, were obtained by averaging across all the trials of Time [s] -2 of spectral power for the passive motor task and the solid red lines represent them for the active all the participants. The dashed blue lines represent the time courses -1000 0 2000 3000 5000 motor task. The point marks above and below the lines signify time points when spectral Time [s] power showed a statistically significant change relative to baseline power (p < 0.01). The blue mark represents significant changes for the passive motor task, and the red mark represents them for the active motor task. The vertical lines connecting the two curves depict time points when the spectral powers of the active and passive motor tasks were significantly different (p < 0.01).

active and passive motor tasks. Fig. 4 shows total average EEG spectrograms in bilateral sensorimotor areas and SMAs for two motor tasks and two functional movements. ERD was observed after the motor task cue and this ERD phenomenon was visible in the entire frequency range of the mu and beta bands. For investigation of spectral power differences between active and passive movements during the motor tasks, event-related changes in mu and beta band power for every case shown in Fig. 5. Overall, both active and passive grasping/supination movements induced significant decreases in mu and beta band power following the movement cue (ERD; p < 0.01). However, there were still differences in the temporal patterns of ERD between active and passive tasks. The active task consistently induced ERD in mu and beta band power during the entire post-stimulus period in bilateral sensorimotor areas and the SMA. The passive task also induced ERD in mu and beta power during the motor tasks in all three areas. However, during the period immediately after the task, it induced ERD for most bands, areas, and movement types outside of the mu band in the SMA for both movements and the beta band in the SMA for supination. During the return period in which the training device returned to its initial position, the passive task induced ERD for everything except the mu band in the SMA during grasping. Thus, it appears that the active task induces ERD more consistently throughout the whole movement period than the passive task.

We also identified time windows in which spectral power was significantly different between active and passive tasks. Differences were observed in the beta band of all areas for grasping and supination. This difference was observed to be greatest during the stay period immediately following the motor task. We also observed differences in the mu band, but only for a few windows within this same period for supination in bilateral sensorimotor cortical areas and grasping in the SMA. Paired t-tests revealed that beta band differences between the two tasks over the entire period (0–5000 ms) were larger than mu band differences for both grasping and supination movements (p < 0.01) in bilateral sensorimotor areas and SMA. Fig. 6 illustrates EEG spectral power topography of the grand average band power across 64 EEG channels at 553 ms and 2815 ms following the movement cue. Average ERD during the active motor task was at its maximum at 553 ms following the movement cue in bilateral sensorimotor areas and the SMA for all bands, movement types, and tasks. ERD in sensorimotor areas was also larger during the active task than the passive task (p < 0.05) except for the beta band with grasping movements 553 ms after movement cues. Furthermore, the associated ERD was larger in the affected hemisphere than in the unaffected hemisphere in the case of the beta band with supination movements 553 ms after movement cues (p < 0.05). Average spectral power during the passive motor task peaked 2815 ms after the movement cue, during the

1534-4320 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSRE.2014.2356472, IEEE Transactions on Neural Systems and Rehabilitation Engineering Passive 8_13Hz 552 ms

Passive 8_13Hz 552 ms

Task is gr

Passive 8_13Hz 2814 ms

Passive 8_13Hz 2814 ms

4

4

4

4

3

3

3

3

2

2

2

2

TNSRE-2013-00345.R1 Task is gr

Passive 13_32Hz 552 ms

Task is gr

Task is gr

Task is su

μ

Grasping

Task is su

Task is su

1

1

1

0

0

0

0

-1

-1

-1

-1

-2

Passive 13_32Hz 552 ms

-3

Task is su

Passive 13_32Hz 2814 ms

-3

Active 8_13Hz 552 ms

Passive 8_13Hz 552 ms

Passive 8_13Hz 552 ms

Active 13_32Hz 552 ms

Active Task 13_32Hz is 552 gr ms

4 -4

4 -4

3

3

2

Passive 2 8_13Hz

1 4

552 ms1 4

4

4

0 3

0 3

3

3

-1 2

-1 2

2

2

-2 1

-2 1

-3 0 4 -4 -1 3 -2 2 -3 1 4 -4 0 3

-3 0 4 -4 -1 3 Passive-2 2 13_32Hz -3 552 ms1 4 -4 0 3 Active -1 2 8_13Hz -2 552 ms1

-2 1

Passive 13_32Hz 552 ms

Passive 13_32Hz 552 ms

-3 0 4-4 -1 3 -2 2 -3 1 4 -4

Active Task 8_13Hz 552is ms su

Active Task 8_13Hz is 552 su ms

0 3

μ Active 13_32Hz 552 ms

Active Task 13_32Hz is 552 su ms

-3 0

-3 0

-4 -1

-4 -1

-2

-2 Passive 13_32Hz -3 552 ms

4 -4

4 -4

3

3

1

Active2 8_13Hz 552 ms 1

0

0

-1

-1

-2

-2

-3

-3

-4

-4

2

β

-2 Passive 2 8_13Hz -3 552 ms 1 4 -4 0 3 Active -1 2 13_32Hz 552 ms1-2

-3

Active 8_13Hz 2814 ms

Passive 13_32Hz 552 ms

Active 8_13Hz 552 ms

Active 13_32Hz 552 ms

Passive 8_13HzPassive 8_13Hz 552 ms 0.5s

Active 13_32Hz 552 ms

Passive 13_32Hz Active 552 ms8_13Hz 0.5s

Active 8_13Hz 552 ms

Active 13_32Hz 552 ms

-2

-2

Passive 13_32Hz 2814 ms

7

-3

-3

Stay period 2815 ms Active Passive

Passive 8_13Hz 552 ms

-3 0 4 -4 -1 3

-2 1

-1 2

Task is su

-2

Motor task period 553 ms Active Passive

-1 2

β

Supination

Passive 8_13Hz 2814 ms

1

1

0

0

-1

-1

-2

-2

-3

-3

4 -4

4 -4

3

3

2

2

1 4 0 3 -1 2 -2 1 -3 0 4 -4 -1 3 -2 2 -3 1 4 -4 0 3

1 4 0 3 -1 2 -2 1 -3 0 44 -4 -13 3

Active 13_32Hz 2814 ms

Active 8_13Hz 2814 ms

Passive 8_13Hz 2814 ms

Active 13_32Hz 2814 ms

4 -4

4-4

3

3

2

Passive 2 8_13Hz

1 4

2814 ms 1 4

4

4

0 3

0 3

3

3

-1 2

-1 2

2

2

-2 1

-2 1

1

1

-3 0 4-4 -1 3 -2 2 -3 1 4 -4 0 3

-3 0 4 -4 -1 3 Passive-2 2 13_32Hz -3 2814 ms 1 4 -4 0 3 Active-1 2 8_13Hz -2 2814 ms 1

0

0

-1

-1

-1 2 Passive 13_32Hz 2814 ms

Passive 13_32Hz 2814 ms

Active 8_13Hz 2814 ms

Active 8_13Hz 2814 ms

-2 1

Passive 8_13Hz 2814 ms

Passive 13_32Hz 2814 ms

Active 8_13Hz 2814 ms

-3 0 4 -4 -1 3

-3 0 4 -4 -1 3 -2 2 -3 1 4 -4 0 3

2

Passive Passive-2 2 Passive 8_13Hz 8_13Hz 8_13Hz -3 2814 ms 2814 ms 3s 1 4 -4 0 3

-2 21

-1 2

-3 10 4 -4-1 0 3 -2

-2 1

Active -1 2 13_32Hz -2 2814 ms 1

-3 0

-3 0

-4 -1

-4 -1

Active 13_32Hz 2814 ms

-2

-3

4 -4

4 -4

3

3

2

2

1 4 0 3 -1 2 -2 1 -3 0 4 4-4 3-1 3 2 -2 2 1 -3 0 1 4 -1 -4 0 3 -2

1 4 0 3 -1 2 -2 1 -3 0 4 -4 -1 3

-1 -3 2

-1 2

-4 -2

1

-2 1

-3 0 4 -4 4-1 3 3 -2 2 2 1-3 1 4-4 0 0 -1 3 -2 -1 2 -3 -2 1 -4 -3 0 -4 -1

-3 0 4 -4 -1 3 -2 2 -3 1 4-4 0 3 -1 2 -2 1 -3 0 -4 -1

-2

-2

-3

3

-3 4 -4 3

-1-3 2 -2-4 1

-1 2 -2 1 -3 0 -4 -1

-3 0 4 -44 -1 33 -2 22 -31 1 4 -40 0-1 3 -1-2 2 -3 -2 1 -4 -3 0 -4 -1

-2

-2

-3

3

-3 4 -4 3

2

2

2

2

1

1

1

1

0

0

0

0

-1

-1

-1

-1

-2

-2

-2

-2

-3

-3

-3

-3

-4

-4

-4

Active 13_32Hz 2814 ms

-2 Passive Passive 13_32Hz 13_32Hz Active -3 2814 ms 8_13Hz 2814 ms 4 -43s

-2

Active 13_32Hz 2814 ms

-3 4 -4

3

3

1

Active 2 8_13Hz 2814 ms1

0

0

-1

-1

-2

-2

-3

-3

2

-4

Active 8_13Hz 2814 ms

[dB]

-4

AH

Active 13_32Hz 2814 ms

UH

Active 13_32Hz 2814 ms

4 -4

stay period that followed the motor task, in bilateral sensorimotor areas and the SMA for all bands. There was no clear ERD detected in sensorimotor areas during passive tasks.

Grasping movement Grasping movement 100

50

Task Stay Task and Stay 3

4

5

6

7

8

Participants Participants

B. Single-trial classification of active and passive motor tasks We classified EEG features from a single trial into two classes based on whether they were associated with active or passive motor tasks. For grasping, classification accuracy was at 76.4 ± 0.1% when using features extracted from the motor task period only, 76.8 ± 0.1% when using features extracted from the stay period only, and 80.7 ± 0.1% when using a combination of both (Fig. 7a). For supination, classification accuracy was at 79.3 ± 0.1%, when using features from the motor task period only, 78.5 ± 0.1% when using features from the stay period only, and 82.8 ± 0.1% when using a combination of both (Fig. 7b). There was no significant difference in classification accuracy between supination and grasping. Classification accuracy was significantly higher than chance level for all participants (p < 0.01) except for three (3, 6, and 9) for grasping and two (2 and 8) for supination. Classification accuracy using features from both task and stay periods was significantly higher than from each alone for grasping (p < 0.05). For supination, classification accuracy using features from a combination of both periods was significantly higher than from just using features related to the stay period (p < 0.05). This was not significant when looking at the task period only (p > 0.05). Table II shows the number of trials correctly or incorrectly classified for active and passive tasks across all the participants. For grasping, the rate of misclassifying active tasks as passive was reduced by 6.6% when the data from stay and task periods were combined (24.5% using the task period only ® 17.9% using both periods). However, the rate of misclassifying the passive task as the active was only reduced by 2.4% (23.0% ® 20.6%). Similarly, for supination, the rate of misclassifying the active task as passive was reduced by 5.0% (19.9% ® 14.9%), while the rate of misclassifying the passive task as active was reduced by only 2.2% (21.3% ® 19.1%). Thus, incorporating EEG data from the stay period following the motor task tended

Supination movement Supination movement

(b)

* *

2

-3 1 4 -4 0 3

-3 0 4 -4 -1 3 -2 2 -3 1 4 -4 0 3

-4

1

-2 2

-2 1

Fig. 6. EEG spectral power topography during active and passive motor tasks. The topographies of EEG spectral power patterns 553 ms after motor task onset are shown in the left two columns, and those 2815 ms after motor task onset are shown in the right two columns. EEG spectral power was calculated from the mu (8-13 Hz) and beta (13-32 hz) bands. The left two columns represent EEG patterns during the motor task period while the right two columns represent those during the stay period. More specifically, the first and the third column show the topographies for the active motor task and the second and the fourth column show those for the passive motor task. The power representations denoted as colors are power changes relative to baseline (in dB units). AH and UH denote the affected hemisphere and unaffected hemisphere, respectively.

(a)

-2

-3

-1 2

4 -4

Classification accuracy Classification accuracy (%)(%)

Task is su

Task is gr

Active Task 8_13Hz is 552 gr ms

1

9 10 11 AVG

100 *

Classification accuracy Classification accuracy (%)(%)

Task is gr

50

Task Stay Task and Stay 1

2

3

4

5

6

7

8

Participants Participants

9 10 11 AVG

Fig. 7. Classification accuracies for grasping (a) and supination (b). Classification accuracies for the active and passive motor tasks are shown for every participant and for grasping and supination movements, respectively. In each participant, the first bar represents classification accuracy using EEG data from the task period, the second bar represents accuracy using the EEG data from the stay period, and the third bar represents accuracy using EEG data from both periods. The last bars show average classification accuracies across all the participants. The result of a pairwise statistical analysis on differences in classification accuracies between three different cases is also marked (*: p < 0.05).

1534-4320 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSRE.2014.2356472, IEEE Transactions on Neural Systems and Rehabilitation Engineering

TNSRE-2013-00345.R1

8

TABLE II THE NUMBER OF CORRECTLY AND INCORRECTLY CLASSIFIED TRIALS FOR ACTIVE (A) AND PASSIVE (P) MOTOR TASKS ACROSS 11 STROKE PATIENTS (a) Grasping movement Task period

Stay period

P 103 344 447 Σ 439 453 892

93

445

P 114 333 447 Σ 445 426 892

Stay period

A 365

80

P

355 447

92

445

Σ 445 435 892

Task and Stay period

Estimated task

Estimated task

A

A

A

P

Σ

A 338

84

422

P 95

350 445

Σ 433 434 867

A 346

P

Σ

76

422

P 110 335 445 Σ 445 411 867

True task

Estimated task

True task

True task

(b) Supination movement Task period

A 352

Estimated task A P Σ

True task

A 336 109 445

Task and Stay period

Estimated task A P Σ

True task

True task

Estimated task A P Σ

P

Σ

A 359

63

422

P

360 445

85

Σ 445 423 867

* Σ: the sum of each row or column.

to increase classification accuracy for the active task more than the passive task. IV. DISCUSSION Our study used EEG signals to predict active engagement during motor tasks in stroke patients. Previous studies on determining motor intention from EEG signals have been conducted in healthy individuals [50]-[53]. These studies relied on ERD/ERS of SMRs to detect information related to imagined, planned, or executed movements. However, since the EEG signals of patients with brain lesions due to stroke may exhibit different neuronal characteristics, it is valuable to verify if ERD/ERS of SMRs can be directly studied in stoke patients to extract motor-related information. Our results demonstrate that we can infer information about motor task engagement using ERD/ERS of SMRs in chronic stroke patients undergoing physical therapy. Our previous study also investigated brain oscillations in the beta and mu bands [33]. This previous study showed ERD in the alpha band during passive or motor imagery tasks but no significant changes in the beta band. This might be the result of the small sample size and the simplicity of the tasks for healthy subjects. By contrast, the present study in stroke patients revealed significant ERD in the beta and mu band for both active and passive tasks. It suggests that beta oscillations can be modulated by active or passive tasks in stroke patients. Additionally, the observation in the present study of larger differences in beta ERD between the active and passive tasks indicates that beta oscillations may serve to discriminate active engagement in stroke patients. Our analysis revealed that larger differences are shown between active and passive tasks in beta band relative to mu band power in both bilateral sensorimotor areas and the SMA. Comparison studies between active and passive tasks have previously been reported for healthy participants. In a PET study, Weiller et al. showed that the activation area of the SMA

was larger in the active task than in the passive task, whereas the activation of bilateral areas in the depth of the sylvian fissure was smaller in the active task than the passive task [54]. By contrast, Mima et al. observed stronger activation in the active task than the passive task in the SMA, secondary somatosensory cortex, primary sensorimotor cortex, and premotor cortex areas [55]. Using EEG, Alegre et al. showed that there were no differences in ERD patterns during the active and passive tasks [56]. Previous studies describe greater ERD in the passive task than in the active task [42], [57], [58], while other studies describe greater ERD in the active task than in the passive task [59]. Therefore, even for similar upper limb movements, conflicting results for active and passive movements have been reported. A direct comparison is impossible because of the different rehabilitation devices, movements, and experimental protocols. In regard to experimental protocols, the passive task was performed separately from the other active and motor imagery tasks in the study by Kaiser et al. [57]. In the case of the passive task experiment alone, this did not include conditions of motor intention. Therefore, the task ERD was distinctly larger than baseline. The experimental protocol consisted of active and MI tasks with the same conditions; thus, the ERD may have been smaller in the active task because less motor intention existed at base line. Ramos-Murguialday et al. used a baseline during the instruction period [42]. Therefore, the ERD of the active task was smaller than the other task because the ERD is occurred during pre-movement in the active task. However, Formagio et al. showed that the ERD of the active task was stronger than passive task, which is similar to our results [59]. We suspect that the lack of a pre-cue and the random order may have influenced these results. In other words, the presence of motor intention at baseline might lead to a reduced ERD, relative to baseline, during the motor task period. Our results also implies that beta oscillations exhibit larger differences between active and passive upper limb movements than mu oscillations, and therefore, may be more useful for monitoring engagement in stroke patients. This is in line with previous reports that showed larger post-movement increases in beta band power relative to mu band power [60], [61]. In our study, we observed clear differences in beta and mu bands, especially during the post-movement period. Furthermore, there were significant differences between the active and passive motor tasks during the post-movement period, even though participants were supposed to stop moving during this time. We suspect that this result is dependent upon our experimental protocol. We performed video recordings for behavior analysis and observed hand movements during the task. We confirmed that patients did not immediately stop the end of the motor task, and remained moving during active tasks. We suspect that these extra movements might be due to difficulties in motor control by stroke patients; the patients may not be able to stop moving in a timely manner as healthy individuals do. Also, we suspect that the one-second stay period in a resting position after the motor task might be too short for stroke patients to follow. Thus, the patients seemed to recognize the stay period as not a stop but instead used it to

1534-4320 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSRE.2014.2356472, IEEE Transactions on Neural Systems and Rehabilitation Engineering

TNSRE-2013-00345.R1 enable a smooth transition from the motor task to a return position. Therefore, ERD of SMRs associated with active movement may be sustained during the stay period following active movement (see Fig. 5). On the contrary, for the passive motor task, ERD may have diminished quickly after the motor task period because patients did not have to volitionally stop movement. Instead, they did not receive proprioceptive feedback. These observations suggest that incorporating a holding task into the design of motor tasks may lead to better assessment of patient active engagement. There were slight changes in power during the pre-movement period especially during the passive task. As these changes occurred particularly during the more difficult supination movement, we suspect that some on-going neural activity in preparation for the supination movement may have caused these pre-movement changes in sensorimotor rhythms. However, further investigations are required to address this issue. Our results revealed the ability to classify whether stroke patients were involved in training tasks actively or passively (average accuracy of single-trial classification was 80.7 ± 0.1% for grasping and 82.8 ± 0.1% for supination). There was no significant difference in classification results between grasping and supination. Thus, this suggests that our method can be applied to various upper limb rehabilitation tasks. However, different tie-up methods of hand impairment, movement speed, and the range of movement of the device could influence the results. In future experiments, it would be valuable to investigate the relationship between individual differences in classification performance, motor function measures, and brain lesion characteristics. While it was beyond this study’s initial scope, future investigations will take this data into account. We will also work to develop an online engagement monitoring system and investigate how one might adjust the design of rehabilitation protocols to encourage more active patient participation in rehabilitation training. The temporal resolution of this active engagement detection system will depend on the trial length of the rehabilitation protocol. From our results, for instance, a two second motor task period would be necessary to determine active engagement in real time. Our target application is different from typical motor intention detection devices used to initiate robotic assistance [27]. The primary application of our findings is to monitor whether stroke patients are actively participating in motor training tasks during robot-assisted therapy. In robot-assisted passive training, patients often do not pay much attention to training. This may reduce the effects of rehabilitation due to lack of motivation and feedback [62]. In clinical therapy, therefore, it is important to encourage patients to be actively engaged in a training task even though their limbs are passively moved. Monitoring active engagement may be better achieved by analyzing brain activity because active engagement is mostly a cognitive effort, making limb movements indistinguishable from passive movements. EMG or force sensing would also be unstable and inconsistent due to motor

9 impairment. Our study focused on this point and aimed to develop an EEG-based active engagement monitoring system to provide real-time feedback of active engagement, thereby maintaining patient motivation throughout therapy. Latent sensorimotor function can be retained through goal-directed therapy, even when a long time has passed following a stroke [36]. Rehabilitation goals may be better achieved when encouraging motivation and active engagement in stroke patients. For stroke rehabilitation therapy, an online monitoring tool that can deliver real-time feedback on performance and engagement level may lead to better rehabilitation outcomes for patients. Patient engagement may be further optimized when using an online monitoring tool with additional tools like video games and virtual reality [63], [64]. We recruited patients who were categorized as moderate or mild according to FMA scoring. However, severe stroke patients may also benefit from our proposed system. Although the present study recruited stroke patients with moderate and mild FMA scores and obtained obviously active or passive training data to determine the feasibility of active engagement monitoring with EEG, the experimental protocol can easily be modified to collect data even in the absence of physical movement, as would be expected in severely affected patients. For instance, a training paradigm could be developed in which active and passive movements are discriminated by visual cues to obtain training data for these two classes for further analysis. We would expect to observe similar ERD/ERS patterns of sensorimotor rhythms to those shown in the present study. A follow-up clinical study will investigate this possibility in severe stroke patients. V. CONCLUSIONS In the present study, we investigated whether we could assess active engagement in stroke patients during rehabilitation training using a non-invasive BCI. We observed that in bilateral sensorimotor cortical areas and the SMA, active movement induced larger ERD in the beta band than passive movement. A larger ERD associated with active movement was observed when participants executed motor tasks. We extracted spectral features of sensorimotor rhythms in the regions above and classified them into active or passive motor tasks on each trial. Classification accuracy was 80.7 ± 0.1% for grasping and 82.8 ± 0.1% for supination. Our results demonstrate the importance and feasibility of developing an online monitoring system of active engagement for stroke patients during motor rehabilitation training. ACKNOWLEDGMENT The authors would like to thank all patients for their participation in the data acquisitions, the editors and anonymous reviewers who have given insightful comments. REFERENCES [1]

W. H. Organization. (2013, July). The top 10 causes of death [Online]. Available: http://www.who.int/mediacentre/factsheets/fs310/en/index.html

1534-4320 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSRE.2014.2356472, IEEE Transactions on Neural Systems and Rehabilitation Engineering

TNSRE-2013-00345.R1 [2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

J. Toole, "Vascular diseases. Etiology and pathogenesis," in Merritt's Textbook of Neurology, 7th Ed., L. Rowland, Eds., Philadelphia: Lea & Febiger, 1984, p. 152. S. A. Sharp, and B. J. Brouwer, “Isokinetic strength training of the hemiparetic knee: effects on function and spasticity,” Arch. Phys. Med. Rehabil., vol. 78, no. 11, pp. 1231-1236, Nov. 1997. E. B. Petcu, T. Kocher, A. Kuhr, A.-M. Buga, I. Kloting, J. G. Herndon, C. Kessler, and A. Popa-Wagner, “Mild systemic inflammation has a neuroprotective effect after stroke in rats,” Curr. Neurovasc. Res., vol. 5, no. 4, pp. 214-223, Nov. 2008. B. E. Fisher, and K. J. Sullivan, “Activity-dependent factors affecting poststroke functional outcomes,” Top. Stroke Rehabil., vol. 8, no. 3, pp. 31-44, 2001. J. D. Schaechter, “Motor rehabilitation and brain plasticity after hemiparetic stroke,” Prog. Neurobiol., vol. 73, no. 1, pp. 61-72, May 2004. R. Parry, N. Lincoln, and C. Vass, “Effect of severity of arm impairment on response to additional physiotherapy early after stroke,” Clin. Rehabil., vol. 13, no. 3, pp. 187-198, Jun. 1999. R. W. Teasell, N. C. Foley, S. K. Bhogal, and M. R. Speechley, “An evidence-based review of stroke rehabilitation,” Top. Stroke Rehabil., vol. 10, no. 1, pp. 29-58, Jul. 2003. L. E. Kahn, P. S. Lum, W. Z. Rymer, and D. J. Reinkensmeyer, “Robot-assisted movement training for the stroke-impaired arm: Does it matter what the robot does?,” J. Rehabil. Res. Develop., vol. 43, no. 5, pp. 619, Aug./Sep. 2006. V. Squeri, L. Masia, P. Giannoni, G. Sandini, and P. Morasso, “Wrist Rehabilitation in Chronic Stroke Patients by Means of Adaptive, Progressive Robot Aided Therapy,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. PP, no. 99, pp. 1, Mar. 2013. W.-w. Liao, C.-y. Wu, Y.-w. Hsieh, K.-c. Lin, and W.-y. Chang, “Effects of robot-assisted upper limb rehabilitation on daily function and real-world arm activity in patients with chronic stroke: a randomized controlled trial,” Clin. Rehabil., vol. 26, no. 2, pp. 111-120, Feb. 2012. J. J. Daly, Y. Fang, E. M. Perepezko, V. Siemionow, and G. H. Yue, “Prolonged cognitive planning time, elevated cognitive effort, and relationship to coordination and motor control following stroke,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 14, no. 2, pp. 168-171, Jun. 2006. L. Petrella, M. McColl, T. Krupa, and J. Johnston, “Returning to productive activities: Perspectives of individuals with long-standing acquired brain injuries,” Brain Inj., vol. 19, no. 9, pp. 643-655, Aug. 2005. M. Ylvisaker, K. Mcpherson, N. Kayes, and E. Pellett, “Metaphoric identity mapping: Facilitating goal setting and engagement in rehabilitation after traumatic brain injury,” Neuropsychol. Rehabil., vol. 18, no. 5-6, pp. 713-741, Oct. 2008. L. Marchal-Crespo, R. Zimmermann, O. Lambercy, J. Edelmann, M. Fluet, M. Wolf, R. Gassert, and R. Riener, “Motor execution detection based on autonomic nervous system responses,” Physiol. Meas., vol. 34, no. 1, pp. 35-51, Dec. 2013. R. A. R. C. Gopura, and K. Kiguchi, "EMG-based control of an exoskeleton robot for human forearm and wrist motion assist," in Proc. Int. Conf. IEEE Robotics Automat., Pasadena, 2008, pp. 731-736. K. Suzuki, G. Mito, H. Kawamoto, Y. Hasegawa, and Y. Sankai, “Intention-based walking support for paraplegia patients with robot suit HAL,” Adv. Robot., vol. 21, no. 12, pp. 1441-1469, Jan. 2007. G. C. Rodriguez, M. J. Fraile, T. J. Perez, and F. P. Rivera, "Bio cooperative robotic platform for motor function recovery of the upper limb after stroke," in Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., Buenos Aires, Argentina, 2010, p. 4472. D. D. Ebaugh, P. W. McClure, and A. R. Karduna, “Three-dimensional scapulothoracic motion during active and passive arm elevation,” Clin. Biomech., vol. 20, no. 7, pp. 700-709, Aug. 2005. C. I. Price, P. Franklin, H. Rodgers, R. H. Curless, and G. R. Johnson, “Active and passive scapulohumeral movement in healthy persons: a comparison,” Arch. Phys. Med. Rehabil., vol. 81, no. 1, pp. 28-31, Jan. 2000. D. K. Sommerfeld, E. U.-B. Eek, A.-K. Svensson, L. W. Holmqvist, and M. H. von Arbin, “Spasticity after stroke its occurrence and association with motor impairments and activity limitations,” Stroke, vol. 35, no. 1, pp. 134-139, Jan. 2004. I. O. Sorinola, C. M. White, D. N. Rushton, and D. J. Newham, “Electromyographic response to manual passive stretch of the hemiplegic wrist: accuracy, reliability, and correlation with clinical

10 spasticity assessment and function,” Neurorehabil. Neural Repair, vol. 23, no. 3, pp. 287-294, Mar./Apr. 2009. [23] J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, “Brain–computer interfaces for communication and control,” Clin. Neurophysiol., vol. 113, no. 6, pp. 767-791, May 2002. [24] S. Ortigue, J. C. Thompson, R. Parasuraman, and S. T. Grafton, “Spatio-Temporal Dynamics of Human Intention Understanding in Temporo-Parietal Cortex: A Combined EEG/fMRI Repetition Suppression Paradigm,” PLoS One, vol. 4, pp. 6962, Sep. 2009. [25] L. Naci, M. M. Monti, D. Cruse, A. Kubler, B. Sorger, R. Goebel, B. Kotchoubey, and A. M. Owen, “Brain-Computer Interfaces for Communication with Nonresponsive Patients,” Ann. Neurol., vol. 72, no. 3, pp. 312-323, May 2012. [26] N. Birbaumer, T. Hinterberger, A. Kubler, and N. Neumann, “ The thought-translation device (TTD): neurobehavioral mechanisms and clinical outcome,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 11, no. 2, pp. 120-123, Jun. 2003. [27] K. K. Ang, and C. Guan, “Brain-Computer Interface in Stroke Rehabilitation,” J. Comput. Sci. Eng., vol. 7, no. 2, pp. 139-146, Jun. 2013. [28] G. Prasad, P. Herman, D. Coyle, S. McDonough, and J. Crosbie, "Using motor imagery based brain-computer interface for post-stroke rehabilitation," in Proc. 4th Int. Conf. IEEE/EMBS Neural Eng., Antalya, Turkish, 2009, pp. 258-262. [29] K. K. Ang, C. Guan, K. S. G. Chua, B. T. Ang, C. Kuah, C. Wang, K. S. Phua, Z. Y. Chin, and H. Zhang, "Clinical study of neurorehabilitation in stroke using EEG-based motor imagery brain-computer interface with robotic feedback," in Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., Buenos Aires, Argentina, 2010, pp. 5549-5552. [30] A. Ramos-Murguialday, D. Broetz, M. Rea, L. Läer, Ö. Yilmaz, F. L. Brasil, G. Liberati, M. R. Curado, E. Garcia-Cossio, and A. Vyziotis, “Brain-machine-interface in chronic stroke rehabilitation: A controlled study,” Ann. Neurol., Jul. 2013. [31] M. Gomez-Rodriguez, M. Grosse-Wentrup, J. Hill, A. Gharabaghi, B. Scholkopf, and J. Peters, "Towards brain-robot interfaces in stroke rehabilitation," in Proc. Int. Conf. IEEE Rehabil. Robotics, ETH Zurich Science City, Switzerland, 2011, pp. 1-6. [32] V. Kaiser, I. Daly, F. Pichiorri, D. Mattia, G. R. Müller-Putz, and C. Neuper, “Relationship between electrical brain responses to motor imagery and motor impairment in stroke,” Stroke, vol. 43, no. 10, pp. 2735-2740, Oct. 2012. [33] H. Kang, W. Park, J.-H. Kang, G.-H. Kwon, S.-P. Kim, and L. Kim, "A neural analysis on motor imagery and passive movement using a haptic device," in Proc. Int. Conf. IEEE Control, Automation and Systems, Jeju Island, Korea, 2012, pp. 1536-1541. [34] G. Pfurtscheller, and F. Lopes da Silva, “Event-related EEG/MEG synchronization and desynchronization: basic principles,” Clin. Neurophysiol., vol. 110, no. 11, pp. 1842-1857, Nov. 1999. [35] M. A. Dimyan, and L. G. Cohen, “Neuroplasticity in the context of motor rehabilitation after stroke,” Nat. Rev. Neurol., vol. 7, no. 2, pp. 76-85, Feb. 2011. [36] B. H. Dobkin, “Strategies for stroke rehabilitation,” Lancet Neurol., vol. 3, no. 9, pp. 528-536, Sep. 2004. [37] P. Langhorne, J. Bernhardt, and G. Kwakkel, “Stroke rehabilitation,” Lancet, vol. 377, no. 9778, pp. 1693-1702, May 2011. [38] N. Takeuchi, and S.-I. Izumi, “Rehabilitation with Poststroke Motor Recovery: A Review with a Focus on Neural Plasticity,” Stroke research and treatment, vol. 2013, pp. 1-13, 2013. [39] W. Park, S. Shin, H. Cho, S. Park, and L. Kim, "Enhanced hand rehabilitation using a haptic interface," in Proc. Int. Conf. IEEE Consumer Electron., Las Vegas, 2011, pp. 55-56. [40] W. Park, W. Jeong, G.-H. Kwon, Y.-H. Kim, and L. Kim, "A Rehabilitation Device to Improve the Hand Grasp Function of Stroke Patients using a Patient-Driven Approach," in Proc. Int. Conf. IEEE Rehabil. Robotics, Seattle, 2013. [41] D. Tombari, I. Loubinoux, J. Pariente, A. Gerdelat, J.-F. Albucher, J. Tardy, E. Cassol, and F. Chollet, “A longitudinal fMRI study: in recovering and then in clinically stable sub-cortical stroke patients,” Neuroimage, vol. 23, no. 3, pp. 827-839, Nov. 2004. [42] A. Ramos-Murguialday, M. Schürholz, V. Caggiano, M. Wildgruber, A. Caria, E. M. Hammer, S. Halder, and N. Birbaumer, “Proprioceptive feedback and brain computer interface (BCI) based neuroprostheses,” PLoS One, vol. 7, no. 10, pp. 1-10, Oct. 2012.

1534-4320 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSRE.2014.2356472, IEEE Transactions on Neural Systems and Rehabilitation Engineering

TNSRE-2013-00345.R1 [43] W. Park, J.-H. Kang, G. Kwon, L. Kim, and S.-P. Kim, "Sample-by-Sample Detection of Movement Intention from EEG Using a Classifier with Optimized Decision Parameters," in Converging Clinical Eng. Res. Neurorehabilitation, Heidelberg, Berlin, 2013, pp. 653-658. [44] A. Delorme, and S. Makeig, “EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis,” J. Neurosci. Methods, vol. 134, no. 1, pp. 9-21, 2004. [45] S. Vorobyov, and A. Cichocki, “Blind noise reduction for multisensory signals using ICA and subspace filtering, with application to EEG analysis,” Biol. Cybern., vol. 86, no. 4, pp. 293-303, Nov. 2002. [46] C. Binnie, R. Cooper, F. Mauguiere, P. Prior, and B. Tedman, EEG, Paediatric Neurophysiology, Special Techniques and Applications, 1st ed., Amsterdam, Netherland: Elsevier, 2003. [47] A. Kübler, F. Nijboer, J. Mellinger, T. M. Vaughan, H. Pawelzik, G. Schalk, D. McFarland, N. Birbaumer, and J. R. Wolpaw, “Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface,” Neurology, vol. 64, no. 10, pp. 1775-1777, May 2005. [48] G. McLachlan, Discriminant analysis and statistical pattern recognition, New York: Wiley, 2004. [49] R. Kohavi, "A study of cross-validation and bootstrap for accuracy estimation and model selection," in Proc. Int. Joint Conf. Artificial Intell., San Francisco, 1995, pp. 1137-1145. [50] C. S. Nam, Y. Jeon, Y.-J. Kim, I. Lee, and K. Park, “Movement imagery-related lateralization of event-related (de) synchronization (ERD/ERS): Motor-imagery duration effects,” Clin. Neurophysiol., vol. 122, no. 3, pp. 567-577, Mar. 2011. [51] X. Liao, D. Yao, D. Wu, and C. Li, “Combining spatial filters for the classification of single-trial EEG in a finger movement task,” IEEE Trans. Biomed. Eng., vol. 54, no. 5, pp. 821-831, May 2007. [52] O. Bai, P. Lin, S. Vorbach, J. Li, S. Furlani, and M. Hallett, “Exploration of computational methods for classification of movement intention during human voluntary movement from single trial EEG,” Clin. Neurophysiol., vol. 118, no. 12, pp. 2637-2655, Dec. 2007. [53] S. L. Gonzalez, R. Grave de Peralta, G. Thut, J. d. R. Millán, P. Morier, and T. Landis, “Very high frequency oscillations (VHFO) as a predictor of movement intentions,” Neuroimage, vol. 32, no. 1, pp. 170-179, 2006. [54] C. Weiller, M. Jüptner, S. Fellows, M. Rijntjes, G. Leonhardt, S. Kiebel, S. Müller, H. Diener, and A. Thilmann, “Brain representation of active and passive movements,” Neuroimage, vol. 4, no. 2, pp. 105-110, Jun. 1996. [55] T. Mima, N. Sadato, S. Yazawa, T. Hanakawa, H. Fukuyama, Y. Yonekura, and H. Shibasaki, “Brain structures related to active and passive finger movements in man,” Brain, vol. 122, no. 10, pp. 1989-1997, Oct. 1999. [56] M. Alegre, A. Labarga, I. Gurtubay, J. Iriarte, A. Malanda, and J. Artieda, “Beta electroencephalograph changes during passive movements: sensory afferences contribute to beta event-related desynchronization in humans,” Neurosci. Lett., vol. 331, no. 1, pp. 29-32, Oct. 2002. [57] V. Kaiser, A. Kreilinger, G. R. Müller-Putz, and C. Neuper, “First Steps Toward a Motor Imagery Based Stroke BCI: New Strategy to Set up a Classifier,” Front. Neurosci., vol. 5, pp. 86-86, Jul. 2011. [58] G. Müller, C. Neuper, R. Rupp, C. Keinrath, H. Gerner, and G. Pfurtscheller, “Event-related beta EEG changes during wrist movements induced by functional electrical stimulation of forearm muscles in man,” Neurosci. Lett., vol. 340, no. 2, pp. 143-147, Apr. 2003. [59] E. Formaggio, S. F. Storti, I. B. Galazzo, M. Gandolfi, C. Geroin, N. Smania, L. Spezia, A. Waldner, A. Fiaschi, and P. Manganotti, “Modulation of event-related desynchronization in robot-assisted hand performance: brain oscillatory changes in active, passive and imagined movements,” J. NeuroEng. Rehabil., vol. 10, no. 24.10, pp. 1186, 2013. [60] M. T. Jurkiewicz, W. C. Gaetz, A. C. Bostan, and D. Cheyne, “Post-movement beta rebound is generated in motor cortex: evidence from neuromagnetic recordings,” Neuroimage vol. 32, no. 3, pp. 1281-1289, Sep. 2006. [61] H.-C. Chang, P.-L. Lee, M.-T. Lo, Y.-T. Wu, K.-W. Wang, and G.-Y. Lan, “Inter-Trial Analysis of Post-Movement Beta Activities in EEG Signals Using Multivariate Empirical Mode Decomposition,” IEEE Trans. Neural Syst. Rehabil. Eng., vol. 21, no. 4, pp. 607-615, Jul. 2013. [62] J. J. Daly, and J. R. Wolpaw, “Brain–computer interfaces in neurological rehabilitation,” Lancet Neurol., vol. 7, no. 11, pp. 1032-1043, Oct. 2008. [63] J. W. Burke, M. McNeill, D. K. Charles, P. J. Morrow, J. H. Crosbie, and S. M. McDonough, “Optimising engagement for stroke rehabilitation using serious games,” Visual Comput., vol. 25, no. 12, pp. 1085-1099, Dec. 2009.

11 [64] A. S. Merians, D. Jack, R. Boian, M. Tremaine, G. C. Burdea, S. V. Adamovich, M. Recce, and H. Poizner, “Virtual reality–augmented rehabilitation for patients following stroke,” Phys. Ther., vol. 82, no. 9, pp. 898-915, Sep, 2002.

Wanjoo Park is a research scientist at the Center for Bionics, Korea Institute of Science and Technology in Seoul since 2008. He entered the Ph.D. program of the Department of Brain and Cognitive Engineering at Korea University in spring 2012. He received his B.S. in electronics engineering from the Korea Aerospace University, Korea, in 2006, and M.S. in electronics engineering from Sogang University, Korea, in 2008. His research interests include haptics, brain-computer interfaces, neuromarketing and rehabilitation engineering.

Gyu Hyun Kwon is a senior research scientist at the Center for Bionics, Korea Institute of Science and Technology in Seoul, and an assistant professor of HCI & Robotics at the University of Science and Technology. He received a Ph.D. in Industrial and Systems Engineering from Virginia Tech in 2010.

Da-Hye Kim began her M.S. and Ph.D. course in the Department of Brain and Cognitive Engineering at Korea University in spring 2013. Until 2012, she had been a researcher at the National Rehabilitation Center. Since 2013, she has performed research in the Center for Bionics at the Korea Institute of Science and Technology. She interests research for neural network, BCI, and rehabilitation.

Yun-Hee Kim is a MD and PhD working as a professor in the Department of Physical & Rehabilitation Medicine at the Samsung Medical Center, Sungkyunkwan University School of Medicine since 2003. Her research interests include neuroplasticity research using functional neuroimaging and noninvasive brain stimulation for stroke patients.

1534-4320 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TNSRE.2014.2356472, IEEE Transactions on Neural Systems and Rehabilitation Engineering

TNSRE-2013-00345.R1

Sung-Phil Kim is an assistant professor working in Department of Human and Systems Engineering at the Ulsan National Institute of Science and Technology. He earned Ph.D. and M.S. degrees from the Department of Electrical and Computer Engineering at the University of Florida and conducted postdoctoral research at Brown University. His research interests include brain-computer interfaces, computational neuroscience and statistical signal processing.

Laehyun Kim is a principal research scientist and has been working at the Center for Bionics at the Korea Institute of Science Technology since 2003. His research interests include haptics, medical robotics, rehabilitation, and BCI. He received a PhD degree in computer science from the University of Southern California.

1534-4320 (c) 2013 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

12

Assessment of cognitive engagement in stroke patients from single-trial EEG during motor rehabilitation.

We propose a novel method for monitoring cognitive engagement in stroke patients during motor rehabilitation. Active engagement reflects implicit moti...
13MB Sizes 0 Downloads 7 Views