Journal of Neuroscience Methods 253 (2015) 10–17

Contents lists available at ScienceDirect

Journal of Neuroscience Methods journal homepage: www.elsevier.com/locate/jneumeth

Computational Neuroscience

Sequence detection analysis based on canonical correlation for steady-state visual evoked potential brain computer interfaces Lei Cao a,b , Zhengyu Ju a , Jie Li a , Rongjun Jian a , Changjun Jiang a,∗ a b

Department of Computer Science and Technology, Tongji University, 201804 Shanghai, China Institute of Medical Psychology and Behavioral Neurobiology, University of Tuebingen, D-72074 Tuebingen, Germany

h i g h l i g h t s • A novel approach based on sequence detection (SD) is proposed for improving the performance of SSVEP recognition. • In comparison with other resultful algorithms, experimental accuracy of the SD approach was better than those using other methods. • It was implicated that our approach could improve the speed of BCI system in contrast to other methods.

a r t i c l e

i n f o

Article history: Received 17 March 2015 Received in revised form 15 May 2015 Accepted 18 May 2015 Available online 23 May 2015 Keywords: Brain computer interface (BCI) Steady-state visual evoked potential (SSVEP) Sequence detection Canonical correlation analysis (CCA)

a b s t r a c t Background: Steady-state visual evoked potential (SSVEP) has been widely applied to develop brain computer interface (BCI) systems. The essence of SSVEP recognition is to recognize the frequency component of target stimulus focused by a subject significantly present in EEG spectrum. New method: In this paper, a novel statistical approach based on sequence detection (SD) is proposed for improving the performance of SSVEP recognition. This method uses canonical correlation analysis (CCA) coefficients to observe SSVEP signal sequence. And then, a threshold strategy is utilized for SSVEP recognition. Results: The result showed the classification performance with the longer duration of time window achieved the higher accuracy for most subjects. And the average time costing per trial was lower than the predefined recognition time. It was implicated that our approach could improve the speed of BCI system in contrast to other methods. Comparison with existing method(s): In comparison with other resultful algorithms, experimental accuracy of SD approach was better than those using a widely used CCA-based method and two newly proposed algorithms, least absolute shrinkage and selection operator (LASSO) recognition model as well as multivariate synchronization index (MSI) method. Furthermore, the information transfer rate (ITR) obtained by SD approach was higher than those using other three methods for most participants. Conclusions: These conclusions demonstrated that our proposed method was promising for a highspeed online BCI. © 2015 Elsevier B.V. All rights reserved.

1. Introduction Steady-state visual evoked potentials (SSVEPs) are elicited with one certain frequency by rapidly repetitive flickering stimulus. The SSVEP contains a series of discrete frequency components. The frequency component is derived from the fundamental frequency of the visual stimulus and its harmonics (Middendorf et al., 2000; Gao

∗ Corresponding author. Tel.: +86 02169589867; fax: +86 02169589867. E-mail addresses: [email protected] (L. Cao), [email protected] (Z. Ju), [email protected] (J. Li), [email protected] (R. Jian), [email protected] (C. Jiang). http://dx.doi.org/10.1016/j.jneumeth.2015.05.014 0165-0270/© 2015 Elsevier B.V. All rights reserved.

et al., 2003; Müller-Putz et al., 2005; Cecotti, 2010; Gollee et al., 2010). This signal is often extracted noninvasively from electroencephalography (EEG) for brain computer interfaces (BCIs). Recent studies indicated that SSVEPs had higher classification accuracy than other EEG patterns, such as P300 and event-related desynchronization/synchronization (ERD/ERS) (Friman et al., 2007; Parini et al., 2009; Guger et al., 2012). And a great number of SSVEP-based BCIs had been developed for human–computer communication (Wolpaw et al., 2000, 2002; Moore, 2003; Kelly et al., 2005; Xia et al., 2013). Several classification methods based on frequency features had been proposed for SSVEP-based BCIs (Palaniappan et al., 2002; Lotte et al., 2007; Middendorf et al., 2000; Gao et al., 2003; Lalor

L. Cao et al. / Journal of Neuroscience Methods 253 (2015) 10–17

et al., 2005; Mukesh et al., 2006; Muller-Putz and Pfurtscheller, 2008; Liavas et al., 1998; Müller-Putz et al., 2005, 2008; Wu and Yao, 2008). The traditional frequency domain analysis for SSVEP detection was power spectral density-based analysis (PSDA) (Middendorf et al., 2000; Gao et al., 2003; Lalor et al., 2005; Mukesh et al., 2006; Liavas et al., 1998). Power spectral density was evaluated from the user’s EEG signal within a time window. It could be estimated by fast Fourier transform (FFT). Subsequently, the frequency corresponding to the peak value was considered as the evoked target stimulus. Müller-Putz et al. (2005, 2008) made use of distinctive sensitive learning vector quantization (DSLVQ) and lock-in analyzer system (LAS) to improve the feature extraction based on spectral information. This strategy significantly increased the classification accuracy compared to PSDA. Moreover, an assisted closed loop (ACL) algorithm was present for optimizing spectrum-based SSVEP recognition (Fernandez-Vargas et al., 2013). Wu and Yao (2008) proposed the stability coefficient (SC) model to improve the performance of SSVEP-based BCIs within a short time window of EEG signals. In addition to spectral features, it was demonstrated that phasecoded information was feasible to decode SSVEP signals (Jia et al., 2011; Manyakov et al., 2012, 2013). Typically, Jia et al. (2011) presented frequency and phase mixed coding to improve the target identification accuracy. Although useful, this approach was still restricted by individual different phase-lags. The individual precise phase-lag was mainly dependent on the preliminary training for phase coding. Above algorithms required a mass of training samples to construct the classifier except for PSDA. However, PSDA was sensitive to external noise. Friman et al. (2007) proposed a minimum energy method (MEC) without calibration working for SSVEP detection. This method canceled the strong nuisance signals to obtain a better signal-to-noise ratio (SNR). Nevertheless, the computations cost had an adverse effect on the real-time BCI application. And canonical correlation analysis (CCA) method using channel covariance information was applied to increase the SNR and reduce the computational cost for online systems (Lin et al., 2006). CCA reflected the correlation relationship between EEG response signals and classical Fourier series at the stimulus frequency and its harmonics. Bin et al. (2009) used CCA algorithm to develop an online BCI system for detecting SSVEP signals without complicated training procedures. In addition, an unsupervised least absolute shrinkage and selection operator (LASSO) model was applied to recognize SSVEP signals to achieve the better effect than that of CCA in a short time window (e.g. 2 s) (Zhang et al., 2012). Recently, a new multivariate synchronization index (MSI) algorithm was proposed for SSVEP recognition (Zhang et al., 2014). This method showed better performance than CCA and MEC when using short length data and small number of channels. However, these methods just reflected the correlation between EEG time series and reference signals at a given time. It was difficult to judge the exact time point when the spectral feature achieved the steady state affected by individual difference, background noise and environmental factors. In this paper, we proposed an innovative frequency recognition approach based on sequence detection (SD), which made use of CCA coefficients for solving this problem. The method was widely used for predicting the exponent of the probability of symbol error in the communication engineering (Bussgang and Middleton, 1955; Hayes et al., 1982; Chaudhari et al., 2009). The theory of SD was proposed by Wald in 1943 (Wald, 1943). The statistical hypothesis of this method was meant any statistical test procedure which gave a rule, at any time point of the experiment, for making one of the following three decisions: (1) to accept the hypothesis, (2) to reject the hypothesis, and (3) to continue the experimental procedure for receiving additional information. Note that such a test experiment was carried out sequentially.

11

When the first or second decision was made at one certain time point, the experiment was terminated. Whereas the experiment was performed continuously while the third decision was made. This procedure had not been terminated until the first or second decision was made in one certain time window. In previous studies, Jin et al. (2011) regulated the time window of the average trial length to reduce the time costing of target detection for a P300-based BCI. Differently, another adaptive approach was used for regulating the time window of one subtrial and moving time window between consecutive calculations to reduce the recognition time (Wang et al., 2006). And a decision would be made if the same frequency was detected in several consecutive calculations. In our study, this SD method based on CCA coefficients was utilized for selecting the most probable stimulus frequency. The identical parameters were regulated for increasing the SNR and classification accuracy. Compared with the simple method of voting, our proposed method took advantage of the cumulative effect of multiple recognizing CCA coefficients to eliminate the noise disturbance. Then, a novel threshold strategy was used for making the decision of frequency detection. This methodology improved the systematical efficiency for SSVEP-based BCI under the condition of the simple parameters optimization. 2. Methodology The flowchart of sequence detection analysis based on canonical correlation is illustrated in Fig. 1. Raw data are segmented in several subtrials based on sequence detection. Then, it is calculated by CCA method and frequency component analysis is performed for target recognition. Other comparative methods are also listed in this section. 2.1. Instantaneous probability analysis based on CCA The instantaneous probability represents the probability that the frequency of the corresponding SSVEP component is recognized at the instantaneous time point. The effectiveness of CCA has been demonstrated in Bin et al. (2009). Thus, we use CCA coefficients to reflect the instantaneous state in our method. CCA is a wellknown multivariable method for two sets of data, which may have the underlying correlation. The working hypothesis of the method is that the source signal for SSVEP, X, is the output of a linear system with the stimulus signal, Y, as the input. Y, at a certain frequency f can be decomposed into the Fourier series of its harmonics (sin(2ft), cos(2ft), sin(4ft), . . .):

Y=

⎧ sin(2ft) ⎪ ⎪ ⎪ ⎪ cos(2ft) ⎪ ⎪ ⎨ sin(4ft) cos(4ft) ⎪ ⎪ ⎪ ⎪ sin(6ft) ⎪ ⎪ ⎩

t=

1 2 T , , . . ., S S S

(1)

cos(6ft)

where f is the fundamental frequency, T is the number of sampling points and S is the sample rate. The algorithm can find a pair of linear combinations, x = XT WX and y = YT WY , for X and Y, to maximize the correlation between two canonical variables, x and y, by solving the following optimization problem: max (x, y)

WX ,WY



=



E[xT y]

=

E[xT x]E[yT y] E[WXT XY T WY ] E[WXT XX T WX ]E[WYT YY T WY ]

(2)

12

L. Cao et al. / Journal of Neuroscience Methods 253 (2015) 10–17

Fig. 1. The flowchart of sequence detection analysis based on canonical correlation.

where S(f) is the SD coefficient. The X and Y are defined in (1). In the off-line analysis, the trial will be invalid if fs is under the threshold T after all subtrials are calculated for SD. Thus, this trial should be discarded and the classification rate will be computed by the valid statistical result. However, there are at least 80% of trials (60 × 0.8 = 48) achieving the valid level for the off-line analysis. Fig. 2. The principle of data segmentation in one trial. The subtrial is constituted by the EEG data within a time window (TW) and it is slid with a moving window (MW) between consecutive subtrials.

The canonical correlation  is utilized as the CCA coefficient obtained with the frequency of reference signals. The detailed work can be reviewed in Lin et al. (2006). 2.2. Frequency component analysis based on SD In a trial of our experiment, the EEG signal is divided into several sequential subtrials for SD. Fig 2 illustrates the principle of data segmentation. The subtrial is constituted by the EEG data within a time window (TW) and it is slid with a moving window (MW) between consecutive subtrials. In a subtrial, we compute the CCA coefficients  firstly. Then the instantaneous probability ratio Pri of the stimulus frequency i is calculated as Pr i =

i M

(3)

where M is defined as



 n j

M=

n

j = 1, 2, . . ., n

(4)

And n is the number of stimulus frequencies. After m subtrials, the SD coefficient Sfm , which denotes the probability ratio of the stimulus frequency f, can be formulated as Sfm = Pr 1f × Pr 2f × · · · × Pr m−1 f

(5)

A defined threshold of SD, T, is used for making a decision. If Sfm ≥ T , the stimulus frequency f is determined as the target frequency. If Sfm < T , the SD is continued to compute the Sfm+1 in the next subtrial. In this paper, the TW and MW are respectively set to a number of 2, 3, 4 and 5 as well as one of five parameters (i.e., 0.2, 0.4, 0.6, 0.8 and 1) for data analysis. And the threshold T is ranged from 1.1 to 3 with the step size of 0.1 to observe the experimental performance. 2.3. Recognition strategy The core problem in the SSVEP recognition is to determine the exact time point when the evoked potential reaches the steady state. Suppose that there are K stimulus frequencies f1 , f2 , . . ., fK and the EEG signal has been acquired from N channels within L s window. In our strategy, the stimulus frequency fs , corresponding with the flickering LED button concerned by the subject, should satisfy fs = maxS(f ) S(f ) > T, f = f1 , f2 , . . ., fK f

(6)

2.4. Comparison methods In our study, we implemented the CCA, LASSO and MSI methods to compare the results with that of our proposed SD method based on CCA coefficients. CCA is a famous multivariable statistical method used for identifying the correlation relationship between two sets of data. This algorithm has been introduced in Lin et al. (2006). The canonical correlation coefficient i reflects the relevance between the stimulus source of ith frequency fi and acquired EEG response signals. In our study, the fundamental frequency and its two harmonics are still used for the CCA. Then, the target stimulus frequency ftarget can be recognized as follows: ftarget = max(1 , 2 , . . ., K )

(7)

fi

LASSO, as a popular model selection and shrinkage estimation method, can provide an analytical solution and a low-variance estimate with high interpretability for a linear regression due to its sparsity constraint. This method has been applied for SSVEP recognizing proposed by Zhang et al. (2012). The LASSO model considers the EEG signal response x as the linear output of standard squarewave signals Y elicited by flickering lights at different frequencies: x = Yˇ + ε

(8)

where ε represents a noise vector with the zero mean and constant variance. And the LASSO estimate is used for solving the optimizaˆ given by: tion sparse vector ˇ ˆ = argmin(x − Yˇ2 + ˇ ) ˇ 2 1

(9)

ˇ

where  1 and  2 denote the l1 -norm and l2 -norm respectively. ˆ  is a penalty parameter which encourages a sparse solution ˇ. This problem can be resolved by quadratic programming. Then the contribution degrees of all these stimulus frequencies and their harˆ Finally, the frequency monics to the EEG signal are computed by ˇ. corresponding to the maximal value of the contribution degree is selected as the target stimulus. MSI mainly reflects the synchronization between the actual mixed signals and the reference signals (Zhang et al., 2014). A novel S-estimator is proposed as the index to measure the synchronization. Firstly, a transformed correlation matrix is calculated as

 R=

IN×N

C11−(1/2) C12C22−(1/2)

C22−(1/2) C21C11−(1/2)

INh ×Nh

(10)

L. Cao et al. / Journal of Neuroscience Methods 253 (2015) 10–17

13

Fig. 3. The maximum classification accuracies by the SD method at various time lengths of TW for all subjects. The corresponding MWs, thresholds and average time costing per trial are listed for further discussion.

where C11 =

1 XX T M

(11)

Here, P = N + Nh . Next, we can calculate the synchronization index between the signals and each reference signal Y and then obtain K indices S1 , S2 , . . ., SK . The target frequency T satisfies the following:

C22 =

1 YY T M

(12)

T = maxSi ,

C12 = C21 =

1 XY T M

(13)

The EEG signal is a matrix X of size N × M and the reference signal is a matrix Y of size Nh × M. Here, N is the number of channels. M is the number of samples. And Nh is the number of harmonics for the reference signal. X and Y are normalized to have a zero mean and unitary variance. I represents the identity matrix. Let 1 , 2 , . . ., P be the eigenvalues of matrix R. Then the synchronization index S can be computed as P

S =1+

i log(i )

i=1

log(P)

(14)

where i =

i



(15)

P

i=1

i

i

i = 1, 2, . . ., K

(16)

2.5. Subjects and experimental settings A SSVEP-based BCI system was developed for acquiring EEG data and an off-line analysis was performed. Fourteen healthy volunteers (12 males and 2 females), aged from 21 to 28, participated in our experiments. All of them had normal or corrected to normal vision and no experience with a BCI experiment. A high-performance bio-signal amplifier (g.Tec) was used to acquire scalp EEG signals. The signal was recorded from four channels POz, O1, Oz, O2 which were placed on the standard positions of the 10–20 international system. It was referenced to the unilateral ear. Impedances of all electrodes were kept below 5 k. EEG data were amplified, digitalized with a sampling frequency of 256 Hz, notchfiltered with 50 Hz and bandpass-filtered between 0.1 and 30 Hz. A LED stimulus panel was used for presenting flickering LED buttons at four frequencies of 6, 7, 8, 9 Hz. The subjects seated in a chair and paid attention to the center of the screen. In a run, a random digital sequence composed of 12 trials. For one trial, one of four numbers (i.e., 1, 2, 3, 4), was presented for instructing the subjects to

14

L. Cao et al. / Journal of Neuroscience Methods 253 (2015) 10–17

Fig. 4. The optimal recognition accuracies by LASSO, CCA, MSI and SD. The mean performance of each method is calculated and showed at the last column.

gazing the corresponding LED buttons. The subjects were instructed to gaze binocularly at one visual cue for 10 s and the following 0.5 s interval was given to the subject to shift his/her gaze. Each subject carried out five runs and had a rest to eliminate the affect of visual fatigue between two runs. 2.6. Evaluation methods In our study, the classification accuracy was adopted to evaluate all methods. The EEG data had 60 trials (12 × 5 =60) and valid trials were used for frequency recognition. In off-line analysis, all trials were valid for CCA, LASSO and MSI classification algorithms. And the validity of a trial by our method had been described in the above paragraph. The classification accuracy was defined as the percentage of valid trials whose classification results were consistent with stimulus frequencies. Let fs be the classification result determined by the recognizing method and fstim be the real stimulus frequency of a trial. Thus, the classification accuracy could be formulated as follows. number of valid trials(fs = fstim ) Acc = × 100% number of valid trials

(17)

At the same time, information transfer rate (ITR) (Cheng et al., 2002) was adopted to evaluate the communication capacity of our BCI in the study. The definition of ITR (Br ) is depicted as: Br = log2 N + P × log2 P + (1 − P) × log2

1−P N−1

(18)

where N denoted possible choices existing in one trial and each choice was of the identical probability to be recognized by the user. And P indicated the probability of selection. It was always invariant for each choice. Then, the ITR could be calculated by Eq. (11). 3. Results Firstly, the recognition precisions and average time costing per trial were analyzed by the SD method based on CCA coefficients for various TWs, MWs and thresholds. Fig. 3 presented the maximum classification accuracies with corresponding MWs, thresholds and average time costing per trial at various time lengths of TW for 14 subjects. It implied that the classification result with the longer duration of TW achieved the higher accuracy for most subjects. And the average time costing per trial was well lower than the predefined recognition time (10 s). Furthermore, Fig. 4 listed the optimal recognition accuracies by these four methods. The corresponding parameters of SD method were given in Table 1. This figure indicated that the classification accuracy by our proposed approach was better than other methods for most subjects. And the paired ttest and Bonferroni correction for classification accuracies further

Fig. 5. The chart shows the changing procedure of SD coefficients at all stimulus frequencies for Subject 1 in one trial. The SD coefficients of target frequency (i.e. 7 Hz) are less than those of the interference frequency (i.e. 6 Hz) at the first two subtrials. However, the SD coefficient of target frequency surpasses that of the interference frequency at the last subtrial by achieving the threshold.

demonstrated that the SD approach was significantly better than CCA (t = 2.665, p < 0.05) and LASSO (t = 8.130, p < 0.01) while there was no significant difference between MSI and SD (t = 1.334). However, the mean classification accuracy of SD was higher than that of MSI. Meanwhile, the change of SD coefficient in one trial was typically showed in Fig. 5 from Subject 1. The SD coefficients of target frequency (i.e. 7 Hz) were less than those of the interference frequency (i.e. 6 Hz) at the first two subtrials. However, the SD coefficient of target frequency surpassed that of the interference frequency at the last subtrial by achieving the threshold. The corresponding spectral characteristics of subtrials were presented in Fig. 6. It reflected the changing process of spectral powers for each frequencies. The largest ITRs were also enumerated for evaluating the performance of these four algorithms (see Table 2). For most participants, the largest ITRs of SD were higher than those of other algorithms. And the paired t-test and Bonferroni correction results indicated that the SD algorithm was significantly better than LASSO (t = 6.072, p < 0.01), CCA (t = 3.160, p < 0.05) and MSI (t = 4.228, p < 0.01). This result implicated that our approach could improve the speed of BCI system in contrast to other methods. 4. Discussion In our study, the cumulative effect of multiple recognizing coefficients may have contributed to the improvement for the classification accuracy in according with the principle of the SD method. For signal detection, the background noise produces the signal disturbance in a limited period of time. And the accumulative operation has a great tolerance to the fluctuation of CCA coefficients

L. Cao et al. / Journal of Neuroscience Methods 253 (2015) 10–17

15

Table 1 The maximum classification accuracies by the SD method at the parameters of TW, MW and threshold. Parameters

Sub 1

Sub 2

Sub 3

Sub 4

Sub 5

Sub 6

Sub 7

Sub 8

Sub 9

Sub 10

Sub 11

Sub 12

Sub 13

Sub 14

TW (s) MW (s) Threshold

5 1 1.9

5 0.8 1.9

5 1 2.9

4 0.8 2

5 1 2.6

4 1 2

5 0.8 2.9

2 0.6 3

4 0.8 2

5 1 1.8

5 1 3

5 1 2.7

5 0.6 2.8

5 1 2.1

Fig. 6. The spectral characteristics of subtrials corresponding to the above example of the changing procedure of SD coefficients. It reflected the changing process of spectral powers for each frequencies.

Table 2 The largest ITRs (bits/min) by four algorithms for 14 participants. Method

Sub 1

Sub 2

Sub 3

Sub 4

Sub 5

Sub 6

Sub 7

Sub 8

Sub 9

Sub 10

Sub 11

Sub 12

Sub 13

Sub 14

Mean

LASSO CCA MSI SD

5.11 5.56 6.51 6.48

1.35 3.58 2.82 3.67

1.50 6.77 5.41 10.12

0.93 4.38 4.15 6.33

4.94 19.43 20.10 21.61

1.96 7.91 5.41 9.64

1.72 12.86 9.81 13.63

4.75 17.36 17.14 29.29

0.81 9.78 11.75 11.56

2.14 9.78 6.14 11.08

9.65 12.45 11.09 15.67

3.73 10.22 6.14 11.92

0.59 4.48 4.15 5.54

2.52 10.22 9.23 10.49

2.98 9.63 8.56 11.93

caused by external noise. As shown in Fig. 5, the SD coefficients of target frequency were less than those of interference frequency at the first two subtrials. However, the SD coefficient of interference frequency did not achieve the threshold for target selection. Then, the interference of nontarget frequency was weakened and the target frequency was recognized by surpassing the threshold. It was clearly demonstrated that our recognizing strategy created greater robustness against noise interference. Moreover, our method accepted the final recognition result only when the EEG signal reached the steady state by our threshold strategy. The proper threshold strategy played an important role in our proposed approach to avoid noise interference. This principle indicated that the SD approach had a very high SNR. For EEG classification, an accuracy of 70% correctly selected characters was often considered as the index for satisfactory communication with a BCI (Furdea et al., 2009; Kübler et al., 2001; Nijboer et al., 2008; Brunner et al., 2011). From Fig. 4, the classification results of all subjects using our proposed approach achieved this standard value or above while there were several participants not to get the satisfaction accuracy of 70% for other methods. It was validated that the SD algorithm was more robust and applicable in comparison with other algorithms, especially for new BCI users. In our experiment, our selected channels were mainly located in the occipital region. And it was the first time all subjects experienced the BCI experiment. However, the number of the electrodes was less than those of other studies (Lin et al., 2006; Bin et al., 2009). For practical application, a wearable device should be lightweight and convenience. Few channels may be beneficial for developing easy-used detectors in the real environment. Therefore, the economical BCI system has to keep the balance between the portability and the measure precision. In this paper, the selection of fundamental frequencies was based on Pastor et al. (2003). In the recognition, CCA coefficients were selected as the basis of one evaluation in a subtrial. And our proposed approach employed the product of CCA coefficients

indirectly to improve the SNR. It efficiently solved the problem that how to use more than one coefficient, which was discussed in Lin et al. (2006). There had been many supervised classification methods, such as DLSVQ, LAS and joint time–frequency analysis (JTFA) (Müller-Putz et al., 2005; Parini et al., 2009), proposed and utilized for SSVEPbased BCI systems. The recognition accuracy by our approach was nearly equal to those of these algorithms. But our approach had the advantages of easy implementation and economical time cost. Furthermore, we compared our approach with three unsupervised classification methods (CCA, LASSO and MSI). Similarly, parameters calibration was used by these methods. And the experimental result indicated that our approach had the higher classification accuracy and ITR. Recently, several SSVEP-based hybrid BCIs had been developed for increasing classification accuracies (Allison et al., 2014; Wang et al., 2014; Pan et al., 2014). Our algorithm could be considered as a reliable method for improving the efficiency of these hybrid BCIs. Our approach has been demonstrated to be robust in the off-line analysis. The fact implied that it was feasible to use this method for online BCI system. The result of ITR included in Table 2, indicated that SD was more efficient than other methods in the aspect of information transmission. This property gave the operation facilitation for developing the online system. Moreover, the cost of the computation was negligible in this experiment. Computations could be performed in 0.094 s in Matlab 2011b environment on a notebook with dual 2.4 GHz CPU configured for four electrodes, 10 s time window and four candidate stimulus frequencies. The time loss could be acceptable for practical use. With this advantage of the communication speed, BCI users may communicate faster and avoid visual fatigue caused by staring at the flashing stimulus for a long period of time. In comparison with other representative publications (Bin et al., 2009; Cheng et al., 2002; Zhang et al., 2014), the ITR of our proposed method is lower than those of other methods. Nevertheless, it is

16

L. Cao et al. / Journal of Neuroscience Methods 253 (2015) 10–17

Table 3 The optimal parameters of the 10-fold cross validation for SD method. Parameters

Sub 1

Sub 2

Sub 3

Sub 4

Sub 5

Sub 6

Sub 7

Sub 8

Sub 9

Sub 10

Sub 11

Sub 12

Sub 13

Sub 14

TW (s) MW (s) Threshold

5 1 1.9

5 0.8 1.9

5 1 2.9

3 0.6 2.4

5 1 2.6

4 1 2

5 0.8 2.9

2 0.6 3

4 0.8 2

5 1 1.8

5 1 3

5 1 2.7

5 0.6 2.8

5 1 2.7

dependent on the limitation of possible choices and user selection. In our experiment, the number of possible choices, i.e. the number of stimulus frequencies (N = 4), was much less than those in the above mentioned publications (N = 6 or 8). And all the users were naive to BCI equipment and paradigm. As a result, the performance of ITR will be improved greatly if the possible choices are increased and the users are familiar with the BCI experiment. On the other hand, we have given the comparative result of the ITR between SD method and other representative algorithms based on the identical experimental condition. It was implicated that our proposed method is eligible for an online SSVEP-based BCI. In future, we may increase the number of visual stimuli like dual-frequency stimulation method (Hwang et al., 2013), to improve the ITR of our BCI system. From Fig. 3, we can find that, the values of parameters are subject-dependent under the condition of the maximum accuracy. Thus, it is difficult to select appropriate parameters for a new subject. If the values are too small, the BCI system may not achieve the optimum performance. And if they are too large, the time cost is not economical for communicating. Therefore, a validation procedure should be implemented to select the optimal parameter. Cross validation is an important technique for estimating the performance of a predictive parameters model. For our proposed method, a 10fold cross validation is implemented for parameters optimization. The result has been given in Table 3. By comparison, it is verified that the selective optimal parameters of cross validation are mostly consistent with those of the test performance (13 of 14 subjects). Generally, this step can be performed before real-time tasks for several seconds. The computational cost is acceptable for training model. In our feature work, the cross validation will be a good choice for parameters optimization. Furthermore, we use CCA coefficients as the evaluation index in a subtrial because they contain most of frequency features. However, SSVEP signals may have discontinuous phase transitions in fact. Consequently, we intend to combine frequency and phase information for instantaneous probability analysis. It may be effective for solving the problem of the distraction stimulation.

5. Conclusions This paper presented the SD method based on CCA coefficients for the first time. This approach improved the systematical efficiency for SSVEP-based BCI under the condition of the simple parameters optimization. Experimental results showed that our algorithm reached the higher classification accuracy and ITR than the widely used CCA algorithm, as well as newly proposed LASSO recognition model and MSI method. It was an effective improvement strategy for CCA-based SSVEP recognition in BCI systems.

Acknowledgements The work was supported by the Education Committee Funds for Scientific Innovation Program of Shanghai, China (Grant No. 12ZS033).

References Allison Z, Jin B, Zhang JY, Wang X. A four-choice hybrid p300/SSVEP BCI for improved accuracy. Brain Comput Interfaces 2014];1(1):17–26. Bin G, Gao X, Yan Z, Hong B, Gao S. An online multi-channel SSVEP-based brain–computer interface using a canonical correlation analysis method. J Neural Eng 2009];6(4):046002. Brunner C, Allison B, Altstätter C, Neuper C. A comparison of three brain–computer interfaces based on event-related desynchronization, steady state visual evoked potentials, or a hybrid approach using both signals. J Neural Eng 2011];8(2):025010. Bussgang J, Middleton D. Optimum sequential detection of signals in noise. IRE Trans Inf Theory 1955];1(3):5–18. Cecotti H. A self-paced and calibration-less SSVEP-based brain–computer interface speller. IEEE Trans Neural Syst Rehabil Eng 2010];18(2):127–33. Chaudhari S, Koivunen V, Poor HV. Autocorrelation-based decentralized sequential detection of OFDM signals in cognitive radios. IEEE Trans Signal Process 2009];57(7):2690–700. Cheng M, Gao X, Gao S, Xu D. Design and implementation of a brain–computer interface with high transfer rates. IEEE Trans Biomed Eng 2002];49(10):1181–6. Fernandez-Vargas J, Pfaff HU, Rodríguez FB, Varona P. Assisted closed-loop optimization of SSVEP-BCI efficiency. Front Neural Circuits 2013]:7. Friman O, Luth T, Volosyak I, Graser A. Spelling with steady-state visual evoked potentials. In: 3rd International IEEE/EMBS Conference on Neural Engineering, 2007. CNE’07; 2007]. p. 354–7. Friman O, Volosyak I, Graser A. Multiple channel detection of steady-state visual evoked potentials for brain–computer interfaces. IEEE Trans Biomed Eng 2007];54(4):742–50. Furdea A, Halder S, Krusienski D, Bross D, Nijboer F, Birbaumer N, Kübler A. An auditory oddball (p300) spelling system for brain–computer interfaces. Psychophysiology 2009];46(3):617–25. Gao X, Xu D, Cheng M, Gao S. A BCI-based environmental controller for the motiondisabled. IEEE Trans Neural Syst Rehabil Eng 2003];11(2):137–40. Gollee H, Volosyak I, McLachlan AJ, Hunt KJ, Graser A. An SSVEP-based brain–computer interface for the control of functional electrical stimulation. IEEE Trans Biomed Eng 2010];57(8):1847–55. Guger C, Allison BZ, Großwindhager B, Prückl R, Hintermüller C, Kapeller C, Bruckner M, Krausz G, Edlinger G. How many people could use an ssvep bci. Front Neurosci 2012];6:169. Hayes J, Cover T, Riera J. Optimal sequence detection and optimal symbol-by-symbol detection: similar algorithms. IEEE Trans Commun 1982];30(1):152–7. Hwang H-J, Kim DH, Han C-H, Im C-H. A new dual-frequency stimulation method to increase the number of visual stimuli for multi-class SSVEP-based brain–computer interface (BCI). Brain Res 2013];1515:66–77. Jia C, Gao X, Hong B, Gao S. Frequency and phase mixed coding in SSVEP-based brain–computer interface. IEEE Trans Biomed Eng 2011];58(1):200–6. Jin J, Allison BZ, Sellers EW, Brunner C, Horki P, Wang X, Neuper C. An adaptive p300-based control system. J Neural Eng 2011];8(3):036006. Kelly SP, Lalor EC, Reilly RB, Foxe JJ. Visual spatial attention tracking using highdensity SSVEP data for independent brain–computer communication. IEEE Trans Neural Syst Rehabil Eng 2005];13(2):172–8. Kübler A, Kotchoubey B, Kaiser J, Wolpaw JR, Birbaumer N. Brain–computer communication: unlocking the locked in. Psychol Bull 2001];127(3):358. Lalor EC, Kelly SP, Finucane C, Burke R, Smith R, Reilly RB, Mcdarby G. Steadystate VEP-based brain–computer interface control in an immersive 3D gaming environment. EURASIP J Appl Signal Process 2005];2005:3156–64. Liavas AP, Moustakides GV, Henning G, Psarakis EZ, Husar P. A periodogram-based method for the detection of steady-state visually evoked potentials. IEEE Trans Biomed Eng 1998];45(2):242–8. Lin Z, Zhang C, Wu W, Gao X. Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. IEEE Trans Biomed Eng 2006];53(12): 2610–4. Lotte F, Congedo M, Lécuyer A, Lamarche F, Arnaldi B, et al. A review of classification algorithms for EEG-based brain–computer interfaces. J Neural Eng 2007];4:1–13. Manyakov NV, Chumerin N, Robben A, Combaz A, van Vliet M, Van Hulle MM. Sampled sinusoidal stimulation profile and multichannel fuzzy logic classification for monitor-based phase-coded SSVEP brain–computer interfacing. J Neural Eng 2013];10(3):036011. Manyakov NV, Chumerin N, Van Hulle MM. Multichannel decoding for phase-coded SSVEP brain–computer interface. Int J Neural Syst 2012];22(05). Middendorf M, McMillan G, Calhoun G, Jones KS. Brain–computer interfaces based on the steady-state visual-evoked response. IEEE Trans Rehabil Eng 2000];8(2):211–4. Moore MM. Real-world applications for brain–computer interface technology. IEEE Trans Neural Syst Rehabil Eng 2003];11(2):162–5.

L. Cao et al. / Journal of Neuroscience Methods 253 (2015) 10–17 Mukesh TS, Jaganathan V, Reddy MR. A novel multiple frequency stimulation method for steady state VEP based brain computer interfaces. Physiol Measure 2006];27(1):61. Müller-Putz GR, Eder E, Wriessnegger SC, Pfurtscheller G. Comparison of DFT and lock-in amplifier features and search for optimal electrode positions in SSVEPbased BCI. J Neurosci Methods 2008];168(1):174–81. Muller-Putz GR, Pfurtscheller G. Control of an electrical prosthesis with an SSVEPbased BCI. IEEE Trans Biomed Eng 2008];55(1):361–4. Müller-Putz GR, Scherer R, Brauneis C, Pfurtscheller G. Steady-state visual evoked potential (SSVEP)-based communication: impact of harmonic frequency components. J Neural Eng 2005];2(4):123. Nijboer F, Furdea A, Gunst I, Mellinger J, McFarland DJ, Birbaumer N, Kübler A. An auditory brain–computer interface (BCI). J Neurosci Methods 2008];167(1):43–50. Palaniappan R, Raveendran P, Omatu S. VEP optimal channel selection using genetic algorithm for neural network classification of alcoholics. IEEE Trans Neural Netw 2002];13(2):486–91. Pan J, Xie Q, He Y, Wang F, Di H, Laureys S, Yu R, Li Y. Detecting awareness in patients with disorders of consciousness using a hybrid brain–computer interface. J Neural Eng 2014];11(5):056007. Parini S, Maggi L, Turconi AC, Andreoni G. A robust and self-paced BCI system based on a four class SSVEP paradigm: algorithms and protocols for a high-transfer-rate direct brain communication. Comput Intell Neurosci 2009]; 2009.

17

Pastor MA, Artieda J, Arbizu J, Valencia M, Masdeu JC. Human cerebral activation during steady-state visual-evoked responses. J Neurosci 2003];23(37):11621–7. Wald A. Tests of statistical hypotheses concerning several parameters when the number of observations is large. Trans Am Math Soc 1943];54(3):426–82. Wang M, Daly I, Allison BZ, Jin J, Zhang Y, Chen L, Wang X. A new hybrid BCI paradigm based on p300 and SSVEP. J Neurosci Methods 2014];244:16–25. Wang Y, Wang R, Gao X, Hong B, Gao S. A practical VEP-based brain–computer interface. IEEE Trans Neural Syst Rehabil Eng 2006];14(2):234–40. Wolpaw JR, Birbaumer N, Heetderks WJ, McFarland DJ, Peckham PH, Schalk G, Donchin E, Quatrano LA, Robinson CJ, Vaughan TM, et al. Brain–computer interface technology: a review of the first international meeting. IEEE Trans Rehabil Eng 2000];8(2):164–73. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM. Brain–computer interfaces for communication and control. Clin Neurophysiol 2002];113(6):767–91. Wu Z, Yao D. Frequency detection with stability coefficient for steady-state visual evoked potential (SSVEP)-based BCIs. J Neural Eng 2008];5(1):36. Xia B, Li X, Xie H, Yang W, Li J, He L. Asynchronous brain–computer interface based on steady-state visual-evoked potential. Cogn Comput 2013];5(2):243–51. Zhang Y, Jin J, Qing X, Wang B, Wang X. Lasso based stimulus frequency recognition model for SSVEP BCIs. Biomed Signal Process Control 2012];7(2):104–11. Zhang Y, Xu P, Cheng K, Yao D. Multivariate synchronization index for frequency recognition of SSVEP-based brain–computer interface. J Neurosci Methods 2014];221:32–40.