Rousseau Sandra
email: sandra.rousseau@gipsa-lab.grenoble-inp.fr

Jutten Christian

Congedo Marco

Time window selection for improving error-related potential detection

Keywords: Error-related potential, BCI, Spatial filtering, Classification

Time window selection for improving error-related potential detection

Sandra Rousseau, Christian Jutten, Marco

Introduction

The error-related potential (ErrP) is an eventrelated potential (ERP) which is generated when a subject commits or observes the commitment of an error. It was first reported in 1991 by Falkenstein et al. [START_REF] Falkenstein | Effects of crossmodal divided attention on late ERP components.II. Error processing in choice reaction tasks[END_REF] and has been since the subject of growing interest. This potential is time-locked to the observation of the error and is mostly characterized by a negative deflection (Ne) [START_REF] Gentsch | Dissociable medial frontal negativities from a common monitoring system for self-and externally caused failure of goal achievement[END_REF], followed by a large positivity (Pe) [START_REF] Steinhauser | Performance monitoring and the causal attribution of errors[END_REF]. There exists different kind of ErrPs depending on the agent committing the error. In BCIs we generally observe what is called the interaction ErrP which occurs when a subject observe an external device committing an error and the feedback ErrP which is observed when a subject commits an error but becomes aware of it only after external feedback. Lately several authors have become interested in its integration in BCI systems as a control loop, for example as an error correction system. The integration of the ErrP in BCIs involves two main operations: its single trial detection and the use of this information to on-line modify the system. Since its signal to noise ratio (SNR) is very low, as any ERP, the ErrP can easily be seen by summing up several trials but is much less detectable on a single-trial basis. However, the single trial detection of the ErrP is mandatory for its integration in BCIs. In addition to its low SNR, ErrP characteristics can also be highly variable from one subject to another and from one experiment to another. Thus to ensure satisfying detection rates, it is necessary to design filtering methods that will increase the ErrP SNR. Moreover, designing subject's specific fil-ters could improve detection accuracy. In this paper, we first present an experiment we designed to obtain ErrP data in high cognitive load conditions, enabling us to approximate BCI use conditions. We study the occurrence of the ErrP and observe its inter-subject latency variability. Then we classify this potential on a single-trial basis using spatial filtering methods. Based on our observations of the inter-subject latency variability we develop a new adaptive spatial filtering method which allows the selection of subjects specific time windows in order to improve single trial detection results. Our method aims at selecting subjects specific time windows based on each subject signal characteristics and optimized for the spatial filter we use. We compare the classification results we obtain with other less adaptive methods. Finally we comment on the relevance of the selected time window and its link with our signals characteristics.

Data

Experiment

The experiment we designed involves a memory game where performance feedback is given after the subject answers questions. No specific reward is given and the experiment is designed so as to induce a high cognitive load. The experiment involves two sessions that last approximately half an hour. Each session consists of six blocks of six trials, for a total of 72 trials. Stimuli are presented on a computer screen and consist of digits displayed in square boxes arranged in circles. Each trial starts with the display of the score for 3000 ms followed by a fixation cross (see Figure 1.a) also displayed for 3000 ms. Then the memorization sequence starts, each memorization consists in a random sequence of two to nine digits appearing sequentially in random positions (see Figure 1.b). Subjects are instructed to retain positions of all digits. At the end of the sequence the target digit is displayed (see Figure 1.c) and subjects have to click on the box where it has appeared. Once the subject has answered, the interface waits for 1500 ms in order to avoid any contamination of ErrP by beta rebound motor phenomena linked to mouse clicking [START_REF] Pfurtscheller | Central beta rhythm during sensorimotor activities in man[END_REF]. Then, if the answer is correct, the chosen box background color turns into green ("correct" feedback), otherwise it turns into red ("error" feedback). Subjects are then asked to report if the feedback (error/correct) matches their expectation by a mouse click ("yes"/ "no") (see Figure 1.d). A random break of 1000 ms to 1500 ms precedes the beginning of the new trial (see Figure 2). The number of digits in the sequence is fixed within blocks (between two and nine) and updated, according to subjects performance, at the beginning of each block. The first block starts with four digits. The number of digits is adapted with an algorithm tuned to allow about 20% of errors for all subjects, regardless the working memory ability and adapting to fatigue. Between the two sessions the screen is shut down to allow a rest break of 2 -3 minutes.

Participants

25 healthy volunteer subjects participated, 14 males and 11 females. Subjects were informed of the procedure before the experiment and filled an information form. All subjects were BCI-naifs at the time of the experiment and none of them reported neurological or psychiatric disorders in the past. Due to the presence of artifacts, four subjects were excluded from analysis. The age of participants ranged from 20 to 30 with a mean of 24 (±2.5). The mean error rate was equal to 18 (±4.6)% of the trials.

Acquisition and preprocessing

EEG recordings were made from 31 sensors using the extended 10/20 system. Both earlobes, digitally linked, were used as electrical reference. The ground sensor was positioned on the forehead. The impedance of each sensor was kept below 5kΩ. The EEG was band-pass filtered in the range 0.1-70 Hz and digitized at 500 Hz using the Mitsar 202 DC EEG acquisition system. Data were bandpass-filtered between 1-40 Hz using an order 4 Butterworth filter with linear phase response. Eye blinks were extracted using ICA (independent component analysis), [START_REF] Comon | Handbook of Blind Source Separation: Independent component analysis and applications[END_REF]. SOBI algorithm was used to perform ICA. One EOG source was suppressed for each subject. It was manually selected using both the temporal shape of the source and its topography.

Observation of the ErrP and its latency variability

In Figure 3 we plot the event-related potential averaged over subjects for correct trials and for error trials for one second post-stimulus (observation of the error). We can see that for error trials the ERP is characterized by a sharp negativity (Ne) followed by a small positivity (Pe) which is consistent with previous reports [START_REF] Yeung | ERP correlates of feedback and reward processing in the presence and absence of response choice[END_REF], [START_REF] Hajcak | It's worse than you thought: The feedback negativity and violations of reward prediction in gambling tasks[END_REF]. In addition to these peaks one can also see a first positive peak which has also been reported by [START_REF] Ferrez | You are wrong!-automatic detection of interaction errors from brain waves[END_REF] in the case of interaction error-related potential. For correct trials, a negativity is also observed but with a much lower intensity. Let us define Ne latency as the time where the mean potential shows the high variability of Ne latency since its variation corresponds to half the width of the peak. In Figure 4 we plot the mean event-related potentials for error trials for three different subjects. We can see that the shape and the latency of this potential can be highly variable.

Single trial classification

ErrP could be used in BCI systems to detect and correct error trials. However, to do so, one has to be able to detect it on a single trial basis. Some authors already studied this issue [START_REF] Dal Seno | Online detection of P300 and error potentials in a BCI speller[END_REF], [START_REF] Ferrez | EEG-based brain-computer interaction: Improved accuracy by automatic singletrial error detection[END_REF]. Most of them only used raw data with no spatial filtering. Here we propose to apply spatial filtering methods to our data in order to try to improve our classification results.

Filtering method

xDAWN algorithm

Spatial filtering was done using xDAWN algorithm [START_REF] Rivet | P300 speller brain-computer interface: Enhancement of P300 evoked potential by spatial filters[END_REF]. We consider that the signal is the sum of one target evoked potential plus other possible superimposed evoked potentials (but non-target) and noise:

X = ∑ i D i A i + N (1)
where X represents the signal, D i is a Toeplitz matrix whose first column entries are set to zero except for those that correspond to the stimuli of type i, A i represents the responses synchronized with the stimuli of type i and N represents the noise. Evoked responses A i are estimated as:

  A 1 ... A n   = (D T D) -1 D T X (2) with D = [D 1 , D 2 ...D n ].
The aim of xDAWN is to find the filter U which maximizes the signal to signal plus noise ratio (SSNR):

U = argmax Tr(U T ∑ i U) Tr(U T ∑ X U) (3)
with

∑ i = (D i A i ) T (D i A i ) and ∑ X = X T X
. This is a Rayleigh quotient ( [START_REF] Fukunaga | Introduction to statistical pattern recognition[END_REF], [START_REF] Gray | Toeplitz and circulant matrices: A review[END_REF])) which can be solved using the generalized eigenvalue decomposition (GEVD) of matrices Σ 1 and Σ X .

Application to our problem

In this problem we consider two potentials: one corresponding to the occurrence of an error (the error potential) and one corresponding to the occurrence of answer display (may it be correct or error). Thus our signal can be written as :

X = D 1 A 1 + D 2 A 2 + N
where D 1 A 1 corresponds to error feedbacks and D 2 A 2 corresponds to feedbacks (errors and corrects). The evoked potentials are usually defined as a one second post-stimulus window of the EEG signal. However, the longer will be this window and the more our potential will contain noise. Thus, it could be interesting to reduce the size of this window. One way of doing it is to determine the position of our peak of interest (defined here as Ne) and to select a smaller time window centered on this peak. However as we have seen before Ne latency can be highly variable from one subject to another, making it impossible to select a prior time window that could be used for all subjects.

Adaptive time window selection

We propose a modification of xDAWN that allows a subject's specific selection of the time window within the spatial filtering algorithm. Here we span the whole ERP and find the time window where the target potential (error trials) contains the most information. The time-window will then be used to estimate the spatial filters and to classify data. Let be L the length of the window of interest (here set to 250 ms), N the maximal length of the potential under study (here 1000 ms) and x the EEG signal of total length N T with N c electrodes. Then the proposed algorithm is as follows:

We first calculate the matrices

A 1 ∈ R N×N c , D 1 ∈ R N T ×N and Σ X ∈ R N c ×N c then ∀t ∈ [0, ..N -L]:
• We create matrices D 1new et A 1new defined so that they contain only the signal corresponding to the window of length L starting at time point t after the feedback:

D 1new = D 1 (:,t : t + L) and A 1new = A 1 (t : t + L, :);
• We create matrix Σ 1new defined as :

Σ 1new = (D 1new .A 1new ) (D 1new .A 1new );
• We compute the generalized eigenvalue decomposition (GEVD) of Σ 1new and Σ X (U t , λ) = GEV D(Σ 1new , Σ X );

• Finally we define our decision value as:

β(t) = λ(1) (4)
with λ(1) being the first eigenvalue of the GEVD. The optimal position of our window (t opt ) is then defined as the time sample corresponding to the maximum of β. Thus, for spatial filtering we will use filters computed on this time window (U t opt ).

Preprocessing and classification

Data were band-pass filtered between 1-10 Hz and then spatially filtered using the previously described methods. Data were then classified using Bayesian LDA classifier [START_REF] Mackay | Bayesian interpolation[END_REF]. xDAWN algorithm returns filters classified in descending order of performance thus we used the first two components as features for our classifier. Data were subsampled at 32 Hz. Data were also classified before spatial filtering using EEG signal from electrodes FCz and Cz as features, as it has been done in most previous studies [START_REF] Ferrez | EEG-based brain-computer interaction: Improved accuracy by automatic singletrial error detection[END_REF]. Classification was performed on 21 subjects. For each subjects a leave-one out validation method was used: spatial filters and classifier were learned on the whole data except one and then tested on the remaining data.

Results

Evaluation of the algorithm

In figure 5 we plot the evolution of our decision value (β) along our window position and the evolution of our mean classification accuracy. We can see that β is highly influenced by the window position. Moreover we can also see that β and our classification accuracy seem to have a similar evolution. Thus it seems that the choice of our decision value is relevant for the optimization of the single trial detection of the ErrP.

Selection of the time window

In figure 6 we plot the mean error potential for three different subjects and the corresponding selected time window. We can see that for these three subjects the shape and latency of the ErrP is very different and that our algorithm adapts to the shape of the potential under study and select the time window where the difference between errors and corrects is huger.

Classification Results

We report the mean classification results (and their standard deviation) on table 1 for our three different methods. It has to be noted that given the very low number of trials available we get quite encouraging results even on raw data.

Secondly we can see that xDAWN allows significantly better classification results for correct trials than classification on raw data (p ≤ 0.01). However for classification on error trials xDAWN is worse than using raw data (p = 0.01). Thus even if xDAWN improves the mean classification rate ( T e +T c 2 ), it gives very poor results for error classification. However for most types of ErrP integration, correct trials identification is more important than error trials identification. A misidentification of a correct trial (correct trial is identified as an error trial) will lead to the modification of a correct order (and thus leading to an error) which will diminish our system's performance. The misidentification of an error will just lead to no specific action, which will bring no modification.

Finally it turns out that our adaptive algorithm gives similar results as the classical xDAWN for correct trials (p = 0.2) (and thus significantly better than raw data, p ≤ 0.01) but significantly better results for error trials (p ≤ 0.01) (and similar as raw data , p = 0.3). The main advantage of our method is that it is optimized for the spatial filter we use and each time window is selected specifically for one subject. Our algorithm allows a significant improvement of detection of correct trials along with no deterioration of error trials detection (but no improvement) as compared to classification on raw data. Using this method we get similar results as those found by previous studies (( [START_REF] Chavarriaga | Learning from EEG error-related potentials in noninvasive brain-computer interfaces[END_REF], [START_REF] Ferrez | EEG-based brain-computer interaction: Improved accuracy by automatic singletrial error detection[END_REF]) but with only very small training sets while other studies used large training sets. This is of great interest for ErrP integration in BCIs. Indeed one huge problem in actual BCI systems is the training session which can be quite long for some applications. Here we get significant detection results even when using only 15 error trials (30 trials in total, errors and corrects) which would correspond to a very short training session. In figure 7 we plot the classification results for the ten first subjects for error trials and correct trials using the different preprocessing methods. We can see that for classification on error trials xDAWN only outperforms our adaptive algorithm for two subjects and for one of them results are very bad for any method. For correct trials spatial filtering clearly outperforms classification on raw data, however no clear difference can be drawn between our two methods, with xDAWN outperforming the adaptive one for half the subjects.

Conclusion

In this paper we have studied the error-related potential and its single-trial detection. We have seen that the ErrP did occur even in high cognitive load conditions and that its latency could be highly variable from one subject to another. Based on this observation we have proposed a new algorithm for adaptive spatial filtering and time window selection based on xDAWN algorithm. Classification results have shown that our algorithm performs better than the xDAWN algorithm and than classification on raw data with an improvement up to 17% for correct trials (which is the most important one as we have said before). Thus, we have developped a new algorithm that allows our classifier to adapt to ErrP latency variability. This is of great interest given its high inter-subject variability. Moreover it is known that ErrP characteristics may vary with experimental and environmental conditions. ErrP latency could then evolve within an experiment, using an adaptive filtering method (that would be updated along the experiment) would allow consistent classification results along time. In conclusion we have shown that using adaptive filters for ErrP single trial detection could significantly improve classification results. Moreover we have developped a new algorithm that could tackle the problem of ErrP latency variability. Further studies should be done in this way, as for example to adapt to latency variability between trials or maybe to find otpimal time windows for the different ErrP peaks (since it is known that the ErrP is characterized by a negative peak and a positive one).

Figure 1 :

 1 Figure 1: Screen shots from the experiment representing different steps.a)Fixation cross. b)Memorization sequence. c)Target digit appearing. d) Feedback report question: 'Did you expect this result?'.

Figure 2 :

 2 Figure 2: Time course of one trial

Figure 3 :

 3 Figure 3: Mean potential averaged over 21 subjects at electrode FCz. Blue thick line corresponds to error trials and red dashed line corresponds to correct trials

Figure 4 :

 4 Figure 4: Temporal course of the mean error potential for different subject.Each line corresponds to one subject. Data are averaged over error trials at electrode FCz.

Figure 5 :

 5 Figure 5: Effect of window position. (a)Evolution of β as a function of window position. (b)Evolution of classification accuracy as a function of window position. Thick line corresponds to errors, dashed line corresponds to corrects.

Figure 6 :

 6 Figure 6: Selected time windows for different subjects. Yellow area corresponds to the selected time window. Thick line is the averaged potential for errors and dashed line is the averaged potential for corrects, at electrode FCz.

Figure 7 :

 7 Figure 7: Classification results for the first ten subjects. (a) for error trials. (b) for correct trials. Black bars correspond to raw data, dark brown bars to xDAWN filtering and light brown bars to adapative xDAWN filtering.

ACKNOWLEDGEMENTS

We are grateful to Dr. Bertrand Rivet for making us available the code of his xDAWN algorithm and explaining us his method. We are grateful to the project ANR OpenVibe.


Rousseau Sandra
email: sandra.rousseau@gipsa-lab.grenoble-inp.fr

Jutten Christian

Congedo Marco

Closed-looping a P300 BCI using the ErrP

Keywords: BCI, Error-related potential, closed-loop

come   L'archive ouverte pluridisciplinaire

Introduction

BCI are systems that aim at decoding brain signals and convert them into orders for driving an external device. One very well known BCI is the P300 speller which allows a subject to spell words using a virtual keyboard. Actual BCIs performances need to be improved in order for them to be more easily usable. One way of improving this accuracy is to include a control loop based on error detection which can be performed with the error-related potential. Errorrelated potentials (ErrP) are evoked potentials which are generated when a subject makes or observes an error [START_REF] Falkenstein | Effects of crossmodal divided attention on late ERP components.II. Error processing in choice reaction tasks[END_REF], [START_REF] Gehring | A neural system for error detection and compensation[END_REF]. It is time-locked to the observation of the error and is mostly characterized by a negative deflection (Ne) [START_REF] Gentsch | Dissociable medial frontal negativities from a common monitoring system for self-and externally caused failure of goal achievement[END_REF], followed by a large positivity (Pe) [START_REF] Steinhauser | Performance monitoring and the causal attribution of errors[END_REF]. Single-trial detection of this potential is now the subject of many studies. As any ERP, the signal to noise ratio of the ErrP is very low and it is hard to see it on a single trial basis. However most studies reached in average a rate of 70% of good detection for error trials and a rate of 80% of good detection for correct trials [START_REF] Ferrez | You are wrong!-automatic detection of interaction errors from brain waves[END_REF], [START_REF] Chavarriaga | To err is human: Learning from error potentials in braincomputer interfaces[END_REF], [START_REF] Bollon | EEG error-related potentials detection with a Bayesian filter[END_REF]. In this paper we study the possible integration of the ErrP in a P300 BCI as an error correction system. Indeed, when the BCI makes an error, an ErrP appears in the subject's EEG. If one can detect this ErrP then it could be used as a control loop to correct errors made from the BCI. Some authors already studied thie issue in two to four clas BCIs [START_REF] Ferrez | EEG-based brain-computer interaction: Improved accuracy by automatic singletrial error detection[END_REF], [START_REF] Ferrez | Simultaneous real-time detection of motor imagery and error-related potentials for improved BCI accuracy[END_REF] or in multiclass BCIs [START_REF] Dal Seno | The utility metric: a novel method to assess the overall performance of discrete brain-computer interfaces[END_REF].

Here we present different kind of strategies that can be used to correct BCI errors in a P300 speller and what kind of improvement can be achieved. First we study the characteristics of different P300 datasets in order to see the repartition of errors and corrects probabilities. Then, using these observations, we propose different stragegies for ErrP integration and test them on our datasets according to different performances of our system. Finally we discuss on their different advantages and disadvantages.

Studying a P300 speller

In order to determine what kind of strategy could be used to integrate ErrPs in a P300 speller we study the characteristics of a P300 system.

Datasets

For this study we used two datasets available online from BCI competitions [START_REF] Blankertz | The bci competition 2003: progress and perspectives in detection and discrimination of eeg single trials[END_REF] (http://www.bbci.de). The two data sets come from the same paradigm [START_REF] Farwell | Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials[END_REF]. The user was presented with a 6 by 6 matrix of characters . The user's task was to focus attention on characters that were prescribed by the investigator (one character at a time). For each character, all rows and columns of this matrix were successively and randomly intensified (leading to 12 intensifications). This was repeated 15 times (leading to 15 × 12 = 180 intensifications for each character). For each character, user display was as follows: the matrix was blank for 2.5 s. Then each row and column was randomly intensified for 100 ms.

After intensification of a row/column, the matrix was blank for 75 ms. Then a 2.5 s period of rest was allowed during which the matrix was blank. Subjects were asked to spell predefined words separated into different sessions. For the first dataset, signals were collected from two subjects in five sessions each.For the second dataset signals were collected from one subject in three sessions. All data were digitized at 240Hz.

Preprocessing and classification

The same method was used for every subject. Data were bandpass filtered between 0.2-20Hz and spatially filtered using xDAWN algorithm [START_REF] Rivet | xDAWN algorithm to enhance evoked potentials: Application to brain-computer interface[END_REF]. For each character and repetition the corresponding signal was defined as the sum of the one second EEG signal following its corresponding row intensification and the one following its corresponding column intensification. Trials were then classified using a Bayesian LDA classifier [START_REF] Mackay | Bayesian interpolation[END_REF]. This type of classifier returns for each letter a probability that it belongs to the target class. Classification is calculated for each repetition, the final score for each character is computed as the sum of its probabilities for each repetition:

s(l) = n ∑ i=1 p(l, i) (1)
where s(l) is the score of the letter l and p(l, i) is the probability given to the letter l by the LDA classifier at repetition i. The character to be spelled is chosen as the one having the highest final score. We will define this highest score s 1 , the second highest score will be refered to as s 2 and so on. A ten-fold cross validation method was used.

Results

Classification results

In Figure 1 we plot the classification results for each subject versus the number of repetitions used. Without surprise the classification rate improves with the growing number of repetitions.

It is clear that one way of improving a P300 speller is simply to increase the number of repetitions, however this also increases the time spent to spell a letter and thus slows down the system.

Second best score

In a second time we got interested in the conditions of failure of our classifier. Thus we studied what happened when the highest score (s 1 ) did not correspond to the target character. Especially we looked at the probability that the second highest score (s 2 ) corresponds to the target letter (when s 1 does not). In figure 2 we plot this probability for different subjects. We can see that this probability can be very low and very different from one subject to another and that no clear dependency on the number of repetitions can be deduced.

(a) Subject 1 (dataset 2) Figure 2: Classification results using second highest score (when highest score is not the target) for different subjects. Each line corresponds to a subject.

Highest score repartition

Finally we studied the repartition of the highest score (s 1 ) and the second highest score (s 2 ). We separated our data into three classes:

• s 1 corresponds to the target letter.

• s 1 does not correspond to the target letter, s 2 does.

• neither s 1 nor s 2 correspond to the target letter.

Then we looked at the evolution of our scores s 1 and s 2 and of their difference (s 1s 2 ) in these different classes. In figure 3 we plot this repartition for the different subjects. In this figure we can see that for the three subjects s 1 is much lower when the target character corresponds to the second highest score (s 2 ). The same observation can be made for their difference (s 1 -s 2 ) for which the effect is even stronger. This observation suggests that when s 1s 2 is low the second highest score would correspond to the target character and inversely when s 1s 2 is high the second highest score would not correspond to the target character. We plot the mean value (standard error) fors 1 , s 2 and their difference (s 1s 2 ) in three different conditions: 1)s 1 corresponds to the target letter, 2)s 2 corresponds to the target letter c)Neither s 1 nor s 2 correspond to the target letter.

Propositions of integration

Now that we have observed the performance of our P300 system and its characteristics we propose to simulate the integration of the ErrP in this system as an error correction system. As we have said before ErrP detection is not perfect and is subject to errors. Thus we will study the impact of our error-detection (ED) performances on our results.

Integration

ErrP can be integrated in different ways in BCI systems in order to correct errors. Using the previously presented observations we have selected four different strategies of integration:

• C: Canceling the command (i.e. erasing the letter). This method is the one that has been mostly used in previous studies on ErrP integration [START_REF] Dal Seno | The utility metric: a novel method to assess the overall performance of discrete brain-computer interfaces[END_REF].

• CI: Canceling the command and starting a new shorter session. The scores resulting from this new session will then be added to our original scores and a new decision will be made from our BCI system.

• R: Replacing the command by the one corresponding to the second highest score.

• RC: Replacing the command by the one corresponding to the second highest score only if the difference s 1s 2 is lower than a predetermined threshold. If it is higher nothing is done.

Performance measurement

The performance of a BCI can be assessed in different ways depending on what we want to optimize. Here we used three different performance measures which evaluate different qualities of our system:

• Classification rate (T f ): this is the classification accuracy of our system for one trial. This is of great use if one desires a BCI that is very precise.

The main disadvantage of this measure is that it does not take time into account. It is defined as:

T f = NbC NTot (2)
with NbC being the number of trials correctly classified and NTot the total number of trials.

• Information transfer rate (ITR): this measure is a more complex one which takes into account both the classification accuracy, the time spent for a trial and the information it contains. With a classification accuracy of T f , a trial duration of c and N possible outcomes, the ITR is defined as:

B = log 2 (N) + T f log 2 (T f ) + (1 -T f ) log 2 ( 1-T f N-1 ) c
(3)

• Utility Metric (U): it was proposed by [START_REF] Dal Seno | The utility metric: a novel method to assess the overall performance of discrete brain-computer interfaces[END_REF]. The main difference with the ITR is that here we consider the special case where the subject has to perform perfectly his task. Thus, when an error occurs the subject has to cancel the order and restart the trial. Let us define TL the time to perform correctly a task:

T L = T f c + (1 -T f )(c + T B + T L) = c 2T f -1
where TB is the time spent to cancel the command, here we assume it is equivalent to any other command with T B = T L. The utility measure is then defined as:

U = bL T L (4)
where bL is the benefit of a letter, if each letter is equiprobable then bL = log 2 (N -1) bits.

Threshold

To determine the threshold for the RC strategy data were classified using the trained classifier, for each trial i we have D(i) = s 1 (i)s 2 (i). D values are then divided into two groups: Group 1 contains trials for which s 1 corresponds to the target letter and Group 2 contains trials for which s 2 corresponds to the target letter. The threshold T is defined as the frontier between these two groups using LDA.

Derivation of the performance measures

Here we will present the different performance measures of the closed-loop system according to the chosen strategy.

Local Variables

Let us first define several variables:

• T f (n): P300 classification rate with n repetitions.

• T c : ErrP classification rate for correct trials.

• T e : ErrP classification rate for error trials.

• T f 2 : P300 classification rate using s 2 when s 1 is not correct.

• T r1 : Probability that (s 1s 2 ) < T when s 1 was not correct (i.e. error will be corrected).

• T r2 : Probability that (s 1s 2 ) > T when s 1 was correct (i.e. correct will not be modified).

• T f c : P300 classification rate with n+p repetitions when T f (n) was correct.

• T f e : P300 classification rate with n+p repetitions when T f (n) was not correct.

Canceling strategy (C)

Classification accuracy and ITR are defined as (refer to [START_REF] Ferrez | You are wrong!-automatic detection of interaction errors from brain waves[END_REF] for more details):

T f b = T f T c T f T c + (1 -T e )(1 -T f ) (5) IT R b = s t log 2 (N) + T f b log 2 (T f b ) + (1 -T f b )log 2 ( 1-T f b N-1 ) c (6) with s t = (T f T c + (1 -T e )(1 -T f )).
The time spent to correctly spell a letter is (refer to [START_REF] Dal Seno | The utility metric: a novel method to assess the overall performance of discrete brain-computer interfaces[END_REF] for more details):

T L = c T f T c + (1 -T f )T e + T f -1 (7)

Canceling and iteration strategy (CI)

Here trials detected as errors will be restarted and will have a new probability of being corrects giving a classification accuracy of:

T f b = T f T c + (1 -T f )T e T f e + T f (1 -T c )T f c (8)
Thus TL will be defined as:

T L = (1 + a(r p (1 -r c ) + r e (1 -r p )))c (1 -2(r p (1 -r c )(1 -r pn1 ) + (1 -r p )(1 -r e r pn2 )) (9)
Now we will only present T f b , ITR and U can be derived by replacing T f by T f b in their definition.

Replacement strategy (R)

Here trials detected as errors are replaced by another letter thus we get for the classification accuracy:

T f b = T f T c + (1 -T f )T e T f 2 (10)

Replacement under condition strategy (RC)

Here the difference with the previous method is that only a fraction of erroneously detected correct trials will be modified and only a proportion of correctly detected error trials will be corrected:

T f b = T f T c + T f (1 -T c )T r2 + (1 -T f )T e T f 2 T r1 (11)

Results

Integration was tested for n ∈ [1, 15] (n is the number of repetitions). We used three new repetitions (a = 3) for the iteration strategy (CI). Different performances of our error-detection system (ED) were tested: T e ∈ {60%, 70%, 80%} and T c ∈ {70%, 80%, 90%}. In figure 4 we plot the results for one subject. First of all one can see how much our performance measures are different. Each strategy performs differently for each performance measure. Here we present figures for only one subject but the observations made also apply to the other subjects.

Classification accuracy

Concerning the classification accuracy, one can see that this measure improves with growing number of repetitions for any type of integration strategy. Moreover, it seems that strategies C and CI allow improvement of the BCI system even for low performances of our ED (T e = 0.6 and T c = 0.7), with strategy C being the most performant one. Strategies R and RC allow improvement of the initial system only for high performances of our ED (T e ≥ 0.7 and T c ≥ 0.8) and never outperform strategy C. Thus it seems that if one aims at simply improving the classification accuracy of a system the best strategy of integration would be C (ie, cancellation of a trial).

Information transfer rate

One can see that it does not increase with the growing number of repetitions, because the duration of a trial is taken into account. Integration results are very different from those obtained with T f . Here strategy C allows improvement only for high performance of our ED (T e ≥ 0.8 and T c ≥ 0.9). Strategies R and RC are the best ones with strategy RC being more stable than R (for low ED performances RC does not degrade our system while R does). Finally one can note that the ITR follows the same evolution (for growing number of repetitions) before and after integration of our error correction system. If one aims at improving the ITR it seems that the best strategy would be RC (ie, replacement under conditions).

Utility metric

Once again, the utility metric (U) does not necessarily increase with growing number of repetitions. For low performances of our ED (T e = 0.6 and T c = 0.7) there does not seem to be one particular strategy clearly outperforming the others. However one can see that strategy R gives very poor results. For higher performances of our ED (T e ≥ 0.8 and T c ≥ 0.9) it seems that strategy A is slightly better than the other but this is not significant. However one can note that with these performances each strategy brings high improvement.

Discussion

In this paper we have simulated the integration of the ErrP as an error-correction system in a P300 speller BCI . We have presented different strategies of integration and studied their impact on the BCI performances. Using different types of performance measures we have seen that each strategy had some advantages and disadvantages. The overall best strategy seems to be the strategy of canceling the erroneous order when it is detected since it gives the best results in terms of classification accuracy and of utility measure. However the measure used in most studies to assess the performance of multiclass BCIs is the ITR. For the ITR it seems that the best strategy is the strategy of replacing the erroneous command by the one obtaining the second best score under condition. Thus, one can not state for a best strategy. However what can be noted is that for high performances of our error-detection system every strategy allows improvement of our system for any type of performance measure. Moreover for T e = 0.8 and T c = 0.9 we get a mean improvement (over subject) of 11% of the classification accuracy , of 9% of the ITR and of 10% of the utility metric (each time for the best integration strategy). Thus we have seen that the ErrP could be used as an error-correction system in a multiclass BCI and that, even if it did not bring results as performant as for a two class BCI, one can get an improvement of around 10% of our system which is very encouraging. However we have also seen that the integration strategy should be carefully chosen according to the systems characteristics and to what one wants to improve.

Figure 1 :

 1 Figure 1: Classification results for different subjects in function of the number of repetitions. Each line corresponds to a subject.

  Figure 3: Highest final scores after 10 repetitions.We plot the mean value (standard error) fors 1 , s 2 and their difference (s 1s 2 ) in three different conditions: 1)s 1 corresponds to the target letter, 2)s 2 corresponds to the target letter c)Neither s 1 nor s 2 correspond to the target letter.

Figure 4 :

 4 Figure 4: Evolution of our different performance measures (Tf, ITR and U) with the number of repetitions used for different performances of our ED.Each column of plots correspond to a different measure of performance. Each line of plots correspond to a different performance of our ED. In each plot, each line corresponds to a different strategy of integration.

ACKNOWLEDGEMENTS

We are grateful to the project ANR OpenVibe.