Keywords: Brain-computer interface (BCI), deep learning, electroencephalogram (EEG), intraoperative awareness during general anesthesia, machine learning, median nerve stimulation, motor imagery. Conclusions . . . . . . . . . . . . . . . . . . . . . .

the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER, and several Universities as well as other organizations.

I would like to dedicate this thesis to my grandfather Yakiv and all the people, who, unfortunately, died because of the COVID-19 global pandemic. My grandfather passed away during the last weeks of dissertation preparation. He was a man of strong will and spirit, who survived the Holodomor of 1932-33, the Second World War, and recovered from three strokes. He lived a wonderful 91 years and continued to smile even in the most difficult times. Your smile will always cheer me up and stay in my heart forever.

Carrying out the required work and then writing this thesis was, undoubtedly, the toughest task I have undertaken. However, one of the pleasures of having completed the thesis is looking back at everyone who has helped me over the past four, eleven, and twenty-eight years.

I would like to begin by thanking my four supervisors: Anton Popov, Laurent Bougrain, Patrick Hénaff, Vladimir Timofeyev, whose expertise was invaluable. Your insightful feedback pushed me to sharpen my thinking and brought me and my work to a higher level.

I would particularly like to single out my supervisor, comrade, and life mentor Anton Popov. Anton, I want to thank you for your patient support, for the inspiration, for the desire and interest in learning and research that you instilled in me, and for all of the opportunities I was given during all these years of study.

I would like to thank Sebastien Rimbert and Pierre Riff for the help during the experiments. I also wish to give thanks to all the participants.

I am also infinitely thankful to my Mum Liudmyla for her love, generosity, and support. I would also like to thank my family: my brother Oleksandr, father Oleksandr, grandfather Yakiv, and grandmother Nina for their belief in me and constant support. It was extremely important for me to feel your support.

Also, I would like to thank my teachers from the high school Tamara Lenova, Valentyna Struk, and Lesia Zapadlovska who instilled in me curiosity and love to learn.

I want to thank the Neurorhythms members for speaking French all the time during lunch and encouraging and pushing me to learn French more.

I would like to thank Francesco Giovannini for your attempts to dissuade me from doing a Ph.D. In the last year, I finally understood what you mean. But have you see, I did it! I would like to thank Dominique Martinez for all our talks about startups and business plans. I would like to thank Rahaf Al-Chwa and Nathalie Azevedo-Carvalho for your help and for all our interesting conversations in the office.

I would like to thank my French friend Valentin Calisti for your help, support, and time we spent together in Ukraine and France. Thanks to you, I fell in love with Marseille and road bikes. And thank you for explaining Riemannian Geometry and teaching me how to flirt in French. Now I'm pro in both! I would like to thank Annik and François Grande and the people from ARIV Association, who became my French family, always were open and kind to me and helped me a lot during my stay in France. Thanks to Annik and François I discovered French culture and a completely different way of life. It made me rethink many of my life's values. And I really appreciate the time and moments we spent together.

But not only French families. During my Ph.D. and four stays in Nancy, I met and discovered many very interesting people, some of them became my close friends.

I would like to thank Valeria Razinina, Daryna Malashenko, and Ivan Kotiuchyi, we actually lived together in the first year of my Ph.D. I would also like to thank Max Fedorchuck, Yuliia Kunyk, Olha Kurhanska, Juliia Miliar, Nikita Vasilenko, Ksenia Yankina, Anastasya Kharchenko, Anna Liednikova, Luigi Penco, Erika Bartolomei, Houda Ennaim, Joanna Rogińska, and Taisiia Sikolenko, all of you supported me all the way to the goal. How many unforgettable moments we have experienced together. These 4 years would have been much harder without you.

I would like to thank my friends Dmytro Domashenko, Kirill Teplyakov, and Anna Turchaninova for your support. You were so brave to travel to Nancy for visiting me.

I want to thank my friend and colleague Dr. Daryna Panicheva, who provided stimulating discussions till 8 a.m. as well as happy entertainments and sports activities to rest my mind outside of the research.

Also, I want to say thanks to everyone who was around, supported and helped me during this long 4 years. My Ukrainian and French friends, thank you for your mental and motivational support! I am very grateful to many of you! I want to say special thanks to my friend Denys Ladchenko, for supporting my family when they needed help.

Also, I would like to thank Taylor Swift for two wonderful albums in 2020. Most of this thesis was written while I was listening to these albums.

The work reported in this thesis would not have been possible without the financial support of the LORIA laboratory, Erasmus, and the French Embassy to Ukraine. Experiments presented in this paper were carried out using

Résumé

Cette thèse présente trois contributions pour améliorer la reconnaissance d'imaginations motrices utilisées par de nombreuses interfaces cerveau-ordinateur (BCI) comme moyen d'interaction. Tout d'abord, nous proposons d'estimer la qualité des images motrices en détectant des valeurs aberrantes et de les supprimer avant apprentissage. Ensuite, nous étudions la sélection des caractéristiques pour sept imaginations de mouvements. Enfin, nous présentons une architecture d'apprentissage profond reprenant les principes du réseau EEGnet applicable directement sur des signaux électroencéphalographiques simplement filtrés et adaptés au nombre d'électrodes. Nous montrons en particulier ses bénéfices pour l'amélioration de la détection des réveils peropératoires et d'autres applications.

Nous introduisons d'abord les concepts de base des interfaces cerveau-ordinateur basées sur l'imagerie motrice. Nous présentons les bases neurophysiologiques des rythmes cérébraux et de leurs localisations.

Nous abordons les méthodes de prétraitement, d'extraction de caractéristiques, de sélection de caractéristiques, de détection des valeurs aberrantes et de classification ainsi que des méthodes d'apprentissage automatique fréquemment utilisées dans les BCI basées sur l'IM pour la tâche de classification, telles que l'analyse discriminante linéaire, les machines à vaste marge et les algorithmes de régression logistique.

Nous introduison le concept de réseaux de neurones artificiels, leurs architectures et algorithmes d'optimisation. Enfin, nous présentons les architectures de réseaux de neurones de convolutions (CNN) et leurs adaptations pour la classification des signaux EEG bruts tels que Deep Convolutional Network, Shallow Convolutional Network et EEGNet.

Pour implémenter et valider les méthodes proposées, nous avons présenté trois jeux de données différents. Le dernier jeu de données se compose de conditions qui nous permettent de comparer deux tâches de discrimination : l'imagination motrice et l'état de repos d'une part et la réponse à une stimulation du nerf médian et la réponse à une stimulation du nerf médian pendant une imagination motrice d'autre part.

Les résultats sur l'estimation de la qualité de l'imagerie motrice à l'aide de la détection des valeurs aberrantes a montré qu'elle permet d'augmenter considérablement la précision pour un groupe de sujets.

Nous avons également présenté une approche pour la classification de l'imagerie motrice de mouvement combinée qui comprend la sélection et la classification des caractéristiques à l'aide d'une architecture avec trois classificateurs binaires en parallèle. Nous avons montré qu'en utilisant cette approche, il est possible d'atteindre le même niveau de précision qu'en utilisant l'approche multi-classes standard, mais avec un temps réduit pour la calibration de la BCI.

Enfin, nous avons proposé d'adapter la taille temporelle du noyau de la première couche convolutive et la profondeur du multiplicateur (qui détermine le nombre de filtres spatiaux) de la couche de convolution pour mieux gérer les composantes de moyenne et haute fréquences à partir de données EEG brutes Nous avons montré que l'architecture de réseau neuronal convolutif EEGNet-4.8 proposée pour la tâche de discrimination MNS vs MI + MNS ainsi qu'un ensemble de paramètres spécifiques (telles que la bande de fréquences, la disposition des électrodes, la valeur de dropout) améliorent de manière significative les prédictions en particulier avec un petit nombre d'électrodes : 84% pour 6 électrodes, 88% avec 13 électrodes; 94,5% avec 128 électrodes).

Enfin nous avons analysé les résultats, examiné les limites des solutions proposées et discuté des moyens de les dépasser dans les travaux futurs. Ces perspectives sont très prometteuses en particulier pour l'amélioration de la détection des réveils peropératoires.

Mots-clés: Apprentisssage profond, Electroencéphalographie, Interfaces cerveau-ordinateur, extraction de caractéristiques, imagination motrices. Brain-computer interfaces (BCI) are comparably new technologies that are becoming more and more used. The goal of BCI is to build a communication channel for interaction between the human brain and physical electronic devices. For example, such application might be to create applications for paralyzed patients that allow to control robots or wheelchairs. Recently, however, the areas of potential application of the technology have gone beyond communication and control and could be found in the fields of rehabilitation [1,2,3,4], treatment [5,6], mental state monitoring [START_REF] Zhang | Deep Learning Decoding of Mental State in Non-invasive Brain Computer Interface[END_REF][START_REF] Myrden | A Passive EEG-BCI for Single-Trial Detection of Changes in Mental State[END_REF][START_REF] Rached | Emotion Recognition Based on Brain-Computer Interface Systems[END_REF][START_REF] Mohammadpour | Classification of EEG-based emotion for BCI applications[END_REF][START_REF] Torres | EEG-Based BCI Emotion Recognition: A Survey[END_REF] and entertainment [START_REF] Maby | BCI Could Make Old Two-Player Games Even More Fun: A Proof of Concept with "Connect Four[END_REF][START_REF] Martišius | A Prototype SSVEP Based Real Time BCI Gaming System[END_REF][START_REF] Liao | Gaming control using a wearable and wireless EEG-based brain-computer interface device with novel dry foambased sensors[END_REF][START_REF] Oum | MindTactics: A Brain Computer Interface gaming platform[END_REF][START_REF] Ahn | A Review of Brain-Computer Interface Games and an Opinion Survey from Researchers, Developers and Users[END_REF].

List of Figures

Motor imagery (MI) is the most frequent interaction paradigm in the Brain-Computer Interface (BCI) domain and useful therapy for motor recovery, but the accuracy remains low and should be improved to create a reliable device that can be used in hospitals.

The use of machine learning in the field of BCI is very promising because machine learning techniques can detect activity and simulate complex processes in the human brain, recognize subject profiles, identify, assess, classify events, all of these allow to speed up and automate many processes in BCI. Most often, EEG signals are used to obtain information about the electrical activity of the brain, as the safest technology due to its noninvasiveness.

Typically, features are extracted from filtered EEG signals in alpha and beta frequency bands using various algorithms directly before classification. Common spatial pattern (CSP) algorithm is still the most popular method for feature extraction [START_REF] Fang | Extending CSP to Detect Motor Imagery in a Four-class BCI[END_REF][START_REF] Wang | Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface[END_REF][START_REF] Blankertz | Optimizing spatial filters for robust EEG single-trial analysis [revealing tricks of the trade[END_REF] from EEG signal and showed both advantages (i.e., high accuracy, flexibility, simplicity) and limitations (i.e., need for frequency band selection). Recently, other algorithms based on Riemannian geometry (such as tangent space (TS) algorithm [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF][START_REF] Barachant | Riemannian geometry applied to BCI classification[END_REF] and minimum distance to Riemannian mean (MDRM) algorithm [START_REF] Barachant | Riemannian geometry applied to BCI classification[END_REF][START_REF] Congedo | A New Generation of Brain-Computer Interface Based on Riemannian Geometry[END_REF][START_REF] Barachant | Multiclass Brain-Computer Interface Classification by Riemannian Geometry[END_REF]) showed promising results for MI detection.

BCI classification accuracy and performance levels have been greatly improved in the past decade mainly due to machine learning algorithms. However, there are still several issues that need to be addressed before BCI can be applied in real-life conditions. These problems include unstable classification performance, long calibration time, and poor reliability. Research on new developments is expected to bring solutions in this regard.

Along with the development of BCI, deep learning has been successfully applied in a variety of areas such as computer vision and natural language processing, significantly outperforming traditional machine learning approaches. Due to the considerable amount of investigations in this area, there are such technologies as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Generative Adversarial Networks (GANs).

While deep neural networks show better performances compared to standard machine learning methods in almost all application domains (such as computer vision or natural language processing), analysis of EEG signals particularly in the BCI domain remains an area where the use of deep learning techniques is still limited [START_REF] Roy | Deep learning-based electroencephalography analysis: a systematic review[END_REF][START_REF] Craik | Deep learning for electroencephalogram (EEG) classification tasks: a review[END_REF]. First of all, the reason is the small amount of data available for training compared to image data in computer vision domains. Moreover, EEG signals are high-dimensional, non-stationary, and have a low signal-to-noise ratio. The potential of deep learning architectures that can deal with raw EEG signals especially for motor imagery-based tasks is rather unexplored and is active research field [START_REF] Craik | Deep learning for electroencephalogram (EEG) classification tasks: a review[END_REF]. Several deep learning architectures already showed competitive results for MI vs. rest task in BCI domain [START_REF] Roy | Deep learning-based electroencephalography analysis: a systematic review[END_REF][START_REF] Craik | Deep learning for electroencephalogram (EEG) classification tasks: a review[END_REF][START_REF] Schirrmeister | Deep learning with convolutional neural networks for EEG decoding and visualization[END_REF][START_REF] Lawhern | EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[END_REF]. At the same time, we already showed the potential of convolutional neural networks to classify MI + MNS vs. MNS task on the smaller dataset [START_REF] Avilov | Deep learning techniques to improve intraoperative awareness detection from electroencephalographic signals[END_REF]. Thus, further research on applications of deep learning in the BCI domain and an assessment of how these applications can be used in practice to implement stable robust BCI would be beneficial.

Hundreds of millions of general anesthesia are performed each year on patients all over the world [START_REF] Weiser | Size and distribution of the global volume of surgery in 2012[END_REF]. Among these patients, 0.2-1.3% suffer from Accidental Awareness during General Anesthesia (AAGA) [START_REF] Mashour | Intraoperative awareness: controversies and non-controversies[END_REF], i.e. an unexpected awakening during a surgical procedure under general anesthesia. This happens when general anesthesia is not deep enough to compensate for all surgical stimuli related to the surgical procedure [START_REF] Macgregor | A waking nightmare: how can we avoid accidental awareness during general anaesthesia?[END_REF]. While the precise number of incidents is still debated, there is no doubt about the negative consequences for a patient experiencing an AAGA. Indeed, it can lead to physical suffering or psychological damage called post-traumatic stress disorder (PTSD) [START_REF] Almeida | Awake and unable to move: what can perioperative practitioners do to avoid accidental awareness under general anesthesia?[END_REF][START_REF] Pandit | Royal College of Anaesthetists, and Association of Anaesthetists of Great Britain and Ireland, "5th national audit project (NAP5) on accidental awareness during general anaesthesia: summary of main findings and risk factors[END_REF]. People with explicit memories of PTSD may have long-term post-operative effects including anxiety, irritability, fear of death, sleep disturbances, and depressive syndromes [START_REF] Blacher | On awakening paralyzed during surgery. a syndrome of traumatic neurosis[END_REF]. In cases of implicit memories, the consequences are not fully established, but all evidence suggests that the patient may suffer from it [START_REF] Tasbighou | Accidental awareness during general anaesthesia -a narrative review[END_REF][START_REF] Mashour | Intraoperative awareness: controversies and non-controversies[END_REF]. AAGAs also generate a high level of anxiety for anesthesiologists and are one of the three main causes of legal attacks by patients against the hospital [START_REF] Myles | Bispectral index monitoring to prevent awareness during anaesthesia: the b-aware randomised controlled trial[END_REF], with a significant economic impact.

Prevention of AAGA during surgery is a challenge because technologies used by anesthesiologists are not able to predict all patients' recovery. Currently, there are two ways to monitor the depth of general anesthesia: (i) by observing the clinical features (i.e. heart rate, blood pressure, movements, sweating), which may be objective or subjective in nature, (ii) or by recording a portion of the electroencephalographic (EEG) signal at the frontal cortex (i.e. Bispectral Index, Patient State Index, Entropy) [START_REF] Kent | Depth of anesthesia[END_REF]. Unfortunately, both clinical signs and new brain indices (based on frontal lobe activity) are not satisfactory enough to successfully assess the depth of general anesthesia and prevent AAGA [START_REF] Purdon | Clinical Electroencephalography for Anesthesiologists Part I: Background and Basic Signatures[END_REF][START_REF] Punjasawadwong | Bispectral index for improving anaesthetic delivery and postoperative recovery[END_REF][START_REF] Myles | Bispectral index monitoring to prevent awareness during anaesthesia: the b-aware randomised controlled trial[END_REF]. In particular, these cerebral indices do not ensure that consciousness during general anesthesia can be detected [START_REF] Avidan | Anesthesia awareness and the bispectral index[END_REF][START_REF] Avidan | Prevention of intraoperative awareness in a high-risk surgical population[END_REF]. Indeed, some studies have failed to demonstrate the superiority of these monitors over physical monitoring [START_REF] Avidan | The incidence of intraoperative awareness in the UK: under the rate or under the radar?[END_REF][START_REF] Mashour | Intraoperative awareness: controversies and non-controversies[END_REF] and others have demonstrated the unreliability of these techniques [START_REF] Schuller | Response of bispectral index to neuromuscular block in awake volunteers[END_REF].

The NeuroRhythms team in Loria proposed concept of an innovative passive brain-computer interface (BCI) based on detection of movement intentions to prevent AAGA earlier [START_REF] Rimbert | Motana: study protocol to investigate motor cerebral activity during a propofol sedation[END_REF][START_REF] Rimbert | Innovative brain-computer interface based on motor cortex activity to detect accidental awareness during general anesthesia[END_REF][START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF][START_REF] Avilov | Deep learning techniques to improve intraoperative awareness detection from electroencephalographic signals[END_REF]. Indeed, during AAGA patients typically try to move to alert the medical staff during an AAGA, but discover that they are unable to move [START_REF] Ghoneim | Awareness during anesthesia: risk factors, causes and sequelae: a review of reported cases in the literature[END_REF][START_REF] Pandit | Royal College of Anaesthetists, and Association of Anaesthetists of Great Britain and Ireland, "5th national audit project (NAP5) on accidental awareness during general anaesthesia: summary of main findings and risk factors[END_REF]. The proposed solution incorporates a Median Nerve Stimulation (MNS) causing specific modulations of activity in the sensorimotor cortex [START_REF] Salenius | Modulation of human cortical rolandic rhythms during natural sensorimotor tasks[END_REF], which are confirmed to be altered by an intention of movement [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF][START_REF] Neuper | Event-related dynamics of cortical thythms: frequency-specific features and functional correlates[END_REF]. Indeed, during a movement or motor imagery, a minimal power level is maintained in both alpha and beta bands [START_REF] Pfurtscheller | Event-related EEG/MEG synchronization and desynchronization: basic principles[END_REF], called event-related desynchronization (ERD). Then, 300-500 ms after the end of the task, there is an increase of power referred to as an event-related-synchronization (ERS) in the beta band, also known as a post-movement beta rebound, lasting 1 s [START_REF] Salenius | Modulation of human cortical rolandic rhythms during natural sensorimotor tasks[END_REF][START_REF] Kilavik | The ups and downs of beta oscillations in sensorimotor cortex[END_REF]. Concurrently, in the alpha band, the power returns to a baseline after several seconds. Typically, these patterns occur in the motor cortex, in the contralateral hemisphere [START_REF] Pfurtscheller | Event-related cortical desynchronization detected by power measurement of the scalp EEG[END_REF][START_REF] Pfurtscheller | Motor imagery and direct brain-computer communication[END_REF][START_REF] Bai | Asymmetric spatiotemporal patterns of event-related desynchronization preceding voluntary sequential finger movements: a high-resolution EEG study[END_REF] but can appear bilaterally [START_REF] Pfurtscheller | Post-movement beta synchronization. a correlate of an idling motor area?[END_REF][START_REF] Fok | An eeg-based brain computer interface for rehabilitation and restoration of hand control following stroke using ipsilateral cortical physiology[END_REF][START_REF] Formaggio | Modulation of event-related desynchronization in robot-assisted hand performance: brain oscillatory changes in active, passive and imagined movements[END_REF]. More interestingly, a MNS induces an ERD during the stimulation while an ERS appears after the stimulation [START_REF] Schnitzler | Involvement of primary motor cortex in motor imagery: a neuromagnetic study[END_REF][START_REF] Neuper | Event-related dynamics of cortical thythms: frequency-specific features and functional correlates[END_REF][START_REF] Salenius | Modulation of human cortical rolandic rhythms during natural sensorimotor tasks[END_REF] but the ERS is strongly abolished if a MNS is performed during an intention of movement [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF][START_REF] Rimbert | Innovative brain-computer interface based on motor cortex activity to detect accidental awareness during general anesthesia[END_REF][START_REF] Neuper | Event-related dynamics of cortical thythms: frequency-specific features and functional correlates[END_REF]. This interesting process could make the detection of AAGA with a passive BCI possible. Indeed, we can imagine a routine system where the patient would be stimulated at the median nerve, and the analysis of ERD and ERS modulations of the motor cortex would be used to find out if the patient has an intention to move [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF].

This new MNS-based BCI provides a solution for two major challenges: (i) be able to detect the intention to move of a person who is a victim of AAGA without any time marker, which is equivalent to continuously analyzing the EEG signal with limited indications concerning the temporal phases to be investigated. A second challenge is, therefore (ii) to obtain appropriate accuracy with an asynchronous BCI, which guarantees the reliability of a device that can be used in a medical context for which the motor intention is self-paced. Previous results show that this may provide a foundation for an innovative BCI that would allow the detection of AAGA [START_REF] Avilov | Deep learning techniques to improve intraoperative awareness detection from electroencephalographic signals[END_REF][START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF]. In previous studies [START_REF] Avilov | Deep learning techniques to improve intraoperative awareness detection from electroencephalographic signals[END_REF], it was already shown that MI + MNS vs. MNS classification results have better accuracy than a MI vs. rest classification, suggesting that MNS could be used as a trigger. Although the use of median nerve stimulation is promising, it still requires more detailed investigations, particularly concerning which frequency bands can be used to maximize discrimination between the two classes.

Firstly, the goal of this work is to increase classification performance for the task of MI detection. Secondly, it is to decrease the number of EEG electrodes. Since applying EEG electrodes takes time and increases the preparation time it is often not possible to install 64 or 128 electrodes before the surgery. The solution might be to select the electrode setup which will show acceptable classification accuracy with a minimum number of electrodes. As a threshold accepted in the present study, 80% of accuracy were selected.

The objective of this thesis is to better investigate the ability of deep learning techniques such as convolutional neural networks (CNN) [START_REF] Krizhevsky | ImageNet Classification with Deep Convolutional Neural Networks[END_REF] to extract features from raw EEG signals applied to the problem of detection of intraoperative awareness during general anesthesia in the case when MNS is applied. Our goal is not only to increase accuracy for MI + MNS vs. MNS task, but also to optimize the proposed method by (i) selecting the best frequency band, (ii) selecting the best feature extraction and classifier method and its parameters, and (iii) reducing the number of EEG electrodes to use.

Present thesis work is structured as follows.

In chapter 1 we have introduced the concepts of motor imagery (MI) based BCI. Signals which are frequently used in the BCI domain were described and we selected EEG technique for the further investigations because it is non-invasive, mobile, cheap, and easy to use method. We have presented neurophysiological concepts of brain rhythms and their localizations. Motor imagery as a frequently used interaction method for BCIs has been introduced as well as changes in the electrical activity of the human brain that usually comes with motor intentions.

Finally, the concept of a novel method based on median nerve stimulation for detection of accidental awareness during general anesthesia was presented and will be used as target application. Moreover we have shown influence of the median nerve stimulation on the oscillations in the sensorimotor cortex.

In chapter 2 we have presented preprocessing, feature extraction, feature selection, outlier detection and classification methods used in motor imagery based BCI.

As feature extraction methods, we have introduced time-frequency analysis, band power features, spatial analysis, and Riemannian geometry techniques. Common spatial pattern and filter bank common spatial pattern algorithms are still the most popular methods for feature extraction from EEG signals. Algorithms based on Riemannian geometry (such as tangent space algorithm and minimum distance to Riemannian mean algorithm) are becoming the new standard methods and could achieved better classification performance.

Also, we have presented machine learning methods frequently used in MI-based BCIs for the classification task, such as linear discriminant analysis, decision trees, gradient boosting, and support vector machines.

In chapter 3 we have introduced the concept of artificial neural networks and their architectures. We have selected Adam as an optimization algorithm for our work as it fixes vanishing learning rate and high variance problems and at the same time it progress faster comparing to other algorithms.

After analyzing of the existing solutions for the EEG signal processing, we have selected CNN architectures and their adaptations for raw EEG signals classification such as Deep Convolutional Network, Shallow Convolutional Network, EEGNetfor the following implementation and investigations.

In chapter 4 to implement and validate proposed methods, we have presented 3 different datasets, which were used in our experiments: motor imagery 2-class dataset, motor imagery 8-class dataset containing combined movements, and median nerve stimulation dataset. The latest dataset consists of conditions that allow us to compare two discriminative tasks: standard MI vs. rest and newly proposed MNS vs. MI+MNS task.

We have described participants, experimental tasks, time schemes, EEG data, preprocessing, feature extraction approaches, and cross-validation strategies that were used to obtain results for all datasets.

In chapter 5 we have represented results of classification using standard and deep learning techniques. Firstly, we have introduced a study about motor imagery quality estimation using outlier detection before BCI training.

Secondly, we have presented an approach for combined movement motor imagery classification which includes feature selection and classification using an architecture with three binary parallel-connected classifiers for detecting three basic movements and further detection of combinations of these movements.

Finally, motor imagery detection using deep learning methods on two different discriminating tasks (median nerve stimulation (MNS) vs. MI+MNS and MI vs. rest) was investigated. We have represented adaptation of deep learning architecture EEGNet that could better handle mid and high-frequency components from raw EEG data. Also, we have investigated the influence of parameters, such as frequency band, electrode layout and dropout date on the classification performance.

In chapter 6 we have analyzed results, considered limitations of proposed solutions, and discussed ways of solving them in future work. We have presented perspectives of proposed methods in the field of MI-based BCI. Also we have discussed the possibility of proposed setup application in clinical conditions with anesthetics for accessing the intraoperative awareness.

Chapter Conclusion finalizes the work and summarizes the application of deep learning techniques for MIbased BCI.

BCI definition and architecture

Brain-Computer Interfaces (BCIs) are systems created to exchange information between the brain and physical electronic external device (for example, a computer). In unidirectional BCI, external devices can either receive signals from the brain or send signals to it (for example, mimicking the retina of the eye when restoring vision with an electronic implant). Bidirectional BCI allows the brain and external devices to exchange information in both directions [START_REF] Boi | A Bidirectional Brain-Machine Interface Featuring a Neuromorphic Hardware Decoder[END_REF]. Nowadays using BCI it is possible to control computers, robots [START_REF] Irimia | BCI Application in Robotics Control[END_REF][START_REF] Tariq | EEG-Based BCI Control Schemes for Lower-Limb Assistive-Robots[END_REF][START_REF] Meng | Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks[END_REF], wheelchairs [START_REF] Abiyev | Brain-Computer Interface for Control of Wheelchair Using Fuzzy Neural Networks[END_REF][START_REF] Tang | Towards BCI-actuated smart wheelchair system[END_REF], and prostheses [START_REF] Li | An Approach for Brain-Controlled Prostheses Based on a Facial Expression Paradigm[END_REF][START_REF] Yahud | Prosthetic Hand for the Brain-computer Interface System[END_REF] but for simple tasks (Figure 1.1). Moreover, it is widely spread in different fields, like rehabilitation of stroke patients [1,2,3,4], treatment [5,6], mental state monitoring [START_REF] Zhang | Deep Learning Decoding of Mental State in Non-invasive Brain Computer Interface[END_REF][START_REF] Myrden | A Passive EEG-BCI for Single-Trial Detection of Changes in Mental State[END_REF][START_REF] Rached | Emotion Recognition Based on Brain-Computer Interface Systems[END_REF][START_REF] Mohammadpour | Classification of EEG-based emotion for BCI applications[END_REF][START_REF] Torres | EEG-Based BCI Emotion Recognition: A Survey[END_REF] and even in the entertainment [START_REF] Maby | BCI Could Make Old Two-Player Games Even More Fun: A Proof of Concept with "Connect Four[END_REF][START_REF] Martišius | A Prototype SSVEP Based Real Time BCI Gaming System[END_REF][START_REF] Liao | Gaming control using a wearable and wireless EEG-based brain-computer interface device with novel dry foambased sensors[END_REF][START_REF] Oum | MindTactics: A Brain Computer Interface gaming platform[END_REF][START_REF] Ahn | A Review of Brain-Computer Interface Games and an Opinion Survey from Researchers, Developers and Users[END_REF]. BCI technology is a potentially powerful communication and control option between users and systems [START_REF] Erp | Brain-computer interfaces: Beyond medical applications[END_REF].

BCI could have different architectures according to the purpose of the system. But usually, all BCIs consists of five main modules, namely, input signal acquisition module, signal preprocessing module, feature extraction, classification, and feature translation (Figure 1.2).

The signal acquisition module is responsible for signal recording from the brain. It could be implemented using invasive or non-invasive procedures i.e. using intracranial or extracranial sensors.

The signal preprocessing module includes artifact removal (such as eye-blink artifacts), sampling, and filtering in required frequency bands.

The feature extraction module analyzes brain signals obtained previously. At this stage features, suitable are identified using mathematical methods. This allows defining useful information (power changes, peak amplitudes, etc) for classification. At this stage dimensionality reduction and feature selection occurs. The goal of the classification module is to map feature vectors obtained on the previous stage to the correct class among N . Here a class corresponds to a brain condition that the system detects and passes to the output module to generate different commands.

Translation module is responsible for translation classification results into the commands for device interaction which may consist of device control or commands transmission. The number of previously considered classes corresponds to several control commands. These actions could be used to control computers, wheelchairs, robotics, or produce commands for rehabilitation, treatment, mental state monitoring devices, and video games.

Besides, BCIs could include an additional feedback module to provide users with information about commands or brain activity. It could be helpful during training sessions to assist users in providing them information about errors or the quality of their mental actions. The aim is to improve the learning stage of the device and increase performance. Occasionally BCIs can have two feedback modules: one after the feature extraction part which responsible for the quality of the motor task performance and brain activity in general and other feedback after the translation module.

Figure 1.2 represents the general architecture of BCI, including modules listed above.

Neurophysiologcal aspects of BCI 1.2.1 Signals for BCI

The electrical activity of the brain is the main source of information for the BCI. There are many techniques for recording brain signals: electroencephalography (EEG), electrocorticography (ECoG), magnetoencephalography (MEG), functional near-infrared spectroscopy (fNIRS), functional magnetic resonance imaging (fMRI), and intracranial microelectrode array (MEA) recording. All of these techniques have their advantages and disadvantages according to scale level, noise, cost of technology, and invasiveness [START_REF] Nicolas-Alonso | Brain Computer Interfaces, a Review[END_REF]. Further, these techniques are presented in more detail.

Electrocorticography (ECoG)

Electrocorticography (ECoG) is an intracranial method recording brain potentials using electrodes applied directly on the surface of the cortex. This method has a high spatial resolution with interelectrode spacing near 1 mm [START_REF] Freeman | Spatial spectral analysis of human electrocorticograms including the alpha and gamma bands[END_REF] and temporal resolution about 1kH, as well as a high signal-to-noise ratio (SNR). The main and extremely important disadvantage of this method is the need to perform surgery following postoperative recovery. Due to the invasiveness of this method, it makes its potential application inconvenient and limited to patients with strong disabilities.

Magnetoencephalography (MEG)

Magnetoencephalography is a functional neuroimaging method for mapping brain activity by a recording of magnetic fields produced by electric currents occurring in the neurons of the brain. The advantages of this method are high spatial (millimeters) and temporal (milliseconds) resolution [START_REF] Gage | Chapter 3 -Observing the Brain[END_REF]. But the high cost of equipment and the additional requirements, such as a magnetically shielded room from external magnetic signals, including the Earth's magnetic field make applications of this technology limited and not suited for many BCI cases.

Functional Magnetic Resonance Imaging (fMRI)

This method is based on the fact that cerebral blood flow and neuronal activation are interrelated. Changes in blood flow (hemodynamic response) are associated with the consumption of energy by brain cells. When an area of the brain is used, blood flow to that area also increases. So functional MRI measures brain activity by detecting changes in blood flow. This method has a high spatial resolution (millimeters), but at the same time it has several disadvantages: first of all, the equipment is expensive and cumbersome and this technique also has a low temporal resolution, because changes in the oxygenation of the cells are not instant and it takes from 1 up to 8 seconds.

Functional Near Infrared Spectroscopy (fNIRS)

Functional Near-Infrared Spectroscopy (fNIRS) is a non-invasive technique that measures cerebral hemodynamic responses using near-infrared light. Cerebral hemodynamic responses correspond to activation or deactivation of the parts of the brain. This method capable to track changes both in oxy-and deoxyhemoglobin concentration. The advantage of fNIRS is its spatial resolution (1cm 2 ) [START_REF] Torricelli | Time domain functional NIRS imaging for human brain mapping[END_REF] which is suitable for the BCI domain and portability of the system. A Head cap with sensors makes this technique comfortable for users. fNIRS could be used for BCI to monitor the depth of anesthesia.

Electroencephalography (EEG)

Electroencephalography (EEG) is a non-invasive technique, which measures the electrical activity of the brain. EEG signal is a complex oscillating electrical process that is the result of electrical summation and filtration of elementary processes occurring in the neurons of the brain. EEG is widely used in the diagnosis of various diseases or disorders of the brain. This method is cheap, safe, and easy to use, that's why it is widely spread in the diagnosis of Parkinson's and Alzheimer's disease, epilepsy, panic attacks, hysteria, drug poisoning, and other disorders of the brain. Moreover, it has a high temporal resolution (milliseconds). EEG recordings use a head cap with electrodes located all over the scalp or over specific regions of the head.

The main disadvantages of EEG are low noise-to-signal ratio and relatively low spatial resolution (1cm 2 ). Now EEG systems become more and more comfortable for users due to developments in areas of dry electrodes and wireless EEG head caps that allow full mobility for the user. A comparison of the main advantages and disadvantages of the signal recording techniques are presented in the table 1.1.

Brain rhythms

Brain rhythm is a dynamic process characterized by simultaneous activation of a specific population of neurons, or synchronization between local oscillations of two or more populations of neurons [START_REF] Buzsáki | Rhythms of the brain. Rhythms of the brain[END_REF]. The synchronized activity of large populations of neurons is the main mechanism for the formation of rhythms in the human brain. Rhythms play an important role in the transmission and processing of information in the central nervous system (sleep, memory, attention, spatial navigation, etc.) Brain rhythms can be divided into alpha-, beta-, theta-and delta-rhythms (Figure 1.3). Each of these rhythms is a special type of cortical activity characterized by brain waves and is associated with states of consciousness such as anxiety, rest, dreaming, or sleep [START_REF] Buzsáki | Rhythms of the brain. Rhythms of the brain[END_REF]. Brain activity changes continuously. Therefore, our mental state reflects the mix of different waves of brain activity at different localizations.

Brain rhythms are characterized by the largest power of wave activity. For example, a person with closed eyes may have a high power of alpha and low power of beta waves in the visual cortex because it does not process visual information. On the other hand, when the eyes are open, the power of the alpha waves decreases sharply, and the beta waves increase as a result of processing the visual information coming to the visual part of the cortex.

Examples of EEG waves are given below and are shown in Figure 1.3.

• Delta waves (< 4 Hz): include all EEG waves with a frequency below 4 Hz and often have amplitude 2-4 times higher than most other types of brain waves. They occur during deep rest, dreamless sleep, frontally in adults, posteriorly in children, and with severe brain damage [START_REF] Kumar | Analysis of Electroencephalography (EEG) Signals and Its Categorization-A Study[END_REF]. • Theta waves (4 -8 Hz): This rhythm is usually observed during deep relax state, tiredness, sleepiness, REM (i.e., Rapid Eye Movement) sleep, and could be caused by emotional stress, alcohol consumption and fatigue [START_REF] Matsuoka | Theta rhythms: State of consciousness[END_REF]. Theta waves are divided into two types: the hippocampal theta waves and cortical theta waves.

• Alpha waves (8 -13 Hz): are rhythmic oscillations with a frequency within 8-13 Hz, which are detected in the EEG of almost all healthy adults at rest state, during daydream and calm. These waves are most intense in the occipital lobe, but they can also be observed in the parietal and frontal areas of the scalp. Their value is usually about 50 µV . Alpha waves disappear during deep sleep. Alpha activity varies throughout the day according to our natural biorhythms, so this factor must be taken into account when performing an EEG analysis.

• Mu waves (8 -12 Hz): It is a specific rhythm associated with motor activity and observed mostly under the primary motor cortex. Its peculiarity is that Mu waves decrease during the preparation and the execution of the movement and this is called Event-Related Desynchronization (ERD) [START_REF] Niedermeyer | Electroencephalography: Basic Principles, Clinical Applications, and Related Fields[END_REF]. While at rest state it increases and called Event-Related Synchronization (ERS) [START_REF] Pfurtscheller | Event-related synchronization (ERS): an electrophysiological correlate of cortical areas at rest[END_REF].

• Beta waves (13 -30 Hz): When a person's attention is focused on a certain type of mental activity, alpha waves are replaced by asynchronous, high-frequency, but low-amplitude beta waves. Beta waves oscillations with a frequency from 13 to 30 Hz appeared during the alert, active thinking, anxiety, panic attack, focus, concentration [START_REF] Kumar | Analysis of Electroencephalography (EEG) Signals and Its Categorization-A Study[END_REF]. This rhythm is most strongly expressed in frontal areas, but various types of intensive mental activity sharply amplify and extend to other areas of the brain. Thus, the power of beta-rhythm increases with mental stress or emotional arousal. Their amplitude is 4-5 times smaller than the amplitude of alpha waves and is 5-30 µV .

• Gamma waves (> 30 Hz): These waves are responsible for perception and consciousness. Often appeared during the combination of two senses in somatosensory cortex [START_REF] Kumar | Analysis of Electroencephalography (EEG) Signals and Its Categorization-A Study[END_REF]. Also, their presence may be caused by electromyographic artifacts [START_REF] Whitham | Scalp electrical recording during paralysis: quantitative evidence that EEG frequencies above 20 Hz are contaminated by EMG[END_REF].

Brain rhythms localization

The cerebral cortex is an accessible source of information about motor and executive behavior, thus it represents an area of interest to researchers in the BCI domain. Interacting through synaptic connections, neurons form electrical impulses that control the activity of the entire body [START_REF] Buzsaki | Rhythms of the Brain[END_REF].

The left hemisphere of the human brain is responsible for the right half of the body movement, the right one for the left. Right and left hemispheres have different functions. Visual signals go to the visual cortex (in the occipital lobe), tactile to the somatosensory cortex (in the parietal lobe), olfactory signals to the olfactory cortex on the base of the frontal lobe and medial aspect of the temporal lobe, etc. In the associative areas of the cortex, the integration of sensory signals of different types takes place [START_REF] Buzsaki | Rhythms of the Brain[END_REF].

The motor areas of the cortex (the primary motor cortex and other areas of the frontal lobes showed in Figure 1.4) are responsible for regulating movement.

The brain has the property of plasticity. If one of its parts is affected, other parts after a while can compensate for its loss of function. Brain plasticity also plays a role in learning new skills.

The frontal lobe represents the most interest for the BCI domain. It is the largest lobe and is located at the front of each hemisphere of the brain. The frontal lobe is covered by the frontal cortex. The frontal cortex includes the premotor cortex, and the primary motor cortex responsible for movements. The front part of the frontal lobe is covered by the prefrontal cortex.

Prefrontal cortex

The prefrontal cortex is the part of the cerebral cortex which covers the front part of the frontal lobe. This area of the brain is involved in the planning of complex cognitive behavior, personality manifestation, decision-making, and regulation of social behavior [START_REF] Yang | Prefrontal Structural and Functional Brain Imaging findings in Antisocial, Violent, and Psychopathic Individuals: A Meta-Analysis[END_REF]. The main purpose of this area of the brain is considered as the coordination of thoughts and actions in correspondence with internal goals [START_REF] Miller | The prefrontal cortex: categories, concepts and cognition[END_REF].

The main function of the prefrontal cortex is an executive function.

Primary motor cortex

One of the main functions of the primary motor cortex is to control voluntary discrete movements performed by muscle groups.

The primary motor cortex contains the topographic representation of various areas of the body upside down, from the legs (top of the hemisphere) to the mouth (bottom of the hemisphere) (Figure 1.5). The primary motor cortex of each hemisphere controls muscles on the opposite side of the body. The surface area of the regions of the primary motor cortex that control various parts of the body is proportional not to the physical surface of these parts, but to the number and degree of complexity of their motor responses. Because of this, about 75% of the main volume of the primary motor cortex is represented by insignificant (by weight), but performing subtle nonstereotypical movements of the muscles of the hands, fingers, facial muscles of the face, lips, tongue [START_REF] Riehle | Motor Cortex in Voluntary Movements: A Distributed System for Distributed Functions[END_REF][START_REF] Rothwell | Control of Human Voluntary Movement[END_REF][START_REF] Rosenbaum | Human Motor Control[END_REF].

Along with the primary, there is a secondary motor cortex, which includes the premotor cortex and supplementary motor area.

Electrode placement

The way the electrodes are placed on the scalp to obtain an EEG is called a derivation scheme. The standard method for covering different areas of the brain is known as the International 10-20 system [START_REF] Klem | The ten-twenty electrode system of the International Federation[END_REF]. Figure 1.6 represents the electrode placement over the scalp. The main idea of this system is the uniform coverage of the cortex based on the proportional distance separating the electrodes from each other. For this purpose, nasion and inion are taken as references, of which five aligned electrodes are placed at 10%, 20%, 20%, 20%, 20%, and 10% of the distance separating both points.

On the International 10-20 system electrodes C3, Cz and C4 are located above the areas of the cerebral cortex, which are responsible for the motor activity of the right hand, legs, and left hand, respectively.

There are also other layouts to cover a scalp, such as the ABC layout represented in Figure 1.7. ABC layouts partly overlap with the international 10-20 system.

Motor imagery

Motor imagery (MI) is a mental process during which a person imitates or repeats a certain movement without execution. At the same time, the person feels its performance [START_REF] Avanzino | Motor imagery influences the execution of repetitive finger opposition movements[END_REF]. It is widely used for sports training as a mental practice of movement, for neurological rehabilitation, as well as for studies of the structure of unconscious processes that precede the performance of movement [START_REF] Decety | Action representation and its role in social interaction[END_REF]. MI represents the result of conscious access to the content of the intention of a movement [START_REF] Jeannerod | Mental imagery in the motor context[END_REF], which is usually performed unconsciously during movement preparation [START_REF] Lotze | Motor imagery[END_REF].

MI has two different components, namely the visual imagery (VI) and the kinesthetic motor imagery (KMI) [START_REF] Neuper | Imagery of motor actions: Differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG[END_REF]. VI activated predominantly the occipital regions and the superior parietal lobules [START_REF] Guillot | Brain activity during visual versus kinesthetic imagery: an fMRI study[END_REF], whereas KMI generates an event-related desynchronization and an event-related synchronization in the contralateral sensorimotor area, which is similar to the one observed during the preparation of a real movement (RM) [START_REF] Pfurtscheller | Event-related EEG/MEG synchronization and desynchronization: basic principles[END_REF].

MI is widely used in the BCI domain and it is the most common discriminating task. MI-based BCIs are used in the recovery of the motor capacity after neurological damage such as rehabilitation for stroke patients [START_REF] Butler | Mental practice with motor imagery: evidence for motor recovery and cortical reorganization after stroke[END_REF] or monitoring consciousness during general anesthesia [START_REF] Blokland | Decoding motor responses from the eeg during altered states of consciousness induced by propofol[END_REF]. For example, MI training is a promising approach in facilitating paretic limb recovery [START_REF] Liu | Newer challenges to restore hemiparetic upper extremity after stroke: HANDS therapy and BMI neurorehabilitation[END_REF].

Imaginary movements are used to train brain-computer interfaces, but it is often impossible to assess the quality of a subject's performance before training the BCI. The process of performing imaginary movement for the user is not an easy task and requires prior training. It could be tiring, boring, and hard to concentrate on the subject. The condition when BCI users fail to reach competence in using a BCI during a standard training process is called BCI illiteracy [START_REF] Vidaurre | Towards a Cure for BCI Illiteracy[END_REF][START_REF] Thompson | Critiquing the Concept of BCI Illiteracy[END_REF]. Therefore, the classification accuracy for some subjects is frequently many times lower than for other subjects. But the reason why is not well-known. All existing methods could be improved if there is a method of assessing the quality of subjects MI performance. Training data could be filtered to make models generalize better, and therefore increase the accuracy of detecting imaginary movements on new data. We will propose a method based on outlier detection to estimate the quality of imaginary movements before model training (section 5.1).

Event-related synchronization and desynchronization

Dynamic of motor activity is observed by the index called Event-related synchronization/desynchronization (ERD/ERS%). Event-Related Desynchronization (ERD) is the suppression of oscillations in the alpha and beta rhythms that takes place in the sensorimotor cortex during the movement [START_REF] Pfurtscheller | Event-related cortical desynchronization detected by power measurement of the scalp EEG[END_REF]. ERD could be observed not only during a real and imaginary movement itself but also before them, during the preparation of movement [START_REF] Pfurtscheller | Patterns of cortical activation during planning of voluntary movement[END_REF][START_REF] Klimesch | Event-related desynchronization, erd-mapping and hemispheric differences for words and numbers[END_REF]. Event-Related Synchronization (ERS) is an opposite occurrence of ERD which is characterized by increasing of power in alpha and beta bands just after the motor task [START_REF] Pfurtscheller | Event-related synchronization (ERS): an electrophysiological correlate of cortical areas at rest[END_REF].

More precisely, compared to a resting state taken before motor imagery, several power modulations are observed in the alpha and in the beta bands of the EEG signal measured over the sensorimotor area corresponding to the body part involved in the motor imagery.

At the beginning of the motor task, there is a gradual power decrease (ERD) in the alpha and in the beta bands (Figure 1.8). Then, as seen from Figure 1.8 a low power level is maintained during the movement.

Finally, from 300 to 500 milliseconds after the end of the motor imagery, there is an ERS or post-movement beta rebound with a duration of about one second. Although several studies showed an activity uniquely in the contralateral area [START_REF] Pfurtscheller | Motor imagery and direct brain-computer communication[END_REF], other studies showed that ERD and ERS are also in the ipsilateral area [START_REF] Fok | An eeg-based brain computer interface for rehabilitation and restoration of hand control following stroke using ipsilateral cortical physiology[END_REF].

Detailed algorithm for computing ERD/ERS% will be presented in the section 2.2.2.1

The emergence of ERD and ERS patterns during and after a MI has been intensively studied in the BCI domain [START_REF]Brain-Computer Interfaces: Principles and Practice[END_REF] to define detectable commands for the system.

BCI application for general anesthesia monitoring

General anesthesia is an artificially induced reversible depression of the central nervous system with a switchoff of consciousness, suppression of sensitivity (primarily pain) and reflex reactions, and a decrease in skeletal muscle tone. All this occurs when one or more general anesthetics are administered, the optimal dose and combination of which is selected by the anesthesiologist, taking into account the individual characteristics of a patient and depending on the type of medical procedure [START_REF] Lin | Fundamentals of Anaesthesia[END_REF][START_REF] Barash | Clinical Anesthesia[END_REF]. Anesthesia drugs inhibit interneuronal (synaptic) transmission of excitation in the central nervous system. At the same time, the transmission of afferent impulses is disrupted [START_REF] Gray | General Anaesthesia[END_REF][START_REF] Lin | Fundamentals of Anaesthesia[END_REF].

The main goal of anesthesia is to slow down the body's reactions to surgery, primarily the sensation of pain. At the same time, drug sleep, with which the concept of "anesthesia" is most often associated, is only one component of anesthesia. When carrying out anesthesia, it is also important to suppress or significantly reduce the severity of the autonomic (automatic) reactions of the body to surgical trauma, which are manifested by an increase in heart rate, an increase in blood pressure, and other phenomena that can occur even when consciousness is switched off. This suppression of autonomic responses is called pain relief or analgesia. Another component of anesthesia is muscle relaxation, which is necessary to ensure normal conditions for the work of surgeons [START_REF] Barash | Clinical Anesthesia[END_REF].

Accidental Awareness during General Anesthesia (AAGA)

Accidental Awareness during General Anesthesia (AAGA) is an unexpected awakening during a surgical procedure under general anesthesia [START_REF] Mashour | Intraoperative awareness: controversies and non-controversies[END_REF]. This happens when general anesthesia is not deep enough to compensate for all surgical stimuli related to the surgical procedure [START_REF] Macgregor | A waking nightmare: how can we avoid accidental awareness during general anaesthesia?[END_REF]. While the precise number of incidents is still debated, there is no doubt about the negative consequences for a patient experiencing an AAGA. Indeed, it can lead to physical suffering or psychological damage called post-traumatic stress disorder (PTSD) [START_REF] Almeida | Awake and unable to move: what can perioperative practitioners do to avoid accidental awareness under general anesthesia?[END_REF][START_REF] Pandit | Royal College of Anaesthetists, and Association of Anaesthetists of Great Britain and Ireland, "5th national audit project (NAP5) on accidental awareness during general anaesthesia: summary of main findings and risk factors[END_REF]. People with explicit memories of PTSD may have long-term post-operative effects including anxiety, irritability, fear of death, sleep disturbances, and depressive syndromes [START_REF] Blacher | On awakening paralyzed during surgery. a syndrome of traumatic neurosis[END_REF]. In cases of implicit memories, the consequences are not fully established, but all evidence suggests that the patient may suffer from it [START_REF] Tasbighou | Accidental awareness during general anaesthesia -a narrative review[END_REF][START_REF] Mashour | Intraoperative awareness: controversies and non-controversies[END_REF]. AAGAs also generate a high level of anxiety for anesthesiologists and are one of the three main causes of legal attacks by patients against the hospital [START_REF] Myles | Bispectral index monitoring to prevent awareness during anaesthesia: the b-aware randomised controlled trial[END_REF], with a significant economic impact.

Prevention of AAGA during surgery is a challenge because technologies used by anesthesiologists are not able to predict all patients' recovery. Currently, there are two ways to monitor the depth of general anesthesia: (i) by observing the clinical features (i.e. heart rate, blood pressure, ciliary reflex, sweating), which may be objective or subjective in nature, (ii) or by analyzing indices from the electroencephalographic signal at the frontal cortex (i.e. Bispectral Index, Patient State Index, Entropy) [START_REF] Kent | Depth of anesthesia[END_REF]. Unfortunately, both clinical signs and brain indices based on frontal lobe activity are not satisfactory enough to successfully assess the depth of general anesthesia and prevent AAGA [START_REF] Purdon | Clinical Electroencephalography for Anesthesiologists Part I: Background and Basic Signatures[END_REF][START_REF] Punjasawadwong | Bispectral index for improving anaesthetic delivery and postoperative recovery[END_REF][START_REF] Myles | Bispectral index monitoring to prevent awareness during anaesthesia: the b-aware randomised controlled trial[END_REF]. In particular, these cerebral indices do not ensure that consciousness during general anesthesia can be detected [START_REF] Avidan | Anesthesia awareness and the bispectral index[END_REF][START_REF] Avidan | Prevention of intraoperative awareness in a high-risk surgical population[END_REF]. Indeed, some studies have failed to demonstrate the superiority of these monitors over physical monitoring [START_REF] Avidan | The incidence of intraoperative awareness in the UK: under the rate or under the radar?[END_REF][START_REF] Mashour | Intraoperative awareness: controversies and non-controversies[END_REF] and others have demonstrated the unreliability of these techniques [START_REF] Schuller | Response of bispectral index to neuromuscular block in awake volunteers[END_REF].

BCI for AAGA detection

In 2016, Blokland et al. [START_REF] Blokland | Decoding motor responses from the eeg during altered states of consciousness induced by propofol[END_REF] firstly proposed an approach to use BCI for detection of AAGA. The idea of such a BCI is to detect the intention of moving from the patient which usually preceded an AAGA. In this work, Blokland et al. investigated the effect of propofol on the EEG signals recorded over the motor cortex. In this investigation, subjects were not completely anesthetized and were asked to perform movements according to sound beeps while an increasing dosage of anesthetic was administered to them. This first approach shows that the BCI domain could contribute to the issue of AAGA. However, the study conducted by Blokland et al. was based on synchronous active communication, i.e., the voluntary subject was explicitly asked to perform a movement during the experiment after a visual and audible signal, which does not realistically reflect the conditions during intraoperative awareness. Therefore, a strategy must be found to design a passive BCI whose task would be to detect the intention of movement of an AAGA victim [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF].

NeuroRhythms team from LORIA laboratory proposed concept of an innovative passive brain-computer interface (BCI) based on detection of movement intentions to prevent AAGA [START_REF] Rimbert | Motana: study protocol to investigate motor cerebral activity during a propofol sedation[END_REF][START_REF] Rimbert | Innovative brain-computer interface based on motor cortex activity to detect accidental awareness during general anesthesia[END_REF][START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF][START_REF] Avilov | Deep learning techniques to improve intraoperative awareness detection from electroencephalographic signals[END_REF]. Indeed, during AAGA patients typically try to move to alert the medical staff during an AAGA, but discover that they are unable to move [START_REF] Ghoneim | Awareness during anesthesia: risk factors, causes and sequelae: a review of reported cases in the literature[END_REF][START_REF] Pandit | Royal College of Anaesthetists, and Association of Anaesthetists of Great Britain and Ireland, "5th national audit project (NAP5) on accidental awareness during general anaesthesia: summary of main findings and risk factors[END_REF]. The proposed solution incorporates a Median Nerve Stimulation (MNS) causing specific modulations of activity in the sensorimotor cortex [START_REF] Salenius | Modulation of human cortical rolandic rhythms during natural sensorimotor tasks[END_REF], which are confirmed to be altered by an intention of movement [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF][START_REF] Neuper | Event-related dynamics of cortical thythms: frequency-specific features and functional correlates[END_REF]. Experimental design of such a median nerve stimulation based BCI presented in section 4.3.

This new MNS-based BCI provides a solution for two major challenges: (i) be able to detect the intention to move of a person who is a victim of AAGA without any time marker, which is equivalent to continuously analyzing the EEG signal with limited indications concerning the temporal phases to be investigated. A second challenge is, therefore (ii) to obtain appropriate accuracy with an asynchronous BCI, which guarantees the reliability of a device that can be used in a medical context for which the motor intention is self-paced. Results show that this may provide a foundation for an innovative BCI that would allow the detection of AAGA [START_REF] Avilov | Deep learning techniques to improve intraoperative awareness detection from electroencephalographic signals[END_REF][START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF].

Median nerve stimulation (MNS) in BCI

Electrical stimulation of the median nerve is a safe, inexpensive, non-invasive technique that is commonly used for neuroresuscitation of coma patients [START_REF] Cooper | Electrical treatment of coma via the median nerve[END_REF][START_REF] Wu | Right median nerve electrical stimulation for acute traumatic coma (the Asia Coma Electrical Stimulation trial): study protocol for a randomised controlled trial[END_REF] and to alleviate symptoms of nausea and vomiting [START_REF] Maharjan | Non-invasive High Frequency Median Nerve Stimulation Effectively Suppresses Olfactory Intensity Perception in Healthy Males[END_REF].

As classification performance of standard MI vs. rest discriminating task remains low in the BCI domain there is a requirement for improvement. MNS could be one of the possible triggers for BCIs [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF]. Of course, the use of this technique reduces the user's mobility. But MNS can be used for BCI that does not require mobility, such as BCI for AAGA detection.

Influence of MNS on ERD/ERS

It was observed that there is a difference in oscillations in the sensorimotor cortex during motor activity depending on whether the median nerve of a person was stimulated or not [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF].

While, during a movement or motor imagery, a minimal power level is maintained in both alpha and beta bands [START_REF] Pfurtscheller | Event-related EEG/MEG synchronization and desynchronization: basic principles[END_REF], called event-related desynchronization (ERD). Then, 300-500 ms after the end of the task, there is an increase of power referred to as an event-related-synchronization (ERS) in the beta band, also known as postmovement beta rebound (Figure 1.9), lasting 1 s [START_REF] Salenius | Modulation of human cortical rolandic rhythms during natural sensorimotor tasks[END_REF][START_REF] Kilavik | The ups and downs of beta oscillations in sensorimotor cortex[END_REF]. Concurrently, in the alpha band, the power returns to a baseline after several seconds. Typically, these patterns occur in the motor cortex, in the contralateral hemisphere [START_REF] Pfurtscheller | Event-related cortical desynchronization detected by power measurement of the scalp EEG[END_REF][START_REF] Pfurtscheller | Motor imagery and direct brain-computer communication[END_REF][START_REF] Bai | Asymmetric spatiotemporal patterns of event-related desynchronization preceding voluntary sequential finger movements: a high-resolution EEG study[END_REF] but can appear bilaterally [START_REF] Pfurtscheller | Post-movement beta synchronization. a correlate of an idling motor area?[END_REF][START_REF] Fok | An eeg-based brain computer interface for rehabilitation and restoration of hand control following stroke using ipsilateral cortical physiology[END_REF][START_REF] Formaggio | Modulation of event-related desynchronization in robot-assisted hand performance: brain oscillatory changes in active, passive and imagined movements[END_REF].

More interestingly, a MNS induces an ERD during the stimulation while an ERS appears after the stimulation [START_REF] Schnitzler | Involvement of primary motor cortex in motor imagery: a neuromagnetic study[END_REF][START_REF] Neuper | Event-related dynamics of cortical thythms: frequency-specific features and functional correlates[END_REF][START_REF] Salenius | Modulation of human cortical rolandic rhythms during natural sensorimotor tasks[END_REF] but the ERS is strongly abolished if a MNS is performed during an intention of movement [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF][START_REF] Rimbert | Innovative brain-computer interface based on motor cortex activity to detect accidental awareness during general anesthesia[END_REF][START_REF] Neuper | Event-related dynamics of cortical thythms: frequency-specific features and functional correlates[END_REF]. This interesting process could make the detection of AAGA with a passive BCI possible. Indeed, we can imagine a routine system where the patient would be stimulated at the median nerve, and the analysis of ERD and ERS modulations of the motor cortex would be used to find out if the patient has an intention to move [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF].

Results in chapter 5 show that a MI + MNS vs. MNS classification results have better accuracy than a MI vs. rest classification, suggesting that MNS could be used as a trigger.

Conclusion

In this chapter, the basic concepts of brain-computer interfaces were represented. Signals which are used in the BCI domain were described and the EEG technique was selected as a non-invasive, mobile, cheap, and easy to use method.

Also in this chapter were discussed neurophysiological concepts of brain rhythms and their localization. Motor imagery as a frequently used trigger for BCIs was represented as well as event-related synchronization and desynchronization which takes place in the brain rhythms during motor imagery and motor tasks.

This thesis will present how to improve it using classic machine learning and deep learning techniques to create a reliable device that can be used in hospitals.

Firstly, in chapter 2 we will present classic machine learning methods for feature extraction and classification in the MI-based BCI domain.

Secondly, we will consider deep learning techniques as universal feature extractor combined together with the classification stage for raw EEG signals in chapter 3.

All the results obtained in this thesis will be carried out on real EEG data described in chapter 4.

Results of machine learning and deep learning techniques application to improve MI detection from the EEG signals will be represented in chapter 5.

The problem of accidental awareness during general anesthesia was posed during this chapter and current techniques for its detection were discussed. Moreover, the concept of the novice method based on BCI and median nerve stimulation for detection of accidental awareness during general anesthesia was presented. Although the use of median nerve stimulation is promising, it still requires more detailed investigations, particularly concerning which frequency bands and electrode layouts can be used to maximize the detection of accidental awareness during general anesthesia. These investigations will be presented in section 5.3.

Another challenge for this kind of BCI remains to be used in clinical practice. We will discuss it further in section 6.4.

Chapter 2

Classic machine learning methods for motor imagery detection Machine learning is a subsection of artificial intelligence that includes a set of methods used to create complex models and algorithms. These models allow researchers, engineers, and analysts to produce reliable solutions and results and reveal the hidden dependence in data through training [START_REF] Kohavi | Special issue on applications of machine learning and the knowledge discovery process[END_REF].

The use of machine learning in the field of BCI is important because machine learning techniques can detect brain activities and reveal complex processes in the human brain, recognize subject profiles, identify, assess, classify events, all of these allow to speed up and automate many processes in BCI. Due to the rapid development of 19 technology, particularly in the field of parallel computing, cloud technology and computing capabilities usage of machine learning has increased.

The whole process includes important steps such as data preprocessing, feature extraction, feature selection, model training, prediction. Some of these parts used in our projects will be described in this chapter.

Preprocessing

After the acquisition, EEG signals are preprocessed. Typically, the signals pass through several stages of preprocessing: spatial filtering, frequency filtering, and artifact removal.

Spatial filtering

Spatial filtering is done to highlight the specific information at one location compared to the neighborhood. To accomplish this, the mean or weighted mean of the signals from the other electrodes is subtracted from the target signal.

There are a few different methods of spatial filtering for large electrode setups (Figure 2.1). A ground electrode for EEG recordings is often placed on the forehead, but could be placed anywhere else on the body. Usually, receiving a signal, each electrode referenced to an electrode behind one ear or both, this is called an "ear reference" (Figure 2.1a). Common average reference (CAR) refers to a filter for which the mean of the signals from all electrodes is taken as a baseline (Figure 2.1b). A bipolar filter is a subtraction one signal from another (not represented in Figure 2.1a). The Laplacian filter approximates the second spatial derivative of the signal. For a small Laplacian filter, the distance-weighted mean of the four nearest electrodes is calculated and used as a reference (Figure 2.1c). For a large Laplacian filter, the next-nearest neighbors are used [START_REF] Mcfarland | Spatial filter selection for EEG-based communication[END_REF] (Figure 2.1d).

Common average reference (CAR)

Common average reference (CAR) is the most often used spatial filter [START_REF] Alhaddad | P300 Speller Efficiency with Common Average Reference[END_REF][START_REF] Syam | Comparing Common Average Referencing to Laplacian Referencing in Detecting Imagination and Intention of Movement for Brain Computer Interface[END_REF]. In CAR, the potential at each electrode is measured with respect to the average of all electrodes [START_REF] Syam | Comparing Common Average Referencing to Laplacian Referencing in Detecting Imagination and Intention of Movement for Brain Computer Interface[END_REF]. The CAR defined by the following equation 2.1:

V CAR i = V i - 1 n n j=1 V j , (2.1)
where V i is an amplitude of each site (electrode) and n -number of sites. Bad electrodes should be detected (for example using a threshold on amplitude or impedance) and removed from the processing stage before CAR because in this method each electrode contributes to others.

Laplacian filter

Laplacian filter is obtained by subtracting the sum of the weighted potential of the surrounding electrodes from the considered electrode potential where the weight is electrode distance-dependent [START_REF] Solis-Escalante | Overt foot movement detection in one single Laplacian EEG derivation[END_REF][START_REF] Lu | Adaptive Laplacian filtering for sensorimotor rhythm-based brain-computer interfaces[END_REF].

The small and large Laplacian filters are calculated according to equations 2.2 and 2.3:

V Lap i = V i - n j g ij V j , (2.2)
with

g ij = (d ij n j 1 d ij ) -1 (2.3)
where V is a signal value, Lap means Laplacian, i is an index that corresponds to the considered electrode, j is an index of the nearest neighbor electrodes and n is the total number of electrodes, d ij is the distance between electrodes i and j. Laplacian filters are usually used with electrode layouts that include a large number of electrodes covering the scalp as a grid. Nearest neighbors selected as North, South, East, and West electrodes relative to the considered electrode [START_REF] Solis-Escalante | Overt foot movement detection in one single Laplacian EEG derivation[END_REF].

Filtering in the frequency domain

Filtering in the frequency domain is one of the main steps in signal processing. It helps to separate useful data from noise.

In this section we used EEG signal from the dataset described in detail in section 4.2 to show the examples of applications for different types of filters in BCI domain. For this purpose we used raw EEG signal of right hand motor imagery from channel C3 of subject 2. We used 5th order Butterworth filters of different types with different cutoff frequency ranges. Figure 2.2 represents raw and filtered signals. Filters are divided into four main types according to the frequency response:

• low-pass filter (LPF) has a relatively high transmission rate at frequencies that are below the cutoff frequency, and relatively low at other frequencies. Therefore, such filters keep the value of the spectral components of the input signal at low frequencies. LPF does transmit low-frequency signals and does not transmit highfrequency signals. LPF is used in BCI for downsampling, as a first step to reduce high-frequency signal components before decimation. Also, LPF can be considered as a part of band-pass filtering widely used in the BCI domain. Filtered signal using LPF filter with cutoff frequency 30 Hz showed on Figure 2.2a;

• high-pass filter (HPF) has a high transmission rate at frequencies that are higher than the cutoff frequency, and low transmission rate at frequencies from zero to the cutoff frequency f p . Therefore, they pass without changes part of the spectrum of the input signal, which is at frequencies from the cutoff frequency f p to +∞ and do not pass the components of the signal spectrum from 0 to f p . HPF transmits high-frequency signals.

HPF is used in BCI as a part of a band-pass filter. Filtered signal using HPF filter with cutoff frequency 8 Hz showed on Figure 2.2b;

• band-pass filter (BPF) has a high transmission factor only in a certain frequency band between frequencies f p1 and f p2 . Outside this range, the transmission rate is small. Therefore, only those spectral components of the signal that are in the bandwidth of the filter will pass to the output, and those that lie from zero to f p1 and from f p2 to +∞ will not pass.

BPF is widely used in the BCI domain to obtain frequency ranges that respond to the specific brain rhythms. For example, signals usually filtered between 8-13 Hz to obtain activity only in alpha waves, 13-30 Hz to obtain beta waves, or 8-30 Hz range to obtain alpha+beta waves (see section 1.2.2). The order of the filter is one of the characteristics of the filter and it is equal to the number of poles of the transfer function.

In our studies, we filtered EEG-signals using 5th order Butterworth band-pass filter. The main feature of Butterworth filter [START_REF] Butterworth | On the theory of filter amplifiers[END_REF] is that the frequency response is maximally flat (i.e. has no ripples) in the pass-band and rolls off towards zero in the stop-band [START_REF] Bianchi | Electronic filter simulation and design[END_REF]. Raw and filtered signal shown on Figure 2.2c; • band-stop filter (BSF) is inverse to BPF: it has a large transmission factor outside the frequency range from f p1 to f p2 . The transmission factor is large starting from zero and up to f p1 frequency, as well as from f p2 to +∞. Therefore, when passing a signal through such a filter, the components lying between f p1 and f p2 will be removed from its spectrum, and all other components will pass to the filter output without changes. A Band-stop filter is also known as a notch filter or band-reject filter.

In particular, such filters (called anti-hum filters) are used to eliminate network interference when it is necessary to cut from the signal spectrum only the spectral component in a narrow range of 50 Hz or 60 Hz depending on the electric power system of the country. For countries using 60 Hz power lines: low frequency: 59 Hz, middle frequency: 60 Hz, high frequency: 61 Hz. This means that the filter passes all frequencies, except for the range of 59-61 Hz. This would be used to filter out the mains hum from the 60 Hz power line, though its higher harmonics could still be present. For countries where power transmission is at 50 Hz, the filter would have a 49-51 Hz range. Filtered signal using BSF filter with cutoff frequency (49-51 Hz) showed on Figure 2.2d;

Artifact removing

Signal artifacts contaminate significantly the quality of EEG data. Artifacts mainly can be grouped as environmental noise, subject's mistakes during experiments, and physiological artifacts.

Artifact removing in terms of EEG preprocessing often refers to the eye-blink removing. Furthermore, the environment artifacts and experiment error, which come from external factors, are classified as extrinsic artifacts, whereas the physiological from the body itself (e.g., eye blink, muscle activity, heartbeat) can be categorized as intrinsic artifacts. The environment artifacts can be eliminated by a simple notch filter due to the frequency of such artifacts are inconsistent with desired signals [START_REF] Jiang | Removal of Artifacts from EEG Signals: A Review[END_REF].

Eye movement and eye blink are the main reason for ocular artifacts. They generate a significant number of artifacts in EEG recordings. Electrooculogram (EOG) and/or eye-tracking are usually recorded to track eye movement and help to manage this type of artifact. Ocular artifacts have a low frequency (< 8 Hz), so they could be removed by a band-pass filter.

Muscle artifacts have a much larger frequency range (0 Hz to >200 Hz). They appeared because of subject talks, sniffs, swallows [START_REF] Urigüen | EEG artifact removal -State-of-the-art and guidelines[END_REF] and measured by electromyogram (EMG). Independent Component Analysis (ICA) is reported as a good method to remove this type of artifacts [START_REF] Chen | Removing Muscle Artifacts From EEG Data: Multichannel or Single-Channel Techniques?[END_REF][START_REF] Devuyst | Removal of ECG artifacts from EEG using a modified independent component analysis approach[END_REF].

Another type of artifact is cardiac artifacts. These artifacts usually appear when some electrode is placed over or near the blood vessel. Such artifacts are also called pulse artifacts and their frequency is near 1.2 Hz. Removal of these artifacts could be easier using the reference waveform of electrocardiogram (ECG).

There are a lot of methods used for artifact removal, such as Blind Source Separation, Wavelet transform, Principal Component Analysis, Independent Component Analysis, Canonical Correlation Analysis, Sparse Decomposition Methods, and others (see [START_REF] Jiang | Removal of Artifacts from EEG Signals: A Review[END_REF] for a review).

Feature extraction

Feature extraction is an important part of data preprocessing which has a strong influence on final results. In machine learning feature extraction starts from an initial set of measured variables and creates derived variables (features) intended to be more informative and non-redundant, facilitating the subsequent learning and generalization step.

Feature extraction reduces the number of resources needed to describe a large data set. One of the main problems when performing the analysis of complex data is the number of involved variables. Analysis with a large number of variables in the general case requires a large amount of memory and computing power, and it can lead to overfitting of classification, and poor generalization to new samples. Feature extraction is a general term for methods of constructing such combinations of variables to solve these problems while maintaining sufficient accuracy of the data description.

Not all feature extraction methods reduce dimensionality, some of them increase data dimension to improve class separability, others enhance signal-to-noise ratio without changing dimensionality [START_REF] Lotte | A review of classification algorithms for EEG-based brain-computer interfaces: a 10 year update[END_REF].

As features for classification in BCI domain different statistics of EEG signals, power features of different frequency bands (such as ERS/ERD features), covariance matrices, etc. are usually used. In this section, we will compare different methods for feature extraction in the MI-based BCI domain. We will use a 2-classes problem from the dataset (section 4.2) discriminating right hand MI vs. rest as an example. Electrodes C3 and C4 are considered. For these examples, we will use data recorded from subjects 1 and 2 to show difference not only between different algorithms but also between different subjects. Firstly, let's consider EEG data, filtered in alpha+beta (8-30 Hz) frequency range and averaged over time samples duration of 2 seconds, half a second after the trigger corresponded to the start of MI. Trials representation according to such average amplitude feature extraction technique showed in Figure 2.3.

As it could be seen from Figure 2.3 classes are hardly separable using just average amplitude as a feature for both subjects.

To maximize separation between classes EEG signals could be squared before averaging (it is equal to computing signal power, this will be discussed in detail in section 2.2.2). Trials representation using the mean of signal power showed in Figure 2.4.

We can see, that in contrast to Figure 2.3b in Figure 2.4b, the classes are well separable for subject 2. However, classes for subject 1 (Figure 2.4a) remain heavily separable using this method.

Time-frequency analysis 2.2.1.1 Windowed Fourier transform

Spectral analysis with window functions (Fourier transform with window, window processing, windowing) is commonly used to reduce the presence of unwanted components in the signal spectrum.

To do this, before calculating the Fourier transform, the signal x[n] is multiplied by some weight (window) function w(t), where n is a number of samples in the signal and t is a time. Window function w(t) must fall (usually to zero) at the ends of the segment in which we consider the signal. In the middle of the segment, it must have constant values.

As a result of multiplying the window by the signal, we obtain the area of the signal that is located on the time interval which is placed in the window w(t). This area at the edges is slightly distorted, but in the middle of the interval coincides with the original signal. In addition, the signal values at the end and at the beginning of the interval are equal to each other (zero), which is required to get rid of the Gibbs effect [START_REF] Popov | Theory of signals: a textbook for applicants for a bachelor's degree in the speciality Micro-and nanosystem technology[END_REF].

The expression for the direct discrete windowed Fourier transform in this case will look like:

c w [m] = N -1 n=0 x[n]w[n]e -2πj m N n , (2.4)
where, m = 0, . . . , N -1 and e j2π N is a primitive N th root of 1. Two-dimensional frequency-time representation is often useful for analyzing signals with a variable spectrum. It is necessary to be able to know both the spectral composition and the duration of the presence of components at different frequencies in the signal.

The mathematical expression for the spectral-temporal decomposition of a signal is called the Short-Time Fourier Transform. The calculation of the spectral-temporal representation of the signals is as follows. Initially, the window function w(t) is placed in the neighborhood of the point t = 0. The Fourier spectrum of the product x(t)w(t) is then calculated. Then the window function is shifted by a certain step τ 1 , the function w(t -τ 1 ) is formed and the spectrum product x(t)w(t-τ 1 ) is calculated. Then the procedure is repeated for all possible offsets of the window function. The expressions of the spectral-temporal decomposition of discrete signals have the form [START_REF] Popov | Theory of signals: a textbook for applicants for a bachelor's degree in the speciality Micro-and nanosystem technology[END_REF]:

c w [m, k] = N -1 n=0 x[n]w[n -k]e -2πj m N n , (2.5)
where w(t), w[n] is window function; and τ, k is parameter of window function offset in time.

It transforms an array x[n] with a d-dimensional vector of indices n = (n 1 , n 2 . . . , n d ) by a set of d nested summations (over n i = 0 . . . N i -1 for each i), where the division n N , defined as n N = n1 N1 , . . . , n d N d , is performed element-wise.

Representation of the power distribution of spectral components, which reflects the change in the spectrum over time, is called a spectrogram:

c w [m, k] = N -1 n=0 x[n]w[n -k]e -2πj m N n 2 .
(2.6)

Therefore, it is necessary to apply time-frequency transformations for signal analysis, which allows observing the change of the spectral composition of the signal in time. To perform time-frequency analysis, the window type, window duration, window overlap, and offset step must be specified.

Wavelet transform

A wavelet transform is a tool that cuts up data or functions or operators into different frequency components, and then studies each component with a resolution matched to its scale [START_REF] Daubechies | Ten Lectures on Wavelets[END_REF]. For several decades Wavelet transform has become very popular when it comes to analysis, de-noising, and compression of signals and images [START_REF] Popov | Theory of signals: a textbook for applicants for a bachelor's degree in the speciality Micro-and nanosystem technology[END_REF].

So wavelet analysis is the decomposition of signals into components of different shapes and durations. Like any decomposition, the wavelet decomposition of the analyzed signal x(t) aims to present it as a series -a weighted finite or infinite sum of some components:

x(t) = n c n ψ n (t), (2.7)
where c n the decomposition coefficient, which corresponds to the contribution of the corresponding component in the signal; ψ n function from some set. In the case of wavelets, this set does not have to be an orthogonal basis, but can also be a semi-orthogonal, non-orthogonal basis, or dictionary of functions.

Let scale a ∈ R, a = 0 and offset parameter τ ∈ R, using a rule:

ψ a,τ (t) = a -1 2 ψ t -τ a (2.8)
we can get a set of wavelet functions ψ a,τ (t) that are scalable (stretched or compressed depending on the size of the scaling factor) and shifted by argument (depending on the size τ ) copies of a single generating mother wavelet function ψ(t).

Graphs of some mother wavelets are shown in Figure 2.5. The goal of signal wavelet analysis is to decompose the analyzed signal into the sum of scaled wavelet functions (2.7) [START_REF] Popov | Theory of signals: a textbook for applicants for a bachelor's degree in the speciality Micro-and nanosystem technology[END_REF].

The values of the corresponding decomposition coefficients are defined as the scalar products of the signal for each scaled and shifted wavelet function from the set ψ a,τ (t) . The Continuous Wavelet Transform (CWT) of continuous signal is presented by the following expression:

W ψ (a, τ ) = x(t), ψ a,τ (t) = +∞ -∞ x(t), ψ * a,τ (t)dt = 1 √ a +∞ -∞ x(t), ψ * t -τ a dt, (2.9)
a, τ ∈ R, a = 0, where ψ * a,τ complex-conjugate mother function; W ψ (a, τ ) -wavelet decomposition of the signal x(t).

It is seen that W ψ (a, τ ) is a function of two variables: scale a and shift τ . This function shows the value of the amplitude with which the decomposition 2.7 includes each wavelet for all possible values a and τ . Such a decomposition reflects the function of one variable x(t) ∈ L 2 (R) into the function of two variables W ψ (a, τ ) ∈ L 2 (R × R). Function (2.9) is the distribution of the signal for all possible compressions and displacements of one mother wavelet function [START_REF] Mallat | A Wavelet Tour of Signal Processing: The Sparse Way[END_REF].

In order to reduce the number of coefficients, parameters a and τ of wavelet functions are discretized [START_REF] Daubechies | Ten Lectures on Wavelets[END_REF][START_REF] Mallat | A Wavelet Tour of Signal Processing: The Sparse Way[END_REF]. Parameters are limited, first, only to the positive values of the scale factor a. To discretize the scale parameter a is selected

a = a m 0 , m ∈ Z, a 0 > 0, a 0 = 1, (2.10)
where a 0 is a fixed scaling step. To discretize the parameter of the shift τ of the mother wavelet along the time axis, values that are multiples to some fixed-step τ 0 (τ 0 > 0) are chosen.

The most common case is when a 0 = 2 and τ 0 = 1 so τ = na. In this case, the family of wavelet functions will be obtained by the expression:

ψ m,n (t) = 2 -m 2 ψ 2 -m t -n , m, n ∈ Z. (2.11)
The resulting wavelets are given on the dyadic grid, ie the step of changing the scale values is a power of two, and the time offset is a multiple of two as well. The wavelet transform of the signal for this choice of coefficients is called the dyadic wavelet transform (DWT) [START_REF] Daubechies | Ten Lectures on Wavelets[END_REF][START_REF] Mallat | A Wavelet Tour of Signal Processing: The Sparse Way[END_REF].

In this case, instead of a continuous wavelet transform as a continuous function of two arguments (scale and offset), we have a finite number of values:

D m,n = x(t), ψ m,n (t) = +∞ -∞ x(t)ψ m,n (t)dt (2.12)
Equation (2.12) determine the direct orthogonal dyadic wavelet transform of a continuous signal.

It should be noted that the concept of "discrete" or "dyadic" in the wavelet transform refers only to the method of selecting the scaling factors, which will not now change continuously but should take only certain values. It is possible to speak about both CWT, and about DWT of continuous or discrete signals [START_REF] Popov | Theory of signals: a textbook for applicants for a bachelor's degree in the speciality Micro-and nanosystem technology[END_REF].

If the signal is discrete, then the scaling factors can still take any value to conduct its CWT. The offset can also be arbitrary. But it is necessary to replace the integrals with sums in the expressions for direct and inverse CWT .

To perform DWT of the discrete signal, all of the above is also true, but the scale and offset are dyadic [START_REF] Popov | Theory of signals: a textbook for applicants for a bachelor's degree in the speciality Micro-and nanosystem technology[END_REF].

Band power features

One of the most widely used methods in EEG signal processing is based on extracting band power features. These features represent the average power of a signal in a specific frequency ranges.

This method consists of signal decomposition into functionally different frequency ranges, such as delta (0.5-4 Hz), theta (4-8 Hz), alpha (8-13 Hz), beta (13-30 Hz), and gamma .

This involves the decomposition of the EEG signal into frequency components, which is usually obtained using a Fourier transform. The most popular algorithm to compute the Fourier transform is the Fast Fourier Transform (FFT), which returns, for each frequency bin, a complex number from which amplitude and phase of the signal at that a specific frequency can be extracted. Magnitude-squared of the FFT is used to obtain the power spectral density.

The average band power consists of computing a single number that summarizes the contribution of a given frequency band to the overall power of the signal. This could be particularly useful to summarize a particular aspect of the data.

Event-related synchronization and desynchronization

A common method for feature extraction in BCI is event-related desynchronization/synchronization (ERD/ERS). It is used to evaluate the modulation produced by the tasks. ERD/ERS can be computed using the band power method [START_REF] Pfurtscheller | Event-related EEG/MEG synchronization and desynchronization: basic principles[END_REF]. Using this measure it possible to see decreasing and increasing oscillations in alpha and beta bands which take place in localized regions over sensorimotor cortices. Event-related desynchronization (ERD) can be observed during and even before sensory and cognitive processing and motor behavior of the brain [START_REF] Pfurtscheller | Patterns of cortical activation during planning of voluntary movement[END_REF][START_REF] Klimesch | Event-related desynchronization, erd-mapping and hemispheric differences for words and numbers[END_REF]. The opposite phenomenon event-related synchronization (ERS) is characterized by increasing power in alpha and beta bands accurate after imaginary or real movement. It was described in more detail in the section 1.2.5. The filtered signal is squared for each trial and averaged over trials. Then it is smoothed using a 2-second sliding window with a 150-250 ms shifting step. We have chosen a specific sliding window because the nature of the real and imagined movement, as well as the components ERD/ERS that underline them, require a short window. Finally, the averaged power computed for each window was subtracted and then divided by the averaged power of a baseline corresponding 2 seconds before each trial. This transformation was multiplied by 100 to obtain percentages. This process can be summarized by the following equation:

ERD/ERS% = x 2 -BL 2 BL 2 × 100 , (2.13)
where x 2 is the average of the squared signal over all trials and samples of the studied window, BL 2 is the mean of a baseline segment taken at the beginning of the corresponding trial, and ERD/ERS% is the percentage of the oscillatory power estimated for each step of the sliding window. It is done for all channels separately.

ERD and ERS are difficult to observe from the EEG signal. Indeed, an EEG signal expresses the combination of activities from several neuronal sources. One of the most effective and accurate techniques used to extract events is the average technique [START_REF] Quiroga | Single-trial event-related potentials with wavelet denoising[END_REF]. We decided to use this technique to represent the modulation of the power of the Alpha and Beta rhythms.

Logarithmic ERD/ERS also could be used as feature, it can be presented by the following equation:

ERD/ERS = log (x 2 + 1) (2.14)
We introduce a 3-classes problem using motor imageries of the right hand, left hand, and rest class from the dataset (section 4.2). We have used EEG data for subjects 1 and 2 in this example. Figure 2.6 shows logarithmic ERD/ERS features, computed for EEG channels C3 and C4, which are located above areas of the brain responsible for right and left hand activity respectively.

ERD is observed during and immediately after the motor imagery task and better represented in the alpha band, while ERS is observed after the task and better expressed in the beta band. That's why features filtered in alpha bands (ERD) were calculated during 0.5-6 seconds of each trial, and features filtered in beta bands (ERS) were calculated beginning with 7 and till 12 sec.

Spatial analysis

Spatial analysis is another group of frequently used techniques for feature extraction and dimensional reduction in the MI-based BCI domain. In this section, we will introduce covariance matrices as one of the ways of representing EEG signals. Covariance matrices are used in the common spatial pattern method, its adaptations, and in Riemannian geometry-based methods. We will present methods like common spatial pattern and filter bank common spatial pattern also along this section. These techniques are standard benchmark methods for feature extraction in BCI.

Covariance matrix

A covariance matrix is a matrix made up of pairwise covariances of elements of one or two random vectors. Let x ∈ R n be a multidimensional random vector containing scalar random variables.

x =      x 1 x 2 . . . x n      (2.15)
The covariance matrix C i,j of a random vector x is a square symmetric positive semi-definite matrix, on the diagonal of which the variances of the vector components are located, and the off-diagonal elements in the (i, j) positions are the covariances between the ith and jth elements of x.

C i,j = cov(x i , x j ) = E (x i -µ i )(x j -µ j ) , (2.16)
where

µ i = E[x i ]
is the expected value of the ith element of vector x. Thus the equation 2.16 takes the form:

C =         E (x 1 -µ 1 )(x 1 -µ 1 ) E (x 1 -µ 1 )(x 2 -µ 2 ) • • • E (x 1 -µ 1 )(x n -µ n ) E (x 2 -µ 2 )(x 1 -µ 1 ) E (x 2 -µ 2 )(x 2 -µ 2 ) • • • E (x 2 -µ 2 )(x n -µ n ) . . . . . . . . . . . . E (x n -µ n )(x 1 -µ 1 ) E (x n -µ n )(x 2 -µ 2 ) • • • E (x n -µ n )(x n -µ n )         (2.

17)

As every covariance matrix is positive semi-definite, it means that their eigenvalues are non-negative.

The covariance matrix of a random vector is a multidimensional analog of the variance of a random variable for random vectors. The covariance matrix of two random vectors is a multidimensional analog of the covariance between two random variables.

In the case of a normally distributed random vector, the covariance matrix together with the expected value (average) of this vector completely determine its distribution.

Figures 2.7a and 2.7b represents covariance matrices for two different subjects. For these examples we used EEG data from subject 1 (Figure 2.7a) and subject 2 (Figure 2.7b) from the dataset described in section 4.2. Data for two classes (rest and right hand MI) were used. We have calculated covariance matrices from filtered (8-30 Hz) EEG signals containing 26 channels. From Figure 2.7b we could see that trials from different classes has different structure of covariance matrices for subject 2. But for subject 1 difference is not so notable (Figure 2.7a).

Common Spatial Pattern (CSP)

An algorithm called Common Spatial Pattern (CSP) is the most commonly used technique to extract features from EEG signals in the BCI domain. This algorithm generates a series of spatial filters that applies to decompose multi-dimensional data into a set of uncorrelated components [START_REF] Blankertz | Optimizing spatial filters for robust EEG single-trial analysis [revealing tricks of the trade[END_REF]. These filters aim to extract features that simultaneously maximize the variance of one class while minimizing the variance of the other one [START_REF] Ramoser | Optimal spatial filtering of single trial eeg during imagined hand movement[END_REF][START_REF] Blankertz | Optimizing spatial filters for robust EEG single-trial analysis [revealing tricks of the trade[END_REF].

CSP is a supervised decomposition technique for feature extraction in a two-class decoding problem for multichannel signals. The CSP algorithm considers the whole set of electrodes that are placed during the recording session, which normally includes those covering the sensorimotor cortex. The number of resulting filters equals the number of EEG channels and, in most of the cases, at least two pairs (i.e., four filters) are selected. All trials have to be previously filtered between the selected frequency range. CSP is very useful for motor imagery BCIs (based on oscillatory activity). Usually, the signal is previously filtered in alpha and beta bands before CSP application typically between 8-30 Hz [START_REF] Ramoser | Optimal spatial filtering of single trial eeg during imagined hand movement[END_REF].

First step to compute CSP is to find covariance matrices (see Section 2.2.3.1) of filtered signals X:

C k i = 1 t -1 X k i (X k i ) (2.18)
where C k i is the spatial covariance matrix for a band-pass filtered trial i of signal X k i ∈ R n×t from class k (with samples t as columns and channels n as rows); with representing the transpose. Then means of the covariance matrices C 1 and C 2 are computed by averaging each of these matrices over corresponding trials. Note that, the original paper describes a step for covariance matrix trace normalization to eliminate magnitude variations in the EEG between subjects [START_REF] Koles | Spatial patterns underlying population differences in the background eeg[END_REF]. But it is not applied in more recent works [START_REF] Blankertz | Optimizing spatial filters for robust EEG single-trial analysis [revealing tricks of the trade[END_REF][START_REF] Grosse-Wentrup | Multi-class common spatial patterns and information theoretic feature extraction[END_REF]. Then CSP analysis is given by the simultaneous diagonalization of two covariance matrices. In other words, we should find a single invertible matrix W such that W C k W is a diagonal matrix for every C k . Technically, this can easily be computed by solving the following generalized eigenvalue decomposition problem [START_REF] Blankertz | Optimizing spatial filters for robust EEG single-trial analysis [revealing tricks of the trade[END_REF]:

C 1 W = λC 2 W (2.19)
where C 1 and C 2 are n-by-n covariance matrices, λ is the diagonal matrix of generalized eigenvalues of C 1 and W is a matrix whose columns are eigenvectors. These column vectors of W represent spatial filters, and the number of columns equals the number of channels in the EEG signal. The spatially filtered signal X k CSP can be obtained from the original EEG signal X k as

X k CSP = W X k (2.20)
Note that the CSP filters are paired, so, after being sorted according to the values of λ, only the first m and the last m columns of W are selected. Typically, 6 filters (i.e., 3 pairs) are used corresponding to the 3 largest and 3 lowest eigenvalues. But not all of the achieved filters are relevant for the following classification. Once selected filters W are obtained, a CSP features f for band-pass EEG signal X can be computed as follows [START_REF] Lotte | A tutorial on EEG signal-processing techniques for mental-state recognition in brain-computer interfaces[END_REF]:

f = log(W XX W ) = log(W CW ) = log(var(W X)) (2.21)
CSP requires more channels than spatial filters such as Bipolar or Laplacian, however in practice it usually achieved higher classification performance [START_REF] Ramoser | Optimal spatial filtering of single trial eeg during imagined hand movement[END_REF]. Figure 2.8 shows the application of CSP we have done on the EEG data from the dataset (section 4.2) with 2-class problem (right hand MI vs. rest). The same data from subjects 1 and 2 as considered earlier for Figure 2.3. From Figure 2.8 we could see that rest and right hand MI classes are better separated than before using just average (Figure 2.3) or average of power (Figure 2.4) for both subjects. However, for subject 2 (Figure 2.8b) we can more easily separate both classes than for subject 1 (Figure 2.8a).

The CSP algorithm has numerous advantages: it leads to high classification performance, it is flexible, computationally efficient, and simple to implement. This makes CSP one of the most popular approaches for the oscillatory activity-based BCI [START_REF] Blankertz | Optimizing spatial filters for robust EEG single-trial analysis [revealing tricks of the trade[END_REF][START_REF] Lotte | A tutorial on EEG signal-processing techniques for mental-state recognition in brain-computer interfaces[END_REF]. Though despite all these advantages CSP has flaws and limitations. Notably it has been shown that CSP is non-robust to noise, to non-stationarities and prone to over-fitting when small training set is available [START_REF] Grosse-Wentrup | Multi-class common spatial patterns and information theoretic feature extraction[END_REF][START_REF] Grosse-Wentrup | Beamforming in noninvasive brain-computer interfaces[END_REF][START_REF] Reuderink | Robustness of the Common Spatial Patterns algorithm in the BCI-pipeline[END_REF]. Also, CSP identifies only the important spatial information but does not consider the spectral one. Moreover, the CSP algorithm has a limitation of classes number. It is applicable only to discriminate between two classes. That's why this algorithm can not be used to solve multi-class problems. Though there are some multi-class extensions [START_REF] Grosse-Wentrup | Multi-class common spatial patterns and information theoretic feature extraction[END_REF].

CSP algorithm requires parameters to setups, such as frequency range for band-pass filtering, the time interval of the signal trial to use for decomposition, and the number of CSP filters to use. Most commonly for motor intention detection tasks in BCI domain frequency range that includes both alpha and beta bands (8-30 Hz) is used [START_REF] Ramoser | Optimal spatial filtering of single trial eeg during imagined hand movement[END_REF]. Time segment usually is taken after 0.5-1 second after the cue, and 2-3 pairs of filters (4-6 filters) are taken into account [START_REF] Wang | Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface[END_REF].

Filter Bank Common Spatial Pattern (FBCSP)

The goal of the Filter Bank Common Spatial Pattern (FBCSP) algorithm is to improve the standard CSP method taken into account specific parameters and profiles of the subjects. This is achieved by automatic selection of the most discriminative subject-specific frequency range for band-pass filtering [START_REF] Ang | Filter bank common spatial pattern algorithm on bci competition iv datasets 2a and 2b[END_REF].

The FBCSP algorithm includes 4 different stages:

1. Band-Pass Filtering.

In this stage, a filter bank is used to decompose EEG signals into different frequency bands. According to the original algorithm [START_REF] Ang | Filter bank common spatial pattern algorithm on bci competition iv datasets 2a and 2b[END_REF] nine 4 Hz wide band-pass filters are used: 4-8, 8-12, ..., 36-40 Hz. These filters totally cover the frequency range from 4 to 40 Hz which corresponds to theta, alpha, beta, and low delta brain rhythms. The number and range of band-pass filters could be changed for a specific task.

2. Spatial Filtering.

On the second stage CSP algorithm is used to employ spatial filtering. CSP filters apply to the signals filtered in the 9 frequency ranges obtained in the previous step. Thus the equation 2.20 takes the form:

X F BCSP b,i = W b X b,i , (2.22)
where X b,i is an i-th trial of the EEG signal filtered in the b-th frequency band. X F BCSP b,i ∈ R n×t denotes X b,i after spatial filtering, W b ∈ R n×n is a CSP filters coefficient matrix, where n is a number of EEG channels, t is a number of time samples per channel and denotes transpose operator.

Then transformation matrix W b with optimal coefficients for discriminating 2 classes of EEG data is computed by solving eigenvalue decomposition problem (equation 2.19) as showed in section 2.2.3.2.

The FBCSP feature vector for the i-th trial is then formed as follows:

v i = [v 1,i , v 2,i , ..., v 9,i ], (2.23)
where v i ∈ R 1×(9 * 2m) , i = 1, 2, ..., n; n denotes the total number of trials in the data; m is number of CSP filters pairs which corresponds to the first m and the last m columns of W b .

3. Feature selection.

On the third stage a feature selection technique applied to select discriminative CSP features from equation 2.23. Different feature selection algorithms can be used in this step.

Classification.

In the final stage classification algorithm is applied to the selected CSP features from the previous step.

Here there are also many choices, such as linear discriminant analysis algorithm or support vector machine classifier. Both methods will be represented later in this chapter.

Riemannian geometry

As covariance matrices are symmetric positive-definite, this means that they belong to a Riemannian manifold from where their classification can be directly reached. Square real symmetric positive-definite (SPD) matrices have the characteristic of being always diagonalizable with strictly real positive eigenvalues, whose space represents a differentiable Riemannian manifold M . There are several advantages of using this manifold that can be easily applied in feature extraction and classification [START_REF] Barachant | Multiclass Brain-Computer Interface Classification by Riemannian Geometry[END_REF].

In differential geometry, a N -dimensional smooth manifold is a topological space that is locally similar to the Euclidean space and has a globally defined differential structure. A smooth Riemannian manifold (or Riemannian space) M is a real smooth manifold equipped with an inner product on the tangent space T Ω M defined at each point Ω that varies smoothly from point to point. The tangent space T Ω M at point Ω is the Euclidean vector space containing the tangent vectors to all curves on M passing through Ω (Figure 2.11) [START_REF] Congedo | Approximate Joint Diagonalization and Geometric Mean of Symmetric Positive Definite Matrices[END_REF].

Riemannian geometry has a specific measure of the distance between positive semi-definite matrices. This measure will be used to find the closest distance between the covariance matrix describing the window under study and the averaged covariance matrix of each class, e.g., rest and motor imagery.

Riemannian distance

The minimum length curve connecting two points in the manifold M called the geodesic γ(t) and the distance of this curve is Riemannian distance. It is one of the basic metrics in the Riemannian Geometry, also called Affine Invariant Riemannian Metric (AIRM), and it computes as follows:

d R (C 1 , C 2 ) = ||log C -1 1 C 2 || F = ||log C -1 2 1 C 2 C -1 2 1 || F , (2.24)
where C i are covariance matrices (2.18) distance between which is computed, λ i are eigenvalues of C -1 1 C 2 and F denotes Frechet norm [START_REF] Congedo | Approximate Joint Diagonalization and Geometric Mean of Symmetric Positive Definite Matrices[END_REF][START_REF] Barachant | Riemannian geometry applied to BCI classification[END_REF].

Riemannian centre of mass

To be able to compare distances between particular trials and certain classes we can compute Riemannian centre of mass for each class in Riemannian space. Riemannian (Frechet) mean or Riemannian centre of mass is computed using equation 2.25. Let's say it will be point Ω (Figure 2.11) and SPD covariance matrix in this point is denoted as C.

C = arg min C ( 1 I I i=1 d 2 R (C, C i )), (2.25)
where I is a number of trials of the observed class. Figure 2.9 shows centers of mass for two classes (rest and right hand MI) of subject 2. We have considered 26-channel EEG data from the dataset described in section 4.2.

Centers of mass for the same subject but only for two EEG channels (C3 and C4) represented in Figure 2.10.

In both cases, it is clear that each class yields a different structure of the covariance matrix.

Tangent space

To be able to use linear classifiers such as LDA or SVM it is necessary to have a feature vector. The next algorithm shows how to get a feature vector from Riemannian space projecting covariances matrices on the tangent space of the current covariance matrix. Consider a point Ω on M and construct the tangent space T Ω M on it. Now take a tangent vector V departing from Ω, which is our reference point. There exists one and only one geodesic on the manifold starting at Ω that corresponds to V ; think of rolling the plane (tangent space) on the surface (manifold) in such a way that the vector always touches the surface. The endpoint on M is Φ. We see that the geodesics on The Manifold and the tangent space at a point. Consider a point Ω on M and construct the tangent space TΩM on it. Now take a tangent vector V departing from Ω, which is our reference point. There exists one and only one geodesic on the manifold starting at Ω that corresponds to V ; think of rolling the plane (tangent space) on the surface (manifold) in such a way that the vector always touches the surface. The endpoint on M is Φ. We see that the geodesics on M through Ω are transformed into straight lines and the distances along all geodesics are preserved (from [START_REF] Congedo | Approximate Joint Diagonalization and Geometric Mean of Symmetric Positive Definite Matrices[END_REF][START_REF] Goh | Unsupervised Riemannian Clustering of Probability Density Functions[END_REF]).

M through Ω are transformed into straight lines and the distances along all geodesics are preserved (this is true in the neighborhood of Ω) [START_REF] Congedo | Approximate Joint Diagonalization and Geometric Mean of Symmetric Positive Definite Matrices[END_REF][START_REF] Goh | Unsupervised Riemannian Clustering of Probability Density Functions[END_REF].

In this case the shortest curve between two points Ω and Φ on M is called the geodesic γ(Ω → Φ) and computes as follows:

γ(Ω → Φ) = Ω 1 2 (Ω -1 2 ΦΩ -1 2 ) β Ω 1 2 , β ∈ [0, 1], (2.26)
where β is the arc length parameter. When β = 0 we are at Ω, when β = 1 we are at Φ and when β = 0.5 we are at the geometric mean of the two points [START_REF] Congedo | Approximate Joint Diagonalization and Geometric Mean of Symmetric Positive Definite Matrices[END_REF].

The exponential and logarithmic maps

The exponential and logarithmic maps are shown graphically in Figure 2.11. The function that maps a vector V ∈ T Ω M to the point Φ ∈ M following the geodesic starting at Ω, is named the exponential map and denoted by Φ = Emap Ω (V ) [START_REF] Congedo | Approximate Joint Diagonalization and Geometric Mean of Symmetric Positive Definite Matrices[END_REF]. It is defined as:

Φ = Emap Ω (V ) = Ω 1 2 exp(Ω -1 2 V Ω -1 2 )Ω 1 2
(2.27)

The inverse operation is the function mapping the geodesic relying Ω to Φ back to the tangent vector V ∈ T Ω M . It is named the logarithmic map and denoted V = Lmap Ω (Φ) [START_REF] Congedo | Approximate Joint Diagonalization and Geometric Mean of Symmetric Positive Definite Matrices[END_REF]. It is defined as:

V = Lmap Ω (Φ) = Ω 1 2 ln(Ω -1 2 ΦΩ -1 2 )Ω 1 2
(2.28) So each covariance matrix (trial) could be projected to tangent space at some point Ω using equation 2.29. Then S i can be vectorized to turns it into Euclidean R n space where the basis is orthonormal. This transformation also preserves the distances between the computed center of mass C (identity on the tangent space) and each covariance matrix C i (but not the distances between covariance matrices). After finishing these manipulations linear classifier (as LDA, LR or SVM) can be applied (Figure 2.12). Vectorized S i can be used as an input to these classifiers.

Feature selection

So taking into account equation 2.28, tangent space projection of C i at C will be defined as:

S i = C 1 2 ln(C -1 2 C i C -1 2 )C 1 2 (2.29)
And according to equation 2.27 to projecting S i back to the manifold next equation should be used:

C i = C 1 2 exp(C -1 2 S i C -1 2 )C 1 2
(2.30)

Using equation 2.29 trials and centers of mass (represented by SPD matrices) could be projected on the tangent space.

Figure 2.13 shows trials and centers of mass for 2-channel EEG data (previously introduced in Figure 2.10) represented in 3-dimensional tangent space. We used EEG data from 2 channels (C3 and C4) of subject 2 from the dataset described in section 4.2. We have previously filtered signals in alpha+beta (8-30 Hz) frequency range. Now we could apply linear classifiers in this space to separate classes.

Feature selection

When the input of the algorithm is too large to be processed and is suspected to be redundant (for example, the same measurements in meters and feet, or repeatability in images represented by pixels), then they can be converted to a shortened set of features (also called a feature vector). Determining a subset of initial features is called feature selection. It is expected that the selected features contain relevant information from the input data so that the desired task can be performed using this reduced representation instead of the complete primary data.

Feature selection, also known as variable selection, attribute selection, or variable subset selection, is the process of selecting a subset of relevant features for use in model construction. Feature selection techniques are used for four reasons:

• simplification of models to make them easier to interpret [START_REF] Gareth James | Statistical learning[END_REF];

• shorter training times;

• to avoid the curse of dimensionality; • enhanced generalization by reducing overfitting [START_REF] Bermingham | Application of high-dimensional feature selection: evaluation for genomic prediction in man[END_REF] (formally, reduction of variance)

In the thesis we used the method which accepts a threshold parameter and selects the features whose importance (defined by the coefficients) are above this threshold. For this purpose, we select a meta-transformer SelectFrom-Model, realized in Python library Scikit-learn for feature selection in section 5.2.3. This algorithm could be used along with any estimator that has a coefficients or feature importance's attribute after fitting. The features are considered unimportant and removed if the corresponding coefficients or feature importance's values are below the provided threshold parameter.

In section 5.2.3 we selected a threshold value equals to 1e -5. A threshold used to demarcate features: it kept features whose importance is greater or equal compared to the threshold while the others are discarded.

In the thesis, we use linear classifiers with L1 penalized parameter, so many estimated coefficients equal to zero. In this case, SelectFromModel selects the non-zero coefficients to reduce the dimensionality of the data.

We will apply feature selection to investigate variability between bands in section 5.2.3. It can help to understand which features (bands) are more useful for the classification.

Outlier detection

Outlier or anomaly detection is the identification of elements that do not match the expected other elements or patterns of the dataset. This technique widely used in detecting fraud among banking operations, text errors, structural defects, and medical problems [START_REF] Aggarwal | Outlier Analysis[END_REF].

Four outlier detection methods were investigated in this work: local outlier factor [START_REF] Breunig | LOF: identifying density-based local outliers[END_REF], Isolation forest [START_REF] Liu | Isolation Forest[END_REF], one-class SVM [START_REF] Schölkopf | Estimating the Support of a High-Dimensional Distribution[END_REF], and elliptic envelope method (or robust covariance) [START_REF] Rousseeuw | A Fast Algorithm for the Minimum Covariance Determinant Estimator[END_REF]. Figure 2.14 presents the application of these methods on ERD/ERS features extracted from the right hand MI class. For the illustrative purpose in this section, we computed results only on EEG signals for subject 2 from the dataset described in section 4.1.

One common way to detect outliers is to assume that the data comes from a known distribution (e.g., a Gaussian distribution). From this assumption, the shape of the data distribution density is determined, and remote observations that go far beyond the shape of the distribution density are defined as outliers.

Local Outlier Factor (LOF) is an unsupervised algorithm for outlier detection [START_REF] Breunig | LOF: identifying density-based local outliers[END_REF]. It assigns some score to each observed sample. This score measures the local deviation of the density of a given sample with a respect to its neighbors and is called the Local Outlier Factor. K-nearest neighbors estimate the local density and determine the degree of isolation of an object from its neighborhood. Then LOF compares the local densities of a sample and its neighbors detecting samples that have a significantly lower density than others. These samples are considered outliers.

The elliptical envelope method (or robust covariance) [START_REF] Rousseeuw | A Fast Algorithm for the Minimum Covariance Determinant Estimator[END_REF] determines the covariance estimate of the data so that the central data points are in the ellipse, ignoring points (outliers) outside the ellipse (see Figure 2.14.d).

One-class implementation of the Support Vector Machine (SVM) method, on the contrary, works better when the data distribution is not strongly Gaussian because this algorithm can closely approximate the distribution of the data set. The one-class SVM method detects the soft boundary of a given data set and classifies whether new observations belong inside the boundary or not [START_REF] Schölkopf | Estimating the Support of a High-Dimensional Distribution[END_REF]. To determine the boundary, it is needed to select the core and the parameter responsible for the probable saturation of the data with outliers. A RBF (radial basis function) kernel can be used (see Figure 2.14.c).

The Isolation Forest (IF) algorithm [START_REF] Liu | Isolation Forest[END_REF] distinguishes (isolates) observations by randomly selecting a feature and a split value between the maximum and minimum values of this feature. As recursive separation can be represented as a tree, the number of splits required to allocate observations is equivalent to the path length from the root node to the end node. Then the estimation is calculated as the path length to isolate the observation. The length of this path, averaged over all such random trees, determines the decision function. Random separation creates noticeably shorter paths for anomalies. Thus, when the forest of random trees often reveals shorter path lengths for the same observations, they are probably anomalies (see Figure 2.14.b). There are few parameters to set the Isolation Forest algorithm such as the number of base estimators in the ensemble and maximum features parameter, which defines the number of features to draw from the data to train each base estimator. Also, the contamination parameter defines the proportion of outliers in the dataset as well as the saturation parameter for One-class SVM.

Isolation Forest with the number of trees equal to 100 and the maximum number of features to train each base estimator chosen equal to five.

Improving the model performance using outlier detection as quality index of imaginary movements estimation

To improve BCI training and classification performance we propose to detect outliers in the data and exclude them before training.

For each class, outliers are detected in the training set and removed using different saturation parameter. Saturation is a parameter responsible for the percentage of data that will be selected as outliers. Outlier detection is based on one of the clustering algorithms presented in section 2.4.

After outlier detection logistic regression is used as a classification method. For each outlier saturation parameter, classification performance is computed. The parameter with the maximum classification accuracy is selected. This is followed by the learning of outlier detectors (clustering models) with the best parameter selected at the previous step for each class. We trained outlier detectors on the training set, saved them and then used for cleaning the test set.

Since the true class of the test data is unknown, it is not known which of the anomaly detectors to use. Therefore, we have proposed to use the method of logical disjunction: for an observation to be marked as an anomaly, it must be marked as an anomaly by each outlier detector. After removing the anomalies, classifiers predict classes on the cleared test set.

Elliptical envelope and local outlier factor methods were prone to overfitting and showed very bad generalization on our high dimensional data. As a result, the models trained with these methods removed about 90% of the data from the test set, considering them as outliers. Therefore in section 5.1, we will present results obtained using Isolation Forest and One-class SVM algorithms because these methods obtained the best performances on our datasets.

Classification

Classification is an instance of supervised learning. It is the problem of identifying to which of a set of categories a new observation belongs, on the basis of a training set of data containing observations whose category membership is known.

Many different classification methods exists (see Figure 2.15). Some of classification techniques that were used for our problems will be described below. Here again, for illustrative purpose, we will consider EEG signals for subjects 1 and 2 from the dataset (section 4.2). We used data for three different classes: right hand MI, left hand MI and rest. We filtered signals previously in the alpha+beta (8-30 Hz) frequency range and ERD/ERS features from two channels C3 and C4 were computed.

From Figure 2.15a we could see that all presented classifiers straggle separating 3 classes for subject 1 having average classification accuracy around 50-55 %. At the same time, for subject 2, classes are well separated by many classification algorithms (Figure 2.15b) having classification accuracy around 82-88 %. The best classifiers are linear ones, such as linear discriminant analysis and logistic regression, support vector machines with RBF kernel, and neural networks. However, low results of classification for subject 1 and the huge difference between results for two subjects shows that another approach should be found to improve both feature extraction and classification stages. For this purpose method based on parallel connection of binary classifiers will be presented (section 2.5.6) together with classic machine learning (sections 2.5.1-2.5.4) and Riemannian geometry approaches (section 2.2.4.3, 2.5.5) along this section. Moreover, in chapter 3, we will consider deep learning methods such as convolutional neural networks for direct feature extraction and classification from raw EEG signals.

Linear discriminant analysis

Linear discriminant analysis (LDA) is a generalization of Fisher's linear discriminant, a method used in statistics, pattern recognition, and machine learning to find a linear combination of features that characterizes or separates two or more classes of objects or events. The resulting combination may be used as a linear classifier, or, more commonly, for dimensionality reduction before later classification.

LDA is a classifier with a linear decision boundary, generated by fitting class conditional densities to the data and using Bayes' rule. The model fits a Gaussian density to each class, assuming that all classes share the same covariance matrix.

In this thesis, we also used shrinkage (or regularized) linear discriminant analysis (sLDA) [START_REF] Friedman | Regularized discriminant analysis[END_REF] which is a version of LDA with a regularisation parameter. This strategy commonly used when it is deals with small sample data set.

For illustrative purpose, we consider EEG signals for subjects 1 and 2 from the dataset (section 4.2). Decision boundary of LDA classifier is represented on the Figure 2.16 for 3 classes: right hand MI, left hand MI, and rest. Classification decision based on 2 ERD/ERS features from channels C3 and C4 computed as described earlier in section 2.2.2.1

Support vector machine

Support vector machine (SVM) is a supervised learning model with associated learning algorithms that analyze data used for classification and regression analysis. Given a set of training examples, each marked as belonging to one or the other of two categories, an SVM training algorithm builds a model that assigns new examples to one category or the other, making it a binary linear classifier. An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on which side of the gap they fall.

In addition to performing linear classification, SVMs can efficiently perform a non-linear classification using what is called the kernel trick, implicitly mapping their inputs into high-dimensional feature spaces.

The kernel function can be any of the following:

• Linear;

• Polynomial;

• Radial Basis Function (RBF);

• Sigmoid. One of the main parameters of SVMs is a regularization parameter C. This parameter controls slack variables. In this thesis we choose C equals 0.01.

In the thesis, we used linear, polynomial, and RBF kernels. SVM classifiers with different kernels (linear, RBF and polynomial (3rd degree)) are represented on Figure 2.17. For illustrative purpose, we consider EEG signals for subjects 1 and 2 from the dataset (section 4.2). Classification decision based on ERD/ERS features obtained from two channels C3 and C4, acquired above motor cortex as described in section 2.2.2.1.

Decision tree classifier

Decision tree learning [START_REF] Quinlan | Induction of decision trees[END_REF] uses a decision tree (as a predictive model) to go from observations about an item (represented in the branches) to conclusions about the item's target value (represented in the leaves). It is one of the predictive modeling approaches used in statistics and machine learning. Tree models where the target variable can take a discrete set of values are called classification trees; in these tree structures, leaves represent class labels and branches represent conjunctions of features that lead to those class labels.

For illustrative purpose, we consider EEG signals for subjects 1 and 2 from the dataset (section 4.2). Example of decision boundary for decision tree classifier for 3 class problem (right hand MI, left hand MI, and rest) is represented on Figure 2.18. Classification decision based on ERD/ERS features obtained from two channels C3 and C4, acquired above motor cortex as described in section 2.2.2.1.

Gradient Boosting

Gradient Boosting (GB) is a supervised ML method for regression and classification problems that creates a prediction model as an ensemble of "weak learners". Weak learners are simple, typically linear prediction models. For gradient boosting usually used decision tree prediction models. GB algorithm builds the model in stages like other boosting methods and generalizes them to optimize an arbitrary differentiable loss function [START_REF] Friedman | Greedy Function Approximation: A Gradient Boosting Machine[END_REF][START_REF] Mason | Boosting Algorithms as Gradient Descent in Function Space[END_REF].

For illustrative purpose, we consider EEG signals for subjects 1 and 2 from the dataset (section 4.2). Example of decision boundary for gradient boosting classifier represented on Figure 2.19 for 3 classes: right hand MI, left hand MI and rest. Classification decision based on ERD/ERS features obtained from two channels C3 and C4, acquired above motor cortex as described in section 2.2.2.1.

The Minimum Distance to Riemannian Mean Algorithm

As was mention in section 2.2.4 covariance matrices of EEG signals belongs to the Riemannian manifold. So one of the approaches is to use distance-based classification directly in the Riemannian manifold without projecting it to tangent space and turning to the Euclidean space [START_REF] Barachant | Riemannian geometry applied to BCI classification[END_REF][START_REF] Barachant | Multiclass Brain-Computer Interface Classification by Riemannian Geometry[END_REF] (as was described in the section 2.2.4.1).

This method represented on the Figure 2.20. At first it is necessary to compute the Riemannian Center of Mass of each class [START_REF] Fréchet | Les éléments aléatoires de nature quelconque dans un espace distancié[END_REF]: Both hands and feet

C k = arg min C ( 1
I I i=1 d 2 R (C, C k i )) (2.31)
Than for the new observations distances to each centers of mass are computed. The minimum distance defines the classification result:

class = arg min k d R (C k , C new ) (2.32)

Proposed solution: a parallel connection of binary classifiers

As usual binary classifier can distinguish only two classes, it is not possible to use them for multi-class tasks. But in the task with combined movements, we proposed a solution of 3 binary classifiers that were connected in a parallel way.

Initially, we had a dataset with 8 classes (7 imaginary movements and rest). There are left hand, right hand, feet, and combinations of these movements: left hand and right hand, left hand and feet, right hand and feet, both hands and feet.

So each of 3 binary classifiers predict presence of only one motor imagery (right hand / left hand / feet) in the signal. In the end of the classification we have combination of 3 states (True or False). Then to decode this we used following equation 2.33:

P redictedLabel = 2 0 × pred 1 + 2 1 × pred 2 + 2 2 × pred 3 , (2.33)
where pred 1 -predicted label ('0' or '1') of the first classifier, which corresponds to the left hand activity; pred 2 -prediction of the right hand activity; pred 3 -prediction of the feet activity.

The code for each of the combined movements is presented in Table 2.1.

Cross-validation

Cross-validation (CV) is a model validation technique for estimating how the results of a statistical analysis will generalize to an independent data set. It is mainly used in settings where the goal is to predict, and it is necessary to assess how accurately the predictive model will work in practice. In the prediction task, models typically provide a set of known data on which training is done (a training data set) and a set of unknown data on which the model is tested (called validation data set or test data set).

The purpose of cross-validation is to test the model's ability to predict new data that was not used in its definition to find problems such as overfitting and selection bias and to give an idea of how this model will be generalized to unknown data for a real problem.

One round of cross-validation involves partitioning a sample of data into complementary subsets, performing an analysis on one of the subsets (called a training set), and validating the results on another subset (called validation or test set). To reduce variability, most methods perform several rounds of cross-validation using different partitions. Then results are averaged over the CV rounds to assess the predicted performance of the model.

Testing the quality of machine learning models should be carried out on examples that did not participate in its learning. The number of test samples should be the greater, the higher the quality of training is. If model errors have a probability close to one billion, then a billion test examples are needed to confirm this probability. It turns out that testing well-trained models becomes a very difficult task.

Cross-validation strategies that were used in this work will be reported later in chapter 4.

Statistical analysis 2.7.1 Wilcoxon signed-rank test

We performed a Wilcoxon signed-rank test (two-sided) to find the significant difference in accuracy obtained by different models. We report different significance levels (p -value < 0.05, p -value < 0.01, p -value < 0.001) in the plots.

The Wilcoxon signed-rank test is a non-parametric statistical hypothesis test used to compare the differences between two samples of paired or independent measurements in terms of the level of any quantitative characteristic, measured on a continuous or ordinal scale [START_REF] Wilcoxon | Individual Comparisons by Ranking Methods[END_REF].

The method consists of the comparison of absolute values of expression shifts in one direction or another. To do this, first, all the absolute values of the shifts are ranked, and then the ranks are summed up. If shifts in one direction or another occur randomly, then the sums of their ranks will be approximately equal. If the shifting intensity in one direction is greater, then the sum of the ranks of the absolute values of shifts in the opposite direction will be significantly lower than it could be with random changes.

One of the main advantages of this test is that the Wilcoxon test can be confidently used with a sample size of up to 25 elements 1 2 . This is because, with a larger number of observations, the distribution of the values of this criterion is rapidly approaching normal. Therefore, in the case of large samples, it is advisable to transform the Wilcoxon test into a z-score [START_REF] Siegel | Nonparametric Statistics for the Behavioral Sciences[END_REF].

Topographies

To analyze the differences between conditions, we performed an event-related spectral perturbation (ERSP) analysis for different frequency bands using EEGLAB.

Brain topography allowed us to display the possible changes over different electrodes on the scalp to localize which part of the brain was involved when the subject performed the requested task. In particular, it allowed us to understand how different conditions can be discriminated and which time parameters we can choose to guide the classification.

A surrogate permutation test (4,000 permutations) from the EEGLAB toolbox was used to validate differences in terms of localization of these ERSPs.

In addition to this analysis, we applied a false discovery rate (FDR) correction test to clarify how the FDR was controlled for multiple comparisons. This test consists of repetitively shuffling values between conditions and recomputing the measure of interest using the shuffled data. It was performed by drawing data samples without replacement and is considered suitable to show the difference between different conditions [START_REF] Manly | Randomization, Bootstrap and Monte Carlo Methods in Biology, ch. The Generation of Random Permutations[END_REF]. Figure 2.21 shows a topographic representation for subject 2 from the dataset described in section 4.2 along with all conditions of the ERD/ERS% values obtained for all electrodes and averaged over the 4-second period starting one second after the task cue was presented. It can be observed that the lowest values appear on the regions associated with the limbs that are engaged at each motor task (broadly, right hand over electrode C3, feet over electrode C z , and left hand over electrode C4).

Conclusion

MI-based BCIs is a domain that requires improvements and a lot of researchers working on new algorithms and methods for feature extraction and classification. In this chapter, we have presented algorithms frequently used in preprocessing, feature extraction, and classification modules of BCI. We have introduced time-frequency analysis, band power features, spatial analysis, and Riemannian geometry-based methods for feature extraction in MI-based BCI. Nevertheless common spatial pattern (CSP) algorithm is still the most popular method for feature extraction from EEG signal. The CSP algorithm finds spatial filters to maximize the variance of the filtered signal for one class and minimize it for another class at the same time. It has numerous advantages: it leads to high classification performance, it is flexible, computationally efficient, and simple to implement. Though despite all advantages CSP algorithm has limitations: it identifies only the important spatial information, but not the spectral one; frequency bands should be selected manually. To solve this issue, a filter bank common spatial pattern (FBCSP) algorithm has been proposed . The FBCSP algorithm uses several small different frequency bands (usually 4 Hz length) in parallel, which led to better classification performance. Other common approaches for the extraction of useful information from EEG data are algorithms based on Riemannian geometry (such as tangent space (TS) algorithm and minimum distance to Riemannian mean (MDRM) algorithm . This approaches usually outperforms the standard FBCSP+LDA method on classification sensory-motor rhythm (SMR) data.

We have introduced classical machine learning methods such as linear discriminant analysis (LDA), support vector machines (SVMs), gradient boosting, and decision tree algorithms that have been commonly used in MIbased BCIs for the classification task. Also, we have proposed a method based on three parallel-connected binary classifiers for the detection of seven MI combined movements and rest. Results of this proposed solution will be represented in section 5.2.2.

In this chapter, we also presented feature selection and outlier detection techniques that could be used in MIbased BCIs. We proposed a method based on outlier detection to improve classification performance and estimate the quality of imaginary movements before training the BCI.

In further chapter 3 we will present deep learning methods, such as convolutional neural networks for the direct feature extraction from raw EEG signals with the following classification.

Then, in chapter 4, we will present datasets with motor imagery data that were used in our investigations. Classification performance results of algorithms represented in this chapter together with their comparison will be introduced in chapter 5. Artificial neural networks (ANNs) are machine learning methods, inspired by neural networks in the biological brain, based on an oriented graph where nodes are artificial neurons and oriented arcs are artificial connections [START_REF] Bishop | Pattern Recognition and Machine Learning[END_REF]. Each connection between artificial neurons can transmit a signal (or value) from one neuron to another. The biological synaptic modulation is reproduced by a connection weight. The artificial neuron that receives the values from some other neurons via the weighted connection can process them and then transfer an output value to the following artificial neurons via a transfer function (or activation function) [START_REF] Hassoun | Fundamentals of Artificial Neural Networks[END_REF]. Weights and functions that calculate arousal can be learned from samples [START_REF] Zell | SNNS (Stuttgart Neural Network Simulator)[END_REF]. The output of each artificial neuron is calculated by a nonlinear function of the sum of its inputs. Artificial neurons and connections usually have a weight that adjusts by training. Weight increases or decreases the signal strength of the connection. Artificial neurons are usually arranged in layers. Different layers can perform different types of transformations of their inputs. The signals pass from the first (input) to the last (output) layer, possibly after passing through the layers several times [START_REF] Bishop | Pattern Recognition and Machine Learning[END_REF][START_REF] Hagan | Neural Network Design[END_REF][START_REF] Livingstone | Artificial Neural Networks: Methods and Applications[END_REF].

ANNs have been used in a variety of domains including computer vision, speech recognition, machine translation, natural language processing, and medical diagnosis [START_REF] Rosa | Artificial Neural Networks -Models and Applications[END_REF].

A deep neural network (DNN) is an artificial neural network that organizes neurons in multiple layers between the input and output layers [START_REF] Bengio | Learning Deep Architectures for AI[END_REF]. Additional layers enable the composition of features from first layers, potentially modeling complex data with fewer units than a similar ANN.

ANNs can be used for classification, segmentation and other tasks presented in section 2.5. But due to the complex nature of the models, numerous variants, and the fact that ANN can be used as a universal approximator of functions performing both processing and classification steps on the raw data forced us to devote a separate chapter to deep neural networks.

There are many types of artificial neural networks such as Feedforward Neural Networks, Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM) cells, Deep Belief Networks (DBNs). All of these artificial neural networks are different in their architecture and intended tasks (e.g. classification, object detection or segmentation).

ANN architecture design is determined depending on the problem being solved. But a lot of ANN architectures could not be used (or their application is not efficient) to properly solve BCI problems due to their complex structure and a small amount of data usually available for training in this domain. In the following sections, only the types of ANN which were used in our study will be reviewed.

Basics of artificial neural networks

There are many neural networks but they always consist of the same components: neurons, connections, weights, biases, and functions.

A neuron j at time t, receiving the input p j (t) from preceding neurons, consists of the following components [START_REF] Zell | SNNS (Stuttgart Neural Network Simulator)[END_REF]:

• activation a j (t), which depends on the discrete time parameter. Activation is a weighted sum of all the inputs, weighted by the weights of the connections from the inputs to the neuron;

• optional bias b j , which learnt by the learning function;

• activation function f (usually nonlinear), which calculates the new activation at a given time t+1 with a j (t), b j and network input p j (t), resulting in the relation:

a j (t + 1) = f (a j (t), p j (t), b j ); (3.1)
• and output function f out , which calculates the activation output for the last layer

y j (t) = f out (a j (t)). (3.2)
The input neuron has no predecessor but serves as an input interface for the entire network. The initial inputs are external data, such as image and signal values. Similarly, an output neuron has no successor and therefore serves as an output interface for the entire network [START_REF] Zell | SNNS (Stuttgart Neural Network Simulator)[END_REF]. The outputs forecast the value of the dependent variable accomplishing the task, such as classification or object recognition in an image.

In artificial neural networks, the activation function determines the output signal, which is determined by the input signal or a set of input signals. However, only nonlinear activation functions allow such networks to solve non-trivial problems using a small number of nodes. In artificial neural networks, this function is also called transfer function [START_REF] Hagan | Neural Network Design[END_REF].

In the case of using linear or binary functions, it is needed to use many neurons for computations. In addition, networks built using such activation functions have unstable convergence, since the excitation of priority inputs of neurons tends to an unlimited increase since this function is not normalizable [START_REF] Livingstone | Artificial Neural Networks: Methods and Applications[END_REF][START_REF] Hagan | Neural Network Design[END_REF]. Problems mentioned above can be solved with the nonlinear activation functions (for example Sigmoid function).

There are many activation functions that are used for ANNs. Most common activation functions are: Logistic (Sigmoid) function, Hyperbolic tangent (tanh), Rectified Linear Unit (ReLU) [START_REF] Nair | Rectified Linear Units Improve Restricted Boltzmann Machines Vinod Nair[END_REF], Leaky Rectified Linear Unit (Leaky ReLU) [START_REF] Maas | Rectifier nonlinearities improve neural network acoustic models[END_REF], Exponential Linear Unit (ELU) [START_REF] Clevert | Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)[END_REF], Scaled Exponential Linear Unit (SELU) [START_REF] Klambauer | Self-Normalizing Neural Networks[END_REF], Maxout unit [START_REF] Goodfellow | Maxout Networks[END_REF] and Softmax function [START_REF] Behnke | Hierarchical Neural Networks for Image Interpretation[END_REF]. Some of these activation functions are shown in the Figure 3.1.

For the classification task, the last layer is usually the logistic function for the binary classification problem and the softmax function for multi-class classification. Whereas for the hidden layers a sigmoid function (logistic function or hyperbolic tangent) is traditionally being chosen. But currently used a lot of different activation functions, with ReLU and it's varieties being common. Some desirable mathematical properties of the activation function:

• Non-linearity If the activation function is non-linear, it can be proven that the two-level neural network will be a universal function approximator [START_REF] Cybenko | Approximation by superpositions of a sigmoidal function[END_REF]. The identity activation function does not satisfy this property. If multiple layers use the identity activation function, the entire network is equivalent to a single-layer model.

• Continuous Differentiability

This property is desirable to provide gradient descent optimization techniques. For example, ReLU is not continuously differentiable and has some problems with gradient descent optimization, but remains a valid option. The binary step activation function is not differentiable at point 0 and its derivative is 0 at all other points, so gradient descent methods do not give any success for it [START_REF] Snyman | Practical Mathematical Optimization: An Introduction to Basic Optimization Theory and Classical and New Gradient-Based Algorithms[END_REF].

• Range of values

If the set of values of the activation function is limited, the gradient-based learning methods are more stable, since the representations of the standards significantly affect only a limited set of link weights. If the range of values is infinite, training is generally more efficient, since reference representations significantly affect most weights. In the latter case, a lower learning rate is usually required.

• Monotonicity

If the activation function is monotonic, the error surface associated with a sibling model is guaranteed to be convex [START_REF] Wu | Global stability analysis of a general class of discontinuous neural networks with linear growth activation functions[END_REF].

• Smooth functions with monotonic derivative

It is shown that in some cases they provide a higher degree of generalization [START_REF] Hagan | Neural Network Design[END_REF].

• Approximates the identity function near the origin If the activation functions have this property, the neural network will train efficiently if its weights are initialized with small random values. If the activation function does not approximate the identity around the origin, it is needed to be careful when initializing the weights [START_REF] Sussillo | Random Walk Initialization for Training Very Deep Feedforward Networks[END_REF] In this thesis we used ReLU as an activation function for the hidden layers and softmax function for the output layer. Rectified linear unit (ReLU) is an activation function defined as the positive part of its argument [START_REF] Nair | Rectified Linear Units Improve Restricted Boltzmann Machines Vinod Nair[END_REF]:

f (x) = max(0, x) = x, x ≥ 0 0, x < 0 (3.3)
ReLU has a range [0, +∞) and can provide better training for deep neural networks, compared to widely used activation functions, such as logistic sigmoid and its more practical analog, hyperbolic tangent [START_REF] Lecun | Efficient BackProp[END_REF]. It was demonstrated for the first time in 2011 [START_REF] Glorot | Deep Sparse Rectifier Neural Networks[END_REF]. And soon in 2017, ReLU was the most popular activation function [START_REF] Ramachandran | Searching for Activation Functions[END_REF].

The softmax function is a generalization of the logistic function to multiple dimensions. This activation function takes as input a vector x of J real numbers and normalizes it into a probability distribution consisting of J probabilities proportional to the exponentials of the input numbers. The softmax function has a range (0, 1) [START_REF] Behnke | Hierarchical Neural Networks for Image Interpretation[END_REF] and defined by the formula: for i = 1, ..., J For instance, j could be iterating through the number of kernels of the previous neural network layer while i iterates through the number of kernels of the current layer [START_REF] Behnke | Hierarchical Neural Networks for Image Interpretation[END_REF].

Training of artificial neural networks

In the learning process, the network views the training set in a certain order. The order of viewing can be sequential, random, etc. Some networks for unsupervised learning (for example, Hopfield networks) view the sample only once. Others (such as Kohonen networks), as well as networks for supervised learning, view the training set many times. One complete pass through the training set called the learning epoch [START_REF] Bishop | Pattern Recognition and Machine Learning[END_REF]. In the case of supervised learning, the set of initial data is divided into two parts -the training data and test data; the principle of separation can be arbitrary. Training data is provided to the network for training, and test data is used to calculate the network error. Thus, if the error on the test set decreases, the network does perform a generalization. If the error on the training data continues to decrease, but the error on the test data increases, then the network has stopped generalizing and is simply "remembering" the training data. This phenomenon is called network overfitting. In such cases, training is usually terminated [START_REF] Hagan | Neural Network Design[END_REF].

Other problems may occur during the training process, such as vanishing gradient problem or the network could fall into the local minimum [START_REF] Hagan | Neural Network Design[END_REF][START_REF] Livingstone | Artificial Neural Networks: Methods and Applications[END_REF].

Optimisation algorithms for ANN

Optimization algorithms are techniques and methods used to modify the attributes of the neural network (for example weights and learning rates), to reduce losses, and to provide the most accurate results possible.

In this thesis different optimization strategies were studied. Three variants of gradient descent (such as standard Gradient Descent (GD) Stochastic Gradient Descent (SGD) [START_REF] Bottou | Stochastic Learning[END_REF][START_REF] Bottou | The Tradeoffs of Large Scale Learning[END_REF] and mini-batch GD) have been reviewed, among which mini-batch gradient descent is the most popular and has fewer disadvantages [START_REF] Ruder | An overview of gradient descent optimization algorithms[END_REF]. Recently more algorithms for optimization appeared, such as Adagrad [START_REF] Duchi | Adaptive Subgradient Methods for Online Learning and Stochastic Optimization[END_REF], Adadelta [START_REF] Zeiler | ADADELTA: An Adaptive Learning Rate Method[END_REF], RMSprop (proposed by Geoff Hinton in Lecture 6e of the Coursera Class 3 ), and Adam [START_REF] Kingma | Adam: A Method for Stochastic Optimization[END_REF].

We choose Adam for the implementation in this thesis due to its speed and stability [START_REF] Kingma | Adam: A Method for Stochastic Optimization[END_REF][START_REF] Ruder | An overview of gradient descent optimization algorithms[END_REF]. Adam [START_REF] Kingma | Adam: A Method for Stochastic Optimization[END_REF] (Adaptive Moment Estimation) works with momentums of the first and second order. The intuition behind Adam is that it doesn't run so fast just to jump over the minimum, it decreases a velocity gradually to search carefully. In addition to keeping the exponentially decaying mean of the previously squared gradients like AdaDelta, Adam also stores the exponentially decaying mean of the previous gradients mt .

Values mt and vt are the first and the second moment, which is the mean and uncentered variance of the gradients, respectively [START_REF] Ruder | An overview of gradient descent optimization algorithms[END_REF].

mt = m t 1 -β t 1 (3.5)
3 http://www.cs.toronto.edu/ tijmen/csc321/slides/lecture_slides_lec6.pdf

vt = v t 1 -β t 2 (3.6)
Here the average of mt and vt has taken, so E[m(t)] can be equal to E[g(t)], where E[f (x)] is the expected value of f (x).

Then mt and vt used to update the parameters, as in Adadelta and RMSprop, which gives Adam update rule:

Θ t+1 = Θ t - η √ vt + mt (3.7)
In this work we used default values of β 1 equal to 0.9, and β 2 equal to 0.999, and 10 -8 for (as proposed by authors [START_REF] Kingma | Adam: A Method for Stochastic Optimization[END_REF]).

Adam has several advantages, this method is fast and converges quickly [START_REF] Kingma | Adam: A Method for Stochastic Optimization[END_REF] and it fixes vanishing learning rate and high variance [START_REF] Ruder | An overview of gradient descent optimization algorithms[END_REF]. In the works [START_REF] Kingma | Adam: A Method for Stochastic Optimization[END_REF][START_REF] Dozat | Incorporating nesterov momentum into adam[END_REF][START_REF] Ruder | An overview of gradient descent optimization algorithms[END_REF] it is shown that Adam makes progress faster in terms of epochs number comparing to other algorithms. But at the same time it is computationally expensive.

Convolutional Neural Networks (CNN)

Convolutional neural network (CNN) is a special architecture of artificial neural networks proposed by Yann Lecun in 1988 [START_REF] Lecun | Backpropagation applied to handwritten zip code recognition[END_REF] and aimed at efficient pattern recognition.

The idea behind convolutional neural networks is an alternation of convolution layers and pooling layers. The structure of the network is unidirectional, fundamentally multilayer. For training, standard methods are used, most often the backpropagation method. Any activation function could be chosen, but commonly the activation function is a ReLU layer [START_REF] Nielsen | Neural Networks and Deep Learning[END_REF].

The network architecture got its name due to the presence of the convolution operation, the point of which is that each fragment of the image is multiplied by the convolution matrix (kernel) element by element, and the result is summed up and written into a similar position in the output image [START_REF] Goodfellow | Deep Learning[END_REF].

The operation of a convolutional neural network usually performs a transition from specific features of an image to more abstract details, and then to even more abstract details up to the isolation of high-level features. At the same time, the network self-adjusts and develops the very necessary hierarchy of abstract features (feature maps), filtering unimportant details and highlighting important ones [START_REF] Goodfellow | Deep Learning[END_REF].

In fact, the features generated by a complex CNN are usually difficult to interpret, so in practice, it is not particularly recommended to try to understand the contents of these features, instead, it is recommended to improve the structure and architecture of the network itself to obtain better results [START_REF] Nielsen | Neural Networks and Deep Learning[END_REF].

CNNs are good in classification, object detection and segmentation [START_REF] Nielsen | Neural Networks and Deep Learning[END_REF] of the images and signals (Figure 3.2).

General architecture of CNN

In CNN, the convolution operation uses only a limited matrix of small weights, which is moved over the entire processed layer (at the very beginning directly over the input image or signal), forming after each shift an activation signal for the neuron of the next layer with a similar position.

The same weight matrix (convolution kernel) is used for different neurons of the layer. It is interpreted as a graphic encoding of some feature, for example, the presence of an oblique line at a certain angle. Then the next layer, resulting from the convolution operation with such kernel, shows the presence of this feature in the processed layer and its coordinates, forming a feature map. Naturally, in CNN, there is a set of different kernels that encodes image elements. These convolution kernels are not set manually in advance but are formed independently during the training of the network [START_REF] Goodfellow | Deep Learning[END_REF].

Each kernel generates its own instance of a feature map, making the neural network multichannel (many independent feature maps on one layer). It should be noted that when a kernel is iterated over a layer, it usually moves not for a full step (the size of this matrix), but a short distance (one or two neurons (pixels)).

Subsampling or pooling is the operation of combining, performs a dimension reduction of the generated feature maps. In CNN architecture information about the presence of the desired feature is more important than accurate knowledge of its coordinates, therefore, the maximum (or average) is selected from several neighboring neurons of the feature map and is saved as one neuron of the condensed feature map of lower dimension. Due to this operation, in addition to speeding up further calculations, the network becomes more invariant to the scale of the input image [START_REF] Nielsen | Neural Networks and Deep Learning[END_REF].

Let's consider a typical structure of CNN in more details (see Figure 3.3). The network consists of a large number of layers. After the input layer (input image), the signal passes through a series of convolutional layers, in which the actual convolution and pooling alternate. Usually, after passing through several layers, the feature map degenerates into a feature vector. At the output of the convolutional layers, several fully connected layers are additionally installed, at the input of which flatten feature maps are fed.

Convolution layer

The convolution layer is the main block of CNN. The convolution layer includes its own kernel (filter) for each channel, the convolution kernel processes the previous layer fragment by fragment (summing the results of the element-wise product for each fragment). The weights of the convolution kernel are unknown and are set during training.

A feature of the convolutional layer is the relatively small number of parameters set during training. So for example, if the original image has a dimension of 12 × 12 × 3 pixels (this means 432 input neurons), and the convolutional layer uses filters with a 5x5 pixel kernel with an output of 1 channel, then only 25 kernel weights are determined during the training process. However, because we have 3 channels image our kernel needs to have also 3 channels. That's why we actually have 5 × 5 × 3 = 75 learnable parameters, which is significantly less than the number of required parameters of a fully connected neural network [START_REF] Wang | A basic introduction to separable convolutions[END_REF].

Convolutional layer with regularized weights over fewer parameters allows avoiding the vanishing gradient and exploding gradient problems that can be observed during backpropagation in FNN [START_REF] Venkatesan | Convolutional Neural Networks in Visual Computing: A Concise Guide[END_REF]. The result of each convolution goes to the activation function, which is a non-linear function. The activation layer is usually logically combined with the convolution layer.

Figure 3.4 shows the example of the convolutional layer with one channel output. Consider a signal with 8 samples and 3 channels as an input. In this case, there exists only one kernel 3 × 1 × 3. And resulting feature map has a size 8 × 1 × 1 in the case with the same padding and a stride of one.

Same and valid padding

Padding is an addition of pixels to the edge of the image. Adding padding allows creating very deep models so that feature maps don't fizzle out [START_REF] Nielsen | Neural Networks and Deep Learning[END_REF].

The same padding calculates and adds the padding required to the input image (or feature map) to control that the output has the same shape as the input.

The same padding tries to add columns evenly to the left and right, but if the number of columns to add is odd, it will add an extra column to the right (same logic applies vertically: there might be an extra row of zeros at the bottom) [START_REF] Nielsen | Neural Networks and Deep Learning[END_REF].

With the same padding, if a stride of one is used, the layer's output will have the same spatial dimensions as its inputs.

With valid padding, there are no augmented input fields and the layer uses only valid input data. That's why some number of columns or rows could be dropped in this case. Valid padding discards only the right-most columns (or the bottom-most rows) [START_REF] Nielsen | Neural Networks and Deep Learning[END_REF].

Depthwise convolutional layer

Depthwise convolution is a type of convolutional layer. With regular convolution performed on multiple input channels (Figure 3.4), the filter has a depth equal to the input and allows to mix channels freely to generate each element in the output. Depthwise convolution performs other way -each channel is kept separately [START_REF] Chollet | Xception: Deep Learning with Depthwise Separable Convolutions[END_REF][START_REF] Howard | MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications[END_REF]. The Figure 3.5 showed how it works [START_REF] Bendersky | Depthwise separable convolutions for machine learning[END_REF][START_REF] Wang | A basic introduction to separable convolutions[END_REF].

Depthwise convolution has three steps:

• Divide the input data into channels and divide the kernel into channels (the input and the kernel have the same number of channels).

• For each of the channels, convolve the input with the appropriate kernel, creating an output feature maps.

• Stack the output kernels together.

Note that the channel multiplier depth parameter of the depthwise convolutional layer lets us compute multiple outputs for each input channel [START_REF] Chollet | Keras[END_REF][START_REF] Bendersky | Depthwise separable convolutions for machine learning[END_REF]. Example in Figure 3.5 has multiplier depth equals to one. But it can be n ≥ 2. In this case, n different kernels for each input channel will be created, allowing to compute more variations of the feature maps. The output channel number will be n times greater than the input channel number.

Depthwise separable convolutional layer

Depthwise separable convolutional [START_REF] Chollet | Xception: Deep Learning with Depthwise Separable Convolutions[END_REF][START_REF] Howard | MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications[END_REF] is a composition of two convolutions: depthwise convolution and following pointwise convolution. In Figure 3.6 it could be seen that after the depthwise convolution is completed, an additional step is performed: 1 × 1 spatial convolution across channels. Pointwise convolution kernel iterates through every single point. This kernel has a depth of however many channels the input image has; in our example, 3 (Figure 3.6) [START_REF] Wang | A basic introduction to separable convolutions[END_REF].

Similarly to depthwise convolution depthwise separable convolution has the parameter "channel multiplier depth" which allows computing several outputs channels for one input channel [START_REF] Bendersky | Depthwise separable convolutions for machine learning[END_REF][START_REF] Chollet | Xception: Deep Learning with Depthwise Separable Convolutions[END_REF].

Depthwise and separable convolutions are efficient alternatives to standard convolution layers, which reduces the number of required weights and computations [START_REF] Chollet | Xception: Deep Learning with Depthwise Separable Convolutions[END_REF][START_REF] Howard | MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications[END_REF][START_REF] Wang | A basic introduction to separable convolutions[END_REF][START_REF] Bendersky | Depthwise separable convolutions for machine learning[END_REF] and thus are less prone to overfitting.

Pooling layers

A pooling layer is nonlinear downsampling of a feature map, while a group of samples is compressed to one sample. The maximum function [START_REF] Yamaguchi | A neural network for speaker-independent isolated word recognition[END_REF] is most commonly used in this case. The transformations involve nonintersecting rectangles or squares, each of which is compressed into one sample, and the sample with the maximum value is selected. The pooling operation can significantly reduce the feature map size. Pooling filters unnecessary data and prevents overfitting. The pooling layer is usually inserted after the convolution layer, and before the next convolution, layer [START_REF] Nielsen | Neural Networks and Deep Learning[END_REF].

In addition to pooling with a maximum function, average pooling could be used. In this case, the average of the samples in the square is used for the output [START_REF] Nielsen | Neural Networks and Deep Learning[END_REF]. Average and max-pooling showed in Figure 3.7.

Fully-connected layer

After several passes of convolution and pooling layers, the system rebuilds from a concrete representation of the signal to more abstract feature maps. Usually, after each convolutional layer, the number of channels increases, and the signal dimension decreases in each channel. In the end, there remains a large set of channels storing a small amount of data (even one parameter), which are interpreted as the most abstract features identified from the original data [START_REF] Nielsen | Neural Networks and Deep Learning[END_REF]. This data is combined and transmitted to a standard fully connected neural network, which can also consist of several layers. In this case, fully connected layers have a relatively small dimension (relative to the number of samples in the original image).

Fully connected layers could have different activation functions. ReLU activation function is commonly used. But the last layer of the ANN uses softmax activation function to predict several classes [START_REF] Venkatesan | Convolutional Neural Networks in Visual Computing: A Concise Guide[END_REF]. In general, this problem requires separate treatment. This is especially important for cases where only a very limited number of training samples are available as in the case of the BCI domain. One of the most simple methods to learn on limited data is called early stopping [START_REF] Prechelt | Early Stopping -But When?[END_REF]. Usage of this method often ensures that the network generalizes well examples outside of the training set. This is achieved by finding the moment when the model stops learning and starts overfitting, usually checking the model on a validation dataset.

Overfitting and regularization techniques

Regularization is another method used to avoid overfitting. There are regularization techniques that could be applied directly to learnable parameters inside layers of ANN and also particular regularization layers, such as dropout [START_REF] Srivastava | Dropout: a simple way to prevent neural networks from overfitting[END_REF] or batch normalization layers [START_REF] Ioffe | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[END_REF]. These techniques could be used separately or collectively because they regularize different parts of ANN. So in this thesis to prevent overfitting, we decided to use L1 and L2 regularization for CNN kernels; dropout before fully connected layer and between convolutional layers; and batch normalization layer after convolution layer and before the activation layer. We will introduce these techniques further in current section.

L1 and L2 regularization

The first regularizer allows for applying penalties to layer parameters or layer activity during optimization. These penalties are summed up in a loss function that the network optimizes.

Regularizers can be of three types:

• Regularizer to apply a penalty on the layer's kernel

• Regularizer to apply a penalty on the layer's bias

• Regularizer to apply a penalty on the layer's output

In this case regularization terms are "constraints" that the optimization algorithm must "adhere to" when minimizing the loss function, in addition to the need to minimize the error between true y and predicted ŷ. Loss function without regularization looks as follows:

Loss = Error(y, ŷ) (3.8)
Regularizers could apply different penalties:

• A regularizer that applies a L1 regularization penalty:

Loss = Error(y, ŷ) + λ 1 N i=1 |w i | (3.9)
• A regularizer that applies a L2 regularization penalty:

Loss = Error(y, ŷ) + λ 2 N i=1 w 2 i (3.10)
• Regularizer that applies both L1 and L2 penalties:

Loss = Error(y, ŷ) + λ 1 N i=1 |w i |+λ 2 N i=1 w 2 i (3.11)

Dropout layer

Dropout is a method for regularization of ANN, designed to reduce overfitting of the network by preventing complex coadaptation of individual neurons to training data during learning [START_REF] Srivastava | Dropout: a simple way to prevent neural networks from overfitting[END_REF].

Dropout characterizes the exclusion of a certain percentage (for example, 30% or 50%) of random neurons at different epochs during the training of a neural network. This is a very efficient way of averaging models within a neural network. As a result, more trained neurons receive bigger weight in the network [START_REF] Warde-Farley | An empirical analysis of dropout in piecewise linear networks[END_REF]. This technique significantly decreases the learning time, increase the quality of learning on training data, and also improves the quality of model predictions on new test data [START_REF] Srivastava | Dropout: a simple way to prevent neural networks from overfitting[END_REF]. [START_REF] Srivastava | Dropout: a simple way to prevent neural networks from overfitting[END_REF]). On the left is the neural network before dropout was applied to it, on the right is the same network after dropout. The network shown on the left is used for testing, after parameters training.

The main idea behind dropout is to train an ensemble of several ANNs instead of training one ANN and then average the results.

Networks for training are obtained by dropping out neurons with probability p, so the probability that a neuron will remain in the network is 1 -p. Dropping out a neuron means that for any input and with any parameters, it returns 0.

The excluded neurons do not contribute to the learning process at any stage of the backpropagation algorithm. Consider h(x) = xW + b to be a linear projection of the input d i -dimensional vector x on the d h -dimensional space of output values; and activation function a(h). Applying Dropout at the training stage can be represented as a modified activation function:

f (h) = D a(h) (3.12)
where D = (X 1 , ..., X dx ) -d x -dimensional vector of random variables X i distributed according to Bernoulli's principle.

X i has the following probability distribution:

f (k, p) = p, if k = 1 1 -p, if k = 0 , (3.13)
where k are all possible output values. Obviously, this random variable fits perfectly if Dropout is applied to one neuron. Indeed, the neuron is turned off with probability p = P (k = 1), otherwise, it is left on.

Let's look at applying Dropout to the i-th neuron:

O i = X i a( di k=1 w k x k + b) =    a( di k=1 w k x k + b), if X i = 1 0, if X i = 0 , (3.14)
where P (X i = 0) = p. Since at the training stage the neuron remains in the network (is not excluded) with probability 1 -p, at the testing stage we need to emulate the behavior of the ensemble of neural networks used in training. For this, the authors propose at the testing stage to multiply the activation function by the coefficient q = 1 -p. Thereby, At the training stage:

O i = X i a( di k=1 w k x k + b) (3.15)
At the testing stage:

O i = qa( di k=1 w k x k + b) (3.16)

Batch normalization layer

Batch normalization is a method that improves performance and stabilizes ANN through normalization of the input layer by re-centering and re-scaling [START_REF] Ioffe | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[END_REF].

Normalization of the input layer of the neural network is usually done by scaling the data supplied to the activation functions. For example, when there are features with values from 0 to 1 and some features with values from 1 to 1000, then they need to be normalized to speed up learning. Data normalization can be performed in the hidden layers of neural networks, which is what the batch normalization method does.

Batch normalization reduces internal covariate shift [START_REF] Ioffe | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[END_REF]. Covariate shift is a situation when the distributions of feature values in the training and test sets have different parameters (mathematical expectation, variance, etc.).

Consider values x from batch B = x 1 , ..., x m ; adjustable parameters γ, β; constant for computational stability.

The batch normalization algorithm can be represented as follows:

1. Determine batch expected value:

µ B = 1 m m i=1
x i (3.17)

2. Determine batch variance:

σ 2 B = 1 m m i=1 (x i -µ B ) 2 (3.18)
3. Compute normalization:

xi = x i -µ B σ 2 B + (3.19)
4. Compute compression and shift:

BN γ,β (x i ) = γ xi + β (3.20)
Besides, using batch normalization has several additional useful properties:

• faster convergence of models is achieved [START_REF] Ioffe | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[END_REF], despite performing additional calculations;

• batch normalization allows each layer of the network to learn more independently of other layers;

• it becomes possible to use a higher learning rate since batch normalization ensures that the outputs of the neural network nodes will not have too large or small values [START_REF] Ioffe | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[END_REF];

• batch normalization is also a regularization mechanism: this method introduces some noise into the outputs of hidden layer nodes, similar to the dropout method;

• models become less sensitive to initial weight initialization [START_REF] Ioffe | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[END_REF].

Review of deep learning for motor imagery detection in BCI

With the growth of computing powers interest in artificial neural networks was renewed recently. Every year models become more complex and demonstrate better results what is proved by ImageNet Large Scale Visual Recognition Challenge [START_REF] Russakovsky | Imagenet large scale visual recognition challenge[END_REF]. In 2011 classification error rate was 25%. Deep CNN AlexNet [START_REF] Krizhevsky | Imagenet classification with deep convolutional neural networks[END_REF] achieved winning top-5 test error rate of 15.3%, compared to runners-up's 26% in 2012. AlexNet was consisted of 8 layers and had 650 000 neurons. From that time AlexNet was outperformed numerous times and in 2017, 29 of 38 competing teams had greater than 95% accuracy. With increasing accuracy, the performance number of layers also increased, and in 2017 winning SENet [START_REF] Hu | Squeeze-and-Excitation Networks[END_REF] had 154 layers 4 .

Naturally, the growth of deep learning popularity did not leave aside the BCI community. However, there is a problem with a very limited amount of labeled data in the BCI domain. A large amount of data and the development of computing power allowed and played a significant role in the success of deep learning techniques and ANNs in the areas of computer vision and natural language processing. A small amount of data limited the application of deep learning in the BCI domain.

Below is an overview of the application of deep learning in motor imagery based BCIs. This section presents the data from the 71 reviewed articles, Table 3.1 summarizes all the obtained data. We selected all articles (including conference papers) written in English which we were able to find as of November 29, 2020, using next key words: artificial neural network, deep learning, motor imagery, BCI, EEG. We used ieeexplore.ieee.org, scholar.google.com and google.com in our research.

In the reviewed studies different datasets were used, some of them are available, such as BCI Competition datasets [START_REF] Blankertz | The BCI Competition 2003: progress and perspectives in detection and discrimination of EEG single trials[END_REF][START_REF] Blankertz | The BCI competition. III: Validating alternative approaches to actual BCI problems[END_REF][START_REF] Sajda | A data analysis competition to evaluate machine learning algorithms for use in brain-computer interfaces[END_REF][START_REF] Tangermann | Review of the bci competition iv[END_REF], EEG motor movement/imagery dataset (eegmmidb) dataset [START_REF] Schalk | BCI2000: a general-purpose Brain-Computer Interface (BCI) system[END_REF] from Physionet [START_REF] Goldberger | PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals[END_REF], upper limb movement (ULM) dataset [START_REF] Ofner | Upper limb movements can be decoded from the time-domain of low-frequency EEG[END_REF], High-Gamma Dataset (HGD) [START_REF] Schirrmeister | Deep learning with convolutional neural networks for EEG decoding and visualization[END_REF], and some of the reviewed studies used private datasets.

Feature extraction methods, deep learning classifier architectures, and optimization strategies that have been used by the reviewed articles are presented in Table 3.1.

There are different deep learning architectures used in MI-based BCI domain. Most reviewed studies used CNN architectures (62.2%), while portion of following LSTM (8.5%), RNN (7.3%) and DBN (6.1%) is significantly lower. The use of the remaining architectures, such as CapsNet [START_REF] Ha | Decoding Two-Class Motor Imagery EEG with Capsule Networks[END_REF][START_REF] Ha | Motor Imagery EEG Classification Using Capsule Networks[END_REF], Stacked Autoencoder (SAE) [START_REF] Tabar | A novel deep learning approach for classification of EEG motor imagery signals[END_REF][START_REF] Hassanpour | A novel end-to-end deep learning scheme for classifying multi-class motor imagery electroencephalography signals[END_REF], Autoencoder (AE) [START_REF] Saha | Deep Learning the EEG Manifold for Phonological Categorization from Active Thoughts[END_REF][START_REF] Zhang | Converting Your Thoughts to Texts: Enabling Brain Typing via Deep Feature Learning of EEG Signals[END_REF], Variational Autoencoder (VAE) [START_REF] Dai | EEG Classification of Motor Imagery Using a Novel Deep Learning Framework[END_REF], Gated Recurrent Unit (GRU) [START_REF] Luo | Exploring spatial-frequency-sequential relationships for motor imagery classification with recurrent neural network[END_REF] and Extreme Learning Machine (ELM) [START_REF] She | A hierarchical semi-supervised extreme learning machine method for EEG recognition[END_REF] even lower (only 10.8% in total). Figure 3.9 represents proportion use of the DNN architectures across all the reviewed articles.

Some architectures contains stacked layers of different types, such as CNN + Stacked Autoencoders (SAE) [START_REF] Tabar | A novel deep learning approach for classification of EEG motor imagery signals[END_REF], CNN + RNN [START_REF] Zhao | On the improvement of classifying EEG recordings using neural networks[END_REF] and RNN + Autoencoder (AE) + XGBoost [START_REF] Saha | Deep Learning the EEG Manifold for Phonological Categorization from Active Thoughts[END_REF][START_REF] Zhang | Converting Your Thoughts to Texts: Enabling Brain Typing via Deep Feature Learning of EEG Signals[END_REF].

The reviewed studies used different activation functions in the proposed architectures of neural networks. The employed activation functions in the reviewed articles were as follows: 53% ReLU, 19.7% ELU, 9.1% Sigmoid, and 6.1% SELU. The proposed architectures in the remaining studies used the tanh and linear activation functions as well as S-shaped Rectified Linear Unit (SReLU) and Leaky ReLU. Figure 3.10 represents the proportion use of the different activation functions across all the reviewed articles where it was mentioned.

During the training procedure, the parameters of the neural networks updates using different optimization methods. The larger part of the reviewed studies used one of the followed optimization algorithms: Adam, gradient descent (GD), and stochastic gradient descent (SGD). And only a few studies used RMSProp and Adadelta. Figure 3.11 represents the proportion use of the different optimization algorithms across all the reviewed articles. That's why it was decided to divide all the reviewed studies into three conditional categories according to input formulation types. The number of articles that have used each of the input types is as follows: features (30), timeseries [START_REF] Congedo | A New Generation of Brain-Computer Interface Based on Riemannian Geometry[END_REF], and images [START_REF] Blankertz | Optimizing spatial filters for robust EEG single-trial analysis [revealing tricks of the trade[END_REF] Short-Time Fourier Transform (STFT), FFT, and Fast Fourier Transform Energy Map (FFTEM) were used by 47.5% of the studies that adopted images in the classification process. While wavelet transforms takes 38%. to the neural network and thus could be recommended for this task. While wavelet and Fourier transform-based methods are recommended in case of using images as inputs to the DNN.

Fourier methods analyze signals using a window of fixed length, which is an obvious limitation of these methods due to the non-stationarity of EEG signals. Wavelet transforms could be better suited for capturing images from the raw EEG data than Fourier transform. The wavelet methods analyze the signals with numerous frequencies and time resolutions.

At the same time there are a lot of studies which used raw EEG signal as an input for the models. And the number of such articles grows every year, 12 of the 22 reviewed articles were published after beginning of 2019. Most of these articles used CNN architecture [START_REF] Kar | A Deep Convolutional Neural Network Based Classification Of Multi-Class Motor Imagery With Improved Generalization[END_REF][START_REF] Jeon | Domain Adaptation with Source Selection for Motor-Imagery based BCI[END_REF][START_REF] Zhao | On the improvement of classifying EEG recordings using neural networks[END_REF][START_REF] Huang | Deep Neural Network with Attention Mechanism for Classification of Motor Imagery EEG[END_REF][START_REF] Lawhern | EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[END_REF][START_REF] Amin | Multilevel Weighted Feature Fusion Using Convolutional Neural Networks for EEG Motor Imagery Classification[END_REF][START_REF] Zhao | Learning joint space-time-frequency features for EEG decoding on small labeled data[END_REF][START_REF] Wu | A Parallel Multiscale Filter Bank Convolutional Neural Networks for Motor Imagery EEG Classification[END_REF][START_REF] Dai | HS-CNN: a CNN with hybrid convolution scale for EEG motor imagery classification[END_REF][START_REF] Li | A Channel-Projection Mixed-Scale Convolutional Neural Network for Motor Imagery EEG Decoding[END_REF][START_REF] Amin | Deep Learning for EEG motor imagery classification based on multi-layer CNNs feature fusion[END_REF][START_REF] Schirrmeister | Deep learning with convolutional neural networks for EEG decoding and visualization[END_REF][START_REF] Wu | Classification of EEG Motor Imagery Using Support Vector Machine and Convolutional Neural Network[END_REF][START_REF] Tayeb | Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals[END_REF][START_REF] Zhang | Making Sense of Spatio-Temporal Preserving Representations for EEG-Based Human Intention Recognition[END_REF][START_REF] Zhang | Converting Your Thoughts to Texts: Enabling Brain Typing via Deep Feature Learning of EEG Signals[END_REF][START_REF] Dose | An end-to-end deep learning approach to MI-EEG signal classification for BCIs[END_REF].

Application of other architectures such RNN [START_REF] Ko | Deep recurrent spatio-temporal neural network for motor imagery based BCI[END_REF][START_REF] Zhang | Intent Recognition in Smart Living Through Deep Recurrent Neural Networks[END_REF], LSTM [START_REF] Zhang | Making Sense of Spatio-Temporal Preserving Representations for EEG-Based Human Intention Recognition[END_REF][START_REF] Zhang | Intent Recognition in Smart Living Through Deep Recurrent Neural Networks[END_REF][START_REF] Ma | Improving EEG-Based Motor Imagery Classification via Spatial and Temporal Recurrent Neural Networks[END_REF], SAE [START_REF] Hassanpour | A novel end-to-end deep learning scheme for classifying multi-class motor imagery electroencephalography signals[END_REF], DBN [START_REF] Tang | Semisupervised Deep Stacking Network with Adaptive Learning Rate Strategy for Motor Imagery EEG Recognition[END_REF] remains limited for the raw EEG analysis. Some of the studies used stacked variations of different networks, such as RNN + AE + XGBoost [START_REF] Zhang | Converting Your Thoughts to Texts: Enabling Brain Typing via Deep Feature Learning of EEG Signals[END_REF] or CNN + RNN [START_REF] Zhao | On the improvement of classifying EEG recordings using neural networks[END_REF].

Proposed architectures for motor intention detection

Summarizing the review in section 3.5, there were observed numerous studies and the most promising architectures were defined. In the case of analyzing small datasets of EEG signals, which are usually noisy and non-stationary, CNN architectures with different regularizing techniques were chosen for the following implementation. A CNN architecture allows working with a small amount of data because of its reduced number of parameters. It was also decided to use raw EEG signals as an input to neural networks to allow CNN to extract features by itself. In this section, different architectures of DNNs, which will be used to obtain results in chapter 5 will be described.

(4000) Dense(Softmax) N = 2

(2)

Deep Convolutional Network

Deep Convolutional Network (D-ConvNet) has four convolution-max-pooling blocks, with a special first block designed to handle the EEG input. It is followed by three standard convolution-max-pooling blocks and a dense softmax classification layer. The first convolutional block was split into two convolutional layers to better handle a large number of input channels -one input channel per electrode. D-ConvNet was proposed in [START_REF] Schirrmeister | Deep learning with convolutional neural networks for EEG decoding and visualization[END_REF] and details of this network architecture can be found in Table 3.2.

Shallow Convolutional Network

The architecture designed for the Shallow Convolutional Network (S-ConvNet) has been inspired by the Filter Bank Common Spatial Pattern (FBCSP) algorithm. The transformations performed by the S-ConvNet are similar to the transformations of FBCSP. Concretely, the first two layers of the S-ConvNet perform temporal and spatial convolutions, as in the D-ConvNet. These steps are analogous to the band-pass filter and the CSP spatial filter steps in FBCSP. In contrast to the D-ConvNet, the temporal convolution of the S-ConvNet had a larger kernel size, allowing a larger range of transformations in this layer. After the two convolutions of the S-ConvNet, a squaring nonlinearity x 2 activation function, a mean pooling layer, and a logarithmic activation function log(x) followed; together these steps are analogous to the trial log-variance computation in FBCSP.

S-ConvNet was proposed in [START_REF] Schirrmeister | Deep learning with convolutional neural networks for EEG decoding and visualization[END_REF] and details of this network architecture can be found in Table 3.3.

EEGNet-2.32

EEGNet architecture is presented on Figure 3.14. The first layer of the EEGNet is a temporal convolution to learn frequency filters. Then follows a depthwise convolution layer, connected to each feature map individually to learn frequency-specific spatial filters. The separable convolution is the next convolutional layer; it is a combination of a depthwise convolution, which learns a temporal summary for each feature map individually, followed by a pointwise convolution, which learns how to optimally mix the feature maps together [START_REF] Lawhern | EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[END_REF].

EEGNet-4.8

We tested different sets of parameters (not detailed in the thesis) and select 8 models derived from the original EEGNet which allow us to obtain the best result. We kept 8 temporal filters F 1 = 8 and 16 pointwise filters F 2 = 16 for all models presented in this thesis. The parameters which have been changing were lengths of temporal kernel size for the first convolutional layer K and multiplier depth D (which determines the number of spatial filters) of the depthwise convolution layer. Thus we set multiplier depth to 2 or 4.

The first layer of filters of the original EEGNet [START_REF] Lawhern | EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[END_REF] extracts mainly a low frequency information since it has large kernel size (1,[START_REF] Almeida | Awake and unable to move: what can perioperative practitioners do to avoid accidental awareness under general anesthesia?[END_REF], and ignores mid-and high frequency information [START_REF] Zeiler | Visualizing and Understanding Convolutional Networks[END_REF]. To remedy this problem, we reduced the first layer kernel size (as in [START_REF] Zeiler | Visualizing and Understanding Convolutional Networks[END_REF]). So for this study the kernel size was set to (1, 32) (as recommended in [START_REF] Lawhern | EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[END_REF] for sensory motor data), (1,[START_REF] Ahn | A Review of Brain-Computer Interface Games and an Opinion Survey from Researchers, Developers and Users[END_REF], (1,[START_REF] Myrden | A Passive EEG-BCI for Single-Trial Detection of Changes in Mental State[END_REF] and (1,4). These new architectures retain much more information in the first layer features concerning the mid and high-frequency bands. More importantly, they also improve the classification performance as shown in section 5.3.2.

We use the notation EEGNet-D.K to denote the number of spatial filters to learn and kernel length for the first convolutional layer; i.e.: EEGNet-4.8 denotes learning 4 spatial filters per temporal filter and kernel length (1,[START_REF] Myrden | A Passive EEG-BCI for Single-Trial Detection of Changes in Mental State[END_REF]. Full details about the network architecture can be found in Table 3.4.

Training procedure

All deep neural networks were implemented with the TensorFlow framework [START_REF] Abadi | TensorFlow: Large-scale machine learning on heterogeneous systems[END_REF] and Keras [START_REF] Chollet | Keras[END_REF] and trained using the Grid'5000 testbed [START_REF] Balouek | Adding virtualization capabilities to the Grid'5000 testbed[END_REF] to speed up the computations. The training of the networks was performed with the following configurations.

• ADAM [START_REF] Kingma | Adam: A Method for Stochastic Optimization[END_REF] is used as the optimization method. The parameters are set to default values as proposed in [START_REF] Kingma | Adam: A Method for Stochastic Optimization[END_REF].

• Categorical cross-entropy is used as the optimization criterion.

• Batch size is selected as 16.

• We run 300 training iterations (epochs) and perform validation stopping, saving the model weights which produced the lowest validation set loss.

• We used a batch normalization [START_REF] Ioffe | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[END_REF] layer before the activation layer

• We used a dropout layer after the activation with a probability of 50%. Our investigations showed that 50% dropout probability leads to better (approximately 2%) performance comparing to 25% (as proposed in [START_REF] Lawhern | EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[END_REF]) and helps to prevent overfitting on datasets with a small number of samples.

The number of training parameters for each neural network is represented in Table 3.5. We could see from the table that EEGNet architectures have much smaller number of parameters comparing to D-ConvNet and S-ConvNet.

Conclusion

In this chapter, we have introduced the concept of artificial neural networks. We have represented activation functions and optimization algorithms for artificial neural networks. We have selected Adam for our experiments in section 5.3 as it allows us to use less time and train more efficiently.

Also, we have presented convolutional neural networks, as they could be used for the direct feature extraction and classification from raw EEG signals. We have introduced their architecture and layers, including depthwise convolutional layer and depthwise separable convolutional layer.

Moreover, we have presented the overfitting problem and regularization techniques to prevent it. Overfitting of neural networks and bad generalization due to the small amount of data available for training is one of the reasons why deep learning application is still limited in the MI-based BCI domain. Another challenge is the nonstationarity and low signal-to-noise ratio of the EEG signals recorded in a realistic setting. The above problems make it impossible to use very deep architectures with many layers and require the application of regularization techniques. In this thesis, we will use L1 and L2 regularization for layer's kernels, dropout layer, and batch normalization layer in proposed architectures for MI detection.

In this chapter, we reviewed existing deep learning solutions for the analysis of EEG signals for MI-based BCI in section 3.5. We have selected CNN architectures for raw EEG signal classification such as Deep Convolutional Network, Shallow Convolutional Network, and EEGNet. Also, we proposed the adaptation of EEGNet, namely EEGNet-4.8 for the following investigations for the motor imagery detection in section 3.6).

Also, we have presented the procedure and parameters that will be used for neural network training in further investigations.

In the next chapter, we will present datasets with motor imageries, which we will use in our experiments. Classification results for deep learning architectures presented in this chapter will be introduced in section 5. In the last chapters, we have presented machine learning and deep learning methods that could be used to improve feature extraction and classification of motor imageries in BCIs.

In this chapter, we present motor imagery (MI) data sets that will be used for our experiments. Participants, EEG channel layouts paradigms, time schemes, and experimental tasks will be described. Preprocessing and cross-validation details will be discussed as well.

To improve motor imagery detection, we have selected 3 different datasets for experiments. Firstly in section 4.1, we will present a classic 2-class MI dataset [START_REF] Shin | Open Access Dataset for EEG+NIRS Single-Trial Classification[END_REF] containing right hand and left hand motor imageries.

Secondly, in section 4.2, we will introduce a 8-class MI dataset with combined movement MIs. This is a database that consisted of 7 different MI tasks and resting state. All the data was recorded by members of the NeuroRhythms team in the LORIA laboratory.

Finally, in section 4.3, we will present a dataset consisted of median nerve stimulations during motor imagery and during rest, as well as MI condition separately. All the data was recorded by members of the NeuroRhythms team in the LORIA laboratory.

2-class motor imagery dataset

We used an open-source 2-class motor imagery dataset described in details in [START_REF] Shin | Open Access Dataset for EEG+NIRS Single-Trial Classification[END_REF]. It is available via Phys-ioNet [START_REF] Goldberger | PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals[END_REF] web-site. This dataset contains right hand and left hand motor imageries. We used this dataset in the investigation on estimating the quality of motor imageries in BCI using outlier detection. Results are represented in section 5.1.

Participants

The database contains EEG signals for BCIs recorded by NIRx GmbH, Berlin, Germany [START_REF] Shin | Open Access Dataset for EEG+NIRS Single-Trial Classification[END_REF]. The experiment involved 29 right-handed and one left-handed healthy subjects (14 males and 15 females, average age 28.5 ± 3.7 years) who were asked to perform imaginary movements with their right and left hands. All of them were healthy, without reported neurological, psychiatric or other brain-related diseases. All volunteers were informed about the experimental procedure and written consent was obtained from all participants. This study was conducted according to the Declaration of Helsinki [START_REF] Association | World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects[END_REF] and was approved by the Ethics Committee of the Institute of Psychology and Ergonomics, Technical University of Berlin [START_REF] Shin | Open Access Dataset for EEG+NIRS Single-Trial Classification[END_REF].

Experimental tasks

Subjects were supposed to perform kinesthetic MI of their right or left hand at the audible signal for three sessions. More precisely, task consisted of imaginations the opening and closing hands as they were grabbing a ball. An arrow pointing to the left or right side appeared on the screen for two seconds as a visual instruction. After the arrow disappeared, a black fixation cross was displayed during the task period for ten seconds. The subjects were asked to mentally open and close their hands with a 1 Hz pace. MI was accomplished continuously over the task period. The task period was finished with a short beep sound and a 'stop' displayed for 1 s on the screen. Each session contained 20 imaginary movements (10 trials for each left and right hand MI in a single session). So 30 trials for each MI in total [START_REF] Shin | Open Access Dataset for EEG+NIRS Single-Trial Classification[END_REF].

Paradigm and time scheme

The subjects sat on a comfortable armchair in front of a 50-in white screen. The distance between their heads and the screen was 1.6 m. They were asked not to move any part of the body during the data recording. The experiment consisted of three sessions of left and right hand MI. Each session comprised a 1 min pre-experiment resting period, 20 repetitions of the given task and a 1 min post-experiment resting period. The task started with 2 s of a visual introduction of the task, followed by 10 s of a task period and resting period which was given randomly from 15 to 17 s. At the beginning and end of the task period, a short beep (250 ms) was played. Figure 4.1 shows the schematic diagram of the experimental paradigm.

Electroencephalographic data

EEG data was recorded by a multichannel BrainAmp EEG amplifier with thirty active electrodes (Brain Products GmbH, Gilching, Germany) with linked mastoids reference. In the experiment a custom-made stretchy fabric cap (EASYCAP GmbH, Herrsching am Ammersee, Germany) consisted of 30 EEG electrodes placed according to the international 10-5 system (AFp1, AFp2, AFF1h, AFF2h, AFF5h, AFF6h, F3, F4, F7, F8, FCC3h, FCC4h, FCC5h, FCC6h, T7, T8, Cz, CCP3h, CCP, CCP5h, CCP6h, Pz, P3, P4, P7, P8, PPO1h, PPO2h, POO1, POO2, and Fz) was used.

Original dataset includes also electrooculogram (EOG), electrocardiogram (ECG), respiration with a piezo based breathing belt and NIRS data collected from subjects. But we have used only EEG signals in our experiments.

EEG signals were recorded at a sampling rate of 1000 Hz and downsampled to 200 Hz.

Data preprocessing

Signals from eight electrodes (FCC3h, FCC4h, FCC5h, FCC6h, CCP3h, CCP4h, CCP5h, CCP6h) corresponding to the motor areas of the motor cortex are filtered in the alpha and beta frequency bands. The 5th order Butterworth bandpass filter is used in the alpha (8-12 Hz), lower-alpha (7-10 Hz), upper alpha (11-13 Hz), beta [START_REF] Fang | Extending CSP to Detect Motor Imagery in a Four-class BCI[END_REF][START_REF] Wang | Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface[END_REF][START_REF] Blankertz | Optimizing spatial filters for robust EEG single-trial analysis [revealing tricks of the trade[END_REF][START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF][START_REF] Barachant | Riemannian geometry applied to BCI classification[END_REF][START_REF] Congedo | A New Generation of Brain-Computer Interface Based on Riemannian Geometry[END_REF][START_REF] Barachant | Multiclass Brain-Computer Interface Classification by Riemannian Geometry[END_REF][START_REF] Roy | Deep learning-based electroencephalography analysis: a systematic review[END_REF][START_REF] Craik | Deep learning for electroencephalogram (EEG) classification tasks: a review[END_REF], and alpha + beta (7-30 Hz) ranges.

Feature extraction

For this dataset ERD/ERS feature extraction method was used (see section 2.2.2.1). Based on the obtained filtered signals, a set of features based on event-related desynchronization (ERD) and synchronization (ERS) was calculated [START_REF] Pfurtscheller | Event-related EEG/MEG synchronization and desynchronization: basic principles[END_REF].

Firstly, we filtered signals from 8 electrodes (described earlier in section 4.1.5) in different bands (four frequency bands were used to obtain ERD features and two frequency bands were used to obtain ERS features).

To obtain ERD features, signals from eight electrodes (described earlier in section 4.1.5) were filtered in (7-30 Hz), (7-10 Hz), [START_REF] Torres | EEG-Based BCI Emotion Recognition: A Survey[END_REF][START_REF] Maby | BCI Could Make Old Two-Player Games Even More Fun: A Proof of Concept with "Connect Four[END_REF][START_REF] Martišius | A Prototype SSVEP Based Real Time BCI Gaming System[END_REF], and (8-12 Hz) frequency bands for the time window from 0.5 to 5.5 sec. ERS features are obtained from the cropped signals in another time window (7-12 sec) using (7-30 Hz) and (17-25 Hz) frequency ranges.

These features describe processes that occur in localized areas of the motor cortex of the brain [START_REF] Pfurtscheller | Patterns of cortical activation during planning of voluntary movement[END_REF][START_REF] Klimesch | Event-related desynchronization, erd-mapping and hemispheric differences for words and numbers[END_REF]. Thereby using 8 electrodes and 6 filters 48 features were obtained.

Cross-validation

We have adopted a subject-specific approach for which data from each subject are investigated separately. In this experiment, we used two loops of cross-validation.

For the outer loop, we used the One-Session-Out approach. And for the inner loop of cross-validation, we used the Leave-One-Out approach. An inner loop was used to select the best parameter of saturation data with outliers.

Subject-specific One-Session-Out approach

For the cross-validation, it was chosen subject-specific One-Session-Out approach which is the variation of the K-Fold cross-validation with K equals to the number of sessions in the experiment. This approach is close to real conditions as it does not allow data leakage between different sessions.

Data are splitted into test and training sets as follows: two sessions for training, one session for testing.

Summarizing training set consisted of 40 observations for each subject, while the test set consisted of 20 observations. Three round of cross-validation was performed and then results were averaged.

Subject-specific Leave-One-Out approach

Using this cross-validation strategy, one observation remains for the single-item test set, and all the others are used for training. Therefore, the number of folds using Leave-One-Out cross-validation equals the number of observations in the data set.

Motor imagery 8-class database

This is a database that consisted of 7 different MI tasks and resting state. All the data was recorded by members of the team NeuroRhythms (Inria/LORIA) during the PhD thesis of Cecilia Lindig-Léon [START_REF] León | Classification multilabels à partir de signaux EEG d'imaginations motrices combinées : application au contrôle 3D d'un bras robotique[END_REF]. All the experiments were carried out with the consent agreement (approved by the ethical committee of INRIA) of each participant and following the statements of the WMA Declaration of Helsinki on ethical principles for medical research involving human subjects [START_REF] Association | World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects[END_REF].

Participants

Ten right-handed healthy volunteer subjects took part in this experiment (7 men and 3 women, 19 to 43 years old). They had no medical history which could have influenced the experiment.

Paradigm and time scheme

Subjects were seated in a comfortable chair with the arms at their sides in front of a screen showing the task cue to be performed, which consisted of one of the seven kinesthetics motor imageries that it is possible to generate with all combinations including the use of the right hand, left hand, both feet together, and rest (i.e., right hand; left hand; feet; both hands together; left hand in combination with feet; the right hand in combination with feet; both hands together with feet, and rest). The whole session consisted of 4 runs, containing each one 10 trials per task, which totals 40 trials per class (320 trials considering the 8 classes).

For the stimulation, three panels were simultaneously displayed on the screen, which was associated from left to right, to the left hand, feet, and right hand. Each trial was randomly presented and lasted for 12 seconds, starting at second 0 with a cross at the center of each panel and an overlaid arrow indicating for the next 6 seconds the task to be performed: an arrow pointing to the left side on the left panel for the left hand, an arrow pointing down on the central panel for feet, an arrow pointing to the right side on the right panel for the right hand, and the simultaneous combination of all of them for the corresponding combined tasks (see Figure 4.2. The rest condition was also considered and it was indicated by the absence of arrows. After the second 6, the task cue disappeared and the crosses were remaining for the next 6 seconds indicating the pause period before the next trial started.

Electroencephalographic data

EEG channels layout

For this experiment EEG cap was fitted with 26 passive electrodes, namely Fp1; Fp z ; Fp2; F z ; FC5; FC3; FC1; FC z ; FC2; FC4; FC6; C5; C3; C1; C z ; C2; C4; C6; CP5; CP3;CP1; CP z ; CP2; CP4; CP6 and P z .

Such electrodes were placed by using the international 10-10 system positions to cover the primary sensorimotor cortex.

Data preprocessing

Firstly, EEG signals were re-referenced with respect to the common average reference across all channels. Trials were considered 2 seconds before the task cue appeared on the screen and 6 seconds after it disappeared so that each segment corresponds to a 14s-length signal segment over 26 electrodes.

Raw EEG signals were filtered by 5th order Butterworth band-pass filter between alpha, beta, and alpha+beta bands.

Classification using multi-label approach 4.2.6.1 Feature extraction

To evaluate more precisely the modulation which appeared during the two different time windows, we computed the ERD/ERS using the "band power method" [START_REF] Pfurtscheller | Event-related EEG/MEG synchronization and desynchronization: basic principles[END_REF] (see Figure 5.4, Figure 5.5 and Figure 5.6).

From each trial, 26 features (one per channel) have been extracted from the filtered signal computing the logarithm of the variance of the amplitude values between 0.5 and 3.5s after the GO signal.

Linear Discriminant Analysis (LDA) [START_REF] Friedman | Regularized discriminant analysis[END_REF] (see section 2.5.1) has been used via the Python library scikit-learn [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF].

Cross-validation

For the cross-validation, it was chosen the One-Session-Out approach as well as described in section 4.1.7. Due to the small number of samples per class, cross-validation has been done using 3 runs for train and 1 run for test. Thus it was used 240 trials for the train and 80 trials for the test in every step of the cross-validation.

Multi-label classification using 3 binary classifiers 4.2.7.1 Feature extraction and classification

As we planned to detect electrical activity in the areas of the primary motor cortex, only 3 channels of EEG (C3, C4, Cz) were chosen for this experiment. Channel C3 corresponds to the right hand movement/imagery, C4left hand, Cz -feet.

Six sets of features were investigated:

• 3-feature set (ERD): alpha+beta band (7-30 Hz) for 3 channels of EEG described above;

• 3-feature set (ERS): alpha+beta band (7-30 Hz); For all filtered signals ERD/ERS and logarithmic ERD/ERS were calculated. ERD is observed during and immediately after the motor imagery task and better represented in the alpha band, while ERS is observed after the task and better expressed in the beta band. That's why features filtered in alpha bands (ERD) were calculated during 0.5-6 seconds of each trial, and features filtered in beta bands (ERS) were calculated from 7 to 12 sec.

As a classifiers were used following algorithms:

• Shrinkage Linear Discriminant Analysis (see section 2.5.1);

• Support Vector Machine with linear, rbf, sigmoid and polynomial kernels (see section 2.5.2) ;

• Decision Tree Classifier (see section 2.5.3).

Cross-validation

For the cross-validation, it was chosen the One-Session-Out approach as well as described in section 4.1.7. Cross-validation was implemented using 3 runs for train and 1 run for test. To create the training set we used the 'One vs. All' approach, which means that we labeled one class as '1' and the other three classes as '0' for each of three classifiers:

• classifier to detect right hand activity (right hand vs. rest & left hand & feet).

• classifier to detect left hand activity (left hand vs. rest & right hand & feet).

• classifier to detect feet activity (feet vs. rest & left hand & right hand).

Each training set had the same number of observations of each of the classes in every step of cross-validation (for example 30 samples of right hand imageries vs. 10 rest + 10 left hand + 10 feet).

The test set consists of 1 run of each of the 8 classes, it equals 80 trials (10 per class).

Median nerve stimulation database

A median nerve stimulation database is used in our investigation of motor intention detection with deep learning to prevent intraoperative awareness.

This database consisted of median nerve stimulations during motor imagery and during rest, as well as MI condition separately. All the data was recorded by members of the team NeuroRhythms (Inria/LORIA) Pierre Riff and Sébastien Rimbert during the PhD thesis of Sébastien Rimbert [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF].

This experiment followed the statements of the WMA Declaration of Helsinki on ethical principles for medical research involving human subjects [START_REF] Association | World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects[END_REF]. Besides, participants signed an informed consent which was approved by the ethical committee of Inria (COERLE, approval number: 2016-011/01) as it satisfied the ethical rules and principles of the institute.

The aim of the experiments with this database is to detect the occurrence of motor patterns under two different conditions: median nerve stimulation during motor imagery (MI + MNS) and median nerve stimulation during rest (MNS) (see Figure 4.3). These two conditions aim at showing that a MNS can be used and is more helpful as a trigger to improve the detection of intraoperative awareness. The reason for using MNS is its ability to modulate the motor cortex activity by first generating an ERD during stimulation and then an ERS post-stimulation, which resulted in a higher accuracy of classification [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF] (see section 1.3.4).

Participants

Twenty-two right-handed healthy volunteers (12 females; 19 to 57 years old; 28.56 ± 13.3 years old) were recruited for this study. All subjects satisfied the inclusion criteria (right-handed, between 18-60 years old, without medical history which could have influenced the task), such as diabetes, antidepressant treatment or neurological disorders [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF].

Paradigm and time scheme

During EEG recordings (Figure 4.3), the subject was lying on a comfortable chair with eyes closed. Their legs rest on a footrest and the right forearm rests on a cushion to prevent movement. The subject mentally presses and releases a remote button. The operator monitors the EEG signals during the experiment.

Participants performed each condition randomly during two runs of 26 trials per condition. So in total, the database consists of 52 recordings of each class for every subject.

As an additional inclusion criterion for participants, we considered the fact that the stimulation was not felt as painful and that it caused a slight movement between the thumb and the index finger. The stimulation intensity was adapted for each subject and varied between 8 and 15 mA.

Condition 1: motor imagery

For the MI condition (C1), subjects had to imagine an isometric grasp between the thumb and the index finger on a pointer button, i.e., they had to try to feel a maximum of sensations caused by the real movement, but without any movement. A low-frequency beep indicated when the subject had to start the motor imagery, the grasping MI was maintained during 2 s, then a second beep indicated the end of the imagined task [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF] (Figure 4.4).

Condition 2: motor imagery with a median nerve stimulation

During the MI + MNS condition (C2), subjects had to perform a motor imagination while their median nerve was stimulated at 750 milliseconds after the start of the motor imaginary task (Figure 4.4). A low-frequency beep was indicated when the subject had to start the motor imagery and a high-frequency beep indicated when to stop it. We chose the 750 milliseconds of delay according to the reaction time of the average person, to stimulate during the ERD corresponding to the imagination started. The duration of stimulation was 100 ms [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF].

Condition 3: median nerve stimulation only

The MNS only condition (C3) consisted of a series of stimulation of the median nerve during rest (Figure 4.4). We placed two stimulating electrodes on the wrist according to the standards [START_REF] Schnitzler | Involvement of primary motor cortex in motor imagery: a neuromagnetic study[END_REF][START_REF] Kumbhare | Buschbacher's Manual of Nerve Conduction Studies, 3rd Edition, ch. Mediane nerve to the abductor pellicis brevis[END_REF] and recorded the EEG data from subjects [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF]. . For all motor tasks, one low-frequency beep indicates when to start the task. For the MNS+MI condition, the MNS occurs at 750 milliseconds after the first beep. The end of the MI is announced by a high-frequency beep and followed by a rest period of 6 seconds (adopted from [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF]).

Electroencephalographic data

EEG signals were acquired using the OpenViBE platform [START_REF] Renard | Openvibe: An open-source software platform to design, test and use brain-computer interfaces in real and virtual environments[END_REF] with a BioSemi Active Two 128-channel EEG system, arranged in the BioSemi's ABC system covering the entire scalp (Figure 4.5) at 2048Hz. Among all registered sites, some of the electrodes were localized around primary motor cortex, motor cortex, somatosensory cortex, and occipital cortex (Figure 4.5), which allowed to observe the physiological changes due to kinesthetic motor imagery and median nerve stimulation [START_REF] Filgueiras | The neural basis of kinesthetic and visual imagery in sports: an ale meta-analysis[END_REF][START_REF] Guillot | Brain activity during visual versus kinesthetic imagery: an FMRI study[END_REF][START_REF] Schnitzler | Involvement of primary motor cortex in motor imagery: a neuromagnetic study[END_REF][START_REF] Salenius | Modulation of human cortical rolandic rhythms during natural sensorimotor tasks[END_REF].

In BioSemi TM system the ground electrodes used were two separate electrodes: Common Mode Sense (CMS) active electrode and Driven Right Leg (DRL) passive electrode located over the parietal lobe.

Impedance was kept below 10kΩ for all electrodes to ensure that the background noise in the acquired signal was low.

An external electromyogram (EMG) electrode was added to verify that there was no movement during the MI task.

EEG channel layouts

Initially 128 EEG channels were registered with BioSemi's ABC layout (Figure 4.5). The recordings with a reduced number of channels were constructed to investigate the possible influence of using a smaller number of electrodes on the motor imagery detection performance.

We also investigated additional channels from the frontal lobe as an interesting area for our experiment because it is considered as the emotional controller center and also responsible for motor functions, motor speech functions and impulse control [START_REF] Collins | Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making[END_REF].

The following electrode setups covering mostly brain areas responsible for movements were tested (see Figure 4.6):

• 128 channels (all channels as present on Figure 4.5);

• 47 channels (Figure 4.6a) covered mainly motor cortex area (A1, A2, A3, B1, B2, B15, B16, B17, B18, B19, B20, B21, B22, B23, B24, B25, B28, B29, B30, B31, B32, C1, C2, C23, C24, C22, C11, D1, D2, D9, D10, D11, D12, D13, D14, D15, D16, D17, D18, D19, D20, D21, D22, D25, D26, D27, D28);

• 13 channels (Figure 4.6b) over motor cortex (D12, C23, B31, D21, D19, D14, A1, B20, B22, B24, D28, A3, B18);

• 13 channels (Figure 4.6c) over motor cortex and frontal lobe (C29, C17, C16, D11, D13, D19, D27, D17, B32, B30, B22, B19, B17); • 3 channels over frontal lobe (C29, C17, C16)

• 3 channels over only left side of frontal lobe (C30, C29, C17)

Data preprocessing

After selecting a certain channel layout, EEG signals were re-referenced to a common average reference, then EEG signals were resampled at 128 Hz. While power modulations during motor activity are observed mainly in mu, alpha and beta bands of an electroencephalographic signal measured over the sensorimotor area of the brain [START_REF] Ramoser | Optimal spatial filtering of single trial eeg during imagined hand movement[END_REF][START_REF] Pfurtscheller | On the existence of different types of central beta rhythms below 30 hz[END_REF][START_REF] Neuper | Handbook of electroencephalography and clinical neurophysiology. Eventrelated desynchronization[END_REF] sampling rate of 128 Hz is enough to represent mu, beta, and low gamma frequency range according to the Nyquist-Shannon-Kotelnikov sampling theorem.

Then each signal was filtered in a specific frequency band using a 4th-order Butterworth band-pass filter [START_REF] Butterworth | On the theory of filter amplifiers[END_REF] and divided into trials of 3 s duration starting at 250 ms after the first beep (Figure 4.4). This corresponds to 384 samples (3 s at 128 Hz) per each channel.

We investigated filtering in few frequency bands to see is it efficient enough to add theta (4-8 Hz) or low gamma (30-38 Hz) range to standard mu+beta (8-30 Hz) frequency band especially for MNS vs. MI+MNS discriminating task because it has not been studied before. Thereby we used the following bands: theta-beta (4-30 Hz), standard mu-beta (8-30 Hz), mu-gamma (8-38 Hz), and theta-gamma (4-38 Hz) as was observed in the section 3.5. We did not apply any feature extraction technique to these data. So filtered EEG signals were directly fed to the deep learning architectures as an input.

Conclusion

In this chapter three different datasets were introduced that were used in our experiments: motor imagery 2-class dataset, motor imagery 8-class dataset containing combined movements, and median nerve stimulation dataset.

Participants, experimental tasks, time schemes, EEG data and channel layouts, preprocessing, feature extraction, classification approaches (if exists), and cross-validation strategies were described for 3 datasets.

Results of investigations on these datasets will be introduced in the following chapter 5. Firstly, we will represent results of outlier detection as quality index of motor imagery estimation in section 5.1 on 2-class motor imagery dataset introduced in section 4.1. Secondly, we will use data from the 8-class motor imagery dataset described in section 4.2 to improve combined movement motor imagery classification. We will show results for the classification of combined movement motor imageries using binary classifiers in section 5.2.2. Also, we will introduce results of feature selection for combined movement MI classification on the 8-class motor imagery dataset in section 5.2.3. Finally, we will represent results for motor imagery detection using deep learning to prevent intraoperative awareness in section 5.3 on the dataset introduced in section 4.3.

Chapter 5

Experimental results In previous chapters, we have presented materials and methods for motor imagery detection from raw EEG signals.

More precisely, in chapter 1, we have introduced the background to BCIs, EEG signals, brain rhythms, and motor imageries as the most popular paradigm in the MI-based BCI domain. Also, we have presented the concept of BCI for accidental awareness during general anesthesia based on median nerve stimulation as one of the possible applications of BCI.

In chapter 2, we have introduced machine learning techniques for motor imagery detection. We have presented different stages of EEG signal processing, such as filtering, feature extraction, feature selection, outlier detection, and classification. Also, we have proposed new methods to improve motor imagery classification in this chapter (see section 2.4.1 and section 2.5.6).

In chapter 3, we have represented background to deep neural networks as a combined solution for feature extraction and classification from raw EEG signals. We have reviewed the application of deep learning for motor imagery detection and proposed convolutional neural network architectures for implementation.

In chapter 4, we have presented three different datasets as well as details of preprocessing, feature extraction, classification, and validation techniques used for these data.

In this chapter, we will present a series of results on different parts of the BCI processing. Each section presents results on the improvement for motor imagery detection.

Results of machine learning and deep learning application will be presented through this chapter for three different tasks:

• Outlier detection method presented in section 2.4.1 will be applied for motor imagery quality estimation prior training a BCI to improve accuracy (section 5.1);

• Application of three parallel-connected binary classifiers presented in section 2.5.6 for combined movement MI classification improvement (section 5.2);

• Deep learning methods presented in chapter 3 will be applied to median nerve stimulation-based BCI for the task of intraoperative awareness prevention (section 5.3).

Improving a BCI using outlier detection as quality index of motor imagery estimation

In this section, we propose to detect outliers in the data and exclude them before training. We use the method presented in section 2.4.1 for outlier detection to analyze the quality of data [START_REF] Avilov | Estimation of imaginary movements quality based on machine learning for brain computer interface applications[END_REF].

We use 2-class motor imagery dataset described previously in section 4.1. Dataset consists of EEG signals from 29 subject performing the right hand and the left hand motor imagery.

We use 48 ERD/ERS features from different channels filtered in different bands (described in section 4.1.6). Then, we apply outlier detection techniques on the feature set to find outliers in the training set and remove them using different saturation parameter separately for each class. We calculate classification accuracy using logistic regression for each parameter of saturation to choose outlier detectors with the best parameter of saturation and use them to clean test set. After removing the outliers, classifiers predict classes on the cleared test set.

As cross-validation, we have applied a subject-specific strategy with two loops: the One-Session-Out approach for the outer loop and the Leave-One-Out approach for the inner loop. An inner loop was used to select the best parameter of saturation data with outliers.

Classification accuracies obtained using the model described in section 2. As we can see from the represented data the average accuracy across all subjects remained at the same level for both methods. However, accuracy increased for 10 subjects (+11.3% in average) using isolation forest method (Figure 5.2) and for 9 subjects (+9.6% in average) using one-class SVM method (Figure 5.3). This suggests that removing outliers from the training data helps to build more accurate classification models. But this approach is subject-specific and can be successfully applied to a specific group of subjects (see section 6.1). At the same time, the method of outlier detection in the initial data has to be improved.

Improving combined movement motor imagery classification using machine learning

In this section, data from the database described in section 4.2 is used. Data consists of 7 different MI tasks (right hand; left hand; feet; both hands together; left hand in combination with feet; the right hand in combination with feet; both hands together with feet) and resting-state recorded from 10 right-handed healthy volunteer subjects.

Each subject performed 4 sessions of motor imageries contains 10 trials for each class. Thus, 40 trials per class are available for each subject.

Figure 5.4 shows the time course of ERD/ERS over electrode C3 (i.e, the activity source associated with the use of the right hand). We can observe that all motor imageries involving the use of the right hand (i.e., right hand, right hand in combination with feet, both hands, and both hands in combination with feet) present considerably lower ERD/ERS values than those that do not include it (i.e., rest, left hand, feet, and left hand in combination with feet).

Similarly, Figure 5.5 presents the time course of ERD/ERS over electrode C z (i.e, the activity source associated with the use of the feet). Over this region, it can also be observed that the motor imageries involving the use of the feet (i.e., feet, left hand in combination with feet, right hand in combination with feet, and both hands in combination with feet) present lower ERD/ERS values than those that do no it include it (i.e., rest, left hand, right hand, and both hands). However, in comparison to the contrast observed over electrode C3 between the tasks involving the use of the right hand and those that do not involve it, differences are not as visible. In fact, we can observe that the use of both left and right hands induce some desynchronization over the central region, possibly because the associated sources are quite close to electrode C z , which might cause it to incorporate some of this activity.

Finally, the same comparison is illustrated in Figure 5.6 for the time course of ERD/ERS generated over electrode C4 (i.e, the activity source linked to the use of the left hand). In this case, similarly to the analysis over electrode C3, it is possible to observe a significant contrast between the low ERD/ERS values obtained from the activity induced by the motor imageries involving the use of the left hand (i.e., left hand, left hand in combination with feet, both hands, and both hands in combination with feet) from the high ERD/ERS values associated to the tasks that do not involve the use of the left hand (i.e., rest, feet, right hand, and right hand in combination with feet).

Combined movement MI classification using classic methods

Table 5.3 reports classification accuracy for 8 classes obtained by a standard shrinkage LDA for 10 subjects. These results were obtained using all available data from 8 classes for train according to One-Session-Out crossvalidation approach. From each trial, 26 ERD/ERS features (one per channel) have been extracted from the filtered (8-30 Hz) EEG signal.

The mean accuracy of 32.7% is quite low. However, for two subjects (S2 and S10) there is higher accuracy compared with other subjects. It is interesting to see that the best subject in term of accuracy had also the powerful ERD during the MI-task (Figure 5.4, Figure 5.5 and Figure 5.6). All of the above once again confirms the fact that this is a subject-specific task. In the next section, we will discuss possible solutions to improve the classification accuracy. & Feet → Both hand). One explanation could be that the activity of the right hand (respectively the left hand and both hands) goes over the Cz location. The accuracy obtained by a shrinkage linear discriminant analysis is not good enough to allow devises control with combined movements. Further work will be focused on building the ensemble of binary classifiers (section 5.2.2) to save time on BCI training and feature selection for better condition separation (section 5.2.3). Then one-versus-one approach, one-versus-the-rest approach, or a hierarchical approach should improve the detection of combined motor imageries.

Classification of combined movement motor imageries using binary classifiers

The aim of this experiment was to investigate the performance of the classification using 3 parallel connected binary classifiers instead of using a multi-label classifier. The idea was to train 3 binary classifiers on data with single motor imageries and then test it on the set which contains all possible combinations of these movements and rest [START_REF] Avilov | Combined imaginary movements classification using binary classifiers in brain-computer interfaces[END_REF]. It is useful because it can help to save a lot of time on recording the EEG while collecting data for MI classification. Instead of recording all combinations of combined movements, it can permit to collect of the sets only with single movements.

In this section different methods of supervised learning were used to estimate the performance of our approach with 3 binary classifiers in the task of the multi-label combined movement classification.

The following algorithms of supervised learning were selected: SVM with different kernels (section 2.5.2), LDA with shrinkage regularization (section 2.5.1) and Decision Tree Classifier (section 2.5.3).

Investigation was conducted with six sets of features: 3-feature set (ERD) (table 5.5), 3-feature set (ERS) (table 5.6), 6-feature set (ERD & ERS on alpha+beta band) (table 5.7), 6-feature set (ERD on alpha and ERS on beta band) (table 5.8), 18-feature set (table 5.9) and 30-feature set (table 5.10) (described in section 4.2.7).

Average accuracies and standard deviation were calculated for all subjects with leave-one-session-out crossvalidation (see section 4.2.6.2). SVM with sigmoid and polynomial kernels are not presented due to low-level performance. As we can see from the tables 5.5 -5.10 the best performance is demonstrated by linear classifiers (Linear SVM and LDA). Average accuracies of combined MI movements classification for different feature sets are represented in Table 5.11 to simplify comparison. But this level of accuracy is lower than the standard multi-label approach. An important moment is that level of classification of the "rest" class is very low, although this class was known to the classifier. It can suggest that the "rest" class was confused with the "feet" class, as was presented in Table 5.4. Also, the accuracy of the "both hands" classification is higher than the accuracy of other unknown classes. But in general, we have a good result on known classes and bad performance on the unknown classes (accuracy of the "both hands and feet" classification even equals to 0.8-2.5% in the 30-feature set).

It is also showed that with an increasing number of different bands the accuracy also increases (26.1% for LDA classifiers on 3-ERD feature set (see Table 5.5) increased to 32.1% for the 30-feature set (see Table 5.10)). But it is doubtful that all features are equally useful for the classification. It is also known that with a small number of data examples, a large number of features may lead to model overfitting, which results in low accuracy on the validation data. That's why using feature selection techniques (see section 2.3) could be promising to improve model performance. Results obtained using feature selection techniques are represented in section 5.2.3.

Feature selection for combined movement MI classification

Observing modulation in a wide frequency band such as the alpha+beta bands together (8-30 Hz) ensures having desynchronization and synchronization. In this section, we investigated features to know the variability between bands. It can help to understand which bands and features are more useful.

For this task meta-transformer, SelectFromModel was used. This algorithm was realized in the Python library Scikit-learn. More information about this algorithm and its parameters see in section 2.3.

We test this algorithm on 30-feature set (see section 4.2.7) with linear SVM (C = 0.01, L1 penalty, threshold = 10 -5 ) for each of 3 binary classifiers for all subjects. Results shows how frequently features were marked as important by classifier (maximum mark is 40 -it equals 10 subjects with 4-fold cross-validation). Results are represented in the tables 5.12-5.14. From the results given in the Tables 5.12 and 5.13 we can see that for the separation of left and right hand activity from other activities alpha ERD and wide beta ERS are useful, and other ERS features can be removed. As for the feet activity detection we have to use ERS beta (17-25 Hz, 7-30 Hz) features on Cz and ERD alpha band feature (8-12 Hz) on C3 and C4 to separate feet activity from others.

To see variability between these features histograms of distribution logarithmic ERD/ERS depending on the state (class) also were created. Histograms are represented in Figures 5. [START_REF] Oum | MindTactics: A Brain Computer Interface gaming platform[END_REF]5.16. Figures 5.15,5.16 shows that separation is not clear even for the features, which was chosen by feature selector. The best separation is in the low-alpha ERD. But even here overlapping between "feet" and "rest" classes exists. The same could be seen on the 3D plot of the sample distribution (Figure 5.7a). Interesting thing that ERD feature on 8-12 Hz band (Figure 5.7c) have much better separation of classes than both ERD on 8-10 (Figure 5.7a) and 11-13 Hz and even than 7-13 Hz (Figure 5.7b). But the separation of 'rest' and 'feet' classes is still very bad. It also could be seen on histograms (table 5.15). Another useful feature could be ERD/ERS in the alpha+beta band (8-30 Hz). As we can see, using features from this band separation of all classes is not perfect, but the separation of the 'rest' and 'feet' classes are much better than in other features.

In this investigation, we also computed accuracies after feature selection for the 30-feature set (Table 5.17). We can see that after feature selection (Table 5.17) classification accuracy is higher. But at the same time, we have not a significant difference between results.

Furthermore, the best accuracy obtained using binary classifies with feature selection (33.2% + -1%) is even higher than for standard multilabel approach (32.7% + -4.2%) (see Table 5.3).

Optimizing motor imagery detection with deep learning to prevent intraoperative awareness

In this section, we have presented results obtained on the dataset introduced in section 4.3. This database consisted of median nerve stimulations during motor imagery and during rest, as well as MI condition separately. EEG signals were recorded from 22 right-handed healthy volunteers.

The goal of this experiment is to improve and optimize motor intention detection using different deep learning architectures, frequency bands, electrode layouts, and classification techniques. Also, the aim of the experiments with this database is to detect the occurrence of motor patterns under two different conditions: median nerve stimulation during motor imagery (MI + MNS) and median nerve stimulation during rest (MNS). These two conditions aim at showing that a MNS can be used and is more helpful as a trigger to improve motor imagery detection (see section 1.3.4).

The standard machine learning classification methods (such as CSP+LDA, MDRM and TS+LR) were used for MI-based BCI described in chapter 2, as well as CNN architectures described in chapter 3.

We selected three state-of-the-art deep neural network architectures (such as D-ConvNet, S-ConvNet, EEGNet) to compare their results to the best-known approaches for classification in the MI-based BCI domain.

For feature extraction we used two efficient methods: CSP filters [START_REF] Fang | Extending CSP to Detect Motor Imagery in a Four-class BCI[END_REF][START_REF] Blankertz | Optimizing spatial filters for robust EEG single-trial analysis [revealing tricks of the trade[END_REF][START_REF] Wang | Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface[END_REF] (see section 2.2.3.2) and Riemannian geometry [START_REF] Barachant | Riemannian geometry applied to BCI classification[END_REF][START_REF] Barachant | Multiclass Brain-Computer Interface Classification by Riemannian Geometry[END_REF][START_REF] Congedo | A New Generation of Brain-Computer Interface Based on Riemannian Geometry[END_REF] (see section 2.2.4).

For classification we used first a linear discriminant analysis (LDA) [START_REF] Vidaurre | Towards a Cure for BCI Illiteracy[END_REF] (see section 2.5.1). We also selected approaches based on Riemannian geometry: MDRM algorithm [START_REF] Barachant | Riemannian geometry applied to BCI classification[END_REF][START_REF] Congedo | A New Generation of Brain-Computer Interface Based on Riemannian Geometry[END_REF][START_REF] Barachant | Multiclass Brain-Computer Interface Classification by Riemannian Geometry[END_REF] (see section 2.5.5), tangent space projection of covariance matrices with following logistic regression classifier application on projected data (TS+LR) as described in [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF][START_REF] Barachant | Riemannian geometry applied to BCI classification[END_REF] (see section 2.2.4.3).

Our motivation to use convolutional neural networks is to not predefine features and expect better accuracy. We used i) a deep convolutional network (D-ConvNet) as reference for deep networks (section 3.6.1), ii) a shallow convolutional network (S-ConvNet) because they are designed specifically for oscillatory signal classification (section 3.6.2), and iii) a generic architecture (EEGNet) proposed in [START_REF] Lawhern | EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[END_REF] for EEG data that already demonstrated good results on several BCI paradigms (section 3.6.3); iv) adaptation of EEGNet, namely EEGNet-4.8, proposed in the section 3.6.4, that better handle mid-and high frequency information.

We present how deep neural nets architectures can improve classification accuracy for two MI discriminating paradigms: the standard (MI vs. rest) BCI task and the same task when a stimulation of the median nerve is added (MI+MNS vs. MNS). We addressed the following aspects in the experiments:

• using extended frequency ranges with theta and low gamma bands (section 5.3.1);

• using various deep learning architectures and standard classifiers (section 5.3.4);

• EEGNet adaptation to better handle medium frequency components (section 5.3.2);

• dropout parameter optimization (section 5.3.3);

• comparing the two discriminating tasks: MNS vs. MNS+MI and MI vs. rest (section 5.3.5);

• frontal electrodes influence investigation (section 5.3.6);

• comparing different EEG montages with reduced electrodes number.

Frequency band selection

In the first set of experiments, we investigate the influence of the EEG frequency band on the classification accuracy. The recommended frequency band for detecting motor imagery is 8-30 Hz [START_REF] Lotte | Comparison of designs towards a subject-independent braincomputer interface based on motor imagery[END_REF][START_REF] Lotte | Signal processing approaches to minimize or suppress calibration time in oscillatory activity-based brain-computer interfaces[END_REF], but sometimes it could be useful to extend the analysis to wider frequency bands such as [START_REF] Ang | Filter bank common spatial pattern algorithm on bci competition iv datasets 2a and 2b[END_REF][START_REF] Dharamsi | Neurology-as-a-Service for the Developing World[END_REF][START_REF] Yang | On the use of convolutional neural networks and augmented CSP features for multi-class motor imagery of EEG signals classification[END_REF] (see Table 3.1). We investigated separately and together the impact of adding the theta 4-8 Hz and the low gamma 30-38 Hz frequency ranges on the classification performance. We did this study for all models, i.e. CSP+LDA, MDRM, TS+LR, EEGNet, D-ConvNet, and S-ConvNet. Similar behavior was obtained for all models, so we only present the most representative part of it (see section 5.3.4) in Figure 5.11 showing classification accuracies for TS+LR and EEGNet-4.8 models trained on data filtered with different bands for MNS vs. MNS+MI and MI vs. rest tasks. To analyze the differences between MI + MNS and MNS conditions, we performed an event-related spectral perturbation (ERSP) analysis for two different frequency bands (8-30 Hz vs. 4-38 Hz) and separately (4-8 Hz and 30-38 Hz) using EEGLAB. This analysis allows us to understand how MI + MNS and MNS conditions can be discriminated and which time parameters we can choose to guide the classification. We have decided to compute ERSPs in a merged band (mu+beta, 8-30 Hz) and a wider band (4-38 Hz) for MI + MNS and MNS conditions (Figure 5.8). A surrogate permutation test (p < 0.01; 4,000 permutations) from the EEGLAB toolbox was used to validate differences in terms of localization of these ERSPs. In addition to this analysis, we applied a false discovery rate (FDR) correction test (see section 2.8) in order to clarify how the FDR was controlled for multiple comparisons to show the difference between MI + MNS and MNS conditions. As we can see from Figure 5.9 (4-8 Hz) frequency band could be very useful in case of discriminating MNS condition from the MI+MNS. In addition, FDR shows a huge number of electrodes all over the cortex which indicates a significant difference between the two conditions. At the same time Figure 5.10 indicates that (30-38 Hz) frequency band is not so useful for discriminating MNS vs. MI+MNS task.

Classification accuracy obtained by different classifiers for 22 subjects using different electrode layouts and different frequency bands for MNS vs. MI+MNS discriminating task represented in Tables A.1 -A.48.

Wilcoxon signed-rank test shows significant (p -value < 0.01 or higher) difference between EEGNet-4.8 performance on data filtered in 8-30 Hz and 4-38 Hz frequency bands for MNS vs. MI+MNS discriminating task (Figure 5.12). This is the case for 128, 47, 13, 9 and 6-electrode setups. At the same time, for 47, 13, and 6electrode setups the difference was even more pronounced (p -value < 0.001). The advantage of using a 4-38 Hz band can achieve accuracy improvement up to 4% for MNS vs. MI+MNS task. Unlike EEGNet-4.8, the TS+LR model showed a highly significant (p -value < 0.01) difference in accuracy obtained on data filtered in 8-30 Hz and 4-38 Hz frequency bands only for 128 electrode setup, although the average growth in accuracy is around 2%.

Motor imagery vs. rest

Although for the MI vs. rest task the addition of the theta band effects classification accuracy, but no significant difference has been noticed based on Wilcoxon signed-rank test (Figure 5.11d).

Classification accuracy obtained by different classifiers for 22 subjects using different electrode layouts and different frequency bands for MI vs. rest discriminating task represented in Tables B.1 -B.48.

Recommendations

When using MNS it is better to use the wider frequency band 4-38 Hz. At the same time, there are no reasons to change the standard 8-30 Hz band for the Rest vs. MI task.

EEGNet architectures

EEGNet is a very promising architecture [START_REF] Lawhern | EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[END_REF] for spatio-temporal analysis of EEG signals. To better investigate this architecture, many parameters of EEGNet may be adjusted. In the present investigation, we focus on the temporal kernel size K for the first convolutional layer and the multiplier depth D of the depthwise convolution layer which to our best knowledge were not investigated deeply earlier. The accuracy obtained using different architectures of EEGNet for MNS vs. MI+MNS task represented in Figure 5.13a. We can see that decreasing the kernel size of the first convolutional layer and increasing the multiplier depth outperforms the standard set of parameters of EEGNet-2.32 [START_REF] Lawhern | EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[END_REF] for a kernel size of 2 and a multiplier depth of 32. We selected EEGNet-4.8 for future investigations based on its higher classification performance for almost all electrode layouts.

MNS vs. MI+MNS

More precisely comparing EEGNet-4.8 proposed in this work with standard EEGNet-2.32 architecture for sensory-motor data [START_REF] Lawhern | EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[END_REF] we can see that it works especially better for larger setups of electrodes for MNS vs. MI+MNS task (Figure 5.13a). Wilcoxon signed-rank test showed a highly significant (p-value < 0.01) difference between performances of EEGNet-4.8 and EEGNet-2.32 for all setups of electrodes except setup with 3 electrodes over the motor cortex.

Classification accuracy obtained by different variations of EEGNet for 22 subjects using different electrode layouts for MNS vs. MI+MNS discriminating task represented in Tables C.1 -C.12.

Motor imagery vs. rest

Classification accuracy obtained by different variations of EEGNet for 22 subjects using different electrode layouts for MI vs. rest discriminating task represented in Tables D.1 -D.12.

For MI vs. rest task (Figure 5.13b) the difference between EEGNet-2.32 and EEGNet-4.8 is not statistically significant, but the classification accuracy of EEGNet-4.8 is better for most electrode layouts.

Dropout investigation

Also we investigated the influence of dropout (section 3.4.2) on classification performance of EEGNet variants for MNS vs. MNS+MI discriminating task. Results for electrode setup with 128 and 47 electrodes represented in Table 5.18 and 5.19 respectively. It is showed that classification accuracy obtained with dropout rate D = 0.5 is higher for all models in contrast to D = 0.25 proposed in [START_REF] Lawhern | EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[END_REF].

Classification results

In this section classification results obtained by all models will be compared using graphical representations and statistical tests both for MNS vs. MI+MNS task and MI vs. rest task.

MNS vs. MI+MNS task

Classification accuracies obtained by EEGNet and other standard and deep learning models for all electrode layouts for MNS vs. MI+MNS task are presented in Figure 5.14 and Table 5. [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF].

Classification accuracies obtained by different models for 22 subjects using layouts with 128, 47, 13 and 6 electrodes, averaged over all folds (k= 10) for MNS vs. MI+MNS condition presented in Figures 5. 15 -5.18.

Boxplots showing the distribution of different models classification accuracy using layouts with 128, 47, 13 and 6 electrodes, averaged over all folds (k = 10) and all subjects (n = 22) for MNS vs. MI+MNS discriminating task represented in Figures 5. 19-5.22. Significance levels of Wilcoxon signed-rank test comparisons showed in these figures means: NS -not significant, *p < 0.05, **p < 0.01, ***p < 0.001, ****p < 0.0001.

From the results presented in this section we can see that EEGNet-4.8 significantly (p -value < 0.01) outperforms all standard and deep learning models for all electrode setups (Figures 5. 19-5.22), except 3-electrode setups where difference to second best model (EEGNet-2.32) is not significant (Figure 5.14) for MNS vs. MI+MNS task. The second best model for all electrodes setups is also an EEGNet architecture (EEGNet-2.32).

Also from Figure 5.14 we can see that performance of D-ConvNet and S-ConvNet become better when the electrode number decreases from 128 to 47, while it becomes worse for all other models. All deep learning models outperform standard models starting from setups with 47 electrodes and less.

Motor imagery vs. rest

Classification accuracies obtained by EEGNet and other standard and deep learning models for all electrode layouts for MI vs. rest task are presented in Figure 5.23 and Table 5. [START_REF] Barachant | Riemannian geometry applied to BCI classification[END_REF].

We can see that TS+LR outperforms other models for 128 and 47 electrodes for MI vs. rest task (Figure 5.23) but then performance decreases a lot when decreasing number of electrodes. At the same time, deep learning models are more stable to decreasing of electrodes number. And we even observe that a decreasing number of electrodes leads to increasing classification performance for all deep learning models with a peak at 9 electrode setup.

Statistical tests show that EEGNet-4.8 significantly (p-value < 0.05) outperforms TS+LR (and other standard methods) for 9 and 6 electrode setups.

Comparing paradigms

Classification performance obtained by EEGNet and other standard and deep learning models for several electrode layouts for MNS vs. MI+MNS and MI vs. rest tasks could be compared in Figure 5.24. The difference between the results of MNS vs. MI+MNS and MI vs. rest paradigms was demonstrated earlier in [START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF]. But previous investigations were done only for 128 electrode setup. In this work, we also present it for a smaller number of electrodes. Boxplots with classification accuracy distribution for EEGNet-4.8 model for setups with 13, 9, 6 electrodes located over motor cortex are shown in Figure 5.25. Statistical tests indicate highly significant (p < 0.001) difference between EEGNet-4.8 accuracies of both tasks for 13, 9 and 6 electrodes.

Frontal electrodes investigation

To explore the opportunities of including the EEG activity from different functional areas of the brain, we investigated setups with electrodes not only from motor cortex but also adding electrodes from frontal lobe (Figures 4.6c, 4.6e, 4.6g) because frontal electrodes are used in many monitoring systems to determine the depth of anesthesia [START_REF] Purdon | Clinical Electroencephalography for Anesthesiologists Part I: Background and Basic Signatures[END_REF].

Figure 5.26 shows classification performance of models for MI + MNS vs MNS discriminating task on the data from 13, 9, and 6 electrodes over the motor cortex and frontal lobe (distinguished by prefix +f r in the chart), and motor cortex only.

Results show that the addition of frontal lobe electrodes increases accuracy almost for all models and all numbers of electrodes. Wilcoxon signed-rank tests confirm significant differences between setups with and without frontal lobe electrodes for many models. For example, the difference is significant (p -value < 0.05) for TS+LR and EEGNet-2.32 for setups with 9 and 6 electrodes. Moreover, it is highly significant for the TS+LR model using 6 electrodes. At the same time, statistical tests show that there is no significant difference between setups with and without frontal lobe electrodes for EEGNet-4.8.

Conclusion

In this chapter results for 3 different machine and deep learning applications to improve BCI performance are represented.

First, it was presented an approach of using outlier detection to estimate the quality of motor imageries prior to BCI training. Results showed that this approach is subject-specific and increases classification accuracy in BCI for a group of subjects without significant average accuracy improvement.

Secondly, the enhancement of combined movement motor imagery classification was studied. Method using 3 binary classifiers was proposed for combined MI movement classification mainly to reduce the time for signal acquisition, training, and calibration of BCI. Using this approach together with the feature extraction method it is possible to achieve the same results as using the standard multi-class approach. Add here why this study and result is useful for mankind and science, anyway.

In this chapter, we have investigated motor imagery detection using deep learning methods. During this study, two different discriminating tasks were considered: median nerve stimulation (MNS) vs. MI+MNS and MI vs. rest as a standard task for the BCI domain. It is a complex study that includes several components:

• frequency band selection, where using wider (4-38 Hz) frequency band was proposed instead of standard (8-30 Hz) band. It was showed that using a wider (4-38 Hz) band allows to significantly increase classification performance;

• EEGNet architectures investigation, where lengths of temporal kernel size for the first convolutional layer and multiplier depth, which determines the number of spatial filters of the depthwise convolution layer were studied. The parameter of the dropout rate for the model also was investigated. EEGNet adaptation, namely EEGNet-4.8 showed the best performance for different electrode layouts for both tasks. Results of EEGNet-4.8 classification are significantly higher than any other classical and deep learning models;

• classification performance for two paradigms MNS vs. MI+MNS and MI vs. rest were compared. It was showed that using MNS vs. MI+MNS discriminating task allows achieving significantly higher classification accuracy than standard MI vs. rest for all investigated electrode layouts.

• frontal electrode investigation, where it was showed that electrode layouts with electrodes over the frontal lobe together with electrodes over the motor cortex allows achieving significant classification accuracy improvement comparing with electrode layouts without frontal electrodes;

• electrode layout investigation where 12 setups with different channel numbers and localization were studied. Results showed that it is possible to achieve high classification performance using a reduced number of electrodes (84.0% using just 6 electrodes over one side of the motor cortex and frontal lobe).

All of the above allows us to discuss the application of deep learning to MNS-based BCI for intraoperative awareness prevention in the next chapter. In previous chapters, we have proposed and considered three different ways of improving motor imagery detection in the BCI domain on tree different datasets described in chapter 4.

Firstly, we have proposed to assess the quality of motor imageries using outlier detection in section 2.4.1. This was followed by the cleaning of the original data from outliers to improve model training and accuracy of constructing decision boundaries for classifiers. Results for improving a BCI using outlier detection as a quality index of motor imagery estimation are presented in section 5.1.

Secondly, to improve combined movement motor imagery classification and reduce efforts for EEG signal recording and BCI calibration, we have proposed the solution based on three parallel-connected binary classifiers in section 2.5.6. We have represented results for classification of combined movement motor imageries using binary classifiers in section 5.2.2. Also, we have introduced results for feature selection applied to parallel-connected binary classifiers in section 5.2.3.

Finally, we have presented four convolutional neural networks (D-ConvNet, S-ConvNet, EEGNet-2.32, and an adaptation of the latest EEGNet-4.8) as advanced feature extraction and classification techniques for motor imagery detection from raw EEG signals in section 3.6. Results for optimizing MI detection with deep learning using MNS-based BCI are presented in section 5.3.

In this chapter, proposed methods and results presented in the previous chapter will be analyzed and discussed. We will discuss the following points:

• Outlier detection for automatic assessing the quality of imaginary movements in MI-based BCI in section 6.1;

• Combined movement motor imagery classification in section 6.2, where we will review the classification of combined movement motor imageries using binary classifiers and feature selection;

• Imaginary movements detection using deep learning to prevent intraoperative awareness in section 6.4. Here we will touch on questions about different deep learning architectures, frequency bands, and layout selections. We will discuss the importance of frontal electrodes for classification accuracy. Also will discuss perspectives of the proposed method for prevention of intraoperative awareness during general anesthesia.

Outlier detection for automatic assessing the quality of imaginary movements in MI-based BCI

Assessing the quality of imaginary movements before training a BCI using outlier detection is an important problem that could increase the accuracy of the classification and improve BCI training. The clustering algorithms used in the work (one-class SVM method and isolation forest) allow to detect and remove around 30% of outliers from the data (see Tables 5.1 and 5.2) and thus perform classification on the cleaned dataset.

However, the obtained classification accuracy of imaginary movements before and after the outlier detection allows us to conclude that this approach is subject-specific and requires additional adjustment of parameters separately for each subject. Imaginary movement classification accuracy increased for ten over twenty-nine subjects (+11.3% in average) using isolation forest method (Figure 5.2 page 83) and for nine subjects (+9.6% in average) using one-class SVM method. But for other subjects classification accuracy was not changed or decreased. This suggests that this approach is subject-specific and can be successfully applied to a specific group of subjects. At the same time, average accuracy across all subjects was not changed so this means that the method of outlier detection in the initial data has to be improved.

In our opinion, this can be achieved by one or more of the following options: changing the architecture of the method; using more discriminative features to better separate classes before outlier detection; using deep learning methods for both feature extraction and outlier detection parts.

Classification of combined movement motor imageries using binary classifiers

Our study shows that the electrical activity of combined motor imageries measured on the scalp is very similar to the electrical activity measured for each simple motor imagery taken individually. Thus an efficient classification should be able to distinguish them.

The low classification rate obtained with classical methods in section 5.2.1 could be linked with the difficulty to perform combined movement motor imagery in a proper way as well as with bad class separation using current feature extraction techniques.

We tried to achieve mainly three goals in our study about combined movement classification:

• to reduce the time for signal acquisition, BCI training, and calibration using an architecture with 3 binary classifiers, which does not require combined movement motor imageries for training, just three classes of MIs: right hand, left hand, and feet.

• to investigate the importance of features obtained from the signals filtered in different bands;

• to improve the performance of combined movement MIs classification overall.

Results for 3 parallel stacked binary classifiers represented in section 5.2.2. It is obvious that the proposed solution could be useful to save time for recording, training, and calibration stages as we will not collect data for combined movements before training. But at the same time the results (Tables 5.5 -5.10) of such approach are worth than using multi-label classification (Table 5.3).

The best accuracies are obtained using a 30-feature set, which confirms the hypothesis that usage of features filtered in different frequency bands could be useful. We show that an increasing number of features leads to increased accuracy of classification even with our small number of training examples. But the final results are still too small to implement this approach to combined movements classification.

It can be seen that the accuracy of the resting state classification is low (Table 5.10), even considering the fact that the resting state was in the training data. But in general, we have a predictable tendency to obtain a good result for known classes and bad performance on the unknown classes.

To start with, detection of the rest state should be improved because accuracy for these conditions is very low. It was shown on histograms -'rest' and 'feet' classes are not well separated (Figures 5. 15-5.16). The solution could be in addition one more serial connected classifier before 3 parallel to detect any movement activity. If there is no movement activity -it is a resting state.

Feature selection for combined movement motor imagery detection

The proposed method for feature selection showed that not all of the features in our feature set are equally useful for the classification. Most prominent features are alpha ERD and wide beta ERS features (Tables 5.12 and 5.13) for the right and left hand MI detection. And ERS beta (17-25 Hz) together with ERS alpha+beta (7-30 Hz) features on Cz were chosen by feature selection for the feet MI detection. It corresponds to described in the literature earlier [START_REF] Pfurtscheller | Event-related EEG/MEG synchronization and desynchronization: basic principles[END_REF][START_REF] Barachant | Multiclass Brain-Computer Interface Classification by Riemannian Geometry[END_REF]. But it is interesting that the feature selection method also chooses ERD alpha band (8-12 Hz) features from C3 and C4 channels to separate feet activity from others (Table 5.14)). It could be evident that this feature was used not for feet detection, but for the detection of right and left hand activity in the data which was trained versus the feet class.

Histograms of logarithmic ERD/ERS features distribution (Figures 5. 15, 5.16) demonstrate that interclass separation is low even for the best features chosen by the feature selection method. Feet and rest classes overlap (Figure 5.7a) which can be a reason for low classification accuracy for the Rest class.

Another useful feature could be ERD/ERS in the alpha+beta band (8-30 Hz). As we can see the separation of the 'rest' and 'feet' classes are much better than in other features for this band (table 5.15).

It was showed that classification accuracy is higher after feature selection (Table 5.17). But at the same time, we have not a significant difference between results.

As already showed and described in the section 5.2.3 the best accuracy obtained using binary classifies with feature selection (33.2% + -1%) is higher than using standard multi-label approach (32.7% + -4.2%) (see Table 5.3). But the difference is not significant and quite low.

Results of feature selection investigation and classification represented only for 10 subjects and require further investigations. But even with this amount of data, we can make a generalized view of the vulnerabilities of our approach. The main achievement of this approach is reduced time for model training and calibration (using 4 classes for training to predict 8 classes). But the result remains low for implementing this approach to control a BCI.

To improve the current method more features could be considered as well. As was presented in section 5.2.3, usage of alpha, beta, and alpha+beta bands together can be a solution to better separate classes. And it can have a positive influence on the classification accuracy. Furthermore, as known from [START_REF] Pfurtscheller | On the existence of different types of central beta rhythms below 30 hz[END_REF][START_REF] Neuper | Handbook of electroencephalography and clinical neurophysiology. Eventrelated desynchronization[END_REF] it is better to use different bands for detection of different movements (for example for finger movement the largest beta ERS was found in the 16+ -21 Hz band and for foot movement in the slightly higher 19+ -26 Hz band). But for the current experimental design, it seems impossible due to combined several different movements.

Also, classification in the BCI domain is subject-specific and there are a lot of parameters that could increase accuracy for each individual subject (for example we use only one timer for ERD calculation and one timer for ERS calculation). Thus creating user-profiles and an automated recognition system is required.

Moreover, better feature extraction techniques should be found. Results obtained in section 5.2.3 showed that classes are not well separated.

An incremental motor imagery learning using first real movements and then imaginary movements could be useful to train the user to produce good motor imagery.

Also, each class is represented by a few numbers of samples (30 for training and 10 for testing). Increasing the number of samples for each class should improve performance.

Imaginary movements detection using deep learning to prevent intraoperative awareness

Results in section 5.3 show the ability of deep learning techniques to extract features from a few EEG channels to detect motor imagery when the subject is being stimulated over the median nerve, which could represent in particular a regain of consciousness during general anesthesia.

Different deep learning architectures

We tested eight different modifications of EEGNet, where we i) reduced the first layer kernel size from 32 to 16, 8 and 4 to better handle mid-and high-frequency features; and ii) changed multiplier depth from 2 to 4, which determines the number of spatial filters of the depthwise convolution layer. We choose EEGNet-4.8 (see section 5.3.2) as most prominent architecture based on the results. Also, we tested another deep learning models such as D-ConvNet and S-ConvNet. S-ConvNet and EEGNets showed high performances. Statistical tests show that EEGNet-4.8, proposed in this paper, significantly outperformed all standard and deep learning approaches especially for MNS + MI vs. MNS task in terms of accuracy.

The higher performance of EEGNet-4.8 compare to EEGNet-2.32 for MNS + MI vs. MNS task can be explained by the presence of useful high-frequency components in the EEG signals for this task unlike MI vs. rest task. These components are better kept using a shorter kernel size. It is also shown in Figure 5.11, where we can see better performances for 8-38 Hz than for standard 8-30 Hz frequency band. Usually, there is intuition to use smaller kernels for detecting high-frequency features and larger kernels for low-frequency features [START_REF] Zeiler | Visualizing and Understanding Convolutional Networks[END_REF]. That's why reducing the first layer kernel size from 32 to 8 is useful for MNS + MI vs. MNS task.

Also as we use the same kernel for a different set of samples in a signal, the same weights are shared across these sample sets. This means that a smaller kernel size leads to using fewer weights to back-propagate on. So the benefit of using a smaller kernel is a reduction in computational costs.

Changing multiplier depth from 2 to 4 in the second layer to increase the number of spatial filters allowed to store more information in this layer. Moreover, EEGNet models have a significantly lower number of training parameters comparing to other deep learning models D-ConvNet and S-ConvNet (Table 3.5), which results in better overfitting resistance.

Frequency band selection

We found that using a 4-38 Hz frequency band is helpful for better distinguishing motor patterns resulting in MNS + MI and MNS. Previously, the 8-30 Hz frequency range was mainly used in MI-based BCI (see Table 3.5) , but adding the theta and low gamma frequency ranges resulted in an increase of accuracy for each electrode layout studied in our experiments.

Though, the effect of hand motor function in the theta 4.8-7.9 Hz band of somatosensory electrical stimulation has already been shown [START_REF] Tu-Chan | Effects of somatosensory electrical stimulation on motor function and cortical oscillations[END_REF]. They observed a significant decrease in mean motor ipsilesional resting-state power.

On the other hand, there is evidence of motor activities in the gamma band [START_REF] Buzsáki | Mechanisms of gamma oscillations[END_REF][START_REF] Ahn | Gamma band activity associated with BCI performance: simultaneous MEG/EEG study[END_REF][START_REF] Cheyne | MEG studies of motor cortex gamma oscillations: evidence for a gamma "fingerprint" in the brain?[END_REF][START_REF] Muthukumaraswamy | Functional properties of human primary motor cortex gamma oscillations[END_REF].

Addition of the frontal electrodes to improves motor imaginary detection

When investigating the effect of adding channels from the frontal cortex as additional input data to the models, we noticed that for many models the accuracy increases comparing to the case when only the channels from the motor cortex were used. This was the case for the less accurate models (e.g. TS+LR and EEGNet-2.32), while there were no significant improvements for the best-obtained model (EEGNet-4.8). The evident increase in classification accuracy between MI + MNS vs. MNS after adding frontal electrodes might have different causes, leading to models perceiving the presence of MI conditioned to nerve stimulation.

Possible reasons may be rooted in participation of the prefrontal cortex in response implementation, providing a functional substrate for the transformation of perception into action [START_REF] Haller | Persistent neuronal activity in human prefrontal cortex links perception and action[END_REF], or regions of the frontal cortex being a part of the cortical networks involved in the complex experience of pain [START_REF] Bushnell | Pain perception: Is there a role for primary somatosensory cortex?[END_REF]. In [START_REF] Guillot | Functional neuroanatomical networks associated with expertise in motor imagery[END_REF] is reported that individuals with the poor ability of motor imagery demonstrated the recruitment of orbitofrontal cortex, among others, while performing MI, and results of [START_REF] Wei | Sport expert's motor imagery: functional imaging of professional motor skills and simple motor skills[END_REF] demonstrated that the sports experts had activation patterns in prefrontal areas while performing motor imagery tasks. These results suggest that characteristic activity related to our MI + MNS vs. MNS might be encountered in the motor cortex area.

Although current results show that the proposed EEGNet-4.8 model with the highest accuracy demonstrates that the inclusion of the frontal electrodes does not lead to even more statistically significant improvements in terms of accuracy, we suggest to use frontal electrodes for discriminating MI + MNS and MNS since adding them statistically significantly improves the performance for most of the classifiers (Figure 5.26). Therefore we believe that the potential of looking at the frontal area is high and this direction will be addressed in our future research. Information from the frontal cortex is useful and the ways of integrating it into classification models in order to obtain a more significant increase in accuracy should be explored further.

Using reduced electrode layouts for motor imagery detection

In parallel with testing the performance of many models, we addressed the problem of minimization of the electrode number in the recording setup, to reduce the preparation and calibration time in clinical settings. It is especially important for the task of detecting intraoperative awareness, where preparation time before surgery is a precious resource. Starting from 128 electrodes layout which is standard for the Biosemi system (Figure 4.5), we tested the dependency of the performance on the electrode number for 48, 13, 9, 6, and 3 electrodes covering mainly the motor cortex area.

We found that it is possible to reduce the number of electrodes to 13 or even 6 electrodes without substantial loss of classification performance. The best model (EEGNet-4.8) achieved 88% accuracy for 13 electrodes setup (84% for 6 electrodes setup) for MNS + MI vs. MNS task. Since such a task with a few electrodes was not addressed in the literature previously, this is the highest accuracy to the best of the authors' knowledge. These values are higher than the available values for a model that distinguishes MI vs. rest, which is 73.8% for 13 electrodes setup (68.9% for 6 electrodes setup). Deep learning models showed higher stability when decreasing the number of electrodes for MNS + MI vs. MNS task. At the same time, the best classification performance for MI vs. rest task has been achieved on the setup with 9 electrodes.

Conclusions

This manuscript presents three main contributions to improve the detection of motor imageries which are used by numerous brain-computer interfaces as a means of interaction. Firstly, we proposed to estimate the quality of motor images by detecting outliers and removing them before learning. Secondly, we studied the selection of features (timing, frequency bands, and electrode) for motor imagery classification and created a method based on three binary parallel-connected classifiers to distinguish 7 motor imageries using just 3 of them for training the BCI. Thirdly, we presented a large review on deep learning and proposed a deep learning architecture based on EEGNet applicable directly to filtered electroencephalographic signals adapted to the number of electrodes. We showed in particular its benefits for improving the detection of intraoperative awareness and other MI-based BCI applications.

Application of the outlier detection method for preliminary preprocessing a dataset and remove bad trials allowed to significantly increase classification accuracy of motor imagery detection for 10 over 29 subjects (+11.3% on average) using isolation forest method and for 9 over 29 subjects (+9.6% on average) using one-class SVM method. But for other subjects classification accuracy was not changed or decreased. Thus, removing outliers from the training data helps to build more accurate classification models for a group of subjects.

The proposed approach for classifying combined motor imageries consists of feature selection and classification using an architecture with three binary parallel-connected classifiers for detecting three basic movements (right hand, left hand, and feet) and further detection of any combinations of these movements. We have studied the selection of different band-power features according to timing windows, frequency bands, and electrodes. We have found that the best features were obtained from the alpha (8-12 Hz) and wider alpha+beta (8-30 Hz) band just after the beginning of the MI task (0.5-6 seconds), corresponding to the event-related desynchronization over C3, C4, and Cz electrodes. We have shown that using this approach it is possible to achieve the same classification accuracy level (33.2% ± 1%) as using ERD/ERS+LDA standard multi-class approach (32.7% ± 4.2%) but with reduced efforts for signal acquisition, training, and calibration of a BCI because we use only 3 electrodes instead 26 for EEG acquisition and train BCI using only 3 MI classes (right hand, left hand, and feet) to detect 7 classes (combinations of these basic MI movements are added to the previously mentioned ones).

In this work, it was proposed to use median nerve stimulation together with deep learning on the filtered EEG data for motor imagery detection. We demonstrated that for motor imagery detection while the subject is being stimulated via the median nerve, it is advisable to use EEG signals recorded from as few as 6 electrodes in an extended 4-38 Hz frequency range processed by an adapted architecture of EEGNet deep neural network, which can give 84% of classification accuracy, comparing to 74.2% obtained with the best performed standard method (TS+LR). The combination of the novel approach of MNS with a larger number of electrodes results in even higher accuracy (94.5% for 128 electrodes and 88.0% for 13 electrodes, comparing to 85.4% and 78.7% respectively, obtained with TS+LR). These results set the background for clinical application with anesthetics of the proposed setup for accessing the intraoperative awareness.

Contribution

The main contribution of this thesis is a demonstration that the deep learning approach to motor imagery detection in raw EEG signals is also more advanced than standard existing machine learning methods.

We have done a complex inclusive review of deep learning for EEG signal analysis in the motor imagery-based BCI domain. Data from 71 reviewed articles were summarized. The main trends in the application of deep learning architectures, optimization algorithms, activation functions, and feature extraction techniques for MI-based BCIs 117 were considered.

We have improved EEGNet architecture to better handle mid and high-frequency components from filtered EEG data, identified wider frequency band, determining the importance of frontal electrodes, and showed on a dataset with 22 subjects that these improvements all together allows us to obtain a classification accuracy of 94.5% for 128 electrode layout and 84% for 6 electrode layout for the proposed MNS vs. MI+MNS discriminative task. This result is the state of the art in the task of MI detection while the median nerve is stimulated. This result is evidence that deep learning architectures not only can work well on raw EEG data with a small number of samples but also beat and replace existing standard machine learning techniques in this domain.

Thus, the current thesis contributes to the MI-based BCI domain. Moreover, the findings of this work could be applied to the task of accidental awareness detection during general anesthesia. Also, the proposed deep learning architecture with a set of parameters for feature extraction and classification from raw EEG signals might be beneficial to the EEG signal analysis community in general.

Future work

We introduce time horizons for future work highlighting short-term, intermediate-term, and long-term planning.

• Short-term plans

The main short-term purpose is to integrate the three contributions of this thesis in one pipeline. We already applied our knowledge obtained in the study on the selection of timing, frequency band, and electrode characteristics to the application of accidental awareness detection during general anesthesia. The integration of unsupervised outlier detection for cleaning the dataset requires further investigations to improve classification performance.

• Intermediate-term plans Although the use of median nerve stimulation is promising, it still requires more detailed investigations. Another challenge for this kind of BCI remains to be used in clinical routine.

Although most anesthetics reduce the global metabolic activity of the central nervous system by decreasing neuronal excitability [START_REF] Hentschke | Disruption of cortical network activity by the general anaesthetic isoflurane[END_REF][START_REF] Mashour | Neuroscientific Foundations of Anesthesiology[END_REF], the effect of anesthetics on the sensorimotor cortex appears to be relatively well preserved [START_REF] Blokland | Decoding motor responses from the eeg during altered states of consciousness induced by propofol[END_REF][START_REF] Rimbert | Innovative brain-computer interface based on motor cortex activity to detect accidental awareness during general anesthesia[END_REF]. In fact, regions of the cortex would still be receptive to information, but their ability to communicate with other regions seems to be damaged during general anesthesia [START_REF] Peltier | Functional connectivity changes with concentration of sevoflurane anesthesia[END_REF], suggesting that our method can be used in patients with clinical conditions [START_REF] Rimbert | Innovative brain-computer interface based on motor cortex activity to detect accidental awareness during general anesthesia[END_REF].

Further experiments are planned to be done in clinical conditions under general anesthesia to investigate the influence of anesthetics on the ability to detect motor activity in real-life conditions. Also, it is important to continue studying user profiles with a set of parameters in case of subject-specificity due to high variance between subjects. Further investigations should be aimed at application different crossvalidation strategies, such as leave-one-subject-out and leave-one-group-of-subjects-out. Such an experimental design is much closer to real-life experience and requires subject-to-subject transfer learning, so this might lead to a decrease in classification performance. The current cross-validation approach presented in this thesis can be well-suited if we consider taking some training examples from an observed subject for training or calibration which is not always possible.

In the context of detection of intraoperative awareness from EEG signals, no training patterns of the intention of movement under propofol are available in advance to train a specific model for the current subject. So the findings of this thesis concerning the benefits of an extension of the frequency band to 4-38 Hz and of using frontal electrodes are useful but the calibration phase has to be adapted. A transfer learning method from other subjects performing a motor intention under propofol should be done.

In order to tackle this challenge, two different machine learning approaches could be explored: i) transfer learning approaches, with the idea to learn a motion intention detection model on a subject excluding anesthesia (where we can have a lot of data) and to transfer this model to a context (where we have little data), in particular by transforming data under anesthesia into data outside anesthesia. The transformation necessary to pass from the covariance matrix of the signals recorded during rest under anesthesia (for example at the very beginning of the operation) to that of the rest signals recorded without anesthesia will provide us with an estimate of the matrix of covariance of motion intent signals during anesthesia from motion intent signals outside anesthesia; ii) adaptive learning approaches, where the idea is to build adaptively, starting from zero, a model describing the properties of EEG signals [START_REF] Kumar | Analysis of Electroencephalography (EEG) Signals and Its Categorization-A Study[END_REF]. An intention to move will then be identified by rupture detection when the property of the EEG signals changes.

One of the possible approaches is to adapt Riemannian geometry to transfer learning. We already have a database with 15 subjects for this experiment and a method such proposed by [START_REF] Rodrigues | Riemannian procrustes analysis: Transfer learning for brain-computer interfaces[END_REF] can be applied in the intermediate-term perspective.

• Long-term plans Another approach based on transfer learning from other subjects performing a motor intention under propofol could be done in the long-term perspective. This method consists of training a network using a large EEG database such as ones available for automatic sleep scoring [START_REF] Mcnicholas | Dublin Sleep Apnea Database[END_REF][START_REF] Kemp | Analysis of a sleepdependent neuronal feedback loop: the slow-wave microcontinuity of the EEG[END_REF][START_REF] Terzano | Atlas, rules, and recording techniques for the scoring of cyclic alternating pattern (CAP) in human sleep[END_REF] to extract principal EEG features. Then, pre-trained layers of this network could be used to train the last layers [START_REF] Vilamala | Deep Convolutional Neural Networks for Interpretable Analysis of EEG Sleep Stage Scoring[END_REF] on our specific EEG data under general anesthesia. Thus, transfer learning can allow avoiding a long calibration of the model on patient-specific EEG of motor imageries using data and models of other anesthetized subjects.

In particular, it is not possible to ask the patient to follow a protocol for several minutes to calibrate a model before a surgery. Deep learning can also be used for collecting larger synthetic amounts of data [START_REF] Luo | EEG Signal Reconstruction Using a Generative Adversarial Network With Wasserstein Distance and Temporal-Spatial-Frequency Loss[END_REF][START_REF] Hartmann | EEG-GAN: Generative adversarial networks for electroencephalograhic (EEG) brain signals[END_REF]. A generative adversarial network could be applied in this case to generate and collect missing training patterns of the intention of movement under propofol using only EEG signals acquired while subjects stimulated via the median nerve. Also, in the long-term, we plan to study the application of median nerve stimulation to detect several various motor imageries (for example, right arm, left arm, head, and full body) to increase the number of possible interactions of BCI.

My publications International journals

1. 1

 1 BCI applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 General BCI architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Examples of EEG waves characterizing brain rhythms . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Types of cortical areas of the human brain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 The motor strip contains an somatotopic representation of the contralateral half of the body . . . . 1.6 The international 10-20 system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.7 EEG electrode placement according to the ABC layout . . . . . . . . . . . . . . . . . . . . . . . 1.8 Example of ERD and ERS occurrence during real movement and motor imagery . . . . . . . . . 1.9 Example of ERD and ERS occurrence during real movement, motor imagery and median nerve stimulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.10 Example of ERD and ERS occurrence during motor imagery, motor imagery during median nerve stimulation and median nerve stimulation itself . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Different spatial filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Example of raw and filtered signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Trials representation according to average amplitude feature extraction technique . . . . . . . . . 2.4 Trials representation according to average of signal power . . . . . . . . . . . . . . . . . . . . . 2.5 Descrete and continuous mother wavelet function . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 Representation of motor imagery of the right hand, left hand and rest class using logarithmic ERD/ERS features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Covariance matrices for two classes representing EEG data . . . . . . . . . . . . . . . . . . . . . 2.8 Trials representation according to CSP feature extraction technique . . . . . . . . . . . . . . . . . 2.9 Centers of mass for two classes representing EEG data from 26 channels . . . . . . . . . . . . . . 2.10 Centers of mass for two classes representing EEG data from 2 channels . . . . . . . . . . . . . . 2.11 The Manifold and the tangent space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.12 Classification on the tangent space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.13 Centers of mass and trials for two classes represented in tangent space . . . . . . . . . . . . . . . 2.14 Results of outlier detection for right hand MI using different methods . . . . . . . . . . . . . . . 2.15 Decision boundary of different classifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.16 Decision boundary of LDA classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.17 SVM classifiers with different kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.18 Results of decision tree classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.19 Results of gradient boosting classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.20 Classification on the Riemannian manifold using distance based approach . . . . . . . . . . . . . 2.21 Distribution of the relative oscillatory power along the sensorimotor cortex during the course of the different motor tasks for the 8-class database. . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Activation functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 CNN applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 CNN Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Convolutional layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Depthwise convolutional layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii xiv List of Figures 3.6 Depthwise separable convolutional layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.7 Examples of max and average pooling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 3.8 Graphical representation of the Dropout method . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 3.9 Proportion use of the different DNN architectures across all the reviewed articles . . . . . . . . . 59 3.10 Proportion use of different activation functions across all the reviewed articles . . . . . . . . . . . 60 3.11 Proportion use of various optimization algorithms across all the reviewed articles. . . . . . . . . . 60 3.12 Proportion use of input formulation types across all the reviewed articles with the used feature extraction methods for achieving them. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 3.13 Detailed proportion use of 3 different input formulation types across all the reviewed articles with the used feature extraction methods for achieving them. . . . . . . . . . . . . . . . . . . . . . . . 65 3.14 EEGNet architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67 4.1 Schematic sequence diagram of the experimental paradigm . . . . . . . . . . . . . . . . . . . . . 73 4.2 Time scheme for motor imagery 8-class database . . . . . . . . . . . . . . . . . . . . . . . . . . 74 4.3 Schematics of the experimental setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 4.4 Representation scheme for one trial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 4.5 BioSemi 128 channels headcap ABC layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 4.6 Layouts with electrodes placement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 5.1 Results of outlier detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.2 Classification accuracy difference after outlier detection using Isolation forest method . . . . . . . 83 5.3 Classification accuracy difference after outlier detection using one-class SVM method . . . . . . . 83 5.4 ERD/ERS% time series within the mu range over electrode C3 . . . . . . . . . . . . . . . . . . . 86 5.5 ERD/ERS% time series within the mu range over electrode C z . . . . . . . . . . . . . . . . . . . 86 5.6 ERD/ERS% time series within the mu range over electrode C4 . . . . . . . . . . . . . . . . . . . 86 5.7 3D plot of the sample distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 5.8 Topographic map of ERD/ERS% in the 8-30 Hz and 4-38 Hz bands during MI + MNS and MNS only conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 5.9 Topographic map of ERD/ERS% in the 4-8 Hz band during MI + MNS and MNS only conditions 97 5.10 Topographic map of ERD/ERS% in the 30-38 Hz band during MI + MNS and MNS only conditions 97 5.11 Classification accuracy for TS+LR and EEGNet-4.8 . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.12 Boxplots showing the distribution of EEGNet-4.8 classification accuracy . . . . . . . . . . . . . . 98 5.13 Classification accuracy for different variants of EEGNet . . . . . . . . . . . . . . . . . . . . . . . 99 5.14 Grand average classification accuracy for different electrode layouts using different models for MNS vs. MI+MNS condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 5.15 Classification accuracies obtained by different models using layout with 128 electrodes . . . . . . 103 5.16 Classification accuracies obtained by different models using layout with 47 electrodes . . . . . . . 103 5.17 Classification accuracies obtained by different models using layout with 13 electrodes . . . . . . . 103 5.18 Classification accuracies obtained by different models using layout with 6 electrodes . . . . . . . 104 5.19 Boxplots showing the distribution of different models classification accuracy using layout with 128 electrodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 5.20 Boxplots showing the distribution of different models classification accuracy using layout with 47 electrodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 5.21 Boxplots showing the distribution of different models classification accuracy using layout with 13 electrodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.22 Boxplots showing the distribution of different models classification accuracy using layout with 6 electrodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 5.23 Grand average classification accuracy for different electrode layouts using different models for MI vs. rest condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 5.24 Comparison of classification accuracy for different electrode layouts . . . . . . . . . . . . . . . . 107 5.25 Boxplots showing the distribution of EEGNet-4.8 classification accuracy for different setups of electrodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 1 . 1 :

 11 Figure 1.1: BCI applications.

Figure 1 . 2 :

 12 Figure 1.2: General BCI architecture. There are five main modules that every BCI consists of, namely, input signal acquisition module, signal preprocessing module, feature extraction, classification, and translation. Besides some BCIs could include feedback modules.

Figure 1 . 3 :

 13 Figure 1.3: Examples of EEG waves characterizing brain rhythms (from [73]).

Figure 1 . 4 :

 14 Figure 1.4: Types of cortical areas of the human brain[80].

Figure 1 . 5 :

 15 Figure 1.5: The motor strip contains an somatotopic representation of the contralateral half of the body, from the toes to the face and tongue [86].

Figure 1 . 6 :

 16 Figure 1.6: The international 10-20 system seen from (A) left and (B) above the head. A = Ear lobe, C = central, Pg = nasopharyngeal, P = parietal, F = frontal, Fp = frontal polar, O = occipital. (C) Location and nomenclature of the intermediate 10% electrodes, as standardized by the American Electroencephalographic Society [73].

Figure 1 . 7 :

 17 Figure 1.7: EEG electrode placement according to the ABC layout.

Figure 1 . 8 :

 18 Figure 1.8: Example of ERD and ERS occurrence during real movement and motor imagery (adopted from [20]).

Figure 1 . 9 :

 19 Figure 1.9: Example of ERD and ERS occurrence during real movement, motor imagery and median nerve stimulation (from [20]).

Figure 1 . 10 :

 110 Figure 1.10: Example of ERD and ERS occurrence during motor imagery, motor imagery during median nerve stimulation and median nerve stimulation itself (from [20]).

  Ear reference (b) Common Average Reference (c) Small Laplacian (d) Large Laplacian

Figure 2 . 1 :

 21 Figure 2.1: Different spatial filters. Mean or weighted mean of green electrodes would be subtracted from the red electrode for each spatial filter presented on the figure.

( a )

 a Raw and filtered signal using LPF (30 Hz) (b) Raw and filtered signal using HPF (8 Hz) (c) Raw and filtered signal using BPF (8-30 Hz) (d) Raw and filtered signal using BSF (49-51 Hz)

Figure 2 . 2 :

 22 Figure 2.2: Example of raw and filtered signal of right hand MI from channel C3 using 5th order Butterworth filters. (a) LPF with cutoff frequency 30 Hz. (b) HPF with cutoff frequency 8 Hz (c) BPF with cutoff frequency range in alpha+beta band (8-30 Hz) (d) BSF with cutoff frequency range in (49-51 Hz).

( a ) 2 Figure 2 . 3 :

 a223 Figure 2.3: Trials representation according to average amplitude feature extraction technique. Two-classes problem with right hand MI and rest. EEG data is filtered in the alpha+beta (8-30 Hz) frequency range and averaged over time samples. Electrodes C3 and C4 are considered. In these examples, we used EEG data from two different subjects: (a) subject 1; (b) subject 2.

(a) Data from subject 1 (b) Data from subject 2 Figure 2 . 4 :

 1224 Figure 2.4: Trials representation according to average of signal power. Two-classes problem with right hand MI and rest. EEG data is filtered in the alpha+beta (8-30 Hz) frequency range, squared and then averaged over time samples. Electrodes C3 and C4 are considered. In these examples, we used EEG data from two different subjects: (a) subject 1; (b) subject 2.

Figure 2 . 5 :

 25 Figure 2.5: Descrete and continuous mother wavelet functions: a) Daubechie wavelet of the 6th order, b) Symlet wavelet of the 6th order, c) Coiflet wavelet of the 6th order, d) Biorthogonal wavelet bior2.6, e) Mexican hat wavelet, f) Morlet wavelet, g) Complex Gaussian wavelet of the 6th order, h) Gaussian wavelet of the 6th order.

( a ) 2 Figure 2 . 6 :

 a226 Figure 2.6: Motor imagery of the right hand, left hand and rest class are represented. Logarithmic ERD/ERS features are computed for EEG channels C3 and C4, which are located above areas of the brain responsible for right and left hand activity respectively. In these examples, we used EEG data from two different subjects: (a) subject 1; (b) subject 2.

( a )

 a Data from subject 1 (b) Data from subject 2

Figure 2 . 7 :

 27 Figure 2.7: Covariance matrices for two classes (rest and right hand MI) representing EEG data (26 channels). Signals are previously filtered in the alpha+beta (8-30 Hz) frequency range. In these examples, we used EEG data from two different subjects: (a) subject 1; (b) subject 2.

(a) Data from subject 1 (b) Data from subject 2 Figure 2 . 8 :

 1228 Figure 2.8: Trials representation according to CSP feature extraction technique. Two-classes problem with right hand MI and rest. EEG data (26 channels) from the dataset 4.2 is filtered in the alpha+beta (8-30 Hz) frequency range, and decomposed to 2 CSP components. In these examples, we used EEG data from two different subjects: (a) subject 1; (b) subject 2.

Figure 2 . 9 :

 29 Figure 2.9: Centers of mass for two classes (rest and right hand MI) representing EEG data (26 channels). Signals are preliminary filtered in alpha+beta (8-30 Hz) frequency range.

Figure 2 . 10 :

 210 Figure 2.10: Centers of mass for two classes (rest and right hand MI) representing EEG data (from channels C3 and C4). Signals are previously filtered in alpha+beta (8-30 Hz) frequency range.

Figure 2 .

 2 Figure 2.11:The Manifold and the tangent space at a point. Consider a point Ω on M and construct the tangent space TΩM on it. Now take a tangent vector V departing from Ω, which is our reference point. There exists one and only one geodesic on the manifold starting at Ω that corresponds to V ; think of rolling the plane (tangent space) on the surface (manifold) in such a way that the vector always touches the surface. The endpoint on M is Φ. We see that the geodesics on M through Ω are transformed into straight lines and the distances along all geodesics are preserved (from[START_REF] Congedo | Approximate Joint Diagonalization and Geometric Mean of Symmetric Positive Definite Matrices[END_REF][START_REF] Goh | Unsupervised Riemannian Clustering of Probability Density Functions[END_REF]).

Figure 2 . 12 :

 212 Figure 2.12: Classification on the tangent space (from [134]).

Figure 2 . 13 :

 213 Figure 2.13: Centers of mass and trials for two classes (rest and right hand MI) represented in tangent space. We used EEG data from 2 channels (C3 and C4). We have previously filtered signals in alpha+beta (8-30 Hz) frequency range.

Figure 2 . 14 :

 214 Figure 2.14: Results of outlier detection for right hand MI using different methods: local outlier factor, Isolation forest, oneclass SVM, robust covariance.

Figure 2 . 15 :

 215 Figure 2.15: Decision boundary of different classifiers for 3 classes: right hand MI, left hand MI and rest. Classification decision based on ERD/ERS features from two channels, C3 and C4, acquired above motor cortex. In these examples, we used EEG data from two different subjects: (a) subject 1; (b) subject 2. Here rest class marked in orange, right hand MI marked in blue and left hand MI marked in purple.

Figure 2 . 16 :

 216 Figure 2.16: Decision boundary of LDA classifier for 3 classes: right hand MI, left hand MI and rest. Classification decision based on ERD/ERS features from two channels C3 and C4, acquired above motor cortex. In these examples, we used EEG data from two different subjects: (a) subject 1; (b) subject 2.

( a )

 a Data from subject 1 (b) Data from subject 2

Figure 2 . 17 :

 217 Figure 2.17: SVM classifiers with different kernels (linear, RBF and polynomial (3rd degree) for 3 classes: right hand MI, left hand MI and rest. Classification decision based on ERD/ERS features from two channels C3 and C4, acquired above motor cortex. In these examples, we used EEG data from two different subjects: (a) subject 1; (b) subject 2.

Figure 2 . 18 :

 218 Figure 2.18: Results of decision tree classifier for 3 classes: right hand MI, left hand MI and rest. Classification decision based on ERD/ERS features obtained from two channels C3 and C4, acquired above motor cortex as described in section 2.2.2.1 In these examples, we used EEG data from two different subjects: (a) subject 1; (b) subject 2.

( a )

 a Data from subject 1 (b) Data from subject 2

Figure 2 . 19 :

 219 Figure 2.19: Results of gradient boosting classifier for 3 classes: right hand MI, left hand MI and rest. Classification decision based on ERD/ERS features obtained from two channels C3 and C4, acquired above motor cortex as described in section 2.2.2.1 In these examples, we used EEG data from two different subjects: (a) subject 1; (b) subject 2.

Figure 2 . 20 :

 220 Figure 2.20: Classification on the Riemannian manifold using distance based approach [134].

Figure 2 . 21 :

 221 Figure 2.21: Distribution of the relative oscillatory power along the sensorimotor cortex during the course of the different motor tasks for subjet 2.ERD/ERS% values were obtained for all electrodes within the mu frequency range and averaged over the 4-second period starting one second after the task cue was presented. Negative values indicate ERD% modulation, which is characteristic of a motor task execution that appears over the contralateral hemisphere of the body part used in the process (broadly, electrode C4 for the left hand, Cz for feet, and C3 for right hand). Note that each topographic map is independently normalized to enhance the corresponding patterns[START_REF] Lindig-Leon | Scalp EEG activity during simple and combined motor imageries to control a robotic ARM[END_REF].

  Figure 3.1: Activation functions

Figure 3 . 2 :

 32 Figure 3.2: CNN applications (from [181]).

Figure 3 . 3 :

 33 Figure 3.3: CNN architecture (from [182]).

Figure 3 . 4 :

 34 Figure 3.4: Convolutional layer with 1 channel output (adapted from [185]).

Figure 3 . 5 :

 35 Figure 3.5: Depthwise convolutional layer with 1 channel output (adapted from [185]).

Figure 3 . 6 :

 36 Figure 3.6: Depthwise separable convolutional layer with 1 channel output (adapted from [185]).

Figure 3 . 7 :

 37 Figure 3.7: Examples of max and average pooling.

  Overfitting is one of the main problems of ANN training. It consists of the following: the model explains well only examples from the training set, adapting to training examples, instead of learning to classify examples that were not involved in training and fail to capture statistical processes which described the data, thereafter losing the model's ability to generalize.

Figure 3 . 8 :

 38 Figure 3.8: Graphical representation of the Dropout method (from[START_REF] Srivastava | Dropout: a simple way to prevent neural networks from overfitting[END_REF]). On the left is the neural network before dropout was applied to it, on the right is the same network after dropout. The network shown on the left is used for testing, after parameters training.

Figure 3 . 10 :

 310 Figure 3.10: Proportion use of different activation functions across all the reviewed articles.

Figure 3 . 11 :

 311 Figure 3.11: Proportion use various optimization algorithms across all the reviewed articles.

Figure 3 .

 3 12 represents the proportion use of input formulation types across all the reviewed articles with the used feature extraction methods for achieving them. About 27.5% of all the reviewed studies used raw EEG data as input for deep learning models (Figure 3.13a). About 40% of all the reviewed articles used CSP or FBCSP in the process of features extraction for feature input calculation. While time-frequency analysis methods such as Fast Fourier transform (FFT), Power Spectral Density (PSD), Continuous Wavelet Transform (CWT), Wavelet Packet Decomposition (WPD), Wavelet Packet Transform (WPT), and Continuous Wavelet Decomposition (CWD) take about 24% in total.

Figure 3 .

 3 13b represents detailed proportion use of feature extraction methods for features input formulation across all the reviewed articles.

Figure 3 .

 3 13c represents detailed proportion use of feature extraction methods for images input formulation across all the reviewed articles. Different tasks activate different electrical signals on the human scalp. The frequency range of the signals differs. The reviewed studies have suggested different frequency ranges for dealing with the MI tasks during the investigation and classification. However, from the reviewed articles and as shown in Table 3.1, the investigated starting point of the frequency range was within 4-8 Hz; while the observed ending point of the range was within 35-40 Hz for MI related tasks. According to observed information, CSP and FBCSP are frequently used in case of extraction features as inputs

Figure 3 . 12 :

 312 Figure 3.12: Proportion use of input formulation types across all the reviewed articles with the used feature extraction methods for achieving them.

( a )

 a Proportion use of feature extraction methods for timeseries input formulation across all the reviewed articles. (b) Proportion use of feature extraction methods for features input formulation across all the reviewed articles. (c) Proportion use of feature extraction methods for images input formulation across all the reviewed articles.

Figure 3 . 13 :

 313 Figure 3.13: Detailed proportion use of 3 different input formulation types across all the reviewed articles with the used feature extraction methods for achieving them.

Figure 3 . 14 :

 314 Figure 3.14: EEGNet architecture. Where the number of channels C = 6, the number of time samples T , kernel length for the first convolutional layer K = 8, multiplier depth D = 2, the number of temporal filters F1 and the number of pointwise filters F2.

Figure 4 . 1 :

 41 Figure 4.1: Schematic sequence diagram of the experimental paradigm.Each session comprised a 1 min pre-experiment resting period, 20 repetitions of the given task and a 1 min post-experiment resting period. The task started with 2 s of a visual introduction of the task, followed by 10 s of a task period and resting period which was given randomly from 15 to 17 s. At the beginning and end of the task period, a short beep (250 ms) was played[START_REF] Shin | Open Access Dataset for EEG+NIRS Single-Trial Classification[END_REF].

Figure 4 . 2 :

 42 Figure 4.2: Time scheme for motor imagery 8-class database (from [150]).

  EEG signals were recorded by the OpenViBE [279] platform from ten right-handed subjects at 256 Hz using a commercial REFA amplifier developed by TMS International TM . Each subject performed 4 runs of motor imageries. Each run contains 10 trials for each class (Rest, Left Hand, Right Hand, Both Hands, Feet, Left hand & Feet, Right hand & Feet, Both hands & Feet). Thus 40 trials per class are available for each subject.

• 6 -

 6 feature set (ERD & ERS on alpha+beta band): a combination of two sets -alpha+beta band (7-30 Hz); • 6-feature set (ERD on alpha and ERS on beta band): alpha band (8-12 Hz) and beta band (17-25 Hz) each for 3 channels of EEG described above; • 18-feature set: low-alpha band (7-10 Hz), upper-alpha band (11-13 Hz), low-beta band (15-17 Hz), mid-beta band (18-20 Hz), mid-beta band (21-23 Hz) and upper-beta band (24-26 Hz) each for 3 channels of EEG described above; • 30-feature set: ERD filtered in following bands (7-10 Hz, 11-13 Hz, 8-12 Hz, 7-30 Hz) and ERS filtered in following bands (15-17 Hz, 18-20 Hz, 21-23 Hz, 24-26 Hz, 17-25 Hz, 7-30 Hz) each for 3 channels of EEG described above.

Figure 4 . 3 :

 43 Figure 4.3: Schematics of the experimental setting (from [20]).

Figure 4 . 4 :

 44 Figure 4.4: Representation scheme for one trial. Timing schemes of a trial for C1, C2, and C3. For all motor tasks, one low-frequency beep indicates when to start the task. For the MNS+MI condition, the MNS occurs at 750 milliseconds after the first beep. The end of the MI is announced by a high-frequency beep and followed by a rest period of 6 seconds (adopted from[START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF]).

Figure 4 . 5 :

 45 Figure 4.5: BioSemi 128 channels headcap ABC layout

9 (

 9 e) 9 (+fr) (f) 6(g) 6 (+fr) (h) 3

Figure 4 . 6 :

 46 Figure 4.6: Layouts with electrodes placement

Contents 5 . 1

 51 Improving a BCI using outlier detection as quality index of motor imagery estimation . . . .[START_REF] Miller | The prefrontal cortex: categories, concepts and cognition[END_REF] 5.2 Improving combined movement motor imagery classification using machine learning . . . . 85 5.2.1 Combined movement MI classification using classic methods . . . . . . . . . . . . . . . 87 5.2.2 Classification of combined movement motor imageries using binary classifiers . . . . . . 87 5.2.3 Feature selection for combined movement MI classification . . . . . . . . . . . . . . . . 90 5.3 Optimizing motor imagery detection with deep learning to prevent intraoperative awareness 95 5.3.1 Frequency band selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 5.3.2 EEGNet architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 5.3.3 Dropout investigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 5.3.4 Classification results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 5.3.5 Comparing paradigms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 5.3.6 Frontal electrodes investigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 5.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

  4.1 are represented in Tables 5.1 and 5.2).

( a )Figure 5 . 1 :

 a51 Figure 5.1: Results of outlier detection for three classes: (a) right hand MI, (b) left hand MIusing different methods: local outlier factor, Isolation forest, one-class SVM, robust covariance.

Figure 5 . 2 :Figure 5 . 3 :

 5253 Figure 5.2: Classification accuracy difference after outlier detection using Isolation forest method for different subjects

Figure 5 . 4 :

 54 Figure 5.4: ERD/ERS% time series within the mu range for all motor tasks over electrode C3 (i.e, the activity source associated with the use of the right hand). ERD/ERS values were estimated over the EEG trials from subject 2 (from [150]).

Figure 5 . 5 :

 55 Figure 5.5: ERD/ERS% time series within the mu range for all motor tasks over electrode Cz. ERD/ERS values were estimated over the EEG trials from subject 2 (from [150]).

Figure 5 . 6 :

 56 Figure 5.6: ERD/ERS% time series within the mu range for all motor tasks over electrode C4. ERD/ERS values were estimated over the EEG trials from subject 2 (from [150]).

Table 5 . 15 :Table 5 . 16 :Figure 5 . 7 :

 51551657 Figure 5.7: 3D plot of the sample distribution for different feature sets: (a) ERD (8-10 Hz); (b) ERD (7-13 Hz); (c) ERD (8-12 Hz); (d) ERD (8-30 Hz).

Figure 5 . 8 :

 58 Figure 5.8: Topographic map of ERD/ERS% (grand average, n = 22) in the 8-30 Hz and 4-38 Hz bands during two conditions: MI + MNS and MNS only. A red color corresponds to a strong ERS and a blue one to a strong ERD. Red electrodes indicate a significant difference between the two conditions (p < 0.01).

5. 3

 3 Figure 5.11c shows that including separately and both together the theta and the low gamma bands significantly (p-value < 0.01) improves the accuracy. Indeed, discrimination power using 4-30 Hz and 8-38 Hz bands exceeds the one of the 8-30 Hz band. Moreover, both together they improve even more accuracy. The positive effect on the accuracy with the low gamma band is stronger (p -value < 0.001) for 128 electrodes. That may indicate the presence of useful information in theta and low gamma oscillations outside the motor cortex and the frontal lobe since for the other montages with 47 electrodes or less recorded brain information is reduced to these regions.To analyze the differences between MI + MNS and MNS conditions, we performed an event-related spectral perturbation (ERSP) analysis for two different frequency bands (8-30 Hz vs. 4-38 Hz) and separately (4-8 Hz and 30-38 Hz) using EEGLAB. This analysis allows us to understand how MI + MNS and MNS conditions can be discriminated and which time parameters we can choose to guide the classification. We have decided to compute ERSPs in a merged band (mu+beta, 8-30 Hz) and a wider band (4-38 Hz) for MI + MNS and MNS conditions (Figure5.8). A surrogate permutation test (p < 0.01; 4,000 permutations) from the EEGLAB toolbox was used to validate differences in terms of localization of these ERSPs. In addition to this analysis, we applied a false discovery rate (FDR) correction test (see section 2.8) in order to clarify how the FDR was controlled for multiple comparisons to show the difference between MI + MNS and MNS conditions. Figures 5.9 and 5.10 shows grand average topographic maps of ERD/ERS% for all subjects in the 4-8 Hz and 30-38 Hz bands during two conditions: MI + MNS and MNS only. The red color corresponds to a strong ERS and a blue one to a strong ERD. Red electrodes indicate a significant difference between the two conditions (p < 0.01).Figure 5.8 shows difference between two bands (8-30 Hz and 4-38 Hz) for MNS vs. MI+MNS task. It could be seen that (4-38 Hz) band better handle ERD during MI+MNS condition. Moreover, the FDR test shows a larger number of electrodes that have a significant difference for two conditions for (4-38 Hz) frequency band.As we can see from Figure5.9 (4-8 Hz) frequency band could be very useful in case of discriminating MNS condition from the MI+MNS. In addition, FDR shows a huge number of electrodes all over the cortex which indicates a significant difference between the two conditions. At the same time Figure5.10 indicates that[START_REF] Mashour | Intraoperative awareness: controversies and non-controversies[END_REF][START_REF] Macgregor | A waking nightmare: how can we avoid accidental awareness during general anaesthesia?[END_REF][START_REF] Almeida | Awake and unable to move: what can perioperative practitioners do to avoid accidental awareness under general anesthesia?[END_REF][START_REF] Pandit | Royal College of Anaesthetists, and Association of Anaesthetists of Great Britain and Ireland, "5th national audit project (NAP5) on accidental awareness during general anaesthesia: summary of main findings and risk factors[END_REF][START_REF] Blacher | On awakening paralyzed during surgery. a syndrome of traumatic neurosis[END_REF][START_REF] Tasbighou | Accidental awareness during general anaesthesia -a narrative review[END_REF][START_REF] Myles | Bispectral index monitoring to prevent awareness during anaesthesia: the b-aware randomised controlled trial[END_REF][START_REF] Kent | Depth of anesthesia[END_REF][START_REF] Purdon | Clinical Electroencephalography for Anesthesiologists Part I: Background and Basic Signatures[END_REF]

Figures 5 .

 5 Figure 5.11c shows that including separately and both together the theta and the low gamma bands significantly (p-value < 0.01) improves the accuracy. Indeed, discrimination power using 4-30 Hz and 8-38 Hz bands exceeds the one of the 8-30 Hz band. Moreover, both together they improve even more accuracy. The positive effect on the accuracy with the low gamma band is stronger (p -value < 0.001) for 128 electrodes. That may indicate the presence of useful information in theta and low gamma oscillations outside the motor cortex and the frontal lobe since for the other montages with 47 electrodes or less recorded brain information is reduced to these regions.To analyze the differences between MI + MNS and MNS conditions, we performed an event-related spectral perturbation (ERSP) analysis for two different frequency bands (8-30 Hz vs. 4-38 Hz) and separately (4-8 Hz and 30-38 Hz) using EEGLAB. This analysis allows us to understand how MI + MNS and MNS conditions can be discriminated and which time parameters we can choose to guide the classification. We have decided to compute ERSPs in a merged band (mu+beta, 8-30 Hz) and a wider band (4-38 Hz) for MI + MNS and MNS conditions (Figure5.8). A surrogate permutation test (p < 0.01; 4,000 permutations) from the EEGLAB toolbox was used to validate differences in terms of localization of these ERSPs. In addition to this analysis, we applied a false discovery rate (FDR) correction test (see section 2.8) in order to clarify how the FDR was controlled for multiple comparisons to show the difference between MI + MNS and MNS conditions. Figures 5.9 and 5.10 shows grand average topographic maps of ERD/ERS% for all subjects in the 4-8 Hz and 30-38 Hz bands during two conditions: MI + MNS and MNS only. The red color corresponds to a strong ERS and a blue one to a strong ERD. Red electrodes indicate a significant difference between the two conditions (p < 0.01).Figure 5.8 shows difference between two bands (8-30 Hz and 4-38 Hz) for MNS vs. MI+MNS task. It could be seen that (4-38 Hz) band better handle ERD during MI+MNS condition. Moreover, the FDR test shows a larger number of electrodes that have a significant difference for two conditions for (4-38 Hz) frequency band.As we can see from Figure5.9 (4-8 Hz) frequency band could be very useful in case of discriminating MNS condition from the MI+MNS. In addition, FDR shows a huge number of electrodes all over the cortex which indicates a significant difference between the two conditions. At the same time Figure5.10 indicates that[START_REF] Mashour | Intraoperative awareness: controversies and non-controversies[END_REF][START_REF] Macgregor | A waking nightmare: how can we avoid accidental awareness during general anaesthesia?[END_REF][START_REF] Almeida | Awake and unable to move: what can perioperative practitioners do to avoid accidental awareness under general anesthesia?[END_REF][START_REF] Pandit | Royal College of Anaesthetists, and Association of Anaesthetists of Great Britain and Ireland, "5th national audit project (NAP5) on accidental awareness during general anaesthesia: summary of main findings and risk factors[END_REF][START_REF] Blacher | On awakening paralyzed during surgery. a syndrome of traumatic neurosis[END_REF][START_REF] Tasbighou | Accidental awareness during general anaesthesia -a narrative review[END_REF][START_REF] Myles | Bispectral index monitoring to prevent awareness during anaesthesia: the b-aware randomised controlled trial[END_REF][START_REF] Kent | Depth of anesthesia[END_REF][START_REF] Purdon | Clinical Electroencephalography for Anesthesiologists Part I: Background and Basic Signatures[END_REF]

Figure 5 .

 5 Figure 5.11c shows that including separately and both together the theta and the low gamma bands significantly (p-value < 0.01) improves the accuracy. Indeed, discrimination power using 4-30 Hz and 8-38 Hz bands exceeds the one of the 8-30 Hz band. Moreover, both together they improve even more accuracy. The positive effect on the accuracy with the low gamma band is stronger (p -value < 0.001) for 128 electrodes. That may indicate the presence of useful information in theta and low gamma oscillations outside the motor cortex and the frontal lobe since for the other montages with 47 electrodes or less recorded brain information is reduced to these regions.To analyze the differences between MI + MNS and MNS conditions, we performed an event-related spectral perturbation (ERSP) analysis for two different frequency bands (8-30 Hz vs. 4-38 Hz) and separately (4-8 Hz and 30-38 Hz) using EEGLAB. This analysis allows us to understand how MI + MNS and MNS conditions can be discriminated and which time parameters we can choose to guide the classification. We have decided to compute ERSPs in a merged band (mu+beta, 8-30 Hz) and a wider band (4-38 Hz) for MI + MNS and MNS conditions (Figure5.8). A surrogate permutation test (p < 0.01; 4,000 permutations) from the EEGLAB toolbox was used to validate differences in terms of localization of these ERSPs. In addition to this analysis, we applied a false discovery rate (FDR) correction test (see section 2.8) in order to clarify how the FDR was controlled for multiple comparisons to show the difference between MI + MNS and MNS conditions. Figures 5.9 and 5.10 shows grand average topographic maps of ERD/ERS% for all subjects in the 4-8 Hz and 30-38 Hz bands during two conditions: MI + MNS and MNS only. The red color corresponds to a strong ERS and a blue one to a strong ERD. Red electrodes indicate a significant difference between the two conditions (p < 0.01).Figure 5.8 shows difference between two bands (8-30 Hz and 4-38 Hz) for MNS vs. MI+MNS task. It could be seen that (4-38 Hz) band better handle ERD during MI+MNS condition. Moreover, the FDR test shows a larger number of electrodes that have a significant difference for two conditions for (4-38 Hz) frequency band.As we can see from Figure5.9 (4-8 Hz) frequency band could be very useful in case of discriminating MNS condition from the MI+MNS. In addition, FDR shows a huge number of electrodes all over the cortex which indicates a significant difference between the two conditions. At the same time Figure5.10 indicates that[START_REF] Mashour | Intraoperative awareness: controversies and non-controversies[END_REF][START_REF] Macgregor | A waking nightmare: how can we avoid accidental awareness during general anaesthesia?[END_REF][START_REF] Almeida | Awake and unable to move: what can perioperative practitioners do to avoid accidental awareness under general anesthesia?[END_REF][START_REF] Pandit | Royal College of Anaesthetists, and Association of Anaesthetists of Great Britain and Ireland, "5th national audit project (NAP5) on accidental awareness during general anaesthesia: summary of main findings and risk factors[END_REF][START_REF] Blacher | On awakening paralyzed during surgery. a syndrome of traumatic neurosis[END_REF][START_REF] Tasbighou | Accidental awareness during general anaesthesia -a narrative review[END_REF][START_REF] Myles | Bispectral index monitoring to prevent awareness during anaesthesia: the b-aware randomised controlled trial[END_REF][START_REF] Kent | Depth of anesthesia[END_REF][START_REF] Purdon | Clinical Electroencephalography for Anesthesiologists Part I: Background and Basic Signatures[END_REF]

Figure 5 . 9 :

 59 Figure 5.9: Topographic map of ERD/ERS% (grand average, n = 22) in the 4-8 Hz band during two conditions: MI + MNS and MNS only. A red color corresponds to a strong ERS and a blue one to a strong ERD. Red electrodes indicate a significant difference between the two conditions (p < 0.01).

Figure 5 . 10 :

 510 Figure 5.10: Topographic map of ERD/ERS% (grand average, n = 22) in the 30-38 Hz band during two conditions: MI + MNS and MNS only. A red color corresponds to a strong ERS and a blue one to a strong ERD. Red electrodes indicate a significant difference between the two conditions (p < 0.01).

Figure 5 . 11 :

 511 Figure 5.11: Classification accuracy for TS+LR (5.11a, 5.11b) and EEGNet-4.8 (5.11c, 5.11d), averaged over all folds (k = 10) and all subjects (n = 22) for different electrodes setups of MNS vs. MI+MNS (5.11a, 5.11c) and MI vs. rest (5.11b, 5.11d) discriminating task. Input data filtered in 4-38 Hz, 4-30 Hz, 8-30 Hz and 8-38 Hz frequency ranges.

Figure 5 . 12 :

 512 Figure 5.12: Boxplots showing the distribution of EEGNet-4.8 classification accuracy using two frequency bands (4-38 Hz) and standard (8-30 Hz) for different electrode layouts, averaged over all folds (k = 10) and all subjects (n = 22) for MNS vs. MI+MNS discriminating task. Significance levels of Wilcoxon signed-rank test comparisons: **p < 0.01, ***p < 0.001.

Figure 5 . 13 :

 513 Figure 5.13: Classification accuracy for different variants of EEGNet, averaged over all folds (k = 10) and all subjects (n = 22) for different electrodes setups of MNS vs. MI+MNS (5.13a) and MI vs. rest (5.13b) discriminating task. Input data filtered in 4-38 Hz frequency band for 5.13a and in 8-30 Hz frequency band for 5.13b.

Figure 5 . 14 :

 514 Figure 5.14: Grand average classification accuracy for different electrode layouts using different models, averaged over all folds (k = 10) and all subjects (n = 22) for MNS vs. MI+MNS condition. Input data filtered in 4-38 Hz frequency band.

Figure 5 . 15 :

 515 Figure 5.15: Classification accuracies obtained by different models for 22 subjects using layout with 128 electrodes, averaged over all folds (k = 10) for MNS vs. MI+MNS condition. Input data filtered in 4-38 Hz frequency band.

Figure 5 . 16 :

 516 Figure 5.16: Classification accuracies obtained by different models for 22 subjects using layout with 47 electrodes, averaged over all folds (k = 10) for MNS vs. MI+MNS condition. Input data filtered in 4-38 Hz frequency band.

Figure 5 . 17 :

 517 Figure 5.17: Classification accuracies obtained by different models for 22 subjects using layout with 13 electrodes over motor cortex and frontal lobe, averaged over all folds (k = 10) for MNS vs. MI+MNS condition. Input data filtered in 4-38 Hz frequency band.

Figure 5 . 18 :

 518 Figure 5.18: Classification accuracies obtained by different models for 22 subjects using layout with 6 electrodes over one side of motor cortex and frontal lobe, averaged over all folds (k = 10) for MNS vs. MI+MNS condition. Input data filtered in 4-38 Hz frequency band.

Figure 5 . 19 :

 519 Figure 5.19: Boxplots showing the distribution of different models classification accuracy using layout with 128 electrodes, averaged over all folds (k = 10) and all subjects (n = 22) for MNS vs. MI+MNS discriminating task. Significance levels of Wilcoxon signed-rank test comparisons: NS -not significant, *p < 0.05, **p < 0.01, ***p < 0.001, ****p < 0.0001.

Figure 5 . 20 :

 520 Figure 5.20: Boxplots showing the distribution of different models classification accuracy using layout with 47 electrodes, averaged over all folds (k = 10) and all subjects (n = 22) for MNS vs. MI+MNS discriminating task. Significance levels of Wilcoxon signed-rank test comparisons: NS -not significant, *p < 0.05, **p < 0.01, ***p < 0.001, ****p < 0.0001.

Figure 5 . 21 :

 521 Figure 5.21: Boxplots showing the distribution of different models classification accuracy using layout with 13 electrodes over motor cortex and frontal lobe, averaged over all folds (k = 10) and all subjects (n = 22) for MNS vs. MI+MNS discriminating task. Significance levels of Wilcoxon signed-rank test comparisons: NS -not significant, *p < 0.05, **p < 0.01, ***p < 0.001, ****p < 0.0001.

Figure 5 . 22 :

 522 Figure 5.22: Boxplots showing the distribution of different models classification accuracy using layout with 6 electrodes over one side of motor cortex and frontal lobe, averaged over all folds (k = 10) and all subjects (n = 22) for MNS vs. MI+MNS discriminating task. Significance levels of Wilcoxon signed-rank test comparisons: NS -not significant, *p < 0.05, **p < 0.01, ***p < 0.001, ****p < 0.0001.

Figure 5 . 23 :

 523 Figure 5.23: Grand average classification accuracy for different electrode layouts using different models, averaged over all folds (k = 10) and all subjects (n = 22) for MI vs. rest condition. Input data filtered in 8-30 Hz frequency band.

Figure 5 . 24 :

 524 Figure 5.24: Comparison of classification accuracy for different electrode layouts (128, 47, 13, 9, 6, 3), averaged over all folds (k = 10) and all subjects (n = 22) for MNS vs. MI+MNS (5.24a) and MI vs. rest (5.24b) conditions. Input data filtered in 4-38 Hz frequency band for 5.24a and in 8-30 Hz frequency band for 5.24b.

Figure 5 . 25 :

 525 Figure 5.25: Boxplots showing the distribution of EEGNet-4.8 classification accuracy for different setups of n electrodes over motor cortex and frontal lobe, averaged over all folds (k = 10) and all subjects (n = 22) for both discriminating tasks. Significance levels of Wilcoxon signed-rank test comparisons ***p < 0.001, ****p < 0.0001.

Figure 5 . 26 :

 526 Figure 5.26: Models classification accuracy for electrode setups over motor cortex and over motor cortex and frontal lobe, averaged over all folds (k = 10) and all subjects (n = 22) for MNS vs. MI+MNS condition.

Table A. 5 :

 5 Classification accuracy obtained by different classifiers for 22 subjects using 47 channel layout for MNS vs. MNS+MI discriminating task. Signals filtered in frequency band, results averaged across 10 folds of cross-validation 0.827 0.750 1.000 0.471 0.644 0.731 0.885 0.990 0.558 0.510 0.837 0.971 0.894 0.731 0.894 0.856 1.000 0.894 0.808 0.606 0.808 0.792 MDM 0.394 0.837 0.760 1.000 0.596 0.548 0.808 0.817 0.923 0.596 0.596 0.779 0.933 0.721 0.750 0.865 0.827 0.904 0.769 0.721 0.721 0.702 0.753 TS+LR 0.760 0.837 0.750 1.000 0.692 0.615 0.808 0.885 0.990 0.587 0.587 0.885 0.952 0.933 0.788 0.885 0.865 0.981 0.904 0.827 0.779 0.788 0.823 D-ConvNet 0.777 0.898 0.910 0.981 0.713 0.627 0.765 0.970 0.991 0.685 0.751 0.865 0.981 0.922 0.894 0.943 0.923 0.944 0.905 0.934 0.623 0.818 0.855 S-ConvNet 0.815 0.887 0.904 0.990 0.753 0.623 0.739 0.943 0.963 0.805 0.885 0.752 0.970 0.924 0.876 0.942 0.925 0.990 0.943 0.885 0.771 0.787 0.867 EEGNet-2.32 0.835 0.896 0.915 0.910 0.739 0.547 0.845 0.933 1.000 0.827 0.865 0.940 0.990 0.923 0.895 0.905 0.865 0.990 0.883 0.875 0.762 0.868 0.873 EEGNet-4.8 0.922 0.954 0.915 0.970 0.895 0.529 0.990 0.952 1.000 0.790 0.971 0.960 0.972 0.933 0.855 0.942 0.943 0.990 0.980 0.903 0.867 0.932 0.916Table A.6: Classification accuracy obtained by different classifiers for 22 subjects using 47 channel layout for MNS vs. MNS+MI discriminating task. Signals filtered in frequency band, results averaged across 10 folds of cross-validation 0.817 0.788 1.000 0.558 0.606 0.750 0.875 0.981 0.548 0.481 0.769 0.952 0.923 0.769 0.846 0.856 1.000 0.885 0.788 0.625 0.769 0.787 MDM 0.510 0.837 0.779 1.000 0.615 0.596 0.808 0.808 0.837 0.606 0.596 0.769 0.904 0.721 0.750 0.865 0.817 0.904 0.769 0.721 0.673 0.673 0.753 TS+LR 0.721 0.837 0.740 1.000 0.615 0.596 0.846 0.885 0.952 0.635 0.606 0.875 0.971 0.923 0.779 0.894 0.865 0.952 0.913 0.788 0.788 0.779 0.816 D-ConvNet 0.830 0.936 0.903 0.990 0.730 0.567 0.631 0.940 0.991 0.568 0.720 0.734 0.972 0.972 0.805 0.952 0.907 0.971 0.914 0.865 0.635 0.710 0.829 S-ConvNet 0.795 0.887 0.864 1.000 0.744 0.545 0.729 0.933 0.888 0.825 0.856 0.702 0.893 0.902 0.914 0.942 0.924 1.000 0.923 0.892 0.771 0.758 0.849 EEGNet-2.32 0.823 0.906 0.943 0.960 0.719 0.558 0.748 0.943 1.000 0.778 0.875 0.922 0.972 0.932 0.855 0.915 0.905 0.990 0.894 0.894 0.838 0.821 0.872 EEGNet-4.8 0.862 0.954 0.893 0.940 0.758 0.548 0.815 0.952 1.000 0.802 0.961 0.932 0.953 0.933 0.934 0.924 0.971 0.990 0.970 0.921 0.887 0.944 0.902 Table A.7: Classification accuracy obtained by different classifiers for 22 subjects using 47 channel layout for MNS vs. MNS+MI discriminating task. Signals filtered in (8-30 Hz) frequency band, results averaged across 10 folds of cross-validation 0.760 0.750 1.000 0.529 0.615 0.692 0.817 0.981 0.558 0.471 0.769 0.904 0.865 0.673 0.885 0.750 1.000 0.913 0.683 0.692 0.827 0.765 MDM 0.394 0.846 0.760 1.000 0.606 0.548 0.817 0.837 0.856 0.538 0.558 0.750 0.913 0.721 0.721 0.885 0.846 0.904 0.760 0.721 0.654 0.731 0.744 TS+LR 0.644 0.827 0.769 1.000 0.635 0.644 0.827 0.846 0.962 0.587 0.519 0.913 0.971 0.923 0.702 0.894 0.865 0.942 0.894 0.817 0.760 0.817 0.807 D-ConvNet 0.617 0.890 0.901 0.990 0.595 0.586 0.598 0.875 0.962 0.616 0.645 0.885 0.971 0.837 0.786 0.756 0.838 0.981 0.850 0.694 0.556 0.682 0.778 S-ConvNet 0.779 0.888 0.865 0.971 0.707 0.584 0.718 0.865 0.876 0.681 0.877 0.750 0.962 0.789 0.729 0.876 0.807 0.961 0.838 0.741 0.703 0.738 0.805 EEGNet-2.32 0.765 0.886 0.952 0.892 0.693 0.519 0.834 0.865 1.000 0.846 0.762 0.924 0.990 0.893 0.785 0.863 0.857 0.990 0.943 0.865 0.808 0.845 0.854 EEGNet-4.8 0.855 0.935 0.892 0.940 0.751 0.538 0.882 0.867 1.000 0.714 0.820 0.960 0.962 0.931 0.855 0.902 0.903 0.990 0.943 0.852 0.859 0.924 0.876 Table A.8: Classification accuracy obtained by different classifiers for 22 subjects using 47 channel layout for MNS vs. MNS+MI discriminating task. Signals filtered in (8-38 Hz) frequency band, results averaged across 10 folds of cross-validation 0.808 0.760 1.000 0.587 0.625 0.712 0.808 0.990 0.452 0.433 0.904 0.971 0.827 0.596 0.894 0.788 1.000 0.913 0.596 0.740 0.875 0.774 MDM 0.231 0.846 0.740 1.000 0.606 0.558 0.817 0.846 0.913 0.538 0.548 0.779 0.952 0.712 0.731 0.865 0.846 0.904 0.750 0.712 0.712 0.721 0.742 TS+LR 0.731 0.827 0.750 1.000 0.731 0.673 0.798 0.837 0.981 0.606 0.519 0.923 0.962 0.923 0.721 0.885 0.865 0.981 0.894 0.837 0.750 0.827 0.819 D-ConvNet 0.675 0.936 0.921 0.971 0.666 0.608 0.794 0.776 0.991 0.688 0.685 0.923 0.972 0.874 0.757 0.826 0.778 0.981 0.935 0.749 0.585 0.838 0.815 S-ConvNet 0.749 0.889 0.885 0.981 0.689 0.605 0.729 0.836 0.951 0.713 0.904 0.816 0.944 0.815 0.755 0.873 0.857 0.980 0.809 0.817 0.683 0.779 0.821 EEGNet-2.32 0.845 0.925 0.950 0.930 0.819 0.528 0.882 0.904 1.000 0.700 0.791 0.942 0.990 0.905 0.806 0.865 0.865 1.000 0.923 0.835 0.799 0.884 0.868 EEGNet-4.8 0.915 0.935 0.922 0.940 0.896 0.529 0.961 0.886 1.000 0.794 0.868 0.980 0.972 0.924 0.845 0.895 0.895 0.990 0.914 0.855 0.877 0.953 0.898 Table A.13: Classification accuracy obtained by different classifiers for 22 subjects using 13 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (4-38 Hz) frequency band, results averaged across 10 folds of cross-validation 0.827 0.740 0.654 0.702 0.635 0.740 0.856 0.721 0.750 0.452 0.683 0.933 0.913 0.750 0.808 0.875 1.000 0.875 0.654 0.740 0.663 0.758 MDM 0.538 0.721 0.769 0.817 0.779 0.635 0.683 0.808 0.683 0.692 0.577 0.702 0.779 0.779 0.769 0.769 0.856 0.990 0.760 0.625 0.663 0.654 0.729 TS+LR 0.692 0.846 0.779 0.769 0.750 0.740 0.769 0.894 0.760 0.788 0.471 0.702 0.942 0.933 0.760 0.769 0.904 1.000 0.865 0.731 0.779 0.673 0.787 D-ConvNet 0.650 0.906 0.881 0.875 0.711 0.632 0.605 0.770 0.770 0.616 0.779 0.657 0.982 0.961 0.854 0.769 0.895 0.971 0.963 0.794 0.570 0.673 0.786 S-ConvNet 0.728 0.866 0.864 0.894 0.846 0.599 0.750 0.855 0.792 0.778 0.866 0.701 0.962 0.941 0.722 0.856 0.945 1.000 0.933 0.845 0.798 0.770 0.832 EEGNet-2.32 0.777 0.925 0.885 0.863 0.759 0.643 0.746 0.911 0.820 0.741 0.864 0.885 0.981 0.933 0.837 0.874 0.876 1.000 0.925 0.865 0.811 0.799 0.851 EEGNet-4.8 0.838 0.935 0.894 0.893 0.798 0.665 0.769 0.893 0.850 0.810 0.952 0.940 0.972 0.953 0.884 0.882 0.935 1.000 0.981 0.834 0.916 0.780 0.880 Table A.14: Classification accuracy obtained by different classifiers for 22 subjects using 13 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (4-30 Hz) frequency band, results averaged across 10 folds of cross-validation 0.808 0.788 0.702 0.692 0.625 0.721 0.827 0.712 0.740 0.500 0.663 0.865 0.865 0.740 0.817 0.885 1.000 0.875 0.635 0.731 0.683 0.753 MDM 0.625 0.740 0.750 0.817 0.769 0.625 0.673 0.808 0.702 0.673 0.587 0.673 0.740 0.817 0.779 0.779 0.846 0.990 0.760 0.606 0.702 0.625 0.731 TS+LR 0.712 0.808 0.788 0.769 0.760 0.721 0.779 0.904 0.769 0.760 0.500 0.712 0.817 0.913 0.740 0.817 0.885 0.990 0.865 0.731 0.788 0.683 0.782 D-ConvNet 0.722 0.865 0.892 0.941 0.730 0.653 0.612 0.770 0.768 0.615 0.824 0.704 0.963 0.971 0.873 0.681 0.943 0.953 0.952 0.784 0.581 0.672 0.794 S-ConvNet 0.709 0.743 0.845 0.904 0.865 0.609 0.759 0.855 0.791 0.787 0.885 0.682 0.885 0.934 0.752 0.875 0.934 0.991 0.933 0.815 0.749 0.777 0.822 EEGNet-2.32 0.750 0.828 0.874 0.844 0.816 0.577 0.815 0.882 0.819 0.712 0.854 0.817 0.953 0.895 0.856 0.874 0.895 1.000 0.945 0.865 0.788 0.759 0.837 EEGNet-4.8 0.840 0.934 0.854 0.904 0.818 0.662 0.709 0.892 0.841 0.790 0.952 0.845 0.962 0.925 0.875 0.864 0.924 1.000 0.961 0.805 0.885 0.809 0.866 Table A.15: Classification accuracy obtained by different classifiers for 22 subjects using 13 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (8-30 Hz) frequency band, results averaged across 10 folds of cross-validation 0.731 0.788 0.654 0.740 0.721 0.587 0.817 0.663 0.702 0.490 0.692 0.894 0.865 0.692 0.712 0.798 1.000 0.865 0.615 0.788 0.692 0.733 MDM 0.462 0.731 0.788 0.808 0.779 0.673 0.673 0.779 0.606 0.663 0.606 0.673 0.760 0.788 0.808 0.750 0.750 0.990 0.760 0.587 0.721 0.673 0.719 TS+LR 0.519 0.788 0.788 0.740 0.760 0.750 0.769 0.837 0.740 0.760 0.538 0.740 0.865 0.933 0.779 0.769 0.875 0.990 0.904 0.615 0.827 0.673 0.771 D-ConvNet 0.675 0.732 0.844 0.757 0.672 0.595 0.518 0.596 0.713 0.667 0.663 0.674 0.944 0.839 0.597 0.510 0.817 0.962 0.925 0.605 0.587 0.664 0.707 S-ConvNet 0.779 0.839 0.817 0.711 0.807 0.598 0.691 0.770 0.772 0.652 0.790 0.702 0.916 0.912 0.738 0.749 0.780 0.971 0.877 0.683 0.816 0.711 0.776 EEGNet-2.32 0.700 0.869 0.814 0.825 0.788 0.556 0.709 0.790 0.724 0.685 0.915 0.848 0.962 0.923 0.854 0.795 0.837 1.000 0.907 0.691 0.799 0.625 0.801 EEGNet-4.8 0.733 0.895 0.823 0.816 0.836 0.615 0.682 0.788 0.783 0.705 0.961 0.835 0.943 0.895 0.826 0.738 0.894 1.000 0.945 0.777 0.850 0.838 0.826 Table A.16: Classification accuracy obtained by different classifiers for 22 subjects using 13 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (8-38 Hz) frequency band, results averaged across 10 folds of cross-validation 0.788 0.731 0.673 0.731 0.760 0.625 0.827 0.625 0.702 0.490 0.663 0.952 0.865 0.702 0.673 0.837 0.990 0.865 0.673 0.846 0.683 0.741 MDM 0.346 0.731 0.769 0.817 0.788 0.673 0.673 0.798 0.606 0.673 0.596 0.663 0.846 0.779 0.798 0.721 0.788 0.990 0.750 0.606 0.683 0.702 0.718 TS+LR 0.567 0.808 0.779 0.740 0.760 0.760 0.731 0.827 0.731 0.760 0.529 0.740 0.942 0.913 0.798 0.760 0.904 1.000 0.894 0.635 0.827 0.673 0.776 D-ConvNet 0.653 0.905 0.862 0.700 0.748 0.536 0.627 0.664 0.660 0.618 0.638 0.796 0.981 0.791 0.716 0.585 0.738 0.971 0.905 0.568 0.615 0.625 0.723 S-ConvNet 0.800 0.888 0.863 0.732 0.835 0.628 0.680 0.771 0.790 0.719 0.838 0.712 0.981 0.863 0.748 0.786 0.836 1.000 0.905 0.671 0.800 0.741 0.799 EEGNet-2.32 0.703 0.867 0.902 0.786 0.715 0.565 0.729 0.758 0.705 0.705 0.850 0.856 0.981 0.913 0.815 0.844 0.846 1.000 0.917 0.750 0.800 0.762 0.808 EEGNet-4.8 0.755 0.915 0.892 0.807 0.768 0.558 0.739 0.799 0.781 0.754 0.904 0.950 0.990 0.933 0.828 0.805 0.835 1.000 0.924 0.747 0.866 0.773 0.833 Table A.21: Classification accuracy obtained by different classifiers for 22 subjects using 9 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (4-38 Hz) frequency band, results averaged across 10 folds of cross-validation 0.760 0.712 0.702 0.683 0.683 0.692 0.808 0.721 0.769 0.442 0.692 0.433 0.856 0.750 0.856 0.750 0.990 0.885 0.654 0.750 0.644 0.729 MDM 0.731 0.740 0.760 0.760 0.788 0.644 0.712 0.760 0.673 0.702 0.548 0.721 0.529 0.731 0.769 0.760 0.625 0.962 0.769 0.606 0.654 0.606 0.707 TS+LR 0.779 0.798 0.750 0.692 0.731 0.654 0.740 0.798 0.625 0.779 0.490 0.740 0.577 0.894 0.769 0.769 0.760 0.981 0.856 0.673 0.788 0.596 0.738 D-ConvNet 0.727 0.855 0.891 0.805 0.751 0.699 0.595 0.750 0.779 0.635 0.883 0.733 0.931 0.954 0.834 0.738 0.934 0.954 0.972 0.735 0.699 0.626 0.795 S-ConvNet 0.748 0.811 0.825 0.864 0.807 0.635 0.691 0.788 0.839 0.719 0.817 0.734 0.719 0.942 0.796 0.865 0.903 1.000 0.934 0.734 0.767 0.759 0.804 EEGNet-2.32 0.845 0.885 0.885 0.854 0.778 0.617 0.752 0.883 0.800 0.703 0.951 0.749 0.884 0.944 0.876 0.825 0.904 1.000 0.934 0.797 0.835 0.673 0.835 EEGNet-4.8 0.806 0.867 0.884 0.835 0.788 0.623 0.767 0.884 0.840 0.811 0.932 0.809 0.933 0.944 0.875 0.854 0.914 1.000 0.981 0.756 0.855 0.757 0.851 Table A.22: Classification accuracy obtained by different classifiers for 22 subjects using 9 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (4-30 Hz) frequency band, results averaged across 10 folds of cross-validation 0.692 0.721 0.721 0.663 0.683 0.721 0.788 0.663 0.721 0.442 0.712 0.481 0.846 0.760 0.808 0.740 1.000 0.894 0.654 0.712 0.596 0.717 MDM 0.731 0.712 0.750 0.760 0.750 0.663 0.712 0.760 0.683 0.702 0.567 0.769 0.510 0.740 0.769 0.760 0.654 0.971 0.769 0.606 0.692 0.577 0.709 TS+LR 0.779 0.740 0.740 0.721 0.721 0.654 0.760 0.798 0.663 0.760 0.529 0.740 0.567 0.885 0.769 0.798 0.779 0.971 0.875 0.692 0.760 0.577 0.740 D-ConvNet 0.799 0.745 0.872 0.802 0.770 0.625 0.630 0.690 0.694 0.502 0.883 0.797 0.678 0.952 0.854 0.777 0.943 0.924 0.952 0.710 0.699 0.645 0.770 S-ConvNet 0.729 0.770 0.815 0.864 0.826 0.645 0.700 0.788 0.822 0.718 0.828 0.763 0.732 0.932 0.798 0.875 0.865 0.971 0.934 0.714 0.739 0.729 0.798 EEGNet-2.32 0.835 0.838 0.863 0.825 0.777 0.635 0.762 0.873 0.839 0.719 0.941 0.768 0.729 0.934 0.895 0.836 0.865 1.000 0.954 0.789 0.816 0.683 0.826 EEGNet-4.8 0.808 0.865 0.874 0.882 0.827 0.612 0.757 0.875 0.849 0.760 0.932 0.826 0.789 0.933 0.885 0.864 0.903 1.000 0.961 0.804 0.798 0.690 0.841 Table A.23: Classification accuracy obtained by different classifiers for 22 subjects using 9 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (8-30 Hz) frequency band, results averaged across 10 folds of cross-validation 0.750 0.702 0.702 0.683 0.692 0.654 0.817 0.606 0.731 0.433 0.683 0.462 0.856 0.808 0.750 0.702 0.981 0.885 0.654 0.837 0.577 0.710 MDM 0.615 0.731 0.779 0.760 0.760 0.663 0.702 0.740 0.635 0.683 0.577 0.625 0.471 0.731 0.779 0.760 0.538 0.962 0.760 0.567 0.702 0.615 0.689 TS+LR 0.635 0.740 0.731 0.721 0.712 0.740 0.721 0.808 0.673 0.760 0.529 0.683 0.558 0.894 0.769 0.760 0.635 0.981 0.875 0.587 0.798 0.635 0.725 D-ConvNet 0.684 0.693 0.834 0.690 0.722 0.664 0.624 0.665 0.638 0.576 0.676 0.699 0.602 0.815 0.669 0.510 0.790 0.952 0.903 0.539 0.777 0.585 0.696 S-ConvNet 0.711 0.827 0.824 0.760 0.785 0.559 0.633 0.798 0.763 0.702 0.817 0.731 0.665 0.846 0.796 0.826 0.749 0.980 0.925 0.605 0.771 0.673 0.761 EEGNet-2.32 0.759 0.771 0.807 0.807 0.768 0.595 0.739 0.790 0.753 0.683 0.914 0.742 0.737 0.876 0.857 0.786 0.828 1.000 0.963 0.656 0.825 0.603 0.785 EEGNet-4.8 0.787 0.847 0.872 0.837 0.788 0.639 0.755 0.771 0.784 0.762 0.912 0.788 0.799 0.914 0.855 0.796 0.865 1.000 0.945 0.655 0.826 0.680 0.813 Table A.24: Classification accuracy obtained by different classifiers for 22 subjects using 9 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (8-38 Hz) frequency band, results averaged across 10 folds of cross-validation 0.779 0.712 0.692 0.683 0.702 0.673 0.837 0.577 0.721 0.423 0.654 0.519 0.865 0.779 0.712 0.750 0.981 0.885 0.625 0.760 0.663 0.711 MDM 0.615 0.750 0.740 0.779 0.798 0.673 0.721 0.740 0.635 0.692 0.567 0.615 0.490 0.702 0.779 0.731 0.538 0.962 0.760 0.548 0.673 0.606 0.687 TS+LR 0.635 0.779 0.721 0.712 0.740 0.740 0.712 0.837 0.673 0.750 0.510 0.673 0.615 0.904 0.769 0.740 0.721 0.971 0.865 0.577 0.817 0.663 0.733 D-ConvNet 0.735 0.876 0.842 0.730 0.759 0.613 0.723 0.643 0.547 0.586 0.695 0.694 0.894 0.730 0.682 0.528 0.775 0.954 0.876 0.483 0.728 0.655 0.716 S-ConvNet 0.729 0.887 0.827 0.797 0.795 0.557 0.661 0.809 0.781 0.713 0.790 0.721 0.750 0.875 0.712 0.789 0.827 0.991 0.915 0.566 0.769 0.665 0.769 EEGNet-2.32 0.788 0.887 0.846 0.826 0.730 0.595 0.748 0.789 0.725 0.751 0.855 0.779 0.943 0.886 0.857 0.776 0.875 1.000 0.963 0.686 0.806 0.663 0.808 EEGNet-4.8 0.799 0.867 0.863 0.856 0.798 0.600 0.796 0.829 0.837 0.800 0.913 0.845 0.942 0.915 0.854 0.784 0.855 1.000 0.935 0.673 0.807 0.771 0.834 Table A.29: Classification accuracy obtained by different classifiers for 22 subjects using 6 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (4-38 Hz) frequency band, results averaged across 10 folds of cross-validation 0.827 0.740 0.673 0.750 0.692 0.731 0.788 0.779 0.788 0.587 0.692 0.452 0.846 0.750 0.798 0.750 0.971 0.865 0.529 0.769 0.490 0.727 MDM 0.760 0.760 0.760 0.740 0.769 0.635 0.673 0.798 0.663 0.673 0.558 0.683 0.558 0.721 0.779 0.769 0.596 0.971 0.750 0.567 0.712 0.433 0.697 TS+LR 0.798 0.808 0.740 0.721 0.769 0.712 0.663 0.856 0.712 0.779 0.558 0.731 0.538 0.856 0.731 0.817 0.740 0.971 0.894 0.558 0.788 0.577 0.742 D-ConvNet 0.751 0.866 0.863 0.719 0.741 0.667 0.555 0.874 0.789 0.705 0.753 0.655 0.718 0.925 0.577 0.652 0.944 0.943 0.963 0.586 0.746 0.596 0.754 S-ConvNet 0.778 0.857 0.903 0.786 0.834 0.645 0.625 0.807 0.782 0.645 0.762 0.645 0.758 0.911 0.787 0.826 0.866 1.000 0.924 0.693 0.796 0.722 0.789 EEGNet-2.32 0.765 0.867 0.835 0.797 0.750 0.770 0.661 0.894 0.789 0.752 0.825 0.808 0.894 0.886 0.767 0.795 0.885 1.000 0.955 0.675 0.816 0.681 0.812 EEGNet-4.8 0.808 0.905 0.875 0.845 0.795 0.698 0.633 0.894 0.877 0.774 0.818 0.769 0.902 0.935 0.815 0.795 0.925 1.000 0.915 0.738 0.806 0.791 0.832 Table A.30:Classification accuracy obtained by different classifiers for 22 subjects using 6 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (4-30 Hz) frequency band, results averaged across 10 folds of cross-validation 0.769 0.769 0.721 0.740 0.712 0.760 0.788 0.769 0.740 0.587 0.683 0.442 0.865 0.779 0.827 0.740 0.971 0.885 0.606 0.740 0.548 0.733 MDM 0.731 0.731 0.779 0.721 0.750 0.635 0.673 0.798 0.663 0.673 0.558 0.663 0.529 0.731 0.760 0.788 0.644 0.971 0.750 0.567 0.721 0.433 0.694 TS+LR 0.817 0.779 0.779 0.731 0.769 0.712 0.683 0.865 0.750 0.779 0.548 0.721 0.519 0.846 0.721 0.837 0.760 0.971 0.904 0.558 0.740 0.587 0.744 D-ConvNet 0.752 0.817 0.844 0.797 0.710 0.706 0.546 0.854 0.817 0.619 0.758 0.657 0.725 0.923 0.567 0.625 0.905 0.904 0.954 0.528 0.710 0.576 0.741 S-ConvNet 0.759 0.818 0.893 0.785 0.816 0.616 0.606 0.797 0.790 0.681 0.750 0.654 0.768 0.931 0.729 0.865 0.835 0.990 0.905 0.673 0.758 0.693 0.778 EEGNet-2.32 0.815 0.857 0.844 0.796 0.815 0.759 0.661 0.894 0.840 0.722 0.816 0.779 0.721 0.885 0.767 0.795 0.894 1.000 0.954 0.724 0.825 0.675 0.811 EEGNet-4.8 0.797 0.817 0.864 0.826 0.789 0.737 0.644 0.884 0.849 0.696 0.834 0.785 0.755 0.916 0.767 0.805 0.857 1.000 0.953 0.711 0.807 0.722 0.810Table A.31:Classification accuracy obtained by different classifiers for 22 subjects using 6 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (8-30 Hz) frequency band, results averaged across 10 folds of cross-validation 0.740 0.712 0.663 0.692 0.760 0.692 0.856 0.692 0.779 0.471 0.683 0.404 0.846 0.798 0.750 0.644 0.942 0.856 0.587 0.788 0.644 0.712 MDM 0.625 0.760 0.692 0.798 0.769 0.731 0.683 0.760 0.625 0.663 0.625 0.635 0.404 0.712 0.779 0.740 0.548 0.942 0.760 0.538 0.740 0.615 0.688 TS+LR 0.702 0.769 0.731 0.740 0.731 0.769 0.683 0.837 0.712 0.750 0.510 0.721 0.394 0.827 0.788 0.760 0.663 0.971 0.856 0.596 0.827 0.644 0.726 D-ConvNet 0.646 0.855 0.783 0.808 0.781 0.659 0.541 0.761 0.682 0.695 0.647 0.721 0.531 0.827 0.558 0.540 0.788 0.913 0.886 0.589 0.645 0.595 0.702 S-ConvNet 0.682 0.847 0.866 0.758 0.785 0.655 0.649 0.825 0.771 0.651 0.831 0.750 0.645 0.856 0.708 0.817 0.749 0.991 0.866 0.664 0.767 0.690 0.765 EEGNet-2.32 0.709 0.827 0.741 0.780 0.809 0.756 0.653 0.840 0.724 0.645 0.799 0.799 0.641 0.897 0.739 0.795 0.808 1.000 0.896 0.645 0.863 0.644 0.773 EEGNet-4.8 0.825 0.750 0.807 0.759 0.799 0.737 0.691 0.846 0.742 0.783 0.816 0.789 0.710 0.935 0.747 0.815 0.807 1.000 0.857 0.712 0.796 0.680 0.791 Table A.32: Classification accuracy obtained by different classifiers for 22 subjects using 6 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (8-38 Hz) frequency band, results averaged across 10 folds of cross-validation 0.788 0.692 0.673 0.721 0.721 0.692 0.865 0.692 0.750 0.471 0.702 0.356 0.808 0.779 0.702 0.625 0.981 0.817 0.567 0.808 0.606 0.706 MDM 0.635 0.779 0.692 0.788 0.779 0.731 0.663 0.769 0.635 0.673 0.625 0.635 0.413 0.692 0.788 0.721 0.538 0.962 0.760 0.548 0.740 0.606 0.690 TS+LR 0.731 0.808 0.712 0.769 0.760 0.750 0.663 0.846 0.692 0.750 0.500 0.731 0.404 0.827 0.750 0.740 0.683 0.971 0.846 0.606 0.827 0.635 0.727 D-ConvNet 0.627 0.855 0.806 0.700 0.760 0.641 0.559 0.790 0.691 0.734 0.665 0.703 0.767 0.754 0.518 0.520 0.748 0.952 0.868 0.596 0.765 0.615 0.711 S-ConvNet 0.681 0.896 0.857 0.775 0.835 0.684 0.670 0.874 0.788 0.673 0.819 0.712 0.732 0.876 0.682 0.798 0.855 1.000 0.856 0.661 0.733 0.652 0.778 EEGNet-2.32 0.729 0.847 0.796 0.771 0.732 0.768 0.702 0.906 0.715 0.675 0.759 0.788 0.835 0.867 0.769 0.785 0.883 1.000 0.896 0.596 0.798 0.618 0.784 EEGNet-4.8 0.806 0.858 0.885 0.796 0.816 0.727 0.683 0.780 0.769 0.764 0.827 0.831 0.885 0.907 0.717 0.797 0.845 1.000 0.830 0.699 0.796 0.661 0.804 Table A.37: Classification accuracy obtained by different classifiers for 22 subjects using 3 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (4-38 Hz) frequency band, results averaged across 10 folds of cross-validation 0.740 0.683 0.760 0.519 0.558 0.577 0.827 0.625 0.625 0.577 0.644 0.433 0.750 0.731 0.740 0.558 0.875 0.769 0.558 0.635 0.538 0.655 MDM 0.702 0.644 0.673 0.740 0.490 0.490 0.606 0.750 0.596 0.644 0.596 0.625 0.433 0.663 0.712 0.779 0.510 0.885 0.750 0.538 0.683 0.385 0.632 TS+LR 0.779 0.731 0.683 0.750 0.510 0.548 0.596 0.798 0.606 0.635 0.567 0.663 0.413 0.721 0.712 0.779 0.538 0.885 0.731 0.606 0.683 0.510 0.656 D-ConvNet 0.809 0.869 0.835 0.767 0.663 0.589 0.589 0.864 0.693 0.605 0.638 0.722 0.737 0.799 0.692 0.661 0.837 0.895 0.883 0.615 0.586 0.674 0.728 S-ConvNet 0.749 0.866 0.836 0.767 0.683 0.587 0.642 0.874 0.667 0.589 0.734 0.722 0.719 0.815 0.702 0.768 0.827 0.905 0.816 0.683 0.646 0.645 0.738 EEGNet-2.32 0.775 0.867 0.815 0.806 0.644 0.548 0.634 0.845 0.684 0.665 0.744 0.721 0.843 0.835 0.709 0.795 0.808 0.953 0.882 0.639 0.696 0.635 0.752 EEGNet-4.8 0.750 0.916 0.866 0.804 0.701 0.624 0.652 0.856 0.683 0.685 0.712 0.749 0.863 0.827 0.670 0.767 0.818 0.962 0.893 0.670 0.664 0.665 0.764 Table A.38: Classification accuracy obtained by different classifiers for 22 subjects using 3 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (4-30 Hz) frequency band, results averaged across 10 folds of cross-validation 0.731 0.740 0.750 0.500 0.567 0.577 0.846 0.596 0.615 0.596 0.644 0.413 0.798 0.712 0.721 0.529 0.875 0.740 0.567 0.663 0.538 0.654 MDM 0.702 0.625 0.721 0.750 0.490 0.490 0.615 0.750 0.596 0.654 0.596 0.635 0.433 0.712 0.721 0.779 0.519 0.875 0.750 0.538 0.712 0.394 0.639 TS+LR 0.779 0.721 0.721 0.750 0.519 0.548 0.615 0.808 0.596 0.663 0.567 0.654 0.413 0.750 0.712 0.788 0.558 0.865 0.769 0.606 0.683 0.510 0.663 D-ConvNet 0.827 0.799 0.825 0.775 0.635 0.607 0.558 0.844 0.703 0.605 0.675 0.665 0.650 0.798 0.682 0.690 0.758 0.905 0.846 0.587 0.566 0.653 0.712 S-ConvNet 0.760 0.828 0.767 0.750 0.674 0.588 0.585 0.865 0.678 0.609 0.675 0.694 0.682 0.814 0.691 0.779 0.781 0.884 0.826 0.605 0.696 0.645 0.722 EEGNet-2.32 0.757 0.808 0.835 0.777 0.644 0.557 0.633 0.874 0.731 0.643 0.732 0.741 0.699 0.855 0.767 0.777 0.702 0.942 0.853 0.633 0.638 0.702 0.741 EEGNet-4.8 0.761 0.827 0.836 0.815 0.691 0.604 0.623 0.846 0.693 0.695 0.722 0.741 0.717 0.827 0.727 0.777 0.769 0.961 0.864 0.595 0.664 0.664 0.746 Table A.39: Classification accuracy obtained by different classifiers for 22 subjects using 3 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (8-30 Hz) frequency band, results averaged across 10 folds of cross-validation 0.702 0.683 0.788 0.404 0.567 0.625 0.779 0.692 0.635 0.529 0.635 0.462 0.712 0.712 0.740 0.510 0.827 0.721 0.500 0.654 0.644 0.644 MDM 0.606 0.606 0.673 0.760 0.423 0.481 0.635 0.750 0.596 0.625 0.596 0.615 0.365 0.596 0.721 0.779 0.481 0.856 0.702 0.567 0.692 0.529 0.621 TS+LR 0.663 0.654 0.692 0.779 0.471 0.548 0.606 0.760 0.654 0.635 0.490 0.663 0.413 0.635 0.702 0.788 0.529 0.846 0.750 0.567 0.654 0.596 0.641 D-ConvNet 0.643 0.750 0.864 0.698 0.587 0.575 0.595 0.828 0.625 0.598 0.608 0.625 0.575 0.704 0.661 0.671 0.604 0.922 0.785 0.491 0.588 0.720 0.669 S-ConvNet 0.748 0.828 0.816 0.729 0.605 0.595 0.701 0.865 0.687 0.596 0.745 0.692 0.645 0.771 0.672 0.770 0.596 0.856 0.808 0.635 0.610 0.656 0.710 EEGNet-2.32 0.796 0.779 0.749 0.723 0.615 0.585 0.664 0.801 0.723 0.547 0.684 0.702 0.635 0.685 0.712 0.732 0.711 0.923 0.768 0.555 0.674 0.569 0.697 EEGNet-4.8 0.770 0.731 0.846 0.733 0.645 0.605 0.680 0.770 0.703 0.635 0.695 0.714 0.635 0.720 0.748 0.769 0.732 0.951 0.749 0.571 0.665 0.711 0.717 Table A.40: Classification accuracy obtained by different classifiers for 22 subjects using 3 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (8-38 Hz) frequency band, results averaged across 10 folds of cross-validation 0.721 0.673 0.788 0.404 0.587 0.625 0.760 0.663 0.644 0.519 0.635 0.462 0.615 0.702 0.692 0.558 0.885 0.712 0.529 0.654 0.615 0.641 MDM 0.567 0.625 0.644 0.740 0.433 0.500 0.625 0.760 0.606 0.635 0.615 0.596 0.356 0.606 0.721 0.760 0.538 0.875 0.702 0.558 0.683 0.529 0.622 TS+LR 0.644 0.692 0.673 0.769 0.452 0.548 0.606 0.750 0.625 0.644 0.500 0.663 0.394 0.644 0.712 0.769 0.577 0.865 0.740 0.567 0.673 0.606 0.642 D-ConvNet 0.703 0.831 0.797 0.669 0.519 0.604 0.647 0.781 0.610 0.586 0.599 0.645 0.787 0.694 0.613 0.682 0.713 0.895 0.728 0.618 0.598 0.678 0.682 S-ConvNet 0.747 0.849 0.788 0.748 0.595 0.578 0.692 0.826 0.725 0.637 0.752 0.703 0.730 0.760 0.680 0.760 0.788 0.905 0.771 0.675 0.607 0.683 0.727 EEGNet-2.32 0.778 0.859 0.846 0.721 0.588 0.615 0.634 0.820 0.683 0.584 0.655 0.701 0.854 0.740 0.700 0.720 0.829 0.943 0.825 0.632 0.636 0.655 0.728 EEGNet-4.8 0.808 0.898 0.872 0.762 0.655 0.614 0.711 0.750 0.703 0.615 0.638 0.725 0.835 0.792 0.680 0.807 0.817 0.933 0.785 0.621 0.605 0.703 0.742

  .26 Classification accuracy obtained by different classifiers using 6 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (4-30 Hz) frequency band . . . . . A.27 Classification accuracy obtained by different classifiers using 6 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (8-30 Hz) frequency band . . . . . A.28 Classification accuracy obtained by different classifiers using 6 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (8-38 Hz) frequency band . . . . . A.29 Classification accuracy obtained by different classifiers using 6 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (4-38 Hz) frequency band . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.30 Classification accuracy obtained by different classifiers using 6 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (4-30 Hz) frequency band . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.31 Classification accuracy obtained by different classifiers using 6 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (8-30 Hz) frequency band . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.32 Classification accuracy obtained by different classifiers using 6 channel layout over motor cortex and frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (8-38 Hz) frequency band . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.33 Classification accuracy obtained by different classifiers using 6 channel layout over motor cortex and frontal lobe from left side of the brain for MNS vs. MNS+MI discriminating task. Signals filtered in (4-38 Hz) frequency band . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.34 Classification accuracy obtained by different classifiers using 6 channel layout over motor cortex and frontal lobe from left side of the brain for MNS vs. .37 Classification accuracy obtained by different classifiers using 3 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (4-38 Hz) frequency band . . . . . A.38 Classification accuracy obtained by different classifiers using 3 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (4-30 Hz) frequency band . . . . . A.39 Classification accuracy obtained by different classifiers using 3 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (8-30 Hz) frequency band . . . . . A.40 Classification accuracy obtained by different classifiers using 3 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (8-38 Hz) frequency band . . . . . A.41 Classification accuracy obtained by different classifiers using 3 channel layout over frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (4-38 Hz) frequency band . . . . . A.42 Classification accuracy obtained by different classifiers using 3 channel layout over frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (4-30 Hz) frequency band . . . . . A.43 Classification accuracy obtained by different classifiers using 3 channel layout over frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (8-30 Hz) frequency band . . . . . A.44 Classification accuracy obtained by different classifiers using 3 channel layout over frontal lobe for MNS vs. MNS+MI discriminating task. Signals filtered in (8-38 Hz) frequency band . . . . . A.45 Classification accuracy obtained by different classifiers using 3 channel layout over frontal lobe from left side of the brain for MNS vs. MNS+MI discriminating task. Signals filtered in (4-38 Hz) frequency band . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.46 Classification accuracy obtained by different classifiers using 3 channel layout over frontal lobe Classification accuracy obtained by different variations of EEGNet using 13 channel layout over motor cortex for MI vs. rest discriminating task . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 D.4 Classification accuracy obtained by different variations of EEGNet using 13 channel layout over motor cortex and frontal lobe for MI vs. rest discriminating task . . . . . . . . . . . . . . . . . . 175 D.5 Classification accuracy obtained by different variations of EEGNet using 9 channel layout over motor cortex for MI vs. rest discriminating task . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 D.6 Classification accuracy obtained by different variations of EEGNet using 9 channel layout over motor cortex and frontal lobe for MI vs. rest discriminating task . . . . . . . . . . . . . . . . . . 175 D.7 Classification accuracy obtained by different variations of EEGNet using 6 channel layout over motor cortex for MI vs. rest discriminating task . . . . . . . . . . . . . . . . . . . . . . . . . . . 176 D.8 Classification accuracy obtained by different variations of EEGNet using 6 channel layout over motor cortex and frontal lobe for MI vs. rest discriminating task . . . . . . . . . . . . . . . . . . 176 D.9 Classification accuracy obtained by different variations of EEGNet using 6 channel layout over motor cortex and frontal lobe from left side of the brain for MI vs. rest discriminating task . . . . 176 D.10 Classification accuracy obtained by different variations of EEGNet using 3 channel layout over Classification accuracy obtained by different variations of EEGNet using 3 channel layout over frontal lobe from left side of the brain for MI vs. rest discriminating task . . . . . . . . . . . . . . 177

	LSTM : Long Short-Term Memory
	MEG : Magnetoencephalography
	MI : Motor Imagery
	ML : Machine Learning
	MNS : Median Nerve Stimulation
	PCA : Principal Component Analysis
	PSD : Power Spectral Density PSO : Particle Swarm Optimization	Glossary
	PTSD : post-traumatic stress disorder
	RBF : radial Basis Function
	RBM : Restricted Boltzmann Machine
	1d-AX : OneDimension-Aggregate Approximation ReLU : Rectified Linear Unit
	AAGA : Accidental Awareness during General Anesthesia RM : Real Movement
	AAR : AutomaticArtifact Removal RNN : Recurrent Neural Network
	ADAM : Adaptive Moment Estimation SAE : Stacked Autoencoder
	AE : Autoencoder SELU : Scaled Exponential Linear Unit
	ANN : Artificial Neural Networks SGD : Stochastic Gradient Descent
	AVG : Average
	BCI : Brain-Computer Interface
	BGRU : Bidirectional Gated Recurrent Unit
	BSS : Blind Source Separation
	CAR : Common Average Reference
	CNN : Convolutional Neural Network
	CSP : Common Spatial Pattern
	CWD : Choi-Willimas Distribution
	CWT : Continuous Wavelet Transform
	DBN : Deep Belief Networks
	DL : Deep Learning
	DNN : Deep Neural Network
	ECoG : Electrocorticography
	EEG : Electroencephalography
	ELM : Extreme Learning Machine
	ELU : Exponential Linear Unit
	EMD : Empirical Mode Decomposition
	EOG : Electrooculogram
	ERD : Event-Related Desynchronization
	ERP : Event-Related Potential
	ERS : Event-Related Synchronization
	FBCSP : Filter Bank Common Spatial Pattern
	FC : Fully Connected
	FFT : Fast Fourier Transform
	FFTEM : Fast Fourier Transform Energy Map
	fMRI : Functional Magnetic Resonance Imaging
	fNIRS : Functional Near-Infrared Spectroscopy
	FPR : False-Positive Rate
	GAN : Generative Adversarial Network
	GD : Gradient Descent
	GPU : Graphical Processor Unit
	GRU : Gated Recurrent Unit
	HGD : High-Gamma Dataset
	ICA : IndependentComponent Analysis
	KMI : Kinesthetic Motor Imagery
	LDA : Linear Discriminant Analysis
	LOF : Local Outlier Factor
		1

. . . . . 107 5.26 Models classification accuracy for electrode setups over motor cortex and over motor cortex and frontal lobe for MNS vs. MI+MNS condition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 AMNS+MI discriminating task. Signals filtered in (4-30 Hz) frequency band . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.35 Classification accuracy obtained by different classifiers using 6 channel layout over motor cortex and frontal lobe from left side of the brain for MNS vs. MNS+MI discriminating task. Signals filtered in (8-30 Hz) frequency band . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A.36 Classification accuracy obtained by different classifiers using 6 channel layout over motor cortex and frontal lobe from left side of the brain for MNS vs. MNS+MI discriminating task. Signals filtered in (8-38 Hz) frequency band . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Afrom left side of the brain for MNS vs. MNS+MI discriminating task. Signals filtered in (4-30 Hz) frequency band . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . D.1 Classification accuracy obtained by different variations of EEGNet using 128 channel layout for MI vs. rest discriminating task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 D.2 Classification accuracy obtained by different variations of EEGNet using 47 channel layout for MI vs. rest discriminating task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 D.3 motor cortex for MI vs. rest discriminating task . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 D.11 Classification accuracy obtained by different variations of EEGNet using 3 channel layout over frontal lobe for MI vs. rest discriminating task . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 xxi D.12 SNR : Signal-to-Noise Ratio SReLU : S-shaped Rectified Linear Unit SSVEP : Steady-StateVisual Evoked Potentials STFT : Short-Time Fourier Transform SVM : Support Vector Machine ULM : Upper Limb Movement VAE : Variational Autoencoder VEP : Visually Evoked Potential VI : Visual Imagery WPD : Wavelet Package Decomposition

  BCI definition and architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 1.2 Neurophysiologcal aspects of BCI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

	Chapter 1
	Brain-Computer Interfaces (BCI)
	Contents
	1.1

1.2.1 Signals for BCI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 1.2.2 Brain rhythms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.2.3 Brain rhythms localization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2.4 Motor imagery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.2.5 Event-related synchronization and desynchronization . . . . . . . . . . . . . . . . . . . 14 1.3 BCI application for general anesthesia monitoring . . . . . . . . . . . . . . . . . . . . . . . . 15 1.3.1 Accidental Awareness during General Anesthesia (AAGA) . . . . . . . . . . . . . . . . . 16 1.3.2 BCI for AAGA detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.3.3 Median nerve stimulation (MNS) in BCI . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.3.4 Influence of MNS on ERD/ERS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Table 1 . 1 :

 11 Advantages and disadvantages of different signal recording techniques , and location are the main characteristics of the EEG waveform. The following methods are used to analyze EEG signals: frequency domain, time-frequency, and non-linear techniques like correlation dimension, largest Lyapunov exponent, Hurst exponent, different entropies, fractal dimension, higher-order spectra, phase space plots, and recurrence plots.

		Advantages	Disadvantages
	ECoG	High spatial and temporal resolution, High SNR	Invasive
		High spatial and temporal resolution,	Big room,
	MEG	High SNR,	Expensive,
		Non-invasive	High degree of immobility
	fMRI	High spatial resolution, Non-invasive	Low temporal resolution, Expensive, High degree of immobility
	fNIRS	Non-invasive, Portable	Low temporal resolution
		Non-invasive,
	EEG	High temporal resolution, Portable,	Low SNR
		Cheap
	Frequency, amplitude

  Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.1 Spatial filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.2 Filtering in the frequency domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.3 Artifact removing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Feature extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.1 Time-frequency analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.2 Band power features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cross-validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 Statistical analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7.1 Wilcoxon signed-rank test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 Topographies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

	Contents
	2.1 2.2.3 Spatial analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
	2.2.4 Riemannian geometry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
	2.3 Feature selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
	2.4 Outlier detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
	2.4.1 Improving the model performance using outlier detection as quality index of imaginary
	movements estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
	2.5 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
	2.5.1 Linear discriminant analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
	2.5.2 Support vector machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
	2.5.3 Decision tree classifier . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
	2.5.4 Gradient Boosting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
	2.5.5 The Minimum Distance to Riemannian Mean Algorithm . . . . . . . . . . . . . . . . . .
	2.5.6 Proposed solution: a parallel connection of binary classifiers . . . . . . . . . . . . . . . .
	2.6

Table 2 . 1 :

 21 Codes for different states of binary classifier decoder

	Code State
	0	Rest
	1	Left hand
	2	Right hand
	3	Both hands
	4	Feet
	5	Left hand and feet
	6	Right hand and feet
	7

  Basics of artificial neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.2 Training of artificial neural networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.2.1 Optimisation algorithms for ANN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 3.3 Convolutional Neural Networks (CNN) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Overfitting and regularization techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Review of deep learning for motor imagery detection in BCI . . . . . . . . . . . . . . . . . . 58 3.6 Proposed architectures for motor intention detection . . . . . . . . . . . . . . . . . . . . . . 65 3.6.1 Deep Convolutional Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 3.6.2 Shallow Convolutional Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 3.6.3 EEGNet-2.32 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 3.6.4 EEGNet-4.8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 3.6.5 Training procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 3.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

	Chapter 3
	Deep neural networks for motor imagery
	detection
	Contents
	3.1

3.3.1 General architecture of CNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 3.3.2 Convolution layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 3.3.3 Same and valid padding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.3.4 Depthwise convolutional layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 3.3.5 Depthwise separable convolutional layer . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.3.6 Pooling layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.3.7 Fully-connected layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 3.4 3.4.1 L1 and L2 regularization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.4.2 Dropout layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3.4.3 Batch normalization layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 3.5

Table 3 . 1 :

 31 Summary of a deep learning application in MI-based BCIs

	Ref	Dataset	Freq. range (Hz)	Feature extraction	Input	DL architecture	Activation function	Optimizer
	[214] BCIC IV 2a [200]	N/M	Raw EEG	time-series	CNN	ELU	Adam
	[227] BCIC IV 2a [200]	0.5-100	Raw EEG	time-series	RNN	ReLU	N/M
	[215] BCIC IV 2a [200]	8-30	Raw EEG	time-series	CNN	Leaky ReLU	N/M
	[213] BCIC IV 2a [200]	1-40	Raw EEG	time-series	CNN, CNN+RNN	ELU	RMSProp
	[216] BCIC IV 2a [200]	1-38	Raw EEG	time-series	CNN	ELU	Adam
							Continued on next page

Table 3 . 1 -

 31 Continued from previous page

	Ref	Dataset	Freq. range (Hz)	Feature extraction	Input	DL architecture	Activation function	Optimizer
	[234]	BCIC III [198], BCIC IV [200], Private	8-26	CNN+ DWT	features LSTM	ReLU	N/M
	[235] BCIC III 4a [198]	N/M	EMD, CSP	features CNN	ReLU	N/M
	[236] BCIC III 4a [198]	0.5-90	BSS, CWT	features CNN	ReLU	N/M
				CSP+
	[237] BCIC III 4a [198]	8-15	Sparse	images	CNN	ReLU	N/M
				representation
	[238]	BCIC III 4a [198], ULM [203]	N/M	CWT	images	CNN	ReLU	GD
	[239] BCIC IV 2a [200]	4-38	3D matrices	images	CNN	ELU	Adam
	[240] BCIC IV 2a [200]	8-30	CSP	features CNN	ReLU	Adam
	[241] BCIC IV 2a [200]	4-38	FBCSP	features	CNN, LSTM	ReLU	Adam
	[242] BCIC IV 2a [200]	8-35	1d-AX	features LSTM	Sigmoid	Adam
	[243] BCIC IV 2a [200]	4-40	FBCSP	features CNN	ReLU	Adam
	[244] BCIC IV 2a [200]	N/M	FBCSP	features CNN	ReLU	Adam
				FBCSP,
	[245] BCIC IV 2a [200]	4-40	Hilbert	features CNN	ReLU	SGD
				transform
	[246] BCIC IV 2a [200]	4-30	CSP	features CNN	N/M	N/M
	[247] BCIC IV 2a [200]	8-30	CSP + FFTEM	images	CNN	ReLU	N/M
	[248]	BCIC IV 2a [200], BCIC IV 2b [200]	8-36	FBCSP, PSO	features CNN	ReLU	Adam
	[211]	BCIC IV 2a [200], BCIC IV 2b [200]	8-30	FBCSP	features GRU	ReLU, tanh	SGD
	[249]	BCIC IV 2a [200], Private	9-30	FBCSP	features CNN	ReLU	SGD
	[250] BCIC IV 2b [200] 4-35	CWT	images	CNN	ReLU	GD
	[251] BCIC IV 2b [200] 4-32	STFT	images	CNN	ReLU	N/M
	[212] BCIC IV 2b [200] 8-30	CSP	features ELM	Sigmoid	N/M
	[252] BCIC IV 2b [200] 8-35	FFT, WPD	images	DBN	Sigmoid	Conjugate GD
	[204] BCIC IV 2b [200] 8-30	STFT	images	CapsNet	SELU	SGD
	[205] BCIC IV 2b [200] 4-38	STFT	images	CapsNet	SELU	SGD
	[253]	BCIC IV 2b [200], BCIC II 3 [197]	0.1-50	CWT	images	CNN	ReLU	GD
	[206]	BCIC IV 2b [200], BCIC II 3 [197]	6-30	STFT	images	CNN + SAE	ReLU	GD
	[254]	BCIC IV 2b [200], Private	8-30	EMD	features CNN	ReLU	SGD
							Continued on next page

Table 3 . 2 :

 32 Parameters of the D-ConvNet Architecture. C is the number of channels and N = 2 is number of classes. T = 3 s x 128 Hz = 384 is the number of samples to consider for classification.

	Layer	# filters / (kernel size) Output
	Input		(1, C, T = 384)
	Conv2D	25 / (1, 5)	(25, C, 380)
	Conv2D	25 / (C, 1)	(25, 1, 380)
	BatchNorm		(25, 1, 380)
	Activation(ELU)		(25, 1, 380)
	MaxPool2D	(1, 2)	(25, 1, 190)
	Dropout		(25, 1, 190)
	Conv2D	50 / (1, 5)	(50, 1, 186)
	BatchNorm		(50, 1, 186)
	Activation(ELU)		(50, 1, 186)
	MaxPool2D	(1, 2)	(50, 1, 93)
	Dropout		(50, 1, 93)
	Conv2D	100 / (1, 5)	(100, 1, 89)
	BatchNorm		(100, 1, 89)
	Activation(ELU)		(100, 1, 89)
	MaxPool2D	(1, 2)	(100, 1, 44)
	Dropout		(100, 1, 44)
	Conv2D	200 / (1, 5)	(200, 1, 40)
	BatchNorm		(200, 1, 40)
	Activation(ELU)

Table 3 . 3 :

 33 Parameters of the S-ConvNet Architecture. C is the number of channels and N = 2 is number of classes. T = 3 s x 128 Hz = 384 is the number of samples to consider for classification. The square and log activation functions correspond to x 2 and log(x), respectively.

	Layer	# filters / (kernel size) Output
	Input		(1, C, T = 384)
	Conv2D	40 / (1, 13)	(40, C, 372)
	Conv2D	40 / (C, 1)	(40, 1, 372)
	BatchNorm		(40, 1, 372)
	Activation(square)		(40, 1, 372)
	AveragePool2D	(1, 35), stride (1, 7)	(40, 1, 49)
	Activation(log)		(40, 1, 49)
	Flatten		(1960)
	Dropout		(1960)
	Dense(Softmax)	N = 2	(2)

Table 3 . 4 :

 34 Parameters of the EEGNet architecture. Where C is the number of channels, T is the number of time samples per channel, F1 is the number of temporal filters, D is the multiplier depth, F2 is the number of pointwise filters and N is the number of classes. We use 8 temporal filters F1 = 8, 4 multiplier depth D = 4, 16 pointwise filters F2 = 16 and (1, K = 8) for the lengths of temporal kernel size for the first convolutional layer.

	Layer	# filters / (kernel size) Output
	Input		(1, C, T = 384)
	Conv2D	8 / (1, K)	(8, C, 384)
	BatchNorm		(8, C, 384)
	DepthwiseConv2D 4 * 8 / (C, 1)	(32, 1, 384)
	BatchNorm		(32, 1, 384)
	Activation(ELU)		(32, 1, 384)
	AveragePool2D	(1, 4)	(32, 1, 96)
	Dropout*		(32, 1, 96)
	SeparableConv2D	16 / (1, 16)	(16, 1, 96)
	BatchNorm		(16, 1, 96)
	Activation(ELU)		(16, 1, 96)
	AveragePool2D	(1, 8)	(16, 1, 12)
	Dropout*		(16, 1, 12)
	Flatten		(192)
	Dense(Softmax)	N = 2	(2)

Table 3 . 5 :

 35 Number of parameters for different neural networks

				Number of channels
	Model	128	47	13	9	6	3
	D-ConvNet	214,077 163,452 142,202 139,702 137,827 135,952
	S-ConvNet	209,442 79,842	25,442	19,042	14,242	9,442
	EEGNet-2.32 3,362	2,066	1,522	1,458	1,410	1,362
	EEGNet-4.8	5,794	3,202	2,114	1,986	1,890	1,794

  -class motor imagery dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Motor imagery 8-class database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Median nerve stimulation database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3.4 EEG channel layouts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.5 Data preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.6 Cross-validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

	Chapter 4
	Materials
	Contents
	4.1 2
	3
	of chapter 5.

4.1.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.2 Experimental tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.3 Paradigm and time scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.4 Electroencephalographic data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.5 Data preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.6 Feature extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1.7 Cross-validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 4.2.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 Paradigm and time scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.3 Electroencephalographic data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.4 EEG channels layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.5 Data preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.6 Classification using multi-label approach . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.7 Multi-label classification using 3 binary classifiers . . . . . . . . . . . . . . . . . . . . . 4.3 4.3.1 Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.2 Paradigm and time scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3.3 Electroencephalographic data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Table 5 . 1 :

 51 Classification accuracy before and after outlier detection using Isolation forest method

	Subject	Accuracy before outlier detection	Accuracy after outlier detection	The best parameter of data saturation by outliers (%)
	S1 S2 S3 S4 S6 S7 S8 S9 S10 S11 S12 S13 S14 S15 S16 S17 S18 S19 S20 S21 S24 S25 S26 S27 S28 S29 AVG	0.73 + -0.06 0.75 + -0.0 0.5 + -0.04 0.58 + -0.02 0.48 + -0.08 0.52 + -0.05 0.6 + -0.08 0.67 + -0.09 0.62 + -0.09 0.58 + -0.06 0.47 + -0.05 0.55 + -0.11 0.48 + -0.02 0.7 + -0.11 0.75 + -0.07 0.47 + -0.05 0.52 + -0.09 0.78 + -0.08 0.6 + -0.04 0.42 + -0.06 0.47 + -0.05 0.93 + -0.09 0.58 + -0.06 0.75 + -0.04 0.38 + -0.05 0.72 + -0.02 0.6 + -0.06	0.97 + -0.05 0.85 + -0.11 0.46 + -0.1 0.54 + -0.1 0.54 + -0.14 0.33 + -0.27 0.52 + -0.1 0.66 + -0.01 0.66 + -0.09 0.55 + -0.17 0.53 + -0.1 0.54 + -0.14 0.37 + -0.09 0.56 + -0.09 0.59 + -0.07 0.54 + -0.05 0.5 + -0.13 0.78 + -0.05 0.44 + -0.14 0.58 + -0.12 0.56 + -0.04 0.91 + -0.08 0.53 + -0.1 0.86 + -0.1 0.58 + -0.11 0.64 + -0.06 0.6 + -0.1	40.0 + -22.86 48.0 + -5.66 28.0 + -14.24 33.33 + -20.74 34.67 + -16.11 50.67 + -6.8 42.67 + -14.73 24.0 + -17.28 32.0 + -9.8 45.33 + -10.5 40.0 + -14.97 41.33 + -23.63 38.67 + -27.39 44.0 + -5.66 26.67 + -18.57 14.67 + -15.08 32.0 + -11.78 17.33 + -21.75 25.33 + -19.14 26.67 + -16.44 34.67 + -6.8 5.33 + -7.54 45.33 + -8.22 20.0 + -11.78 42.67 + -12.36 38.67 + -27.39 33.54 + -14.89

Table 5 . 2 :

 52 Classification accuracy before and after outlier detection using One-class SVM method

	Subject	Accuracy before outlier detection	Accuracy after outlier detection	The best parameter of data saturation by outliers (%)
	S1 S2 S3 S4 S6 S7 S8 S9 S10 S11 S13 S14 S15 S16 S17 S18 S19 S20 S21 S22 S24 S25 S26 S27 S28 S29 AVG	0.73 + -0.06 0.75 + -0.0 0.5 + -0.04 0.58 + -0.02 0.48 + -0.08 0.52 + -0.05 0.6 + -0.08 0.67 + -0.09 0.62 + -0.09 0.58 + -0.06 0.55 + -0.11 0.48 + -0.02 0.7 + -0.11 0.75 + -0.07 0.47 + -0.05 0.52 + -0.09 0.78 + -0.08 0.6 + -0.04 0.42 + -0.06 0.5 + -0.07 0.47 + -0.05 0.93 + -0.09 0.58 + -0.06 0.75 + -0.04 0.38 + -0.05 0.72 + -0.02 0.6 + -0.06	0.73 + -0.12 0.75 + -0.04 0.42 + -0.07 0.51 + -0.04 0.48 + -0.11 0.36 + -0.25 0.56 + -0.21 0.7 + -0.03 0.7 + -0.12 0.61 + -0.1 0.55 + -0.05 0.33 + -0.09 0.45 + -0.32 0.7 + -0.02 0.56 + -0.09 0.57 + -0.14 0.77 + -0.09 0.57 + -0.27 0.56 + -0.11 0.45 + -0.15 0.5 + -0.12 0.92 + -0.08 0.5 + -0.14 0.82 + -0.15 0.72 + -0.21 0.71 + -0.05 0.6 + -0.12	37.33 + -3.77 12.0 + -11.31 18.67 + -13.2 24.0 + -16.97 29.33 + -21.25 32.0 + -6.53 40.0 + -17.28 12.0 + -6.53 28.0 + -9.8 42.67 + -12.36 8.0 + -5.66 42.67 + -24.51 44.0 + -16.97 22.67 + -15.43 50.67 + -7.54 24.0 + -8.64 12.0 + -8.64 40.0 + -22.63 21.33 + -7.54 22.67 + -20.74 32.0 + -14.97 20.0 + -25.51 25.33 + -16.44 21.33 + -1.89 30.67 + -17.99 32.0 + -22.63 27.9 + -13.72

Table 5 . 3 :

 53 Accuracy obtained by a shrinkage LDA for 10 subjects for 8-class MI classification task.

	Subject Accuracy & STD
	1	0.219 ± 0.34
	2	0.581 ± 0.50
	3	0.347 ± 0.27
	4	0.284 ± 0.66
	5	0.269 ± 0.40
	6	0.237 ± 0.18
	7	0.278 ± 0.66
	8	0.266 ± 0.35
	9	0.353 ± 0.30
	10	0.434 ± 0.52
	AVG	0.327 ± 0.42

Table 5 .

 5 4 shows the main misclassifications. In particular, feet motor imageries have been confused with a resting state (Rest → Feet, Feet → Rest, Both hands → Feet, Left hand & Feet → Left hand, Right hand & Feet → Right hand, Both hands

Table 5 . 4 :

 54 Confusion Matrix (%) over all subjects

					Predicted labels
		Rest LH RH BH F LH & F RH & F BH & F
	Rest	44	5	7	5	18	9	6	6
	LH	6	22	11	15	7	16	9	14
	RH	6	10	25	17	9	6	16	11
	BH	5	11	12	28	7	10	8	19
	F	17	6	8	5	42	8	9	5
	LH & F	6	13	6	9	11	35	9	11
	RH & F	6	8	14	9	7	9	36	11
	True labels BH & F	6	13	9	20	6	12	11	23

Table 5 . 5 :

 55 Accuracies of combined MI movements classification on 3-ERD-feature set

	State	SVM (linear)	Classifiers SVM (RBF)	LDA Decision Tree
	Average 0.264 + -0.08 0.255 + -0.08 0.261 + -0.09 Rest 0.152 + -0.11 0.163 + -0.15 0.16 + -0.12 Left hand 0.348 + -0.2 0.287 + -0.2 0.342 + -0.19 Right hand 0.367 + -0.29 0.35 + -0.3 0.37 + -0.28 Both hands 0.282 + -0.24 0.282 + -0.26 0.29 + -0.24 Feet 0.427 + -0.25 0.443 + -0.23 0.412 + -0.24 Left hand and feet 0.265 + -0.24 0.24 + -0.21 0.255 + -0.24 Right hand and feet 0.185 + -0.13 0.22 + -0.13 0.173 + -0.14 Both hands and feet 0.085 + -0.1 0.053 + -0.05 0.085 + -0.09	0.194 + -0.08 0.198 + -0.11 0.258 + -0.15 0.297 + -0.25 0.16 + -0.08 0.24 + -0.17 0.165 + -0.12 0.133 + -0.08 0.1 + -0.06

Table 5 . 6 :

 56 Accuracies of combined MI movements classification on 3-ERS-feature set

	State	SVM (linear)	Classifiers SVM (RBF)	LDA Decision Tree
	Average 0.168 + -0.03 0.168 + -0.03 0.166 + -0.03 Rest 0.043 + -0.06 0.045 + -0.06 0.048 + -0.05 Left hand 0.338 + -0.17 0.282 + -0.15 0.335 + -0.16 Right hand 0.233 + -0.13 0.223 + -0.12 0.248 + -0.12 Both hands 0.115 + -0.08 0.118 + -0.08 0.11 + -0.08 Feet 0.283 + -0.16 0.31 + -0.17 0.263 + -0.13 Left hand and feet 0.11 + -0.07 0.098 + -0.09 0.09 + -0.07 Right hand and feet 0.22 + -0.13 0.26 + -0.16 0.227 + -0.15 Both hands and feet 0.005 + -0.01 0.008 + -0.01 0.008 + -0.01	0.146 + -0.03 0.138 + -0.06 0.17 + -0.05 0.16 + -0.09 0.133 + -0.04 0.17 + -0.07 0.128 + -0.07 0.155 + -0.07 0.113 + -0.06

Table 5 . 7 :

 57 Accuracies of combined MI movements classification on 6 ERD & ERS-feature set

	State	SVM (linear)	Classifiers SVM (RBF)	LDA Decision Tree
	Average 0.285 + -0.09 0.277 + -0.09 0.284 + -0.09 Rest 0.205 + -0.09 0.195 + -0.14 0.23 + -0.1 Left hand 0.467 + -0.22 0.43 + -0.22 0.44 + -0.19 Right hand 0.455 + -0.28 0.46 + -0.28 0.46 + -0.27 Both hands 0.265 + -0.19 0.183 + -0.14 0.268 + -0.19 Feet 0.455 + -0.19 0.487 + -0.22 0.463 + -0.19 Left hand and feet 0.17 + -0.16 0.188 + -0.17 0.18 + -0.15 Right hand and feet 0.227 + -0.12 0.247 + -0.16 0.202 + -0.12 Both hands and feet 0.033 + -0.04 0.028 + -0.04 0.03 + -0.04	0.203 + -0.07 0.232 + -0.13 0.288 + -0.15 0.34 + -0.26 0.133 + -0.05 0.318 + -0.14 0.143 + -0.07 0.113 + -0.06 0.063 + -0.04

Table 5 . 8 :

 58 Accuracies of combined MI movements classification on 6-feature set: alpha band (8-12 Hz) and beta band[START_REF] Fang | Extending CSP to Detect Motor Imagery in a Four-class BCI[END_REF][START_REF] Wang | Common Spatial Pattern Method for Channel Selelction in Motor Imagery Based Brain-computer Interface[END_REF][START_REF] Blankertz | Optimizing spatial filters for robust EEG single-trial analysis [revealing tricks of the trade[END_REF][START_REF] Rimbert | Median Nerve Stimulation Based BCI: A New Approach to Detect Intraoperative Awareness During General Anesthesia[END_REF][START_REF] Barachant | Riemannian geometry applied to BCI classification[END_REF][START_REF] Congedo | A New Generation of Brain-Computer Interface Based on Riemannian Geometry[END_REF][START_REF] Barachant | Multiclass Brain-Computer Interface Classification by Riemannian Geometry[END_REF][START_REF] Roy | Deep learning-based electroencephalography analysis: a systematic review[END_REF][START_REF] Craik | Deep learning for electroencephalogram (EEG) classification tasks: a review[END_REF]

	State	Classifiers SVM (linear) SVM (RBF)	LDA Decision Tree
	Average Rest Left hand Right hand Both hands Feet Left hand and feet Right hand and feet Both hands and feet	0.275 + -0.09 0.272 + -0.09 0.205 + -0.12 0.225 + -0.14 0.228 + -0.14 0.281 + -0.1 0.467 + -0.2 0.477 + -0.2 0.458 + -0.23 0.412 + -0.27 0.425 + -0.28 0.423 + -0.25 0.25 + -0.12 0.205 + -0.14 0.268 + -0.15 0.497 + -0.14 0.507 + -0.18 0.488 + -0.14 0.168 + -0.1 0.15 + -0.09 0.2 + -0.12 0.165 + -0.09 0.168 + -0.11 0.158 + -0.1 0.035 + -0.05 0.02 + -0.02 0.03 + -0.04	0.222 + -0.08 0.35 + -0.14 0.338 + -0.18 0.372 + -0.25 0.12 + -0.08 0.36 + -0.12 0.103 + -0.06 0.11 + -0.05 0.025 + -0.03

Table 5 . 9 :

 59 Accuracies of combined MI movements classification on 18-feature set

	State	SVM (linear)	Classifiers SVM (RBF)	LDA Decision Tree
	Average 0.286 + -0.08 0.294 + -0.09 0.288 + -0.09 Rest 0.253 + -0.12 0.255 + -0.13 0.253 + -0.11 Left hand 0.492 + -0.12 0.487 + -0.16 0.505 + -0.13 Right hand 0.45 + -0.21 0.435 + -0.31 0.463 + -0.23 Both hands 0.232 + -0.12 0.253 + -0.15 0.247 + -0.14 Feet 0.45 + -0.18 0.507 + -0.18 0.443 + -0.19 Left hand and feet 0.19 + -0.07 0.188 + -0.11 0.192 + -0.07 Right hand and feet 0.185 + -0.08 0.19 + -0.12 0.172 + -0.09 Both hands and feet 0.033 + -0.04 0.035 + -0.06 0.028 + -0.03	0.205 + -0.06 0.487 + -0.13 0.27 + -0.1 0.263 + -0.14 0.103 + -0.07 0.328 + -0.12 0.098 + -0.07 0.073 + -0.05 0.017 + -0.02

Table 5 .

 5 10: Accuracies of combined MI movements classification on 30-feature set

	State	SVM (linear)	Classifiers SVM (RBF)	LDA Decision Tree
	Average 0.303 + -0.11 Rest 0.358 + -0.14 0.375 + -0.15 0.357 + -0.14 0.305 + -0.1 0.321 + -0.09 Left hand 0.535 + -0.21 0.522 + -0.24 0.533 + -0.22 Right hand 0.583 + -0.23 0.555 + -0.25 0.595 + -0.23 Both hands 0.22 + -0.16 0.21 + -0.14 0.253 + -0.19 Feet 0.497 + -0.19 0.588 + -0.21 0.508 + -0.17 Left hand and feet 0.193 + -0.11 0.158 + -0.07 0.202 + -0.1 Right hand and feet 0.187 + -0.1 0.175 + -0.1 0.19 + -0.1 Both hands and feet 0.015 + -0.02 0.025 + -0.05 0.015 + -0.02	0.213 + -0.07 0.422 + -0.09 0.315 + -0.16 0.398 + -0.25 0.072 + -0.04 0.325 + -0.12 0.105 + -0.07 0.073 + -0.06 0.008 + -0.02

Table 5 .

 5 11: Average accuracies of combined MI movements classification for different feature sets

	Feature set	SVM (linear)	Classifiers SVM (RBF)	LDA Decision Tree
	3-ERD (7-30) 3-ERS (7-30) 6: ERD (7-30) & ERS (7-30) 6: ERD (8-12) & ERS (17-25) 0.275 + -0.09 0.272 + -0.09 0.264 + -0.08 0.255 + -0.08 0.261 + -0.09 0.168 + -0.03 0.168 + -0.03 0.166 + -0.03 0.285 + -0.09 0.277 + -0.09 0.284 + -0.09 0.281 + -0.1 18-feature set 0.286 + -0.08 0.294 + -0.09 0.288 + -0.09 30-feature set 0.303 + -0.11 0.305 + -0.1 0.321 + -0.09	0.194 + -0.08 0.146 + -0.03 0.203 + -0.07 0.222 + -0.08 0.205 + -0.06 0.213 + -0.07

Table 5 .

 5 12: Result of feature selection for 'Right vs. rest & Left & Feet' classifier. Frequently used features are indicated in green, occasionally used features indicated in yellow, features that are rarely used are indicated in red.

	Feature	Channels C3 Cz C4
	ERD 8-10	2	2	3
	ERD 11-13 30	4	7
	ERD 8-12	22	6	14
	ERD 8-30	25 12 18
	ERS 15-17	2	0	2
	ERS 18-20	9	0	2
	ERS 21-23	4	1	3
	ERS 24-26	0	2	0
	ERS 17-25 16	4	10
	ERS 8-30	24	8	16

Table 5 .

 5 13: Result of feature selection for 'Left vs. rest & Right & Feet' classifier. Frequently used features are indicated in green, occasionally used features indicated in yellow, features that are rarely used are indicated in red.

	Feature	Channel C3 Cz C4
	ERD 8-10	11	7	9
	ERD 11-13 16	3	32
	ERD 8-12	14 10 20
	ERD 8-30	16 11 21
	ERS 15-17	4	0	6
	ERS 18-20	3	0	7
	ERS 21-23	9	0	0
	ERS 24-26	2	3	0
	ERS 17-25 12	6	18
	ERS 8-30	12	6	22

Table 5 .

 5 14: Result of feature selection for 'Feet vs. rest & Right & Left' classifier. Frequently used features are indicated in green, occasionally used features indicated in yellow, features that are rarely used are indicated in red.

	Feature	Channels C3 Cz C4
	ERD 8-10	10	9	4
	ERD 11-13 14 14 12
	ERD 8-12	19 11 21
	ERD 8-30	12 16 12
	ERS 15-17	3	0	1
	ERS 18-20	5	2	3
	ERS 21-23	6	9	1
	ERS 24-26	2	9	0
	ERS 17-25	5	24	5
	ERS 8-30	18 10 18

Table 5 .

 5 17: Accuracies of combined movement classification on 30-feature set using binary classifiers and feature selection

	State	SVM (linear)	Classifiers sLDA Decision Tree
	Average before feature selection 0.303 + -0.11 0.321 + -0.09 Average after feature selection 0.323 + -0.11 0.332 + -0.10	0.213 + -0.07 0.215 + -0.07

Table 5 .

 5 18: Classification accuracy obtained by different variations of EEGNet with different dropout rate (D) for 22 subjects using 128 channel layout for MNS vs. MNS+MI discriminating task. Signals filtered in (4-38 Hz) frequency band, results averaged across 10 folds of cross-validation

	Subject	EEGNet-2.32 D = 0.25 D = 0.5 D = 0.25 D = 0.5 EEGNet-4.8
	S1	0.873636	0.923 0.941818	0.931
	S2	0.972727	0.954 0.990909	0.991
	S3	0.95	0.961	0.98	0.970
	S4	0.93	0.961	0.96	0.980
	S5	0.779091	0.825 0.847273	0.843
	S6	0.595455	0.587 0.538182	0.548
	S7	0.99	1.000	1	0.990
	S8	0.962727	0.971 0.981818	0.972
	S9	0.99	1.000 0.990909	1.000
	S10	0.829091	0.804 0.858182	0.945
	S11	0.871818	0.923 0.950909	0.960
	S12	0.940909	0.951	0.96	0.950
	S13	0.990909	0.991 0.980909	0.990
	S14	0.943636	0.962 0.941818	0.972
	S15	0.930909	0.865 0.950909	1.000
	S16	0.961818	0.990	0.99	1.000
	S17	0.858182	0.915 0.943636	0.954
	S18	0.970909	0.990	0.99	0.990
	S19	0.934545	0.916 0.962727	0.925
	S20	0.95	0.901 0.940909	0.970
	S21	0.868182	0.857 0.933636	0.926
	S22	0.916364	0.886 0.971818	0.972
	AVG	0.910	0.915	0.937	0.945

Table 5 .

 5 19: Classification accuracy obtained by different variations of EEGNet with different dropout rate (D) for 22 subjects using 47 channel layout for MNS vs. MNS+MI discriminating task. Signals filtered in (4-38 Hz) frequency band, results averaged across 10 folds of cross-validation

	Subject	EEGNet-2.32 D = 0.25 D = 0.5 D = 0.25 D = 0.5 EEGNet-4.8
	S1	0.766364	0.835 0.932727	0.922
	S2	0.906364	0.896 0.926364	0.954
	S3	0.942727	0.915 0.951818	0.915
	S4	0.99	0.910	1	0.970
	S5	0.713636	0.739 0.916364	0.895
	S6	0.576364	0.547 0.536364	0.529
	S7	0.774545	0.845 0.941818	0.990
	S8	0.885455	0.933 0.931818	0.952
	S9	1	1.000	1	1.000
	S10	0.763636	0.827 0.820909	0.790
	S11	0.770909	0.865 0.924545	0.971
	S12	0.930909	0.940	0.99	0.960
	S13	0.981818	0.990 0.971818	0.972
	S14	0.912727	0.923 0.904545	0.933
	S15	0.847273	0.895 0.895455	0.855
	S16	0.873636	0.905 0.932727	0.942
	S17	0.886364	0.865 0.940909	0.943
	S18	0.98	0.990	0.99	0.990
	S19	0.943636	0.883 0.951818	0.980
	S20	0.855455	0.875 0.882727	0.903
	S21	0.787273	0.762	0.82	0.867
	S22	0.78	0.868 0.942727	0.932
	AVG	0.858	0.873	0.914	0.916

Table 5 . 20 :

 520 Grand average classification accuracy obtained by different classifiers using different layouts for MNS vs. MI+MNS discriminating task. Signals filtered in (4-38 Hz) frequency band, results averaged across 10 folds of cross-validation and across 22 subjects

	Electrode layout	CSP+LDA MDRM TS+LR D-ConvNet S-ConvNet EEGNet-2.32 EEGNet-4.8
	128	0.839	0.756	0.854	0.835	0.834	0.915	0.945
	47	0.792	0.753	0.823	0.855	0.867	0.873	0.916
	13_mc	0.728	0.698	0.759	0.805	0.813	0.838	0.866
	13_mc+fr	0.758	0.729	0.787	0.786	0.832	0.851	0.880
	9_mc	0.678	0.657	0.706	0.766	0.793	0.812	0.846
	9_mc+fr	0.729	0.707	0.738	0.795	0.804	0.835	0.851
	6_mc	0.664	0.655	0.672	0.718	0.769	0.776	0.817
	6_mc+fr	0.727	0.697	0.742	0.754	0.789	0.812	0.832
	6_mc+fr_os	0.704	0.687	0.722	0.781	0.789	0.828	0.840
	3_mc	0.655	0.632	0.656	0.728	0.738	0.752	0.764
	3_fr	0.633	0.632	0.644	0.621	0.635	0.665	0.662
	3_fr_os	0.641	0.638	0.642	0.650	0.660	0.669	0.693

Table 5 . 21 :

 521 Grand average classification accuracy obtained by different classifiers using different layouts for MI vs. rest discriminating task. Signals filtered in (8-30 Hz) frequency band, results averaged across 10 folds of cross-validation and across 22 subjects

	Electrode layout	CSP+LDA MDRM TS+LR D-ConvNet S-ConvNet EEGNet-2.32 EEGNet-4.8
	128	0.639	0.654	0.706	0.615	0.639	0.677	0.699
	47	0.644	0.670	0.722	0.642	0.681	0.709	0.703
	13_mc	0.662	0.658	0.708	0.640	0.669	0.706	0.713
	13_mc+fr	0.674	0.661	0.692	0.652	0.684	0.706	0.738
	9_mc	0.664	0.653	0.679	0.644	0.685	0.718	0.730
	9_mc+fr	0.672	0.653	0.695	0.642	0.670	0.690	0.711
	6_mc	0.672	0.641	0.668	0.640	0.662	0.697	0.700
	6_mc+fr	0.656	0.635	0.660	0.605	0.661	0.696	0.689
	6_mc+fr_os	0.653	0.628	0.666	0.619	0.677	0.676	0.686
	3_mc	0.629	0.612	0.629	0.608	0.631	0.669	0.657
	3_fr	0.544	0.542	0.557	0.515	0.562	0.537	0.541
	3_fr_os	0.525	0.534	0.529	0.514	0.555	0.529	0.539

  Outlier detection for automatic assessing the quality of imaginary movements in MI-based BCI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 6.2 Classification of combined movement motor imageries using binary classifiers . . . . . . . . 112 6.3 Feature selection for combined movement motor imagery detection . . . . . . . . . . . . . . 113 6.4 Imaginary movements detection using deep learning to prevent intraoperative awareness . . 113

	Chapter 6
	Discussion
	Contents
	6.1

6.4.1 Different deep learning architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 6.4.2 Frequency band selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 6.4.3 Addition of the frontal electrodes to improves motor imaginary detection . . . . . . . . . 114 6.4.4 Using reduced electrode layouts for motor imagery detection . . . . . . . . . . . . . . . 115

https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.wilcoxon.html

https://data.library.virginia.edu/the-wilcoxon-rank-sum-test/

https://paperswithcode.com/sota/image-classification-on-imagenet/ Figure 3.9: Proportion use of the DNN architectures across all the reviewed articles.

Remerciements

Appendix A

Classification results for MNS vs.

MI+MNS task using different classifiers

In this section, classification accuracy results obtained by different classifiers (CSP+LDA, MDRM, TS+LR, D-ConvNet, S-ConvNet, EEGNet-2.32, EEGNet-4.8) for 22 subjects from dataset introduced in Section 4.3 are presented. Results are represented for 12 different channel layouts for MNS vs. MNS+MI discriminating task. Signals filtered in 4 different frequency bands, results averaged across 10 folds of cross-validation. 0.817 0.827 0.827 1.000 0.827 0.663 0.933 0.952 0.856 0.615 0.577 0.865 1.000 0.798 0.731 0.990 0.894 0.981 0.846 0.808 0.779 0.865 0.839 MDM 0.240 0.817 0.769 1.000 0.779 0.731 0.962 0.933 0.692 0.510 0.615 0.625 0.962 0.769 0.702 0.760 0.827 0.952 0.750 0.827 0.683 0.731 0.756 TS+LR 0.779 0.894 0.837 1.000 0.769 0.769 0.942 0.952 0.856 0.692 0.606 0.808 0.981 0.779 0.798 0.904 0.846 0.971 0.942 0.904 0.904 0.865 0.854 D-ConvNet 0.823 0.879 0.883 0.980 0.670 0.641 0.885 0.833 0.832 0.645 0.800 0.845 1.000 0.883 0.798 0.950 0.849 0.981 0.842 0.849 0.645 0.857 0.835 S-ConvNet 0.691 0.850 0.814 0.893 0.798 0.555 0.864 0.867 0.788 0.777 0.895 0.665 0.854 0.954 0.836 0.932 0.943 0.990 0.905 0.863 0.809 0.805 0.834 EEGNet-2.32 0.923 0.954 0.961 0.961 0.825 0.587 1.000 0.971 1.000 0.804 0.923 0.951 0.991 0.962 0.865 0.990 0.915 0.990 0.916 0.901 0.857 0.886 0.915 EEGNet-4.8 0.931 0.991 0.970 0.980 0.843 0.548 0.990 0.972 1.000 0.945 0.960 0.950 0.990 0.972 1.000 1.000 0.954 0.990 0.925 0.970 0.926 0.972 0.945

Table A.2:

Classification accuracy obtained by different classifiers for 22 subjects using 128 channel layout for MNS vs. MNS+MI discriminating task. Signals filtered in (4-30 Hz) frequency band, results averaged across 10 folds of cross-validation 0.702 0.779 0.875 1.000 0.788 0.712 0.933 0.933 0.942 0.596 0.471 0.808 0.952 0.769 0.721 0.981 0.865 0.971 0.913 0.817 0.798 0.827 0.825 MDM 0.365 0.817 0.788 1.000 0.769 0.740 0.952 0.923 0.683 0.510 0.625 0.606 0.923 0.769 0.702 0.769 0.808 0.962 0.750 0.769 0.712 0.760 0.759 TS+LR 0.615 0.885 0.846 1.000 0.769 0.769 0.971 0.933 0.760 0.654 0.654 0.750 0.981 0.779 0.788 0.865 0.827 0.971 0.923 0.904 0.885 0.798 0.833 D-ConvNet 0.645 0.859 0.915 0.961 0.672 0.605 0.776 0.829 0.795 0.617 0.770 0.739 0.933 0.943 0.747 0.923 0.878 0.981 0.743 0.836 0.596 0.693 0.793 S-ConvNet 0.692 0.803 0.815 0.834 0.788 0.585 0.815 0.848 0.817 0.758 0.913 0.760 0.816 0.913 0.797 0.905 0.944 0.990 0.944 0.882 0.846 0.816 0.831 EEGNet-2.32 0.834 0.916 0.941 0.921 0.788 0.585 0.990 0.961 0.964 0.813 0.884 0.895 0.961 0.895 0.846 0.925 0.875 0.970 0.935 0.913 0.885 0.935 0.892 EEGNet-4.8 0.923 1.000 0.960 0.950 0.855 0.576 0.980 0.972 0.981 0.831 0.942 0.900 0.981 0.925 0.944 1.000 0.915 0.981 0.971 0.970 0.914 0.955 0.928 Table A.3: Classification accuracy obtained by different classifiers for 22 subjects using 128 channel layout for MNS vs. MNS+MI discriminating task. Signals filtered in Hz) frequency band, results averaged across 10 folds of cross-validation 0.673 0.837 0.788 1.000 0.750 0.788 0.933 0.904 0.856 0.663 0.567 0.894 0.933 0.798 0.846 0.952 0.846 0.962 0.904 0.923 0.808 0.904 0.842 MDM 0.260 0.808 0.808 1.000 0.779 0.712 0.923 0.904 0.663 0.510 0.615 0.587 0.942 0.760 0.712 0.769 0.827 0.952 0.750 0.798 0.760 0.788 0.756 TS+LR 0.577 0.885 0.846 1.000 0.779 0.769 0.904 0.952 0.769 0.692 0.625 0.750 0.971 0.769 0.788 0.894 0.837 0.971 0.904 0.875 0.885 0.817 0.830 D-ConvNet 0.578 0.857 0.846 0.952 0.613 0.555 0.805 0.673 0.633 0.646 0.681 0.745 0.953 0.798 0.731 0.808 0.847 0.972 0.716 0.656 0.510 0.663 0.738 S-ConvNet 0.681 0.831 0.798 0.835 0.675 0.595 0.806 0.759 0.737 0.645 0.858 0.575 0.815 0.874 0.770 0.837 0.859 0.990 0.735 0.575 0.692 0.765 0.759 EEGNet-2.32 0.768 0.944 0.934 0.940 0.767 0.577 1.000 0.924 0.964 0.810 0.724 0.882 0.980 0.933 0.915 0.913 0.845 0.980 0.876 0.911 0.865 0.896 0.879 EEGNet-4.8 0.865 0.982 0.892 0.910 0.810 0.576 0.990 0.944 0.991 0.830 0.884 0.950 0.982 0.963 0.923 0.980 0.866 0.990 0.943 0.960 0.923 0.955 0.914 Table A.4: Classification accuracy obtained by different classifiers for 22 subjects using 128 channel layout for MNS vs. MNS+MI discriminating task. Signals filtered in Hz) frequency band, results averaged across 10 folds of cross-validation 0.779 0.856 0.769 1.000 0.769 0.654 0.942 0.904 0.933 0.567 0.567 0.894 1.000 0.779 0.750 0.971 0.913 0.971 0.913 0.827 0.856 0.904 0.842 MDM 0.202 0.808 0.779 1.000 0.788 0.702 0.933 0.923 0.673 0.500 0.596 0.625 0.952 0.769 0.702 0.760 0.837 0.952 0.750 0.837 0.712 0.779 0.753 TS+LR 0.760 0.894 0.885 1.000 0.779 0.769 0.894 0.952 0.885 0.731 0.587 0.808 0.971 0.769 0.798 0.923 0.856 0.971 0.904 0.885 0.875 0.885 0.854 D-ConvNet 0.718 0.953 0.826 0.971 0.632 0.659 0.960 0.742 0.727 0.733 0.710 0.823 0.990 0.866 0.845 0.970 0.878 0.991 0.704 0.749 0.556 0.815 0.810 S-ConvNet 0.663 0.851 0.825 0.864 0.657 0.586 0.932 0.768 0.710 0.759 0.905 0.614 0.972 0.905 0.759 0.854 0.848 0.990 0.795 0.635 0.731 0.837 0.794 EEGNet-2.32 0.905 0.954 0.971 0.941 0.826 0.585 1.000 0.972 1.000 0.819 0.800 0.942 0.990 0.972 0.925 0.950 0.925 0.970 0.915 0.912 0.830 0.963 0.912 EEGNet-4.8 0.945 0.991 0.961 0.922 0.912 0.548 0.990 0.954 1.000 0.895 0.972 0.971 1.000 0.972 0.971 0.980 0.934 0.990 0.953 0.970 0.933 0.962 0.942 Table A.9: Classification accuracy obtained by different classifiers for 22 subjects using 13 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (4-38 Hz) frequency band, results averaged across 10 folds of cross-validation 0.808 0.788 0.702 1.000 0.567 0.635 0.673 0.808 0.673 0.615 0.558 0.577 0.538 0.856 0.702 0.846 0.721 0.885 0.894 0.673 0.683 0.817 0.728 MDM 0.596 0.817 0.769 1.000 0.462 0.596 0.673 0.798 0.615 0.654 0.577 0.654 0.490 0.779 0.702 0.817 0.779 0.817 0.817 0.635 0.654 0.654 0.698 TS+LR 0.760 0.837 0.769 1.000 0.548 0.673 0.779 0.885 0.683 0.596 0.558 0.731 0.606 0.885 0.740 0.827 0.788 0.885 0.894 0.750 0.740 0.769 0.759 D-ConvNet 0.766 0.897 0.891 1.000 0.789 0.566 0.626 0.921 0.753 0.616 0.686 0.752 0.824 0.885 0.895 0.855 0.845 0.943 0.933 0.817 0.567 0.885 0.805 S-ConvNet 0.825 0.898 0.845 0.951 0.722 0.575 0.692 0.951 0.715 0.785 0.790 0.767 0.747 0.874 0.788 0.894 0.903 0.970 0.905 0.797 0.695 0.796 0.813 EEGNet-2.32 0.815 0.887 0.873 0.940 0.789 0.500 0.701 0.931 0.751 0.726 0.884 0.837 0.860 0.896 0.885 0.903 0.905 0.980 0.885 0.844 0.767 0.885 0.838 EEGNet-4.8 0.873 0.906 0.843 0.960 0.789 0.482 0.718 0.951 0.739 0.772 0.953 0.932 0.973 0.925 0.934 0.886 0.895 0.981 0.952 0.893 0.820 0.874 0.866 Table A.10: Classification accuracy obtained by different classifiers for 22 subjects using 13 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (4-30 Hz) frequency band, results averaged across 10 folds of cross-validation 0.837 0.817 0.721 1.000 0.596 0.654 0.683 0.846 0.692 0.548 0.596 0.635 0.558 0.894 0.683 0.837 0.731 0.856 0.856 0.673 0.692 0.779 0.736 MDM 0.663 0.808 0.798 1.000 0.433 0.587 0.683 0.798 0.615 0.644 0.587 0.673 0.442 0.808 0.702 0.808 0.760 0.808 0.817 0.673 0.673 0.625 0.700 TS+LR 0.740 0.827 0.769 1.000 0.529 0.692 0.750 0.885 0.663 0.615 0.577 0.731 0.558 0.894 0.721 0.846 0.798 0.875 0.894 0.760 0.740 0.712 0.753 D-ConvNet 0.805 0.907 0.902 0.990 0.769 0.604 0.615 0.960 0.799 0.684 0.732 0.730 0.575 0.875 0.864 0.845 0.895 0.915 0.933 0.817 0.530 0.730 0.794 S-ConvNet 0.845 0.867 0.806 0.980 0.712 0.635 0.679 0.923 0.715 0.776 0.790 0.757 0.690 0.855 0.778 0.865 0.893 0.952 0.923 0.807 0.724 0.800 0.808 EEGNet-2.32 0.854 0.897 0.854 0.970 0.760 0.473 0.672 0.940 0.770 0.745 0.875 0.826 0.727 0.924 0.875 0.884 0.865 0.971 0.903 0.864 0.758 0.837 0.829 EEGNet-4.8 0.844 0.925 0.872 0.941 0.818 0.492 0.686 0.942 0.750 0.799 0.943 0.876 0.824 0.914 0.924 0.903 0.864 0.971 0.951 0.864 0.846 0.895 0.856 Table A.11: Classification accuracy obtained by different classifiers for 22 subjects using 13 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in (8-30 Hz) frequency band, results averaged across 10 folds of cross-validation 0.683 0.788 0.721 1.000 0.500 0.663 0.683 0.750 0.606 0.606 0.538 0.721 0.490 0.846 0.644 0.856 0.740 0.846 0.865 0.606 0.635 0.740 0.706 MDM 0.587 0.788 0.779 1.000 0.452 0.587 0.644 0.798 0.625 0.606 0.587 0.625 0.462 0.769 0.702 0.846 0.769 0.846 0.798 0.615 0.673 0.683 0.693 TS+LR 0.606 0.808 0.827 1.000 0.500 0.692 0.740 0.798 0.683 0.625 0.538 0.721 0.462 0.846 0.740 0.885 0.808 0.865 0.894 0.673 0.788 0.750 0.739 D-ConvNet 0.714 0.896 0.903 1.000 0.646 0.535 0.590 0.839 0.752 0.619 0.724 0.737 0.668 0.790 0.747 0.653 0.786 0.944 0.887 0.743 0.606 0.730 0.750 S-ConvNet 0.777 0.905 0.835 0.990 0.731 0.556 0.721 0.866 0.674 0.644 0.796 0.703 0.755 0.855 0.767 0.778 0.808 0.932 0.830 0.656 0.715 0.787 0.776 EEGNet-2.32 0.785 0.877 0.854 0.980 0.626 0.509 0.720 0.943 0.694 0.769 0.745 0.855 0.766 0.807 0.836 0.785 0.762 0.971 0.885 0.805 0.740 0.826 0.797 EEGNet-4.8 0.851 0.926 0.853 0.960 0.729 0.500 0.688 0.895 0.721 0.731 0.814 0.895 0.808 0.873 0.855 0.885 0.827 0.980 0.914 0.749 0.797 0.865 0.824 Table A.12: Classification accuracy obtained by different classifiers for 22 subjects using 13 channel layout over motor cortex for MNS vs. MNS+MI discriminating task. Signals filtered in 0.683 0.788 0.750 1.000 0.433 0.625 0.721 0.750 0.615 0.606 0.471 0.683 0.558 0.827 0.712 0.865 0.740 0.923 0.875 0.644 0.673 0.769 0.714 MDM 0.548 0.808 0.750 1.000 0.433 0.567 0.644 0.798 0.606 0.625 0.587 0.606 0.462 0.750 0.702 0.837 0.750 0.846 0.788 0.606 0.635 0.692 0.684 TS+LR 0.606 0.798 0.798 1.000 0.433 0.663 0.731 0.798 0.683 0.663 0.490 0.740 0.577 0.856 0.750 0.865 0.808 0.865 0.894 0.683 0.788 0.760 0.739 D-ConvNet 0.693 0.915 0.912 1.000 0.627 0.589 0.643 0.752 0.766 0.658 0.665 0.769 0.855 0.836 0.700 0.778 0.779 0.952 0.868 0.713 0.520 0.778 0.762 S-ConvNet 0.786 0.895 0.846 0.980 0.731 0.539 0.702 0.865 0.675 0.684 0.778 0.732 0.747 0.847 0.729 0.862 0.858 0.961 0.829 0.685 0.769 0.818 0.787 EEGNet-2.32 0.755 0.898 0.835 0.970 0.693 0.509 0.632 0.933 0.741 0.675 0.783 0.922 0.981 0.875 0.827 0.925 0.847 0.990 0.905 0.826 0.769 0.874 0.826 EEGNet-4.8 0.805 0.916 0.874 0.970 0.759 0.482 0.738 0.905 0.758 0.733 0.842 0.904 0.970 0.884 0.856 0.855 0.856 0.990 0.914 0.809 0.808 0.884 0.841 0.558 0.471 0.529 0.500 0.529 0.510 0.606 0.625 0.587 0.740 0.606 0.606 0.567 0.606 0.452 0.596 0.558 0.538 0.529 0.490 0.510 0.529 0.556 MDM 0.596 0.413 0.567 0.490 0.519 0.635 0.577 0.567 0.625 0.644 0.644 0.606 0.567 0.538 0.481 0.587 0.500 0.548 0.538 0.538 0.538 0.538 0.557 TS+LR 0.558 0.413 0.538 0.471 0.519 0.577 0.596 0.606 0.615 0.750 0.635 0.615 0.587 0.548 0.452 0.577 0.577 0.538 0.558 0.490 0.558 0.577 0.562 D-ConvNet 0.519 0.460 0.536 0.508 0.540 0.459 0.470 0.540 0.539 0.560 0.568 0.480 0.507 0.510 0.481 0.537 0.500 0.619 0.516 0.491 0.556 0.530 0.519 S-ConvNet 0.423 0.490 0.615 0.469 0.567 0.451 0.535 0.536 0.607 0.656 0.522 0.584 0.595 0.505 0.536 0.641 0.531 0.598 0.613 0.479 0.457 0.606 0.546 EEGNet-2.32 0.596 0.434 0.520 0.491 0.529 0.421 0.491 0.558 0.588 0.643 0.490 0.568 0.587 0.505 0.577 0.633 0.532 0.604 0.481 0.459 0.565 0.565 0.538 EEGNet-4.8 0.462 0.500 0.555 0.466 0.673 0.490 0.484 0.567 0.515 0.606 0.519 0.618 0.538 0.544 0.562 0.654 0.520 0.579 0.541 0.509 0.472 0.566 0.543

Table B.44:

Classification accuracy obtained by different classifiers for 22 subjects using 3 channel layout over frontal lobe for MI vs. rest discriminating task. Signals filtered in