Keywords: 5, 1 ANOVA Results -Batch receiving feedback, Comparison Ttest with Bonferroni correction -Batch receiving feedback

Je parle pour les gens habitués à trouver de la sagesse dans la feuille qui tombe, des problèmes gigantesques dans la fumée qui s'élève, des théories dans les vibrations de la lumière, de la pensée dans les marbres, et le plus horrible des mouvements dans l'immobilité. Je me place au point précis où la science touche à la folie, et je ne puis mettre de garde-fous.

-Honoré de Balzac, Théorie de la démarche, 1833

Pour emprunter une métaphore surannée, je dirais que je n'ai peut-être pas été assis sur les épaules de géants, mais plutôt suspendu à leurs mains. Fabrizio, Marie-Constance, et Juliana ont été ces "géants" pour moi. Je tiens également à remercier les « anciens », Alexandre Routier, Arnaud Valladier, Simona Bottani, Tiziana Cattai, et Raphaël Couronne pour m'avoir accueilli si chaleureusement dans le laboratoire et de m'avoir tant inspiré dans leurs domaines de recherche respectifs. Je n'aurais jamais imaginé que le laboratoire, en particulier Aramis et l'Institut du Cerveau, puisse être le berceau de tant de réflexions scientifiques fructueuses. Je remercie chaleureusement Camille, Élise, Sophie, Domitille, Charley, Rémi, Matthieu, Sofia, Pierre-Emmanuel, Nemo, Ravi, et Arya pour les discussions passionnantes au laboratoire et en dehors. Merci pour les moments de vie, de "vélo", de rires, et pour tous les soirs à débriefer sur le monde et la science. L'institut acceuille des personnes venant de domaines de compétences et de connaissances extrêmement variés ce qui permet de regarder les choses sous des angles variés, de prendre du recul, mais aussi construire des réflexions profondes, avec des enjeux de poids.

En ce qui concerne les deux personnes suivantes, je ne peux que remercier la fortune de les avoir mises sur mon chemin. Je cite ici Montaigne : "Au demeurant, ce que nous appelons ordinairement amis et amitiés, ce ne sont qu'accointances et familiarités nouées par quelque occasion ou commodité, par le moyen de laquelle nos âmes s'entretiennent. En l'amitié de quoi je parle, elles se mêlent et confondent l'une en l'autre, d'un mélange si universel qu'elles effacent et ne retrouvent plus la couture qui les a jointes." Je conclus donc cette partie consacrée au laboratoire en remerciant Élisa de Launoit, une amie absolument extraordinaire. Elle est irradiante de bonté et d'optimisme. Partir en conférence avec elle est la meilleure des expériences, réfléchir sur la science et le monde avec elle est une chance et ses savoirs toujours partagés sans artifices ni prétentions sont des cadeaux. Enfin, au cours de ces trois années, j'ai eu le privilège de côtoyer un être exceptionnel en la personne de Vito Dichio. Être son ami est un privilège, et nos innombrables échanges ont été une source d'inspiration pour moi. Son amitié a été une grande motivation pour affronter les hésitations et les doutes. Je le remercie de m'avoir fait découvrir le second plus beau pays du monde, je resterai éternellement attaché à cette terre et à cette langue.

En dehors du laboratoire, j'ai pu compter sur le soutien indéfectible d'une bande éclectique d'amis qui m'a toujours distrait de mes inquiétudes. Je remercie donc Poeiti, Alésia, Clément, Luc, Gonzague, Hermès, et Luc, qui me connaît mieux que moi-même, je leur suis éternellement reconnaissant pour cette amitié exceptionnelle que nous partageons.

Pour finir, je lance à mes parents et mon frère un millier de mercis pour leur soutien sans failles et leur sagesse, pour être là à chaque instant, pour me rappeler de relativiser mes échecs, de m'encourager quand je pense que rien ne va, merci de m'aimer malgré mes colères et mon impatience. Votre amour à lui seul me permet de faire face. successive samples classified as either motor imagery in red or resting state in blue. The illustration shows two different features considered as the feature vector. If more samples are classified as belonging to motor imagery than as belonging to resting state, the robot's gripper closes. . . . . 3. [START_REF] Subasi | Chapter 2 -Biomedical Signals[END_REF] The can to reach is selected based on the average of the gaze position along the x-axis for 2 seconds 3.6 Placing the robot arm in the environment: Considered positions of the robot with regards to the table with cans reachable, on the right of the subject, tilted at 45 or 90 degrees, or in front of the subject. The position kept for the protocol will be the one where the robot is facing the subject at the over end of the table. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Robot placement and visual feedback: Robot from above and the motor imagery stimulus (in red) below the cans. The outside circle will be later on describe in the Neurofeedback section, it is a continuous feedback linked the neural activity of subjects during their motor imagery trial. . 3.8 From EEG to ML algorithm: Chain of treatment from EEG signal to classification. First we acquire the raw EEG signal corresponding to trials of motor imagery (in red) or resting state (in blue).

From theory to experience, first successes, first failures

We estimate for each trial the power spectrum to obtain an average over trial power spectrum. The average power spectrum for a specific bin of frequency for a specific electrode for each class (MI or rest) is then considered as a feature. If the features are discriminant enough, a machine learning algorithm can draw a separation between those classes (an hyperplane). . . . . . . . . 1 Results of a "prodigy" : left, 𝑅 2 map corresponding to the difference between motor imagery and resting state over trial of a specific phase (here the calibration). Right: The average powerspectrum over the phase for both condition (Top), The associate topography to the 𝑅 2 map to indicate the localization of the desynchronization -the sensori motor cortex (Bottom) . . . . . . . . . . . . . 4.2 Evolution during session: Evolution of the 𝑅 2 map, computed on the power spectrum difference between motor imagery and resting state over the trials of each phase, from the calibration to the driving phase for a "prodigy" subject. Here, what is noticeable is the spread of high statistical differences between resting state en motor imagery PSD in the sensory motor area in the low 𝛽 band from the calibration to the control phase. This spread out of the intensity can be the mark of the engagement of the subject but also his or ability to greatly perform the task. . . . . . . . . . 4.3 Results of a "sysyphus" : left, 𝑅 2 map corresponding to the difference between motor imagery and resting state over trial of a specific phase (here the calibration). Right: The average powerspectrum over the phase for both condition (Top), The associate topography to the 𝑅 2 map to indicate the localization of the desynchronization -the sensori motor cortex (Bottom) . . . . . . . . . . . . . 4.4 Evolution during session: Evolution of the 𝑅 2 map, computed on the power spectrum difference between motor imagery and resting state over the trials of each phase, from the calibration to the driving phase for a "sysyphus" subject. Here what is noticeable compared to the "prodigy" behaviour is that the highest statistical difference is not in the sensory motor area and the spread out in the 𝛼 band across all the electrodes is a mark of the inability to perform the task (yet). . . 4.5 Variability of resting state and MI: Distribution of the calibration trials' power spectrum for the specific CP3 electrode at 13Hz, Red for motor imagery , Blue for resting state, points on the line corresponds to the new features of trials of control 1. Each point represent a trial of the control phase whereas the histogram represents the trials of the calibration phase. . . . . . . . . . . . .

Comparing variability of prodigy and sysyphus: Features by Principal Component Analysis,

Red for motor imagery , Blue for resting state, left "sysyphus" -distribution of the control 2, right "prodigy" -distribution of the calibration. First, we select using the 𝑅 2 map the features (specific electrodes at a specific frequencies) that present the highest statistical differences. Second, we use over the trials of a block (calibration or control 1) a PCA on the feature matrix consisting of the observations for the different features selected to retrieve only the principal 3 eigenvectors (based on the 3 highest eigenvalues), third we plot the 3 components in the different trials for the two conditions. The purpose of the plot is to have a synthetic representation of the features to see their distribution. . between subjects for the band ) in first phase with no control over the robot (Calibration), threshold of display set to 𝑝 < 0.05. Down : Evolution of the cluster size (in decimeter) in function of the negative ERD threshold. To do so, we average across subjects the ERD for each electrodes, we then evaluate the diameter of the cluster formed by the sensori motor cortex area. We increase the negative ERD threshold to see how the cluster diminishes, this way we have an insight on the relevant electrodes at the group level. . . . . . . 4.13 ERD/ERS spatial distribution in control 1:Top : Cluster permutation based permutation test perform on power spectrum 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡 between subjects for the band in first phase of pseudo control over the robot (Control 1), threshold of display set to 𝑝 < 0.05. Down,left Evolution of the cluster size (in decimeter) in function of the negative ERD threshold. To do so, we average across subjects the ERD for each electrodes, we then evaluate the diameter of the cluster formed by the sensori motor cortex area. We increase the negative ERD threshold to see how the cluster diminishes, this way we have an insight on the relevant electrodes at the group level. Down, right, Number of electrodes in each cluster for each strategy. . . . . . . . . . . . . . . . . . . . . . . . 4.14 ERD/ERS spatial distribution in control 2:Top : Cluster permutation based permutation test perform on power spectrum 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡 between subjects for the band in second phase of pseudo control over the robot (Control 2), threshold of display set to 𝑝 < 0.05. Down,left Evolution of the cluster size (in decimeter) in function of the negative ERD threshold. To do so, we average across subjects the ERD for each electrodes, we then evaluate the diameter of the cluster formed by the sensori motor cortex area. We increase the negative ERD threshold to see how the cluster diminishes, this way we have an insight on the relevant electrodes at the group level. Down, right, Number of electrodes in each cluster for each strategy.

Evolution of the computation of the feedback:

On top, the computation of the feedback for batch 1, at each power estimation, each feature (C3 at 13 Hz for example) is classified by the LDA using the distance to the hyperplane (negative values associated to class one, positive values associated to class 2), then at the end of a trial, the sum of the distances to the hyperplane gives a positive or a negative value resulting in the closing of the gripper. This method is sensitive to power spectrum variations. Bottom, the computation of the feedback for batch 2, the average PSD over time windows for each feature is classified by the LDA, only one hyperplane distance is output, either negative or positive. This method is less sensitive to power spectrum variations over trial.

5.1 Braccio protocol -a multimodal BCI platform experimentation. a) BCI platform composed of the 64 EEG BrainAmp device, the Tobii Pro Glasses 3 Eyetracker, the augmented table with the display monitor underneath the glass and Pollen Robotics's Reachy arm facing the subject. The robot goes to the desired can position accoordingly to the gaze position and closes its gripper based on motor imagery activity. b) The three different timings of control investigated in the protocol, strategy one (in green) -performing cognitive tasks before the robot reaches the can, strategy two (in red) -performing cognitive tasks meawhile the robot reaches the can, strategy three (in blue) -performing cognitive tasks after the robot reaches the can. . . . . . . . . . . . .

5.2

Behavioral scores and Performance. a) Correlation between the highest performance of subjects across strategies in accuracy and sensitivity in function of their motivation, rho established with spearman correlation,𝑝 𝑣𝑎𝑙𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡 𝑦 = 0.024, 𝑝 𝑣𝑎𝑙𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = 0.007. b) Average Accuracy in the two phases of control for each strategy for the subject batch that received feedback. The two-way Anova test was performed between phases and strategies, test revealed that the phase was the only significant factor at 𝑝 < 0.0002, in the post-hoc analysis with bonferonni correction, significant difference was only found between control phases of strategy 3 at 𝑝 < 0.003. . . . . . . . . . . . across subjects for the band in second phase of robot control with feedback (Control 2), threshold of display set to 𝑝 < 0.05. The distance matrix needed for the cluster computation was set considering a threshold of 40 mm between EEG channels. We use a threshold computed using percent point function at 𝛼 = 0.05 for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15 b) Cluster based permutation test performed in the source space power spectrum 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 across subjects for the band in second phase of robot control with feedback (Control 2). Source space dipoles estimated using weight minimum norm (wMNE). Only strategy 3 present significant clusters (presented here at 𝑝 < 0.05, the clusters of the two other strategies are displayed but are not significant. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cluster based permutation test performed on node strength functional connectivity using welch spectral coherence 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 across subjects for the band in second phase of robot control with feedback (Control 2). Cluster based permutation evaluated at a threshold determined by quantile function evaluated for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15 and 𝑝 < 0.05 , clusters kept at 𝑝 < 0.05. The distance matrix needed for the cluster computation was set considering a threshold of 40 mm between EEG channels. Only strategy 3 presented statistical differences. Links are computed using cluster permutation test on the difference of functional connectivity matrix between MI and resting state across subjects to show the different levels of connectivity between strategies. Number of links shown corresponded to 𝑝 < 0.001 for strategy 1 and 2, 𝑝 < 10𝑒 -06 for strategy 3. . . . . . . . . . . . . . . . . . . . .

In all phases of the experimentation for all strategies:

Cluster based permutation test performed in the source space power spectrum 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 across subjects for the band in second phase of robot control with feedback (Control 2). Source space dipoles estimated using weight minimum norm (wMNE). Significant clusters at 𝑝 < 0.05. . . . . . . . . . . . . . . . . . . . . .

Subject level analysis of their patterns variations during the experimentation: Top: Comparison

for each strategy of the distances in the different configurations. Each strategy respectively : Left -distances between calibration and control 1 of the matrix of 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 power spectrum for 64 electrodes on the frequency bins ranging from 7 to 35 Hz, Center -distances between calibration and control 2 of the matrix of 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡 power spectrum for 64 electrodes on the frequency bins ranging from 7 to 35 Hz, Right -distances between calibration and control 1 of the matrix of 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡 power spectrum for 64 electrodes on the frequency bins ranging from 7 to 35 Hz. Green -Strategy 1, Red -Strategy 2,Blue -Strategy 3, from dark to light colour : Frobenius, Spectral form, Mahalanobis distance.Bottom: Boxplot associated representation of the distances in the different strategies, Wilcoxon test to assess the level of differences between the distances. * : 𝑝 < 0.05 . . .

Introduction 1

Among subjects of science that unleash passions, Brain Computer Interface (BCI) is appearing often. Its ability to nourish the wildest and sometimes scariest imaginations makes it complex to apprehend and even more to work on. It is rather difficult to juggle between our own expectations and the reality of the field which is struggling with a lot of insoluble problems, largely due to our poor understanding of brain mechanisms. BCI exited science fiction in the 1970s with Jacques Vidal's [START_REF] Vidal | Toward Direct Brain-Computer Communication[END_REF] reflection on the feasibility to use electro-encephalography signals to establish communication with an external device. Later on, in the beginning of the 1990s, with the improvement of computers, the reflection became experiments forming the premises of a field that would be growing until today.

During this growth, many different tracks were followed to make it possible to actually control devices based on human's brain activity. The acquisition techniques took many forms ranging from invasive, which gave tremendous information on brain signals to non invasive, which kept improving usability and facility to deploy, both having their pros and cons. With the improvement of computer capability and the dawn of machine learning, methods to classify brain data have become more refined, subtle and elegant, capable of capturing complex information of brain data. The BCI field has the particularity to advance in parallel with the understanding we have of the brain in fundamental neuroscience. Experimentation is done based on this acquired knowledge. Sometimes, by creating new interactions, the field also contributes to understand better brain behaviours we would not have suspected. However, it must be said, that this particular field is largely invested by a trial-error mindset, fumbling around the darkness with the failure and successes of a few for unique light.

To this day, it would seem rather unwise to advocate for a daily use of such a system for the simple reason that the technology itself is not ready yet to enter in our life. Nevertheless, it could easily find its use for the ones that need it the most. Before enhancing humans, let us try to repair them. In the case of loss of movement capability -symptomatic of many diseases or accidents among which we can evoke amyotrophic lateral sclerosis (ALS) or spinal cord injury (SCI) -providing to patients a way to communicate or to interact with the external world is absolutely necessary. BCIs can be a response to this challenge as their capacity to interact with the users' brain and their intentions is what the interface is all about. Will we be able to make patients walk again ? It is not certain yet with non invasive techniques, but it seems more and more imaginable with invasive methods. Another promising lead to help patients regards the stroke rehabilitation process. Indeed, by performing movements or at least by translating to patients their brain activity, we can try to retrain their brain and to overcome their injury by synaptic rewiring process. BCIs here play the role of a crutch and a rewarding element in the meantime patients cannot fully move, that being said, they should not be thought as the only solution as subjects' improvement could be limited by the sole use of the technique.

Introduction

In all of this, how should we contribute to BCI systems, what are the elements that should be tackled and what should we choose to tackle based on our own expertise ? A particular challenge for subjects is to create brain patterns that are differentiable at the EEG level. To create those patterns, we rely on cognitive tasks that change the brain activity profile. Among those cognitive tasks, one that changes the activity is the motor imagery of limb movements, a task which consists in imagining movements without performing them. This peculiar task is unfamiliar to many and therefore rather complex to execute. A way to help subjects is to use evocative feedback in the BCI context such as robotic arms. Nourishing the sentiment of control over the arm as well as the movements produced by the arm help to elicit stronger differentiable brain patterns. But, due to the current limitations regarding degrees of control permitted by BCI systems, a full control over a robotic arm is not possible. To go around those limitations, a possible solution is to couple the BCI with another technology to increase the degrees of control and to reinforce subjects' sense of control over the arm. Among the possible technologies that offer a window on the intention of subjects without making them move, the eye tracker appear to be an elegant solution. It accesses this intention through pupil position that infers the gaze direction. The integration of the two components creates an hybrid BCI system capable of controlling the arm in an intuitive manner.

This hybridisation has been demonstrated as a proof of concept in the BCI field, but the impact of the integration of those modalities on the brain as well as how we should merge those modalities remain to be studied. Indeed, understanding better how to shape the interaction between the eye tracker, the BCI and the robotic arm, would result in a better comprehension of why we obtain good performances and how to elicit those discriminant brain patterns. This work consisted in the creation of an experimental platform that entangles the different modalities to establish a strong control over the arm. Because we create such a platform, in the process, we raise up interrogations concerning its use, its strength and its flaws. By conducting an experimental protocol, we assess how we should define the interaction by a thorough analysis which covers pure performance of the system, physiological and neuro-physiological responses of subjects. During two different campaign of experimentation which lead to two scientific contributions, we investigated the time to perform the cognitive task in a multimodal control over a robotic arm, the moment being related to the arm's movement (before, during or after it reaches an object). We find out that consistency is key in the interaction, we also demonstrate the importance of movement to elicit brain responses in this particular context. Furthermore, we highlight the notion of intentional binding, at the heart of the sense of agency, that stresses the importance of time between intention of action and its realization. Our work paves the road for a better comprehension of the dynamic of the brain in its control over external devices in a multimodal setup.

This thesis is structured in five different chapters covering the overall BCI context, the development of the experimental platform, the results from the experimental protocol associated with their discussion and finally a general conclusion on the work realised and its openings.

▶ Chapter 2: This chapter presents the overall state of the art with an emphasis on the notion of intuitiveness. It enters into the details of what is a brain computer interface, first by presenting briefly the brain mechanisms responsible for the observation of brain signals at the surface of the scalp. It then presents the different brain signals that can be used as commands for external devices. The chapter then thoroughly discusses the motor imagery paradigm for BCI and the leads to improve the system. It also presents the key elements to have in mind in robotic control and its association to BCI and the eye tracking technology is also evoked. The chapter concludes by introducing the multimodal approach to BCI and its current state-of-the-art. ▶ Chapter 3: This chapter presents the development of the prototype platform, the focus is on the initial interrogations and the technological development both from the hardware and the software perspectives. The chapter ends by introducing the experimental protocol, the methods finally used and the different hypotheses on the results. The interrogation risen by the protocol concerns the dynamic of control of the robot. In this multimodal paradigm, we interrogate the moment to perform the cognitive task either prior to any movements of the robot or after it moves or meanwhile it moves to maximize the performance of the system and the brain responses. ▶ Chapter 4: This chapter presents the results from the first batch of 11 subjects as well as some general observations from the two batches related to their general ability to perform the experience. It establishes a synthesis of all the statistical analysis and it shows the differences online and offline performances leading to changes in the experimental setup. It brings conclusions and insights on the development of the platform and the solutions to improve the protocol. ▶ Chapter 5: This chapter explores results of the second batch of 15 subjects, it discusses regarding those results and points out the strength and weaknesses of the protocol. It is this chapter that brings most of the answers to our hypotheses and opens to the different phenomena at stake when designing multimodal setups of control. ▶ Chapter 6: The thesis ends by a chapter that summarizes the work done and puts it back in the general scope of BCI. It also establishes the main leads of future development that could be directly extrapolate from the reflection and from the platform itself. It then concludes on the general need for the use of movement in the BCI field. In this chapter, we will try to cover the overall context in which the thesis took place. Certain themes will be simply mentioned as they are part of the general BCI ecosystem as others will be more thoroughly introduced as they had direct implication in the development of this work. This overview however cannot be considered as a pure review as it is oriented towards a specific direction regarding our own developments. Without further introduction, this chapter will cover five different topics, EEG patterns that can be exploited as machine commands, possible ways to improve BCI, robotic control through BCI, eye tracking technology and multimodal approach to BCI.

From brain patterns to command

To understand the patterns that we are going to introduce, it is necessary to present the human brain both structurally and functionally. The brain is structured around four different porous lobes represented on Fig 2 .1 that serve different cognitive functions [START_REF] Sure | The human brain: surface, blood supply, and three-dimensional anatomy, 2nd edn, completely revised[END_REF] in addition to the cerebellum, the zones interact between themselves even though specific behaviour can be found in specific regions. Those zones cover two hemispheres, that serve different functions while being constantly in interaction [START_REF] Dharani | Chapter 1 -Functional Anatomy of the Brain[END_REF].

The brain is irrigated via a complex vasculature. At a microscopic scale, the brain tissue is composed of many different cells. The neural cells are responsible for the brain functioning and the electrical information transfer. Whereas the glial cells (astrocytes,oligodendrocytes,Schwan cells,microglia and ependymocytes) in a broad sense principally serve a role of support [START_REF] Allen | Glia -more than just brain glue[END_REF], bringing nutriments to the neurons and creating the myelin as represented in Fig 2 .2. They serve other functions not all fully understood yet. They, for instance, play a role in modulating the electrical transmission and in forming the blood brain barrier. Neurons' functions in the other hand, are better understood (because studied for a longer period), they transmit electrons through the exchange of sodium and potassium ions. Which in fine generates electric differences of potential that propagate from one neuron to the next thanks to synaptic connections as represented below in Fig 2 .3.

This difference of potential creates an electric signal in microvolts that is going to gain amplitude the more neurons burst synchronously at different frequency rate [START_REF] Subasi | Chapter 2 -Biomedical Signals[END_REF]. To sense this electrical activity, different solutions can be considered, the two main non-invasive techniques are magneto-encephalography (MEG) [6][7] which is based on the electromagnetic field generated by the local potential difference and the electroencephalogram (EEG) technology which senses the electrical activity directly at the scalp level. When the electric signal arrives at the scalp, it is highly noisy due to the propagation from deep zones that diminish its ratio signal over noise. Moreover, because of the spatial constraints given by the shape of the brain and skull, the signal that arrives is not exclusively coming from the area beneath the electrode as shown in Fig 2 .4. A possible metaphor for that phenomenon is the circus tent you see from above, if someone lights projectors inside illuminating the scene and the roof, you do not know from where the light is coming from. Taking that into account, there are still a relevant number of information coming from this electric signal, and it is still the quickest non invasive access to the neuronal communication in terms of response time in comparison to other technologies that would sense for instance the blood flow which produces a delay from the moment the information has been generated in the brain to the moment it is sensed [START_REF] Li | Concurrent fNIRS and EEG for Brain Function Investigation: A Systematic, Methodology-Focused Review[END_REF]. The brain signals give temporal information, if we extract directly the time series, and spectral information, if we estimate the power spectrum from the time series.

Brain activity consists in changes between oscillatory behaviours and electric waves passing through the different zones. In this misty environ- Four different types of signatures will be mentioned, the evoked potentials, the event related potentials, the movement related potentials and the event related synchronization/desynchronization which is going to be more thoroughly explored as it is this specific signature we rely on in the thesis to build the BCI system. All these signatures, once presented will be introduced in the specific context of BCI later on.

The Electro-encephalogram technology

Before talking about the patterns, it is necessary to introduce more precisely the EEG technology [START_REF] Simpson | Chapter 16 -The electroencephalogram[END_REF]. This method of acquisition, invented in the 19 𝑡 ℎ century and perfected in the 20s by H.Berger with the amplification of the signal, relies on electrodes sensing microvolt (𝜇𝑉) activity. The electrodes are usually made of metal such as silver, tin, or gold, and are connected to wires that transmit signals to amplifiers [START_REF] Cooper | EEG Technology. en. Google-Books[END_REF].

The conductive gel or paste used to make the junction with the scalp helps to reduce the electrical resistance, allowing for a more accurate recording of the brain activity. Each electrode is placed at a specific location on the scalp, based on a standardized system of placement. Among the different systems, we can mention the International 10-20 system, and the 10-10 system that comes from it (represented in Fig 2 .5). This system is based on the distance between key landmarks on the head, such as the nasion (the point where the forehead meets the nose) and the inion (the lowest point of the skull at the back of the head). The amplification is separated into two phases. In the first phase, the pre-amplifier amplifies the analog signal and filters out unwanted noise and artifacts that may be present in the signal at the electrode level. The signal is then transmitted to the main amplifier. In the second phase, the main amplifier has a high gain, high input impedance and low noise, and is designed to amplify the signal without introducing any additional noise or distortion. After amplification, the signal is digitized by an analog-to-digital converter and sent to the computer.

The next sections present the different patterns we can extract from the EEG system, there are meant to be presented as a list to indicate that 2: A stimuli is an input to the brain, an event occurring at a giving time and for a certain period of time in order to produce a change in the brain activity. there are plenty of patterns exploitable and that we deliberately choose one of them based on what we want to achieve.

Exploitable brain patterns

Evoked Potentials (EPs)

Evoked potentials are brain patterns elicited by an external stimuli 2 [START_REF] Sörnmo | Chapter 4 -Evoked Potentials[END_REF].

In other words they are not the result of an internal process. The signal in itself has low amplitude. This event relies on the different senses the brain has at its disposal. The event can be auditory (producing a sound), visual (a change of light, an object appearing in the vision field), touch (touching the skin), or nervous (inducing an electrical signal in a nerve).

Visual stimuli can be used to elicit a specific pattern such as steady state visual potentials (SSVEPs).

Steady State Visual Potentials (SSVEPs)

SSVEPs rely on the fact that the brain can "resonate" at the frequency of the visual stimuli presented [START_REF] Norcia | The steady-state visual evoked potential in vision research: A review[END_REF]. The choice of the frequency is large as some studies suggest, it can range from 2 to 100 Hz. The brain reproduces a signal at the frequency given which makes it convenient to detect as represented in Fig 2 .6. The detection is robust due to the high signal over noise ratio and observable from one subject to the next.

Event Related Potentials (ERPs)

There is a debate whether event related potentials or evoked potentials are the same or not. We are not going to enter into those considerations. We choose to separate the two based on the fact that ERPs seem to involve higher cognitive process and even though the end result is that an external stimulus elicits a brain pattern, the way the pattern is created has a higher dependence on the choice of the user. This is developed by Brandeis and Lehman who characterize the different event related potentials [START_REF] Brandeis | Event-related potentials of the brain and cognitive processes: Approaches and applications[END_REF]. Amongst the visual event related potentials, one of the most famous is the P300, a positive amplitude response 300 milliseconds after the presentation of an unfamiliar stimuli as shown in Fig 2 .7.

Error Related Potentials (ErrPs)

One stimulus in particular can give place to a slightly different response, it is the error of the system. In a preconceived logical sequence of events, if one of them changes from its logical track, it becomes an error for the brain. A specific brain signal appears, up to one second after, and characterizes the "surprise" of the brain to the incorrect behaviour. This was first discovered by Falkenstein et al [START_REF] Falkenstein | Effects of crossmodal divided attention on late ERP components. II. Error processing in choice reaction tasks[END_REF] in a study on reaction to error in bimanual choice reaction task. The ErrPs include two patterns that follow each other as shown in Fig 2 .8: first, a negative deflection in the EEG waveform occurring up to 100 ms after the event, second, a positive deflection in the EEG waveform approximately 300 to 500 ms after the error.

Movement Related Potentials (MRPs)

As presented by Shakeel et al [START_REF] Shakeel | A Review of Techniques for Detection of Movement Intention Using Movement-Related Cortical Potentials[END_REF] in their review, MRPs are a sequence of patterns starting prior to the movement. It begins with the Bereitschaftspotential [START_REF] Shibasaki | What is the Bereitschaftspotential?[END_REF] (BP), then the motor potential (MP) and finishes with the movement monitoring potential (MMP). It occurs in the motor cortex, it is generated up to one second before the actual action and it is a representative signature of motor task planning. MRPs typically have a characteristic waveform as shown in Fig 2 .9 consisting of a negative peak around 200-300 milliseconds after movement onset, followed by a positive peak around 400-500 milliseconds after movement onset. These components are often referred to as the negative motor potential (NMP) and the positive motor potential (PMP), respectively.

Event Related Synchronization/Desynchronization (ERD/ERS)

ERD refers to a decrease in the power of neural oscillations in specific frequency bands (e.g., 𝛼 or 𝛽) happening during the execution of a motor task, while ERS refers to an increase in power in the same frequency bands during the preparatory phase before the motor task. Characterized by Pfurtscheller for the first time in 1977 for the ERD [START_REF] Pfurtscheller | Event-related cortical desynchronization detected by power measurements of scalp EEG[END_REF], and later on for the ERS, those two phenomenon [START_REF] Pfurtscheller | Event-related EEG/MEG synchronization and desynchronization: basic principles[END_REF] are shown in Fig 2 .10. Pfurtscheller advanced the idea that the event related synchronization or desynchronization [START_REF] Pfurtscheller | Event-related EEG/MEG synchronization and desynchronization: basic principles[END_REF] corresponds to a change in the number of neurons bursting at the same frequency when an internal process occurs as shown in Fig 2 .11.

Attributing commands

The different patterns mentioned before have the advantages of being reproducible and observable for a large amount of subjects. Therefore, it is possible to trick the brain in order to create those specific activities and associate the detection of those particular activities to a command. By "tricking", we mean that we can make subjects generate those patterns voluntarily and sense this induced activity with EEG for instance. The most elementary step being: 0 not detected -no action; 1 detected -action.

In the case of SSVEPs. Guger's review [START_REF] Guger | How Many People Could Use an SSVEP BCI?[END_REF] on SSVEP BCI describes the different applications and limitations of the approach. The main method is to make several zones blink at different frequencies, based on where the subject is looking, we can retrieve the different frequencies and from that issue several commands corresponding to the different zones of interest.

For ERPs, we can present different stimuli with only one being of interest for the subject, we can then know what was the stimulus of interest based on the brain pattern detection and create a specific command for this specific stimuli. This was extensively used with the P300 wave in the context of spelling words [START_REF] Pan | Advances in P300 brain-computer interface spellers: toward paradigm design and performance evaluation[END_REF]. Furthermore, we can design a controller that takes into account the ErrPs to correct itself if the sequence seems odd to the subject [START_REF] Kumar | A Review of Error-Related Potential-Based Brain-Computer Interfaces for Motor Impaired People[END_REF]. And it can serve to correct other BCIs if they wrongly issue a command as Ferrez and Millan described in [START_REF] Ferrez | Error-related EEG potentials generated during simulated brain-computer interaction[END_REF] and put to use by Lopes-Dias [START_REF] Lopes-Dias | Online asynchronous decoding of error-related potentials during the continuous control of a robot[END_REF] to correct a robot trajectory. Finally, we can use MRPs or ERD/ERS to create commands based on the detection of the motor-cortex activity which is presented in the next section.

Once you have come up with possible commands based on the acquisition of signals, it is necessary to interact what could be the use of such technology.

Why do we do BCI ?

To start this introduction on brain computer interface, it is necessary to talk about its use. The question could be asked of the relevancy to decode brain signals to control external devices when we could issue commands from all the other outputs provided by the body which are far more accessible (gaze, speech, touch, etc). Accessing the brain information presents, despite all the challenges, an incredible advantage in the case of severe impairments. So, the principal use of BCI, at least for now, is to serve in priority the disabled rather than providing the general population new tools of control, the BCIs can serve either to restore the function or to compensate for the loss by providing another channel of communication towards the outer environment. Far from modern concepts of augmenting humans [START_REF] Kleinpeter | L'humain augmenté Series Title: Les essentiels d'Hermès[END_REF], it is more reasonable to think of repairing them first. Many diseases and accidents create a situation of disability due to an impairment (the loss of a limb or its functioning) [START_REF] Menter | Impairment, disability, handicap and medical expenses of persons aging with spinal cord injury[END_REF]. 3 Indeed, BCIs find their immediate use in the context of spinal cord injuries [START_REF] Birbaumer | Breaking the silence: Brain-computer interfaces (BCI) for communication and motor control[END_REF] [START_REF] Cajigas | Brain-Computer Interface, Neuromodulation, and Neurorehabilitation Strategies for Spinal Cord Injury[END_REF][START_REF] Dunkelberger | A review of methods for achieving upper limb movement following spinal cord injury through hybrid muscle stimulation and robotic assistance[END_REF], they provide a source of command left intact that can be used for restoration of the walk or even in reaching and grasping objects. In the context of amyotrophic lateral sclerosis [START_REF] Vaughan | Chapter 4 -Brain-computer interfaces for people with amyotrophic lateral sclerosis[END_REF] or multiple sclerosis, they provide the last access to patients desires when the disease is at its latest stage. And finally, BCIs can be used in the rehabilitation process of some cases of stroke [START_REF] Silvoni | Brain-Computer Interface in Stroke: A Review of Progress[END_REF]. It appears crucial to initiate a recovering process over motor action quickly after the stroke to regain motor capabilities. But, after the stroke, some phenomenons including muscle spasticity [START_REF] Kuo | Post-stroke Spasticity: A Review of Epidemiology, Pathophysiology, and Treatments[END_REF] prohibit from moving. In that case, motor imagery (and therefore motor imagery BCIs) can serve in stimulating the brain to help restore motor functions. Here, BCIs serve multiple roles, among them, we can mention that they give knowledge to subjects about their activity and their recovering progress. For caregivers, it is an access through the non invasive techniques to the remaining active motor areas [START_REF] Benzy | Motor Imagery Hand Movement Direction Decoding Using Brain Computer Interface to Aid Stroke Recovery and Rehabilitation[END_REF]. It is to mention that BCI performances are tremendously impacted by brain lesions' severity and locations. It means that systems must be even more tailored than for healthy subjects to patients to compensate for the handicap (in the control). This is a sine qua non condition if we want BCI to become true assistive technologies. On other aspects of handicap that do not concern motor impairment, we can mention psychiatric disorders with, for example, children with attention deficit who seem to respond positively to intervention using BCI [START_REF] Newson | EEG Frequency Bands in Psychiatric Disorders: A Review of Resting State Studies[END_REF][START_REF] Amaral | A Feasibility Clinical Trial to Improve Social Attention in Autistic Spectrum Disorder (ASD) Using a Brain Computer Interface[END_REF][START_REF] Arpaia | Robotic Autism Rehabilitation by Wearable Brain-Computer Interface and Augmented Reality[END_REF][START_REF] Fan | A Step towards EEG-based brain computer interface for autism intervention[END_REF].

Other uses are now considered such as in video games as an additional source of command in more immersive environment. The principal axis of development in this sector concerns responses to evoked potential. Their use remains however marginal and some might even say not ready [START_REF] Cattan | The Use of Brain-Computer Interfaces in Games Is Not Ready for the General Public[END_REF][START_REF] Bos | Brain-Computer Interfacing and Games[END_REF] to be used. On a completely different topic, it is possible to find some military application for BCI [START_REF] Miranda | DARPA-funded efforts in the development of novel brain-computer interface technologies[END_REF], especially related to passive BCIs [START_REF] Zander | Towards passive brain-computer interfaces: applying brain-computer interface technology to human-machine systems in general[END_REF][START_REF] Alimardani | Passive Brain-Computer Interfaces for Enhanced Human-Robot Interaction[END_REF] that monitor the brain activity to overwatch subjects. This is especially applied in the context of aviation to monitor fatigue and attention [START_REF] Dehais | Monitoring Pilot's Cognitive Fatigue with Engagement Features in Simulated and Actual Flight Conditions Using an Hybrid fNIRS-EEG Passive BCI[END_REF].

Brain Interfaces -types and axis of improvement

Using brain patterns as commands is the initial brick to build a brain machine interface. Certain brain patterns are easier to detect than other, and since there are plenty of patterns, there are plenty of brain machine interfaces existing. But all brain patterns do not carry the same meaning and are more or less feasible to apply in a general context. Moreover, the intuitiveness in the link between the brain signature and the command is going to be of key importance in our approach to design the brain machine interface system.

In this manuscript, we focus on the brain pattern of ERD/ERS elicited by motor imagery task. This task has to be introduced thoroughly for several reasons, first, because it is a complex task, second because the BCI paradigm relies in its proper execution, and third, its use can benefit subjects in the context of stroke for instance.

Use your imagination !

Motor imagery is a cognitive process which consists in creating a kinesthesic and visual representation of a motor action without performing it as defined by Guillot et al [START_REF] Guillot | Contribution from neurophysiological and psychological methods to the study of motor imagery[END_REF]. This task has been extensively studied from the 80s with Parsons [START_REF] Lawrence | Imagined spatial transformations of one's hands and feet[END_REF] reporting his subjects imagining their own hand and feet to move accordingly to the stimuli. It continued to be characterized over the years as a specific cognitive task that could be observed in the brain as described by Decety[49]. This mental task has the particularity to activate sensorimotor regions of the brain similarly to real motor actions [START_REF] Mulder | Motor imagery and action observation: cognitive tools for rehabilitation[END_REF] as we can observe in Fig 2 .12.

As mentioned before, this creates from a EEG perspective first an ERD followed by an ERS with regards to a baseline state 4 . One does not simply do motor imagery, it is a difficult, tiring and not an intuitive process which requires strong body ownership and intense focus. It is important to mention that this process can also be elicited by the observation of someone else performing a movement [START_REF] Robert | Neural correlates of action: Comparing meta-analyses of imagery, observation, and execution[END_REF]. Moreover using motor imagery contributes in integrating a movement in one's self internal workflow. This particular property has been studied thoroughly in sport as it allows to continue practicing without hurting oneself for better 5: Spasticity defines an excessive muscle contraction caused by the loss of motor neuron inhibition [START_REF] Kuo | Post-stroke Spasticity: A Review of Epidemiology, Pathophysiology, and Treatments[END_REF].

6: To use an analogy lacking of originality, it is similar to computer's standby mode.

[53]: Baker (2007), 'Oscillatory interactions between sensorimotor cortex and the periphery' recovery. An overview of the application of motor imagery is given by Macintyre [START_REF] Tadhg | Chapter 9 -Motor imagery, performance and motor rehabilitation[END_REF]. In a similar way, motor imagery has been used for stroke recovery as it allows severe hemiplegic or paraplegic patients suffering from spasticity 5 to initiate a neuronal activity in the sensorimotor areas which is essential to recreate or at least stimulate synaptic connections to regain some level of mobility.

Nonetheless, motor imagery remains a challenge as we are basically asking for moving without moving which is rather complex to apprehend. To second that, subjects have absolutely no clue if they are doing the task the right way or even if they are doing it at all. To answer this tedious question, we rely on the observation of the ERD/ERS with EEG for instance and provide feedback to give them an idea of what they are performing mentally. This is the elementary brick of a brain interface with motor imagery. But, to observe the apparition of this ERD/ERS, we need to oppose it to another state. This state is called the resting state. This mental task focuses on the brain with zero activity, not dead, fortunately, but without any cognitive process. It is a state of relaxation closed to sleep without being in fact asleep. In this state, 𝛼 and 𝛽 (to some extent) oscillations have high power amplitude, corresponding to a synchronous activity of the neurons bursting at the same rhythm 6 . In opposition to motor imagery which sees a reduction of the synchronicity in those bands resulting in a decrease of power amplitude in those same bands.

Brain interfaces -Description

In the next two sections, we are going to explore more deeply the different possibilities to associate the ERD/ERS signature to a command. But, we are going to leave the terms of ERD/ERS to focus solely on power spectrum. Indeed, the ERD/ERS as presented above, shows how the desynchronization and synchronization of neurons ends up to be a decrease or an increase of power spectrum in time (from an EEG perspective). Therefore we will rely on the change of power spectrum in the frequency bands associated to motor activity (𝛼 8-12 Hz and 𝛽 13-30 Hz) to determinate if we are in a motor imagery or a resting state. The literature on frequency bands is varying a lot, sometimes 𝛼 and 𝛽 are completed by 𝜇 rhythm, and the separation between them is arbitrary. The complexity lies in the boundaries between those bands rather than the bands themselves. Indeed, 𝛽 as defined between 15 to 30 Hz by Baker [START_REF] Stuart | Oscillatory interactions between sensorimotor cortex and the periphery[END_REF] and 𝛼 at ∼ 10 Hz leaves quite the choice for selecting the bounds. We can mention Davis [START_REF] Davis | The Role of Beta-Frequency Neural Oscillations in Motor Control[END_REF] establishing 𝛼 between 8 and 12 Hz and 𝛽 between 15 to 30 Hz and Marzbani [START_REF] Marzbani | Neurofeedback: A Comprehensive Review on System Design, Methodology and Clinical Applications[END_REF] introducing the sensori motor rhythm (SMR) as an intermediate band [START_REF] Norcia | The steady-state visual evoked potential in vision research: A review[END_REF][START_REF] Mandel | Navigating a smart wheelchair with a brain-computer interface interpreting steady-state visual evoked potentials[END_REF][START_REF] Brandeis | Event-related potentials of the brain and cognitive processes: Approaches and applications[END_REF][START_REF] Eisenbarth | Methods in Cognitive Neuroscience: A primer for Forensic Psychologists[END_REF]. We deliberately choose to base ourselves on Marcuse et al. [START_REF] Lara V Marcuse | 2 -The normal adult EEG'. en. In: Rowan's Primer of EEG[END_REF] for the sake of simplicity.

Introducing Motor Imagery Neurofeedback

Neurofeedback consists in the simplest connection between the power spectrum and the command by having a direct link between the power spectrum value and the feedback. Marzbani [START_REF] Marzbani | Neurofeedback: A Comprehensive Review on System Design, Methodology and Clinical Applications[END_REF], again, reviews a great quantity of papers on the topic covering the different frequency bands and their related applications. There are two possible ways to exploit the 7: Such as motor imagery of the right or left hand, or resting state. link, either through a continuous feedback (visual, auditory or touch) establishing a variation of the feedback based on the variation of the power spectrum amplitude or through a discrete feedback which relies on passing a threshold with the power spectrum amplitude which is going to switch the state of a feedback (on/off for instance). In a neurofeedback, the emphasis is on subjects' ability to modulate their brain rhythm to generate robust commands. However, the power spectrum variations are quite high, therefore a direct command will suffer from instability which makes the overall behaviour erratic in certain cases. On this note Geppert [START_REF] Enriquez-Geppert | EEG-Neurofeedback as a Tool to Modulate Cognition and Behavior: A Review Tutorial[END_REF] points out the ways to create protocols that limit those effects. Nonetheless, because it is based on the powerspectrum directly, neurofeedback remains with the time series displayed the mirror of brain activity through the EEG scope.

Introducing Motor Imagery BCI

The point until here was to show how broad the field can be and how easy we can lose ourselves in it. It shows that to actually create a Brain Computer Interface, we need to choose what we are going to exploit and it forces to ask why we should exploit one specific pattern more than another one. But also, by getting familiar with the different approaches to BCI, we might end up retrieving some crucial information that could serve our specific case. In the next section, we are going to enter more in details the specific branch of BCI we used in the thesis which is the Motor imagery based BCI.

Knowing that subjects who perform motor imagery with regards to a resting state change their brain patterns, brain computer interfaces consist in introducing a classification algorithm that will learn to discriminate the different patterns with a training dataset. Then new samples will be classified as belonging to one class 7 or the next. From this classification, we can create a continuous feedback. Indeed, the probability to belong to a class varies between samples therefore the feedback will vary accordingly to its probability "continuously". Or, we can create a discrete feedback with a single feedback response based on the accumulation of responses of the classification over time or a single classification response throughout a certain period of time. Although we are going to go deeper in explaining those concepts of feedback, we can already point out that BCI is now built upon a strong established literature with books covering various aspects of the field. We care to mention several references that help to give an idea of the present field such as Wolpaw's Brain-Computer Interfaces: Principles and Practice [START_REF] Wolpaw | Brain-Computer Interfaces: Principles and Practice[END_REF], Clerc's two volumes Les interfaces cerveau-ordinateur [START_REF] Clerc | Les interfaces cerveau-ordinateur 2: technologie et applications[END_REF][START_REF] Clerc | Les interfaces cerveau-ordinateur 1: fondements et méthodes. fr. Google[END_REF] (for our french readers) and finally Mueller-Putz's Neuroprosthetics and Brain-Computer Interfaces in Spinal Cord Injury: A Guide for Clinicians and End Users [START_REF]Neuroprosthetics and Brain-Computer Interfaces in Spinal Cord Injury: A Guide for Clinicians and End Users[END_REF].

In BCI, the emphasis is made on the subject's ability to be consistent in the cognitive task performed.The difference with the neurofeedback is quite subtle. Indeed, the modulation of amplitude is directly linked to the cognitive task to perform. What needs to be kept is that BCI uses machine learning algorithm to issue a feedback whereas neurofeedback directly issues an information from the brain signals. On this, some even argue that BCI is a sub-field of neurofeedback. 8: The point here is not to antagonize the approaches nor the people behind them, but more to express that BCI requires to juggle with an incredible set of skills. To create the perfect interface, there is a need for cross-disciplinary research.

A brain computer interface belongs to the category of the numerous human machine interface. From an engineering perspective, the brain/human part is mostly seen as a black box with inputs and outputs sending orders to the machine another black box itself receiving inputs and issuing outputs. From a unidirectional paradigm (open-loop system), the general effort has been for years to develop a bidirectional paradigm (close-loop system). In doing so, the outputs of the machine become also inputs of the brain.

Before going into the two main branches of development of BCI, it might be necessary to do an analogy. In known systems of human interface, we rely on different approaches towards control, such as improving the devices or the interaction or understanding the user. Let's take two examples, a mechanical one and an electronic one. First, let's talk about the plane, this system has been optimized again and again to serve its purpose but it also changed the user in doing so to create a good (and surely not the best) human machine interface. From a pure engineering point of view, engines have been optimized to be more reliable, more trustworthy and efficient, structures are lighter and stronger, flight envelopes are more mastered. However, the interaction has been overall the same since the 40's, the commands are exactly the same in a modern aircraft (Fig 2 .13), a stick and pedals, even if there are some additional components, the user was the one that adapted to the device, learning to master it even though it is not intuitive [START_REF] Grant | Smithsonian Institution. Flight: The Complete History of Aviation[END_REF]. A more recent machine to control at the antipodes of the plane is the smartphone. From an engineering point of view, processors are more energy efficient, have more computing capabilities, internal OS are more optimized in resources as well as what they can provide. But it is not the tools of computation which are solely responsible for the smartphone prosperity. Indeed, the incredible power of smartphone relies on its interaction with the user. Over the years, the emphasis has been made on the intuitiveness of the device, the user adapted to the phone of course but the phone are now designed to maximize the interaction creating an extension of oneself through the device [START_REF] Wang | Why We Cannot Resist Our Smartphones: Investigating Compulsive Use of Mobile SNS from a Stimulus-Response-Reinforcement Perspective[END_REF][START_REF] Kushlev | The Social Price of Constant Connectivity: Smartphones Impose Subtle Costs on Well-Being[END_REF].From a pure human machine interface point of view, the phone is far better than the plane.

Those two examples are not made in the sole purpose of discussing human machine interfaces. The plane demonstrates that human capability of outstanding adaptation when presenting a new device to master. We can fly using tools that are 100 years old and bend to rules of those tools established a long time ago due to numerous technical constraints. The phone demonstrates our complete integration of external device to our body when they are thought to be as ultra intuitive as possible.

Coming to the design of a brain computer interface, we have the choice to focus on the machine part by improving the computation and acquisition methods and let users adapt to the device or we can try to improve the interaction by giving more intuitive feedback and consider that the algorithms are in fact efficient enough. And, of course, we can do both, but the fields of research on those areas are far from each other and do not involve the same researchers. Improving the computation methods rely on complex engineering tools whereas improving the interaction rely on strong knowledge in ergonomics and psychology 8 .

Computer approach

In EEG BCI, the computer focus is straightforward, EEG is a temporal signal with high ratio of noise over signal, therefore, techniques are going to focus on extracting and finding relevant information differentiable from the time series while reducing the noise.

Filtering the data

Since motor imagery signature is present in the 𝛼 and 𝛽 bands, the initial filters are often bandpass filters either Finite Impulse Response (FIR) or Infinite Impulse Response (IIR). Those filters use specific windows that will distort the original signal at the extremities of the band which result in a loss of information. Other techniques are used to refine the data in a more clever way. The laplacian filter for instance, by doing the second spatial derivative of the EEG signals allows to stress on the local field potentials. Carvalhaes [START_REF] Carvalhaes | The surface Laplacian technique in EEG: Theory and methods[END_REF] presents a review on the mathematical background of the method and its numerical implementation. Mcfarland [START_REF] Dennis | The advantages of the surface Laplacian in brain-computer interface research[END_REF] demonstrated the relevance of the method to reduce noise and to constrain source localization (limiting the influence of a source over an ensemble of sensors). The Laplacian filter consists in subtracting the average activity of neighboring electrodes from the activity of a specific electrode [START_REF] Kayser | Issues and considerations for using the scalp surface Laplacian in EEG/ERP research: A tutorial review[END_REF]. Another technique, quite similar is the common average reference (CAR) which consists in subtracting the average signal across electrodes to all the electrodes [START_REF] Yao | Which Reference Should We Use for EEG and ERP practice?[END_REF] as shown in Fig 2 .14. CAR is a common method used as a standard to assess the relevance of other methods as presented by Togha [START_REF] Mahdi | Improving the performance of the motor imagery-based brain-computer interfaces using local activities estimation[END_REF] that evaluates another spatial filter : local activities estimation. Other filters exist and focus on different properties (spatial or spectral) of the signal; however, filters have the drawback of removing information that might be interesting to keep. So, a trade-off must be made. Furthermore, some filters alter the signal in a way certain techniques of computation become wrong 9

𝑎 𝑖 = 𝑎 𝑖 -𝑘 𝑗 𝑎 𝑗-𝑖+1 * (2.7) 𝜎 2 𝑗 (1 -|𝑘 𝑗 | 2 )𝜎 2 𝑗-1
(2.8)

Figure 2.15: PSD between MI and Rest:

Typical difference of power spectrum between motor imagery and resting state for a given electrode, computed with Burg auto regressive method.

Spectral estimation

As mentioned before, the known neurophysiological information of the motor imagery is in the power spectrum as observed in Fig 2 .15. It is therefore necessary to estimate the power spectrum from the EEG signal and in real time additionally 10 . The academic way to compute the power spectrum over time series is to use the Fourier transform. The Fourier transform formula requires that the time signal is the same between -∞ and +∞ to give the spectral form.

f (𝜔) = ∫ ∞ -∞ 𝑓 (𝑡)𝑒 -𝑖𝜔𝑡 𝑑𝑡 (2.1)
But, by definition, the EEG signal is not stationary, therefore it is necessary to use other techniques more suited to this problem [START_REF] Subasi | Chapter 3 -Biomedical Signal Processing Techniques[END_REF]. In addition to that and as evoked before, the signal is noisy and full of artifacts (muscular, heart rate,eye movements). On top of those considerations, the EEG signal depends on the resolution of acquisition. Based on all those difficulties, spectral estimation has to establish a trade off between temporal resolution 11 and spectral resolution 12 . Among the different techniques reviewed using the same dataset by Diez et al. [START_REF] Diez | A Comparative Study of the Performance of Different Spectral Estimation Methods for Classification of Mental Tasks[END_REF], you can find the discrete time fourier transform,

𝑋 𝑘 = 𝑁-1 𝑛=0 𝑥 𝑛 • 𝑒 -𝑗 2𝜋 𝑁 𝑘𝑛 (2.2)
a standard periodogram that is not costly in computation, which gives an exact representation of the data by decomposing it to a sum of sinusoidal however it assumes periodicity in the signal which is not true in EEG signal which creates a boundary effect distorting the signal at its edges. Another solution, used largely in the BCI field is the welch periodogram,

𝑃 𝑤 ( 𝑓 ) = 1 𝑀 𝑀-1 𝑚=0 |𝑋 𝑚 ( 𝑓 )| 2 (2.3)
which estimates the power spectrum over segments, the main effects are to increase the resolution and limit the boundary effect however by its nature the calculation is more costly and the spectral and temporal resolutions depends on the choice of the parameters of window length and overlap. Finally, other methods consist on modeling the signal as an auto-regressive model where we find the optimal parameters of the filter corresponding to the signal. This approach allows a higher resolution and is more robust to noise. It is however, more sensitive to outliers and the choice for the filter order is tedious. This is the case of Burg auto-regressive method 13 and its spectral estimation:

𝑃(𝑤) = 𝜎 2 𝑤 |1 + 𝑝 𝑖=1 𝑎 𝑖 𝑒 -𝑗𝜔𝑖 | 2
(2.9)

Finding new discriminant markers

This part is partially at the crossroads of the user center approach in the sense that we want to investigate subject dependent new bio-markers. The power spectrum desynchronization approach lacks of subtlety as it is highly localized. Indeed, we investigate changes primarily in the sensorimotor cortex. Even though this measure is robust and should appear at some point for all subjects, sometimes subjects do not manage to produce it. We will come back to this idea when we introduce more in detail the concept of illiteracy in BCI [START_REF] Allison | Brain-Computer Interfaces: Applying our Minds to Human-Computer Interaction[END_REF]. Furthermore, it does not take into account the interconnected nodes the brain is structured upon. The activity in the brain spreads out from one place to another. Therefore, the spatial information or more exactly its distributed information plays a crucial role not used in most of the BCI paradigms. Gonzalez-Astudillo's work [START_REF] Gonzalez-Astudillo | Network-based brain-computer interfaces: principles and applications[END_REF], for example, focuses on finding those distributed activities and tries to characterize them to extract what information could be differentiable from one state -motor imagery-to the next -resting state-. Among the metrics that are studied, we can mention functional connectivity that has been investigated and characterized thoroughly by Corsi [START_REF] Corsi | Functional Connectivity Ensemble Method to Enhance BCI Performance (FUCONE)[END_REF] and in an extended review by Leewis [START_REF] Leeuwis | Functional Connectivity Analysis in Motor-Imagery Brain Computer Interfaces[END_REF] recently published. Another metric also highlighted by Corsi is neuronal avalanches [START_REF] Corsi | Measuring Neuronal Avalanches to inform Brain-Computer Interfaces[END_REF], Corsi tries to focus on the displacement of the information in time from a region to the next using the raw EEG information. Those features could then be used in classification algorithms to increase the performances of the BCI.

Concerning functional connectivity, different estimators can be used to determine the connectivity matrix. Because the information related to the motor imagery state was in the spectral domain, the emphasis is made on the spectral connectivity. There are several methods of estimation for this spectral connectivity, we can mention two of them, the spectral coherence read as :

𝐶 = 𝐸[𝑆 𝑥𝑦 ] 𝐸[𝑆 𝑥𝑥 ] • 𝐸[𝑆 𝑦𝑦 ]
(2.10)

and the imaginary coherence read as :

𝐶 = Im(𝐸[𝑆 𝑥𝑦 ]) 𝐸[𝑆 𝑥𝑥 ] • 𝐸[𝑆 𝑦𝑦 ]
(2.11)

The computation also relies on a spectral estimator. This means that we need to ask ourselves the same question regarding which estimator to use. In the context of spectral connectivity, the main ones used are fourier, multitaper and welch [START_REF] André | A Tutorial Review of Functional Connectivity Analysis Methods and Their Interpretational Pitfalls[END_REF]. Based on the computation of the functional connectivity, different metrics can be computed and present relevant information that can be exploited to discriminate between cognitive states. One of the initial computation that can be done on the matrix is the node strength, a measure used in network analysis to quantify the importance or influence of individual nodes within a network. The node strength [START_REF] Barrat | The architecture of complex weighted networks[END_REF], a tool of high dimensional data modeling to measure centrality, can be used to characterize the brain network as defined by De Vico Fallani [START_REF] De | Graph analysis of functional brain networks: practical issues in translational neuroscience[END_REF] (in a weighted network). A specific node is calculated by summing the weights of the connections (edges) that are linked to that node. In the context of motor imagery with regards to resting state, the node strength of certain nodes localized in the sensorimotor cortex tends to increase. It means that regions of the brain get more synchronized during a motor imagery task than during a resting state task.

Those new metrics put the emphasis on the connection between regions of the brain rather than focusing on a local source of information. In [START_REF] Subasi | Chapter 4 -Feature Extraction and Dimension Reduction[END_REF] doing so, they provide new discriminant features that can be mixed with standard ones to make machine learning algorithm more robust and to exploit different information present in the brain. This could benefit subjects that might encounter difficulties to produce the brain patterns of spectral desynchronization.

Techniques of classification

In BCI, the key is to have different cognitive states creating data that are differentiable statistically. This statistical difference between signals allows the use of machine learning algorithm, most of the time supervised, capable of establishing a separation between brain patterns and inferring the associated class for new samples. What comes as an input of the classifier (which is based on a ML algorithm) is called a feature and can be either explainable (related directly to neurophysiological signatures) or inexplicable (a modification of the signal to enlarge its differentiable nature) [START_REF] Subasi | Chapter 4 -Feature Extraction and Dimension Reduction[END_REF]. A classifier is the association of the feature and the ML algorithm. In the most regular configuration, to build a classifier, we first need a training set of data, where the algorithm learns to differentiate between classes 14 and then, a testing set where the algorithm assigns a class to new signals. If the algorithm predicts correctly, the accuracy 15 will be high. On this area, Lotte's review [START_REF] Lotte | A review of classification algorithms for EEG-based brain-computer interfaces: a 10 year update[END_REF] needs to be mentioned as it covers a great deal of machine learning methods highlighting their strength and weaknesses in their application to BCI.

Power spectrum based Classifier

Since the entry point for motor imagery was originally with differences of power spectrum in the 𝛼 and 𝛽 bands, the most standard approach is to rely on those features as an entry point of the classification. The input of the classifier is going to be the power spectrum of specific electrodes 16 at specific frequency bins or averaged across frequency bins. The resulting feature vector can be used by different linear and non linear algorithms such as LDA or SVM (to cite the most used one). Subasi's chapter [START_REF] Subasi | Chapter 5 -Biomedical Signal Classification Methods[END_REF] covers in depth the use of LDA and its implementation as a basis for EEG classification.

Covariance based Classifier

The other major approach to the EEG signal is to use the covariance matrix (positive semi-definite 17 ), that puts the focus on the joint variability of the electrodes between themselves. Instead of having a vector for each class, corresponding to the different cognitive states -MI or rest for instance, it is now a matrix for each class and the information it contains should differ from a class to the next. Then, several solutions are possible. The first known one is the common spatial pattern (CSP) [START_REF] Fu | Improvement motor imagery EEG classification based on sparse common spatial pattern and regularized discriminant analysis[END_REF] filter considered as a gold standard in BCI pipeline as covered by Lotte's work [START_REF] Lotte | Regularizing common spatial patterns to improve BCI designs: unified theory and new algorithms[END_REF]. It needs to be mentioned that this work warns on the possible over-fitting of the method with few data and its sensitivity to noise. Technically, it maximizes the variance between conditions by computing the eigen vectors of the classes' covariance matrices 18 . Then the filter is applied [87]: to the EEG data to form the CSP features (the weights that give more importance to certain electrodes and discard the others) which are then considered as input of a regular machine learning algorithm such as LDA [START_REF] Wu | Common spatial pattern and linear discriminant analysis for motor imagery classification[END_REF] or SVM.

The new state of the art approach consists in considering the covariance matrix as a riemannian manifold. A type of smooth manifold in differential geometry, equipped with a Riemannian metric, a smoothly varying inner product on the tangent space at each point. The Riemannian metric is a positive-definite inner product that allows for various geometric notions such as angles, lengths of curves, areas and curvature. The Fig 2 .16 is shown to help explain how the covariance can be represented as a Riemannian manifold and how to project the tangent space on a local point.

By doing so, the covariance matrix becomes an enriched object which can be manipulated with new method. The covariance matrices can be directly exploited using riemannian distance as presented by Yger's review of the different Riemannian method usable in BCI [START_REF] Yger | Riemannian Approaches in Brain-Computer Interfaces: A Review[END_REF] 19 . In the calibration phase, the covariance matrices are averaged for each class. In the testing part, the distance between the averaged matrix and the new samples is computed, the shorter the distance, the higher the probability to belong to a class. The other solution is to project the covariance matrices into a tangent space (a vector space that approximates the manifold at a specific point). The vectors for each class can then be considered as input for more standard algorithm such as LDA, SVM or Logistic Regression. This was introduced by Barachant in his work and validated as one of the most efficient method [START_REF] Barachant | Riemannian Geometry Applied to BCI Classification[END_REF]. The approaches based on Riemannian geometry are the one presenting the highest performances to this day. However, they suffer from a lack of neurophysiological interpretability 20 . Indeed, the covariance matrix object in itself is complex to interpret and to link to the knowledge we have on the ERD/ERS during the cognitive task. On this note, the projection on a tangent space gives at least the spatial information of what is discriminant -we would want the sensori motor cortex to appear. It remains that the information must be present in some form in those covariance matrices which means that either there is a way to come back to the spectral information through the covariance or that some Riemannian markers should be defined to characterize the difference between motor imagery and resting state.

21: Explanability in machine learning is the possibility to explain the steps of manipulation on the data from input to output. This is crucial to know exactly what will be used to discriminate between classes. This is to limit the black box effect that would lead to an absence of knowledge on why performances are good (or bad).

22: It is important to note that the parameters of the artificial neural network (number of layers, number of neurons per layer, etc) are important to avoid overfitting.

[90]:

Deep learning approach

Deep learning differs in the approach of other methods to classify the data. Deep learning relies on artificial neural network, layers of nodes that transform successively the data from a representation to the next. Based on the transformation, they issue a class based on the output representation.

The particularity is that the output representation cannot be interpreted directly, making those neural network algorithms black boxes that could be criticized for their lack of explanability 21 . However this approach allows to extract more information from the data, making it most of the time the best suited solution to separate classes. A representation of this is shown in Fig 2.17 below. Nevertheless, deep learning approach relies on a massive amount of data in order to find the discriminative information between classes 22 . This is a tedious matter in BCI as the EEG data classes do not have a high number of samples. Despite those difficulties, some algorithms seem promising to be used in the matter especially Convolutional Neural Networks (CNN). Zhang's specific review on the topic goes in more details than Lotte's review especially concerning BCI using ERD/ERS patterns (motor imagery paradigm) [START_REF] Zhang | A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers[END_REF]. This technique was originally used in image recognition for its ability to extract relevant information from images and its application in EEG showed encouraging results. Even though CNN is frequently used as mentioned by Hossain's up to date review [START_REF] Murad | Status of deep learning for EEG-based brain-computer interface applications[END_REF], other methods have risen such as Long Short-Term Memory (LSTM), Reccurent Neural Network (RNN) and Autoencoders (AE) and Variational AE (VAE). Their flexibility presents the advantage that they can be combined together, certain properties of the CNNs on the creation of new representations leading to new features can be implemented into other deep learning methods that are even allowing a thinner separation of the classes. In addition to that, deep learning presents the advantage of opening the track of transfer learning from one subject to the next but also from one session of a subject to the next one.

Current challenges in computer approach

The challenges risen by the BCI technology are tremendous, especially in EEG. The relevant information is present in a noisy signal, unstable and varying from one subject to the next. Methods to filter the data need to be subtle to not lose too much information. Machine learning algorithms have to be robust, but adaptive to changes in accordance to the mental strategies of subjects. Those algorithms need to train on few data with few periods of calibration or even none. On this note, using transfer learning 23 techniques is more than relevant. For instance, training on one subject to predict on the results of a second subject. Wang et al [START_REF] Wang | A review on transfer learning for brain-computer interface classification[END_REF] in their review established a clear definition of transfer learning and the different focus types (transferring feature representation, instance/data or classifier). Wu [START_REF] Wu | Transfer learning for motor imagery based brain-computer interfaces: A tutorial[END_REF] detailed a more applicative paper focusing on transfer learning specifically for MI BCI. In the case of performance, some machine learning are able to give good accuracy but those good performances need to be explained by neuro physiological signature and that is not always the case.

What's at stake here is the trust we put in the system. In one hand, if cognitive tasks are rightly done by subjects but they are not well classified by algorithms, subjects loose the trust they put in the device because it does not respond well to what they think they do. In the other hand, too powerful algorithm that are capable of differentiating states but not on the right information (if subjects do not perform the task for instance) will provide maybe a good control but will not be trustworthy as we do not know why the algorithms gave a certain choice.

In a talk presented at the BCI Society in 2023, Pr.J Wolpaw simply described it as the "cliff challenge", would you put yourself in an exoskeleton controlled by a BCI to step back from the cliff? Even today, with all the methods that are used, not a soul would dare.

Subject approach

Since we just covered the C in BCI, let's now focus on the H. When dealing with a human in the loop, everything that you show and do not show have a significant impact on the interaction and more critically to brain patterns. Unfortunately, if we consider humans as black boxes with input and output, we will rapidly miss how a good interaction works. The issue with centering on humans rather than machines in BCI is that it creates an insane number of parameters to take into account. Jeunet describes in the experimental part of [START_REF] Jeunet | Mind the Traps! Design Guidelines for Rigorous BCI Experiments[END_REF] the number of biases coming from working with subjects and the necessity to clearly define the hypotheses to avoid being overwhelmed by the different variables. Among the questions needed to be addressed, we can find the question of maintaining the attention and the involvement of subjects without disturbing them, the question of creating simple cognitive task (easier to detect via EEG) that are challenging for subjects, the question of the instruction given. This approach could be seen as empiric in comparison to the computer science techniques introduced previously but they actually carry maybe more meaning to subjects and therefore they are essential to study.

An experience tailored to the subject

In this effort, some studies aim to focus on the subjects' profile, knowing more their personality and their cognitive traits may allow to create stimuli that are more significant to their eyes. This is the subject of Jeunet's work [START_REF] Jeunet | Predicting Mental Imagery-Based BCI Performance from Personality, Cognitive Profile and Neurophysiological Patterns[END_REF] on measuring traits of subjects that could impact performance (forms of IQ, memory ability, personality traits, anxiety). Jeunet identified, for instance, that tension and self-reliance showed high correlation with performance. Tailoring subjects' experience relies on a strong psychology background which allows to target precisely behaviours to maximize at the end performances by eliciting neurophysiological responses. The strength of this approach is that it maximizes the probability for subjects to perform complex task such as motor imagery because it makes the system adapted to their internal representations. The other side of the coin is that tailoring the experience to subjects takes time, needs to be done thoroughly and prohibits from doing large and rapid application to a high number of participants.

Instruction and task to perform

To perform motor imagery or resting state, it is necessary to talk about it and describe it to subjects naive to BCI. Let's take the example of the motor imagery of the right hand, in simple words, that would give "imagine that you are closing your right hand in your head, the feedback you see corresponds to the intensity of your mental task towards this action". This sentence that could appear simple is flawed. What do imagine or intensity of a mental task mean ? To add to this complexity, the language of modern science is english, the words we use are conditioned by this language. But, this work took place in France, subjects are to be spoken in their native language, French. French and English are different regarding the way we carry meaning in sentences. And translation of instruction are not easy at all, every word has the power to bias the experimentation. Even the words to talk about cognitive tasks are different, in english "motor imagery" becomes "imagination motrice", it is subtle but imagery could be translated by "imagerie" which means "a group of images sharing the same characteristics" so the key information here is that it is an image where as the actual translation "imagination" means "the faculty to make a representation or form images", and the key information becomes the representation. This subtle difference highlights how complex it is to characterize this cognitive task. In opposition to that, the alternative task asked to subjects is the resting state, simply formulated as "do not think about anything". Trying to say more is already too much, if we say that it is meditation, it allows subjects to let their mind wonder and by doing so it might activate their memory or more complex process 24 . So the resting state condition might be even harder to explain than motor imagery.

In that effort, Roc [START_REF] Roc | A review of user training methods in brain computer interfaces based on mental tasks[END_REF] characterized the different types of instruction present in the literature and paved the way for better instructions to give to subjects. Roc also established general guidelines to design BCI protocols that take into account the biases created by the instructions.

Lights out ! -Stimuli

Instructions take another form during the experimentation itself. Indeed, the main MI BCI paradigm relies on doing alternatively tasks of motor imagery and resting state therefore a key step in designing a BCI experimentation is the stimuli 25 given to subjects. Different stimuli correspond to different tasks to perform, they must carry inherent meaning because 27: Those types of stimuli are slightly outdated. From the literature putting the subject at the center, these are not the most suitable stimuli to present. This is a good case of conceptual stimuli that are far fetched, the association between a rectangle placed up on a screen and the motor imagery of the right hand is not straight forward. they are what starts the cognitive process. If stimuli are too far fetched, the inner process that associates the trigger to the task will take more time 26 . However, if the triggers are too framing, the lack of evocative freedom given by stimuli could bother subjects in their inner process. Maybe it is necessary here to formulate differently by taking an example. Let's say that we want subjects to do motor imagery and we present a stimulus to indicate that they have to imagine closing their hands. We might want to show a hand that is closed, and, in opposition, the resting state will be an opened hand. The content is explicit but during the resting state we still show an hand. So it might remain confusing for subjects because they need to focus on nothing but the hand is shown.

On this matter, there is an extensive literature presenting the different types of stimuli [START_REF] Singh | A Comprehensive Review on Critical Issues and Possible Solutions of Motor Imagery Based Electroencephalography Brain-Computer Interface[END_REF]. Historically, two different experimentation were used, first in the BCI 2000 framework [START_REF] Wilson | Using an EEG-Based Brain-Computer Interface for Virtual Cursor Movement with BCI2000[END_REF], a pong like display where a ball has to reach either the upper part of the screen or the lower part of the field depending on if it is motor imagery of closing the right hand (up) or resting state (down) 27 , this is represented in Mattia's work [START_REF] Mattia | The Promotoer, a brain-computer interface-assisted intervention to promote upper limb functional motor recovery after stroke: a study protocol for a randomized controlled trial to test early and long-term efficacy and to identify determinants of response[END_REF] on the promotoer where a hand is presented to initiate the MI task is a good example in the context of stroke rehabilitation that takes into account those notions to keep the subject interested and ensure a level of concentration without parasites sources.

Maintaining subject's attention -Feedback interrogation

One of the complexity related to BCI experimentation is the time it takes. First, from a practical point of view, the installation of the EEG cap takes time. Second, phases of training to calibrate the classifier are long because you need enough samples to train on. And third, you need to control the system for a certain time to really assess its efficiency. This ends up being boring and tiring for subjects. Therefore, the feedback, the essential information linked to the brain activity has to entertain the subjects, it gives meaning to the experience but also ensure stronger interactions between subject and machine. In that effort, a huge amount of feedback exists, of different nature, shown at different intervals but they need to follow a certain framework. On this subject, Kosmyna and Lecuyer [START_REF] Kosmyna | Designing Guiding Systems for Brain-Computer Interfaces[END_REF] did a brief review of the different feedback available in BCI and established the different criterion regarding their conception (timing, triggering types of response).

As mentioned before in the classification part, there are two main types of feedback [START_REF] Thomas | Effect of Continuous and Discrete Feedback on Agency and Frustration in a Brain-Computer Interface Virtual Reality Interaction[END_REF]. First, there is the continuous feedback, where an information is shown during all the period of the cognitive task. Taking back the example of the Graz visualization, the feedback is a blue bar oriented towards the left or right depending on the cognitive task (MI of the left or right hand) changing its length with the distance to the hyperplane 28 . Meanwhile, this feedback provides the intensity meaning of the cognitive task, its representation is extremely far from the motor imagery task (or the resting state) which makes it difficult to associate and might end up disengaging subjects. Those feedback present the advantage of being by definition continuous meaning that subjects have an interaction that lasts for a period of time, they ensure the system is working properly. More over, they allow to keep the subject hooked to the task (in trying to get the best feedback possible related to their performance). However, they need to be recomputed often which means that the classification must be light in calculation. Furthermore, the feedback can become erratic if wrongly calibrated. This can result in disturbing subjects and in the end even in disengaging them from the interaction.

The second possibility is to use discrete feedback presented at the end of a trial. These types of feedback have the strength of being more robust to variation of mental strategy during the trial. As they have a longer window to classify, they introduce a reward mechanism where subjects perform to obtain a certain outcome. In addition to that, stronger algorithm can be used because there is less need for real-time result. But, the main drawback is, as one can expect, its discrete nature. Indeed, during a certain amount of time, subjects do not receive any information regarding their inner activity resulting in not knowing if the system is working and if they are doing the task correctly which is deplorable.

The overall conclusion of studies on the choice of feedback do not point towards an advantage of one of the method or the other. However the nature of the feedback is extremely important. Visual feedback are the most used as presented by Alimardani's review advocating for their use [START_REF] Alimardani | The Importance of Visual Feedback Design in BCIs; from Embodiment to Motor Imagery Learning[END_REF]. But other leads are investigated such as auditory feedback with different approaches to it, such as modulating frequency [START_REF] Gaetano | Investigating the role of combined acoustic-visual feedback in onedimensional synchronous brain computer interfaces, a preliminary study[END_REF], modulating volume [START_REF] Mccreadie | Is Sensorimotor BCI Performance Influenced Differently by Mono, Stereo, or 3-D Auditory Feedback?[END_REF][START_REF] Mccreadie | Sensorimotor learning with stereo auditory feedback for a brain-computer interface[END_REF] or in combination with other modalities as an additional reward [START_REF] Li | Sensorimotor Rhythm-Brain Computer Interface With Audio-Cue, Motor Observation and Multisensory Feedback for Upper-Limb Stroke Rehabilitation: A Controlled Study[END_REF]. Another promising perspective is to use the vibro-tactile feedback. On this, Fleury [START_REF] Fleury | A Survey on the Use of Haptic Feedback for Brain-Computer Interfaces and Neurofeedback[END_REF] presented a review of the different uses of haptic feedback and Kauhanen [START_REF] Kauhanen | Haptic Feedback Compared with Visual Feedback for BCI[END_REF] compared it to visual feedback in terms of performance to validate the method. This feedback allows a certain integration in the body of the user which reinforces the strength of the interaction. In this effort of integration, others tend to focus on electrical stimulation, the feedback being a direct control over muscles and therefore the body of the subject. This lead is difficult to put in place because of the intrusion to the subject's body, however it demonstrated encouraging results especially in the domain of rehabilitation. An exploratory study conducted by Sinha [START_REF] Sinha | Brain-Computer Interface Training With Functional Electrical Stimulation: Facilitating Changes in Interhemispheric Functional Connectivity and Motor Outcomes Post-stroke[END_REF] showed encouraging results on rehabilitation of chronic stroke patients with the use of FES, but the few number of subjects limits the results to be generalized.

A feedback that presents the advantage of stimulating the subject and that puts the emphasis on the gesture associated to the cognitive task is the robotic arm where movements or actions are controlled by the BCI. Many approaches integrate a robotic arm, either by making the robot accompany the user in doing the movement (this can be considered as an exoskeleton) or as an external agent, a third arm to control. Those types of feedback create a high level of engagement and an immediate share of space which is necessary to create ecological environments. Although we will enter more deeply on the use of robot controlled by BCI, we can mention [START_REF] Aljalal | Comprehensive review on brain-controlled mobile robots and robotic arms based on electroencephalography signals[END_REF] review on the field presenting the advantages of the approach.

Embodiment and intentional binding

Before going further, it is important to take a moment to talk about certain concepts that are essential to understand why the use of a robotic arm can be beneficial to the BCI field. To do so, we need to introduce the concept of embodiment. Embodiment is a general concept difficult to define. Here, we base ourselves on studies mostly related to virtual reality as defined by Kilteni's work on the topic [START_REF] Kilteni | The Sense of Embodiment in Virtual Reality[END_REF], they approach the term from an "avatar" [START_REF] Gonzalez-Franco | Avatar Embodiment. Towards a Standardized Questionnaire[END_REF] 29 perspective, other would be more focused on the interaction of the body with the environment to define the concept. Embodiment means stricto sensu the act to put into (em-) the body. This simple definition allows to understand that we are going to place ourselves "in the shoes" of something. In other words, embodiment gathers the different notions that make our interaction with our own body. Our own body being the ultimate embodiment for ourselves.The sense of embodiment relies on three different senses (we keep here the definition established by Kilteni).

▶ Sense of body ownership. Body ownership is the deep feeling that our body is our own, and we are in control of it. It connects our physical form to our identity and is influenced by sensory inputs like awareness of body position and movement, tactile sensations, and visual feedback. This sense of ownership shapes our self-perception and strengthens our overall embodiment. ▶ Sense of agency. It refers to the perception of being in control and actively causing our actions. It involves the understanding that we are the agents behind our body's movements and the initiators of events in the world. This perception is closely tied to our awareness of the outcomes of our actions and the feedback we receive from the environment. ▶ Sense of self-location. Sense of self-location, or spatial presence, relates to the sensation of occupying a distinct position in space. It is crucial for our embodiment and fundamental to our interaction with the surroundings. This perception is closely connected to body ownership since the perceived location of our body plays a vital role in our sense of presence within the environment.

Embodiment can be observed in many situations and does not only concern avatars that look like humans as demonstrated in Aymerich-Franch's work [START_REF] Aymerich-Franch | Non-human Looking Robot Arms Induce Illusion of Embodiment[END_REF], a known example is the avatar in a video game (in VR or not). The gamers share a common space, have control and to some extent feel a certain owning of the avatar's body. These mechanisms are extremely important in a game to keep players hooked in. And of course, they (the mechanisms of embodiment) present a perfect utility in the BCI field. On this, the work of Alimardani was groundbreaking on using a complete android controlled by BCI in a first personal view (FPV) [START_REF] Alimardani | BCI-teleoperated androids; a study of embodiment and its effect on motor imagery learning[END_REF]. Maintaining a high level of engagement through the experimentation and feeling in control of an external body are keys to elicit strong motor imagery patterns. On this note, the sense of agency (SoA) is linked to the reduction of the error. Being agent tends to reduce the frustration, a frustration which is partially responsible for the decrease of the BCI performances. The end result of the association of agency and frustration reduction is at the heart of building a strong BCI protocol.

Linked to the sense of agency (with some debates in which we do not enter), there is a strong effect called intentional binding which will be one of the initial thinking brick of the thesis study [START_REF] Moore | Intentional binding and the sense of agency: a review[END_REF]. Intentional binding refers to a perceived shrinking of time between a voluntary action and its consequence. This mechanism is a signature of being in control over a system. What is to keep from the literature on the field is that some inner phenomenons are at stakes in creating a strong human machine interaction and those phenomenons can be useful to the BCI field if exploited wisely. In that context, embodiment showed to be useful in eliciting and reinforcing motor imagery patterns which is beneficial to BCI. Therefore, some studies integrated robot or virtual avatars on which subjects can identify to trigger the brain in doing better ERD/ERS responses [START_REF] Škola | Progressive Training for Motor Imagery Brain-Computer Interfaces Using Gamification and Virtual Reality Embodiment[END_REF][START_REF] Petit | An integrated framework for humanoid embodiment with a BCI[END_REF][START_REF] Martens | Towards robotic re-embodiment using a Brain-and-Body-Computer Interface[END_REF]. Moreover the use of robots that present similarities with human arm can become integrated to the framework of the user thanks to an embodiment process.

Current challenges and limitations

BCIs have gain popularity over the years. However, to echo what has been said on the challenges in computer science, the reflection on the human interface was largely ignored for a long period as efforts were prioritized on signal acquisition and treatment. But, since machine learning techniques are more and more reliable, placing the human at the heart of the study has gain popularity and it opens large fields of researches as described by Sahan et al. on a review of BCI's progresses [START_REF] Saha | Progress in Brain Computer Interface: Challenges and Opportunities[END_REF]. On those new fields, the task itself is problematic as it remains complex to perform by subjects. Therefore, new descriptions are needed as well as more suited tasks and more generally more ecological as defined by Vincente in the early 90s experimentation 30 . [START_REF] Vicente | Ecological interface design: theoretical foundations[END_REF] Another problem which has to be considered is the BCI illiteracy. A high number of subjects (between 15 to 30%) are not able to use BCIs because of unknown reasons. The concept of illiteracy was largely investigated by Allison and Neuper in their work considered a milestone in its characterization [START_REF] Allison | Brain-Computer Interfaces: Applying our Minds to Human-Computer Interaction[END_REF]. This is absolutely impossible to conceive if we want the system to be controllable by anyone. Understanding the human subtlety leading to this inability of operating BCI devices is a central question.

In this reasoning of creating BCIs for all, the highest challenge will be to make a transition from pure research framework with a controlled environment with few electric noises and research equipment to the "real world" with all its additional constraints. This is today still not on the table at least with non invasive techniques even though certain promising works are today conducted with implanted electrodes [START_REF] Louis | An exoskeleton controlled by an epidural wireless brain-machine interface in a tetraplegic patient: a proof-of-concept demonstration[END_REF].

[123]: [START_REF] Taylor | Robotic Control[END_REF], Robotic Control 31: 3 for the shoulder, 1 for the elbow and 3 for the wrist 32:

Δq = J -1 Δx (2.15)
where q represents the angles and x the position.

33: It represents the partial derivatives of each component of the function with respect to each input variable, providing information about the rate of change and sensitivity of the function with respect to its inputs.

J =            𝜕 𝑓 1 𝜕𝑥 1 𝜕 𝑓 1 𝜕𝑥 2 . . . 𝜕 𝑓 1 𝜕𝑥𝑛 𝜕 𝑓 2 𝜕𝑥 1 𝜕 𝑓 2 𝜕𝑥 2 . . . 𝜕 𝑓 2 𝜕𝑥𝑛 . . . . . . . . . . . . 𝜕 𝑓𝑚 𝜕𝑥 1 𝜕 𝑓𝑚 𝜕𝑥 2 . . . 𝜕 𝑓𝑚 𝜕𝑥𝑛            (2.16)
x being the position of the end effector.

The robotic arm integration to BCI offers an answer to a variety of the challenges presented, by presenting a congruent gesture feedback associated to the cognitive task and integrating itself into a real environment. It also contributes to elicit stronger brain patterns due to embodiment. However, its integration bring other challenges especially regarding its control.

How to control a robotic arm

Following what was exposed in the embodiment section, it is necessary to introduce some basic robotic concepts to understand why using a BCI for a robotic arm is not completely straightforward.

Concerning robotics

Robotics is on its own a whole field of research, here the emphasis is made on robotic control and gesture. The control of an arm depends mainly on its nature, the more degrees of freedom the more complex it is to control. The concept is simple : to reach a desired position, an arm needs to go from its original configuration to a new one in order to have its end effector positioned to the desired 3D point. The configuration of a robot is characterized by its joints forming the kinematic chain as shown in Fig 2 .20 from its basis to the end effector. Here, Robotic Control from Taylor [START_REF] Taylor | Robotic Control[END_REF] is a solid reference to understand the initial concepts even though more recent ones are not present but the proceedings of the International Symposium on Advances in Robot Kinematics gives a good overview of the modern trends regarding the field [START_REF]Advances in Robot Kinematics[END_REF].

Each joint has a certain number of degrees of freedom (DoF). In a determined system, since we need a 3d position to go to, we would need theoretically only 3 degrees of freedom in all the kinematic chain. The robotic arm are in these case simple and systems of control are easy to develop. However, those systems lack flexibility and are far from what an actual human arm is. A real human arm presents some redundancy as it is composed of at least seven degrees of freedom from shoulder to hand 31 and at least one for the hand itself as represented in Fig 2 .21.

We use the term redundancy because there is an infinite number of solution to reach a desired position with that many degrees of freedom hence some redundancy between degrees. But the infinite number of solution has the advantage of being adaptable to an infinite number of situations. That is why, robotic arms tend to possess those number of degrees of freedom.

Mimicking human movement

Even though, some robotic arms possess the same number of joints as a human arm, they do not have the same behaviour. Indeed, inverse kinematics 32 methods which compute the angles based on the inversion of the Jacobian matrix 33 provide a solution for the end effector to reach the desired position. But the movement, hence the different joints displacements, needs to be in accordance with human constraints. Free-hand movement in reaching task relies on the concept of minimum effort. To achieve minimum effort, the main model in place is the minimum jerk trajectory. On this, the main two hypotheses are that movements should relatively fast and discrete. Human movement are described in Sha's paper [START_REF] Sha | Minimum Jerk Reaching Movements of Human Arm with Mechanical Constraints at Endpoint[END_REF] and it gives a good idea of what's necessary for a robotic arm to mimic human arm movement. Jerk is the derivative of acceleration. To mimic human reaching trajectories, robotic trajectory is interpolated by the minimum jerk method 34 . In doing so, it allows to be more precise and to reduce the vibrations. This produces an effect of abrupt increase of acceleration at the start of the trajectory until a maximum at mid-distance followed by a decrease of acceleration until the position is reached.

Here, even if we are not talking about an exoskeleton but fully externalized robots, it is necessary to stress the necessity to obtain those type of movements. Indeed, one of the core idea is to use embodiment on one hand and motor imagery in the over hand. We push the idea that it is needed to have a robot movement as similar as the one of a human to ensure its acceptability [START_REF] Ventre-Dominey | Embodiment into a robot increases its acceptability[END_REF] and to limit the dissonance between the motor imagery task and the real movement as it has been demonstrated that robot gesture changes the perception we have of them [START_REF] Lastrico | Robots with Different Embodiments Can Express and Influence Carefulness in Object Manipulation[END_REF]. However, this idea is not pushed to trying to mimic the specific user's arm movement.

Sharing control

Before evoking robotic integration to BCI which is a sub branch of robot human interaction work flow, it is necessary to introduce elements of human robot shared control. In a shared control framework, the robot is meant to relieve the user from a certain load to maximize the performance in doing a task. In that context, the robot flexibility, strength and precision team up with human adaptability and creativity [START_REF] Musić | Control sharing in human-robot team interaction[END_REF]. The core idea in a shared control is to have a context and a task to realize both known to the robot and the user, here the robot takes into account information provided by a user through posture [START_REF] Zhu | Human-robot shared control for humanoid manipulator trajectory planning[END_REF] or haptics [START_REF] Musić | Haptic Shared Control for Human-Robot Collaboration: A Game-Theoretical Approach[END_REF] and the user adapts the command based on the robot response. This teaming framework finds its immediate use in industry but this can also benefit patients especially in stroke rehabilitation [START_REF] Ghannadi | Configuration-Dependent Optimal Impedance Control of an Upper Extremity Stroke Rehabilitation Manipulandum[END_REF]. The approach is slightly different in the BCI context as the robot is principally used as an actuator (doing the action) but not in a spirit of shared control. However it is the robot itself that is going to "close the loop" by its behaviour.

The usage of a robot for BCI

As mentioned before, embodiment can be useful to improve BCI performance. But, it is necessary to show more in details how robotic arms are integrated to the BCI framework. BCI relies on commands issued from classification algorithm, the tasks can be at best "motor imagery of the left or right hand closing", "left or right foot stretching" and "tongue pulling". In the hypothesis that we are able to detect those different tasks with EEG, there is still not enough classes to control all the joints. In addition to that, controlling an arm with a combination of those tasks is absolutely counter-intuitive, when our arm reaches an object, we don't think about pulling our tongue. Human movement planning relies on synergy principles defined by Bernstein (1896-1966) in his work both from a neuro-physiological and behavioral perspective [START_REF] Latash | On Dexterity and Its Development[END_REF] 35 . Most of the time, BCIs solely focus on controlling the gripper closing as presented by Zheng's review [START_REF] Zhang | A survey on robots controlled by motor imagery brain-computer interfaces[END_REF] on BCI with robotic control, or act as a trigger to reach or not a target. On this note, it has already been demonstrated that ERD/ERs could be induced by robotic movement [START_REF] Pablo | An ERD/ERS analysis of the relation between human arm and robot manipulator movements[END_REF]. Recent studies undertook the challenge to decode the complete arm movement directly from the EEG signal [START_REF] Korik | Decoding Imagined 3D Arm Movement Trajectories From EEG to Control Two Virtual Arms-A Pilot Study[END_REF] and to reproduce this movement with a robotic arm. This work has been done with invasive EEG 36 and Ecog [START_REF] Louis | An exoskeleton controlled by an epidural wireless brain-machine interface in a tetraplegic patient: a proof-of-concept demonstration[END_REF] 37 first but also using movement related potential in non invasive BCIs [START_REF] Gernot | Feel Your Reach: An EEG-Based Framework to Continuously Detect Goal-Directed Movements and Error Processing to Gate Kinesthetic Feedback Informed Artificial Arm Control[END_REF]. However motor imagery remains complex to decode for complex movement at the EEG level [START_REF] Müller-Putz | From classic motor imagery to complex movement intention decoding: The noninvasive Graz-BCI approach[END_REF]. As evoked earlier on the feedback, an intrinsic complexity of the BCI lies on the delay between the mental action and its response. This is still the case for a robotic arm control, even in a continuous feedback configuration [START_REF] Ruben | Continuous versus discrete robotic feedback for brain-computer interfaces aimed for neurorehabilitation[END_REF], we will observe a certain delay between the user's brain response and the issued control. Those delays are still poorly studied in the BCI field even though it was demonstrated that the delay (if limited) could be integrated to subjects' framework [START_REF] Keller | Towards Understanding Adaptation Latency in Self-adaptive Systems[END_REF] who adapt to a certain extent after what, they loose their agency [START_REF] Moore | What Is the Sense of Agency and Why Does it Matter?[END_REF].

A poor player upon the stage ?

It is, here, necessary to interrogate the dual role of the robot 38 especially in the context of brain machine interfaces. On one hand, because we want to obtain a certain level of embodiment through sense of agency and ownership, the arm is considered as being integrated in the workspace of the user. Subjects control the arm therefore the arm belong to them. The more we have this result, the better the performances. In this paradigm, robot could be assimilated as a third arm or a prosthetic.

On the other hand, the robotic arm is a complete entity in itself which is giving the lines to the subject in the form of dialogue between two performers on stage. And this changes the framework of the interaction because by no longer considering the robotic arm as an extension of one self (just like a tool) but as an external agent (another actor) capable of The control is sometimes 2d [START_REF] Úbeda | Brain-machine interface based on EEG mapping to control an assistive robotic arm[END_REF] or starting to integrate true human movement information [START_REF] Schwarz | Decoding hand movements from human EEG to control a robotic arm in a simulation environment[END_REF] "some level of intelligence", the interaction is built on a collaboration as defined by Terveen [START_REF] Terveen | Overview of human-computer collaboration[END_REF]. Those collaborations have been deeply studied in other field. Fong's chapter [START_REF] Fong | Collaboration, Dialogue, Human-Robot Interaction[END_REF] on collaboration established the key principles of robotic collaboration and emphasised the possible effects of empathy towards a robot. On this, the concept of empathy towards a robot is largely described by Malinowska [START_REF] Malinowska | What Does It Mean to Empathise with a Robot?[END_REF] which can be used to strength the link between the two agents. 39 In BCI, was introduced by Pillette [START_REF] Pillette | A physical learning companion for Mental-Imagery BCI User Training[END_REF] a robot as a comforting agent (a robot displaying simple emotions to indicate success or failure). A key idea is to interrogate how the robot is perceived by the end users in order to maximize the reward and the engagement of the user. But robotic arms because they are just arms are principally used in a "prosthetic paradigm" in the BCI field, which is maybe lacking of subtlety regarding what is their role in the collaboration with users.

In that sense, a robot is not a poor player upon stage * because its interaction with the subject is based on an explicit code of dialogue. Subjects expect a certain behaviour from the robot based on their brain commands and the robot by its response orient subjects towards a certain direction, in doing so changing their behaviour. Considering the robot as an external agent allows some liberty regarding its failure that we would not have if it is considered as a prosthetic which should answer au doigt et à l'oeil. Honig [START_REF] Honig | Understanding and Resolving Failures in Human-Robot Interaction: Literature Review and Model Development[END_REF] presented an extensive state of the art regarding the acceptation of the failure in the system. It evoked notably a higher level of engagement when the robot is failing as well as a certain level of empathy regarding the robot when it fails. In our quest to use robots at the end of the feedback chain, it might come handy to rely on these notions. Robotic failures (and this is the case for all machines) can result in a decrease of trust [START_REF] Klüber | When Robots Fail-A VR Investigation on Caregivers' Tolerance towards Communication and Processing Failures[END_REF]. However, knowing that robot's mistake can be accepted to some extent by the end users if it does not disturb them, BCI could take advantage on it. BCI could keep the robot as an external agent that could elicit some embodiment effect meanwhile staying external.

We have evoked so far aspects directly or indirectly linked to BCI. It is necessary here to take a step back. We are in a pickle since we cannot control a robot completely in an intuitive manner with non invasive BCI 40 .

In that context, it might be useful to introduce a new way to control the arm.

EyeTracking Technology

On the matter, the eye tracking technology suits the requirements to access the intention of subjects. The gaze allows to indicate the subjects' interest position. This "interest position" can be converted into a "reaching position" allowing to create a control of the robot based on the 3d position to reach.

Technicalities

Eye properties

The elements constitutive of the eye that interest us are the pupil, the iris and the sclera [START_REF] Purves | Anatomy of the Eye[END_REF]. The pupil is responsible of the aperture for the light to enter. The iris which surrounds the pupil is the diaphragm that controls the amount of light. And sclera, the white part, offers a certain protection to the inside of the eye. Eye direction is dealt by 6 different muscles surrounding the eye and allowing to rotate on two axis. They also serve in the stabilization of vision when the head is in movement. This is represented in detail in Fig 2 .22.

To track relevant information in the visual field, eye movements play obviously a crucial role. There exist different types of movement either involuntary or voluntary to track those information. First, there is the direct control over the eyes which is responsible for smoothly following movements. Second, it represents most of the movements, eyes oscillate between saccadic movement to track possible new information and focus points of interest. Those periods of focus opposed to the period of saccades are key in the process of attention.

Acquisition methods

To capture gaze data, the main solutions rely on exploiting the difference of brightness between the pupil and the rest of the eye. The pupil appears as a black dot at the center of the eye. Using cameras especially infrared to maximize the difference of brightness, the center can be found by image processing. In the case of infrared cameras, two different phenomenon are used. Toboada and Robinson's patent explain how Tobii's technology uses those principles [START_REF] Taboada | Eye tracking system and method[END_REF]. The dark pupil and the bright pupil effects shown in Fig 2 .23. The dark pupil effect consists in placing a light source far from the camera axis with a certain angle, the reflection of the light tends to darken the pupil. On the opposite, the bright pupil effect consists in placing the light source close to the camera axis, the direct reflection of the light on the pupil tends then to lighten the pupil. The combination of those two effects makes it easier for image processing to search in the image the pupil and therefore the center of the eye.

Performance and robustness

Eye tracking technology has demonstrated high robustness in the last decades. The level of precision is linked to the resolution of the camera, its sampling rate and the image processing algorithm in order to recover the pupil position. Despite faces having different physiognomy, eye tracker technology remains quite effective. Niehorster [START_REF] Diederick | What to expect from your remote eye-tracker when participants are unrestrained[END_REF] characterized on this the limit of those modern devices. Furthermore, image processing algorithms have become more and more accurate, especially with the arrival of deep learning algorithms and their use in this technology to reduce the estimation error. However, some drawbacks are to be mentioned especially with subjects wearing glasses as the infrared light is distorted reflected in an non predictive way. More over, eye tracking technology still suffers from a lack of flexibility. Indeed most of the devices rely on fixed cameras and light source forcing subjects to be placed in the same way. On that note, new systems offer to be mounted directly on the head of the subject via glasses as presented in Tobii's patent [START_REF] Gustafsson | Portable eye tracking device[END_REF]. This comes as a solution to multiple problems evoked before, even though Hooge [START_REF] Ignace | How robust are wearable eye trackers to slow and fast head and body movements?[END_REF] showed that some that there were still limitation to those devices in their use.

Physiological signature

Far from saying that eyes are the mirror of the soul † , some interesting signatures can be observed through eye and gaze analysis. One of the initial measure is the blinking rate obtained by the period during which the signal is lost, it is a key signature of fatigue. This is covered in detail by Schleicher's article [START_REF] Schleicher | Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired?[END_REF] but other physiological phenomenons can be found such as helping to increase focus or as a buffer to process relevant information [START_REF] Ranti | Blink Rate Patterns Provide a Reliable Measure of Individual Engagement with Scene Content[END_REF]. From the gaze perspective, attention level of subjects can be measured by the amount of saccadic eye movements and numbers of fixation points in a time sequence. This was demonstrated by Zhao [START_REF] Zhao | Eye movements and attention: The role of pre-saccadic shifts of attention in perception, memory and the control of saccades[END_REF] even though the number of subjects was quite low. The measure is however not entirely binary (fixation = attention would be an oversimplification, saccadic movements also play an important role in maintaining the attention). In addition to that, the number of saccadic movements can be used as a first measure of tiredness. Since, the eyetracking technology relies on finding the pupil in the eye, its size is also determined. Pupil diameter does not change exclusively with light, other complex mechanisms are at stakes, the variation are also linked to attention and concentration level [START_REF] Ruud | Pupil Diameter Tracks Lapses of Attention[END_REF]. Those variations are studied in depth and even used to reinforce commands in BCI for instance. This is the case of Rozado's study [START_REF] Rozado | Improving the Performance of an EEG-Based Motor Imagery Brain Computer Interface Using Task Evoked Changes in Pupil Diameter[END_REF] that focused on improving MI BCI classifier offline thanks to this new modality.

EEG artifacts

Here it is necessary to evoke the gaze responsibility in EEG artifacts [START_REF] Lins | Ocular artifacts in EEG and event-related potentials. I: Scalp topography[END_REF]. The 6 muscles produce a strong electrical signal (Electro-myographic signals) that contaminates the EEG signal especially in the frontal regions. 41: This word describes something absolutely horrible, it comes from latin meaning "bastard, mixed-blood" which is itself coming from the idea of chimera (a counter nature monster).

In addition, the blinking elicits an electrical wave coming from the electrical potential difference from eye rolling as shown in Fig 2 .24.

Furthermore, the eye muscle activity is dealt by the brain sensori-motor cortex which induces, naturally, an activity contaminating other possible cognitive tasks.

Controlling a robot with gaze

As evoked before, gaze is defined by alternating between phases of fixation and phases of saccadic movements. The variation makes it more complex to establish smooth commands, however if fixation points tend to be on the same location for a certain time, the position can be considered as a command. Shazad [START_REF] Imran | Control of Articulated Robot Arm by Eye Tracking[END_REF] and Ban [START_REF] Ban | Persistent Human-Machine Interfaces for Robotic Arm Control Via Gaze and Eye Direction Tracking[END_REF] presented two good examples of this type of control. In the case of robotic control, two strategies are used. The first consists in using the gaze as a direct command (right, left, up, down) which will in fine control the orientation of the robot effector.

A derived version of this technique will be to use gaze to point towards objects with already known positions and the robot will reach the desired object. The second strategy consists in calibrating the gaze position with the robot end effector position to create a direct control where the robot follows the gaze, this is largely done in 2D but can also be achieved in 3D.

To strengthen the approach, the robot can be equipped with a camera, to have a certain intelligence in the sense that objects pointed out by gaze will be recognized by the robot in order to seize them more efficiently. This was tempted by Ciao [START_REF] Cio | Proof of Concept of an Assistive Robotic Arm Control Using Artificial Stereovision and Eye-Tracking[END_REF] on a proof of concept paper.

Multimodal approach, state of the art and development

To answer the possible limitation of control by the sole BCI over the robot, and to reinforce the sense of agency in an intuitive way, the coupling of the eyetracking to the BCI is an appropriate response.

Context and advances in hybrid BCIs

To merge the two components of control, the name essentially given is "Hybrid" which was defined by Millan in 2010 [START_REF] Del | Combining Brain-Computer Interfaces and Assistive Technologies: State-ofthe-Art and Challenges[END_REF]. 41 Far from the chimeras, hybrid technologies consists in a fusion of modalities to create We use this duality to enrich the interaction a richer control. Eye tracker associated to BCI have been broadly used in the field, firstly in the purpose of removing artifact more efficiently. But it did not wait long to serve a role of dual command. Its feasibility was first studied by Meena [START_REF] Meena | Towards increasing the number of commands in a hybrid brain computer interface with combination of gaze and motor imagery[END_REF] and Doherty [START_REF] Doherty | Exploring Gaze-Motor Imagery Hybrid Brain-Computer Interface design[END_REF]. Hence, eye tracker would be principally used to control the position of the robotic arm, by indicating a position to reach before letting the motor imagery BCI would be responsible of the seizing or the initiation of the movement. Studies remained however at the stage of the proof of concept as explored by Wang [START_REF] Wang | Hybrid gaze/EEG brain computer interface for robot arm control on a pick and place task[END_REF] and focused and demonstrating a gain in performance with a few number of subjects.

Because of the technology constraints, most of the eye tracking available used to be mounted on a screen. The control allowed by this configuration had to go through a monitor to point most of the time in 2D object to reach. The introduction of the monitor enables complete ecological environment especially in the control of a robotic arm. Moreover, studies rely on the integration of the Graz protocol (and its visualization) into the screen for the BCI approach (as presented by Zeng et al. [START_REF] Zeng | Closed-Loop Hybrid Gaze Brain-Machine Interface Based Robotic Arm Control with Augmented Reality Feedback[END_REF]). It appears difficult to say that it is an hybrid system (creating a fusion of information) but it is more two separate systems put together via a monitor in order to achieve control.

But, recent development and industrial advances led to more ergonomic devices especially in the eye tracker field. In addition to that, the rising use of augmented reality in BCI [START_REF] Si-Mohammed | Brain-Computer Interfaces and Augmented Reality: A State of the Art[END_REF] allows to get rid of monitors blocking the interaction. The overall result is a control in an intuitive environment environment as shown by Chen et al [START_REF] Chen | Combination of Augmented Reality Based Brain-Computer Interface and Computer Vision for High-Level Control of a Robotic Arm[END_REF].

Scientific interrogations, limitation of the approach

In this context, we need to evoke what is still to address in this new field of research. Indeed, the coupling of those technologies creates possible complication, especially from the BCI perspective. First because involving the eye can create artifact or additional motor activity that will be sensed in the sensori-motor cortex. This is not necessary a bad thing 42 but the methodology of the experimentation must integrate this point with caution. More over, our lack of knowledge regarding the underlying mechanisms of the brain when involved in a complex task blocks from providing strong conclusions on performances. On this note, many parameters are yet to analyse regarding the dual modality to understand better the keys of the interaction. In a first contribution, we analysed brain activity in a direct control over a virtual arm via eyetracker to assess the involvement of gaze control on the sensori-motor cortex.

Contribution 1

Following those interrogations, at the very beginning of my researches on BCI, we wanted to assess brain activity when controlling a virtual robotic arm, this work allowed to reflect upon the gaze modality and to better shape the hybrid system. This preliminary work was of crucial importance to characterize the gaze impact on the sensori-motor cortex. It allowed when developing the multimodal plaform to know that the gaze would not be neutral hence that a timing of control could overlap motor imagery and gaze activity. Furthermore, it highlighted the need to disentangle the different modalities of control -motor imagery and BCIfor the arm and to find ways to focus the gaze during the motor imagery task in order to limit the overlapping -in addition to limit the artifacts produced by eye.

Towards multimodal

INTRODUCTION

Brain Computer Interface is a wide field of study [START_REF] Wolpaw | Brain-Computer Interfaces: Principles and Practice[END_REF] that focuses on extracting brain signals and the ways to interconnect them with a computer, however despite the breakthrough in machine learning [START_REF] Zhang | A survey on robots controlled by motor imagery brain-computer interfaces[END_REF] and optimisation techniques [START_REF] Lotte | A review of classification algorithms for EEG-based brain-computer interfaces: a 10 year update[END_REF] , it remains extremely difficult to find robust control over a robot for instance in the case of BCI based on scalp EEG using motor imagery (the mental process of imagining a movement without actually performing it [START_REF] Guillot | Contribution from neurophysiological and psychological methods to the study of motor imagery[END_REF]). To strengthen the BCI, a few studies tried to combine it with other more reliable technologies such as eye tracker which shows less variability across subjects [START_REF] Wang | A Human-Robot Interaction System Based on Hybrid Gaze Brain-Machine Interface and Shared Control[END_REF][START_REF] Schiatti | Soft brain-machine interfaces for assistive robotics: A novel control approach[END_REF]. Despite expected performance improvement, the issue of how this can generate overlapping brain activity processes, is still poorly understood.

Indeed the literature is unclear if the control by gaze of the robot activates the zone used for motor imagery task. Furthermore this goes to the extent of any sentiment of control which could produce an activity in the motor cortex activity. Hence we need to ask about the possible link between the sense of agency, " a subjective experience of action control, intention, [...] " [START_REF] Kilteni | The Sense of Embodiment in Virtual Reality[END_REF], and motor imagery.

To answer this question we created a protocol with 3 tasks, two types of control of a virtual robotic arm by movement or gaze and a robot observation taks.

Method, material and protocol

Strategy employed

The principal criterion of the study is the level of agency. There are three level of control: arm mimicking his right arm's movement, we are in the 3rd level of agency where his control is complete, the phenomenon of mirror is expected to give the best scoring of SoA. This will results in a separation between feeling and visualisation, it will be the feeling of the arm but the visualisation of the robotic arm. Second the subject controls the robot by his gaze. We still have a SoA which is no more linked to motor control, we expect that the scoring of SoA will decrease. We want to observe if it still present an activity in the zone of the motor cortex with the control by gaze.

In opposition to what could be observed if the robot is acting on its own where there is supposed to be no SoA. It is to ensure that what we observe during the control by gaze with the EEG cap is indeed the motor cortex activity and not only the parietal lobe activity which is responsible with visual treatment and trajectory planning and could be the only one involved during this controlled task.

The EEG data are processed to extract their spectral density dynamic, we use Matlab with the EEGlab [START_REF] Delorme | EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis[END_REF] plugin for ICA treatment and Brainstorm [START_REF] Tadel | Brainstorm: a user-friendly application for MEG/EEG analysis[END_REF] for some of the statistical analysis.

Material

We created a structure to support a projector, an eye tracking device in order to display a virtual robot. Through the frame, a virtual scene is projected composed of virtual moving red targets and a robotic arm. We use a Tobii Pro X3 in our experimentation. The second part of the user's environment is a Kinect v1 mounted on a tripod coupled with a screen where the same virtual environment is displayed. The EEG cap used is a Enobio 8 electrodes.

Control by motion capture

To perform control by motion capture, we use joints estimation (shoulder, elbow, wrist) by computing a distance transformation as presented by Quoc and al. [START_REF] Pham | Skeleton Formation From Human Silhouette Images Using Joint Points Estimation[END_REF]. From this joints estimation we obtain the vectors chest to shoulder, shoulder to elbow and elbow to wrist. This vectors are reproduced by the robotic arm. We achieve direct control of the virtual by mimicking the movement of the user. Later on we will refer to this type of control as "Mirror Control". In addition to that, we perform a Principal Component Analysis on the vectors obtained before in order to find a common pattern in the movement of a grasping task. From that, we extract the implication of each joint in the movement. That will be used later on to control the behaviour of the arm when it is controlled by the gaze.

Control by gaze

From the eye tracker, we obtain a 2d estimation of the gaze position on the surface. Since our robotic arm is a 3R planar robot, we use a pseudo inverse of a damped Jacobian [START_REF] Samuel R Buss | Introduction to Inverse Kinematics with Jacobian Transpose, Pseudoinverse and Damped Least Squares methods[END_REF] to obtain stable smooth movements that can integrate the characteristics of the human movement. As a result the user focuses is gaze for a period of at least 200 ms on a targeted area which can be a moving target and the effector (the last part of the arm) reaches the position.

Experimental protocol

8 healthy subjects (aged 26 ± 3 years, 4 men) volunteered for the experience. They all came from scientific background and were not familiar with eye tracking technologies. One of the subjects was not a french speaker therefore he received the original questionnaire. They all signed informed consent according to institutional guidelines. The protocol was reviewed by the Sorbonne's ethical committee. The EEG cap is installed on the placement Cz, FC1,C1, CP1, P3, CP5, C3 and FC5.

A robotic arm is displayed on a monitor that moves accordingly to the movement of the subject's right arm, virtual objects are moving randomly on the screen as targets to catch. During 4 sessions of 5 minutes, he is required to perform two different tasks :

1. Move its arm and observe the robotic arm 2. Rest, trying to relax and not to think After the sessions in front of the screen, the subject is placed in front of a frame where he controls the robotic arm with his gaze. The arm is projected on a tilted plane. The subject is asked either to control the robot or to rest for 2 sessions of 5 minutes. After that, the subject is presented the same robotic arm but has no control over its movement. The subject is asked either to control the robot or to rest for 2 sessions of 5 minutes.

Between each set of tasks, the subject answers a SoA questionnaire which assess its sensation of control over the robotic arm. In total, the subject performs the motor activity 36 times, the control by gaze 24 times, the absence of control 15 times and the resting state 45 times.

Data Analysis

We compute the spectral density for each trial and in average for every conditions. We perform a comparison on the average of all the trials by conditions using the two tailed Wilcoxon test in 𝛼 (8-12 Hz) and 𝛽 (15-29 Hz) bands, this allows to check if there is a trend across subjects regarding the different conditions. Secondly, we perform a permutation paired student t-test on the different conditions using a subset of samples to have the same number of trials in each condition for each subjects.

Results

Based on the results of the questionnaires Fig. 2.25, we observe individual differences between the condition no control and the two other conditions. There is not a significant difference between the conditions Mirror Control and Gaze control. Our comparison on spectral density between conditions on average across subjects allowed us to be aware of a trend. results for control by gaze compared to resting state with also a decrease of power at p<0.05 in both frequency bands. However between the situation no control and resting state, we only observed decrease of power at p<0.05 in the 𝛼 band. We do a comparison of the spectral density subject by subject between the different conditions, we observe a difference of activity in the alpha and beta bands in the motor region at p<0,05 and negative t-values indicating a decrease of power. This is shown both by Fig. 2.26 establishing the number of times we observe negative t-values for each electrode from a subject to another and by the table showing the most interesting tvalues for each comparison Tab. 2.1. As expected, the decrease of power corresponding to negative t-values at p<0.05 occurred in most of the case (6 subjects) between Mirror control and resting state. Between the condition control by gaze and resting state for each subjects we observe a difference of spectral activity for 5 of the subjects at 𝑝 < 0.05 and negative t-values indicating a decrease of power. This indicates that being in control of the robot with the gaze can have an impact on the motor cortex activity. Mirror control and control by gaze present similar results both on sense of agency rating and in number of occurrences.

Between no control and resting state, we observe that in the 𝛼 band, for 4 of the subjects, at p<0,05 t-values are negative indicating a possible

Discussion

In the context of hybrid motor imagery BCI, the link between the observation of a robot moving or its control by gaze and the activation of the motor cortex region in bands associated with attention and motor activity must be addressed. Here, we show that there are significant differences of spectral density between resting state and the three other conditions (eg Mirror Control, Control by gaze and No control) with negative t-values. This indicates a decrease of power in the frequency bands of interest (alpha and beta), a marker of motor activity. More than that it seems that the notion of control established by the SoA scoring does not have to be necessary linked to the notion of motor action to generate an activity in the motor cortex. The difference between no-control and resting state can be explained by two different reasons. First motor imagery can be triggered by the observation of someone else's movement [START_REF] Mulder | Motor imagery and action observation: cognitive tools for rehabilitation[END_REF], a similar mechanism might occur when observing a robotic arm moving that has been controlled before. Second, the phase of no control comes right after the phase of control by gaze. The subjects are maybe still trying to control the robotic arm at the beginning of the session resulting in an activity in the same region. From Fig. 2.27, we can advance the hypothesis that the no control state is closer to the resting state, which would explain why there is a decrease of power density in the 𝛼 and 𝛽 bands(negative t-values) for 6 of the subjects. The two subjects presenting an increase in the 𝛼 band (positive t-values) might have been paying more attention to the robot moving freely, this being an interpretation of the alpha band in accordance with the literature [START_REF] Klimesch | $\alpha$ band oscillations, attention, and controlled access to stored information[END_REF]. However we must keep in mind that we only have 8 electrodes over the sensorimotor area, and we cannot exclude other more distributed significant activation. We conclude that in the context of MI BCI mixed with eye tracker, one does not simply treat the moment of control by gaze as a resting state, and there can be a possible overlap of brain activity if motor imagery and control by gaze are not complete distinct tasks.

Our work has some limitations that might be addressed in future studies. First, it was not possible to switch between control by gaze, mirror control and no control at all in an instant. It means that we could only randomise the trials between resting and the other conditions but not between the active conditions. This could introduce a bias. Secondly, some subjects were left-handed (2 over 8) but the virtual arm was placed on the right side of the projected scene and its behaviour was based on right arm movements. This laterality could affect the performance of the subjects in their motor imagery task [START_REF] Gentili | Laterality effects in motor learning by mental practice in right-handers[END_REF].

This closes up the introduction on the knowledge of BCI we relied on for the development of the platform. The next chapter is going to go more deeper into the development of the platform. We will go more on specific aspects of the literature that are useful for the choices made. We will also present the protocol and the main scientific question we ask with regarding to such a platform.

Creating a brain robot interface

What I cannot create, I cannot understand

Development of the prototype, an hybrid solution -the Braccio platform 3

Key aspects of prototype design

▶ Interrogations on the way to build a strong interaction with the robotic arm through BCI. ▶ Justifying the choices of stimuli, feedback and robot control in the prototype : visual stimuli and neuro feedback embedded in an augmented table, robot controlled in position by gaze and the closing by MI BCI. ▶ Covering the methods of computation to create the EEG pipeline. ▶ Covering the different engineering developments in robotic control, eye tracking acquisition and BCI. ▶ Presenting the system's architecture. ▶ Presenting the experimental protocol and its associated hypotheses : investigating different dynamics of control over the robotic arm, either the robot goes to the object after or prior or meanwhile the motor imagery task is performed. ▶ Presenting the material and the statistical methods of analysis.

In the previous chapter, we interrogate the reasons why we end up creating a device that fuses technologies for the control of a robotic arm, here we are going to show more specifically what the different technologies bring to the prototype as well as asking how we should merge those technologies. We will present the different choices that are based on the literature but also ones solely based on the results of reasoning and ideas taken from other fields of research but also linked to experimental and time constraints. All of this makes the platform original but with some remaining parameters that should be addressed.

We aim for a control of the robot based on the eyetracker for the position to reach to grasp an object and the closing of the gripper based on the motor imagery BCI as shown in Fig 3 .1, a link for the final platform overview is provided here to see where we are going ( Braccio Video).

Interrogations

Creating an experimental platform is truly about bouncing between questions that are of many different nature. They can be either engineering or ergonomic or also neuro-scientific and even philosophical questions. Their answers are as well of those same different natures. The complexity lies on the fact that the nature of the question might not be aligned with the nature of its answer, meaning that a neuro scientific question about the experimental platform might be answered by an engineering answer on the constraints of the same platform. We must build our platform above different pillars of concepts (Fig 3 .2) to ensure a good BCI platform and a great interaction with subjects.

Creating a sense of agency -the trust issue

One of the notions that has been central to this work is the sense of agency. As mention before, this sense is part of the embodiment process that shows to benefit the performances of BCI. As introduced in the previous chapter, the feedback BCI is linked to the brain activity. But, if at any point in the process, the link is not good meaning that the acquisition, the spectral estimation or the classification are erroneous, subjects lose their feeling of control leading in fine to a loss of involvement and the failure of the interaction. And this is not unusual as mentioned before when evoking the illiteracy issue but it can concern more generally every subject. Here, the additional source of control that is gaze can come to compensate this possible loss. By ensuring that the system keeps working with the control by gaze no matter what the BCI does, we create an initial level of trust in the system. In a way, the eye tracker is meant to be a crutch to the BCI system. Subjects will be more involved in the task, and in fine trust the overall system, which is key to focus on the motor imagery task and therefore having better performance resulting in a stronger interaction.

How to control a robotic arm ?

In this context, it is essential to know how the robotic arm will be controlled. Defining the control commands is what is going to shape the interaction. We need to remember that BCIs with motor imagery rely on intense focus. This means that a balance is needed between phases of interaction and phases of introversion 1 . Based on the assumption that it is intuitive to perform motor imagery (or resting state) in order to achieve the closing of the gripper which becomes the closing of subjects own hand by association. We could imagine that the motor imagery would be associated to the initiation of any movement, but cognitive tasks must remain simple such as imagining closing the hand. It is to note that it is not possible to differentiate motor imagery power spectrum patterns of the closing hand from the complete movement of the arm. By having in one hand, the imagination of a whole arm movement and in the other hand, the imagination of the sole hand closing, we already create a dissonance. This goes in opposition to the need for congruent feedback. So, to limit the dissonance, it is important to keep a consistent feedback associated to the MI task. Therefore, if the motor imagery task is simple, i.e the closing of the hand, then the reward should be the closing of the robotic arm gripper. Since we are focusing on the gripper, we need to study its movement. The movement is a simple closing. We already established in the previous chapter the two different feedback at our disposal : continuous or discrete. In the case of the closing, you can then control it two different ways. The first way is to associate direct results of the classifier continuously. The gripper will close in an iterative manner based on the correctness of the classification, this is shown in Fig 3 .3 . The other way is to wait until the end of the trial to have a closing if the

Choosing Discrete feedback

In our framework, we choose to focus on the discrete feedback, meaning that subjects will perform the cognitive task for a period of time and then they will receive a reward based on the result of the classification. We will compensate the delay in the response by an intermediate continuous neurofeedback presented later. In doing so, we also limit the intrinsic delays of the classification that could come from a continuous mode. Furthermore, it allows the users to be fully focus on their mental task during the time sloth dedicated.

Concerning the eye tracker control over the position, as presented before, a lot of solutions are on the table. To control the arm based on gaze, a simple and standardized solution is to point at desired objects already known by the system. As mentioned before, the eye can be responsible for a lot of the artifacts in the EEG, if the system of control relies too much on the eyes meaning that the robot is always controlled by gaze for instance , we run the risk of polluting the data from the start. Pointing at few objects represents a good compromise between full control and no control at all. The solution adopted is as followed and can be seen in Fig3.5: first gaze information is retrieved in a 3d space for a short period of 2 secondsthis allows to be sure of users' intent and limits the impact of saccadic eye movement. If the gaze is focused for this time on right or left (defined by the sign along the x axis centered on the glasses middle), the order is sent to the robot to go to the designated object (placed either on the left or on the right). This method provides from having objects on the sideline and objects position must be known to the robot. The trajectories of the robot will be described more in details later on.

Creating the stage

We have our actors, it is time to place them on stage. Concerning subjects, it is pretty obvious, they actually need to be placed in a chair looking towards the device, the EEG cap and the eyetracking glasses will be worn. This is a standard situation in EEG experimentation and allows for clean signals. If the robotic arm is reaching objects, we need a table to put the objects upon. Finally, where should the robot be placed to grasp objects ? The orientation of the robot plays an important role in the embodiment process as described by Lienkamper that assessed that we could observe loss of embodiment by rotating and shifting robot's position [START_REF] Lienkämper | Quantifying the alignment error and the effect of incomplete somatosensory feedback on motor performance in a virtual brain-computer-interface setup[END_REF]. One option is to place the robotic arm at subjects' level on their right, this is Lienkamper best position to obtain embodiment. But in our framework this is not possible for the simple reason it blocks the sight over the table. We can place it on the right of subjects tilted of 90 degrees but it means that it is highly lateralized and it is considered to be the worst configuration for embodiment. Furthermore, we can only reach objects on the right field of vision of the subjects which is not intuitive as well. So we need to abandon the fact that the robot will be close to subjects. The next solution is to place it facing the subject, as two chess players on a table. This configuration has been largely explored in robot collaboration (co-bot) which is one of the source of inspiration of the protocol design. As evoked before, this changes slightly the paradigm as subjects might lose some sense of embodiment due to the position of the robot but they get an "interactive" framework where the robot becomes another agent. On this note, facing the robot has not been explored extensively in terms of embodiment, but Inoue and Kitazaki [START_REF] Inoue | Virtual Mirror and Beyond: The Psychological Basis for Avatar Embodiment via a Mirror[END_REF] have revealed certain effects on virtual avatar presented in a mirror. We might imagine that an effect remains but this is speculative.

Lighting of the beacons -Stimuli

So far, we did not mention the way subjects are able to know when to perform the task. It is necessary to interrogate what should be indicated to subjects to perform the task. Moreover, where and how can we put stimuli in order to continue the creation of the ecological environment ?

One of the initial incentive of the thesis was to think that in order to do motor imagery, we need to anchor our mind on a meaningful object following the idea of a reach to grasp movement to imagine rather than a free movement. So the object to grasp has to be meaningful to be seized in real life -hence the use of soda cans in our case 2 . Everything needs to 3: This is also important to ensure a low level of EEG artifact coming from the eyes.

4: Supposedly to find red fruits in a green landscape as explained by Lee on colour vision [START_REF] Barry | The evolution of concepts of color vision[END_REF] go towards this object : 1) the gaze -hence the attention of the subject 2) the robot -which takes it. So we need to find a way to show both the cans and the stimuli at the same location. To highlight the object, a possible solution is to use a display screen as a table and doing so creating an augmented table. Stimuli will be presented below the cans to indicate what to do, this way, the main focus will remain the cans 3 . Now, what should we show, since the object is already carrying meaning (we, through the robot, have to take it). It must be simple and straightforward in order to limit any disturbing effect. We choose disks with different colours associated to the different cognitive tasks (MI of the closing of the hand or resting state). Since we have the same shape for both stimuli, the colour can be a way to differentiate between tasks. But what should be the colours associated to the cognitive task ? The choices made were the following, red for the motor imagery and blue for the resting state. To understand the motivation behind, we need to leave the field of BCI to enter the field of colour meaning. In human eye evolution, the red colour has been perceived a bit later than the two other colours 4 . The overall consequence is that red is perceived more intensely than the other colours, its "natural" excitation properties explain partially why we associate it to ardour, passion, power and by extension speed as presented by Pastoureau in his essays on colour signification [START_REF] Pastoureau | Rouge: Histoire d'une couleur[END_REF]. The blue colour on the opposite is associated to relaxed and peaceful states also developed by Pastoureau [START_REF] Pastoureau | Histoire d'une couleur[END_REF]. Therefore, it makes sense to associate it to resting state. This argumentation can maybe not be generalized based on color meaning being different around the globe. The Fig 3 .7 presents a view of the experimental platform from above which integrates the stimuli. The disks will appear under the table during a sequence that is generated by the BCI software, it will associate every apparition of the disks with a timestamp to the EEG data.

When should subjects perform the cognitive tasks ?

Now that the pieces come together and that the main design of the platform is set, it is necessary to interrogate the way we are shaping the interaction. The question came from an initial interrogation concerning gesture in reaching task. When do we trigger the closing of our own hand ? Do we already know prior to the movement that we will grasp the object or is it when arriving to the target ? Although this interrogation has been investigated from a pure human movement perspective [START_REF] Betti | Reach-To-Grasp Movements: A Multimodal Techniques Study[END_REF], we wanted to draw the possible parallels in the integration of the robot movement to the motor imagery task.

In the standard literature, discrete feedback and movement are provided at the end of the cognitive task as a reward to ensure a complete focus of the subjects on the task. In our environment, it means that the robot would go to seize the object after the motor imagery task has been performed. But this framework presents some limitations regarding the interaction. The interaction is directed, first an order from the user, then a sequence of actions from the robot. The directed interaction makes it less about collaborating with the robot. Second, if we follow the leitmotiv established so far of intuitiveness, we encounter an issue. The motor imagery of the closing of the hand is not consistent with the action performed by the robot which is also a displacement towards the target before closing the gripper. A possible other way to consider the interaction is to make the robot move towards the target and then perform the motor imagery. This solution has the advantage to create a balancing sequence in the interaction, first a robot movement, then a cognitive task and then a robotic closing. The idea is to benefit from the robotic movement to elicit a preparatory MI state. Furthermore, the motor imagery becomes congruent with the action it creates. Finally, we can consider another approach, where movement and cognitive task become one. By performing the motor imagery meanwhile the robot is moving, this removes the sequential character of the interaction. It merges it which creates synergy. Coming back to the intentional binding effect, our idea when we developed the framework was to artificialize this effect: by creating an initial movement linked with gaze, we create a first brick of agency that is completed by the motor imagery task.

Even though, the relevance of a timing being better than the rest seems logical from the product of a reflection, it remains purely speculative. We do not know what is the best way to control the robotic arm in this specific context. That is why this three timings have been implemented but also they are going to become our scientific question. Indeed, how should we control the robotic arm ? 5: The implementations of some of the methods were done either by or in collaboration with A. Desbois, therefore, the next parts will highlight my contributions but will also mention ones not developed by me that were essentials for the realization of the project. Chain of treatment from EEG signal to classification. First we acquire the raw EEG signal corresponding to trials of motor imagery (in red) or resting state (in blue). We estimate for each trial the power spectrum to obtain an average over trial power spectrum. The average power spectrum for a specific bin of frequency for a specific electrode for each class (MI or rest) is then considered as a feature. If the features are discriminant enough, a machine learning algorithm can draw a separation between those classes (an hyperplane).

Creating the conditions for a robust platform

The reader will now leave the realm of scientific interrogations to enter in the realm of the engineering implementation. Indeed, the next sections will be dedicated to technical implementations that have been explored during this work. In this specific part, I will speak at the first person to make things more clear on the contributions especially in Optimizing standard BCI paradigm Fig. 3.2.1.

Certain part were not exploited for the experimental protocol but their engineering development might interest some. 5

Optimizing standard BCI paradigm

To optimize the BCI system, it needs to be studied in details to identify its strength and its weakness. The main paradigm we are in is the Graz BCI, a sequence of visual stimuli indicating different phases of a trial. The data follow a chain of treatment largely described in the state of the art. The sequence is as follow, a pre-stimulus (a cue to prepare the subject), a stimulus (to ask the subject what to to), a feedback (to show what the subject is doing) and a post-stimulus (to indicate the end of the trial). The time duration varies from a protocol to the next but the sequence is always as presented. The data treatment chain as represented in Fig 3 .8 starts with the time series of the electrodes, after presenting the stimuli, the power spectrum is estimated, then for electrodes at frequency bins of interest, the estimated power enter as features in the classifier algorithm which issues a probability to belong to a class which is represented during the feedback. This pipeline can be created in a dedicated software, in our case Open-ViBE [START_REF] Renard | Openvibe: An open-source software platform to design, test, and use brain-computer interfaces in real and virtual environments[END_REF] (OV). OpenViBE is an Inria software in C++ developed for BCI application. The software integrates the treatment chain from acquisition of the EEG to the restitution of visual or auditory feedback. A detailed presentation is in annex to explain the overall software architecture and how bricks can and have been integrated. The first step for us is to choose what power spectrum estimator will be used in the chain of treatment.

Spectral estimation II

As mentioned in the initial section on the spectral estimation, there are several techniques that can be used. In the initial methods proposed in OpenViBE (OV), there is only Fourier transform and energy of the signal to estimate spectral density. These two methods are not suitable for a rapid estimator of the spectrum therefore our need to develop new estimator in OpenViBE. A "box" dedicated to the Burg autoregressive method was developed by A. Desbois to have a real time estimator inside OpenViBE. Real time is not not stricto sensu usable here. Because the estimator is based on aggregated timing windows, you do not have one time sample converted to one spectral sample (for frequency resolution). In parallel, I developed a python software 6 built largely with MNE which reproduces offline the pipeline of OpenViBE in which I integrated a burg estimator to make sure OV was having the right behaviour. However, the Burg spectral estimator has some parameters which are important 6: I mention this here as it was this tool which was used to determine the different parameters that would be used later on online with OpenViBE. The number of particles, the cognitive and social coefficients and the number of iterations [START_REF] Piotrowski | Population size in Particle Swarm Optimization[END_REF] to study in order to prove their efficiency. Both Krusienski's [START_REF] Krusienski | A comparison of classification techniques for the P300 Speller[END_REF] and Bufalari's [START_REF] Bufalari | Autoregressive spectral analysis in Brain Computer Interface context[END_REF] found an optimal filter order around 20 to obtain the best statistical differences between cognitive states while not increasing too much the need for computing resources. However, both their work solely studied the filter order and did not study the window and overlap parameters. From what you read in section 9, these parameters represent a trade-off between frequency resolution and number of samples for online feedback. We first based our parameters on BCI 2000 historic software but in an effort to make sure the parameters were rightly chosen I used a Particle Swarm Algorithm to assess those parameters.

Particle Swarm Algorithm

Particle Swarm Optimization (PSO) algorithm [START_REF] Kennedy | Particle swarm optimization[END_REF] belongs to the list of non-gradient algorithms which allow to converge towards a solution by searching in the multi dimensional space an optimal position 7 of a given function with multiple parameters without making assumptions on the function. The algorithm is based on an analogy with bird flocks or bee swarms, each particle is a multi dimensional point with a velocity and 2 parameters associated: a cognitive one and a social parameter one. The particles are guided by their local optima and the overall known optima. Through a certain number of iterations, the particles exchange the information of their position to orient the search towards a better optima in a multi dimensional area as represented in Fig 3 .10. At the end of the iterations or if the particles do not vary much in positions anymore, the optimal position is supposed to be found. The parameters of the swarm 8 depend on the nature of the problem, it is crucial to identify with precision those parameters before tackling the problem and to be sure that a convergence can occur. We base our approach on studies that use [START_REF] Akilandeswari | Swarm Optimized Feature Selection of EEG Signals for Brain-Computer Interface[END_REF] the algorithm directly for features extraction [START_REF] Akilandeswari | Swarm Optimized Feature Selection of EEG Signals for Brain-Computer Interface[END_REF] [START_REF] Qi | Channel and Feature Selection for a Motor Imagery-Based BCI System Using Multilevel Particle Swarm Optimization[END_REF]. We will use as an ending result the 𝑅 2 map presenting the statistical differences between MI and resting state for the relevant electrodes of the sensori motor cortex. We then create the negative of this matrix and compute the power spectrum estimation on pilot subjects data inside the particle swarm algorithm. Meaning that for a subject, each particle which has a position in 3 dimensions (filter order, windowing, overlap) corresponds to the spectral estimation both for motor imagery and resting state compared via a 𝑅 2 test for a specific electrode at frequency bands of interests (in 𝛼 and 𝛽), this is represented in Fig 3 .11. Although this approach allows to explore optimal parameters, it is extremely time consuming for certain window and overlap. This is the main limitation that forces to limit the number of iteration before finding an optimal solution. However the empirical analysis arrived to similar results as the one presented in the works of Krusienski and Bufalari as well as allowed to clearly show the importance of those three parameters in the issued 𝑅 2 map. The limitation of this approach is mainly the computation time of each particle which enables any use during the time of an experimentation, it shows however strength as it would be a good strategy to tailor the experience to the subjects.

Adaptive classification

In a standard BCI protocol, the experience integrates two phases, first a calibration phase or training phase where subjects perform the different cognitive tasks without receiving feedback and second, a testing phase where subjects do receive a feedback based on results of the classification and are indeed in control. But features which were used for the training of a classifier have a tendency to change over time due to a number of reasons. Among them, we can mention the shift of the EEG signal due to the sensors and the gel conductivity, the tiredness of subjects, the possible differences in brain activation with the appearance of a feedback and the exploratory mental strategies of subjects during sessions. Hence, the need for adaptive classifier that changes over time and that can start after few trials to give some responses to the user. Several options are available in adaptive classification but to stay close to the neurophysiological signatures present in the EEG signal, one lead that has been explored extensively by Vidaurre [START_REF] Vidaurre | Toward Unsupervised Adaptation of LDA for Brain-Computer Interfaces[END_REF]. It concerns adaptive LDA that either updates the covariance matrix issued from the feature vectors or the average feature vector of the class to give more importance to the new incoming samples. I implemented this method in OpenViBE but it stayed at the sole stage of the implementation as it would have required more time to evaluate its relevance in an online context with subjects.

Developing a new software of analysis -HappyFeat

One of the idea that emerged during the PhD was the need for an analysis tool that could show statistical results and comprehensive representation of the data such as topography map. Fig 3 .12 gives an overview of the representations needed to analyse data in a synthetic way. for motor imagery (in red) and resting state (in blue) for a specific electrode. Bottom Left: Topography map for a certain frequency bin, colormap corresponds to the level of significant differences between the two cognitive states. Right : Statistical difference map (here coefficient of determination) for each electrode for each frequency bin computed over trials of the different cognitive states.

The idea was that we could use such a tool to adapt the experimentation by training more efficiently the classifier based on relevant features in a short time period. The first initial bricks I developed were based on Qt in python where computation were dealt by python libraries. The visualization and statistical analysis were separated from OpenViBE. In parallel, A. Desbois was starting the building of a complete software that would integrate analysis based on OV computation in an effort to be consistent between analysis and what was actually done in OV in experimentation. The initial visual toolkits and statistical analysis I developed were integrated and rebuilt within the team creating the software that would become HappyFeat.

HappyFeat is a python graphical user interface (GUI) wrapping OpenViBE scripts (coded in C++) that has 3 main functions :

▶ it allows to generate scenari 9 in OV with pre-sets parameters which allow for an important gain of time. ▶ it processes data in order to perform visualization, and statistical analysis. ▶ it trains the OV classifier based on features entered in the software and with the dataset of our choice.

Those three functions make it possible to adapt the features rapidly accordingly to the change of mental states of users within sessions.

Because we analyse the data almost in real time, we can decide when

𝐷(𝑥) = 𝑏 𝑤 1 𝑥 (3.1)
where 𝑤 = Σ -1 (𝜇 2 -𝜇 1 ) is the discriminant vector, 𝑏 = -𝑤 𝑇 𝜇 is the bias, and 𝜇 = 1 2 (𝜇 1 + 𝜇 2 ) is the global mean. If 𝐷(𝑥) > 0, the observation 𝑥 is classified as class 2; otherwise, it is classified as class 1.

it is relevant to train again the classifier and to see if basically subjects are able to perform the task of motor imagery or at least if they can produce subsequent differentiable activity between resting and MI in the sensori-motor cortex.

On another note, this software development contributed in letting go the implementation of the adaptive classification in the future experimental protocol. Indeed, since the software allows to quickly train again the classifier based on features we can identify as relevant even if they evolve in time, it appears less relevant to use an adaptive classification algorithm.

Classification process

In our pipeline, we use standard Linear Discriminant Analysis (LDA) algorithm for a binary problem (Right hand Motor imagery vs Resting state) trained on significant features 10 selected by hand for each subject for each session and in each phase of the protocol. LDA is built on the hypothesis that the covariance matrix of the two classes is equal. Each new sample is associated to a distance to an hyperplane making a separation between the two classes feature vector. The projection of the feature vectors minimizes the inter-class variance and maximizes the distance between classes 11 . In our case, we perform an estimation of the power spectrum for the duration of the motor imagery trial and classify it at the end of the trial as either a resting state or a motor imagery task. We rely on a discrete feedback which will be the closing of the hand but the way to do so will change from the beginning to the end of the PhD. The first version was an average of classified samples as shown before in Fig 3 .4. But the second version consisted in estimating the power spectrum average over all trial, and issuing a single feature vector by trial and a single choice for the classifier at each trial. We will evoke more in details why this change in chapter 3 concerning results.

Providing a neuro-feedback

Following the discussion on spectral estimation, a central question in the work was to keep the attention of subjects in the task without disturbing them. Since our classification algorithm would be used to create discrete feedback (the closing of the robot's hand), we lack a continuous information. As presented in the state of the art on feedback, the continuous feedback presents the advantage of maintaining the attention of subjects, moreover, it contributes to close the loop of the system. Indeed, by indicating in real time what is subjects' brain activity, we establish a direct channel of communication, and, in doing so we allow subjects to change accordingly their mental strategy to obtain the most rewarding output.

The idea to integrate the neuro-feedback was introduced to answer the limitation of the robot closing as a discrete reward. The neuro-feedback provides a direct information on the subject's spectral power over one frequency bin or the average of many. In our introduction of the neurofeedback, we advanced the idea that neuro-feedback was used to learn how to modulate brain's power amplitude in certain frequency bands. Here, we want to propose another use for the neuro-feedback that does not focus on modulating subjects' brain frequency amplitudes. Indeed, our approach is task driven meaning that the end purpose is to grasp an object with the robotic arm. Therefore if the neuro-feedback does not match with the end goal and the MI task (imagining closing the right hand), we might introduce some dissonance, and as mentioned before, this is what needs to be avoided. So, we introduced the neuro-feedback as an halo circling the object to grasp. This halo changes its radius based on the power spectrum of the most discriminating feature (hence an electrode at a given frequency bin). This is represented in Fig 3 .13. The more the radius diminishes the better. The reducing radius is supposed to create an implicit association with the hand closing. The conception of the feedback was the product of a reflection on how in ergonomics design we indicate to users that a computer process is occurring as described by Vladic in their review of loading animation [START_REF] Vladić | Analysis of the loading animation performance and viewer perception[END_REF]. There are several indicators known to the general public such as the blue circle for MicroSoft or the rainbow disk for MacOS when loading a file to evoke known representations. Those loading animation are among the basis of Graphic User Interface to maintain users attention while waiting for a final outcome as described by Meyer [START_REF] Meyer | Displaying a boundary in graphic and symbolic "wait" displays: Duration estimates and users' preferences[END_REF]. The other inspiration for this design was linked to concepts of video game design where the camera locks itself to an object by a cursor surrounding the object. This design in video games was used (as it is a bit outdated) to focus on the actions to do regarding the object of interest and freed the player from the control over the in game camera. In our case, this is in the scope of trying to force subjects focus on the targeted object, all the attention should be driven to it.

This neuro-feedback will serve two purposes in this protocol. Because of how it works, it indicates subjects that the BCI has sensed a brain activity and that it is actually working. This serves the purpose of trust in the system and introduces a waiting mechanism, it also contributes to the general sense of agency as the brain activity has an impact on what is presented. The second purpose it serves is the focus: by narrowing the attention towards the target and making it appealing by an additional visual cue, we push subjects to focus even more on the object to grasp.

On the necessity to randomize between trials of motor imagery and resting state

One could argue that the training of motor imagery and resting state could be done separately as we would do in normal sessions of training.

We could design the system to ask subjects to perform the same task over and over for a certain time and then proceed to do the exact same thing with another task. This approach has to be put aside for several reasons.

From an attention perspective, introducing randomness in the presentation of stimulus allows to maintain a certain level of attention. Here, what is sought is to always challenge subjects and to keep them from falling into a sense of routine.

From a pure data acquisition perspective, having the same task over and over can be tiring which leads to changes in the signal, this might end up worsening results over time and the data collection would be less homogeneous. More over, this also means that we need to start first with one task, train on it for a certain time, and then change completely. The slight variations in the EEG signal occurring because of time will increase differences between tasks without the certainty that their are linked to the task, therefore it creates a bias due to time. Last but not least, the fact that the task is performed again and again introduces mechanisms of preparation. Indeed, because subjects know what is appearing next, they can anticipate the task resulting in a variability in the ERD/ERS through trials based on their anticipation towards the next trial. By removing the anticipation, each trial becomes a reset where the signal should appear at the same moment in all configuration. This anticipation would appear similarly if we would alternate between cognitive tasks with a known sequence.

On the choice of providing or not feedback in the cognitive states

The question of the resting state arrived quite early on in the development of the platform. The main question was : what should we show during the resting state ? Providing a feedback has been an endless discussion where effects on performance can be either positive or negative as described extensively by Carabalona [START_REF] Carabalona | Attitude of the subject towards feedback and its implications for BCI use in neurorehabilitation[END_REF][START_REF] Carabalona | To feedback or Not to Feedback? Some Open Questions for Brain-Computer Interfaces Use[END_REF]. Moreover, positive feedback are shown to provide better performances in specific cases as shown by Mladenovic work funded on the paper of Barbero and Grosse-Wentrup [START_REF] Mladenovic | Computational Modeling of User States and Skills for Optimizing BCI Training Tasks[END_REF][START_REF] Barbero | Biased feedback in brain-computer interfaces[END_REF]. The argument was to say that no matter what, the resting state should be a moment of self relaxation none driven by any information to focus on not focusing. In order to achieve that, we should not provide a negative feedback and even no feedback at all after a resting state, this is advanced by Vaslyev on displaying real time feedback in MI practive [START_REF] Vasilyev | Does Real-Time Feedback Affect Sensorimotor EEG Patterns in Routine Motor Imagery Practice?[END_REF]. So, the immediate solution would be to not move the arm at all during the resting state. But, in doing so, it creates a major bias in the sense that it is the robot movement that could elicit some desynchronization as mentioned already in Lana's work [START_REF] Pablo | An ERD/ERS analysis of the relation between human arm and robot manipulator movements[END_REF]. To avoid this possible bias, the considered solution was to have the robot moving based on the eye tracker (the same way as for the motor imagery state), the only difference would be that 12: Mahalanobis distance:

𝐷(𝑥, 𝑦) = (𝑥 -𝜇) 𝑇 Σ -1 (𝑥 -𝜇) (3.2)
with x a multivariate vector, 𝜇 an average multivariate vector and Σ the covariance matrix The end result is that we will provide only a positive feedback in motor imagery. On this note, during the calibration phase, the robot always closes its hand when the task to perform is motor imagery. Two arguments support this approach: first, to be as close as possible of the feedback phase in an effort to limit the differences between the two phases. Second, to already make subjects associate the reward to what they are imagining, this is supported again by Vasilyev in the same article [START_REF] Vasilyev | Does Real-Time Feedback Affect Sensorimotor EEG Patterns in Routine Motor Imagery Practice?[END_REF].

We have covered until here the BCI part of our multimodal system, we are going next to develop the gaze modality.

Optimizing gaze acquisition

We use for gaze acquisition the Tobii Pro glasses 3 that work at 50 Hz sampling rate. Those eyetracking glasses rely on the combination of the light pupil and the dark pupil effect, the light emitters and infrared cameras are blend in the glasses making it highly robust to the change of subjects. The patent associated to Tobii's method of acquisition also indicated the use of deep learning to reduce the error of the gaze position estimation. Due to its sampling rate, some information regarding saccades cannot be totally observed. This technology can nevertheless generate some outliers (aberrant values) that need to be filtered. A possible solution regarding this topic is to evaluate the euclidean distance to a cluster of points, by doing so we create circles in which the data are considered as true and outside they are considered as outliers. This first technique is straightforward to implement, we however need to choose the radius parameter as well as the refreshing rate of the cluster center (the average position of the previous points). But here, it does not cope with the way data spreads out, indeed you could have a cluster larger in the x-axis than on the y-axis (forming an ellipsoid) due to the vision field presented to the user where there are more elements of interests along the x-axis, as represented in Fig 3 .14.

Therefore, another solution which takes more into account the data dispersion is the Mahalanobis distance 12 . This distance measures the dissimilarity between two vectors. The idea in our case is that new incoming samples will be similar to an average vector resulting in a short distance, at the opposite, outlier will appear as extremely dissimilar resulting in a long distance. If the distance exceeds a certain threshold defined based on previous knowledge concerning the data distribution, it is considered as an outlier. This filtering method is well known and has demonstrated its relevancy [START_REF] Leys | Detecting multivariate outliers: Use a robust variant of the Mahalanobis distance[END_REF] therefore it has been applied as a filter for the acquisition of the eyetracking data to ensure a good quality of control over the robotic arm.

Exploration not kept

The next section explores a way to estimate the 3d gaze position based on the 2d gaze and a central camera. This development was set as a proposition of technology but was not kept in the final version of the protocol for its complexity.

Exploring 3D position estimation

The first idea of control for the robot was to have a 3D estimation of the position to reach by the robot. Although Tobii's API provides a 3D estimate, the system has not prove itself entirely reliable as the 3D estimation for gaze is an open problem as explained for example by Xia et al exploring a method to estimate the 3d gaze [START_REF] Xia | High-Accuracy 3D Gaze Estimation with Efficient Recalibration for Head-Mounted Gaze Tracking Systems[END_REF]. Indeed, the estimated vector for each eye which is reliable on a (x,y) plan becomes erratic in 3d because a small angle error substantially changes the estimated depth. To answer this technical challenge, we propose an original engineering solution that couples several techniques from computer vision. First, by using the frontal camera of the glasses, we can use simultaneous localization and mapping (SLAM) to estimate 3d points of the environment. We then use Delaunay's triangulation method to estimate triangles between 3d points (which are in 2d in each camera frame) for each frame. We then estimate the 2d Gaze and associate its found position to one of the triangles. We then project the triangle in the 3d space to get a 3d gaze point. The projection can be seen in Fig 3 .15. But unfortunately le mieux est l'ennemi du bien, and this technique requires high computation resources for the SLAM and the Delaunay triangulation and could not possibly be used in real time. Moreover, to create a standardized protocol where subjects behave similarly, this is not applicable because it increases the variability in the experimentation. By having the robot exploring in its workspace to multiple positions not predetermined, we cannot standardize the movement shown to subjects. Thus, it increases the number of possible sources of errors which could result in disturbing subjects and even in disengaging them which is what we want to avoid.

Robot trajectory

The robot used in the thesis is a Reachy right arm from Pollen Robotics as described in their associated paper [START_REF] Mick | Reachy, a 3D-Printed Human-Like Robotic Arm as a Testbed for Human-Robot Control Strategies[END_REF]. The arm is a seven degrees of freedom (DoF) arm capable of lifting light objects with its gripper. The robot is going to follow pre-known trajectories based on the object's position.

It is first positioned in a standing by position. Then, based on the choice of the object to grasp, it positions its gripper above the object using minimum jerk interpolation for its displacement to mimic natural human reach to grasp movement. Then the robot adjusts itself to the level of the object, grasps and raises it up and puts it back down. Then it is going back to its standing by position. All those steps are shown in Fig 3 .16. The position of the cans are pre-set in the robot space, inverse kinematic is computed to know the ending configuration of the joints, then using the position of the arm in the standing by position and the target configuration joint, the trajectory is computed using minim jerk trajectory. There are no possibilities to change the trajectory of the robot during its movement. The robot's behaviour has all its sequence based on the BCI sequence set by the server that relays the information coming from the EyeTracker and OpenViBE that sets the time of acquisition and the orders as presented in Fig 3 .17.

System architecture

This section presents in a concise structure the different elements that work together. Through different schematics, we dig into the complexity of the system. It also gives an idea of the engineering bricks needed behind. First, we will show the overall system, then, we will take the different actors and dissect them to sublevels of engineering. The first diagram represented in 13: This will later be described as the dataset Batch 1 14: Later described as the dataset Batch 2

Experimental protocol

We want to investigate how the association of Eye tracker and motor imagery BCI should play their distinct role in controlling the robot to obtain good performance from a session to another. In our state of the art, we presented different proof of concepts that have used the combination of 2d gaze estimation by eye tracker and Motor Imagery BCI for the control of a robotic arm. Most of them use a monitor as a way to get the 2d gaze estimation as well as the display of the stimuli. We want to stand out of those approaches. By providing interesting new visual feedback integrated in the environment in order to create an augmented reality feedback. The objective is to create a rich environment to involve the subjects as much as possible. In our case we want to elicit better differentiable brain patterns by providing a realistic or ecological environment which supposedly modifies the implication of the subjects in the experimentation.

In the next two sections, we introduce the experimental sequence as well as the different dynamics that will be called strategies of control. It is to note that certain small elements differ between the first version 13 of the protocol and the final version 14 of it.

Automatic sequences of the robot

The next paragraph presents the different phases involving the robot displacement. It is to note that the robot lifting the can, putting it back and going back to the "standby" position are part of the automatic sequence during the motor imagery trial.

Phases

Here, we are going to describe the different phases of a session. In terms of vocabulary, a session is considered to be all recording once the EEG cap is worn (it basically corresponds to a visit of a subject). A session is composed of different phases themselves composed of different runs. Runs are EEG acquisition composed of different trials where subjects perform different cognitive task.

Hypothesis

Based on the protocol, we formulate several hypothesis:

-[H 1] We should observe differences in terms of performance between strategies.

-[H 2] We should observe differences from a neuro physiological perspective between strategies.

The first two are the principle hypotheses we need to investigate, they are based on the fact that the robotic arm induces some embodiment effect which results in improving the elicitation of motor imagery.

-[H 3] We should observe differences between phases of calibration and control for each strategy.

The third hypothesis is linked to the sense of agency effect, subjects will go through a non-feedback phase where they are not fully in control and phases of full control. By changing the rules of the experimentation, we change the brain activity.

-[H 4] We should observe a delay in the apparition of the ERD/ERS after the stimulus presentation.

The fourth hypothesis is supported by the reaction time of subjects when presented stimuli both in a general sense and in the apparition of the ERD/ERS in the standard literature.

-[H 5] We should observe a strong ERD/ERS for all subjects in all strategies.

The fifth hypothesis is based on our general approach to anchor subjects into an ecological environment, providing them with an intuitive feedback directly linked to their motor imagery task with an eye tracker to ensure an initial level of agency useful to keep the subject engaged in the task.

-[H 6] We should find that strategies involving the robotic arm (hence strategy 2 and 3) have a different behaviour than the one where the robotic arm is used at the end.

The sixth hypothesis is linked to the first two but here the focus is put on the fact that the robotic arm should help the subject in producing discriminant features. This means that the robot interaction with the user should help in observing a decrease of power spectrum during motor imagery states in 𝛼 and 𝛽 bands with regards to the resting state.

-[H 7] We should not observe an important training effect.

The seventh hypothesis relies on the fact that we randomize sessions between subjects so they always receive a new strategy, more over, training effects occur in longer periods.

-[H 8] We should see an improvement of the performance from Control 1 to Control 2 both online and offline.

The eighth hypothesis is linked to the fact that we train the algorithm again to match features between Control 1 and 2 and these features should more stable between control phases which means that the classifier should perform better.

Material and methods

We present in the next section the different methods used for the analysis of the EEG data as well as the different technicalities regarding the hardware acquisition. To complete this section, a dictionary can be found in Appendix that covers some methods and terms in more details.

EEG acquisition

EEG signals is acquired using a 64 electrode BrainAmp system, with TP9 and TP10 as Reference and Ground respectively. The sampling rate is set to 500 Hz. Impedance level is set to 15 𝑘Ω with a tolerance of 10 𝑘Ω with ActiCap Control software. The acquisition is done under OpenViBe 3.3.0 with BrainAmp drivers. Common average reference (CAR) is applied to the data both in online and offline analysis except for connectivity analysis.

Eye tracking acquisition

Gaze is recorded and used as a command with a Tobii Pro Glasses 3 set to 50 Hz. Only the x-axis direction is used to choose between right and left. Negative values are the indication to seize the left can and positive values are the indication to seize the right can.

Signal Processing

Online signal processing

Online processing was performed with OpenViBE 3.3.0 (Inria software, France) through a dedicated pipeline. Sixty-four electrodes which covered the scalp and especially the sensori-motor cortex were used for online process. The EEG were sampled at 500 Hz. On one hand, the power spectrum estimated with the auto regressive method is computed using a window of 250 ms and an overlap of 161 ms for a specific bin for a certain electrode determined by the pre-analysis of the data and used for the neurofeedback halo radius. The radius was changing smoothly its value with a logarithmic iteration of the power estimation following Ramoser work on the computation of the amplitude of the visual feedback [START_REF] Ramoser | EEG-based communication: evaluation of alternative signal prediction methods[END_REF] . On another hand, the signal is buffered for 3 seconds to compute the average power spectrum over the windows of power spectrum estimation 17 , the result is sent to the online 2-class LDA classifier which then issues a probability. The probability value is sent to the robotic arm to determine if it closes or not its hand.

Offline signal processing

Muscular activity data contained the labeled 64 EEG signals with the timestamps corresponding to muscle activity, resting state and cue.

Common average reference was applied to the data before computing the power spectrum estimation with Burg auto-regressive method. The window of analysis corresponded to the MI/rest task with an offset of 1 second determined as a reaction time before the true activation of either Rest or MI. During this moment eye activity is very low as the subjects are focused on the target during this period.

Pupil diameter analysis is performed by suppressing moments of blink and interpolating data to limit loss of information and jump effect.

Statistical analysis

The next section presents the methods used in the journal paper to characterize the different dynamics of the system.

Classifier Performance

We compare online sensitivity and accuracy using a Wilcoxon ranksum 18 test. We compare the average score for each subject between strategies. We also compare scores between phases (Control 1 and 2) for each strategies.

Power spectrum analysis

We perform an average across trials of the power spectrum of the motor imagery and the resting state and compute the 𝐸𝑅𝐷 = (𝐴 -𝑅)/𝑅 * 100 based on Pfurtscheller et al [START_REF] Pfurtscheller | Functional brain imaging based on ERD/ERS[END_REF](A: the motor imagery average power spectrum, R: the resting state average power spectrum). We want to assess two effects, first if the phases are different, second if the strategies are different as well in terms of ERD for each electrode. To assess the distribution of the ERD across the scalp at the group level, we perform cluster based permutation test [207][208]. This method is used to assess significance of clusters in neuro-imaging data. It identifies brain regions exhibiting significant differences between conditions. The test calculates a statistical test (student t-test for instance) for each electrode in the case of EEG. These electrodes are then thresholded by their level of significance creating binary maps.Adjacent electrodes that exceed the threshold are regrouped in clusters that sum the statistics of every element. Then the group labels are shuffled randomly to recompute all the previous steps. The distribution of tests is used as a threshold to assess the cluster significance. The clusters that remain under significant threshold can be considered as meaningful and interpretable.

We define a threshold computed using percent point function at 𝛼 = 0.01 for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15, we follow the documentation for MNE function permutation cluster 1samp test. The adjacency matrix is set to 40 mm for the electrodes considered as adjacent. We establish profiles for the 3 strategies in the three phases of the session.

Time frequency analysis

The ERD is computed based on the power spectrum computed for parameters 𝑛 𝑤𝑖𝑛𝑑𝑜𝑤𝑠 = 0.25𝑚𝑠, 𝑛 𝑜𝑣𝑒𝑟𝑙𝑎𝑝 = 0.1𝑚𝑠 and 𝑛 𝑓 𝑓 𝑡 = 5000 to ensure enough samples for analysis. We search how the spectro temporal distribution is different between strategies for each electrode. To do so, we average over trials for each of the subjects the difference of MI and resting state time frequency maps. We then perform cluster based permutation test on the 𝛼 band (8-12 Hz) and on the 𝛽 (13-35 Hz) for the 4 s of cognitive task. The cluster permutation test considers adjacency in time and frequency as nearest neighbours in the 2 dimensions. We use a threshold computed using percent point function at 𝛼 = 0.01 for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15, we follow documentation for MNE function permutation cluster 1samp test. We only keep clusters below 𝑝 < 0.01.

Functional Connectivity analysis

We perform the spectral coherence using welch power spectrum estimation for each subjects for each trial in all 250 frequency bins. We then average over trials and between frequency bins of interests (13-25Hz) for the 4 s of cognitive task. We then compute the cluster based permutation test at the group level at 𝛼 = 0.05 for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15 and we keep clusters below 𝑝 < 0.05.

Source Space analysis

To perform the source space analysis, we follow those steps : first, we perform the cross spectral density (CSD) using morlet spectral estimator in the 𝛽 band (13-25 Hz) to get an average CSD over trials of calibration, Control 1 and Control 2 for each strategy and for motor imagery and resting state trials. We then compute for each subject the 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 CSD. We compute the forward model based on the free surfer average MRI scan (average scan computed over 40 healthy subjects). We compute Dynamic Imaging of Coherent Sources (DICS) beamformer weights over the CSD to obtain vertices in the source space. We then compute the cluster based permutation test on 15 observations/subjects' vertices in each phase for each strategy at 𝛼 = 0.01 and we keep clusters below 𝑝 < 0.05. A thorough review on the use of beamformer has been presented by Westner et al. [START_REF] Westner | A unified view on beamformers for M/EEG source reconstruction[END_REF]. In this review, Westner comes back to the origin of the spatial 19: This is the case of beamformer spatial filter.

filter, its different variant and its application to neuroscience first through MEG and later on through EEG. This was performed for both batch of analysis, in an effort to stay close to the steps in the sensor space. We also perform source space reconstruction using weight minimum norm estimate and compute in the source space directly the power spectrum estimation using multitaper with an overlap of 500ms. This method was prioritized for the main contribution of the thesis as it does not apply a spatial filter that tends to search for focused activity 19 .

Pupil diameter analysis

The pupil diameter for each eye is extracted and the derivative is computed though discrete difference for each subject. We evaluate the statistical differences between strategies using a wilcoxon test on subject's observations.

Conclusion

This chapter introduced the many aspects present in the PhD thesis from the scientific interrogations to the pure engineering development. The chapter is meant to answer the possible questions of the reader regarding the different choices that have been made and also to explain why certain technological solutions were not used for the final version of the experimental protocol. The last part of the chapter serves an introductory role as it presents the different aspects of the experimental protocol centered on healthy subjects, it also presents the different hypotheses regarding what we expect to observe. Certain of them are already validated in the literature but we should observe them once again and others are specific to the protocol and are original to the field to our knowledge. Finally, the chapter presented the different methods of analysis that would be used which includes the different statistical tests and ways to process the EEG data.

From theory to experience, first successes, first failures

Errare humanum est, sed perseverare diabolicum

This pleasant sentence might be a fraud as the original source was not found.

Another good example of how internet can be deceiptive. So, to complete this, it seems that Alexander Fleming said it as well when finding the penicillin as told in Introduction to the History of Mycology by G. C. Ainsworth. Words, words, words...

Forging the protocol through experience -General observations on subjects and Batch 1 first study 4

The most exciting to hear in science, the one that heralds new discoveries is not "Eureka" but "That's funny".

Attributed to I. Asimov

Key Results

▶ Characterization of subjects of the different batches, assessing the difficulty of producing the different cognitive task tanks to the dispersion of data in trials. ▶ Offline performance evaluation of the first batch of subjects. ▶ Neuro-physiological analysis of first batch through ERD/ERS and node strength based on spectral coherence functional connectivity evaluated with cluster based permutation test. Establishing difference of patterns at the group level between strategies and highlighting the consistency of one timing/strategy over the rest. ▶ Discussion on batch 1. Evaluating the limitation of the dataset which receives a non rewarding feedback, argumentation on the use of the method as a sham protocol.

In this chapter, we are going to present the different results from the collected datasets on healthy right handed subjects. The protocol was approved by Inria's national ethical committee as part of the BCIPRO protocol ( authorization number 2021-35 -ref SICOERLE n°179). Experiments took place in the extremely controlled environment of the EEG/MEG center within the neuroimaging core facility of the Paris Brain Institute.

To give an idea of the time spent and the amount of work of the data collection. From March to July 2022 we had a first group of 11 subjects which came for 3 sessions. Based on the knowledge we built, a second group of 15 subjects came for 3 sessions from October 2022 to April 2023. In total and counting the different pilot subjects that shaped the experimentation, I recorded 96 experiences. The dataset is aimed to be stored on Mother of all BCI Benchmark (MOABB) [START_REF] Jayaram | MOABB: trustworthy algorithm benchmarking for BCIs[END_REF] to share with the community.

We will first evoke the general observations on subjects' performances, then present results from the first batch of subjects and lessons we learned from them. In a second time (in chapter 5), we will present results of the second batch which received the final version of the protocol and ended up being used for the main publication of the thesis. We later observed that the activation for right handed of their right hand is not solely localized in the left hemisphere.

2:

The main recruitment came from inside of the Paris Brain Institute.

3: This is not part of the study of course but it is always relevant to mention as the inability to control a BCI is a complex recurrent theme in the field. More over the physical activity ranged from sport, dance and acting. The main idea behind is probably the sense of proprioception which was developed for their use.

General observations regarding the dataset 4.1.1 Subject characterization

We had few criteria of selection for subjects, they needed to be right handed, over 18 and healthy, moreover the batches had to be gender balanced. We only kept right handed subjects to standardize the protocol. Indeed, it is necessary to ask the same task for all subjects which in our case would be the motor imagery of the right hand closing. More over the robot was a right arm. We wanted to avoid a possible bias in the activation patterns of the sensorimotor cortex [START_REF] Gentili | Laterality effects in motor learning by mental practice in right-handers[END_REF] (the handedness tends to activate both hemisphere with the "weak" hand 1 ). Our batches were quite homogeneous in age, the first ranged from 25 to 37, and the second from 22 to 35. A possible critic we can formulate is that the method of recruitment made it difficult to cover different social background 2 , but this also allows to have homogeneous batches. As expected from the literature, the subjects who perform the best (where the motor imagery state was highly different from the resting state) were the ones who did a physical activity 3 or did it to a high level before.

Global Analysis

Here we are going to introduce results across batches to describe subjects responses to the experimentation. We are going to assess their ability to perform the two cognitive task and in the process to see if they have a differentiable brain patterns. To do so, it is necessary to evaluate for each subject in each phase if at some point we observe differences between the power spectrum of the MI and resting state in the sensorimotor cortex. Also this will be an assessment of a possible training effect. Indeed in combination with performance metrics (accuracy or sensitivity), we will be able to observe possible changes which are closer to the real evolution of subjects in the task. It is to say that, based on the knowledge acquired during the experimental campaign, the task which appeared to be the most difficult is the resting state. The subjects judged the task hard to perform, complex to maintain and difficult to replicate from a session to the next. We will introduce on this note an idea regarding subjects ability to perform the cognitive tasks we call the Eureka effect we observed with subjects that did not perform well at first. This can be seen as the result of training but since the effect sometimes does not last in time, it might go beyond training and it lights another intake on the illiteracy.

Qualitative analysis

In the next section, we will try to qualitatively characterize the subjects based on their ability to change their power spectral amplitude from resting state to motor imagery state. We will establish a dichotomy between ones that are skilled in the task compared to the ones that are less skilled. Having a high difference between power spectrum of resting state and MI for a specific electrode at a given frequency bin is considered to be a discriminant feature useful for the classification algorithm (LDA).

General observations regarding the dataset 71

4: To compute the 𝑅 2 , we base ourselves on the BCI2000 software implementation.

The prodigies -Subjects gifted

Those subjects distinguish themselves by their ability to create discriminant features even during the calibration phase, we will use the term prodigies to name them. This ability is observable from the 𝑅 2 test 4 but also from the classification accuracy. We can observe for a specific subject what the ability looks like in We observed that these subjects change voluntarily (or maybe involuntarily) their brain patterns passing from the calibration to the control phase. The overall consequence is a more intense desynchronization and this is shown by the 𝑅 2 coefficients increasing in more sensori motor areas and in more spectral frequencies as shown in Fig 4 .2. Due to their impressive ability to produce differentiable brain patterns in which ever configuration right away, those subjects make it more complex to identify a more relevant strategy which means that if differences there are, subtle they will be. In terms of number, we found three subjects per dataset that could be referred as prodigious. They represent 37% of the subjects that were already presenting relevant features in batch 1 and 20% of them in batch 2. Here, what is noticeable is the spread of high statistical differences between resting state en motor imagery PSD in the sensory motor area in the low 𝛽 band from the calibration to the control phase. This spread out of the intensity can be the mark of the engagement of the subject but also his or ability to greatly perform the task.

5: This is also supported by the literature [START_REF] Alimardani | The Importance of Visual Feedback Design in BCIs; from Embodiment to Motor Imagery Learning[END_REF] 6: A state where they have focus, trust themselves and follow the flow. Those subjects are still searching for an optimal mental strategy, in that sense, they are learning to create the differentiable patterns. We will consider that their efforts is their trait and name them Sysyphus. In both cases (not able at first or relatively able) we saw some improvements but how the improvement occurs is not straight forward. It is clear that the feedback apparition presents a clear advantage (Fig 4 .4). 5 But it is unclear if it is the sessions that present an effect on the apparition of the differentiable patterns as it was not shown statistically that the performance increased from one session to the next. It is however clear that getting familiar with the device and its mechanisms is crucial.

The Eureka effect introduced concerns all those Sysyphus. During one of the phases of one of the sessions, they finally "get it" and patterns of desynchronization from resting to motor imagery (resulting in the decrease of power spectrum) appear. The principle hypothesis that would remain to be proven is that it is the let go 6 that allows them to get to an exploitable resting state which is then differentiable from their active state. In both batches, we found that approximately 30% of subjects were not able to perform the task right away and control the BCI [START_REF] Becker | BCI Illiteracy: It's Us, Not Them. Optimizing BCIs for Individual Brains[END_REF]. This follows the general trend present in the literature. Even though they are unable at first to generate the power spectrum difference between the MI Here what is noticeable compared to the "prodigy" behaviour is that the highest statistical difference is not in the sensory motor area and the spread out in the 𝛼 band across all the electrodes is a mark of the inability to perform the task (yet).

and resting state, they manage after some time to do it as well, therefore they are kept in the study.

Variability between subjects

It is important to note that the differences between "prodigies" and "Sysyphus" are a mark of subjects' inter-variability. Indeed, subjects are different in their ability to perform the task but also in their the way they do it (in the sensori motor cortex and in the frequency bands in which the desynchronization occurs). It is to not that for the two batches, all subjects were able to have a significant desynchronization in the control 2 phase even though certain had a stronger desynchronization. this means that group level analysis can be performed if we normalize the difference between MI and rest PSD for all subjects to compensate for the possible effect of specific subjects' influence.

Motor imagery or resting state ?

As mentioned before, it appears rather clear that the motor imagery is performed better than the resting state, this is shown by the variability in the features that are exploited for the classification. This is described in Fig 4 .5 which shows how the features of the two classes for a specific electrode at a specific frequency bin are distributed in the calibration 7: By presenting higher variability in its features, resting state seems to remain quite exploratory, the mark of the mind wandering during the trial.

and how the new points corresponding to the new phase belong to those distributions.

The motor imagery features are often less variable from a trial to the next than the resting state ones Fig 4 .6 shows the distribution of the power spectrum amplitude in the two cognitive states for the different features used for the classification algorithm synthesized by the Principal Component Analysis (PCA) to limit to the 3 most relevant components. 7 . This is observable for all subjects (especially for the prodigies). It is also how we can observe that subjects start performing the task well as the MI features become less variable. Features by Principal Component Analysis, Red for motor imagery , Blue for resting state, left "sysyphus" -distribution of the control 2, right "prodigy" -distribution of the calibration. First, we select using the 𝑅 2 map the features (specific electrodes at a specific frequencies) that present the highest statistical differences. Second, we use over the trials of a block (calibration or control 1) a PCA on the feature matrix consisting of the observations for the different features selected to retrieve only the principal 3 eigenvectors (based on the 3 highest eigenvalues), third we plot the 3 components in the different trials for the two conditions. The purpose of the plot is to have a synthetic representation of the features to see their distribution. Before presenting results of the first batch, it is necessary to establish something clear. The feedback given to subjects was based on a computation sensitive to more randomness of the classification. The feedback was computed based on the results of the classification as always but, in details, the classifier issued a distance to the hyperplane for each power spectrum estimate during the 3 seconds of feedback. The average distance over the hyperplane was the indicator of the closing or not of the gripper. This solution is highly sensitive to the variation of power during the trial and resembled more to a continuous feedback adapted as a discrete feedback. Therefore, the performance results online were extremely low comparatively to the subjects' potential. This forced us to be cautious in conclusions made from the performances standpoint. More over, the offline analysis leads us to think that something was odd with the feedback given. On this note, the offline analysis was published in a conference paper and focused on the comparison between two of the three strategies based on the subjects we had at the time. This analysis was completed by an additional one based on a new method of classification -Riemannian geometry. It is safe to say that the calibration part of the dataset can be analysed thoroughly as it presents patterns prior to any feedback. However, the results obtained on the two phases of control where a feedback was displayed (supposedly a full control over the robotic arm) should be carefully put into perspective and we should not formulate strong conclusions based on them. 8: 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡 𝑦 = 𝑇𝑃 𝐹𝑁+𝑇𝑃 9: It is important to say that the accuracy will be later introduced to be sure that our approach is not giving high sensitivity but low accuracy which is simply a biased classifier.

Analysis of the first batch

Using the blocks

There

Performance Evaluation Online Performance

We evaluated the number of time the robot seizes an object in the motor imagery state, this means that we evaluated the True Positive Rate (TPR) but through sensitivity 8 , the accuracy was not at first studied because we did not have any feedback on the Resting state, more over we inspired ourselves from Pereira's work [START_REF] Pereira | Online detection of movement during natural and self-initiated reach-and-grasp actions from EEG signals[END_REF] focusing on TPR on the results 9 . As we observed in the Fig 4 .7, the number of time the robot closes its gripper out of ten trials of MI was quite low. But actually, since the result of the classification was not what was closing the gripper but the average of choices over a trial, it is not per say a true TPR from a machine learning standpoint. And in addition, we could not observe any differences between strategies. This was quite strange when we compared those scores to the neurophysiological responses of the subjects (highlighted by the 𝑅 2 maps). More over, training the data a second time did not have a significant effect. And finally within phases, the score was varying often with no clear direction. Those low results were in opposition with what we could find offline.

Offline Performance

Parts of the results were published in IEEE MetroXRAINE conference.

Exploring strategies for multimodal

Results

The first paper only dealt with 7 subjects, did not take into account what will be later the strategy 2 (with the robot moving meanwhile subjects perform cognitive tasks) and focused on the training within datasets of calibration and control 1. The idea of the paper was to introduce the framework and to justify the relevance of studying the timing in the robot control. To observe if the trend disappeared with a higher number of subjects, we computed the offline accuracy using a SVM with radial basis function on feature selected by hand based on the different 𝑅 2 maps throughout the sessions to evaluate if we observed the same behaviour when training on calibration and then on control 1 (Fig 4 .8).

We found again that strategy 3 (motor imagery after the robot reached the can to seize) is higher than the two others and that in all training configuration but not at significant level anymore. To complete the analysis, we also compared between phases how the accuracy evolves (Fig 4 .9). Even if we could observe relative improvement between phases of control and strategy 3 remaining the highest one in both phases, it was not statistically different between control 1 and 2. Following the work we did before, we also assessed the evolution of accuracy from one session to the next (Fig 4 .10). We did not find significant differences between sessions Tab4.1 and no trends are observable that would indicate an improvement between sessions. We later on completed this analysis with an evaluation of the accuracy for each strategy using a state of the art method, a linear regression on tangent space features from covariance matrices described as a Riemannian manifold [START_REF] Barachant | Multiclass Brain-Computer Interface Classification by Riemannian Geometry[END_REF] ( Fig 4.11). This presents the advantage of not selecting features by hand which standardizes the evaluation. This second analysis also helped demonstrate that something was definitely odd with online results in comparison to the high accuracy we could obtain from our subjects indicating that they were indeed performing the task well at some point.

Before going further, it is necessary to indicate that it was at this point that some changes were made to the protocol. These changes will be elaborated in section 4.3.5. The next part we are going to cover is the neurophysiological analysis that was conducted on batch 1. This analysis takes a new look at the dataset with fresh eyes and it will greatly serve the global analysis with the objective of answering the different hypotheses formulated.

NeuroPhysiological Analysis -Investigating brain patterns to characterize the strategies ERD/ERS spatial distribution reveals consistency of activation

To assess the brain responses of our subjects, we use the relative difference of powerspectrum between the motor imagery state and the resting state 10 for each electrode on specific frequency bands. We mainly focus on the 𝛽 band where we could obtain the most relevant information. Actually, we could observe differences at statistical level only in this specific band. By performing the average ERD across subjects we observed some different spatial behaviour between strategies and between phases. This lead to the idea that we needed a tool to assess how the spatial distribution is statistically different. Based on Corsi's work [START_REF] Corsi | Functional disconnection of associative cortical areas predicts performance during BCI training[END_REF] and an extent research on the literature [START_REF] Bullmore | Global, voxel, and cluster tests, by theory and permutation, for a difference between two groups of structural MR images of the brain[END_REF][START_REF] Maris | Nonparametric statistical testing of EEG-and MEG-data[END_REF][START_REF] Pantazis | A comparison of random field theory and permutation methods for the statistical analysis of MEG data[END_REF], we decided to use the cluster based permutation test.

Strategies presented different distributions of power spectrum desynchronization. At the group level, during the calibration (Fig 4 .12), we only observe significant cluster for strategy 3. During the control 1 phase (Fig 4 .13), we observe that the difference of power spectrum is statistically different for all strategies but strategy 3 remains the strongest both in terms of intensity (the highest negative T values 11 ) and in cluster size, we should note that the cluster size is not a criterion to describe if a strategy is better than another. And finally, during control 2 (Fig 4 .14), it appears that strategy 2 presents a broader spatial distribution but strategy 3 between subjects for the band (13-25Hz) in first phase with no control over the robot (Calibration), threshold of display set to 𝑝 < 0.05. Down : Evolution of the cluster size (in decimeter) in function of the negative ERD threshold. To do so, we average across subjects the ERD for each electrodes, we then evaluate the diameter of the cluster formed by the sensori motor cortex area. We increase the negative ERD threshold to see how the cluster diminishes, this way we have an insight on the relevant electrodes at the group level.

12: The end result of having a variability in the feedback response with the closing of the gripper. remains the most intense (the highest negative t-values). Even though in phases of control 1 and control 2, subjects did not receive a true reward for their task (based on what we have established before) we still can draw some interesting points. First, strategy 3 is consistent throughout all phases even with a flaw in the feedback. Second, there is a reinforcing effect through phases for strategy 1. Third, having the robot involved in the interaction tends to activate a broader range of electrodes in the driving phases even when subjects do not have full sense of agency 12 . between subjects for the band (13-25Hz) in first phase of pseudo control over the robot (Control 1), threshold of display set to 𝑝 < 0.05. Down,left Evolution of the cluster size (in decimeter) in function of the negative ERD threshold. To do so, we average across subjects the ERD for each electrodes, we then evaluate the diameter of the cluster formed by the sensori motor cortex area. We increase the negative ERD threshold to see how the cluster diminishes, this way we have an insight on the relevant electrodes at the group level. Down, right, Number of electrodes in each cluster for each strategy. between subjects for the band (13-25Hz) in second phase of pseudo control over the robot (Control 2), threshold of display set to 𝑝 < 0.05. Down,left Evolution of the cluster size (in decimeter) in function of the negative ERD threshold. To do so, we average across subjects the ERD for each electrodes, we then evaluate the diameter of the cluster formed by the sensori motor cortex area. We increase the negative ERD threshold to see how the cluster diminishes, this way we have an insight on the relevant electrodes at the group level. Down, right, Number of electrodes in each cluster for each strategy.

Studying subjects separately suggests the need for congruent feedback

Since we observe differences of patterns at the group level between strategies, it might be interesting to see at the subject level how those differences characterize themselves. To do so, we estimate the power spectrum for each bin for each electrode and compute the difference 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 , we only keep the relevant bins to form the 𝑒 𝑙𝑒 𝑐𝑡𝑟𝑜𝑑𝑒 𝑠𝑥𝑏𝑖𝑛𝑠 matrix. We do this for the three phases (calibration, control 1, control 2). Then, we compute different matrix distances (Frobenius distance, spectral form and mahalanobis distance) between the phases for each subject. Even though the distances appear to be shorter in some configurations (Fig 4 .15), it is difficult to establish a trend. This absence of trend might be an indicator of subjects exploring "mental strategies" in all phases because of the absence of congruent feedback.

Time Frequency Analysis allows to reinforce the idea of consistency of one specific strategy

In order to refine the analysis, we need to evaluate how the desynchronization occurs within trials. For all subjects, there is the same amount of time to perform the motor imagery (and resting state) but, since some robotic movements occur prior or during the cognitive task, we can interrogate how it affects subjects. To do so, we compute the power spectrum for a high number of windows and with a frequency resolution increased to get a smoother variation of the power spectrum over time.

Based on Brinkman's work [START_REF] Brinkman | Distinct Roles for Alpha-and Beta-Band Oscillations during Mental Simulation of Goal-Directed Actions[END_REF], we compute cluster permutation test on time frequency maps across subjects to assess statistically the time period in which the desynchronization occurs. From that, we compute for each electrode the sum of the elements inside those significant clusters. We then plot on a topography map what electrodes present the highest cluster size and importance. By doing so, we obtain an information both spatial, spectral (since we observe a cluster on time frequency maps) and temporal as presented in Fig 4 .16. As mentioned before, we mainly focus on the calibration part in the batch 1 as it is unbiased in opposition to the rest of the session, but for the sake of clarity and because we could end up finding relevant information, we also evaluate control 1 and 2.

𝐶𝑜 ℎ(𝑤 𝑘 ) 𝑖𝑗 = |𝑃 𝑖𝑗 (𝑤 𝑘 )| 2 𝑃 𝑖𝑖 (𝑤 𝑘 ) • 𝑃 𝑗 𝑗 (𝑤 𝑘 ) (4.1) 14: 𝑠 𝑖 = 𝑗 𝐴 𝑖𝑗 (4.2)
where A is the functional connectivity matrix.

[217]: Cattai et al. (2021), 'Phase/Amplitude Synchronization of Brain Signals During Motor Imagery BCI Tasks' higher activity which is more spatially distributed and more spread out in the time frequency domain. We note that strategy 3's profile remains from one phase to the next whereas the two other strategies do not show a clear improvement especially for strategy 2 in batch 1. This results come to support the claim already established in the ERD/ERS part that strategy 3 is consistent through out the session despite the lack of agency. This tends also to support why we observe a trend in our offline analysis over performance between strategy 1 and 3. Indeed, the desynchronization being our classification basis, it does not come in opposition of the previous result.

Functional connectivity analysis -Brain networks reveal other differences between strategies

In the previous sections we have assessed the information contained by each electrode separately. Here, we are going to explore the interaction between electrodes thanks to functional connectivity estimated with spectral coherence. Spectral coherence measure is a good transition from the ERD/ERS traditional analysis to the brain networks analysis because it estimates the correlation of power spectral across electrodes 13 . We want to assess which are the most connected electrodes. To do so, we rely the node strength of the complex network field. 14 Following Cattai's work [START_REF] Cattai | Phase/Amplitude Synchronization of Brain Signals During Motor Imagery BCI Tasks[END_REF], we establish for each subject for each phase the electrode list and their associated node strength in the motor imagery and the resting state. The associated signature to motor imagery is an increase of the node strength from a resting state to a motor imagery state in the sensori motor cortex. We use the cluster permutation test on the 15 subjects to assess if some statistical information remains at the group level using the same parameters of clusters as for the ERD/ERS analysis. In the calibration phase (which is the main source of our analysis), we find that all three strategies present an increase in the node strength from resting to motor imagery. In all phases, strategy 3 presents the highest t values in the sensori motor area even though the other strategies present significant activities.

Pushing the results to the source space to know if strategies involve different brain regions

Another approach to the data is to evaluate how brain regions are activated in the different phases and in the different strategies. Since we do not have the MRI scan of subjects, the level of precision we can obtain by projecting on an average MRI scan is limited, this has been highlighted by NeugeBauer et al. on a specific study on the estimation of Epileptogenic zones of the brain using the method [START_REF] Neugebauer | Validating EEG, MEG and Combined MEG and EEG Beamforming for an Estimation of the Epileptogenic Zone in Focal Cortical Dysplasia[END_REF]. But, it remains relevant to see if some specific regions present different degrees of activity depending on the strategy. To do so, we compute a forward model on free surfer average scan (over 40 subjects), we estimate cross spectral density over the 𝛽 band for each condition over trials of each phase and perform 𝑀𝐼-𝑅𝐸𝑆𝑇 𝑅𝐸𝑆𝑇 . We finally compute the inverse model using dynamic imaging of coherent sources (DICS) beamformer for each subject. We then perform the cluster based permutation test at the group level. In Fig The main observation we can formulate is that strategies and phases produce various profile. During calibration, it seems that only strategy 1 presents an activity in the sensori motor area but this becomes the case for all strategies in control 1. Strategy 2 and 3 seems to activate premotor cortex area in control 1 and control 2. We might consider here the fact that the number of subjects limits the statistical power allowing to highlight specific regions. We can however note that the profile of strategy 1 and 3 appears to be similar to their profiles in the time frequency analysis.

Beamformer technique is however used to estimate a focal zone of activation, which might show limitations in its interpretation, especially when we compare it to what we obtain in terms of distribution in the sensor space. So, it is necessary to use an additional method in the source space. Using wMNE, we observe more relevant information, strategy 3 is active in both phases of control. Nevertheless, strategy 1 is the only one active for the calibration 1. If we come back to the time frequency analysis, strategy 1 in the calibration was extremely localized, at the opposite strategy 3 was quite spread out and less intense. However in the phases of control, the intensity of the activity is almost exclusively present in strategy 3 which matches with what we observe in the source space. It brings a new brick of analysis pledging for this specific strategy.

Questionnaire Evaluation -Is agency too biased by gaze control ?

We wanted to assess the level of agency felt by users in the experimentation, we had previously a translation of a Van Acken questionnaire [START_REF] Van Acken | Tracking the Sense of Agency in BCI Applications[END_REF] that tracked agency in BCI protocol. We ask participants how they felt regarding the experience and their level of control over the arm. The results (Fig 4 .20) we obtained did not establish a clear difference between 15: On this note, subjects reported that they had difficulty answering certain questions and that they did not perceive the questionnaire as relevant.

strategies. On this note, we can formulate three hypothesis. The first one is that the way we design the experience in batch 1 with the downgraded feedback did not allow to assess clearly if a strategy was better perceived than the others even though, since the three strategies were based on the same calculation for the feedback, this should be limited. Moreover, the questionnaire results were high indicating that subjects kept feeling being in control despite the flawed feedback. The second hypothesis is that the questionnaire is not suited for our experience and does not provide enough information to know if a strategy is better. 15 The third hypothesis is that there can be a bias in the link between the experimenter (myself) and the subjects where they want to answer what they think we want them to answer and that for all strategies making it impossible to know what they truly feel.

Discussion on Batch 1

Results from batch 1 are halftone for all the reasons evoked before. Nevertheless, if we reflect upon the work done, they were absolutely necessary to improve the protocol, and a lot was learnt thanks to this first dataset. Here, we are going to discuss those results, how they can fall within the general BCI framework and what they tell of some brain mechanisms.

Performance result

It is not relevant to base our analysis on the online sensitivity which is more a sum of True Positives and False Negatives. In this case, it seems more appropriate to look at the data from the offline perspective. It was observed that the 𝑅 2 coefficient increased for most of the subjects from the training in the calibration phase to the driving phases. This results in a 16: In the batch 1, the sense of agency is quite relative, because even if the subjects have indeed an influence on the feedback, the way it is computed limits the effect.

[117]: significant improvement of the accuracy between phases for all strategies. This improvement can take its root from two different origins. First, it can be the result of the training effect during session as presented for instance by Alimardani et al. [START_REF] Alimardani | Brain-Computer Interface and Motor Imagery Training: The Role of Visual Feedback and Embodiment[END_REF] concerning motor imagery learning skill, linked to what was introduced as the eureka effect which is that subjects need some time to understand the nature of the task and to know how to create perceivable differences between their motor imagery and resting state. Second, the driving phases creates an additional sense of agency 16 with the knowledge that their brain activity has a direct influence on the feedback, this was mentioned by Skola et al. [START_REF] Škola | Progressive Training for Motor Imagery Brain-Computer Interfaces Using Gamification and Virtual Reality Embodiment[END_REF]. This change in the interaction modifies the brain activity which ends up modifying the features that become more discriminant. On this note, it is plausible that it is a combination of those two origins that produces this noticeable effect.

When comparing the performances between strategies, we find significant differences in sensitivity for a subset of subjects 17 between strategy 1 and 3. When computing on the complete batch 1 dataset with a selection of feature by hand, the differences are not significant anymore. Nevertheless, the trend points in the same direction indicating an advantage for strategy 3 in both phases. This is also confirmed when we compute the accuracy with the Riemannian geometry approach that prevents from possible biases of features selection by hand as mentioned on another topic (evaluating biases when selecting features to discriminate genes dataset) by Krawczuk et al. [START_REF] Krawczuk | The feature selection bias problem in relation to highdimensional gene data[END_REF]. Even though we cannot conclude on a strategy being better than the rest from a performance perspective, it is reassuring to see the same trend between classification methods and also that our approach to the protocol is relevant. Indeed, having 2 phases of control where we train 2 times in the experiences allows to obtain better features.

Neurophysiological result

In this section we will focus on the calibration as the main result. In the calibration phase, subjects control the arm with gaze and receive positive feedback after the motor imagery task (the robot's gripper closes). We observe differences of distribution in the sensor space from a strategy to the next. Clusters were only found in strategy 3. It contributes to the idea that this strategy is relevant to integrate in the multimodal framework. This is reinforced by the fact that it is the same strategy that presents higher offline performances. If we take a look at the time frequency analysis we find again that this strategy is responsible for activating more regions of the brain and for a longer time period.

For the phases of control 1 and control 2, it is more difficult to conclude for the reasons we mentioned before. However, we can still indicate that strategy 3 remains consistent from a phase to the next and that overall, strategies involving the robot in the loop (strategy 2 & 3) present a higher distributed activity than the strategy which follows the main BCI standards (strategy 1).

From a functional connectivity perspective, we show that for the three strategies, we observe an increase (from resting state to motor imagery) of node strength in the sensori motor regions in the calibration phase at significant levels which corresponds to what was described by Van Wijk et al on neural synchrony during motor action [START_REF] Van Wijk | Neural synchrony within the motor system: what have we learned so far?[END_REF]. Strategy 3 presents a stable patterns across phases at the highest t-values. S1 and S2 are however triggering activity profiles in the second phase of control. It is interesting to note that using another approach on the EEG data, we keep finding the same profile indicating a consistency in strategy 3 and a variability in the phases especially for strategy 2. This could be the mark of the apparition of the feedback which affect more strategy 2 than strategy 3. Strategy 1 is the standard of literature (movement of the robot after the MI task) and the profile in node strength indicates fewer nodes connected. It is to note that the control which is going to be present in the batch 2 is going to reveal more discriminant differences between strategies.

Because we can observe broader activity across electrodes both in ERD/ERS and functional connectivity for the two strategies involving the robotic arm during the cognitive tasks, it might be the mark of this interaction. The broad activity observed at the sensor level does not appear clear at the source level. In the driving phases, we can observe activity around the sensori motor cortex, more localized in the premotor cortex area involved in simulation of movements. We can evoke two plausible explanations to the spread out of the activity. First, from a pure statistical point of view, the number of subjects is maybe not high enough to highlight effects of strategies, this was mentioned by Pernet et al on their evaluation of the cluster permutation test on similated EEG data [START_REF] Pernet | Cluster-based computational methods for mass univariate analyses of event-related brain potentials/fields: A simulation study[END_REF]. Second, it might be that the absence of congruent feedback provokes mental strategies exploratory behaviour from subjects who try to obtain one way or another a response from the system.

It is important to note that by doing the group level analysis, subjects who produce the strongest desynchronization can mask possible patterns on the weakest ones. However, we normalize the difference of power spectrum between motor imagery and resting state which means that the effect will be limited. The reasons why some subjects perform better than others is still unclear in the literature, it is known that subjects with good proprioception, having athletic background or dancing abilities or manual activities perform better but it is not certain if it is because they are better at doing the motor imagery task, or the resting state or both.

Those results cannot suffice to conclude on the relevance of one strategy being better than the other but it highlights the idea to study those differences which seems important. With a refined version of our protocol, we can present some key information regarding the control of a robotic arm using a multimodal BCI and how to design those types of control.

A possible way to create sham experimentation

To assess the relevance of a method, it is necessary to see how it places itself in the literature. But, this can become biased because of poor control design [START_REF] Mansour | Efficacy of Brain-Computer Interface and the Impact of Its Design Characteristics on Poststroke Upper-limb Rehabilitation: A Systematic Review and Meta-analysis of Randomized Controlled Trials[END_REF]. Indeed, if the control group of a study is too different, we end up comparing results with two different experimentations and we cannot really assess what is causing the differences of those same results.

A possible solution to assess the relevance of a protocol is to use a sham design where the feedback is the result of chance, the principles of sham design were described by Miller and Kaptchuk [START_REF] Franklin | Sham procedures and the ethics of clinical trials[END_REF] to explain how it relates to the placebo effect. All the framework is the same except this part. It allows to see if better results are linked to the control or to the experimentation itself. The only thing is that if the subjects know that they receive a sham feedback, they are biased and disengaged in the task (they need to believe that they are in control). Designing within-subject sham condition is tricky as they should be blind to this control condition so should be the experimenters. On this He et al. [START_REF] He | Neurofeedback-linked suppression of cortical beta bursts speeds up movement initiation in healthy motor control: A double-blind sham-controlled study[END_REF] investigated a trial to trial sham approach to investigate the impact of the neurofeedback. Our approach, a discrete feedback based on the average of a continuous feedback (ACF) ( the average result of classification over trial) brings a certain balance to this effect. It introduces chance because of what we said before but it also convinces subjects that they are still in control (because to a certain extent, they do).

To validate this method as a new sham, it would be nevertheless necessary to create an experimentation where 3 methods are used :

▶ A random closing of the gripper or closer to reality a closing linked to a previous session or from a different subject. ▶ Average of continuous feedback (ACF) ▶ A discrete feedback computed over the trial -Over trial discrete feedback (OTDF) and test how the subjects' responses evolve.

General discussion on the results

Subjects of batch 1 received a very poor reward comparatively to the effort they put. We could easily say that those poor performances which were not the product of their doing induced frustration an disbelief in the system. Nevertheless, subjects ERD/ERS kept being present through the experimentation indicating that despite not being presented a rewarding, they continue to do the task. In that sense, their frustration was contained by their motivation and when the robot was closing, the reward appeared as even greater. Skola et al. already described how motivation played an important role in BCI experience [START_REF] Škola | Progressive Training for Motor Imagery Brain-Computer Interfaces Using Gamification and Virtual Reality Embodiment[END_REF]. To temper this, throughout the sessions, subjects were shown their motor imagery vs resting state patterns during the breaks between runs. This helped them in being confident in what they were performing as well as being less affected by the negative reward when the robot is not closing the gripper. Overall, it seems that their motivation did not suffer too much from this lack of reward. This can be explained by the engaging environment, the higher mechanism of reward it puts in place or even the natural motivation of those subjects.

The design of the experimentation was meant to ensure a certain level of agency thanks to the use of gaze to control the position of the arm. This was in the fear that the robotic arm would not close as often as expected resulting in the decrease of this same sense. What the questionnaire revealed was that we could obtain stable sense of agency even though the robot was not closing as often as it should. Therefore, we can with caution say that the experimentation tricks enough the sense of agency on the robot behaviour to limit the negative effect the robot can have when it is not closing.

18: In the sense that this always occurs. On another note, the offline performances revealed themselves to be quite high (SVM and Riemannian geometry) pushing on the idea that the task was properly executed. Moreover it says something of the feedback, indeed, it is not so much the feedback that is important as the fact of being in charge.

Finally, the neurophysiological analysis helped to show that strategies did not show the same pattern at the group level both spatially, temporally and in connectivity. It points towards strategy 3 as being the one presenting the most interesting desynchronization in the sense that it seems to be stable from a phase to the next (calibration to control 2).

Drawing the first conclusions -Improving the protocol

After analysing the first subjects and establishing that there was something different between strategies (from an offline perspective), it was necessary to take a step back and reflect on the dataset. The differences between the offline performance and online performance were too striking to be the only fruit of natural 18 differences which are inherent to the passage from offline to online. Furthermore, the high variability of performances even for subjects that had 𝑅 2 maps that showed statistical differences between MI and resting state made it impossible to validate the protocol with certainty.

After carefully analysing the steps of the classification process, it appeared clear that the way we computed the feedback was not the most appropriate. This was mentioned by Perdikis et al in [START_REF] Perdikis | Evidence Accumulation in asynchronous BCI[END_REF] on the variability of the classifier directly linked to the oscillatory nature of the spectral information. This led to a major change in the OpenViBE scenario to compute the power spectrum over all the trial to create a single vector of features instead of a feedback computed based on the average of the choices of the classifier over trial. Fig 4.21 shows the differences between the two methods. In addition to that, the HappyFeat software had been improved which resulted in a easier and more straightforward to use design of the pipeline.

From the questionnaire perspective, some key elements should have been addressed in order to keep it and to use the results. Moreover, a more thorough conception of the questions should have been conducted as well as a more intensive search of the literature to support the claim we would do. For all those reasons as well as the results obtained that did not show any trend, the questionnaire was not used for the batch 2.

All of this taken into consideration, despite the feedback part that casts a shadow on the dataset, some relevant observations should be made from the first batch. First, with a refined process, some differences between strategies are found from a performance perspective. Those differences are even more present and highlighted from neuro physiological point of view with spatial, spectral and temporal differences. So these encouraging results push to go forward but with a revised protocol. Furthermore, we can already mention that strategy presents a behaviour consistent throughout the experimentation and that the robotic arm tends to elicit a broader desynchronization across electrodes when comparing cognitive tasks. Also, the node strength analysis reveals a dense brain network especially in strategy 3. ), then at the end of a trial, the sum of the distances to the hyperplane gives a positive or a negative value resulting in the closing of the gripper. This method is sensitive to power spectrum variations. Bottom, the computation of the feedback for batch 2, the average PSD over time windows for each feature is classified by the LDA, only one hyperplane distance is output, either negative or positive. This method is less sensitive to power spectrum variations over trial.

Movement and time -Investigating their role in an hybrid BCI

Ts:Every time science is taking a new step, it is because a fool pushed it by accident.

Mental imagery timing affects hybrid BCI control of robotic arms 5

Chaque fois que la science avance d'un pas, c'est qu'un imbécile l'a poussée sans le faire exprès.

Lazare character from Emile Zola, La joie de vivre, 1884 (chap. IX)

Key Results

▶ Results and discussion from the main contribution of the thesis. ▶ Offline performance and online evaluation of the second batch of subjects. All timings/strategies presented high performance both offline and online using various techniques advocating for the use of such device to elicit brain patterns usable by machine learning algorithms. ▶ Eye tracker analysis of the second batch. Establishing that timings/strategies have an effect on pupil dilation. ▶ Evaluation of correlation between performance and motivation in the second batch. ▶ Neuro-physiological analysis of both batches through ERD/ERS and node strength based on spectral coherence functional connectivity evaluated with cluster based permutation test. Establishing difference of patterns at the group level between strategies and highlighting the consistency of one timing/strategy (3) over the rest.

Protocol summary

▶ Position reached by the robot thanks to gaze position (eyetracker) ▶ Closing of the gripper based on results of the classifier (Discrete feedback of the MI BCI). ▶ Stimuli in an augmented table with a blue disk indicating resting state and red disk indicating MI via OpenViBE software. ▶ Neurofeedback during the 3 seconds of MI ▶ 3 sessions for 3 strategies of control over the arm 1. Robot reaches target after subject's cognitive task. 2. Robot reaches target meanwhile subject's cognitive task. 3. Robot reaches target before cognitive task.

The results we obtained so far were shadowed by the feedback given to subjects of the first batch. The principal correction made is the computation of the feedback. Instead of issuing a choice for every power estimation sample over a MI/Rest trial and summing the choices to get the feedback (closing or not the gripper at the end of the trial), we do the average power estimation over the MI/Rest trial and issue a single choice from the classifier to get the feedback. In addition to that, we use an updated and optimized version of HappyFeat.

Here, based on the corrections we implemented, we propose the final results and discussion of a study on 15 subjects that performed the three strategies of control with the best methods of control we could come up with. The results spotlight how one specific strategy appears to be relevant from many perspective, by being consistent from one phase to the next and also consistent throughout our metrics of analysis. In a seconde time, we discuss those results and enrich the discussions we had on the first batch to answer the different hypotheses we had when the protocol was designed.

Results

Behavioral performance

Figure 5.1: Braccio protocol -a multimodal BCI platform experimentation. a) BCI platform composed of the 64 EEG BrainAmp device, the Tobii Pro Glasses 3 Eyetracker, the augmented table with the display monitor underneath the glass and Pollen Robotics's Reachy arm facing the subject. The robot goes to the desired can position accoordingly to the gaze position and closes its gripper based on motor imagery activity. b) The three different timings of control investigated in the protocol, strategy one (in green) -performing cognitive tasks before the robot reaches the can, strategy two (in red) -performing cognitive tasks meawhile the robot reaches the can, strategy three (in blue) -performing cognitive tasks after the robot reaches the can.

Fifteen healthy right-handed subjects (8 females) participated in a randomized longitudinal EEG study consisting in controlling the reachand-grasp action of a robotic arm via a hybrid-BCI (Fig. 5.1a). The goal was to use the eye-gaze to select a target object and grasp it by means of a right-hand motor imagery (MI). Across sessions, subjects were instructed to perform the MI task in different moments, i.e., 1) before, 2) during and 3) after the reaching phase (Fig. 5.1b). Each session started with a calibration, where subjects were instructed through a visual cue prompted on the table monitor to perform several trials of MI (grasp) and resting state (no-grasp) tasks (Fig. 5.1a). At this stage the robotic arm reached the target and the grasping depended on the given cue and not on the recorded brain activity (neurofeedback off). After selecting the most relevant controlling EEG channels in terms of discriminant power spectra, subjects performed the same task but the robotic hand action was now controlled by the brain activity (neurofeedback on). Based on the input controlling features, a linear discriminant classifier determined the type of action, i.e. grasp/no grasp. Two consecutive control blocks were then realized to allow subjects practicing and learning the task.

To assess the role of the intrinsic subjects' motivation on their ability to control the BCI, we first measured their reward/effort ratio via an online questionnaire before the experiment [START_REF] Pessiglione | Why not try harder? Computational approach to motivation deficits in neuro-psychiatric diseases[END_REF]. Results showed that the highest classification accuracies (correct/total trials) tended to be reached by the most motivated subjects (𝑅 = 0.683,𝑝 = 0.007, Fig 5 .22a). Then, we investigated how subjects became proficient and whether one timing strategy gave better performance. In average subjects exhibited a significant learning effect across the control blocks regardless of the timing strategy. However, only strategy 3 gave a significantly higher accuracy at the end of the session (Fig. 5.2b, Tab5.2). In terms of sensitivity (correct/total MI trials), the scores were in general very high (> 83%) and no significant effects were reported across blocks or strategies. Average Accuracy in the two phases of control for each strategy for the subject batch that received feedback. The two-way Anova test was performed between phases and strategies, test revealed that the phase was the only significant factor at 𝑝 < 0.0002, in the post-hoc analysis with bonferonni correction, significant difference was only found between control phases of strategy 3 at 𝑝 < 0.003.

Motor-related spatiotemporal brain dynamics

To understand how the brain responded to the different timing strategies, we focused on the last control block of the experiment corresponding to the best achieved accuracy in average. First, we computed the autoregressive-based power spectrum of the EEG signals corresponding to the MI and rest trials in the four characteristic frequency bands theta (4 -7𝐻𝑧), alpha (8 -12𝐻𝑧), beta (13 -25𝐻𝑧) and gamma (26 -35𝐻𝑧) [START_REF] Lara V Marcuse | 2 -The normal adult EEG'. en. In: Rowan's Primer of EEG[END_REF]. Then, we considered the trial-averaged power spectra so to have a more At the group-level, all strategies exhibited a significant beta power decrease in the MI condition as compared to the rest one, while no differences were found in the other bands (Fig. 5.3a). In terms of spatial distribution, all strategies involved the contralateral sensorimotor area of the brain, while strategy 2 further exhibited a wider extension notably including the frontal premotor regions in both hemispheres. In the source space, we could only observe a significant power decrease in the 𝛽 band for strategy 3 (Fig. 5.3b).

A more detailed time-frequency analysis of the EEG signals, revealed that strategy 2 also elicited a more temporally sustained motor-related activity as compared to strategies 1 and 3. This was particularly evident for the EEG channels in the bilateral sensorimotor area (Fig. 5.4). However in the contra lateral hemisphere in C1 (one of the most relevant electrode), the highest desynchronization was found with S3, meaning again that the intensity of the MI could be consistent to what we had previously observed spatially and in terms of performance.

This findings indicated that performing MI during the reaching phase elicits larger brain activity responses both in space and time. While strategy 2 elicited higher attentional levels as measured by the pupil diameter derivative data (Fig. 5.5), the wider motor-related activation was not associated with the best performance, which was instead achieved when MI was performed just after the reaching (strategy 3).

Brain network changes during motor imagery

To better understand the brain organizational properties in the different timing strategies, we performed a functional connectivity network analysis of the recorded EEG signals. To this end, we computed the Welch-based spectral coherence in the same frequency bands considered for the power spectra. The resulting brain networks consisted of nodes (the EEG channels) and weights links (the amount of signal synchronization between two channels). At the group-level, strategy 3 elicited a across subjects for the band (13-25Hz) in second phase of robot control with feedback (Control 2). Source space dipoles estimated using weight minimum norm (wMNE). Only strategy 3 present significant clusters (presented here at 𝑝 < 0.05, the clusters of the two other strategies are displayed but are not significant.

higher number of motor-related functional interactions as compared to the other timing strategies. While this tendency was reported in every frequency band, a stronger effect was observed for the beta band (Fig. 5.6). To quantify how those links were spatially distributed and whether they concentrated in specific brain regions, we then computed the so-called node strength which measured the total connection intensity for each node.

Here, only the beta-node strength showed significant increments in the MI condition as compared to the rest one, while no differences were found in the other bands (Fig. 5.7). Notably, the most significant EEG channels were all located over the sensorimotor area contralateral to the imagined movement and exhibited a preferential information integration with pre-frontal and frontal brain regions in the contralateral hemisphere and, to a less extent, in the ipsilateral one. across subjects for the band (13-25Hz) in second phase of robot control with feedback (Control 2). Source space dipoles estimated using weight minimum norm (wMNE). Significant clusters at 𝑝 < 0.05.

We perform the wMNE source reconstruction for all strategies in the different phases of the experimentation Fig 5 .8 to evaluate the potential activity in the different strategies. We confirm the consistency of the patterns of strategy 3. We however also find for calibration and control 1, an activity for strategy 1, however this activity seems to reduce in intensity to disappear eventually in control 2, a possible explanation is that the level of commitment needed.

Subject level analysis, finding in the inter subjects variation an additional explanation on strategy 3

Based on the observation that the strategies are not giving the same profile, we want to assess per subject at the ERD/ERS level how those changes occur the same way as in batch 1. To do so, we compute for each subject different matrix distances between the phases of the experimentation to establish per subject their variations. The matrices for each phase are the powerspectrum 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 for each electrode for bins of frequency ranging from 7 to 35 Hz. The distances computed are the Frobenius distance, the spectral norm and malahanobis distance between couples calibration -control 1, control 1 -control 2 and calibration -control 2. The different comparisons per strategy are presented in Fig. 5.9, what we observe corresponds to patterns observable at the group level with the cluster permutation test, indeed the distances between control 1 and control 2 are lower for strategy 1 and 2 than the ones between calibration and control 1 indicating that features are close between control 1 and 2. This is however not the case for strategy 3, the same way patterns were different between phases of calibration and control 1, distances remain high from the couple calibration -control 1 to the couple control 1 -control 2, and the distances get shorter when comparing calibration to control 2 106 5 Mental imagery timing affects hybrid BCI control of robotic arms indicating that the features are close between those two phases.

Source space analysis using beamformer to identify possible zones of interest Combining batches -A careful endeavour

Before presenting those results, it is necessary to be cautious because combining those datasets is not rigourous at all the levels. The batch 1 did not receive the same feedback (a downgraded one let's say) as the batch 2. Comparing the phases of drive 1 and 2 across subjects of the two batches rely on the idea that the effect of the strategy is stronger than the effect of the sense of agency. This hypothesis is not ascertainable, to do this we would need an additional protocol that establishes a control group receiving true random feedback and a one with the real feedback. In our case, batch 1 is not exactly random stricto sensu. Nevertheless, the results we observed separately in the different batches tend to be similar. Therefore, we can with caution perform the analysis. In any case, the calibration phase is the same in batch 1 and 2 so we can at least perform this analysis and conclude safely for this part. Since we have more subjects we can try to search for high significance, therefore we can lower the threshold of significant, to do so we iterated until no more significance was found.

We combined the two batches to increase our statistical power and to see if regions were activated differently between strategies Fig 5 .10.

Even though the forward model is based on the free surfer average MRI scan and we cannot have strong conclusions regarding the precise brain areas that are activated, it seems that some specific zones are active for the different strategies. We observe broader activation for strategy 1 and 2 across the left hemisphere whereas strategy 3 is more frontal. Results get more specific during the driving phase. We find that strategy 1 and 3 present a specific region of activation stable between phases with a higher negative t-value for strategy 3 but the regions are different. Strategy 1 involves the medial premotor cortex. This region is specialized in initiating movements based on internal process related to memory as described Purves et al [START_REF] Purves | The Premotor Cortex[END_REF]. This could make sense because in strategy 1, subjects are not presented any movement before or during the motor imagery task so it is more centered on users' inner state. Strategy 3 involves the left primary sensory cortex.This region is processing the somatosensory input from the thalamus and is involved in the proprioception as mentioned by Delhaye [START_REF] Benoit | Neural Basis of Touch and Proprioception in Primate Cortex[END_REF]. In our framework, this could imply that the robot displacement prior to the motor imagery task creates an integration effect resulting in an embodiment effect linked to the robotic arm. Finally strategy 2 involves the posterior partial area which is responsible for linking visual information and decision making as described by Zhou [START_REF] Zhou | Posterior Parietal Cortex Plays a Causal Role in Perceptual and Categorical Decisions[END_REF]. The fact that strategy 2 is bi-lateralized could mean that it is more demanding as it requires to both process decision to go to target, observing robot's movement and initiating motor imagery which is this time dealt by the right hemisphere with the premotor cortex. By computing additionally the node strength for the different strategies in the first phases of the experimentation Fig. 5.11, we continue to reveal that strategy 3 present the most consistent behaviour, it also shows that the other robotic interaction (strategy 2) could reveal to be active in the first control phase. Estimation of the time frequency ERD/ERS ( 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡 ) between 4 and 32 Hz using multitaper on the 4 second of trials. The ERD is evaluated using cluster permutation test across subjects in the different brain regions of the left and right hemisphere. Only clusters at 𝑝 < 0.05 are kept. We measure the size of the cluster in each strategy and normalize the value by the maximum across regions and strategies. Is displayed the normalized size of the significant regions for the different strategies.b) Brain cortical parcellation of freesurfer, are studied only a subset of regions that could be involved in motor imagery based on our previous observations of the source space.

Studying by regions the profile of the ERD/ERS allows several thing, first to further describe the characteristics of the different strategies and to verify if the consistency effect of strategy 3 that we observed is also present, second to pinpoint zones of activation that could be more involved and in doing so to give us insight on the brain behaviour temporally Fig 5 .12. The analysis focused on zones known to be involved to some extent to motor action.

We observe that in control 2, the information reveals a different profile from what we could observe in the sensor space. Indeed, it is not anymore strategy 2 that presents the broader profile but strategy 3. This could be linked to the source estimation that tends to look for more localized (and more intense) source of information to propagate at the brain level. In any case, the end results is that strategy 3 again appears to be presenting a high desynchronization in the sensorimotor area, 3 different regions present higher ERD/ERS profiles (postcentral -involved in proprioception [START_REF] Diguiseppi | Neuroanatomy, Postcentral Gyrus[END_REF], inferior parietal -involved in sensory integration [START_REF] Clower | The Inferior Parietal Lobule Is the Target of Output from the Superior Colliculus, Hippocampus, and Cerebellum[END_REF], caudal middle frontal -involved in the control of saccadic movement [START_REF] Rami | Neuroanatomy, Frontal Cortex[END_REF]). It is however surprising to see strategy 2 being activated in the calibration phase. It might be that the robotic arm displacement elicit an ERD/ERS in the early stage or that it is easier to generate MI, however this is not confirmed by the other analyses in the sensor space or in the source space.

Discussion

Mental imagery timing and intentional binding

In an effort to improve sense of agency in BCI systems leading to overall better performances [START_REF] Schofield | Characterization of the Sense of Agency over the Actions of Neuralmachine Interface-operated Prostheses[END_REF], recent development oriented themselves on the use of hybrid systems combining gaze with MI BCI. Those hybrid systems were principally investigated for the control of robots which could already reinforce sense of embodiment leading as well to better performances of the BCI [START_REF] Vavoulis | A Review of Online Classification Performance in Motor Imagery-Based Brain-Computer Interfaces for Stroke Neurorehabilitation[END_REF][START_REF] Xu | Robotic neurorehabilitation system design for stroke patients[END_REF]. Those studies focus on the feasibility and their resulting performance. More over most of the systems still rely on an intermediate monitor for MI/rest stimuli which limits the direct interaction with the robotic arm. Studies on multimodal BCI for the control of a robot limit themselves to the motor imagery phase prior to any movement of the robot. But this choice could be questioned. It is necessary to interrogate how one should integrate the mental task inside those sophisticated setup. To support the relevancy of this interrogation, we evaluated performance of three different timing of control, one where the robot moved after the whole MI process, one meanwhile and one before. Those three timings introduced different delays in the accomplishment of the gripper 's closing. We found out that the best improvement of performance occurred with the strategy with the reaching phase before the motor imagery task. This finding could find its root in the concept of intentional binding, intrinsically linked to sense of agency [START_REF] Moore | Intentional binding and the sense of agency: a review[END_REF]. Indeed, by reducing the time between the mental task and its result on the robot, we comfort the user in its intention which creates a stronger bond leading to a higher sense of agency which in the end benefit the BCI system.

Motor-related brain activity and embodiment

Embodiment has shown to play a crucial role in reaching better performance of MI BCI systems. However the associate neural correlates of embodiment are still to identify especially in EEG. Indeed, from an fMRI perspective, it seems that several zones of sensorimotor areas are involved in the process ranging from parietal to frontal [START_REF] Arzy | Neural Basis of Embodiment: Distinct Contributions of Temporoparietal Junction and Extrastriate Body Area[END_REF]. Meanwhile BCI systems tended to rely on this sense to achieve better results by assessing its level through questionnaires, studies were not oriented in defining the mental correlates of the process and how it impacted the performance especially in EEG. Our approach extensively analyzes spatial and temporal neural correlates of distinct control dynamics. Spatially, electrode count and variations between Motor Imagery (MI) and resting state power spectra signify pronounced group-level activity influenced by strategies, with profiles shifting across control phases. Source space analysis reveals that only Strategy 3 presents a significant level of activation between cognitive states in the sensorimotor cortex as well as in the premotor area known for memory-based initiation of movements [START_REF] Purves | The Premotor Cortex[END_REF] and the primary sensory area linked to somatosensory input and proprioception [START_REF] Benoit | Neural Basis of Touch and Proprioception in Primate Cortex[END_REF], it is to note that this is also reported from the time frequency analysis standpoint which contributes to argue for the integration of the robot into the brain framework. The fact that it is the only one presenting a significant desynchronization in the source space could be the mark of a more consistent behaviour across subjects whereas the two other strategies do not possess the focal activation needed for source space consistent profile.

Brain connectivity networks and BCIs

We investigated so far changes primarily in the sensorimotor cortex. Even though this measure is robust it does not take into account the interconnected nodes the brain is structured upon which makes it a network. The interconnected nature of brain behaviour is inherent, so studying local activity through ERD/ERS gives a partial information. For this reason, studying networks associated metrics could help better characterize the brain behaviour properties in its different dynamics of control over the arm. Among the different tool, functional connectivity allows to analyse the interaction between different brain regions through its sensors during motor imagery tasks [START_REF] Leeuwis | Functional Connectivity Analysis in Motor-Imagery Brain Computer Interfaces[END_REF], its measure has been demonstrated to be efficient as a correlate of performance [START_REF] Corsi | Functional disconnection of associative cortical areas predicts performance during BCI training[END_REF]. With functional connectivity associate metrics we can mention node strength which has become a common as a first characterization of a network especially to discriminate between motor imagery and resting state [START_REF] Cattai | Phase/Amplitude Synchronization of Brain Signals During Motor Imagery BCI Tasks[END_REF]. Studying robotic control in a multimodal settings has not been to our knowledge from a connectivity perspective, more over, our originality comes from evaluating different dynamics of control using this metric. Strategy 3 is the only one to showcase notable increase of node strength from Rest to MI during the second control phase, aligning itself with already observed difference of performance with regards to the two other strategies. Those results tend to demonstrate the limitations of the ERD/ERs analysis to explain why strategy 2 is giving better performance while stressing for the use of functional connectivity as a relevant performance measure. More over, functional connectivity have shown to be a signature of sense of agency as a result of connected brain regions [START_REF] Cavazzana | SENSE OF AGENCY AND INTENTIONAL BINDING: How does the brain link voluntary actions with their consequences?[END_REF]. In our case, we could be witnessing this effect of agency from a brain network perspective in the specific strategy of reaching before performing the mental task which match with intentional binding.

Gaze analysis spotlights expected attentional marker

Gaze analysis allows to determine other physiological signatures which are more stable from one subject to the next. The gaze analysis allows to extract information related to the attention and the engagement of subjects. In our case, they allow to check if some differences can be spot between strategies and if they are congruent with our neuro-physiological analysis. The trend is found from a pure physiological perspective with a gaze analysis that indicates that attention level is higher for the strategy 2 where the robot goes to the target selected by gaze during the cognitive task.

The physiological responses (the pupil dilation) indicated that the two strategies involving the robot were different from the one where the robots movement is not integrated to dynamic of the cognitive task. The eye is more involved in the robot movement, this is absolutely logic as the movement creates attention [START_REF] Kang | Pupil Dilation Dynamics Track Attention to High-Level Information[END_REF]. But also, it could be destructive, too much eye implication would mean possible motor activity linked to the eye activity resulting in an overlapping of information in the sensori motor cortex, and this comes in addition to possible artifacts. Here, the strategy 3 arrives as the right balance between maintaining the attention and the involvement of the subject, it keeps some level of restraint which allows to focus on the motor imagery and the resting state with less contamination of the gaze.

Introducing a feedback might disturb at first

subjects, and the markers of the second control could be the return to a more stable state

Mental dynamics in the context of hybrid systems, to our knowledge have not been studied especially from a neuro physiological perspective as most of the systems present proof of concept or focus on performances. We conducted a thorough neurophysiological analysis. First, from a spatial perspective, the number of electrodes and the intensity in the difference between MI and resting state tasks indicates that the strategies induce different activity at the group level and also those profiles change between phases of control. Those differences are also there from a time frequency perspective. To summarize, strategy 2 and 3 are more active and more spread out spatially and temporally in the calibration phase as well as in the second phase of control whereas the strategy 1 and 3 are more active in the first phase of control. From a functional connectivity perspective, statistical differences are found for strategy 2 and 3 in control 1 and only for strategy 3 in control 2.

The difference we observe between the different phases can be interpreted as followed, in the calibration part, we observe the situation of "pure" motor imagery with a positive feedback corresponding to the best scenario possible, this is supported by the work of Mladenovic [START_REF] Mladenovic | Computational Modeling of User States and Skills for Optimizing BCI Training Tasks[END_REF] investigating the advantages of positive feedback in BCI experience but also by the study conducted by Barbero and Grosse-Wentrup on bias feedback in BCI [START_REF] Barbero | Biased feedback in brain-computer interfaces[END_REF]. But during the control 1, we start presenting a feedback linked to the brain activity, and the machine learning algorithm is trained on the calibration phase. This introduces a perturbation meanwhile giving a higher sense of agency to the subjects, this is supported by Carabalona's paper on the attitude of subject towards feedback [START_REF] Carabalona | Attitude of the subject towards feedback and its implications for BCI use in neurorehabilitation[END_REF]. Hence the observation of different patterns in control 1. Finally, control 2 represents a return to a stable behaviour, first because subjects become accustomed to the perturbation and second, by retraining on control 1 phase the machine learning algorithm, we slightly improve the performance which makes the interaction closer to the calibration phase. All taken together, the results point towards strategy 3 as the best suited strategy of control as the different angles of analysis -performance (i), spatial ERD/ERS (ii), spectro temporal ERD/ERS (iii), source space ERD/ERS (iv) and network analysis (iv) -all contributed to highlight the relevancy of this specific timing to execute the motor imagery task.

Limitations

In addition to what was said on the Batch 1, it is necessary to assess the limitations of the overall scientific study. First, and this goes for numerous studies, the number of subjects prevent from generalizing the conclusions. We can only say that for the subjects of our batches, we observe a certain behaviour and brain patterns at the group level 1 So, it would be necessary to have a higher number of subjects to strengthen the approach.

Another limitation we have, comes from an initial choice concerning the feedback on the resting state. We voluntarily decided to not provide a feedback to not disturb the subject. But, since we are evaluating the difference between motor imagery and resting state, subjects might at the end of the day, need to be provided a feedback in order to know if they are doing the task the right way. On this, we could argue that we provided a "delayed personal feedback". Indeed, we showed to them at the end of each phase their MI vs Rest brain patterns to indicate them from a pure neurophysiological perspective if they managed to do the task. In addition to that, we also mentioned between runs of control if the classifier managed to classify correctly their resting state to orient them in doing the task if they were not doing it correctly. This approach allows to give an idea to subjects of what they are doing but is not a direct feedback from the machine.

Concerning tiredness, the experimentation is long and requires high concentration throughout the session. Even though breaks were done between runs in order to release subjects' tension and to relax them. Even though we observed in all configurations an increase of performance from control 1 to control 2, we cannot discard the hypothesis that subjects get tired at the end of the experimentation and that might affect their brain activity as mentioned in the literature [240, 241][242].

From a performance perspective, it is rather difficult to compare results between strategies. This is because the classification methods used (LDA online, SVM offline, and even Riemannian geometry offline) do not encapsulate the shape of the distribution of the activation in the sensor space. We could use certain spatial filters to further characterize the distribution of the data. For that reason, it might be more relevant to compare for instance weights issued from a CSP and its associated performance score. In a way, the analysis of the source space using beamformer is an approach to this problem -in the sense that it is a powerful spatial filter that could help in discriminating between strategieseven though the resulting data are not meant to be used by a machine learning algorithm.

The conclusion we can formulate concerning the brain regions activated during the motor imagery trials with respect to the resting state have to be tempered by the absence of subjects' MRI recordings. Indeed, head and brain morphology differ from one subject to the next, therefore, when

Hypothesis 7

We should not observe an important training effect.

Hypothesis 5

We should observe a strong ERD/ERS for all subjects in all strategies computing the forward model to access the source space, we are limited in the precision of the zones to analyse. Furthermore, the number of electrodes could be in itself an issue to perform source reconstruction.

Concerning time frequency cluster analysis, it could hide the possible temporal variability of ERD/ERS at the subject level (when they start and end there ERD/ERS) in the different strategies which in the end could explain why the time frequency results differ in the source space and in the sensor space as well as from one batch to the next.

The main limitation might be on the design of the experimentation itself. Each subject receives the three strategies in a random order to limit the bias of the training effect. But here, what it might quantify instead of the strategies relevance is the effect of the adaptation to a new paradigm of control, on this interrogation, we did not investigate how the order of the strategies could have an impact -and that could help to answer the interrogation. In this sense, we might in reality assess if a strategy is easier to adapt to which is slightly different. It might have been better to have 3 groups of subjects each training on a separate strategy and assess how they manage to control the robotic arm rapidly, on this we could also assess if the 3 groups present different brain patterns of activation.

General scientific discussion

In this work, we investigated if there is a key influence over subjects of the involvement of the robot in the time of the cognitive task in the context of a multimodal BCI. Once we established the existence of this influence we characterized it using statistical and machine learning tools which allowed to get a clearer picture of the interaction created with the robotic arm.

The information we obtained from the two batches are quite meaningful. First, regarding the possible training effect from a session to the next, we did not observe any effect on the performance nor any trend concerning the neurophysiological analysis, this contributes to answer our seventh hypothesis. We can nevertheless consider that subjects get accustomed to the task and the overall setup, allowing them to display rapidly differentiable brain pattern. On this, we can note that all subjects at some point in a relatively short amount of time (only 3 sessions) could produce the desynchronization which is the main indicator of the cognitive task being performed, it answers our fifth hypothesis but it also advocates for the use of robotic device in the loop as well as multimodal approach to BCIs. Since we observed in all batches an improvement in the performance (only offline in the case of batch 1 and both online and offline in the case of batch 2), it means that the effect of fatigue is less strong than the fact to have a phase of training and a phase of testing sharing the same characteristics (control 1 and control 2). Something else can be mentioned that joins the supposition of the eureka effect and the let go feeling. It might be possible that a certain level of tiredness is not so bad to the experimentation. Indeed, by having a certain level of fatigue, the excitation level of subjects -ergo not being fully focused on themselves or trying too much to intellectualize the task-diminishes and allows to be present in the task focusing on what is important. This let go 2: It indicates the beginning of the MI or resting trial.

Hypothesis 4

We should observe a delay in the apparition of the ERD/ERS after the stimulus presentation.

Hypothesis 3

We should observe differences between phases of calibration and control for each strategy.

Hypothesis 8

We should see an improvement of the performance from Control 1 to Control 2 both online and offline.

3: Performance of machine learning classifier are not only seen as performance in the BCI field. Indeed, because the feedback is directly linked to the classifier, improving results offline is not completely relevant. The results influence subjects in their task during the experimentation.

could be beneficial to be more aware of oneself and to perform more accurately both resting and motor imagery tasks. Performance has been demonstrated to not decrease even though fatigue increases in [START_REF] Li | Exploring Fatigue Effects on Performance Variation of Intensive Brain-Computer Interface Practice[END_REF] even though this must be linked to subject ability to perform naturally. On this note, subjects were in a separate room, where the only noises were the ones produced by the robot, they were few sources of light in the room, one small projector directed towards the robot, the screen displaying the stimuli and a circling blue light surrounding the setup to voluntarily place the subjects in a relaxing mode. This argumentation is a little speculative as quantifying the let go effect is tricky but it could be an interesting lead of development in BCI.

In the design of a motor imagery BCI experimentation, the cue is often integrated to the training of the classification algorithm 2 . The reaction time to the stimulus is not taken into account and often not mentioned. We designed our system in the hypothesis that the start of the desynchronization occurs after a certain time corresponding to the reaction after the cue, therefore we trained the classifier with 1 second of delay. We could observe this from the performance stand point where accuracy increases from the first second to the next, in doing so, we answer our fourth hypothesis . It seems that studying this reaction time in the start of motor imagery presents some relevance that could be studied in the BCI field more thoroughly. But, this raises a new interrogation. Why strategies do not have an impact on this timing ? Indeed, the robotic movement (especially for strategy 2 and 3) should induce a preparatory mechanism and an early desynchronization but this is not so much the case. We could observe from our spectro-temporal analysis that they were some differences in the start of the desynchronization at the group level but it is quite unnoticeable, and what we can basically say is that this ERD starts roughly around one second after the apparition of the stimulus. If an ERD is supposed to be observable linked to a movement, it means that subjects inhibit it as long as the stimulus does not appear and that after the stimulus' presentation they still have a certain reaction time. To add more weight to this hypothesis, subjects do not know if they will get a resting task or a motor imagery task, so they might force this inhibition in order to be sure to perform the correct task.

The first batch of analysis gave us several information regarding the protocol and the interaction between robot and subjects. First, we could establish that brain patterns evolve between phases alongside a session. This supports the relevance of re-training the classification algorithm during the session and to have 2 phases of control. This observation is also a first step to answer our third hypothesis . Related to this change, the offline analysis allowed us to say that there is an improvement of the separateness of brain patterns through phases leading to an improvement of the classification performance which contributes to answer the eighth hypothesis . However those remain offline performances. They are less usable than online performances in the BCI field 3 . More over, the first batch's performance do not allow to fully conclude on a dynamic of control being better than another. But the combination of neurophysiological analysis and offline performances permits to push the claim that strategies present different signatures which need be characterized more thoroughly.

The second batch and its analysis are the answer to this need for characteri-

Hypothesis 1

We should observe differences in terms of performance between strategies.

[ We should observe differences from a neuro physiological perspective between strategies.

Hypothesis 6

We should find that strategies involving the robotic arm (hence strategy 2 and 3) have a different behaviour than the one where the robotic arm is used at the end.

zation. After improving the protocol to obtain better online performances, we could study more in detail the influence of the different dynamics. We were limited in the conclusion regarding the performance as they became high especially in terms of online sensitivity. However, we could observe that for one of the strategies, the difference between brain patterns of the phases created a greater improvement of the accuracy, which completes our answer to the third hypothesis. Furthermore, this strategy was already the one spotted in batch 1 as the one with the most interesting trend. This partially answers our first hypothesis which remains halftone. But, it also signifies that differences in the dynamics are thin and cannot be only judged by the performance criterion which is the first indicator but lacks of subtlety, we can refer ourselves to A Tale of two learners [START_REF] Perdikis | Brain-Machine Interfaces: A Tale of Two Learners[END_REF] on the matter. The neurophysiological analysis came to endorse our observation that those mental dynamics create different response involving broader regions. To push the argumentation, we could say that the sole neurophysiological information is the basis to indicate what strategy to choose. We indeed observe major differences of activity distribution between strategies with broader profiles for strategies involving the robot (i.e strategy 2 and 3). Those observations contribute to answer our second and sixth hypotheses.

Where does consistency come from ?

Defining what is the best strategy to use is tricky. The general demonstration of the protocol is that integrating the robot in a multimodal framework is useful to obtain strong desynchronization resulting in good performances in whatever configuration. The subtlety in the protocol lies in the changes between the phases of control. Meanwhile, it stresses the need for retraining machine learning algorithm based on the changes of features, as demonstrated in batch 2 by the reduction of the differences between control 1 and 2 for the strategy 1 and 3 at the subject level. It also shows that features that are consistent throughout the experimentation are also key. Indeed, one of the many challenges in BCI is to obtain stable features or at least stable signatures [START_REF] Hwang | Evaluation of feature extraction methods for motor imagery-based bcis in terms of robustness to slight changes of electrode locations[END_REF]. Those stable signatures are important in creating robust framework, invariant (to an extent) to changes which in fine is important for new algorithms to be used especially if we think of transfer learning. In that regard, the strategy where the robot comes to the object, then the subject performs the motor imagery task and then the robot seizes it (strategy 3) demonstrated a consistent behaviour at the group level throughout the experimentation in both batches even with all the flaws of the batch 1. The constancy in the patterns of activation might be related to the framing created by the experience. Indeed, strategy 1 is maybe allowing to much liberty in the movement imagery with the lack of external stimuli (robot's placement and movement) which results in being inconsistent. Strategy 2 on the opposite side might be too framing and could be destabilizing subjects that could feel lacking of time to perform the task (this could explain why it is the only strategy bi-lateralized in the source space) as they need to watch the robot going to the target in the meantime. Strategy 3 in that sense appears to be a compromise that frames subjects by introducing a movement prior to the task with destabilizing them during its realization. On this note, we could argue something else. After checking whether the arm would seize the right or left can, subjects must integrate a waiting/expecting mechanism where they indeed expect the arm to go on the right or the left direction, during this small "checking state" they might be less willing to start right away motor imagery in strategy 2. This is not the case in strategy 3 where the time of "checking" occurs prior to the cognitive task to execute. All taken together, the two batches revealed the same effect from an ERD/ERS perspective and a connectivity perspective that pointed out towards strategy 3.

"It did not get it."

This simple sentence was often pronounced by subjects throughout the experimentation in the different batches. Those simple words indicate something very important, subjects did not consider the robot as part of their own body but as an external agent. This was to be expected from the placement of the arm in front of them but also from the dialogue we created between the two protagonists (the user and the robot). The dialogue is composed of the moment where we select the can, the robot goes to it, the subject performs the cognitive task and the robot closes its gripper accordingly. The movement, the robot's task of grasping the object and the sense of control over the arm are surely helping the users in producing the ERD but without having a complete integration.

But is this to consider as a bad thing ? Considering the robot as another actor on stage allows a certain dissociation. Indeed, the robot and the user share the same goal, grasping the can but if the robot fails to do so, the frustration is contained by the fact that the failure can be imputed to the robot and not oneself. Of course, if the robot never seizes the object, and that the patterns shown to subjects do not present any differences between the two cognitive tasks, then it is to impute to the users and they know it. But if the robot makes a mistake (i.e the classification algorithm did not rightly classify the incoming sample), the system's failure gets impersonated by this external agent. In this paradigm, subjects could be more inclined to make more effort in order to help the robot achieve its task at the next trial without suffering from the frustration of being presented a bad feedback.

Following this statement, it might interest the reader to come back to the notions of embodiment. Embodiment has been studied thoroughly in the last decades as it is puzzling to understand how the brain mixes the 3 different senses to know that it is embodied in its own body. Furthermore, it is complex to disentangle this sense from motor action. One of the approach is to use the famous protocol of the rubber hand illusion to fool the brain in thinking that a rubber arm is its own arm. Ehrsson [START_REF] Ehrsson | The concept of body ownership and its relation to multisensory integration[END_REF] demonstrated in a fMRI study of the illusion that the ventral premotor cortex played a preponderant role in the illusion. Furthermore, it was indicated by Tsakiris [START_REF] Tsakiris | The rubber hand illusion revisited: visuotactile integration and self-attribution[END_REF] that the ownership was strongly correlated with posterior insular cortex. The main limitation is that ownership does not cover fully the embodiment also composed of agency and self-location. On this Ohata [START_REF] Ohata | Sense of Agency Beyond Sensorimotor Process: Decoding Self-Other Action Attribution in the Human Brain[END_REF] showed that supplementary motor area, cerebellum, and posterior parietal cortex were involved in the sense of agency. Nahab [START_REF] Fatta | Impaired sense of agency in functional movement disorders: An fMRI study[END_REF] goes a bit further in its review finding that the correlated nodes associated to sense of agency (SoA) are pre-supplementary motor area (pre-SMA), dorsolateral prefrontal area (DLPFC), anterior insula, tempero-parietal 4: Even in the source space.

junction (TPJ), and precuneus/posterior cingulate. Even though the study is not conducted in fMRI but in EEG, it is worth mentioning Arzy's work [START_REF] Arzy | Neural Basis of Embodiment: Distinct Contributions of Temporoparietal Junction and Extrastriate Body Area[END_REF] who presents a interesting access to embodiment by visual representations and identifies as well the tempo parietal junction (TPj) and the Extra Striate body area as nodes for the embodiment process. It is to mention that the embodiment process has been thoroughly covered in the domain of prosthetic as a way to assess their level of acceptability as presented by Segil [START_REF] Segil | Measuring embodiment: A review of methods for prosthetic devices[END_REF] which details the use of fMRI for ownership, body representation and agency. This acceptability of prosthesis could be measured by an assessment of the activity in the different zones of the brain mentioned before. If we come back to our own results, it appeared that one strategy activated in the source space brain regions usually associated to. embodiment which could mean that despite a low felt acceptability as their own limb, the robot can still produce an activity echoing what a prosthesis would do.

Could we target specific zones of activation using strategies ?

The overall analysis in the source space indicated different regions being activated depending on the phases and the strategies. It seems that the strategy 1 activates in the driving phases the medial premotor cortex in the central area whereas strategy 3 activates the primary somato sensory cortex and strategy 2 has a bi-lateral activation, one in the parietal region (left hemisphere) and the other in the premotor cortex (right hemisphere). We already mentioned the possible interpretations regarding why those zones might get activated for each of the strategies. Here, we advance the idea that since those strategies present different patterns at the source level, we could target those specific regions for their activation. If we come back to the reflection on motor rehabilitation following a stroke, the zone affected by the stroke could be more precisely targeted by providing a specific strategy. In that effort, the BCI program would be even more tailored to patients to maximize their rehabilitation. This proposition contains some speculations, indeed, it might not be the right solution to target a specific zone to initiate a rewiring of the synaptic connections. An interesting aspect regarding those results is that it proposes two axes of reflection. The first concerns consistency, indeed, strategy 3 seems 4 to be the stable strategy with a region being activated in a similar manner in the three phases. Thus, to create a robust BCI, it might be interesting to keep this specific strategy. The second concerns versatility, indeed, what we have kept showing is differences between strategies profile, defining what is the one to keep might not be the right interrogation. In a way, controlling this versatility by targeting specific zones all the time might be what we should aim for and give us a better idea of the brain mechanisms in the integration of the robotic arm to its new workspace.

The Final problem, closing and opening

it is a far, far better thing that i do, than i have ever done; it is a far, far better rest that i go to than i have ever known.

General Conclusions and future developments 6

In this work, we designed a platform and a framework for multimodal brain computer interface. The platform merges different acquisition techniques for the control of a robotic arm in an augmented setup which is realised on a blended monitor in a table that creates an enriched ecological environment. Gaze is exploited for the position to reach by the robotic arm and for physiological analysis. Non invasive EEG through MI BCI is used to control the gripper closing and for neuro physiological analysis. The augmented table gives the stimuli associated to the cognitive task to perform and an additional neuro-feedback. The framework we built consists in different ways to sequence the control over the robotic arm based on the combination of the different modalities.

The platform called Braccio can serve over purposes, it has been tailored to offer adjustments based on scientific questions.

Towards a new framework ?

We have different solutions based on the platform we designed to improve the experience, we choose here to evoke two different axes of research, one more oriented towards the technology, one towards the user.

Bringing intelligence to the robot

One of the numerous point to tackle for those types of devices to go out of the safe laboratory environment is to provide the robot some "intelligence". We cannot restrain the system to preregistered objects that would be always of the same shape and at the same position. Predictors of the object shape in movement related potential via the grasping exist and could be translated to motor imagery. However, they would add another layer of complexity to the BCI system which lead to more mistake. A possible solution to avoid this problem is to give the robot a level of intelligence through computer vision to choose on its own how to grasp the object through the concept of hand-to-eye [START_REF] Bae | A dynamic visual servoing of robot manipulator with eye-in-hand camera[END_REF][START_REF] Ito | Visual tracking of hand-eye robot for moving target object with multiple feature points[END_REF]. Those system already exist and could be applied with eye tracker technology in order to issue a choice in the environment with a probability associated to the choice. The robot becomes "aware" of its own environment and can help the user in realizing the task. By doing so, we would also create a safer environment with the robot being conscious of obstacles on the way and on it could be integrated into a more dense environment. On this note, a possible interrogation we could have is on the evocative effect of certain objects to grasp. Are there objects that can elicit more the motor imagery or the resting state if they are seized by the robot in the complex environment ? And related to this, could the robot gesture towards those objects affect the user? In that framework, some augmented reality features could be used to make sure the object selected by the robot is the one desired by the user (through eye tracking). Furthermore, augmented reality could be also used to indicate the stimuli (motor imagery or resting state) if we keep within a standard MI BCI protocol.

Towards self initiated BCI, focused gaze as trigger

Based on the framework we developed, the strategy where the robot goes to the target and then the cognitive task is performed seems to be relevant to keep. We could easily make the transition from the supervised BCI paradigm to the self initiated one [START_REF] Pereira | Online detection of movement during natural and self-initiated reach-and-grasp actions from EEG signals[END_REF][START_REF] Scherer | Self-initiation of EEG-based brain-computer communication using the heart rate response[END_REF][START_REF] Andreea | Continuous detection of the selfinitiated walking pre-movement state from EEG correlates without session-to-session recalibration[END_REF]. Indeed, once the object has been selected, we could propose that the robot reaching the position is the trigger to start the detection of the motor imagery, since we know it should occur. We would oppose those patterns to the one before the selection of the object by gaze for instance. Doing so, we would create a higher level of sense of agency and we would reduce the time of detection of self initiated patterns. The scientific question could then be to investigate how to reduce this time of MI apparition with the robot gesture.

Is intuitiveness the key ?

We could interrogate the relevancy of intuitiveness in those types of interaction. Even if it is a cornerstone of the work presented here and something we have advocated in the creation of the protocol, it is necessary to discuss it. There are some striking works on the matter especially from the Plasticity lab that points in the opposite direction [START_REF] Maimon-Mor | Is an artificial limb embodied as a hand? Brain decoding in prosthetic limb users[END_REF]. Indeed, thanks to our adaptability, we can integrate new commands in our framework to perform actions. One result on this which is counter intuitive is that there is not a high increase of the cognitive load after being trained between intuitive control and not intuitive control. Coming back to my initial example of the plane, after a time you are able to fly the machine, which means you integrate the non intuitive commands in the motor framework. So, should we be intuitive if other solutions are possible ? In other words, should we design experience accordingly to this parameter, should the cognitive task be of the right hand closing to control a robotic arm, or should we even use a robotic arm in that context?

Intuitiveness serves several purposes. First, it is easy to understand the instruction right away especially in the motor imagery task. Having a comprehensive instruction which makes sense to the subject helps in having good performances right away. In a EEG experience, time itself is extremely precious and should not be wasted. The installation of the cap and the moments of recordings are long and time cannot be allocated to getting accustomed to the task for a long period. Second, having congruent tasks associated to what is given as feedback is essential to not disturb the subjects. And third, from a clinical perspective, in the case of stroke rehabilitation, the cognitive task has to be congruent with the motor restoration, if there is motor deficit in the hand, we want to restore the motor action of the hand, so the BCI needs to be based on tasks related to the hand.

All those arguments strongly pledge for the use of intuitive systems in BCI but they also admit the use of non intuitive configurations. In fact, the time for the integration of new commands is fast, vary from one subject to another and is in any case necessary to learn the motor imagery task. Second, once the task has been associated to the feedback, subjects integrate it and adapt to it 1 . On this note, it has been advocated in studies to use subject tailored MI tasks 2 . And finally, the mechanisms of motor restoration using BCI are a hard topic of research . The rehabilitation process can occur in any case -with more heterogeneity-without the BCI even though it is certainly helping to have it. An honest answer would be that we do not fully know how the brain works in integrating new commands to its framework. The underlying networks of motor actions 3 might need to be helped by giving a intuitive feedback and task associated to be integrated rapidly. Contrarly it might be that those networks have intrinsic power of adaptation and simply need to be provided a feedback to perform actions. The answer probably lies in between, non intuitive feedback might demonstrate good performances because the brain is able to adapt but intuitive feedback, easy to handle, are more comfortable at first. In the case of BCI, a tremendous challenge concerns the illiterate subjects who are not able to do right away the cognitive task. It might be possible -and it is even certain-that the threshold for intuitiveness is not the same for everyone and some might need a refined experience demanding less time of adaptation. Our protocol aimed for that, indeed all our subjects managed at some point to do the task and control the arm and they only had 3 sessions.

Robot movement, agency, binding and networks, is it telling a new story ?

This work was set with a clear course to begin with, building a multimodal BCI and interrogate the timing regarding its control. Of course, its multimodal aspect and the overall context of robotic control altered deeply our ways of thinking and our navigation across the many different fields ended up deviating our course. But, we must, at the end, return to Ithaca and to do so, we need to reflect on what we truly ended up finding. The interrogation we raised regarding the timing of the movement came from the initial notion of intentional binding that links our sense of agency to time. Agency is already an obsession in BCI and hybrid systems are built upon the idea to reinforce this sentiment. Taming agency comes from understanding what makes us agent and perceived time regarding our action appears to be essential. But, sense of agency is not just about being in control, it is strongly bound to our body. And this is paramount to understand why the sense of agency and its associate intentional binding can be observed through brain patterns of activation. We uncovered the fact that this binding between movement, intention and expected result could be observed from a network perspective compared to a traditional approach towards brain activity. But, understanding how agency, binding, brain networks and robot movement are intertwined is a mystery. Descartes in is Méditations Métaphysiques * described he was not just inside his body as the pilot of a ship, this was to signify that soul and body could not be seen as two separate entities -even though he was the one to advocate for a clear separation between the two, one more paradox ... 4 If we remove the notion of soul and replace it by the idea of the brain, the metaphor of embodiment becomes clear and it is probably essential to have in mind to think about how agency occurs and what is needed to fool the brain in activating itself the same way it would "naturally". We might not fully understand the brain mechanisms regarding embodiment and sense of agency. Yet, if we come back to Ithaca, we can surely say what is needed to build our multimodal BCI: a robot (i), an hybrid control (ii) and a precise timing to execute the mental task (iii).

Final words

It is rather strange to bring closure to a work that required so much energy, time and dedication, blood, toil tears and sweat would have said Mr.Churchill. It seems to me at my very young age a long time ago when I first encountered the notions of brain computer interface. I began my journey with the great assurance (of course too much) that I could solve many things, that as an engineer, I was just setting of for a simple quest, how hard could it be, we all have brains, we are all able to move, it should not be to hard to connect a robot to this pink porridge and we would be done by Christmas. Maybe, I am making myself more a fool than what I was but still, I was convinced that bringing robotics, movement and agency would be useful and with great optimism we would make it work. Interestingly, I have met in those years many different scientists from the field and outside, and BCI always was welcomed by either optimism, scepticism or both. I could hear researchers from the BCI community saying "it does not work", biologists saying that our engineering approach was absolutely blunt and without finesse, physicists saying that the number of parameters was too high and I admit as a simple engineer that they are absolutely all correct, and yet, we could talk about it for hours because this field is fascinating. The richest conversations I had were with a PhD student (hopefully graduated by the time I am writing) with who we questioned the reasons why we were doing research on the brain and the approaches on the field especially from the engineer perspective that shows limitations by its inability to apprehend that we do not understand everything. A thing quite strange is that the community built on not so solid foundations an entire field of research and I personally contributed to put another brick (more a roof tile) on this unstable house. Nevertheless, we all have brains and some of us tend to use it and to observe ourselves using it. And that simple sentence is probably why we continue on doing BCI. We are brain body interfaces, intertwined, interconnected, not differentiable but still in interaction. And that is why I would say, BCI is done with guts (ironically), guts that are of course relying on the knowledge of the giants but still the field is pushed this way. I find personally quite surprising that movement is not often regarded as a topic of research in BCI at least to my approach of the literature. We, of course, ask subjects to perform imagery, movement activity, but we build interactions in a strangely frozen approach. The way we interact and succeed in working with moving machines -car, planes, bikes, boats-is I think relying far more than we tend to imagine on the fact that we move and integrate their movement as ours. I allow myself two more paintings, one by Da Vinci 6.1 who defined capturing the movement as the mark of fine art in its treaty of painting, one by Kupka(Fig. 6.2) who considered movement as the elementary common form to all sensations.

We often think about the brain as a complex system but it is always centered on the brain with a clear delimitation separating sensory input, body and external environment. I would argue on this that we are maybe more on a continuum with no clear separations between the entities, in a way our interactions with the environment forms a complex being with many new properties to characterize, and machines become an extension of one self [START_REF] Lebedev | Brain-Machine Interfaces: From Basic Science to Neuroprostheses and Neurorehabilitation[END_REF]. On a final note that follows the challenge of movement integration, I am very sensitive to words we use, their roots and their meaning and often, I found that the scientific language suffocates nuances. My mentions of multimodal, imagery, resting state or hybrid have been examples throughout the manuscript of this. The interface in BCI might be misleading, its root is simple inter -between, face -form/appearance/figure/visage. The word carries the meaning of a frozen interaction where the two players are not modified which is not relevant as closing the loop is all about having the two actors -the brain and the computer/machine/robot evolving together in synergy. Defining association of those two elements might push our imagination where we do not have anymore a pilot and its ship. Beamforming is particularly valuable in functional brain mapping, as it allows researchers to estimate the sources of brain activity with high spatial resolution and improved signal-to-noise ratio compared to traditional sensor-level analyses. However, it also has its limitations, such as sensitivity to model inaccuracies and assumptions, and the requirement for accurate sensor and head model information. Careful consideration of these factors is necessary for successful beamforming analysis in neuroscience.

Burg Autoregressive method: The Burg autoregressive (AR) method is a technique used for modeling time-series data. It is a type of linear prediction method that estimates the coefficients of an autoregressive model using a method called maximum entropy spectral analysis. The Burg AR method is particularly useful when the data is characterized by a wide-sense stationary stochastic process, meaning that its statistical properties do not change over time.The Burg AR method works by minimizing the forward and backward prediction errors of an autoregressive model of the data. The algorithm starts by assuming that the time series data can be modeled using an AR model of order p. The algorithm then estimates the coefficients of the model by minimizing the prediction error of the model. This is done by iteratively updating the coefficients of the AR model to minimize the mean-squared error between the predicted values and the actual values. The power spectrum of the time series can be estimated by computing the Fourier transform of the estimated autoregressive coefficients. The power spectrum represents the distribution of power across different frequencies in the signal, and can provide insights into the underlying dynamics of the system generating the time series.One advantage of the Burg method is that it is relatively robust to noise and outliers in the time series. Additionally, it can estimate the power spectrum of non-stationary signals, which can be useful in many applications. However, it can be computationally expensive, particularly for large time series.

CAR : In EEG data analysis, the common average reference (CAR) is a method to reduce the effects of common noise sources shared across multiple electrodes. This technique is based on the idea that noise sources such as muscle activity or electrical interference are common to all electrodes to some extent, and their contribution can be removed by computing the average signal across all electrodes and subtracting it from each individual electrode signal. The CAR technique involves taking the average of all the electrode signals and subtracting that value from each electrode signal. This effectively removes any common noise that is present in all electrodes, as well as any electrical activity that is not specific to a particular area of the brain. After applying the CAR, the resulting signal is often referred to as the "re-referenced" signal. The re-referencing process can help to improve the signal-to-noise ratio and reduce the impact of common noise sources, making it easier to identify specific patterns of activity related to the brain function of interest.

Cluster based permutation test: The cluster-based permutation test is a statistical method used to determine whether there is a significant difference between two or more groups of data. It is commonly used in neuroimaging research, where it is used to analyze the differences between brain activity patterns in different conditions or groups.The method is based on the concept of clustering, which involves grouping together adjacent data points that have similar values. In the clusterbased permutation test, the data are first grouped into clusters based on their similarity. The size and significance of these clusters are then determined through permutation testing.Permutation testing involves randomly reassigning the group labels of the data and recalculating the test statistic (e.g., t-value, F-value) for each permutation. This process generates a null distribution of the test statistic, which represents the distribution of the test statistic under the null hypothesis of no difference between groups.The cluster-based permutation test then identifies clusters of adjacent data points with test statistics that exceed a predefined threshold (e.g., p < 0.05). The size and significance of each cluster are then determined by comparing its test statistic to the null distribution generated by the permutation test. If the cluster's test statistic is higher than a predetermined threshold value of the null distribution, the cluster is considered significant.The advantage of the cluster-based permutation test is that it accounts for multiple comparisons by taking into account the spatial structure of the data, which is often important in neuroimaging research. Additionally, the method is less sensitive to noise and outliers than other statistical methods, such as the standard t-test or ANOVA.However, the cluster-based permutation test can be computationally intensive and may require large sample sizes to achieve sufficient power. Additionally, the method requires careful consideration of the choice of threshold for determining significant clusters and may be sensitive to the specific clustering algorithm used.

CSP: CSP is commonly used in EEG-based brain-computer interfaces (BCIs) to enhance the discriminability of brain activity patterns associated with different mental tasks or motor actions. The main idea of CSP is to project multichannel EEG signals onto a spatial subspace that maximizes the separation between two classes of signals. In the case of BCI, these two classes correspond to different mental states or motor actions. The projection is achieved by applying a spatial filter that is designed based on the covariance matrices of the two classes of signals. The CSP algorithm consists of the following steps:

1. Collect EEG signals from two different conditions, for example, left hand and right hand motor imagery. 2. Segment the EEG signals into epochs of fixed length. 3. Compute the covariance matrices of the two conditions. 4. Compute the spatial filter that maximizes the variance ratio between the two conditions. This filter is obtained by computing the generalized eigenvectors of the two covariance matrices. 5. Apply the spatial filter to the EEG signals to obtain the CSP features. 6. Classify the CSP features using a suitable machine learning algorithm, such as linear discriminant analysis (LDA) or support vector machines (SVM).

By using CSP, it is possible to enhance the discriminability of EEG features and improve the performance of BCI systems.

Covariance matrix and tangent space logistic regression : Classification using tangent space logistic regression on covariance matrix with Riemannian geometry is a method for analyzing multichannel EEG/MEG data. It involves constructing a covariance matrix for each trial, which contains information about the statistical relationships between the signals from each channel. The covariance matrix can be considered a point in a high-dimensional space, and so Riemannian geometry can be used to analyze the properties of these matrices. In Riemannian geometry, the tangent space at a point on a manifold is a vector space that approximates the manifold near that point. In the case of covariance matrices, the tangent space at a point is the space of symmetric matrices that are close to the covariance matrix at that point. By projecting the covariance matrices onto the tangent space, we can reduce the dimensionality of the data and create a more manageable feature space for classification. Tangent space logistic regression is a method for performing classification on data that lie on a Riemannian manifold. In this case, the data are the covariance matrices projected onto the tangent space. Logistic regression is used to fit a linear decision boundary to the data, which separates the classes. The classification is performed by computing the probability that each trial belongs to each class based on its tangent space covariance matrix, and assigning the trial to the class with the highest probability. In summary, classification using tangent space logistic regression on covariance matrix with Riemannian geometry is a method for analyzing multichannel EEG/MEG data. It involves constructing a covariance matrix for each trial, projecting it onto the tangent space, and performing logistic regression to classify the data based on the tangent space covariance matrices. This method can improve the accuracy of classification by accounting for the Riemannian structure of the data.

Embodiment: Sense of Body Ownership: Body ownership refers to the feeling that one's body is one's own, and that one is located within it. It is the sense that our body is an extension of ourselves, and that we have control over it. This sense is closely linked to the sensory and perceptual information that we receive from our body, including proprioceptive, tactile, and visual information. Sense of agency: refers to the feeling that one is the agent of one's own actions. It is the sense that we are the ones controlling our body and making things happen in the world. This sense is closely linked to our perception of the consequences of our actions, and the feedback that we receive from the environment. Sense of Self-location: also known as spatial presence, refers to the feeling of being located in a specific position in space. It is a key aspect of embodiment, as it is a fundamental component of our experience of being in and interacting with the world. Sense of self-location is closely linked to the sense of body ownership, as the perceived location of one's body is a critical factor in determining one's sense of presence in the environment.

EyeTracker : The dark pupil technique is based on the fact that the infrared light emitted by the eye-tracking system is absorbed by the retina and the iris, but not by the pupil. Therefore, the pupil appears as a dark region in the image captured by the camera. The center of this dark region is considered as the pupil center. This technique is widely used because it is simple and reliable. The bright pupil technique, on the other hand, is based on the reflection of the infrared light by the cornea. The camera captures the reflected light, which appears as a bright region in the image. The center of this bright region is considered as the pupil center. This technique is less common than the dark pupil technique because it is more sensitive to changes in lighting conditions and requires more complex algorithms to locate the pupil center accurately.

Functional Connectivity: Functional connectivity refers to the statistical associations between different brain regions or networks. In EEG analysis, functional connectivity can be assessed using coherence, which measures the consistency of the phase relationship between two signals at different frequency bands. Coherence is a measure of linear correlation between two signals, and it varies between 0 (no coherence) and 1 (perfect coherence).Spectral coherence is the coherence between two signals at a given frequency, and it is often used to study frequency-specific functional connectivity in EEG. Spectral coherence can be calculated using the Fourier transform of the signals, which reveals the power spectrum of the signals at each frequency.On the other hand, imaginary coherence is a measure of non-linear correlation between two signals. It is calculated by first computing the complex coherence, which is the coherence between two signals at a given frequency that includes both magnitude and phase information. Then, the imaginary part of the complex coherence is extracted, which reflects the phase difference between the two signals. Imaginary coherence has been shown to be more sensitive than spectral coherence in detecting non-linear interactions between brain regions.In summary, spectral coherence and imaginary coherence are two complementary measures of functional connectivity in EEG. Spectral coherence is useful for studying frequency-specific interactions between brain regions, while imaginary coherence is more sensitive to non-linear interactions.

Inverse Kinematics: In robotics and kinematics, inverse kinematics is the problem of determining the joint parameters that will achieve a desired end-effector position and orientation. The problem is typically formulated as finding a function that maps the Cartesian position and orientation of the end-effector to the joint parameters of the robot. One common approach to solve the inverse kinematics problem is to use the Jacobian matrix, which relates the velocity of the end-effector to the velocity of the robot's joints. Specifically, the Jacobian matrix maps changes in the joint angles to changes in the position and orientation of the end-effector. To solve for the joint angles that correspond to a desired end-effector position and orientation, one approach is to invert the Jacobian matrix. This involves solving a linear system of equations that relates the desired end-effector velocities to the joint velocities required to achieve them. However, the Jacobian matrix may not always be invertible, and even when it is invertible, the inverse may not be well-conditioned or numerically stable. In these cases, other techniques such as gradient descent or optimization-based methods may be used instead.

LDA: Linear Discriminant Analysis (LDA) is a classical supervised machine learning technique used for dimensionality reduction, classification, and feature extraction. It is widely used in pattern recognition, image processing, and signal processing. The goal of LDA is to find a linear combination of features that maximally separates different classes of data while minimizing the variation within each class. Specifically, LDA finds a projection of the data onto a lower-dimensional space that preserves the most discriminatory information between classes. This projection is achieved by maximizing the ratio of the between-class variance to the within-class variance. In LDA, the input data is a set of n samples, each with p features, belonging to one of k classes. The algorithm first computes the mean vectors and covariance matrix for each class, and then computes the pooled within-class covariance matrix. The projection matrix, which maps the original p-dimensional space into a lower-dimensional subspace, is then obtained by solving a generalized eigenvalue problem. Once the projection matrix is computed, LDA can be used for classification by transforming the data into the lower-dimensional space and applying a simple decision rule, such as the nearest centroid rule or a linear discriminant function. LDA has several advantages, including its simplicity, effectiveness in high-dimensional settings, and interpretability of the results. However, it assumes that the data is normally distributed and that the covariance matrix is the same for all classes, which may not be the case in practice.

Laplacian filter: Laplacian filtering involves computing the second spatial derivative of the signal at each electrode location, which is equivalent to computing the difference in activity between neighboring electrodes. This has the effect of emphasizing local changes in the EEG signal while suppressing global changes that affect all electrodes in the same way. The main advantage of Laplacian filtering is that it can help to improve spatial resolution by reducing the blurring effects of volume conduction, which can make it difficult to localize the sources of EEG activity accurately.

Mimimum Jerk and Robot Trajectories:

The minimum jerk model is a widely used mathematical model to describe human movement. The model proposes that the human nervous system is optimized to minimize the rate of change of acceleration (jerk) during motion.According to this model, when a human performs a movement, the trajectory of the movement is not a straight line, but instead follows a smooth and curved path. The velocity and acceleration of the movement are controlled in such a way that the rate of change of acceleration (jerk) is minimized.The minimum jerk model has been found to accurately describe a wide variety of human movements, from reaching and grasping to walking and running. It has also been used to design robotic and prosthetic devices that mimic human movements.Overall, the minimum jerk model provides a useful framework for understanding the underlying principles of human movement control and for designing devices that can assist or augment human movement.The minimum jerk trajectory is a popular way to generate smooth and natural-looking trajectories. Inverse kinematics using the minimum jerk can be done by finding the joint positions that result in a minimum jerk trajectory that passes through a set of desired end-effector positions. Here are the basic steps to do this:

1. Define the desired end-effector positions: The first step is to define a set of desired end-effector positions that the robot needs to reach. These positions are usually specified in the task space. 2. Generate the minimum jerk trajectory: Next, a minimum jerk trajectory needs to be generated that passes through these desired end-effector positions. The minimum jerk trajectory can be parameterized by time and can be expressed as a function of time. The trajectory is characterized by its initial and final positions, velocities, and accelerations, and its duration. 3. Compute the inverse kinematics: Once the minimum jerk trajectory is generated, the inverse kinematics problem needs to be solved to determine the corresponding joint positions that result in this trajectory. This involves finding the set of joint angles that place the robot's end-effector at the desired positions at each point along the minimum jerk trajectory. 4. Solve the optimization problem: Inverse kinematics using the minimum jerk can be formulated as an optimization problem, where the objective is to minimize the difference between the actual joint positions and the joint positions that result in the minimum jerk trajectory. This can be achieved by using an optimization algorithm, such as gradient descent or a genetic algorithm. 5. Update the joint positions: Finally, the joint positions are updated based on the results of the optimization problem. These joint positions are then used to control the robot's movements to follow the minimum jerk trajectory and reach the desired end-effector positions.

PSO: Particle Swarm Optimization (PSO) is a metaheuristic optimization algorithm inspired by the social behavior of bird flocking and fish schooling. The algorithm maintains a population of particles that move in the search space, trying to find the optimal solution.Each particle in the swarm represents a candidate solution to the problem being optimized, and its position and velocity in the search space are updated at each iteration based on the best solutions found by the particles itself and the entire swarm. The position of a particle in the search space is represented as a vector of real-valued numbers, and its velocity is a vector of the same dimensionality. At each iteration, the velocity of each particle is updated by a weighted combination of its current velocity, its personal best position, and the global best position found by the swarm. This update rule allows the particles to move towards promising regions of the search space. The new position of a particle is then obtained by adding its updated velocity to its current position. The personal best position of each particle is updated if its current position improves its fitness, while the global best position is updated if any particle in the swarm finds a better solution than the current global best. The algorithm continues until a stopping criterion is met, such as a maximum number of iterations or a satisfactory solution is found. The final solution is then given by the best position found by any particle in the swarm. PSO has been successfully applied to a wide range of optimization problems, including feature selection, neural network training, and parameter optimization for machine learning algorithms. One of the advantages of PSO is its simplicity and ease of implementation, but its performance can be sensitive to the choice of algorithm parameters, such as the number of particles and the weight parameters used in the velocity update rule.

𝑅 2 test: When comparing conditions in a neuroimaging study, researchers often use R-squared maps to examine the differences in brain activity between the conditions. R-squared (𝑅 2 ) represents the proportion of variance in the data that can be explained by the independent variable(s). In this case, the independent variable is the condition being compared. To generate an𝑅 2 map, the data are first analyzed using a statistical method, such as a general linear model (GLM), to estimate the parameters of interest, such as the mean activation levels for each condition. The 𝑅 2 map is then generated by calculating the proportion of the total variance in the data that is accounted for by the model, which reflects the degree of difference between the conditions. A high 𝑅 2 value indicates that there is a large difference between the conditions in terms of brain activity, and that the model is able to explain a large proportion of the variance in the data. This suggests that the independent variable (i.e., condition) is a strong predictor of the dependent variable (i.e., brain activity). 𝑅 2 is defined as follows :

𝑟 2 = 𝑐𝑜𝑣(𝑥,

II. Materials

▶ Connect all devices: computer, battery, amplifier, BrainAmp, and electrodes. ▶ Turn on computer, launch Acticap software. ▶ Connect optical fibers (EEG1-32 to 1, EEG33-63 to 2). ▶ Open relevant xlsx file (PreAnalysis_Sub_XX_Sess_XX.xlsx) and complete required details. ▶ Connect cap and electrodes as specified in EEG input box.

III. Quality Check

▶ Launch Acticap software. ▶ Check impedance using Acticap software or visually via OV. ▶ Load Braccio.rwksp workspace to ensure proper setup. ▶ Save impedance and Acticap software information.

IV. Information

▶ Complete "Infos" section in xlsx file. ▶ Fill out first part of sheet for current visit.

1: Strictly speaking, from a software engineering perspective, real time is not achieved since the data is buffered and computation establishes a delay between the original signal and its back end. One must deal with this problematics when building the BCI system. ▶ A server dealing with the acquisition of the EEG device through the different protocols of communication between the hardware and the computer, it can be LabStreamingLayer(LSL) in the most common situations. The server broadcasts the EEG signal to the designer.

▶ A designer which serves two functions, it is the pipeline of data treatment and collection and also the executing system that creates the BCI through visualization/audio feedback or broadcasting (through LSL/TCP,UDP protocols) to external devices.

The designer is a graphical programming language relying on boxes that executes specific functions of treatment on the original EEG signal. Those boxes communicate together via the transfer of specific buffered data in a pseudo real time manner. 1 The coding of those boxes structured around different functions.

▶ Initialization -creation of the attribution of the variables values. ▶ Uninitialization-destruction of the variables. ▶ Process Input -The incoming data arriving to the box. ▶ Process -the computation performed on the incoming data.

Of course those functions may vary based on the different treatment done on the data and different call to over processes and libraries are necessary (such as Eigen).

E Additional Results

E.1 Batch 2 additional results on performances

In an effort to assess the quality of the dataset, we perform an offline analysis on the data using Riemannian geometry approach with the tangent space logistic regression between the calibration phase and the drive 1 phase as well as between drive 2E.1. The results were outstanding in terms of accuracy (between 85 to 90 % of accuracy between control 1 and 2).

Between sessions analysis

We already show the fact in batch 1 that the session did not seem to be the main effect ergo the training of subjects is limited. Here we come back to this statement because the batch 2 presented the subjects a higher sense of agency since the classification method was more appropriate and we could expect changes. From a pure performance point of view (Fig E .2, we do not observe any significant differences between sessions nor observe any trend that would indicate an improvement linked to a training effect. In addition to that, we performed the same analysis on power spectrum using the cluster permutation test to be sure that we would not miss any substantial information. In the calibration phase, we did not find any differences between sessions, and for drive 1, session 1 presents higher activity than the two others. In drive 2, we observe a wider distribution for session 3 but the differences between sessions are not tremendous. But, it is possible there might be an effect in drive 2 of the training even though it does not appear to be striking.

Evaluating the motivation per strategy

We wanted to evaluate between strategies how the motivation score can be correlated to the performance to see if some effects were observable.

To encapsulate the evolution of performance, we defined for each subject and each strategy the slope between accuracy of drive 1 and drive 2 (we also do it for the sensitivity). We only could observe high correlation between strategy 3 and the slope both in accuracy and sensitivity.

Figure 1 :

 1 Figure 1: Sunset in Tuscany, among friends, 2023

Errare humanum est, sed perseverare diabolicum 4 first study 1 1 . 1 2 . 1 2 . 2 2 . 3 4 5 9 2 3

 411121222345923 Forging the protocol through experience -General observations on subjects and Batch 1 Sunset in Tuscany, among friends, 2023 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Braccio platform: a multi modal BCI approach for the control of a robot . . . . . . . . . . . . . Brain structure: The different lobes of the brain, lateral representation. . . . . . . . . . . . . . . Brain cells schematic: Glia representation alongside the different cells interacting, image from[START_REF] Allen | Glia -more than just brain glue[END_REF]. Neuron schematics: Detailed representation of the neuron composed of its nucleus, axon and synapse, image from[START_REF] Dharani | Chapter 1 -Functional Anatomy of the Brain[END_REF]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.From initial source to non invasive sensor: Diagram representation of the transfer of information from the cortex to the scalp, image taken from [8]. . . . . . . . . . . . . . . . . . . . . . . . . . . 2.System of place of EEG sensors on the scalp: 10-10 International system, adapted from Laurens R. Krol. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.6 SSVEP pattern: Typical SSVEP response of an EEG signal acquired during visual stimulation at 9 Hz, Figure from [13]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.7 ERP pattern: Representation of the event related potential after presentation of a stimulus of an EEG signal, Figure from [15]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.8 ErrP patter: Typical Error Related Potential response after presentation of an error in a feedback, Figure from [17]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.MRCPs pattern: Movement related cortical potentials from an healthy subject for real imaginary movement of right ankle dorsi flexion, Figure from [18]. . . . . . . . . . . . . . . . . . . . . . . . 2.10 ERD/ERS pattern: Typycal Event Related Desynchronization/Event Related Synchronization response after the generation of a motor movement, Figure from [22]. . . . . . . . . . . . . . . . 2.11 Power amplitude linked to neurons activity Simulation of the relation between power spectrum amplitude and connectivity between neurons, Figure from [21]. . . . . . . . . . . . . . . . . . . 2.12 Motor imagery,Motor Action,Motor observation: Brain activity for the same motor action but either imagined, performed or observed. Image from [51]. . . . . . . . . . . . . . . . . . . . . . 2.13 Cockpit evolution: On the left a Supermarine Spitfire from 1941, on the right a EuroFighter Typhoon from 2015. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.14 CAR modification on signal: Before and after applying the Common Average Reference filter to the EEG data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.15 PSD between MI and Rest: Typical difference of power spectrum between motor imagery and resting state for a given electrode, computed with Burg auto regressive method. . . . . . . . . . 2.16 Riemannian manifold representation for covariance: Representation of the covariance matrix as a Riemannian manifold and the local projection on the Euclidean tangent space. . . . . . . . . . 2.17 Layers of a NN algorithm: Representation of the steps of manipulation of the EEG data in a deep learning architecture, on the left the different layers of the artificial neural network, on the right, the representation of the successive transformations. . . . . . . . . . . . . . . . . . . . . . . . . 2.18 BCI2000 visualization: BCI 2000 early visual stimuli and feedback, the yellow signifies the target to go to. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.19 Graz Visualization Graz visual stimuli and feedback, the blue bar evolves with time in function of the classification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.20 Kinematic chain : Representation from the base to the end effector of a robot arm. . . . . . . . 2.21 Robot arm opposed to human arm: Difference of joints and degrees of liberty between a human arm and a robotic arm, Figure from [125]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.22 Eye musculature: Muscle and eye anatomy, figure obtained from Elsevier's complete anatomy software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.23 Effects of light on pupil: Representation of the different pupil effect based on the placement of the light with regards to the eye. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.24 Blinks: Typical signature of blinks in EEG signal. . . . . . . . . . . . . . . . . . . . . . . . . . . 2.25 Questionnaire Score: Average score of the sense of agency for the different tasks. * * * : 𝑝 < 0.001, Wilcoxon test. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.26 Decrease power among subjects: Occurrence from one subject to another of negative t-values at p<0.05 per electrodes for each comparison. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.27 Comparison between conditions in the sensor space: T values between the conditions Control by gaze and No Control in the most interesting frequency bands (𝛼 or 𝛽) with FDR correction, 𝑝 < 0.05. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Left: Position towards an object to reach by the robot based on gaze. Right: Grasping of the object based on the MI BCI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.Pillars of development: Principles for the subject upon which we must build the BCI platform : Trust, Agency, Involvement and Disturbance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.Continuous feedback with a robot:Iterative closing of the gripper based on the successive samples classified as either motor imagery in red or resting state in blue. The illustration shows two different features considered as the feature vector. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Discrete feedback with a robot: Closing of the gripper at the end of the trial based on the average

3 . 9

 39 Photography of a bird flock, Author: S.Solker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.10 PSO: Representation of the Particle swarm algorithm searching for the optimal negative position in a 3d gaussian via its 3d particles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.11 PSO in the selections of AR parameters: Representation of the Particle swarm algorithm searching for the optimal R2 value of certain electrodes of the sensori motor cortex in the multi dimensional space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.12 Needed representation to evaluate subjects' performance: Top Left : visualization of the average spectral estimation (Burg estimator in this case) for motor imagery (in red) and resting state (in blue) for a specific electrode. Bottom Left: Topography map for a certain frequency bin, colormap corresponds to the level of significant differences between the two cognitive states. Right : Statistical difference map (here coefficient of determination) for each electrode for each frequency bin computed over trials of the different cognitive states. . . . . . . . . . . . . . . . . 3.13 NeuroFeedback visualization: Representation of the neuro-feedback during a trial of motor imagery. The halo changes its radius with the amplitude of the power for in a specific frequency band for a specific electrode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.14 Filter using Mahalanobis distance: Representation of the dispersion of the eye data and their integration with the average euclidean distance and the Mahalanobis distance . . . . . . . . . . 3.15 Proposition of 3d gaze estimator:Delaunay triangulation in 2d over the points established with the SLAM, projection in 3d to infer the 3d gaze position. . . . . . . . . . . . . . . . . . . . . . . 3.16 Stages of robot displacement: The different steps in time of the robot trajectory from its origin position to reaching the object and going back. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.17 Robot's behaviour based on the different orders sent from OpenViBE and the eye tracker via the server that does the synchronization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.18 Multimodal architecture: General architecture presenting the different protagonists and the exchange of information between them. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.19 BCI system: Architecture of the BCI system with the inner interactions between the two main software used (HappyFeat and OpenViBE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.20 Robotic and Eye tracker: Technical architectures. . . . . . . . . . . . . . . . . . . . . . . . . . . 3.21 Session: Phases of the protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.22 Strategy 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.23 Strategy 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.24 Strategy 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.

4 . 7 8 9

 4789 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Online performance: Online sensitivity corresponding to the number of time the can is seized by the robot for each strategy in the two phases of control (control 1 -Left, control 2 -Right). . . . 4.Offline performance evaluation: Offline accuracy computed with SVM with RBF kernel using the most relevant subjects' features for each strategy in the two phases of control. Left : Algorithm trained on Calibration and tested on control 1, Right : Algorithm trained on control 1 and tested on control 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.Evolution of performance for the different strategies: Evolution from control 1 to control 2 offline accuracy computed with SVM with RBF kernel using the most relevant subjects' features for each strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.10 Offline accuracy computed with SVM with RBF kernel using the most relevant subjects' features for each sessions in the two phases of control. Left : Algorithm trained on Calibration and tested on Drive 1, Right : Algorithm trained on Drive 1 and tested on Drive 2. . . . . . . . . . . . . . . 4.11 Performance score with Riemannian geometry: Offline accuracy computed with Linear Regression classifier on Tangent space of the covariances matrices for each strategy in the two phases of control. Left : Algorithm trained on Calibration and tested on control 1, Right : Algorithm trained on control 1 and tested on control 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.12 ERD/ERS spatial distribution in calibration:Top : Cluster permutation based permutation test perform on power spectrum 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡

  . . . . . . . . . . . . . . . . . . . . . . . 4.15 Subject's level analysis comparing phases: Comparison for each strategy of the distances in the different configurations. Each strategy respectively : Left -distances between calibration and control 1 of the matrix of 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡 power spectrum for 64 electrodes on the frequency bins ranging from 7 to 35 Hz, Center -distances between calibration and control 2 of the matrix of 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 power spectrum for 64 electrodes on the frequency bins ranging from 7 to 35 Hz, Right -distances between calibration and control 1 of the matrix of 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 power spectrum for 64 electrodes on the frequency bins ranging from 7 to 35 Hz. Green -Strategy 1, Red -Strategy 2,Blue -Strategy 3, from dark to light colour : Frobenius, Spectral form, Mahalanobis distance. . . . . . . . . . . . . 4.16 Spectro temporal distribution: Cluster size found in the time frequency map for each electrode for each strategy in Calibration, control 1 and control 2 phase; cluster based permutation evaluated at a threshold determined by quantile function evaluated for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 11 and 𝑝 < 0.05 , clusters kept at 𝑝 < 0.05. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.17 Brain networks in the different phases for the different strategies:Cluster permutation based permutation test performed on node strength of Functional connectivity 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 between subjects for the band (13-25Hz) in the three phases of Experimentation, Cluster evaluated at 𝑝 < 0.05. . 4.18 Source space estimation: Cluster found in the vertices estimated using DICS beamformer over cross spectral density in the 𝛽 band between the states of motor imagery and resting state for each strategy in Calibration, control 1 and control 2 phase; cluster based permutation evaluated at a threshold determined by quantile function evaluated for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15 and 𝑝 < 0.01 , clusters kept at 𝑝 < 0.05 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.19 In all phases of the experimentation for all strategies: Cluster based permutation test performed in the source space power spectrum 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 across subjects for the band (13-25Hz) in second phase of robot control with feedback (Control 2). Source space dipoles estimated using weight minimum norm (wMNE). Significant clusters at 𝑝 < 0.05. . . . . . . . . . . . . . . . . . . . . . 4.20 Agency perceived:Score of the sense of agency questionnaire established for each strategy at the end of each session. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

5. 3

 3 ERD Spatial distribution for the different strategies. a) Cluster based permutation test performed on power spectrum 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡

5. 4 5 . 5 5 . 6 5 . 7

 4555657 ERD Spectro temporal distribution for the different strategies. a) Cluster size found in the time frequency map for each electrode for each strategy in Control 2 phase; cluster based permutation evaluated at a threshold determined by quantile function evaluated for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15 and 𝑝 < 0.05 , clusters kept at 𝑝 < 0.05. b) Time frequency map for C4 electrode in terms of t-values and true ERD/ERS of motor imagery task vs resting state for the three different strategies, pointed line indicates the cluster found using cluster based permutation test. . . . . . . . . . . . . . . . Attention level through gaze:Average derivative of the pupil diameter across subjects between strategies, wilcoxon test, ( * * )𝑝 < 0.01. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Connectivity analysis across frequency bands.Cluster permutation test performed on the difference between MI and resting state in spectral coherence functional connectivity in the different frequency bands 𝜃,𝛼,𝛽,𝛾. Are plotted all the t-values to indicate the level of activation in each strategies in the different bands. Only strategy 3 in the 𝛽 band reveals to be significant in terms of node strength. Cluster based permutation evaluated at a threshold determined by quantile function evaluated for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15 and 𝑝 < 0.05 . . . . . . . . . . . . . . . . . . . Brain networks behaviour in the different strategies.

5. 10 6 . 1 2 F. 1

 106121 Beamformer revealing focused activity in all strategies: Cluster found in the vertices estimated using DICS beamformer over cross spectral density in the 𝛽 band between the states of motor imagery and resting state for each strategy in Calibration, control 1 and control 2 phase; cluster based permutation evaluated at a threshold determined by quantile function evaluated for Batch 1 and 2 combined meaning 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 26 and 𝑝 < 0.001 , clusters kept at 𝑝 < 0.001. . . . . . . 5.11 Brain networks behaviour in the different strategies in calibration and control 1. Cluster based permutation test performed on node strength functional connectivity using welch spectral coherence 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡 across subjects for the band (13-25Hz) in calibration and first phase of robot control with feedback (Control 2). Cluster based permutation evaluated at a threshold determined by quantile function evaluated for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15 and 𝑝 < 0.05 , clusters kept at 𝑝 < 0.05. The distance matrix needed for the cluster computation was set considering a threshold of 40 mm between EEG channels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.12 Spectro temporal activation in different regions of the brain for the different phases of the different strategies:a) Estimation of the time frequency ERD/ERS ( 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 ) between 4 and 32 Hz using multitaper on the 4 second of trials. The ERD is evaluated using cluster permutation test across subjects in the different brain regions of the left and right hemisphere. Only clusters at 𝑝 < 0.05 are kept. We measure the size of the cluster in each strategy and normalize the value by the maximum across regions and strategies. Is displayed the normalized size of the significant regions for the different strategies.b) Brain cortical parcellation of freesurfer, are studied only a subset of regions that could be involved in motor imagery based on our previous observations of the source space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Movimento del braccio L.Da Vinci, 1510-1511, Da Vinci built his pictures on the notion of movement which he considered key, who am I to say otherwise. . . . . . . . . . . . . . . . . . . . . . . . . 6.Autour d'un point F.Kupka, 1920-1930 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E.1 Top: Accuracy in the different phases using the tangent space logistic regression on the covariance matrix computed on the time series of motor imagery and resting state. Bottom: Average accuracy for the different strategies with training on the different intervals of time to assess the evolution of the accuracy with regards to the interval. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . E.2 Online accuracy in the different phases of control for each session, Left : Drive 1, Right : Drive 2. E.3 Slope of performance between drive 1 and 2 in function of the reward/effort score computed for each subject for strategy 3. Correlation using Spearman test. 14 subjects used (One of the subjects did not answer the online motivation questionnaire). . . . . . . . . . . . . . . . . . . . . . . . . The Creation of Adam, Michelangelo, Sistine Chapel . . . . . . . . . . . . . . . . . . . . . . . . F.2 Sysyphus, Titian, Museo del Prado . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Figure 1 . 1 :

 11 Figure 1.1: Braccio platform: a multi modal BCI approach for the control of a robot

Figure 2 . 1 :

 21 Figure 2.1: Brain structure: The different lobes of the brain, lateral representation.

Figure 2 . 2 :

 22 Figure 2.2: Brain cells schematic: Glia representation alongside the different cells interacting, image from [4].

Figure 2 . 3 :

 23 Figure 2.3: Neuron schematics: Detailed representation of the neuron composed of its nucleus, axon and synapse, image from [3].

Figure 2 . 4 :

 24 Figure 2.4: From initial source to non invasive sensor: Diagram representation of the transfer of information from the cortex to the scalp, image taken from [8].

Figure 2 . 5 :

 25 Figure 2.5: System of place of EEG sensors on the scalp: 10-10 International system, adapted from Laurens R. Krol.

[ 11 ]

 11 : Sörnmo et al. (2005), 'Chapter 4 -Evoked Potentials' [12]: Norcia et al. (2015), 'The steadystate visual evoked potential in vision research' [14]: Brandeis et al. (1986), 'Event-related potentials of the brain and cognitive processes'

Figure 2 . 6 :

 26 Figure 2.6: SSVEP pattern: Typical SSVEP response of an EEG signal acquired during visual stimulation at 9 Hz, Figure from [13].

Figure 2 . 7 :

 27 Figure 2.7: ERP pattern: Representation of the event related potential after presentation of a stimulus of an EEG signal, Figure from [15].

Figure 2 . 8 :

 28 Figure 2.8: ErrP patter: Typical Error Related Potential response after presentation of an error in a feedback, Figure from [17].

[ 18 ]

 18 : Shakeel et al. (2015), 'A Review of Techniques for Detection of Movement Intention Using Movement-Related Cortical Potentials' [21]: Pfurtscheller et al. (1999), 'Eventrelated EEG/MEG synchronization and desynchronization'

Figure 2 . 9 :

 29 Figure 2.9: MRCPs pattern: Movement related cortical potentials from an healthy subject for real imaginary movement of right ankle dorsi flexion, Figure from [18].

Figure 2 . 10 :

 210 Figure 2.10: ERD/ERS pattern: Typycal Event Related Desynchronization/Event Related Synchronization response after the generation of a motor movement, Figure from [22].

Figure 2 . 11 :

 211 Figure 2.11: Power amplitude linked to neurons activity Simulation of the relation between power spectrum amplitude and connectivity between neurons, Figure from [21].

Figure 2 . 12 :

 212 Figure 2.12: Motor imagery,Motor Action,Motor observation: Brain activity for the same motor action but either imagined, performed or observed. Image from [51].

[ 56 ]

 56 : Marcuse et al. (2016), '2 -The normal adult EEG' [55]: Marzbani et al. (2016), 'Neurofeedback'

Figure 2 . 13 :

 213 Figure 2.13: Cockpit evolution: On the left a Supermarine Spitfire from 1941, on the right a EuroFighter Typhoon from 2015.

[ 65 ]

 65 : Carvalhaes et al. (2015), 'The surface Laplacian technique in EEG' [67]: Kayser et al. (2015), 'Issues and considerations for using the scalp surface Laplacian in EEG/ERP research' [68]: Yao et al. (2019), 'Which Reference Should We Use for EEG and ERP practice?'9: This is the case with spectral coherence estimation to assess functional connectivity and CAR[START_REF] Greenblatt | Connectivity measures applied to human brain electrophysiological data[END_REF][START_REF] Fein | Common reference coherence data are confounded by power and phase effects[END_REF]

Figure 2 . 14 :

 214 Figure 2.14: CAR modification on signal: Before and after applying the Common Average Reference filter to the EEG data.

  .

10 : 2 𝑖( 2 . 5 )

 10225 If we want to deliver continuous feedback to users 11: Estimating power spectrum at almost each time point. 12: Estimating power spectrum with enough information therefore requiring many time points for each spectral estimation. [73]: Diez et al. (2008), 'A Comparative Study of the Performance of Different Spectral Estimation Methods for Classification of Mental Tasks' (𝑛 -𝑖) + 𝑤(𝑛) (2.4) 𝑘 𝑗 = 𝑗 𝑖=1 𝑎 𝑗-𝑖+1 𝑟 𝑖 𝑗 𝑖=1 𝑟 𝑎 𝑗 = 𝑎 𝑗-1 + 𝑘 𝑗 𝑎 𝑗-1 * (2.6)

Figure 2 . 16 :

 216 Figure 2.16: Riemannian manifold representation for covariance: Representation of the covariance matrix as a Riemannian manifold and the local projection on the Euclidean tangent space.

Figure 2 . 17 :

 217 Figure 2.17: Layers of a NN algorithm: Representation of the steps of manipulation of the EEG data in a deep learning architecture, on the left the different layers of the artificial neural network, on the right, the representation of the successive transformations.

[ 95 ]

 95 : Jeunet et al. (2015), 'Predicting Mental Imagery-Based BCI Performance from Personality, Cognitive Profile and Neurophysiological Patterns' 24: An it is the exact opposite of what we want to obtain. [96]: Roc et al. (2021), 'A review of user training methods in brain computer interfaces based on mental tasks' 25:The initial "order" given to subjects to start a sequence of actions or tasks. It is a trigger which can be visual, auditory or via touch.

Figure 2 . 18 :

 218 Figure 2.18: BCI2000 visualization: BCI 2000 early visual stimuli and feedback, the yellow signifies the target to go to.

  Fig 2.18. The second is the Graz Visualization as first designed by Pfurtscheller et al. in the late 90s[99] framework where red arrows (pointing right or left most of the time) indicate to do the motor imagery of the right or the left hand (or to indicate to do motor imagery of the right hand or resting state) depending on the orientation. Here, we gain an element of information, the orientation. By providing the direction, the association with right or left becomes more straight forward. However it lacks a component associated to the notion of motor action, this is even more the case in the feedback phase (evoked later). This is as well shown in Fig 2.19 .

Figure 2 . 19 :

 219 Figure 2.19: Graz Visualization Graz visual stimuli and feedback, the blue bar evolves with time in function of the classification.

[ 111 ]

 111 : Aljalal et al. (2020), 'Comprehensive review on brain-controlled mobile robots and robotic arms based on electroencephalography signals' [112]: Kilteni et al. (2012), 'The Sense of Embodiment in Virtual Reality' 29: Avatar is an extremely strong word to use. It comes from Hindu and it means an incarnation of the Vishnu divinity. It acquired in the 19 𝑡 ℎ the meaning of metamorphosis and transformation of someone into something else. [115]: Alimardani et al. (2015), 'BCIteleoperated androids; a study of embodiment and its effect on motor imagery learning'

[ 116 ]

 116 : Moore et al. (2012), 'Intentional binding and the sense of agency' [120]: Saha et al. (2021), 'Progress in Brain Computer Interface'30: An ecological environment refers to an immersive environment that provides the user with a sense of presence. The goal of an ecological environment in a BCI context is to create a realistic and interactive environment that can be controlled by the user's brain signals. [74]: Allison et al. (2010), 'Could Anyone Use a BCI?' [122]: Benabid et al. (2019), 'An exoskeleton controlled by an epidural wireless brain-machine interface in a tetraplegic patient'

Figure 2 . 20 :

 220 Figure 2.20: Kinematic chain : Representation from the base to the end effector of a robot arm.

Figure 2 . 21 : 5 [ 129 ]

 2215129 Figure 2.21: Robot arm opposed to human arm: Difference of joints and degrees of liberty between a human arm and a robotic arm, Figure from [125].

Figure 2 . 22 :

 222 Figure 2.22: Eye musculature: Muscle and eye anatomy, figure obtained from Elsevier's complete anatomy software.

Figure 2 . 23 :

 223 Figure 2.23: Effects of light on pupil: Representation of the different pupil effect based on the placement of the light with regards to the eye.

[ 152 ]

 152 : Niehorster et al. (2018), 'What to expect from your remote eye-tracker when participants are unrestrained' [155]: Schleicher et al. (2008), 'Blinks and saccades as indicators of fatigue in sleepiness warnings' [157]: Zhao et al. (2012), 'Eye movements and attention' [159]: Rozado et al. (2015), 'Improving the Performance of an EEG-Based Motor Imagery Brain Computer Interface Using Task Evoked Changes in Pupil Diameter' [160]: Lins et al. (1993), 'Ocular artifacts in EEG and event-related potentials. I'

Figure 2 . 24 :

 224 Figure 2.24: Blinks: Typical signature of blinks in EEG signal.

Figure 2 . 25 :

 225 Figure 2.25: Questionnaire Score: Average score of the sense of agency for the different tasks. * * * : 𝑝 < 0.001, Wilcoxon test.

Figure 2 . 26 :

 226 Figure 2.26: Decrease power among subjects: Occurrence from one subject to another of negative t-values at p<0.05 per electrodes for each comparison.

[ 178 ]Figure 2 . 27 :

 178227 Figure 2.27: Comparison between conditions in the sensor space: T values between the conditions Control by gaze and No Control in the most interesting frequency bands (𝛼 or 𝛽) with FDR correction, 𝑝 < 0.05.

Figure 3 . 1 :

 31 Figure 3.1: Left: Position towards an object to reach by the robot based on gaze. Right: Grasping of the object based on the MI BCI.

Figure 3 . 2 :

 32 Figure 3.2: Pillars of development: Principles for the subject upon which we must build the BCI platform : Trust, Agency, Involvement and Disturbance

Figure 3 . 3 :

 33 Figure 3.3: Continuous feedback with a robot:Iterative closing of the gripper based on the successive samples classified as either motor imagery in red or resting state in blue. The illustration shows two different features considered as the feature vector.

Figure 3 . 4 :

 34 Figure 3.4: Discrete feedback with a robot: Closing of the gripper at the end of the trial based on the average successive samples classified as either motor imagery in red or resting state in blue. The illustration shows two different features considered as the feature vector. If more samples are classified as belonging to motor imagery than as belonging to resting state, the robot's gripper closes.

Figure 3 . 5 :

 35 Figure 3.5: The can to reach is selected based on the average of the gaze position along the x-axis for 2 seconds

  Fig 3.6 shows the different positions considered for the robotic placement.

Figure 3 . 6 :

 36 Figure 3.6: Placing the robot arm in the environment: Considered positions of the robot with regards to the table with cans reachable, on the right of the subject, tilted at 45 or 90 degrees, or in front of the subject. The position kept for the protocol will be the one where the robot is facing the subject at the over end of the table.

Figure 3 . 7 :

 37 Figure 3.7: Robot placement and visual feedback: Robot from above and the motor imagery stimulus (in red) below the cans. The outside circle will be later on describe in the Neurofeedback section, it is a continuous feedback linked the neural activity of subjects during their motor imagery trial.

Figure 3 . 8 :

 38 Figure 3.8: From EEG to ML algorithm:Chain of treatment from EEG signal to classification. First we acquire the raw EEG signal corresponding to trials of motor imagery (in red) or resting state (in blue). We estimate for each trial the power spectrum to obtain an average over trial power spectrum. The average power spectrum for a specific bin of frequency for a specific electrode for each class (MI or rest) is then considered as a feature. If the features are discriminant enough, a machine learning algorithm can draw a separation between those classes (an hyperplane).

[ 190 ]

 190 : Kennedy et al. (1995), 'Particle swarm optimization' 7: An optimal position to find usually consists in finding the point where the derivative becomes equal to zero.

Figure 3 . 9 :

 39 Figure 3.9: Photography of a bird flock, Author: S.Solker

Figure 3 . 10 :

 310 Figure 3.10: PSO: Representation of the Particle swarm algorithm searching for the optimal negative position in a 3d gaussian via its 3d particles

Figure 3 . 11 :

 311 Figure 3.11: PSO in the selections of AR parameters: Representation of the Particle swarm algorithm searching for the optimal R2 value of certain electrodes of the sensori motor cortex in the multi dimensional space

[ 194 ]

 194 : Vidaurre et al. (2011), 'Toward Unsupervised Adaptation of LDA for Brain-Computer Interfaces' 9: Scenari in OV are data treatment pipeline offline or online.

Figure 3 . 12 :

 312 Figure3.12: Needed representation to evaluate subjects' performance: Top Left : visualization of the average spectral estimation (Burg estimator in this case) for motor imagery (in red) and resting state (in blue) for a specific electrode. Bottom Left: Topography map for a certain frequency bin, colormap corresponds to the level of significant differences between the two cognitive states. Right : Statistical difference map (here coefficient of determination) for each electrode for each frequency bin computed over trials of the different cognitive states.

[ 196 ]

 196 [START_REF] Meyer | Displaying a boundary in graphic and symbolic "wait" displays: Duration estimates and users' preferences[END_REF], 'Displaying a boundary in graphic and symbolic "wait" displays'

Figure 3 . 13 :

 313 Figure 3.13: NeuroFeedback visualization: Representation of the neurofeedback during a trial of motor imagery.The halo changes its radius with the amplitude of the power for in a specific frequency band for a specific electrode.

[ 199 ]

 199 : Mladenovic (2019), 'Computational Modeling of User States and Skills for Optimizing BCI Training Tasks' [200]: Barbero et al. (2010), 'Biased feedback in brain-computer interfaces'

[ 202 ]

 202 : Leys et al. (2018), 'Detecting multivariate outliers' the robot would not close its hand no matter what when it is a resting state both in calibration and control phases.

Figure 3 . 14 :

 314 Figure 3.14: Filter using Mahalanobis distance: Representation of the dispersion of the eye data and their integration with the average euclidean distance and the Mahalanobis distance

Figure 3 . 15 :

 315 Figure 3.15: Proposition of 3d gaze estimator:Delaunay triangulation in 2d over the points established with the SLAM, projection in 3d to infer the 3d gaze position.

Figure 3 . 16 :

 316 Figure 3.16: Stages of robot displacement: The different steps in time of the robot trajectory from its origin position to reaching the object and going back.

Fig 3 .

 3 18 shows how users are placed between all the elements, they interact and receive feedback from the different systems. The second diagram (Fig 3.19) goes in more details with the architecture of the BCI system, it shows for instance the elements that belong to OpenViBE and the ones that belong to HappyFeat software. It also allows to see how HappyFeat modifies the parameters of OV. The last diagram (Fig 3.20) presents the two other crucial elements of the

Figure 3 . 17 :

 317 Figure 3.17: Robot's behaviour based on the different orders sent from OpenViBE and the eye tracker via the server that does the synchronization.

Figure 3 . 18 :

 318 Figure 3.18: Multimodal architecture: General architecture presenting the different protagonists and the exchange of information between them.

Figure 3 . 19 :

 319 Figure 3.19: BCI system: Architecture of the BCI system with the inner interactions between the two main software used (HappyFeat and OpenViBE)

Figure 3 . 20 :

 320 Figure 3.20: Robotic and Eye tracker: Technical architectures.

Figure 3 . 21 :

 321 Figure 3.21: Session: Phases of the protocol

Figure 3 . 22 : Strategy 1 Figure 3 . 23 : Strategy 2 ▶

 32213232 Figure 3.22: Strategy 1

Figure 3 . 24 :

 324 Figure 3.24: Strategy 3

[ 209 ]

 209 : Westner et al. (2022), 'A unified view on beamformers for M/EEG source reconstruction'

[ 180 ]

 180 : Gentili et al. (2015), 'Laterality effects in motor learning by mental practice in right-handers' 1:

  Fig 4.1.

Figure 4 . 1 :

 41 Figure 4.1: Results of a "prodigy" : left, 𝑅 2 map corresponding to the difference between motor imagery and resting state over trial of a specific phase (here the calibration). Right: The average powerspectrum over the phase for both condition (Top), The associate topography to the 𝑅 2 map to indicate the localization of the desynchronization -the sensori motor cortex (Bottom)

Figure 4 . 2 :

 42 Figure 4.2: Evolution during session: Evolution of the 𝑅2 map, computed on the power spectrum difference between motor imagery and resting state over the trials of each phase, from the calibration to the driving phase for a "prodigy" subject. Here, what is noticeable is the spread of high statistical differences between resting state en motor imagery PSD in the sensory motor area in the low 𝛽 band from the calibration to the control phase. This spread out of the intensity can be the mark of the engagement of the subject but also his or ability to greatly perform the task.

[ 211 ]

 211 : Becker et al. (2022), 'BCI Illiteracy'The Sysyphus -Subjects who search Beside the prodigies, most of subjects perform relatively well from the start but still have room for improvement as we can observe for a specific subject in Fig 4.3, and they either improve during the same session or later on in the next ones.

Figure 4 . 3 :

 43 Figure 4.3: Results of a "sysyphus" : left, 𝑅 2 map corresponding to the difference between motor imagery and resting state over trial of a specific phase (here the calibration). Right: The average powerspectrum over the phase for both condition (Top), The associate topography to the 𝑅 2 map to indicate the localization of the desynchronization -the sensori motor cortex (Bottom)

Figure 4 . 4 :

 44 Figure 4.4: Evolution during session: Evolution of the 𝑅 2 map, computed on the power spectrum difference between motor imagery and resting state over the trials of each phase, from the calibration to the driving phase for a "sysyphus" subject.Here what is noticeable compared to the "prodigy" behaviour is that the highest statistical difference is not in the sensory motor area and the spread out in the 𝛼 band across all the electrodes is a mark of the inability to perform the task (yet).

Figure 4 . 5 :

 45 Figure 4.5: Variability of resting state and MI: Distribution of the calibration trials' power spectrum for the specific CP3 electrode at 13Hz, Red for motor imagery , Blue for resting state, points on the line corresponds to the new features of trials of control 1. Each point represent a trial of the control phase whereas the histogram represents the trials of the calibration phase.

Figure 4 . 6 :

 46 Figure 4.6: Comparing variability of prodigy and sysyphus:Features by Principal Component Analysis, Red for motor imagery , Blue for resting state, left "sysyphus" -distribution of the control 2, right "prodigy" -distribution of the calibration. First, we select using the 𝑅 2 map the features (specific electrodes at a specific frequencies) that present the highest statistical differences. Second, we use over the trials of a block (calibration or control 1) a PCA on the feature matrix consisting of the observations for the different features selected to retrieve only the principal 3 eigenvectors (based on the 3 highest eigenvalues), third we plot the 3 components in the different trials for the two conditions. The purpose of the plot is to have a synthetic representation of the features to see their distribution.

Figure 4 . 7 :

 47 Figure 4.7: Online performance: Online sensitivity corresponding to the number of time the can is seized by the robot for each strategy in the two phases of control (control 1 -Left, control 2 -Right).

  are 3 blocks in a session used in the experimental protocol. The first block Calibration where subjects train to do MI and Rest but are not in full control of the arm (just controlling the position with their gaze). The second block Control 1 where subjects perform MI and resting state and the results of their MI trials close or not the gripper on the can selected. To do so, a LDA classifier has been trained on the calibration block. The third block Control 2 where subjects perform MI and resting state and the results of their MI trials close or not the gripper on the can selected. To do so, a LDA classifier has been trained on the Control 1 block. All training of the LDA were done by checking on the statistical differences of PSD for the different electrodes in the different frequency bins.

Figure 4 . 8 :

 48 Figure 4.8: Offline performance evaluation: Offline accuracy computed with SVM with RBF kernel using the most relevant subjects' features for each strategy in the two phases of control. Left : Algorithm trained on Calibration and tested on control 1, Right : Algorithm trained on control 1 and tested on control 2.

  BCIs in an enriched environment T. Venot;A. Desbois;M.C Corsi;L. Hugueville;L. Saint-Bauzel,F. De Vico Fallani, 2022 IEEE International Conference on Metrology for Extended Reality, Artificial Intelligence and Neural Engineering (MetroX-RAINE)

Figure 4 . 9 :

 49 Figure 4.9: Evolution of performance for the different strategies: Evolution from control 1 to control 2 offline accuracy computed with SVM with RBF kernel using the most relevant subjects' features for each strategy

Figure 4 . 10 : 10 : 11 :

 4101011 Figure 4.10: Offline accuracy computed with SVM with RBF kernel using the most relevant subjects' features for each sessions in the two phases of control. Left : Algorithm trained on Calibration and tested on Drive 1, Right : Algorithm trained on Drive 1 and tested on Drive 2.

Figure 4 . 11 :

 411 Figure 4.11: Performance score with Riemannian geometry: Offline accuracy computed with Linear Regression classifier on Tangent space of the covariances matrices for each strategy in the two phases of control. Left : Algorithm trained on Calibration and tested on control 1, Right : Algorithm trained on control 1 and tested on control 2.

Figure 4 . 12 :

 412 Figure 4.12: ERD/ERS spatial distribution in calibration:Top : Cluster permutation based permutation test perform on power spectrum 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡

Figure 4 .

 4 Figure 4.13: ERD/ERS spatial distribution in control 1:Top : Cluster permutation based permutation test perform on power spectrum 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒𝑠𝑡

Figure 4 . 14 :

 414 Figure 4.14: ERD/ERS spatial distribution in control 2:Top : Cluster permutation based permutation test perform on power spectrum 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒𝑠𝑡

Figure 4 . 15 :

 415 Figure 4.15: Subject's level analysis comparing phases: Comparison for each strategy of the distances in the different configurations. Each strategy respectively : Left -distances between calibration and control 1 of the matrix of 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡 power spectrum for 64 electrodes on the frequency bins ranging from 7 to 35 Hz, Center -distances between calibration and control 2 of the matrix of 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡 power spectrum for 64 electrodes on the frequency bins ranging from 7 to 35 Hz, Right -distances between calibration and control 1 of the matrix of 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒𝑠𝑡 power spectrum for 64 electrodes on the frequency bins ranging from 7 to 35 Hz. Green -Strategy 1, Red -Strategy 2,Blue -Strategy 3, from dark to light colour : Frobenius, Spectral form, Mahalanobis distance.

Figure 4 . 16 :

 416 Figure 4.16: Spectro temporal distribution: Cluster size found in the time frequency map for each electrode for each strategy in Calibration, control 1 and control 2 phase; cluster based permutation evaluated at a threshold determined by quantile function evaluated for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 11 and 𝑝 < 0.05 , clusters kept at 𝑝 < 0.05.

Figure 4 . 17 :

 417 Figure 4.17: Brain networks in the different phases for the different strategies:Cluster permutation based permutation test performed on node strength of Functional connectivity 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡 between subjects for the band (13-25Hz) in the three phases of Experimentation, Cluster evaluated at 𝑝 < 0.05.

Figure 4 . 18 :

 418 Figure 4.18: Source space estimation: Cluster found in the vertices estimated using DICS beamformer over cross spectral density in the 𝛽 band between the states of motor imagery and resting state for each strategy in Calibration, control 1 and control 2 phase; cluster based permutation evaluated at a threshold determined by quantile function evaluated for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15 and 𝑝 < 0.01 , clusters kept at 𝑝 < 0.05

Figure 4 . 19 :

 419 Figure 4.19: In all phases of the experimentation for all strategies: Cluster based permutation test performed in the source space power spectrum 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 across subjects for the band (13-25Hz) in second phase of robot control with feedback (Control 2). Source space dipoles estimated using weight minimum norm (wMNE). Significant clusters at 𝑝 < 0.05.

Figure 4 . 20 :

 420 Figure 4.20: Agency perceived:Score of the sense of agency questionnaire established for each strategy at the end of each session.

[ 219 ]

 219 : Alimardani et al. (2018), 'Brain-Computer Interface and Motor Imagery Training'

[ 223 ]

 223 : Mansour et al. (2022), 'Efficacy of Brain-Computer Interface and the Impact of Its Design Characteristics on Poststroke Upper-limb Rehabilitation'

[ 225 ]

 225 : He et al. (2020), 'Neurofeedbacklinked suppression of cortical beta bursts speeds up movement initiation in healthy motor control'

[ 226 ]

 226 : Perdikis et al. (2011), 'Evidence Accumulation in asynchronous BCI'

Figure 4 . 21 :

 421 Figure 4.21: Evolution of the computation of the feedback: On top, the computation of the feedback for batch 1, at each power estimation, each feature (C3 at 13 Hz for example) is classified by the LDA using the distance to the hyperplane (negative values associated to class one, positive values associated to class 2), then at the end of a trial, the sum of the distances to the hyperplane gives a positive or a negative value resulting in the closing of the gripper. This method is sensitive to power spectrum variations. Bottom, the computation of the feedback for batch 2, the average PSD over time windows for each feature is classified by the LDA, only one hyperplane distance is output, either negative or positive. This method is less sensitive to power spectrum variations over trial.

Figure 5 . 2 :

 52 Figure 5.2: Behavioral scores and Performance. a) Correlation between the highest performance of subjects across strategies in accuracy and sensitivity in function of their motivation, rho established with spearman correlation,𝑝 𝑣𝑎𝑙𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡 𝑦 = 0.024, 𝑝 𝑣𝑎𝑙𝐴𝑐𝑐𝑢𝑟 𝑎𝑐 𝑦 = 0.007. b)Average Accuracy in the two phases of control for each strategy for the subject batch that received feedback. The two-way Anova test was performed between phases and strategies, test revealed that the phase was the only significant factor at 𝑝 < 0.0002, in the post-hoc analysis with bonferonni correction, significant difference was only found between control phases of strategy 3 at 𝑝 < 0.003.

5. 1 101 Figure 5 . 3 :

 110153 Figure 5.3: ERD Spatial distribution for the different strategies. a) Cluster based permutation test performed on power spectrum 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 across subjects for the band (13-25Hz) in second phase of robot control with feedback (Control 2), threshold of display set to 𝑝 < 0.05. The distance matrix needed for the cluster computation was set considering a threshold of 40 mm between EEG channels. We use a threshold computed using percent point function at 𝛼 = 0.05 for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15 b) Cluster based permutation test performed in the source space power spectrum 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡

Figure 5 . 4 :

 54 Figure 5.4: ERD Spectro temporal distribution for the different strategies. a) Cluster size found in the time frequency map for each electrode for each strategy in Control 2 phase; cluster based permutation evaluated at a threshold determined by quantile function evaluated for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15 and 𝑝 < 0.05 , clusters kept at 𝑝 < 0.05. b) Time frequency map for C4 electrode in terms of t-values and true ERD/ERS of motor imagery task vs resting state for the three different strategies, pointed line indicates the cluster found using cluster based permutation test.

Figure 5 . 5 :

 55 Figure 5.5: Attention level through gaze:Average derivative of the pupil diameter across subjects between strategies, wilcoxon test, ( * * )𝑝 < 0.01.

Figure 5 . 6 :Figure 5 . 7 :

 5657 Figure 5.6: Connectivity analysis across frequency bands.Cluster permutation test performed on the difference between MI and resting state in spectral coherence functional connectivity in the different frequency bands 𝜃,𝛼,𝛽,𝛾. Are plotted all the t-values to indicate the level of activation in each strategies in the different bands. Only strategy 3 in the 𝛽 band reveals to be significant in terms of node strength. Cluster based permutation evaluated at a threshold determined by quantile function evaluated for 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 15 and 𝑝 < 0.05

1 Figure 5 . 8 :

 158 Figure 5.8: In all phases of the experimentation for all strategies: Cluster based permutation test performed in the source space power spectrum 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒 𝑠𝑡

Figure 5 . 9 :

 59 Figure 5.9: Subject level analysis of their patterns variations during the experimentation: Top: Comparison for each strategy of the distances in the different configurations. Each strategy respectively : Left -distances between calibration and control 1 of the matrix of 𝑀𝐼-𝑅𝑒𝑠𝑡 𝑅𝑒𝑠𝑡 power spectrum for 64 electrodes on the frequency bins ranging from 7 to 35 Hz, Center -distances between calibration and control 2 of the matrix of 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡 power spectrum for 64 electrodes on the frequency bins ranging from 7 to 35 Hz, Right -distances between calibration and control 1 of the matrix of 𝑀𝐼-𝑅𝑒 𝑠𝑡 𝑅𝑒 𝑠𝑡 power spectrum for 64 electrodes on the frequency bins ranging from 7 to 35 Hz. Green -Strategy 1, Red -Strategy 2,Blue -Strategy 3, from dark to light colour : Frobenius, Spectral form, Mahalanobis distance.Bottom: Boxplot associated representation of the distances in the different strategies, Wilcoxon test to assess the level of differences between the distances. * : 𝑝 < 0.05

Figure 5 . 10 : 5 . 2 . 4 Brain networks in the calibration an control phase also reveals consistency of strategy 3 Figure 5 . 11 :

 5105243511 Figure 5.10: Beamformer revealing focused activity in all strategies: Cluster found in the vertices estimated using DICS beamformer over cross spectral density in the 𝛽 band between the states of motor imagery and resting state for each strategy in Calibration, control 1 and control 2 phase; cluster based permutation evaluated at a threshold determined by quantile function evaluated for Batch 1 and 2 combined meaning 𝑛 𝑜𝑏𝑠𝑒𝑟𝑣𝑎𝑡𝑖𝑜𝑛𝑠 = 26 and 𝑝 < 0.001 , clusters kept at 𝑝 < 0.001.

5. 2 . 5 Figure 5 . 12 :

 25512 Figure 5.12: Spectro temporal activation in different regions of the brain for the different phases of the different strategies:a)

[ 238 ]

 238 : Cavazzana (2016), 'SENSE OF AGENCY AND INTENTIONAL BIND-ING: How does the brain link voluntary actions with their consequences?'

  ]: Perdikis et al. (2020), 'Brain-Machine Interfaces' Hypothesis 2

Figure 6 . 1 :

 61 Figure 6.1: Movimento del braccio L.Da Vinci, 1510-1511, Da Vinci built his pictures on the notion of movement which he considered key, who am I to say otherwise.

▶

  Training and test data: 2 sets of 3 runs and 1 set of 2 runs. ▶ Eye Tracker data from experiment. ▶ Save data on hard drive and chandelier. EEG Protocol I. Interaction with the Patient ▶ Place the patient comfortably. ▶ Explain the protocol's objective and steps briefly. ▶ Describe the preparation process, including gel application and alcohol scrubbing on forehead and mastoids. ▶ Identify the cap's reference point. ▶ Measure nasion to inion distance (approximately 36 cm) before cap installation. ▶ Install the cap and measure the following distances (to be recorded in xlsx file): • Distance from top stitching to nasion (about 3.6 cm or 10 • Distance from T8 to right preauricular • Distance from T7 to left preauricular • Distance from FPz to Cz • Distance from Nasion to Oz ▶ Apply gel to electrodes. ▶ During preparation, have the patient perform hand movements.

  OpenViBE is built upon a C++ architecture open source originally licenced by Mensia Technologies. It has evolved through the last decade into a reliable software offering versatility and high liberty of modification. The software has 2 main different attributes.

Figure E. 1 :

 1 Figure E.1: Top: Accuracy in the different phases using the tangent space logistic regression on the covariance matrix computed on the time series of motor imagery and resting state. Bottom: Average accuracy for the different strategies with training on the different intervals of time to assess the evolution of the accuracy with regards to the interval.

Figure E. 2 :

 2 Figure E.2: Online accuracy in the different phases of control for each session, Left : Drive 1, Right : Drive 2.

Figure E. 3 :Figure F. 2 :

 32 Figure E.3: Slope of performance between drive 1 and 2 in function of the reward/effort score computed for each subject for strategy 3. Correlation using Spearman test. 14 subjects used (One of the subjects did not answer the online motivation questionnaire).

  3.1.4 When should subjects perform the cognitive tasks ? . . . . . . . . . . . . . . . . . . . 3.2 Creating the conditions for a robust platform . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Optimizing standard BCI paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Providing a neuro-feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 On the necessity to randomize between trials of motor imagery and resting state . . 3.2.4 On the choice of providing or not feedback in the cognitive states . . . . . . . . . . . 3.2.5 Optimizing gaze acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.6 Robot trajectory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.7 System architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Experimental protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Material and methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 EEG acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Eye tracking acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.3 Signal Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.4 Statistical analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

the-Art in brain machine interfaces 2

	Sed nihil dulcius est, bene quam munita tenere
	edita doctrina sapientum templa serena,
	despicere unde queas alios passimque videre
	errare atque viam palantis quaerere vitae, certare
	ingenio, contendere nobilitate,
	noctes atque dies niti praestante labore
	ad summas emergere opes rerumque potiri.
	o miseras hominum mentes, o pectora caeca!
	Lucrece, De Rerum Natura, (II,7-19)
	Ts: But nothing is sweeter than to dwell on
	the serene heights that science defends, the
	refuge of the wise; and to be able to cast one's
	eyes over other men from this asylum, and to
	see them here and there wandering, seeking
	the road of life, making a show of genius,
	arguing over the nobility of blood, night and
	day striving with all-consuming labor to rise
	to fortune and possess power. O wretched
	hearts of men! O blinded minds!
	[2]: Sure (2007), 'Henri M. Duvernoy
	(ed)'

[START_REF] Dharani | Chapter 1 -Functional Anatomy of the Brain[END_REF]

:

[START_REF] Dharani | Chapter 1 -Functional Anatomy of the Brain[END_REF]

, 'Chapter 1 -Functional Anatomy of the Brain'

State-of-

Key aspects of the state of the art ▶ Overview of brain mechanisms and EEG patterns. ▶ Introducing neurofeedback and motor imagery brain computer interface. ▶ Covering the two research focus of BCI : computational improvement and human centered design. ▶ Introducing robot involvement in BCI. ▶ Introducing EyeTracker technology and its use for robotic control. ▶ Introducing multimodal BCI, presenting the advantages and limitations of the technique.

BCIs: the impact of peripheral control on motor cortex activity and sense of agency

	,T. Venot; M.C. Corsi;
	L. Saint-Bauzel; F. de Vico Fallani, 2021 43rd Annual International
	Conference of the IEEE Engineering in Medicine Biology Society
	(EMBC)

Table 2 . 1 :

 21 Most significant t-values associated with their p-values for each subject in each comparison, the frequency band is indicated.

	PairSubjects	1	2	3	4	5	6	7	8
		t=-5, 34	t=-1.80	t=-2.51	t=1.15	t=-8.91	t=1.98	t=-8.29	t=-2.69
	Mirror Vs Rest	p=0,001	p=0.001	p=0.015	p=0.015	p=0.001	p=0.019	p=0.001	p=0.015
		(𝛽)	(𝛼)	(𝛼)	(𝛽)	(𝛽)	(𝛽)	(𝛼)	(𝛼)
			t=-1.81			t=-1.00	t=-2.17	t=-1.13	t=-4.66
	Gaze Vs Rest	NS	p=0.017	NS	NS	p=0.012	p=0.045	p=0.019	p=0.001
			(𝛼)			(𝛽)	(𝛼)	(𝛽)	(𝛼)
		t=3.73	t=-1.72			t=-3.1943	t=-1.78		t=-3.98
	No Control Vs
		p=0.010	p=0.010	NS	NS	p=0.002	p=0.006	NS	p=0.001
	Rest
		(𝛽)	(𝛼)			(𝛼)	(𝛽)		(𝛼)
						[179]: Klimesch (2012), '$\alpha$ band
						oscillations, attention, and controlled ac-
						cess to stored information'
						[180]: Gentili et al. (2015), 'Laterality ef-
						fects in motor learning by mental practice
						in right-handers'

  : Akilandeswari et al. (2014), 'Swarm Optimized Feature Selection of EEG Signals for Brain-Computer Interface'

Table 4 . 1 :

 41 Analysis of Variance (ANOVA) Table -Batch receiving random feedback

	Source	Sum of Squares Degrees of Freedom F-Statistic p-value
	C(Group)	0.013622	2.0	0.414009	0.662869
	C(Session)	0.122551	1.0	7.449113	0.008316
	C(Group):C(Session)	0.007057	2.0	0.214490	0.807569
	Residual	0.987102	60.0	-	-

Table 5 .

 5

1: ANOVA Results -Batch receiving feedback

Table 5 . 2

 52

	: Test Multiple Comparison	group1	group2	stat	pval	pval_corr reject
	Ttest with Bonferroni correction -Batch
		Strat 1 control 1 Strat 1 control 2	-2.64	0.0194	0.2911	False
	receiving feedback
		Strat 1 control 1 Strat 2 control 1	1.0373	0.3172	1.0	False
		Strat 1 control 1 Strat 2 control 2	-1.628	0.1258	1.0	False
		Strat 1 control 1 Strat 3 control 1	0.8587	0.405	1.0	False
		Strat 1 control 1 Strat 3 control 2	-2.519	0.0245	0.3682	False
		Strat 1 control 2 Strat 2 control 1	3.8115	0.0019	0.0286	True
		Strat 1 control 2 Strat 2 control 2	0.6168	0.5473	1.0	False
		Strat 1 control 2 Strat 3 control 1	3.0636 0.0084	0.1263	False
		Strat 1 control 2 Strat 3 control 2 -0.2192 0.8297	1.0	False
		Strat 2 control 1 Strat 2 control 2 -2.8707 0.0123	0.185	False
		Strat 2 control 1 Strat 3 control 1	0.1989	0.8452	1.0	False
		Strat 2 control 1 Strat 3 control 2 -4.0225 0.0013	0.0189	True
		Strat 2 control 2 Strat 3 control 1	2.7535	0.0155	0.233	False
		Strat 2 control 2 Strat 3 control 2 -0.6117 0.5506	1.0	False
		Strat 3 control 1 Strat 3 control 2 -4.9837 0.0002	0.003	True
		robust estimation of the MI and rest condition for each subject and
		strategy.

Dictionary regrouping key notions and methods 10-20 EEG system:

  What I cannot create, I cannot understand, quote from the blackboard of Richard Feynman. ▶ Errare humanum est, sed perseverare diabolicum, latin phrase attributed to Seneca. ▶ It is a far, far better thing that I do, than I have ever done; it is a far, far better rest I go to than I have ever known. A Tale of two cities, C.Dickens, The 10-20 system is a standardized method used to describe and apply electrode placements for electroencephalography (EEG) recordings. The system is based on the relationship between the location of an electrode on the scalp and the underlying area of the brain. The name "10-20" refers to the distances between adjacent electrode placements, which are either 10% or 20% of the total frontback or right-left distance of the skull.The system is based on a grid of electrodes placed at specific locations on the scalp. Each electrode is labeled with a letter and a number, indicating its position on the grid. The letters F, T, C, P, and O refer to the frontal, temporal, central, parietal, and occipital lobes of the brain, respectively. The numbers indicate the distance between electrode placement along the front-back and right-left dimensions of the skull.The 10-20 system is widely used in clinical and research settings for EEG recordings and has become a standardized method for electrode placement in EEG experiments. It allows for consistent and reproducible recordings across different studies and laboratories, facilitating comparison and pooling of data. The covariance matrix of the sensor data is computed. It represents the cross-correlations and power of the signals at different sensors, capturing the spatial properties of the brain activity. 4. Spatial Filter (Beamformer): The core of the beamforming technique lies in the design of the spatial filter. The spatial filter aims to enhance signals arising from a specific brain region (source location) while suppressing signals from other areas. The filter is computed based on the forward model, the covariance matrix, and the desired source location. 5. Source Reconstruction: The spatially filtered sensor data is used to reconstruct the neural activity at the desired source location. This provides an estimate of the neural activation in the brain region of interest. There are different types of beamformers, each with its specific characteristics and applications. Some common types include: ▶ Minimum Variance Beamformer (MV Beamformer): Aims to minimize the variance of the reconstructed signal subject to certain constraints. ▶ Linearly Constrained Minimum Variance (LCMV) Beamformer: An extension of MV beamformer that includes linear constraints to enhance signals coming from specific directions. ▶ Dynamic Imaging of Coherent Sources (DICS): A type of beamformer optimized for frequency-specific source localization. ▶ Sparse Beamformers: Incorporate sparsity constraints to improve localization accuracy and reduce interference from distributed sources.

	A Enlightening
	▶ Book 3, chapter 15.
	Beamformer: A beamformer is a signal processing technique used in
	various fields, including neuroscience and audio engineering. It is a spa-
	tial filtering method designed to enhance or suppress signals at specific
	spatial locations. In the context of neuroscience, beamformers are often
	used for analyzing brain signals measured by EEG (electroencephalogra-
	phy) or MEG (magnetoencephalography) to estimate the neural activity
	in the brain with high spatial resolution. The basic idea behind beamform-
	ing is to create a spatial filter that emphasizes the signals arriving from
	a specific direction (source location) while attenuating signals coming
	from other directions (interference or noise). It is analogous to aiming a
	directional microphone or antenna at a particular sound source or radio
	transmitter to pick up its signal more clearly.The general steps involved
	in the beamforming process are as follows:
	1. Sensor Data Acquisition: EEG or MEG sensors are placed around
	the head to measure the electromagnetic activity generated by the
	brain.
	2. Forward Model: A forward model is constructed, which describes
	the relationship between the brain activity at different source lo-
	cations and the measurements at the sensor locations. It involves
	modeling the brain's anatomy, sensor positions, and the conductiv-
	ity properties of the head.
	3. Covariance Estimation:

B

  𝑠 1 𝑛 2 -𝑠 2 𝑛 1 (𝑛 1 + 𝑛 2 ) 2 (B.2) 4𝑛 1 𝑛 2 (𝑛 1 + 𝑛 2 ) 2 (B.4) where 𝑠 𝑘 = 𝑖 𝑥 𝑘 𝑖 and 𝑞 𝑘 = 𝑖 𝑥 𝑘 2 𝑖 . The Wilcoxon test, also known as the Wilcoxon signedrank test, is a nonparametric statistical test used to compare the median of two related samples. It is used when the assumptions of the paired t-test, such as normality and homogeneity of variances, are violated. The Wilcoxon test is often used in situations where the data is ordinal, skewed, or has outliers. It is a useful alternative to the paired t-test when the data does not meet the assumptions of parametric tests. The Wilcoxon test works by comparing the differences between pairs of observations within each group. The differences are then ranked in order of magnitude, regardless of the direction of the difference. The sum of the ranks for the positive differences and the sum of the ranks for the negative differences are calculated separately. The test statistic is the smaller of the two sums, and its significance is determined by comparing it to a table of critical values based on the sample size. The Wilcoxon test is a one-tailed test, meaning that it tests for the hypothesis that one group is larger than the other. If the null hypothesis is rejected, it means that the median of one group is significantly greater than the median of the other group.The Wilcoxon test can also be used for matched-pairs data, where each subject is measured twice, such as before and after a treatment. In this case, the differences between the two measurements are calculated, and the Wilcoxon test is applied to these differences. In summary, the Wilcoxon test is a nonparametric test used to compare the median of two related samples when the assumptions of parametric tests are not met. It is useful in situations where the data is ordinal, skewed, or has outliers. Start the computer, remain on Windows without powering the room. ▶ Launch the virtual machine, ensuring a private host network connection. 2. Virtual Machine ▶ Launch Terminator terminal, create two subwindows. ▶ Verify selection of FTDI FT230X in USB device. ▶ Navigate to desktop in terminal, launch RobotLauncher.sh. ▶ In second terminal, initiate desired strategy in Train or Test mode. ▶ To shut down, utilize TurnOffSafe.sh script. Generate scenarios (once only, do not modify). ▶ Launch OpenVibe acquisition. ▶ Process in HappyFeat. ▶ Output training scenarios and train on selected features. ▶ Launch online scenario Eye Tracker and Sending: Explain motor imagination and rest states. Subjects should imagine a slow but firm hand closure for three seconds. ▶ Imagined hand closure resembles a familiar manual or sport-related gesture (e.g., bicycle brake), emphasizing hand closure. ▶ Explain protocol details. ▶ Complete Motor Imagination Questionnaire. ▶ Conduct gesture training before proceeding. ▶ Complete Agency Questionnaire at each session's end.

	BRACCIO Checklist Augmented Table ▶ Wired ▶ Powered Wilcoxon test: C ▶ Turned on HDMI 4
		Lights
		▶ Spotlights
		▶ LEDs
	Material
		Lever Break for Motor Learning
	EEG	Setup Protocol
	▶ 2 EEG boxes
	▶ 2 EEG cables ("nappe")	1. Preparations
	▶ 2 optical fibers
	▶ ActiCap box ▶ 2 charged batteries ▶ 2 short cables for batteries ▶ 56 or 58 electrodes ▶ 32 x 2 active electrodes ▶ GND and REF electrodes Generalities ▶ Gel ▶ 2 syringes	▶ 3. Principal Computer (Windows)
	▶ Papers ▶ Towels ▶ Compresses (check the date) ▶ Shampoo	▶ Start OpenVibe server acquisition. ▶ Launch Feature Extraction (green arrow on desktop). ▶ Launch GoodVibes GUI (bcipipeline.py) with specified parameters:
	▶ Hair washing tray	• 10 trials per condition
	▶ Alcohol	• 60-second baseline
	▶ Product for scrubbing	• 5.5-second choking
		• 11-second trial end
		• 3-second feedback
	𝑦) 2 𝑣𝑎𝑟(𝑥)𝑣𝑎𝑟(𝑦) ▶ • Launch Eye Tracker Receiver and Sender. ▶ Cable plugged into the computer PC Windows in the room -Eye Tracker:	(B.1)
	▶ Transmission box turned on	• Start ConsoleApplication1 after initiating scenario.
	▶ Charged batteries	with
		4. Hardware Setup	𝑐𝑜𝑣(𝑥, 𝑦) = 2
	Robot Reachy
	▶ Wired
	▶ Powered

,

𝑣𝑎𝑟(𝑥) = 𝑞 1 + 𝑞 2 𝑛 1 + 𝑛 2 -(𝑠 1 + 𝑠 2 ) 2 (𝑛 1 + 𝑛 2 ) 2 (B.3)

𝑣𝑎𝑟(𝑦) =

▶ Install 64-electrode bonnet, reference on TP9 and ground on TP10.

5. Instructions

▶

State-of-the-Art in brain machine interfaces

* Macbeth, Shakespeare

† De Oratore, Cicero

Mental imagery timing affects hybrid BCI control of robotic arms

General Conclusions and future developments

General Conclusions and future developmentsFigure 6.2: Autour d'un point F.Kupka, 1920-1930

Remerciements

A.2 Sources of chapters' subtitles

▶ Concerning BCIs. This chapter is largely concerned with the brain and the connected machine, and from its pages a reader may discover much of their character and a little of their story. Adapted from the prologue of The Lord of the Rings, J.R.R Tolkien.