Martijn Arns

Jean-Marie Batail

Stéphanie Bioulac

Marco Congedo

Christophe Daudet

Dominique Drapier

Thomas Fovet

Renaud Jardri

Fabien Le Van Quyen

Lotte

Le Van Quyen

Fabien Lotte

David Mehler

Jean-Arthur Micoulaud
email: jarthur.micoulaud@gmail.com

Franchi

Diane Purper-Ouakil

François Vialatte

Dr Micoulaud Franchi

Neurofeedback: One of today's techniques in psychiatry?

Keywords: Neurofeedback, EEG, real-time fMRI, psychiatric disorder 29

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Mots clefs 32

Neurofeedback ; Électroencéphalographie ; imagerie fonctionnelle par résonance magnétique 33 en temps réel ; Troubles psychiatriques 34 However, the use of neurofeedback as a therapeutic tool for psychiatric disorders remains 5 controversial. The aim of this review is to summarize and to comment the level of evidence of 6 electroencephalogram (EEG) neurofeedback and real-time functional magnetic resonance 7 imaging (fMRI) neurofeedback for therapeutic application in psychiatry. 8

Method 9

Literature on neurofeedback and mental disorders but also on Brain Computer Interfaces 10 (BCI) used in the field of neurocognitive science has been considered by the group of expert 11 of the NExT (Neurofeedback Evaluation & Training) section of the French Association of 12

Biological Psychiatry and Neuropsychopharmacology (AFPBN). 13

Results 14

Results show a potential efficacy of EEG-neurofeedback in the treatment of attentional-15 deficit/hyperactivity disorder (ADHD) in children, even if this is still debated. For other 16 mental disorders, there is too limited research to warrant the use of EEG-neurofeedback in 17 clinical practice. Regarding fMRI-neurofeedback, the level of evidence remains too weak, for 18 now, to justify clinical use. The literature review highlights various unclear points, such as 19 indications (psychiatric disorders, pathophysiologic rationale), protocols (brain signals 20 targeted, learning characteristics), and techniques (EEG, fMRI, signal processing). 21

Conclusion 22

The field of neurofeedback involves psychiatrists, neurophysiologists and researchers in the 23 field of brain-computer-interfaces. Future studies should determine the criteria for optimizing 24 neurofeedback sessions. A better understanding of the learning processes underpinning 25 neurofeedback could be a key element to develop the use of this technique in clinical practice. Neurofeedback can be considered as a biofeedback technique (i.e. a technique which consists 2 in measuring a physiological activity using a technical interface to extract a parameter of 3 interest; this parameter is then presented in real-time to the participant, typically via visual or 4 auditory feedback [1]; the goal is to teach the subject to modify the parameter). When the 5 physiological activity is a brain activity, biofeedback is called neurofeedback. Thus, 6 neurofeedback allows the subject to voluntary modulate his/her related brain and cognitive 7 activities [1, 2](see Figure 1). 8

The first observation of neurofeedback, was based on the classical conditioning principles 9 controversy exists, particularly in psychiatry and neurology [START_REF] Micoulaud--Franchi | Neurofeedback: time needed for a promising non--17 pharmacological therapeutic method[END_REF]11]. With regard to the 26 efficacy of neurofeedback in brain disorders, opinions within the scientific community appear 27 to be rather sharply divided [START_REF] Micoulaud--Franchi | Neurofeedback: time needed for a promising non--17 pharmacological therapeutic method[END_REF][START_REF] Micoulaud--Franchi | Electroencephalographic 21 neurofeedback: Level of evidence in mental and brain disorders and suggestions for 22 good clinical practice[END_REF]12] However, studies that added EEG neurofeedback to methylphenidate treatment did not report 6 'add-on' improvements on clinical symptoms [START_REF] Sherlin | Neurofeedback and basic lerning therory: 3 implications for research and practice[END_REF]22] or cognitive function [23]. 7

Other psychiatric disorders 8

There has been too limited research (i.e. lack of RCTs and independent replications) on the 9 following indications to warrant its use in clinical practice: Depression [24], Addictions [25, 10 26], Anxiety disorders [27,28]. 11

Advantages and pitfalls of EEG neurofeedback 12

Despite the meta-analyses presented before, the effectiveness of EEG neurofeedback in 13 treating ADHD remains debated because of the studies that were included [12,[29][START_REF] Sonuga--Barke | Response to Chronis--Tuscano et al. and 29 Arns and Strehl[END_REF][START_REF] Van Dongen--Boomsma | Dr van Dongen--Boomsma replies[END_REF][START_REF] Van Dongen--Boomsma | Dr van Dongen--33 Boomsma and colleagues reply[END_REF][33][START_REF] Dagenais | Doubting the 38 efficacy/effectiveness of electroencephalographic neurofeedback in treating children 39 with attention--deficit/hyperactivity disorder is as yet unjustified[END_REF]. These 14 choices warrant some explanations. For example, in the meta-analysis of Micoulaud-Franchi analysis, who reported that when focusing on 'standard neurofeedback protocols' significant 25 effects are found for both parent as well as teacher rated symptoms. Further emphasizing the 26 need to evaluate neurofeedback not as a singular phenomenon (neurofeedback as an umbrella 27 term i.e. medication) but evaluate it based on the specific protocol used (specific protocol i.e. 28 antidepressant, psychostimulant) [15]. These aspects are too rarely discussed in the debate of 29 EEG neurofeedback efficacy. Considering the absence of a current consensus [12, [START_REF] Zuberer | Are treatment effects of neurofeedback training 1 in children with ADHD related to the successful regulation of brain activity? A review on 2 the learning of regulation of brain activity and a contribution to the discussion on 3 specificity[END_REF][39][40], 30 these points will be crucial in the next years to gradually improve the practice of EEG 31 neurofeedback in psychiatry [41].

Two groups of technical issues can be identified in EEG neurofeedback protocols: i) 1 electrophysiology because the practice of EEG neurofeedback requires high quality 2 recordings of EEG signal [START_REF] Micoulaud--Franchi | Electroencephalographic 21 neurofeedback: Level of evidence in mental and brain disorders and suggestions for 22 good clinical practice[END_REF]36]; ii) learning because the practice of EEG neurofeedback 3 requires attention to some important technical aspects as described below and in Table 1. 4

The number of sessions is the first technical aspect, which is usually between 20 and 30, one 5 to three times per week, but the ideal number and the optimum inter-session duration have not 6

been defined yet [START_REF] Micoulaud--Franchi | The covariation of independent and 5 dependant variables in neurofeedback: a proposal framework to identify cognitive 6 processes and brain activity variables[END_REF]. It should be noted that efficacy with regards to the inattention 7 dimension in ADHD is proportional to the number of neurofeedback sessions [16] and 8 seemed to be maintained over time [43]. 9

Second is the choice of the threshold of reward, which is essential. Adjusting a threshold (and 10 a given occupation time) determines the number of positive reinforcements required to 11 strengthen the subject in a type of neurocognitive strategy. The threshold may be set 12 automatically or manually. When the threshold is determined automatically there is a 13 continuous updating of a threshold in order to give positive reinforcement to the subject for a 14 given percentage of occupation time below or above the threshold. The threshold is 15 continuously calculated according to signal just before. When the threshold is determined 16 manually, the professional determines the threshold based on a baseline recorded before the 17 neurofeedback session. If the number of positive reinforcement is too high or too low during 18 the session, the professional can adjust the threshold. The manual threshold seems to lead to 19 better learning [1,[START_REF] Micoulaud--Franchi | The covariation of independent and 5 dependant variables in neurofeedback: a proposal framework to identify cognitive 6 processes and brain activity variables[END_REF]. Indeed, if the subject is being asked to increase the amplitude of a 20

given brain activity and the threshold is calculated automatically, he will always be getting a 21 percentage of feedback even if the amplitudes are decreased across time. However, the 22 manual threshold requires performing a baseline measurement before each session and the 23 adjustment during the session by the professional complicates the standardization of 24 neurofeedback protocol. 25

Third is the type of positive reinforcement. This can be visual or auditory, proportioned 26 (graduated) or binary (present or absent), immediate or delayed, simple or complex, and 27 frequent or rare. Visual feedback, which is proportionate, immediate and simple, seems to 28 allow for better learning [START_REF] Micoulaud--Franchi | The covariation of independent and 5 dependant variables in neurofeedback: a proposal framework to identify cognitive 6 processes and brain activity variables[END_REF]. The number of reinforcements must be sufficient to maintain 29 the motivation of the subject. However, if the number of reinforcement is too high the 30 learning process can be altered [39,[START_REF] Micoulaud--Franchi | The covariation of independent and 5 dependant variables in neurofeedback: a proposal framework to identify cognitive 6 processes and brain activity variables[END_REF]. Note that positive reinforcement incorporated in an 31 entertaining interface (such as video games) may increase the motivation of the subject but 32 could impair learning according to some authors [1,14]. 33

Fourth is the evaluation of the training parameter during one session (evolution of the 1 performance), and the evaluation of the learning curve across the sessions (evolution of the 2 training parameter) that should be determined to ensure that a learning process occurs during 3 neurofeedback treatment. Lastly, the "transfer sessions" allow for the generalization of skills 4 learned in daily life [12,14,40]. 5

EEG neurofeedback and the vigilance system 6

Neurophysiological targets for EEG neurofeedback in ADHD are underpinned by 7 pathophysiological relevance related to the vigilance system. EEG neurofeedback 8 traditionally records a limited amount of information provided by a single electrode placed on 9 the scalp. This information concerns the EEG power in certain spectral bands: the beta band 10 (12-21 Hz) and the theta band (4-8 Hz) [START_REF] Oken | Vigilance, alertness, or sustained attention: 17 physiological basis and measurement[END_REF]45]. In a simple manner, an increase in the central 11 frontal beta band can be related to an increase in vigilance [46], and an increase in central 12 frontal theta band is related to a decrease in vigilance with subjective diurnal sleepiness and 13 possibly entering the first stage of sleep [45,[START_REF] Strijkstra | Subjective sleepiness correlates 24 negatively with global alpha (8--12 Hz) and positively with central frontal theta (4--8 Hz) 25 frequencies in the human resting awake electroencephalogram[END_REF]. Interestingly, an increase in theta power and 14 a decrease in beta power were observed in a subgroup of ADHD patients (greater theta/beta 15 (TBR) ratio) [48]. These EEG patterns suggest a link between the vigilance system, sleep 16 problems and ADHD (particularly in the subgroup with the greater TBR ratio) [49]. As a 17 result, decreasing TBR can be a potentially interesting target for EEG neurofeedback [START_REF] Arns | Differential effects of theta/beta and SMR 32 neurofeedback in ADHD on sleep onset latency[END_REF][51][START_REF] Monastra | The effects of stimulant therapy, EEG 36 biofeedback, and parenting style on the primary symptoms of attention--37 deficit/hyperactivity disorder[END_REF]. 18 Indeed, it was shown that TBR neurofeedback is more effective in the subgroup of patients 19 with the greater TBR ratio [53]. 20

Several studies have also demonstrated that sensori-motor rhythm neurofeedback (SMR), a 21 More specifically, it was recently demonstrated that SMR neurofeedback in ADHD resulted 24 in reduced inattention, hyperactivity and impulsivity, and these effects were mediated by 25 reduced sleep onset latency [START_REF] Arns | Differential effects of theta/beta and SMR 32 neurofeedback in ADHD on sleep onset latency[END_REF], further demonstrating a causal link between delayed sleep 26 onset latency and ADHD symptoms, specifically inattention. The TBR neurofeedback 27 overlaps with the SMR protocol, with clinical effects on ADHD indistinguishable from SMR 28 neurofeedback. However, the effect of TBR neurofeedback was not be mediated via sleep 29 onset latency normalization [START_REF] Arns | Differential effects of theta/beta and SMR 32 neurofeedback in ADHD on sleep onset latency[END_REF]. The effect of TBR neurofeedback could be mediated via a 30 reduction in diurnal sleepiness [49], but further research is needed to investigate the exact 31 working mechanism of TBR neurofeedback in ADHD [14]. 32

EEG neurofeedback and new target methods 1

The major limitation of "traditional" neurofeedback resides in the limited information 2 provided by a single electrode placed on the scalp, which is a differentially measured 3 potential with respect to a reference electrode. It is known that the EEG signal reflects mainly 4

the superposition of the electric potential created by ionic charge oscillation (due to 5 postsynaptic potentials) around the pyramidal cells found in the neocortex [START_REF] Congedo | On the blind source separation of human 1 electroencephalogram by approximate joint diagonalization of second order statistics[END_REF]. The potential 6 generated from a large population of neurons beneath the electrode are superimposed to create 7 the measurable EEG. Put differently, the response of the electrode is highly spatially 8 unspecific. It has been suggested that this lack of spatial specificity may impede the ability of 9 subjects to acquire control over the region of interest (ROI), i.e., the brain structures to be 10 trained [58]. Another limitation of traditional neurofeedback is the filtering resulting from the 11 choice of the reference electrode placement; depending on the position of the active and 12 reference electrode on the scalp, the measurement is sensitive to current flowing in the ROI 13 along one direction only. Therefore, a considerable improvement in the neurofeedback 14 technique can be obtained considering spatial-specific brain activity, solving implicitly the 15 issue of the chosen reference. Two possible improvements in this sense have been proposed, 16 namely, basing the neurofeedback not on the signal captured by the two scalp electrodes but 17 on EEG inverse solutions or on EEG blind source separation. Both methods require the use of 18 multiple electrodes (a minimum of eight); it is indeed the spatial information contained in 19 such a multivariate EEG recording that allow for better estimates of the ROI's current. 20

EEG neurofeedback based on inverse solutions 21

An EEG inverse solution is a mathematical method used to estimate the intracranial current 22 generated in the observed scalp potential. Once the current is estimated in the ROI, its density 23 (energy) provides an appropriate feedback signal. By acquiring data from 19 electrodes, 24

Congedo, Lubar and Joffe (2004) demonstrated learned control of the cognitive division of 25 the anterior cingulate cortex using the inverse solution known as low resolution 26 electromagnetic tomography (LORETA) [59,60]. Subsequent studies confirmed the viability 27 and further explored the correlates of LORETA-neurofeedback of the anterior cingulate 28 cortex [START_REF] Cannon | Differentiating a network of executive 11 attention: LORETA neurofeedback in anterior cingulate and dorsolateral prefrontal 12 cortices[END_REF]62]. This preliminary work was replicated and reiterated later by several other 29 research groups using other inverse solutions in proof-of-concept studies [START_REF] Liechti | First clinical trial of tomographic 16 neurofeedback in attention--deficit/hyperactivity disorder: evaluation of voluntary 17 cortical control[END_REF]64].

EEG neurofeedback based on BSS/ICA 1

Over the past 20 years, research on blind source separation (BSS) has developed into a 2 burgeoning signal processing method with applications across a wide variety of fields. It has 3 since been proven valuable in identifying cortical sources of brain activity associated with 4 cognitive task performance [START_REF] Onton | Frontal midline EEG dynamics during working 21 memory[END_REF]. Such a spatial filtering technique may provide an ideal way 5 to train specific brain regions or networks in a neurofeedback setting. In fact, a blind source 6 separation filter can estimate both the location and the direction of current, thus yielding a 7 sharper filter compared to an inverse solution [66]. Further advantages of such spatial filters 8 are that they are computationally inexpensive (important for 'real-time' feedback) and 9 potentially more robust in the presence of artefacts. The viability of BSS neurofeedback has 10 been explored in two studies; the first aimed to suppress excessive theta in deep frontal 11 medial regions for the treatment of obsessive-compulsive disorder [START_REF] Koprivova | Prediction of treatment response and the 27 effect of independent component neurofeedback in obsessive--compulsive disorder: a 28 randomized, sham--controlled, double--blind study[END_REF], the second aimed to 12 enhance theta activity on a source localized into deep medial-temporal regions associated 13 with spatial-navigation abilities [68]. 14

EEG neurofeedback based on stereotactic EEG 15

As early as the 1960s, the important work by [START_REF] Fetz | Operant conditioning of cortical unit activity[END_REF] on primates showed the operant 16 conditioning of single cell spike trains in the motor cortex [START_REF] Fetz | Operant conditioning of cortical unit activity[END_REF]. The motor cortex is probably 17 neuropsychiatric conditions. Such a technique has been called BrainTV [START_REF] Durup | L'électroencéphalogramme de l'homme. Observations psycho--8 physiologiques relatives à l'action des stimuli visuels et auditifs[END_REF]. The technique 27 enables to combine the spatial resolution of fMRI neurofeedback and the temporal resolution 28 of scalp-level EEG neurofeedback [74]. Thus, despite the invasive nature of BrainTV, these 29 protocols could be a response to some limitations of neurofeedback protocols in the future. 30

In this context, neurofeedback can indeed be performed in patients with drug resistant 31 epilepsy undergoing long-term monitoring, where depth electrodes are implanted for clinical 32 diagnostics. The effects of self-induced intracortical oscillatory activity (4-8 Hz) were studied 33 in several neurosurgical patients. It was found that subjects learned to robustly and 1 specifically induce oscillations in the target frequency, confirmed by increased oscillatory 2 event density [75]. As controls improved during learning, induced oscillatory activity at the 3 target electrode became functionally decoupled from distant sites, which predicted the 4 individual session-to-session performance variability. Furthermore, in another study [75], 5 patients were trained to up-regulate the relative proportion of the gamma rhythm at different 6 fronto-temporal cortical locations. In line with previous findings, on monkeys using direct 7 cortical recordings [START_REF] Engelhard | Inducing gamma oscillations and precise spike 47 synchrony by operant conditioning via brain--machine interface[END_REF], it was found that most subjects learned to specifically increase local 8 cortical gamma power. These findings suggest that the effects of voluntary control of 9 intracortical oscillations can be exploited to specifically target plasticity processes to 10 reconfigure network activity, with a particular relevance for memory function or skill 11 acquisition [START_REF] Bagdasaryan | Experiencing your brain: neurofeedback as a new 1 bridge between neuroscience and phenomenology[END_REF]. In particular, abnormalities in gamma oscillations exist in a number of 12 neurologic and psychiatric diseases [78]. Thus, the specific rectification of gamma 13 oscillations could ameliorate some of the deficits caused by these pathological conditions 14

Level of Evidence 26

Due to the novelty of the technique, the studies that have so far provided evidence for the 27 clinical use of fMRI neurofeedback are limited. This section will focus on recent 28 developments in the field and on clinical and translational applications. A more 29 comprehensive review on relevant designs and training paradigms can be found elsewhere 30

[10]. 31

Major Depressive Disorder, the emblematic disorder 1

The psychiatric disorder most studied in the context of fMRI neurofeedback is major 2 depressive disorder. The use of fMRI neurofeedback in treating depression is based on the 3 pathophysiological model of emotional dysfunction during a depressive episode [START_REF] Giacobbe | Treatment resistant depression as a failure of 9 brain homeostatic mechanisms: implications for deep brain stimulation[END_REF]82]. 4

Therefore, published studies have so far mainly focused on the up-regulation of brain areas or 5 even on specific structures that are involved in emotions, including parts of the limbic system 6 (e.g., the amygdala) and the ventral prefrontal cortex [START_REF] Micoulaud--Franchi | Towards a new approach of 15 neurophysiology in clinical psychiatry: Functional magnetic resonance imaging 16 neurofeedback applied to emotional dysfunctions[END_REF]. To date, no randomized control 7 trials (RCTs) have been published, and the current literature consists exclusively of open label 8 and pilot studies [START_REF] Hamilton | Effects of salience--network--node 18 neurofeedback training on affective biases in major depressive disorder[END_REF][START_REF] Young | Real--time FMRI neurofeedback training of 21 amygdala activity in patients with major depressive disorder[END_REF][START_REF] Linden | Real--time self--regulation of emotion networks 23 in patients with depression[END_REF]. These studies have demonstrated the feasibility of the technique and 9

suggested that patients are able to self-regulate their brain activity in target areas. Further, 10 improvements in mood were only found in the group that received fMRI neurofeedback 11 training but not in a control group, suggesting a link between neurofeedback success, positive 12 emotions (as accessed by self-reports in autobiographic memory recall and happiness ratings) 13 and clinical improvement (e.g., HDRS-17). To rule out the unspecific effects (e.g., regression 14 to the mean) of these pilot findings, RCTs are needed that are based on larger samples and 15 appropriate clinical control conditions, including randomization and blinded assessments. 16

Advantages and pitfalls of fMRI neurofeedback 3

The gold standard for evaluating a therapeutic technique requires assessing its efficacy in a 4 double-blind randomized and placebo-controlled trial. However, some of these requirements 5 can pose a challenge for the evaluation of fMRI neurofeedback training. First, implementing a 6 double-blind design can be limited because most current training protocols require (at least in 7 the early learning phase) that patients engage in specific conscious processes in the form of 8 explicit mental strategies. 9

Second, designing an appropriate placebo-controlled condition for neurofeedback protocols 10 requires careful consideration depending on the study type. Three main types of controls have 11 been proposed and tested so far: 12

• Transfer runs, during which patients are instructed to engage in the same cognitive 13 strategies in or outside the scanner but without being provided with neurofeedback. 14

• "Sham" neurofeedback, which entails either random or yoked feedback based on some 15 other patient's brain activity. However, sham feedback bears the risk that patients 16 notice the non-contingency of the feedback [10]. 17

• An active control group that receives veridical feedback from target areas of another 18 functional system that is neither involved in the pathophysiology of the respective 19 condition nor in the task (i.e., cognitive strategy) of interest. However, a recent study 20 has demonstrated that neurofeedback training itself involves various brain regions 21 besides the individual target areas, including structures of reward circuitry (basal 22 ganglia, striatum) and parts of the prefrontal cortex [START_REF] Emmert | Meta--analysis of real--time fMRI neurofeedback 25 studies using individual participant data: How is brain regulation mediated? 26[END_REF]. Further, such a control group 23 cannot control for potential unspecific effects due to the high-tech laboratory setting. 24

Including a third treatment as a usual control group that receives standard therapy 25 could address this problem at the expense of increased trial costs. 26

Third, it remains to be tested how to optimize neurofeedback protocols for psychiatric 27 conditions. This includes: 28

• Defining effective target areas or the networks for a particular psychiatric condition 29 based on a pathophysiological model. Target areas can either be chosen a priori based 30 on anatomical landmarks, or they can be functionally defined using a so-called 31 "localizer" task (e.g., presenting emotionally valenced visual stimuli in a 32 neurofeedback protocol for depression [START_REF] Linden | Real--time self--regulation of emotion networks 23 in patients with depression[END_REF]). Similarly, target areas for functional 33 connectivity-based neurofeedback are determined by the correlation of activity among 1 brain areas that belong to a network of interest. 2

• Determining efficient study designs with regard to the duration and number of 3 sessions to exploit regarding the learning capacities of patients who have cognitive 4 impairments (e.g., attention and memory deficits). 5

• The nature of task instructions for patients, either given explicit strategies at hand 6 (e.g., imaging positive autobiographical memories) or task instructions that rather 7 focus on the goal to achieve a certain target level in the feedback while patients learn 8 implicitly the effect of various strategies [88]. 9

• The design of the interface, such as the modality of feedback (e.g., visual, auditory or 10 tactile), the mode of feedback presentation (e.g., continuous or intermittent) and the 11 complexity of the presented feedback (e.g., for visual feedback, a thermometer display 12 or more complex scenes based on virtual reality) 13

fMRI neurofeedback and new target method 14

As previously described, different strategies exist to optimally define the brain target, or the 15 region(s) of interest (ROI), in fMRI neurofeedback protocols [10]. This ROI can be localized 16 using structural information but can also be functionally defined. In the latter, the patient is 17 asked to perform a specific task in the scanner, and the highlighted areas can be used as the 18 ROI for the fMRI-neurofeedback in a second step (e.g., in [START_REF] Linden | Real--time self--regulation of emotion networks 23 in patients with depression[END_REF]). 19

For fMRI neurofeedback with a therapeutic purpose, both of these methods rely on our a 20 priori knowledge of the underlying neural mechanisms of the disorder/symptom we want to 21 relieve. Such strategies appear very relevant for disorders with persistent (or tonic) symptoms, 22 i.e., symptoms that do not change much over time (e.g., depressive mood) but pose special 23 challenges for more acute symptoms, characterized by intrusiveness and phasic activity (e.g., 24 hallucinations in schizophrenia or obsessions in obsessive compulsive disorder). For the latter 25 symptoms, which are associated with transitory brain-states, strategies using pre-defined 26 anatomical targets appear poorly appropriate. On the contrary, training patients to self-27 regulate the activity of brain regions that re-activate during the occurrence of subjective 28 symptoms could be an interesting alternative. 29

To address this issue, a first method could be to induce symptoms while scanning to localize 30 functional activations associated with the targeted subjective experience that can then be used 31 as the ROI for fMRI neurofeedback. However, in some cases (such as hallucinations [89]), 32 symptom provocation may not be possible, and another method to detect the onset of 1 symptoms together with the associated brain activation patterns is needed. 2 Machine-learning, and particularly the recent development for fMRI analysis of linear support 3 vector machines (lSVMs), offers several advantages in this context. Such techniques classify 4 functional or anatomical patterns using a multivariate strategy and thus allow for decoding 5 and capturing the fine-grained spatial pattern of BOLD activity to predict future mental states, 6 such as perception or free choices [START_REF] Kriegeskorte | Visual Population Codes: Toward a Common 33 Multivariate Framework for Cell Recording and Functional Imaging[END_REF]. In the same way, it is now possible to develop 7 classifiers able to quickly detect the emergence of subjective symptoms by detecting specific 8 patterns of brain activity identified during symptomatic periods [91,92]. Such fine-grained 9 activity patterns can be used as the signal that is fed back to the patient during neurofeedback 10 protocols. However, to be eligible for this strategy, the patient's symptoms must exhibit some 11 specific features, such as frequent occurrence (i.e., the symptom must occur several times 12 during the fMRI session) [93]. 13

Combining lSVM (or other advanced machine learning classifiers) and fMRI neurofeedback 14 could constitute a promising way to develop fMRI neurofeedback for the treatment of phasic 15 psychiatric symptoms. However, considering the potential cost necessary to implement fMRI 16 neurofeedback, proof-of-concept studies are urgently required. 17

Human learning and neurofeedback 18

The learning process is crucial in neurofeedback and requires models to understand the 19 mechanism of feedback learning [94]. A good practice guide is also of critical importance for 20 the evaluation of these interventions and to reach higher standards in clinical practice [START_REF] Micoulaud--Franchi | Electroencephalographic 21 neurofeedback: Level of evidence in mental and brain disorders and suggestions for 22 good clinical practice[END_REF]. 21

Learning during neurofeedback can be either explicit or implicit [94]. In the explicit learning 22 process, the user observes a feedback signal, which is a direct correlate of the neurosignal to 23 be regulated. In the implicit learning process, the signal is not explicitly presented to the 24 subject but instead changes some detail(s) of the experimental conditions. For example, a 25 person using a videogame whose content (e.g., changing levels of difficulty or access to 26 bonus items) evolves depending upon his frontal alpha rhythm is receiving implicit feedback; 27 he/she does not know directly that his brainwaves have changed, but he/she experiences 28 indirect effects of this physiological change. 29

Theory of human learning 30

From the perspective of the experimenter, operant conditioning has historically been the 31 dominant interpretation of neurofeedback mechanisms; in this case the feedback is modeled 32 as an implicit infra-cognitive reinforcement learning (RL) signal [1]. Such an approach is 1 indeed supported by animal studies: for example, prefrontal cortical neurons can be controlled 2 by rhesus monkeys through an operant conditioning paradigm [START_REF] Schafer | Selective attention from voluntary control of neurons in 44 prefrontal cortex[END_REF]. The problem lies with the 3 definition of the reward: the interpretation of the biosignal depends upon the motivational 4 state of the subject. Furthermore, RL has two possible mechanisms [96]: 5

• either the subject is in a goal-directed setup and supports his learning from an internal 6 model, in which case learning is termed as model-based RL; 7

• or the subject has no model of the outside events and learning arises from simple 8 associations, termed as model-free RL. 9

The two issues associated with operant conditioning are therefore to determine the reward 10 mechanisms and the type of RL. 11

From the perspective of the subject, neurofeedback relies on two specific biofeedback skills 12 [START_REF] Epstein | Biofeedback, self--control, and self--management[END_REF]: 13

• discrimination, which is the aptitude to achieve an inner perception of the biological 14 variable, 15

• and self-maintenance, which is the ability to affect the biological variable and to 16 effectively change it in the intended direction. 17

The acquisition of these skills could be either explicit or implicit, depending on the type of 18

neurofeedback. 19

During an implicit neurofeedback procedure, learning is more likely to follow a model-free 20 RL mechanism. The subject scans the different percepts available to him/her at a given time. 21

Several levels of salience filters attribute weights to both external and internal percepts based 22 on their physical, temporal, motivational, and emotional properties [98]. The resulting neural 23 representations then go through a competitive selection process to determine which 24 information enters working memory (WM). This filtering layer is referred to as bottom-up 25 attention and will, for example, allow a loud, unexpected sound to enter almost anyone's WM 26 (in addition to triggering subcortical responses). 27

During an explicit neurofeedback procedure, a model-based RL is triggered: the subject seeks 28 to reach a goal (regulating the feedback signal). Top-down signals may therefore alter the 29 bottom-up selection process by modifying the behavior of salience filters (e.g., emotional 30 regulation) or by enhancing or inhibiting a neural representation that has already entered WM 31 and has gained or lost salience through high-level processing (voluntary attention and percept 32 inhibition, respectively). The subjects will then manipulate their different neural 33 representations to determine if a correlation between the feedback and the neural 34 representation can be established with the feedback, which is a typical set-shifting task. Set-1 shifting indeed refers to the ability to switch between different high-level neural 2 representations of a percept on the basis of a feedback [99]. Sustained attention is another top-3 down component of attention and refers to the ability to maintain neural representations in 4 WM over time [START_REF] Gazzaley | Top--down modulation: bridging selective attention and 6 working memory[END_REF], which is necessary for long-lasting neurofeedback sessions. 5

The interaction between these top-down and bottom-up processes lead to the dual-process 6 theory for neurofeedback mechanisms [START_REF] Wood | On the need to better specify the concept of 8 "control" in brain--computer--interfaces/neurofeedback research[END_REF] (Figure 2), a theory that categorizes the 7 cognitive functions supporting neurofeedback into two main types of processing: 8

• more automatic and capacity-free processes 9

• vs. more controlled and capacity-limited processes. 10

These two processes lead to opposing perspectives on proper feedback designs: 11

• one based on bottom-up operant conditioning strategies [START_REF] Sterman | Foundation and Practice of Neurofeedback for the 11 Treatment of Epilepsy[END_REF]; 12

• and another based on a top-down cognitive paradigm where higher cognitive functions 13 percolate down from large-scale oscillations to small-scale and single-neuronal 14 activities [START_REF] Bagdasaryan | Experiencing your brain: neurofeedback as a new 1 bridge between neuroscience and phenomenology[END_REF]. 15

Recent models of explicit neurofeedback learning are based on a top-down skill learning 16 paradigm [START_REF] Micoulaud--Franchi | The covariation of independent and 5 dependant variables in neurofeedback: a proposal framework to identify cognitive 6 processes and brain activity variables[END_REF]. Skill learning is a paradigm that describes the mechanisms involved in the 17 acquisition of complex perceptual, cognitive, or motor skills. One can identify two significant 18 properties of a motor action [START_REF] Salmoni | Knowledge of results and motor learning: a 13 review and critical reappraisal[END_REF]: 19

• its performance, i.e., the quality of the subject's own movement (how to do the 20 action); 21

• and its result, i.e., the success or failure of the action (what shall be performed). 22

The subject can learn about these two properties either by himself or with external help. 23

When the subject has direct access to these two observables, it is termed "intrinsic feedback." 24 When the information comes from an external source (for example, a sports coach or a 25 device), it is termed "external feedback." Extrinsic feedback helps to accelerate and facilitate 26 the learning process [104], especially when it is not redundant with internal feedback. It has 27 informational functions and motivational properties with important influences on learning 28 [START_REF] Wulf | Motor skill learning and performance: a review of 17 influential factors[END_REF]. Successful feedback learning is an adaptation of internal feedback in a way that 29 incorporates the external feedback [START_REF] Synofzik | Internalizing agency of self--action: perception of 19 one's own hand movements depends on an adaptable prediction about the sensory 20 action outcome[END_REF]. Neurofeedback provides scaffolding for the subject, 30 helping him/her to acquire or improve task-related discrimination and self-maintenance skills. 31

A possible resolution of the apparent contradiction between top-down and bottom-up models 32 would be to postulate the existence of interactions between these two types of processing. 33

Model-free RL and model-based RL form two cooperative systems with model-free RL 34 driving online behavior and model-based RL working offline in the background to 1 continuously adjust model-free RL. Once the subject becomes proficient with the task, model-2 free RL progressively dominates with time. As a consequence, early explicit neurofeedback 3 learning can become implicit with time, and there is a continuum between the two learning 4 mechanisms [1]. 5

Human learning and Brain Computer Interface 6

A brain-computer interface (BCI) can be defined as a system that translates the brain activity 7 patterns of a user into messages or commands for an interactive application, this activity 8 being measured and processed by the system [START_REF] Lotte | Electroencephalography (EEG)--based Brain--22 Computer Interfaces[END_REF]. With a BCI, the user's brain activity is 9 usually measured via EEG and processed by the system. For instance, a BCI can enable a user 10 to move a cursor to the left or to the right of a computer screen by imagining left or right hand 11 movements, respectively. Because they make computer control possible without any physical 12 activity, EEG-based BCIs have revolutionized many applications areas, notably enabling 13 severely motor-impaired users to control assistive technologies, e.g., to control text input 14 systems or wheelchairs, as a rehabilitation device for stroke patients, or as new gaming input 15 device, for example [START_REF] Birbaumer | Brain--computer interfaces: communication and 25 restoration of movement in paralysis[END_REF][START_REF] Daly | Brain--computer interfaces in neurological rehabilitation[END_REF][START_REF] Wolpaw | Brain--computer interfaces for 29 communication and control[END_REF]. 16

Such BCI-based systems are used for communication and control applications in which the 17 user voluntarily sends mental commands to the application. These types of BCIs are known 18 either as active BCI (or explicit), when the user performs mental tasks (e.g., imagining 19 movement), or as reactive BCI, when the users have to attend to stimuli (e.g., flickering visual 20 images) [START_REF] George | An overview of research on "passive" brain--computer 31 interfaces for implicit human--computer interaction[END_REF][START_REF] Zander | Towards passive brain--computer interfaces: applying brain--35 computer interface technology to human--machine systems in general[END_REF]. There is yet another category of BCI: passive BCI (or implicit), for which 21 the mental state of the user is passively estimated, without any voluntary mental command 22 from the user, to adapt the application in real-time to this mental state [START_REF] George | An overview of research on "passive" brain--computer 31 interfaces for implicit human--computer interaction[END_REF][START_REF] Zander | Towards passive brain--computer interfaces: applying brain--35 computer interface technology to human--machine systems in general[END_REF]. 23 BCIs, similarly to neurofeedback, thus rely on a closed loop that exploits brain activity in real 24 time, specifically by acquiring EEG signals, preprocessing them (filtering), extracting 25 relevant features describing the user's state or intent and translating them into feedback to 26 close the loop. Although both BCIs and neurofeedback share similar technological tools, their 27 original purposes were very different: BCIs enable users to control an external object, such as 28 a computer or an orthosis, whereas neurofeedback enables their users to acquire control of 29 themselves. Although some BCIs, e.g., BCIs based on mental imagery tasks, involve a 30 learning process, and thus require the user to perform self-regulation, self-regulation is not the 31 final objective [START_REF] Lotte | Towards Improved BCI based on Human Learning Principles[END_REF]. As such, it can be said that neurofeedback is used to train users to learn 32 how to control a BCI. 33

It should be noted though that the boundaries between BCI and neurofeedback remain blurry 1 and are a subject of debate (see [START_REF] Perronnet | Brain Training with Neurofeedback[END_REF] for more detailed discussions). For instance, recently, 2 active BCI systems that can detect imagined movements of the hands have been used to 3 perform stroke rehabilitation by guiding users to self-regulate their brain activity in motor 4 brain areas damaged by stroke [START_REF] Ramos--Murguialday | Brain--machine interface in chronic 42 stroke rehabilitation: a controlled study[END_REF], similar to neurofeedback. Passive BCIs can also be used 5 to give feedback to a user regarding his own high-level mental states, such as mental stress or 6 attention, to implicitly help him/her to self-regulate those states [START_REF] Ramos--Murguialday | Brain--machine interface in chronic 42 stroke rehabilitation: a controlled study[END_REF], again, similar to 7 neurofeedback. 8

In these examples above, there are nonetheless differences between BCIs and neurofeedback. 9

Indeed, contrary to classical neurofeedback approaches, BCIs usually heavily rely on machine 10 learning tools to estimate some specific mental states [START_REF]A Tutorial on EEG Signal--processing Techniques for Mental--state 44 Recognition in Brain----Computer Interfaces[END_REF]. BCIs typically use a set of 11 example of EEG data that are recorded while the target user is in the mental state to be 12 detected. Such data are used to calibrate a classifier to recognize this mental state using 13 machine learning. Most neurofeedback approaches do not use a data-driven approach or 14 machine learning to provide feedback to the user. Nevertheless, there is no fundamental 15 constraint preventing neurofeedback from using machine learning as BCIs do, and future 16 neurofeedback approaches could benefit from machine learning algorithms initially developed 17 for BCI to provide more specific and robust feedback. 18

Overall, BCIs (both active/explicit and passive/implicit) and neurofeedback are clearly related 19 approaches and technologies. Although they are primarily studied separately, they could both 20 benefit from one another, notably in terms of EEG signal processing, feedback design and 21 user training. In the future, it is not unlikely that BCI and neurofeedback share similar 22 research paths. 23

Conclusion 24

This review highlights the growing body of evidence for use of neurofeedback in the field of 25 psychiatry. Neurofeedback remains a very promising technique thanks to the progress of i) 26 the techniques used (such as multivariate EEG recording for a better ROI localization, or 27 coupled EEG-fMRI neurofeedback protocols), ii) signal processing (such as EEG-low 28 resolution electromagnetic tomography or linear support vector machines in fMRI for phasic 29 psychiatric disorders), and iii) understanding of the learning skills (both model-free and 30

model-based reinforcement learning). 31

Thus, neurofeedback is a today's technique that is largely inspired by the original works of 32 Durup and Fessard. However, it remains to be clarified whether the therapeutic effect of 33 neurofeedback is clinically meaningful and how to optimally perform neurofeedback in a 1 clinical setting. The respective place of neurofeedback techniques in the clinical 2 armamentarium has to be defined. The field of neurofeedback involves psychiatrists, 3 neurophysiologists and researchers in the field of brain-computer-interfaces. Future studies 4 should determine the criteria for optimizing neurofeedback sessions. A better understanding 5 of the learning processes underpinning neurofeedback could be a key element to develop the 6 use of this technique in clinical practice. Dual process theory of neurofeedback. Bottom-up operant conditioning and top-down skill 2 learning processes improve self-maintenance and discrimination skills. Implicit feedbacks 3 interact mostly with the bottom-up system, whereas explicit feedbacks first interact with the 4 top-down system, before becoming progressively integrated as the subject becomes 5 independent from the feedback, which becomes then mostly a bottom-up reinforcement 6 signal, migrating towards the operant conditioning mechanism.

Introduction 2 8 9

 289 Le neurofeedback consiste à mesurer, chez un sujet, une activité cérébrale et à traiter le signal 3 au moyen d'une interface technique afin d'en extraire un paramètre d'intérêt qui sera présenté 4 en temps réel au participant sous la forme d'une information visuelle ou auditive. L'objectif 5 est d'apprendre au sujet à modifier ce paramètre et donc à moduler son activité cérébrale et 6 cognitive. Cependant, l'utilisation du neurofeedback en pratique clinique pour la prise en 7 charge des troubles psychiatriques reste controversée. Méthode Cet article présente une synthèse de la 1 ère journée nationale sur le neurofeedback organisé par 10 la section NExT (Neurofeedback Evaluation & Training) de l'Association française de 11 psychiatrie biologique et de neuropharmacologie (AFPBN). Un état des lieux de l'utilisation 12 du neurofeedback en électroencéphalographie (EEG) et en imagerie par résonance 13 magnétique fonctionnelle (IRMf) est proposé. Pour intégrer l'arsenal thérapeutique, cette 14 technique doit en effet répondre aux exigences de l'evidence based medicine. 15 Résultats 16 Les études montrent une efficacité probable du neurofeedback en EEG pour le trouble du 17 déficit de l'attention / hyperactivité (TDAH) chez les enfants. Pour les autres troubles 18 psychiatriques, le nombre d'études est encore trop limité pour se positionner. En ce qui 19 concerne le neurofeedback en IRMf, le niveau de preuve reste, pour l'heure, trop faible pour 20 justifier une utilisation clinique. Les modalités d'emploi du neurofeedback, notamment en ce 21 qui concerne les indications médicales, les protocoles d'utilisation (activité(s) cérébrale(s) 22 ciblée(s), caractéristiques d'apprentissage) et les outils de mesure employés (EEG, IRMf, 23 mode de traitement du signal) restent donc à clarifier. 24 Conclusion 25 Le vaste champ de recherche du neurofeedback implique à la fois des psychiatres, des 26 neurophysiologistes et des chercheurs du domaine des interfaces cerveaux-ordinateurs. Les 27 futurs travaux devront s'attacher à déterminer les critères permettant d'optimiser les séances 28 de neurofeedback afin de mieux comprendre ses effets, le tout dans l'optique d'une utilisation 29 en pratique clinique dans certaines indications. L'étude des processus d'apprentissage 30 constitue un élément clé autour duquel les futures recherches devront se focaliser. 31

26 Plan 1 12

 261 Introduction .....................................................................................................................................................5 2 Electroencephalographic neurofeedback (EEG-NF) .........................................................................6 3 2.1 Level of Evidence .................................................................................................................................6 4 2.1.1 Attentional-Deficit/Hyperactivity Disorder, the emblematic disorder .......................6 5 2.1.2 Other psychiatric disorders .......................................................................................................7 6 2.2 Advantages and pitfalls of EEG neurofeedback.........................................................................7 7 2.3 EEG neurofeedback and the vigilance system ............................................................................9 8 2.4 EEG neurofeedback and new target methods...........................................................................10 9 2.4.1 EEG neurofeedback based on inverse solutions .............................................................10 10 2.4.2 EEG neurofeedback based on BSS/ICA............................................................................11 11 2.4.3 EEG neurofeedback based on stereotactic EEG .............................................................11 12 3 Functional magnetic resonance imagery and neurofeedback ......................................................12 13 3.1 Level of Evidence ..............................................................................................................................12 14 3.1.1 Major Depressive Disorder, the emblematic disorder...................................................13 15 3.1.2 Other psychiatric disorders ....................................................................................................13 16 3.2 Advantages and pitfalls of fMRI neurofeedback ....................................................................14 17 3.3 fMRI neurofeedback and new target method ...........................................................................15 18 4 Human learning and neurofeedback..................................................................................................... 16 19 4.1 Theory of human learning...............................................................................................................16 20 4.2 Human learning and Brain Computer Interface.......................................................................19 21 5 Conclusion ...................................................................................................................................................

  applied to the electroencephalogram (EEG). Classical conditioning involves learning new 10 behaviors through the process of association. Neurofeedback originates from the 1930s based 11 on the work of Gustave Durup and Alfred Fessard, who were two emblematic figures of 12 psychophysiology and neurophysiology in France. They observed that brain activity (alpha 13 blocking response) could be modified according to the classical conditioning principles (i.e. 14 to develop an association between an EEG activity (alpha blocking response), a behavior and 15 cognitive response, and a signal of feedback [3]. In 1941, Jasper & Shagass published the first 16 systematic study that investigated classical conditioning of EEG [4]. Subsequent studies in the 17 1960s confirmed that alpha blocking could indeed be conditioned and related to some specific 18 cognitive activities of the trained subject [5]. 19 After a serious decline during the 1980s and 1990s, mainly due to the poor reliability of 20 methods used for recording brain activity, the technique gained ground again in the early 21 2000s with a renewed interest both in scientific and societal terms [6]. Thanks to the principle 22 on which it is based and to the fertile dynamic nature of ongoing research in a range of 23 clinical, therapeutic and fundamental topics, neurofeedback can be considered a technology of 24 today [6, 7]. However, despite great interest in neurofeedback research [8-10], significant 25

  15 et al. (2014), the well-controlled, randomized and blinded study conducted by Arnold et al. 16 (2013) [35] was not included because the EEG neurofeedback protocol was not based on the 17 basic learning theory used in standard EEG neurofeedback protocols (particularly because of 18 the type of reinforcement chosen) [1]. Moreover, the EEG recording was carried out using an 19 unconventional setup, with electrodes placed on the forehead, a region known to be 20 problematic for recording because of muscular artefacts. The study by Arnold et al. thus 21 highlights the need to avoid some pitfalls regarding technical issues of electrophysiology [36] 22 and technical issues of learning [1, 37] when a study on neurofeedback is conducted. In 23 further support of this notion is the above reported result from the Cortese et al. (2016) meta-24

  frequency that overlaps with to the TBR protocol, results in increased sleep spindle density 22 during sleep [54, 55], decreased sleep latency [54] and increased total sleep time [54, 56]. 23

  the most obvious place to search for cortical signals directly associated with volitional 18 movement [70]. This may be one of the reasons why a substantial part of invasive 19 neurofeedback research has been conducted on paralyzed or lock-in patients, recognizing the 20 need of people with disabilities and aiming to restore their communicative or motor functions. 21 In this context, brain-computer interfaces (BCIs) were tested in amyotrophic lateral sclerosis, 22 brain stem stroke and spinal cord lesions using cortical neuronal activity recorded by 23 implanted electrodes [71]. Nevertheless, conscious control has also been shown to be possible 24 at the cellular level in human temporal lobe structures [72]. The successful cases in these 25 applications encouraged the usage of invasive neurofeedback for other neurological and 26

[ 77 ]. 15 3

 7715 Functional magnetic resonance imagery and neurofeedback 16Real-time functional magnetic resonance imaging neurofeedback (fMRI neurofeedback) is a 17 rather recent development for providing neurofeedback training based on blood oxygenation 18 contrasts (blood-oxygen level dependent, BOLD) [79]. fMRI neurofeedback training can 19 overcome some limitations of more traditional forms of neurofeedback, such as EEG-20 neurofeedback, because of its better spatial resolution and whole brain coverage. In particular, 21 the whole brain coverage makes fMRI neurofeedback a promising technique for non-invasive 22 psychiatric rehabilitation because it allows for training patients in self-regulating subcortical 23 brain areas [80]. Depending on the disease model of interest, patients can be either trained to 24 increase or decrease the activity of relevant brain areas [10]. 25

  Two ongoing (Young, clinicaltrials.gov: NCT02709161; Moll et al., NCT01920490), one 17 completed single blind (Linden et al., NCT01544205), and one completed double blind 18 (Young et al., NCT02079610) RCTs are currently listed. 19 3.1.2 Other psychiatric disorders 20 For other psychiatric conditions, such as schizophrenia, addiction, obsessive compulsive 21 disorder and eating disorder, the feasibility of fMRI neurofeedback training has been 22 investigated in pilot studies with small sample sizes (for review [10]). These studies used 23 different target areas such as the insula in schizophrenia (based on a facial emotion 24 recognition paradigm) and in psychopathic personality disorder (regulation of fear circuitry) 25 and the anterior cingulate cortex in controlling cravings in nicotine addiction. The 26 Collaborative Research Project BRAINTRAIN is a European consortium that focuses on the 27 improvement and translation of real-time fMRI neurofeedback protocols for clinical 28 applications (braintrainproject.eu). Current registered RCTs investigate therapeutic effects of 29 fMRI neurofeedback in alcohol addiction (Linden et al., NCT02486900), Anxiety in 30 adolescents (Cohen-Kadosh et al., NCT02440451) and autism spectrum disorder (Castelo-functional connectivity between reward-and impulse-related brain areas in eating disorders 1 (Hallschmid et al., NCT02148770). 2

7

 7

  Figure 1 1 Principle of neurofeedback 2 3

7

 7

2 Electroencephalographic neurofeedback (EEG-NF) 3 2.1 Level of Evidence 4

	1	research on the learning process through the link between neurofeedback and brain computer
	2	interfaces (BCIs) is discussed.
	5	Most trials on the efficacy of EEG neurofeedback in psychiatric disorders have significant
	6	methodological weaknesses (in particular: size of the population studied, none randomized or
	7	none blinded protocol, inadequate control group, low quality of the EEG neurofeedback
	8	session) [13]. This point could explain the skepticism of many researchers and clinicians
	9	concerning the effectiveness of EEG neurofeedback to treat psychiatric disorders [12].
	10	However, a number of studies have presented good methodological criteria (studies designed
	11	with controlled, randomized, and open or blind protocols, a primary endpoint related to the
	12	treated disorder and assessed using standardized measurement tools, and an identifiable EEG
	13	neurophysiological target) particularly in the field of attentional-deficit/hyperactivity disorder
	14	(ADHD) [9, 12, 14].
		2
	28	comprising an optimistic group who consider
	29	neurofeedback to be effective and a skeptical group who do neither assign scientific or
	30	therapeutic value to neurofeedback training. This article aims to review the evidence of EEG
	31	neurofeedback (EEG NF) and real-time functional magnetic resonance imaging
	32	neurofeedback (fMRI NF) in psychiatric disorders. The advantages and pitfalls for each of
	33	both neurofeedback techniques are discussed, and new perspectives are highlighted. Lastly,

.1.1 Attentional-Deficit/Hyperactivity Disorder, the emblematic disorder 15

	16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 1 2 3 4 5	Four meta-analyses discussed the therapeutic interest of EEG neurofeedback in ADHD [15-18]. Computed effect size (ES) in the meta-analyses can be considered as small between 0.2 and 0.5, medium between 0.5 and 0.8 and large above 0.8. The first meta-analysis conducted by Arns et al. (2009) found an effect size (ES) that was more larger for the domain of inattention (ES=0.81, 95% CI=0.39-1.23) than for the domain of hyperactivity (ES=0.39, 95% CI=0.05-0.75) in ADHD [16]. The second meta-analysis of Sonuga-Barke et al. (2013) found a significant ES using parent ratings in randomized controlled trials (RCTs) (ES=0.59, 95% CI=0.31-0.87), but this result was no longer significant (ES=0.29, 95% CI=-0.02-0.61, though trend, p=0.07) when looking at "probably blinded" teacher ratings [17]. The third meta-analysis of Micoulaud-Franchi et al. (2014) found an ES that was significantly higher than in the control group on "probably blinded" teacher ratings for the inattention dimension of ADHD in RCTs (ES=0.30, 95% CI=0.03-0.58) [18]. The fourth meta-analysis of Cortese et al (2016) is the updated Sonuga-Barke et al. meta-analysis and reported similar results (ADHD total symptoms, ES=0.35, 95% CI=0.11-0.59; inattention, ES= 0.36, 95% CI= 0.09-0.63; hyperactivity/impulsivity, ES=0.26, 95% CI=0.08-0.43 for parent ratings, but non (ADHD total symptoms ES=0.35, 95% CI=0.04-0.69) [12]. RCTs that have compared EEG neurofeedback with medication found that methylphenidate was not superior to EEG neurofeedback training [19, 20]. In the study of Meisel et al. (2013), significant pre-post significant ES for "probably blinded" teacher ratings) [15]. However, a sub-analysis in this [12]), and for this sub-analysis a significant ES for probably blinded ratings was found academic performance improvements were obtained only in the neurofeedback group [19].

Table 1 1

 1 Principles and technical aspects of learning during neurofeedback 2 3 Aim of the learning during neurofeedback Learnability The parameters of interest can be regulated by the learner Perceptibility The parameter of interest can be perceived by the learner without exceeding his/her perception capabilities Mastery The learner gains progressively control over the sessions Motivation The learner should be preserved from boredom and not experience disengagement from the task Autonomy The learner achieves progressive independence from the feedback and can self-regulate the brain signal of interest without feedback Technical aspects related to the learning Quality of signal recording Quality of the signal-to-noise ratio / Method to avoid artefact Signal processing Signal processing method to compute the parameter of interest Occupation time Time above or below a threshold until a reward is given Threshold Automatically adapted or manually Number of positive reinforcements Number of positive reinforcements above or below a certain number until the threshold is modified Perceptual modality of feedback Type of cue used to provide feedback (e.g. visual, auditory or tactile)

	Mode	of	feedback	Continuous or intermittent
	presentation
	Complexity	of	the	e.g., for visual feedback, a thermometer display or more complex
	feedback				scenes based on virtual reality
	Number of sessions		Number of session to obtain a learning
	Duration of a session		Duration of a session and number of block per session
	Inter session duration		Duration between two sessions
	Training curve			Evaluation of the training parameter during the session
	Learning curve			Evolution of the training across the sessions
	Role of the professional Task instructions and motivation given to the subject before,
					during and after the session
	Transfer sessions		Generalization of learned skills to activities of daily living i.e. in
					an ecologically relevant setting
	4
	5
	6
	7

meta-analysis focused on standard neurofeedback protocols (based on theArns et al. criteria


Nicolas Huin

Andrea Tomassilli

Frédéric Giroire

Brigitte Jaumard

Energy-Efficient Service Function Chain Provisioning

Keywords: Network Function Virtualization, Service Function Chains, Software Defined Networks, Energy Efficiency, Optimization, Column Generation

Network Function Virtualization (NFV) is a promising network architecture concept to reduce operational costs. In legacy networks, network functions, such as firewall or TCP optimization, are performed by specific hardware. In networks enabling NFV coupled with the Software Defined Network (SDN) paradigm, network functions can be implemented dynamically on generic hardware. This is of primary interest to implement energy efficient solutions, which imply to adapt dynamically the resource usage to the demands. In this paper, we study how to use NFV coupled with SDN to improve the energy efficiency of networks. We consider a setting in which a flow has to go through a Service Function Chain, that is several network functions in a specific order. We propose a decomposition model that relies on lightpath configuration to solve the problem. We show that virtualization allows to obtain between 30% to 55 % of energy savings for networks of different sizes.

Affectation économe en énergie de chaînes de services

Résumé : La virtualisation des fonctions réseaux (NFV) est un concept d'architecture réseaux prometteur pour réduire les coûts opérationnels. Dans les réseaux traditionnels, les fonctions réseaux, comme un firewall ou un optimisateur TCP, sont effectués sur des équipements spécifiques. Dans les réseaux permettant la NFV ainsi que les réseaux logiciels (Software Defined Networks), les fonctions réseaux peuvent être implémentée dynamiquement sur des équipements génériques. Cela est très prometteur pour mettre en pratique des solutions efficaces en énergie qui implique une adaptation dynamique des ressources aux demandes. Dans ce paper, nous étudions comment utiliser les technologies NFV et SDN pour améliorer l'éfficacité énergétique des réseaux. Nous considérons un cadre dans lequel un flôt doit passer au travers d'une chaîne de fonctions de service, c'est-à-dire par une séquence de fonctions réseaux dans un ordre donné. Nous proposons un modèle de décomposition pour résoudre le problème. Nous montrons que la virtualisation permet d'obtenir entre 30 et 55% de gains énergétiques pour des réseaux de différentes tailles.

Mots-clés : Virtualisation des fonctions réseaux, Chaînes de fonctions de service, Réseaux logiciels, Efficacité énergétique, Optimisation, Génération de Colonnes

Introduction

Nowadays, network operators are trying to reduce their operational costs by smartly upgrading their networks. To this end, two recent paradigms are particularly promising: Software Defined Networks (SDNs) and Network Function Virtualization (NFV). SDN decouples the data plane from the control plane, and puts the intelligence into a centralized controller. This allows the deployment of advanced dynamic protocols and, thus, a finer network optimization based on metrology data, regularly collected by network nodes and sent to the controller.

For security and efficiency reasons, network flows have to go through a number of network functions. Examples of network functions are firewall, load balancers, Video Optimizer Controller,... Network functions are traditionnally implemented on specific harware. This leads to inefficiency as the hardware is provisionned for peak traffic. The principle of the NFV paradigm is to virtualize the functions to obtain Virtualized Network Functions (VNFs). This allows to implement these functions on generic hardware. Coupled with SDN, VNFs can be installed dynamically when needed, at the best position in the network, at the best time, and for only the adequate traffic. This allows a reduction of hardware cost (Capex) and of operational cost (Opex).

One of the important operational cost is the energy used by the nework. With the sharp increase of Internet traffic, networks have to be more energy efficient [START_REF] Matsubara | Toward future networks: a viewpoint from itu-t[END_REF]. To this end, Energy-Aware Routing solutions have been proposed, see e.g. [START_REF] Chiaraviglio | Minimizing isp network energy cost: Formulation and solutions[END_REF][START_REF] Modrzejewski | Energy Efficient Content Distribution in an ISP Network[END_REF]. The principle is to aggregate traffic on a number of network equipements as small as possible, in order to turn off the unused ones. SDN and NFV will allow to put the proposal into practice by enabling dynamic routing and network function provisionning.

In this paper, we explore the potential of network virtualization to reduce network energy consumption. We consider a setting in which a network flow has to go through a Service Function Chain (SFC), that is an ordered sequence of VNFs. We study the problem of minimizing network energy consumption, while satisfying at the same time the link and node capacity constraints and the SFCs constraints. We call this problem the Capacitated Energy Efficient Service Function Chain Provisioning Problem (EE-SFCP), Several optimization models were proposed in the litterature to solve the problem of SFCprovisioning. [START_REF] Savi | Impact of processing costs on service chain placement in network functions virtualization[END_REF][START_REF] Gupta | Joint virtual network function placement and routing of traffic in operator networks[END_REF][START_REF] Martini | Latency-aware composition of virtual functions in 5g[END_REF][START_REF] Mohammadkhan | Virtual function placement and traffic steering in flexible and dynamic software defined networks[END_REF][START_REF] Riggio | Virtual network functions orchestration in wireless networks[END_REF]. However, these works do not consider the minimization of the network energy consumption under a dynamic traffic. In [START_REF] Tomassilli | Energy-efficient service chains with network function virtualization[END_REF], we introduce a new Integer Linear Program to model the problem of Energy-Efficient SFC provisioning and a heuristic algorithm to solve large instances. However, no garantees are provided on the heuristic algorithm efficiency. In this paper, we remedy this limitation by comparing it with a decomposition model. Our contributions are the following:

• We propose a column generation model to solve the problem of EE-SFC-provisioning on large instances.

• The energy-efficiency problem is a difficult optimization problem. As a matter of fact, it contains a sharp On-Off phenomena, as a network device consumes a large portion of its energy as soon as it is used, even if very lightly used. We thus discuss several variants of the model to reduce the integrality gap.

• We use the decomposition model to carry out extensive simulations on network of different sizes. We show that between 30% to 55% of energy can be saved while respecting the constraints of the service chains.

• We compare the results obtained from the decomposition model to the one of Green-Chains. The decomposition model succeeds in finding better solutions than the heuristic algorithm. However, the results are not very far. This validates the heuristic algorithm, which finds solutions not far from optimal for an important number of cases.

2 Statement of the Problem: SFC and VNF Placement

Notations

We assume the network to be represented by a directed graph G = (V, L), where V is the set of nodes (indexed by v) and L is the set of links (indexed by ). Each node v ∈ V has a a set of compute, storage and network resources denoted by C v to host network functions. Within the cope of this study, we will assume that the resources are described by a given number of cores.

Traffic is described by a set of demand requests D, in which each demand d is defined by a 3-tuple (v s , v d , c), where v s is the source of the demand, v d its destination, and c the requested service chain. Let D c sd be the bandwidth requirement of demand d. Indeed, each demand d is associated with a given application, which is required to pass through a given service function chain (SFC), i.e., a sequence of virtual functions. Let D c sd be the bandwidth requirement of demand d.

Let F be the overall set of virtual functions arising in the service chains, indexed by f , and C be the set of service chains, indexed by c. Each service chain c corresponds to a sequence of n c functions f c 1 , . . . , f c i , . . . , f c nc , where f c i denotes the ith function of chain c. Note that some functions may appear more than once in a given chain and that for various reasons (e.g., resiliency, privacy), two different functions f and f may not be allowed to be installed in the same node. Each virtual function f has its one resource requirement and we denote by ∆ f the number (fraction) of cores required by function f per bandwidth unit.

The Energy Efficient Service Function Chain Provisioning (EE-SFCP) consists in jointly provisioning a set D of demand requests coupled with service function chains C and placing virtual functions arising in the chains, in order to minimize the network energy consumption, subject to link and node capacities.

Power Model

Campaigns of measures of power consumption (see, e.g., [START_REF] Chabarek | Power awareness in network design and routing[END_REF]) show that a network device consumes a large amount of its power as soon as it is switched on and that the energy consumption does not depend much on the load. Following this observation, on/off power models have been proposed and studied. Later, researchers and hardware constructors have proposed more energy proportional hardware [START_REF] Niccolini | Building a power-proportional software router[END_REF]. To cover the different models for different hardware, we use a hybrid power model in which the power of an active link is expressed as follows:

P = P on + bw C P max . ( 1
)
where P on represents the energy used when the link is switched on, bw the bandwidth that is carried on , and P max the energy consumed by when it is fully capacitated, i.e., when the amount of carried bandwidth equals the transport capacity (C link ) of link . We assume that links can be put into sleep mode, by putting to sleep both endpoint interfaces. Two links in opposite direction ( = (v, v ) and = (v , v)) between a pair of nodes will be assumed to be in the same state (active or in sleep mode), as the send and receive elements of a unidirectional fiber are usually controlled by the same interface. Routers cannot be put into sleep mode, as there are the sources or destinations of network traffic. However, cores may be Inria put into sleep mode and the power used by nodes is given by

P v = P unit v × #cores, (2)
where P unit v represents the energy consumption of a single core.

Layered Graph

Following a similar idea as in [START_REF] Dwaraki | Adaptive service-chain routing for virtual network functions in software-defined networks[END_REF][START_REF] Tomassilli | Energy-efficient service chains with network function virtualization[END_REF], we use a layered graph G l that is defined as follows. The initial network graph G is transformed into a layered graph G l by adding max c∈C n c layers to the graph (counting G as the base layer) and each layer is an exact copy of the original graph. For every node v ∈ V , let v i denote the corresponding node in the ith layer (i = 0, 1, . . . , n c ). Every (i -1, i) layer pair is connected vertically by links from v i-1 to v i . We denote by L(G l ) and by V (G l ) the set of links and nodes of graph G l , respectively. Provisioning of a chain and node placement of its functions amounts to find a path from node v s on the first layer (graph G) to node v d on the n c th layer. Placement of a function on a node is given by the endpoints of the link used to switch between layers.

We next set a column generation model for EE-SFCP, which makes use of the layered graph G l .

Decomposition Models

We first present here a model using Column Generation, CG-simple. We then introduce two variants of the models, CG-cuts, and CG-cut+. Indeed, problems dealing with energy-efficiency frequently lead to large integrality gap and bad precision. This is due to the On-Off phenomena of power models, which translates into large steps of the objective function. We thus try to improve the precision of the model by introducing different sets of constraints. We discuss the precision of the models in Section 5.2.

Column Generation Formulation

We propose a column generation formulation that relies on the concept of chaining & function placement configurations: each configuration γ is associated with a 3-uplet (v s , v d , c) and defines: (i) a potential route for demand D c sd associated with node pair (v s , v d ) for chain c and, (ii) node placement of the functions of chain c along the potential route. Route is described by parameters δ γ , equal to 1 if link belongs to the path, 0 otherwise. Node placement is given by a γ vfi , equal to 1 if the ith function f i of c is located at node v, 0 otherwise. We denote by Γ the overall set of configurations.

We now define the set of variables.

• x ∈ {0, 1}. x = 1 if link is on (active), 0 otherwise. Note that links are powered off by pair, i.e.,

x =(v,v ) = x =(v ,v) • y γ d ∈ {0, 1}. y γ d = 1 if demand d is routed using configuration γ, 0 otherwise. • req v ∈ N. req v = # required cores in node v.
The objective, i.e., the minimization of the energy, can be written

The constraint set decomposes into several sets of contraints, which are next described. One path per demand

γ∈Γ d y γ d = 1 d = (v s , v d , c) ∈ D. (4)
Link capacity

d=(vs,v d ,c)∈D γ∈Γ d D c sd δ γ y γ d ≤ x C link ∈ L. (5)
Node capacity

d=(vs,v d ,c)∈D γ∈Γ d D c sd nc i=1 ∆ fi a γ vfi y γ d ≤ req v ≤ C node v v ∈ V nfv . ( 6
)
The CG-cuts model. To reduce the integrality gap, we add cuts, which are next described. Inequalities [START_REF] Mohammadkhan | Virtual function placement and traffic steering in flexible and dynamic software defined networks[END_REF] states that, for each node, at least one incident link should always be on. Moreover, at least n -1 links should be on to have a connected network (or different if not all-to-all). Inequality (8) enforce that last condition.

∈ω + (v) x ≥ 1 v ∈ V (7) ∈L x ≥ n -1. (8)
The CG-cut+ model. For the second variant, we add the following set of constraints:

x ≥ γ∈Γ d δ γ y γ d ∈ L, γ ∈ Γ d (9)
Following Equation (4), we have that

γ∈Γ d δ γ y γ d ≤ 1.
This allows us to circumvent the use of a big M formulation at the expense of a large number of constraints.

Solution Scheme

There is a configuration generator, i.e., pricing problem, for each d = (v v , v d , c) ∈ D. Two sets of decision variables are required. First set is made of variables ϕ i such that ϕ i = 1 if the provisioning of demand d uses link in layer i of the layered graph G l , 0 otherwise. Second set contains variables a i v such that

a i v = 1 if the ith function (f i ) of chain c for demand d = (v v , v d , c
) is placed on NFV node v, 0 otherwise.

min ∈L P max D c sd C link nc i=0 ϕ i -u (4) sd + ∈L u (5) D c sd nc i=0 ϕ i + D c sd v∈V u (6) v nc i=0 ∆ fi a i v . ( 10
)

Inria

Path computation (flow conservation constraints):

∈ω + (v) ϕ i - ∈ω -(v) ϕ i + a i v -a i-1 v = 0 v ∈ V, 0 < i < n c ( 11
) ∈ω + (v) ϕ 0 - ∈ω -(v) ϕ 0 + a 0 v = 1 if v = v s 0 else v ∈ V ( 12
) ∈ω + (v) ϕ n c - ∈ω -(v) ϕ n c -a n c v = -1 if v = v d 0 else v ∈ V. (13)
Link capacity

D c sd nc i=0 ϕ i ≤ C link ∈ L. (14)
Node capacity

D c sd nc i=0 ∆ fi a i v ≤ C node v v ∈ V nfv . ( 15
)
4 GreenChains Heuristic

In [START_REF] Tomassilli | Energy-efficient service chains with network function virtualization[END_REF], we proposed a heuristic algorithm called GreenChains to solve EE-SFCP. We briefly recall it below. Heuristic GreenChains solves EE-SFCP in three steps.

-First, the routing problem is to compute a path for each demand, respecting the link capacity constraints.

-Second, the goal of the service chain placement problem is to find a placement of the NVF respecting the capacities of the nodes and the order defined by the service chains.

-Last, the energy saving problem tries to put into sleep mode as many links and cores as possible to decrease the energy consumption of the network.

Routing

Step. Demands are considered one at a time. For each demand, we compute a weighted shortest path on a residual graph. The weight of a link is equal to the inverse of its spare capacity. This favours the links with lower loads. The idea is to spread as much as possible the demands to find a feasible routing. To compute the residual graph, when the path is computed, we reduce the link capacities by the value of the demand. Furthermore, when considering a new demand to be routed, we remove links with a residual capacity smaller than the demand. Service Chain Placement Step. When the paths for all demands are set, we place the service chains. We consider the different service chains separately. For a given chain, we consider the paths of all the demands with this chain. We then sort the nodes of the networks in the decreasing order of the number of such paths they are involved in. We then try to place all the functions of the chains on the nodes following this order. If it is possible in a node u, we place the functions.

We then assign some of the paths to the cores of node u, till its capacity is full. We then actualize the ordered list of nodes considering only the paths not yet assigned.

Energy Saving

Step. We attempt to greedily put links into sleep mode. It starts with the graph G = (V, L). It first launches the routing module and then the service chain placement module. If they both succeed, it creates a list U of all links according to their usage (volume of traffic). It then chooses the less loaded link min as a candidate to be put in sleep mode. It now The second element of U is then considered. The algorithm goes on till all links have been tried and set either as definitely in sleep mode or definitely active.

Numerical Experiments

In this section, we investigate the energy savings obtained by the Column Generation model. We compare the results with ones of the GreenChains heuristic algorithm, proposed in [START_REF] Tomassilli | Energy-efficient service chains with network function virtualization[END_REF]. We first present the data sets we use for the experiments. We then take a look at the precision of the solutions obtained by the Column Generation model and GreenChains. We investigate different improvements of the model presented in Section 3. We then present the energy savings obtained for network topologies of different sizes. Last, we discuss the impact of the solutions on link usage and path lengths.

Data sets

In networks, each type of flows has to go through a different chain of network services. In our experiments, we consider the four of the most frequent types of flows as presented in Table 1: Video Streaming, Web Service, Voice-over-IP (VoIP), and Online Gaming. The traffic percentage are from [START_REF] Index | Cisco visual networking index: Forecast and methodology 2015-2020[END_REF]. For each one, we give the ordered set of functions required and the bandwidth used. In total, 6 different functions are used, and each function requires a different amount of cores to be executed. We tested the CG model and GreenChains on three topologies of different sizes from SNDlib [START_REF] Orlowski | SNDlib 1.0-Survivable Network Design Library[END_REF]: pdh (11 nodes and 64 directed links), alanta (15 nodes and 44 directed links), and germany50 (50 nodes and 176 directed links).

For each network, we generate a set of demands from the traffic matrices provided in SNDLib: we divide each aggregate flow from a source to a destination into four demands corresponding to the four different types of traffic. The original load of the flow is conserved and each subflow load is given by the distribution of the last column of Table 1. For example, a flow with a charge of 1 is splitted into a Web Service, a VoIP, a Video Streaming and an Online Gaming sub-flows with a respective charge of 0.182, 0.118, 0.699 and 0.001.

We tested the solution on a daily traffic to see how much energy can be saved during the day or at night. The variations of traffic come from a trace of a typical France Telecom link shown in Figure 1. Previous work [START_REF] Araujo | Energy efficient content distribution[END_REF] shows that most of the energy savings can be obtained by using a small number of configurations during the day. In our case, we considered 5 different levels of traffic called D1 to D5. D1 represents the period with the lowest amount of traffic and D5 the one with the highest.

Quality of the Column Generation models

In Section 3, we propose three different models using Column Generation, CG-simple, CG-cuts, and CG-cut+. We now study the impact of the different sets of constraints on the final objective value of the Column Generation and on the ratio between the Integer Reduced Master Problem (RMP) and its Fractional counterpart, ε = (z ilp -z lp )/z lp . In Figure 2, we compare the solutions founded by the three CG models for the pdh network and for the 5 different levels of traffic. Energy used is given in Figure 2a (Error bars represents the energy used found by the RMP) and ε ratio in Figure 2b. The first observation is that the three models obtain similar energy savings: almost always equal for CG-simple and CG-cuts, and a little bit better (put percent) for CG-cut+. However, the ε ratio is dramatically different. Adding the cuts improves greatly the ratio: ε for CG-simple varies between 169% for the D1 period and 64% for the D5 period, when, for CG-cuts, ε is between 8 to 30%. The ratio is further improved with CG-cut+: between 2 and 7%. As the energy savings are similar for the three models, this shows that the three CG models provide solutions that are few percent from optimal, but that this is easier for Cplex to prove it for the CG-cuts and CG-cut+ models.

For the following results, we focus on the CG-cuts model, as its ratio is a lot better than the one of CG-simple, and, as CG-cut+ requires too much memory to solve large networks. We are not able to solve the problem for the pdh network for the D3 and D4 period.

We provide in Figure 3 the values of the ε ratio for the three studied topologies from SNDLib. On pdh, the ratio increases from 7% at the D1 period to 17% at the D5 period. For atlanta, the ratio decreases from 19% at D1 to 8% at D3, before increasing back to 27% at D5. For germany50, the ratio steadily increases from 20 to 26%. Energy Savings. We compare the savings obtained by our solutions with the legacy scenario, which serves as a baseline, and a hardware scenario. The legacy scenario considers a legacy network which does not implement SDN nor NFV. The routing is thus static and the network functions are not virtualized. They are executed by specific hardware, which are installed at given positions in the network. The demands use the shortest paths between the sources, hardwares implementing the network functions of their services chains, and the destination. The hardware scenario corresponds to the one of an SDN (non virtualized) networks in which an operator tries to reduce its energy consumption by adapting the routing to the demands. In this scenario, the network functions are carried out by some specific hardware placed at given positions in the network. We provide in Figure 4 the energy used for the 5 levels of demands D1, D2, D3, D4 and D5 for the SNDLib networks. The values are normalized: 100 corresponds to the legacy scenario. We also present in Figure 5 the corresponding energy savings during the day for the germany50 network. We see that we obtained important savings using virtualization: between 25 and 62 % for pdh, 15 and 30 % for atlanta, and 38 and 42 % for germany50.

Heuristic vs the Column Generation model. We compare in Figure 5 the Column Gen- Again, when some links are put into sleep mode, some of the paths between sources and destinations are becoming longer. We investigate the impact of these changes on the delays in the network. To compute the delays, we assume that they are proportional to the distances traveled along the paths [START_REF] Choi | Analysis of point-to-point packet delay in an operational network[END_REF]. SNDLib provides the geographical coordinates of the nodes of germany50. We thus derived the link lengths based on these coordinates. For the other two networks, no data is available. We thus use the average delay of 1.8 ms founded in germany as the link length.

In Figure 7, we look at the delay of the request for each period of time. The median remains the same on pdh and atlanta all day long. On germany50, it slightly decreases for higher traffic periods, as more links are active. At most, requests experience delay of about 7, 14 and 25 on pdh, atlanta and germany50 respectively. An important observation is that the maximum delay Inria of every path stays below 50 ms, which is a classic value for the maximum delay in Service Level Agreements [START_REF] Giroire | Increasing the robustness of ip backbones in the absence of optical level protection[END_REF].

Conclusion

In this work, we investigate the potential of network virtualization to reduce the energy consumption of networks. We propose a Column Generation model to solve the problem of minimizing network energy consumption while satisfying the constraints of service chains of virtualized network functions. We obtain between % of energy savings for network topologies of different sizes.

Figure 1 :

 1 Figure 1: Normalized daily variation of traffic of a France Telecom network link and multi-period approximation.

Figure 2 :Figure 3 :

 23 Figure 2: Comparison of the performance of different Column Generation Models for pdh.

Figure 4 :

 4 Figure 4: Comparison of the energy used and saved between GreenChains and the Column Generation model for pdh network.

Figure 5 :

 5 Figure 5: Energy savings during the day.

Figure 7 :

 7 Figure 7: Distribution of delay for the 5 traffic periods on the pdh, atlanta and germany50. The box represents the 1st and 3rd quartile. The median is represented by the red line and the whiskers represents the 1st and 9th decile.

RR n°8980

route des Lucioles -BP 93 06902 Sophia Antipolis Cedex Publisher Inria Domaine de Voluceau -Rocquencourt BP 105 -78153 Le Chesnay Cedex inria.fr ISSN 0249-6399

* N. Huin, A. Tomassilli, and F. Giroire have been partially supported by ANR program ANR-11-LABX-0031-01 and Mitacs. † B. Jaumard was supported by a Concordia University Research Chair (Tier I) and by an NSERC (Natural Sciences and Engineering Research Council of Canada) grant.

Column Generation

eration model and the heuristic. Error bars on the CG solutions represent the lower bounds, that is the relaxation of the RMP (z lp ). For all three topologies and for all time periods, the difference between the heuristic and the Column Generation model is small. For the lowest traffic periods (D1 and D2), both methods provide similar solutions. The CG model provides slightly better solutions when the traffic is higher, with a difference of 4, 7 and 5% for pdh, atlanta and germany50 respectively in the D5 period. The important thing to note here is that, even though the Column Generation does not provide a much better solution than the heuristic, it does show (by providing the ε ratio) that the heuristic gives good results, regardless of the traffic period.

Load of links

To reduce the amount of energy used by the network, we reroute some of the flows to be able to put links into sleep. This means that the remaining active links are more loaded. In Figure 6, we look at the link load for the highest and lowest traffic periods. First, we see that for atlanta and germany50, with no surprise, the number of links with no traffic is higher when the traffic is low, with around 40% of the links that forward no traffic at all. When the network is at its highest utilization it drops to 20% or less. The pdh network, due to its higher link density, can have more links put into sleep mode. Indeed, between 45% and 70% of the links have no traffic. Moreover, at the lowest traffic period no links are used at 100%. They are at most used up to 50%, 22% and 63% of their capacity, for pdh, atlanta and germany50 respectively. At rush hour, germany50 and pdh have about 5 and 10% at full capacity, while atlanta links are at most used at 92%.