Mohammed Mokhtari
email: mohammed.mokhtari@outlook.com

Smail Zouggar
email: szouggar@gmail.com

Nacer M ' Sirdi
email: nacer.msirdi@univ-amu.fr

M L Elhafyani
email: elhafyani77@gmail.com

Behavior Analysis Of Asynchronous Wind Turbine In Presence Of Static Synchronous Compensator Operating With Fuzzy Logic Voltage Controller

Keywords: Asynchronous Wind Turbine, Reactive Energy Compensation, STATCOM, Voltage Regulation, Fuzzy Logic Controller Synchronous angular velocity

The aim of this paper is to discuss and demonstrate how a Static Synchronous Compensator (STATCOM) is used with a Single Input Fuzzy Logic Controller (SIFLC) to improve voltage profile and stability of an asynchronous wind turbine. In the literature, it is well known that reactive power management is the greatest challenge in wind turbine based on a three-phase Self-Excited Induction Generator (SEIG). Any variation of wind speed or load causes a variation on the needed reactive power and thus a voltage fluctuation. Flexible AC transmission (FACT) device such as STATCOM become then a necessity to prevent voltage instability and hence voltage collapse at the point of common coupling (PCC). The performance of the proposed compensator can be significantly improved when combined with a SIFLC. The present document traits so the modeling of the power system, the simulation results, the control scheme and the design of the proper SIFLC.

INTRODUCTION

As wind energy is abundant and endowed with inexhaustible potential, it is today one of the best sources of sustainable electricity supply for global development [START_REF] Elhafyani | Permanent and dynamic behaviours of self-excited induction generator in balanced mode[END_REF]. In the last decades, the use of renewable energy sources has increased considerably in rural areas mainly those in hilly and remote regions. Often, the electric power demanded by these communities is in the range of 1 to 100 kW which makes an isolated micro-grids appears as the best solution to choose, eliminating so the high-cost investment required for transmission line by conventional system.

Compared with other generators, Self-excited induction generator (SEIG) applied in variable speed wind turbine system for an isolated micro-grid presents more advantages. Its robustness and low cost qualities makes it perfectly suited for use in some extreme wind and load conditions [START_REF] Li | High power density STATCOM with extended reactive power control range[END_REF]. However, the terminal voltage stability of the induction generator (IG) cannot be guaranteed just with a shunt connection of fixed capacitor bank [START_REF] Anu | STATCOM based controller for a three phase system feeding single phase loads[END_REF]. Thus, several voltage regulating schemes have been proposed by researchers using Flexible AC transmission (FACT) devices such as Static Var Compensator (SVC) or more recently Static Synchronous Compensator (STATCOM) [START_REF] Singh | STATCOM-based controller for a three-phase SEIG feeding single-phase loads[END_REF].

In this work, the topology using STATCOM have been chosen since it offers enhanced performance in terms of dynamic response and regulation compared with other available schemes specially when the produced voltage is low [START_REF] Sethy | Design, Analysis and Simulation of Linear Model of a STATCOM for Reactive Power Compensation with Variation of DC-link Voltage[END_REF]. The fundamental principle of STATCOM is the generation of a controllable ac voltage source by a voltage source inverter (VSI) based on three phase sinusoidal pulse width modulation (SPWM) power converter technology such as IGBTs [START_REF] Jain | Voltage regulation with STATCOMs: Modeling, control and results[END_REF]. The ac side of the VSI is connected to the asynchronous wind turbine through a coupling reactance and its dc side to a capacitor. The active and reactive power transfer between the two systems is caused by the voltage difference across the reactance and can be controlled by the modulation index MI and the relative phase angle , Fig. 1.

Basically, there are two control objectives implemented in STATCOM. One is the PCC voltage regulation and the other is dc voltage regulation across its capacitor. In conventional scheme, a decoupled dq axis control approach based on four proportional integral (PI) type cascaded controllers is used [START_REF] Baran | STATCOM with energy storage for smoothing intermittent wind farm power[END_REF]. Aside from experimental procedures, to the authors' knowledge, there is no standard procedure for designing the ac voltage regulator that ensures the required stability and robustness to system variations. In general, the chosen set of PI gains remain fixed in daily operation of STATCOM. Since wind speed and load changes with time those parameters may not be suitable for all operating points [START_REF] Perumal | Analysis of a self excited induction generator with STATCOM/battery energy storage system[END_REF]. Therefore, to reduce voltage fluctuation at all possible conditions we propose a new structure for ac voltage regulator based on Single-Input Fuzzy Logic Mamdani-type that depends only on the error between the reference value and the measured RMS value of the asynchronous wind turbine produced voltage.

The organization of this paper is as follows. It starts with the modeling of the studied asynchronous wind turbine and STATCOM, then focuses on the description of the conventional control scheme and the design of the proper current and voltage controllers, to finally discuss and analysis the simulated results.

POWER SYSTEM MODELING Asynchronous Wind Turbine Model

Several models of wind turbines have been developed and can be found throughout the bibliography [START_REF] Elhafyani | Permanent and dynamic behaviours of self-excited induction generator in balanced mode[END_REF]. Since the electrical behavior of the system is our main point of interest the following model is assumed: The electromagnetic torque produced by the asynchronous generator is defined as:

𝐶 𝑒𝑚 = 𝑝. 𝐿 𝑚 𝐿 𝑟 (𝜑 𝑟𝑑 . 𝑖 𝑠 -𝜑 𝑟𝑞 𝑖 𝑠 ) (1)

Static Synchronous Compensator Model

The modeling of STATCOM is reviewed in the lines below and described in Park reference frame [START_REF] Perumal | Analysis of a self excited induction generator with STATCOM/battery energy storage system[END_REF]. We define the reference frame coordinate where the d_axis is always coincident with the instantaneous system voltage vector and the q_axis is in quadrature with it. The equations describing the AC side of STATCOM are given by:

𝑑 𝑑𝑡 [ 𝑖 𝑜𝑡𝑑 𝑖 𝑜𝑡𝑞 ] = [ - 𝑅 𝑐 𝐿 𝑐 𝜔 𝑠 -𝜔 𝑠 - 𝑅 𝑐 𝐿 𝑐 ] . [ 𝑖 𝑜𝑡𝑑 𝑖 𝑜𝑡𝑞 ] + 1 𝐿 𝑐 [ 𝑉 𝑑𝑠 -𝑉 𝑜𝑡𝑑 𝑉 𝑞𝑠 -𝑉 𝑜𝑡𝑞 ] (2)
Where 𝜔 𝑠 the pulsation of the generated voltage 𝑉 𝑠 .

The voltage and current are related in the DC side of STATCOM by the following equation:

𝑑𝑉 𝑑𝑐 𝑑𝑡 = 𝑖 𝑑𝑐 𝐶 𝑑𝑐 (3)
The switching function S of the VSI including the modulation index MI and the phase angle 𝛼 can be defined as:

𝑆 = [ 𝑆 𝑢 𝑆 𝑣 𝑆 𝑤 ] = 𝑀𝐼. [ 𝑠𝑖𝑛 (𝑤𝑡 + 𝛼) 𝑠𝑖𝑛 (𝑤𝑡 - 2𝜋 3 + 𝛼) 𝑠𝑖𝑛 (𝑤𝑡 + 2𝜋 3 + 𝛼) ] (4)
The output voltages and currents of STATCOM are expressed as follows:

[ 𝑉 𝑜𝑡𝑑 𝑉 𝑜𝑡𝑞 ] = 𝑀𝐼. 𝑉 𝑑𝑐 . [ 𝑐𝑜𝑠 (𝛼) 𝑠𝑖𝑛 (𝛼) ] (5)
𝑖 𝑑𝑐 = 𝑀𝐼. [𝑖 𝑜𝑡𝑑 𝑖 𝑜𝑡𝑞 ]. [ 𝑐𝑜𝑠 (𝛼) 𝑠𝑖𝑛 (𝛼) ] (6)

VOLTAGE CONTROL ARCHITECTURE Description

The main objective of the adopted command law is to regulate the AC voltage at PCC and the DC voltage of the VSI by performing a decoupled active and reactive power circulation between STATCOM and induction generator. This can be achieved by feeding or draining a sinusoidal current with a displacement of 90 degrees in relation to PCC voltage. STATCOM control parameters modulation index MI and phase angle 𝛼 are deduced from the inverter voltage references in Park reference frame. Thus, a PLL is used to synchronize the voltage output of STATCOM with the asynchronous generator voltage. The complete description of the control scheme is presented in Fig. 4.

Current Controller Tuning

The references of current loops 𝑖𝑜𝑡𝑑 * and 𝑖𝑜𝑡𝑞 * are deduced from the active and reactive references of the converter 𝑄 * and 𝑃𝑑𝑐 * as shown in Fig. 4. The current loop implemented uses a PI controller. Its frequency response is plotted in Fig. 5. The phase margin of the system is about 90º and the setting time is 0.0109 seconds which allows a stable and fast response of the current controller as references change to meet the need for active and reactive power.

DC Voltage Regulator Tuning

By sensing the voltage across the capacitor Cdc and comparing it to the reference value Vdc*, the DC voltage error can be eliminated using also a PI controller as shown in Fig. 6. The inner d_axe current loop have a setting time faster than the DC voltage loop implemented.

AC Voltage Regulator Design

The Single-Input Fuzzy Logic Controller proposed in this work is constructed by choosing the error between the reference and the measured RMS value of the generated voltage as an input signal and dQ* as output signal, Fig. 7. Hence, the reference reactive power to exchange with the IG is calculated using the following expression:

𝑄 * = ∑ 𝑑𝑄 * (𝑡) (7)
o If Vs_RMS is equal to the reference voltage value then error is equal to zero and dQ* must also be equal to zero to keep Q* at its previous value to avoid injection or absorption of any reactive power. o If Vs_RMS is greater than the reference voltage value (inductive mode) the error will be negative and dQ* must be positive as well as Q* and its value should increase to absorb the excess of reactive power forcing then the generated voltage to drop to the rated value. o If Vs_RMS is less than the reference voltage value (capacitive mode) the error will be positive and dQ* must be negative as well as Q* and its value should decrease to inject the needed reactive power forcing then the generated voltage to rise up to the rated value.

We define a simple structure for the fuzzy controller Mamdani-type with seven triangular rule-based-membership functions. The seven linguistic variables used are Negative Big (NB), Negative Medium (NM), Negative Small (NS), Zero (ZO), Positive Small (PS), Positive Medium (PM) and Positive Big (PB). The 7 rules have been built as represented in Fig. 7.

SIMULATION RESULTS & DISCUSSION

At no compensation mode (STATCOM disconnected), during the starting phase from the instant 0s to 2s, Fig. 8, the SEIG is not yet excited so the voltage generated (phase to ground) is equal to 0V. After the second two, the output voltage grows exponentially until the steady-state RMS value of 254.6V. At the 5 th s it drops to 224.5V when wind speed changes from 10.5m/s to 8.5m/s at no load, and drops again at the 9 th s to 204V when connected to a resistive load of 100Ohm. Then, at the 12 th s, the generated voltage starts collapsing when wind speed changes from 8.5m/s to 7.5m/s at full load. The observed change is caused by the reduction of the reactive power required by the asynchronous generator to keep the produced voltage at its rated value (Vs_ref). Once the STATCOM connected, Fig. 8. The new simulation result shows that the desired steady state RMS value of the generated voltage is reached despite variation in wind speed or load. At t=2s the peak voltage reach 254.6V and decreases in 1.03 seconds creating so an overshoot of 10.7%. At t=5s Vs_RMS drops to 192.2V to create a deep of 16.52% which only lasts 0.3 seconds. At t=12s the PCC voltage drops again to 221.3V during 0.2s and rises up to stabilize at the reference value preventing the asynchronous wind turbine from collapsing. The SIFLC can so increases the performance of to improve the asynchronous wind turbine voltage profile and stability when operating in both capacitive and inductive mode, Fig. 9. On the other hand, simulation results observed in Fig. 10 shows that Vdc rises up at t=2s just after the excitation of the SEIG and settles at the reference value proving so the efficiency of the designed DC voltage controller. The active power provided by the asynchronous wind turbine to STATCOM reach 23.06W to supply enough power to regulated Vdc then falls down to zero after 1.2 seconds since all losses are neglected.

CONCLUSION

In this paper, the model of an asynchronous wind turbine operating with STATCOM has been presented. The performance of the studied system under close loop condition at different wind speed and load circumstances using a SIFLC for PCC voltage regulation has been also investigated. Behavior analysis of the power system have shown the efficiency of the proposed command law which will be validated using real-time control board dspace 1104.

Figure 1 .

 1 Figure 1. Schematic diagram of asynchronous wind turbine with STATCOM.

Figure 2 .

 2 Figure 2. Schematic diagram of the turbine model.The induction generator model developed in this work is represented in  reference frame, Fig.3. It should be noted that Lm is not a constant but a function of the magnetizing current im. This dependency have been experimentally determined and incorporated into Matlab/Simulink environment.

Figure 3 .

 3 Figure 3. Equivalent circuit of the SEIG in  reference frame.

Figure 4 .

 4 Figure 4. Control Scheme.

Figure 5 .

 5 Figure 5. Current loops frequency response with controller parameters and performance.

Figure 6 .

 6 Figure 6. DC voltage loop frequency response with controller parameters and performance.

Figure 7 .

 7 Figure 7. Inputs and outputs membership functions of the SFLC (left) & Controller diagram (right)

Figure 8 .

 8 Figure 8. Asynchronous wind turbine voltage with and without compensation at different wind speed and load state.

Figure 9 .

 9 Figure 9. STATCOM reactive power variation as function of PCC voltage.

Figure 10 .

 10 Figure 10. STATCOM active power (left) & Vdc response (right).


Herilalaina Rakotoarison

Michèle Sebag
email: sebag@lri.fr

International Workshop on AutoML AutoML with Monte Carlo Tree Search

Keywords: Model selection, Hyper-parameter optimization, Monte Carlo Tree Search, AutoML

. In this paper, a new approach, called Monte Carlo Tree Search for Algorithm Configuration (Mosaic), is presented, fully exploiting the tree structure of the algorithm portfolio and hyper-parameter search space. Experiments (on 133 datasets of the OpenML repository) show that Mosaic performances match that of Auto-Sklearn.

Introduction

The progress of the machine learning (ML) field is witnessed as an explosion of applications is seen in all fields, from computer vision (Krizhevsky et al., 2012;[START_REF] Karpathy | Large-scale video classification with convolutional neural networks[END_REF][START_REF] Shaoqing Ren | Faster r-cnn: Towards realtime object detection with region proposal networks[END_REF][START_REF] He | Deep residual learning for image recognition[END_REF] to recommendation systems [START_REF] Ricci | Introduction to recommender systems handbook[END_REF][START_REF] Michael D Ekstrand | Collaborative filtering recommender systems[END_REF]. However, the diversity of ML algorithms and their sensitivity w.r.t. their hyper-parameters make it a difficult task to find the approach best suited to the application at hand. This difficulty makes all the more serious the announced shortage of machine learning experts in the next decade. The problem of automatically finding the best setting for an ML problem, referred to as AutoML, has attracted interest since the late 1980s (see [START_REF] Brazdil | Metalearning and algorithm selection: progress, state of the art and introduction to the 2018 special issue[END_REF] for a survey), with several AutoML international challenges organized in the last decade [START_REF] Guyon | Design of the 2015 chalearn automl challenge[END_REF][START_REF] Guyon | Automatic machine learning challenge 2018: Towards ai for everyone[END_REF]. These challenges have primed the design and deployment of numerous automated machine learning platforms (AutoMLP in the following) (Feurer et al., 2015;[START_REF] Li | Hyperband: Bandit-based configuration evaluation for hyperparameter optimization[END_REF][START_REF] Chen | Autostacker: A compositional evolutionary learning system[END_REF][START_REF] Olson | Evaluation of a tree-based pipeline optimization tool for automating data science[END_REF].

The contribution of the paper is to tackle AutoML as a one-player game, adapting Monte-Carlo Tree Search (MCTS) [START_REF] Kocsis | Bandit based monte-carlo planning[END_REF] to the exploration of the complex and structured decision space Λ encompassing all ML options (pre-processing, feature selection, model selection and optimization hyper-parameters). Formally, the proposed approach called Monte-Carlo Tree Search for AlgorIthm Configuration (Mosaic), tackles the following optimization problem:

Find λ * ∈ argmin λ∈Λ L(λ, D train , D valid ), ( 1
)
where Λ is the set of all hyper-parameter settings, loss function L measures the learning performance on the validation set D valid of the model learned on the training set D train .

A main difficulty of the above optimization problem lies in the fact that the optimization objective (Eq. 1) usually is a non-convex and expensive one, all the more so when large datasets and/or complex model spaces are considered.

This paper is organized as follows. Section 2 describes and discusses the state-of-theart of AutoML. For the sake of self containedness, the formal background in Monte Carlo tree search is given in section 3. A detailed overview of Mosaic is given in section 4. The experimental validation of the approach is given in section 5 and the paper concludes with some perspectives for further research in section 6.

Related Work

The domain of automated machine learning involves several subproblems: i) selection of the ML algorithm most appropriate to the current dataset; ii) optimization of the associated hyper-parameters; iii) optionally, data preprocessing.

The field of hyperparameter optimization is currently dominated by Bayesian optimization methods [START_REF] Snoek | Scalable bayesian optimization using deep neural networks[END_REF][START_REF] Hutter | Sequential model-based optimization for general algorithm configuration[END_REF][START_REF] James S Bergstra | Algorithms for hyperparameter optimization[END_REF]. These methods proceed by estimating the conditional probability p(f |λ) of the performance f given a hyperparameter setting λ, and exploit this probabilistic model to select the next most promising candidate λ * ; after evaluating the performance associated to this candidate, the model is updated and the search is iterated. Other approaches handle the expensive optimization problem by considering cheap approximations thereof, e.g. sub-sampling the data or using a surrogate objective (estimating the performance) [START_REF] Swersky | Freeze-thaw bayesian optimization[END_REF][START_REF] Li | Hyperband: Bandit-based configuration evaluation for hyperparameter optimization[END_REF][START_REF] Klein | Fast bayesian optimization of machine learning hyperparameters on large datasets[END_REF]. Yet another approach is the Bandit-based approach Hyperband [START_REF] Li | Hyperband: Bandit-based configuration evaluation for hyperparameter optimization[END_REF], tackling hyperparameter optimization as a resource allocation task.

At the current state-of-the-art in AutoMLPs are Auto-Sklearn (Feurer et al., 2015) and Auto-Weka [START_REF] Thornton | Auto-weka: Combined selection and hyperparameter optimization of classification algorithms[END_REF]. They rely on the Bayesian-based approach SMAC. Auto-Sklearn involves two extra components: meta-learning components to warm-start the Bayesian optimization procedure, and model ensemble strategy described in [START_REF] Caruana | Ensemble selection from libraries of models[END_REF] to build a more robust classifier. Auto-Sklearn involves 15 classifiers, 14 feature preprocessing methods, and 4 data preprocessing methods. Other AutoMLPs based on evolutionary algorithms have recently emerged: TPOT [START_REF] Olson | Evaluation of a tree-based pipeline optimization tool for automating data science[END_REF] and AutoStacker [START_REF] Chen | Autostacker: A compositional evolutionary learning system[END_REF], with the caveat that they might need many evaluations to yield a good result, thus with a high computational cost.

Interestingly, MCTS has been applied to feature selection [START_REF] Gaudel | Feature selection as a one-player game[END_REF]: the so-called Fuse (Feature UCT Selection, details of UCT in section 3) extends MCTS to tackle the combinatorial optimization of selecting a best subset of features, cast as a oneplayer game. Wistuba (2017) also proposed to tune deep neural network with MCTS but only on limited list of parameters. In this work, we would like to extend those approaches to the full autoML problem.

Monte-Carlo Tree Search

Monte-Carlo Tree Search (MCTS) extends the celebrated Multi-armed Bandit algorithm [START_REF] Auer | Using confidence bounds for exploitation-exploration trade-offs[END_REF] to tree-structured search spaces.

Multi-armed Bandit Let k arms or options be defined, the i-th arm being associated with an (unknown) probabilistic bounded reward, e.g. a Bernoulli variable of probability µ i . Based on the arms selected in the past and the associated reward, the goal is to find an arm selection policy such that it maximizes the expected cumulative reward, or equivalently, minimizes the regret (difference with the oracle strategy, always selecting the arm with highest µ i ). The UCB1 algorithm [START_REF] Auer | Using confidence bounds for exploitation-exploration trade-offs[END_REF] defines such an arm selection policy, selecting in each time step the arm maximizing the following:

Select arg max i {µ i + 2 log n n i , } (2)
where µ i is the sampled average obtained by playing arm i, n i is the number of times arm i has been chosen, n is the number of iterations so far.

The MCTS algorithm iterates over four phases [START_REF] Chaslot | Monte-carlo tree search: A new framework for game ai[END_REF]: selection, expansion, playout and backpropagation:

• Selection: In each node of the tree, the child node is selected after a Multi-armed Bandit strategy, e.g. the UCT (Upper Confidence bound applied to Trees) algorithm [START_REF] Kocsis | Bandit based monte-carlo planning[END_REF] selects the child node such that it maximizes:

µ i + c log N n i , (3)
where µ i is the value of node i, N is the number of times the parent node was visited, n i is the number of times node i is visited, and c is a problem-dependent constant which balances exploration and exploitation.

• Expansion: The algorithm adds one or more nodes to the tree. This node corresponds to the first encountered position that was not added in the tree.

• Playout: When reaching the limits of the visited tree, a roll-out strategy is used to select the options until reaching a terminal node and computing the associated reward.

• Backpropagation: The reward value is propagated back, i.e. it is used to update the value associated to all nodes along the visited path up to the root node.

Remarks. A Multi-armed Bandit problem defines a reinforcement learning (RL) problem with a single state. MCTS relaxes this single-state limitation and is known as an efficient RL approach, at the core of the RL breakthroughs related to the Atari games [START_REF] Mnih | Playing atari with deep reinforcement learning[END_REF] and the game of Go [START_REF] Silver | Mastering the game of go with deep neural networks and tree search[END_REF].

Monte-Carlo Tree Search for Algorithm Configuration

After formalizing the AutoML process as a Markov Decision Process problem, this section gives an overview of the proposed MOSAIC approach and discusses its components.

AutoML as a Markov Decision Process

In the following, the dataset D at hand is assumed to be fixed. AutoML thus aims to find a 4-tuple m ∈ M where m consists of (1) a pre-processing method b, (2) the hyper-parameter setting of b, noted p b (p b in IR q , and q depends on b), (3) a learning algorithm a and (4) the hyper-parameter setting p a of algorithm a (where p a is in IR d , and d depends on a). The goal is to find the best tuple m * in the sense of the performance (e.g. predictive accuracy) of the learned model on distribution D underlying the finite sample D. AutoML problem defines an ill-posed and expensive structured optimization problem, which is tackled as a sequential decision problem for tractability.

In MOSAIC, the order of the choice follows the domain knowledge: the first choice is that of the pre-processing method b; the choice of pre-processing hyper-parameters p b depends on b, the choice of the learning algorithm a and the choice of a hyper-parameters p a itself depends on a.

Finally, Mosaic follows the standard MCTS procedure, except for two specific issues: handling of continuous hyper-parameter values (section 4.2) and overcoming of the combinatorial nature of the optimization (section 4.3). The MOSAIC strategy is described as follows:

• The tree-path from the root node to an internal node represents a partial solution (incomplete tuple);

• In each non-terminal node, the choice of a child node is conducted using the standard UCB rule in the finite case (see section 4.2 for the continuous case); When reaching the limits of the visited tree, a roll-out random strategy is applied until reaching a terminal node and computing the associated reward (more section 4.3).

• The reward associated to a full tree-path (terminal node, complete solution m) is computed.

• After the evaluation of a terminal node, the reward is backpropagated to update the value in each node of the visited tree path; this value will support the choice among the child nodes in the next tree-walk.

Continuous hyper-parameter values

As said, a main difficulty of the AutoML optimization problem is its mixed, continuous and discrete, search space. Multi-armed Bandit approaches have been extended to continuous search spaces (see in particular [START_REF] Wang | Algorithms for infinitely many-armed bandits[END_REF] and [START_REF] Bubeck | X-armed bandits[END_REF]), notably exploiting dichotomic divisions of the search space. Another approach has been taken here for two reasons. A first motivation is to enforce the scalability of the approach w.r.t. the dimension of the continuous hyper-parameter space preventing the use of regular decomposition of the search space (exponential in its dimension) along a grid. A second motivation is that, after [START_REF] James S Bergstra | Algorithms for hyperparameter optimization[END_REF], the uniform sampling of continuous hyper-parameter space outperforms the grid-based approaches, everything else being equal. Accordingly, the first value selected for a continuous hyper-parameter is uniformly selected in its range. The number of values considered is gradually extended with the number of visits n i to the choice node, along the Progressive Widening strategy [START_REF] Couëtoux | Continuous upper confidence trees[END_REF]. Formally, when the integer value of √ n i is incremented, a new value (uniformly selected in the value range), is considered.

Details of Mosaic Implementation

In its current version, the search space of MOSAIC is built upon the scikit-learn machine learning environment [START_REF] Pedregosa | Scikit-learn: Machine learning in Python[END_REF], with 13 data preprocessing methods and 17 classifiers. An extensive list of available data pre-processing methods / machine learning algorithms with their respective hyper-parameters is presented in Appendix A.

Mosaic approach relies on two remarks. On the one hand, as said the uniform exploration of the configuration search space is an excellent strategy except for its huge computational cost. On the other hand, as shown by [START_REF] Li | Hyperband: Bandit-based configuration evaluation for hyperparameter optimization[END_REF], there is (only) a factor two between the computational resources required by state-of-the-art Bayesian optimization SMAC [START_REF] Hutter | Sequential model-based optimization for general algorithm configuration[END_REF]TPE (Bergstra et al., 2011), and those of a pure random search needed to reach the same performance level. Mosaic accordingly attempts to get the best of both worlds: using the MCTS randomized exploration strategy and search space customized to reflect the domain structure, and its ability to achieve exploitation and intensify the search in the neighborhood of the most promising solutions.

The main limitation of MCTS for AutoML is related to the non-smoothness of the performance landscape. Typically, if a pre-processing algorithm is first evaluated with a poor algorithm (considering the dataset at hand), this pre-processing will not be further considerd for a long time. For this reason, each time a new node is created, k playout (k = 3 in the experiments) are launched and the maximum reward (over the k runs) is backpropagated to the ancestor nodes1 .

Experimental Results

All AutoMLP approaches can be combined with an ensemble strategy, retaining the best solutions found along the search and averaging them. Focusing on the comparative evaluation of the MCTS and SMAC search strategies, this section will be restricted to the evaluation of the vanilla Mosaic and Auto-Sklearn (without the ensembling and meta-learning options). The same configuration as in Feurer et al. (2015) is used, considering 133 (binary and multi-class) datasets2 and averaging the results over 10 independent runs. For each dataset, Mosaic and Auto-Sklearn are ranked according to their average test performance, and the rank averaged over all datasets of Mosaic and Auto-Sklearn is finally reported. The performance metric, used to rank two approaches on a fixed dataset, is the Balanced accuracy.

Training setup : Hyper-parameters of Auto-Sklearn are all set to default. For Mosaic, k (the number of evaluations to do when new node is expanded) is set to 3. Mosaic searches over a structured space of 173 hyper-parameters instead of 110 for Auto-Sklearn. The time budget for each run is set to one hour with six minutes timeout for one hyper-parameter setting evaluation. The memory is limited to 3GB for each run. All runs are launched on the same CPU model (Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz).

Results

A close look at the results show that the performance gap between Mosaic and Auto-Sklearn is quite small for several datasets (also stated in [START_REF] Li | Hyperband: Bandit-based configuration evaluation for hyperparameter optimization[END_REF]). For the sake of a fair assessment, we rank the two methods only when their performances are statistically different according to the Mann-Whitney test with p = 0.05. We then end up with 102 datasets on which one of the two methods is statistically better than the other. Figure 1 reports the average rank of the two methods showing that Auto-Sklearn outperforms Mosaic at the beginning. It is due to the Progressive Widening strategy (Section 4.2) used by Mosaic. However, Mosaic starts gaining performance when the tree search increases and it ends up with a slightly better performance than Auto-Sklearn.

Discussion and Perspectives

The main contribution of the paper is the Mosaic AutoML platform, adapting and extending the Monte-Carlo Tree Search setting to tackle the structured optimization problem of algorithm selection and configuration. The key ingredients of Mosaic are: i) the structure of the search space; ii) the ability to handle continuous hyper-parameter ranges. The merits of the approach are demonstrated as it matches the performance of the mature Auto-Sklearn which dominates the state of the art in the last 3 years. Two perspectives for further research will be considered. The first one is concerned with improving the sampling of continuous hyper-parameter values, by taking inspiration from AlphaGo [START_REF] Silver | Mastering the game of go with deep neural networks and tree search[END_REF]. A longer term research is to take advantage in the search of the meta-features describing the current dataset, and to capitalize the models learned from the past datasets, e.g. to achieve a smart initialization of the dataset-dependent hyper-parameter optimization as in Auto-Sklearn.

Algorithms

Figure 1 :

 1 Figure 1: Average rank (lower is better) of Mosaic and Vanilla Auto-Sklearn across 102 datasets (Datasets on which the performance of both methods differs statistically according the Mann-Whitney rank test with p = 0.05).

Table 2 :

 2 List of machine learning algorithms with their respective hyper-parameters.

		Parameters
	LinearDiscriminantAnalysis	solver, shrinkage
	QuadraticDiscriminantAnalysis	reg param
	DummyClassifier	-
	AdaBoostClassifier	base estimator, n estimators, learning rate, algorithm
		n estimators, criterion, max depth,
	ExtraTreesClassifier	min samples split, min samples leaf, min weight fraction leaf, max features,
		max leaf nodes, class weight
		n estimators, criterion, min samples split,
	RandomForestClassifier	min samples leaf, min weight fraction leaf, max features, max leaf nodes,
		class weight, bootstrap
		loss, learning rate, n estimators,
	GradientBoostingClassifier	max depth, criterion, min samples split, min samples leaf, min weight fraction leaf,
		subsample, max features, max leaf nodes
		learning rate, penalty,
	SGD Classifier	alpha, l1 ratio, loss, epsilon,
		eta0, power t, class weight, max iter
	KNN classifier	K, metric, weights
	Perceptron	penalty, alpha, max iter, tol, shuffle, eta0
	RidgeClassifier	alpha, max iter, class weight, solver
	PassiveAggressiveClassifier	C, max iter, tol, loss, class weight
	KNeighborsClassifier	n neighbors, weights, algorithm, leaf size, p, metric
		hidden layer sizes, activation, solver,
		alpha, batch size, learning rate,
	MLPClassifier	learning rate init, power t, max iter, shuffle, warm start, momentum,
		nesterovs momentum, early stopping,
		validation fraction, beta 1, beta 2, epsilon
	SVC	C, max iter, tol, loss, class weight, kernel, degree, gamma, coef0
		criterion, splitter, max depth, min samples split,
	DecisionTreeClassifier	min samples leaf, min weight fraction leaf, max features, max leaf nodes,
		min impurity decrease, class weight
		criterion, splitter, max depth, min samples split,
	ExtraTreeClassifier	min samples leaf, min weight fraction leaf, max features, max leaf nodes,
		min impurity decrease, class weight

Another strategy, left for further work, is to use specific policy based on extreme value estimates[START_REF] Achab | Max k-armed bandit: On the extremehunter algorithm and beyond[END_REF][START_REF] Bubeck | Bandits with heavy tail[END_REF].

Specifically, Feurer et al. (2015) uses 140 datasets but we were unable to get 7 of them.

Appendix A. List of machine learning algorithms and data preprocessing methods with their corresponding hyper-parameters


Jorge G Madrid

Hugo Jair Escalante

Eduardo F Morales
email: emorales@inaoep.mx

Wei-Wei Tu

Yang Yu

Lisheng Sun-Hosoya

Isabelle Guyon
email: guyon@chalearn.org

Michele Sebag
email: michele.sebag@lri.fr

Towards AutoML in the presence of Drift: first results

Keywords: AutoML, Life Long Machine Learning, Concept Drift, AutoSKLearn

Research progress in AutoML has lead to state of the art solutions that can cope quite well with supervised learning task, e.g., classification with AutoSklearn. However, so far these systems do not take into account the changing nature of evolving data over time (i.e., they still assume i.i.d. data); even when this sort of domains are increasingly available in real applications (e.g., spam filtering, user preferences, etc.). We describe a first attempt to develop an AutoML solution for scenarios in which data distribution changes relatively slowly over time and in which the problem is approached in a lifelong learning setting. We extend Auto-Sklearn with sound and intuitive mechanisms that allow it to cope with this sort of problems. The extended Auto-Sklearn is combined with concept drift detection techniques that allow it to automatically determine when the initial models have to be adapted. We report experimental results in benchmark data from AutoML competitions that adhere to this scenario. Results demonstrate the effectiveness of the proposed methodology.

Introduction

Autonomous Machine Learning (AutoML) is the field focusing on methods that aim at automating different stages of the machine learning process. AutoML solutions are increasingly receiving more attention from both the ML community and users because of (1) the large amounts of data readily available everywhere, and (2) the lack of domain and/or ML experts who can advise/supervise the development of ML-based systems.

Although progress in AutoML is vast, the considered scenarios are somewhat constrained, e.g., in the type of approached problem, in the assumptions on data, in the size c 2018 J.G. Madrid, H.J. Escalante, E.F. Morales, W.-W. Tu, Y. Yu, L. Sun-Hosoya, I. Guyon & M. Sebag.

of datasets, etc. In this context, one of the most desirable features for AutoML methods is to work under a lifelong machine learning (LML) setting. LML refers to systems that can sequentially learn many tasks from one or more domains in its lifetime [START_REF] Daniel | Lifelong machine learning systems: Beyond learning algorithms[END_REF], these systems (not restricted to supervised learning) require the ability to retain knowledge, adapt to changes and transfer knowledge when learning a new task. An AutoML method that learns from different tasks and that is able to adapt itself during its lifetime would comprise a competitive and robust all-problem machine learning solution.

This paper aims at exploring the viability of AutoML methods to operate in a LML setting, in particular in a scenario where the targets evolve over time, that is, in the presence of concept drift. We modify the Auto-Sklearn method with mechanisms that allow it to deal with the drift phenomenon in a simplified LML evaluation scenario. The proposed mechanisms are sound and highly intuitive, yet very useful. We perform experiments in benchmark data from concept drift and AutoML with drift. Experimental results reveal that the proposed mechanisms allow Auto-Sklearn to successfully cope with drift. To the best of our knowledge these are the first results reported on AutoML for a simplified LML scenario in the presence of concept drift1 .

Related work

Although the term 'AutoML' was coined recently by F. Hutter and collaborators (Thornton et al., 2013;[START_REF] Hutter | Automated configuration of algorithms for solving hard computational problems[END_REF], the problem of hyper-parameter selection has been studied for several decades in the machine learning community [START_REF] Bozdogan | Model selection and akaike's information criterion (aic): The general theory and its analytical extensions[END_REF][START_REF] Bengio | Gradient-based optimization of hyperparameters[END_REF][START_REF] James S Bergstra | Algorithms for hyperparameter optimization[END_REF][START_REF] Bergstra | Random search for hyper-parameter optimization[END_REF]. However, the original emphasis was on over-fitting avoidance while, with the emergence of "big data" the current emphasis is on search efficiency. Novel effective approaches have been proposed in the academic literature that have become wide spread among practitioners because they are both theoretically well founded and practically efficient: (1) Bayesian Optimization (BO) methods build a posterior p(model|data) by applying candidate models to the input data and use this posterior distribution to guide the search (e.g. [START_REF] Hutter | Sequential model-based optimization for general algorithm configuration[END_REF][START_REF] Swersky | Freeze-thaw bayesian optimization[END_REF]). (2) Complementary to BO, Meta Learning develops a set of meta-features capturing the nature of data, which are then used to infer the model performance based on past experiences on similar data, without actually training the model (e.g. [START_REF] Muñoz | Instance spaces for machine learning classification[END_REF]). (3) Evolutionary Algorithms (EA) learn a distribution over hyper-parameters and updates it to help the search (e.g. [START_REF] Real | Large-scale evolution of image classifiers[END_REF]). ( 4) Reinforcement learning approaches (RL), where the hyper-parameter optimization problem is formulated as learning an efficient policy to move in the hyper-parameter space and solved using RL techniques (e.g. [START_REF] Zoph | Neural architecture search with reinforcement learning[END_REF][START_REF] Baker | Designing neural network architectures using reinforcement learning[END_REF]). The algorithm Auto-sklearn used in this paper is based on a BO approach initialized with Meta Learning.

Hyper-parameter selection usually focuses on solving tasks in isolation, but, if tasks show some sequential dependency, a lifelong learning (LML) approach can be applied to continuously retain and immigrate knowledge across tasks and make the future learning more efficient. The concept of LML was first introduced in robot control [START_REF] Thrun | Lifelong robot learning[END_REF], where, inspired by the fact that a robot might be exposed to various learning tasks during their lifetime and the knowledge learned might be transfered to make future learning more efficient, they proposed EBNN (explanation-based neural network): suppose the robot perform all tasks in the same environment (e.g. housekeeping robot), then its knowledge about the environment (called action model, the mapping between stateaction pairs to the next state) can be transfered to new task using back-propagation. Since then, diverse techniques have been developed in lifelong supervised learning [START_REF] Daniel | The task rehearsal method of life-long learning: Overcoming impoverished data[END_REF], 1996, 2001;[START_REF] Chen | Lifelong learning for sentiment classification[END_REF][START_REF] Fei | Learning cumulatively to become more knowledgeable[END_REF].

Due to the big volume of data being constantly generated, many prediction problems need a model that continuously receives data and thus cannot work in an off-line/static mode with historical data. Some of these environments are non-stationary where data distributions change over time, this phenomenon is known as concept drift [START_REF] Jeffrey | Beyond incremental processing: Tracking concept drift[END_REF][START_REF] Widmer | Learning in the presence of concept drift and hidden contexts[END_REF][START_REF] Gama | A survey on concept drift adaptation[END_REF]. Numerous research work have been done for adapting models to data streams with presence of concept drift. [START_REF] Hulten | Mining time-changing data streams[END_REF] developed CVFDT where a Decision Tree is maintained up-to-date with regard to a sliding window of examples: an alternative sub-tree is built when drift is detected in the window, and is used to replace the old sub-tree when this new one becomes more performing. Then, by proving the out-performance of ensembling compared to single classifiers in the concept drift environment, Wang et al. (2003b) proposed weighted classifier ensemble, where adaptation is done by dynamically updating the weight for each base classifier according to its expected prediction accuracy on current test stream. To address the problem of sliding window size, [START_REF] Bifet | Learning from time-changing data with adaptive windowing[END_REF] introduced Adwin in which the window is resized based on changing rate observed in the window, Adwin is also used widely as a change detector as in [START_REF] Bifet | Leveraging bagging for evolving data streams[END_REF] Although work for processing streams and data in the presence of drift is vast, to the best of our knowledge existing work as not approaching the AutoML setting: automatically building and updating a full model [START_REF] Escalante | Particle swarm model selection[END_REF], that is a model that comprises data preprocessing, feature selection and classification model and that optimizes hyperparameters of the whole model.

In the scenario of LML-AutoML, the learning algorithm should be able to incorporate new data and update previous predictive models. In this paper, we follow the 3 steps of on-line adaptive learning procedure proposed in [START_REF] Gama | A survey on concept drift adaptation[END_REF]: (1)Predict. When new example X t arrives, a prediction ŷt is made using the current model L t . (2)Diagnose. After some time the true label y t is received and the loss can be estimated as f (ŷ t , y t ), and (3)Update. use the example (X t , y t ) to update the model to obtain L t+1 . Several methods that cope with concept drift use an explicit drift detection algorithm (Wang et al., 2003a;[START_REF] Beyene | Improved concept drift handling in surgery prediction and other applications[END_REF][START_REF] Street | A streaming ensemble algorithm (sea) for large-scale classification[END_REF]. One of the most used detector which is used as a benchmark in new detection methods is the Drift Detection Method (DDM) [START_REF] Gama | Learning with drift detection[END_REF]. It is a method that controls the trace of the on-line error by modeling the classifi-cation errors with a Binomial distribution. Some extensions to DDM have been developed to improve its performance. Fast Hoeffding Drift Detection Method (FHDDM) (Pesaranghader and Viktor, 2016) uses a sliding window and Hoeffding's inequality to detect drifting points earlier, experimental results also show that the number of false positive and false negatives is minor [START_REF] Pesaranghader | Fast hoeffding drift detection method for evolving data streams[END_REF][START_REF] Pesaranghader | Reservoir of diverse adaptive learners and stacking fast hoeffding drift detection methods for evolving data streams[END_REF]. Other methods extend FHDDM, however, these require additional parameters.

AutoML scenario and proposed methods

As previously mentioned we consider a simplified LML -AutoML with a concept drift scenario, which is precisely the scenario considered in the forthcoming AutoML32 challenge. The aim is assessing the robustness of methods to concept drift and its lifelong learning capabilities. It is assumed that there is an initial training (labeled) dataset available, and multiple batches of test data. Also, we assume that data are temporally dependent and are subject to an underlying form of the concept drift phenomenon. For the LML -AutoML evaluation, datasets will be split into sequential batches so that the lifelong scenarios can be evaluated: the test data will come in the form of sequential batches of data, each batch needs to be predicted at first, then the target values will be revealed and thus become a batch of new training data. From the model predictions and the revealed target values of a specific test batch, its performance can be evaluated. The performance of the model is given by taking the average across batches. Figure 1 illustrates the considered scenario.

Auto-sklearn

Auto-sklearn is an AutoML solution that has succeed in recent academic competitions (Feurer et al., 2015;[START_REF] Guyon | A brief review of the chalearn automl challenge: Any-time any-dataset learning without human intervention[END_REF][START_REF] Guyon | [END_REF]. In our opinion, it is THE state of the art on AutoML and for that reason we considered it for our study. Auto-Skelearn is implemented in scikitlearn [START_REF] Pedregosa | Scikit-learn: Machine learning in python[END_REF], it initially comprised 15 classification algorithms, 14 preprocessing methods, and 4 data preprocessing methods. Similarly to Auto-WEKA [START_REF] Thornton | Auto-weka: Combined selection and hyperparameter optimization of classification algorithms[END_REF], Feurer et al. approach AutoML as a Combined Algorithm Selection and Hyperparameter (CASH) or full model selection problem (Feurer et al., 2015;[START_REF] Escalante | Particle swarm model selection[END_REF], which they tackle using SMAC, a three-based Bayesian optimization method. There are two key components that make Auto-Sklearn so competitive. The first is based on metalearning, complementary to Bayesian optimization, it is used to warmstart the optimization by quickly suggesting instantiations of a framework that are likely to perform well. The meta-learning was done in an off-line phase where 38 meta-features were learned from 140 OpenML datasets [START_REF] Vanschoren | Openml: Networked science in machine learning[END_REF]. The second feature is the automated ensemble construction of models evaluated during optimization, when finding the best model instead of discarding the rest of the models found in the Bayesian optimization process, Feurer, et. al. store them and then build and ensemble using a greedy ensemble selection algorithm.

Proposed method

Auto-Sklearn was modified with basic mechanisms for allowing it to cope with incoming data as depicted in Figure 1. The overall proposed procedure is shown in Algorithm 1. With a first batch, an initial ensemble model is learned by using Auto-Sklearn. With the generated model predictions are made for the next batch. After the predictions are made, feedback for this batch is received (i.e., ground truth), using a drift detector these predictions are diagnosed to determine if drift has happened. If the detector triggers the drift alarm, the current ensemble model is adapted by using one of the mechanisms described below. After this, the system receives the next batch and the process is repeated until no batches are left. In the rest of this section we describe the components of the proposed method, namely: the drift detector and the proposed adaptation mechanisms.

Data: D(X, y) examples Take a batch D t of size n, D t (X , y ) ∈ D(X, y); T t ← learn a model with auto-sklearn using D t while there is data in D do Take next batch D t+1 ; ŷ ← Make predictions with T t ; for y j ∈ ŷ do drift detected = Detector(y j == y j ∈ y ) //drift will be detected with model performance end if drif detected then T t+1 = adapt(T, D t+1 ); Detector.reset(); t = t + 1 else T t+1 = T t ; t = t + 1 end end Algorithm 1: Drift adaption schema for Auto-sklearn

Drift detector

In a strict LML -AutoML context, where no human intervention is expected, selecting a drift detection algorithm is not trivial, different methods perform differently depending on the type of drift in the data: incremental, abrupt, reoccurring, gradual. Some of these methods also require parameter tuning to achieve better results. For this experiment, we decided to use a particular drift detector with default hyper-parameters to make the problem more manageable and postpone to future work the automatic selection of this type of models. Specifically, the Fast Hoeffding Drift Detection Method (FHDDM) method was selected because according to experiments in [START_REF] Pesaranghader | Reservoir of diverse adaptive learners and stacking fast hoeffding drift detection methods for evolving data streams[END_REF] it was found that in many datasets its performance can be compared to the state-of-the-art. The implementation of FHDDM in Tornado [START_REF] Pesaranghader | Reservoir of diverse adaptive learners and stacking fast hoeffding drift detection methods for evolving data streams[END_REF] was integrated with AutoSKlearn.

Model adaptation methods

Three approaches were implemented for adapting the model generated by Auto-Sklearn, a global replacement strategy where the initial model found by Auto-Sklearn is retrained with new examples and two variants of model management strategies. Model replacement. A traditional concept drift adaptation where the model is globally replaced with a new one. Taking advantage of the Auto-Sklearn improvements, the model is not trained from scratch. Having the meta-features learned with the meta-learning process from Auto-Sklearn, these are used for the creation of the new model, which is a new ensemble with the best models found by the SMAC process, the learning for the experimentation was performed storing all data received instead of only the current batch. Model management. Two variants of this formulation were considered. The first strategy assumes the data is generated by a mixture distribution (that formed by previous batches and the current one) and the ensemble weights from the initially learned ensemble are updated with new data, either: (1) using the latest batch or (2) using all the stored data up to the current batch, we call these variants WU-latest and WU-all, respectively. Creators of AutoSKlearn (Feurer et al., 2015) found that uniformly weighted ensembles did not work well. Instead, a greedy ensemble selection algorithm was used to build the ensemble. Such same algorithm is used in this work to update the weights of each model under the model management variants. With the new data every model performance is computed again and the ensemble is rebuild with new information.

In a second model management variant called Add New, new models are learned with all the stored data, meta-learning process is also skipped in this case, and using auto-sklearn ensemble construction methods these new models are integrated in the initial ensemble. However, the implementation of this last strategy did not work as expected since the original design of auto-sklearn does not consider adding new models to the ensemble so a new model selection algorithm for the ensemble construction ought to be implemented.

Experiments and results

Data

For the empirical evaluation we used benchmark data form both concept drift and the AutoML fields, where the AutoML data sets considered are known to incorporate temporal dependencies across instances. The considered data sets are described in Table 1.

Concept drift data sets have been widely used for the evaluation of drift detection techniques. Chess comprises 2 years of chess games collected from chess.com, the task is to predict whether a player will win or lose a match given players and game setup features, the player increases its abilities through time and faces more skilled players, this is where the concept drift is expected. In the Poker dataset each instance is an example of a hand having five playing cards drawn from a standard deck. Each card is described by two attributes (suit and rank) for a total of ten attributes. The class predicts the poker hand, 10 classes are considered. Since the poker hands were generated in order (varying suits and ranks in order), the concept drifts multiple times. The electricity market dataset was collected from the Australian New South Wales Electricity Market. Each instance represents the market state in a period of 30 minutes. The task in this dataset is to predict if the market prices are going up or down for the next period of time. Finally, a synthetic Stagger dataset was generated to test the methods, three drift points are artificially induced were the concept changes abruptly. According to previous work, we used accuracy as evaluation measure for these datasets.

AutoML data sets comprise undisclosed data from click user data, these datasets were used in the AutoML2 challenge [START_REF] Guyon | Analysis of the automl challenge series 2015-2018[END_REF]. Although the data has been keep confidential, it is known that there is a temporal dependency across instances. We expect the proposed methods to capture these dependencies and improve the performance of a straightforward AutoML technique. We know RH and RI were collected from applications in startup phase, and behavior changed dramatically during that time. RL was collected from content recommendation applications, the items were user generated contents, there were a certain proportion of new contents generated each day, the user preferences of the items were changed along time. PM and RM were collected from a mature recommender system in different time periods, the items were cars, the candidate set of cars was relatively stable along time, the user preferences were also relatively stable compared to other datasets. For these datasets we used the normalized area under the ROC curve as evaluation measure.

Results

For each of the considered datasets the following setup was considered. Training partitions were used to obtain an AutoSKlearn model. For concept drift datasets, the data was divided into batches of equal size, and the first batch was used as training set. For the AutoML datasets the predefined training partitions were used for generating the initial model. After that, the test set was processed in batches. The actual model was used to predict labels for the batch t and performance was estimated, after that, test labels for this batch were made available to the method. Then we applied the drift detector. If drift was detected, the methods described in Section 3 were applied to adapt the current model, which was then used for batch t+1. In experiments, we compare the performance of the initial AutoSKlearn model (Base) with the replacement, WU-all, WU-batch and the Add New methods.

Table 2 reports the average (across batches) performance for the concept drift datasets. Regarding the real datasets, in all but one configuration (Poker dataset with WU-latest method) the drift aware AutoML solutions outperformed the base model. As it can be expected, replacing the initial model with a new model obtained by applying AutoSKlearn with all available data obtained the best performance. Model replacement, however, is the most computationally expensive solution. A competitive alternative is the WU-all method, in which ensemble weights are updated by using all of the available data. In our experiments, WU-all method was in the worse case about 5 times faster than the replacement technique. The speedup was more noticeable in large datasets.

Interestingly, most drift aware methods failed with the synthetic data set (Stagger ), this was due in part to the abrupt drift present in data: synthetic drift is simulated by inverting the concept (negative instances are considered as positive ones). WU-all and WU-latest models only consider models built before the drift (i.e. with the rule before the drift) and thus fail to classify correctly after drift detected. The Add new method also tries to incrementally adjust to changes, but AutoSKlearn fails to find an ensemble that classifies correctly both instances before and after the drift. On the other hand, the replacement method builds models from scratch after drift is detected and is able to improve the accuracy of the ensemble. Figure 2 shows the per-batch improvement of AutoML drift aware methods vs. using the initial model. The figure shows a somewhat erratic behavior at times, although, in general, these plots confirm the results presented in Table 2. It can be seen from these plots that the WU-batch method is very sensitive to the batch used to adjust weights (see rightmost top plot). The Replacement and WU-all strategy consistently outperform the base model in the real data sets, and for the synthetic dataset (rightmost bottom) all methods but replacement perform poorly.

Table 3 shows the results obtained in the AutoML2 challenge data sets, performance is reported in terms of the normalized AUC. For PM and RM datasets the drift detector did not detect drift in any batch, and therefore the base model was used for making predictions in all the batches. This could be due to the maturity of the application from which PM and RM were collected from. The drift of concept evolves slowly and it cannot be captured by the batch size. On the other hand, for datasets collected from startup businesses or UGC recommender systems, due to the changing nature of the applications, for RH, RI and RL, at least one of the adaptive AutoML methods outperformed considerably the base performance. For RL, the Replacement method improved relatively the performance of the base model by 47%. For RI the WU-latest method improved the initial performance by more than 60%. Whereas for RH all of the adaptive AutoML variants outperformed the reference performance; the highest relative improvement being of 73% obtained by the WU-all strategy. Although more experiments are needed, the results in AutoML2 data suggest/confirm that each dataset presents a different variant of the concept drift phenomenon. RL seems to follow the behavior of datasets in Table 2. RH seems to exhibit a slowly progressing drift, where WU-all takes advantage. While RI seems to show a drift aligned with the concept drift detector (hence the WU-latest method is able to find quite useful weights).

Conclusions

AutoML is an increasingly trending topic because of its relevance given the amount of data being generated nowadays. Although very effective AutoML methods have been proposed so far, to the best of our knowledge all of them focus on the assumption of static (iid) data. However, on-line data is constantly changing and therefore, static solutions may not be useful for evolving data streams. This paper described preliminary experimentation on the evaluation of the drift adaptive capabilities of a state of the art AutoML solution. To the best of our knowledge this is the first work dealing with AutoML in the presence of drift. Experimental results in benchmark data confirm the usefulness of the proposed mechanisms for dealing with drift. Although our results are far from being conclusive, they bring some light into the performance of AutoML for evolving data streams.

  and van Rijn et al. (2014). The same authors then came up with Leverage Bagging (Bifet et al. (2010)) where examples are weighted by a well parameterized Poisson distribution to add randomization in input data that improve the accuracy of ensembling. van Rijn et al. (2014), Rossi et al. (2014) and van Rijn et al. (2015) collaborated meta-learning techniques to solve the algorithm selection problem: predict best model for next upcoming stream based on meta-knowledge collected from previous streams.

Figure 1 :

 1 Figure 1: Evaluation scenario considered in the AutoML3 challenge.

Figure 2 :

 2 Figure 2: Performance of drift aware AutoML variants. From top to bottom and left to right results for: Electricity, Poker, Chess and Stagger are shown, respectively.

Table 1 :

 1 Datasets considered for experimentation.

			Concept drift datasets
	Dataset	instances attributes Reference
	Chess	503	8	( Žliobaite, 2010)
	Poker	100,000	10	(Olorunnimbe et al., 2015)
	Electricity 45,312	8	(Baena-García et al., 2006)
	Stagger	70,000	3	(Gama et al., 2014)
		AutoML2 challenge data sets
	Dataset	instances attributes Reference
	PM	49,964	89	(Guyon et al., 2018)
	RH	60,042	76	(Guyon et al., 2018)
	RI	57,306	113	(Guyon et al., 2018)
	RL	56,209	22	(Guyon et al., 2018)
	RM	55,239	89	(Guyon et al., 2018)

Table 2 :

 2 Results on benchmark data.

	Method	Electricity Poker Chess Stagger Rank
	Base	67.15	67.97	38.23	54.09	4.5
	Replacement	76.44	90.38 58.13 78.81	1
	WU-all	70.23	76.89	52.62	54.09	2.75
	WU-latest	67.95	67.49	53.24	54.09	3.5
	Add new model 67.47	74.98	47.28	54.14	3.25

Table 3 :

 3 Results on data from the AutoML2 challenge.

	Method	PM * RH	RI	RL	RM *
	Base	0.433 0.192 0.299 0.340 0.264
	Replacement 0.433 0.197 0.092 0.478 0.264
	WU-all	0.433 0.370 0.199 0.212 0.264
	WU-latest	0.433 0.270 0.450 0.405 0.264
	Add new	0.433 0.298 0.184 0.277 0.264

One should note that our work is different from standard data-stream / concept drift learning methods in the sense that in AutoML we aim to autonomously find preprocessing, feature selection and classification methods (together with hyper parameter optimization), while in most of the existing literature the problem is reduced to modify the classification model.

https://www.4paradigm.com/competition/nips2018


Jean-Charles Quinton

Annique Smeding

Dynamic competition and binding of concepts through time and space Quinton & Smeding

Models of implicit stereotypes (e.g. association of male with math, or female with language) usually explain the faster responses observed for stereotype congruent trials in the Implicit Association Test (IAT) by requiring a fundamental opposition between the malefemale concepts (or math-language), limiting the decision-making dynamics to abstract dimensions. This paper introduces alternate models exploiting the sensorimotor dimensions of the IAT, which naturally account for the opposition between concepts, because typically mapped on opposite corners of the screenspace and different response actions. In addition to the emergence of the IAT effect, dynamic characteristics of the decision-making process within these models are tested against human data, obtained with a mouse-tracking adapted IAT procedure.

Introduction

Stereotypes have been defined as "associations of a social group concept with one or more (nonvalence) attribute concepts" [START_REF] Greenwald | A unified theory of implicit attitudes, stereotypes, self-esteem, and self-concept[END_REF]. For instance, in the case of a gender-math stereotype which will be the target illustration of this paper, math is more strongly linked to the social group of males than to females (Nosek, Banaji, & Greenwald, 2002). Such associations are constitutive of a social knowledge structure, formed and Dynamic competition and binding of concepts through time and space Quinton & Smeding reinforced on the long term, and can be measured at the implicit level. This is the case in the Implicit Association Test (IAT; [START_REF] Greenwald | A unified theory of implicit attitudes, stereotypes, self-esteem, and self-concept[END_REF], where stimulus words such as man and equation (appearing in the middle of a computer screen) must respectively and quickly be categorized into target categories male and math (appearing at the top left or top-right corner of the screen, depending on experimental condition). The IAT effect corresponds to the slowing down of responses and increase in errors when moving from stereotype-congruent trials (male and math categories grouped on one corner of the screen and opposed to female and language categories, grouped on the other corner) to stereotype-incongruent trials (male and language grouped and opposed to female and math ). This opposition occurs both at the visual level (target categories being located on opposite sides of the computer screen) and at the motor level (a single key press being used for categories on the same side of the screen).

In the theoretical framework behind IAT effects (i.e. Unified Theory, [START_REF] Greenwald | A unified theory of implicit attitudes, stereotypes, self-esteem, and self-concept[END_REF]), it is posited that associations between stereotypical concepts are reinforced (balancecongruity principle), and that bipolar opposition of concepts such as male and female (imbalance-dissonance principle) must be present. A direct competition is also invoked in more recent developments, where attribute concepts as well as social group concepts are made mutually inhibitory in neural network based computational models [START_REF] Freeman | A dynamic interactive theory of person construal[END_REF].

On the one hand, the IAT literature thus supports that social knowledge (i.e., stereotypes) can influence our behavior and decisions. On the other hand, the dynamic categorization literature shows that categories themselves, or at least the binding or competition of their constitutive features or elements, directly depend on the task to be performed. Words such as grandmother, grandfather, daughter and son (all part of the gender-math IAT stimuli set) will be bound differently depending on whether target categories are male vs. female or young vs. old. Even if we cannot rule out the possibility that competition between concepts (such as male and female) may be permanent, competition must be modulated by the context to not hinder performance when target categories change.

For categorization tasks with non linguistic stimuli, including stereotype related tasks (e.g. [START_REF] Freeman | A dynamic interactive theory of person construal[END_REF], the competition and binding of features occur in space and time. Indeed, and especially for visual stimuli, objects are often described by a set of more or less complex features, and active capabilities of the human perception system allow to sequentially focus on, or search for, discriminative features (Quinton, Catenacci Volpi, Barca, & Pezzulo,2014). These discriminative features depend on the target categories, so that even the exact same stimuli might be explored, processed and categorized differently depending on context and a prioris [START_REF] Kietzmann | Overt visual attention as a causal factor of perceptual awareness[END_REF]. Therefore both the sensorimotor apparatus and the experimental design put constraints on the dynamic decision process.

Taken together, these two literatures acknowledge the wide range and huge amount of information that can be quickly integrated, as well as the complementary processes that influence decisions. However, they diverge when it comes to the characteristics of competition. Indeed, the literature on social stereotypes assumes a permanent male-female competition at the conceptual level, while dynamic categorization research indicates that this competition may be temporary and may depend on task constraints. This partial incompatibility is not surprising, as Unified Theory was developed to provide theoretical support to the IAT effect, with a disciplinary focus from social psychology. Reciprocally, research on dynamic categorization originated in cognitive psychology and neuroscience, and is only progressively expanding to encompass social factors.

Our aim is to reconcile the two literatures around these notions of binding and competition, by adopting a dynamical system perspective and considering the different levels at which competition might occur (i.e., conceptual vs. perceptual). In the present research, we combine an empirical study with a computational study to test the hypothesis that dynamic Dynamic competition and binding of concepts through time and space Quinton & Smeding competition is required to correctly model human behavior, but that such competition is not required at the conceptual level to explain the emergence of the IAT effect. The next section introduces the empirical procedure of the present research. This procedure relies on a mousetracking adapted IAT [START_REF] Yu | Beyond reaction times: Incorporating mousetracking measures into the implicit association test to examine its underlying process[END_REF] specifically designed to collect continuous data and to allow the analysis of dynamical characteristics of categorization processes under the influence of stereotypes. A set of connectionist models are then introduced as proofs of concepts. These models are tested against human data, demonstrating that the IAT effect might actually depend on the experimental design and response modalities, involving sensorimotor dimensions and not being limited to abstract concepts. Nevertheless, the same models also suggest that the effect is not an artifact of the procedure, as it also requires and reflects stereotypical associations between concepts.

Empirical Study

Method

The standard IAT procedure involves a comparison of latencies and error rates between congruent and incongruent trials. It therefore provides little information about the dynamics of the decision, nevertheless required to reach a deep understanding of the processes underlying categorization under the influence of stereotypes, and thus to test the associated theoretical or computational models. On the contrary, mouse-tracking, i.e. the tracking of computer mouse movements, has proven to be effective in revealing the online decision-making dynamics underlying (social) categorization [START_REF] Freeman | A dynamic interactive theory of person construal[END_REF].

Except for a research targeting valenced associations [START_REF] Yu | Beyond reaction times: Incorporating mousetracking measures into the implicit association test to examine its underlying process[END_REF], the IAT literature has remained silent with respect to these theoretical and methodological advances.

To investigate the dynamics of categorization and obtain human data to confront computational models to, we used a mouse-tracking adapted IAT procedure investigating Dynamic competition and binding of concepts through time and space Quinton & Smeding gender-math associations (see Fig. 1 for the setup, [START_REF] Nosek | Math = Male, Me = Female, therefore Math≠Me[END_REF] for details on the original procedure). Twenty-two female humanities undergraduates took part in the study and were chosen because of the typical IAT effect observed in this sample [START_REF] Smeding | Women in Science, Technology, Engineering, and Mathematics (STEM): An Investigation of Their Implicit Gender Stereotypes and Stereotypes' Connectedness to Math Performance[END_REF]. Data were part of a larger study.

Results

Trials where the final categorization was erroneous (6.7%) were removed. Coordinates (x,y) of mouse trajectories were remapped rightward in [-5, 5]×[-0.5, 9.5] starting from (0,-0.5) Analyses on global statistics over trajectories were performed using mixed-effects models, with standard temporal and geometrical mouse-tracking statistics (reflecting attraction toward the non-selected response) regressed on trial congruency: RT (Response Time), AUC (Area Under the Curve) and MD (Maximum Deviation). The effect was significant for RT, B = 65.5, t(2440) = 3.21, p = .001. As expected, RT for female humanities students was higher for incongruent trials (M = 901ms, SD = 452) than for congruent trials (M = 832ms, SD = 347). Consistent yet not significant results were obtained for MD, B = 0.18, p = .05, and AUC, B = 0.07, p = .07, reflecting a larger deviation for incongruent trials(see Fig. 2).

Turning to the dynamical characteristics of trajectories, we computed the average speed profile on the X axis, on which categories are opposed (Fig. 3). ANOVA was performed between congruent and incongruent trials for all participants. Speed profiles showed a significant difference in mean speed amplitude, B = -0.21, t(42) = -2.22, p = 0.03, with slower movements for incongruent trials (M = 1.44, SD = 0.35) than congruent trials (M = 1.65, SD = 0.26). Similar results were obtained for the center of mass of the distribution, reflecting a shift in time B = 130.1, t(42) = 3.27, p = .002, with slower onset for incongruent trials (M = 815ms, SD = 174.3) than congruent trials (M = 685ms, SD = 66.9).

Dynamic competition and binding of concepts through time and space Quinton & Smeding

Computational Study

Method

Human data from our mouse-tracking IAT demonstrate that hypotheses about the interpretation of stimuli influence the behavior in real-time and compete from an early stage of the decision-making process [START_REF] Spivey | The Continuity of Mind[END_REF]. To develop the connexionist models introduced in this section, we have introduced representations of these hypotheses and made them interact. The behavior and final decision are thus emergents of the interactions between the task environment (e.g. stimuli) with a network of concept nodes c i  {math, language, male, female...} (see Fig. 4). Pairs of nodes (c i , c j ) may interact through a link of strength w ij , which can be related to a wide variety of representations, including (a) social knowledge (e.g. math-male stereotype), (b) linguistic knowledge and task constraints (e.g. father-male association) and (c) knowledge about experimental design constraints (e.g. male-left mapping on the screen). To each node c i is associated a dynamic activity a i , governed by Eq.1. This is a continuous time version of existing stereotype models based on recurrent neural network [START_REF] Freeman | A dynamic interactive theory of person construal[END_REF], including a relaxation term (-a i ), the nonlinear influence of connected nodes (w ij (a j )) and noise ( ).

To make the comparison of models relevant to the stereotype literature, we only initially consider excitatory links between nodes (see [START_REF] Greenwald | A unified theory of implicit attitudes, stereotypes, self-esteem, and self-concept[END_REF]. As we are interested in generative models able to reproduce mouse trajectories toward responses, we however need to introduce additional nodes for left and right associated to sensorimotor behavior (linked to either target location or movement orientation). The choice of approximating continuous dimensions by two opposing nodes was made for simplification, but results could be extended by using dynamic neural field models [START_REF] Quinton | Exploring and optimizing dynamic neural fields parameters using genetic algorithms[END_REF]. The Dynamic competition and binding of concepts through time and space Quinton & Smeding current model can indeed be considered as the limit case of the classical neural field equation, that is extensively used to account for decision dynamics on continuous feature spaces.

At any time during the simulations, the mouse position is updated based on a speed vector, computed as a weighted sum of vectors pointing to the targets from the current position, scaled to roughly reproduce the human hand dynamics, and with weights set to the activity of the respective left or right nodes [START_REF] Quinton | The cat is on the mat. Or is it a dog? Dynamic competition in perceptual decision making[END_REF]. Taking the maximum likelihood over any activity distribution (e.g. left-right) instead leads to initial jerky movements not found in human data. Adding an activity threshold from which movement is initiated in part solves this problem, but the simulated data then cannot account for the progressive bifurcation (with initial straight-up movement) and hesitations (i.e. changes in direction) observed in human trajectories, showing that a continuous integration and competition of hypotheses seems required, either at the neuronal or body apparatus level.

To further test this hypothesis, we introduce 3 different models in addition to the one previously described, that lacked inhibition (model A). These models simply differ by the level at which competition takes place, introducing an inhibitory link between nodes either at the conceptual level (model B), sensorimotor level (C), or both (D).

Results

In order to ensure a fair comparison, a metaheuristic was applied to select the best parameters for each individual model (see [START_REF] Quinton | Exploring and optimizing dynamic neural fields parameters using genetic algorithms[END_REF] for methodological details).

Optimization criteria were selected to reflect the capability to generate: (i) correct responses, (ii) convergence to a clear decision when the time constraint is relaxed, (iii) smooth trajectories with progressive bifurcation, (iv)significant differences between the congruent and incongruent blocks on RT, AUC and MD.

Taken separately, criteria were correctly satisfied by all models, yet optimization led to different parameter values. Satisfying criteria (iv) alone for instance means that all models are able to account for the IAT effect. However, taking all criteria together leads to different results, with model A generating wrong decisions or chaotic trajectories, and model B not satisfying criteria (ii). The time constraint is a requirement of the mouse-tracking paradigm, because if relaxed, the participant may decide before engaging in action. With this in mind, authors of previous computational studies argue that conceptual nodes could settle into an attractor with a mixed activity, allowing non stereotypical attributes to remain marginally active, while still being able to reach a decision (see [START_REF] Freeman | A dynamic interactive theory of person construal[END_REF] for a type B model, [START_REF] Greenwald | A unified theory of implicit attitudes, stereotypes, self-esteem, and self-concept[END_REF] for type A). Although we defend the same position, any model willing to account for human decision-making should also be able to generate straight trajectories if movement starts once the dynamical system has converged to its fixed attractor. This is only the case for model C and D, where a distributed activity is maintained at the conceptual level (e.g. a male =0.8, a female =0.2), but where competition at the sensorimotor level guarantees a clear decision after an infinite amount of time (e.g. a left =0, a right =1).

Discussion

The present research relied on a mouse-tracking adapted IAT to investigate the dynamical processes underlying implicit gender-math associations. In addition to replicating the IAT effect, our results demonstrate that, beyond reaction times, the influence of congruency influences the whole behavior, as reflected by early mouse movements. Mouse trajectories were correctly simulated by a set of generative dynamical connectionist models, using a few conceptual and sensorimotor nodes, with block dependent binding.

However, only models including competition at the sensorimotor level were able to satisfy all criteria characterizing human decision-making. Although we cannot rule out that the modeling choices are responsible for the lack of solution satisfying all criteria when competition is absent or limited to the conceptual level, the computational models were designed to be as similar as possible to references from the literature. In virtue of Occam's razor, also known as the law of parsimony, competition at the sensorimotor level alone should be preferred, as it is sufficient to explain the decision dynamics. Additionally, not requiring fixed inhibitory links between abstract concepts (such as male and female) reconciles the IAT literature with the flexible binding capabilities demonstrated by humans in dynamic decisionmaking. During the simulation, activity of the stimulus node is maintained to 1.0, generating indirect activations from first-order links (e.g. probability of math category to be relevant when observing the word father, even if there is no direct math-father link). In the end, mouse movements are generated from the activities of the left and right nodes associated with the onscreen targets. The dynamical binding between the math and male through left illustrated here corresponds to stereotypical trials. Models A, B, C and D differ by the inhibitory links introduced (dashed lines).

Figure 1 .

 1 Figure 1. Setup used in both experimental and computational studies. Participants were

Figure 2 .Figure 3 .Figure 4 .

 234 Figure 2. Averaged trajectories for congruent and incongruent trials for all participants,

Acknowledgments

This research was supported by the French National Center for Scientific Research (CNRS) through the PEPS « Humanités -Mathématiques -Sciences de l'information » scheme and Université Savoie Mont Blanc through the International Relations Grant (RI) Dynamic competition and binding of concepts through time and space Quinton & Smeding