Nicolas Gutowski
email: nicolas.gutowski@univ-angers.fr

Daniel Schang
email: daniel.schang@eseo.fr

Olivier Camp
email: olivier.camp@eseo.fr

Pierre Abraham
email: piabraham@chu-angers.fr

A novel multi-objective medical feature selection compass method for binary classification

Keywords: Genetic Algorithm Application, Multi-objective Feature Selection, Extreme Learning Machine, Machine Learning, Cardiology, Medicine 2020 68T05, 68T20, 68W50, 90C29, 92C50

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Predictive medicine is commonly used in the medical world: from radiotherapy to pharmacotherapy or health-care management. One of the great challenges such predictions face is to reduce the complexity of the decision problem while maintaining acceptable decision performances.

Typically, when taking a decision or making a diagnosis, practitioners can only consider a limited number of decision variables. It is therefore important for them to be able to rely on automatic methods aimed at selecting the most relevant subset of features to take into account in their decision problem, while maintaining good classification performances such as accuracy or Area Under the ROC Curve (AUC). Moreover, this helps reducing cognitive load upon the practitioners while decreasing the economical cost of variable collection (e.g., Vitamin D deficiency diagnosis [START_REF] Annweiler | Derivation and validation of a clinical diagnostic tool for the identification of older community-dwellers with hypovitaminosis D[END_REF]).

To cope with such issues encountered in many fields of application (e.g., finance [START_REF] Ng | Lg-trader: Stock trading decision support based on feature selection by weighted localized generalization error model[END_REF], bank [START_REF] Kozodoi | A multi-objective approach for profit-driven feature selection in credit scoring[END_REF], business & marketing [START_REF] Huang | Multi-objective feature selection by using NSGA-II for customer churn prediction in telecommunications[END_REF], medicine [START_REF] Lorenz | Feature selection with NSGA and GAAM in EEG signals domain[END_REF]), Multi-Objective Feature Selection (MOFS) problems have been the subject of many research works [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF][START_REF] Sohrabi | Multi-objective feature selection for warfarin dose prediction[END_REF][START_REF] Deniz | Robust multiobjective evolutionary feature subset selection algorithm for binary classification using machine learning techniques[END_REF][START_REF] Zhu | An improved NSGA-III algorithm for feature selection used in intrusion detection[END_REF][START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF][START_REF] Al-Tashi | Approaches to multi-objective feature selection: A systematic literature review[END_REF][START_REF] Dong | A multi-objective multi-label feature selection algorithm based on shapley value[END_REF][START_REF] Deniz | Evolutionary multiobjective feature selection for sentiment analysis[END_REF][START_REF] Gao | Multi-objective optimization of feature selection using hybrid cat swarm optimization[END_REF][START_REF] Nouri-Moghaddam | A novel multi-objective forest optimization algorithm for wrapper feature selection[END_REF]. MOFS problems can be formalized as a kind of Multi-Objective Optimization problems [START_REF] Deb | Multi-objective optimization[END_REF] which consists in minimizing the number of features to select from a dataset while maximizing one or several performance metrics such as accuracy of the decision.

The main contribution of this work is the definition of a novel method to solve such MOFS problems. It combines one of the most popular Genetic algorithm (GA) in the literature [START_REF] Holland | Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence[END_REF] with a Compass based fitness computation method [START_REF] Maturana | A compass to guide genetic algorithms[END_REF][START_REF] Gutowski | Gorthaur-exp3: Bandit-based selection from a portfolio of recommendation algorithms balancing the accuracy-diversity dilemma[END_REF] promoting optimal solutions respecting a given trade-off between the problem's objectives. The use of such a fitness function eases the tuning of the objective as it is defined by a single 3D reference vector. Moreover, such a parameter makes the search direction for the solutions more explicit. As a secondary contribution, this paper studies the interest of using a classification qualification metric seldom used when solving MOFS problems, namely the Area Under the ROC Curve (AUC), as an objective. This article presents an approach for multi-objective feature selection, the Genetic Algorithm with Multi-Objective Compass (GAwC). It uses a GA hybridized with a 3-Dimensional Compass method guided by a reference vector set to : 1) minimize the number of selected features; 2) maximize the classifier's accuracy; 3) maximize the classifier's AUC.

The novelty between GAwC and other competitive methods is twofold : 1) It includes AUC, a classification quality metric seldom considered in the objective function for feature selection; 2) It provides an aid to the user in the parameterization of the objective function. This is obtained by only requiring the setting of a 3D reference vector to define the influence of each objective in the solution. This makes the search direction more explicit and visual for the user.

To evaluate the performances of GAwC on various real-world medical datasets, the method is compared with several Multi/Many-Objective Evolutionary Algorithm (MOEA, MaOEA) based state-of-the-art competitive methods such as QEISS-like [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF] or DMEA-FS-like [START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF] methods. The method's different performance metrics -i.e.; specificity, sensitivity, Matthews correlation coefficient and AUC obtained when applying GAWC on a specific Drug Intake prediction dataset collected by the cardiology department of the University Hospital of Angers in France (CHU Angers) are discussed.

The paper is organised as follows: Section 2 reviews the related work and clarifies the background along with the problem. Section 3 sheds light on the GAwC method while section 4 depicts how its empirical evaluation against other state-of-the-art methods was conducted. Furthermore, Section 5 provides a discussion which is twofold : 1) it discusses results obtained by GAwC in terms of classification performance metrics only;

2) it gives a clinical view point on the results obtained by GAwC with the ASA Drug Intake (ASA-DI) real-life dataset and especially provides a justification of the results obtained from a clinical point of view. Finally, a section for conclusion and perspectives brings the paper to a close.

Related Work and Background

This section starts by defining Multi-Objective Optimisation Problems (MOPs), and the main algorithms used to solve them. Then, in a second part, it focuses on multiobjective approaches applied to medical feature selection. Finally, it depicts the background methods the GAwC method relies on.

Multi-objective Optimization Problems

Multi-objective Optimization problems (MOP) [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF] and, by extension, Many-objective Optimization Problems (MaOP) [START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF], are a kind of optimization problems involving k conflicting objectives where 2 ≤ k ≤ 3 for MOPs and k ≥ 4 for MaOPs.

The vector-valued objective function of a MOP or a MaOP is defined as:

F : X → R k , F( x) = (F 1 ( x), . . . , F k ( x)) ⊺ (1)
where x is a feasible decision vector and set X ⊆ R d is the d-dimensional feasible decision space where d depends on the application and corresponds to the number of decision variables.

Thus, a MOP or MaOP can be formalized as follows:

max (F 1 ( x), F 2 ( x), . . . , F k ( x)) , s.t. x ∈ X (2)
In MOP/MaOP, a feasible decision that maximizes all objective functions concomitantly cannot generally be obtained and a trade-off needs to be found. It should be noted that problems aiming at finding the minimum of a function can be reformulated as an equivalent maximization problem applied to the inverse or the negative of the function.

In such MOP/MaOP problems, the aim is to find Pareto optimal (or Pareto efficient) solutions. A feasible decision x ∈ X is Pareto-optimal if, and only if, it Pareto dominates all other existing solutions. More formally, a feasible solution x ∈ X is Pareto-optimal if, and only if:

∀ x ′ ∈ X; ∀j ∈ [1, k]; F j ( x ′ ) ≤ F j ( x) (3)
Otherwise, the corresponding outcome of F for the feasible decision x can be considered inefficient in the objective space and solution x is said to be Pareto dominated. The set of all non Pareto dominated solutions is known as the Pareto front.

There exist numerous methods to solve MOPs/MaOPs [START_REF] Marler | Survey of multi-objective optimization methods for engineering[END_REF][START_REF] Caramia | Multi-objective optimization[END_REF], for example methods based on Operational Research [START_REF] Marler | The weighted sum method for multi-objective optimization: new insights[END_REF] or Evolutionary Computation [START_REF] Abbass | Pde: a pareto-frontier differential evolution approach for multiobjective optimization problems[END_REF]. In the end of the 20 th century, Evolutionary Algorithms (EAs) were found useful for solving MOPs/MaOPs [START_REF] Zitzler | Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach[END_REF] these algorithms are nowadays known as multi-objective evolutionary algorithms (MOEAs). The new method proposed in this paper is based on the principles of multi-objective evolutionary algorithms to solve MOPs/MaOPs.

Multi-objective Approaches Applied to Medical Feature Selection

In the field of medicine, feature selection is a common challenge frequently encountered in real-case applications [START_REF] Mezdad | Identification of new factors associated to walking impairment in patients with vascular-type claudication[END_REF]. For example, [START_REF] Chen | Selection of effective features for ECG beat recognition based on nonlinear correlations[END_REF] proposes a method to select the most effective and least redundant features from an ECG beat classification system.

More recently, [START_REF] Jimenez | Feature selection based multivariate time series forecasting: An application to antibiotic resistance outbreaks prediction[END_REF] proposes feature selection based multivariate time series forecasting aiming at generating precise models for antibiotic resistance outbreak prediction. Furthermore, [START_REF] Bania | R-hefs: Rough set based heterogeneous ensemble feature selection method for medical data classification[END_REF] proposes a heterogeneous ensemble feature selection method for medical data classification which aims at selecting the less redundant and most relevant features during the aggregation of different subsets of features.

The problem of obtaining the highest classification accuracy using the smallest number of features is encountered in several real-life applications such as in medicine. To solve the problem of finding the pareto subset, numerous methods reminded in [START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF] have been proposed: Particle Swarm Optimization (PSO) [START_REF] Yan | An efficient particle swarm optimization for large-scale hardware/software co-design system[END_REF], Improved Binary Particle Swarm Optimization (IBPSO) [START_REF] Chuang | Improved binary PSO for feature selection using gene expression data[END_REF], Artificial Bee Colony algorithms (ABC) [START_REF] Hancer | A binary abc algorithm based on advanced similarity scheme for feature selection[END_REF], Genetic Algorithms (GA) [START_REF] Huang | A GA-based feature selection and parameters optimizationfor support vector machines[END_REF], Bat algorithm [START_REF] Yong | A novel bat algorithm based on cross boundary learning and uniform explosion strategy[END_REF], Simulated Annealing (SA) [START_REF] Lin | A SA-based feature selection and parameter optimization approach for support vector machine[END_REF], Tabu Search (TS) [START_REF] Hou | An efficient gpu-based parallel tabu search algorithm for hardware/software co-design[END_REF], Ant Colony Optimization (ACO) [START_REF] Kashef | An advanced ACO algorithm for feature subset selection[END_REF], and Krill Head (KH) algorithms [START_REF] Abualigah | Hybrid clustering analysis using improved krill herd algorithm[END_REF].

Thus, Multi-objective approaches have been extensively studied in different fields of application. Some of them rely on Evolutionary Algorithms (EA) and are referred to as Multi-Objective Evolutionary Algorithms (MOEAs) and Many-Objective Evolutionary Algorithms (MaOEAs). These works have led to the emergence of numerous Multi-Objective Feature Selection (MOFSs) methods based on MOEAs (e.g., NSGA-II [START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF]) and in derivation Many-Objective Feature Selection (MaOFS) methods based on Many-Objective Evolutionary Algorithms (MaOEAs) (e.g., NSGA-III [START_REF] Li | An evolutionary many-objective optimization algorithm based on dominance and decomposition[END_REF]). To the best of our knowledge, among the most efficient methods, that either rely on Non-dominated Sorting Genetic Algorithm II (NSGA-II) [START_REF] Deb | A fast and elitist multiobjective genetic algorithm: NSGA-II[END_REF] or on Borg-MOEA [START_REF] Hadka | Borg: An auto-adaptive many-objective evolutionary computing framework[END_REF], are W-QEISS,F-QEISS, and W-MOSS [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF]. Those three methods were very recently outperformed in most datasets by two MOEAs methods: W-DMEA-FS and F-DMEA-FS [START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF].

Moreover, QEISS-like and DMEA-FS-like methods have been compared to another popular and older methods: mRMR [START_REF] Peng | Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy[END_REF], and with more traditional methods (e.g., based on PSO). Since, the four competitive methods presented in [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF][START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF] show better results both in terms of measured accuracy and number of selected features, in this paper

GAwC is compared to these methods.

Thus, to explain those competitive approaches more precisely, QEISS methods (Quasi Equally Informative Subset Selection) are built on the formulation of a four-objective optimization problem, aiming: at maximizing the accuracy of a classifier, minimizing the number of features, and optimizing two entropy-based measures of relevance and redundancy [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF]. Likewise, the DMEA-FS (Dividing-based Many-objectives Evolutionary Algorithm for Large-scale Feature Selection) method, [START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF] establishes four reasonable objectives, including the number of selected features, accuracy, intra-class distance and inner-class distance, for exploring the optimal feature subsets.

Efficient approaches such as DMEA-FS-like methods [START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF] or QEISS-like methods [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF] are applied to various fields of application and types of classifications (either bi-class or multi-class cases). They have been experimented on two binary classification medical datasets (diabetes dataset and the statlog heart dataset) available in the UCI Machine Learning repository.

However, those approaches are not medicine-centered and especially they do not consider any relevant metrics that should be taken into account by medical applications i.e., specificity and sensitivity. Yet, those criteria are important to consider in the field of medicine in order to measure and ensure: 1) the ability to correctly detect ill patients;

2) the ability to correctly reject healthy patients. Since they consider a combination of specificity and sensitivity, ROC curves are typically used to evaluate the quality of a classifier. A ROC curve is a graphical representation used to evaluate the quality of a binary classifier. It is a plot of the true positive rate (sensitivity) against the false positive rate (1-specificity) at various threshold settings [START_REF] Fawcett | An introduction to ROC analysis[END_REF]. AUC, gives an indication of how much the model is able to distinguishing between classes.

In addition to accuracy, AUC summarizes all four metrics of the confusion matrix: tp (true positives), tn (true negatives), f p (false positives), and f n (false negatives). Hence, the higher the value of AUC the better the classifier in terms of quality. Sample ROC curves and their associated AUC, from a perfect classifier (AUC = 1) to a random choice classifier (AUC = 0.5) are given in Figure 1 and Table 1 gives an interpretation of AUC values. Values of AUC below 0.5 are not considered here. When faced with classifiers with AUC < 0.5 the inverse classifier with an AUC > 0.5 should be considered. Based on this idea, the novel approach for multi-objective medical features selection proposed here considers three objectives: number of selected features, accuracy, and AUC. By attempting to maximize AUC, we believe that our proposal could find a Pareto optimal solution that other methods do not find.

The next sub-section reminds the foundations the proposed GAwC method is built upon.

Background

As specified hereafter, the GAwC method uses a hybrid algorithm comprising a genetic algorithm and a 3-Dimensional Compass method.

Genetic algorithms (GAs) [START_REF] Holland | Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence[END_REF] are one of the most popular techniques among evolutionary algorithms [START_REF] Katoch | A review on genetic algorithm: past, present, and future[END_REF]. They deal with the evolution of a population of artificial individuals and operate in a manner similar to Darwinian natural selection. GAs can achieve a rapid global search in large search spaces [START_REF] Goldberg | Genetic Algorithms in Search, Optimization, and Machine Learning[END_REF]. As mentioned in [START_REF] Ghaheri | The applications of genetic algorithms in medicine[END_REF], GAs are used in many medical specialties including radiology, pulmonology, radiotherapy, oncology, pediatrics, cardiology, neurology, endocrinology, surgery, obstetrics, gynecology, infectious diseases, rehabilitation medicine, pharmacotherapy, orthopedics, and health care management.

The second part of the hybrid algorithm is a 3-Dimensional Compass based fitness calculation relying on the basics of its original 2-Dimensional version proposed by [START_REF] Maturana | A compass to guide genetic algorithms[END_REF] and experimented with the Gorthaur method in [START_REF] Gutowski | Gorthaur: A portfolio approach for dynamic selection of multi-armed bandit algorithms for recommendation[END_REF][START_REF] Gutowski | Gorthaur-exp3: Bandit-based selection from a portfolio of recommendation algorithms balancing the accuracy-diversity dilemma[END_REF] in the context of recommendation systems. This method is detailed in the next section.

Methods

In this section the original genetic algorithm underlying the GAwC approach, the baseline considered for comparison, is first reminded. The different criteria considered by our objective functions are presented and finally, our main contribution: the Genetic Algorithm with multi-objective Compass (GAwC) is presented.

Original Genetic Algorithm

Below is a summary of the GA used in our experiments :

Algorithm: General principle of the genetic algorithm Input:

n: Number of individuals.
individualSize: Size of each individual.

nbGens: Number of generations (iterations).

mutationRate: Probability to mutate a bit of an individual.

Output:

Best individual of the population. Selection by tournament.

6:

Crossover to build the offspring generation: A two-points operator is used.

7:

Evaluate each child, according to the fitness function.

8:

Merge parents (n/2 individuals) and children (n/2 individuals) to re obtain a population of n individuals.

9:

Mutate each bit of the population with a probability equal to mutationRate (the bits of the best individual are spared from the mutation).

10:

Evaluate each mutated individual, according to the fitness function.

11:

Save in xMaxGen the fittest individual in the current generation's population. Precision regarding the specific configuration details of the above algorithm are given below :

• Input: n, the number of individuals in each generation is set to 200, the size of an individual individualSize depends on the problem, the number of generations, nbGens, is set to 100, finally, mutationRate is set to 0.10.

• Output: After the complete run, the algorithm returns the individual with the best fitness encountered throughout all generations.

• Initialization: Each individual in the first generation, a vector of size individualSize, is generated randomly.

• Selection by tournament: the population is randomly grouped in pairs. The fitness of both parents in each pair are compared and only the one with the best fitness is kept.

• Crossover: the population is randomly grouped in pairs. A crossover operation is performed between both parents of each pair to obtain the child generation. A twopoint crossover is used: two crossover points are picked randomly in the 2 parents.

Bits are swapped between the crossover points to obtain the 2 new children.

Multi-objective problem setting

Selecting a low number of features from a dataset while maintaining high levels of accuracy and AUC for a given classifier, can quite naturally be set as a multi-objective problem. In what follows we formally define the objective functions of our problem.

Objective Functions

Let V be the set of candidate features (variables) and S a subset of V. The cardinality |S| is the number of selected features in V and represents the first objective to be minimized. It is defined as

f 1 (S) = |S| (4)
The second objective to be maximized is the learning algorithm's binary classification performance as quantified by its accuracy. It is defined as

f 2 (S) = tp + tn tp + tn + f p + f n (5)
where tp, tn, f p, f n stand for true positives, true negatives, false positives, and false negatives respectively.

Finally, the last objective is AUC. We argue such criterion is the most important to consider in the field of medicine since (as Matthew's Coefficient Correlation) it represents one of the best metric for observing classification quality [START_REF] Chicco | The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation[END_REF].

This last objective to be maximized can be more precisely defined as:

f 3 (S) = 1 0 ROC(x) dx (6)
where ROC(x) designs the ROC curve. In the experiments presented in this article AUC is computed by the Scikit-learn Python library1 using the Trapezoidal rule of numerical integration [START_REF] Fawcett | An introduction to ROC analysis[END_REF].

GAwC: Genetic Algorithm with multi-objective Compass

This section presents our contribution: GAwC, the evolutionary multi-objective feature selection method. This hybrid method combines a GA and a Compass based multiobjective fitness function. It is used in this work to select a minimal subset of relevant features while maintaining a high classification quality. First, the 3D Compass method is presented and the way it is used to compute the multi-criteria score of a candidate solution is described. The computation of scores from the objective functions and their use by the Compass method are then explained. Finally, the GAwC algorithm is presented and it is shown how the multi-criteria score of each candidate solution can be taken into account by a GA to solve the multi-objective problem. Note that a Python implementation of the GAwC algorithm is shared in the following github repository:

https://github.com/ngutowski/gawc.

The 3-Dimensional Compass

The 3-Dimensional Compass is inspired by the 2-Dimensional Compass [START_REF] Maturana | A compass to guide genetic algorithms[END_REF] and the Gorthaur [START_REF] Gutowski | Gorthaur: A portfolio approach for dynamic selection of multi-armed bandit algorithms for recommendation[END_REF][START_REF] Gutowski | Gorthaur-exp3: Bandit-based selection from a portfolio of recommendation algorithms balancing the accuracy-diversity dilemma[END_REF] methods. It is aimed at aggregating the results of 3 score functions (sc 1 , sc 2 , sc 3 ) with values in [0, 1] in a single multi-criteria score.

The multi-criteria score S is obtained by projecting the score vector o = (sc 1 , sc 2 , sc 3 ) on a reference vector c representing a target trade-off between the 3 scores (see figure 2).

We have :

Sc = || o|| cos α with cos α = o. c
|| o||×|| c|| and thus :

S = o. c || c||
As detailed in [START_REF] Gutowski | Gorthaur-exp3: Bandit-based selection from a portfolio of recommendation algorithms balancing the accuracy-diversity dilemma[END_REF], such an aggregated score can be used to evaluate the candidate solutions of a multi-objective optimization problem with respect to a given trade-off.

Prior to using them in a 3D Compass, objective functions of the optimization problem should be scaled so as to be uniformly distributed between 0 and 1.

Figure 2: Projection of a score vector on the reference vector with the 3D Compass.

Objective functions and relative scores

The multi-objective method GAwC is designed for the optimization of 3 parameters:

Here, the number of features, the accuracy and the AUC. More precisely, it consists in the maximization of all parameters that it is provided with. If accuracy and AUC have to be maximized, the number of features (equation 4), on the contrary, should be minimized. Therefore the minimization of the number of features must be transformed in a maximization problem. The achievement of this transformation could be done by using the inverse of equation 4 for example. However, to better take into account wide data sets, we have added a factor ρ (ρ will be explained in the next paragraph). Thus, the "number of features score" Sc Feat described in Equation 7 will be used as a scoring function for the number of features and will be maximized.

Sc Feat = 1 - ρ f 1 (S) (7)
The factor ρ to be applied to high dimensional datasets aims at more rapidly converging on a solution with a low number of features. For example, in [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF] the minimum number of features is given an upper bound of 20. Thus, herein ρ can be considered both as a way to set an upper bound for the minimum number of features to select and as a sort of learning rate ensuring that convergence on an acceptable optimal solution does not require too many generations.

The scoring function used for the maximization of objective function f 2 (S) (See Equation 5) during the search process is the "accuracy score" Sc Acc . Since accuracy ranges from 0 to 1 and since it should be maximized, it can directly be used as a score function.

Hence,

Sc Acc = f 2 (S) (8)
The AUC of a random guess classifier is 0.5. It will thus be considered that AUC ranges from 0.5 to 1. Therefore a relevant score for AUC is a normalized value of f 3 (S)

on a scale from 0 to 1. We refer to this score as the "score of AUC" (Sc AUC ) in order to differentiate it from the raw AUC.

The "score of AUC" (Sc AUC ) can be defined by:

Sc AUC = ( f 3 (S) -0.5) × 2 (9)

The GAwC Algorithm

The GAwC algorithm is an implementation of the GA inspired by [START_REF] Holland | Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence[END_REF] and [START_REF] Mezdad | Identification of new factors associated to walking impairment in patients with vascular-type claudication[END_REF] and presented in section 3.1. In this work it is used to find a Pareto optimal solution to the 3-objective problem consisting in selecting a minimal subset of features to be fed to a classifier while maximizing both, the accuracy and the AUC-ROC score of the classifier.

As presented in section 3.3.1 the search of the solution in the search space is directed by a reference vector that indicates a trade-off between all three objectives with which the optimal solution should align.

Individuals of the GAwC algorithm are represented by a binary vector of size equal to the total number of features (for example 48 for the ASA-DI dataset). The bits in the vector indicate the presence or absence of each feature in the corresponding individual.

The fitness of an individual is computed using the 3-Dimensional Compass method presented in section 3.3.1.

More precisely, for each generation g i ; i ∈ [1, nbGens] generated by GAwC and each individual x j,g i ; j ∈ [1, n] in the generation's population, the algorithm computes the individual's accuracy score Sc Acc (x j,g i ) (See Equation 8), its number of features score ScFeat(x j,g i ) (See Equation 7) and its score of AUC Sc AUC (x j,g i ) (See Equation 9) when using the features represented by the individual. The ability of the individual to be altogether, accurate, with a low number of features, and a good classification quality (AUC) is then expressed by score vector --→ o x j,g i such that:

--→ o x j,g i = (Sc Feat (x j,g i ), Sc Acc (x j,g i ), Sc AUC (x j,g i ))

As presented in section 3.3.1, the individual's fitness is obtained by projecting its score vector --→ o x j,g i onto the predefined reference vector c and can be calculated as follows:

f (x j,g i ) = | --→ o x j,g i | cos α x j,g i (10)
where, as presented in figure 2, α j,g i is the angle between reference vector c and the individual's score vector --→ o x j,g i .

At each generation g i generated by GAwC, the algorithm determines x max (g i ), the fittest individual in the population of generation g i , such that:

x max (g i ) = argmax j∈[1,n] f (x j,g i )
Finally, if necessary, x max (g i ) is used to update, x max , the fittest individual encountered throughout all generations:

x max = max i∈[1,nbGens]
x max (g i )

This way, once all nbGens generations have been generated by the GA, x max is the fittest individual encountered throughout all generations.

Figure 3 depicts the evolution of the score vector of x max over generations i, i + 50 and i + 100 and shows the fitness of the corresponding individual obtained by projecting the score vector onto reference vector c.

For the purpose of this work and according to the protocol followed by [START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF], the GA's number of generations nbGens, is set to 100. according to the fitness f g i computed according to Equation 10.

Empirical Evaluation

This section, describes the datasets used for experimentation (See Table 2) and the algorithms that are compared. After detailing the experimental protocol and settings, the preliminary results obtained using the baseline are presented and analyzed. 2.

More precisely, the evaluation of the GAwC method, presented in this article is based on six real-world datasets in the field of Life Sciences and Medicine:

• Breast Cancer2 is a well-known dataset of 30 continuous features that has been often used in the field of artificial intelligence in medicine [START_REF] Sridevi | A novel feature selection method for effective breast cancer diagnosis and prognosis[END_REF][START_REF] Bhardwaj | Breast cancer diagnosis using simultaneous feature selection and classification: A genetic programming approach[END_REF][START_REF] Antunes | Feature Selection Optimization for Breast Cancer Diagnosis[END_REF]. Its first usage appears in [START_REF] Street | Nuclear feature extraction for breast tumor diagnosis[END_REF]. The dataset is used to predict whether a given patient has a malignant or a benign tumor. It should be noticed that the features in this dataset are computed from digitized images of a fine needle aspirate of a breast mass. They describe the characteristics of the cell nuclei present in the images.

• Cardiotocography3 is a fetal cardiotocography dataset. The 21 continuous features are used to predict if a cardiotocogram belongs to the "normal" category or to the "suspicious or pathologic" category. The dataset consists of measurements of fetal heart rate (FHR) and uterine contraction (UC) features on cardiotocograms classified by expert obstetricians. Its first usage appears in [START_REF] Ayres-De Campos | Sisporto 2.0: A program for automated analysis of cardiotocograms[END_REF]. Though this dataset is not used as frequently as the Wisconsin breast cancer dataset, a recent use of if can be quoted for the classification of anticipation of fetal risks using bagging ensemble classifiers [START_REF] Subasi | Classification of the cardiotocogram data for anticipation of fetal risks using bagging ensemble classifier[END_REF].

• Diabetes 4 is a dataset from the National Institute of Diabetes and Digestive and Kidney Diseases. Its first usage appears in [START_REF] Smith | Using the adap learning algorithm to forecast the onset of diabetes mellitus[END_REF] and for recent feature selection methods in [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF][START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF] which are the main competitive methods we compare to. Relying on the 8 categorical features of diagnostic measurements, the goal is to predict whether a patient has diabetes or not;

• Heart (Statlog) 5 is a well-known dataset of 13 features (7 categorical, 6 continuous) that has extensively been used in the field of Artificial Intelligence [START_REF] Kononenko | Overcoming the myopia of inductive learning algorithms with relieff[END_REF][START_REF] Brown | Diversity in neural network ensembles[END_REF][START_REF] Smirnov | Unanimous voting using support vector machines[END_REF] and for feature selection [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF][START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF]. The goal here is to predict whether a patient has a heart disease or not;

• Musk16 is a dataset from the AI Group at Arris Pharmaceutical Corporation.

Musk1 is used to scale-up the experiment in terms of number of features (166 continuous features). This dataset, where the goal is to predict whether a molecule will be judged to be a musk or not by a human expert, was originally used by [START_REF] Dietterich | Solving the multiple instance problem with axis-parallel rectangles[END_REF] and also employed for features selection method evaluation [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF][START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF]. Due to the number of features it provides, it represents a high dimensional case that should be experimented in the field of features selection;

• ASA (acetylsalicylic acid) Drug Intake (ASA-DI) is a new dataset [START_REF] Ramondou | Current-induced vasodilation specifically detects, and correlates with the time since, last aspirin intake: An interventional study of 830 patients[END_REF] that was collected by the University Hospital of Angers (France) and provided to us for experimentation following a real need: decreasing the number of features to be considered for medical decision while keeping both good accuracy and AUC. Based on 48 (30 categorical, 18 continuous) diagnostic measurements (features), ASA-DI corresponds to a dataset of patients in Cardiology from which the objective is to predict whether a patient has taken ASA or not. This dataset represents thus our direct real-world case on which selected features will be analyzed and validated by an expert as a clinical assessment. Note that ASA-DI has first been used and provided by [START_REF] Ramondou | Current-induced vasodilation specifically detects, and correlates with the time since, last aspirin intake: An interventional study of 830 patients[END_REF]

Competitive Methods

In order to evaluate the performance of the GAwC method, it is first compared with the baseline reminded in Section 3.1. Then, it is further compared with some of the most efficient, popular and related state-of-the-art methods reminded in Section 2.2 and listed below:

1. Baseline (original method without compass) [START_REF] Holland | Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence[END_REF][START_REF] Mezdad | Identification of new factors associated to walking impairment in patients with vascular-type claudication[END_REF] (see Section 3.1);

2. W-DMEA-FS, F-DMEA-FS [START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF] (see Section 2.2);

3. W-QEISS, F-QEISS, and W-MOSS [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF] (see Section 2.2).

Experimental Protocol

Simulation protocol. The protocol proposed by [START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF] is adopted for the comparison of GAwC with the different competitive methods. Hence, 10 repeated runs of 10 crossvalidations are computed to mitigate accuracy deviations. Then, the best and worst results in terms of accuracy and number of features are highlighted. Finally, the average of both metrics and their rank compared to that of the other methods are calculated. The same protocol is applied for the six datasets.

Datasets pre-processing. Due to the imbalance in the distribution of the target classes, a stratified shuffle sampling is used. The Scikit-learn library 7 provides such a package to ensure that relative class frequencies are approximately preserved in each train and validation fold. The GAwC implementation can be set to use a new seed for each run and to save all seeds. To guarantee the reproductibility of results, another setting allows the tool to use a given given set of predetermined seeds for the runs.

Technical settings. For all experiments, software methods relying on machine learning have been implemented using the Scikit-learn library and executed on a high performance computing cluster [START_REF]High performance computing cluster of leria, slurm/debian cluster of 27 nodes[END_REF] of 27 nodes (composed of 700 logical CPU, 2 nvidia GPU tesla k20m, 1 nvidia P100 GPU, 120 TB of beegfs scratch storage).

However, both the Genetic Algorithm and the 3-Dimensional Compass used by GAwC, are personal implementations.

Machine Learning method used for comparison. To guarantee a fair comparison of

GAwC with the algorithms presented in [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF][START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF], the same classifier -i.e.; Extreme Learning Machine, (ELM) [START_REF] Huang | Extreme learning machine: theory and applications[END_REF] is used by GAwC. Moreover, it is set with identical parameters following the guidelines given by [START_REF] Huang | Extreme learning machine: theory and applications[END_REF][START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF], i.e., a number of hidden layer of 10 and a sigmoid function f (x) = 1 (1+e -x ) as activation function in the hidden layer.

GAwC special settings. In this paper, the goal is to compare the multi-objective medical feature selection method GAwC with other competitive methods to observe whether it obtains better results in terms of accuracy and number of features. Thus, the motivation for the algorithm's setting is not "medical-driven" but "competition-driven". For this, the Compass is first set to be as competitive as possible in order for it to search for solutions in a relevant search space. This will allow our method to obtain similar results to those obtained by the five competitive feature selection algorithms [START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF]. Therefore, we set the Compass score objectives (the reference vector) as follows:

• AUC score objective is constant and set to its maximum value (i.e.: 1);

• The desired tradeoff between accuracy and number of features scores is given as the 2D Compass angle θ between the projection of the reference vector on the Sc Feat Sc Acc -plane and the Sc Feat axis (see Figure 3). To help choose this parameter, the tool can be set to perform a coarse exploration of the results, for a single run, for ranges of values of θ. In the present experimental case, a high accuracy is a priority and should not be sacrificed for the sake of a lower number of features. Acceptable solutions should thus be found for values of 45°≤ θ ≤ 90°, more likely on the upper part of the range. Depending on the dataset, experiments exhibit the best results for specific values of θ ranging from 80°to 86°. For each dataset, the value of θ selected by the coarse exploration is used as a starting Compass angle for a finer exploration computing 10 runs with three angles: θ, θ + 1°and θ + 2°.

Finally, as explained in Equation 7, the computed number of features score should be scaled in order to bound subset cardinality and define a maximum number of selected features; as in [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF] where the maximum is limited to 20. For this, constant ρ presented in Equation 7is set as follows:

• for Breast Cancer dataset, ρ = 4;

• for Cardiotocography dataset, ρ = 2;

• for Diabetes and Heart datasets, ρ = 1;

• for Musk1 dataset, ρ = 16;

• for ASA-DI dataset, ρ = 6.

Results

This subsection presents the results obtained by GAwC using ELM for all simulations, compared with the baseline [START_REF] Holland | Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence[END_REF] and with the following competitive methods:

W-DMEA-FS, F-DMEA-FS [START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF], W-QEISS, F-QEISS, and W-MOSS [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF].

Preliminary results for the baseline are presented in Table 3, an example of the best individual over generations is depicted in Figure 4 and the performances of GAwC and of all the competitive methods on all datasets are summarised in Table 4. Note that, Tables 3 and4 use the same presentation and use the same criteria as [START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF] in order to ease comparison with the original work. Thus, the rows correspond to 5 UCI datasets plus the ASA-DI dataset and the columns to the experimented methods (i.e., in Table 3:

Baseline [START_REF] Holland | Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence[END_REF]; in Table 4: GAwC and the results of the five competitive feature selection algorithms [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF][START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF]). The results are presented as: accuracy (as a percentage -%) followed by the number of selected features, presented between parentheses. 10 repeated tests are implemented for each methods for fair comparison with [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF][START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF] (See protocol description in Section 4.1.3), and both the best and worst results obtained in terms of accuracy and number of features along with their average over the 10 runs are reported. Each method is ranked according to its average results for each dataset. The rank of each method on both metrics (Accuracy Rank:NumberOfFeatures Rank) is reported in Table 4 for each dataset.

Finally, the results obtained by GAwC are presented graphically in Figure 5. They show the two part solutions obtained by GAwC for the six different datasets and the best solutions found by the competitive methods. The black points are GAwC, W-DMEA-FS is represented by a red cross marker, F-DMEA-FS by an orange cross marker, W-QEISS by a blue cross marker, F-QEISS by a cyan cross marker, and W-MOSS by a green cross marker.

Baseline Results

Before experimenting the new GAwC method, the results obtained by the original method [START_REF] Holland | Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence[END_REF] without Compass are first measured (See Table 3). On the six datasets it underperforms the competitive methods' results published in [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF][START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF] (See Table 4), either with regard to both number of selected features and accuracy, or solely for the Thus, even though it obtains similar results on experiments with a low dimensional dataset (Diabetes) in comparison with the competitive methods, it does not suitably operate in mid or high dimensional cases.

Hence, those previous considerations support the idea that the total number of fea-tures needs to be considered as part of the objective functions of the baseline method.

Generations

Before comparing the results obtained by GAwC in terms of accuracy and number of features with those of the competitive methods and, in order to better understand the method, the results obtained by the Compass over generations are presented in Fig- (Figure 4c) and number of features (Figure 4d).

Hence, in Figure 4, it can be observed that the projection for generation g 88 on the reference vector is higher than it is for other generations. This directly expresses the methods ability to best fit to the desired trade-off between accuracy, number of selected features, and AUC. Hence, since the goal of GAwC is to find the best individual (a set of selected features) encountered throughout all generated populations according to the chosen {Accuracy, Number of features, AUC} trade-off, the results presented in Table 4 are those obtained after the last population has been generated by GAwC. Note that GAwC iterates over 100 generations to allow for fair comparison with the competitive methods [START_REF] Karakaya | Identifying (quasi) equally informative subsets in feature selection problems for classification: a max-relevance min-redundancy approach[END_REF][START_REF] Li | A dividing-based many-objective evolutionary algorithm for large-scale feature selection[END_REF] that also use this same setting.

Results Obtained by GAwCs and Competitive Methods: Global Analysis

The results obtained by GAwC and the competitive methods are presented in Observation of the two objectives in solutions. Figure 5 represents the number of features selected by GAwC (black points) along with each classification accuracy. Those are compared to the competitive methods' best results (colored crosses). Figure 5 shows that for all datasets and for all selected number of features, the GAwC algorithm obtains a higher accuracy than that obtained by the competitive methods. This figure is an interesting complement to Table 4 since it shows the values of both objectives in the solutions computed by GAwC in the search space defined by the reference vector (See Figure 3 or Figure 4 for illustration of the Compass).

Discussion

In this section we discuss our results; first by explaining why, according to our assumptions the method outperforms the competitive methods, second by precising why and how the compass can respond to different Medical practitioner's requirements.

Then, with the help of a cardiologist's analysis, we observe, discuss and validate the medical features GAwC selected.

Comparison with Competitive Methods

In Section 4.2.3 we described the results obtained by observing Table 4 and Figure 5.

GAwC obtains the best results both in terms of Accuracy and number of selected features.

These results can be explained by the fact that the three-objective search space includes a criteria reflecting the quality of prediction: AUC (with the target AUC objective set to its maximum and Accuracy expectation greater than number of variables selection, see Section 4.1.3). Thus, aiming at, together, maximizing accuracy and AUC, and minimizing the number of selected features, guarantees high accuracy level and classification quality while selecting the satisfactory subset of features. Note that, depending on the Medical practitioner's requirements, the Compass angle can be set to favor a low number of features rather and be less demanding with the maximization of Accuracy and AUC. This can be the case when searching for the smallest number of features to consider yet accepting a low level of either accuracy or AUC to reach. In this case, the two objectives of the solutions may vary and propose solutions with both lower number of features and accuracy (up to a desired threshold). anti-TNFa agents or other immunosuppressors, antihistamines, allopurinol, betablockers, betamimetics, and ASA. For ASA, the dose regimen and the time since the last intake were recorded. The presence of diabetes mellitus was assessed by a prescription (intake or administration) for antidiabetic drugs.

Some Clinical

Moreover, as part of microvascular investigations, the TIssue Viability Imaging (TiVi) features were collected and analyzed from patients who were placed in a semirecumbent position in a quiet air-conditioned room (22 °C ±3) with moderate lighting and no direct sunlight. Skin blood flow was thus studied on the volar aspect of the forearm using a TiVi700 imager (WheelsBridge AB, Link öping, Sweden).

Variables validation. The experiment with GAwC using ELM with the ASA-DI database, conduced to retain the following features:

• Age;

• Cholesterol lowering drug;

• Anticoagulants;

• Non-ASA Antiplatelet AGents (AAG);

• TIssue Viability Imaging (TiVi) which is also directly correlated to CIV and ASA intake [START_REF] Ramondou | Current-induced vasodilation specifically detects, and correlates with the time since, last aspirin intake: An interventional study of 830 patients[END_REF].

From a cardiologist's point of view: Current-induced vasodilation (CIV) has been proposed as a way of detecting aspirin intake without the need of blood sampling [START_REF] Ramondou | Current-induced vasodilation specifically detects, and correlates with the time since, last aspirin intake: An interventional study of 830 patients[END_REF].

Predicting aspirin intake by a bedside test could be of particular interest when it comes to define adherence to treatment. Nevertheless, which of the co-variate are associated to aspirin intake among which clinical factors could be important to determine whether adjustment should be performed when analyzing CIV response. Among possible factors are those that are the causes of ASA treatment or are associated with an increased risk of arterial disease (such as obesity, age, etc..), others are treatments that are generally associated to ASA treatment in recommendations (Cholesterol lowering drugs, non-ASA antiplatelet agents, sartans).

Thus, the features selected by GAwC are considered as fully relevant and valid.

Evaluation metrics. The five features subset GAwC has selected as optimal solution (See Table 4 and detailed results in Table 5) obtains an accuracy of 78.34% which outperforms all competitive methods. Nevertheless, in this paragraph, we deepen our evaluation on several other metrics identified as more relevant in the field of medicine i.e., Specificity, Sensitivity, MCC and AUC. Thus, we observe that the optimal solution found by GAwC obtains a higher AUC with the ASA-DI Dataset than the original methods used in [START_REF] Ramondou | Current-induced vasodilation specifically detects, and correlates with the time since, last aspirin intake: An interventional study of 830 patients[END_REF] i.e., GAwc obtains an AUC of 0.74 (i.e., Excellent discrimination, see Table 1)

versus an AUC of 0.679 (i.e., Acceptable discrimination, see Table 1) in [START_REF] Ramondou | Current-induced vasodilation specifically detects, and correlates with the time since, last aspirin intake: An interventional study of 830 patients[END_REF] with linear models (the authors did not compute accuracy that was not relevant considering the objective of their paper).

Conclusion and Perspectives

This study has taken a decisive step in the direction of feature selection methods. In the area of binary classification problems, we have developed a novel hybrid approach for multi-objective feature selection called Genetic Algorithm with multi-objective Compass (GAwC). Namely, this method is a GA hybridized with a 3-Dimensional Compass method. According to a user defined trade-off, the method aims at :

1. minimizing the number of features to select, 2. maximizing the accuracy the classifier obtains, 3. maximizing the classifier's AUC.

The performances of this approach overtake recent competitive methods on 5 wellkown datasets (Breast Cancer, Cardiotocography, Diabetes, Heart, and Musk1 from UCI ML Repository). Moreover, these performances are very interesting from a practitioner's point of view on a new medical dataset (ASA (acetylsalicylic acid) Drug Intake).

Furthermore, these performances could reduce cognitive load upon the practitioners while decreasing the economical cost of variables collection.

Further research in this field could include three directions depicted as follows. The first perspective would be the validation of this method on other new datasets. Secondly, the extension of compass to 4D or 5D could be very interesting. For example, the addition of AUK (Area under the Kappa curve) [START_REF] Kaymak | The AUK: A simple alternative to the AUC[END_REF] could be promising : this score is particularly suitable for measuring classifiers' performance on skewed datasets; note that the mathematical relationship between AUC and AUK is non linear. The addition of MCC (Matthew's Coefficient Correlation) would conduce to a 5D compass that may have an interesting potential in the future. Finally, the last perspective deals with the replacement of the evolutionary part of the method by other methods such as Particle Swarm Optimization [START_REF] Chuang | Improved binary PSO for feature selection using gene expression data[END_REF], Ant Colony Optimization [START_REF] Kashef | An advanced ACO algorithm for feature subset selection[END_REF], or Artificial Bee Colony algorithm [START_REF] Hancer | A binary abc algorithm based on advanced similarity scheme for feature selection[END_REF],

1 : 2 : 3 :

 123 Initialization: Create a random population of n individuals. Evaluate each individual of the population according to the fitness function. Save in xMax the fittest individual in the population 4: for i ← 1 to nbGens do 5:

12 : 15 :

 1215 if xMax < xMaxGen then xMax = xMaxGen return xMax, the fittest individual encountered throughout all generations.

Figure 3 :

 3 Figure 3: Score vectors and fitnesses of the best individuals at generations g i , g i+50 and g i+100

  number of features criterion. With the Diabetes dataset, which is a low dimensional dataset, it obtains an equivalent average number of features to the best competitive method (4.0 versus 4.1 for W-DMEA-FS) but a lower accuracy (76.42 versus 77.44 for W-DMEA-FS). Nevertheless, when the baseline is applied to higher dimensional cases as the Breast Cancer, Cardiotocography, Heart, Musk1, or the ASA-DI datasets, it performs very poorly in terms of feature selection (21.8 versus 5.75 for W-DMEA-FS with the Breast Cancer dataset; 10.6 versus 5.69 for W-DMEA-FS with the Cardiotocography dataset; 8.0 versus 6.2 for W-DMEA-FS with the Heart dataset; 81.2 versus 7.5 for W-DMEA-FS with the Musk1 dataset; 13.8 versus 5.4 for W-DMEA-FS with the ASA-DI dataset).

ure 4 .

 4 They illustrate the evolution of the best individual encountered after 3 generations (g 1 ,g 30 , and g 88 ) on one specific run with the Musk1 dataset. Moreover, each sub-figure shows the results from a different point of view: accuracy (Figure4b), AUC

Figure 4 :

 4 Figure 4: Results obtained by the Compass for 3 enhanced generation from g 1 to g 88 (example from a run using ELM on Musk1 dataset) observable through four points of view).

Figure 5 :

 5 Figure 5: The two objectives of part solutions on six datasets by GAwC (ELM) method against the competitive methods.

Figure 1 :

 1 ROC curves and AUC values.

	AUC = 0.5	No discrimination, e.g., randomly flip a coin
	0.6 ≥ AUC > 0.5	Poor discrimination
	0.7 ≥ AUC > 0.6	Acceptable discrimination
	0.8 ≥ AUC > 0.7	Good discrimination
	0.9 ≥ AUC > 0.8	Excellent discrimination
	AUC > 0.9	Outstanding discrimination

Table 1 :

 1 Interpretation of AUC values[START_REF] Yang | The receiver operating characteristic (ROC) curve[END_REF].

Table 2 :

 2 a study conducted in accordance with the Declaration of Helsinki, approved by the local Ethics Committee, and registered in the clinicaltrial.gov database under reference NCT:03357367 before the first patient was included. Used Bi-Class Datasets.

	Dataset	Number of Number of	Dataset
	name	instances	features	source
	Breast Cancer	569	30	UCI ML Repository
	Cardiotocography	2126	21	UCI ML Repository
	Diabetes	768	8	UCI ML Repository and Kaggle
	Heart	270	13	UCI ML Repository
	Musk1	476	166	UCI ML Repository
	ASA-DI	822	48	University Hospital of Angers

Table 3 :

 3 Preliminary results of accuracy and number of features obtained by the baseline method[START_REF] Holland | Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence[END_REF].

Table 4

 4

	,

Table 4 :

 4 The two objectives of part solutions on six datasets by the novel GAwC approach against competitive methods.

	Analysis on best results. Table 4 shows that GAwC outperforms the competitive meth-
	ods for all datasets in terms of best accuracy (i.e., 98.2% versus 98.06% for W-DEMA-
	FS with the Breast Cancer dataset, 90.87% versus 90.51% for W-DEMA-FS with the

Table 5 ,

 5 also reports the sensitivity (88.59% ±5.84) and specificity (59.4% ±8.34) results along with an important metric to be considered: Matthew's Coefficient Correlation (MCC) (0.513). The MCC metric is a classification quality criteria complementary to AUC that is nowadays underused in the field of Machine Learning and that we argue should be further considered in future evaluations.

	Metrics	Results St. deviation
	Sensitivity 88.59% ±5.84
	Specificity 59.4%	±8.34
	Accuracy	78.34% ±4.56
	MCC	0.513	±0.101
	AUC	0.74	±0.009

Table 5 :

 5 Results obtained for the optimal solution found by GAwC on Sensitivity, Specificity, Accuracy, Matthew's Coefficient Correlation (MCC) and AUC in ASA-DI dataset (10 CV).

Python library http://scikit-learn.org/

Available at https://archive.ics.uci.edu/ml/datasets/breast+cancer+wisconsin+(diagnostic)

Available at https://archive.ics.uci.edu/ml/datasets/cardiotocography

Originally provided by UCI Machine Learning, it can be found now in Kaggle https://www.kaggle. com/mathchi/diabetes-data-set

Available at https://archive.ics.uci.edu/ml/datasets/statlog+(heart)

Available at https://archive.ics.uci.edu/ml/datasets/Musk+(Version+1)

Python library http://scikit-learn.org/