Nguyen-Khang Pham

Thanh-Nghi Do

Philippe Lenca

Stéphane Lallich

Using local node information in decision trees: coupling a local labeling rule with an off-centered entropy

Dealing with skewed class distribution and costsensitive data has been recognized as one of the 10 most challenging problems in data mining research. These problems have been reported to hinder the performance of classifiers, especially on the minority class. To deal with this problem in decision tree induction we proposed an off-centered entropy while other authors proposed an asymmetric entropy. Compared to Shannon's entropy both of them take their maximum value for a distribution fixed by the user instead of an uniform distribution. We here also propose to use in each leaf of the tree a local class labeling rule instead of the classical majority rule that mechanically favors the majority class i.e. the negative one.

In this paper we briefly present the concepts of the three entropies and the new class labeling rule. This allows us to propose an adaptive learning of decision trees. We then compare their effectiveness on 25 imbalanced data sets. All our experiments are founded on the C4.5 decision tree algorithm, in which only the function of entropy and class labeling rule are modified. The results are promising and show the interest of our proposal.

I. INTRODUCTION

Dealing with imbalanced and cost-sensitive data has recently been recognized as one of the 10 most challenging problems in data mining research [START_REF] Yang | 10 challenging problems in data mining research[END_REF]. In supervised learning, the data set is said imbalanced if the class prior probabilities are highly unequal. In the case of two-class problems (we will only consider binary classification in this paper), the larger class is called the majority class and the smaller the minority class. Such imbalanced data sets are thus an important problem from both real applications and research points of view.

Firstly, real-life two-class problems have often minority class prior under 0.10 (e.g. fraud detection, medical diagnostic or credit scoring). Secondly, in such a case the performances of data mining algorithms are lowered, especially the error rate corresponding to the minority class. The main reason underlying is that most of data mining algorithm try to maximize the accuracy. What is more, it is the minority class that the practitioner is interested in i.e. the minority class corresponds to positive cases and the cost of misclassifying the positive examples is higher than the cost of misclassifying the negative examples.

Nguyen-Khang Pham is with IRISA, Rennes, France (email: pnguyenk@irisa.fr).

Thanh-Nghi Do is with Cantho University, Vietnam (email: dtnghi@gmail.com).

Philippe Lenca is with the Institut TELECOM, TELECOM Bretagne, UMR 3192 LabSTICC, Brest, France (email: philippe.lenca@telecombretagne.eu).

Stéphane Lallich is with the University of Lyon, ERIC Laboratory, Lyon 2, France (email: stephane.lallich@univ-lyon2.fr).

This problem gave rise to many papers, from which one can cite papers from two workshops associated which AAAI and ICML conferences respectively [START_REF] Japkowicz | AAAI'Workshop on Learning from Imbalanced Data Sets[END_REF] and [START_REF] Chawla | ICML'Workshop on Learning from Imbalanced Data Sets[END_REF] and a special issue of SIGKDD [4]. As summarized by the review papers of [START_REF] Japkowicz | The class imbalance problem: Significance and strategies[END_REF], [START_REF] Japkowicz | The class imbalance problem: A systematic study[END_REF] and [START_REF] Visa | Issues in mining imbalanced data sets -A review paper[END_REF] or by the very comprehensive papers of [START_REF] Weiss | The effect of class distribution on classifier learning[END_REF] and [START_REF]Learning when training data are costly: The effect of class distribution on tree induction[END_REF], solutions to the class imbalance problems were proposed both at the data and algorithmic level.

At the data level, these solutions change the class distribution. They include different forms of sampling, such that over-sampling (which increases the number of minority class data points, [4], [START_REF] Liu | Generative oversampling for mining imbalanced datasets[END_REF]) or under-sampling (which decreases the number of majority class data points, [START_REF] Kubat | Addressing the curse of imbalanced data sets: One-sided sampling[END_REF]) on a random or a directed way. [START_REF] Liu | Exploratory under-sampling for class-imbalance learning[END_REF] proposed two ways of class imbalance learning that are designed to utilize the major class examples ignored by under-sampling. For a recent and comprehensive study of sampling one may refer to [START_REF] Van Hulse | Experimental perspectives on learning from imbalanced data[END_REF].

At the algorithmic level, one solution is to re-balance the error rate by weighting each type of error with the corresponding cost [START_REF] Domingos | Metacost: A general method for making classifiers cost sensitive[END_REF]. A study of the consistency of the costs re-balancing, for misclassification costs and class imbalance, is presented in [START_REF] Zhou | On multi-class cost-sensitive learning[END_REF]. For a comparison of cost sensitive approach and sampling approach one can see for example [START_REF] Liu | The influence of class imbalance on costsensitive learning: An empirical study[END_REF] and [START_REF] Weiss | Cost-sensitive learning vs. sampling: Which is best for handling unbalanced classes with unequal error costs?[END_REF].

We focus in this paper on decision tree induction, especially C4.5 [START_REF] Quinlan | C4.5: Programs for Machine Learning[END_REF]. In a simple way, one have thus to consider the sampling problem, the split function, the pruning scheme and the class labeling rule. A comparative study using C4.5 decision tree shown that under-sampling beat over-sampling [START_REF] Drummond | C4.5, class imbalance, and cost sensitivity: Why under-sampling beats over-sampling[END_REF]. [START_REF] Ling | Decision trees with minimal costs[END_REF] propose to use a criterion of minimal cost, while [START_REF] Du | Cost-sensitive decision trees with pre-pruning[END_REF] explore efficient pre-pruning strategies for the costsensitive decision tree algorithm to avoid overfitting. Some algorithmic solutions consist in adjusting the probabilistic estimates at the tree leaf or adjusting the decision thresholds [START_REF] Elkan | The foundations of cost-sensitive learning[END_REF]. [START_REF] Chawla | C4.5 and imbalanced datasets: Investigating the effect of sampling method, probalistic estimate, and decision tree structure[END_REF] studied the quality of probabilistic estimates, the pruning scheme and the effect of preprocessing the imbalanced data set concerning C4.5.

We first proposed a method to off-center whichever kind of entropy [START_REF] Lallich | Construction of an off-centered entropy for supervised learning[END_REF]. The interest of the off-centered entropy for decision tree is that it could take its maximal value for the a priori distribution of the class in any considered node. We then defined an adaptive learning strategy which replace the usual entropy used in tree induction algorithms by an offcentered entropy [START_REF] Lenca | A comparison of different off-centered entropies to deal with class imbalance for decision trees[END_REF]. Experiments with C4.5, using majority rule to label the leaves of the tree, led to promising results. These previous works are clearly at the algorithmic level as we only modified the split function used in decision tree induction.

At the algorithmic level an another challenge concerns the labeling rule used in each leaf, specially in case of imbalanced data sets. Indeed the majority rule used in C4.5 is clearly not adapted and will mostly predict the majority class. In this paper we extend our previous considerations of using local information in each leave of the tree. We here consider the labeling rule in each leaf and propose to use a nearest neighbors approach. What is more one should notice that, even if the data are not initially imbalanced, a decision tree may process imbalanced data in any node. The rest of the paper is organized as follows. In section II, we first review splitting criterion based on Shannon's entropies. We recall basic considerations on Shannon's entropy and then briefly present our off-centered entropy and the asymmetric entropy proposed by [START_REF] Marcellin | An asymmetric entropy measure for decision trees[END_REF]. In section III we present labeling strategies that could be applied in decision trees. Then, in section IV we compare the performances of different solutions based on the coupling of the three entropies used in this study and three labeling rules. These experiments are based on 25 imbalanced data sets and deliver interesting results. Finally, Section V draws conclusions and suggests future work.

II. FROM SHANNON'S ENTROPY TO NON-CENTERED ENTROPIES

We first recall basic considerations on Shannon's entropy and then present briefly two non-centered entropies in the boolean case and mention the results in the general case.

A. Usual measures based on Shannon's entropy

Many induction tree algorithms on categorical variables use predictive association measures based on the entropy of Shannon [START_REF] Shannon | A mathematical theory of communication[END_REF]. Let us consider a class variable Y having q modalities, p = (p 1 , . . . , p q ) being the vector of frequencies of Y , and a categorial predictor X having k modalities. The joint relative frequency of the couple (x i , y j ) is denoted p ij , i = 1, . . . k; j = 1, . . . q. What is more, we denote by h(Y ) = -q j=1 p .j log 2 p .j the a priori Shannon's entropy of Y and by h(Y /X) = E(h(Y /X = x i )) the conditional expectation of the entropy of Y with respect to X.

Shannon's entropy h(p), is a real positive function of p = (p 1 , . . . , p q ) to [0..1], verifying notably interesting properties for machine learning purposes [START_REF] Zighed | Graphes d'Induction -Apprentissage et Data Mining[END_REF]:

• h(p) is invariant by permutation of the modalities of Y ;

• h(p) reaches its maximum log 2 (q) when the distribution of Y is uniform (each modality of Y has a frequency of 1/q); • h(p) reaches its minimum 0 when the distribution of Y is sure (centered on one modality of Y and the others modalities being of null frequency); • h(p) is a strictly concave function. The behavior of Shannon's entropy is illustrated in Fig. 1 in the boolean case.

As example of measures based on Shannon's entropy, one can mention:

• the entropic gain h(Y ) -h(Y /X) [START_REF] Quinlan | Induction of decision trees[END_REF];

• the gain-ratio h(Y )-h(Y /X) h(X) [START_REF]C4.5: Programs for Machine Learning[END_REF] which relates the entropic gain of X to the entropy of X, rather than to the a priori entropy of Y in order to discard the predictors having many modalities.

For more measures and details one can refer to [START_REF] Wehenkel | On uncertainty measures used for decision tree induction[END_REF] and [START_REF] Loh | Split selection methods for classification trees[END_REF].

The particularity of these coefficients is that Shannon's entropy of a distribution reaches its maximum when this distribution is uniform. That is to say that the reference value corresponds to the uniform distribution of classes. This characteristic could be a major problem especially in case of highly imbalanced classes, or when the classification costs differ largely. It would seem more logical to evaluate h(Y ) and h(Y /X = x i ) used in the above measures on a scale for which the reference value is centered on the independence situation i.e. on the a priori distribution of classes.

B. Off-centered entropy

The construction principle of an off-centered entropy is sketched out in the case of a class variable Y made of q = 2 modalities in [START_REF] Lallich | Parametrised measures for the evaluation of association rule interestingness[END_REF] and [START_REF]A probabilistic framework towards the parameterization of association rule interestingness measures[END_REF]. The frequencies distribution of Y for the values 0 and 1 is noted (1 -p, p).

This off-centered entropy associated with (1 -p, p) and noted η θ (p) is maximal when p = θ, θ being fixed by the user and not necessarily equal to 0.5 (in the case of a uniform distribution). It is constructed with a simple transformation of the (1 -p, p) distribution into a (1 -π, π) distribution such that π increases from 0 to 1/2 when p increases from 0 to θ, and π increases from 1/2 to 1 when p increases from θ to 1. By looking for a linear expression of π as π = p-b a , on both intervals 0 ≤ p ≤ θ and θ ≤ p ≤ 1, we obtain:

• π = p 2θ if 0 ≤ p ≤ θ; • π = p+1-2θ 2(1-θ) if θ ≤ p ≤ 1.
The off-centered entropy η θ (p) is then defined as the entropy of (1 -π, π):

η θ (p) = -π log 2 π -(1 -π) log 2 (1 -π) .
Note that the thus transformed frequency depends of θ and should be noted as π θ . We simply use π for clarity reasons.

With respect to the distribution (1 -p, p), clearly η θ (p) is not an entropy strictly speaking. Its properties must be studied considering the fact that η θ (p) is the entropy of the transformed distribution (1 -π, π), i.e. η θ (p) = h(π) and thus possesses such characteristics. Obviously invariance by permutation of modalities of Y is not more true and η θ (p) is maximal for p = θ i.e. for π = 0.5. Proofs are given in detail in [START_REF]Construction d'une entropie décentrée pour l'apprentissage supervisé[END_REF]. The behavior of this entropy is illustrated in Fig. 1 for θ = 0.2.

Following a similar way as in the boolean case we extended the definition of the off-centered entropy to the case of a variable Y having q > 2 modalities and proposed a general decentring framework that can be applied to any measure of predictive association based on a gain of uncertainty [START_REF]Construction d'une entropie décentrée pour l'apprentissage supervisé[END_REF], [START_REF] Lallich | Construction of an off-centered entropy for supervised learning[END_REF]. This allows to define a set of off-centered generalized entropies.

C. Asymmetric entropy

Directly related to the construction of a predictive association measure, especially in the context of decision trees, [START_REF] Marcellin | An asymmetric entropy measure for decision trees[END_REF] proposed an asymmetric entropy for a boolean class variable. This measure is asymmetric in the sense that one may choose the distribution for which it will reach its maximum. They preserve the strict concavity property but alter the maximality one in order to let the entropy reach its maximal value for a distribution chosen by the user (i.e. maximal for p = θ, where θ is fixed by the user). This implies revoking the invariance by permutation of modalities. They thus proposed:

h θ (p) = p(1 -p) (1 -2θ)p + θ 2
It can be noticed that for θ = 0.5, this asymmetric entropy corresponds to the quadratic entropy of Gini. The behavior of this entropy is illustrated in Fig. 1 for θ = 0.2. The authors extended also this approach in particular to the situation where the class variable has q > 2 modalities [START_REF] Zighed | Mesure d'entropie asymétrique et consistante[END_REF].

× × × × × × × × × ×××× × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × Off-centered × × × × × × × × × ××××××× ×× ×× × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × × ×

III. THE CLASS LABELING RULE

The majority decision rule is by far the most commonly used labeling rule in tree induction. This can be explained by two reasons. Firstly, the majority rule exploits at best the intelligibility of decision trees. In particular, it is opposable; in the case of scoring, you can explain to a customer for which reason his credit demand is refused. Secondly, it is known that the majority rule is optimal in case of balanced classes and symmetrical costs of misclassification.

Despite its popularity, the majority rule is particularly ill-suited to the case of imbalanced or cost-sensitive data. Indeed, consider the example of a cancer whose a priori probability is 0.02 for the individuals of a population. If in a leaf, the likelihood of cancer increases to 0.32, it means that the risk of cancer is multiplied by 16 for individuals of the leaf. Nevertheless, the application of majority rule remains insensitive to such a change. For each case of the leaf, the majority rule still predicts the absence of cancer.

Among the different labeling rules, we distinguish between the global rules, the individualized rules and the aggregated rules. The global rules will assign the same class to all individual to be classified which belong to the same leaf. An individualized rule uses further information to select the class to be attributed to each individual of the considered leaf. An aggregated rule will consider a combination of rules issued from multiple classifiers. An individual rule is well suited to imbalanced data, but the intelligibility of the rules is lost, even if the intelligibility of the tree is preserved. In the case of aggregated rules, both the tree and the rules intelligibility are lost.

When using a global rule one choose the same class for all individuals from the same leaf on the basis of a rule which may be different from the majority rule. [START_REF] Ritschard | Données déséquilibrées, entropie décentrée et indice dimplication[END_REF] have proposed two different labeling rules. The first one uses an implication index where the chosen class is the one which maximizes the index of the rule "if the leaf is F then the class is C i ". The second one is the contribution to the entropy: the chosen modality is the one for which the contribution to the entropy of the leaf is the lowest among the modalities for which there are more cases than expected by chance. The main feature of this type of rule is to favor the minority class accuracy in the trade off between minority class and majority class.

Beyond the majority rule (denoted by MR in the tables), another rule is known, the proportional random rule, which is naturally associated with the normalized gain of quadratic entropy [START_REF] Goodman | Measures of association for cross classifications, i[END_REF]. The principle of this rule is to attribute the class at random, while respecting the distribution of the class in the considered leaf. This kind of rule is known for providing as good results as those offered by the majority rule, but it has the advantage of being less affected than the majority rule in case of imbalanced data.

To settle the class labeling problem, we propose an individualized rule (denoted by k-NN-rule). The class assigned to an individual is the majority class among its k-nearest neighbors in the corresponding leaf. The nearest neighbors are based on the variables that have not yet been used to construct the leaf. If the topology induced by these variables adds some information about the class, the proposed rule is a kind of proportional rule which chooses the class better than chance. Otherwise, the proposed rule is confused with the rule of proportional allocation at random. We here introduce a specific bias in favor of rare class. As pointed out by [START_REF] Weiss | Mining with rarity: A unifying framework[END_REF] if a general bias is good for common cases, it is not appropriate for rare cases and may even cause rare cases to be totally ignored. These problems were also previously discussed by [START_REF] Holte | Concept learning and the problem of small disjuncts[END_REF].

A final way to fight against the crushing effect of the majority rule is to generate a large diversity of situations by using a set of trees and to aggregate their results, i.e decision tree bagging [START_REF] Breiman | Bagging predictors[END_REF].

IV. EXPERIMENTS

We have empirically studied behaviors of decision tree algorithms using our proposed off-centered entropy [START_REF] Lallich | Construction of an off-centered entropy for supervised learning[END_REF] to classify imbalanced data sets. We are also interested in the comparison of performances with state-of-the-art entropies including Shannon's entropy [START_REF] Shannon | A mathematical theory of communication[END_REF] and asymmetric entropy [START_REF] Marcellin | An asymmetric entropy measure for decision trees[END_REF]. Due to the evaluation we have added the off-centered and asymmetric entropies to the publicly available decision tree algorithm, C4.5 [START_REF]C4.5: Programs for Machine Learning[END_REF].

The experimental setup used the twenty five data sets described in Table I including seventeen first ones from the UCI repository [START_REF] Blake | UCI repository of machine learning databases[END_REF], six next ones from the Statlog repository [START_REF] Michie | Machine Learning, Neural and Statistical Classification[END_REF], an another one from the DELVE repository (http://www.cs.toronto.edu/∼delve/) and a last one from [START_REF] Jinyan | Kent ridge bio-medical data set repository[END_REF]. In order to evaluate performances for classification tasks of imbalanced data sets, we pre-processed multi-class (more than two classes) data sets as two-class problems. The columns 5 and 6 in table I show how we convert multiclass to minority and majority classes, for example with the OpticDigits data set, the digit "0" is mapped to the minority class (Class min: 10%) and the remaining data are considered as the majority class (Class maj: 90%).

We will denote by (***) significant result at 1/1000 level, (**) at 1/100, (*) at 5/100 and ( 0 ) when results are not significant.

A. Comparison of the three entropies two by two, using majority rule

For this first comparison published in [START_REF] Lenca | A comparison of different off-centered entropies to deal with class imbalance for decision trees[END_REF], the majority rule is applied. The results of the different entropies are compared in terms of the length of the produced tree, of error rate on the minority class and the majority class, as well as the global error rate (tables II, III, IV). The main result of the carried out experiments is that our strategy of adaptive learning, using the non-centered entropies, particularly the off-centered entropy, outperform the Shannon entropy. These both entropies significantly improve Amin, the minority class accuracy, without penalizing Amaj, the majority class accuracy and TS, the tree size.

Indeed, facing the Shannon entropy (SE), the off-centered entropy (OCE) improves Amin 23 times out of 25, with 1 defeat and 1 tie, which corresponds to a p-value of 0.000. The corresponding average gain is 1.98 points of percent (***). Amaj is not significantly improved (0.58), but the global accuracy Acc is improved 21 times out of 25, against 3 decrease and 1 tie (***), while the average corresponding gain is equal to 0.76. Moreover, the trees provided by the offcentered entropy are often of a more reduced size, without this reduction being significant.

The asymmetric entropy gives results slightly less significant when compared to Shannon entropy. It improves Amin 20 times out of 25 (***), with an average gain close to 1.44 (***). However, the improvement of Amaj is only of 0.29, which is not significant. Furthermore, Amaj is improved only 10 times out of 25, with 6 ties. As a result, the small increase of Acc (0.34) is not entirely significant (p-value = 0.067), corresponding to 18 wins out of 25 and 1 tie (*). There is no significant difference for the size of the tree. If the two non centered entropies OCE and AE are compared, one can observe a slight but not significant superiority of OCE for each criterion, in particular a gain of 0.54 for Amin, and 0.42 for Acc.

B. Comparison of the different entropies two by two using majority decision rule and bagged decision trees

In the second run of experimentations, bagged decision trees are performed, using each of the three entropies, In comparison with SE, improvements due to OCE are very highly significant, both for the frequency of improvement in the accuracy and the gain precision. Accuracy has been increased 21 times against 2 (***) on the minority class, 17 times (*) against 6 on the majority class and 23 times (***) against 0 globally. The average gain is 0.65 for Amaj ( 0 ), 1.47 for Amin (***), and 0.75 for Acc (**).

The superiority of AE against SE is slightly modified. For Amin, the average gain of AE compared to SE is no more significant with bagging, only 0.51 instead of 1.44 without bagging. On the other hand, the average gain for Amaj is slightly increased (0.50 instead of 0.29) while the gain for Acc (0.44 instead 0.39) becomes significant (*). The superiority of OCE against AE becomes significant for both Amin and Acc. Compared with AE, OCE improves Amin 16 times against 5 (*), with an average gain of 0.96 (**). By the same way, OCE improves Acc 17 times against 3 (**), with an average gain of 0.37 (*).

C. Comparison of the k-NN-rule with the majority rule, for each entropy

For each entropy, we examine whether the k-NN-rule (k = 3 in our experiments) gives better results than the majority rule (tables VIII, IX and X). Generally speaking, whatever the entropy, the use of local rule significantly increases the value of Amin, but slightly reduces the values of Amaj and Acc. More specifically, in the case of Shannon entropy, the average gain on Amin due to k-NN-rule is 1.7 (**) corresponding to 19 wins against 6 defeats (*). The drops in Amaj (0.5) and Acc (0.1) are not significant. Regarding OCE, the average gain on Amin (1.1) was not significant, but the k-NN-rule prevailed 18 times out of 25, which was significant (*). For AE, the average gain on Amin (1.1) was not entirely significant, but drops in Amaj (0.8) and Acc (0.3) owed to the k-NN-rule are significant. In this new series of experiments, we compare two by two the results obtained with the three entropies, when using the k-NN-rule (tables XI, XII and XIII). The best outcome The last type of experiments compares the results obtained when using usual strategy (SE + MR) with those issued from the proposed adaptive learning strategy (OCE or AE) associated with k-NN-rule. Figure 2 illustrates the gain of accuracy on Amin for each data base when using AE or OCE with k-NN-rule instead of SE with MR. From results presented in Table XIV it is clear that the proposed adaptive learning strategy associated with the k-NN-rule outperforms very significantly the usual strategy. The most conclusive results are obtained with OCE. Indeed, using OCE and the k-NN-rule, the average gain in Amin is equal to 3,0 percent points (***), corresponding to 23 wins and 2 defeats (***), while maintaining Amaj, which gives an average improvement of 0.5 for Acc, corresponding to 19 wins against 6 defeats (**). Using AE in association with the k-NN-rule, also improves the results of the usual strategy. In average, the increase on Amin worth 2.5 (***), corresponding to 21 wins against 4 defeats (***), but the overall accuracy is not improved, as there is a loss of about 0.5 on Amaj.

V. CONCLUSION

Standard decision trees like C4.5 perform poorly on imbalanced data sets. To settle this problem, solutions can be proposed at different levels: resampling methods, split function, pruning-scheme, labeling rule. Previously, we defined an off centered entropy named OCE, which takes its maximum value for the a priori distribution of the class in the considered node. Others authors have proposed an asymmetrical entropy AE which has the same property. In this paper we suggest and test two propositions at the labeling level. The first one consists in using bagged decision trees while the second one lies in the individualization of the labeling rule inside each leaf. The class assigned to an individual is the majority class among the k nearest neighbors of the considered individual inside the leaf (k-NN-rule).

We then present several experiments regarding the different strategies that we could apply. The experiments show that in case of bagging, our off-centered entropy OCE outperforms very significantly Shannon entropy SE (Amin, Amaj and Acc are all the three very significantly increased (respectively 1.5, 0.7 and 0.8 points of percent). The comparison of our strategy (OCE + k-NN-rule) with the usual strategy (SE + majority rule) is in favor of (OCE + k-NN-rule). The experiments show that the average gain is 3.0 for Amin and 0.5 for Acc. Using AE in place of OCE leads to similar result but slightly less significant.

Overall, these results show the relevance of the adaptive learning strategy that we propose to deal with imbalanced data sets when performing decision trees.

In the future, we intend to experiment other labeling rules. Furthermore, we want associate to our strategy a pruning procedure well suited to imbalanced data sets. In addition the use of measures which synthesize both the minority and majority class performance like the area under the ROC curve or the F-measure should be considered.

Fig. 1 .

 1 Fig. 1. Off-centered, asymmetric and Shannon's entropies

Fig. 2 .

 2 Fig. 2. Gain of OCE/k-NN-rule and AE/k-NN-rule against SE/MR

TABLE V OCE

 V

		VS. SE WITH BAGGING
		Acc	Amin	Amaj
	avg.	0.75	1.47	0.65
	std. dev.	1.22	1.58	2.93
	Student ratio 3.11	4.66	1.11
	p-value	0.0048 0.0001 0.2786
	OCE	23	21	17
	=	2	2	2
	SE	0	2	6
	p-value	0.0000 0.0001 0.0347
		TABLE VI
		AE VS. SE WITH BAGGING
		Acc	Amin	Amaj
	avg.	0.39	0.51	0.50
	std. dev.	0.79	1.59	1.58
	Student ratio 2.45	1.61	1.59
	p-value	0.0218 0.1214 0.1249
	AE	16	14	13
	=	4	3	4
	SE	5	8	8
	p-value	0.027	0.286	0.383
	associated with the majority rule. The corresponding results
	are synthesized in tables V, VI and VII.

TABLE VIII K

 VIII -NN-RULE VS. MR FOR SE

	SE	Acc	Amin Amaj
	avg.	-0,10	1,66	-0,48
	std. dev.	1,08	2,83	1,74
	Student ratio	-0,46	2,94	-1,38
	p-value	0,648 0,007	0,180
	result	lost	gain	lost
	k-NN-rule wins	14	19	9
	=	1	0	2
	MR wins	10	6	14
	p-value (sign test) 0,541 0,015	0,405
	result	gain	gain	lost

TABLE XIII OCE

 XIII

	VS. AE WITH K-NN-RULE
	OCE-AE	Acc	Amin Amaj
	avg.	0,48	0,49	0,55
	std. dev.	1,53	1,95	1,84
	Student ratio	1,57	1,26	1,48
	p-value	0,129 0,219	0,151
	result		gain	gain
	OCE wins	13	14	12
	=	4	3	7
	AE wins	8	8	6
	p-value (sign test) 0,383 0,286	0,238
	result	gain	gain	gain
	is clearly that achieved by OCE. In fact, OCE significantly
	surpasses SE, increasing on average of 1.4 point for Amin
	(**), 0.5 for Amaj ( 0 ) and 0.6 for Acc (**). The sign test
	confirms the significance of this superiority, since OCE is
	defeated by SE only 2 times out of 25 for Amin (***), and 1
	time out of 25 for Acc (***). The comparison of OCE with
	AE shows that empirically OCE has better results than AE, but
	improvements are too small to be significant. AE also prevails
	on SE, though giving less significant results than OCE. The
	sole significant result concerns Amin. AE gets the better of
	SE 16 times out of 25 (**), but the average improvement on
	Amin (0.9) is not significant. The changes in Amaj and Acc
	are very small.
	E. Comparison of OCE and AE using k-NN-rule with SE
	using the majority rule

TABLE XIV NON

 XIV -CENTERED ENTROPIES WITH K-NN-RULE VS. SE WITH MR AE/k-NN-rule vs. SE/MR OCE/k-NN-rule vs. SE/MR

		Acc	Amin Amaj	Acc	Amin Amaj
	avg. gain	0,03	2,54	-0,49	0,51	3,03	0,06
	std. dev.	1,01	3,46	2,25	1,74	4,05	,08
	Student ratio	0,13	3,67	-1,09	1,45	3,74	0,09
	p-value	0,900 0,001	0,288	0,159 0,001	0,926
	result	gain	gain	lost	gain	gain	gain
	AE/OCE (k-NN-rule) wins 17	21	9	19	23	10
	=	0	0	2	0	0	2
	SE (MR) wins	8	4	14	6	2	13
	p-value (sign test)	0,108 0,001	0,405	0,015 0,000	0,678
	result	gain	gain	gain	gain	gain	gain