Mathieu Vu

Emilie Chouzenoux

Jean-Christophe Pesquet

Ismail Ben Ayed

Aggregated f -average Neural Network for Interpretable Ensembling

Aggregated f -average Neural Network for Interpretable Ensembling

Mathieu Vu, Émilie Chouzenoux, IEEE Senior Member, Jean-Christophe Pesquet, IEEE Fellow, and Ismail Ben Ayed

Abstract-Ensemble learning leverages multiple models (i.e., weak learners) on a common machine learning task to enhance prediction performance. Basic ensembling approaches average the weak learners outputs, while more sophisticated ones stack a machine learning model in between the weak learners outputs and the final prediction. This work fuses both aforementioned frameworks. We introduce an aggregated f -average (AFA) shallow neural network which models and combines different types of averages to perform an optimal aggregation of the weak learners predictions. We emphasise its interpretable architecture and simple training strategy, and illustrate its good performance on the problem of few-shot class incremental learning.

Index Terms-ensemble learning, estimator aggregation, weakly supervised learning, few-shot learning, incremental learning.

I. INTRODUCTION

A. Ensemble learning

Ensemble learning (or ensembling) is a set of methods which leverage an ensemble of models (also called weak learners), instead of relying on a single learner to perform a given machine learning task (e.g., classification). While ensembling is obviously more demanding in terms of computing resources, it can achieve better accuracy and generalisation, improve overall stability, and reduce prediction variance and bias. Two main phases are identified in the process of building an ensemble model, namely the training of weak learners and the fusion of outputs [START_REF] Sagi | Ensemble learning: A survey[END_REF]. The former focuses on producing an ensemble of diverse models, which is a crucial step in ensemble learning [START_REF] Lahat | Multimodal data fusion: An overview of methods, challenges, and prospects[END_REF], [START_REF] Chen | Class-imbalanced deep learning via a class-balanced ensemble[END_REF], [START_REF] Cheplygina | Dissimilarity-based ensembles for multiple instance learning[END_REF], [START_REF] Hassan | A distributed ensemble of diverse deep learning models for predicting covid-19 cases[END_REF]. For example, bootstrap aggregating (or bagging) [START_REF] Breiman | Bagging predictors[END_REF] trains each model on a different subset of the training data to produce diverse weak learners. Output fusion gathers outputs from each weak learner of the ensemble and combines them to produce the final prediction [START_REF] Kviman | Multiple importance sampling elbo and deep ensembles of variational approximations[END_REF], [START_REF] Luengo | Efficient linear fusion of partial estimators[END_REF], [START_REF] Zhang | Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics[END_REF].

One could distinguish two categories of methods for output fusion in ensemble learning [START_REF] Sagi | Ensemble learning: A survey[END_REF]. The most basic one is to average weak learners outputs or, in the case of classification, to use a majority voting scheme [START_REF] Biggs | On margins and generalisation for voting classifiers[END_REF]. Different types of averages could be used (e.g., arithmetic, geometric, harmonic, etc) and weights could be included to further refine results. Those weights can be set using various kinds of criteria, for example based on weak learners isolated performance [START_REF] Freund | A decision-theoretic generalization of on-line learning and an application to boosting[END_REF]. The second category of methods uses meta-learners. They consist in plugging an additional model, responsible for taking advantage of the weak learners. Mixture of experts is a popular variant of meta-learners, where a gating network selects the weak learner that is most suited to produce the correct prediction given a certain input [START_REF] Jacobs | Adaptive mixtures of local experts[END_REF], [START_REF] Bian | Subarchitecture ensemble pruning in neural architecture search[END_REF]. A more straightforward output fusion based on meta-learners is the stacking of an additional learning model. Taking weak learners output as input, it learns the best combination to assemble the unified prediction [START_REF] Wolpert | Stacked generalization[END_REF], [START_REF] Kumar | An ensemble of fine-tuned convolutional neural networks for medical image classification[END_REF].

B. Contribution

Our contribution, in this work, is to introduce aggregated f -average (AFA) neural networks (NNs), based on a novel architecture for the output fusion phase of ensemble learning. It consists of a shallow neural network modelling different types of averages (arithmetic, geometric, harmonic, etc.) and is able, through supervised learning, to combine and/or select them optimally. Thanks to a specific architecture including original nonlinear activations and constrained weights, it is easily interpretable. To illustrate the performance of AFA neural networks upon the state-of-the-art, we describe their application and implementation in the currently popular setting of few-shot class incremental learning (FSCIL).

The paper is organised as follows. First, in section II, we present the architecture of our AFA model along with its training process. We then introduce, in section III, the FSCIL problem and describe our ensembling approach in this context. Experiments on several datasets highlight the benefits of our model, when compared with other ensemble output fusion methods.

II. METHODOLOGY

A. Ensembling through averaging

Let K machine learning models trained for a common task (e.g., classification), produce K outputs (x k ) 1≤k≤K , assumed to be vectors in R N . In ensemble learning, those K outputs are combined during an output fusion phase in order to produce a single, expectably better, prediction for the task at hand. A naive method is to average the outputs. We summarize in Table I

K k=1 ω k x k K k=1 x ω k k K k=1 ω k x k -1 K k=1 ω k x q k 1/q Validity x k ∈ R N x k ∈ [0, +∞) N x k ∈ (0, +∞) N x k ∈ [0, +∞) N , q > 0 B. f -average
Following Kolmogorov's mean framework [START_REF] Kolmogorov | Mathematics and Mechanics[END_REF], we rewrite the above examples under the generalised form:

x = f -1 K k=1 ω k f (x k ) . (1)
Hereabove, f is a bijective function from [0, +∞) N to some convex C of R N , and f -1 is its inverse function from C to [0, +∞) N . We will subsequently assume that f operates componentwise, in the sense that it consists of the application of the same scalar function to each of the components of its argument. Let us now express functions f and f -1 to retrieve the popular averaging rules from Table I. We denote (ξ n ) 1≤n≤N the components of x ∈ R N . We set ϵ ∈ (0, +∞) and, to circumvent the indefiniteness of the harmonic mean for vectors with a zero component, we define the leaky hyperbolic function h ϵ as

(∀ξ ∈ R) h ϵ (ξ) =            1 ξ + ϵ -ϵ if ξ ∈ [0, 1/ϵ -ϵ] - ξ ϵ 2 + 1 ϵ -ϵ if ξ < 0 -ϵ 2 ξ - 1 ϵ + ϵ if ξ > 1/ϵ -ϵ.
(2) Table II summarizes the expression for f , f -1 , as well as their associated definition domains, recovering the averaging rules from Table I. Exact geometric and harmonic means formula are retrieved when ϵ goes to zero.

We now propose to extend the generalised average framework (1) to the case when scalars (ω k ) 1≤k≤K are replaced by matrices (Ω k ) 1≤k≤K in R N ×N , so as to allow a full mixing of the weak learners. Given some functions (f, f -1 ) defined as previously, the f -average output x ∈ R N is obtained as

x = f -1 K k=1 Ω k f (x k ) = f -1 W f (x) . (3)
Hereabove,

W = [Ω 1 , . . . , Ω K ] ∈ R N ×KN , (4)
and

f : [0, +∞) KN → C K (5) x = (x k ) 1≤k≤K → f (x k ) 1≤k≤K (6)
applies f in a parallel manner to the vector inputs (x k ) 1≤k≤K .

In order to ensure the interpretability of the averaging operation in (3), W is chosen such that

W ∈ [0, +∞) N ×KN , W 1 KN = 1 N . (7)
This guarantees, in particular, that W f (x) belongs to the definition domain of f -1 (since this domain has been assumed to be convex). For instance, if x ∈ [0, 1] KN (e.g., in a classification context), the constraint on W ensures that the output x also belongs to [0, 1] N .

x f W f -1 x Fig. 1.
Structure of a neural network that performs an f -average for ensembling Remarkably, operation (3), that we call an f -average, can be represented as the application, on x, of a two-layer neural network whose structure is drawn in Figure 1. This neural network is parametrized by the choice of f (along with its inverse function f -1 ) and by the weight matrix W . The former can be set by the user, while the latter can be determined through supervised learning, by minimizing a loss associated to the task at hand. The f -average network is interpretable, as the contribution of each output in the final prediction can easily be retrieved using the weights in matrix W , and the averaging operation is determined by the choice of f .

C. Aggregated f -averages

The previous approach requires the prior choice for the average rule (i.e., f ). To wave this restriction, we suggest aggregating J > 1 f -averages, associated to different functions (f j ) 1≤j≤J . Resorting to the same structure as the one presented in the previous section, we define

(∀j ∈ {1, . . . , J}) xj = f -1 j W j f j (x) , (8)
where, for every j ∈ {1, . . . , J},

W j ∈ [0, +∞) N ×KN W j 1 KN = 1 N , (9)
and f j is a function operating component-wise from [0, +∞) KN to C j , associated to a given mean function f j , f -1 j is its inverse function operating component-wise from some convex set C j ⊂ R N to [0, +∞) N , and x concatenates

Mean

f (x) f domain f -1 (x) f -1 domain Arithmetic Id [0, +∞) N Id [0, +∞) N Geometric ln(ξ n + ϵ) 1≤n≤N [0, +∞) N exp(ξ n ) -ϵ 1≤n≤N [ln ϵ, +∞) N Harmonic h ϵ (ξ n ) 1≤n≤N [0, +∞) N h ϵ (ξ n ) 1≤n≤N (-∞, ϵ -1 -ϵ] N Power-q (ξ q n ) 1≤n≤N [0, +∞) N (ξ 1/q n ) 1≤n≤N [0, +∞) N
columnwise the inputs (x k ) 1≤k≤K . The resulting joint aggregate estimate of the J outputs (x k ) 1≤j≤J is defined as

x = J j=1 A j xj = A    x1 . . . xJ    , (10)
with, for every j ∈ {1, . . . , J}, A j ∈ [0, +∞) N ×N , and A ∈ [0, +∞) N ×N J is the rowwise stacking of (A j ) 1≤j≤J matrices. Operations ( 8)-( 10) are equivalent to plug x as the input of a neural network with J sub-networks of the form presented in Figure 1, operating in parallel, followed by a linear layer involving the weight matrix A. We further add a final activation function, g : R N → R N , to control the domain of the output. For instance, a softmax activation can be used to get nonnegative outputs summing to one, in a classification context. The resulting network, called aggregated f -average, is displayed in Figure 2. It has a limited number JN 2 (K + 1) of linear parameters, namely the entries of matrices W 1 , . . . , W J , and A. The training of these parameters can follow a classical supervised learning approach. Given a sample and its ground truth, the task model loss is computed (e.g., a cross-entropy loss for classification), before updating the weights from all layers using a backpropagation algorithm (e.g., Adam optimizer [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF]). Constraints on weight matrices (W j ) 1≤j≤J can simply be imposed through a projection step of each row of these matrices on the convex unit simplex set [START_REF] Condat | Fast projection onto the simplex and the ℓ 1 ball[END_REF], performed after each backpropagation update. Let us remark that this model maintains the interpretability properties of the J f -average sub-models it contains. Furthermore, weights from matrix A can be viewed as the contribution level of each type of average model.

III. AGGREGATED f -AVERAGE FOR FEW SHOT INCREMENTAL LEARNING

A. FSCIL framework and datasets

We now illustrate the potential of the proposed AFA framework by applying it to the problem of few-shot class incremental learning. FSCIL, recently introduced in [START_REF] Tao | Few-shot class-incremental learning[END_REF], focuses on designing machine learning methods that can deal with both incremental [START_REF] Castro | End-to-end incremental learning[END_REF], [START_REF] Hou | Learning a unified classifier incrementally via rebalancing[END_REF] and few-shot situations [START_REF] Singh | Semisupervised deep convolutional transform learning for hyperspectral image classification[END_REF], [START_REF] Tang | Joint concept matching based learning for zero-shot recognition[END_REF], [START_REF] Martin | Towards practical few-shot query sets: Transductive minimum description length inference[END_REF], [START_REF] Boudiaf | Information maximization for few-shot learning[END_REF]. Specifically, FSCIL aims at including an increasing number of categories in a classification problem with the extra constraint that only a small number of training samples are available for upcoming classes. This is motivated by the frequent practical situation in computer vision when a model, built to classify certain categories seen during training phase, must be adjusted to classify images belonging to new classes with only (very) few annotations. The main difficulty revolves around the ability to learn new classes while preventing catastrophic forgetting of classes previously learned, yielding poor performance of standard incremental learning methods.

x f 2 f 1 . . . f J W 1 W 2 . . . W J f -1 1 f -1 2 . . . f -1 J A g x
The FSCIL setting considered here consists of K successive sessions that incrementally provide new categories of images to be classified. First session (k = 1), also called base session, is a standard classification problem with a large number n train of training samples for each of the n class_base base classes. In subsequent sessions (k ∈ {2, . . . , K}), only a small additional number n shots of training samples is provided for a limited number n way of novel classes. At each session k, the number of classes to predict is

N k = n class_base + (k -1)n way . ( 11
)
Our experiments focus on four typical FSCIL datasets, which characteristics are summarized in Table III. The closer the F 1 score is to 1, the better. As sessions are incrementally added, the classification task becomes increasingly more difficult because of the additional categories to be handled.

B. Application of aggregated f -average

We address FSCIL as the ensembling of successive few-shot learning problems [START_REF] Chen | A closer look at few-shot classification[END_REF]. For each session, we train a few-shot learning classifier that serves as a weak classifier (i.e., weak learner) specialised on its own session in our ensemble model. During base session, the weak classifier is a standard classifier with a ResNet-18 architecture [START_REF] He | Deep residual learning for image recognition[END_REF] trained thanks to the larger training set. Then, for the remaining sessions (k ≥ 2), its backbone (i.e., all layers except the last fully connected layer) is frozen and used as feature extractor. For these sessions, that last fully connected layer (specialised on base session) is replaced by a few-shot learning method. We use a state-of-theart nearest-neighbour classifier that determines the estimated class of a test image by retrieving the closest mean centroid in the feature space [START_REF] Wang | Simpleshot: Revisiting nearest-neighbor classification for few-shot learning[END_REF]. It is trained to classify the n way new classes provided by the current session.

As described in Figure 5, for a given K, our aggregated f -average model takes as input the outputs from sessions k ∈ {1, . . . , K}, of size N = N K (i.e., the total number of classes, defined in [START_REF] Freund | A decision-theoretic generalization of on-line learning and an application to boosting[END_REF]), to form a vector x ∈ R KN K . As the weak classifiers from earlier sessions are trained to predict a fewer number of classes, a zero-padding is performed for its prediction vector in order to reach output size N K . Our AFA model is trained for 100 epochs, minimising a crossentropy loss with the Adam algorithm [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF], using an initial learning rate of 10 -1 decreased by a factor 10 at epochs 40 and 70. The training uses a prototype rehearsal strategy [START_REF] Rebuffi | icarl: Incremental classifier and representation learning[END_REF] which consists of collecting predictions from all previous weak classifiers on training sets from all previous sessions to form the training set of the ensemble learning model. It is then fed to the model as mini-batches of size 64. We set J = 4 and (f j ) 1≤j≤J chosen so as to aggregate four different types of means, namely arithmetic, geometric, harmonic, and quadratic (power-2), using the activation functions presented in Table II with ϵ = 10 -4 . Weights of matrices (W j ) 1≤j≤J and matrix A are initialised with entries sampled randomly following a uniform distribution between 0 and 1 KN K before being projected onto the simplex to comply with the sum-to-one constraint. Our model and all experiments were implemented using Tensorflow [START_REF] Abadi | TensorFlow: Large-scale machine learning on heterogeneous systems[END_REF].

C. Results

We compare our proposed model performance, for increasing values of K, against classic ensembling methods for classification performing output fusion. We try out different types of averages (namely, arithmetic, geometric, harmonic), as well as a majority vote scheme. Performance were also compared against three different types of ensembling neural network: 1) a shallow neural network with similar number of parameters and layers as our AFA model, 2) a deeper neural network with five fully connected layers, and 3) a neural network specifically designed for ensembling including a weighted average layer followed by a fully connected layer for the output [START_REF] Ozechi | Ensembling neural network models with tensorflow[END_REF]. All neural network models, including the AFA model, were trained with the same process with only slight adjustments on learning rate parameters to adapt to each model architecture. The results, in terms of accuracy and F 1 scores (the larger, the better), are summarized in Table IV and Figure 3. Note that the mean accuracy over all classes might hide performance discrepancies between base and new classes [START_REF] Branco | A survey of predictive modeling on imbalanced domains[END_REF], as can be seen in Table IV. Typically, a model trained on base session only (called No Fine-tuning in Table IV), reaches a decent mean accuracy thanks to its performance on base session classes (k = 1), while it yields very poor performance on new classes (k > 1). In contrast, our approach AFA performs an effective incremental learning, with good accuracy on both base and new classes. Classic averaging shows inconsistent results (see Figure 3), as the optimal type of average depends on the session number and on the dataset. The majority vote scheme seems to produce worse results in early sessions (low K), when few voters are available, and a gain in performance with an increasing number of voters. By design, the proposed ensemble method models those different types of average, and learns their optimal combination, thus producing significantly better performance in every session.

AFA also outperforms other NN approaches: among the three models compared with the AFA model, best results were achieved by the weighted average neural network model. Despite its dedicated design, its performance drop more significantly than our AFA model with more sessions incrementally added. Because of its generic architecture, the shallow NN produces even worse results while still improving with respect to classic averages. Finally, the deep NN shows the most inconsistent results; its performance catastrophically drop in later sessions (higher K) showing training issues. Indeed, these later sessions drastically increase its architecture size due to the higher number K of models to ensemble and the higher number of classes N K to predict, while not providing a significantly larger training set because of the few-shot constraint of our task at hand. Figure 4 illustrates these behaviours. It presents training losses of the different stacked neural network models, including the proposed AFA, on a specific example. The plots demonstrate our model' stability and fast convergence to a lower loss value than other methods which exhibit a plateau at a higher value. The deep NN loss behavior shows the inability to train a large number of parameters on such limited amount of data, with a validation loss diverging.

IV. CONCLUSION

In this article, we presented a novel output fusion method for ensemble learning. Inspired by basic methods, namely averaging and model stacking, our method relies on an original neural network architecture that models multiple types of averages. It is trained to learn an optimal and interpretable selection and/or combination of the multiple weak learners inputs. We illustrated its operability and efficiency on the problem of few-shot class incremental learning where it significantly outperforms standard ensemble learning output fusion methods.

Fig. 2 .

 2 Fig.2. Structure of the proposed aggregated f -average neural network. It aggregates J f -averages for ensembling, with A ∈ [0, +∞) N ×N J . The activation function g : R N → R N is selected according to the task (e.g, softmax for classification, linear for regression).

Fig. 3 .

 3 Fig.3. Comparison of ensemble learning output fusion methods, on FSCIL datasets, in terms of averaged F 1 score over all classes, for various sessions k. The closer the F 1 score is to 1, the better. As sessions are incrementally added, the classification task becomes increasingly more difficult because of the additional categories to be handled.

Fig. 4 .Fig. 5 .

 45 Fig.[START_REF] Cheplygina | Dissimilarity-based ensembles for multiple instance learning[END_REF]. Training and validation losses of different stacked NN models for ensembling. Models are trained on the same data, at the same session (K=5) on CUB-200, and with the same training parameters, except for learning rates adapted to each architecture. The AFA model presents a loss that converges faster to its plateau, at a lower value than other models. AFA and weighted average losses are smooth and decrease regularly, while the deep and shallow NN models present more instability on the validation loss. Especially, the deep NN shows a divergent validation loss.

  common expressions for weighted averages, with (ω k ) 1≤k≤K nonnegative reals such that

K k=1 ω k = 1.

TABLE II EXAMPLES

 II FOR f , f -1 , AND DEFINITION DOMAINS, FOR x = (ξn) 1≤n≤N . GEOMETRIC AND HARMONIC FORMULAS ARE RETRIEVED WHEN ϵ → 0.

TABLE IV DETAILED

 IV METRICS COMPARISON ON THE LAST SESSION (K = 9) OF MINI-IMAGENET DATASET. FOR ALL METRICS, THE HIGHER THE BETTER.

	Metrics	No Fine-tuning	Shallow NN Deep NN Weighted Avg.	AFA
	Accuracy	39.26	32.69	27.04	38.63	41.29
	Base Accuracy	65.43	47.38	38.80	55.23	55.12
	New Accuracy	00.00	10.65	09.40	13.73	20.55
	F1(%)	30.01	32.11	25.24	37.06	40.73