Adrian Popescu
email: adrian.popescu@cea.fr

IL2M: Class Incremental Learning With Dual Memory

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Incremental learning (IL) is the ability of artificial agents to learn from data that are presented to them sequentially. Our focus is on class IL which assumes that data are labeled. The problem is trivial if enough computational power and storage are available and if long delays are allowed for model updates. These conditions are often not met in real applications and class IL becomes hard to solve. This is the case in contexts such as robotics, mobile apps and military applications, where visual recognition capacities need to be incremented without access to large infrastructures [START_REF] German | Continual lifelong learning with neural networks: A review[END_REF].

Recent class IL methods exploit Deep Neural Networks (DNNs) which obtain very good performance for many AI tasks, including image recognition [START_REF] He | Deep residual learning for image recognition[END_REF]. The main problem faced by DNN based IL methods is catastrophic forgetting [START_REF] Mccloskey | Catastrophic interference in connectionist networks: The sequential learning problem[END_REF], i.e. their inability to integrate new data without forgetting previously learned knowledge. Minimizing computation, storage and time requirements simultaneously is not doable and existing methods make compromises on one or two of these conditions. A stream of research [START_REF] Aljundi | Expert gate: Lifelong learning with a network of experts[END_REF][START_REF] Mallya | Packnet: Adding multiple tasks to a single network by iterative pruning[END_REF][START_REF] Rusu | Progressive neural networks[END_REF]29] assumes that deep architectures can grow to some extent so as to integrate new data. Under this assumption, no memory of the past is needed. Another research trend [START_REF] Francisco | End-to-end incremental learning[END_REF]8,12,[START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] posits that the DNN architectures should be fixed. They adapt the DNN fine tuning process by adding a distillation loss and use a bounded memory of the past to limit catastrophic forgetting.

Our method, Incremental Learning with Dual Memory (IL2M ) is summarized in Figure 1. with an example which includes an initial and two incremental states. IL2M uses a fixed DNN architecture and a bounded memory of the past. Our main contribution is to propose a second memory which stores initial class statistics in a very compact format. The introduction of this memory is based on the intuition that classes are best modeled when first learned, with all data available. Initial class statistics are reused in each subsequent incremental state to rectify the prediction scores of past classes. Rectification is necessary because class IL models are trained with imbalanced datasets in which past classes have fewer examples. Consequently, their prediction scores are generally lower than those of new classes.

A second contribution is of practical nature and consists in using vanilla fine tuning as basis for class IL. This use challenges the common hypothesis that a distillation loss term is necessary in IL with memory [START_REF] Francisco | End-to-end incremental learning[END_REF]8,12,[START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF]. We show that, if each past class has at least a few exemplars, the distillation loss actually hurts performance and vanilla fine tuning provides significantly better performance.

The evaluation is done against strong baselines and their adaptations based on vanilla fine tuning. Three large public datasets with different memory sizes and number of IL states are used. Results indicate that IL2M obtains stateof-the-art results in a wide majority of tested conditions.

Related work

Different methods were proposed for class IL. We group them in three classes and list their advantages and limits.

A first class of algorithms is focused on adapting the parameters of the deep model in order to accommodate new classes. Growing a Brain [29] proposes to widen a part of the layers or/and increase network depth. Deep Adaptation Networks [START_REF] Rosenfeld | Incremental learning through deep adaptation[END_REF] are an alternative to fine tuning to fit models to new tasks. Each new task requires approximately 13% supplementary parameters. While manageable for one task, this amount becomes important for a large number of increments. Progressive neural networks [START_REF] Rusu | Progressive neural networks[END_REF] train several models for initial tasks and exploit them when adding new tasks in order to preserve old knowledge. Preservation is notably done by using lateral connections between all models. A committee of expert networks is introduced in [START_REF] Aljundi | Expert gate: Lifelong learning with a network of experts[END_REF] to handle the different tasks learned. The most adapted expert is chosen via a gating mechanism which exploits training samples. The authors of [START_REF] Sylvestre-Alvise Rebuffi | Efficient parametrization of multi-domain deep neural networks[END_REF] introduce universal parametric families of neural networks which share a majority of parameters and use small modular adapters that are attached to the network to specialize it for new tasks. A key finding is that both shallow and deep layer adaptation is needed for successful adaptation. PackNet [START_REF] Mallya | Packnet: Adding multiple tasks to a single network by iterative pruning[END_REF] is a very interesting approach that accommodates new tasks by iteratively pruning redundant parameters for previous tasks. The number of parameters grows slowly but only a limited number of new tasks can be included with reasonable performance loss. Also, the inference is longer since it cannot be applied simultaneously to all trained tasks. Piggyback [START_REF] Mallya | Piggyback: Adapting a single network to multiple tasks by learning to mask weights[END_REF] combines PackNetand network quantization works to propose masks for individual weights. It thus learns a large number of tasks with a single base network. While rather light, each task requires specific masks and the number of parameters increases when adding new tasks. Approaches in this group cope well with new data, do not depend on past memory and can integrate new tasks if the number of model parameters is allowed to grow. However, they tend not to scale well, either because new parameters need to be added each time or because a limited number of tasks can be included.

A second class of algorithms keeps the number of DNN parameters constant and memorizes a part of past data to limit catastrophic forgetting. Here, the class IL problem becomes akin to an imbalanced learning one [START_REF] He | Learning from imbalanced data[END_REF]. The challenge is to ensure similar performance for past and new classes, given that the number of images for past classes can be orders of magnitude lower than that for new classes [START_REF] Buda | A systematic study of the class imbalance problem in convolutional neural networks[END_REF]. Adapted fine tuning is generally applied to update the model incrementally. A modification of loss function to include a distillation component alongside the classification one is widely used [START_REF] Francisco | End-to-end incremental learning[END_REF]8,12,[START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF]. These approaches are inspired by Learning without Forgetting (LwF ) [ 15] which was an early attempt to exploit knowledge distillation [START_REF] Hinton | Distilling the knowledge in a neural network[END_REF] as an antidote to catastrophic forgetting. The distillation loss reduces the discrepancy between the activations of past classes in the initial and the updated network. LwF has the particularity of not needing a memory of old tasks, which is an important advantage in IL. However, its performance is lower compared to approaches that exploit a bounded memory. iCaRL [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] is an influential algorithm from this class. It builds on the combination of classification and distillation losses from LwF and adds a bounded memory, as well as a nearest-exemplars-mean (NEM) classifier. NEM is inspired by nearest-class-mean [START_REF] Mensink | Distance-based image classification: Generalizing to new classes at near-zero cost[END_REF], which tackles class imbalance. iCaRL is notably tested on the Imagenet LSVRC dataset and it outperforms several baselines, including LwF and fixed representations. The authors of [12] propose a detailed analysis of iCaRL and show that its most important component is the bounded memory. They replace the NEM classification with a dynamic threshold moving method and obtain a marginal improvement. An end-to-end IL (EtEIL) algorithm is introduced in [START_REF] Chu | Best practices for fine-tuning visual classifiers to new domains[END_REF] which also exploits a combined loss. The main novelties come from: (1) the proposal of a distillation term per incremental state and (2) a classification step done with a balanced fine tuning that tackles class imbalance. As a result, a 7 points improvement compared to iCaRL is reported for ILSVRC. We note that, while implemented in different deep learning frameworks and with different formulations, the distillation based backbones from [START_REF] Francisco | End-to-end incremental learning[END_REF]12,[START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] have rather similar similar results. The use of GANs was explored in [8] as an alternative to the storage of raw images for past classes. While conceptually interesting, the quality of generated exemplars is not yet sufficient for them to efficiently replace real images. The results from [8] indicate that only a combination of both types of images provides a slight performance improvement compared to the sole use of real images. The use of the adapted fine tuning is an adequate solution if the model complexity needs to be constant across incremental states. This is the case of embedded systems which have limited computing power and need to adapt continuously to their environment [START_REF] Kemker | Fearnet: Braininspired model for incremental learning[END_REF][START_REF] German | Continual lifelong learning with neural networks: A review[END_REF]. However, partial access to past data is a necessary condition for this type of methods to work well. This condition cannot be met in contexts such as that of medical data where data privacy is of utmost importance [START_REF] Shokri | Privacy-preserving deep learning[END_REF].

A third, less frequent, class of algorithms exploits initial fixed representations as feature extractors for IL. FearNet [START_REF] Kemker | Fearnet: Braininspired model for incremental learning[END_REF] is a biologically inspired such method. Separate networks are used for long and short term memories to represent past and new classes. A decision mechanism is implemented to decide which network should be used for each test example. While FearNet outperforms iCaRL, its memory increases significantly with time since the algorithm needs to store detailed statistics for each class learned. DeeSIL [START_REF] Belouadah | Deesil: Deep-shallow incremental learning[END_REF] is a simple take at class IL with bounded memory. A fixed representation is learned in the initial state and is then reused as feature extractor for all incremental states. Shallow classifiers are learned independently for each new class. This approach is a direct application of transfer learning schemes [START_REF] Kornblith | Do better imagenet models transfer better?[END_REF][START_REF] Sharif Razavian | CNN features off-the-shelf: An astounding baseline for recognition[END_REF]. Despite its simplicity, it provides 14 and 7 points performance gain over iCaRL [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] and end-to-end learning [START_REF] Francisco | End-to-end incremental learning[END_REF] for ILSVRC. FearNet and DeeSIL have interesting performance but are heavily dependent on the quality of their initial fixed representation. If it is learned with a small number of classes or if the new classes are very different from the initial ones, the generalization ability of the feature extractor is likely to be low.

Class IL problem formulation

The class IL problem was described in [START_REF] Francisco | End-to-end incremental learning[END_REF]8,[START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] and we present an adaptation here. A dataset

X P = {X 1 ,X 2 ,...,X P } is composed of P different classes such that X i = {x 1 i ,x 2 i ,...,
x ni i } is the set of n i labeled examples for the i th class. In DNNs, a model M is composed of a feature extractor F : X i → R d , with d the size of the feature vector, followed by a classifier C : R d → P . The prediction score for class C i is noted p(C i ) and is the raw output of the DNN classification layer (without softmax). The class IL problem is defined as follows:

Given a model M P trained on X P , the objective is to use M P to train an updated model M N which recognizes N classes based on the dataset X N . The access to X P is partially provided by a bounded memory K and the number of parameters of M N and M P is identical.

Each set of N -P new classes forms an incremental batch and the N classes form an incremental state. A loss adaptation to class IL is widely used to move from M P to M N [START_REF] Francisco | End-to-end incremental learning[END_REF]8,12,[START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF]. It can be written as L = L c +L d , where L c and L d are classical cross-entropy and distillation terms respectively. L d is meant to reduce catastrophic forgetting. M can be modeled in an end-to-end fashion to combine F and C in a single deep architecture [START_REF] Francisco | End-to-end incremental learning[END_REF]. The two components can also be separated. For instance, [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] uses a deep architecture F which is retrained at each incremental step to extract features and a nearest-mean-of-exemplars to implement C. Alternately, [START_REF] Belouadah | Deesil: Deep-shallow incremental learning[END_REF] exploits a fixed deep representation to extract features for all incremental states and a set of independently trained SVMs to implement C.

The bounded memory K which provides partial access to past training data reduces the effect of catastrophic forgetting. Since the size of the memory is constant across incremental states, the training set of past classes is progressively reduced when more classes are added. Assuming a balanced representation of past classes in memory, each class will have K P images when incrementing from P to N classes and K N for the following incremental state. We note Z the total number of states, including the first nonincremental one.

Proposed method

We focus on a class IL scenario in which the DNN model complexity is constant and a bounded memory of the past is allowed. Adapted fine tuning methods [START_REF] Francisco | End-to-end incremental learning[END_REF]8,12,[START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] update the model M for each incremental state. However, only a small fraction of past data can be used due to the bounded memory and imbalance worsens as more classes are learned ( K N < K P ). Fixed representation based methods [START_REF] Belouadah | Deesil: Deep-shallow incremental learning[END_REF][START_REF] Kemker | Fearnet: Braininspired model for incremental learning[END_REF] ex- ploit all available data but their models are frozen after the initial non-incremental state. They are thus heavily dependent on the quality of this initial representation.

We introduce Incremental Learning with Dual Memory (IL2M ) and illustrate it in Figure 1. The method aims to partially reconcile the fine tuning and fixed representation based approaches. IL2M uses vanilla fine tuning as backbone to update deep models M for each incremental state, as proposed in fine tuning approaches. Similar to fixed representation methods, IL2M exploits class related knowledge from the initial state in which they were learned across incremental states. Due to deep model updating, initial class models cannot be fully reused in later states. Instead, IL2M exploits past class statistics from their initial state to rectify their prediction scores in the current incremental state. This rectification is supported by two related hypotheses: (1) classes are best modeled when all their data are available and (2) class prediction scores are higher on average when more training data are available. We illustrate the validity of these hypotheses in Figure 2. It plots the averaged predictions of past and new classes for the ILSVRC dataset with Z = 10 states and memory sizes K = {20000, 10000, 5000}.

The scores in Figure 2 confirm that vanilla fine tuning generates a prediction bias in favor of new classes. This bias is mainly due to the imbalance in favor of new classes which appears in class IL. As a result, a large part of images from past classes are predicted as belonging to new classes (see supplementary material for a detailed analysis of error types). The comparison of the three subfigures shows that score gap between past and new classes is higher when the memory capacity is lower. The average difference over all incremental states is 2.42, 4.02 and 6.45 for K = {20000, 10000, 5000} respectively. This is intuitive since the imbalance between past and new classes is higher for lower memories. The gap also tends to grow from left to right in each subfigure due to the increasing number of classes to fit in the bounded memory. For instance, the difference is 2.26, 4.16 and 4.67 for states 1, 5 and 9 with K = 10000 exemplars.

To compensate for the bias toward new classes, we rectify predictions of past classes C i (i =1,...,P) using:

p r (C i )=    p(C i ) × µ P (Ci) µ N (Ci) × µ(M N ) µ(M P ) , if pred = new p(C i ), otherwise (1)
with: P -the initial state in which C i was learned; Nthe current incremental state; p(C i ) -the raw prediction for C i in state N ; µ P (C i ) and µ N (C i ) -the mean classification scores of C i in states P and N obtained from all training data and the current exemplar set respectively; µ(M N ) and µ(M P ) -the model confidences in states N and P given by the averaged prediction scores of all new training data. In Eq. 1, rectification is applied to past class predictions only if an image is initially predicted as belonging to a new class. This situation is the riskiest in terms of imbalance-driven errors in favor of new classes. Otherwise, we consider that the rectification is not necessary since a past class is directly predicted and there is no prediction bias toward new classes. The effect of the rectification restriction to past images initially associated to a new class is studied in the ablation study from Subsection 5.4.

Since classes are initially learned in different incremental states, the following conditions need to be met for the proposed rectification to be useful in class IL:

1. the scores p r () for classes in range {1,P} and p() from {P +1,N} should be comparable;

2. the statistics stored in the statistical memory S should be very compact in order to increase memory needs only marginally;

3. model level normalization should be introduced to limit the influence of combining the outputs of models learned in different incremental states.

The first condition is handled via the use of class related statistics in the first term which modifies p() in Eq. 1. More specifically, we use the means of class C i in its initial and current states P and N . The intuition here, supported by Figure 2, is that since the class is first learned with all training images in state P when it was new, its mean prediction score µ P (C i ) is likely to be higher than µ N (C i ). Consequently, this term of the equation generally increases p r (C i ) compared to p(C i ). The second condition listed above is related to the introduction of the statistical memory S which makes the IL2M rectification possible. S includes a float value per class to store µ P (C i ) and the induced memory requirement is negligible. As for the model level knowledge, only one float per incremental state is needed to store µ(M). The third condition is necessary since the averaged scores for new classes are not equivalent in the different incremental states which are combined. This is clear in Figure 2, where, for instance, the new class mean scores for state 8 are higher than those of state 7 for K = 10000. The last term of Eq. 1 provides a global harmonization of the score rectification across the different states that are combined in IL2M .

The complexity of the supplementary arithmetic operations from Eq. 1 is very low compared to the overall complexity to a deep neural network architecture. For each class score rectification, a division and a multiplication are needed to introduce the second term. The division in third term can be computed only once the training of the current incremental state is ready. This term is thus integrated through a simple multiplication. For 1000 past classes, IL2M adds 1000 divisions and 2000 multiplications. This is to be compared to the tens to hundreds of million of multiplications done in typical DNN architectures.

The rectification introduced here is an alternative to the NEM classification from iCaRL [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] and to the balanced fine tuning step of end-to-end learning from [START_REF] Francisco | End-to-end incremental learning[END_REF]. The three methods are compared in the following section.

Experiments

Baseline methods

IL2M is designed for IL with bounded memory and is compared to strong methods which address the problem:

• iCaRL -the public implementation from [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] is reused here. It includes a fine tuning with classification (L c ) and distillation (L d ) losses for representation learning followed by nearest-exemplars-mean (NEM) component for classification. When no memory is available, iCaRL is equivalent to LwF.M C, the adaptation of Learning without Forgetting to a multiclass context also introduced in [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF].

• DeeSIL -the fixed-representation based algorithm [START_REF] Belouadah | Deesil: Deep-shallow incremental learning[END_REF] is implemented without external data to ensure comparability. Each class is learned with all its training images as positives. The negative set includes all training images of other classes from the same incremental batch and the exemplars of past classes stored in memory. A grid search for the optimal regularization parameter is applied to the first batch and the parameter is then frozen.

• FT -fine tuning with classification loss only (L c ) constitutes the basis for IL2M and for the proposed modifications of two strong baselines described below.

Each incremental state uses the model learned in the previous state to initialize the training process. Training is done with the exemplars of past classes and with all available images of new classes. In [START_REF] Francisco | End-to-end incremental learning[END_REF], herding has marginal effect and we perform a simpler random selection of exemplars.

• FT NEM -a version of FT which uses the nearestexemplars-mean classifier from [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] instead of the classification layer of the deep network. FT NEM is a modified version of iCaRL in which the distillation loss L d is ablated.

• FT BAL -a version of FT in which a balanced fine tuning is performed for classification after the initial imbalanced vanilla FT following [START_REF] Francisco | End-to-end incremental learning[END_REF]. FT BAL is a modified version of EtEIL [START_REF] Francisco | End-to-end incremental learning[END_REF] in which we again ablate L d . The balancing step starts with the latest learning rate of the imbalanced FT. Note that original EtEIL [START_REF] Francisco | End-to-end incremental learning[END_REF] is not fully evaluated because the only available implementation uses MathConvNet based on non-free Matlab. However, a top-5 accuracy comparison of EtEIL and FT BAL for ILSVRC is clearly favorable to the latter method (69.4 vs. 77.52).

In addition, we provide Full, the non incremental learning training with all data available. This is an upper bound performance for class IL algorithms.

Datasets and methodology

We evaluate all methods on three datasets designed for the following visual recognition tasks: (1) objects in ILSVRC [26], (2) faces in VGGFace2 [START_REF] Cao | Vggface2: A dataset for recognising faces across pose and age[END_REF]

and (3) tourist landmarks in Google Landmarks [20] (Landmarks below).

A summary of the datasets is presented in Table 1. In VGGFace2 [START_REF] Cao | Vggface2: A dataset for recognising faces across pose and age[END_REF] and Landmarks [20], we kept the 1,000 classes which include the largest number of examples. For ILSVRC, we use the train and test sets from [START_REF] Francisco | End-to-end incremental learning[END_REF][START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] have standard test sets for IL. We randomly select 50,000 and 20,000 images respectively for testing, with a balanced distribution among classes (see supplementary material for more details). Note that, due to sequential nature of incremental learning, model training is rather expensive. As a result, the usual evaluation protocols include two [START_REF] Francisco | End-to-end incremental learning[END_REF]8,12,[START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] or three [START_REF] Kemker | Fearnet: Braininspired model for incremental learning[END_REF] datasets which are generally smaller than the ones used here. The memory K and the number of states Z were shown to be the most important parameters of the class IL algorithms tested here [START_REF] Francisco | End-to-end incremental learning[END_REF]12]. We fix each parameter and vary the other as follows: (1) for Z = 10, we test K = {20000, 10000, 5000, 0} and ( 2) for K = 5000, we test Z = {5, 10, 20}.

A ResNet-18 architecture [START_REF] He | Deep residual learning for image recognition[END_REF] was used in [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] and then in [12] and [START_REF] Francisco | End-to-end incremental learning[END_REF]. We reuse it here with the standard Pytorch version which essentially follows the original implementation from [START_REF] He | Deep residual learning for image recognition[END_REF]. Further details of the training process are provided in the supplementary material.

All methods are evaluated using top-1 accuracy, a metric which is well suited when each image has only one label in the ground truth, as it is the case here. This metric is more informative of the actual performance than the top-5 accuracy which is often used following its introduction in the popular ImageNet challenge [26]. However, to facilitate comparability with class IL results presented in previous works [START_REF] Francisco | End-to-end incremental learning[END_REF]12,[START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF], we also provide top-5 results in the supplementary material.

Discussion of results

The comparison of the methods tested in Table 2 shows that IL2M has the best performance in a wide majority of configurations with memory (K>0). Our method outperforms previous algorithms (iCaRL [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] and DeeSIL [START_REF] Belouadah | Deesil: Deep-shallow incremental learning[END_REF]), FT the vanilla fine tuning baseline and its variants FT NEM and FT BAL , which use the classification components from [START_REF] Francisco | End-to-end incremental learning[END_REF] and [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF].

Among published baselines, FT consequently outperforms iCaRL for Z = 10 and K = {20000, 10000}. For K = 5000, it is better for Z = {5, 20} states and slightly falls behind for ILSVRC with Z = 10 states. Naturally, iCaRL is better when no memory is allowed and distillation reduces catastrophic forgetting. The comparison of FT to DeeSIL [START_REF] Belouadah | Deesil: Deep-shallow incremental learning[END_REF] is also favorable for all settings where K>0, except for Z =5and ILSVRC with Z = 10 and K = {5000, 10000}.

The detailed results for the three datasets with K = 10000 and Z = 10 from Figure 3 confirm the above findings. IL2M has the best performance for a wide majority of IL states. It is also interesting to see that our method provides good results for later incremental states. This is clear for ILSVRC, where IL2M has similar performance with that of FT NEM and DeeSIL for states 7 to 9 and is better than them in earlier states. The gap between iCaRL performance and all FT methods introduced here is large overall and clearly increases for in later states for VGGFace2 and Landmarks. This finding indicates that vanilla FT is a much better base for IL when the number of classes is large.

While our focus is on class IL with a memory, we also present results with no memory (K =0 ). Here distillation clearly has a positive effect and outperforms fine tuning, thus confirming the results from [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF]. All methods derived from FT have the same performance because all score rectification methods rely on exemplars. DeeSIL [START_REF] Belouadah | Deesil: Deep-shallow incremental learning[END_REF] is the best method when K =0because it has low dependence on memory. Except for 20 states, its performance is better than that of iCaRL by a consequent margin. This result is at odds with the conclusion of [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF], where the authors found their fixed representation to be less effective than iCaRL. The difference is explained by the fact that fixed representations of past classes in [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] were learned only with exemplars from the current state. This restriction is unnecessary since the representation is fixed and each class can be learned the first time it is seen without violating memory requirements and then reused across IL states.

When compared to Full, the upper-bound nonincremental learning, the results obtained by all incremental method are lower in all configurations. This is particularly the case for ILSVRC, the hardest task among the three tested, where the gap reaches 16.6 top-1 accuracy points for Z = 10 states and K = 20000. Naturally, this gap grows for all datasets when the memory is reduced. This finding confirms the conclusions of [START_REF] Francisco | End-to-end incremental learning[END_REF][START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] that class IL remains a hard problem if it operates under computational and memory constraints.

Effect of score rectification

IL2M , FT NEM and FT BAL all use vanilla FT with memory as IL backbone. The three methods differ in the way final classification scores are obtained. FT NEM uses the NEM method [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] as external classifier. FT BAL classifier adds a balanced fine tuning step for classification following [START_REF] Francisco | End-to-end incremental learning[END_REF]. IL2M notably exploits the content of statistical memory to rectify scores. The results from Table 2 show that our method yields better performance than FT NEM and FT BAL for almost all configurations tested.

Equally important, IL2M is useful for all memory sizes while this is not the case for NEM in FT NEM , which actually hurts FT performance for Landmarks in three tested Table 2: Top-1 average accuracy (%) for the different methods tested. To test robustness, the available memory (in thousand exemplars) and the number of states are varied to the left and the right of the table. Each time, the other parameter is fixed. Following [START_REF] Francisco | End-to-end incremental learning[END_REF], accuracy is averaged only for incremental states (i.e. excluding the initial, non-incremental state). Full is the non-incremental upper-bound performance obtained with all data available for all classes. Best results are in bold. configurations. The balanced fine tuning in FT BAL also improves performance for all memory sizes but to a lesser extent than IL2M . With lower memory, FT BAL is more prone to catastrophic forgetting than IL2M and FT NEM because a larger extent of data needs to be dropped during balancing. It is noticeable that the usefulness of score rectification grows when exemplar memory is lower and imbalance between past and new classes is consequently higher. For instance, IL2M gains 5.3 and 11.9 top-1 accuracy points for ILSVRC with K = 20000 and K = 5000 exemplars respectively when Z = 10.

Effect of distillation

The results from Table 2 and Figure 3 show that the use of distillation loss is detrimental in class IL if at least a few exemplars per past class are allowed. The ablation of L d in iCaRL to obtain FT NEM is beneficial for all datasets and memory sizes K = {20000, 10000, 5000} and Z = 10.

The results presented here are at odds with the conclusion of [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] about the low performance of vanilla fine tuning in class IL with memory. That conclusion was based on a biased comparison of iCaRL and FT since the first method used an exemplar memory and the second did not. Naturally, distillation is useful when no memory is allowed, the setting for which it was initially designed [15] and which is not in focus here. While we do not have a complete set of results for EtEIL, we note that distillation is also harmful for this method on the ILSVRC dataset with K = 20000. The original top-5 result reported in [START_REF] Francisco | End-to-end incremental learning[END_REF] is 69.4 while the modified FT BAL version introduced here reaches 77.52.

In Table 3, we analyze the behavior of hybrid1, the version of FT with distillation which serves as backbone for iCaRL [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] ) is comparable for the two methods, although slightly higher when distillation is used. Consequently, data imbalance is not the main factor which explains the difference between the two methods. This difference comes mostly from the distribution of wrong classifications between past classes (e(p, p)). While distillation is assumed to preserve accuracy for past classes, the obtained results indicate that hybrid1 makes between two and three times more mistakes than vanilla fine tuning. A possible explanation for this situation is that distillation usually assumed to be initialized with a strong model learned on a large balanced dataset [START_REF] He | Deep residual learning for image recognition[END_REF]. This condition is not met in IL since the models from the previous state are trained on an imbalanced dataset.

Ablation study

We analyze the contribution of the IL2M components in an ablation study with the ILSVRC dataset for Z = 10 states and memory K = {20000, 10000, 5000}. We test the following changes on top of the FT baseline: IL2M 1 -activation of the first component of the rectification which works with class level means; IL2M 2 -activation of the second component which works with model level means; IL2M 1+2 -both mean based components are activated; IL2M -full version in which we also add the restriction of rectifying past class scores only if an image is initially predicted as belonging to a new class (given by Eq. 1).

The results from Table 4 indicate that each component has a positive effect compared to FT. The largest single contribution is the use of class means from statistical memory S in IL2M 1 . The gain is particularly interesting for the lower memory sizes, where the effect of catastrophic forgetting on FT is higher. The model level means have a small positive contribution for K = {20000, 10000} and a slight negative effect for K = 5000. The final restriction of rectification has moderate positive effect in all settings.

Conclusion

We introduce IL2M , a new method designed for class IL with memory. Extensive experiments show that IL2M outperforms very competitive algorithms which are either based on adapted fine tuning [START_REF] Francisco | End-to-end incremental learning[END_REF][START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] or fixed representations [START_REF] Belouadah | Deesil: Deep-shallow incremental learning[END_REF]. IL2M gets significantly better results than existing adapted fine tuning based methods for almost all configurations with memory and falls behind the fixed representation in a single case. The rectification method from Eq. 1 improves FT results in all configurations tested. The balanced fine tuning from [START_REF] Francisco | End-to-end incremental learning[END_REF] is also beneficial, but to a lesser extent. NEM [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] has a mixed effect because it actually hurts performance in some cases. The IL2M ablation study from Subsection 5.4 shows that the obtained gain is mainly due to the use of the statistical memory S introduced here. The method has negligible supplementary cost, both in terms of memory and computation. It is thus fitted for deployment in computationally constrained environments. Interestingly, the largest gains compared to FT, FT NEM and FT BAL are obtained for lower memory sizes. This makes IL2M very interesting from an application perspective since it reduces the memory requirements.

We also find that, surprisingly, vanilla fine tuning is a very effective baseline for class IL with memory. FT compares favorably with existing algorithms [START_REF] Belouadah | Deesil: Deep-shallow incremental learning[END_REF][START_REF] Francisco | End-to-end incremental learning[END_REF][START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF]. The ablation of the distillation component from iCaRL [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF] and end-to-end incremental learning [START_REF] Francisco | End-to-end incremental learning[END_REF] in FT NEM and FT BAL improves the performance of original methods. This improvement of state-of-the-art methods is an interesting by-product of our work. Although IL2M is designed for class IL with memory, we also test it without memory for completeness. As expected, adding a distillation component is beneficial in this configuration. However, the use of fixed representations [START_REF] Belouadah | Deesil: Deep-shallow incremental learning[END_REF] provides the best performance when no memory is allowed and is thus preferable.

We test the proposed method and the baselines with three large scale datasets dedicated to distinct visual tasks and with different memory sizes.

The evaluation setting can be reused to ensure a robust testing of class incremental learning algorithms. The code and dataset details are publicly available at:

https://github.com/EdenBelouadah/ class-incremental-learning.

The reported results reduce the performance gap between incremental and non-incremental learning. However, this gap is still large, especially for the harder visual datasets, such as ILSVRC. The class IL research problem remains an open one if we work under strong computational and memory constraints. We will pursue work along the following lines: (1) test a constant complexity method such as IL2M for multitask IL to replicate real-life scenarios in which more diversified visual content is encountered, (2) enhance vanilla fine tuning by leveraging recent results which improve imbalanced learning [START_REF] Buda | A systematic study of the class imbalance problem in convolutional neural networks[END_REF] and make curriculum learning [7] scalable and (3) explore alternative score rectification methods to further improve performance.

Figure 1 :

 1 Figure 1: Illustration of the proposed IL2M training process. The deep models associated to the three states recognize 2, 4 and 6 classes respectively. The bounded memory includes K =4image exemplars of past classes and is represented on light blue background. The number of class exemplars class stored in memory decreases when adding new classes to keep memory requirements constant. The IL training process is more and more prone to catastrophic forgetting because the dataset is increasingly imbalanced. The second memory S, represented in light pink, stores statistics which are obtained when classes are initially learned. IL2M makes these class statistics usable across different incremental states to rectify the raw prediction scores of past classes in order to make them more comparable to those of new classes. (Best viewed in color.)

Figure 2 :

 2 Figure 2: Prediction scores for the ILSVRC dataset [26] with Z = 10 states and memory K = {20000, 10000, 5000} exemplars. We select the scores of the true class for train images and then average them for past and new classes. Incremental states from 1 to 9 are represented. The initial state (0) does not include past classes and is not represented. (Best viewed in color.)

FT

  BAL 52.1 47.0 37.2 18.3 91.5 88.6 82.1 20.8 93.2 90.2 85.7 21.0 IL2M 56.4 50.8 44.1 18.3 92.0 89.7 86.5 20.8 93.4 90.8 86.9 21..6 74.1 49.5 73.8 52.6 50.9 28.4 89.3 69.3 88.3 74.9 35.4 36.8 85.7 83.3 85.4 84.1 44.1 46.2 87.4 85.7 83.4 84.4 44.7 41.6 87.7 83.9 88.2 84.8 44.9 42.0 90.1 85.7 88.5

Figure 3 :

 3 Figure 3: Top-1 accuracy for object, face and landmark recognition with memory K = 10000 and Z = 10 states. To be aligned with the results from Table 2, only the incremental states are represented. (Best viewed in color.)

  The number of class exemplars class stored in memory decreases when adding new classes to keep memory requirements constant. The IL training process is more and more prone to catastrophic forgetting because the dataset is increasingly imbalanced. The second memory S, represented in light pink, stores statistics which are obtained when classes are initially learned. IL2M makes these class statistics usable across different incremental states to rectify the raw prediction scores of past classes in order to make them more comparable to those of new classes. (Best viewed in color.)

Table 1 :

 1 Summary of the datasets used in evaluation.

	to fa-

  DeeSIL 47.3 47.2 47.0 46.5 81.5 81.3 80.9 80.0 82.8 82.6 82.4 81.2 FT 51.1 42.3 32.2 18.3 91.1 87.6 82.0 20.8 93.2 90.1 84.7 21.0 FT NEM 54.9 49.1 42.8 18.3 91.1 87.6 84.2 20.8 91.1 88.5 84.7 21.0

	States				Z = 10
	Dataset	ILSVRC		VGGFace2		Landmarks
	K	20k 10k	5k	0k	20k 10k	5k	0k	20k 10k	5k	0k
	iCaRL	35.1 33.6 32.9 20.8 66.8 65.3 64.4 26.1 68.9 66.9 65.6 27.0

Table 3 :

 3 Top-1 analysis for hybrid1 the FT with distillation used as backbone for iCaRL[START_REF] Rosenfeld | Incremental learning through deep adaptation[END_REF] and for vanilla FT using Z = 10 and K = 10000. c(•) e(•, •) stand for correct and erroneous predictions and p and n stand for past and new classes. For instance, e(p, p) designates the number of past samples wrongly predicted as other past classes.

Table 4 :

 4 and of vanilla FT for ILSVRC with K = Top-1 average ILSVRC accuracy for different versions of IL2M evaluated in the ablation study with Z = 10 states and memory K = {20000, 10000, 5000}. 10000 images and Z = 10 states. The bias toward new classes (e(p, n)

			Z = 10
	IL Method		K
		20k	10k	5k
	FT	51.13 42.29 32.23
	IL2M 1	53.45 47.64 42.20
	IL2M 2	51.94 43.63 31.74
	IL2M 1+2 55.15 49.57 42.51
	IL2M	56.37 50.82 44.05


Craig Stuart Carlson

Jean De Bruin Jordaan

Ken J Nixon

Michiel Postema

Michiel Postema Quantifying

Quantifying the dynamic hydrophobicity of carbon black tattoo ink owing to ultrasound

• Cuvettes containing 1.5ml each of n-octanol and reverse osmosis distilled, degassed water.

• Zuper black pigment dispersion in a dilution of 0.25‰.

• Custom-built unfocused single-element transducers. Despite growing academic interest of tattoos in society, the reason why tattoos are permanent has predominantly been studied from a biological perspective. In preliminary physics studies of tattoo ink, it has been presumed that the hydrophobicity of the pigment prevents absorption and therefore may be a contributing factor to a tattoo's permanence. According to these early studies, ultrasound might change the hydrophobicity of microparticles. The purpose of this study was to confirm the presence of hydrophobic components in carbon black tattoo ink and to relate sonication to such presence.

Materials and methods

Quantifying hydrophobicity

Conclusions

• Carbon black tattoo ink was found to be very hydrophobic.

• Sonication made the dispersion less hydrophobic, with some dynamic changes observed.

• Influencing the hydrophobicity of tattoo ink might change the permanence of a tattoo.

Results

Hydrophilic Hydrophobic P I < 10 0 P I > 10 0 (extremely: >10 4 )

• Hydrophobicity is quantified by the water-octanol partition coefficient.

• Determined using colourimetry.

octanol water

Light intensity measurements

Attenuation change

Mean differences in absorption coefficients between unsonicated and sonicated dispersions Δα, as a function of time.

Close-up frames of photographs of unsonicated (u) and sonicated (s) ink dispersions in cuvettes.

Light intensity partition coefficients of carbon black tattoo ink as a function of pre-applied ultrasound pressure amplitude. Coefficients have been represented on a logarithmic scale. Cropped frontal photograph of a representative experiment inset, showing a cuvette with no ink, a cuvette with unsonicated ink (green), and ten cuvettes with sonicated ink (red) at different pressure amplitudes.

Quantifying the dynamic hydrophobicity of carbon black tattoo ink owing to ultrasound Craig

  S. Carlson, Jean de Bruin Jordaan, Ken J. Nixon, and Michiel Postema

Poster number 72 : Craig.Carlson@tuni.fi

Research funding: This work was supported by the National Research Foundation of South Africa, Grant No. 127102 , and by the Academy of Finland, Grant No. 340026. Conflict of interest: Authors state no conflict of interest. Ethical approval: Ethical clearance was obtained from the Human Research Ethics Committee (Medical) of the University of the Witwatersrand, Johannesburg (clearance certificate no. M190808 MED19-07-006). from volunteer participants where required.


Jocelyn Aznar

Christian Döhler

Jozina Vander Klok

Preface to the Special Issue of NUSA on applicatives in Austronesian languages 1

In the grammar of many Austronesian and Papuan languages, applicatives are a prevalent morpho-syntactic construction, but also play a role beyond its morpho-syntactic function of promoting a peripheral semantic role to direct object as the "applied phrase". Work investigating the morphosyntax of applicative constructions in Austronesian languages in particular has explored the types of semantic roles applicative morphology targets, the role of stacking or doubling of different applicative morphology, as well as its role in relation to its use in deriving causative constructions (e.g., [START_REF] Donohue | Coding choices in argument structure: Austronesian applicatives in texts[END_REF][START_REF] Kroeger | Morphosyntactic vs. morphosemantic functions of Indonesian -kan[END_REF][START_REF] Son | An Event-Based Account of -kan Constructions in Standard Indonesian[END_REF][START_REF] Chang | Extractions in Tsou causative applicatives[END_REF][START_REF] Nie | Raising Applicatives and Possessors in Tagalog[END_REF], among many others). In addition, several authors have explored non-syntactic functions of applicative morphology (cf. Pacchiarotti & Zúñiga 2022). Among these functions are pluractionality, iteration, topicality of the applied phrase, specificity of the applied phrase, completeness of the event or affectedness of the applied phrase. These functions, while less studied, seem to be common cross-linguistically. Moreover, applicative morphology and structures are an important avenue of research with respect to pathways of change and variation into and/or beyond applicatives (e.g., [START_REF] Peterson | Applicative constructions[END_REF], which provide insights into better understanding the relations across the languages of this region.

There are four papers that address different topics within this theme: The first two papers, by Aldridge and by Krauße and François, both take a diachronic approach in investigating the development of applicative morphology, but in different areas of the Austronesian language family and also from different perspectives of historical development. Aldridge's paper proposes reconstructions and diachronic developments of applicative constructions. It focuses on the Nuclear Austronesian subgroup, with an emphasis on Formosan languages and dialects (16 of them) and with data coming from Philippine language as well. Krauße and François examine Oceanic languages spoken in Vanuatu, distantly related to the languages in Aldridge's paper, and show five different pathways of *-akin, one into the applicative use, and four pathways beyond the prototypical applicative use.

The last two papers, by Avelino and by Rajeg and Arka, are case studies on the applicative construction in two different languages, also spanning different areas of the Austronesian language family. Avelino shows how case and pivot morphology in Tboli gives important clues as to the structure of applicative constructions in this Philippine-type language. He analyzes both applicatives in Tboli as low applicatives, where the applied argument is structurally introduced within the VP (Verb Phrase), within the Generative grammar framework. Rajeg and Arka's paper moves to Indonesian, and investigates the use of the verb beli 'buy' in the -kan applicative construction across Actor Voice and Passive Voice, using a usage-based and constructional grammar approach. One of the main results is that the applicative construction with beli 'buy' in usage is most often realized with only one overt internal argument (as monotransitive), in comparison to the typically reported double-object construction as the standard. Moreover, Rajeg and Arka found that the use of this verb in the Passive Voice (di-beli-kan) shows a unique argument realization with the role of RATE used as the subject argument, which is otherwise not found in Actor Voice.

i.

  Diachronic development of the Nuclear Austronesian locative and patient applicatives by Edith Aldridge ii. The road to opacity: Reflexes of Proto-Oceanic *-akin in northern Vanuatu by Daniel Krauße and Alexandre François iii. Usage-based perspective on argument realisation: A corpus study of Indonesian BUY verbs in applicative construction with -kan by Gede Primahadi Wijaya Rajeg and I Wayan Arka iv. Case, Pivothood, and the Tboli Applicative by Gérard Avelino This special thematic issue of NUSA on Applicatives in Austronesian Languages arose out of a thematic session as part of the 14th annual Austronesian and Papuan Languages and Linguistics conference (APLL14). The event was co-hosted by Leibniz Centre of General Linguistics (ZAS) and the Humboldt University of Berlin on June 9-11, 2022. This special issue is also supplemented by the article by Rajeg and Arka, originally presented at the workshop on Austronesian argument alternations in discourse at the Australian National University [ANU], November 28-29, 2022, co-hosted by ANU and the University of Oslo.

The editors of this special issue would like to thank and acknowledge Karlina Denistia, Bethwyn Evans, Harold Koch, Simon Musgrave, Yining Nie, Sonja Riesberg, Stefan Schnell, and one anonymous reviewer for their helpful comments and constructive criticism on the manuscripts.


Vincenzo Lomonaco
email: vincenzo.lomonaco@unipi.it

Lorenzo Pellegrini

Andrea Cossu

Antonio Carta

Gabriele Graffieti

Tyler L Hayes

Matthias De Lange

Marc Masana

Jary Pomponi

Gido M Van De Ven

Martin Mundt

Qi She

Keiland Cooper

Jeremy Forest

Eden Belouadah

Simone Calderara

German I Parisi

Fabio Cuzzolin

Andreas S Tolias

Simone Scardapane

Luca Antiga

Subutai Ahmad

Adrian Popescu

Christopher Kanan

Joost Van De Weijer

Tinne Tuytelaars

Davide Bacciu

Davide Maltoni

Avalanche: an end-to-end library for continual learning

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Continual Learning (CL), also referred to as Lifelong or Incremental Learning, is a challenging research problem [START_REF] Chen | Lifelong Machine Learning, Second Edition[END_REF]. Lately, it has become the object of fast-growing interest from the research community, especially thanks to recent investigations leveraging gradient-based deep architectures [START_REF] Hadsell | Embracing Change: Continual Learning in Deep Neural Networks[END_REF][START_REF] German I Parisi | Continual lifelong learning with neural networks: A review[END_REF]. In the last few years, machine learning has witnessed a prolific, variegated, and original research production on the topic: from Computer Vision [START_REF] Delange | A continual learning survey: Defying forgetting in classification tasks[END_REF][START_REF] Lomonaco | Continual Learning with Deep Architectures[END_REF][START_REF] Masana | Classincremental learning: survey and performance evaluation[END_REF] Figure 1: Operational representation of Avalanche with its main modules (top), the main object instances (middle) and the generated stream of data (bottom). A Benchmark generates a stream of experiences e i which are sequentially accessible by the continual learning algorithm A CL with its internal model M . The Evaluator object directly interacting with the algorithm provides a unified interface to control and compute several performance metrics (p i ), delegating results logging to the Logger(s) objects.

to Robotics [START_REF] Lesort | Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges[END_REF][START_REF] German | Online continual learning on sequences[END_REF][START_REF] Pellegrini | Latent replay for real-time continual learning[END_REF], from Reinforcement Learning [START_REF] Khetarpal | Towards continual reinforcement learning: A review and perspectives[END_REF][START_REF] Lomonaco | Continual Reinforcement Learning in 3D Non-Stationary Environments[END_REF] to Sequence Learning [START_REF] Cossu | Continual learning for recurrent neural networks: an empirical evaluation[END_REF], among others. However, continual learning algorithms today are often designed and implemented from scratch with different assumptions, settings, and benchmarks that make them diffi-cult to compare among each other or even port to slightly different contexts. A crucial factor for the consolidation of a fast-growing research topic in the machine learning domain is the availability of tools and libraries easing the implementation, assessment, and replication of models across different settings, while promoting the reproducibility of results from the literature [START_REF] Pineau | Improving reproducibility in machine learning research (a report from the neurips 2019 reproducibility program[END_REF].

In this work, we propose Avalanche, an open-source (MIT licensed) end-to-end library for continual learning based on PyTorch [START_REF] Paszke | Pytorch: An imperative style, high-performance deep learning library[END_REF], devised to provide a shared and collaborative codebase for fast prototyping, training, and evaluation of continual learning algorithms.

We designed Avalanche according to a set of fundamental design principles (Sec. 2) which, we believe, can help researchers and practitioners in a number of ways: i) Write less code, prototype faster, and reduce errors; ii) Improve reproducibility; iii) Favor modularity and reusability; iv) Increase code efficiency, scalability and portability; v) Foster impact and usability.

The contributions of this paper can be summarized as follows:

1. We propose a general continual learning framework that provides the conceptual foundation for Avalanche (Sec. 3). 2. We discuss the general design of the library based on five main modules: Benchmarks, Training, Evaluation, Models, and Logging (Sec. 4). 3. We release the open-source, collaboratively maintained project at https : / / github . com / ContinualAI/avalanche, as the result of a collaboration powered by the non-profit ContinualAI 1 and involving over 15 organizations across Europe, United States, and China. Prior work related to this project is discussed in Section 5. Lastly, we discuss in Section 6 the importance that an all-in-one, community-driven library may have for the future of continual learning and its possible extensions.

Design Principles

Avalanche has been designed with five main principles in mind: i) Comprehensiveness and Consistency; ii) Ease-of-Use; iii) Reproducibility and Portability; iv) Modularity and Independence; v) Efficiency and Scalability. These principals, we argue, are important for any continual learning project but become essential for tackling the most interesting research challenges and real-world applications.

Comprehensiveness and consistency

The main design principle for Avalanche follows from the concept of comprehensiveness, the idea of providing an exhaustive and unifying library with end-to-end support for continual learn-Ease-of-Use The second principle is the focus on simplicity: simple solutions to complex problems and a simple usage of the library. We concentrate our efforts on the design of an intuitive Application Programming Interface (API), an official website, and rich documentation with a curated list of executable notebooks and examples 2 .

Reproducibility and Portability

Reproducing research paper results is a difficult but much needed task in machine learning [START_REF] Hutson | Artificial intelligence faces reproducibility crisis[END_REF]. The problem is exacerbated in continual learning. A critical design objective of Avalanche is to allow experimental results to be seamlessly reproduced. This allows researchers to simply integrate their own original research into the shared codebase and compare their solution with the existing literature, forming a virtuous circle. Hence, reproducibility is not only a core objective of sound and consistent scientific research, but also a means to speed up the development of original continual learning solutions.

Modularity and Independence Modularity is another fundamental design principle. In Avalanche, simplicity is sometimes bent in favor of modularity and reusability. This is essential for scalability and to collaboratively bring the codebase to maturity. A particular focus on module independence is maintained to guarantee the stand-alone usability of individual module functionalities and facilitates learning of a particular tool.

Efficiency and Scalability Computational and memory requirements in machine learning have grown significantly throughout the last two decades [START_REF] Neil C Thompson | The computational limits of deep learning[END_REF]. Standard deep learning libraries such as TensorFlow [START_REF] Abadi | Tensor-Flow: Large-scale machine learning on heterogeneous systems[END_REF] or PyTorch [START_REF] Paszke | Pytorch: An imperative style, high-performance deep learning library[END_REF] already focus on efficiency and scalability as two fundamental designing principles, since modern research experiments can take months to complete [START_REF] Schwartz | Green ai[END_REF]. Avalanche is based on the same principles: offering the end-user a seamless and transparent experience regardless of the use-case or the hardware platform that the library is run on.

Continual Learning Framework

Recently, we have witnessed a significant attempt to formalize a general framework for continual learning algorithms [START_REF] Lomonaco | Continual Learning with Deep Architectures[END_REF][START_REF] Lomonaco | Continual Reinforcement Learning in 3D Non-Stationary Environments[END_REF][START_REF] Gido M Van De Ven | Three scenarios for continual learning[END_REF]. These proposals often categorize scenarios and algorithms based on their unique properties and specific settings. However, as outlined in this paper, within the formal design of Avalanche, we take a different approach.

Given the fast-evolving, often conflicting views of the problem, we aim to lower the number of assumptions to a minimum, favoring simplicity and flexibility. In practice, this translates into providing users with a set of building blocks that can be used in any continual learning solution without imposing any strong nomenclature, constraining abstractions, or assumptions.

In Avalanche, data is modeled as an ordered sequence (or stream) composed of n, usually non-iid, learning experiences: e 0 , e 1 , . . . , e n .

A learning experience is a set composed of one or multiple samples which can be used to update the model. This is often referred to in the literature as batch or task. This formulation is general enough to be used in several continual learning contexts, such as supervised, reinforcement, or unsupervised continual learning. Avalanche provides a general set of abstractions that do not impose any particular constraints on the content of the experiences. For example, in a supervised training regime, each learning experience e i can be seen as a set of triplets x i , y i , t i , where x i and y i represent an input and its corresponding target, respectively, while t i is the task label, which may or may not be available.

During training, a continual learning algorithm A CL processes experiences sequentially and uses them to update the model and its internal state. In Avalanche, each algorithm has a training mode, used to update the model, and an evaluation mode, which may be used to process streams of experiences for testing purposes.

The continual learning framework we propose can be formalized as follows.

Definition (Continual Learning Framework). A continual

Learning algorithm A CL is expected to update its internal state (e.g. its internal model M or other data structures) based on the sequential exposure to a non-stationary stream of experiences (e 1 , . . . , e n ). The objective of A CL is to improve its performance on a set of metrics (p 1 , . . . , p m ) as assessed on a test stream of experiences (e t 1 , . . . , e t n ).

Main Modules

The library is organized into five main modules: Benchmarks (Sec. 4.3), Models (Sec. 4.5), and Logging (Sec. 4.4). Table 1 summarizes the features provided by Avalanche at the current stage of development. In Fig. 1, an operational representation of the library modules and their interplay within the aforementioned reference framework is shown.

Benchmarks

Continual learning revolves around the idea of dealing with a non-stationary stream of experiences. An example stream from the standard SplitMNIST benchmark [START_REF] Zenke | Continual Learning Through Synaptic Intelligence[END_REF] composed of five experiences is shown in Fig. 2. A target system powered by a continual learning strategy is required to learn from experiences (e.g., by considering additional classes in a class-incremental setting [START_REF] Masana | Classincremental learning: survey and performance evaluation[END_REF]) in order to improve its performance or expand its set of capabilities. This means that the component in charge of generating the data stream is usually the first building block of a continual learning experiment. It is no surprise that a considerable amount of time is spent defining and implementing the data loading module. The benchmarks module offers a powerful set of tools one can leverage to greatly simplify this process.

The term benchmark is used in Avalanche to describe a recipe that specifies how the stream of data is created by defining the originating dataset(s), the contents of the stream, the amount of examples, task labels and boundaries [START_REF] Aljundi | Task-Free Continual Learning[END_REF], etc. When defining such elements, some degree of freedom is retained to allow obtaining different benchmark instances. For example, different instantiations of the SplitMNIST benchmark [START_REF] Zenke | Continual Learning Through Synaptic Intelligence[END_REF] can be obtained by setting different class assignments. Alternatively, distinct instances of the PermutedMNIST [START_REF] Goodfellow | An empirical investigation of catastrophic forgetting in gradient-based neural networks[END_REF] benchmark can be obtained by choosing different pixel permutations.

The benchmarks module is designed with the idea of providing an extensive set of out-of-the-box loaders covering the most common benchmarks (i.e. SplitCIFAR [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF], PermutedMNIST [START_REF] Goodfellow | An empirical investigation of catastrophic forgetting in gradient-based neural networks[END_REF], etc.) through the classic submodule. A simple example illustrating how to use the "SplitM-NIST" benchmark [START_REF] Zenke | Continual Learning Through Synaptic Intelligence[END_REF] is shown in Fig. 3. Moreover, a wide

Supported features

Benchmarks Split/Permuted/Rotated MNIST [START_REF] Lopez | Gradient Episodic Memory for Continual Learning[END_REF], Split Fashion Mnist [START_REF] Farquhar | A Unifying Bayesian View of Continual Learning[END_REF], Split Cifar10/100/110 [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF][START_REF] Maltoni | Learning in Single-Incremental-Task Scenarios[END_REF],

Split CUB200, Split ImageNet [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF], Split TinyImageNet [START_REF] Delange | A continual learning survey: Defying forgetting in classification tasks[END_REF], Split/Permuted/Rotated Omniglot [START_REF] Schwarz | Progress & Compress: A scalable framework for continual learning[END_REF], CORe50 [START_REF] Lomonaco | CORe50: A New Dataset and Benchmark for Continuous Object Recognition[END_REF], OpenLORIS [START_REF] She | OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning[END_REF], Stream51 [START_REF] Roady | Stream-51: Streaming classification and novelty detection from videos[END_REF].

Scenarios

Multi Task [START_REF] Lesort | Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges[END_REF], Single Incremental Task [START_REF] Lesort | Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges[END_REF], Multi Incremental Task [START_REF] Lesort | Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges[END_REF], Class incremental [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF][START_REF] Gido M Van De Ven | Three scenarios for continual learning[END_REF], Domain Incremental [START_REF] Gido M Van De Ven | Three scenarios for continual learning[END_REF], Task Incremental [START_REF] Gido M Van De Ven | Three scenarios for continual learning[END_REF], Task-agnostic, Online, New Instances, New Classes, New Instances and Classes.

Strategies

Naive (Finetuning), CWR* [START_REF] Lomonaco | Rehearsal-Free Continual Learning over Small Non-I.I.D. Batches[END_REF], Replay, GDumb [START_REF] Prabhu | GDumb: A Simple Approach that Questions Our Progress in Continual Learning[END_REF], Cumulative, LwF [START_REF] Li | Learning without Forgetting[END_REF], GEM [START_REF] Lopez | Gradient Episodic Memory for Continual Learning[END_REF], A-GEM [START_REF] Chaudhry | Efficient Lifelong Learning with A-GEM[END_REF], EWC [START_REF] Kirkpatrick | Overcoming catastrophic forgetting in neural networks[END_REF], Synaptic Intelligence [START_REF] Zenke | Continual Learning Through Synaptic Intelligence[END_REF], AR1 [START_REF] Maltoni | Learning in Single-Incremental-Task Scenarios[END_REF], Joint Training.

Metrics

Accuracy, Loss (user specified), Confusion Matrix, Forgetting, CPU Usage, GPU usage, RAM usage, Disk Usage, Timing, Multiply, and Accumulate [START_REF] Jeangoudoux | A Correctly Rounded Mixed-Radix Fused-Multiply-Add[END_REF][START_REF] Díaz-Rodríguez | Don't forget, there is more than forgetting: New metrics for Continual Learning[END_REF].

Loggers

Text Logger, Interactive Logger, Tensorboard Logger [START_REF] Abadi | Tensor-Flow: Large-scale machine learning on heterogeneous systems[END_REF], Weights and Biases (W&B) Logger [START_REF] Biewald | Experiment tracking with weights and biases[END_REF] (in progress).

Table 1: Avalanche supported features for the Alpha release (v0.0.1).

Classic Benchmarks range of tools are available that enables the creation of customized benchmarks. The goal is to provide full support to researchers implementing benchmarks that do not easily fit into the existing categories.

Most out-of-the-box benchmarks are based on the dataset implementation provided by the torchvision library. A proper implementation is provided for other datasets (such as CORe50 [START_REF] Lomonaco | CORe50: A New Dataset and Benchmark for Continuous Object Recognition[END_REF], Stream-51 [START_REF] Roady | Stream-51: Streaming classification and novelty detection from videos[END_REF], and Open-LORIS [START_REF] She | OpenLORIS-Object: A Robotic Vision Dataset and Benchmark for Lifelong Deep Learning[END_REF]). The benchmark preparation and data loading process can seamlessly handle memory-intensive benchmarks, such as Split-ImageNet [START_REF] Sylvestre-Alvise | icarl: Incremental classifier and representation learning[END_REF], without the need to load the whole dataset into memory in advance.

Further, the benchmarks module is entirely standalone, meaning that it can be used independently from the rest of Avalanche.

Benchmarks creation

The benchmarks module exposes a uniform API that makes it easy to define a new continual learning benchmark.

The classic package hosts an ever-growing set of common benchmarks and is expected to cover the usage requirements of the vast majority of researchers. However, there are situations in which implementing a novel benchmark is required. Avalanche offers a flexible API that can be used to easily handle this situation.

Starting from the higher-level API, Avalanche offers explicit support for creating benchmarks that fit one of the ready-to-use scenarios. The concept of scenario is slightly different from that of 'benchmark' as it describes a more general recipe independent of a specific dataset. If the benchmark to be implemented fits either in the New Instances or New Classes scenarios [START_REF] Maltoni | Learning in Single-Incremental-Task Scenarios[END_REF], one can consider using one of the specific generators nc scenario or ni scenario. Both generators take a pair of train and test datasets and produce a benchmark instance. The New Classes generator splits all the available classes in a number of subsets equal to the required number of experiences. Patterns are then allocated to each experience by assigning all patterns of the selected classes. This means that the New Classes generator can be used as a basis to set up Task or Class-incremental learning benchmarks [START_REF] Gido M Van De Ven | Three scenarios for continual learning[END_REF]. The New Instances generator splits the original training set by creating experiences containing an equal amount of patterns using a random allocation schema. The main intended usage for this generator is to help in setting up Domain-Incremental learning benchmarks [START_REF] Gido M Van De Ven | Three scenarios for continual learning[END_REF]. Most classic benchmarks are based on these generators. Fig. 4 shows a simple example of using nc scenario.

If the benchmark does not fit into a predefined scenario, a generic generator can be used. At the moment, Avalanche allows users to create benchmark instances from lists of files, Caffe-style filelists [START_REF] Jia | Caffe: Convolutional architecture for fast feature embedding[END_REF], lists of PyTorch datasets, or even directly from Tensors. We expect that the number of generic generators will rapidly grow in order to cover the most common use cases and allow for maximum flexibility. [6] proposes a benchmark where a separate stream of data is used for cross-validation, while [START_REF] Roady | Stream-51: Streaming classification and novelty detection from videos[END_REF] defines an out-ofdistribution stream used to evaluate the novelty detection capabilities of the model.

Streams and Experiences

Motivated by this remark, we decided to model benchmark instances as a composition of streams of experiences. This choice has two positive effects on the resulting API. Firstly, the way streams and experiences can be accessed is shared across all benchmark instances. Secondly, this modeling of benchmark instances does not force any preconceived schema upon researchers. Avalanche leaves the semantic aspects regarding the definition and usage of each stream to the creator of the benchmark.

A simple example showing the versatility of this design choice concerns the test stream: in order to allow for a proper evaluation of a continual learning strategy, benchmarks do not only need to describe the stream of training experiences but also need to properly describe a testing protocol. Such protocol is, in turn, based on one or more test datasets on which appropriate metrics can be computed. In many cases, the test data may need to be structured into a sequence of 'test experiences', analogously to what happens with the training data stream. For instance, in Class-Incremental learning the test set may be split into different experiences each containing only test patterns related to classes in the corresponding training experience.

Avalanche currently supports two different streams: train and test, while the support for arbitrary streams (for instance, out-of-distribution stream) will be implemented in the near future.

Each experience can be obtained by iterating over one of the available streams. Fig. 5 shows how, starting from a benchmark instance, streams can be retrieved and used. Each experience carries a PyTorch dataset, task label(s) and other useful benchmark-specific information that can be used for introspection. An experience also carries a numerical identifier that defines its position in the originating stream. In fact, experiences in a stream can be also accessed by index. This functionality makes it easy to couple related experiences from different streams. Task Labels and Nomenclature Every mechanism, internal aspect, name of function and class in the benchmarks module were designed with the intent of keeping Avalanche as neutral as possible with respect to the presence of task labels. Task boundaries, task descriptors and task labels are widely used in the continual learning literature to define both semantic and practical aspects of a benchmark. However, the meaning of those concepts is usually blurred with the definition of the specific benchmark to which they are applied to, making it hard to clearly pin-point a generic way to manage them.

Main Training Loop

Based on this observation, and due to the fact that the usage of task-specific information may become more extravagant or sophisticated in the future, we decided that Avalanche should not force any kind of convention. This means that the choice of whether to use task labels and how to use them is left to the user. Following this idea, the GenericCLScenario class, which is the common class for all scenarios instances, allows researchers to assign task labels at pattern granularity, thus allowing for experiences with zero or more task labels. We deemed this the most natural choice for Avalanche: we believe that a continual learning library should not constrain researchers by superimposing a certain view of the field upon them. Instead, the idea of enabling the user to create complex setups in a simple way, without forcing a subjective interpretation, will probably prove to be more robust as the field continues to evolve.

Training

The training module implements both popular continual learning strategies and a set of abstractions that make it easy for a user to implement custom continual learning algorithms. Each strategy in Avalanche implements a method for training (train) and a method for evaluation (eval), which can work either on single experiences or on entire slices of the data stream. Currently, Avalanche provides 11 continual learning strategies (with many more to come), that can be used to train baselines in a few lines of code, as shown in Fig. 6. See Table 1 Under the hood, BaseStrategy deals with most of the boilerplate code. The generic loops are able to seamlessly handle common continual learning scenarios, independently of differences such as the presence or absence of task labels.

Plugin System BaseStrategy provides a simple callback mechanism. This is used by strategies, metrics, and loggers to interact with the training loop and execute their code at the correct points using a simple interface and provides an easy interface to implement new strategies by adding custom code to the generic loops. BaseStrategy provides the global state (current mini-batch, logits, loss, ...) to suitable plugins so that they can access all the information they need. In practice, most strategies in Avalanche are implemented via plugins. This approach has several advantages compared to a custom training loop. Firstly, the readability of the code is enhanced since most strategies only need to specify a few methods. Secondly, this allows for multiple strategies to be combined together. For example, we can define a hybrid strategy that uses Elastic Weight Consolidation (EWC) [START_REF] Kirkpatrick | Overcoming catastrophic forgetting in neural networks[END_REF] and replay using the snippet of code shown in Fig. 8.

Hybrid Strategies

Evaluation

The performance of a CL algorithm should be assessed by monitoring multiple aspects of the computation [START_REF] Lesort | Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges[END_REF]. The evaluation module offers a wide set of metrics to evaluate experiments. Avalanche's design principle is to separate the issues of what to monitor and how to monitor it: the evaluation module provides support for the former through the metrics, while the logging module fulfills the latter (Section 4.4). Metrics do not specify in which format their output should be displayed, while loggers do not alter metrics logic. Metrics can work even without a logger: the strategy's train and eval methods return a dictionary with all the metrics logged during the experiment.

Few lines of code are sufficient to monitor a vast set of metrics: accuracy, loss, catastrophic forgetting, confusion matrix, timing, ram/disk/CPU/GPU usage and Multiply and Accumulate [START_REF] Jeangoudoux | A Correctly Rounded Mixed-Radix Fused-Multiply-Add[END_REF] (which measures the computational cost of the model's forward pass in terms of floating point operations). Each metric comes with a standalone class and a set of plugin classes aimed at emitting metric values on specific moments during training and evaluation.

Stand-alone Metrics Stand-alone metrics are meant to be used independently of all Avalanche functionalities. Each metric can be instantiated as a simple Python object. The metric will keep an internal state to store metric values. The state can be reset, updated or returned to the user by calling the related reset, update and result methods, respectively.

Plugin Metrics Plugin metrics are meant to be easily integrated into the Avalanche training and evaluation loops. Plugin metrics emit a curve composed by multiple values at different points in time. Usually, plugin metrics emit values after each training iteration, training epoch, evaluation experience or at the end of the entire evaluation stream. For example, EpochAccuracy reports the accuracy over all training epochs, while ExperienceLoss produces as many curves as the number of experiences. Each curve monitors the evaluation accuracy of an experience at the end of each training loop. Avalanche recommends the use of already implemented helper functions to simplify the creation of each plugin metric. The output of these functions can be passed as parameters directly to the EvaluationPlugin.

Evaluation Plugin EvaluationPlugin is the component responsible for the orchestration of both plugin metrics and loggers. Its role is to gather metrics outputs and forward them to the loggers during training and evaluation loops. All the user has to do to keep track of an experiment is to provide the strategy object with an instance of the EvaluationPlugin with the target metrics and loggers as parameters. Fig. 9 shows how to use the evaluation plugin and metric helper functions.

Avalanche's effort to monitor different facets of performance aims at enabling a wider experimental assessment, which is too often focused only on the forgetting of previous knowledge [START_REF] Díaz-Rodríguez | Don't forget, there is more than forgetting: New metrics for Continual Learning[END_REF] InteractiveLogger and TensorboardLogger [START_REF] Abadi | Tensor-Flow: Large-scale machine learning on heterogeneous systems[END_REF]. They provide reports on textual file, standard output and Tensorboard, respectively. As soon as a metric emits a value, the Text Logger prints the complete metric name followed by its value in the destination file. The InteractiveLogger reports the same output as TextLogger, but it also uses the tqdm package3 to display a progress bar during training and evaluation. TensorboardLogger is able to show images and more complex outputs, which cannot be appropriately printed on standard output or textual file. We are also working on the integration of the Weights and Biases logger [START_REF] Biewald | Experiment tracking with weights and biases[END_REF], which should be released soon. Integrating loggers into both training and evaluation loops is straightforward. Once created, loggers have to be passed to the EvaluationPlugin, which will be in charge of redirecting metrics outputs to each logger during the experiment. See Fig. 9 for an example of loggers creation.

Users can easily design their own loggers by extending the class StrategyLogger, which provides the necessary interface to interact with the EvaluationPlugin.

Models

The Avalanche models module offers a set of simple machine learning architectures ready to be used in experiments. In particular, the module contains versions of feedforward and convolutional neural networks and a pretrained version of MobileNet (v1) [START_REF] Howard | MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications[END_REF]. The main purpose of these models is to let the user focus on Avalanche features, rather than on writing lines of code to build a specific architecture. We plan to extend our model support with more advanced architectures, also tailored to specific CL applications.

Related Works

Reproducibility is one of the main principles upon which Avalanche is based. Experiments in the continual learning field are often challenging to reproduce, due to the different implementations of protocols, benchmarks and strategies by different authors. This issue of insufficient reproducibility is not limited to continual learning. The whole artificial intelligence community is affected; a number of authors have recently discussed some possible solutions to the problem [START_REF] Hartley | dtoolai: Reproducibility for deep learning[END_REF][START_REF] Pineau | Improving reproducibility in machine learning research (a report from the neurips 2019 reproducibility program[END_REF][START_REF] Raff | A step toward quantifying independently reproducible machine learning research[END_REF].

The advent of machine (and deep) learning libraries, mainly TensorFlow [START_REF] Abadi | Tensor-Flow: Large-scale machine learning on heterogeneous systems[END_REF] and PyTorch [START_REF] Paszke | Pytorch: An imperative style, high-performance deep learning library[END_REF] has partially mitigated the reproducibility problem. Using these libraries assures a standard implementation of many machine learning building blocks, reducing ambiguities due to bespoke and different implementations of basic concepts.

In recent times, the continual learning community has put a lot of effort into addressing these problems, by providing code and libraries aimed to increase the reproducibility of continual learning experiments [START_REF] Delange | A continual learning survey: Defying forgetting in classification tasks[END_REF][START_REF] Hsu | Re-evaluating continual learning scenarios: A categorization and case for strong baselines[END_REF][START_REF] Masana | Classincremental learning: survey and performance evaluation[END_REF][START_REF] Serra | Overcoming catastrophic forgetting with hard attention to the task[END_REF][START_REF] Gido M Van De Ven | Generative replay with feedback connections as a general strategy for continual learning[END_REF][START_REF] Gido M Van De Ven | Three scenarios for continual learning[END_REF]. On the one hand, these first attempts lack the generality and the consistency of Avalanche, especially regarding the creation of different and complex benchmarks, and the continual support of a large community. On the other hand, they demonstrate, however, the growing interest of the entire community towards these issues.

Another area other than continual learning which has recently seen a proliferation of libraries and tools similar in spirit to Avalanche is reinforcement learning (RL). One of the most popular such benchmark RL libraries is OpenAI Gym [START_REF] Brockman | Openai gym[END_REF], within which a multitude of different RL environments is available. A similar library is ViZDoom [START_REF] Wydmuch | Vizdoom competitions: Playing doom from pixels[END_REF], in which an agent plays the famous computer game Doom. Other relevant projects in the field of reinforcement learning are Dopamine [START_REF] Samuel Castro | Dopamine: A Research Framework for Deep Reinforcement Learning[END_REF], which focuses on simplicity and easy prototyping, and project Malmo [START_REF] Johnson | The malmo platform for artificial intelligence experimentation[END_REF], which is based on the famous Minecraft game. Many of these libraries, however, only focus on the agent's interaction with the environment. This problem is addressed by other libraries that include standard implementations of baselines algorithms, such as OpenAI baselines [START_REF] Dhariwal | Openai baselines[END_REF] and stable baselines [START_REF] Hill | Stable baselines[END_REF].

Another prominent example of a collection of baseline training strategies and pretrained models is the natural language processing transformers library by Hugging Face [START_REF] Wolf | Transformers: State-of-the-art natural language processing[END_REF]. Many basic concepts upon which Avalanche is based (e.g. plugins, loggers, benchmarks) can also be found in more general machine learning libraries such as PyTorch Lighting [START_REF] Wa Falcon And | Pytorch lightning[END_REF] and fastai [START_REF] Howard | Fastai: A layered api for deep learning[END_REF].

Another important problem in research is the bookkeeping of experiments. As discussed in Sec. 4.4, Avalanche already implements a fine-grained and punctual logging, which allows to visualize and save the results of different experiments. Moreover, Avalanche could be easily integrated with standalone libraries specifically developed for experiments bookkeeping and visualization, such as Sacred [START_REF] Greff | The Sacred Infrastructure for Computational Research[END_REF] or Weights and Biases (wandb) [START_REF] Biewald | Experiment tracking with weights and biases[END_REF].

The motivations and the community needs behind the developing of Avalanche were reinforced by the recent publication of similar continual learning libraries. Continuum [START_REF] Douillard | Continuum: Simple management of complex continual learning scenarios[END_REF] focuses on data loading and processing, and could be related with the functionalities provided by the benchmark module of Avalanche. Sequoia4 is based on Continuum for the data loading, but, in addition, it provides a playground for research at the intersection of Continual, Reinforcement, and Self-Supervised Learning.

Conclusion and Future Work

In the last decade, we have witnessed a significant effort towards making research in machine learning more transparent, reproducible and open-access. However, although research papers are increasingly accompanied by publicly hosted codebases, it is often difficult to run and integrate such software into environments which are typically different from the one within which it was originally designed. This not only hampers reproducibility but also inhibits scalability, for each research paper ends up creating its own implementation almost from scratch. Avalanche aims to provide a coherent, end-to-end, easily extendable library for continual learning research and development; a solid reference point and shared resource for researchers and practitioners working on continual learning and related areas.

As reported in Table 1, the current Avalanche Alpha version focuses on continual supervised learning for vision tasks, as a significant amount of deep learning research in this area was designed and assessed in this context [START_REF] Hadsell | Embracing Change: Continual Learning in Deep Neural Networks[END_REF]. However, being Avalanche a community-driven effort, we plan in both the near and medium terms to support the integration of additional learning paradigms (e.g. Reinforcement or Unsupervised Learning), tasks type (e.g. Detection, Segmentation) and application contexts (e.g. Natural Language Processing, Speech Recognition), depending on the research community demands and priorities.

We hope that this library, as a powerful avalanche, may trigger a positive reinforcement loop within our community, nudging it to shift towards a more collaborative and inclusive research environment and helping all of us tackle together the grand research challenges presented by a frontier topic such as continual learning.

  4.1), Training (Sec. 4.2), Evaluation (Sec.

Figure 2 :

 2 Figure 2: Example of a generated stream in Avalanche, composed by five experiences, implementing the common SplitMNIST benchmark [56]. When accessing experience e 3 , the A CL algorithm has no access to previous or future experiences.

1 benchmark_instance = SplitMNIST( 2 n_experiences=5, seed=1) 3 # Other useful parameters 4 # return_task_id=False/True 5 #

 12345 fixed_class_order=[5, 0, 9, ...]

Figure 3 :

 3 Figure 3: Simple instantiation of a Classic continual learning benchmark.

Figure 4 :

 4 Figure 4: Example of the "New Classes" benchmark generator on the MNIST dataset.

1 train_stream

 1

3 4for

 3 idx, experience in enumerate(train_stream):

Figure 5 :

 5 Figure 5: Example of the main training loop over the stream of experiences.

2 criterion, mem_size) 3 forFigure 6 :Figure 7 :

 2367 Figure 6: Simple instantiation of an already available strategy in Avalanche.

1 replay

 1

Figure 8 :

 8 Figure 8: Example of an on-the-fly instantiation of hybrid strategies through Plugins.

  .

4. 4 .

 4 LoggingNowadays, logging facilities are fundamental to monitor in real time the behavior of an ongoing experiment (which may last from minutes to days). The Avalanche logging module is in charge of displaying to the user the result of each plugin metric during the different experiment phases. Avalanche provides three different loggers: TextLogger,

Figure 9 :

 9 Figure 9: Avalanche evaluation plugin (or evaluator) object instantiation example.

  for a complete list of the available strategies.

		Training Strategies
	1	strategy = Replay(model, optimizer,

https://www.continualai.org ing research and development. A comprehensive codebase does not only provide a unique and clear access point to researchers and practitioners working on the topic, but also favors consistency across the library, with a coherent and easy interaction across modules and sub-modules. Last but not least, it promotes the consolidation of a large community able to provide better support for the library.

The official website, documentation, notebooks, and examples are available at https://avalanche.continualai.org.

https://tqdm.github.io

https://github.com/lebrice/Sequoia

Acknowledgments

The Avalanche project is a community-driven, collaborative project powered and maintained by ContinualAI: a nonprofit research organization and the largest open community on Continual Learning for AI. We would like to thank all its members for the valuable contributions and feedback that significantly improved the quality of the manuscript and the software library.


Magali Coupaud

Jonathan Mirault

Alice Delserieys

come

Effect of the structuring of an instruction on the exploration and comprehension of a text : a dual oculometric and behavioural approach

To identify how the elements of the Darwinium game are perceived by young people and what effects the specificity of a set of instructions has on their understanding of the rules of the game.

Genetic drift Random probabilities

Depending on the size of the biological population and the laws of probability. For a population to evolve, it is not necessarily related to adaptation, it may simply reflect the effect of random probability at any stage of reproduction (Jacob, 1981).

Natural selection Contingence

Intersection of independent causal series: the presence of external factors and the internal variation of an organism Mechanism combining fortuitous events, that are contingent in a historical perspective

• Randomness, intrinsically linked to gaming (Solinski, 2015)

• Games in education:

• Allows to experiment with the right to make mistakes (Sanchez et Romero, 2020) • Engaging et motivating (Alvarez et al., 2016) • Allows access to the complexity of a concept • Must be efficient However, the accuracy seems to be equal in both cases.

  (Sanchez et Romero, 2020) Design We choose a 2x1 design with 'Instruction' as 2-level factor ('detailled' / 'global') Detailled : « Discover the story of the game presented in the image on the first page of the research booklet, then find out what characterises the starting population of animals. » Global : « Explore the image on the first page of the research booklet, which tells the story of the game and describes the starting animal population. » Participant ➢ 8 volonteer student naive to the purpose -6 girls + 2 boys from 14 to 25 y.o (M=21; SD = 3.9) normal or corrected-to-normal vision Conclusion 27 The game's elements seem to be selfsufficient for pupils to play but teachers can improve the experience with focused instructions. Detailed instructions seem to allow children to access faster to the information.

au 6 octobre 2023 Du premier

4 au 6 octobre 2023

au 6 octobre 2023