Ruiqi Dai

Mathieu Lefort

Frédéric Armetta

Mathieu Guillermin

Stefan Duffner

Novelty detection for unsupervised continual learning in image sequences

Keywords: Continual learning, class-incremental learning, novelty Detection, object recognition, unsupervised learning

Recent works in the domain of deep learning for object recognition on common image classification benchmarks often address the representation learning problem under the assumption of i.i.d. input data. Although achieving satisfying results, this assumption seems not realistic when agents have to learn autonomously. An autonomous agent receives a continual visual flow of objects which is far from an i.i.d. distribution of objects. Moreover, agents have to construct their representations of the world and adapt to unknown environments, without relying on external sources of information such as labels that would be provided post-classification and are unavoidable when an oversegmentation is done. Then, in order to exploit the learned representation effectively for object recognition, a clear and meaningful relationship w.r.t. real object categories is required, which has been largely neglected in existing unsupervised algorithms.

In this paper, we propose a novelty detection method for continual and unsupervised object recognition, as an extension for the recent CURL model, which allows to moderate oversegmentation while preserving accuracy, in order to meet the requirements for autonomy. We experimentally validated our approach on two standard image classification benchmarks, MNIST and Fashion-MNIST, in this unsupervised and continual learning setting and improve the state of the art in terms of cluster purity, which is crucial for subsequent object recognition, since it facilitates clustering when information on ground truth labels is not available for free.

I. INTRODUCTION

Let's consider an agent interacting with objects in an unknown environment, continuously perceiving the objects through sensors. Being able to adapt to changes in the environment as well as continuously building a (visual) representation of objects of new classes while exploiting acquired knowledge is a crucial property for such a dynamic machine learning system. Classical deep learning models have shown excellent performance on image classification in an "off-line" setting, making the learning scenario comparatively simple in terms of representation learning since an iterative stochastic optimization of the loss function on i.i.d. data can be applied efficiently. However, when training data is not available all at once but sequentially, these models face some severe limitations. The literature on continual learning with neural networks [START_REF] Rao | Continual unsupervised representation learning[END_REF], [START_REF] Rebuffi | icarl: Incremental classifier and representation learning[END_REF] partially responds to this issue, yet many of them are supervised (i. e. supervised learning) and in order to effectively classify new observations of learned objects, extensive class labels are needed either at training time or after training to correctly attribute the numerous learned clusters to meaningful object categories. Even when addressed specifically, the lack of control over the way objects are introduced to the system lead to catastrophic forgetting phenomena for objects not seen for a long time, which is still a limitation to be overcome and considerably decreases the clustering performance. In fact, being able to learn new knowledge is an advantage coming from the plasticity of the network, but at the same time, the network should be stable enough to maintain the acquired knowledge, according to the stability-plasticity dilemma. A further limitation that we could identify, in the case of online and unsupervised learning, is the tendency to oversegment categories into many additional clusters [START_REF] Furao | An incremental network for on-line unsupervised classification and topology learning[END_REF], which makes the grouping of clusters inefficient during evaluation. Indeed, even if the clustering is done following an unsupervised approach, its evaluation is generally done thanks to ground truth labels assigned to the generated cluster, but hardly available online, which makes the methods ineffective when seeking autonomy. Moreover, we underline that over-segmentation facilitates the achievement of good accuracy while paradoxically reducing the autonomy of the system.

In this paper, we address the problem of unsupervised classincremental representation learning for object recognition, in which objects are observed one after the other for a single period of time without storing any images in the long term. We will name it as class-incremental learning in the following sections. We propose a deep neural network model performing unsupervised class-incremental learning for visual object recognition, which is an extension of CURL [START_REF] Rao | Continual unsupervised representation learning[END_REF], an approach dedicated to continual learning. Our main contribution lies in the integration of a new-class estimator based on statistics of the dynamics of the input sequence leveraging the temporal continuity of objects introduced and allowing to improve the detection of new objects. Furthermore, this guides the training with self-supervision by optimizing an adapted loss function.

The re-identification of learned objects that re-appear at different times during training could be addressed by using a classifier for category prediction which may require dedicated mechanisms. We choose to set aside this problem that can be addressed on its own as a second step. In this paper, we focus on the problem of performing an accurate automatic and unsupervised novelty detection, in order to maintain clustering as close as possible to the original class labels provided by the dataset while keeping high accuracy.

II. RELATED WORK

A. Novelty Detection

In the literature, there are two types of tasks considering novelty detection: one-class classification approaches [START_REF] Markou | Novelty detection: a review-part 1: statistical approaches[END_REF] that consider novelty detection as a binary classification problem of known/unknown, which are limited in scalability when there are numerous categories in the dataset; or multi-class approaches, also called open-set classification in the literature [START_REF] Bendale | Towards open set deep networks[END_REF], [START_REF] Geng | Recent advances in open set recognition: A survey[END_REF]. For multi-class novelty detection, the estimation of the probability for unknown objects is a major challenge because existing classification approaches are usually based on a closed-world assumption [START_REF] Bendale | Towards open set deep networks[END_REF], which estimates the probability distribution only over known categories, thus does not provide an appropriate estimation of the uncertainty when it comes to an unknown object. As a result, the model may wrongly "activate" an existing category with high confidence [START_REF] Liu | Energy-based out-of-distribution detection[END_REF], this creates calibration problems in commonly used classification approaches using the softmax function. Some proposed approaches re-calibrate softmax, for example, ODIN [START_REF] Liang | Enhancing the reliability of out-ofdistribution image detection in neural networks[END_REF] or G-OpenMax [START_REF] Ge | Generative openmax for multi-class open set classification[END_REF], [START_REF] Geng | Recent advances in open set recognition: A survey[END_REF]. Others use ensembles of deep learning models to predict uncertainty [START_REF] Lakshminarayanan | Simple and scalable predictive uncertainty estimation using deep ensembles[END_REF]; or treat this issue with a probabilistic approach based on the likelihood ratio between the inlier distribution and background knowledge [START_REF] Ren | Likelihood ratios for out-of-distribution detection[END_REF], [START_REF] Serrà | Input complexity and out-of-distribution detection with likelihood-based generative models[END_REF]. However, in continual learning, it is much more challenging to have a precise estimate on the background statistics for this sort of calibration.

B. Unsupervised object recognition

Different approaches for unsupervised image classification have been proposed in the literature, contrary to continual learning, common "off-line" approaches assume that training data are i.i.d [START_REF] Goyal | Object recognition using deep neural networks: A survey[END_REF], [START_REF] Szegedy | Inception-v4, inception-resnet and the impact of residual connections on learning[END_REF]. It is usually necessary to present the entire dataset several times in random order during training to ensure convergence and optimal performance. Recent advances in this domain make use of deep neural networks, in particular generative models [START_REF] Bojanowski | Optimizing the latent space of generative networks[END_REF], [START_REF] Goodfellow | Generative adversarial nets[END_REF], [START_REF] Kingma | Auto-encoding variational bayes[END_REF] like Variational Auto-Encoders (VAE) [START_REF] Kingma | Auto-encoding variational bayes[END_REF], [START_REF] Tolstikhin | Wasserstein autoencoders[END_REF] and Generative Adversarial Networks (GAN) [START_REF] Bojanowski | Optimizing the latent space of generative networks[END_REF], [START_REF] Goodfellow | Generative adversarial nets[END_REF]. These models learn to generate new data with the same statistics as a training set. Another family of unsupervised object recognition concerns clustering approaches like k-means [START_REF] Sculley | Web-scale k-means clustering[END_REF] or DBSCAN [START_REF] Xu | Survey of clustering algorithms[END_REF]. These approaches work on the raw data without learning highlevel features as deep neural networks, so applying them to images requires to use "hand-crafted" local feature extractor. Others include incremental clustering, for example SOINN [START_REF] Furao | An incremental network for on-line unsupervised classification and topology learning[END_REF] that learns the topology of dataset distribution, which will be introduced more in detail in section II-C. Common unsupervised object recognition algorithms have difficulties in determining the number of categories, as a result, they tend to mix similar categories, or reversely divide a category into several subcategories, requiring an extra effort of regrouping clusters during evaluation.

C. Continual learning

The literature in continual learning concerns two different scenarios: either solving a sequence of tasks/learning different datasets in the multi-task scenario, or learning new classes [START_REF] Maltoni | Continuous learning in single-incrementaltask scenarios[END_REF] incrementally in the single-task scenario. The state-of-the-art methods for continual learning with neural networks [START_REF] Lesort | Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges[END_REF], [START_REF] Parisi | Continual lifelong learning with neural networks: A review[END_REF] mainly focused on 3 categories of approaches:

• Structural approaches propose approaches that is network structure-related, for example, [START_REF] Rusu | Progressive neural networks[END_REF] proposes to dynamically add new nodes during training that modify the network structure with respect to the arrival of new tasks. Other algorithms [START_REF] Mallya | Piggyback: Adapting a single network to multiple tasks by learning to mask weights[END_REF] selectively activate parts of the network. • Regularization approaches add a task-related regularization term to the cost function [START_REF] Zenke | Continual learning through synaptic intelligence[END_REF] to moderate changes in neurons involved in previous tasks while still allowing the network to learns new tasks. For example, in [START_REF] Kirkpatrick | Overcoming catastrophic forgetting in neural networks[END_REF] the effect of catastrophic forgetting is contained by constraining the update of weights via a regularization term based on the Fisher information matrix extracted from previous tasks. Common incremental clustering methods [START_REF] Dey | Performance comparison of incremental k-means and incremental DBSCAN algorithms[END_REF], [START_REF] Joshi | Incremental learning: Areas and methods-a survey[END_REF] (such as BIRCH [START_REF] Zhang | BIRCH: An efficient data clustering method for very large databases[END_REF], incremental k-means [START_REF] Dey | Performance comparison of incremental k-means and incremental DBSCAN algorithms[END_REF]) are potential approaches to address incremental learning. Other approaches make use of topology learning [START_REF] Furao | An incremental network for on-line unsupervised classification and topology learning[END_REF]. Furao et al. [START_REF] Furao | An incremental network for on-line unsupervised classification and topology learning[END_REF], for example, proposed a model called SOINN, for unsupervised and online topology learning for non-stationary data, with less memory consumption and allows for learning without knowing a priori the number of classes and the distribution of data. Yet in the domain of image sequences, to work effectively with more complex visual data streams, these approaches often require either hand-crafted features or a pre-trained feature extraction model. Comparatively, approaches based on deep learning are more suitable due to the powerful representation capacity for visual data and images. Another limitation of these approaches is that they tend to create a large number of clusters and thus "over-segmenting" the original object classes [START_REF] Furao | An incremental network for on-line unsupervised classification and topology learning[END_REF]. This is also the case for some of the unsupervised continual learning approaches mentioned previously, cf. CURL [START_REF] Rao | Continual unsupervised representation learning[END_REF]. This makes subsequent classification more complex as supervision is required afterwards to assign each cluster to the corresponding object class.

We propose a model that improves the clustering effectiveness by exploiting the constraints of the addressed scenario where objects are presented one after the other in the data stream. Our model is based on a previously proposed generative deep neural network [START_REF] Rao | Continual unsupervised representation learning[END_REF] that we extended by modifying and improving the loss function and the new object class detection process.

III. PROPOSED APPROACH

Regarding the context and the class-incremental setting (cf. section I) of unsupervised and continual learning of object representation, we propose a generative neural network model extending CURL [START_REF] Rao | Continual unsupervised representation learning[END_REF] that has been originally designed for the single-task sequential learning. We will first briefly outline this base model in section III-A, and then present our contributions (sections III-B and III-C). The model optimizes a modified ELBO (Evidence Lower Bound) objective (maximizing the likelihood of the data), with input images x, categorical variable y (the index of the Gaussian component), latent variable z corresponding to the internal representation (formed by the GMM):

A. Model and learning algorithm

E(x) = K n=1 q(y = k|x) log p(x| z (k) ) -KL((q(z|x, y = k)||p(z|y = k)) -KL(q(y|x)||p(y)) , (1)
where q(y = k|x) represents the component posterior, computed by a dense layer with softmax, marked as yellow nodes in Fig. 1, z (k) ∼ q(z|x, y = k) is the latent code sampled from the kth Gaussian component each modelled by a dense layer of latent encoder head, log p(x| z representing the input images, y the category variable. The encoder maps the input images to a shared representation for all the categories. Its output is used as the input of a fully-connected (fc, in yellow) softmax layer to estimate the object category q(y|x); and updates the parameters µ k , σ k of the corresponding component(s) k. Posterior latent variable distribution q(z|x, y) is approximated with component-specific latent encoders (a GMM). Also, the prior p(z|y) of latent variable z follows a Gaussian distribution. Then, the image X is reconstructed from the resampled Ẑ using the decoder. For more détails, refer to section III.

with reconstructed image X at the output, KL((q(z|x, y = k)||p(z|y = k)) is a Kullback-Leibler divergence acting as the component-wise regularizer and enforcing a (Gaussian) embedding and KL(q(y|x)||p(y)) is the categorical regularizer that ensures that classes are well balanced, approaching the assumed uniform prior distribution p(z|y) of each category. By maximizing Eq. 1, the model learns to reconstruct the input images and at the same time, due to the two regularization terms, to cluster objects into different classes in the latent space z by dynamically assigning them to different components. Poorly modelled instances whose ELBO is inferior to a threshold are considered as possible new category candidates and are thus stored in a temporary buffer which, once filled, is used to create and initialize a new component in the model. For more details, refer to [START_REF] Rao | Continual unsupervised representation learning[END_REF]. This base model gives promising clustering performance. However, a major limitation is that the number of clusters resulting from the model does not allow to stay close to the original number of classes. In fact, it tends to create excessive clusters that therefore do not reflect the actual distribution of object categories. The number of introduced clusters is expected to stay close to the ground truth distribution, to facilitate eventually the categorization during evaluation.

Due to the fact that objects are presented sequentially for a certain amount of time, we consider that continuity is present in the perception of an autonomous agent evolving in a continual environment. Thus, the purpose of this study relies on measuring the additional value allowed by considering such a hypothesis on the accuracy of the system, training time and the need to keep under control the number of introduced components.

We propose two modifications of the original CURL model: a new category detection process (section III-B) that guides the learning with self-supervision optimizing a modified loss function (section III-C).

B. Detection of new classes

We hereby consider the case where the agent perceives objects class by class in the environment, not in a completely random way but in a class incremental way as it is mentioned in section I. In this paper, we choose to focus on novelty detection and improvements that can be achieved through the use of the continuity of perceived objects hypothesis, illustrated by the continuity in classes presented to the system. This shows the potential for such an approach, which is generally not exploited in machine learning, to study later the resilience of the process in a more noisy environment.

In this context, our contribution consists in the automatic detection of new classes by integrating an adaptive change detection algorithm, the Page-Hinckley test [START_REF] Page | Continuous inspection schemes[END_REF] applied to ELBO likelihood, a common approach applied in the domain of concept drift detection to detect abrupt changes in sequential input data. Formally, let x t ∈ X = {x 0 , . . . , x T } be the examples presented in sequence of input training examples. In accordance with CURL, in our model, poorly modelled examples, are considered as new category candidates, i. e. for which the unsupervised ELBO objective E(x) (Eq. 1) is below a threshold θ, since the unsupervised ELBO objective E(x) marginalizes over all the existing categories which might reduce false-positive new category detection that corresponds to a category learned in the past instead of a new one. In our model, we apply the Page-Hinckley test that computes the decision function g(t) for each new arriving example. We compare ELBO objective E(x) with a threshold θ, noted by H the Heaviside step function that will equal to 1 if E(x) is smaller than a threshold θ (implying an outlier). It is smoothed by a running average noted by p n (t), counting the average times that the outliers occur. We adopted a variant of the Page-Hinckley test as defined in [START_REF] Montiel | Scikitmultiflow: A multi-output streaming framework[END_REF], with N being the number of samples the agent has seen since the previous category change, and υ being the tolerated change for each step:

g(t) = max(0, g(t -1) + p n (t) -µ pn (t) -υ) (2)
µ pn (t) = (N -1) N µ pn (t -1) + 1 N p n (t) (3)
p n (t) = α * p n (t -1) + (1 -α) * H(θ -E(x t )) . (4)
If g(t) is greater than a threshold θ n , then a new category is detected, i. e. a Gaussian is added to the GMM in the VAE and we reinitialize g(t) to 0. Contrary to CURL that might be affected by noise in the ELBO loss, under the hypothesis of temporal continuity, our proposal of detecting new categories by Eq. 2-Eq. 4. helps to smooth these fluctuations and to obtain a cleaner supervision signal in the presence of outliers and alleviate category "over-segmentation". Another modification of CURL in our model is that we propose for the original CURL model concerns the usage of the buffer storing recent examples in the incoming data stream. In our model since the proposed Page-Hinckley test detects abrupt changes, once a category change is captured, the buffer is filled with all the following instances in the sequence until reaching its maximum size n. However, the examples in the (unfilled) buffer are not used for training immediately to prevent over-fitting resulting from too few training instances and to ensure having enough observations for each object class. Once the buffer is full, the training of the new class is initiated and the buffer is released.

C. Loss function

We use self-supervision deduced from our new-category detection algorithm to adapt the loss function that is used for training the model. We propose to optimize a supervised version of the ELBO objective function E sup (x) that CURL [START_REF] Rao | Continual unsupervised representation learning[END_REF] originally used for a supervised baseline comparison of their algorithm. However, we integrate it differently in our approach. That is, we create an internal supervision signal y m ∈ N based on the detection of new classes for training. y m ∈ N that corresponds to the class of the instance determined by our model. Note that our proposed approach is still completely unsupervised as no ground truth labels are used. More specifically, y m is incremented if the presence of the new, unseen object class is detected and maintained constant otherwise.

y m = y m + 1, if g t ≥ θ n y m , otherwise.
The objective is defined as:

E sup (x) = log q(y = y m |x) + log p(x| z ym , y = y m ) -KL(q(z|x, y = y m )||p(z|y = y m )) , (5)
and we continue to use the same variable definition as in Eq. 1. where the first term trains a fully connected layer with softmax to predict the label, the second term minimises the auto-encoded reconstruction error and the last term again represents the Kullback-Leibler divergence between the variational posterior of z and its corresponding Gaussian prior distribution.

IV. EXPERIMENTS A. Dataset

To compare our approach to the state of the art, we evaluated our model on two standard datasets: MNIST (images of handwritten digits from 0 to 9) [START_REF] Lecun | The MNIST database of handwritten digits[END_REF] and Fashion-MNIST [START_REF] Xiao | Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms[END_REF] (Zalando's images with the classes {T-shirt, trouser, pullover, dress, coat, sandal, shirt, sneaker, bag, ankle boot}). Both datasets contain objects from 10 classes, with 60000 images for training and 10000 images for testing. The size of images in both datasets is 28 × 28. For each class, there are around 6000 images for training and around 1000 for the test set.

During training we present images in a class by class order, from 0 to 9. It can be noticed that compared to the MNIST dataset, the Fashion-MNIST dataset is more complex. In the Fashion-MNIST dataset, images of different classes can be more similar (than in MNIST), for example, dresses resemble coats. The MNIST dataset, however, is a comparatively simple task for the reason of the well-alignment of digits in each category. And comparatively, objects in the Fashion-MNIST dataset are more diverse and more complex.

B. Experimental protocol

Our model is trained in a continuous way, as stated in section I, i. e. the data are presented to the model sequentially and class-by-class. Thus, each class is seen only once, but during training, each image of the current class can be presented several times until a new class is detected. For a fair comparison with CURL, as detailed in section IV-C, we preserved the model architecture and setting for generative replay from [START_REF] Rao | Continual unsupervised representation learning[END_REF], meanwhile tuning the other parameters (the threshold for outliers) with respect to clustering performances, and most importantly, with regard to the optimum number of categories detected that approaches the true distribution of categories and results in the best clustering score of AMI/ARI while preserving the same performance (i. e. accuracy) in clustering. We also compared the clustering performance of our proposal with two incremental clustering algorithms, BIRCH [START_REF] Zhang | BIRCH: An efficient data clustering method for very large databases[END_REF] and incremental k-means [START_REF] Sculley | Web-scale k-means clustering[END_REF]. The two mentioned incremental algorithms do not provide explicit feature extracting methods, to avoid retraining a neural network, we take the flattened image as input without extracting features. In addition, we compare our model with SOINN [START_REF] Furao | An incremental network for on-line unsupervised classification and topology learning[END_REF], which was originally designed for offline clustering, that we adapted in an online setting that presents objects class by class. As an ablation study, we have also tested two variants of our model: Ours w/o P-H test, where we perform a simple running average p n while comparing ELBO objective with a threshold; Ours w/o p n , where we apply the Page-Hinckley test directly on ELBO objective without running average p n smoothing.

C. Hyperparameters

For both datasets, we fixed the neural network architecture and the learning rate to 10 -3 while using the Adam optimizer. To compare our model with CURL, we use the same neural network structure as in [START_REF] Rao | Continual unsupervised representation learning[END_REF]: a 4-layer MLP as encoder {1200, 600, 300, 150}, and a linear layer with 64 dimensions to compute the mean and variance for the 32-dimensional latent variable z. For the decoder a two-layer MLP {500, 500} was used. The total number of iterations is 100000 counting all the categories, for each category 10000 iterations, where at each iteration, the size of batch is 100. We applied the mechanism of generative replay in the same way as CURL, i. e. images of previous classes are generated at fixed intervals (every 10000 steps) and stored into a buffer. For the mixture generative strategy, we continue to use the one of CURL, that is to create a mixture between real images of the current category and generated images of other learned components. To this end, every two steps, a batch of generated images is mixed with the batch of real images for training. We suppose that images of a class are visible for at least 100 steps in both experiments and we use a buffer of size 100 that stores outlier candidates. For the value of θ, the threshold on the ELBO loss, we have chosen θ = -150 for all the experiments on MNIST, resulting in the best accuracy. For Fashion-MNIST, we set θ = -300 for CURL and our model without Page-Hinckley test, and θ = -190 for our model. Concerning the Page-Hinckley test, we set α = 0.85 for both datasets, and υ = 0.3 and θ n = 1.5 for parameters in the experiments with Page-Hinckley test applied on running average p n ; υ = 55.0 and θ n = 1500.0 for parameters in the experiments with Page-Hinckley test applied on negative unsupervised ELBO loss without p n smoothing.

D. Evaluation measures

To evaluate the quality of the learned clustering, we used three standard metrics: the clustering accuracy assigning to each component its most represented class, for labelisation of each component on the test set in correspondence to classes and measuring the proportion of correctly classified instances, the Adjusted Mutual Information (AMI) and the Adjusted Random Index (ARI) computed between learned clustering prediction and that of the ground truth. AMI measures the mutual information between two assignments of partitions. ARI measures the similarity between two partitions by counting the difference of assignment of pairs of samples between two partitions. Both metrics are adjusted w. r. t. the chance to remove the bias induced by the inequality in the number of clusters in both partitions. All measures are in [0, 1], where higher values are better.

The clustering accuracy gives a general idea about the classification performance if labels were available. However, it does not completely reflect the quality of the clustering. For example, let's consider the case where the algorithm creates a partition that correctly separates different classes, but creates many excessive clusters from the same class (oversegmentation). We need at least one ground-truth label per cluster to regroup them into correct classes, i. e. requiring supplementary effort on data annotation, which considerably decreases the level of autonomy of the algorithm in an unsupervised continual learning setting.

E. Results

The results on MNIST and Fashion-MNIST are shown in Table I and in Table II respectively. Note that one needs to choose the trade-off between optimizing the number of clusters, reaching better AMI/ARI scores, while detecting all the changes which allows high clustering accuracy. For MNIST, our model achieves a very good trade-off and creates fewer additional components, i. e. the closest to the real number of classes [START_REF] Kingma | Auto-encoding variational bayes[END_REF], and scores the highest in terms of AMI and ARI compared to CURL and SOINN. For Fashion-MNIST, our model outperforms CURL on the AMI and ARI measures, with a slightly inferior accuracy. But as shown in Table II, CURL creates 120 components exceeding by far the number of real categories in the dataset. This indicates that the clusters created by our model follow the true distribution of different categories and avoid over-segmentation.

In Fig. 2 and Fig. 3, we further show the confusion matrix between ground truth classes and clusters. We can observe that in our model, samples of the same class are represented principally by one cluster. On the contrary, the confusion matrix of CURL shows that CURL tends to separate samples of the same class into different clusters. We equally illustrate the Component id in the creation order To validate the individual contributions of our method, we compared it to a variant of CURL using our loss (Eq. 5) supervised by the ground truth and with buffer, called "CURL supervised" in Tables I and II. These two experiments demonstrate the effectiveness of our new-category detection algorithm, since our model with Page-Hinckley test applied on p n is capable of reaching a comparable performance in terms of new category detection on MNIST with respect to supervision using the ground truth. But both our model and CURL are outperformed by SOINN in terms of clustering accuracy. Only in terms of accuracy, SOINN performs better, which is not surprising given the excessive number of components (about 1200 on MNIST and about 755 nodes on Fashion-MNIST) reducing thus the probability of impure ground-truth clusters but at the same time needing much more additional supervision to label these clusters, as demonstrated in Fig. 5, one could observe that if we only use part of the test set to label components by their majority class, a drop in the clustering performance could be remarked from Fig. 5. The SOINN model converges the slowest compared to CURL and our model.

The results of BIRCH and incremental k-means are much below the performance of the other methods on both datasets showing a clear limitation of such classical incremental clustering algorithms in this context.

V. CONCLUSION AND DISCUSSION

Recent works have focused on creating an efficient neural network model for continual learning, as it is the case for CURL which is unsupervised and provides a generative replay mechanism while making use of a rich multivariate Gaussian Mixture Model. In this paper, we improved the new category detection process by moderating the number of components created for class categorization in order to stay close to the real distribution. We consider that, for an autonomous agent, some continuity is present and images of its environment are not perceived in a totally random order. Thus, we proposed a completely unsupervised approach based on an extension of CURL, a VAE-based model [START_REF] Rao | Continual unsupervised representation learning[END_REF], that takes advantage of continuity in the introduced object class and applying a supervised ELBO loss with self-supervision. To this end, we proposed to use the statistical Page-Hinckley test to improve the performance of new-class detection, and p n a running average for each instance, to smooth fluctuations in the ELBO loss, leading to a robust class change detector. When compared to the baseline, our proposal allows to considerably reduce the introduction of additional clusters while keeping accuracy, which improves autonomy. Indeed, over-segmentation of clusters leads to further supervision for classification which is not always available online, or can only be done in a restrained way. This work appears as a first step and shows how unsupervised learning can take advantage of temporal continuity of objects perceived to better categorize objects online. Further work will study how this proposal behaves under increasing noise in the input sequences.

  CURL is a model that learns robust representations for different classes in a continuous manner based on a derivative of Variational Auto-Encoder (VAE), as shown in Fig. 1. Concretely, the core of the model is a Variational Auto-Encoder (VAE) which allows to approximate the distribution of the latent variable with a Gaussian or component. CURL extends the VAE by dynamically introducing a new dedicated component, for each new outlier image. It alleviates the effect of catastrophic forgetting by continuously generating synthetic training examples of previously learnt classes.

Fig. 1 .

 1 Fig.1. The neural architecture of CURL: a Variational Auto-Encoder, X representing the input images, y the category variable. The encoder maps the input images to a shared representation for all the categories. Its output is used as the input of a fully-connected (fc, in yellow) softmax layer to estimate the object category q(y|x); and updates the parameters µ k , σ k of the corresponding component(s) k. Posterior latent variable distribution q(z|x, y) is approximated with component-specific latent encoders (a GMM). Also, the prior p(z|y) of latent variable z follows a Gaussian distribution. Then, the image X is reconstructed from the resampled Ẑ using the decoder. For more détails, refer to section III.

Fig. 2 .

 2 Fig. 2. Confusion matrices between ground truth and predicted cluster components for CURL on the MNIST test set (above) and the Fashion-MNIST test set (below) (the darker a cell the more instances it represents).

Fig. 3 .Fig. 4 .

 34 Fig.3. Confusion matrices between ground truth and predicted cluster components for our complete approach on MNIST (left) and Fashion-MNIST (right) (the darker a cell the more instances it represents).

Fig. 5 .

 5 Fig. 5. Influence of of the number of annotated examples used for labeling the test set on the accuracy, for MNIST (left) and Fashion-MNIST (right).

  TABLE I COMPARISON OF OUR METHOD WITH THE STATE OF THE ART ON THE MNIST (AVERAGE OVER 3 RUNS) FOR EACH METRIC MEAN±SD.

	Model	accuracy	AMI	ARI	nb components
	CURL [23]	0.822 ± 0.0102	0.557 ± 0.006	0.28 ± 0.025	93.85 ± 1.884
	CURL supervised [23]	0.855 ± 0.006	0.749 ± 0.006	0.6997 ± 0.012	10 ± 0
	SOINN [4]	0.925 ± 0.0011	0.39 ± 0.002	0.018 ± 0.0008	1204 ± 39.6
	BIRCH [35]	0.3026 ± 0.002	0.184 ± 0.014	0.10 ± 0.0113	10 ± 0
	Increm. k-means [27]	0.338 ± 0.017	0.2545 ± 0.013	0.124 ± 0.013	10 ± 0
	Ours w/o P-H test	0.849 ± 0.008	0.735 ± 0.0102	0.685 ± 0.015	22 ± 1.07
	Ours w/o pn	0.854 ± 0.005	0.748 ± 0.00424 0.6996 ± 0.0085	10.67 ± 0.47
	Ours	0.8537 ± 0.006	0.746 ± 0.0096	0.70 ± 0.013	10 ± 0
	Model	accuracy	AMI	ARI	nb components
	CURL [23]	0.686 ± 0.013	0.445 ± 0.004	0.137 ± 0.002	120 ± 0.0
	CURL supervised [23] 0.654 ± 0.007	0.57 ± 0.006	0.4336 ± 0.006	10 ± 0
	SOINN [4]	0.796 ± 0.003	0.365 ± 0.001	0.022 ± 0.008	755 ± 16.54
	BIRCH [35]	0.328 ± 0.023	0.286 ± 0.019	0.124 ± 0.0139	10 ± 0
	Increm. k-means [27]	0.404 ± 0.004	0.38 ± 0.0105	0.237 ± 0.013	10 ± 0
	Ours w/o P-H test	0.644 ± 0.009	0.537 ± 0.015	0.415 ± 0.022	64.6 ± 9.5
	Ours w/o pn	0.65 ± 0.0098	0.547 ± 0.007	0.42 ± 0.007	25.67 ± 9.534
	Ours	0.6526 ± 0.0056 0.558 ± 0.00856 0.442 ± 0.0117	13.0 ± 2.19

TABLE II COMPARISON

 II OF OUR METHOD WITH THE STATE OF THE ART ON FASHION-MNIST (AVERAGE OVER 3 RUNS) FOR EACH METRIC MEAN±SD.

			Confusion matrix of CURL model on MNIST
	True label	0 1 2 3 4 5 6 7 8		500
		9
		0 2 4 6 8	1 0 1 2 1 4 1 6 1 8 2 0 2 2 2 4 2 6 2 8 3 0 3 2 3 4 3 6 3 8 4 0 4 2 4 4 4 6 4 8 5 0 5 2 5 4 5 6 5 8 6 0 6 2 6 4 6 6 6 8 7 0 7 2 7 4 7 6 7 8 8 0 8 2 8 4 8 6 8 8 9 0 9 2 Component id in the creation order	0
		0 2 4 6 8 1 0 1 2 1 4 1 6 1 8 2 0 2 2 2 4 2 6 2 8 3 0 3 2 3 4 3 6 3 8 4 0 4 2 4 4 4 6 4 8 5 0 5 2 5 4 5 6 5 8 6 0 6 2 6 4 6 6 6 8 7 0 7 2 7 4 7 6 7 8 8 0 8 2 8 4 8 6 8 8 9 0 9 2 9 4 9 6 9 8 1 0 0 1 0 2 1 0 4 1 0 6 1 0 8 1 1 0 1 1 2 1 1 4 1 1 6 1 1 8


Ruiqi Dai

Mathieu Lefort

Frédéric Armetta

Mathieu Guillermin

Stefan Duffner

Self-supervised continual learning for object recognition in image sequences

The autonomous learning of different objects in images, with a continual and unsupervised context, relies on detecting unknown objects and recognizing known ones based on the learned visual representation. Novelty detection is challenging because of the internal representation drifts of known objects not been seen for a long time. Most existing approaches either perform offline unsupervised learning on a large dataset, or continual supervised learning. Nevertheless, very few existing approaches propose unsupervised continual learning for object recognition. In this paper, we propose a new neural network-based approach for continually learning representations of objects from image sequences, that is able to autonomously detect novel objects and to recognize previously learned ones during training. It is based on a statistical test, performed on internal representations, adapted to counterbalance the concept drift, without storing any image. Experimental results show that our approach outperforms the state of the art on MNIST and Fashion-MNIST datasets. In particular, our approach avoids to over-segment the distribution of clusters, which artificially increases traditional indicators such as clustering accuracy.

Introduction

For an agent, it is very challenging to autonomously and continuously learn and make use of object representations of its open environment. The system has to detect novelty and introduce a new concept or class if necessary while maintaining the already acquired knowledge. This is part of the more general problem of finding a meaningful and robust representation under the stability-plasticity dilemma. Recently, unsupervised visual representation learning methods based on deep neural networks have been proposed [START_REF] Chen | A simple framework for contrastive learning of visual representations[END_REF][START_REF] Kingma | Auto-encoding variational bayes[END_REF]. But learning these models online with a continuous stream of images is not straightforward due to the nature of stochastic gradient descent optimization and the fact that most of them rely on an i.i.d. data assumption which is not valid in open environments.

The literature on continual learning with neural networks [START_REF] Dushyant | Continual unsupervised representation learning[END_REF][START_REF] Lopez-Paz | Gradient episodic memory for continual learning[END_REF][START_REF] Rebuffi | iCaRL: Incremental classifier and representation learning[END_REF] partially responds to this issue by applying strategies alleviating catastrophic forgetting, such as experience replay [START_REF] Dushyant | Continual unsupervised representation learning[END_REF][START_REF] Rebuffi | iCaRL: Incremental classifier and representation learning[END_REF], which either retain a memory of real images or generate new ones. In addition, novelty detection is an important aspect in unsupervised continual learning. However, in current representation learning approaches based on generative deep neural networks, the problem of correctly reidentifying reappearing objects or detecting unknown categories is not explicitly addressed. In fact, most existing algorithms rely on the observation likelihood, the loss function or a separate classifier [START_REF] Dushyant | Continual unsupervised representation learning[END_REF], but these methods have some serious drawbacks with continuously evolving models and are difficult to calibrate [START_REF] Geng | Recent advances in open set recognition: A survey[END_REF]. As a consequence, they tend to keep expanding ignoring the true number of categories. Moreover, more advanced techniques [START_REF] Hendrycks | A baseline for detecting misclassified and out-ofdistribution examples in neural networks[END_REF][START_REF] Lee | A simple unified framework for detecting out-of-distribution samples and adversarial attacks[END_REF] operate on statistical estimates of sample distributions, but in general they can only be applied offline.

In this paper, we target autonomous agent systems and propose a deep neural network model performing continual learning for visual object recognition. In this setting, the objects are learned in sequential order and are presented class by class, yet can reappear later. Our model is a modified version of a recently proposed Variational Auto-Encoder (VAE) model for continual unsupervised learning [START_REF] Dushyant | Continual unsupervised representation learning[END_REF]. The main contribution of this paper is a new method to detect new object classes and recognize old ones in an image stream based on a statistical hypothesis test on the distributions of internal representation vectors. Our algorithm does not store any previous processed image, it only relies on the first two moments of the distributions that are estimated on-line during training.

Related work

Novelty detection. Detecting novelty, anomalies or outliers in a data sample distribution has been studied for several decades [START_REF] Hodge | A survey of outlier detection methodologies[END_REF][START_REF] Ma | A review of novelty detection[END_REF]. Some of the works consider the problem as one-class classification problem, i. e. only modelling the nominal class, or by creating a single artificial class, but their performances suffer when the number of categories increases [START_REF] Geng | Recent advances in open set recognition: A survey[END_REF]. In classification applications, one usual approach is to infer abnormality from the output scores of the classifier. However, samples from unknown classes may produce high confidence values by strongly activating one of the known classes [START_REF] Geng | Recent advances in open set recognition: A survey[END_REF][START_REF] Lee | A simple unified framework for detecting out-of-distribution samples and adversarial attacks[END_REF]. For example, in neural networks, the softmax function indicates the confidence in the classification, but the interpretation as a true probability estimate is questionable [START_REF] Hess | Softmax-based classification is k-means clustering: Formal proof, consequences for adversarial attacks, and improvement through centroid based tailoring[END_REF]. Multi-class novelty detection can be formulated as "open-set recognition" [START_REF] Geng | Recent advances in open set recognition: A survey[END_REF], i.e. examples of unknown classes are explicitly handled by the model and rejected. For neural networks, for instance, models based on the calibration (of softmax) have been proposed such as ODIN [START_REF] Liang | Enhancing the reliability of out-of-distribution image detection in neural networks[END_REF] or G-OpenMax [START_REF] Ge | Generative openmax for multi-class open set classification[END_REF]. Also, removing the softmax normalisation may improve the discrimination between inliers and outliers [START_REF] Liu | Energy-based out-of-distribution detection[END_REF]. A more probabilistic approach consists in using the likelihood ratio between inliers and a background distribution for outlier detection [START_REF] Ren | Likelihood ratios for out-of-distribution detection[END_REF][START_REF] Serrà | Input complexity and out-of-distribution detection with likelihood-based generative models[END_REF]. Other methods compute the uncertainty of model predictions using ensembles of deep models [START_REF] Lakshminarayanan | Simple and scalable predictive uncertainty estimation using deep ensembles[END_REF].

Despite these advances, two major challenges with generative neural networks remain: it may be difficult to disentangle known from unknown classes [START_REF] Geng | Recent advances in open set recognition: A survey[END_REF][START_REF] Nalisnick | Do deep generative models know what they don't know?[END_REF], and most methods operate offline, which either requires all the feature vectors extracted for known classes or explicitly train a separate mapping function that is independent from the learnt representation [START_REF] Bodesheim | Kernel null space methods for novelty detection[END_REF].

Continual learning with neural networks. Various methods have been proposed to alleviate catastrophic forgetting, such as regularization strategies [START_REF] Kirkpatrick | Overcoming catastrophic forgetting in neural networks[END_REF][START_REF] Zenke | Continual learning through synaptic intelligence[END_REF] that try to conserve weights w.r.t. their relevance to already learned classes. Other strategies include structural approaches [START_REF] Lopez-Paz | Gradient episodic memory for continual learning[END_REF][START_REF] Rusu | Progressive neural networks[END_REF] that dynamically expand network structures for new tasks, or experience replay [START_REF] Rebuffi | iCaRL: Incremental classifier and representation learning[END_REF] with either partial storage of real training examples or generated examples for learned categories [START_REF] Dushyant | Continual unsupervised representation learning[END_REF]. However, these approaches are usually applied to supervised settings.

Among the unsupervised approaches, the Self-Taught Associative Memory (STAM) architecture [START_REF] Smith | Unsupervised progressive learning and the STAM architecture[END_REF] uses a specific hierarchical feature representation based on image patch exemplars of different sizes obtained from clustering, which may limit its scalability. SOINN [START_REF] Furao | An incremental network for on-line unsupervised classification and topology learning[END_REF] proposed a model with Self-Organizing Incremental Neural Network based on a distance metric to incrementally learn the topology of input data. Our model is based on the Continual Unsupervised Representation Learning (CURL) model [START_REF] Dushyant | Continual unsupervised representation learning[END_REF] that will be presented in section 3.

Representation learning algorithm and model

Our model is an extension of [START_REF] Dushyant | Continual unsupervised representation learning[END_REF], a type of Variational Auto-Encoder (VAE) with a Gaussian Mixture Model (GMM), adapted to a class-incremental learning setting. More formally, it estimates the probability p(x, y, z) = p(y)p(z|y)p(x|z) of the input x, the label y and the latent code z, using variational inference, and approximates the posterior p(y, z|x) with q(y, z|x) = q(y|x)q(z|x, y), where q(y|x) is the output of a dense layer followed by softmax determining the component posterior given an input x, and q(z|x, y) is the distribution of z encoded by the component-wise encoder. Instead of a single multivariate Gaussian as in classical VAE, CURL uses several components modelled as dense neural layers that output the different means and variances, and that are dynamically added during training when a certain number of outliers are detected. A modified ELBO (Evidence Lower Bound) objective is optimized during learning:

E(x) = K k=1 q(y = k|x) log p(x| z (k) ) component-wise reconstruction -KL(q(z|x, y = k)||p(z|y = k))
Kullback-Leibler divergence regularization on z -KL(q(y|x)||p(y)) categorical regularization [START_REF] Bodesheim | Kernel null space methods for novelty detection[END_REF] with z (k) ∼ q(z|x, y = k) being the sampled latent code, p(z|y = k) and p(y) being the prior distributions (normal and uniform respectively). By maximizing Eq. 1, the model maximizes the data likelihood via reconstruction (first term), and regularises the model such that z tends to be normally distributed (second term) and samples are evenly distributed over components (third term).

During training, the model alleviates catastrophic forgetting by following the mixture generative replay strategy, mixing real examples of the current category with generated images of past categories. To detect outliers, the model uses the ELBO loss (Eq. 1) as an indicator of learning quality, modelling the likelihood of an example belonging to a learned category. For more details, refer to [START_REF] Dushyant | Continual unsupervised representation learning[END_REF].

The authors also introduced a supervised form of the ELBO loss that replaces the unsupervised ELBO objective (Eq. 1) when used in a supervised setting (in this case the component y to update is not determined by the internal classifier q(y|x) but selected by y m ): E sup (x) = log q(y = y m |x) + log p(x| z ym , y = y m ) -KL(q(z|x, y = y m )||p(z|y = y m )) [START_REF] Chen | A simple framework for contrastive learning of visual representations[END_REF] In our model the labels y m are a self-supervision signal generated automatically based on a statistical hypothesis test -a two-sample t 2 Hotelling test that we adapted (cf. section 4.1) -performed on the internal representation of the model.

Proposed Approach

Continual detection and recognition of objects

In our model, one object category is supposed to be modelled by a single component, contrary to CURL that uses a GMM model allowing multiple components per category. As the latent variable z for each category tends to follow a multivariate normal distribution, because of the KL regularisation term during training, to decide if the current observation batch corresponds to a given class, we perform a two-sample Hotelling t 2 test [START_REF] Hotelling | The generalization of student's ratio[END_REF], which is a statistical test for multivariate normal distributions. To this end, we compute the t 2 statistics:

t 2 = n y * n b n y + n b (z y -zb ) T Σ -1 (z y -zb ) , (3)
with Σ being the pooled covariance matrix determined by

Σ = (n y -1) Σ y sh + (n b -1) Σ b n y + n b -2 , (4)
zy and Σ y sh the sample mean and covariance matrix of latent variable z corresponding to an object category y, zb and Σ b the sample mean and covariance matrix of the input batch and n y , n b the sample sizes of the two distributions respectively. The t 2 distribution follows the F distribution up to a factor, where d is the dimension of z:

n y + n b -d -1 (n y + n b -2)d t 2 ∼ F (d, n y + n b -1 -d) (5)
Our null hypothesis H 0 is that the two means µ y , µ b of object class y and input batch b are equal, and we reject it if the left hand side of Eq. 5 is below a critical value related to a given p-value threshold. In practice, we compute the p-values for the tests between the current batch and each trained object category, and we select the class with the highest p-value if it is above the defined threshold. Otherwise, it is considered belonging to a new object category.

Online parameter estimation

In our continual learning setting, the embedding in the latent space of the VAE gradually evolves. To compute the mean and covariance of previous objects without storing past images, we approximate them by running averages. However, these approximated covariance matrices slightly underestimate the actual variance. This may lead to too large values in the t 2 statistics (Eq. 3) and eventually to very small p-values for known classes. To alleviate this problem, we apply a shrinkage operation to the running covariance matrices such that the diagonal entries are more homogeneous and the difference between eigenvalues is reduced.

zy (t) = (1 -α)z y (t -1) + αz y (t) , (6)
Σ y (t) = (1 -α) Σ y (t -1) + α(z y (t) -zy (t)) T (z y (t) -zy (t)) . (7)
Σ y sh (t) = (1 -γ) Σ y + γ * tr( Σ y ) d I , (8)
where z y (t) is the embedding of class y, α ∈ (0, 1) a small update factor, γ > 0 is the shrinkage coefficient and I the identity matrix. The equations are applied for the current class with real observations and for all the other classes with synthetic examples from generative replay. Finally, we do the Hotelling t 2 test (Eq. 4) using zy and Σ y sh .

Experiments

Protocol

We evaluated our proposed approach on MNIST and Fashion-MNIST, each including 10 classes, a total of 60000 images of size 28x28 for training and 10000 images for test. The VAE architecture we used is the same as the one of CURL, i. e. a 4-layer MLP {1200, 600, 300, 150} as encoder and a two-layer MLP {500, 500} as decoder, with a 32-dimensional latent space. The learning rate is set to 10 -3 with an Adam optimizer. The size of the outlier buffer is 100 for CURL, 200 for our model and for "CURL with HT" (as the test is performed on batches, we slightly increase the buffer to contain 2 batches and avoid potential fluctuations). The batch size is 100, thus n b = 100 and n y is empirically set to 20.

Our protocol consists of two consecutive sequences. In the first one, we present half of the training examples class by class in a random order to test the ability of the model to detect new classes. In the second one, we present the second half of the training data still class by class, in the same order as in the first sequence, to test the recognition performance of the model. We compared our approach with CURL [START_REF] Dushyant | Continual unsupervised representation learning[END_REF], SOINN [START_REF] Furao | An incremental network for on-line unsupervised classification and topology learning[END_REF]

Results

To evaluate the quality of the learned clustering, we compute different standard metrics on the two test sets: accuracy (the label of each component is obtained by post labelling via majority vote), the Adjusted Mutual Information (AMI) and the Adjusted Random Index (ARI) measuring the correspondence of the learned clustering w.r.t. the Ground Truth. Our model considers that all examples of a batch belong to the category, while the other tested models evaluate each example individually. For a fair comparison, the other models are also evaluated batch by batch via majority vote of prediction amongst the batch. On the MNIST dataset (table 1), our model performs better than all other models for all indicators. In particular, the number of components is much closer to the real number of categories, thus greatly improving the AMI and ARI score, while not impacting the accuracy which remains at 100%. It is interesting to note that the application of the Hotelling test on CURL, decreases significantly the number of learned components. However, this does allow the model to reach the AMI and ARI score of our model and induces a drop in accuracy performance. This validates the need of using the supervised ELBO loss in our model.

On the Fashion-MNIST dataset (table 2), the trend for AMI and ARI are similar to the ones observed on MNIST. However, here, this is obtained at the cost of a drop in accuracy. This may be explained as the Fashion-MNIST dataset is harder and by the fact that having a higher number of clusters facilitates a high accuracy (as the chance of mixing different classes in one component is reduced). However, for an autonomous agent, we prefer to have a smaller number of clusters as less examples can be required to label them.

In figure 1 we illustrate the evolution of model performance during training on Fashion-MNIST on the detection accuracy, i. e. the binary classification of "known" and "unknown" classes, and the clustering accuracy of the classes that have been learned by the agent, i. e. known classes (except for those considered by error as unknown). We can observe that each new class is detected at the right time and that the model is not subject to catastrophic forgetting.

Conclusion

In this paper, we proposed a self-supervised continual object representation learning algorithm extending the CURL model [START_REF] Dushyant | Continual unsupervised representation learning[END_REF]. Our contribution lies in combining the model with a statistical hypothesis test allowing to detect unknown categories and to recognize previously learned categories, and in this way, to self-supervise the learning process without storing any previous examples. Compared to the state of the art, our model demonstrates its capacity to recognize learned objects in an online scenario while avoiding creating and reallocating new components for learned categories. Thus the model becomes more effective in automatically detecting the number of categories. Our proposal permits to moderate the amount of over-segmentation and achieves better performance in terms of standard clustering metrics AMI and ARI compared to the state-of-theart algorithms CURL and SOINN [START_REF] Furao | An incremental network for on-line unsupervised classification and topology learning[END_REF] but may have lower accuracy due to fewer component creation. The introduced statistical test allows to detect properly the novel classes and recognize learned categories. In the future, we will evaluate our approach on more realistic scenarios for autonomous agents with video streams of more complex objects in difficult and varying environments. The balance to be found between the quantity of over segmentation and accuracy can be of primary interest in this context as well.

Fig. 1 .

 1 Fig. 1. The evolution of detection (left) and clustering accuracy (right) on Fashion-MNIST during training. The rows represent the 10 classes in the learned order.

Table 1 .

 1 and CURL combined with Hotelling t-squared test as an ablation study. Comparison with the state of the art on MNIST (mean of 3 runs ±SD).

	model	AMI	ARI	# components accuracy
	CURL [3]	0.518 ± 0.013	0.156 ± 0.02	126.6 ± 17.74	1.0 ± 0.0
	CURL with HT 0.6 ± 0.04	0.41 ± 0.025	22.6 ± 1.69	0.87 ± 0.04
	SOINN [4]	0.367 ± 0.002	0.013 ± 0.008	1507 ± 11.34	1.0 ± 0.0
	Ours	0.778 ± 0.012 0.769 ± 0.02	11 ± 1.41	1.0 ± 0.0
	model	AMI	ARI	# components accuracy
	CURL [3]	0.429 ± 0.004	0.1006 ± 0.0111 170.0 ± 0.0	0.993 ± 0.004
	CURL with HT 0.473 ± 0.0054 0.25 ± 0.008	31.3 ± 6.34	0.857 ± 0.03
	SOINN [4]	0.342 ± 0.0016 0.016 ± 0.0003 1009 ± 17.518 1.0 ± 0.0
	Ours	0.57 ± 0.02	0.395 ± 0.03	13.33 ± 0.94	0.798 ± 0.002

Table 2 .

 2 Comparison with the state of the art on Fashion-MNIST (mean of 3 runs ± SD).


domain: sdv scco

text:
Sander Van Bree
email: sandervanbree@gmail.com

Andrea Alamia

Benedikt Zoefel
email: benedikt.zoefel@cnrs.fr

Oscillation or not-Why we can and need to know

Keywords: Entrainment, neural oscillations, brain dynamics, biophysics, cognitive neuroscience Word count

phenomenon of the central nervous system [START_REF] Llinás | The intrinsic electrophysiological properties of mammalian neurons: insights into central nervous system function[END_REF], and evidence for their involvement in cognitive processes has been mounting ever since their discovery [START_REF] Ward | Synchronous neural oscillations and cognitive processes[END_REF].

Yet, identifying oscillations is not always easy. In the presence of a rhythmic stimulus, neural ensembles can show rhythmic behaviour even if they cannot be considered an oscillatori.e., they are unable to produce oscillatory activity on their own, but simply reflect the regularity of the stimulus.

In various fields of research, such as that of "neural entrainment", this has led to immense efforts to distinguish "true" neural oscillations from other neural processes that can produce remarkably similar responses in certain situations [START_REF] Zoefel | The Involvement of Endogenous Neural Oscillations in the Processing of Rhythmic Input: More Than a Regular Repetition of Evoked Neural Responses[END_REF].

In a recent article, [START_REF] Doelling | Neural oscillations are a start toward understanding brain activity rather than the end[END_REF] propose that the question "oscillation or not" hampers progress in these research fields. They argue that the phenomenon "neural oscillation" comprises various heterogeneous neural processes; consequently, the line between oscillating and non-oscillating systems is blurred and therefore meaningless to draw.

Here, we argue against this conclusion and base our argumentation on one crucial notion: Despite their heterogeneity, neural oscillations are a separate class of neural processes. This is because neural oscillations have unique and shared properties. Unique properties are those that only oscillators have. Shared properties are those that all oscillators or all oscillators within a subclass have for example, all oscillators with a specific frequency. Unique properties are the reason we can know whether oscillations are present, as they allow us to distinguish them from non-oscillatory processes.

The existence of such properties therefore contradicts the notion that there is no clear line between oscillating and non-oscillating systems. Shared properties are the reason why we need to know whether oscillations are present, as, once a member of the "oscillation family" (or a subclass) is identified, it can be assumed to possess such properties. Importantly, this means that an answer to the question "oscillation or not" advances certain research fields rather than impedes them, as it constrains the possible neural dynamics involved and allows us to make informed predictions on a variety of levels.

How we can know: Unique properties

The following properties are unique to oscillators and therefore sufficient for their identification.

Unique properties are not necessarily shared by all oscillators. In this case, the absence of such properties does not lead to the conclusion that no oscillator is involved.

• Eigenfrequency: All oscillators have a "natural" frequency, their eigenfrequency. This is the input frequency they respond to most strongly [START_REF] Hutcheon | Resonance, oscillation and the intrinsic frequency preferences of neurons[END_REF], or the frequency they oscillate at in the absence of external stimulation.

• Arnold Tongue: An external, rhythmic stimulus can synchronize a neural oscillator more readily if the stimulus rate is close to the oscillator's eigenfrequency. The more distant the two (stimulus rate and neural eigenfrequency), the higher the stimulus intensity needs to be to reliably synchronize the oscillator. This leads to a characteristic triangular shapean Arnold Tonguewhen the degree of stimulus-brain synchronization is visualized as a function of rate and intensity of the external rhythm (relative to those of the neural oscillator; [START_REF] Pikovsky | Synchronization: a universal concept in nonlinear sciences[END_REF].

• Convergence: If sufficiently close to the eigenfrequency, a periodic force (i.e. an external stimulus) will alter the frequency and phase of an oscillator until they converge to a stable value [START_REF] Fröhlich | Experiments and models of cortical oscillations as a target for noninvasive brain stimulation[END_REF]. The time (e.g., number of cycles) it takes to reach stability depends on the position in the Arnold Tongue (i.e. it varies as a function of rate and intensity of the rhythmic stimulus).

• Independence: Only oscillators show rhythmic behaviour in the absence of a rhythmic stimulus. This property comprises several scenarios: (1) Rhythmic neural activity in the absence of any stimulus input ("spontaneous"); (2) Rhythmic neural activity evoked (and "reset") by non-rhythmic stimulus input (e.g., a pulse); (3) Rhythmic neural activity which outlasts rhythmic stimulus input. How long such "echoes" last (or whether they are present at all) depends on the dampening of the oscillatory system.

Why we need to know: Shared properties

Once the involvement of a neural oscillator in a given phenomenon is established (e.g., based on unique properties), then properties shared by all (or a subclass of) oscillators can be reliably associated with that phenomenon.

• Neurophysiological properties. Having an eigenfrequency is both unique to and shared by all oscillators. In addition, all neural oscillations reflect rhythmic changes in excitability [START_REF] Buzsáki | Neuronal oscillations in cortical networks[END_REF][START_REF] Singer | Neuronal oscillations: unavoidable and useful?[END_REF], produced by neuronal interactions that crucially rely on inhibition [START_REF] Llinás | The intrinsic electrophysiological properties of mammalian neurons: insights into central nervous system function[END_REF]. Oscillations stem from networks hard-wired to produce rhythmic activity on their own, at or close to their eigenfrequency. They therefore entail a component of activity that goes beyond a simple input-output relay of information, and seems optimized for internal control of stimulus processing (see next point). Some properties are shared only by a subclass of oscillators, such as distinct neural architectures underlying different oscillatory frequencies [START_REF] Womelsdorf | Dynamic circuit motifs underlying rhythmic gain control, gating and integration[END_REF].

Knowing the eigenfrequency of an oscillator therefore provides us with valuable information about its neural origins and functional role.

• Functional properties. The functional role of a neural oscillator is fundamentally linked to its rhythmic variation in excitability. If these excitability cycles are synchronised between different neural populations, this leads to efficient neural communication [START_REF] Fries | Rhythms for Cognition: Communication through Coherence[END_REF]. The oscillatory cycle is also a gatekeeper for external stimulation, rhythmically amplifying and attenuating input, a system that can ultimately serve attentional selection [START_REF] Lakatos | Entrainment of neuronal oscillations as a mechanism of attentional selection[END_REF]. Together, a common theme across neural oscillations is the role of gating, routing, and structuring of information. Identifying an oscillation indicates that such a role might be associated with the observed rhythmic brain activity.

• Methodological properties. Once the involvement of a neural oscillator has been established, methods developed to manipulate oscillations can be applied more efficiently. Such methods include transcranial alternating current stimulation (tACS), which has been shown to follow an Arnold Tongue [START_REF] Huang | Transcranial alternating current stimulation entrains alpha oscillations by preferential phase synchronization of fast-spiking cortical neurons to stimulation waveform[END_REF] and produce rhythmic effects that outlast the electric stimulation [START_REF] Van Bree | Sustained neural rhythms reveal endogenous oscillations supporting speech perception[END_REF]. Neural oscillations at specific frequencies can also be manipulated pharmacologically [START_REF] Lozano-Soldevilla | GABAergic Modulation of Visual Gamma and Alpha Oscillations and Its Consequences for Working Memory Performance[END_REF]. We illustrate our line of argumentation with an example. Recent research has shown that the rhythm of visual speech cues modulates the ability to detect an auditory target [START_REF] Biau | Auditory detection is modulated by theta phase of silent lip movements[END_REF]. This effect could merely reflect a passive relay of visual information to the auditory system and lead to a rhythmic modulation of auditory perception only because the information itself is rhythmic. The oscillatory effect, however, only appears after several cycles of visual stimulation, a property that speaks for the involvement of neural oscillations (convergence described above). This conclusion allows us to link the observed phenomenon with other properties oscillations possess. For example, it suggests that vision can control auditory sensitivity by modulating neural excitability, acting as a rhythmic gatekeeper for auditory information. It also predicts that we can use tACS to alter crossmodal perception, but only when we successfully identify and target the eigenfrequency of the oscillating neural system.

Conclusion

We fully agree with [START_REF] Doelling | Neural oscillations are a start toward understanding brain activity rather than the end[END_REF] that oscillations are a "start rather than the end to understanding brain activity". Nevertheless, we have to reach that start. Finding an answer to "oscillation or not" is therefore important and satisfying per se, and will guide the biophysical modelling and theory building that the authors propose. Characterizing the dynamics underlying brain activity should go hand in hand with, rather than replace, testing for the presence of oscillators.

Oscillation or notwe can know and need to know.

Acknowledgements

A.A. is supported via The Agence Nationale de la Recherche, and B.Z. is supported via the Centre National de la Recherche Scientifique.

Conflict of interest

The authors declare no competing interest.