Mira Ait-Saada
email: mira.ait-saada@u-paris.fr

Mohamed Nadif
email: mohamed.nadif@u-paris.fr

Unsupervised Anomaly Detection in Multi-Topic Short-Text Corpora

Unsupervised anomaly detection seeks to identify deviant data samples in a dataset without using labels and constitutes a challenging task, particularly when the majority class is heterogeneous. This paper addresses this topic for textual data and aims to determine whether a text sample is an outlier within a potentially multi-topic corpus. To this end, it is crucial to grasp the semantic aspects of words, particularly when dealing with short texts, since it is difficult to syntactically discriminate data samples based only on a few words. Thereby we make use of word embeddings to represent each sample by a dense vector, efficiently capturing the underlying semantics. Then, we rely on the Mixture Model approach to detect which samples deviate the most from the underlying distributions of the corpus. Experiments carried out on real datasets show the effectiveness of the proposed approach in comparison to state-ofthe-art techniques both in terms of performance and time efficiency, especially when more than one topic is present in the corpus.

Introduction

Anomaly Detection (AD) is a task that can address various objectives such as mining frauds [START_REF] Deng | Combining self-organizing map and k-means clustering for detecting fraudulent financial statements[END_REF], diseases [START_REF] Han | Madgan: Unsupervised medical anomaly detection gan using multiple adjacent brain mri slice reconstruction[END_REF] and intrusions [START_REF] Pu | A hybrid unsupervised clustering-based anomaly detection method[END_REF]. AD takes several forms: supervised, unsupervised or semi-supervised. Unsupervised AD implies that no prior information about the dataset is provided. In this case, the solution usually consists of identifying samples that deviate in a certain way from the others among the same dataset; anomalies being, by definition, rare phenomena. Particularly, anomalies in a textual dataset can be defined as samples having an atypical vocabulary (lexical anomaly) or a deviating global meaning (semantic anomaly). Identifying abnormalities in textual data can be very useful in many industrial use-cases. A good example is the detection of non-eligible and/or fraudu- lent course contents in the public French plateform MonCompteFormation1 where millions of course sessions are available with no possibility of controlling training organizations in a supervised fashion (using labeled data). Hence, to assess the effectiveness of our approach, we rely on an external labeled dataset that closely relates to course contents and that is dedicated to course certifications. The dataset is described in Appendix A. In addition to the difficulty of mining anomalies in short-text corpora of varying sizes, we also have an important computational cost constraint that is also addressed by the proposed solution.

Capturing the semantics of a given text is usually performed using Word Embeddings, which consist in representing a word or a piece of text by a fixed-size vector, supposed to detain its meaning. Several word embedding techniques are available such as word2vec [START_REF] Mikolov | Distributed representations of words and phrases and their compositionality[END_REF], GloVe [START_REF] Pennington | GloVe: Global vectors for word representation[END_REF], fastText [START_REF] Bojanowski | Enriching word vectors with subword information[END_REF], BERT [START_REF] Devlin | BERT: Pre-training of deep bidirectional transformers for language understanding[END_REF], Distil-BERT [START_REF] Sanh | Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter[END_REF], etc. Each of the mentioned works provides ready-to-use models that are pre-trained on very large corpora and intended to be general for a given language and suitable for several NLP downstream tasks. Indeed, relying on such pre-trained models has proved efficient in several tasks [START_REF] Kim | Convolutional neural networks for sentence classification[END_REF][START_REF] Das | Named entity recognition with word embeddings and wikipedia categories for a low-resource language[END_REF] and is particularly useful when dealing with small corpora [START_REF] Buechel | Learning neural emotion analysis from 100 observations: The surprising effectiveness of pre-trained word representations[END_REF]. If we consider the small corpus given in Table 1, we can observe that the inlier samples (training titles about Linux shell programming) do not have any words in common. Thus, based only on the syntactic information of the samples, it would be impossible to isolate the outlier (about learning Spanish), even though it is the only one that does not have anything to do with shell programming. This can easily occur when dealing with short-text corpora, especially when the number of samples is not sufficient to learn the different syntactic variants of a word or a concept.

Depending on the data type and the assumptions that can be made, the definition of an outlier may differ, and the choice of the model is crucial, especially in an unsupervised context [START_REF] Charu | An introduction to outlier analysis[END_REF]. In this paper, we propose a probabilistic AD approach based on Mixture Models, that effectively identifies the most deviating samples in short-text datasets, even in the case where several topics are present in the inlier class. We also show the effectiveness of using the knowledge learned by word embedding models in capturing the underlying semantics of short texts and efficiently identifying outliers. The main contributions of this paper are:

• We address the challenge of mining anomalies in short texts written in French

• We tackle the classical one-class inlier scheme and also a more challenging multi-class inlier setting.

• We propose an effective and efficient anomaly detection approach that outperforms previously proposed AD techniques.

Related Work

Anomaly detection is an active research area, and a large number of approaches are proposed in several application domains. Specifically, our work relates to unsupervised AD for text and clustering-based AD. Unsupervised AD is gaining more and more interest in research due to the constant growth of data volumes while labeling data samples is not getting any cheaper. One of the most important family of methods contains reconstruction-based approaches that assume that a well-generalizing model would struggle at compressing rare anomalous samples. This kind of approaches include linear models such as Robust PCA [START_REF] Kang | Robust pca via nonconvex rank approximation[END_REF] and deep autoencoder models based on convolutional networks [START_REF] Oza | One-class convolutional neural network[END_REF], recurrent networks [START_REF] Hsieh | Unsupervised online anomaly detection on multivariate sensing time series data for smart manufacturing[END_REF], etc.

Clustering-based AD

In the clustering-based AD approaches, anomalies are generally seen as data samples that present a lower adhesion to the underlying groups. Several two-phase approaches have been proposed and consist in using a clustering algorithm such as DB-SCAN (Sheridan et al.), K-means [START_REF] Deng | Combining self-organizing map and k-means clustering for detecting fraudulent financial statements[END_REF] and Affinity Propagation (Marcos [START_REF] Alvarez | Clusteringbased anomaly detection in multi-view data[END_REF], then compute an anomaly score from the obtained clustering partition. Similarly in [START_REF] Mahadevan | Anomaly detection in crowded scenes[END_REF], to detect temporal anomalies in videos, inlier behaviors are modeled as a mixture of Gaussian distributions. A deep GMM-based approach called DAGMM is proposed in [START_REF] Zong | Deep autoencoding gaussian mixture model for unsupervised anomaly detection[END_REF] to detect outliers in numerical data, where the input data are compressed into a lowerdimensional space using an autoencoder and then fed into a GMM component. The autoencoder's reconstruction loss and the log-likelihood of the GMM component are optimized jointly, without performing any pre-training phase.

AD in text data

Unlike images, time series, and numerical data, relatively few AD studies are dedicated to textual data. Document-term matrix representations (also called sparse bag-of-words) have previously been used in (Kannan et al.) to perform AD based on Nonnegative Matrix Factorisation (NMF) and isolate an outlier matrix, used to compute the anomaly scores. Sparse representations are also used by [START_REF] Larry | One-class svms for document classification[END_REF] as input to a One-Class Support Vector Machine (OC-SVM) [START_REF] Schölkopf | Estimating the Support of a High-Dimensional Distribution[END_REF] and later to a shallow autoencoder [START_REF] Manevitz | One-class document classification via neural networks[END_REF]. Word embeddings like word2vec are used for AD in [START_REF] Zhuang | Identifying semantically deviating outlier documents[END_REF] along with a von Mises Fisher (vMF) mixture model where more general words are penalized when computing the overall outlierness score of a given document. Pre-trained fastText word vectors are used in [START_REF] Ruff | Selfattentive, multi-context one-class classification for unsupervised anomaly detection on text[END_REF] as the embedding layer of a multi-head attention network to perform anomaly detection as a one-class classification task.

Recently, a deep end-to-end approach has been proposed by [START_REF] Manolache | DATE: Detecting anomalies in text via selfsupervision of transformers[END_REF] that does not use any knowledge transfer. The authors use the transformer architecture of ELECTRA [START_REF] Clark | Electra: Pre-training text encoders as discriminators rather than generators[END_REF] that contains two adversarial components: a generator and a discriminator. The model is trained from scratch on a given dataset by optimizing a loss function based on token replacement.

Semantic text representations

Tremendous advances in various NLP tasks have been made in recent years thanks to dense vector representations of words and text sequences. Static word embeddings like word2vec [START_REF] Mikolov | Distributed representations of words and phrases and their compositionality[END_REF], GloVe [START_REF] Pennington | GloVe: Global vectors for word representation[END_REF] and fastText [START_REF] Bojanowski | Enriching word vectors with subword information[END_REF] provide one unique dense representation for each word whereas contextual word embedding models like ELMo [START_REF] Peters | Deep contextualized word representations[END_REF], BERT [START_REF] Devlin | BERT: Pre-training of deep bidirectional transformers for language understanding[END_REF] and RoBERTa [START_REF] Liu | RoBERTa: A robustly optimized BERT pretraining approach[END_REF] provide word representations that depend on the surrounding context. Contextual word embedding models are based on deep neural networks, which makes them resource-intensive and difficult to use in some industrial contexts. Both kinds of word embeddings have proved effective in several unsupervised downstream tasks like semantic textual similarity [START_REF] Arora | A simple but tough-to-beat baseline for sentence embeddings[END_REF][START_REF] Ranasinghe | Enhancing unsupervised sentence similarity methods with deep contextualised word representations[END_REF]), clustering (Ait-Saada et al., 2021;[START_REF] Boutalbi | Tensorbased graph modularity for text data clustering[END_REF] and anomaly detection [START_REF] Zhuang | Identifying semantically deviating outlier documents[END_REF].

Gaussian Mixture Models

Given a corpus D of n short texts (d 1 , . . . , d n ), we represent each sample by a fixed size vector, thus obtaining a matrix X = (x 1 , . . . , x n ) of size n × m. To tackle the AD problem, we postulate that the samples follow a mixture of distributions, from which the anomalous samples deviate. Admittedly, the family of t-distributions provides a heavy-tailed alternative to the gaussian family for anomaly detection. However, as pointed out by [START_REF] Yuan | Regularized parameter estimation of high dimensional t distribution[END_REF], although useful from modeling perspective, the practical use of multivariate t-distribution is often limited by the difficulty in parameter estimation, particularly so for high dimensional data. Note that, in our proposal, the consideration of the time consumption is important. Therefore, considering a t mixture model leads to estimate a supplementary parameter (in addition to the estimation of vector means and covariance matrices) that is the degree of freedom of each component. Moreover, since we suggest to consider an ensemble method allowing to combine results by varying the number of components (cf. Section 3.1), we would therefore increase yet the computation time for estimation of the parameters. However, Gaussian Mixture Model (GMM)-based approaches, more parsimonious than t-mixture model, have shown their effectiveness in anomaly detection, such as DAGMM [START_REF] Zong | Deep autoencoding gaussian mixture model for unsupervised anomaly detection[END_REF]. For these reasons we retain GMM to address our purpose.

In a finite GMM, the data (x 1 , . . . , x n ) are taken to constitute a sample of n independent instances of a random variable X in R m . Density can be expressed as:

f (x i ; Θ) = g k=1 π k φ k (x i |µ k , Σ k ), ∀i ∈ {1, . . . , n} where Θ = (π 1 , . . . , π g , µ 1 , . . . , µ g , Σ 1 , . . . , Σ g ), φ k (x i |µ , Σ k ) is the kth component density for observation x i with parameters (µ k , Σ k ), (π 1 , . . . , π g-1 )
are the mixing weights or probabilities (such that π k > 0, g k=1 π k = 1) and g is the number of mixture components. Thus, clusters are ellipsoidal, centered at the mean vector µ k , and with other geometric features, such as volume, shape and orientation, determined by the covariance matrix Σ k [START_REF] Banfield | Modelbased gaussian and non-gaussian clustering[END_REF][START_REF] Celeux | Gaussian parsimonious clustering models[END_REF]. To estimate Θ we rely on the maximisation of the log-likelihood given by:

L(X; Θ) = n i=1 log g k=1 π k φ k (x i |µ k , Σ k ) .
The maximization is commonly performed by Expectation-Maximization [START_REF] Dempster | Maximum likelihood from incomplete data via the em algorithm[END_REF]; an iterative algorithm based on the maximization of the conditional expectation of the complete data log-likelihood given Θ ′ :

Q(Θ|Θ ′ ) = i k s ik log(π k φ k (x i |µ k , Σ k ))
where

s ik ∝ π k φ ℓ (x i |µ k , Σ k ) are the posterior probabilities.
In real terms, the algorithm is broken down into two steps (E-M steps) and the unknown parameters of Θ are updated thanks to the previously computed probabilities. For each component k, we have

π k = i s ik n µ k = i s ik x ij i s ik and Σ k = i s ik (x i -µ k ) ⊤ (x i -µ k ) i s ik .
The procedure used to identify anomalies is described in Algorithm 1. It takes as input a set of short texts and returns the ones that are the most likely to constitute an anomaly. The maximum density as normality score denotes the confidence Algorithm 1: AD with GMM Input: D = {d 1 , . . . d n }, g the number of components, M an embedding module, α the desired number of output samples;

x i ← M(d i ), i = 1, . . . n; X ← (x 1 , . . . , x n ); Initialize Θ ′ from a partition obtained with k-means repeat E-step: Compute Q(Θ|Θ ′ ); M-step: Update π k , µ k and Σ k ; until Convergence; s i ← -max k (s ik ), k = 1, . . . , g; s ← (s i , . . . , s n ); r ←argsort(s); return d j , j = r 1 , . . . , r α ;
of the assignment. Multiplied by -1, it denotes the uncertainty of the assignment and is similar to using the entropy of s since k s ik = 1. The number of returned text samples depends on the user's needs and is specified by the cutoff parameter α. In the evaluation section, we evaluate the AD performance with every possible value of α using the AUROC score.

Proposed solution for multi-class inliers

In the standard setting of AD where we consider one large inlier class, we set the number of components to its smallest possible value g = 2, which provides satisfactory results. In this study, we also consider the more challenging scenario where several underlying topics are present in the dataset. In this context, we make the distinction between extreme values and outliers (Aggarwal, 2017) as shown in Figure 1. A Gaussian mixture model would not have any difficulty in spotting both types of outliers since it is capable of modeling clusters of different shapes. Furthermore, we expect GMM to show good results in the multi-class context, since one of its fundamental assumptions is the multiplicity of inherent distributions among the data samples. However, this property requires to know the number of components in advance, which is not always possible in real life.

To address this issue, we propose to use GMME, an ensemble of several models, obtained with different values of g. To this end, we use Algorithm 1 with varying g k ∈ G and combine the output scores s (g k ) as follows:

e i = - k rank(s (g k ) i
).

The intuition behind using an ensemble approach is to make each of the models separate the dataset into clusters in a different way and assign an anomaly score according to the formed clusters. Combining those different anomaly scores leads to a more robust and meaningful overall score, even when the optimal number of clusters is not included in G. This is corroborated by the empirical study conducted in Section 4.5.

Experimental Study

To assess the effectiveness of our approach and compare it to state-of-the-art, we conduct a set of experiments on real datasets and discuss the results in this section.

Datasets

We run our AD experiments on three datasets described in Table 2. MLSUM [START_REF] Scialom | MLSUM: The multilingual summarization corpus[END_REF] and COVID-news [START_REF] Cortal | Covid-19 -french news dataset[END_REF] Given a classification dataset, we first remove the classes that are too small to constitute an AD dataset, thus obtaining ℓ classes. We then derive ℓ sets of samples in which there is one inlier (majority) class and a certain rate r of outliers picked randomly from the other classes.

Experimental settings

In order to empirically evaluate our approach and compare it to other AD techniques, we rely on the Area Under the Receiver Operating Curve (AU-ROC), originally used as a metric in the classification task. In our case, it takes as input the anomaly scores as well as the ground truth labels and determines at what extent it is possible to accurately identify outliers using the anomaly score. It is equivalent to evaluating the performance of AD at every possible value of α in Algorithm 1. For each AD approach, we compute the AUROC on the test set over 5 different initializations, except for OC-SVM that is a deterministic model.

In our study, we discard the case where we train the model on the majority class as a one-class classification task [START_REF] Ruff | Selfattentive, multi-context one-class classification for unsupervised anomaly detection on text[END_REF][START_REF] Manolache | DATE: Detecting anomalies in text via selfsupervision of transformers[END_REF][START_REF] Larry | One-class svms for document classification[END_REF], for it is not a realistic scenario since one rarely has access to a large enough amount of inlier-only labeled samples in real life. Thus, we consider in this study a fully unsupervised scenario, where no labels are available and both inlier and outlier samples are present in the training set. To this end, we contaminate both sets with up to r = 10% of outliers as in [START_REF] Manolache | DATE: Detecting anomalies in text via selfsupervision of transformers[END_REF]. The set of labels is used only during the evaluation phase.

Text representation. Given a raw corpus D of short texts, we first perform a minimal preprocessing that consists in removing stop words and lowercasing the input text. Then, we use a pretrained fastText model [START_REF] Bojanowski | Enriching word vectors with subword information[END_REF] to represent texts by fixed-size vectors and build the X matrix. The model is trained on French Wikipedia and represents each word by a vector of size m = 300. Using those word vectors, we represent a text sequence by the arithmetic mean of its tokens' representations as in [START_REF] Ranasinghe | Enhancing unsupervised sentence similarity methods with deep contextualised word representations[END_REF][START_REF] Arora | A simple but tough-to-beat baseline for sentence embeddings[END_REF]. We show that this way of representing text sequences is well suited to shorts texts and is very beneficial in capturing text semantics for AD. One noticeable advantage of fastText is its ability to represent out-of-vocabulary words thanks to sub-word embeddings. Another advantage of fastText is that pre-trained word representations are provided in a wide range of languages2 . We do not use contextual word embeddings to represent text sequences since they significantly increase the computational cost and do not seem to bring any performance gain in the AD task [START_REF] Ruff | Selfattentive, multi-context one-class classification for unsupervised anomaly detection on text[END_REF].

Baselines. We compare our approach to other AD techniques: OC-SVM [START_REF] Schölkopf | Estimating the Support of a High-Dimensional Distribution[END_REF], AE, DAGMM [START_REF] Zong | Deep autoencoding gaussian mixture model for unsupervised anomaly detection[END_REF] and DATE [START_REF] Manolache | DATE: Detecting anomalies in text via selfsupervision of transformers[END_REF]. For OC-SVM, AE and DAGMM, we use as input the same matrix X as for GMM. Concerning OC-SVM, we set ν = 0.05, which is the value that presents the best results among {0.05, 0.1, 0.2, 0.5} by far. For the autoencoder (AE) we train a model with three encoder layers and three decoder layers of size 256, 128 and 64, a learning rate of 0.001 and a weight decay of 10 -8 . Concerning DAGMM, the authors choose the parameters relating to the architecture of the neural network according to the dataset and do not provide a method to reproduce this choice. This way of configuring the model is not suitable for the unsupervised case, in which no tuning of the hyperparameters is possible. We therefore opt for a standard architecture in decreasing powers of 2 starting from m = 300 (i.e. 256, 128, . . . ). We choose m ′ = 5 as the encoding dimension because it gives the best overall performance. For DATE, we use the same parameters as used in the original paper for AG-news [START_REF] Manolache | DATE: Detecting anomalies in text via selfsupervision of transformers[END_REF].

Hyperparameter tuning. Our present work falls within the context of unsupervised learning, where we are not supposed to have access to ground truth labels. In this respect, it is not feasible in real life to tune the hyperparameters of a given model to fit unlabeled data, since the performance score is simply impossible to compute. We therefore consider hyperparameter tuning in this context unrealistic and possibly leading to hiding instabilities that can neither be detected nor fixed by practitioners. Hence, robustness and insensitivity to parameters are key in the unsupervised setting. For this reason, regarding the baselines, we use the recommended parameters provided in the original works when available, in order to reproduce real-world conditions. When no recommended parameters are provided, we use the ones that maximize the overall performance even though it does not play in favor of our proposed approach which does not require any parameter tuning. By doing so, we guaran- tee a fair evaluation of our proposal and show its robustness in an unsupervised context.

Results with one-class inliers

The obtained performance is given in Table 3 with an anomaly rate r = 10%. We first observe the effectiveness of GMM on the three datasets, in comparison to all of the baselines, offering the best AUROC in most of the cases. Furthermore, the state-of-the-art DATE shows its limits on short texts and presents competitive but poorer results in comparison to GMM. OC-SVM is competitive on short texts in comparison to DATE but presents poorer overall performance. AE is the model that provides the lowest AUROC values, right after DAGMM. This might be due to the fact that the encoding (or embedding) step of the encoder (for both AE and DAGMM) is performed beforehand using pretrained word embeddings and becomes pointless when applied to this kind of representation.

The results obtained with different values of contamination rate r are summarized by Critical Difference (CD) diagrams in Figure 2. The aim of CD diagrams [START_REF] Demšar | Statistical comparisons of classifiers over multiple data sets[END_REF] is to visualize the performance ranks of each approach over the different datasets. If we take the example of r = 10%, the CD diagram summarizes the scores given in Table 3. It depicts the average rank of each method and the bold line corresponds to the critical difference, based on the post-hoc Nemenyi test [START_REF] Peter | Distribution-free multiple comparisons[END_REF]. Note that the presence of a higher number of outliers increases the difficulty of the AD task and decreases the overall performance scores. We can observe that the more we inject anomalies in the training set, the more GMM gets competitive against the other approaches in terms of AUROC score. This shows GMM's robustness to outliers and its generalization capabilities while other techniques tend to overfit in the presence of noise in the training set. Also, it is worth noting that GMME yields similar results in comparison to GMM, which makes it a universal solution, well suited to AD even in the one-class scenario. Comparison with VMF-Q. To make a fair comparison between GMM and VMF-Q [START_REF] Zhuang | Identifying semantically deviating outlier documents[END_REF], we reproduce the exact same setting for the two approaches. VMF-Q is based on the von Mises-Fisher distribution that relies on a Bessel function to estimate the parameter κ. The model is originally trained using embeddings of size m = 200, but encounters numerical difficulties with higher dimensions, due to the approximations made by the Bessel function that depends, inter alia, on m. We hence use, for both GMM and VMF-Q, another pretrained model of size m = 200, that is provided by [START_REF] Fauconnier | French word embeddings[END_REF]. The gain of performance from VMF-Q to GMM is summarized in Figure 3. We can observe a clear advantage of GMM in comparison to VMF-Q on the three datasets, especially on MLSUM, where GMM outperforms VMF-Q on all the subsets. We also report poorer overall results using word2vec with m = 200 in comparison to the fastText model we use in the rest of our experiments.

Performance and data size

Figure 4 shows the gain of performance from baselines to GMM according to the size of the datasets. We see that the percentage of improvement is greater on small datasets but remains positive on large datasets. Note that detecting anomalies can be trickier on small datasets, especially when dealing with short texts (cf. Table 1) which makes GMM a good solution to tackle this difficulty.

Multi-class inliers

We investigated in the previous sections the detection of semantic anomalies in a dataset classically composed of one unique class. In this section, we consider a dataset with several underlying topics and identify the samples that do not belong to any of them. To evaluate our approach in such a context, we create datasets as described in Section 4.1 but this time, by combining ĝ random classes to form one inlier class then inject anomalous samples with a rate r = 10%. We then proceed similarly to identify anomalies and validate the obtained results. Note that we make sure in our experiments to put aside enough "anomalous" classes so that we have a sufficient diversity of anomalies and avoid forming an additional cluster with the anomalous samples. To this end, we limit ĝ according to the available classes in the dataset. We use the obtained datasets to assess the performance of GMME (cf. Section 3) and compare it to GMM and two other baselines: DBSCAN and DATE. DBSCAN natively deals with outliers and considers the samples that cannot be assigned to any existing cluster as anomalies (they are assigned the -1 label). We use this information to determine whether a text sample is an anomaly. We set the parameters ϵ = 1 and min_samples = 3 which yielded better results than the default values available in the scikit-learn implementation.

Table 4 shows the performance obtained by GMME with values of g k ∈ G = {2, 3, 4}. We first observe that the ensemble approach further improves the performance of the simple GMM, which still performs better in comparison to the other baselines. DBSCAN presents the poorest results after DATE, which is less competitive in the multi-class context, especially on the RNCP dataset. Thus, GMME is the most competitive approach in the multi-topic scenario, even when the real number of clusters is not included in G. We first notice that GMM is the fastest approach both during the training and evaluation phase. It is followed by GMME that is relatively quick, especially during evaluation. GMM scales better with an increasing number of samples in comparison to OC-SVM that takes more than four times as much time to train. DATE is the approach that takes the most time to train and evaluate, which is due to its deep architecture. The computational time of DATE can be partially amortized with a more powerful GPU but can still represent an impediment, especially in an industrial context.

Computation time analysis

We also observe that the computation time of X does not depend much on the size of the dataset, simply because the task that takes the longest is loading the model from memory. Hence, this way of representing text scales well, especially when used along with GMM or GMME.

Improve the results with Transformers

In the previous experiments, we relied on fastText static embeddings in order to show that is it possible to perform effective and fast anomaly detection without having access to important computational resources. In this section, we show that it is possible to achieve even better results using Transformer representations. For our purpose, we use Camem-BERT [START_REF] Martin | CamemBERT: a tasty French language model[END_REF] and FlauBERT [START_REF] Le | FlauBERT: Unsupervised language model pre-training for French[END_REF], both trained on French corpora.

CamemBERT and FlauBERT are both based on the RoBERTa variant of Transformer language models [START_REF] Liu | RoBERTa: A robustly optimized BERT pretraining approach[END_REF]. While FlauBERT uses the exact same objective and tokenization process as in [START_REF] Liu | RoBERTa: A robustly optimized BERT pretraining approach[END_REF], plus a French-specific preprocessing, CamemBERT makes use of the Sen-tencePiece tokenizer and the whole-word masking strategy that consists in masking words instead of sub-word tokens. Another major difference between the two models is the training set of data. CamemBERT uses the French part of OSCAR, a large multilingual corpus extracted from Common Crawl, while FlauBERT is trained on a set of 24 corpora.

Table 6 contains the results obtained using several input representations. We recall the results obtained by DATE for comparison purposes since it is also based on a Transformer architecture. We first observe the significant difference in performance between CamemBERT and FlauBERT with a clear advantage for CamemBERT in both its base and large versions. The large variant of Camem- BERT achieves the best overall results, especially on MLSUM. CamemBERT-base yields competitive results, achieving good AUROC scores on the COVID dataset. We also notice that, in all cases, the two-phased approach that consists in computing Transformer representations and then use an ensemble of model-based clustering models (GMME) is more efficient than the end-to-end Transformerbased approach (DATE). It is worth noting that fastText embeddings remain competitive and surpass the three FlauBERT models. It is therefore a very good alternative in real-time use cases when computational speed is a critical issue.

Conclusion

This paper addresses semantic anomaly detection in short texts with an additional constraint in time efficiency. In addition to the classical framework where one class is used as the inlier class, we also consider the scenario where several underlying subgroups are present in the normal class. We see anomaly detection as a probabilistic clustering problem, in which we learn a Gaussian mixture model and consider the low posterior probability samples as belonging to none of the modeled clusters and more likely to constitute outliers. This uncertainty score proved effective with different numbers of subgroups. In the multi-class setting, we propose GMME, an ensemble approach that improves the performance of GMM when several topics are present in the inlier class. The two approaches outperform state-of-the-art anomaly detection techniques in both scenarios, with an impressively low computation time.

In our proposal, we rely on the Gaussian Mixture model for its flexibility. This choice is motivated by the presence of the proportions π k of each cluster and the spectral decomposition of the covariance matrix Σ k taking into account the volume, shape, and orientation of each cluster (as depicted in Figure 1). The characteristics of the clusters should not be overlooked when tackling the problem of anomaly detection through a clustering approach. Furthermore, note that our approach can be extended to latent block models, devoted to coclustering, which may constitute an interesting and promising future path of research.

Limitations

This paper deals exclusively with French text corpora to answer a specific industrial need. However, we are confident about the fact that this work can easily be extended to other languages, especially English, for which more data and pre-trained models are available. This would constitute an interesting trajectory for future work.

Figure 1 :

 1 Figure 1: Difference between outlier and extreme value. This example illustrates the benefit of the clusteringbased AD approaches in general and Gaussian mixture models in particular.

Figure 2 :

 2 Figure2: CD plots from the Nemenyi test over different datasets. This graphic summarizes the rank of each approach with different contamination rates r.

Figure 3 :

 3 Figure 3: Gain of performance between GMM and VMF-Q using word2vec embeddings of size m = 200. Positive values give an advantage to GMM.

Figure 4 :

 4 Figure 4: Percentage of improvement of GMM in comparison to baselines w.r.t. to the train set's size. Positive values give an advantage to GMM.

Table 1 :

 1 Example illustrating the importance of semantic representations in a small corpus of short texts.

	Les lignes de commande Linux pour débutants	inlier
	Formation: Introduction au Shell Bash	inlier
	Administration système Unix pour les nuls	inlier
	Apprendre à utiliser le terminal Ubuntu/Debian inlier
	Formation en Espagnol pour débutants	outlier

  are both news datasets from which we extract the title to constitute our short-text corpora. RNCP is a dataset we built from an official French repository that lists training certifications (cf. Appendix A).

	Dataset Classes Smallest Largest Medial
	RNCP	16	763	9,510	3,101
	COVID	9	236	3,235	1,270
	MLSUM	10	2,573	26,024 13,054

Table 2 :

 2

Datasets' description. The sizes correspond to the whole set of samples (training and test set).

Table 3 :

 3 AUC scores obtained with anomaly r = 10%. The bold numbers correspond to the best score in each row and the underlined numbers are for the second best performance score.

		Inlier	OC-SVM AE DAGMM DATE GMM
	RNCP	enviro. défens. intell. recher. nautis. aérono. sécuri. multim. humani. nucléa. enfanc. saison. assist. sport ingéni. sans d.	62.2 58.0 66.7 68.6 64.9 63.7 59.8 57.8 56.7 65.1 61.0 50.5 45.1 51.1 66.9 45.4	62.1 46.2 65.4 64.5 62.9 59.4 54.7 56.3 59.6 58.7 60.1 47.5 50.7 56.1 60.1 36.9	52.8 52.5 59.1 58.0 53.0 49.8 51.3 51.3 53.4 59.9 65.0 50.8 56.4 46.8 57.2 53.7	65.8 62.2 79.0 75.3 68.2 76.1 75.0 64.5 69.0 74.9 72.1 48.8 52.5 66.3 75.3 39.3	68.4 74.3 80.1 77.7 73.1 78.0 80.0 71.2 72.1 75.1 78.9 74.9 63.2 73.3 74.2 59.9
	COVID	cultur. enviro. intern. people. politi. scienc. sociét. sport économ.	49.7 56.9 62.0 63.9 67.2 55.5 48.9 68.5 52.7	40.1 55.0 53.7 47.4 60.1 38.3 47.5 35.6 49.4	43.4 50.2 53.1 49.7 51.6 60.6 55.3 55.4 50.7	40.3 55.2 62.4 51.8 68.1 64.5 57.0 52.6 54.0	53.7 66.2 65.1 64.0 79.2 66.6 55.1 69.0 59.4
	MLSUM	afriqu. police. politi. scienc. societ. sante argent. livres. cultur. sport	70.0 78.3 70.3 53.1 73.0 78.7 47.7 71.2 59.5 74.9	50.5 74.4 60.3 35.5 73.0 56.2 36.4 65.8 49.8 56.5	62.0 52.9 49.6 49.6 48.9 55.2 51.7 48.4 52.4 45.2	74.3 77.0 74.6 71.0 75.4 86.5 82.2 58.1 45.8 69.2	75.5 82.9 75.7 55.2 79.0 87.5 63.4 65.9 51.4 81.7

Table 4 :

 4 Comparison of DBSCAN, DATE, GMM and GMME approaches: AUROC scores obtained with multiple classes as inliers.

	# inlier DBSCAN DATE GMM GMME classes
	RNCP	2 3 4 5 6 7 8	63.4 62.8 64.6 65.7 61.7 58.4 60.4	75.6 70.5 77.6 63.5 56.7 48.8 56.8	88.6 82.4 80.2 81.6 65.5 60.6 63.2	89.2 83.9 82.0 80.7 64.5 62.6 63.4
	COVID	2 3 4 5	51.5 50.6 50.7 50.0	59.3 51.3 53.5 51.1	65.2 54.1 56.2 52.6	63.8 54.2 56.3 52.7
	MLSUM	2 3 4 5	50.5 50.2 50.6 50.7	70.0 49.6 47.0 45.7	74.2 55.1 55.0 53.1	74.6 54.4 55.5 53.8

Table 5

 5

	contains the execution time of both the
	training and evaluation steps. It is estimated over
	three different runs, on the three-class datasets used
	in Section 4.5. The sizes of the training sets are
	1034, 6342, 46253 and the test sets are of size 444,
	2718, 5837 for RNCP, COVID-news and MLSUM
	respectively. The experiments on DATE are per-
	formed on an NVIDIA RTX2070 GPU.

Table 5 :

 5 Execution time in seconds. The row X corresponds to the computation time of the embedding matrix X using fastText. "train" stands for the training time and "eval" for the evaluation time.

			1-SVM	GMM	GMME	DATE
	RNCP	X train eval	3e-01 1e-01	3.7 3e-01 3e-03	1.2 3e-02	-78.49 2.0
	COVID	X train eval	5.7 1.8	3.7 3.5 2e-02	16.2 8e-02	-512.97 14.1
	MLSUM	X train eval	450.6 33.0	3.9 10.0 4e-02	94.8 2e-01	-3639 30.2

Table 6 :

 6 Comparison between different embedding representations in terms of AUC score with anomaly rate r = 10%. The bold numbers correspond to the best score in each row and the underlined numbers are for the second-best performance score. flauB and camB stand respectively for FlauBERT and CamemBERT and the subscript is for the number of layers, which indicates the model size (6 for small, 12 for base and 24 for large).

	Dataset	# Inlier classes	DATE	fastText flauB 6	GMME flauB 12 flauB 24	camB 12	camB 24
	RNCP	2 3 4 5 6 7 8	75.58 70.48 77.58 63.50 56.74 48.78 56.82	89.22 83.90 82.00 80.68 64.46 62.60 63.44	87.02 76.10 79.14 69.76 60.48 64.68 53.18	59.78 64.80 70.82 47.78 53.40 67.24 47.28	71.38 63.38 66.06 69.80 60.48 66.42 59.92	83.82 76.98 80.82 77.60 64.60 67.84 65.16	88.44 82.14 84.82 83.54 66.64 68.66 67.00
	COVID	2 3 4 5	59.28 51.32 53.46 51.08	63.82 54.20 56.30 52.70	62.34 54.10 55.40 56.28	56.38 47.28 49.70 49.02	56.56 54.86 51.80 50.48	65.82 56.00 58.00 52.90	63.32 55.48 58.44 52.12
	MLSUM	2 3 4 5	70.00 49.60 47.02 45.70	74.60 54.44 55.52 53.84	68.80 59.70 58.30 58.80	61.00 47.80 39.00 42.30	55.00 56.50 54.70 56.02	79.60 66.44 65.02 65.44	85.10 72.74 70.18 71.08

https://www.moncompteformation.gouv. fr/ developed by « Caisse des Dépôts et Consignations » (CDC)

https://fasttext.cc/

A Construction of the RNCP dataset

The RNCP 3 dataset is composed of certification contents that are provided by an organization named « France-Compétences ». All the data used here are publicly available in https://www.da ta.gouv.fr/fr/datasets/.

In order to label data we first get the ROME codes of each certification that correspond to the 3 Répertoire National des Certifications Professionnelles 1402 professions related to the certification. Each ROME code is assigned to a topic in the file « Arborescence thématique » of the same repository. We use the ROME code as intermediate to have the topic (thématique) of each certification. We hence obtain a multi-label classification dataset.

Given a certification c i , let T i be the set of its corresponding topics. In the one-class setting with inlier class h, the label of c i is established as follows:

In the multi-class setting with inlier classes symbolized by H, the anomaly labels are defined as:

where y i = 1 means c i is categorized as an anomaly.

B AD examples with RNCP

Table 7 presents some examples of anomalies predicted on two subsets of the RNCP dataset: « aéronotique » (meaning aeronautics) and « nucléaire » (meaning nuclear). The aéronautique test set contains 1585 samples, 154 of which are labeled as anomalies, and the nucléaire set contains 1431 samples including 131 anomalies. In both cases we set α to 250 (cf. Algorithm 1). We observe in both cases that DATE has more difficulty in detecting anomalous text sequences when they are very short. For example, in the nucléaire set, the certification Livreur (meaning delivery person) does not have anything to do with the nuclear field. Yet DATE does not place it among the 250 most deviant samples and makes it the 355th anomalous sample while it is only 3rd according to GMM. This might be explained by the fact that DATE is trained from scratch and does not benefit from the semantic knowledge inherited by transfer learning.

Subset


D Zeka

P.-A Guidault

D Néron

M Guiton

Presqu'île de Giens (Var) On the benefits of a multiscale strategy to model-order reduction for frictional contact problems

Keywords: multiscale strategy, model-order reduction, frictional contact

In this work, the efficiency of a multiscale strategy based on a domain decomposition method (DDM) for model-order reduction of friction contact problems is presented. The resolution strategy is based on the LATIN method combined with the Proper Generalized Decomposition (PGD). The effectiveness and possible improvement points of the method will be illustrated on a one-dimensional frictional benchmark problem which features a large frictional contact interface and sliding front.

Introduction

The direct numerical resolution of large-scale, time-dependent nonlinear problems still nowadays remains a cumbersome challenge, especially for frictional contact problems. There are in this regard a range of applications where one should not negotiate on the quality and accuracy of the solution. It is the case, for example, of helically wounded steel wires, widely encountered in the energy and power transmission fields. The life prediction of such structures involving fretting fatigue phenomena between their constituent wires is crucial. However, wire-scale simulations for varying tension and bending during service life require effective computational and modeling strategies [START_REF] Bussolati | Robust contact and friction model for the fatigue estimate of a wire rope in the mooring line of a floating offshore wind turbine[END_REF]. Examples include multigrid acceleration strategies or DDM techniques for numerical resolution parallelization [START_REF] Dostál | A theoretically supported scalable TFETI algorithm for the solution of multibody 3D contact problems with friction[END_REF]. A different path consists in adopting reduced-order models (ROM) methods to decrease the computational cost by seeking the solution of the given problem in a reduced-order basis (ROB). Nevertheless, a ROB may not easily and efficiently capture nonregular and propagating multiscale phenomena that occur at contact interfaces : sliding, sticking and separation zones being difficult to represent. Indeed, for specific problems where high accuracy of local contact quantities is required, ROM alone may not guarantee high accuracy while ensuring at the same time a significant decrease in computational cost. As highlighted in [START_REF] Giacoma | Toward an optimal a priori reduced basis strategy for frictional contact problems with LATIN solver[END_REF][START_REF] Guidault | Model order reduction for the fatigue life prediction of wire ropes in tension and bending[END_REF], frictional contact problems present a multiscale content, with global modes on the structural level and localized modes bringing corrections at the contact interfaces. This suggests that proposing a model reduction method within a multiscale approach may be truly beneficial for problems of this type.

Here, the mixed DDM strategy based on the LATIN method [START_REF] Ladevèze | Nonlinear Computational Structural Mechanics -new approaches and non-incremental methods of calculation[END_REF] is considered. The LATIN (LArge Time INcrement) is a non-incremental solver for nonlinear problems which iterates on the whole spacetime domain with an iterative scheme which shares similarities with augmented Lagrangian formulations and Uzawa-like algorithms, known to be robust for contact problems [START_REF] Simo | An augmented lagrangian treatment of contact problems involving friction[END_REF]. The mixed nature of the method, with both interface displacements and interface forces as unknowns, enables one to deal with interfaces characterized by different behaviours with a single resolution method. Moreover, the global space-time approach makes the method suitable for on-the-fly model-order reduction based on the PGD [START_REF] Nouy | A priori model reduction through proper generalized decomposition for solving time-dependent partial differential equations[END_REF]. A multiscale version of the LATIN-based mixed DDM was introduced in [START_REF] Nouy | Multiscale computational strategy with time and space homogenization : a radialtype approximation technique for solving microproblems[END_REF][START_REF] Ladevèze | A multiscale computational approach for contact problems[END_REF][START_REF] Ladeveze | On a multiscale computational strategy with time and space homogenization for structural mechanics[END_REF]. However, no particular focus has been paid on frictional problems per se, with a few contact interfaces surrounded by other material nonlinearities that reduce the influence of the frictional interfaces on the whole structure.

This work is intended to highlight how and to what extent a multiscale approach, in particular one based on DDM, may be helpful in efficiently solving frictional contact problems with a model reduction approach. To this end, the multiscale strategy is briefly presented and applied to a one-dimensional frictional contact problem. The strengths of the method are presented and possible strategies for improvement are proposed.

A multiscale strategy for frictional contact problems

The reference problem of the equilibrium of an elastic structure, under the assumptions of small displacements and an isothermal quasi-static state, occupying the space domain Ω on the time interval [0, T ], is considered. The structure is subjected to time-dependent body forces f d , imposed loads F d on a portion ∂ 2 Ω of the boundary as well as prescribed displacements u d on the complementary part ∂ 1 Ω. The basic idea consists in describing the structure as an assembly of substructures and interfaces, where each substructure has its own variables and equations.

A substructure Ω E ⊂ Ω, E ∈ E, with E being the set of substructures, is subjected to the action of its environment (i.e., the neighbouring substructures V E ) by a force distribution F E and a displacement field W E on its boundary ∂Ω E . The kinematic admissibility condition for W E is denoted with E E,ad and F E,ad represents the static admissibility conditions for F E . The admissibility of the couple s E = (W E , F E ) is denoted by S E,ad and includes the verification of the linear elastic constitutive relation σ E = K : ε E , with K being Hooke's tensor. An interface Γ EE ′ between substructures Ω E and Ω E ′ , on the other hand, must verify equilibrium conditions and the interface constitutive behaviour b EE ′ = 0, which depends on the nature of the interface (perfect interface, contact interface, boundary condition interface etc.).

The introduction of a multiscale strategy is operated at the interface level. The interface unknowns

s E = (W E , F E ) are split into s E = s M E + s m E
prior to any discretization, where s M E is the set of the macro quantities and s m E the complementary set of micro quantities. The macroscale is defined by the characteristic length of the interfaces, which is a priori greater than the discretization on the microscale. The most important feature of the multiscale strategy is that the equilibrium conditions at the interfaces are partially verified a priori on interface macroquantities :

∀E ∈ E, ∀E ′ ∈ V E , F M E + F M E ′ = 0. (1)
The corresponding admissibility space is denoted with F M ad . For further details on the multiscale strategy one may refer to [START_REF] Ladevèze | A multiscale computational approach for contact problems[END_REF][START_REF] Ladeveze | On a multiscale computational strategy with time and space homogenization for structural mechanics[END_REF].

The multiscale strategy within the LATIN framework

The decomposition into substructures and interfaces with their own equations to be satisfied leads to partition the reference problem into two manifolds of solutions. The manifold A d , corresponding to the set of the linear and possibly global equations pertaining to the substructures, and the manifold Γ representing the set of the local and possibly non-linear equations at the interface level. With this partitioning, it is possible to apply the LATIN method [START_REF] Ladevèze | Nonlinear Computational Structural Mechanics -new approaches and non-incremental methods of calculation[END_REF]. The solution is therefore sought iteratively with a two-search direction alternate algorithm between A d and Γ, as shown schematically in Fig. 1. Other interface behaviours such as perfect linkage interfaces or boundary conditions interfaces can be also taken into account thanks to the mixed nature of the strategy.

Problem 2.1 (local stage). Find s = { s E } E∈E , verifying : equilibrium of forces : F E + F E ′ = 0 Γ interface constitutive behaviour b EE ′ ( W E , W E ′ , F E , F E ′ ) = 0 search direction E + : F E -F E -k( W E -W E ) = 0 From s n+1/2 ∈ Γ,
A d the admissibility of F M : F M ∈ F M ad search direction E -: F E -F E + k(W E -W E -W M E ) = 0
In the search direction E -, the quantity W M E represents a Lagrange multiplier introduced to enforce the equilibrium conditions (1). It is computed at every iteration by solving a coarse-scale problem on the whole structure. The coarse-scale problem allows macro quantities to be propagated among all substructures and ensures numerical scalability over the number of substructures. [START_REF] Ladeveze | On a multiscale computational strategy with time and space homogenization for structural mechanics[END_REF]. The quantity k appearing in the search direction equations is the search direction parameter, homogeneous to a stiffness. The value of k affects the convergence rate of the problem, but not the solution at convergence. Reference values for k usually adopted in each substructure are given by the stiffness of the substructure itself.

In order to control the convergence of the LATIN algorithm, one can build error indicators based on the distance between two consecutive solutions belonging to each of the two manifolds. The usual error indicator adopted, introduced in [START_REF] Ladevèze | Nonlinear Computational Structural Mechanics -new approaches and non-incremental methods of calculation[END_REF], is the LATIN indicator :

η := ∑ E ∥s -s∥ 2 1 2 ∑ E (∥s∥ 2 + ∥ s∥ 2 )
, with ∥□∥ 2 :=

∂Ω E ×[0,T ] (kW 2 + 1 k F 2 )dΓdt. (2)
Such an error indicator characterizes the global distance in space and time between two consecutive solutions of the linear and local stages blending together displacements and interface forces with the search direction parameter k. The LATIN indicator, by definition, does not allow to accurately control the solution locally in space and time. A more stringent indicator has been proposed in [START_REF] Ribeaucourt | A new fatigue frictional contact crack propagation model with the coupled X-FEM/LATIN method[END_REF]. In both cases, however, the indicator depends on k. In [START_REF] Passieux | Approximation radiale et méthode LATIN multiéchelle en temps et espace[END_REF], a novel error indicator was introduced, which is based solely on the non-satisfaction of the constitutive behaviour at the interfaces.

Approximation of the linear stage : Proper Generalized Decomposition

The linear stage at iteration n + 1 consists in solving a set of linear problems for each substructure over all the space-time domain. A PGD approximation of the linear stage can be introduced in order to speed up the computations [START_REF] Ladeveze | On a multiscale computational strategy with time and space homogenization for structural mechanics[END_REF]. First of all, the solution s n+1 at the current iteration can be equivalently written in term of corrections with respect to the previous iteration solution s n+1 = s n + ∆s, with ∆s = (∆W , ∆F). Given the linearity of the equations pertaining to the manifold A d , for each substructure Ω E the admissibility conditions can be equivalently written in terms of corrections, as well as the search direction E -: ∆F E + k∆W Eδ E = 0. The quantity δ E is a residual in the search direction, it is a quantity known at this stage that depends on the iterates coming form the previous local and global stages and on the Lagrange multiplier W M E . Introducing PGD consists in looking for a space-time separated representation in space and time for the corrections of forces and displacements, that is ∆W E = Z(x)ϕ(t) and ∆F E = G(x)ϕ(t). If this separated representation is required to verify admissibility conditions, the following constraint has to be verified for space modes :

Ω E ε E (Z) : K : ε E (Z * )dΩ = ∂Ω E G • Z * dΓ ∀Z * ∈ E E,ad , (3)
By requiring a separated representation of the unknowns, as well as the admissibility condition (3) for space modes, the search direction equation can only be verified in a weak sense through the following minimal residual problem [START_REF] Ladeveze | On a multiscale computational strategy with time and space homogenization for structural mechanics[END_REF][START_REF] Passieux | Approximation radiale et méthode LATIN multiéchelle en temps et espace[END_REF] :

Problem 2.3. Find (Z, G) ∈ E E,ad × F E,ad and ϕ ∈ L 2 [0,T ] satisfying (Z, G, ϕ) = arg min (G,Z,ϕ) ∥(G + kZ)ϕ -δ E ∥ 2 ∂Ω E ×[0,T ] , (4)
as well as the admissibility condition (3).

The resolution of ( 4) is usually performed with a fixed point iterative strategy. The first step consists in computing the space modes knowing the time mode ϕ from the previous step. Then ϕ is updated knowing the space modes from the first step. The process is stopped after few iterations. Such a progressive PGD decomposition where new modes are added along the iterations, is generally not optimal, which means that a large number of modes may be required for a given accuracy. To improve the quality of the basis and control its size so that it does not grow uncontrollably, time modes updating [START_REF] Nouy | A priori model reduction through proper generalized decomposition for solving time-dependent partial differential equations[END_REF] and downsizing algorithms [START_REF] Giacoma | Toward an optimal a priori reduced basis strategy for frictional contact problems with LATIN solver[END_REF][START_REF] Zeka | Preliminary study for the simulation of wire ropes using a model reduction approach suitable for multiple contacts[END_REF] can be adopted and will be briefly explained in the next section.

Numerical application

As a simple but very demonstrative example, an application on a one-dimensional frictional contact problem is reported. The benchmark problem consists of a clamped bar subjected to a time dependent traction loading F(t) on the free side (Fig. 2b). The bar is in contact with a frictional interface by means of a normal pressure p(t) acting on it (Fig. 2a). In practice, it is assumed that the bar is always in contact with the surface due to the pressure p. A monoscale monostructure model reduction analysis of this test case was performed in [START_REF] Zeka | Preliminary study for the simulation of wire ropes using a model reduction approach suitable for multiple contacts[END_REF]. It was shown that the problem presents a low reducibility, especially for frictional forces and, because of this, a lot of PGD modes on the whole contact interface need to be computed. This may lead to loss of efficiency and poor local accuracy for contact quantities.

Here, the benefits that a domain decomposition approach can bring, and especially the computational gain in introducing a multiscale aspect, are studied. The structure is therefore decomposed into several substructures that interact with each other through connecting perfect interfaces and subject also to the frictional contact interface, as in Fig. 2a in the case of decomposition into 5 substructures. The used parameters are shown in Tab. 1. A reference search direction k = ES/l sub was adopted for both frictional and perfect interfaces, with l sub being the length of a substructure.

Parameters

Performance of the multiscale approach

Fig. 3 shows, in the case of 10 substructures, the behavior of LATIN indicator [START_REF] Capaldo | The reference point method, a "hyperreduction" technique : Application to PGD-based nonlinear model reduction[END_REF] in the monoscale version and in the multiscale case. The gain in convergence with the introduction of the multiscale aspect is remarkable. The multiscale approach, in particular, allows for a huge convergence gain in the first iterations, where macroquantities converge rapidly. Subsequently, the quantities converge at the local microlevel. In this second stage the monoscale and multiscale approach exhibit the same convergence rate. To highlight the effect of the multiscale approach on the solution of the problem, Fig. 4 shows, in the case of 10 substructures, the solution of the problem after 10 iterations of the LATIN method for the monoscale approach and the multiscale one, compared to the reference solution at time t = 0.5s and t = 1s. It is evident how, after a a few iterations, the multiscale approach succeeds in capturing the global behavior of the problem very well. Subsequently, further iterations are needed for the microquantities to converge, especially at the sticking-sliding discontinuity zones. The monoscale approach, on the other hand, turns out to be far from the reference solution, with the loading boundary condition still not fully propagated along all the substructures. These results show indeed how the multiscale approach allows for significant gain, especially in the first iterations. However, local convergence of microquantities still continues to maintain the same convergence rate as the monoscale approach, which may still lead iterate a considerable number of times in order to have an accurate computation of local quantities. One method to improve this weakness may be to update the search direction parameter k, taken constant in space and time, according to the contact conditions once the contact status has come to convergence. The search direction parameter would then result as a function of space and time k(x,t). Updating the search direction can be, however, computationally burdensome since it would require to recompute the linear operators of the LATIN. Despite this, still updating few time after the contact status has come to convergence may be beneficial, especially if one can provide a separated representation of k(x,t) to facilitate the integration of the operators.

Preliminary SVD analysis and introduction of PGD

Before proceeding with the application of PGD in the test case, an a posteriori SVD analysis of the previous problem in the case of 5 substructures is given. Fig. 5 shows the trend of the singular values for the frictional contact forces in the different substructures, numbered from 1, which corresponds to the clamped substructure, to 5, corresponding to the substructure where the external force is applied. Different substructures subjected to different sticking-sliding conditions show different potential reducibility. The first two, more under sticking conditions, exhibit singular values that decrease very rapidly. The last three, on the other hand, where a large sliding front is present, display a very slow trend in singular values decreasing In Fig. 6 is shown a comparison between space modes of frictional forces for substructures 4 and 5. A more global aspect of the first modes, more energetic and, one might say, at "low-frequency" is evident, compared with subsequent modes that bring more local, "high-frequency" corrections. The contribution of the first modes in the multiscale approach is well captured in fact by the macroproblem for the whole structure. This corresponds in fact to the rapid initial convergence rate at the beginning in Fig. 3 and can be visualized as well in the solution plot of Fig. 4 where some information is still missing locally in the sticking-sliding transition zones. Moreover Fig. 6 highlights another interesting aspect. Indeed, substructures 4 and 5 exhibit similar sticking-sliding conditions and evolution over time, for this reason their modes present similar shapes. This suggests the idea of being able to share the PGD basis between similar subdomains along the LATIN iterations [START_REF] Ladeveze | On a multiscale computational strategy with time and space homogenization for structural mechanics[END_REF][START_REF] Passieux | Approximation radiale et méthode LATIN multiéchelle en temps et espace[END_REF]. The introduction of PGD allows the linear stage problems of the LATIN to be solved in a separated representation framework and to save computational time. However, even though finding a new pair of PGD modes is less onerous than solving the problem in full format, adding a new mode to each iteration of the LATIN is still expensive and leads to a high ROB size that can cause the method to lose effectiveness. For this reason, an established way to avoid adding modes at each iteration unless necessary and to simultaneously improve the quality of the reduced representation is to perform a preliminary stage by updating all the time functions [START_REF] Nouy | A priori model reduction through proper generalized decomposition for solving time-dependent partial differential equations[END_REF]. However, for strongly non-regular and non-linear problems such as frictional contact problems, this does not prevent the generation of strongly redundant modes that move the PGD basis away from its optimality. It is therefore necessary to contain the size of the PGD basis and control its quality along the iterations of the LATIN. A soft sorting and downsizing algorithm such as the one proposed in [START_REF] Giacoma | Toward an optimal a priori reduced basis strategy for frictional contact problems with LATIN solver[END_REF] may be sufficient, as shown in Fig. 7 and8, where the the first 3 SVD modes of the displacements of substructure 5 are compared with the first 3 PGD modes displaying a good correlation. The non-linear aspect in the LATIN is still integrated in the local stage in full format on the interfaces. This may in fact represent a bottleneck to model reduction, especially in the case of many and large interfaces. One can think of the possibility to integrate the local stage in a reduced format as well by making use, for example, of an a hyperreduction format dedicated to the LATIN-PGD [START_REF] Capaldo | The reference point method, a "hyperreduction" technique : Application to PGD-based nonlinear model reduction[END_REF].

Conclusions

In this paper, some strengths and points of improvement of the LATIN-based multiscale mixed DDM method for the model-order reduction of frictional contact problems were presented. The DDM approach allows to create reduced bases per substructure to better track sliding fronts and propagative phenomena. The multiscale aspect leads to a significant gain in convergence in the early stages, taking advantage of the multiscale nature of the phenomenon itself. However, after few iterations the convergence rate drops and it is necessary to refer to search direction updating techniques, which are scheduled to be proposed. PGD is naturally introduced in the linear stage of LATIN and allows for the addition of new modes where and when required to better follow the evolving frictional phenomena, with the size of the basis which can be efficiently controlled with a downsizing algorithm. The local stage still remains a bottleneck as it is integrated in full format and the potentiality offered by a reduced RPM integration will be considered. Moreover, the need to control the local error with an appropriate convergence indicator is crucial for specific applications. Two-dimensional problems with more complex loadcases and larger number of frictional interfaces will be tested to further comprehend the potentiality of the proposed approach.

FIGURE 1 -

 1 FIGURE 1 -LATIN iterative scheme.

FIGURE 2 -

 2 FIGURE 2 -Benchmark problem set : sketch of the problem (a) and time evolution of the load F(t) (b).

FIGURE 3 -

 3 FIGURE 3 -LATIN indicator evalution for the monoscale and the multiscale approach.

FIGURE 4 -

 4 FIGURE 4 -Frictional contact forces distribution after 10 iterations of the LATIN for the monoscale and the multiscale approach at time t = 0.5s (a) and t = 1s (b).

5 FIGURE 5 -

 55 FIGURE 5 -SVD singular values for frictional forces on the different substrctures.

5 FIGURE 6 -

 56 FIGURE 6 -SVD space modes 1, 3 and 5 of frictional forces for substructures 4 and 5.

3 FIGURE 7 -

 37 FIGURE 7 -SVD space modes 1, 2 and 3 of displacements for substructure 5.

2mode 3 FIGURE 8 -

 38 FIGURE 8 -PGD space modes 1, 2 and 3 of displacements for substructure 5.

  by following a descent search direction E -, the solution s n+1 ∈ A d is found in the linear stage by solving a linear elastic problem (2.2) at the substructural level.

Problem 2.2 (linear stage). Find s = {s E } E∈E , verifying : the admissibility of s E : s E ∈ S E,ad

TABLE 1 -

 1 Used parameters for the benchmark problem.

	Young modulus, E	210 GPa
	Bar cross section, S	3.14 mm 2
	Bar length, L	1 m
	Number of DOFs, N x	51
	Number of time steps, N t	101
	Time interval, T	1 s
	Friction coefficient, µ	0.3
	Pressure load, p(t)	5000 N/L