Christophe Cerisara
email: cerisara@loria.fr

Paul Caillon
email: paul.caillon@loria.fr

Guillaume Le Berre
email: guillaume.le-berre@loria.fr

Unsupervised Post-Tuning of Deep Neural Networks

Keywords: deep learning, unsupervised training, regularization, natural language processing

We propose in this work a new unsupervised training procedure that is most effective when it is applied after supervised training and fine-tuning of deep neural network classifiers. While standard regularization techniques combat overfitting by means that are unrelated to the target classification loss, such as by minimizing the L2 norm or by adding noise either in the data, model or process, the proposed unsupervised training loss reduces overfitting by optimizing the true classifier risk. The proposed approach is evaluated on several tasks of increasing difficulty and varying conditions: unsupervised training, posttuning and anomaly detection. It is also tested both on simple neural networks, such as small multi-layer perceptron, and complex Natural Language Processing models, e.g., pretrained BERT embeddings. Experimental results confirm the theory and show that the proposed approach gives the best results in posttuning conditions, i.e., when applied after supervised training and fine-tuning.

I. INTRODUCTION

A major challenge for deep learning classifiers is to move beyond traditional supervised training and exploit the large quantity of unlabeled data available. A particularly successful approach in this direction consists in training models with auxiliary tasks for which labels are easily available, such as regenerating the observations. This may be realized for instance by plugging the classifier into an autoencoder architecture [START_REF] Tschannen | Recent Advances in Autoencoder-Based Representation Learning[END_REF], or by predicting masked tokens or future events in time series, e.g., with language models [START_REF] Liu | A Survey on Contextual Embeddings[END_REF]. Then, the parameters of these models may be transferred to target tasks with limited amount of annotated data, but such a fine-tuning process is likely to overfit [START_REF] Wang | Integrating Task Specific Information into Pretrained Language Models for Low Resource Fine Tuning[END_REF]. Most of these methods still exploit supervised training losses, because both the observations in autoencoders and the masked words in language models are given at training time. Unsupervised losses that do not depend on the class labels are rather often used in different contexts, for instance to cluster [START_REF] Min | A Survey of Clustering With Deep Learning: From the Perspective of Network Architecture[END_REF] the data or to pretrain a classification model [START_REF] Erhan | Why does unsupervised pre-training help deep learning?[END_REF] in order to facilitate the subsequent supervised training process. We propose in this work an unsupervised loss that is designed to optimize the classifier risk without relying on any labeled training instance. This is made possible by replacing the standard empirical approximation of the risk, which approximates the full data distribution by a limited labeled training set, by another type of approximation that does not require any This work was performed using HPC resources from GENCI-IDRIS (Grant 2020-AD011011668). label. However, this approximation relies on an assumption about the shape of the distribution of the model outputs that is not always fulfilled in practice. We show in this work that carefully initializing the parameters of the model enables to fulfill this necessary assumption and leads to an analytical derivation of the unsupervised loss that can be applied to deep neural networks. Concretely, such an initialization of the parameters may be achieved by first training or fine-tuning the model on the small amount of available labeled data. Then, the unsupervised loss may be applied in a post-training way, i.e., as soon as the model's parameters are in a neighborhood of the optimum. This additional final training stage then improves the generalization capability of the classifier without impacting its performances on the target supervised task. Indeed, the proposed unsupervised risk gives theoretical guarantees that the model converges towards the same optimum than with the supervised classification risk. This is in contrast with most standard regularization methods, such as the L2 norm, dropout and priors: all of these methods push the model's parameters in a direction that improves generalization but that is not related to the target task-dependent classification risk.

We present in Section III the proposed unsupervised training method and evaluate its performances in Section IV. Section V discusses the experimental results, while Section II reviews the related work and Section VI concludes the paper.

II. RELATED WORKS

Regularization is a key component of most deep neural networks, because of their large number of paremeters and the ease with which they might overfit [START_REF] Neyshabur | Geometry of Optimization and Implicit Regularization in Deep Learning[END_REF]. The standard stochastic gradient descent has an implicit regularization property, which is still not well understood theoretically [START_REF] Razin | Implicit Regularization in Deep Learning May Not Be Explainable by Norms[END_REF], but is essential to the success of deep learning. In addition to this implicit regularization, many regularization methods have been proposed for deep neural networks [START_REF] Kukačka | Regularization for Deep Learning: A Taxonomy[END_REF] but, to the best of our knowledge, only a few published methods propose to modify the parameters after the supervised training phase. One of such methods generate new latent data representations that are hard to classify to improve generalization [START_REF] Khan | Post-training iterative hierarchical data augmentation for deep networks[END_REF], but although this data augmentation procedure occurs after the main training step, it is still supervised while our loss is unsupervised. Another classical post-training technique in deep learning is quantization, which aims at compressing the model layers with smaller bit widths [START_REF] Kluska | Post-training Quantization Methods for Deep Learning Models[END_REF]. Although such methods may be viewed as post-training regularizers, because drastic compres-sion introduces noise in the model, such a regularization is not based on any approximation of the classifier risk. The authors of [START_REF] Moreau | Post Training in Deep Learning with Last Kernel[END_REF] propose another post-training strategy for deep neural networks, for training the final layer alone while freezing the rest of the network. They report improvements in generalization and explain them from the kernel theory point of view, but contrary to our proposal, they still use the supervised objective for this post-training step.

Learning with Label Proportions is a different weakly supervised paradigm where the only supervision comes from the known proportion of classes in subsets, or bags, of observed samples [START_REF] Qi | Adaboost-LLP: A Boosting Method for Learning With Label Proportions[END_REF]. This situation occurs in many important applications, for instance to train a neural network on quantum physics experiments, where assigning a precise label to a given instance is not possible [START_REF] Cohen | Machine) learning to do more with less[END_REF]. [START_REF] Lu | On the minimal supervision for training any binary classifier from only unlabeled data[END_REF] further study the minimum amount of unlabeled sets with different label proportions required to train any binary classifier, and prove that three such sets are sufficient. In our approach, we rather use a single such subset and exploit two additional assumptions to train our deep neural networks. More generic weakly supervised learning methods may also exploit uncertainty on labels, such as [START_REF] Zantedeschi | Beta-risk: a new surrogate risk for learning from weakly labeled data[END_REF] or class-conditional label noise, which considers training a classifier with labels that have been corrupted with different probability [START_REF] Reeve | Classification with unknown class-conditional label noise on non-compact feature spaces[END_REF]. Another approach that is related to our work are one-class deep neural networks, in particular the Deep-SVDD [START_REF] Ruff | Deep one-class classification[END_REF] and CVDD [START_REF] Ruff | Self-attentive, multi-context one-class classification for unsupervised anomaly detection on text[END_REF] models. We compare our method with CVDD in Section IV. Although such one-class models and our post-processing approach differ in their theoretical motivation, both optimization algorithms end up being very similar. For instance both approaches rely on a first step to split the classifier scores into two subsets according to a given quantile, followed by a second optimization step. The main difference lies in the loss function that is used during this optimization step. Text anomaly detection methods may also exploit generative models, for instance, [START_REF] Yap | Text anomaly detection with arae-anogan[END_REF] adapts generative adversarial networks, which have already shown remarkable results for image anomalies, to generate normal sentences and then compute an anomaly score for texts. Non-deep learning methods are also used for text anomaly detection, for instance, [START_REF] Kannan | Outlier detection for text data[END_REF] rely on the low-rank matrix factorization of the documentterm matrices.

III. PROPOSED UNSUPERVISED APPROACH

A. Motivation

Let us consider a binary classifier with parameters θ that outputs a scalar score f (x) ∈ IR for input x. Let us call y ∈ {0, 1} the true class of x. The classification decision is class ŷ = 0 iff f (x) ≤ 0, and ŷ = 1 iff f (x) > 0. Supervised training of f (x) classically aims at minimizing the classifier risk θ * = arg min R(θ), which is the expected error of the classifier over the full (unknown) distribution of the data. With a hinge loss, it is:

R(θ) = E p(x,y) (1 -f (x) • (2y -1)) + (1)
= P (y = 0) p(f (x) = α|y = 0)(1 + α)+dα +

P (y = 1) p(f (x) = α|y = 1)(1 -α)+dα
where (x) + = max(0, x). The distribution p(x, y) is usually unknown, and R(θ) is commonly approximated by the empirical risk Remp (θ), which averages the error over a finite dataset, given the supervised labels y. [START_REF] Balasubramanian | Unsupervised supervised learning II: Margin-based classification without labels[END_REF] propose another approximation R(θ) of R(θ) that does not require the knowledge of y, hence leading to an unsupervised training algorithm, which depends on 2 assumptions:

• H1: The class-conditional distribution of the scores p(f (x)|y) is Gaussian; • H2: The class-marginal prior P (y) is known; They thus propose an unsupervised training method for a binary linear classifier, and for three types of error functions: hinge, exponential and log loss. We propose in this work to extend the domain of application of their approximation of the risk to a much larger and powerful class of binary classifiers: any deep neural network (DNN) with a final linear (or softmax) classification layer. To achieve this, we introduce a new simplifying assumption, derive an end-toend differentiable equation of the risk, analyze the resulting function and propose a new post-tuning regularization strategy.

B. Limitations and proposed solutions

While H2 is an assumption that is met in several common tasks, e.g., the prevalence of a disease is known, as well as the ratio of occurrence of a word in a language, H1 is critical for the validity of the previous approximation of the risk. [START_REF] Balasubramanian | Unsupervised supervised learning II: Margin-based classification without labels[END_REF] justifies H1 with a central limit theorem for noniid features: intuitively, a linear classifier computes a sum of random variables that tends towards a Gaussian distribution as the feature vector dimension tends towards infinity. When replacing the linear classifier with a DNN, this theorem may not be used any more. Furthermore, even with a linear classifier, we have observed that H1 does not hold in several practical situations, for instance when initializing the classifier with random parameters. Even after training has started, the fact that the feature vector dimension is small may break the theorem. Therefore, we rather propose a third assumption:

• H3: Both distributions p(f (x)|y = 0) and p(f (x)|y = 1)
are well separated when θ is close to an optimum θ * emp = arg min θ Remp (θ) Our rationale is to not assume any more that H1 is always true, but rather to restrict ourselves to contexts where H1 is valid: H3 serves to define such contexts, and H3 further enables us to derive an end-to-end closed-form solution of the unsupervised loss. Note that, in a neighborhood of θ * emp and when the DNN has enough capacity, the Universal Approximation Theorem [START_REF] Lu | The expressive power of neural networks: A view from the width[END_REF] guarantees that the scores at the output of the DNN of the samples that can be correctly classified are well separated. The samples that can not be correctly classified, i.e., the ones for which the input features do not provide enough information to decide on one class or another, are arbitrarily assigned to the two partitions. In practical situations, the input features are rich enough and this Bayes Risk is small compared to both distributions of well-classified samples. Then, the scores in each partition may be approximated with a Gaussian, which fulfills H1. We have done many experiments that confirm this, and report a few next.

The obvious question is why would we need to optimize the model with our unsupervised loss, when the model is already trained ? Because θ * emp is prone to overfitting, and compensating this issue with excessive regularization during the supervised training stage might move away θ * emp from θ * . Indeed, conversely to standard regularization, our approximation of Eq-1 is designed to converge towards θ * . Furthermore, it is more robust to overfitting than empirical risk minimization, because only 4 parameters (the two Gaussian parameters) are estimated from the data, compared to the large number of parameters classically found in recent DNNs. Our proposed approach may thus be viewed as a post-tuning stage that is applied after fine-tuning the DNN to the target task and that will a posteriori reduce the impact of overfitting.

C. Derivation of the loss

We propose to rewrite Eq-1 as follows:

R(µ, σ) = P (y = 0) N (α; µ0, σ0)(1 + α)+dα + P (y = 1) N (α; µ1, σ1)(1 -α)+dα
where µ = (µ 0 , µ 1 ) ∈ IR 2 and σ 2 = (σ 2 0 , σ 2 1 ) ∈ IR + 2 are respectively the means and variances of the corresponding Gaussians p(f (x)|y = 0; θ) and p(f (x)|y = 1; θ), and

N (α; µ, σ) = 1 √ 2πσ 2 e - (α-µ) 2 2σ 2
Removing the non-linearity, we get: We know that:

b a N (x; µ, σ)dx = 1 2 erf b -µ σ √ 2 -erf a -µ σ √ 2
and:

b a xN (x; µ, σ)dx = µ b a N (x; µ, σ)dx -σ 2 [N (x; µ, σ)] b a So, b a xN (x; µ, σ)dx = µ 2 erf b-µ σ √ 2 -erf a-µ σ √ 2 - σ 2 (N (b; µ, σ) -N (a; µ, σ))
Which gives:

R(µ, σ) = P (y = 0) 2 (1 + µ0) 1 -erf -1 -µ0 σ0 √ 2 + P (y = 0)σ 2 0 N (-1; µ0, σ0) + (2) P (y = 1) 2 (1 -µ1) 1 + erf 1 -µ1 σ1 √ 2 + P (y = 1)σ 2 1 N (1; µ1, σ1)
In order to use Eq-2 as a loss function for Stochastic Gradient Descent in deep learning toolkits, we need to rewrite this expression as a differentiable function that depends on the parameters θ. We analyze next the function R(µ, σ) and finally derive the missing analytical expression to build the end-to-end loss.

D. Analysis of the loss

Let us note p 0 = P (Y = 0) and plot Equation 2 as a function of (µ 0 , µ 1 ) in Figure 1 (top), for p 0 = 0.1 and σ 0 = σ 1 = 1. When we fix µ 1 , we can see in Figure 1 (bottom) that the risk as a function of µ 0 may be well approximated by a scaled and translated rectified linear function, as long as the variances are small enough. Furthermore, the lower σ 0 (and σ 1 ) is, the better the risk is. Varying µ 1 and σ 1 only translates this curve vertically, above the horizontal axis. So, assuming that the risk has first been minimized with respect to µ 1 , then the global minimum of the risk may be obtained by decreasing linearly µ 0 . Conversely, lower risks are obtained when µ 1 is increasing. In conclusion, the risk is lower when µ 0 is small, µ 1 is large and when σ 0 and σ 1 are small, which supports the validity of H3 on reasonably good areas of the parameter space, i.e., when θ * and θ * emp are not too far away one from the other.

E. Approximation of the bi-Gaussian distribution

Given the previous analysis, we may now assume H3 in a neighborhood of θ * emp : both modes (µ 0 , σ 0 ) and (µ 1 , σ 1 ) of the scores distribution are well separated with a small overlap. Then, a good approximation of µ 0 and µ 1 can be computed by splitting all the scores f (x) according to the p 0 -quantile x p0 . Let us call X -the subset of size N -of all data points that are on the left side of the p 0 -quantile:

X -= {x ∈ X s.t. f (x) < f (x p0 )}
and similarly for the other side:

X + = {x ∈ X s.t. f (x) ≥ f (x p0 )}
We can now approximate the Gaussian parameters deterministically:

µ0 1 N - x∈X - f (x) µ1 1 N + x∈X + f (x) (3)
σ 2 0   1 N - x∈X - f (x) 2   -   1 N - x∈X - f (x)   2 σ 2 1   1 N + x∈X + f (x) 2   -   1 N + x∈X + f (x)   2
We can now plug equations 3 into Eq 2 to get a differentiable loss R(θ) with respect to the network parameters θ. Algorithm 1 summarizes the optimization procedure.

IV. EXPERIMENTAL VALIDATION The proposed unsupervised loss is coded in PyTorch [START_REF] Paszke | Automatic differentiation in pytorch[END_REF] and is freely distributed 1 .

A. Unsupervised conditions

We first evaluate our proposed optimization algorithm in unsupervised conditions, i.e., without any label in the training corpus, and thus without any first stage of supervised training or fine-tuning of the model: the models' parameters are initialized randomly before applying algorithm 1. These experiments will help us to evaluate, on various tasks, the impact of not fulfilling assumption H3.

1) Breast Cancer detection: The first task is to detect cancer on the standard Wisconsins Breast Cancer benchmark [START_REF] Wolberg | Breast cancer wisconsin (diagnostic) data set[END_REF] with a 2-layers Multi-Layer Perceptron (MLP). We compare it with k-means clustering in Table I.

This experiment suggests that, on a relatively easy dataset, the proposed optimization procedure may be used as an unsupervised training algorithm and outperforms a simple clustering baseline. We investigate next its unsupervised performances on more complex tasks and models. -Let consider a binary classification task, for which we assume that the proportion of class-0 elements p 0 is known approximately; -Let {x i } 1≤i≤N be a corpus of observations without labels; -Let g φ (x) be a deep neural network with parameters φ that computes a vectorial representation of an input x, which is fed to a final linear classification layer f θ (g φ (x)) with parameters θ; φ and θ should be pretrained.

• Iterate:

-Run a forward pass on the dataset {x i } 1≤i≤N with the current parameters φ, θ. -Compute all classifier scores {s i = f θ (g φ (x i ))} 1≤i≤n over the full corpus N , or over a batch of observations n that is large enough to assume that the distribution of classes in the batch is representative of the distribution in the whole corpus.

-Sort the list of scores (s i ) 1≤i≤n to compute the p 0quantile x p0 . -Compute the Gaussian parameters µ = (µ 0 , µ 1 ), σ = (σ 0 , σ 1 ) with Equations 3. -Compute the risk (Eq 2) with these Gaussian parameters. -Apply automatic differentiation to compute ∇ θ R(µ, σ), and optionally ∇ φ R(µ, σ); -Run a step of Gradient Descent to update θ, and optionally φ. negative movie reviews. These datasets as well as the experimental evaluation protocol that we have used are described in details in [START_REF] Conneau | SentEval: An evaluation toolkit for universal sentence representations[END_REF]. This protocol first computes a sentence representation with the InferSent [START_REF] Conneau | Supervised learning of universal sentence representations from natural language inference data[END_REF] sentence embeddings, and then passes these sentence embeddings into a feed-forward network that is trained on each dataset.

We have adopted the same experimental protocol and the same hyper-parameters, except that we do not train the final feed-forward network with supervised labels and the crossentropy loss, but we rather train it without any label and with our proposed unsupervised loss. Note that in these experiments, the parameters of the InferSent embeddings are pretrained on general English texts, while the parameters of the final classification layer are initialized randomly, because we assume that we do not have access to any task-specific label. Table II summarizes the accuracy of two baselines: the state-of-the-art supervised models trained on the full corpus and on only 100 instances, as well as the accuracy of the proposed unsupervised model. We can observe that the proposed unsupervised method performs much worse than the fully supervised models. We interpret this as a consequence of the fact that H3 is not fulfilled. However, our unsupervised model always matches the performances of a supervised model trained on 100 reviews, and outperforms them (+20% accuracy) on the subjectivity classification task.

B. Post tuning conditions

We evaluate next our unsupervised loss in "post-tuning" conditions: the models are first trained in a supervised way to their optimum with standard regularization methods, and this first training phase is followed by our unsupervised optimization procedure. In order to initialize a model that is as close as possible to its supervised optimum, we exploit the scripts and hyperparameters from state-of-the-art published papers, and report their published results. Two binary classification NLP tasks are considered, plus another text anomaly detection task.

1) Supervised classification: The two classification tasks are MRPC, which goal is to decide whether two sentences are paraphrases, and CoLA, where the model decides whether a sentence is grammatical or not. 400 sentences are extracted from the training sets to find p 0 and early stopping threshold. The state-of-the-art accuracy is 93% on MRPC [START_REF] Ruder | Nlp-progress[END_REF] and 75% on CoLA [START_REF] Warstadt | Neural network acceptability judgments[END_REF], which are obtained with advanced models, such as StructBERT and ensembling. We rather use the reference and ubiquitous BERT pretrained model, which is easier to experiment with and gives 84% on MRPC and 56% on CoLA when trained on the full training corpus [START_REF] Huggingface | Transformers results[END_REF]. We report in Table III the best accuracy (among all training epochs) obtained on the validation set with these baselines, when trained on our reduced training set. For our post-tuned model, we find the best epoch with early stopping on the 400 development sentences, and compute the accuracy on the validation set with this model. The proposed loss improves the accuracy of the fine-tuned BERT model on both tasks.

2) Unsupervised anomaly detection: The post-tuning approach may also be applied on a model previously trained with another unsupervised loss function. We demonstrate this on the text anomaly detection task, where models are classically trained in an unsupervised way, because of the lack of labeled anomaly samples. Unsupervised text anomaly detection is performed on the Reuters-21578 corpus [START_REF] Apté | Automated learning of decision rules for text categorization[END_REF], with the model and experimental conditions described in [START_REF] Ruff | Self-attentive, multi-context one-class classification for unsupervised anomaly detection on text[END_REF]. The standard Area Under Curve (AUC) metric is used to evaluate the quality of the text anomaly detection task. The anomaly detection model transforms a variable-length sequence of Glove [START_REF] Pennington | GloVe: Global vectors for word representation[END_REF] pre-trained word embeddings into a fixed-size document embedding matrix M with multi-head self-attention [START_REF] Lin | A structured self-attentive sentence embedding[END_REF]. Each of the r heads, or columns in the resulting matrix M , is a linear combination of the original word embeddings.

The anomaly score is then the average cosine distance between these r sentence embeddings and a set of r context vectors {c k } 1≤k≤r . These context vectors are trained in [START_REF] Ruff | Self-attentive, multi-context one-class classification for unsupervised anomaly detection on text[END_REF] with the unsupervised CVDD loss, which minimizes the cosine distance on the (unlabeled) training corpus.

We propose to reformulate this last step within the framework of neural networks, by encoding each c k as the parameters θ of an unbiased linear layer, which outputs the dot product between its weights θ and the input x, normalized by both θ and x Frobenius norms:

f cos (x) = i θ i x i ( i θ 2 i ) ( i x 2 i ) (4)
This neural version of the CVDD model is illustrated in Figure 2.

In our experiments, we initialize the parameters of the model in Figure 2 with a standard CVDD training, as described in [START_REF] Ruff | Self-attentive, multi-context one-class classification for unsupervised anomaly detection on text[END_REF]. This first step enables to get parameters that are relatively close to a good solution, with well-separated Gaussians in the score space, hence fulfilling our third assumption. Then, in a second phase, we remove the CVDD loss and replace it with our unsupervised loss.

The version of the Reuters-21578 dataset distributed with the CVDD code is composed of news articles with a single label (topic) per article. Table IV presents the experimental Fig. 2. Text anomaly detection model from [START_REF] Ruff | Self-attentive, multi-context one-class classification for unsupervised anomaly detection on text[END_REF], encoded in the form of a neural network. The word embeddings sequence is encoded into M fixed-size embeddings. Each of these M sentence embeddings is passed to a normalized linear layer that outputs the cosine distance between the sentence embedding and its parameter vector. These r normalized neural layers implement equation 4. The final anomaly score is the average of these distances.

results on this corpus: each of the seven labels listed in the first column is considered as the normal class one after the other, while all the other labels are representatives of the anomaly class. The italic numbers between parenthesis are taken from the paper [START_REF] Ruff | Self-attentive, multi-context one-class classification for unsupervised anomaly detection on text[END_REF], while the plain CVDD numbers are obtained by re-running the code distributed by the authors of [START_REF] Ruff | Self-attentive, multi-context one-class classification for unsupervised anomaly detection on text[END_REF]: there may be some differences, which are likely due to different runtime environments (type of GPUs, library versions...). The columns "Eq 2" correspond to our proposed unsupervised loss. The model and number of parameters are the same between CVDD and our approach: only the loss changes. Following the experimental conditions chosen in [START_REF] Ruff | Self-attentive, multi-context one-class classification for unsupervised anomaly detection on text[END_REF], the training corpus only contains normal samples. The experimental results with both the original CVDD loss and the model fine-tuned with our unsupervised loss are reported in Table IV. All models in this table have the same number of parameters. Fine-tuning is achieved with the Adam criterion over 1000 epochs with a learning rate of 10 -4 and momentum of 10 -6 . The only remaining hyperparameter is our p 0 value, which represents our expected proportion of anomaly instances. We optimize p 0 with a grid search between 0.1 and 0.5 and pick the value with the lowest risk on a development set. The development set is equal to the training set (with normal samples only) merged and shuffled with 50% of random outliers from the training set. There is no label in the development set.

We can observe a quasi-systematic improvement obtained when using our proposed unsupervised loss, as compared to the CVDD objective. The average relative reduction of the error rate obtained with our proposed loss is -32% with r = 3 and -43% with r = 5, which means that our proposed loss nearly halves the number of errors made by CVDD, and -22% with r = 10. A qualitative analysis of the detected anomalies is given in the next section.

V. DISCUSSION OF EXPERIMENTAL RESULTS

We now discuss our experimental results on topic outliers detection on the Reuters corpus, in the light of our modeling assumptions.

a) H1 and H3: The first assumption concerns the bigaussianity of the scores. To get an intuition about this assumption, we plot in Figure 3 the histogram of the scores f (x) over all training samples just before (left) and after (right) unsupervised post-tuning. We can also see in Figure 3 the effect of post-tuning, which increases the interval between both modes from 0.14 to 0.17, as expected. This plot also illustrates a case that is compatible with our third assumption, i.e., that the initial distribution is already composed of two modes that are relatively well separated.

b) H2: Regarding our second assumption about the known class marginal p 0 , we have plotted in Figure 4 the test AUC for the full range of possible values for p 0 .

Note that at training time, following the experimental conditions in [START_REF] Ruff | Self-attentive, multi-context one-class classification for unsupervised anomaly detection on text[END_REF], there is no outlier in the training corpus, so in theory the true p 0 should be null. However, in every realistic corpus, there are always samples that are not prototypical and somehow differ from the bulk of the distribution. Such samples may be considered as outliers, as illustrated in the following qualitative analysis.

Figure 4 shows that the influence of p 0 is significant, but with a moderate degradation within the whole range of possible values. p 0 should then preferably be tuned on a development corpus as we have done in our experiments, whenever it is possible. c) Qualitative analysis: Because the test corpus is created by choosing a single topic as the normal class, and pooling together all other topics as outliers, the resulting corpus is very unbalanced. Figure 5 plots the distribution of the classifier scores on the test corpus, showing with a different color and opacity the normal and anomaly samples.

We observe the following:

• At the extreme left of the distribution, we find all "normal" class examples, e.g.:

A 24-hour strike by Belgian public employees protesting against a government pay ... • The large set of "non-normal" samples is distributed over a large span covering the mid-left, central and right of the

VI. DISCUSSION AND CONCLUSION

Supervised learning aims at minimizing the classifier error for a specific task, for instance predicting the sentiment from short texts or recognizing persons in pictures. The most commonly used function to approximate this objective is the empirical risk, which computes the classifier error on a given labeled training corpus. [START_REF] Balasubramanian | Unsupervised supervised learning II: Margin-based classification without labels[END_REF] proposes another approximation for linear classifiers, which provably converges towards the optimum using a training corpus without labels, i.e., without a precise and detailed information about the task itself. The success of the approach depends on an assumption about the Gaussianity of the marginal scores distribution, which is theoretically fulfilled when the observations have infinite dimension. Experiments confirm that it is also often fulfilled in practice, although regions of the parameter space might exist where this assumption is broken. During training, when the classifier enters such a region, convergence is not any more guaranteed. Therefore, we propose in this work a new constraint that positions the model in regions of the parameter space that are likely to fulfill the Gaussianity assumption, even with non-linear deep neural networks.

Just like other regularization methods, the proposed loss combats overfitting: this is mainly because it does not follow the common empirical risk approximation, which is the root cause of overfitting. Still, one could argue that a finite dataset, even unlabeled, is actually used and that the training process may overfit the distribution of the data in this dataset. This is correct, but the dataset is only used to estimate four parameters: the means and variances of two Gaussians, which strongly limit the risk of overfitting, as compared to the millions of parameters that are trained in the neural network in the preceding supervised training phase. So the proposed loss is extremely robust to overfitting: it will lead the model away from the overfitted local optima and towards the optimum of the classifier risk with much better generalization properties.

R 1 N

 1 (µ, σ) = (y = 0) +∞ -(α; µ0, σ0)dα + P (y = 0) +∞ -1 αN (α; µ0, σ0)dα + P (y = 1) 1 -∞ N (α; µ1, σ1)dα -P (y = 1) 1 -∞ αN (α; µ1, σ1)dα

Fig. 1 .

 1 Fig. 1. Risk as a function of both (µ 0 , µ 1 ) (top), and only µ 0 (bottom) for µ 1 = 2, σ 1 = 1 and σ 0 ∈ {0.1, 1, 3}

1

 1 1 https://github.com/cerisara/unsuprisk.gitAlgorithm Unsupervised optimization• Initialization:

Fig. 3 .

 3 Fig. 3. Histograms of the classifier scores on the training corpus, before (top) and after (bottom) unsupervised training with our loss.

Fig. 4 .

 4 Fig. 4. Test AUC for varying p 0 , with r = 3 and nc ="ship". Because of the symmetry of unsupervised binary classification, p 0 may only vary between 0 and 0.5

Fig. 5 .

 5 Fig. 5. Histogram of the scores of the normal (solid green on the bottom left) and anomaly (translucent pink dominating between x = 0.1 and x = 0.6) test samples.

TABLE II PURELY

 II UNSUPERVISED TRAINING: ACCURACY ON 4 NLP TASKS (BOTTOM LINE). SUPERVISED MODEL TRAINED ON ALL DATA (TOP, FROM [11]), AND ON ONLY 100 TRAINING SAMPLES (MIDDLE).

	System	CR	SUBJ MPQA	MR
	InferSent fully supervised	86.3	92.4	90.2	81.1
	InferSent supervised 100 obs	63.8	62.5	70.1	53.9
	Proposed unsupervised training 66.8	83.0	70.9	59.7

TABLE III POST

 III TUNING EXPERIMENTS WITH A PRETRAINED BERT MODEL ON THE MRPC AND COLA BENCHMARKS.

		MRPC
		Supervised fine-tuning Proposed post-tuning
	Accuracy	78.4	82.3
	F1	85.7	86.7
		CoLA
		Supervised fine-tuning Proposed post-tuning
	Matthew's corr.	47.1	49.6

Hence, although it may seem counter-intuitive to apply unsupervised optimization after supervised training, because of the possibility to destroy information gained from the training labels, this final optimization step is designed to still converge towards the minimum task error rate.

We experimentally validate our proposed approach on tasks and models of increasing complexity: from two-layers MLP on the Wisconsis Breast Cancer dataset, to middle-sized neural networks for anomaly detection on texts and large BERT-based fine-tuned models. We also experiment with models that do not fulfill our proposed constraint on SentEval benchmarks, and models that do, such as CVDD. We show that using our proposed constraint leads to better results, giving a new state-of-the-art performance for text anomaly detection on the Reuters corpus.


Marco Cannone

Ahmad El Hajj

Regis Monneau

A El Hajj

F Ribaud

Global existence for a system of non-linear and non-local transport equations describing the dynamics of dislocation densities

Keywords: AMS Classification: 54C70, 35L45, 35Q72, 74H20, 74H25 Cauchy's problem, system of non-linear transport equations, system of nonlocal transport equations, system of hyperbolic equations, entropy, Riesz transform, Zygmund space, dynamics of dislocation densities

HAL is

Introduction 1.Physical motivation and presentation of the model

Real crystals show certain defects in the organization of their crystalline structure, called dislocations. These defects were introduced in the Thirties by Taylor, Orowan and Polanyi as the principal explanation of plastic deformation at the microscopic scale of materials.

In a particular case where these defects are parallel lines in the three-dimensional space, their cross-section can be viewed as points in a plane. Under the effect of an exterior stress, dislocations can be moved. In the special case of what is called "edge dislocations", these dislocations move in the direction of their "Burgers vector" which has a fixed direction. (cf J. Hith and J. Lothe [START_REF] Hirth | Theory of dislocations, Second Edition[END_REF] for more physical description).

In this work, we are interested in the mathematical study of a model introduced by I. Groma, P. Balogh in [START_REF] Groma | Link between the microscopic and mesoscopic lenght-scale description of the collective behaviour of dislocations[END_REF] and [START_REF] Groma | Investigation of dislocation pattern formation in a twodimensional self-consistent field approximation[END_REF]. In this model we consider two types of dislocations in the plane (x 1 , x 2 ). Typically for a given velocity field, those dislocations of type (+) propagate in the direction + b where b = (1, 0) is the Burgers vector, while those of type (-) propagate in the direction -b (see Figure 1.1). Here the velocity vector field is the shear stress in the material, solving the equations of elasticity. It turns out that this shear stress can be expressed as some Riesz transform of the solution (see Section 2). More precisely our non-linear and non-local system of transport equations is the following:

         ∂ρ + ∂t (x, t) = -(R 2 1 R 2 2 (ρ + (•, t) -ρ -(•, t)) (x)) ∂ρ + ∂x 1 (x, t) in D ′ (R 2 × (0, T )), ∂ρ - ∂t (x, t) = (R 2 1 R 2 2 (ρ + (•, t) -ρ -(•, t)) (x))
∂ρ - ∂x 1 (x, t) in D ′ (R 2 × (0, T )).

(P)

The unknowns of the system (P) are the scalar functions ρ + and ρ -at the time t and the position x = (x 1 , x 2 ), that we denote for simplification by ρ ± . These terms correspond to the plastic deformations in a crystal. Their derivative in the x 1 direction (i.e. the direction of Burgers vector b), ∂ρ ± ∂x 1 represents the dislocation densities of ± type. In our work, we will only consider solutions ρ ± such that ∂ρ ± ∂t , ∇ρ ± and ρ + -ρ -are Z 2 -periodic functions. The operators R 1 (resp. R 2 ) are the Riesz transformations associated to x 1 (resp. x 2 ). More precisely, these Riesz transforms are defined as follows:

Definition 1.1 (Riesz transform in the periodic case) Let the torus T 2 = R 2 /Z 2 . We define for i ∈ {1, 2} the Riesz transforms R i over T 2 as follows. If f ∈ L 2 (T 2 ), the Fourier series coefficients of R i f are given by: i) c (0,0

) (R i f ) = 0, ii) c k (R i f ) = k i |k| c k (f ) for k = (k 1 , k 2 ) ∈ Z 2 \ {(0, 0)},
where we recall that c k (f ) =

T 2 f (x)e -2πik•x d 2 x.
In fact, this 2D model has been generalized later in 2003 by I. Groma, F. Csikor and M. Zaiser in a model taking into account the back stress describing more carefully boundary layers (see [START_REF] Groma | Spatial correlations and higher-order gradient terms in a continuum description of dislocation dynamics[END_REF] for further details). The Groma-Balogh model neglects in particular the short range dislocation-dislocation correlations in one slip direction. For an extension to multiple slip see S. Yefimov and E. Van der Giessen [38, ch. 5.]. This multiple slip version of the Groma-Balogh model presents some analogies with some traffic flow models (see O. Biham et al. [START_REF] Biham | Self-organization and a dynamical transition in traffic-flow models[END_REF]). See also V. S. Deshpande et al. [START_REF] Deshpande | Finite strain discrete dislocation plasticity[END_REF] for a similar model with boundary conditions and exterior forces. Recently, A. EL-Azab [START_REF] El-Azab | Statistical mechanics treatment of the evolution of dislocation distributions in single crystals[END_REF], M. Zaiser, T. Hochrainer [START_REF] Zaiser | Some steps towards a continuum representation of 3d dislocation systems[END_REF] and R. Monneau [START_REF] Monneau | A kinetic formulation of moving fronts and application to dislocations dynamics[END_REF] were interested in modeling the dynamics of dislocation densities in the three-dimensional space, but many more open questions have to be solved for establishing a satisfactory three-dimensional theory of dislocations dynamics and for getting rigorous results.

We stress out the attention of the reader that there was no existence and uniqueness results for (P). In this paper we prove that (P) admits a "global in time" solution.

Main result

In the present paper, we prove a "global in time" existence result for the system (P) describing the dynamics of dislocation densities.

In this work, we consider the following initial conditions:

ρ ± (x 1 , x 2 , t = 0) = ρ ± 0 (x 1 , x 2 ) = ρ ±,per 0 (x 1 , x 2 ) + Lx 1 , (IC)
where ρ ±,per is a 1-periodic function in x 1 and x 2 . The periodicity is a way of studying the bulk behavior of the material away from its boundary. Here L is a given positive constant that represents the initial total dislocation densities of ± type on the periodic cell.

Before to give our main result, we want to show that the bilinear term on the right hand side of (P) is well defined. To this end, we need first to recall the following definition:

Definition 1.2 (The space L log L) We define the space L log L(T 2 ) L log L(T 2 ) = f ∈ L 1 (T 2 ) such that T 2 |f | ln (e + |f |) < +∞ .
This space is endowed with the (Luxemburg) norm

f L log L(T 2 ) = inf λ > 0 : T 2 |f | λ ln e + |f | λ ≤ 1 .
The space L log L(T We can now state the following proposition.

Proposition 1.3 (Meaning of the bilinear term)

Let T > 0, f and g be two functions defined on

T 2 × (0, T ), such that f ∈ L 1 ((0, T ); W 1,2 (T 2 )) and g ∈ L ∞ ((0, T ); L log L(T 2 )) then, f g ∈ L 1 (T 2 × (0, T )).
We will see that the proof of this proposition (given in Subsection 3.2) is a direct consequence of Trudinger inequality.

We can now state our main result (see also our comments in Subsection 1.3 on the unknown uniqueness of the solution).

Theorem 1.4 (Global existence)

For all T, L > 0 , and for every initial data

ρ ± 0 ∈ L 2 loc (R 2 ) with (H1) ρ ± 0 (x 1 + 1, x 2 ) = ρ ± 0 (x 1 , x 2 ) + L, a.e. on R 2 , (H2) ρ ± 0 (x 1 , x 2 + 1) = ρ ± 0 (x 1 , x 2 ), a.e. on R 2 , (H3) ∂ρ 0 ± ∂x 1 ≥ 0, a.e. on R 2 , (H4) ∂ρ 0 ± ∂x 1 L log L(T 2 ) ≤ C, with T 2 = R 2 /Z 2 , the system (P)-(IC) admits solutions ρ ± ∈ C([0, T ); L 1 loc (R 2 )) ∩ L ∞ ((0, T ); L 2 loc (R 2 )
) in the distributional sense, such that, ρ ± (•, t) satisfy (H1), (H2), (H3) and (H4) for a.e. t ∈ (0, T ). Moreover, we have:

(P1) R 2 1 R 2 2 (ρ + -ρ -) ∈ L 2 ((0, T ); W 1,2 loc (R 2 )).

Remark 1.5 (Bilinear term)

It is clear here that the bilinear term on the right hand side of(P) is always defined via (P 1) and Proposition 1.3.

In order to prove our main theorem we regularize the system (P) by adding the viscosity term (ε∆ρ ± ), and regularized also the initial data (IC) by classical convolution. Then, using a fixed point Theorem, we prove that our regularized system admits local in time solutions. Moreover, as we get some ε-independent a priori estimates we will be able to extend our local in time solution into a global one. This turns out to be possible thanks to the entropy inequality (1.1). Then, joined with other a priori estimates, it will be possible to prove some compactness properties and to pass to the limit as ε goes to 0 is the ε-problem.

Remark 1.6 (Entropy and energy inequalities)

It turns out that the constructed solution also satisfies the following fundamental entropy inequality (as a consequence of Lemma 5.4), for a.e. t ∈ (0, T ),

T 2 ± ∂ρ ± ∂x 1 ln ∂ρ ± ∂x 1 + t 0 T 2 R 1 R 2 ∂ρ + ∂x 1 - ∂ρ - ∂x 1 2 ≤ T 2 ± ∂ρ ± 0 ∂x 1 ln ∂ρ ± 0 ∂x 1 (1.1)
Moreover, (at least formally for sufficiently regular solution) the following energy inequality holds:

1 2 T 2 R 1 R 2 (ρ + -ρ -)(•, t) 2 + t 0 T 2 R 2 1 R 2 2 (ρ + -ρ -) 2 ∂ρ + ∂x 1 + ∂ρ - ∂x 1 ≤ 1 2 T 2 R 1 R 2 (ρ + 0 -ρ - 0 ) 2 Remark 1.

(Bounds on the solution)

If we denote ρ = ρ + -ρ -, then there exists a constant C independent on T , and a constant C T depending on T such that,

(E1) ρ ± -Lx 1 L ∞ ((0,T );L 2 (T 2 )) ≤ C T , (E4) R 2 1 R 2 2 ρ L 2 ((0,T );W 1,2 (T 2 )) ≤ C, (E2) ∂ρ ± ∂x 1 L ∞ ((0,T );L log L(T 2 )) ≤ C, (E5) R 2 1 R 2 2 ∂ρ ∂t L 2 ((0,T );W -1,2 (T 2 )) ≤ C, (E3) ∂ρ ± ∂t L 2 ((0,T );L 1 (T 2 )) ≤ C, where W -1,2 (T 2 ) is the dual space of W 1,2 (T 2 ).
In a particular sub-case of model (P) where the dislocation densities depend on a single variable x = x 1 + x 2 , the existence and uniqueness of a Lipschitz viscosity solution was proved in A. El Hajj, N. Forcadel [START_REF] Hajj | A convergent scheme for a non-local coupled system modelling dislocations densities dynamics[END_REF]. Also the existence and uniqueness of a strong solution in W 1,2 loc (R × [0, T )) was proved in A. El Hajj [START_REF] Hajj | Well-posedness theory for a nonconservative Burgers-type system arising in dislocation dynamics[END_REF]. Concerning the model of I. Groma, F. Csikor, M. Zaiser [START_REF] Groma | Spatial correlations and higher-order gradient terms in a continuum description of dislocation dynamics[END_REF] which takes into consideration the short range dislocation-dislocation correlations giving a parabolic-hyperbolic system, let us mention the work of H. Ibrahim [START_REF] Ibrahim | Existence and uniqueness for a non-linear parabolic/Hamilton-Jacobi system describing the dynamics of dislocation densities, to appear[END_REF] where a result of existence and uniqueness of a viscosity solution is given but only for a one-dimensional model.

Our study of the dynamics of dislocation densities in a special geometry is related to the more general dynamics of dislocation lines. We refer the interested reader to the work of O. Alvarez et al. [START_REF] Alvarez | Dislocation dynamics: short-time existence and uniqueness of the solution[END_REF], for a local existence and uniqueness of some non-local Hamilton-Jacobi equation. We also refer to O. Alvarez et al. [START_REF] Alvarez | Existence and uniqueness for dislocation dynamics with nonnegative velocity[END_REF] and G. Barles, O. Ley [START_REF] Barles | Nonlocal first-order Hamilton-Jacobi equations modelling dislocations dynamics[END_REF] for some long time existence results.

Comments on the uniqueness of the solution and related literature

The problem (P) is a system of transport equations with low regularity of the vector field, so that the uniqueness of the solution here is an open question. However, in the following we present some uniqueness results where the vector field has a better regularity.

From a technical point of view, (P) is related to other well known models, such as the transport equation with a low regularity vector field. This equation was studied in the work of R. J. Diperna, P. L. Lions [START_REF] Diperna | Ordinary differential equations, transport theory and Sobolev spaces[END_REF] and L. Ambrosio [START_REF] Ambrosio | Transport equation and Cauchy problem for BV vector fields[END_REF], where the authors showed the existence and uniqueness of renormalized solutions by considering vector fields in L 1 ((0, T ); W 1,1 loc (R N )) and L 1 ((0, T ); BV loc (R N )) respectively in both cases with bounded divergence. On the contrary in system (P), we are only able to prove that for the constructed solution, the vector field is in L 2 ((0, T ); W 1,2 loc (R 2 )) without any better estimate on the divergence of the vector field.

More generally in the frame of symmetric hyperbolic systems, we refer to the book of D. Serre [34, Vol I, Th 3.6.1], for a typical result of local existence and uniqueness in

C([0, T ); H s (R N )) ∩ C 1 ([0, T ); H s-1 (R N )), with s > N 2 + 1, by considering initial data in H s (R N ). This result remains local in time, even in dimension N = 2.
We can also remark that in the case where we multiply the right side of the two equations in system (P) by -1, we get a quasi-geostrophic-like system. For those who are concerned in quasi-geostrophic systems, we refer to P. Constantin et al. [START_REF] Constantin | Formation of strong fronts in the 2-D quasigeostrophic thermal active scalar[END_REF], and to [START_REF] Constantin | Singular front formation in a model for quasigeostrophic flow[END_REF] for certain 2D numerical results. We also refer to A. Córdoba, D. Córdoba [START_REF] Córdoba | Formation of singularities for a transport equation with nonlocal velocity[END_REF], D. Chae, A. Córdoba [START_REF] Chae | Finite time singularities in a 1D model of the quasi-geostrophic equation[END_REF] for blow-up results in finite time, in dimension one.

Let us also mention some related Vlasov-Poisson models (see J. Nieto et al. [START_REF] Nieto | High-field limit for the Vlasov-Poisson-Fokker-Planck system[END_REF] for instance) and a related model in superconductivity studied by N. Masmoudi et al. [START_REF] Masmoudi | Global solutions to vortex density equations arising from sup-conductivity[END_REF] and by L. Ambrosio et al. [START_REF] Ambrosio | A gradient flow approach to an evolution problem arising in superconductivity[END_REF]. These models were derived from some Vlasov-Poisson-Fokker-Planck models (see for instance T. Goudon et al. [START_REF] Goudon | Multidimensional highfield limit of the electrostatic Vlasov-Poisson-Fokker-Planck system[END_REF] for an overview of similar models). It is also worth mentioning that this model is related to Vlasov-Navier-Stokes equation see T. Goudon et al. [START_REF] Goudon | Hydrodynamic limit for the Vlasov-Navier-Stokes equations. I. Light particles regime[END_REF], [START_REF]Hydrodynamic limit for the Vlasov-Navier-Stokes equations. II. Fine particles regime[END_REF].

Notation

In what follows, we are going to use the following notation:

1. ρ = ρ + -ρ -, 2. ρ ±,per (x 1 , x 2 , t) = ρ ± (x 1 , x 2 , t) -Lx 1 ,
3. Let f be a function defined on R 2 × (0, T ) having values in R 2 , we denote by

f (t) = f (., t) : x -→ f (x, t),
4. Throughout the paper, C is an arbitrary positive constant independent on T and C T is an arbitrary positive constant depending on T .

Organization of the paper

First, in Section 2, we recall the physical derivation of system (P). In Section 3, we recall the definitions and properties of some useful fundamental spaces, and we give the proof of Proposition 1.3. We also prove that the bilinear term of our system has a better mathematical meaning (see Proposition 3.4). Next, in Section 4, we regularize the initial conditions and we show that the system (P), modified by a term (ε∆ρ ± ), admits local solutions. Moreover, we show that these solutions are regular and increasing for all t ∈ (0, T ), for increasing initial data. In Section 5, we prove some ε-uniform a priori estimates for the regularized solution obtained in Section 4. Then, thanks to these a priori estimates, we extend the local in time solutions for the ε-problem constructed in Section 4, in to global in time solution. Finally, in Section 6, we achieve the proof of our main theorem, passing to the limit in the equation as ε goes to 0, and using some compactness properties inherited from our a priori estimates.

Physical derivation of the model

In this section we explain how to derive physically the system (P). We consider a threedimensional crystal, with displacement

u = (u 1 , u 2 , u 3 ) : R 3 → R 3 .
For x = (x 1 , x 2 , x 3 ), and an orthogonal basis (e 1 , e 2 , e 3 ), we define the total strain by:

ε ij (u) = 1 2 ∂u i ∂x j + ∂u j ∂x i , i, j = 1, 2, 3.
This total strain is decomposed as

ε ij (u) = ε e ij + ε p ij ,
with ε e ij is the elastic strain and ε p ij the plastic strain which is defined by:

ε p ij = ρε 0 ij , (2.2)
with the fixed matrix

ε 0 ij = 1 2 (1 -δ ij )
, where δ ij is the Kronecker symbol, in the special case of a single slip system where dislocations move in the plane {x 2 = 0} with Burgers vector b = e 1 . Here γ is the resolved plastic strain, and will be clarified later. In the case of linear homogeneous and isotropic elasticity, the stress is given by

σ ij = 2µε e ij + λδ ij k=1,2,3 ε e kk for i, j = 1, 2, 3, (2.3)
where λ, µ are the constant Lam coefficients of the crystal (satisfying µ > 0 and 3λ+2µ > 0). Moreover the stress satisfies the equation of elasticity:

j=1,2,3 ∂σ ij ∂x j = 0.
We now assume that we are in a particular geometry where the dislocations are straight lines parallel to the direction e 3 and that the problem is invariant by translation in the x 3 direction. Moreover we assume that u 3 = 0 and σ i3 = 0 for i = 1, 2, 3. Then, this problem reduces to a two-dimensional problem with u 1 , u 2 only depending on (x 1 , x 2 ) and so we can express the resolved plastic strain ρ as

ρ = ρ + -ρ -,
where ∂ρ + ∂x 1 and ∂ρ - ∂x 1 are respectively the densities of dislocations of Burgers vectors

given by b = e 1 and b = -e 1 .

Furthermore, these dislocation densities are transported in the direction of the Burgers vector at a given velocity. This velocity is indeed the resolved shear stress i,j=1,2,3

σ ij ε 0 ij = σ 12
, up to sign of the Burgers vectors. More precisely, we have:

∂ρ ± ∂t = ±(σ 12 )e 1 .∇ρ ± .
Finally, the functions ρ ± and u = (u 1 , u 2 ) are solutions of the coupled system (see I. Groma, P. Balogh [START_REF] Groma | Investigation of dislocation pattern formation in a twodimensional self-consistent field approximation[END_REF], [START_REF] Groma | Link between the microscopic and mesoscopic lenght-scale description of the collective behaviour of dislocations[END_REF]), on R 2 × (0, T ):

                                     j=1,2 ∂σ ij ∂x j = 0 for i = 1, 2, σ ij = 2µε e ij + λδ ij k=1,2 ε e kk for i, j = 1, 2, ε e ij = 1 2
∂u i ∂x j + ∂u j ∂x i -(ρ + -ρ -)ε 0 ij for i, j = 1, 2, ε 0 ij = 1 2 (1 -δ ij ) for i, j = 1, 2, ∂ρ ± ∂t = ±σ 12 ∂ρ ± ∂x 1 .
(2.4)

Then the following lemma holds.

Lemma 2.1 (Computation of σ 12 ) Assume that (u 1 , u 2 ) and ρ = ρ + -ρ -are Z 2 -periodic functions. If (u 1 , u 2 ), ρ + , ρ -are solutions of problem (2.4), then σ 12 = -C 1 R 2 1 R 2 2 ρ , (2.5)
where

C 1 = 4 (λ + µ)µ λ + 2µ > 0.
Using this expression of σ 12 and rescaling in time with the positive constant C 1 we obtain system (P), from the last equation (2.4).

Proof of Lemma 2.1:

We can rewrite the first equation of (2.4) with div u =

∂u 1 ∂x 1 + ∂u 2 ∂x 2 µ∆u 1 + (λ + µ) ∂ ∂x 1 (div u) = µ ∂ρ ∂x 2 , (2.6a
)

µ∆u 2 + (λ + µ) ∂ ∂x 2 (div u) = µ ∂ρ ∂x 1 . (2.6b) Considering ∂ ∂x 1 (2.6a)+ ∂ ∂x 2 (2.6b), we get (λ + 2µ)∆(div u) = 2µ ∂ 2 ρ ∂x 1 ∂x 2 .
Plugging the expression of div u into (2.6), we get

∆u 1 = ∂ρ ∂x 2 -2 (λ + µ) (λ + 2µ) ∂ ∂x 1 ∆ -1 ∂ 2 ρ ∂x 1 ∂x 2 , (2.7a) ∆u 2 = ∂ρ ∂x 1 -2 (λ + µ) (λ + 2µ) ∂ ∂x 2 ∆ -1 ∂ 2 ρ ∂x 1 ∂x 2 . (2.7b) Considering now ∂ ∂x 2 (2.7a)+ ∂ ∂x 1 (2.7b) , we obtain ∆ ∂u 1 ∂x 2 + ∂u 2 ∂x 1 = ∆(ρ + -ρ -) -4 (λ + µ) (λ + 2µ) ∆ -1 ∂ 4 ∂x 2 1 ∂x 2 2 (ρ + -ρ -). (2.8)
Recalling that

σ 12 = µ ∂u 1 ∂x 2 + ∂u 2 ∂x 1 -(ρ + -ρ -) , (2.9)
this yields

σ 12 = -4 (λ+µ)µ (λ+2µ) ∆ -2 ∂ 4 ∂x 2 1 ∂x 2 2 (ρ + -ρ -) = -C 1 R 2 1 R 2 2 (ρ + -ρ -) . 2
Remark 2.2 (Property of the elastic energy) If we define the elastic energy by

E = R 2 /Z 2 µ i,j=1,2 (ε e ij ) 2 + λ 2 k=1,2 ε e kk 2 .
Using system (2.4) we can show formally that

dE dt = - R 2 /Z 2 (σ 12 ) 2 ∂ρ + ∂x 1 + ∂ρ - ∂x 1 ≤ 0.
where we have used the fact that ∂ρ + ∂x 1 , ∂ρ - ∂x 1 ≥ 0 to see that the elastic energy is a non-increasing in time. Hence, the elastic energy E is a Lyapunov functional for our dissipative model.

3 Concerning the meaning of the solution of (P)

In this section we prove Proposition 1.3. This shows that if (P) admits solutions verifying the conditions of Theorem 1.4, then we can give a mathematical meaning to the bilinear term. In order to do this, we need to define some functional spaces and recall some of their properties, that will be used later in our work.

Properties of some useful Orlicz spaces

We recall the definition of Orlicz spaces and some of their properties. For details, we refer to R. A. • A is a continuous, non-negative, non-decreasing and convex function.

• A(0) = 0 and lim t→+∞ A(t) = +∞.

Let A(•) be a Young function. The Orlicz class K A (T 2 ) is the set of (equivalence classes of) real-valued measurable function h on T 2 satisfying

T 2 A(|h(x)|) < +∞.
The Orlicz space L A (T 2 ) is the linear hull of K A (T 2 ) supplemented with the Luxemburg norm

f L A (T 2 ) = inf λ > 0 : T 2 A |h(x)| λ ≤ 1 .
Endowed with this norm, the Orlicz space L A (T 2 ) is a Banach space. Moreover, for all f ∈ L A (T 2 ), we have the following estimate

f L A (T 2 ) ≤ 1 + T 2 A(|f (x)|) (3.10)
Definition 3.1 (Some Orlicz spaces)

• EXP α (T 2 ) denotes the Orlicz space defined by the function A(t) = e t α -1 for α ≥ 1.

• L log β L(T 2 ) denotes the Orlicz space defined by the function A(t) = t(log(e + t)) β , for β ≥ 0.

Observe that for 0 < β ≤ 1 the space EXP

L log 1 L(T 2 ) = L log L(T 2 ).
Let us recall some useful properties of these spaces. The first one is the generalized Hlder inequality.

Lemma 3.2 (Generalized Hlder inequality) i) Let f ∈ EXP 2 (T 2 ) and g ∈ L log 1 2 L(T 2 ).
Then there exists a constant C such that (see R. O'Neil [START_REF] Neil | Fractional integration in Orlicz spaces. I[END_REF]Th 2.3])

f g L 1 (T 2 ) ≤ C f EXP 2 (T 2 ) g L log 1 2 L(T 2 ) . ii) Let f ∈ EXP 2 (T 2 ) and g ∈ L log L(T 2 ). Then there exists a constant C such that (see R. O'Neil [31, Th 2.3]) f g L log 1 2 L(T 2 ) ≤ C f EXP 2 (T 2 ) g L log L(T 2 ) .
The second property is the Trudinger inequality.

Lemma 3.3 (Trudinger inequality)

There exists a constant γ > 0 such that, for all f ∈ W 1,2 (T 2 ), we have (see N. S. Trudinger [START_REF] Trudinger | On imbeddings into Orlicz spaces and some applications[END_REF])

T 2 e γ f f W 1,2 (T 2 ) 2 ≤ 1.
In particular we have the following embedding

W 1,2 (T 2 ) ֒→ EXP 2 (T 2 ).

Sharp estimate of the bilinear term

Now, we propose to verify with the help of the following proposition that the system (P) has indeed a sense, and first prove a better estimate than those mentioned in Proposition 1.3. Namely, we have the following. Proposition 3.4 (Estimate of the bilinear term) Let T > 0, f and g be two functions defined on T 2 × (0, T ), such that

(1) f ∈ L 2 ((0, T ); W 1,2 (T 2 )), (2) g ∈ L ∞ ((0, T ); L log L(T 2 )). Then f g ∈ L 2 ((0, T ); L log 1 2 L(T 2 ))
and for a positive constant C, we have:

f g L 2 ((0,T );L log 1 2 L(T 2 )) ≤ C f L 2 ((0,T );W 1,2 (T 2 )) g L ∞ ((0,T );L log L(T 2 )) .
For the proof of this Proposition , we use Lemma 3.2 (ii), and integrate in time. Thanks to the Trudinger inequality (Lemma 3.3), we get the result. We do the same way for the proof of the Proposition 1.3.

Local existence of solutions of a regularized system

In this section, we state a local in time existence result for system (P), modified by the term ε∆ρ ± , and for smoothed data. This modification brings us to study, for all 0 < ε ≤ 1, the following regularized system:

         ∂ρ +,ε ∂t -ε∆ρ +,ε = -(R 2 1 R 2 2 ρ ε ) ∂ρ +,ε ∂x 1 in D ′ (R 2 × (0, T )), ∂ρ -,ε ∂t -ε∆ρ -,ε = (R 2 1 R 2 2 ρ ε ) ∂ρ -,ε ∂x 1 in D ′ (R 2 × (0, T )), (P ε )
where ρ ε = ρ +,ε -ρ -,ε , with the following regular initial data:

ρ ±,ε (x, 0) = ρ ±,ε 0 (x) = ρ ±,per 0 * η ε (x) + (L + ε)x 1 = ρ ±,ε,per 0 (x) + L ε x 1 , (IC ε )
where

η ε (•) = 1 ε 2 η( • ε ), such that η ∈ C ∞ c (R 2
) is a non-negative function and R 2 η = 1.

Remark 4.1

We consider L ε = L + ε to obtain strictly monotonous initial data ρ ±,ε 0 . This condition will be useful in the proof of Lemma 5.4. For the regularized system (P ε )-(IC ε ) we have the following result.

Theorem 4.2 (Local existence result of monotone smooth solutions)

For all initial data ρ ± 0 ∈ L 2 loc (R 2 ) satisfying (H1), (H2) and (H3), and all ε > 0, there exists T ⋆ > 0 such that the system (P ε )-

(IC ε ) admits solutions ρ ±,ε ∈ C ∞ (R 2 × [0, T ⋆ )).
Moreover ρ ±,ε (•, t) satisfy (H1), (H2) and ∂ρ ±,ε ∂x 1 > 0, for all t ∈ [0, T ⋆ ).

Before proving Theorem 4.2, let us recall some well known results.

We first recall the Picard fixed point result which will be applied in the proof of this theorem in order to prove, the existence of solutions.

Lemma 4.3 (Picard Fixed point Theorem)

Let E be a Banach space, B is a continuous bilinear application over E × E having values in E, and A a continuous linear application over E having values in E such that:

B(x, y) E ≤ η x E y E for all x, y ∈ E, A(x) E ≤ µ x E for all x ∈ E,
where η > 0 and µ ∈ (0, 1) are two given constants. Then, for every x 0 ∈ E verifying

x 0 E < 1 4η (1 -µ) 2 , the equation x = x 0 + B(x, x) + A(x) admits a solution in E.
For the proof of Lemma 4.3, see M. Cannone [9, Lemma 4.2.14].

We now recall the following decay estimates for the heat semi-group.

Lemma 4.4 (Decay estimate)

Let r, p, q ≥ 1. Then, for all functions f ∈ L q (T 2 ) and g ∈ L p (T 2 ), where

1 r ≤ 1 q + 1 p ,
we have, for S 1 (t) = e t∆ , the following estimates:

i) S 1 (t)(f g) L r (T 2 ) ≤ Ct -( 1 p + 1 q -1 r ) f L q (T 2 ) g L p (T 2 ) for all t > 0, ii) ∇S 1 (t)(f g) L r (T 2 ) ≤ Ct -( 1 2 + 1 p + 1 q -1 r ) f L q (T 2 ) g L p (T 2 ) for all t > 0,
where C is a positive constant depending only on r, p, q.

The proof of this lemma is a direct application of the classical version of the L r -L p estimates for the heat semi-group (see A. Pazy [32, Lemma 1.1.8, Th 6.4.5]) and the Hlder inequality.

Here is now, the demonstration of Theorem 4.2.

Proof of Theorem 4.2:

Frist we prove using Lemma 4.3 the local existence of the regularized system (P ε )-(IC ε ). This result is achieved in a super-critical space. Here particularly we chose the space of functions L ∞ ((0, T );

W 1, 3 2 loc (R 2 )
). The notation "super-critical space" is to say that we are choosing a space where our ε-problem is well defined, and where the right hand term (the bilinear term) is in a space better than L 1 . This premits to use a bootstrap arguments which easily leads to the existence of smooth solution of the regularized problem. Now, we note that, if we let ρ ±,ε,per = ρ ±,ε -L ε x 1 , we know that the system (P ε ) is equivalent to,

∂ρ ±,ε ∂t -ε∆ρ ±,ε,per = ∓(R 2 1 R 2 2 ρ ε ) ∂ρ ∂x 1 ±,ε,per ∓ L ε (R 2 1 R 2 2 ρ ε ) in D ′ (T 2 × (0, T )), (P per ε )
with initial conditions,

ρ ±,ε,per (x, 0) = ρ ±,ε 0 (x) -L ε x 1 = ρ ±,ε,per 0 (x). (IC per ε )
To solve this system in the space L ∞ ((0, T ); W 1, 3 2 (T 2 )) we reduce to construct a solution ρ ±,ε,per to the following integral problem (see A. Pazy [32, Th 5.2, Page 146])

ρ ±,ε,per (•, t) = S ε (t)ρ ±,ε,per 0 ∓L ε t 0 S ε (t -s) R 2 1 R 2 2 ρ ε (s) ds ∓ t 0 S ε (t -s) R 2 1 R 2 2 ρ ε (s) ∂ρ ∂x 1 ±,ε,per (s) ds, (In ε )
where S ε (t) = S 1 (εt), and S 1 (t) = e t∆ is a the heat semi-group. We rewrite the system (In ε ) in t he following vectorial form:

ρ ε v (x, t) = S ε (t)ρ ε 0,v + L ε J1 t 0 S ε (t -s) R 2 1 R 2 2 ρ ε (s) ds + Ī1 t 0 S ε (t -s) R 2 1 R 2 2 ρ ε (s) ∂ρ ε v ∂x 1 (s)ds, where S ε (t) = S 1 (εt), ρ ε v = (ρ +,ε,per , ρ -,ε,per ), ρ ε 0,v = (ρ +,ε,per 0 , ρ -,ε,per 0 ), Ī1 = -1 0 0 1 and J1 = - 1 1
.

Which is equivalent to,

ρ ε v (x, t) = S ε (t)ρ ε 0,v + B(ρ ε v , ρ ε v )(t) + A(ρ ε v )(t), (4.11)
where B is a bilinear map and A is a linear one defined respectively, for every vector u = (u 1 , u 2 ) and v = (v 1 , v 2 ), as follows:

B(u, v)(t) = Ī1 t 0 S ε (t -s) R 2 1 R 2 2 (u 1 -u 2 )
∂v ∂x 1 (s) ds, (4.12)

A(u)(t) = L ε J1 t 0 S ε (t -s) R 2 1 R 2 2 (u 1 -u 2 )(s) ds. (4.13)
Now, we apply Lemma 4.3 to equation (4.11). First of all, we estimate the bilinear term,

B(u, v)(t) (W 1, 3 2 (T 2 )) 2 ≤ Ī1 t 0 S ε (t -s) R 2 1 R 2 2 (u 1 -u 2 ) ∂v ∂x 1 (s) ds (W 1, 3 2 (T 2 )) 2 ≤ t 0 S ε (t -s) R 2 1 R 2 2 (u 1 -u 2 ) ∂v ∂x 1 (s) ds (W 1, 3 2 (T 2 )) 2 .
Then, since L 4 (T 2 ) ֒→ L 3 2 (T 2 ), we have,

B(u, v)(t) (W 1, 3 2 (T 2 )) 2 ≤ t 0 S ε (t -s) R 2 1 R 2 2 (u 1 -u 2 ) ∂v ∂x 1 (s) ds (L 4 (T 2 )) 2 + t 0 ∇S ε (t -s) R 2 1 R 2 2 (u 1 -u 2 ) ∂v ∂x 1 (s) ds (L 3 2 (T 2 )) 2 . ( 4.14)
We use Lemma 4.4 (i) with r = 4, q = 3, p = 3 2 to estimate the first term and Lemma 4.4 (ii) with r = 3 2 , q = 4, p = 3 2 to estimate the second term. We get for 0 ≤ t ≤ T , and with constants C depending on ε,

B(u, v)(t) (W 1, 3 2 (T 2 )) 2 ≤ C t 0 1 (t -s) 3 4 R 2 1 R 2 2 u(s) (L 4 (T 2 )) 2 ∂v ∂x 1 (s) (L 3 2 (T 2 )) 2 ds ≤ C sup 0≤s<T ( u(s) (W 1, 3 2 (T 2 )) 2 ) sup 0≤s<T ( v(s) (W 1, 3 2 (T 2 )) 2 ) t 0 1 (t -s) 3 4
ds.

Here we have used in the second line the property that Riesz transformations are continuous from L ). Hence we have,

B(u, v) L ∞ ((0,T );(W 1, 3 2 (T 2 )) 2 ) ≤ η(T ) u L ∞ ((0,T );(W 1, 3 2 (T 2 )) 2 ) v L ∞ ((0,T );(W 1, 3 2 (T 2 )) 2 ) , (4.15)
with η(T ) = C 0 T 1 4 for some constant C 0 > 0. We estimate the linear term in the same way to get,

A(u) L ∞ ((0,T );(W 1, 3 2 (T 2 )) 2 ) ≤ L ε η(T ) u L ∞ ((0,T );(W 1, 3 2 (T 2 )) 2 ) . (4.16)
Moreover, we know by classical properties of heat semi-group that,

S ε (t)ρ ε 0,v L ∞ ((0,T );(W 1, 3 2 (T 2 )) 2 ) ≤ ρ ε 0,v (W 1, 3 2 (T 2 )) 2 . (4.17)
Now, if we take

(T ⋆ ) 1 4 = min 1 2C 0 L ε , 1 16C 0 ρ ε 0,v (W 1, 3 2 (T 2 )) 2 , ( 4.18)
we can easily verify that we have the following inequalities:

ρ ε 0,v (W 1, 3 2 ) 2 (T 2 ) < 1 4η(T ⋆ ) (1 -L ε η(T ⋆ )) 2 , and L ε η(T ⋆ ) < 1, (4.19)
Using inequalities (4.15), (4.16), (4.17), (4.19) and Lemma 4.3 with the space

E = L ∞ ((0, T ⋆ ); W 1, 3 2 (T 2 )) 2
, we show the local in time existence or the system (4.11) in L ∞ ((0, T ⋆ ); W 1, 3 2 (T 2 ))

2

. As a consequence we prove that the system (P ε )-(IC ε ) admits some solutions ρ ±,ε ∈ L ∞ ((0, T ⋆ ); W Finally, the fact that product (R

2 1 R 2 2 ρ ε ) ∂ρ ∂x 1 ±,ε,per is well defined in L ∞ ((0, T ); L 6 5 (T 2 )) since L ∞ ((0, T ); W 1, 3 2 (T 2 )) ֒→ L ∞ ((0, T ); L 6 (T 2
)), we can prove, by a bootstrap argument, the regularity of the solution. The monotonicity of the solution is a consequence of the maximum principle for scalar parabolic equations the previous result (see G. Lieberman [START_REF] Lieberman | Second order parabolic differential equations[END_REF]Th 2.10]). 2

5 ε-Uniform estimates on the solution of the regularized system

In this section, we prove some fundamental ε-uniform estimates. In Subsection 5.1 we give some general estimates which are independent on the equation. In the second Subsection 5.2 we establish a priori estimates on the solutions of system (P ε ).

Useful estimates

Now we recall some well known properties of Riesz transform, that will be used later in our work.

Lemma 5.1 (Properties of Riesz transform) i) For all g ∈ L p (T 2 ), 1 < p < +∞, we have

R i g L p (T 2 ) ≤ g L p (T 2 ) . ii) If g ∈ L 2 (T 2 ), then R/Z R 1 g(x 1 , x 2 )dx 1 = 0, for a.e. x 2 ∈ R/Z.
iii) For all g ∈ L 2 (T 2 ), we have

∂ ∂x 1 R 2 g = ∂ ∂x 2 R 1 g and R 1 R 2 g = R 2 R 1 g.
iv) For all f, g ∈ L 2 (T 2 ), we have

T 2 (R i f )g = T 2 f (R i g). v) If g ∈ L 2 (T 2
) and does not depend on x 2 , then R 1 g = 0.

Proof of Lemma 5.1:

For the proof of i) (see A. Zygmund [40, Vol I, Page 254, (2.6)]). The proof of iv) this is straightforward, using Fourier series. For the proof of ii), it suffices to note that, if

we denote by f (x 2 ) = R/Z R 1 g(x 1 , x 2 )dx 1 , then we have c k 2 (f ) = c (0,k 2 ) (R 1 g) = 0 by definition of c k for k 1 = 0.
Finally, we prove iii), checking simply that

c k ∂ ∂x 1 R 2 g = 2πik 1 k 2 |k| c k (g) = 2πik 2 k 1 |k| c k (g) = c k ∂ ∂x 2 R 1 g ,
and similar we prove second equality of iii). The prove of v) is direct. In fact,

c (k 1 ,k 2 ) (R 1 g) = k 1 |k| T 2 g(x 2 )e -2πi(k 1 x 1 +k 2 x 2 ) dx 1 dx 2 = 0. 2 Lemma 5.2 (L ∞ estimate) If f ∈ L 2 loc (R 2
) and f verifies (H1), (H2) and (H3) for a.e. t ∈ (0, T ), then there exists a constant C = C(L) such that,

f per - 1 0 f per dx 1 L ∞ (T 2 ) ≤ C.
(5.20)

where

f per = f -Lx 1 .
Proof of Lemma 5.2:

We compute

1 0 ∂f per ∂x 1 dx 1 = 1 0 ∂f ∂x 1 -L dx 1 ≤ L + 1 0 ∂f ∂x 1 dx 1 = L + 1 0 ∂f ∂x 1 dx 1 = 2L,
where we use (H3) in the second line and (H1) in the last line. We next apply a "Poincar-Wirtinger inequality" in x 1 and we deduce the result. 2 We will also use the following technical result. Lemma 5.3 (L log L Estimate) Let (η ε ) ε be a non-negative mollifier, then for all f ∈ L log L(T 2 ) and f ≥ 0, the function

f ε = f * η ε satisfies T 2 f ε ln f ε → T 2 f ln f as ε → 0.
For the proof see R. A. Adams [START_REF] Adams | Sobolev spaces[END_REF]Th 8.20].

A priori estimates

In this subsection, we show some ε-uniform estimates on the solutions of the system (P ε )-(IC ε ) obtained in Theorem 4.2. These estimates will be used, on the one hand to extend the solution in a global one and, on the other hand in Subsection 6.2, for ensuring by compactness the passage to the limit as ε tends to zero. The first estimate concerns the physical entropy of the system, and is a key result. It shows that in our model, the dislocation densities cannot be so concentrated and then can be controlled.

Lemma 5.4 (Entropy estimate)

Let ρ ± 0 ∈ L 2 loc (R 2 ). If ρ ±,ε ∈ C ∞ (R 2 × [0, T ))
are solutions of the system (P ε )-(IC ε ) and ρ ±,ε (•, t) satisfy (H1), (H2), (H3) and (H4), then

T 2 ± ∂ρ ±,ε ∂x 1 ln ∂ρ ±,ε ∂x 1 + t 0 T 2 R 1 R 2 ∂ρ ε ∂x 1 2 ≤ T 2 ± ∂ρ ±,ε 0 ∂x 1 ln ∂ρ ±,ε 0 ∂x 1 , (5.21
)

where ρ ε = ρ +,ε -ρ -,ε .
In particular, there exists a constant C independent of ε ∈ (0, 1] such that

∂ρ ±,ε ∂x 1 L ∞ ((0,T );L log L(T 2 )) + ∂ ∂x 1 (R 1 R 2 ρ ε ) L 2 (T 2 ×(0,T )) ≤ C (5.22) with C = C ∂ρ 0 ± ∂x 1 L log L(T 2 ) .
Proof of Lemma 5.4:

First of all, we denote θ ±,ε = ∂ρ ±,ε ∂x 1 and N ± (t) =

T 2 θ ±,ε (t) ln(θ ±,ε (t)).
Using the fact that ρ ±,ε ∈ C ∞ (R 2 × [0, T )), we can derive N(t) = N + (t) + N -(t) with respect to t, and since θ ±,ε > 0, we obtain:

d dt N(t) = T 2 +,- (θ ±,ε ) t ln(θ ±,ε ) + T 2 +,- (θ ±,ε ) t .
Using system (P ε ) we see that the second term is zero. Moreover, we get

d dt N(t) = T 2 +,- ∓ (R 2 1 R 2 2 ρ ε )θ ±,ε x 1 + ε∆θ ±,ε ln(θ ±,ε ).
Integrating by part in x 1 , we get

d dt N(t) = T 2 +,- ±(R 2 1 R 2 2 ρ ε )θ ±,ε θ ±,ε x 1 θ ±,ε -ε +,-T 2 |∇θ ±,ε | 2 θ ±,ε = T 2 R 2 1 R 2 2 ρ ε ∂θ ε ∂x 1 -ε +,-T 2 |∇θ ±,ε | 2 θ ±,ε
where θ ε = θ +,ε -θ -,ε . We integrate also the first term by part in x 1 , and we deduce that

d dt N(t) = - T 2 R 2 1 R 2 2 θ ε θ ε -ε +,-T 2 |∇θ ±,ε | 2 θ ±,ε = - T 2 (R 1 R 2 θ ε ) 2 -ε +,-T 2 |∇θ ±,ε | 2 θ ±,ε ≤ 0,
where we have used Lemma 5.1 (iii) and (iv) for the second line.

Integrating in time, we get

N(t) + t 0 T 2 (R 1 R 2 θ ε ) 2 ≤ N(0).
Which proves (5.21). Moreover, we have

N(0) ≤ T 2 +,- θ ±,ε (0) log(e + θ ±,ε (0)).
Since the initial data (IC) satisfies (H4), we deduce by Lemma 5.3 that there exists a positive constant C independent of ε ∈ (01] such that

N(t) + t 0 T 2 (R 1 R 2 θ ε ) 2 ≤ C.
Let us now consider

N ± 1 (t) = T 2 θ ±,ε (t) log(e + θ ±,ε (t)).
We deduce, with another constant C ′ > 0, that

N + 1 (t) + N - 1 (t) + t 0 T 2 (R 1 R 2 θ ε ) 2 ≤ C ′
which joint to Lemma 3.10 implies (5.22). 2

Remark 5.5 (L 2 estimate on the gradient of the vector field)

We want to bound ∇ (R 2 1 R 2 2 ρ ε ). To this end, remark that by the property of Riesz transform (see Lemma 5.1 (iii)), we have

∂ ∂x 1 R 2 1 R 2 2 ρ ε = R 1 R 2 ∂ ∂x 1 R 1 R 2 ρ ε and ∂ ∂x 2 R 2 1 R 2 2 ρ ε = R 2 2 ∂ ∂x 1 R 1 R 2 ρ ε ,
where those quantities involve (5.22).

∂ ∂x 1 R 1 R 2 ρ ε which is bounded in L 2 (T 2 × (0, T )) by
Then using the fact the Riesz transforms are continuous from L 2 onto itself (see Lemma 5.1 (i)), we deduce that

∇ R 2 1 R 2 2 ρ ε L 2 (T 2 ×(0,T )) ≤ C, (5.23)
where the constant C is independent on ε.

We now present a second a priori estimate.

Lemma 5.6 (L 2 bound on the solutions)

Let T > 0. Under the condition ρ ± 0 ∈ L 2 loc (R 2 ). If ρ ±,ε ∈ C ∞ (R 2 × [0, T )
) are solutions of system (P ε )-(IC ε ) and ρ ±,ε (•, t) satisfy (H1), (H2), (H3) and (H4), then there exists a constant C T independent of ε ∈ (01], but depending on T , such that:

ρ ±,ε,per L ∞ ((0,T );L 2 (T 2 )) ≤ C T with ρ ±,ε,per = ρ ±,ε -Lx 1 .
Proof of Lemma 5.6:

Let T = R/Z. We want to bound m ±,ε (x 2 , t) = T ρ ±,ε,per (x 1 , x 2 , t)dx 1 . There is no problem of regularity since ρ ±,ε ∈ C ∞ (R 2 × [0, T )).
We integrate equation (P ε ) with respect to x 1 , and we get

∂m ±,ε ∂t -ε ∂ 2 m ±,ε ∂x 2 2 = ± T (R 2 1 R 2 2 ∂ρ ε ∂x 1 )(ρ ±,ε,per -m ±,ε )dx 1 ± m ±,ε T (R 2 1 R 2 2 ∂ρ ε ∂x 1 )dx 1 ∓L ε T (R 2 1 R 2 2 ρ ε )dx 1 , (5.24)
where for the first line we have integrated by part, and introduced the mean value m ±,ε . Therefore, using that ρ ε is a 1-periodic function in x 1 and Lemma 5.1 (ii) and (iii), we deduce that

T (R 2 1 R 2 2 ρ ε )dx 1 = 0 = T (R 2 1 R 2 2 ∂ρ ε ∂x 1 )dx 1 , Equation (5.24) is then equivalent to ∂m ±,ε ∂t -ε ∂ 2 m ±,ε ∂x 2 2 = ± T (R 2 1 R 2 2 ∂ρ ε ∂x 1 )(ρ ±,ε,per -m ±,ε )dx 1 = I ± (x 2 , t).
(5.25)

We now show that I ± ∈ L 2 (T × (0, T )). Indeed, we have

I ± L 2 (T×(0,T )) ≤ T (R 2 1 R 2 2 ∂ρ ε ∂x 1 )(ρ ±,ε,per -m ±,ε )dx 1 L 2 (T×(0,T )) ≤ ρ ±,ε,per -m ±,ε L ∞ (T 2 ×(0,T )) R 2 1 R 2 2 ∂ρ ε ∂x 1 L 2 (T 2 ×(0,T )) ≤ C
where for the last line we have used (5.23) and (Lemma 5.

1 (i)) to bound R 2 1 R 2 2 ∂ρ ε ∂x 1 L 2 (T 2 ×(0,T )) .
Furthermore, the bound ρ ±,ε,per -m ±,ε L ∞ (T 2 ×(0,T )) ≤ C follows from (5.20).

Multiplying (5.25) by m ±,ε and integrating in x 2 , we get 1 2

d dt m ±,ε (•, t) 2 L 2 (T) + ε ∂ ∂x 2 m ±,ε (•, t) 2 L 2 (T) = T (I ± m ±,ε )(,t).
Using Cauchy-Schwarz inequality on the right hand side, we deduce that 1 2

d dt m ±,ε (•, t) 2 L 2 (T) ≤ I ± (•, t) L 2 (T) .
We conclude to the result by integrating in time. 2

Corollary 5.7 (W 1,2 estimate on the vector field) Under the assumptions

ρ ± 0 ∈ L 2 loc (R 2 ). If ρ ±,ε ∈ C ∞ (R 2 × [0, T ))
are solutions of the system (P ε )-(IC ε ) and ρ ±,ε (•, t) satisfy (H1), (H2), (H3) and (H4), then there exists a constant C independent of ε such that: (5.23) and the fact that R 2 1 R 2 2 ρ ε is of null average (see Lemma 5.1 (ii)) and applying "Poincar-Wirtinger inequality", we can prove the result. The following estimate will provide compactness in time of the solution, uniform with respect to ε. Let T > 0. Under the assumptions

R 2 1 R 2 2 ρ ε L 2 ((0,T );W 1,2 (T 2 )) ≤ C, Using
ρ ± 0 ∈ L 2 loc (R 2 ). If ρ ±,ε ∈ C ∞ (R 2 × [0, T )
) are solutions of the system (P ε )-(IC ε ) and ρ ±,ε (•, t) satisfy (H1), (H2), (H3) and (H4), then i) For all ψ ∈ L 2 ((0, T ); W 1,2 (T 2 )), there exists a constant C independent of ε ∈ (0, 1] such that:

T 2 ×(0,T ) ψR 2 1 R 2 2 ∂ρ ε ∂t ≤ C ψ L 2 ((0,T );W 1,2 (T 2 ))
where

ρ ε = ρ +,ε -ρ -,ε .
ii) For all ψ ∈ L 2 ((0, T ); W 2,2 (T 2 )), there exists a constant C T independent of ε ∈ (0, 1] such that:

T 2 ×(0,T ) ψ ∂ρ ±,ε ∂t ≤ C T ψ L 2 ((0,T );W 2,2 (T 2 )) .
Proof of Lemma 5.8:

Proof of (i): The idea is somehow to bound R 2 1 R 2 2 ∂ρ ε ∂t using the available bounds on the right hand side of the equation (P ε ).

We will give a proof by duality. First of all, we subtract the two equations of system (P ε ) and we apply the Riesz transform R 2 1 R 2 2 , to obtain that

R 2 1 R 2 2 ∂ρ ε ∂t = - I 1 R 2 1 R 2 2 (R 2 1 R 2 2 ρ ε ) ∂k ε ∂x 1 + I 2 εR 2 1 R 2 2 (∆ρ ε ) (5.26) with k ε = ρ +,ε + ρ -,ε .
In what follows, we will prove that for a function ψ ∈ L 2 ((0, T ); W 1,2 (T 2 )), we can bound

J i = T 2 ×(0,T ) ψI i for i = 1, 2.
Estimate of J 2 : To estimate J 2 , we integrate by part, to get:

J 2 = -ε T 2 ×(0,T ) ∇(R 2 1 R 2 2 ρ ε ) • ∇ψ.
We deduce that for all ε ∈ (01]:

|J 2 | ≤ R 2 1 R 2 2 ρ ε L 2 ((0,T );W 1,2 (T 2 )) ψ L 2 ((0,T );W 1,2 (T 2 )) ≤ C ψ L 2 ((0,T );W 1,2 (T 2 )) , (5.27)
where we have used Corollary 5.7 in the last line.

Estimate of J 1 : To control J 1 , we rewrite it under the following form:

T 2 ×(0,T ) R 2 1 R 2 2 (R 2 1 R 2 2 ρ ε ) ∂k ε ∂x 1 ψ = T 2 ×(0,T ) (R 2 1 R 2 2 ρ ε ) ∂k ε ∂x 1 (R 2 1 R 2 2 ψ).
We use the fact that

(i) (R 2 1 R 2 2 ρ ε ) is bounded in L 2 ((0, T ); W 1,2 (T 2 )) uniformly in ε (by Corollary 5.7), (ii) ∂k ε ∂x 1 is bounded in L ∞ ((0, T ); L log L(T 2
)), uniformly in ε (by Lemma 5.4).

We deduce from this and from Proposition 3.4, (with

f = R 2 1 R 2 2 ρ ε and g = ∂k ε ∂x 1 ) the following estimate: (R 2 1 R 2 2 ρ ε ) ∂k ε ∂x 1 L 2 ((0,T );L log 1 2 L(T 2 )) ≤ C R 2 1 R 2 2 ρ ε L 2 ((0,T );W 1,2 (T 2 )) ∂k ε ∂x 1 L ∞ ((0,T );L log L(T 2 )) ≤ C ∂k ε ∂x 1 L ∞ ((0,T );L log L(T 2 )) ≤ C.
We use Lemma 3.2 (i), to deduce that

|J 1 | ≤ T 2 ×(0,T ) (R 2 1 R 2 2 ρ ε ) ∂k ∂x 1 ε (R 2 1 R 2 2 ψ) ≤ (R 2 1 R 2 2 ρ ε ) ∂k ∂x 1 ε L 2 ((0,T );L log 1 2 L(T 2 )) R 2 1 R 2 2 ψ L 2 ((0,T );EXP 2 (T 2 )) ≤ C R 2 1 R 2 2 ψ L 2 ((0,T );W 1,2 (T 2 )) ≤ C ψ L 2 ((0,T );W 1,2 (T 2 )) (5.28)
where we have used the Trudinger inequality (see Lemma 3.3) in the third line and the fact that Riesz transforms are continuous from L 2 onto itself in the last line (see Lemma 5.1 (i)). Finally, collecting (5.28) and (5.27) together with (5.26) and the definitions of J i , for i = 1, 2, we get that there exists a constant C independent of ε such that

T 2 ×(0,T ) ψR 2 1 R 2 2 ( ∂ρ ε ∂t ) ≤ C ψ L 2 ((0,T );W 1,2 (T 2 )) . t ∈ (0, T ), that, B(ρ ε v , ρ ε v )(t) (W 1, 3 2 (T 2 )) 2 ≤ C t 0 1 (t -s) 23 24 R 2 1 R 2 2 ρ ε (s) L 8 (T 2 ) ∂ρ ε v ∂x 1 (s) (L 1 (T 2 )) 2 ds ≤ C sup 0≤s<T R 2 1 R 2 2 ρ ε (s) L 8 (T 2 ) sup 0≤s<T ∂ρ ε v ∂x 1 (s) (L 1 (T 2 )) 2 t 0 1 (t -s) 23 24
• That leads,

B(ρ ε v , ρ ε v ) L ∞ ((0,T );(W 1, 3 2 (T 2 )) 2 ) ≤ CT 1 24 R 2 1 R 2 2 ρ ε L ∞ ((0,T );L 8 (T 2 )) ∂ρ ε v ∂x 1 L ∞ ((0,T );(L 1 (T 2 )) 2 ) •
(5.29) Similarly, we show that,

A(ρ ε v ) L ∞ ((0,T );W 1, 3 2 (T 2 )) ≤ CT 1 24 R 2 1 R 2 2 ρ ε L ∞ ((0,T );L 8 (T 2 )) .
(5.30)

By using (5.29), (5.30) and (4.17), and the equation ((4.11)) we get the proof. 2 Proof of Theorem 5.10: We argue by contradiction. Suppose that there exists a maximum time T max such that we have the existence of solutions of (P

ε )-(IC ε ) in C ∞ (R 2 × [0, T max )).
For δ > 0, we reconsider the system (P ε ) with the initial data ρ ±,ε δ,max = ρ ±,ε (x, T max -δ).

We reapply for the second time, the proof of Theorem 4.2, we deduce that there exists a time

T ⋆ δ,max ( ρ ±,ε,per δ,max W 1, 3
such that the system (P ε )-(IC ε ) admits solutions defined until,

T 0 = (T max -δ) + T ⋆ δ,max .
Moreover, from Lemmata 5.2 5.1 (v) and 5.1 (i) with p = 8, we can deduce easily that

R 2 1 R 2 2 (ρ ε ) is bounded on L ∞ ((0, T ), L 8 (T 2
)). Now, by Lemmata 5.11 and 5.4, we know that ρ ±,ε,per δ,max are δ-uniformly bounded in W 1, 32 (T 2 ). By using (4.18), we deduce that there exists a constant C(ε, T max , L) > 0 independent of δ such that T ⋆ δ,max ≥ C > 0. Then lim inf δ→0 T ⋆ δ,max ≥ C > 0. Hence T 0 > T max which gives the contradiction. 2

6 Existence of solutions for the system (P)-(IC)

In this section, we will prove that the system (P)-(IC) admits solutions ρ ± in the distributional sense. They are the limits when ε → 0 of the solution ρ ±,ε given in Theorem 5.10. To do this, we will justify the passage to the limit as ε tends to 0 in the system (P ε )-(IC ε ), using some compactness arguments.

Preliminary results

Before proving the main theorem, let us recall some well known results.

Lemma 6.1 (Trudinger compact embedding)

The following injection (see N. S. Trudinger [START_REF] Trudinger | On imbeddings into Orlicz spaces and some applications[END_REF]):

W 1,2 (T 2 ) ֒→ EXP β (T 2 ), is compact, for all 1 ≤ β < 2.
For the proof of this lemma see also R. A.

ρ n L q ((0,T );B) + ρ n L 1 ((0,T );X) + ∂ρ n ∂t L 1 ((0,T );Y ) ≤ C,
where q > 1 and C is a constant independent of n, then (ρ n ) n is relatively compact in L p ((0, T ); B) for all 1 ≤ p < q.

For the proof, see J. Simon [START_REF] Simon | Compact sets in the space L p (0, T ; B)[END_REF]Th 6,Page 86].

In order to show the global existence of system (P) in Subsection 6.2, we will apply this lemma in the particular cases where B = EXP β (T 2 ), X = W 1,2 (T 2 ) and Y = W -1,2 (T 2 ), for 1 < β < 2.

Lemma 6.3 (Weak star topology in L log L) Let E exp (T 2 ) be the closure in EXP (T 2 ) of the space of functions bounded on T 2 . Then E exp (T 2 ) is a separable Banach space which verifies, i) L log L(T 2 ) is the dual space of E exp (T 2 ).

ii) EXP β (T 2 ) ֒→ E exp (T 2 ) ֒→ EXP (T 2 ) for all β > 1.

For the proof, see R. A. Adams [START_REF] Adams | Sobolev spaces[END_REF]Th 8.16,8.18,8.20].

Proof of Theorem 1.4

Let us fix any T > 0. For any ε ∈ (0, 1], we are considering the solution ρ ±,ε of (P ε )-(IC ε ) given in Theorem 5.10 on R 2 × (0, T ). First, by Lemma 5.6 we know that, the periodic part of the solutions, denoted by ρ ±,ε,per are ε-uniformly bounded in L 2 (T 2 × (0, T )). Hence, as ε goes to zero, we can extract a subsequence still denoted by ρ ±,ε,per , that converges weakly in L 2 (T 2 × (0, T )) to some limit ρ ±,per . Then we want to prove that ρ ± = ρ ±,per + Lx 1 are solutions of the system (P)-(IC). Indeed, since the passage to the limit in the linear term is trivial in D ′ (T 2 × (0, T )), it suffices to pass to the limit in the non-linear term

(R 2 1 R 2 2 ρ ε ) ∂ρ ±,ε ∂x 1 . (6.31)
Step 1 (compactness of (R 2 1 R 2 2 ρ ε )): Now notice that: • From Corollary 5.7 we know that the term (R 2 1 R 2 2 ρ ε ) is ε-uniformly bounded in L 2 ((0, T ); W 1,2 (T 2 )). Then it is in particular ε-uniformly bounded in L 1 ((0, T ); W 1,2 (T 2 )).

• From the previous point and Lemma 6.1, we know that (R 2 1 R 2 2 ρ ε ) is also ε-uniformly bounded in L 2 ((0, T ); EXP β (T 2 )) for all 1 ≤ β < 2.

• From Lemma 5.8, the term R 2 1 R 2 2 ( ∂ρ ε ∂t ) is ε-uniformly bounded in L 2 ((0, T ); W -1,2 (T 2 ))

and then in L 1 ((0, T ); W -1,2 (T 2 )).

Collecting this, we get that there exists a constant C independent on ε such that ρε = R 2 1 R 2 2 ρ ε satisfies for some 1 < β < 2 ρε L 2 ((0,T );EXP β (T 2 )) + ρε L 1 ((0,T );W 1,2 (T 2 )) + ∂ ρε ∂t L 1 ((0,T );W -1,2 (T 2 )) ≤ C.

Then Lemma 6.2 joint to Lemma 6.1, with B = EXP β (T 2 ), X = W 1,2 (T 2 ) and Y = W -1,2 (T 2 ), shows the relative compactness of (R 2 1 R 2 2 ρ ε ) in L 1 ((0, T ); EXP β (T 2 )), and then using Lemma 6.3, we deduce the compactness in L 1 ((0, T ); E exp (T 2 )).

Step 2 (weak-⋆ convergence of ∂ρ ±,ε ∂x 1 ): By Lemma 5.4, we have that ∂ρ ±,ε ∂x 1 is εuniformly bounded in L ∞ ((0, T ); L log L(T 2 )) which is the dual of L 1 ((0, T ); E exp (T 2 )) by Lemma 6.3. Then, this term converges weakly-⋆ in L ∞ ((0, T ); L log L(T 2 )) toward ∂ρ ± ∂x 1 . That enables us to pass to the limit in the bilinear term (6.31) in the sense L 1 ((0, T ); E exp (T 2 )) -strong × L ∞ ((0, T ); L log L(T 2 )) -weak -⋆.

which shows that (R 2 1 R 2 2 ρ ε ) ∂ρ ±,ε ∂x 1 → (R 2 1 R 2 2 ρ) ∂ρ ± ∂x 1 in D ′ (T 2 × (0, T )).
In what precedes, we have shown that ρ ± are solutions of the system (P).

Step 3 (conclusion): Passing to the limit in the estimates of Lammata 5.4, 5.6, 5.8 and Corollary 5.7, we get in particular by Lemma 5.3, the entropy estimates (1.1) and (E1), (E2), (E4), (E5). At this stage we remark that, by Proposition 3.4 that

∂ρ ± ∂t = (R 2 1 R 2 2 ρ)
∂ρ ± ∂x 1 ∈ L 2 ((0, T ); L log Since the function ρ ±,per,ε (•, t) satisfy (H1), (H2), (H3), (H4) (see Theorem 5.10) by passing in the limit ε → 0, we can see that the limit function ρ ±,per (•, t) reserves the same assumptions (H1), (H2), (H3), (H4).

It remains to prove that ρ ± satisfies the initial conditions (IC). Indeed, from the estimates on ρ ±,ε,per given by Lemma 5.6 and ∂ρ ±,ε ∂t given by Lemma 5.8 (ii), we can prove easily, that ρ ±,ε,per (t) -ρ ±,ε,per 0

W -2,2 (T 2 ) ≤ C T t 1 2 .
where C T is constant independent of ε. Hence we can pass to the limit ε → 0, which this implies in particular that ρ ±,per (•, 0) = ρ ±,per 0 in D ′ (R 2 ). 2 Remark 6.4

In our proof, we have indirectly used a kind of compensated compactness technic for Hardy spaces. Nevertheless in our case, we do not have enough regularity to do so.

Figure 1 :

 1 Figure 1: Groma-Balogh 2D model.

3 2

 3 onto itself (see A. Zygmund [40, Vol I, Page 254, (2.6)]) and the Sobolev injection W 1, 3 2 (T 2 ) ֒→ L 4 (T 2

1 , 3 2

 13 loc (R 2 )), satisfying (H1) and (H2) a.e. t ∈ [0, T ⋆ ).

Lemma 5 . 8 (

 58 Duality estimate for the time derivative of the solution)

1 2 L 1 2

 21 (T 2 )), and then ρ ±,per ∈ C([0, T ); L log L(T 2 )), which proves (E3).

  2 ) is a special space of Zygmund spaces (see R. A. Adams[1, (13), Page 234], E. M. Stein [36, Page 43])

École Nationale des Ponts et Chaussées, CERMICS, 6 et 8 avenue Blaise Pascal, Cité Descartes Champs-sur-Marne, 77455 Marne-la-Vallée Cedex

2, France 2 Universit de Marne-la-Valle 5, boulevard Descartes Cit Descartes -Champs-sur-Marne 77454 Marne-la-Valle cedex 2

(T 2 ) , L, ε) > 0, where ρ ±,ε,per δ,max = ρ ±,ε δ,max -Lx 1 ,

Acknowledgements

The second author would like to thank Y. Meyer, F. Murat and L. Tartar for fruitful remarks that helped in the preparation of the paper, and H. Ibrahim for carefuly reading it. The authors also would like to thank the referee who helped to improve drastically the presentation of the paper. This work was partially supported by the contract JC 1025 "ACI, jeunes chercheuses et jeunes chercheurs" (2003)(2004)(2005)(2006)(2007), the program "PPF, programme pluri-formations mathmatiques financires et EDP", (2006-2010), Marne-la-Vallée University and cole Nationale des Ponts et Chausses, and by the project ANR MICA ("Mouvements d'interfaces, calcul et applications").

Proof of ii): The proof of (ii) is similar to that of (i). The only difference is that we integrate by part the viscosity term twice and use the estimate of Lemma 5.6. 2

Remark 5.9 (W -1,2 and W -2,2 estimate) Let W -1,2 (T 2 ) be the dual space of W 1,2 (T 2 ). By point (i) of the previous lemma, we deduce that there exists a constant C independent of ε, such that

However, the point (ii) controls the time derivative of the solution in L 2 ((0, T ); W -2,2 (T 2 )), where W -2,2 (T 2 ) is the dual space of W 2,2 (T 2 ). This control will allows us later to recover the initial conditions in the limit as ε goes to zero.

Theorem 5.10 (Global existence) For all T > 0, ε ∈ (0, 1] and for all initial data ρ ± 0 ∈ L 2 loc (R 2 ) satisfying (H1), (H2), (H3) and (H4), the system H2) and (H3) for all t ∈ (0, T ) and the estimates given in Lemmata 5.4, 5.6, 5.8 and Corollary 5.7.

Before going into the proof, we need the following lemma.

where

) . Proof of Lemma 5.11: If we denote ρ ε v = (ρ +,ε,per , ρ -,ε,per ) and ρ ε 0,v = (ρ +,ε,per 0 , ρ -,ε,per 0

), then we have shown that ρ ε v satisfies (4.11), using (4.14) with u = v = ρ ε v , we get,

.

We use now Lemma 4.4 (i) with r = 4, q = 24 5 , p = 1 to estimate the first term, and Lemma 4.4 (ii) with r = 3 2 , q = 8, p = 1 to estimate the second term. It gives for