Haytham Elghazel

Emmanuel Coquery

Khalid

Thomas Ranvier
email: thomas.ranvier@univ-lyon1.fr

Khalid Benabdeslem

Deep Multi-Source Supervised Domain Adaptation with Class Imbalance

Keywords: Deep Learning, Domain Adaptation, Multi-Source, Supervised, Imbalanced

Introduction

Learning from imbalanced data requires a specific learning approach in order to pay specific attention to the class distribution during the training phase. Deep Learning is notoriously hard when dealing with limited data. Indeed, if the data is too limited, it is impossible to properly train a deep model, and simpler Machine Learning approaches should be preferred. When independent limited but similar datasets are available, it becomes adequate and beneficial to use deep multi-source domain adaptation to improve predictions on the target domain [START_REF] Wilson | A Survey of Unsupervised Deep Domain Adaptation[END_REF][START_REF] Day | A survey on heterogeneous transfer learning[END_REF][START_REF] Pan | A Survey on Transfer Learning[END_REF][START_REF] Kouw | An introduction to domain adaptation and transfer learning[END_REF]. Exploiting knowledge from several source domains can help to minimize the negative impact of both limited data and class imbalance. In this paper, we propose a new interesting multi-source supervised domain adaptation approach, which we ultimately aim to apply on real-world limited and imbalanced medical tabular data.

In transfer learning, we aim to exploit knowledge from one or several source dataset(s) to improve learning performance on another target dataset. For transfer learning to be beneficial, the dataset(s) used as source(s) should be similar enough to the target dataset. A source that is not similar enough to the target will negatively impact learning performance, and should not be used in this context. We talk about domain adaptation when we aim to learn a single and common task by transferring knowledge from one or several source domain(s) to a target domain. A very well-researched area of domain adaptation is single-source domain adaptation [5? -7], that is, when we use only one source domain to transfer towards the target domain. A more complex and less researched area is multi-source domain adaptation [START_REF] Zhao | Adversarial Multiple Source Domain Adaptation[END_REF][START_REF] Zhu | Aligning Domain-specific Distribution and Classifier for Cross-domain Classification from Multiple Sources[END_REF][START_REF] Peng | Moment Matching for Multi-Source Domain Adaptation[END_REF][START_REF] Zuo | Attention-Based Multi-Source Domain Adaptation[END_REF], where we use several source domains to transfer as much knowledge as possible to the target. Domain adaptation can help largely improve prediction performance on the target domain by exploiting more knowledge from source domain(s) than available on the sole target domain. Domain adaptation is often used to make prediction possible on an entirely unlabeled target domain, that is, unsupervised domain adaptation. In our work, we are interested in a case where the target domain is labeled as any standard dataset, in this case we talk about supervised domain adaptation, which is a less researched domain adaptation area, despite being a common real-world occurrence.

In this paper, we propose a new original approach for multi-source domain adaptation in a supervised context, and demonstrate its performance on limited and imbalanced data. Namely, Weighted Multi-Source Supervised Domain Adaptation (WMSSDA). WMSSDA transfers knowledge from s source domains to a similar target domain. It learns a domain invariant latent space, regularized using both statistical and adversarial approaches, where shared knowledge across source domains is transferred to the target domain, and s source domain specific latent spaces, in which source specific knowledge is transferred. With such an architecture, our proposed approach WMSSDA is able to exploit both common knowledge across all domains and source specific knowledge that is useful for inference on the target domain. We compute source domain specific transfer contribution weights during training, those are applied during training to weight the importance of each source domain on learning, reducing as much as possible Negative Learning. We conduct extensive experiments to compare our approach with other baseline and state-of-the-art domain adaptation approaches in a data-limited and class imbalance context. We show that WMSSDA outperforms other state-of-the-art domain adaptation approaches on both image benchmark datasets and real-world medical tabular data. We perform an ablation study to validate the pertinence and positive impact of each component in our method.

The source code used to conduct our complete experiments is available at the following GitHub repository 1 .

In the rest of the paper, we first formally describe our domain adaptation learning scenario in section 2, we then present related works from transfer learning and learning on imbalanced data literature in section 3. Section 4 describes our proposed approach, we show our experimental results in section 5 and conclude with a summary of our contributions, results and future perspectives.

Learning Scenario

In this section, we formally describe the considered learning scenario. 1: Main notations used in this paper.

Notations and Preliminaries

Table 1 summarizes the used notations in this section and the rest of the paper. We introduce definitions and concepts needed for the following sections, our notations are inspired by the work of [START_REF] Cortes | Adaptation Based on Generalized Discrepancy[END_REF], we took liberties to adapt them to our multi-source supervised domain adaptation context.

Let X ∈ R d denote an input feature space, with d the number of features, and Y = {1, . . . , c} a multi-class output label space, with c the total number of classes. We define a domain as a pair formed by a distribution over X and a labeling function mapping from X to Y. We note D = (P (X D ), f D ) the domain D, with P (X D ) the marginal distribution of D over X , f D : X → Y is the labeling function mapping from feature to label space, X D is the data sample Deep Multi-Source Supervised Domain Adaptation with Class Imbalance defined as X D = {x i ∈ X } n D i=1 , with n D the number of instances in the data sample of the domain D.

In the scenario of multi-source supervised domain adaptation we consider, we are given s source domains, noted S i for i ∈ [1, s], that we want to exploit to improve classification over one target domain, noted T. A unique label space Y is shared across all domains, feature space of each domain can be different from other domains, we note X D the feature space of domain D. In our scenario, we have access to s labeled source domains where S i = {(x Si j , y Si j )} ni j=1 , with {x Si 1 , . . . , x Si ni } ∼ P (X Si ), and y Si j = f Si (x Si j ). We have access to a labeled target domain, similarly, T = {(x T j , y T j )} n T j=1 , where {x T 1 , . . . , x T n T } ∼ P (X T ) and y T j = f T (x T j ). We want to exploit knowledge from labeled source domains and the labeled target domain, to improve classification on an unknown and unusable part of T. As we consider a scenario in which the three types of shifts are present between domains, we consider that the covariate shift assumption does not hold, f S1 ̸ = . . . ̸ = f Ss ̸ = f T . Solving such a problem is only possible if the target domain is labeled, as it is necessary to rely on supervision to properly align domains with concept shifts. Therefore, we want our supervised domain adaptation model to learn to estimate the labeling function f T , while exploiting knowledge from the source domains through the learning of the different source labeling functions {f S1 , . . . , f Ss }.

Domain Shift in Our Adaptation Scenario

In their 2012 paper, to unify the terms and definitions used for the various domain shifts that appear in domain adaptation literature and provide consistent terminology, Moreno-Torres et al. [START_REF] Moreno-Torres | A unifying view on dataset shift in classification[END_REF] proposed formalization of three types of shifts: covariate shift, prior shift, and concept shift. More recently, Kouw and Loog [START_REF] Kouw | An introduction to domain adaptation and transfer learning[END_REF] reviewed those defined shifts and provided more precise and up-to-date definitions. To present our adaptation scenario in terms of domain shift we use those same terms and define them mathematically using our notations in the following. Figure 1 illustrates each of the presented domain shifts. Formally, covariate shift is when the data marginal distributions of two domains are different, while their conditional distributions are equal, P (X D1 ) ̸ = P (X D2 ) and P (Y D1 |X D1 ) = P (Y D2 |X D2 ). Intuitively, covariate shift exists between domains when the feature space of a domain is different from the one of another domain, that is X D1 ̸ = X D2 , or when there exists a domain specific form of sample selection bias [START_REF] Kouw | An introduction to domain adaptation and transfer learning[END_REF]. Under covariate shift, a classifier trained on one domain might struggle when applied on another domain. This is a domain shift that is present in almost all domain adaptation applications and is a point of interest of this paper.

There is a prior shift between two domains when the label marginal distributions of both domains are different, while their conditional distributions are equal, P (Y D1 ) ̸ = P (Y D2 ) and P (X D1 |Y D1 ) = P (X D2 |Y D2 ). This happens when the class balance is not the same in each domain. This is a common occurrence that can happen when data from similar domains are gathered differently [START_REF] Redko | Advances in Domain Adaptation Theory[END_REF], leading to different label marginal distributions between the domains. This type of domain shift appears less often in domain adaptation literature, we are specifically interested in prior shift, as it occurs in cases where domains are all differently imbalanced.

Formally, concept shift occurs when the conditional distributions of two domains are different, P (Y D1 |X D1 ) ̸ = P (Y D2 |X D2 ). This means that the decision boundary between classes is not the same from one domain to another, meaning that the causal relation between features and labels is semantically different from one domain to another. Concept shift might occur between two domains if classes are semantically inaccurate, which might lead to slightly differently labeled data between them. In an unsupervised context, concept shift would render transfer from source domains to the target domain impossible. As we are in a supervised domain adaptation setting, transferring knowledge is possible in our scenario.

In this paper, we are specifically interested in the case of multi-source domain adaptation with imbalanced data. We want to use multiple source domains with different feature spaces to improve classification performance over a similar target domain, with the particularity that each used domain is imbalanced in a different way. That is, each domain's data and label marginal distributions are different from other domains. This is a standard case of covariate shift between domains, with the addition of a prior shift, due to domain specific class imbalance. In our learning scenario we assume that the covariate shift assumption does not hold, as we cannot entirely rule out the hypothesis of a concept shift across domains.

Related Works

Domain Adaptation

Domain adaptation (DA) can be considered as a special case of transfer learning [START_REF] Wilson | A Survey of Unsupervised Deep Domain Adaptation[END_REF]. Transfer learning includes all approaches that are able to use knowledge from a source to improve inference on a target. Domain adaptation includes all approaches that aim to learn a single and common task by transferring knowledge from one or several source domain(s) to a target domain [3? ], most domain adaptation scenarios do not include target domain labels, that is, unsupervised DA. Less researched DA scenarios are semi-supervised and supervised DA where the target domain includes partially or entirely labeled target data, the goal of transferring knowledge from source(s) to target in those cases remains identical. As can be understood by the names, single-source DA is transferring knowledge from one source domain to a target domain, while multi-source DA transfers knowledge from multiple source domains to the target. Since about 2015, deep neural networks have been explored for domain adaptation, obtained results are significantly better compared to prior shallow transfer learning approaches, consequently, most recent DA approaches are based on deep models [1, 15? ].

Most domain adaptation approaches in the literature focus on learning a shared domain invariant latent space between domains to capture shared information between source(s) and target [START_REF] Wilson | A Survey of Unsupervised Deep Domain Adaptation[END_REF]. This leads to a latent space where instances of a domain are indistinguishable from instances of other domains, while classification relevant information is conserved, leading to better inference results on the target domain. There are two main ways of reaching this goal, relying on statistic distribution matching, or relying on an adversarial loss that encourages samples from different domains to lose all domain specific information.

Maximum Mean Discrepancy (MMD) is the most commonly used statistic to measure domain discrepancy to match source(s) and target learned distributions in DA literature. MMD has first been used and democratized by DAN [START_REF] Long | Learning transferable features with deep adaptation networks[END_REF], that uses MMD to minimize the distance between the learned representation of a source domain and the representation of the target domain. As an alternative, [START_REF] Peng | Moment Matching for Multi-Source Domain Adaptation[END_REF] proposed M 3 SDA, a multi-source approach that uses Moment matching Distance (MD) to match the distributions moments between domains. They demonstrate that MD is more pertinent than MMD in a multi-source adaptation context.

Other approaches from the DA literature use an adversarial approach to learn a domain invariant latent space. DANN [START_REF] Ganin | Domain-Adversarial Training of Neural Networks[END_REF] is the first DA approach that made use of a domain classifier trained adversarially on the learned latent representation of both target and source data. The domain classifier tries to discriminate the domain in the latent features, while the feature extractor learns to fool the domain classifier, successfully leading to a common invariant latent space between domains. MDAN [START_REF] Zhao | Adversarial Multiple Source Domain Adaptation[END_REF] can be considered as a multi-source version of DANN, with s sources domains, s domain classifiers are trained to discriminate between the i-th source domain and the target domain, leading to an invariant latent space between each source domain and the target domain. Adversarial approaches are notorious for reaching better results than statistic distribution matching in Domain Adaptation [START_REF] Zhu | Deep Subdomain Adaptation Network for Image Classification[END_REF].

We believe that learning a shared domain invariant latent space for multisource domain adaptation is limited, and that learning pairwise invariant latent spaces between the target domain and each source domain allows for the capture and transfer of more useful information between sources and target. When learning a domain invariant latent space between two domains, intuitively, the only information that is captured within the shared representation is the common information. Therefore, when building a shared representation across multiple domains, only the common information across all domains is captured, which becomes a limiting factor as the number of domains and the dissimilarities between them increases. This is why we believe that learning several latent spaces in a pairwise manner between target and sources is pertinent in order to capture and transfer as much relevant information as possible. For this reason, in our proposed multi-source domain adaptation approach, we used an architecture where a shared domain invariant latent space is learned across all domains in one branch, while s source specific latent spaces are learned between the target domain and each of the s source domains in another branch.

There exist two multi-source domain adaptation methods in the literature that also rely on learning both a shared domain invariant latent space while also learning pairwise latent spaces between sources and target domain: ML-MSDA [START_REF] Li | Mutual Learning Network for Multi-Source Domain Adaptation[END_REF], and MLAN [START_REF] Xu | Mutual Learning of Joint and Separate Domain Alignments for Multi-Source Domain Adaptation[END_REF]. Unlike us, their methods have been proposed and applied in a unsupervised domain adaptation context, which differs from supervised domain adaptation. Mutual Learning Network for Multiple-Source Domain Adaptation (ML-MSDA) [START_REF] Li | Mutual Learning Network for Multi-Source Domain Adaptation[END_REF] is composed of two branches. The first branch learns a shared invariant latent space across all domains, while the second learns pairwise latent spaces between the target and each source domain. By jointly learning those multiple latent representations, they obtain better experimental results than all previously presented multi-source domain adaptation approaches. They rely on adversarial learning to ensure the domain invariance of the learned latent spaces. Similarly, [START_REF] Xu | Mutual Learning of Joint and Separate Domain Alignments for Multi-Source Domain Adaptation[END_REF] extended the work of [START_REF] Li | Mutual Learning Network for Multi-Source Domain Adaptation[END_REF] by proposing a Mutual Learning based Alignment Network (MLAN). The model architecture is identical to ML-MSDA, but is trained slightly differently, through the proposed mutual learning module. The module relies on pseudo-labeling of target instances to maximize target prediction performance. With MLAN, [START_REF] Xu | Mutual Learning of Joint and Separate Domain Alignments for Multi-Source Domain Adaptation[END_REF] currently obtains state-of-the-art results compared to other multi-source and single-source models of the domain adaptation literature.

In our work, we propose a multi-source supervised domain adaptation method with a two-branch architecture, similarly to ML-MSDA and MLAN. We exploit the fact that we are working in a supervised domain adaptation context to train the model on labeled instances from all sources and target domains.

Negative Transfer

The goal of domain adaptation is to exploit knowledge from one or several source domain(s) to improve prediction quality on a target domain. But a common issue with domain adaptation is negative transfer [START_REF] Day | A survey on heterogeneous transfer learning[END_REF][START_REF] Pan | A Survey on Transfer Learning[END_REF][START_REF] Zhang | A Survey on Negative Transfer[END_REF]. Negative transfer occurs in domain adaptation when transferring knowledge from a source domain to a target domain harms the learning performance on the target. Consequently, instead of improving the inference model performance, negative transfer leads to a decrease in prediction performance on the target domain. One of the most common reasons for negative transfer is a too large dissimilarity between source and target domains [START_REF] Zhang | A Survey on Negative Transfer[END_REF]. This risk is multiplied in the multi-source domain adaptation field, as multiple sources can contribute to negative transfer. Negative transfer is an important issue in the transfer learning field, limiting negative transfer is an important matter, which should be addressed when designing new domain adaptation approaches.

In their paper, [START_REF] Zuo | Attention-Based Multi-Source Domain Adaptation[END_REF] proposed the ABMSDA method, which avoids negative transfer by weighting each source domain depending on its contribution to the adaptation process. Their model architecture is composed of a domain classifier, a common feature extractor regularized using WMD (a modified version of Moment Distance), and a shared task-specific classifier. They train the domain classifier, separately and prior from the rest of the model, to predict the probability that target images belong to each source domain. They use the probability output of the domain classifier as a metric that indicates the statistical similarity between the target domain and each source domains, with the intuition that source domains that are most similar to the target domain should be attributed a higher weight. They apply those weights to source instances when computing WMD. They also apply those weights when combining the probability outputs of the classifier during training, leading to a classifier less prone to negative transfer. This is a way of avoiding negative transfer during multi-source domain adaptation.

In our proposed approach, we compute transfer contribution weights with a discrepancy measure, directly during training. Those weights are associated to each source domain, based on supervised target domain results, and applied to scale the importance of source instances during training. The goal being to increase the importance of instances from relevant source domains, while decreasing the importance of instances from less related source domains.

Dealing With Class Imbalance

Class imbalance occurs when the labels we aim to predict are not uniformly distributed over the dataset, resulting in certain classes having a much higher, or lower, number of instances compared to others. When trained on an imbalanced dataset, standard Machine Learning approaches will lead to poorer results than when trained with a similar balanced dataset [START_REF] He | Learning from Imbalanced Data[END_REF]. When dealing with class imbalance it is primordial to use approaches that help improve learning performance. Nowadays, two main ways are used to deal with imbalanced data in the literature, sampling approaches, and cost-sensitive approaches [START_REF] Wang | Training deep neural networks on imbalanced data sets[END_REF].

Sampling approaches aim to artificially adjust the class distribution, by either removing instances the majority class(es), and/or adding more instances from the minority class(es). Under-sampling aims to reduce the number of majority class instances to achieve a more balanced class distribution. The simplest under-sampling approach that can be used to artificially re-balance a dataset is random under-sampling [START_REF] Wang | Training deep neural networks on imbalanced data sets[END_REF][START_REF] Fernández | Learning from Imbalanced Data Sets[END_REF][START_REF] Krawczyk | Learning from imbalanced data: open challenges and future directions[END_REF]. With random under-sampling instances from the majority class(es) are randomly removed until the desired class distribution is achieved. This simple approach can lead to improved inference results but can also result in the loss of important information from the majority class(es), and so, reduce the generalization ability of the model as it misses crucial information. To assess this drawback more researched and advanced under-sampling approaches have been proposed, such as: Condensed Nearest Neighbor [START_REF] Hart | The condensed nearest neighbor rule (Corresp.)[END_REF], or Tomek Links [START_REF] Tomek | Two Modifications of CNN[END_REF]. Overall, under-sampling leads to improved inference results but might also lead to losing important information for inference if the amount of instances is too low to afford removing instance from the majority class(es). On the other hand, over-sampling involves increasing the representation of minority classes by duplicating or generating synthetic examples. The simplest over-sampling approach that can be used to artificially re-balance a dataset is random over-sampling [START_REF] Wang | Training deep neural networks on imbalanced data sets[END_REF][START_REF] Fernández | Learning from Imbalanced Data Sets[END_REF][START_REF] Krawczyk | Learning from imbalanced data: open challenges and future directions[END_REF]. Random over-sampling duplicates randomly selected instances from the minority class(es) until the desired class distribution is achieved. As with random under-sampling, over-sampling leads to improved inference results but highly increases the risk of overfitting, leading to biased inference models that lack in generalization capacity. The Synthetic Minority Oversampling Technique (SMOTE) method is the most popular and most widely used advanced oversampling method, it has been proposed in [START_REF] Chawla | SMOTE: Synthetic Minority Over-sampling Technique[END_REF] and it is known to largely improve inference results on imbalanced data [START_REF] Wang | Training deep neural networks on imbalanced data sets[END_REF][START_REF] Fernández | Learning from Imbalanced Data Sets[END_REF]. SMOTE works by creating synthetic examples of the minority class(es) by interpolating between existing instances of the minority class(es). SMOTE is known to largely improve prediction results on imbalanced data. A known drawback of SMOTE, and over-sampling in general, is the risk of leading to overfitting, and the risk of generating synthetic examples that are unrealistic or less informative, leading to a limited improvement in prediction quality.

Removing and generating synthetic data leads to improved learning results, but with important drawbacks. Removing data is often non viable in real-world scenarios with limited data, and generating synthetic data comes with the disadvantage of potentially generating implausible instances. Another approach that can be used in Machine Learning to deal with class imbalance is costsensitive learning, where modifications are made to the algorithm, and/or to the training process, to take account of imbalance and improve prediction results. It has been shown in several empirical studies that cost-sensitive learning leads to superior inference results than sampling approaches on imbalanced data [START_REF] He | Learning from Imbalanced Data[END_REF][START_REF] Kukar | Cost-Sensitive Learning with Neural Networks[END_REF]. Therefore, cost-sensitive techniques are usually a better solution than sampling methods. In neural network training, the most popular way of implementing cost-sensitive learning is to adapt the error function to take into account the class cost of each training instance during the learning phase, as defined in [START_REF] Kukar | Cost-Sensitive Learning with Neural Networks[END_REF]. The error function is corrected by introducing the cost factor of the class as a weight that is applied during training. Class weights applied to the loss function are commonly computed as the inverse of the class distribution of training data, though other weighting techniques can be used. This approach obtained by far the best results in [START_REF] Kukar | Cost-Sensitive Learning with Neural Networks[END_REF], it is still a very commonly used approach as it is very easy to implement and use, and it reaches better results than most other existing approaches to handle imbalanced data. This is why, in this paper, we use a cost-sensitive approach to account for the class imbalance between domains to maximize prediction quality.

Proposed Approach

In this section, we describe in details our proposed approach: Weighted Multi-Source Supervised Domain Adaptation (WMSSDA).

The idea behind WMSSDA is to create a common domain invariant latent space and s source domain specific latent spaces, perform classification on each latent space, and draw final weighted prediction results in an ensemble manner. The common domain invariant latent space is trained on a label classification task on both source and target batches. Two regularization techniques are employed to minimize domain-specific information into the common latent space. First, by using a Moment Distance (MD) regularization to match the distributions of source and target batches, and secondly, through the adversarial training of a domain discriminator. For the s source domain specific latent spaces the ideology is the opposite, we want each latent space to retain as much source specific information as possible. This is possible as in a supervised domain adaptation context, the target domain is labeled, it is therefore possible to train each classifier on a supervised classification task on both source and target instances, leading to a pairwise fine-tuning between each source domain and the target domain. Those specific latent representations are regularized using a collaboratively trained specific domain classifier, while performing label classification, leading to latent spaces retaining as much domain specific information as possible. Using multi-source domain adaptation naturally helps dealing with the class imbalance problem, as it increases the total amount of available data for training. We further deal with the class imbalance in each domain using a cost-sensitive learning approach, by scaling the loss with a class weight, computed as the inverse of the class distribution of training data. We use the output of the Moment Distance measure between the target domain and each source domain to determine transfer contribution weights, higher weights are attributed to source domains with closer latent distributions compared to the target domain latent representation, and inversely. Those transfer contribution weights are then applied to weight the classification loss on source instances, giving less importance to less relevant source domains in training. Ultimately, the final predictions for the target domain are obtained by passing a target batch through all trained modules. The average of the outputs from all specific classifiers is computed and combined with the outputs of the common classifier to obtain the final probabilities for the target classes. Figure 2 shows a simple representation of the architecture of the approach.

When training the approach, we iterate through one batch from each source domain for each target domain batch. Batches are processed as a pair between Fig. 2: Architecture of our approach WMSSDA, common modules appear in purple, i-th source domain specific modules appear in the same color as the i-th associated source domain. Lines of color symbolize the data flow, each color corresponding to a batch of the corresponding domain. Dashed red lines symbolize the computation and application of transfer contribution weights, computed from the MD measure and applied as scaling in the classification loss terms.

the b-th target batch and the b-th batch of the i-th source domain, which we will refer to as a pair of batches in the following. Each pair of batches is fed through a common feature extractor used to extract low-level features on all domains alike. The approach architecture is then divided in two main parts: common modules and source domain specific modules.

• Common modules. They are fed pairs of batches between the target domain and any source domain. The first component of this part is a common neck, comparable to the previous feature extractor, which extracts higher-level features on all domains. We call the output of the previous component the common domain invariant latent space, which we note Z Tcom and Z Scom , for the target and source batch latent representations respectively. To ensure that this latent space is domain invariant, we use both statistical distribution matching and adversarial domain discrimination. We regularize the latent representations by minimizing the standard Moment Distance, such as defined in [START_REF] Peng | Moment Matching for Multi-Source Domain Adaptation[END_REF][START_REF] Zuo | Attention-Based Multi-Source Domain Adaptation[END_REF], between target and source representation,

L M D = k i=1 E(Z i Scom ) -E(Z i Tcom ) F .
We use the output of the MD measure between the target domain batch and batches of all source domains to compute transfer contribution weights. We note those weights α ∈ R s , with α i the weight associated to the i-th source domain. Those transfer contribution weights are used to scale the classification loss of each source domains, giving more weight to close and related source domains and less weight to less useful domains. They are Deep Multi-Source Supervised Domain Adaptation with Class Imbalance computed as:

α i = s + 1 s - e Di-max(D)
e Di-max(D) with D = {MD(Z Sicom , Z Tcom )} s i=1 the set of MD measures between the target domain batch and each source domain batch. The weights are computed such as s i=1 α i = s, which ensures that source instances are given as much importance as target instances overall, while a different weight is given to each source domain. We associate the MD regularization with the training of an adversarial common domain classifier applied to the latent space. This common domain classifier learns to discriminate the domain from which originates each sample in the pair of batches, while the common feature extractor and neck try to fool the discriminator, leading to a domain invariant representation in this latent space. Parameters of the common feature extractor and neck are tuned to maximize the loss of the domain discriminator, while the discriminator parameters are tuned to minimize their own loss. As it is hard to optimize minimax problems using gradient descent algorithms, a common practice in domain adaptation research is to use a gradient reversal operation, and apply it between the latent representation and the domain classifier, such as defined in [START_REF] Ganin | Domain-Adversarial Training of Neural Networks[END_REF]. Which solves the adversarial problem by minimizing a single loss. We note L adv d the loss of the common domain discriminator applied on the gradient reversed common latent representation of the pair of batches. We found that using both statistical and adversarial strategies simultaneously led to better empirical results, which suggests that using both approaches cooperatively leads to a better domain invariant representation. Finally, the latent representation is fed through a task-specific classifier that discriminates samples on their class label.

• Source domain specific modules. They are only fed pairs of batches between the target domain and their associated source domain, that is, a pair of batches between the i-th source domain and the target domain is fed to the i-th specific module. Each specific module is composed of a specific neck, which extracts higher-level features, a specific task-specific classifier that discriminates samples on their class label, and a specific domain classifier that discriminates samples on the domain they originate from. In the opposite way to the above, this specific domain classifier is trained collaboratively, which pushes each specific latent space to retain as much domain specific information as possible. The source and target batches are treated differently:

-The source batch from domain S i is fed through the i-th specific neck, the resulting latent representation is noted Z Si . The i-th domain classifier is fed Z Si and is trained to recognize that those samples originate from domain S i . The i-th task-specific classifier is fed Z Si and is trained to discriminate the class of each sample.

-The target batch is fed through all specific necks, and the resulting latent representations are noted {Z T1 , . . . , Z Ts }. The i-th domain classifier is fed Z Ti and is trained to recognize that those samples originate from domain T. Each task-specific classifier is fed Z Ti , their outputs are noted { ŶT1 , . . . , ŶTs }. Only the i-th classifier is trained to discriminate the class of each target sample to avoid overfitting.

We note L sp d the loss of the i-th specific domain discriminator applied on the i-th latent representations of the pair of batches.

We note L y the global task-specific classification loss, which is the average between common and specific classification probabilities on the pair of batches. To take account of class imbalance during training we compute class weights, noted W , that are applied in a cost-sensitive learning manner during taskspecific classification loss computation, they are computed as the inverse of the class distribution observed in the training data of each domain:

W D = 1/P (Y D ).
Finally, the loss we minimize using gradient descent is computed for each pair of batches in the following way:

L = L y + λ 1 L adv d + λ 2 L sp d + λ 3 L M D
Where λ 1 , λ 2 , λ 3 are hyper-parameters that are defined to balance each component of the final loss.

Final prediction results are obtained by feeding target instances through all task-specific classifiers, the average of source specific classifiers outputs is computed and averaged with the output of the common classifier, leading to final class probabilities.

Pseudo-code 1 describes, in a more formal way, the training steps of the entire approach. In this pseudo-code, model includes a common feature extractor model, common and specific necks, and common and specific classifiers, com clf d is the common domain classifier and sp clf d is the set of i specific domain classifiers. We note adv(•) the gradient reversal operation, classification outputs are noted { ŶDcom , ŶD1 , . . . , ŶDs } and correspond to the label probabilities obtained on the common latent representation and all specific representations of the domain D respectively. Latent spaces are referred to with the same notations as above, Z Dcom for the common latent representation and Z Di for the i-th specific latent representation of domain D. Parameter E is the number of epochs to perform, s is the number of source domains and λ is the set of hyper-parameters used to balance all loss components together. Finally, W are class weights specific to each domain computed as 1/P (Y D ), W i for the i-th source domain and W T for the target domain.

Our proposal combines the advantages of a domain invariant latent space with a set of domain specific representations, with transfer contribution weights applied to minimize negative transfer during training, leading to a supervised domain adaptation approach able to transfer knowledge from multiple source domains to a target domain. Similarly to DANN [START_REF] Ganin | Domain-Adversarial Training of Neural Networks[END_REF], our proposed WMSSDA learns a domain invariant latent space on which is performed classification and

L y ← (L Ty + α i × L Sy )/(1 + α i ; L = L y + λ 1 L adv d + λ 2 L sp d + λ 3 L M D ;
Train model, com clf d and sp clf d by back-propagating L; end end similarly to MFSAN [START_REF] Zhu | Aligning Domain-specific Distribution and Classifier for Cross-domain Classification from Multiple Sources[END_REF], WMSSDA builds s domain specific latent spaces. The main difference between the second part of WMSSDA and MFSAN is that MFSAN aims to match source and target distributions between all specific latent spaces, which we find counterproductive as it means that all specific latent spaces are pushed toward an identical latent space. We choose to combine both the adversarial and statistical approaches to regularize the common latent space, since we found better results in this way, and could use the MD results to compute contribution weights. As in the two recent Unsupervised multi-source domain adaptation approaches, ML-MSDA [START_REF] Li | Mutual Learning Network for Multi-Source Domain Adaptation[END_REF] and MLAN [START_REF] Xu | Mutual Learning of Joint and Separate Domain Alignments for Multi-Source Domain Adaptation[END_REF], we chose to organize our architecture in two branches, one to build a common latent space, and the other to build s domain specific latent spaces. We do so as we believe that only learning a shared latent space across all domains in a multi-source domain adaptation context prevents part of useful knowledge to be transferred from source to target domain. We also noted superior experimental results by using two branches instead of one. We choose to use the Moment Distance to match both target and source common distributions, since [START_REF] Peng | Moment Matching for Multi-Source Domain Adaptation[END_REF] demonstrated that Moment Distance is better suited as a statistical distribution matching approach for multi-source domain adaptation than the most common Maximum Mean Discrepancy. We compute a scale variable γ as described in [START_REF] Zhu | Deep Subdomain Adaptation Network for Image Classification[END_REF], to scale the impact of the MD regularization throughout the training. Its value starts at a 0 and increases logarithmically towards a value of 1 over the total number of epochs, giving more importance to this regularization at the middle and end of the training phase. We apply the computed transfer contribution weights from the MD output on the classification loss of source instances, minimizing as much as possible Negative Learning during training. We apply class weights to scale the computed classification loss to take account of class imbalance during training in a cost-sensitive learning manner. Using a domain adaptation approach also naturally helps in dealing with limited data since source domain knowledge helps improving the overall learning performance. Our proposed WMSSDA approach allows the exploitation of both common knowledge and source domain specific information that is useful for target domain classification.

As we consider, in our learning scenario, that the covariate shift assumption might not hold, meaning that there might be concept shift between domains, we implement a second version of our method. In their paper, [? ] proposed an interesting and easy-to-implement way to allow domain adaptation approaches to handle concept shift by successfully aligning conditional distribution between two domains P (Y D1 |X D1 ) = P (Y D2 |X D2 ). Similarly to [START_REF] Zhu | Aligning Domain-specific Distribution and Classifier for Cross-domain Classification from Multiple Sources[END_REF], we implement a second version of our method in which we include this modification, we name this variation WMSSDA-β. We replace each task-specific classifier in WMSSDA-β with a pair of classifiers and follow the following three steps in our training:

1. We train our entire model as previously defined, where label probabilities are defined as the mean of each pair of classifier outputs. 2. We then fix the feature extractor and necks and train the classifier pairs to maximize their discrepancy. The discrepancy between two classifiers C and C ′ for an instance x is defined as |C(x) -C ′ (x)|. 3. Finally, we train the feature extractor and necks to minimize this same discrepancy with fixed classifiers. Deep Multi-Source Supervised Domain Adaptation with Class Imbalance

We repeat those steps until global convergence to simultaneously align both marginal and conditional distributions, leading to successful domain adaptation given our scenario. This β version of WMSSDA should be able to better handle concept shift than the standard version, leading to better inference results when in presence of concept shift.

Experimental Results

This section presents the experiments we led to evaluate and compare our proposed approach WMSSDA to other state-of-the-art domain adaptation approaches in a data-limited and class imbalance context.

Datasets

With our experiments we aim to show that our method performs well compared to other state-of-the-art approaches, on both popular benchmark domain adaptation datasets and a real-world medical domain adaptation dataset composed of mixed-type tabular data. We are interested in comparing state-of-theart domain adaptation approaches in the context of limited data and class imbalance.

The 5-Digits multi-domain dataset is widely used in domain adaptation studies [6-8, 10, 11], it is composed of five digits recognition datasets, with grayscale or color images of various sizes:

1. MNIST2 , the widely known handwritten digit recognition dataset. 2. MNIST-M3 , a more complex version of MNIST, created by combining MNIST images with randomly extracted patches of photos of the BSDS500 dataset as their background. 3. Street View House Numbers (SVHN) 4 , a Real-World image dataset of house numbers extracted from Google Street View images. 4. Synthetic Digits (SYN) 5 , synthetically generated images of digits with random backgrounds. 5. USPS6 , a handritten digit recognition dataset similar to MNIST.

The DomainNet dataset is a multi-domain dataset recently released with paper [START_REF] Peng | Moment Matching for Multi-Source Domain Adaptation[END_REF], and accessible at the following webpage 7 . It aims to provide a new difficult and more advanced benchmark dataset for domain adaptation approaches. It is composed of six domains of color images of various sizes, with a total of 345 common categories across all domains:

1. Clipart, a collection of clipart images. 2. Infograph, infographic images of specific objects. 3. Painting, artistic depictions of objects in the form of paintings.

4. Quickdraw, drawings of the worldwide players of the game "Quick Draw!". 5. Real, photos and real-world images. 6. Sketch, sketches of specific objects.

The 5-Digits and DomainNet datasets are domain adaptation benchmark image datasets that are known for their covariate shift between domains. We perform experiments on those two datasets to demonstrate the capacity of our approach to perform well on known experimental domain adaptation datasets. As we are interested in an experimental scenario with limited data in each domain and class imbalance, we preprocessed the datasets to follow our setting of interest. We randomly selected subsets of each dataset domain to create both a limited amount of data and a class imbalance. After this selection step, the class representation in the 5-Digits dataset spans from 4.22% (10 samples) for the least represented class to 21.09% (500 samples) for the most represented class in each domain. For the DomainNet dataset, the class representation spans from 1.1% (10 samples) for the least represented class to 13.33% (120 samples) for the most represented class in each domain, out of 17 classes, leading to a total of 900 instances in each domain.

The Covid dataset was provided by the Mexican government and is composed of health data about patients that suffer from, or have symptoms that could be related to, Covid19. It is downloadable from the following Kaggle repository 8 , it originally contains 1, 048, 576 samples from patients suffering from Covid19, with 20 tabular mixed-type (continuous and categorical) features. The goal of this dataset is to predict the survival outcome of patients. The dataset is naturally imbalanced, odds of survival to Covid19 being, fortunately, higher than the odds of death. We used the categorical feature "Medical Unit" to split the original data into 5 domains depending on the type of medical institution that provided the care to the patient. We could not find more information about those kinds of medical institutions, apart from their ID in the dataset, we consider in the following that they correspond to different hospitals as we observe a covariate shift between the institutions. During our preprocessing, we selected 800 unique patients per domain to simulate a limited amount of training data, and we dropped two features containing too many missing values. With the subtraction of the feature used to split the data into separate domains, there remain 17 features after preprocessing.

Figure 3 shows the univariate marginal distributions of four features of the Covid dataset across the five domains. We observe different marginal distributions between all domains, with domain 4 the most different one, with drastically different Sex and Obesity distributions. This simple visualization of the distributions across the Covid dataset pairs of domains is enough to conclude that there is indeed a covariate shift between the five domains.

Table 2 shows the class representation in each domain of the Covid dataset, revealing a prior shift between the domains of the dataset. Indeed, we observe that the label distribution is imbalanced in each domain and the representation 8 https://www.kaggle.com/datasets/meirnizri/covid19-dataset Fig. 3: We observe a covariate shift between the five domains, as univariate marginal distributions are different across pairs of features, P (X D1 ) ̸ = P (X D2 ) ̸ = . . . ̸ = P (X D5 ).

) ̸ = P (Y D2 ) ̸ = . . . ̸ = P (Y D5 ).
With the preprocessing described above for each dataset, we observe both covariate and prior shifts in all datasets, with limited and imbalanced data in each domain. All datasets might suffer from concept shift, in precaution, the β version of our approach is designed to better handle concept shift.

Compared Approaches

We compared our proposed WMSSDA to four single-source and four multi-source domain adaptation state-of-the-art approaches. Most of those approaches are initially unsupervised domain adaptation approaches, that is, approaches that do not use labels from the target domain. We modified those approaches to allow them to use target domain labels in their training phase for a fair comparison. The modification for each method is usually as simple as adding the supervised classification task on the target domain into the loss term of each approach. This phase of adapting unsupervised domain adaptation approaches for the supervised domain adaptation context is crucial as it would not be fair to compare our approach, that is able to use target domain labels in its supervised learning, with unsupervised domain adaptation methods that are able to use them. This would lead to far better inference results for our approach compared to unsupervised ones and would not be representative of the real capacity of unsupervised approaches.

In our experiments, single-source approaches are evaluated using two settings, such as in [START_REF] Peng | Moment Matching for Multi-Source Domain Adaptation[END_REF]. In the single best setting, we evaluate the approach on all possible pairs of domains as source and target and select the best-obtained results for each target domain. In the source combine setting, we combine all source domains as one unique domain to obtain only one source domain. We used the following single-source domain adaptation approaches from the literature:

• DAN, Deep Adaptation Network [START_REF] Long | Learning transferable features with deep adaptation networks[END_REF] is among the first deep learning unsupervised domain adaptation approaches to have been proposed. DAN uses multi-kernel MMD to minimize the distribution divergence between features extracted on the source and target data, fed through common layers followed by domain specific classifiers. We based our implementation on the following PyTorch implementation 9 . • DANN, Domain-Adversarial Neural Network [START_REF] Ganin | Domain-Adversarial Training of Neural Networks[END_REF], a model that builds a domain invariant latent space, using an adversarial domain classifier, on which classification is performed. We based our implementation on this PyTorch implementation 10 . • MCD, Maximum Classifier Discrepancy [? ], an approach that trains a generator and a pair of classifiers by alternating between training the classifiers to maximize their discrepancy, and training the generator to minimize their discrepancy. We based our implementation on this PyTorch implementation 11 . • DSAN, Deep Subdomain Adaptation Network [START_REF] Zhu | Deep Subdomain Adaptation Network for Image Classification[END_REF], a similar approach to DAN that replaces the MK-MMD with a Local MMD that aligns the distributions of the relevant subdomains. We based our implementation on this PyTorch implementation 12 .

We used the following multi-source domain adaptation approaches from the literature:

• MDAN, Multi-source Domain Adversarial Network [START_REF] Zhao | Adversarial Multiple Source Domain Adaptation[END_REF], a similar approach to DANN that uses as many adversarial domain classifiers as source domains. We based our implementation on this PyTorch implementation 13 . • MFSAN, Multiple Feature Spaces Adaptation Network [START_REF] Zhu | Aligning Domain-specific Distribution and Classifier for Cross-domain Classification from Multiple Sources[END_REF], the architecture of this model is composed of a common feature extractor followed by source domain specific parallel layers blocks. Those layers are regularized using MMD and domain specific classifier outputs are aligned with an L1 operation. Classifiers outputs are combined in an ensemble way to obtain the final target prediction. We based our implementation on this PyTorch implementation14 . • M 3 SDA, Moment Matching for multi-source domain adaptation [START_REF] Peng | Moment Matching for Multi-Source Domain Adaptation[END_REF], the architecture is composed of a common feature extractor followed by source domain specific parallel classifiers. The output of the common feature extractor is regularized using a moment-matching distribution distance between source and target data. We based our implementation on this PyTorch implementation 15 . • ABMSDA, Attention-Based Multi-Source Domain Adaptation [START_REF] Zuo | Attention-Based Multi-Source Domain Adaptation[END_REF], the architecture of the model is comparable to that of M 3 SDA, the main difference is that ABMSDA uses a common classifier instead of domain specific ones. ABMSDA computes attention weights with a domain classifier that is fed raw data, the domain classifier outputs are used to derive weights that are applied to the moment matching regularization and to the training loss in an attempt to weight each source domain and avoid negative transfer.

We also used two simple baseline approaches to get reference results:

• NN, a simple neural network, in the first evaluation setting, the model is trained only on the target domain, in the second setting, the model is trained on all combined domains, sources, and target domains alike. • FT, a simple fine-tuning approach in which a neural network is pre-trained on source data and is then fine-tuned on target data.

Experimental Protocol

We performed three main experiments, one to evaluate our approach compared to other domain adaptation methods on popular benchmark datasets, a second one to evaluate WMSSDA on a real-world medical multi-source domain adaptation dataset, and finally, an ablation study to evaluate the impact and usefulness of each component of our method. We evaluate all approaches on a supervised domain adaptation classification task with limited data and class imbalance, all unsupervised domain adaptation methods have been slightly modified to handle a labeled target domain as described in the previous section. All our experimental results are compared using the three following classification metrics: the balanced Accuracy (bACC), the Area Under the Receiver Operating Characteristic Curve (AUC), and the F 1 -score. All datasets are split between a training set and a test set, all approaches are trained on the same training data and evaluated on the same test data. To obtain significant results we conduct each experiment 5 times and report the mean and standard deviation of each evaluation metric. To be as fair as possible in our comparisons, in each experiment, we define the model architecture for each approach as similarly as possible across them, while taking account of their architectural differences. For image datasets, our feature extractor is a convolutional network, and all other modules are fully connected networks, for the Covid tabular dataset all modules are fully-connected. In the same line of thought, identical hyper-parameters, tuned on the NN approach, are used across all compared approaches for each experiment. All approaches are given the same class weights, those are applied in a cost-sensitive learning manner during loss computation to better handle class imbalance. Those class weights are computed, for each domain, as the inverse of the class distribution observed in the training data: W D = 1/P (Y D ).

To better asses the obtained experimental results, we statistically compare WMSSDA results to each of the compared state-of-the-art approaches using ttests. The results of those statistical tests are used to determine if our approach performs significantly better, even, or worse than each other, based on a pvalues set to 0.05. The results of the t-tests are symbolized in result tables as either a bullet •, a circle •, or an equivalent symbol ≡. The bullet is used to signify that our method is significantly better than the method we compared it to, the circle signifies the opposite, and the equivalent means that there is no significant difference between WMSSDA and the compared method. In the following sections, we will refer as "significantly better" all results that have been evaluated using a t-test and that were classified as significantly better in regard to a p-value of 0.05, and "significantly worse" the opposite.

Comparative Study on Benchmark Datasets

This first experiment is a comparative study of several baseline and state-ofthe-art domain adaptation approaches to evaluate WMSSDA performance on benchmark domain adaptation datasets. We aim to show that our approach is able to compete against, and even outperform, other state-of-the-art domain adaptation approaches on well-known image benchmark multi-domain datasets in a context of limited data and class imbalance.

Table 3 reports our entire experimental results on the 5-Digits dataset, with visual indications of the results of the t-tests between our best-performing models and each compared approach in all settings. As can be seen in the table, our two models, WMSSDA and WMSSDA-β, perform largely better than other approaches in the vast majority of cases, obtaining the best or second-best result for almost all metrics and target domains. We note that apart from our approach, other state-of-the-art multi-source approaches do not reach particularly better results than single-source approaches, we hypothesize that this is a manifestation of negative transfer that multi-source approaches are not yet able to fully avoid. The statistical comparison of the results on 5-Digits shows that our best-performing approach, WMSSDA-β, obtains significantly better results in the vast majority of cases, with only a few settings in which our approach leads to equivalent results to those of other methods. If we consider average performance across all target domains, we can conclude that WMSSDA-β obtains the best prediction results overall, with significantly better results than any other compared approach. WMSSDA is: • significantly better, ≡ equivalent, • significantly worse, p-value: 0.05

Table 3: Comparative Study on the 5-Digits domain adaptation Benchmark Dataset, with Limited and Imbalanced Data. The best and second-best results for each metric and each target domain appear in bold and underlined respectively. Results are evaluated on a multi-class classification task using metrics: balanced Accuracy, AUC, and the F 1 -Score.

The fact that we drastically reduced the amount of available training data worsens classification results, but this does not affect the comparison potential of the results. We can see that our proposed WMSSDA-β obtains the best or second-best results in the majority of cases, while WMSSDA obtains overall good results but not better than other approaches. It is important to note that the overall best prediction results for the Quickdraw domain are reached by a simple neural network trained on the target domain only. This means that no domain adaptation method is currently able to avoid negative transfer enough to match those results, nor are they able to surpass them. This shows that negative transfer in domain adaptation is still an open-problem, and that it is crucial to find better ways of solving this issue. The second best results on the Quickdraw domain are reached by the Fine-Tuning approach, Results are evaluated on a multi-class classification task using metrics: balanced Accuracy, AUC, and the F 1 -Score.

followed by our WMSSDA-β approach, showing that our method is currently the best-performing multi-source domain adaptation approach to avoid negative transfer. The simple Fine-Tuning approach leads to significantly better average balanced Accuracy results than ours and almost all other methods on both single best and source combine settings. This probably shows that adding a short phase of pre-training to other state-of-the-art approaches would drastically improve the overall results of all methods. Another well-performing approach in this setting is MFSAN, which leads to significantly better balanced Accuracy than ours on average. Overall, our two versions of WMSSDA lead to competitive experimental results on this dataset, with WMSSDA-β leading to significantly better results than other approaches in the vast majority of cases. When considering the average over all target domains, WMSSDA obtains significantly better results than all other approaches on the three evaluation metrics except for Fine-Tuning and MFSAN on the balanced accuracy metric.

We can conclude from this experiment on benchmark datasets that our proposed approach is able to compete, and even surpass, other baseline and state-of-the-art domain adaptation approaches in our supervised multi-domain with limited and imbalanced data context.

Comparative Study on Real-World Tabular Medical Dataset

The second experiment is a comparative study between all tested domain adaptation approaches and our proposed WMSSDA, on the real-world mixed-type tabular medical Covid dataset. We aim to show that our approach can reach good results on mixed-type tabular data in addition to image data. We also aim to show that WMSSDA competes and outperforms other domain adaptation approaches in a real-world medical context with limited data and class imbalance. Table 5 reports our entire experimental results on the Covid dataset. Results show that both WMSSDA variations lead to the best or second-best results in most cases, and in all cases on average. Those experimental results show that our approach can perform very well on tabular data. In this particular setting, we note that it is our standard version of WMSSDA that performs the best, which is probably an indication that there is almost no concept shift in this dataset, unlike with the two benchmark image datasets, rendering the β version of the approach no better than the standard one. The results between the two versions of WMSSDA are close, with slightly better results for our standard version, thus, we performed the statistical tests evaluation based on the standard version of WMSSDA. The statistical comparison of the results shows that our approach WMSSDA leads to significantly better results than most other approaches, in the majority of cases. Our approach leads to significantly better results than any other state-of-the-art approach when considering the average performance over all target domains.

Ablation Study

Our method WMSSDA is composed of several important elements, in this section we perform an ablation study in order to evaluate the pertinence and usefulness of each component of WMSSDA. An ablation study is a type of experiment that is conducted to investigate the impact of removing, or disabling, specific components or features of an approach. To do so, we eliminate parts of our model and evaluate the results to understand their individual contributions to the overall performance and validate the pertinence of each component.

In this ablation study we compare five ablated versions of our WMSSDA approach with our complete method. Table 6 shows all WMSSDA versions compared in this study. In the column "Branches" it is indicated if the method contains both the common modules branch and the source domain specific modules branch, or only one of the two. Column "Regus" indicates if both Table 7: Ablation study results.

Table 7 shows our experimental results on the Covid dataset for this ablation study. We compare our method with only the common modules branch with method WMSSDA-A and only the source domain specific modules branch with method WMSSDA-B. In both cases, average results are similar, with slightly better results for WMSSDA-A, showing the importance of a shared latent space. Method WMSSDA-E contains both branches and obtains largely better results than WMSSDA-A and WMSSDA-B, showing the pertinence of an architecture combining both the common and specific branches to obtain the best possible results. We believe that the two branches architecture naturally decreases negative transfer as classifiers with higher confidence are given more importance in the ensemble pooling of results, which highly contributes to improving prediction quality. We also evaluate the pertinence of using both statistical and adversarial regularizations to learn a shared domain invariant latent space, with method WMSSDA-C using only the adversarial regularization, and method WMSSDA-D using only the statistical MD regularization. The results of WMSSDA-C and WMSSDA-D show that using a statistical only regularization leads to slightly better results than the adversarial one alone. This is contradictory with the actual consensus in the literature, that states that adversarial regularization is superior to statistical regularization for learning a common domain-invariant latent space. This can probably be explained by the fact that MD has been shown to be more pertinent and lead to better learning performance than MMD in a multi-source domain adaptation context in [START_REF] Peng | Moment Matching for Multi-Source Domain Adaptation[END_REF], making it slightly superior to adversarial alignment in this case. Method WMSSDA-E, which uses both MD and adversarial regularization for its shared latent space obtains better results than both WMSSDA-C and WMSSDA-D, showing that using both kind of regularizations allows to further align the latent space and lead to even better adaptation results. Finally, we observe that our complete WMSSDA approach leads to the best results overall, which seems to indicate that our transfer contribution weights are useful to limit negative transfer during training and have a positive effect on learning performance.

Discussion and Conclusion

In this paper, we proposed an innovative multi-source supervised domain adaptation approach, Weighted Multi-Source Supervised Domain Adaptation (WMSSDA). We evaluated and compared WMSSDA and compared our results to those obtained by other state-of-the-art approaches, on limited and imbalanced data, on both benchmark and real-world medical datasets. Our proposed approach is composed of a two branch architecture, learning both a shared domain invariant latent space and source domain specific latent spaces. The shared latent representation is learned and regularized both statistically and adversarially, the statistical regularization relies on a MD measure between source and target domains. The output of the MD regularization is used to compute transfer contribution weights that are applied to weight the impact of each source domain during training, limiting negative transfer. We show that our proposed WMSSDA outperforms most state-of-the-art approaches on both image benchmarks datasets and a real-world tabular medical dataset. We further analyze the relevance and importance of each component of our method by performing an ablation study, validating the overall architecture of our approach.

Overall, our experimental results seem to show that most multi-source domain adaptation approaches do not obtain significantly better results than single-source approaches in our experimental scenario. This seems to show that despite researchers efforts, negative transfer in multi-source domain adaptation is still an open critical problem that seems to limit the overall potential performance of state-of-the-art multi-source domain adaptation approaches. Best performing domain adaptation approaches are still not able to fully avoid negative transfer. In our proposal of a new multi-source domain adaptation approach, we tried to limit negative transfer through the computation of transfer contribution weights that are applied as a scaling of the impact of each source domain in the training of the entire model. Our experimental results and ablation study show that this element of our approach is relevant and improves overall results. But even with this component, our proposed approach is yet not able to fully avoid negative transfer. Future works in the domain adaptation field should focus on finding better ways to handle this important matter.

Declarations Funding

Fig. 1 :

 1 Fig. 1: Illustration of the three kinds of domain shifts.

  is different from one domain to the other, thus, P (Y D1 ) ̸ = P (Y D2 ) ̸ = . . . ̸ = P (Y D5 ).

  This research is supported by the European Union's Horizon 2020 research and innovation program under grant agreement No 875171, project QUAL-ITOP (Monitoring multidimensional aspects of QUAlity of Life after cancer

  Algorithm 1: WMSSDA Training Pseudo-Code. input: E, s, model, com clf d, sp clf d, W , λ for epoch ← 1 to E do X Ty = cross entropy(Y T , ŶT , W T ); // Feed source batch to model and compute source loss components X S , Y S ←extract batch(S i ); { ŶScom , ŶS1 , . . . , ŶSs } ← forward(model, X S ); ŶS ← ( ŶScom + ŶSi )/2;

T , Y T ← extract batch(T); for i ← 1 to s do // Feed target batch to model and compute target loss components { ŶTcom , ŶT1 , . . . , ŶTs } ← forward(model, X T ); ŶT ← ( ŶTcom + ŶTi )/2; L L Sy = cross entropy(Y S , ŶS , W i ); // Feed adversarially trained common domain classifier { ŶSadv d , ŶTadv d } ←forward(com clf d, adv({Z Scom , Z Tcom })); L Sadv d ← cross entropy( ŶSadv d , {i, . . . , i}); L Tadv d ← cross entropy( ŶTadv d , {0, . . . , 0}); L adv d ← (L Sadv d + L Tadv d )/2; // Feed i-th collaboratively trained specific domain classifier { ŶSsp d , ŶTsp d } ← forward(sp clf d i , {Z Sspi , Z Tspi }); L Ssp d ← cross entropy( ŶSsp d , {1, . . . , 1}); L Tsp d ← cross entropy( ŶTsp d , {0, . . . , 0}); L sp d ← (L Ssp d + L Tsp d )/2; // Compute MD regularization γ ← 2/(1 + exp(-10e/E))) -1; D i ← M D(Z Scom , Z Tcom ); L M D ← γ × D i ; α i ← ((s + 1)/s) -(e Di-max(D) / e Di-max(D) ); // Compute global loss and back-propagate

Table 2 :

 2 Covid label distribution per domain, showing that there is a prior shift across domains, P (Y D1

Table 4

 4 reports our entire experimental results on the DomainNet dataset. Prediction results are overall quite low, as this dataset is notoriously hard.

	Setting	Method	Metric MNIST	MNIST-M	SVHN	SYN	USPS	Avg
	Single Best Source Combine Multi Source	NN DAN DANN DSAN FT MCD NN DANN DAN DSAN FT MCD MDAN MFSAN M3SDA ABMSDA WMSSDA WMSSDA-β	bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1 bACC AUC F1	84.74 ± 0.29 .9915 ± .0004 83.85 ± 0.35 77.56 ± 1.54 .9832 ± .0005 75.10 ± 2.15 90.12 ± 0.51 .9967 ± .0002 89.43 ± 0.71 91.00 ± 0.25 .9971 ± .0001 90.73 ± 0.29 88.95 ± 0.56 .9958 ± .0002 88.54 ± 0.67 92.47 ± 1.00 .9968 ± .0005 92.25 ± 1.10 91.92 ± 1.00 .9971 ± .0005 91.57 ± 1.19 90.12 ± 1.62 .9945 ± .0008 89.94 ± 1.62 75.91 ± 1.34 .9847 ± .0017 72.56 ± 1.96 94.11 ± 0.54 .9978 ± .0003 94.02 ± 0.57 92.76 ± 0.47 .9977 ± .0001 92.62 ± 0.50 93.33 ± 0.91 .9950 ± .0016 93.33 ± 0.91 87.63 ± 0.98 .9946 ± .0005 87.29 ± 1.11 90.97 ± 0.46 .9970 ± .0003 90.67 ± 0.55 92.38 ± 1.41 .9963 ± .0009 92.20 ± 1.57 93.53 ± 0.50 .9977 ± .0003 93.37 ± 0.58 95.30 ± 0.21 .9988 ± .0001 95.25 ± 0.21 95.21 ± 0.53 .9989 ± .0001 95.14 ± 0.56	• 63.08 ± 1.07 • .9353 ± .0046 • 58.98 ± 1.25 • 53.95 ± 2.10 • .8963 ± .0077 • 48.61 ± 2.61 • 69.62 ± 0.85 • .9622 ± .0030 • 68.38 ± 1.14 • 70.42 ± 1.78 • .9549 ± .0052 • 69.44 ± 2.05 • 68.11 ± 0.98 • .9568 ± .0029 • 65.58 ± 0.97 • 74.09 ± 2.43 • .9666 ± .0084 • 73.32 ± 2.27 • 68.66 ± 1.78 • .9572 ± .0029 • 67.77 ± 1.52 • 66.76 ± 1.68 • .9471 ± .0038 • 66.22 ± 1.39 • 53.67 ± 1.02 • .9018 ± .0056 • 47.78 ± 1.02 • 72.40 ± 0.48 • .9608 ± .0017 • 71.88 ± 0.24 • 70.71 ± 0.55 • .9627 ± .0013 • 69.38 ± 0.70 • 64.97 ± 3.06 • .9067 ± .0193 • 66.18 ± 2.56 • 53.44 ± 1.66 • .9194 ± .0036 • 49.77 ± 1.59 • 70.75 ± 1.19 • .9607 ± .0055 • 69.45 ± 1.30 • 65.45 ± 1.19 • .9421 ± .0039 • 64.67 ± 1.46 • 67.01 ± 1.84 • .9471 ± .0041 • 66.78 ± 2.09 75.05 ± 1.06 .9737 ± .0017 74.62 ± 1.17 77.58 ± 1.75 .9790 ± .0025 76.68 ± 2.05	• 46.36 ± 1.07 • .8578 ± .0081 • 31.47 ± 0.81 • 41.12 ± 0.14 • .8196 ± .0036 • 24.88 ± 0.39 • 69.37 ± 0.53 • .9479 ± .0013 • 68.26 ± 0.63 • 67.57 ± 1.03 • .9394 ± .0022 • 67.52 ± 1.04 • 59.32 ± 1.97 • .9274 ± .0049 • 51.94 ± 2.35 • 59.94 ± 4.38 • .8972 ± .0178 ≡ 58.00 ± 4.22 • 62.21 ± 1.05 • .9354 ± .0030 • 61.04 ± 1.99 • 54.77 ± 8.47 • .9078 ± .0291 • 54.63 ± 7.70 • 42.48 ± 1.48 • .8449 ± .0100 • 24.84 ± 1.13 • 65.76 ± 1.01 • .9348 ± .0018 • 63.40 ± 1.08 • 59.97 ± 1.08 • .9296 ± .0032 • 52.48 ± 1.26 • 51.95 ± 3.47 • .8475 ± .0236 • 50.63 ± 4.02 • 43.47 ± 2.08 • .8925 ± .0113 • 26.62 ± 1.26 • 62.76 ± 2.64 • .9378 ± .0050 • 57.28 ± 3.75 • 57.56 ± 2.61 • .9178 ± .0078 • 56.41 ± 2.87 • 53.47 ± 4.63 • .9079 ± .0112 • 52.12 ± 5.24 70.50 ± 1.21 .9519 ± .0014 68.17 ± 2.01 71.83 ± 1.57 .9596 ± .0027 67.80 ± 1.95	• 61.72 ± 0.70 • .9333 ± .0028 • 56.77 ± 1.36 • 56.46 ± 2.01 • .9073 ± .0082 • 49.47 ± 2.79 • 78.40 ± 0.52 • .9734 ± .0008 ≡ 78.06 ± 0.53 • 77.30 ± 0.79 • .9669 ± .0022 ≡ 77.27 ± 0.80 • 67.50 ± 1.13 • .9562 ± .0029 • 65.03 ± 1.57 • 76.84 ± 3.16 • .9665 ± .0081 • 76.38 ± 3.41 • 78.36 ± 0.76 • .9737 ± .0013 • 78.23 ± 0.74 • 73.98 ± 3.86 • .9617 ± .0090 • 73.97 ± 3.77 • 56.00 ± 0.69 • .9138 ± .0059 • 48.14 ± 0.98 • 80.41 ± 0.29 • .9730 ± .0005 • 80.39 ± 0.31 • 73.49 ± 0.22 • .9671 ± .0009 • 72.69 ± 0.30 • 70.68 ± 0.99 • .9315 ± .0040 • 70.52 ± 0.99 • 60.64 ± 1.27 • .9469 ± .0042 • 57.22 ± 1.39 • 75.06 ± 0.97 • .9734 ± .0011 • 74.07 ± 1.14 • 75.85 ± 0.43 • .9660 ± .0012 • 75.56 ± 0.39 • 77.45 ± 0.56 • .9677 ± .0014 • 77.22 ± 0.62 82.85 ± 0.49 .9838 ± .0005 82.70 ± 0.49 84.74 ± 0.59 .9876 ± .0006 84.60 ± 0.62	• 89.49 ± 0.53 • .9934 ± .0007 • 88.91 ± 0.40 • 78.70 ± 1.13 • .9900 ± .0010 • 76.74 ± 1.58 • 96.02 ± 0.24 • .9989 ± .0002 • 96.21 ± 0.28 • 96.30 ± 0.31 • .9988 ± .0002 • 96.35 ± 0.39 • 93.91 ± 0.34 • .9980 ± .0002 • 94.10 ± 0.26 • 94.95 ± 0.28 • .9982 ± .0005 • 94.81 ± 0.29 • 95.77 ± 0.29 • .9983 ± .0002 • 95.93 ± 0.34 • 93.65 ± 1.38 • .9971 ± .0009 • 94.06 ± 1.13 • 81.27 ± 1.65 • .9910 ± .0013 • 79.88 ± 2.36 • 96.35 ± 0.16 • .9986 ± .0004 • 96.38 ± 0.25 • 95.85 ± 0.26 • .9985 ± .0002 • 95.93 ± 0.24 • 93.71 ± 0.91 • .9880 ± .0035 • 94.22 ± 0.78 • 87.58 ± 2.12 • .9946 ± .0016 • 87.85 ± 2.02 • 95.87 ± 0.34 • .9986 ± .0002 • 96.10 ± 0.30 • 94.69 ± 0.46 • .9976 ± .0004 • 94.84 ± 0.52 • 95.23 ± 0.77 • .9983 ± .0004 • 95.57 ± 0.75 96.77 ± 0.30 .9992 ± .0001 96.90 ± 0.29 96.91 ± 0.24 .9993 ± .0002 97.00 ± 0.22	• 69.08 • • .9423 • • 64.00 • • 61.56 • • .9193 • • 54.96 • • 80.71 • • .9758 • • 80.07 • • 80.52 • • .9714 • • 80.26 • • 75.56 • • .9669 • • 73.04 • • 79.66 • • .9651 • • 78.95 • • 79.39 • • .9724 • • 78.91 • • 75.86 • • .9616 • • 75.76 • • 61.87 • • .9272 • • 54.64 • • 81.80 • • .9730 • • 81.21 • • 78.56 • • .9711 • • 76.62 • • 74.93 • • .9337 • • 74.98 • • 66.55 • • .9496 • • 61.75 • • 79.08 • • .9735 • • 77.51 • • 77.19 • • .9639 • • 76.74 • • 77.34 • • .9637 • • 77.01 • 84.10 .9815 83.53 85.26 .9849 84.24

Table 4 :

 4 Comparative Study on the DomainNet domain adaptation Benchmark Dataset, with Limited and Imbalanced Data. The best and second-best results for each metric and each target domain appear in bold and underlined respectively.

Table 5 :

 5 Comparative Study on the Real-World Medical Covid Dataset Dataset, with Limited and Imbalanced Data. The best and second-best results for each metric and each target domain appear in bold and underlined respectively. Results are evaluated on a binary classification task using metrics: balanced Accuracy, AUC, and the F 1 -Score.

Table 6 :

 6 Ablation study compared approaches. the statistical and adversarial regularizations of the common branch are used, or only one of the two, or none in the case where only the specific branch is used. Finally, column "Weights" indicates if transfer contribution weights are computed and used during training to minimize negative transfer or not.

	Method	Metric MNIST	MNIST-M	SVHN	SYN	USPS	Avg
		bACC 94.95 ± 0.58	72.02 ± 1.24	67.69 ± 2.28	80.94 ± 0.49	96.57 ± 0.22	82.43
	WMSSDA-A	AUC	.9985 ± .0001	.9620 ± .0024	.9429 ± .0054	.9783 ± .0012	.9989 ± .0002	.9761
		F1	94.89 ± 0.61	71.20 ± 1.35	65.64 ± 2.70	80.63 ± 0.53	96.83 ± 0.19	81.84
		bACC 93.77 ± 0.90	74.62 ± 1.50	66.34 ± 2.24	79.72 ± 1.20	95.89 ± 0.35	82.07
	WMSSDA-B	AUC	.9980 ± .0003	.9721 ± .0021	.9414 ± .0048	.9805 ± .0013	.9982 ± .0005	.9781
		F1	93.71 ± 0.92	73.79 ± 1.72	62.83 ± 2.40	79.48 ± 1.18	96.06 ± 0.47	81.17
		bACC 93.95 ± 0.61	73.52 ± 1.32	69.61 ± 1.60	81.58 ± 0.95	96.60 ± 0.16	83.05
	WMSSDA-C	AUC	.9982 ± .0002	.9709 ± .0017	.9482 ± .0049	.9813 ± .0014	.9984 ± .0004	.9794
		F1	93.79 ± 0.71	72.98 ± 1.58	67.29 ± 2.87	81.41 ± 0.99	96.73 ± 0.11	82.44
		bACC 95.01 ± 0.46	74.60 ± 1.33	69.27 ± 2.05	81.87 ± 1.42	96.28 ± 0.21	83.41
	WMSSDA-D	AUC	.9988 ± .0002	.9736 ± .0025	.9505 ± .0035	.9828 ± .0019	.9987 ± .0003	.9809
		F1	94.93 ± 0.50	74.19 ± 1.40	67.16 ± 2.34	81.70 ± 1.43	96.38 ± 0.14	82.87
		bACC 95.16 ± 0.25	74.94 ± 0.76	69.20 ± 1.17	82.60 ± 0.46	96.60 ± 0.28	83.70
	WMSSDA-E	AUC	.9989 ± .0002	.9737 ± .0024	.9501 ± .0016	.9833 ± .0008	.9987 ± .0003	.9809
		F1	95.11 ± 0.26	74.50 ± 0.88	67.11 ± 1.56	82.45 ± 0.43	96.65 ± 0.30	83.17
		bACC 95.30 ± 0.21	75.05 ± 1.06	70.50 ± 1.21	82.85 ± 0.49	96.77 ± 0.30	84.10
	WMSSDA	AUC	.9988 ± .0001	.9737 ± .0017	.9519 ± .0014	.9838 ± .0005	.9992 ± .0001	.9815
		F1	95.25 ± 0.21	74.62 ± 1.17	68.17 ± 2.01	82.70 ± 0.49	96.90 ± 0.29	83.53

http://yann.lecun.com/exdb/mnist

https://www.kaggle.com/datasets/aquibiqbal/mnistm

http://ufldl.stanford.edu/housenumbers

https://www.kaggle.com/datasets/prasunroy/synthetic-digits

https://www.kaggle.com/datasets/bistaumanga/usps-dataset

http://ai.bu.edu/M3SDA/#dataset

https://github.com/easezyc/deep-transfer-learning/blob/master/MUDA/MFSAN

Deep Multi-Source Supervised Domain Adaptation with Class ImbalanceImmunoTherapy -an Open smart digital Platform for personalized prevention and patient management).

https://drive.google.com/file/d/ 1XvgWqkHA4LuK6bPE9ktIlO9SFY1MemnC/view?usp=sharing 9 https://github.com/CuthbertCai/pytorch DAN 10 https://github.com/fungtion/DANN 11 https://github.com/mil-tokyo/MCD DA 12 https://github.com/easezyc/deep-transfer-learning/tree/master/UDA/pytorch1.0/DSAN 13 https://github.com/hanzhaoml/MDAN

Availability of data and material

• Covid dataset: https://www.kaggle.com/datasets/meirnizri/ covid19-dataset. • Digits dataset: -MNIST: http://yann.lecun.com/exdb/mnist. -MNIST-M: https://www.kaggle.com/datasets/aquibiqbal/mnistm. -Street View House Numbers (SVHN): http://ufldl.stanford.edu/ housenumbers. -Synthetic Digits (SYN): https://www.kaggle.com/datasets/prasunroy/ synthetic-digits. -USPS: https://www.kaggle.com/datasets/bistaumanga/usps-dataset.

• DomainNet dataset: http://ai.bu.edu/M3SDA/#dataset.

Code availability

Temporary private link, to be updated after acceptance: https://drive.google. com/file/d/1XvgWqkHA4LuK6bPE9ktIlO9SFY1MemnC/view?usp=sharing.

Conflicts of interest/Competing interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Ethics approval

The authors adhere to the accepted ethical standards of a genuine research study.

Consent to participate

All authors give consent for their participation to the research work.

Consent for publication

The authors give to the publisher the permission to publish this work if it is accepted.