Pooja Gupta
email: poojag@iiitd.ac.in

Angshul Majumdar
email: angshul@iiitd.ac.in

Emilie Chouzenoux
email: emilie.chouzenoux@inria.fr

Giovanni Chierchia
email: giovanni.chierchia@esiee.fr

DeConDFFuse : Predicting Drug-Drug Interaction using joint Deep Convolutional Transform Learning and Decision Forest fusion framework

Keywords: information fusion, convolution, drug-drug interaction, multi-channel, transform learning, decision forest

In Drug-Drug-Interaction (DDI), the task is to predict the (adverse) effect of administering two drugs simultaneously. Currently, the techniques proposed in this direction are generally based on either shallow learning paradigms like Random Decision Forest (RDF), Logistic Regression (LR), Support Vector Machines (SVM), etc., or deep Convolutional Neural Networks (CNNs). However, specific works combine traditional machine learning (ML) algorithms such as RDF, LR, SVM, and deep learning paradigms such as CNNs in a piecemeal fashion which might not be optimal. Hence, the present work proposes a framework that presents a joint end-to-end solution. We propose a Siamese-like architecture with two processing channels' networks based on deep convolutional transform learning. Common fused representations as well as channel-wise representations are learnt, in addition with the transform across them. The final representation is passed to a decision forest to give final predictions. The proposed method is thus a supervised end-to-end multi-channel fusion framework that (i) learns unique and interpretable filters in contrast with CNNs, and (ii) jointly learns and optimizes decision forest in contrast with state-of-the-art piecemeal approach. We apply this technique to identify DDIs among 1059 drugs from the DrugBank database showing superiority of our method compared to the state-of-the-art(s).

Introduction

Drug-Drug interactions (DDIs) are the adverse changes or effects or reactions of one drug due to the recent concurrent use of another drug(s). For example, the drug Ceftriaxone should be avoided in children less than 28 days old if they are receiving or expected to receive IV calcium-containing products. Indeed, it might lead to neonatal deaths resulting from crystalline deposits in the lungs and kidneys, as reported in (Sandritter et al.). Such reaction from DDIs is known as adverse drug reactions (ADRs). ADRs are responsible for the threat to a person's life and inadvertently increase overall healthcare costs.

According to the studies [START_REF] Allison | Reinventing clinical trials[END_REF][START_REF] Bouvy | Epidemiology of adverse drug reactions in europe: a review of recent observational studies[END_REF], ADRs contribute to more than 20% of clinical trial failures and are considered the highest load in the modern drug discovery process. Serious ADRs can cause severe disability and even death in patients. Also, from study [START_REF] Bouvy | Epidemiology of adverse drug reactions in europe: a review of recent observational studies[END_REF], it is observed that approximately 3.6% of all hospital admissions are caused by ADRs in Europe. Up to 10% of patients in European hospitals experience an ADR among those patients. Similarly, it has been estimated that more than 2 million severe ADRs occur in hospitalized patients each year in the United States. This results in more than 100, 000 deaths [START_REF] Giacomini | When good drugs go bad[END_REF][START_REF] Lounkine | Large-scale prediction and testing of drug activity on side-effect targets[END_REF]. From a financial perspective, the annual financial cost of drug-related morbidity in the United States (US) was estimated at $528.4 billion in 2016, equivalent to 16% of total US healthcare expenditures that year (Jonathan H. [START_REF] Watanabe | Cost of prescription drug-related morbidity and mortality[END_REF].

It, thus, becomes pertinent to identify in an exhaustive manner, the DDIs that could cause ADRs. This might not avoid all unanticipated drug interactions but it can help lower the drug development costs and optimize the drug design process [START_REF] Zhang | Prediction of adverse drug reactions based on knowledge graph embedding[END_REF]. Initially and primarily now also, the techniques for DDIs identification are based on clinical trials and experimentation conducted within a living organism (in vivo) or outside it (in vitro) [START_REF] Duke | Literature based drug interaction prediction with clinical assessment using electronic medical records: novel myopathy associated drug interactions[END_REF]. While the ideal case would be to identify all the possible interactions during clinical trials, most of these are often practically determined after the approval of drugs for clinical use. The reason is that the sample size and duration of pre-market trials are limited [START_REF] Whitebread | Keynote review: In vitro safety pharmacology profiling: an essential tool for successful drug development[END_REF]. Another problem with the clinical trials is that the trial subjects are at risk for potential adverse effects.

The other strategy relies on computational approaches. This involves predicting DDIs from some drug features via algorithmic approaches such as machine learning, deep learning, etc. Many significant studies are suggestive of such techniques. The features (i.e., the properties) of the drugs are extracted, such as their side effects, their target proteins, enzymes, chemical structures, etc. Then, off-the-shelf supervised machine learning algorithms such as Decision Trees (DT), Naive Bayes (NB), Logistic Regression (LR), Random Forest (RF), Gradient Boosting (GBT), K-nearest neighbors (KNN), etc. are applied to finally predict probable DDIs given these features [START_REF] Abdelaziz | Large-scale structural and textual similarity-based mining of knowledge graph to predict drug-drug interactions[END_REF]. Exploiting and finding similarities between drug features can be useful in that context [START_REF] Zhang | Sflln: A sparse feature learning ensemble method with linear neighborhood regularization for predicting drug-drug interactions[END_REF]. However, this method can deal with only few (typically up to four) drug features and requires learning a high number of learning parameters that might lead to high training costs. Recently, some works have represented the DDI problem as a matrix completion problem [START_REF] Zhang | Manifold regularized matrix factorization for drug-drug interaction prediction[END_REF][START_REF] Mongia | Deepvir -graphical deep matrix factorization for[END_REF]. Nevertheless, this approach too can involve only a limited set of drug features while a high number of various diverse features are usually available for a drug.

Deep Learning (DL) approaches are currently used in almost every area of problemsolving. In the context of DDI, DL allows the automated learning of features from raw data and provides an end-to-end solution. Approaches like Convolutional Neural Networks (CNNs) or Recurrent Neural Networks (RNNs) such as Long-Short Term Memory (LSTM) have been used to predict DDIs. In the study [START_REF] Huang | Drug-drug interaction extraction from biomedical literature using support vector machine and long short term memory networks[END_REF], the authors have investigated twostages framework using feature-based binary classifier first and then uses LSTM network.

Researchers have used Knowledge Graphs (KGs) to represent drugs' information in this respect. They usually combine the diverse information available about drugs across many different sources in the form of Knowledge Graphs. After that, they apply the DL techniques on them [START_REF] Karim | Drugdrug interaction prediction based on knowledge graph embeddings and convolutional-lstm network[END_REF] like CNNs or recurrent neural networks like LSTM. Still, all of these aforementioned techniques suffer from the typical black-box issue reported in DL, with no interpretability nor uniqueness guarantees on the learnt inner representations.

In this work, we propose a novel model to solve the DDI problem in a supervised fashion. We propose to adopt a representation learning paradigm to learn diverse and interpretable features giving a common representation for the two drugs. We then introduce an end-to-end multi-channel framework that fuses the information about the drugs that are present in a drug pair, and finally gives our DDI predictions from a decision forest. The task at hand is to predict DDIs that could be of two types, namely 1 (known-to-interact) and 0 (either unknown or known-not-to-interact) interactions. Hence, it is a binary classification task. The whole pipeline is jointly and globally optimized to further guide learnt representations in the same and optimal decision. Its performance is compared over the state-of-the-arts on a DDI dataset. The rest of the paper is organized as follows. We first perform a bibliographical study in Section 2. We then discuss the proposed method and dataset in Section 3. In Section 4, we proceed with the experimental setup, followed by the results and some discussion in Section 5. Finally, we conclude our work in Section 6.

Related work and positioning

DDIs identification is considered as a non-trivial problem from the research perspective to be solved in the pharmacology discipline. In literature, many different computational strategies have been investigated that we discuss here below. Several families of methods can be identified, relying either on statistical machine learning models, graph models, deep learning models and matrix factorization models.

Statistical Machine Learning based frameworks

The work [START_REF] Sridhar | A probabilistic approach for collective similarity-based drug-drug interaction prediction[END_REF] proposes similarity-based models that compute similarity scores between drug features like chemical structures, side-effects, targets, pathways, etc., and thereafter performs a probabilistic inference of the DDIs. Researchers have explored Bayesian learning models [START_REF] Yu | A bayesian meta-analysis on published sample mean and variance pharmacokinetic data with application to drug-drug interaction prediction[END_REF] under statistical learning paradigms. Another work [START_REF] Zhang | Sflln: A sparse feature learning ensemble method with linear neighborhood regularization for predicting drug-drug interactions[END_REF] uses sparse feature learning ensemble method with linear regularization utilizing four drug features -chemical substructures, targets, enzymes and pathways. In [START_REF] Ferdousi | Computational prediction of drugdrug interactions based on drugs functional similarities[END_REF], the work utilizes the drug similarity function between various drug features such as the chemical structure, enzymes, targets, pathways, etc., to compute DDIs. In [START_REF] Dang | Machine learning-based prediction of drug-drug interactions for histamine antagonist using hybrid chemical features[END_REF], ML algorithms like NB, DT, RF, LR, and XGBoost were used with crossvalidation with input as SMILE values and interaction features based on CYP450 group. In [START_REF] Huang | Drug-drug interaction extraction from biomedical literature using support vector machine and long short term memory networks[END_REF], the two-stages framework is proposed using SVM in first stage as feature based binary classifier and then a RNN based bidirectional LSTM network.

All the aforementioned studies utilize statistical machine learning methods whose performance might depend highly on the quality of features used; thus, it becomes pertinent to explore multiple features than restrict them to some set. Furthermore, overfitting stays a significant issue with these techniques due to their restrictive non-linear mapping and fitting capability.

Matrix Factorization and multi-modal techniques

Let us also mention recent studies that present matrix factorization as the solution to predict DDIs [START_REF] Zhang | Manifold regularized matrix factorization for drug-drug interaction prediction[END_REF][START_REF] Mongia | Deepvir -graphical deep matrix factorization for[END_REF]. Here, the input is the DDI matrix and the similarity scores between the drugs. The pair for which the DDI is to be predicted is treated as a missing value; hence, it is imputed using the inputted similarity scores. Then some works use the Triple Matrix Factorization also [START_REF] Sandritter | Tmfuf: a triple matrix factorization-based unified framework for predicting comprehensive drugdrug interactions of new drugs[END_REF][START_REF] Shi | Predicting comprehensive drug-drug interactions for new drugs via triple matrix factorization[END_REF]. Matrix Factorization based algorithms may have problems if the values are not independent. Additionally, this technique is likely to fail as it assumes missing values at random which is not the case in practice. Some researchers have even proposed multi-modal techniques to predict DDIs. Although there are few, we briefly introduce a few of these studies here. The study that used this technique learned the unified drug representations from multiple drug feature networks simultaneously using multi-modal deep auto-encoders. Then, they applied four operators on the learned drug embeddings to represent drug-drug pairs, and finally, they use a RF classifier to train models for predicting DDIs [START_REF] Zhang | Predicting drug-drug interactions using multi-modal deep auto-encoders based network embedding and positiveunlabeled learning[END_REF].

Graph-based frameworks

Graph-based embedding techniques are also gaining momentum in DDIs prediction. With the advent of the availability of biomedical data, researchers are moving toward KGs to populate and complete the available biomedical information. It is done with the help of the large structured databases and texts available publicly [START_REF] Celebi | Evaluation of knowledge graph embedding approaches for drug-drug interaction prediction in realistic settings[END_REF]. For example, the Bio2RDF project has made 35 life sciences datasets available as Linked Open Data (LOD) in RDF. In this, similar entities are mapped in different KGs, resulting in large heterogeneous biomedical graphs containing drug-related facts. Some works have used the combination of DDI matrix and KG followed by the application of ML algorithms [START_REF] Celebi | Evaluation of knowledge graph embedding approaches for drug-drug interaction prediction in realistic settings[END_REF].

Deep Learning based frameworks

Deep learning is another effective modeling technique that is extensively used in solving most real-world problems these days. It has emerged to be helpful in the said DDI prediction as well. In [START_REF] Yu | Raneddi: Relation-aware network embedding for drug-drug interaction prediction[END_REF], the proposed framework integrates the multi-relational and the relation-aware network structure representations. Finally, the integrated representations via concatenation is passed through neural network to get the final DDI predictions. The study [START_REF] Sahu | Drug-drug interaction extraction from biomedical texts using long short-term memory network[END_REF] proposes attention-based RNNs -LSTM for DDI prediction. In [START_REF] Liu | Enhancing drug-drug interaction prediction using deep attention neural networks[END_REF], the work utilizes deep Neural Networks based on attention technique for predicting DDIs with features from multiple networks are learnt using graph embedding techniques. In [START_REF] Rohani | Drug-drug interaction predicting by neural network using integrated similarity[END_REF], multiple drug similarities based on drug substructure, target, side effect, off-label side effect, pathway, transporter, and indication data are calculated. Then, it uses a neural network for interaction prediction.

In another study [START_REF] Chen | Drug-drug interaction prediction with graph representation learning[END_REF], the authors explore the molecular graphs formed from SMILE inputs and learn the graph representations using Siamese Graph Convolutional Network (GCN) and further pass these representations to the neural networks for the final predictions. Some works have even integrated the CNNS and RNNS for the same. For example, [START_REF] Liu | Efficient prediction of drug-drug interaction using deep learning models[END_REF] integrates CNNs, RNNs and mixture density networks. Another work [START_REF] Wu | Drug-drug interaction extraction via hybrid neural networks on biomedical literature[END_REF] combines RNN-based stacked Gated Recurrent Units (GRU) with CNNS for the prediction task.

Further studies are combining KGs and DL to predict DDIs. In the study [START_REF] Karim | Drugdrug interaction prediction based on knowledge graph embeddings and convolutional-lstm network[END_REF], the DDI matrix and KG form the input to the DL network that has CNN and LSTM. KG is input to the network in the form of learned embeddings like ComplEx, TransE, RDF2Vec, etc. This work [START_REF] Park | Agcn: Attention-based graph convolutional networks for drug-drug interaction extraction[END_REF] uses a Graph CNN and additionally applies attention based strategy to prune the irrelevant information and keep the significant parts only. Another work suggests using a Neural Network instead of CNN that uses the same input as in the preceding study mentioned, except that it is based on previously established Graph Neural Networks (GNN) [START_REF] Hamilton | Inductive representation learning on large graphs[END_REF]. Also, it focuses on neighborhood sampling and aggregates entities and their neighborhood representation into a single vector in 3 different ways [START_REF] Lin | Kgnn: Knowledge graph neural network for drug-drug interaction prediction[END_REF]. Another work [START_REF] Chen | MUFFIN: multi-scale feature fusion for drug-drug interaction prediction[END_REF] proposes the Deep CNN network that learns the cross channel features from two inputs -KG and drug molecular features graph.

From the above mentioned studies related to DDI prediction and otherwise also, CNNs have been regarded as one of the most frequently used paradigms for solving problems. However, there are certain issues that persist with CNNs. There is no surety of learning of distinctive filters and hence there are chances of redundancy in the learned features / representations. Another issue that is encountered with CNNs is dead neuron problem that becomes big in nature if every neuron in a specific hidden layer is dead, it cuts the gradient to the previous layer resulting in zero gradients to the layers behind it. Thus, the weights would not be updated and the learning will be improper.

Also, the current proposed paradigms are usually such that these learn features from CNNs and then explicitly employ some other classifier to perform the classification task, i.e. piecemeal approach instead of joint training that might have chance of loosing important information. With other techniques as well like similarity-based, network-based, graph-based, etc., we do not obtain guarantee of the distinctive learning of features. Thus, from this discussion, these issues lead to the scope for developing supervised frameworks that can combinely tackle them.

Our contribution

Our work relies on a supervised learning strategy formulating the problem as a binary classification one, which makes our approach related to the mentioned DL methods. However, the proposed pipeline involves representation learning steps, namely convolutional transform learning, that aims at gaining interpretability and stability of the decision process. In particular, our specific learning strategy enhances unique features/representations that help in better prediction. The features/representations are also optimized and learned in a direction given by a decision forest predictor with the goal to reach better predictions for the test data. Thus, this work proposes an end-to-end solution that learns parameters for both deep CTL and decision forest at the exception of the RF probability distribution. The latter is computed based on the CTL and decision forest learnt models. The end-to-end training relies efficiently on stochastic gradient updates via automatic differentiation. The details of our pipeline and our training strategy are discussed in the next Section 3. Also, the works discussed above utilize different feature sets for drugs that may not be readily available for some small molecule drugs and hence, limit them to predict interactions for those with the other drugs, for example, [START_REF] Zhang | Sflln: A sparse feature learning ensemble method with linear neighborhood regularization for predicting drug-drug interactions[END_REF]. Thus, another benefit with our approach is that we have used Bioactivity descriptors as features for the drugs in our dataset, generated via Signaturizer tool [START_REF] Bertoni | Bioactivity descriptors for uncharacterized chemical compounds[END_REF]. The pre-trained Siamese Neural Network of the tool helps to infer 25 different types of bioactivity descriptors for the drugs that have little or no information. With our approach, we initially need smile values which we may not get for all the drugs but still will be able to process more drugs when compared with feature sets that reduces the number of drugs significantly. For example, the features used in the study [START_REF] Zhang | Sflln: A sparse feature learning ensemble method with linear neighborhood regularization for predicting drug-drug interactions[END_REF] that restrict the number of the drugs considered and compels to remove them from the dataset due to non-availability of features.

Data and Methodology

This section will first briefly describe the dataset used, followed by the proposed technique for inference and training.

Dataset Description

We use DDI data from Stanford's Biosnap dataset2 , which contains a network of 1514 DrugBank drugs representing nodes and 48514 drug-drug interactions representing edges. This network of interactions between drugs is approved by the U.S. Food and Drug Administration. We have assumed all other interactions apart from approved interactions as either known-not-to-interact or unknown. Here, we represent the known-to-interact interactions by 1 and the others by 0 numerically. We first determined the SMILE values of the drugs using compound IDs taken from the dataset using DrugBank.ca. Since the SMILE values are not available for all the drugs (retrieved using DrugBank IDs), thus, the number of the drugs in the dataset was reduced to 1368 and, accordingly, the number of interactions. Further, we have processed only the drugs that have at least 10 known-to-interact interactions with other drugs. So, there are 1059 drugs and their respective interactions.

Thereafter, we extract the bioactivity descriptors via the Signaturizer tool [START_REF] Bertoni | Bioactivity descriptors for uncharacterized chemical compounds[END_REF] using the determined smile value of each drug. This tool provides bioactivity descriptors that encode the physicochemical and structural properties of small molecule drugs covering all the drugs present in Chemical Checker (CC). The latter has further covered the source databases -DrugBank.ca and ChEMBl. It has a pre-trained Siamese Neural Network via which inputting a smile value for drug, 25 different types of bioactivity descriptors can be inferred for the drugs that have little or no information. The descriptors are fixed-length normalized vectors of size 128. There are broadly five categories of bioactivity descriptors labeled as A to E (A: Chemistry, B: Targets, C: Networks, D: Cells, and E: Clinics). Each has five sub-categories marked as A1 to A5, for example, thus 25 different descriptors. We have taken descriptors from A and B broad categories representing a drug's Chemistry and Targets, respectively. Further, we choose A1 and A2 sub-categories from A, representing 2D and 3D fingerprints, and the B1 sub-category from B, representing the mechanism of action of a drug. Since we have chosen 3 types of bioactivity descriptors out of 25, each having 128 fixed-sized vectors, therefore, we have a total 384(128 × 3) features for every drug. Thus, our dataset comprises 1059 unique drugs with 384 bioactivity descriptors/features for each drug and corresponding interactions.

Proposed Framework

In this section, we discuss our proposed work. We present a fusion framework that combines the benefits of our recently established multi-channel, unsupervised, fusion-based representation learning framework -DeConFuse [START_REF] Gupta | Deconfuse: a deep convolutional transform-based unsupervised fusion framework[END_REF] and jointly optimizes a decision forest with binary decision, that gives the final DDI Predictions. Such a solution has been successfully used before in Deep Neural Decision Forest (DNDF) framework [START_REF] Kontschieder | Deep neural decision forests[END_REF]. Let us mention that DeConFuse architecture is unsupervised. We want to propose a supervised version of this architecture. We previously established a supervised version of this framework namely SuperDeConfuse [START_REF] Gupta | Superdeconfuse: A supervised deep convolutional transform based fusion framework for financial trading systems[END_REF]. However, the supervision in SuperDeConFuse was incorporated by using cross entropy loss in the optimization objective and using softmax in the end of its architecture. Here, our goal is to guide the supervision through a random decision forest. The proposed solution does not utilize features/representations from CNN but instead from the DeConFuse network based on deep CTL, through linear transform learning. The advantage that latter offers is that it promotes unique filters/transforms which is not guaranteed with CNNs. Such advantage helps in learning diverse and interpretable representations. These representations are further guided by the predictions from decision forest whose parameters are jointly optimized. The representations learnt are useful as these help decision forest to correctly identify many known-to-interact (1) DDIs as also corroborated from the experimental results discussed in the section 5 later. We briefly discuss both frameworks and, finally, mention the details of the combined fusion framework.

DeConFuse

Let us start introducing Deep CTL architecture for representation learning. It stacks multiple convolutional layers on top of each other to generate the features, as shown in Figure 1. Here T ℓ , ℓ ∈ {1, . . . , L} filters are convoluted with input sample S. The outputs X are the learned features corresponding to the convoluted output. Optimal filters aim at reducing the quadratic loss

F conv (T 1 , . . . , T L , X | S) = 1 2 ∥T L * ϕ L-1 (T L-1 * . . . ϕ 1 (T 1 * S)) -X∥ 2 F , (1)
where ϕ is the activation function (e.g., SELU), and L represents the number of CTL layers we apply, which in our case is L = 2. However, this objective alone is not enough for proper learning, as it can give trivial solution (e.g., X and T ℓ all equal to 0). To avoid this situation, regularization is added, leading to the objective function

F conv (T 1 , . . . , T L , X | S) = 1 2 ∥T L * ϕ L-1 (T L-1 * . . . ϕ 1 (T 1 * S)) -X∥ 2 F + ι + (X) + L ℓ=1 (µ||T ℓ || 2 F -λ log det(T ℓ )). (2)
Here, the term ι + (X) is a non-negativity constraint on X (equals to 0 for positive valued X, +∞ otherwise). The regularization term "µ ∥•∥ 2 F -λ log det" ensures that the non-zero and unique filters are learnt (which is not guaranteed in CNN). We learn here all the variables in an end-to-end fashion. Next, we extend the deep CTL to a fusion network where we have two separate Deep CTL networks/channels (C = 2) for each drug in a drug pair that give features X = (X (c) ) 1≤c≤C and fused together to give the common representation Z. The latter representation is learnt via linear transforms (i.e. not convolutional) as learnt in original Transform Learning (TL) technique [START_REF] Ravishankar | Learning sparsifying transforms[END_REF]. This part of the architecture is learnt so as to reduce the fusion loss

F fusion ( T , Z, X) = 1 2 Z - C c=1 flat(X (c) ) T c 2 F + ι + (Z) + C c=1 µ∥ T c ∥ 2 F -λ log det( T c ) (3)
where the operator "flat" transforms X (c) into a matrix where each row contains the "flattened" features of a sample. In a nutshell, DeConFuse has two parts, (i) Deep Convolutional part and (ii) Fusion part. The learning procedure aims at solving the joint optimization problem given as: minimize T,X, T ,Z

F fusion ( T , Z, X) + C c=1 F conv (T (c) 1 , . . . , T (c) L , X (c) | S (c) ). (4)
The complete architecture of DeConfuse is given in Figure 2. There are two notable advantages of the DeConFuse approach. Firstly, we rely on automatic differentiation [START_REF] Paszke | Automatic differentiation in pytorch[END_REF] and stochastic gradient approximations to efficiently solve Problem (4). Secondly, we are not limited to ReLU activation in ( 2), but rather we can use more advanced ones, such as SELU [START_REF] Klambauer | Self-normalizing neural networks[END_REF]. It is beneficial for the performance, as shown by our numerical results.

DNDF Framework

We now introduce the DNDF framework from [START_REF] Kontschieder | Deep neural decision forests[END_REF], that will be the last brick of our DDI pipeline. It is different from conventional deep neural networks as it outputs the final predictions from the decision forest, and their split (decision nodes) and leaf (prediction) nodes' parameters are jointly and globally optimized. The technique is stochastic, differentiable, and, thus, gives a backpropagation compatible version of decision trees that guides the representation learning in lower layers of deep CNNs. This reduces the uncertainty on routing decisions of a sample taken at the split nodes, such that the globally defined loss function is minimized. For the leaf nodes, the optimal predictions are achieved by minimizing the convex objective function, which does not require step size selection. We further explain the objective function briefly.

Decision Trees with Stochastic Routing

Consider a classification problem with input space χ and finite output space Y . A decision tree consists of decision (or split) nodes and prediction (or leaf) nodes. Decision nodes, let's say, indexed by N are internal nodes of the tree, and prediction nodes are indexed by L, i.e., terminal/leaf nodes of the tree. Each prediction node ℓ ∈ L is associated with a probability distribution π ℓ = (π ℓy ) y∈Y . Each decision node n ∈ N is assigned a decision function d n (•; θ) : χ → [ 0, 1] parameterized by θ, which routes the samples along the tree branches. A sample x ∈ χ when reaches a decision node n, it will be directed either to the left or right sub-tree based on the output of the function d n (x; θ). Here, it is a probabilistic Image taken from [START_REF] Kontschieder | Deep neural decision forests[END_REF].

routing where the routing direction is the output of a Bernoulli random variable with mean d n (x; θ). As the sample ends in a leaf node ℓ, the related tree prediction is given by the class-label distribution π ℓ = (π ℓy ) y∈Y . In the case of stochastic routings, the leaf predictions will be averaged by the probability of reaching the leaf. Thus, the final prediction for a sample x from tree D with decision nodes parameterized by θ is given as:

(∀y ∈ Y ) P D [ y | x, θ, π] = ℓ∈L π ℓy µ ℓ (x | θ) (5)
where π = (π ℓ ) ℓ∈L . Here above, µ ℓ (x | θ) is regarded as the routing function providing the probability that sample x will reach leaf ℓ. Note that Σ ℓ µ ℓ (x | θ) = 1 for any x ∈ χ.

For an explicit form for the routing function, the following binary relations that depend on the tree's structure are given as: ℓ ↙ n, which is true if ℓ belongs to the left sub-tree of node n, and n ↘ ℓ, which is true if ℓ belongs to the right sub-tree of node n. Hence, these relations can be exploited to express µ ℓ as:

µ ℓ (x | θ) = n∈N d n (x; θ) 1 ℓ↙n dn (x; θ) (6)
where dn (x; θ) = 1 -d n (x; θ), and 1 P is an indicator function conditioned on the argument P . Although the product in ( 6) runs over all nodes, however, only decision nodes along the path from the root node to the leaf ℓ contribute to µ ℓ , because for all other nodes 1 ℓ↙n and 1 n↘ℓ will be both 0 (with the assumption 0 0 = 1). See Figure 3. Decision functions deliver a stochastic routing with decision functions defined as follows:

d n (x; θ) = σ(f n (x; θ)), (7)
where σ(x) = (1 + e -x ) -1 is the sigmoid function, and f n (•; θ) : χ → R is a real-valued function depending on the sample and the parameterization θ.

A forest is an ensemble of decision trees F = {D 1 , . . . , D k }, which delivers a prediction for a sample x by averaging the output of each tree, i.e.

(∀y ∈ Y ) P F [y | x, θ, π] = 1 k k h=1 P D h [y | x, θ, π]. ( 8
)
Figure 4: illustration of how to implement a deep neural decision forest (DNDF). Top: Deep CNN with a variable number of layers, subsumed via parameters θ. FC block: Fully Connected layer used to provide functions f n (•; θ), described in Equ. 7. Each output of f n is brought in correspondence with a split node in a tree, eventually producing the routing (split) decisions d n (x) = σ(f n (x)). The order of the assignments of output units to decision nodes can be arbitrary (the one shown allows a simple visualization). The circles at the bottom correspond to leaf nodes, holding probability distributions π ℓ . Image taken from [START_REF] Kontschieder | Deep neural decision forests[END_REF].

Learning Trees by Back-Propagation

Learning a decision tree, for which the model is explained in the previous sections, requires estimating both the decision node parameterizations θ and the leaf predictions π. The parameters θ are estimated using the Minimum Empirical Risk principle with respect to a given data set T ⊂ χ × Y under the log-loss (also known as the cross-entropy loss), i.e., minimizers of the following risk term are searched:

F tree (θ; π; T ) = 1 |T | (x,y)∈T -log(P D [y | x, θ, π]) (9)
The forest is learned by considering the ensemble of trees F, where all trees can possibly share the parameters in θ. Still, each tree can have a different structure with a different set of decision functions and independent leaf predictions π. The illustration of the forest of decision trees taking the parameters θ and computing routing decisions and prediction nodes probabilities can be referred to from Figure 4. Thus, for the forest, empirical risk is minimized as:

F forest (θ; π; T ) = 1 |T | (x,y)∈T -log(P F [y | x, θ, π]).
(10)

A two-step optimization strategy is followed to minimize the above function, with alternate updates of θ and π. Interested Readers can further know about the detailed updates/learning mechanism for the parameters θ and π from DNDF [START_REF] Kontschieder | Deep neural decision forests[END_REF].

Combined Proposed Framework DeConDFFuse -DeConFuse and Decision Forest

We propose to combine the frameworks explained in the previous sections 3.2.1 and 3.2.2. Specifically, instead of utilizing the features from a CNN network, we propose to inherit the representations learned from the DeConFuse network to peruse them in our decision forest, i.e., the decision forest is jointly trained and optimized within the DeConFuse network. The DeConFuse network learns channel-wise representations corresponding to each drug in a drug pair, that is X (c) with c ∈ {1, 2}, and finally learns common representation Z from X (c) where the fusion takes place. Here, we do not have a positivity constraint on Z and only on channel-wise representations X (c) .

The representation Z is passed to the DF, where it applies the features mask, i.e., randomly selects the features from the representation that will participate in the decision tree's routing process that sends those selected features to the linear fully connected layer parameterized by θ, i.e., given by the function

f n (x; θ n ) = θ ⊤ n x.
The number of features involved is given by feature ratio. Thereafter, the sigmoid activation is applied as given in Eq. 7. Then the routing function is computed, and the prediction probabilities are calculated. Thus, the prediction probabilities having a probability for each class for each tree is likewise obtained. Finally, the probabilities from all the trees of the Forest F are averaged to get the outcome probability for each of the classes 0 and 1 in our case. The negative log-likelihood loss is computed and back-propagated, which guides the representation learning of the DeConFuse framework and learning of the parameters θ. The objective function for this framework that combines the idea of DeConFuse and DF can be deduced from 4 and 10: minimize

T,X, T ,Z,θ,π F fusion ( T , Z, X) + C c=1 F conv (T (c) 1 , . . . , T (c) L , X (c) | S (c) ) + F forest (θ; π; T Z ). J(T,X, T ,Z,θ,π) (11)
Hereabove, the dataset T Z is built with the learned features Z and the known labels. Note that there is no positivity constraint anymore on the learned representations Z.

Experimental Setup

We conduct experiments on drug-drug interaction dataset comprising DDI matrix and bioactivity descriptors/feature vectors for each drug as explained in section 3.1. We divide the DDI matrix dataset into training and testing datasets. We have kept all drugs in the training data so that there are 95 samples per drug. Further, out of 95 samples, there are 60% of 1 interactions for each drug (not exceeding half of the 95, i.e., min (60% of 1 interactions, 95//2)), and the remaining are the samples from 0 interactions. The remaining samples of 0 and 1 interactions per drug are kept in testing data. All the training and test data samples from each interaction category per drug are selected randomly. Also, only one pair of interactions are kept from either the upper triangle or the lower triangle of the DDI matrix. Thus, each training and testing sample is the drug pair and its corresponding interaction value, which we call a label. Approximately there are 1L training samples and 4L testing samples.

We pass the input as a drug pair as a sample during training. For each drug in a pair, we give the 1D feature vectors i.e. bioactivity descriptors to the individual channel/network based on deep CTL, where L = 2 represents the number of CTL layers. Thus, the input S gathers the bioactivity descriptors/1D feature vectors of size 384 for each channel corresponding to each drug. Since there are 1D feature vectors for each drug in the drug pair, thus, we have 1D convolutions in each deep CTL network. The two networks' learned features/representations X (c) are flattened and concatenated. Then we pass these features to the linear Transform learning layer that acts as a fully connected layer where we learn transform T and common representation Z. Further, we send the learned representation Z selectively by applying the feature mask to the decision forest. The final predictions are output by averaging the predictions from each tree in the decision forest.

The complete architecture is shown in Figure 5 and all the architectural and hyperparameter details are given in Table 1. In the set of hyperparameters, the atom ratio signifies the number of features to be kept in the representation Z; and the feature ratio signifies the randomly selected number of features from the representation Z that will participate in the routing decision function of each tree parameterized by θ. We have compared our results with three state-of-the-arts/benchmark techniques, namely:

• KGNN: This technique is used to build the Knowledge Graph (KG) and pass the DDI matrix and KG to the Graph Neural Network (GNN). It focuses on neighborhood sampling and aggregates entities and their neighborhood representation into a single vector in 3 ways -sum, concat, and neighbor [START_REF] Lin | Kgnn: Knowledge graph neural network for drug-drug interaction prediction[END_REF].

• Conv-LSTM: This technique uses the DDI matrix and KG in the form of the input to the DL network that has a CNN and LSTM. KG is input to the network in the form of learned embeddings like ComplEx, TransE, RDF2Vec, etc. (Rezaul Karim et al., 2019). We have compared against the embedding that has given the best results in this work, i.e., ComplEx embedding.

• Graph Embedding DDI: This technique uses KG and DDI matrix as input but experiments with many different types of embedding techniques. Then each of these embeddings, one by one, are passed to machine learning techniques like Random Decision Forest (RF), Gaussian Naive Bayes (GNB), and Logistic Regression (LR) [START_REF] Celebi | Evaluation of knowledge graph embedding approaches for drug-drug interaction prediction in realistic settings[END_REF]. Here also, we have used the embedding type Skip Gram, which gives the best results in their study. For all three benchmarks -KGNN, KG Conv-LSTM, and Graph Embedding DDI-we have used the same DrugBank IDs as present in our training and testing samples. Since these methods rely on KGs, we did not use the bioactivity descriptors/features but recreated KG and embeddings for our dataset, for running these benchmarks.

Results and Analysis

The prediction results are evaluated using the classification metrics -AUC ROC, F1 Score, Precision, Recall, and Accuracy. We have computed all the metrics except Accuracy as weighted metrics since there is a huge class imbalance between 0 and 1 labels. The following Table 2 contains the values of the said evaluation metrics:

We have also computed the confusion matrices (in percentages) for each of the methods. They are displayed in Figure 6.

From Table 2, it is seen that benchmark Graph DDI gives the best values in terms of Accuracy, F1 Score and Recall, and our method for Precision, AUC ROC, and AUPRC. However, no single Benchmark has worked well in terms of all the classification metrics used for evaluation. In fact, the next best performance in terms of Accuracy and F1 is given by our method. Despite the highest F1, Accuracy, and recall values, Graph DDI fails to achieve the highest values for AUC -ROC and AUPRC, which are considered more relevant and important metrics for the performance evaluation in the case of binary classification. The reason for the same can be observed with the help of the confusion matrices in Figure 6 that are represented in the form of the percentages.

We can see that with our method, we can predict the highest number of known-to-interact interactions (1) correctly than any other benchmarks. Also, except for Graph DDI, the False positives, i.e., classifying 0 as 1, are lesser with our method than the other two benchmarks.

Here, the former task of classifying the known-to-interact drug interactions is more important to prevent ADRs, as explained before and which Graph DDI does not achieve at all or is nearly negligible. Thus, with our method, we are able to accomplish the former task of identifying known-to-interact interactions better than any other benchmark, and for the false positives too, it gives good performance compared to the other benchmarks except graph DDI. The latter is the reason why Graph DDI has the 3 metric values higher than our method. With our method, though the percentage of False positives are higher than Graph DDI, however, it is not necessary that these False positives are completely incorrect. The reason is that the 0 interactions do not signify that there is no interaction between those two drugs. It means either known-not-to-interact or unknown.

In summary, Graph DDI classifies almost all 0 interactions correctly; still, it does not correctly classify 1 (known-to-interact) interactions that are against the study's objective, i.e., to identify the known-to-interact DDIs to avoid ADRs. With our method, both types of interactions are classified reasonably well. It is the stable method corroborated from the classification metrics also as it gives good performance in terms of all the metrics. Hence, our proposed framework performs superior to the benchmarks.

Additionally, let us compare representations / features learned from benchmarks with our framework. Our method performs better due to the kind of representations / features learned from our CTL based network. We carefully examine each of the benchmarks here. In KGNN, Knowledge Graph features, different aggregation techniques and Graph Neural Network are utilized. The latter has a lot of parameters and computation cost since the neural network connects neurons in each layer with every neuron in the preceding and consecutive layers. Also, it has no uniqueness guarantee for the kind of weights learned. The poorest performance from Graph DDI is due to a lack of learning ability of the traditional machine algorithms utilized in this framework after learning the embeddings from KG. Lastly, with the Conv-LSTM framework, all the disadvantages of CNN discussed in section 2.4 are applied. Thus, the representations learned with it might have redundancy leading to inferior performance. Therefore, from overall performance, it can be concluded that the representations learned from our method are better than the benchmarks.

The optimizer used for the updating all the parameters of the framework except probability distribution π of decision forest is Adam that uses the automatic differentiation in pytorch for gradient computation. The hyperparameters like learning rate, betas, and eps etc. associated with it are mentioned in the Table 1. We have also plotted the loss plot with our technique, which can be referred to from Figure 7. It can be seen that with Adam optimizer, our solution converges to the point of stability.

Conclusion

In this work, we propose applying our recently established work DeConFuse and combining it with Decision Forest previously established in the DNDF framework. The proposed framework is a deep supervised fusion end-to-end framework for processing 1D multi-channel drug data. Unlike other deep learning models that separately use conventional machine learning algorithms like RDF, our framework is jointly optimized and is not piecemeal. We have applied the proposed model for the binary classification task of DDI prediction leading to good performance. The advantage of our framework is its ability to learn unique filters that are not guaranteed with CNNs. It helps us learn non-redundant common representation for the problem where we have two drugs in a drug pair that is not only guided by the deep CTL, but the jointly optimized Decision Forest loss also directs it. We are achieving reasonably well performance compared to the state-of-the-art(s).

The future scope of the work is to improve performance by reducing the number of false positives. Also, the current solution to the DDI problem considers the event when two drugs are administered together. However, combination of more than two drugs are routinely used. Thus, we would also like to extend the capability of our framework to handle more than two drugs' combinations in future. This could be done with our architecture by increasing the number of channels per increase in number of drugs. Lastly, although, we have applied our architecture for drug-drug-interaction, it is flexible enough to be applicable for other biomedical interaction problems. In the future, we will like to explore other areas such as drug-target prediction [START_REF] Ding | Identification of drug-target interactions via multiple information integration[END_REF][START_REF] Tanoori | Drug-target continuous binding affinity prediction using multiple sources of information[END_REF][START_REF] Turki | Machine learning algorithms for predicting drugs-tissues relationships[END_REF], protein-protein interaction [START_REF] Sun | Identifying influential genes in protein-protein interaction networks[END_REF][START_REF] Yu | Prediction of protein-protein interactions based on elastic net and deep forest[END_REF][START_REF] Lee | Mining protein-protein interaction information on the internet[END_REF] and drug repositioning [START_REF] Zhang | Computational drug repositioning using collaborative filtering via multi-source fusion[END_REF].

Figure 1 :

 1 Figure 1: Deep CTL architecture. The illustration is given for L = 2 layers, with the first layer T 1 composed of M 1 = 8 filters of size 3 × 1, and the second layer composed of M 2 = 16 filters of size 3 × 1.

Figure 2 :

 2 Figure 2: DeConFuse architecture. This is a general architecture representation where C represents the number of DeepCTL networks/channels.

Figure 3 :

 3 Figure 3: Each node n ∈ N of the tree performs routing decisions via function d n (•). The black path shows an exemplary routing of a sample x along a tree to reach leaf ℓ 4 , which has probability µ ℓ4 = d 1 (x) d2 (x) d5 (x).Image taken from[START_REF] Kontschieder | Deep neural decision forests[END_REF].

Figure 5 :

 5 Figure 5: DDI prediction using combined DeConFuse and decision forest architecture-DeConDFFuse. Here C = 2 number of networks/channels via each of which a drug in the drug pair is passed along with its bioactivity descriptors/ features vector respectively.

Figure 6 :

 6 Figure 6: Confusion matrices for different benchmarks and our method-DeConDFFuse

Figure 7 :

 7 Figure 7: Loss plot with our method -DeConDFFuse.

Table 1 :

 1 DDI Prediction DeConDFFuse Architecture Details

	Parameter	Value
	Layer Wise Hyperparameters
	Layer1 -Convolution (CTL) (1,16,3,1,1)
	Maxpool	(2,2)
	Layer2 -Convolution (CTL) (1,32,3,1,1)
	Maxpool	(2,2)
	atom ratio	0.75
	Decision Forest Hyperparameters
	#Trees	90
	tree depth	10
	feature ratio	0.5
	Other Model Hyperparameters
	Epochs	75
	Learning Rate	0.01
	µ	1e-05
	λ	0.0001
	batch size	4096
	weight decay	1e-05
	Optimizer Hyperparameters
	Optimizer Used	Adam
	Ams grad	True
	Learning rate	0.01
	betas	(0.9, 0.999)
	eps	1e-08
	(in˙planes, out˙planes, kernel size, stride, padding)
	(kernel size, stride)

Table 2 :

 2 DDI Prediction Results

	Method	Sub	Accuracy	F1	Precision Recall	AUC	AUPRC
		Method					ROC
	KGNN	Sum	85.8168	89.5672	95.0392	85.8168	82.6508	18.6945
		Concat	86.9723	90.2730	95.0375	86.9723	83.5235	19.8427
		Neighbor	81.7908	86.9563	94.1900	81.7908	74.379	10.7655
	Conv-LSTM	-	86.4785	89.3325	92.5174	86.4785	49.597	3.8164
	ComplEx
	Graph DDI		95.8015	94.0807	92.5087	95.8015	50.1899	3.8546
	Skip Gram	LR	96.1235 94.2236	92.3973	96.1235 50.0603	3.8583
		RF	96.1235 94.2236	92.3973	96.1235 50.0934	3.8439
	DeConDFFuse	-	90.7422	92.7777	95.9478	90.7422 91.4453 34.0847
	(Ours)

https://snap.stanford.edu/biodata/datasets/10001/10001-ChCh-Miner.html#: :text=Dataset %20information&text=Drug%2Ddrug%20interactions%20occur%20when,such%20as%20adverse %20drug%20reactions

Acknowledgement

The authors acknowledge support from Inria Saclay, through the International Inria Team program COMPASS.