Ph.D Eunice Okome Obiang

François, Maxime Eric Daniel

Axel Aurore

Thibault Jean Thomas

Agathe Thomas

Chloé, Maël Sabrina Florian

Contribution à l'étude des

modèles graphiques et statistique en grande dimension appliquée à la modélisation du cancer du sein triple négatif

LIST OF FIGURES

1.1 Examples of undirected (top) and directed (bottom) graphs with their adjacency matrices.

1.2 Example of a directed graph with its associated moral graph. . . . . . . . . . . . . . . . . 2.1 Marginal shape of the generalized Gaussian distribution (d " 1 and V " 1) for some β ă 1 (dotted red), β " 1 (black) and some β ą 1 (dotted blue). The noteworthy cases β " 1{2 (Laplace), β " 1 (Gaussian) and β " `8 (uniform) are highlighted. . . . . . . . . . . . . 2.2 Mean squared prediction error for N " 500 repetitions of the weakly structured Scenario 1.

2.3 Mean squared prediction error for N " 500 repetitions of the strongly structured Scenario 2.

2.4 Mean squared prediction error for N " 500 repetitions of the strongly structured Scenario 3.

2.5 Mean squared prediction error for N " 500 repetitions of the strongly structured Scenario 2 (left) and Scenario 3 (right) for Or, Gm and the unstructured Or (L " I p ), with β " 2.

2.6 Temperature and log-precipitation measured over a year in Montreal (left). Empirical distribution of the minimal and maximal log-precipitation for the 35 weather stations (right).

2.7 Mean squared prediction error for N " 100 repetitions of the experiment. GenGm for β P t0.5, 1, 1.5, 2u is compared with Spr, Gm, Las and GLas. . . . . . . . . . . . . . . . . .

2.8

Variable selection for min p by GenGm with pλ, µ, ηq " p0, 0.05, 1q and, from top to bottom, β P t0.5, 1, 1.5, 2u. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.9

Variable selection for max p by GenGm with pλ, µ, ηq " p0, 0.05, 1q and, from top to bottom, β P t0.5, 1, 1.5, 2u. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.10 Estimated direct links (top) and regression coefficients (bottom) for the pair pmin p , max p q by GenGm with pλ, µ, ηq " p0, 0.05, 1q and β " 2, after the N " 100 experiments. Dotted lines divide the panel into months. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.11 Estimated correlation between min p and max p by GenGm with pλ, µ, ηq " p0, 0.05, 1q and β " 2, after the N " 100 experiments. The off-diagonal entry is approximately 0.32. . . . 3.1 Medians of the mean squared prediction errors obtained after N " 100 repetitions of Scen.

1 (top), Scen. 3 (middle) and Scen. 5 (bottom) with ˘1 standard deviation and n e growing from 100 to 500. The black curves correspond to uncorrelated predictors (ρ " 0) while the blue and red curves correspond to correlated predictors (ρ " 0.5 and ρ " 0.9, respectively).

5.8

Step 3: intersections of the sets of variables selected by each of three methods (left), bmlasso did not select any of the candidate variables; proportion of groups selected by SVSSA in the sets of groups selected by each individual method (right). . . . . . . . . . . . . . . . .

5.9

Step 4: intersections of the sets of variables selected by each of the four methods (left); Les progrès scientifiques et technologiques des deux dernières décennies, en termes de collecte et de traitement de données, ont grandement impacté les méthodes de recherche des facteurs pronostiques. Les différentes unités de recherche ne se limitent plus aux données issues de leurs services respectifs, mais disposent désormais d'un large panel de données biologiques, de types, de formats et de sources divers, offrant de nouvelles perspectives de recherches cliniques. Cette dynamique a entraîné le développement d'études multi-omiques, où les praticiens exploitent plusieurs types de données moléculaires de haute dimension (telles que les données génomiques pour l'ADN, les transcriptomiques pour l'ARN, les protéomiques pour les protéines, etc.), dans le but de comprendre la biologie sous-jacente de maladies complexes comme le cancer. En revanche, si les applications sont en expansion, la recherche statistique sur l'intégration des données multi-omiques au sein d'algorithmes de Machine Learning peut encore gagner en maturité. Pour y voir plus clair, tournons-nous vers les défis liés à ce type de données.

D'abord rappelons que les données mutli-omiques s'inscrivent dans le cadre de la grande dimension, où le nombre d'individus est largement inférieur au nombre de variables explicatives. En particulier, les données exploitées proviennent généralement d'enquêtes de cohortes comportant quelques centaines d'individus, sur lesquels les informations collectées constituent des centaines de milliers de variables, selon les matrices omiques considérées. Ainsi, outre la nécessité de construire des modèles prédictifs offrant une XXVII précision à la hauteur des enjeux sanitaires, il est essentiel de considérer également le caractère pratique de ces modèles. Un modèle construit sur la base de milliers de variables, mobilisant de nombreuses procédures de collecte, aura une utilité pratique limitée face à un modèle de plus petite taille. La recherche de la parcimonie, et donc la sélection de variable constitue le premier enjeu de ces études cliniques. La littérature statistique renferme un large éventail de méthodes de sélection de variables pour l'analyse en grande dimension. La plus connue, et très saluée par la littérature est la méthode Lasso de Tibshirani [START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF]. Considérons un problème de régression linéaire, où nous souhaitons exprimer une variable Y P R à partir d'un vecteur de p variables explicatives X P R p , par la forme Y " β t X `ϵ où β est le vecteur des coefficients de régression et ϵ est un bruit. En s'appuyant sur un échantillon de n observation indépendantes pY i , X i q i"1,..,n , la méthode Lasso résout le problème de régression pénalisé par la norme

ℓ 1 suivant β " arg min βPR p ´n ÿ i"1 pY i ´βt X i q 2 `λ p ÿ j"1 |β j | ¯, (1)
où λ est un paramètre de pénalisation. Ce problème est équivalent à la minimisation de l'erreur quadratique sous une contrainte de la forme |β| 1 ď s, où s est un paramètre de régularisation. Ainsi défini, la méthode Lasso a des vertus à la fois de shrinkage et de parcimonie, et plus précisément l'estimateur obtenu a un support S " tj; βj ‰ 0u vérifiant cardpSq ď n. Toutefois, si le Lasso offre de bonnes performances lorsque la taille du support du modèle oracle est effectivement inférieur à n, cette méthode ne répond pas intégralement aux contraintes liées aux données mutli-omiques. D'abord, la procédure de sélection de variables s'avère trop restrictive en raison de la taille relativement faible des échantillons issus des enquêtes de cohortes. De plus, lorsqu'il est appliqué à des groupes de variables fortement corrélées, le lasso aura tendance à sélectionner arbitrairement une seule variable du groupe, ignorant les actions conjointes des prédicteurs que l'on peut par exemple retrouver au sein des réseaux de régulation de gènes, ou encore au sein d'associations de traits phénotypiques présents dans les mesures omiques [START_REF] Freijeiro-González | A critical review of lasso and its derivatives for variable selection under dependence among covariates[END_REF][START_REF] Martin | Sharp thresholds for high-dimensional and noisy sparsity recovery using ℓ 1 -constrained quadratic programming (lasso)[END_REF][START_REF] Zhao | On model selection consistency of lasso[END_REF]. Cette limite nous mène au deuxième enjeu des études cliniques, la prise en compte de la structure de corrélation des prédicteurs dans les procédures de sélection de variables.

Cette problématique a été traitée par de nombreux auteurs, et sous différents formats des plus remarquables. Considérons à présent que les prédicteurs sont ordonnés en m groupes, tel que X " pX 1 , ..., X m q P R p . Yuan et Lin ont étudié la méthode Group Lasso effectuant la sélection de variables à l'échelle des groupes, en remplaçant la pénalisation ℓ 1 du Lasso par une pénalisation ℓ 1 -ℓ 2 sur le vecteur de coefficients associé à chaque groupe [START_REF] Yuan | Model selection and estimation in regression with grouped variables[END_REF]. Cette approche fut initiée dans le cadre de la régression linéaire par la minimisation de l'erreur quadratique, et a été par la suite étendue aux fonctions de pertes générales dans l'article de Kim et al. [START_REF] Kim | Blockwise sparse regression[END_REF]. Précisons toutefois que l'approche de sélection par groupes a été initialement pensée pour intégrer des variables catégorielles à la régularisation ℓ 1 . Ainsi, si ces méthodes sont efficaces pour le traitement de petits groupes de variables, elles offriraient un résultat peu sparse sur un groupement omique. Le recours à celles-ci doit justifier une segmentation préalable des variables au sein des groupes omiques, pouvant être réalisée selon une connaissance a priori des interactions entre les prédicteurs, ou encore par le biais d'un clustering [START_REF] Ma | Supervised group lasso with applications to microarray data analysis[END_REF]. Afin de pallier l'excès de générosité du Group Lasso, Simon et al. [START_REF] Simon | A sparse-group lasso[END_REF] ont proposé la méthode Sparse-Group Lasso qui effectue la sélection de variables sur deux niveaux ; d'abord à l'échelle des groupes par la pénalisation ℓ 1 -ℓ 2 , puis à l'échelle des XXVIII variables au sein des groupes non nuls par la pénalisation ℓ 1 . La littérature sur ce sujet est très fournie.

Citons simplement les travaux de Li et al. [START_REF] Li | Multivariate sparse group lasso for the multivariate multiple linear regression with an arbitrary group structure[END_REF], la régression linéaire avec spike and slab de Xu et Gosh [START_REF] Xu | Bayesian variable selection and estimation for Group Lasso[END_REF] ainsi que la généralisation vectorielle de Liquet et al. [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF].

La modélisation à sorties multiples constitue précisément le troisième enjeu que nous souhaitons soulever. Supposons maintenant que nous ayons affaire à une régression linéaire multivariée de la forme

Y " B t X `E, (2)
où Y P R q est le vecteur des réponses, X P R q le vecteur des variables explicatives (éventuellement structurées en groupes), B P R pˆq la matrice des coefficients de régression et E P R p un terme de bruit gaussien multivarié. Le modèle graphique gaussien partiel (PGGM), développé par Sohn et Kim [START_REF] Sohn | Joint estimation of structured sparsity and output structure in multiple-output regression via inverse-covariance regularization[END_REF] ou Yuan et Zhang [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF], apparaît comme un outil puissant pour mettre en évidence les relations entre prédicteurs et réponses par le biais de corrélations partielles (appelées désormais 'liens directs', par opposition aux 'liens indirects' résultant de corrélations). En effet, supposons que le couple pY, Xq P R q`p admet également une distribution gaussienne multivariée de moyenne nulle, de covariance Σ et de matrice de précision Ω " Σ ´1. La décomposition en blocs donnée par

Ω " ˜Ωy ∆ ∆ t Ω x ¸ (3)
avec Ω y P S q ``, ∆ P R qˆp et Ω x P S p ``conduit à Y k | X k " N q p´Ω ´1 y ∆ X k , Ω ´1 y q. Cette remarque est cruciale car on peut voir que la régression à sorties multiples Y k " B t X k `Ek avec un bruit gaussien E k " N q p0, Rq peut être reparamétrée avec

B " ´∆t Ω ´1 y et R " Ω ´1 y . (4)
Ainsi, tandis que ∆ ne contient que les liens directs entre les prédicteurs et les réponses, l'estimation de B est impactée par la structure de corrélation des sorties. Afin d'assurer une sélection de variables pertinentes, Yuan et Zhang [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF] proposent d'approcher B à travers une estimation distincte de ∆ et Ω ´1 y par maximum de vraisemblance pénalisée induisant de la sparsité au sein des matrices ∆ et Ω ´1 y .

Cette approche fut ensuite reprise par Chiquet et al. [START_REF] Chiquet | Structured regularization for conditional Gaussian graphical models[END_REF]. Les auteurs proposent une variante dans laquelle aucune hypothèse de sparsité n'est portée sur Ω y , et centrent leur schéma de régularisation sur les liens directs ∆. En particulier, celui-ci prend la forme de deux pénalisations, une pénalisation ℓ 1 d'une part, et une pénalisation structurante d'autre part induite par une connaissance a priori de la structure de corrélation des prédicteurs.

Avec ce panorama en tête, cette thèse s'est articulée autour de deux axes. Le premier axe, théorique, apporte des contributions à la théorie des modèles graphiques gaussiens partiels. Notre première étude reprend les travaux de Yuan et Zhang [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF] et ceux de Chiquet et al. [START_REF] Chiquet | Structured regularization for conditional Gaussian graphical models[END_REF]. En particulier nous proposons une méthode d'estimation et de sélection de variables, où nous appliquons une pénalisation de type ℓ 1 sur ∆ et Ω y afin d'introduire de la sparsité, et une pénalisation structurante sur ∆ reflétant un a priori gaussien généralisé sur les liens directs. Nous apportons une garantie théorique à notre méthode, XXIX et la mettons en application sur des données synthétiques et un ensemble de données réelles sont menées.

Inspirés par les travaux de Xu et Gosh [START_REF] Xu | Bayesian variable selection and estimation for Group Lasso[END_REF], et Liquet et al. [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF] traitant l'estimation bayésienne de B, notre deuxième étude explore diverses approches bayésiennes au sein des modèles graphiques gaussiens partiels. En suivant la stratégie spike and slab de ces auteurs, nous proposons différents modèles hiérarchiques permettant de traiter des régressions linéaires à sorties uni ou multidimensionnelles, en petite ou grande dimension, selon une configuration saturée, sparse, group sparse ou encore sparse-groupsparse de la matrice ∆. Nous fournissons également une garantie théorique sur nos estimations sparse et group-sparse, et montrons l'efficacité de nos modèles d'abord à travers des différents scénarios de données simulées et puis par l'étude d'un jeu de données réel. Les résultats obtenus sont très compétitifs, notamment en termes de récupération de support.

Le deuxième axe de la thèse a un objectif plus appliqué à l'analyse de survie, mais s'intègre également dans la problématique de sélection de variables. Nous nous situons dans le cadre d'une analyse multi-omique où nous souhaitons exprimer la survie de patientes à partir d'un échantillon d'observations contenant de la censure. Les méthodes de sélection de variables en analyse de survie ont été majoritairement été étudiée dans le cadre des modèles de Cox, traitant différents enjeux liés à l'analyse multi-omique.

Nous pensons notamment à la méthode priority-Lasso de Klau et al. [KJH `18] qui tient compte d'une connaissance a priori de l'usabilité des groupes de variables en pratique, ou encore l'IPF-Lasso de Boulesteix et al. [START_REF] Boulesteix | Ipf-lasso: integrative-penalized regression with penalty factors for prediction based on multi-omics data[END_REF] qui attribue différents facteurs de pénalisation aux groupes afin de contrebalancer le déséquilibre induit par les différences de tailles des groupes. Néanmoins, si le modèle de Cox est de loin le plus répandu dans ces études [START_REF] David R Cox | Partial likelihood[END_REF], les modèles de Cure mériteraient une attention particulière, puisqu'ils sont spécifiquement conçus pour traiter les problématique cliniques où l'on peut vraissemblabement considérer qu'une partie de la population censurée est tout simplement immunisée et n'observera pas l'évènement [START_REF] Boag | Maximum likelihood estimates of the proportion of patients cured by cancer therapy[END_REF]. Cependant, la littérature compte peu d'études de sélection de variable reprenant cette approche.

Nous proposons un algorithme de sélection de variables descendante nommée Stepwise Variable Selection for Survival Analysis (SVSSA) dans ce manuscrit. L'idée est d'assurer la fiabilité de la sélection d'une part en la segmentant en quatre étapes et d'autre part en effectuant un consensus entre des méthodes de régularisation bien établies dans la littérature. En particulier, nous supposons qu'une variable définie comme pertinente par différents modèles de sélections, reposant sur des critères et des approches variés, est vraisemblablement significative. Nous testons cette procédure sur un jeu de données portant sur le cancer du sein triple négatif, en comparant les performances des modèles établis dans la littérature à celles des modèles de Cox et de Cure construits à partir des variables sélectionnées par SVSSA). Les résultats sont très compétitifs, mais la méthode gagnerait en optimisation du temps de calcul.

Organisation du manuscrit

Ce manuscrit s'articule principalement autour de deux axes. Le premier axe, théorique, est composé des trois premiers chapitres dédiés aux modèles graphiques gaussiens. Le deuxième axe, appliqué, se compose quant à lui des deux derniers chapitres, et traite de la sélection de variables en analyse de survie en grande dimension. Plus précisément :

( data for RNA, proteomic data for proteins, etc.), in order to understand the underlying biology of complex diseases such as cancer. On the other hand, even though its applications are expanding, statistical research on the integration of multi-omics data within machine learning algorithms is still far from being a mature field. To get a clearer picture, let us turn to the challenges related to this type of data.

First of all, let us recall that mutli-omics data are part of the high-dimensional framework, where the number of individuals is much lower than the number of explanatory variables. In particular, the data used generally comes from cohort surveys comprising a few hundred individuals, on which the information collected constitutes hundreds of thousands of variables, depending on the omic matrices considered. Thus, in addition to the need to build predictive models offering an accuracy commensurate with the health issues at stake, it is essential to also consider the practicality of these models. A model built with thousands of variables, mobilizing many collection procedures, will have limited practical utility compared to a smaller model. The search for sparsity, and therefore the selection of variables, constitutes the first challenge of these clinical studies. The statistical literature contains a wide range of variable selection methods for high-dimensional analysis. The best known, and highly acclaimed in the literature, is the Lasso method of Tibshirani [START_REF] Tibshirani | Regression shrinkage and selection via the lasso[END_REF]. Consider a linear regression problem, where we wish to express a variable Y P R from a vector of p explanatory variables X P R p , by the form Y " β t X `ϵ where β is the vector of regression coefficients and ϵ is a random noise. Based on a sample of n independent observations pY i , X i q i"1,..,n , the Lasso method solves the following ℓ 1 penalized regression problem β " arg min

βPR p ´n ÿ i"1 pY i ´βt X i q 2 `λ p ÿ j"1 |β j | ¯, (1)
where λ is a penalty parameter. This problem is equivalent to minimizing the sum of squared errors under a constraint of the form |β| 1 ď s, where s is a regularization parameter. Thus defined, the Lasso method has both shrinkage and sparsity virtues, and more precisely the estimator obtained has a support S " tj; βj ‰ 0u verifying cardpSq ď n. However, while Lasso performs well when the support size of the oracle model is indeed less than n, this method does not fully meet the constraints related to mutli-omic data. First, the variable selection procedure turns out to be too restrictive due to the relatively small size of the samples from cohort surveys. Moreover, when applied to groups of highly correlated variables, the Lasso will tend to arbitrarily select a single variable from the group, ignoring the joint actions of predictors that can be found, for example, in gene regulation networks, or in associations of phenotypic traits present in omic measures [START_REF] Freijeiro-González | A critical review of lasso and its derivatives for variable selection under dependence among covariates[END_REF][START_REF] Martin | Sharp thresholds for high-dimensional and noisy sparsity recovery using ℓ 1 -constrained quadratic programming (lasso)[END_REF][START_REF] Zhao | On model selection consistency of lasso[END_REF]. This limitation leads us to the second challenge of clinical studies, taking into account the correlation structure of the predictors in variable selection procedures.

This problem has been treated by many authors, and in various remarkable formats. Suppose we have omic predictors which are arranged into m groups, such that X " pX 1 , ..., X m q P R p . Yuan and Lin studied the Group Lasso method, which performs group-wide variable selection, replacing the Lasso ℓ 1 penalty by a ℓ 1 -ℓ 2 penalty on the vector of coefficients associated with each group [START_REF] Yuan | Model selection and estimation in regression with grouped variables[END_REF]. This approach was initiated in the context of linear regression by minimizing the sum of squared errors, and was later extended to general loss functions in the paper by Kim et al. [START_REF] Kim | Blockwise sparse regression[END_REF]. Note however that the group selection approach was initially designed to integrate categorical variables into the ℓ 1 -regularization. Thus, although these methods are efficient at dealing with small groups of variables, they would not offer a result sparse enough on an omic grouping. The use of these methods must justify a preliminary segmentation of the variables within the omic groups, which can be carried out according to prior knowledge of the interactions between predictors, or even by means of a clustering [START_REF] Ma | Supervised group lasso with applications to microarray data analysis[END_REF]. In order to compensate the excessive generosity of the Group Lasso, Simon et al. [START_REF] Simon | A sparse-group lasso[END_REF] proposed the method Sparse-Group Lasso which performs variable selection on two levels ; first at the group level with the ℓ 1 -ℓ 2 penalty, then at the variable level within the non-zero groups with the ℓ 1 penalty. The literature on this subject is extensive.

In particular we note the works of Li et al. [START_REF] Li | Multivariate sparse group lasso for the multivariate multiple linear regression with an arbitrary group structure[END_REF], the linear regression with spike and slab of Xu and Gosh [START_REF] Xu | Bayesian variable selection and estimation for Group Lasso[END_REF] as well as the vector generalization of Liquet et al. [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF].

The third challenge we want to raise is the need for multi-output modeling. Suppose now that we deal with a multivariate linear regression of the form

Y " B t X `E, (2)
where Y P R q is the vector of responses, X P R q the vector of predictors (possibly structured in groups), B P R pˆq the matrix of regression coefficients and E P R p a multivariate Gaussian noise term. The partial Gaussian graphical model (PGGM), developped e.g. by Sohn and Kim [START_REF] Sohn | Joint estimation of structured sparsity and output structure in multiple-output regression via inverse-covariance regularization[END_REF] or Yuan and Zhang [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF],

appears as a powerful tool to exhibit relationships between predictors and responses that exist through partial correlations (called from now on 'direct links', as opposed to 'indirect links' resulting from correlations). Indeed, assume that the couple pY, Xq P R q`p also admits a multivariate gaussian distribution with zero mean, covariance Σ and precision matrix Ω " Sigma ´1. Then, the block decomposition given by Ω " ˜Ωy ∆

∆ t Ω x ¸(3)
with Ω y P S q ``, ∆ P R qˆp and Ω x P S p ``leads to Y i | X i " N q p´Ω ´1 y ∆ X i , Ω ´1 y q. This is a crucial remark because one can see that the multiple-output regression Y i " B t X i `E¨i with Gaussian noise E k " N q p0, Rq may be reparametrized with

B " ´∆t Ω ´1 y and R " Ω ´1 y . ( 4
)
Thus, while ∆ only contains the direct links between the predictors and the responses, the estimation of B is impacted by the correlation structure of the outputs. In order to ensure a selection of relevant variables, Yuan and Zhang [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF] propose to approximate B through distinct estimations of ∆ and Ω ´1 y by penalized maximum likelihood inducing sparsity within the matrices ∆ and Ω ´1 y . This approach was then taken up by Chiquet et al. [START_REF] Chiquet | Structured regularization for conditional Gaussian graphical models[END_REF]. The authors propose a variant in which no sparsity assumption is made on Ω y , and focus their regularization scheme on the direct links ∆. In particular, the latter takes the form of two penalties, a ℓ 1 penalty on the one hand, and a structured penalty on the other hand induced by a prior knowledge of the correlation structure of the predictors.

With this panorama in mind, this thesis is articulated around two axes. The first axis, theoretical, brings contributions to the theory of partial Gaussian graphical models. Our first study takes up the work of Yuan and Zhang [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF] and that of Chiquet et al. [START_REF] Chiquet | Structured regularization for conditional Gaussian graphical models[END_REF]. In particular, we propose a method of estimation and variable selection, where we apply ℓ 1 -type penalty on ∆ and Ω y to introduce sparsity, as well as structural penalty on ∆ reflecting a generalized Gaussian prior on the direct links. We bring theoretical guarantees to our method, and test it on synthetic and real datasets.

Inspired by the work of Xu and Gosh [START_REF] Xu | Bayesian variable selection and estimation for Group Lasso[END_REF], and Liquet et al. [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF] dealing with Bayesian estimation of B, our second study explores various Bayesian approaches within partial Gaussian graphical models.

Following the spike and slab strategy of these authors, we propose different hierarchical models allowing to process linear regressions with uni or multidimensional outputs, in small or high dimension, according to a saturated, sparse, group-sparse or even sparse-group-sparse configuration of the matrix ∆. We also provide a theoretical guarantee on our sparse and group-sparse settings, and show the efficiency of our models first through different simulated data scenarios and then by studying a real dataset. The results obtained are very competitive, especially in terms of support recovery.

The second axis of the thesis is related to survival analysis, but is also linked to the problem of variable selection. We are in the context of a multi-omics analysis where we want to express patient survival times from a sample containing censored observations. Variable selection methods in survival analysis have been mainly studied in the context of Cox models, dealing with different issues related to multi-omics analysis. We think in particular of the priority-Lasso method of Klau et al. [KJH `18] which takes into account a usability criterion. In addition to seeking sparsity, it also prioritize selecting variables that are more routinely or more easily assessed. We should also mention the IPF-Lasso de Boulesteix et al. [START_REF] Boulesteix | Ipf-lasso: integrative-penalized regression with penalty factors for prediction based on multi-omics data[END_REF] which assigns different penalization factors to groups in order to handle the imbalance induced by the differences in group sizes. Even though the Cox model is by far the most widespread in these studies [START_REF] David R Cox | Partial likelihood[END_REF], the Cure models deserve special attention, since they are specifically designed to deal with clinical studies where one can assume that part of the censored population is simply immune and will not observe the event [START_REF] Boag | Maximum likelihood estimates of the proportion of patients cured by cancer therapy[END_REF]. However, there are few variable selection studies in the literature that take this approach.

We propose a stepwise variable selection algorithm named Stepwise Variable Selection for Survival Analysis (SVSSA) in this manuscript. The idea is to ensure the reliability of the selection firstly by segmenting it into four stages et also by making a consensus between regularization methods well established in the literature. In particular, we assume that a variable defined as relevant by different selection models, based on various criteria and approaches, is likely to be significant. We test this procedure on a Triple Negative Breast Cancer dataset, comparing the performances of the models established in the literature with those of the Cox and Cure models built from the variables selected by SVSSA. The results are very competitive, but the computation time must be improved.

Outline

This manuscript is mainly articulated around two axes. The first axis, theoretical, is composed of the first three chapters dedicated to Gaussian graphical models. The second axis, applied, is composed of the last two chapters, and deals with variable selection in high-dimensional survival analysis. More precisely:

(1) Chapter 1 provides a brief introduction to graph theory, and in particular to Gaussian graphical models. We start by defining the basic concepts of graph theory, and highlight the relationship between the structure of conditional independences of the variables and the precision matrix of the graphical model. We continue with a presentation of Gaussian graphical models and their advantages in recovering direct links between predictors and responses compared to standard linear regression models. Next, we present the penalized partial Gaussian graphical model introduced by Sohn and Kim [START_REF] Sohn | Joint estimation of structured sparsity and output structure in multiple-output regression via inverse-covariance regularization[END_REF] and Yuan and Zhang [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF], and its advantages in terms of accuracy and computational time. Finally, we discuss the Bayesian literature on high-dimensional Gaussian graphical models.

(2) Chapter 2 presents the results obtained in the first article of this thesis, published in the journal ESAIM: Probability and Statistics [START_REF] Okome | A partial graphical model with a structural prior on the direct links between predictors and responses[END_REF]. It is devoted to the estimation of a PGGM with a structural penalty on the direct links. After a presentation of the literature that addresses this problem, we formalize the form of our generalized Gaussian penalization and expose the new objective function of our PGGM. We continue on the theoretical guarantees of our estimator, and detail the proof of the main theorem. Finally, we test the performance of our method first by empirical studies on synthetic data, then by processing a real dataset, before concluding on possible improvements.

(3) Chapter 3 presents the results obtained in the second article of this thesis, published in the journal Bayesian Analysis [START_REF] Okome | A bayesian approach for partial gaussian graphical models with sparsity[END_REF]. After a presentation of the works that preceded us in the literature, we formalize our hierarchical models. We continue on the theoretical guarantees associated with sparse and group-sparse configurations. Then test our methods on Gibbs samplers and a real dataset, before concluding.

(4) Chapter 4 serves as an introduction to survival analysis. We begin by defining the framework and the specific problems of survival models, and continue on the main methods for estimating such models in the non-parametric, semi-parametric and parametric frameworks. We then address the issue of variable selection in survival analysis, through penalized estimation methods, and conclude with a discussion of Cure models and their specificities for modeling clinical events.

(5) Chapter 5 presents the last results of this thesis, namely a stepwise variable selection method suitable for multi-omics studies. The first part of the chapter focuses on detailing the architecture of our algorithm, in particular by providing a brief description of the different methods that compose it.

The second part is an application on real data on triple negative breast cancer, in which we compare the results of our approach with well-established regularization methods, according to the relevance of the selection, the predictive performances, and the computation time.

Chapter 1

INTRODUCTION TO GAUSSIAN GRAPHICAL MODELS

T his introductory chapter deals with Gaussian graphical models (GGM). It describes the general framework in which the methods presented in Chapters 2 and 3 are embedded. Throughout these introductory developments, we will rely mainly on the books by Lauritzen [START_REF] Steffen L Lauritzen | Graphical models[END_REF], Whittaker [START_REF] Whittaker | Graphical models in applied multivariate statistics[END_REF] and Maathuis et al. [START_REF] Maathuis | Handbook of Graphical Models[END_REF], to which we refer the reader for a further introduction to graph theory.

We will however borrow some additional references, these will be presented in due time.

Elementary concepts

In this section, we briefly recall the basic notions needed to understand graph theory, and in particular Gaussian graphical models.

Conditional independence

Consider the random vector X " pX 1 , ..., X p q with probability density f X ą 0, and set Γ " t1, ..., pu. For any set A Ă Γ, we denote X A the random vector defined by pX i q iPA and f X A its probability density.

Definition 1.1.1 (Independence). Let A, B be two subsets of Γ, and X A , X B the associated random vectors. X A and X B are independent if and only if

f pX A ,X B q px A , x B q " f X A px A qf X B px B q, @ x A , x B . (1)
This is denoted by

X A K K X B .
Definition 1.1.2 (Conditional independence). Let A, B, C be three subsets of Γ, and X A , X B , X C the associated Gaussian vectors. X A and X B are conditionally independent on X C if and only if

f pX A ,X B q|X C px A , x B ; x C q " f X A |X C px A ; x C qf X B |X C px B ; x C q ðñ f X A |X B ,X C px A ; x B , x C q " f X A |X C px A ; x C q, @ x A , x B , x C .
(

This is denoted by

X A K K X B |X C .
The conditional independence describes the fact that as soon as we know X C , the knowledge of X B will not bring any additional information about X A . We will see later that this notion is fundamental in graph Chapter 1 -Introduction to Gaussian graphical models theory.

Partial correlation

Definition 1.1.3 (Correlation). Let X a and X b be two random variables with strictly positive variances. The linear correlation coefficient between X a and X b is given by

ρpX a , X b q " CovpX a , X b q pVarpX a qVarpX b qq 1 2
.

(3) Definition 1.1.4 (Partial correlation). Let X a , X b and X c be three random variables with strictly positive variances. The (linear) correlation coefficient between X a and X b conditional on X c is given by

ρpX a , X b |X c q " CovpX a , X b |X c q pVarpX a |X c qVarpX b |X c qq 1 2 . ( 4
)
The correlation coefficient expresses the degree of linear relationship between two variables, while the partial correlation coefficient measures the relationship between the two variables conditionally on others.

Correlation and independence are closely related. Independence between two variables implies a zero correlation between them, but the converse is only true in certain special cases, in particular the Gaussian case.

Consider now the Gaussian random vector X " pX 1 , ..., X p q with zero mean and covariance Σ P S p ``, and set Σ ´1 " Ω and Γ " t1, ..., pu. The density of X is given by

f X pxq " 1 p2πq p 2 a detpΣq exp ˆ´xx, Ωxy 2 ˙, @ x P R p . ( 5
)
For any set A Ă Γ, we denote X A the Gaussian vector defined by pX i q iPA , and for more clarity we will denote B the complementary subset of A in Γ, X B being the associated Gaussian vector (i.e. B " ΓzA and X B " pX i q iPΓzA ). X, Σ and Ω thus satisfy the following decompositions into blocks

X " ˜XA X B ¸, Σ " ˜ΣA Σ AB Σ t AB Σ B ¸et Ω " ˜ΩA Ω AB Ω t AB Ω B ¸ (6)
Proposition 1.1.5 (Marginal and conditional density). Let X, X A , X B , Σ and Ω be defined and decomposable as above,

(a) the marginal distribution of X A is N p0, Σ A q ; (b) the conditional probability distribution of X A given X B " x B is N p´Ω ´1 A Ω AB x B , Ω ´1 A q.
Proof. We only focus here on the proof of (b), that of (a) being well known. Using the Schur complement

Ω " ˜pΣ A ´ΣAB Σ ´1 B Σ t AB q ´1 ´pΣ A ´ΣAB Σ ´1 B Σ t AB q ´1Σ AB Σ ´1 B ´Σ ´1 B Σ t AB pΣ A ´ΣAB Σ ´1 B Σ t AB q ´1 Σ ´1 B `Σ ´1 B Σ t AB pΣ A ´ΣAB Σ ´1 B Σ t AB q ´1Σ AB Σ ´1 B " ˜ΩA ´ΩA M ´M t Ω A Σ ´1 B `M t Ω A M ¸, with M " ´Ω ´1 A Ω AB " Σ AB Σ ´1 B . For x B fixed, the conditional density of X A giving X B " x B thus satisfies f X A |X B px A ; x B q " f X px A , x B q f X B px B q 9 exp ˆ´1 2 `px A , x B q t Ω px A , x B q ´x t B Σ ´1 B x B ˘9 exp ˆ´1 2
`x t A Ω A x A `x t B Ω t AB x A `x t A Ω AB x B `x t B Ω B x B ´x t B Σ ´1 B x B ˘9 exp ˆ´1 2 `x t A Ω A x A ´2 x t A Ω A M x B `x t B pΣ ´1 B `M t Ω A M q x B ´x t B Σ ´1 B x B ˘9 exp ˆ´1 2 `px A ´M x B q t Ω A px A ´M x B q `x t B Σ ´1 B x B ´x t B Σ ´1 B x B ˘9 exp ˆ´1 2 x px A ´M x B q, Ω A px A ´M x B q y ˙.
We obtain the form of the desired density.

Since the independence between Gaussian variables can be shown by the nullity of the correlation coefficients, and therefore of the covariances, we can deduce from proposition 1.1.5 the following corollary.

Corollary 1.1.6. Let X, Γ, Σ and Ω defined as above, @ i, j P Γ,

(a) X i K K X j ðñ Σ ij " 0 ; (b) X i K K X j |X Γzti,ju ðñ Ω ij " 0.

Proof. (b)

Let pose A " ti, ju. The conditional covariances can be read on Ω ´1 A , or Let A " ti, ju.

Conditional covariances read on Ω ´1

A , where

Ω ´1 A " 1 detpΩ A q ˜Ωjj ´Ωij ´Ωji Ω ii ¸.
Thus, CovpX i , X j |X Γzti,ju q " 0 ðñ pΩ ´1 A q 12 " 0 ðñ Ω ij " 0.

Graph -notations and terminology

Definition 1.1.7 (Graph). A graph G is a pair G " pΓ, Eq, where Γ is a finite set of elements called vertices or nodes, and E is a set of edges composed of pairs of elements taken from Γ.

A graph G is therefore a set of nodes connected by edges. However, there are several types of graphs defined by the characteristics of their vertices and/or their edges. In this chapter, we will limit ourselves to so-called simple graphs, which are characterized by the absence of loops and multiple edges. In other words, these are graphs that have no more than one edge between a pair of vertices and in which the edges start and end on different vertices. To go further, we refer the interested reader to the beautiful introductions to graph theory offered by the books of [W `01] and [START_REF] Gross | Graph theory and its applications[END_REF].

Definitions 1.1.8 (Undirected edge and graph). Let G " pΓ, Eq be a graph, (i) An edge pi, jq of G is said to be undirected if pi, jq P E and pj, iq P E. It is then denoted by i, Ø j.

(ii) G is said to be undirected (UG) when all its edges are undirected.

Definitions 1.1.9 (Directed Edge and graph). Let G " pΓ, Eq be a graph, (i) An edge pi, jq of G is said to be directed if pi, jq P E et pj, iq R E. It is then denoted by i Ñ j.

(ii) G is said to be directed (DG) when all its edges are directed.

Depending on whether it is directed or undirected, a graph is associated with a specific vocabulary.

In an undirected graph, for any edge i Ø j in E, we will say that i and j are respective neighbors, and we will denote ne G piq " tj P Γ : pi, jq P Eu the set of neighbors of i in G. In the case of a directed graph, for any edge i Ñ j in E, we will say that i is the parent of j (resp. j is the descendant of i), and we will denote pa G pjq " ti P Γ : pi, jq P Eu the set of parents of j in G (resp. de G piq " tj P Γ : pi, jq P Eu).

However, it is possible to harmonize this information whatever the graph using the adjacency matrix defined by

A " pA ij q i,jPΓ , A ij " $ & % 1 if pi, jq P E 0 otherwise. (7)
This one offers a simple representation of a graph that we will know oriented or not depending on whether its adjacency matrix is symmetric. The Figure 1.1 represents a directed graph and its undirected version with their adjacency matrices.

We will see later in the chapter that depending on the phenomenon being studied, it can be more convenient to manipulate undirected graphs than directed ones. It is therefore appropriate to introduce now on the the graph moralization criterion which enables to find the equivalent undirected form of an acyclic directed graph. is the undirected graph G m " pΓ, E m q, such that the set of edges E m is constructed as follow 1 0 1 1 0 0 1 0 0 2 1 0 1 0 0 0 0 0 3 1 1 0 1 0 0 0 0 4 0 0 1 0 1 0 0 0 5 0 0 0 1 0 1 1 0 6 1 0 0 0 1 0 1 0 7 0 0 0 0 1 1 0 0 8 0 0 0 0 0 0 0 0 1 2 3 4 5 6 7 8 1 0 0 1 0 0 0 0 0 2 1 0 1 0 0 0 0 0 3 0 0 0 0 0 0 0 0 4 0 0 1 0 1 0 0 0 5 0 0 0 0 0 1 1 0 6 1 0 0 0 0 0 1 0 7 0 0 0 0 0 0 0 0 8 0 0 0 0 0 0 0 0

@i, j P Γ, i Ø j ðñ i Ñ j ou j Ñ i ou tDk P Γ : i Ñ k et j Ñ ku. ( 8
)

Gaussian graphical models

In this section we will discuss the theoretical aspect of Gaussian graphical models. Let's take again our p-dimensional Gaussian random vector X, with zero mean, covariance Σ P S p ``, and precision matrix Ω " Σ ´1, and consider a graph G " pΓ, Eq with Γ " t1, ..., pu. A Gaussian graphical model of X with respect to G is a probabilistic model, in which the graph G expresses the conditional dependence structure between the random variables of X. In particular, if G is undirected, it will contain the structure of correlations of the variables of X, whereas if it is directed it will contain the causal relations. We will see in this section that the definition of a graphical model relies on the respect of the pairwise Markov property, and we will study its interpretation in the framework of oriented and non-oriented graphs.

Markov properties for undirected graphs

Markov properties distinguish conditional independence structures represented by a graph, by exploiting the factorization 2 for different sets C on which the conditioning is based. Definition 1.2.1. Let be a random vector X and an undirected graph G. The vector X satisfies the pairwise Markov property if for any pair of nodes i, j in Γ,

i Ü j ðñ X i K K X j |X Γzti,ju . ( 9
)
Definition 1.2.2. Let be a random vector X and an undirected graph G. The vector X satisfies the local Markov property if for any nodes i in Γ,

X i K K X Γzne G piqYtiu |X ne G piq . ( 10
)
X A K K X B |X C . ( 11
)
One can easily see that the global Markov property implies the local property which implies the pairwise property, and in well-defined cases, when the density of X is strictly positive, the reciprocal implications are also verified (see observation 1.7.1 of [START_REF] Maathuis | Handbook of Graphical Models[END_REF]). Thus, a Gaussian vector satisfying one of the Markov properties, satisfies them all.

Markov properties for directed acyclic graphs

Since the Markov properties arise from the factorization of the joint distribution over a set of conditional distributions, the presence of cycles in directed graphs setting poses a problem in this writing. Indeed, if we consider the loop 1 Ñ 2 Ñ 3 Ñ 1, there is no coherent way to factorize the distribution of the vector of variables indexed by these nodes, since no node is guaranteed to d-separate its parents from its descendants. Therefore, we will restrict ourselves in this section to directed acyclic graphs. The factorization 2 is then slightly modified in the context of DAGs, in particular for any node i in Γ, the random variable X i is associated to a conditional distribution with respect to the variables corresponding to the parents of i in the DAG. The joint distribution now verifies

f pX 1 , ..., X n q " ź iPΓ f Xi|X papiq . ( 12
)
Definition 1.2.4. Let be a random vector X and a directed acyclic graph G. The vector X satisfies the pairwise Markov property if for any pair of nodes i, j in Γ,

i Û j ðñ X i K K X j |X Γzde G piqYti,ju . ( 13
)
Definition 1.2.5. Let be a random vector X and a directed acyclic graph G. The vector X satisfies the local Markov property if for any node i in Γ,

X i K K X Γzde G piqYtiu |X pa G piq . ( 14
)
G m associated with G, A is separated from B given C), X A K K X B |X C . ( 15
)
The Markov properties associated with DAGs offer a more complex interpretation than those associated with UGs. In order to unify the two cases, we will exploit Theorem 3.5.2 of [START_REF] Whittaker | Graphical models in applied multivariate statistics[END_REF] which states that a DAG satisfies the same Markov properties as its associated moral graph. Moreover, as we do not study the causal structures contained in DAGs, we will only be interested in the correlation structures contained in the associated moral graphs. Thus, in the rest of the manuscript we will focus on the case of undirected graphs.

Gaussian graphical models

Now that we have stated the properties of Markov, we can present the definition of a Gaussian graphical model.

Definition 1.2.7 (Gaussian graphical models). Let be a Gaussian random vector X and an undirected graph G. The distribution of X is a Gaussian graphical model with respect to G if it satisfies the pairwise Markov property.

It follows from this definition that the minimal graph G ˚for which X satisfies the Markov properties describes the sparsity pattern of the precision matrix of X. In particular,

A ij " A ji " 0 ðñ Ω ij " Ω ji " 0, ( 16
)
where A is the adjacency matrix of G ˚. Thus, the estimation of the graph G ˚, and thus of the conditional independence structure of X, requires the estimation of the precision matrix Ω. There is a large variety of methods for estimating Ω, in this introduction we will discuss the maximum likelihood estimation and the Bayesian estimation.

Maximum likelihood estimation

Consider a sample of n independent observations random vectors according to our multivariate pdimensional Gaussian distribution. Let us denote X i P R p the values taken by individual i on the p random variables of X, and X " pX t i q 1ďiďn the matrix of observed data, of dimension n ˆp. The Chapter 1 -Introduction to Gaussian graphical models likelihood of associated to the data is given by

L n pΩq " n ź i"1 f X pX i q " n ź i"1 1 p2πq p 2 a detpΣq exp ˆ´xX i , Ω X i y 2 " " detpΩq
p2πq p ȷ n 2 exp ˆ´xxX t X, Ωyy 2 ˙, (17)
with xx¨, ¨yy refers to the Frobenius scalar product (i.e. xxA, Byy " trpA t Bq). We define S P R pˆp , the empirical covariance matrix of the sample under the assumption of centered data, by

S " 1 n n ÿ i"1 X i X t i . ( 18
)
We can rewrite the likelihood with this quantity, which gives

L n pΩq " " detpΩq p2πq p ȷ n 2 exp ˆ´n 2 xxS, Ωyy ˙. (19)
The estimator of Ω is obtained by maximizing L n pΩq. In order to simplify the problem in an additive form, we will instead use the equivalent optimization problem which consists in minimizing the negative log-likelihood. Theorem 1.3.1 raises two problems. First, the maximum likelihood estimator does not exist in highdimensional setting, but also the latter is not sparse in the classical case. This method of estimation does not meet the needs generally encountered in practice, in particular the treatment of data with a number of predictors largely superior to the number of observations, and for which we are looking for a sparse solution.

Penalized Gaussian graphical models

The respect of the criterion of parsimony is a major concern when creating a high dimensional model. We are looking for a relevant solution involving a minimum of parameters. Many studies dealing with this issue have been done in the context of Gaussian graphical models. The idea is to select a sparse undirected graph, which results in the estimation of a sparse precision matrix, its zeros expressing conditional independences between the variables. We can cite for example the traditional approach, greedy stepwise forward-backward selection, which consists in the iterative selection and/or deletion of the neighbors of each node using multiple tests (e.g. [START_REF] Arthur | Covariance selection[END_REF][START_REF] Wermuth | Analogies between multiplicative models in contingency tables and covariance selection[END_REF][START_REF] Edwards | Introduction to graphical modelling[END_REF]). Another approach is to represent each variable as a linear combination of the other variables of X, and then to perform a penalized regression (a Lasso-type for example) in order to recover the set of neighbors of each node of the graph (e.g.

[MB06, CLL11]).

In this section, we will focus on the penalized maximum likelihood approach. The idea is to take the optimization problem (21) to which we add a penalty on the precision matrix. This means finding Ω P S p `ẁhich minimizes the convex objective function ℓℓ n pΩq " ´ln detpΩq `xxS, Ωyy `λ penpΩq.

(

) 23
The function penpΩq is the penalty function applied to the precision matrix. When one wishes to induce sparsity within it, |Ω| 1 the sum of the absolute value of each element of Ω, or |Ω| 1 this sum deprived of the diagonal elements, are well-known as natural choices. However, note that if some authors use the penalty |Ω| 1 , the estimation algorithms they present are built so as not to penalize the diagonal elements. The parameter λ enables to give more or less weight to the penalty function, thus varying the degree of sparsity and shrinkage within the precision matrix. When λ " 0 we obtain the maximum likelihood estimator.

The practitioner can choose the value of this parameter in order to obtain an estimator Ω P S p ``, but also according to the desired degree of sparsity, or the one considered optimal according to a model selection criterion (e.g. AIC, BIC, cross-validation error).

The idea of using the ℓ 1 regularization as a penalty function is clearly inspired by Tibshirani's Lasso tibshirani96, in order to take advantage of the sparsity and shrinkage qualities related to this estimator.

Efficient algorithms exist to find solutions to the optimization problem (23). Let us cite some of them. which consists in a recursive update of the maximum likelihood estimator. At each iteration, they consider the matrix W pj´1q , which is the estimator updated in the previous step, and the empirical covariance matrix S which verify the following decompositions:

W pj´1q " ˜Wzj W j pW j q t W jj ¸and S " ˜Szj S j pS j q t S jj
where A zj corresponds to the matrix A deprived of the row and the column j, and A j corresponds to the column j of A deprived of the diagonal element A jj . In order to update W pjq , they then estimate W j by solving the following quadratic problem: W j " arg min y ty t pW pj´1q zj q ´1 y : }y ´Sj } 8 ď λu.

(25)

This estimation method echoes that of Meinhausen and Bühlmann [START_REF] Meinshausen | High-dimensional graphs and variable selection with the lasso[END_REF], however the authors guarantee here the uniqueness of the solution obtained at each regression, and the estimator is updated iteratively until convergence. This algorithm offers a computational complexity OpKp 4 q where K denotes the number of iterations before convergence. Once the reference format of the Nesterov problem is obtained, the first order method is articulated in two steps. First, a smooth approximation of the objective function f is constructed by adding a convex penalty on u ; then the first order optimization algorithm, presented in [START_REF] Yurii | A method for solving the convex programming problem with convergence rate Op1{k 2 q[END_REF], is applied to this new function. This algorithm offers a complexity Opp 4.5 {ϵq where ϵ ą 0 denotes the desired accuracy on the optimization problem (23).

Nesterov's first order method by

Block coordinate descent by

W j " min x |Q x ´b| 2 2 `λ|x| 1 , ( 27
)
with Q the square root of W zj and b " 1 2 Q ´1 S j . The estimation algorithm becomes mainly a recursive Lasso, where at each step the diagonal element is omitted from the regression problem to prevent it from being penalized. The computational complexity of this algorithm is Opp 3 q under a dense model (i.e. the covariance matrix is full), and is less in the sparse case. To complete this non-exhaustive list we can also mention the neighborhood selection of Meinshausen and Bühlmann [START_REF] Meinshausen | High-dimensional graphs and variable selection with the lasso[END_REF], which consists in searching for the neighbors of each variable by performing a Lasso regression expressing the latter as a function of the other variables ; or the smooth minimization approach of Lu [START_REF] Lu | Smooth optimization approach for sparse covariance selection[END_REF], also based on Nesterov's first order method, but offering a more attractive complexity than the algorithm of the authors of [START_REF] Banerjee | Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data[END_REF].

In addition to the properties of these algorithms reported in each article, we can find the theoretical guarantees associated with this type of penalization in the paper of Ravikumar et al. [START_REF] Ravikumar | High-dimensional covariance estimation by minimizing ℓ 1 -penalized log-determinant divergence[END_REF].

Partial Gaussian graphical model

In this section we will present the partial Gaussian graphical model (PGGM), introduced by Sohn and Kim [START_REF] Sohn | Joint estimation of structured sparsity and output structure in multiple-output regression via inverse-covariance regularization[END_REF], and Yuan and Zhang [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF] . We will first formulate the motivation of this model by highlighting the link between linear regression and GGM, then we will discuss its theoretical and inferential aspects.

Link with linear regression

We are now in the linear regression setting where we want to express multiple responses Y P R q from a set of predictors X P R p . More precisely we are looking for the matrix of regression coefficients

B P R pˆq such that Y " B t X `E, (28)
with E " N q p0, Rq a Gaussian noise. Let Z " pY, Xq " N q`p p0, Σq be the Gaussian vector associated to the joint distribution of Y and X. The distribution of Z is a Gaussian graphical model for which the associated graph is determined according to the conditional independence structure provided by the precision matrix Ω " Σ ´1. We observe once again the following block decompositions

Z " ˜Y X ¸, Ω " ˜Ωy ∆ ∆ t Ω x ¸and Σ " ˜Σy Σ yx Σ t yx Σ x
where Ω y P S q ``, ∆ P R qˆp and Ω x P S p ``, and where the same goes for Σ x , Σ yx and Σ x . Moreover by Schur, the precision matrix Ω satisfies, by blockwise inversion,

Ω ´1 y " Σ y ´Σyx Σ ´1 x Σ t yx and ∆ " ´pΣ y ´Σyx Σ ´1 x Σ t yx q ´1 Σ yx Σ ´1 x . ( 29
)
If for an individual i we observe the values X i on the p predictors of X, the proposition 1.1.5 notifies that the conditional distribution of Y i given X i satisfies

Y i | X i " N p´Ω ´1 y ∆ X i , Ω ´1 y q. ( 30
)
By associating the conditional distribution (30) with the linear writing (28), we obtain

B " ´∆ t Ω ´1 y , R " Ω ´1 y . ( 31
)
The reparametrization (31) gives a new light on the multiple-output regression. Whereas B contains direct and indirect links between the predictors and the responses (due e.g. to strong correlations among the variables), ∆ only contains direct links, as it is shown by the graphical models theory. In other words, the direct links are closely related to the concept of partial correlations between X and Y . For example, the direct link between predictor k and response ℓ may be evaluated through the partial correlation

CorrpY ℓ , X k | Y ‰ ℓ , X ‰ k q contained,
apart from a multiplicative coefficient, in the ℓ-th row and k-th column of ∆ (see section 1.1.2), with the particularly interesting consequence that the support of ∆ is sufficient to identify direct relationships between X and Y . Thus, when Y is multidimensional, it makes more sense to study the conditional independence between predictors and responses through ∆, B being influenced by the indirect links that propagate through Ω y .

In this vein, Sohn and Kim [START_REF] Sohn | Joint estimation of structured sparsity and output structure in multiple-output regression via inverse-covariance regularization[END_REF] and Yuan and Zhang [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF] proposed a partial estimation of the Gaussian graphical model (PGGM). The objective reduces to the estimation of the direct links ∆ together and the conditional precision matrix of the responses Ω y , possibly penalized. This approach has a huge advantage over GGMs from an inferential point of view. Since we are not interested in partial correlations between the predictors, we do not need to impose any assumption on the structure of X, notably its degree of sparsity. Moreover, since the size of X is, in general, much larger than that of Y (e.g. omics data processing, imaging, NLP), the estimation of the full precision matrix Ω is highly impacted by the estimation of Ω x . Therefore, the bias induced in the estimation of Ω x , due in particular to the difficulties related to the high dimensional analysis, will affect the estimates of ∆ and Ω y . Finally, it is obvious that the computational complexity, which we have not discussed so far since it is closely related to the estimation algorithms, will be strongly reduced with a partial estimation.

Penalized partial likelihood

Let's consider again our n observations. This time for each individual we observe the pair pY i , X i q of values taken on the predictors and the responses. The empirical covariance S satisfies the following block decomposition

S " ˜Sy S yx S t yx S x with S y " 1 n n ÿ i"1 Y i Y t i , S yx " 1 n n ÿ i"1 Y i X t i and S x " 1 n n ÿ i"1 X i X t i .
In the following Proposition, L denotes the negative of the logarithm of the likelihood function corresponding to the GGM.

Proposition 1.5.1 (Proposition 1 of [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF]). Under the transformation Ωx " Ω x ´∆ t Ω ´1 y ∆ we have, LpΩ y , ∆, Ω x q " LpΩ y , ∆, Ωx q " L pa pΩ y , ∆q `Hp Ωx q, (32

)
where Hp Ωx q " ´ln detp Ωx q `xxS x , Ωx yy and L pa pΩ y , ∆q " ´ln detpΩ y q `xxS y , Ω y yy `2xxS yx , ∆yy `xxS x , ∆ t Ω ´1 y ∆yy (33)

Moreover, L pa pΩ y , ∆q is convex.

Proof. Refer to the proof of the proposition provided in Appendix A.1 of [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF].

The proposition 1.5.1 enables to decompose the optimization problem (21) into two distinct problems by the reformulation (32). Since L is jointly convex, Ω is obtained on the one hand by finding the estimators of Ω y and ∆ by minimizing the partial likelihood L pa , and on the other hand by finding the estimator of Ωx , from which we will be able to deduce the estimator of Ω x , by minimizing the function H. After having taken care to reintegrate the penalty functions, we obtain the new convex objective function of the PGGM,

L pa pΩ y , ∆q " ´ln detpΩ y q `xxS y , Ω y yy `2 xxS yx , ∆yy `xxS x , ∆ t Ω ´1 y ∆yy `λ penpΩ y q `µ penp∆q (34)
defined over pΩ y , ∆q P Θ " S q ``ˆR qˆp for some usual penalty functions. It is worth noting that penp∆q often plays a crucial role in modern statistics dealing with high-dimensional predictors, and the natural choice is |∆| 1 to get sparsity. However, since the number of responses is usually small, we can choose λ " 0 and impose no sparsity structure within Ω y . In the seminal papers [START_REF] Sohn | Joint estimation of structured sparsity and output structure in multiple-output regression via inverse-covariance regularization[END_REF] and [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF], the authors consider |Ω y | 1 and |Ω y | 1 for penpΩ y q, respectively. Consider θ " pΩ y , ∆q P Θ the matrix of dimension pqˆpq`pqq of the model parameters. The estimator p θ of θ is obtained by solving the following optimization problem

p θ " p p Ω y , p ∆q " arg min θPΘ L pa pΩ y , ∆q. ( 35
)
Yuan and Zhang propose in [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF] an algorithm for estimating the problem (35) with complexity Opp 3 `p2 q`pq mintn, quq. It is competitive against its counterparts in the GGM setting, the best having a complexity Oppp `qq 3 q (see section 1.4). This algorithm adopts a block coordinate descent approach, that exploits the joint convexity property of (35) by alternating between solving the following two subproblems Ω pt`1q y " arg min ΩyPS q ``L pa pΩ y , ∆ ptq q et ∆ pt`1q " arg min

∆ P R qˆp L pa pΩ pt`1q y , ∆q. (36)
Yuan and Zhang have also provided in [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF] theoretical guarantees on the penalized PGGM estimator, which takes the form of an upper bound on the estimation error. More precisely, they show that when n is large and p " n, with high probability, the estimation error is given by

} p θ ´θ ˚}F À c |S| ln p n , ( 37
)
where θ ˚are the true parameters of the model, and |S| is the cardinal of the support of θ ˚. We will re-demonstrate this result in Chapter 2 when we will provide the model with an additional structural regularization.

Moreover, while Bayesian GGM has been extensively studied in the literature (see e. n the tree of high-dimensional regression problems, many of them embed a correlation structure within the predictors. This is notably the case in genomics with gene regulatory networks that provide information about interactions between genes and their joint actions within cells ; similarly, in signal theory predictors often represent a continuous phenomenon so that consecutive variates act together, this can be observed for example with adjacent pixels in an image. Well-known methods in the literature, such as the Group Lasso of Yuan and Lin [START_REF] Yuan | Model selection and estimation in regression with grouped variables[END_REF], the blockwise sparse regression of Kim et al. [START_REF] Kim | Blockwise sparse regression[END_REF], or the sparse-group Lasso of Simon et al. [START_REF] Simon | A sparse-group lasso[END_REF], integrate in their estimation process a structuring in groups of variables supposedly correlated. One of the main limitations of these methods is their lack of flexibility in group construction. Indeed, each variable is assigned to one group, assuming that it has no significant correlation with variables in other groups. To get around this assumption, which in practice is not always verified (e.g. image processing, genetic data, etc.), studies have been carried out to extend this problem by allowing overlapping groups. To that extent we can cite the CAP penalties of Zhao et al. [START_REF] Zhao | The composite absolute penalties family for grouped and hierarchical variable selection[END_REF], the Graph Lasso and Group Lasso with overlap of Jacob et al. [START_REF] Jacob | Group lasso with overlap and graph lasso[END_REF], the Structured-Lasso and Intersected Structured-Lasso of Jenatton et al. [START_REF] Jenatton | Structured variable selection with sparsity-inducing norms[END_REF], or the Tree-Guided Group Lasso by Kim and Xing [START_REF] Kim | Tree-guided group lasso for multi-response regression with structured sparsity, with an application to eqtl mapping[END_REF]. Although some of these methods borrow elements from graph theory to define complex groupings of predictors, the interactions between the responses remain unexplored. In this vein, graphical models naturally appear as a way to explore, and more precisely Gaussian graphical models with a structural penalty. This is precisely the subject of the article [CMHR17] by Chiquet [START_REF] Chiquet | Structured regularization for conditional Gaussian graphical models[END_REF]. We bring two types of penalty to our optimization problem: a standard ℓ 1 penalty on ∆ and Ω y to induce sparsity within the partial correlations, and possibly within the responses precision matrix ; and a structural penalty reflecting a generalized Gaussian prior on the direct links, which guides the sparsity pattern from a prior knowledge of the interactions between predictors. The second axis aims at providing a theoretical guarantee for our method, taking the form of an upper bound on the estimation error arising with high probability, provided that the model is suitably regularized. This chapter is organized in four parts. In Section 2.1 we start with a reminder of the mathematical formalization of PGGM proposed by [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF]. In Section 2.2, we introduce the model, consisting in putting a generalized Gaussian prior on the direct links before the procedure of estimation of Ω y and ∆, and we detail the new objective function. Then, in Section 2.3 we provide error bounds for our estimates and prove our results in Section 2.4. Section 2.5 is devoted to empirical considerations. We explain how we deal with the minimization of the new objective and we test the method on simulations first, and next on a real dataset (a Canadian average annual weather cycle, see e.g. [START_REF] Ramsay | Functional Data Analysis[END_REF]). Finally, we close the chapter with a short conclusion in the section 2.6.

Introduction

Let us consider, now and in all the study, the sample of n independent observations pY i , X i q, and the empirical covariances denoted by

S y " 1 n n ÿ i"1 Y i Y t i , S yx " 1 n n ÿ i"1 Y i X t i et S x " 1 n n ÿ i"1 X i X t i . ( 1
)
We are in the PGGM setting discussed in Section 1.5. Without going into the details already covered, let's take up the objective function (34), where we consider the penalties penpΩ y q " |Ω y | 1 and penp∆q " |∆| 1

which correspond to the PGGM (Gm) of [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF]. The Spring (Spr) of [START_REF] Chiquet | Structured regularization for conditional Gaussian graphical models[END_REF] can also be seen as a PGGM but with no penalty on Ω y (replaced with an additional structuring one on ∆, we will come back to this point thereafter), so for (Spr) we may consider λ " 0. The generalized procedure (GenGm) at the heart of the study relies on a combination between these two approaches. We will see in due time that we keep both the penalties of (Gm) and the structuring one of (Spr) on ∆.

A generalized Gaussian prior on the direct links

As explained in the introduction, our method integrates a structural regularization on the direct links ∆ through a generalized Gaussian prior. This section will be devoted to the presentation of this prior and the functioning of the associated structural regularization.

Remark 2.2.1. Let us specify that although we use the term prior here, we do not follow a Bayesian approach, the estimation step not being based on the search for an posterior distribution. We will carry out a more detailed study of this approach in Chapter 3.

Generalized Gaussian distribution

Recall that the density of a multivariate Gaussian distribution N p0, V q, with zero mean2 and covariance

V P S d ``
, is given by

@ z P R d , f V pzq " 1 p2πq d 2 a detpV q exp ˆ´xz, V ´1zy 2 ˙.
The generalization of this law involves two new parameters. A scale parameter m P s0, 8r, and a shape parameter β P s0, 8r. However, in our study we will only consider the cases where m " 1. According to the definition given in formulas ( 1)-( 2) of [START_REF] Pascal | Parameter estimation for multivariate generalized Gaussian distributions[END_REF], the density of a d-dimensional multivariate generalized Gaussian distribution GN p0, 1, V, βq takes the form of

@ z P R d , f V, β pzq " β Γp d 2 q π d 2 Γp d 2β q 2 d 2β a detpV q exp ˆ´xz, V ´1zy β 2
ẇhere Γ is the Euler Gamma function.

We clearly recognize the Gaussian N p0, V q setting for β " 1. Moreover, for β " 1{2, it can be seen as a multivariate Laplace distribution whereas it is known to converge to some uniform distribution as

Formulation of the penalty

Our approach consists in integrating information on the structure of the interactions between the predictors within the optimization problem. To this end, although we are not in the Bayesian framework, we borrow a convention from this inference in order to formalize our structural penalization. In particular, the usual Bayesian approach for multiple-output Gaussian regression having B as matrix of coefficients and R as noise variance consists in a conjugate prior vecpBq " N pb, RbL ´1q for some information matrix L P S p ``and a centering value b (see e.g. Section 2.8.5 of [START_REF] Rossi | Bayesian Statistics and Marketing[END_REF]). In the PGGM reformulation, we have R " Ω ´1 y and B " ´∆ t Ω ´1 y as explained in Section 1.5, and of course we shall choose b " 0 to meet our purposes. Thus, if we pose L P S p ``the structural information matrix

vecp∆ t q " ´pΩ y b I p q vecpBq " N p0, Ω y b L ´1q
is a natural prior for the direct links (this is in particular the choice of the authors of [START_REF] Chiquet | Structured regularization for conditional Gaussian graphical models[END_REF]). Indeed, vecpBq " vecp´∆ t Ω ´1 y q " ´pΩ ´1 y b I p q vecp∆ t q, from which we get,

vecp∆ t q " ´pΩ ´1 y b I p q ´1 vecpBq " ´pΩ y b I p q vecpBq " N p´pΩ y b I p q b, pΩ y b I p q pR b L ´1qpΩ y b I p q t q " N p´pΩ y b I p q b, Ω y R Ω y b I p L ´1 I p q.
Following the same logic, let us choose Ω y b L ´1 for scatter parameter and suppose that vecp∆ t q " GN p0, 1, Ω y b L ´1, βq.

(

In this way, we can play on the intensity of the constraint we want to bring on ∆, from a non-informative prior (β " 8) to quasi-boundedness (β Ñ 0) through Laplace (β " 1{2) and Gaussian distributions (β " 1). This prior entails an additional smooth term acting as a structural penalization in the objective (34) that becomes L pa pΩ y , ∆q " ´ln detpΩ y q `xxS y , Ω y yy `2 xxS yx , ∆yy

`xxS x , ∆ t Ω ´1 y ∆yy `η xxL, ∆ t Ω ´1 y ∆yy β `λ |Ω y | 1 `µ |∆| 1 , (4)
with three regularization parameters pλ, µ, ηq. The smooth penalization lends weight to the prior on ∆ and thereby plays on the extent of shrinkage and structuring through β, whereas |∆| 1 and |Ω y | 1 are designed to induce sparsity. One can note that this is closely related to the log-likelihood of a hierarchical model of the form

# Y i | X i , ∆ " N p´Ω ´1 y ∆ X i , Ω ´1 y q vecp∆ t q " GN p0, 1, Ω y b L ´1, βq
where the emphasis is on ∆ in the prior and Ω y remains a fixed parameter. We can indeed see that

f pY i , ∆|X i , Ω y q " f pY i |X i , Ω y , ∆q f p∆|Ω y q.
We know that

Y i | X i , Ω y , ∆ " N p´Ω ´1 y ∆ X i , Ω ´1 y q. Thus, f pY i |X i , Ω y , ∆q " 1 p2πq p 2 detpΩ ´1 y q 1 2 exp ˆ´x pY i `Ω´1 y ∆X i q, Ω y pY i `Ω´1 y ∆X i q y 2 9 detpΩ y q 1 2 exp ˆ´x pY i `Ω´1 y ∆X i q, Ω y pY i `Ω´1 y ∆X i q y 2 ˙.
We also know that vecp∆ t q " GN p0, 1, Ω y b L ´1, βq. Thus,

f p∆|Ω y q 9 1 detpΩ y b L ´1q 1 2 exp ˆ´x vecp∆ t q, pΩ ´1 y b Lq vecp∆ t q y β 2 9 detpΩ y q ´p 2 detpLq q 2 exp ˆ´`v ecp∆ t q t vecpL ∆ t Ω ´1 y q ˘β 2 9 detpΩ y q ´p 2 exp ˆ´xx L, ∆ t Ω ´1 y ∆ yy β 2 ˙.
We recognize the structural term that appears in our penalty. Moreover,

f pY i , ∆|X i , Ω y q 9 detpΩ y q 1 2 exp ˆ´x pY i `Ω´1 y ∆X i q, Ω y pY i `Ω´1 y ∆X i q y `xx L, ∆ t Ω ´1 y ∆ yy β 2 ˙,
from which we get

L n pΩ y , ∆q " n ź i"1 f pY i , ∆|X i , Ω y q 9 detpΩ y q n 2 exp ˆ´ř n i"1 x pY i `Ω´1 y ∆X i q, Ω y pY i `Ω´1 y ∆X i q y 2 ėxp ˆ´n xx L, ∆ t Ω ´1 y ∆ yy β 2 9 detpΩ y q n 2 exp ˆ´xx pY `Ω´1 y ∆Xq, Ω y pY `Ω´1 y ∆Xq yy 2 ėxp ˆ´n xx L, ∆ t Ω ´1 y ∆ yy β 2 ˙,
and so

ℓℓ n pΩ y , ∆q " cst `n 2 ln detpΩ y q ´1 2 xx pY `Ω´1 y ∆Xq, Ω y pY `Ω´1 y ∆Xq yy ´n 2 xxL, ∆ t Ω ´1 y ∆yy β " cst `n 2 ln detpΩ y q ´1 2 xxY Y t , Ω y yy ´1 2 xxΩ ´1 y Ω y Y X t , ∆yy ´1 2 xxX Y t Ω y Ω ´1 y , ∆ t yy ´1 2 xxX X t , ∆ t Ω ´1 y ∆yy ´n 2 xxL, ∆ t Ω ´1 y ∆yy β " cste `n 2 ln detpΩ y q ´n 2 xxS y , Ω y yy ´n xxS yx , ∆yy ´n 2 xxS x , ∆ t Ω ´1 y ∆yy ´n 2 xxL, ∆ t Ω ´1 y ∆yy β
The objective function is obtained by considering the negative log-likelihood on which we add the parameters and the penalty functions. The objective function being defined, we can now study the existence of a global minimum on its domain.

Proposition 2.2.2. Assume that β ě 1. Then, L pa pΩ y , ∆q defined in (4) is jointly convex with respect to pΩ y , ∆q.

Proof. See Section 2.4.2.

Thus, for β ě 1 the estimator of our model corresponds to the global minimum defined by p θ " arg min Θ L λ,µ,η pθq.

(5)

Theoretical guarantees

In this section provide some theoretical guarantees for the estimation of p θ defined in (5). These are valid for any β ě 1 and β " 0, and under the respect of the assumptions (H 2.1 ) and (H 2.2 ) which we will be discussed later in the chapter. However, we shall not theoretically deviate too much from the Gaussianity in the prior (i.e. β " 1 `ϵ for a small ϵ ě 0), even if we will allow ourselves some exceptions in the practical works. We will also come back to this point in due course. Now and throughout the rest of the chapter, denote by θ " pΩ y , ∆q P Θ " S q ``ˆR qˆp the pq ˆpq `pqqmatrix of parameters of the model, with true value θ ˚" pΩ ẙ , ∆ ˚q. As it is usually done in studies implying sparsity, we will also consider S of cardinality |S|, the true active set of θ ˚defined as S " tpi, jq, θ i,j ‰ 0u, and its complement S. Theorem 2.3.1 gives an upper bound on the estimation error. To facilitate reading, we centralized the precise definition of the numerous constants involved and that of the assumptions to be verified in Section 2.4.3.

Theorem 2.3.1. Fix d λ ą c λ ą 1, d µ ą c µ ą 1,
e λ ą 0 and e µ ą 0, and assume that the regularization parameters satisfy pλ, µ, ηq

P Λ " rc λ h a , d λ h a s ˆrc µ h b , d µ h b s ˆr0, ηs, where η " min ! pc λ ´1q λ c λ ℓa , pcµ´1q µ cµ ℓ b , e λ ha ℓa , eµ h b ℓ b ) β s β´1 L
for some non-random constants s L , ℓ a and ℓ b defined in (10) and (11), and the random constants h a and h b given above. Then, under (H 2.1 ), there exists absolute constants b 1 ą 0 and b 2 ą 0 such that, for any 0 ă b 3 ă 1 and as soon as n ą n 0 , with probability no less that 1 ´e´b2n ´b3 , the estimator (5) satisfies

} p θ ´θ ˚}F ď 16 m ˚cλ,µ a |S| γ r,η,β,p c
lnp10pp `qq 2 q ´lnpb 3 q n where γ r,η,β,p , c λ,µ and m ˚are technical constants defined in (15), ( 16) and (17), respectively, and where the minimal number of observations is given by

n 0 " max # plnp10pp `qq 2 q ´lnpb 3 qq c 2 λ,µ |S| p16 m ˚q2 r ˚2 γ 2 r,η,β,p , b 1 pq `rs α s lnpp `qqq, lnp10pp `qq 2 q ´lnpb 3 q + ( 6
)
with s α defined in (13) and r ˚in (14).

Proof. See Section 2.4.3.

Among all these constants, we can note that s L , ℓ a , ℓ b , h a and h b are useful to properly describe and restrict Λ, the domain of validity of pλ, µ, ηq for the theorem to hold. Once Λ is fixed, the other constants take part in the upper bound of the estimation error. However, as it stands, the theorem is very difficult to interpret. The next two remarks seem essential to have an overview of the orders of magnitude involved for the number of observations, for p and q, for the estimation error and for the regularization parameters.

Remark 2.3.1 (Validity band).

Of course the degree of sparsity |S| is crucial in the estimation error, but it also plays an indirect role in the probability associated with the theorem and in the numerous constants. In virtue of Lemma 2.4.13, we can hope that λ and µ have a wide validity band, by playing on c λ , c µ , d λ and d µ . In turn, η also has a non-negligible area of validity, provided of course that ℓ a , ℓ b and s L , all depending on combinations between ∆ ˚, Ω ˚´1 y and L, are small enough. Accordingly, it would be to our advantage if L was both sparse and not chosen with too large elements. As it always appears together with η, we may as well take a normalized version of L (e.g. |L| 8 ď 1).

Remark 2.3.2 (Order of magnitude). Even if the result holds for any β ě 1, the terms 9p β´1 appearing in some upper bounds of the proof clearly argue in favor of a moderate choice β P r1, 1 `ϵs for a small ϵ ą 0, depending on p. In other words, we cannot deviate too much from the Gaussianity in the prior on the direct links. For example in a very high-dimensional setting pp " 10 7 q, choosing ϵ " 0.1 leads to p β´1 « 5 whereas we may try larger values of ϵ for the more common high-dimensional settings p " 10 3 or p " 10 4 . By contrast, we can see that n 0 must (at least) grow like q for the theorem to hold, so highdimensional responses are excluded. However in multiple-output regressions, even when p is extremely large, q generally remains small. According to all these considerations, we may roughly say that, in a high-dimensional setting with respect to p,

} p θ ´θ ˚}F À c |S| ln p n
with a large probability, under a suitable regularization of the model. We recognize the usual terms appearing in the error bounds of regressions with high-dimensional covariates, like the ℓ 2 error of the Lasso (see e.g. Chap. 11 of [START_REF] Hastie | Statistical Learning with Sparsity: The Lasso and Generalizations[END_REF]). This is the same bound as in [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF], but our additional structural penalty restricts Λ.

Technical proofs

This proof section is organized in three parts. First, we introduce some useful linear algebra lemmas that will be repeatedly used subsequently, well-known for most of them. We then prove the joint convexity of the objective function (4), and finish with the proof of our main result. Proof. Since A is symmetric with non-negative eigenvalues, there is an orthogonal matrix P such that Then for all i P t1, ..., du, λ i pABq ě 0.

Linear algebra

A " P DP t with D " diagpsppAqq P S d `. Thus, for all v P R ℓ , it follows that xv, U t A U vy " v t U t P DP t U v " }D 1{2 P t U v} 2 ě 0.
Proof. The equality AB " A 1{2 pA 1{2 B A 1{2 q A ´1{2 shows that AB and A 1{2 B A 1{2 are similar, so they must share the same eigenvalues. From Lemma 2.4.1, λ i pA 1{2 B A 1{2 q ě 0 . trppA ´λmin pAqI d q Bq " trpB 1{2 pA ´λmin pAqI d q B 1{2 q ě 0 from Lemma 2.4.

λ min pAq }U } 2 F ď trpU t AU q ď λ max pAq }U } 2 F .
Proof. Denote by u i the i-th column of U , and let P be the orthogonal matrix such that A "

P DP t with D " diagpsppAqq P S d `. The i-th diagonal element of U t AU satisfies u t i A u i " u t i P DP t u i ě λ min pAq }u i } 2 ě 0. Thus, trpU t AU q " ℓ ÿ i"1 u t i A u i ě λ min pAq ℓ ÿ i"1 }u i } 2 " λ min pAq }U } 2 F .
The upper bound stems from 0

ď u t i A u i ď λ max pAq }u i } 2 .
Lemma 2.4.6. Let A and B be symmetric matrices of same dimensions. Then, λ min pAq `λmin pBq ď λ min pA `Bq and λ max pA `Bq ď λ max pAq `λmax pBq.

Proof. These are just two special cases of Weyl inequalities. We refer the reader to Theorem. 4.3.1 of [HJ12], for example.

Convexity of the objective function

We know from Prop. 1 of [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF] and the convexity of the elementwise ℓ 1 norm that L pa pΩ y , ∆q ή xxL, ∆ t Ω ´1 y ∆yy β is itself convex, but it remains to show that this is still the case with the additional smooth penalty.

Proof of Proposition 2.2.2

We want to show that xxL, ∆ t Ω ´1 y ∆yy β satisfies the convexity inequality. Recall that Θ " S q ``ˆR qˆp and consider the mapping Φ : Θ Ñ S p `defined as @ pA, Bq P Θ, ΦpA, Bq " B t A ´1B.

We can already note from Lemma 2.4.1 that trpΦpA, Bqq ě 0. Moreover, for all 0 ď h ď 1 and all Z i " pA i , B i q P Θ, i " 1, 2, we are interested in the matrix M h pZ 1 , Z 2 q satisfying the following blockwise decomposition

M h pZ 1 , Z 2 q " h ˜A1 B 1 B t 1 B t 1 A ´1 1 B 1 ¸`p1 ´hq ˜A2 B 2 B t 2 B t 2 A ´1 2 B 2 ¸. (7)
The Schur complement S h pZ 1 , Z 2 q of block hA 1 `p1 ´hqA 2 in M h pZ 1 , Z 2 q is defined by

S h pZ 1 , Z 2 q " h pB t 1 A ´1 1 B 1 q `p1 ´hq pB t 2 A ´1 2 B 2 q ´ph B t 1 `p1 ´hq B t 2 q ph A 1 `p1 ´hq A 2 q ´1 ph B 1 `p1 ´hq B 2 q, " h ΦpZ 1 q `p1 ´hq ΦpZ 2 q ´ΦphZ 1 `p1 ´hqZ 2 q. ( 8
)
Moreover, the decomposition 7) is symmetric and positive semi-definite. It is well-known (see e.g.

˜A1{2 A ´1{2 B 0 0 ¸t ˜A1{2 A ´1{2 B 0 0 ¸" ˜A B B t B t A ´1B ḑirectly shows that M h pZ 1 , Z 2 q in (
Appendix A.5.5 of [START_REF] Boyd | Convex optimization[END_REF]) that in that case, the Schur complement (8) must also be positive semidefinite. The trace of S h pZ 1 , Z 2 q is therefore positive. For i " 1, 2 we pose Z i " pΩ i,yy , Ω i,yx L 1{2 q, This convexity inequality concludes the proof.

P h " h ∆ 1 `p1 ´hq ∆ 2 , Q h " h Ω 1,

Theoretical guarantees

Before stating the proof of the theorem, we first present the assumptions related to the covariances ; and then we define the constants appearing in its statement, and which will be used throughout its proof.

Assumptions

Our results depend on some basic assumptions related to the true covariances of the Gaussian observations and the empirical covariances.

Σ x P S p ``, Ω ẙ P S q ``, B ‰ 0 (that is, ∆ ˚‰ 0) and ∆ ˚L ∆ ˚t P S q ``. (H 2.1 ) (H 2.1
) is a natural hypothesis in our framework, in particular we suppose that there is at least a link between X and Y .

Remark 2.4.7 (Null model). Even if it is of less interest, our study does not exclude the case where ∆ ˚" 0. Indeed, we might as well consider that ∆ ˚" 0 and get the same results, but some constants should be refined. On the other hand, Σ x P S p ``and Ω ẙ P S q ``are crucial.

@ u ‰ 0 tel que |u| 0 ď rs α s, 1 2 u t Σ x u ď u t S x u ď 3 2 u t Σ x u. (H 2.2 )
De plus, λ max p∆ ˚Sx ∆ ˚tq ď 7 5 λ max p∆ ˚Σx ∆ ˚tq.

(H 2.2 ) is to be assumed with the smallest integer greater than s α in (13). This is a random hypothesis, which will be controlled with a probability, related to the proximity between the empirical covariance and the true covariance of the predictors, since we recall that S has no reason to be an excellent approximation of Σ ˚when p " n. This is also assumed by the authors of [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF], it is a kind of restricted isometry propertie (RIP), well-known in high-dimensional studies. In particular, we will see through Lemma 2.4.13 that it is satisfied with high probability provided that n is large enough.

Some constants

First we define the random constants. Under (H 2.1 ), the random matrices

A n " pS y ´Σẙ q ´Ω˚´1 y ∆ ˚pS x ´Σx q ∆ ˚t Ω ˚´1 y with h a " |A n | 8 (C 2.1 )
and

B n " 2 ppS yx ´Σẙ x q `Ω˚´1 y ∆ ˚pS x ´Σx qq with h b " |B n | 8 (C 2.2 )
are going to play a fundamental role, especially h a and h b .

Let now define the non-random constants, starting with those related to L and the true values of the model. The bounds

ω L " λ min p∆ ˚L ∆ ˚tq 4 λ max pΩ ẙ q , ω L " 4 λ max p∆ ˚L ∆ ˚tq λ min pΩ ẙ q , ω S " 4 λ max p∆ ˚Σx ∆ ˚tq λ min pΩ ẙ q . ( 9
)
are useful to control the eigenvalues of some recurrent expressions (Lemmas 2.4.8 and 2.4.9), uniformly in a neighborhood of θ ˚" pΩ ẙ , ∆ ˚q. The true value of the term at the heart of the structural regularization is

s L " xxL, ∆ ˚t Ω ˚´1 y ∆ ˚yy. ( 10
)
It plays a role in the proof of Lemma 2.4.10 and, as a consequence, in the definition of the area of validity Λ. This important lemma also requires to define

ℓ a " |Ω ˚´1 y ∆ ˚L ∆ ˚t Ω ˚´1 y | 8 and ℓ b " 2 |Ω ˚´1 y ∆ ˚L| 8 (11)
and, in the context of the theorem,

α " max ! pc λ `1qλ c λ `ηβ s β´1 L ℓ a , pcµ`1qµ cµ `ηβ s β´1 L ℓ b ) min ! pc λ ´1qλ c λ ´ηβ s β´1 L ℓ a , pcµ´1qµ cµ ´ηβ s β´1 L ℓ b ) . ( 12
)
From α and the cardinality of the true active set |S|, let

s α " |S| " 1 `12 α 2 λ max pΣ x q λ min pΣ x q ȷ ( 13
)
which serves as an upper bound in the random hypothesis (H 2.2 ). Similarly, let

r ˚" mintr 1 , r 2 , r 3 , r 4 u (14)
where , a 2 λ min pLq 4 λ max pΩ ẙ q `a3 λ min pΣ x q 40 λ max pΩ ẙ q

r 1 " λ min pΩ ẙ q 2 ,
* (15)
where, as it is detailed in the proof of the lemma in question,

a 1 " 1 ´ϵS ω S ´ηβ p β´1 ω β L ϵ L , a 2 " 2 ϵ S 2 `ϵS and a 3 " ηβ pp ω L q β´1 2 ϵ L 2 `ϵL
for some well-chosen ϵ S ą 0 and ϵ L ą 0. Here again, we make sure that γ r,η,β,p ą 0. In the same way, in the context of the theorem,

c λ,µ " max " pc λ `1q d λ c λ `eλ , pc µ `1q d µ c µ `eµ * (16)
is needed through Lemma 2.4.12. Finally, independently of the structure matrix L,

m ˚" |diagpΣ x q| 8 `|diagpΩ ˚´1 y ∆ ˚Σx ∆ ˚t Ω ˚´1 y q| 8 ( 17
)
is going to play a significative role in the upper bound of the theorem.

Proof of Theorem 2.3.1

Let R n pθq be the the smooth part of the objective (4),

R
Before we start the actual proof, some additional lemmas are needed. They constitute a local study in a sort of r ˚-neighborhood of θ ˚that we define as

N r,α pθ ˚q " ␣ θ P Θ, }δθ} F ď r ˚and |rδθs S | 1 ď α|rδθs S | 1 ( . ( 22
)
Our strategy can be summarized as follows:

Ñ (Lemma 2.4.10) Show that there exists a configuration for the regularization parameters pλ, µ, ηq so that the estimation error satisfies |rδϑs S | 1 ď α|rδϑs S | 1 for some α ą 0.

Ñ (Lemma 2.4.11) Find some r ˚ą 0 and γ r,η,β,p ą 0 such that e 1 pθ, θ ˚q ą γ r,η,β,p }δθ} 2 F as soon as θ P N r,α pθ ˚q.

Ñ (Lemma 2.4.12) Exploit this result to show that the estimation error must also satisfy }δϑ} F ď r provided that maxth a , h b u is small enough. Ñ (Lemma 2.4.13) Conclude that the theorem holds with high probability, provided that n is large enough.

Thereafter, N r,α pθ ˚q will always refer to α in (12) and r ˚in (14). The next two lemmas give some bounds for expressions that will appear repeatedly.

Lemma 2.4.8. Under (H 2.1 ) and (H 2.2 ), for all θ P N r,α pθ ˚q, we have the bound

λ max pΩ ´1 y ∆ S x ∆ t q ď ω S
where ω S is given in (9). In addition,

trpδθ yx S x δθ t yx q ě λ min pΣ x q 10 }δθ yx } 2 F .
Proof. Similar reasonings may be found in the proofs of Lemmas 1-2 of [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF]. We simply reworked the constants to make them stick to our study.

Lemma 2.4.9. Under (H 2.1 ), for all θ P N r,α pθ ˚q, we have the bounds

λ min pΩ ´1 y ∆ L ∆ t q ě ω L and λ max pΩ ´1 y ∆ L ∆ t q ď ω L
where ω L and ω L are given in (9). As a corollary,

p ω L ď xxL, ∆ t Ω ´1 y ∆yy ď p ω L .
Proof. Let θ " θ ˚`δθ P N r,α pθ ˚q, we have ∆ L ∆ t " p∆ ˚`δθ yx q L p∆ ˚t `δθ t yx q. Since L P S p ``from Lemma 2.4.1, δθ yx L δθ t yx P S q `, and in particular λ min pδθ yx L δθ t yx q ě 0. From Lemma 2.4.6,

2 λ min p∆ L ∆ t q ě 2 `λmin p∆ ˚L ∆ ˚tq `λmin pδθ yx L ∆ ˚t `∆˚L δθ t yx q ˘`2 λ min pδθ yx L δθ t yx q ě 2 `λmin p∆ ˚L ∆ ˚tq `λmin pδθ yx L ∆ ˚t `∆˚L δθ t yx q ě 2 `λmin p∆ ˚L ∆ ˚tq ´}δθ yx L ∆ ˚t `∆˚L δθ t yx } 2 ě 2 `λmin p∆ ˚L ∆ ˚tq ´}δθ yx L ∆ ˚t} 2 ´}∆ ˚L δθ t yx } 2 ě 2 `λmin p∆ ˚L ∆ ˚tq ´2 }δθ yx } 2 }L ∆ ˚t} 2 ě 2 `λmin p∆ ˚L ∆ ˚tq ´2 }δθ yx } F }L ∆ ˚t} 2 ě λ min p∆ ˚L ∆ ˚tq
as soon as }δθ yx } F ď r ˚since we know that 4 }L ∆ ˚t} 2 r ˚ď λ min p∆ ˚L ∆ ˚tq. We therefore deduce that ∆ L ∆ t P S q ``. From Lemma 2.4.4, we get

λ min pΩ ´1 y ∆ L ∆ t q ě λ min p∆ L ∆ t q λ max pΩ y q ě λ min p∆ ˚L ∆ ˚tq 4 λ max pΩ ẙ q
where the inequality in the denominator comes from λ max pΩ y q ď λ max pΩ ẙ q`λ max pδθ y q, via Lemma 2.4.6, and the fact that λ max pδθ y q ď }δθ y } F ď r ˚ď λ max pΩ ẙ q. For the upper bound, a similar logic gives, with Lemmas 2.4.6 and 2.4.5,

a λ max p∆ L ∆ t q " }p∆ ˚`δθ yx q L 1{2 } 2 ď }∆ ˚L 1{2 } 2 `}δθ yx L 1{2 } 2 ď a λ max p∆ ˚L ∆ ˚tq `btrpδθ yx L δθ t yx q ď a λ max p∆ ˚L ∆ ˚tq `}δθ yx } F a λ max pLq ď a 2 λ max p∆ ˚L ∆ ˚tq since }δθ yx } F ď r ˚and r ˚aλ max pLq ď p ? 2 ´1q a λ max p∆ ˚L ∆ ˚tq. It follows from Lemma 2.4.4 that λ max pΩ ´1 y ∆ L ∆ t q ď λ max p∆ L ∆ t q λ min pΩ y q ď 4 λ max p∆ ˚L ∆ ˚tq λ min pΩ ẙ q
where the inequality in the denominator comes from λ min pΩ y q ě λ min pΩ ẙ q `λmin pδθ y q, via Lemma 2.4.6, and the fact that 2 λ min pδθ y q ě ´2 }δθ y } F ě ´2 r ˚ě ´λmin pΩ ẙ q. The corollary that concludes the lemma is now immediate.

Lemma 2.4.10. Assume that λ, µ and η are chosen according to the configuration of the theorem. Then, under (H 2.1 ), the estimation error satisfies

|rδϑs S | 1 ď α |rδϑs S | 1
where α ą 0 is given in (12).

Proof. The first derivative of ϕ will be explicitely computed in (26). For t " 0, we find ∇R n pθ ˚q " ´pΩ ˚´1 y , 0 q,p q `pS y , 0 q,p q `2 p0 q,q , S yx q `p´Ω ˚´1 y Whence it follows from the well-known relation

∆ ˚Sx ∆ ˚t Ω ˚´1 y , 2 Ω ˚´1 y ∆ ˚Sx q `η β xxL, ∆ ˚t Ω ˚´1 y ∆ ˚yy β´1 p´Ω ˚´1 y ∆ ˚L ∆ ˚t Ω ˚´1 y , 2 Ω ˚´1 y ∆ ˚Lq, ϕ 1 p0q " ´xxΩ ˚´1 y ,
A n " S y ´Ω˚´1 y ∆ ˚Sx ∆ ˚t Ω ˚´1 y ´Σẙ `Ω˚´1 y Ω ẙ Σ ẙx Σ ˚´1 x Σ x Σ ˚´1 x Σ ˚t yx Ω ẙ Ω ˚´1 y " ´Ω˚´1 y `Sy ´Ω˚´1 y ∆ ˚Sx ∆ ˚t Ω ˚´1 y and B n " 2 S yx `2 Ω ˚´1 y ∆ ˚Sx ´2 Σ ẙx `2 Ω ˚´1 y Ω ẙ Σ ẙx Σ ˚´1 x Σ x " 2 S yx `2 Ω ˚´1 y ∆ ˚Sx . By posing C A " ´Ω˚´1 y ∆ ˚L ∆ ˚t Ω ˚´1
|trpM 1 M 2 q| ď |M 1 | 8 |M 2 | 1 ,
where M 1 and M 2 are compatible matrices, that

ϕ 1 p0q ě ´ha |δϑ y | 1 ´ηβ s β´1 L ℓ a |δϑ y | 1 ´hb |δϑ yx | 1 ´ηβ s β´1 L ℓ b |δϑ yx | 1 ě ´λ c λ |δϑ y | 1 ´ηβ s β´1 L ℓ a |δϑ y | 1 ´µ c µ |δϑ yx | 1 ´ηβ s β´1 L ℓ b |δϑ yx | 1 ,
making use of the constants (11), λ ě c λ h a and µ ě c µ h b . For the sake of clarity, let

∆ n pθ, θ ˚q " R n pθq `λ |Ω y | 1 `µ |∆| 1 ´Rn pθ ˚q ´λ |Ω ẙ | 1 ´µ |∆ ˚|1 .
For all θ P Θ,

|Ω y | 1 ´|Ω ẙ | 1 " |rΩ ẙ `δθ y s S | 1 `|rδθ y s S | 1 ´|rΩ ẙ s S | 1 ě ˇˇ|rΩ ẙ s S | 1 ´|rδθ y s S | 1 ˇˇ`|rδθ y s S | 1 ´|rΩ ẙ s S | 1 ě |rδθ y s S | 1 ´|rδθ y s S | 1
from the triangle inequality and the fact that, as Ω ẙ is positive definite, the diagonal must belong to S, i.e. pj, jq P S for all 1 ď j ď q so that any square matrix M of size q is such that rM s S has diagonal elements all equal to zero. A similar bound obviously holds for |∆| 1 ´|∆ ˚|1 . Now, using the previous information, we can show that

∆ n p p θ, θ ˚q ě ´´λ c λ `ηβ s β´1 L ℓ a ¯|δϑ y | 1 ´´µ c µ `ηβ s β´1 L ℓ b ¯|δϑ yx | 1 `λ`| rδθ y s S | 1 ´|rδθ y s S | 1 ˘`µ `|rδθ yx s S | 1 ´|rδθ yx s S | 1 ˘, " ´´λ c λ ´ηβ s β´1 L ℓ a `λ¯| rδθ y s S | 1 `´µ c µ `ηβ s β´1 L ℓ b `µ¯| rδθ yx s S | 1 `´´λ c λ ´ηβ s β´1 L ℓ a ´λ¯| rδθ y s S | 1 `´µ c µ `ηβ s β´1 L ℓ b ´µ¯| rδθ yx s S | 1 .
Let's pose c " min

" pc λ ´1qλ c λ ´ηβ s β´1 L ℓ a , pc µ ´1qµ c µ ´ηβ s β´1 L ℓ b * and c " max " pc λ `1qλ c λ `ηβ s β´1 L ℓ a , pc µ `1qµ c µ `ηβ s β´1 L ℓ b * .
We obtain

∆ n p p θ, θ ˚q ě c `|rδϑ y s S | 1 `|rδϑ yx s S | 1 ˘´c `|rδϑ y s S | 1 `|rδϑ yx s S | 1 ˘. (23)
Thus, provided that c ą 0, which is stated in the configuration of the theorem, it only remains to note that, necessarily,

∆ n p p θ, θ ˚q ď 0 since p θ is the global minimizer of θ Þ Ñ R n pθq `λ |Ω y | 1 `µ |∆| 1 .
The identification of α given in (12) easily follows.

Lemma 2.4.11. Under (H 2.1 ) and (H 2.2 ), the second-order error term of (19) satisfies, for t " 1 and all θ P N r,α pθ ˚q, e 1 pθ, θ ˚q ą γ r,η,β,p }δθ} 2 F where γ r,η,β,p ą 0 is given in (15).

Proof. From the definition of ϕ in (20) and the fact that ϕ 1 p0q " xx∇R n pθ ˚q, θ ´θ ˚yy, by Taylor-Lagrange there exists h P s0, 1r satisfying

e 1 pθ, θ ˚q " 1 2 ϕ 2 phq. (24)
To simplify the calculations, let

u L " xxL, ∆ t Ω ´1 y ∆yy. ( 25
)
We are going to study the behavior of R n pΩ y , ∆q in the directions Ω y " Ω ẙ `t δθ y and ∆ " ∆ ˚`t δθ yx through ϕptq, where we recall that δθ y " Ω y ´Ωẙ and δθ yx " ∆ ´∆˚. One can see that ϕptq moves from R n pΩ y , ∆q to R n pΩ ẙ , ∆ ˚q as t decreases from 1 to 0. The first derivative is

ϕ 1 ptq " B R n pΩ y , ∆q Bt " ´xxΩ ´1 y , δθ y yy `xxS y , δθ y yy `2 xxS yx , δθ yx yy `2 xxS x , ∆ t Ω ´1 y δθ yx yy ´xxS x , ∆ t Ω ´1 y δθ y Ω ´1 y ∆yy `ηβ u β´1 L " 2 xxL, ∆ t Ω ´1 y δθ yx yy ´xxL, ∆ t Ω ´1 y δθ y Ω ´1 y ∆yy ‰ . ( 26
)
The second derivative is tedious to write but straightforward to establish,

ϕ 2 ptq " xxΩ ´1 y , δθ y Ω ´1 y δθ y yy `2 " xxS x , δθ t yx Ω ´1 y δθ yx yy ´2 xxS x , ∆ t Ω ´1 y δθ y Ω ´1 y δθ yx yy `xxS x , ∆ t Ω ´1 y δθ y Ω ´1 y δθ y Ω ´1 y ∆yy ‰ `2 ηβ u β´1 L " xxL, δθ t yx Ω ´1 y δθ yx yy ´2 xxL, ∆ t Ω ´1 y δθ y Ω ´1 y δθ yx yy `xxL, ∆ t Ω ´1 y δθ y Ω ´1 y δθ y Ω ´1 y ∆yy ‰ `ηβpβ ´1q u β´2 L " 2 xxL, ∆ t Ω ´1 y δθ yx yy ´xxL, ∆ t Ω ´1 y δθ y Ω ´1 y ∆yy ‰ 2 . ( 27
)
First, from the combination of Lemmas 2.4.1 and 2.4.9, we clearly have u L ě 0. We also note that

0 ď } 2 c M 1 ´cM 2 } 2 F " 4 c 2 }M 1 } 2 F ´4 xxM 1 , M 2 yy `c2 }M 2 } 2
F for any c ‰ 0 and any matrices M 1 and M 2 of same dimensions. It follows, after some reorganizations, that for any c ‰ 0 and d ‰ 0,

ϕ 2 ptq ě xxΩ ´1 y , δθ y Ω ´1 y δθ y yy `2 " xxS x , δθ t yx Ω ´1 y δθ yx yy `xxS x , ∆ t Ω ´1 y δθ y Ω ´1 y δθ y Ω ´1 y ∆yy ‰ ´4 xxS 1{2 x δθ t yx Ω ´1{2 y , S 1{2 x ∆ t Ω ´1 y δθ y Ω ´1{2 y yy `2 ηβ u β´1 L " xxL, δθ t yx Ω ´1 y δθ yx yy `xxL, ∆ t Ω ´1 y δθ y Ω ´1 y δθ y Ω ´1 y ∆yy ‰ ´4 xxL 1{2 δθ t yx Ω ´1{2 y , L 1{2 ∆ t Ω ´1 y δθ y Ω ´1{2 y yy ě xxΩ ´1 y , δθ y Ω ´1 y δθ y yy `c1 xxΩ ´1 y , δθ yx S x δθ t yx yy `c2 xxS x , ∆ t Ω ´1 y δθ y Ω ´1 y δθ y Ω ´1 y ∆yy `ηβ u β´1 L " d 1 xxΩ ´1 y , δθ yx L δθ t yx yy `d2 xxL, ∆ t Ω ´1 y δθ y Ω ´1 y δθ y Ω ´1 y ∆yy ‰ , where c 1 " 2 ´4 c 2 , c 2 " 2 ´c2 , d 1 " 2 ´4 d 2 and
where these positive constants are respectively given by

a 1 " 1 ´ϵS ω S ´ηβ p β´1 ω β L ϵ L , a 2 " 2 ϵ S 2 `ϵS and a 3 " ηβ pp ω L q β´1 2 ϵ L 2 `ϵL .
The combination of Lemmas 2.4.1, 2.4.3 and 2.4.5 gives, uniformly in t P r0, 1s, xxΩ ´1 y , δθ y Ω ´1 y δθ y yy ě λ min pΩ ´1 y q trpδθ y Ω ´1 y δθ y q ě }δθ y } 2 F 4 λ 2 max pΩ ẙ q where the inequality in the denominator comes from λ max pΩ y q ď 2 λ max pΩ ẙ q already established in the proof of Lemma 2.4.9. Similarly,

xxΩ ´1 y , δθ yx L δθ t yx yy ě λ min pΩ ´1 y q trpδθ yx L δθ t yx q ě λ min pLq }δθ yx } 2 F 2 λ max pΩ ẙ q .
Lemma 2.4.8 directly enables to bound the last term, xxΩ ´1 y , δθ y S x δθ y yy ě λ min pΩ ´1 y q trpδθ yx S x δθ t yx q ě λ min pΣ x q }δθ yx } 2 F 20 λ max pΩ ẙ q .

In conclusion, combining (24), (28) and the upper bounds above,

e 1 pθ, θ ˚q ě a 1 }δθ y } 2 F 8 λ 2 max pΩ ẙ q `a2 λ min pLq }δθ yx } 2 F 4 λ max pΩ ẙ q `a3 λ min pΣ x q }δθ yx } 2 F 40 λ max pΩ ẙ q ě min " a 1 8 λ 2 max pΩ ẙ q
, a 2 λ min pLq 4 λ max pΩ ẙ q `a3 λ min pΣ x q 40 λ max pΩ ẙ q * }δθ} 2

F

and we clearly identify γ r,η,β,p ą 0.

Lemma 2.4.12. Assume that λ, µ and η are chosen according to the configuration of the theorem. Suppose also that h a in (C

∆ n pθ ˚`t δϑ, θ ˚q " t ϕ 1 p0q `λ |Ω ẙ `t δϑ y | 1 `µ |∆ ˚`t δϑ yx | 1 ´λ |Ω ẙ | 1 ´µ |∆ ˚|1 `et pθ, θ ˚q ě c `|rt δϑ y s S | 1 `|rt δϑ yx s S | 1 ˘´c `|rt δϑ y s S | 1 `|rt δϑ yx s S | 1 ˘`t 2 2 ϕ 2 phq.
From Lemma 2.4.11, we obtain

0 ě c `|rt δϑ y s S | 1 `|rt δϑ yx s S | 1 ˘´c `|rt δϑ y s S | 1 `|rt δϑ yx s S | 1 ˘`γ r,η,β,p }t δϑ} 2 F ě ´c |rt δϑs S | 1 `γr,η,β,p }t δϑ} 2 F .
where we used c ą 0. Note that the configuration of the penalization parameters, which is stated in the theorem, implies the following relations

pc λ `1qd λ h a c λ ě pc λ `1qλ c λ , pc µ `1qd µ h b c µ ě pc µ `1qµ c µ , e λ h a ě η β s β L ℓ a and e µ h b ě η β s β L ℓ b .
From where constant c λ,µ is given in (16). Note that in the proof of Lemma 2.4.10, it was sufficient to see that R n pθq ´Rn pθ ˚q ě ϕ 1 p0q whereas here, we must consider R n pθq ´Rn pθ ˚q " ϕ 1 p0q `e1 pθ, θ ˚q to meet our purposes. That explains the presence of γ r,η,β,p }t δϑ} 2 F in the inequality. We deduce that the error must satisfy

}t δϑ} F ď c λ,µ a |S| maxth a , h b u γ r,η,β,p .
As a corollary, it holds that }δϑ} F ą r ˚ñ c λ,µ a |S| maxth a , h b u ě r ˚γr,η,β,p or, conversely written,

c λ,µ a |S| maxth a , h b u ă r ˚γr,η,β,p ñ }δϑ} F ď r ˚.
Lemma 2.4.13. Assume that λ, µ and η are chosen according to the configuration of the theorem. Then, under (H 2.1 ), there exists absolute constants b 1 ą 0 and b 2 ą 0 such that, for any b 3 P s0, 1r and as soon as n ě max ␣ b 1 pq `rs α s lnpp `qqq, lnp10pp `qq 2 q ´lnpb 3 q ( , with probability no less that 1 ´e´b2n ´b3 both the random hypothesis (H 2.2 ) is satisfied and the upper bound maxth a , h b u ď 16 m ˚c lnp10pp `qq 2 q ´lnpb 3 q n holds, where h a and h b are given in (C 2.1 ) and (C 2.2 ), s α is defined in (13) and m ˚in (17). Hence, one can find a minimal number of observations n 0 such that the theorem holds with high probability as soon as n ą n 0 .

Proof. All the ingredients of the proof are established in [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF]. The authors start by recalling that there exists absolute constants b 1 ą 0 and b 2 ą 0 such that hypothesis (H 2.2 ) is satisfied with probability no less than 1 ´e´b2n as soon as n ě b 1 pq `rs α s lnpp `qqq. We also refer the reader to Lem. 5.1 and Thm.

of [BDDW08]

, or to Lem. 7.4 of [START_REF] Giraud | Introduction to High-Dimensional Statistics[END_REF] for the random bounds of the restricted isometry constants. Afterwards, they prove (see Prop. 4) that, as soon as n ě lnp10pp `qq 2 q ´lnpb 3 q for some b 3 ą 0, with probability 1 ´b3 , maxth a , h b u ď 16 m ˚c lnp10pp `qq 2 q ´lnpb 3 q n .

To find the minimal number of observations, we just need to make sure that the above bound is itself smaller than the one of Lemma 2.4.12. It is then not hard to see that we may retain the minimal size n 0 given in (6).

Simulations and real dataset

Our estimation algorithm consists of a coordinate descent procedure. We exploit the fact that the minimization problem (5) is jointly convex, and alternate between the computation of p Ω y " arg min S q ``ℓ λ,µ,η pΩ y , p ∆q and p ∆ " arg min

R qˆp ℓ λ,µ,η p p Ω y , ∆q.
Each step is done by an Orthant-Wise Limited-Memory Quasi-Newton (OWL-QN) algorithm (see e.g.

[AG07]). The first subproblem is performed through half-vectorization (vech) to ensure symmetry and we set the objective to `8 on S q ``to ensure positive definiteness of the solution. The coordinate descent is stopped when

} p Ω ptq y ´p Ω pt´1q y } 2 ď ϵ maxp1, } p Ω pt´1q y } 2 q and } p ∆ ptq ´p ∆ pt´1q } 2 ď ϵ maxp1, } p ∆ pt´1q } 2 q
following two consecutive iterations t ´1 and t, where ϵ ą 0 is a small threshold depending on the desired precision.

We are now going to try our method on synthetic data first, and then on a real dataset. We will pay attention to the role played by β, in particular we will see that it can be useful as well as counterproductive, depending on the situations.

Simulations

For each scenario, we first generate i.i.d. standard Gaussian vectors X i P R p , then Y i P R q is simulated according to the setting and we estimate Ω y and ∆. From the relations detailed in Section 2.1, we recall that Y i " B t X i `Ei with E i " N p0, Rq is an equivalent formulation, provided that B " ´∆ t Ω ´1 y and R " Ω ´1 y . In a compact form, we may also write Y " XB `E or vecpY q " pI q b Xq vecpBq `vecpEq where the i-th row of Y is Y t i and the i-th row of X is X t i . Thus, we can estimate B using the Lasso (Las) and the Group-Lasso (GLas) in the vectorized form, to provide a basis for comparison between our method and the usual penalized methods. The Lasso penalty is obviously }vecpBq} 1 to promote coordinate sparsity while, for the Group-Lasso, we use the penalty }B 1 } 2 `. . . `}B p } 2 where B i is the i-th row of B, to promote row sparsity and exclude altogether some predictors from the model. We also implement some variants of our generalized graphical model (GenGm): ´the case where Ω y " R ´1 is known and does not need to be estimated is the Oracle (Or) , ´the case where η " 0 so that β has no influence is the classic PGGM (Gm), ´the case where λ " 0 and β " 1 is called the Spring (Spr) by the authors of [START_REF] Chiquet | Structured regularization for conditional Gaussian graphical models[END_REF].

We will focus on structured scenarios. With no structure in ∆, there is no reason why our method should outperform the usual PGGM. In a completely random setting, we have observed that all PGGM procedures perform identically. In fact, a slight gain can be obtained compared to Spr and Gm simply due to the flexibility induced by the additional parameter (Spr and Gm are particular cases of GenGm).

However, that clearly cannot counterbalance the extended computational times, and GenGm should not be used for such situations. The calibration of the regularization parameters is made using a crossvalidation on a training set of size n t " 150 and the accuracy is evaluated thanks to the mean squared prediction error (MSPE) on a validation set of size n v " 1000,

MSPE " › › Y `X p ∆ t p Ω ´1 y › › 2 F q n v . ( 29
)
Due to the large amount of treatments, the grids for cross-validation are not very sharp here but they will be carefully refined for the real datasets of the next section. The covariance between the outputs is R " pr |i´j| q 1 ď i, j ď q for r " 1 2 and we work with p " 100. Each scenario is repeated N " 500 times and GenGm is evaluated with numerous values of β, from 0.25 to 2 with a step of 0.25. The results of the following scenarios are summarized on Figures 2.2, 2.3 and 2.4 below, respectively. Ñ Scenario 1 (q " 1). We draw ω i " ˘1 2 for i " 1, . . . , 10 and we fill 10 randomly selected sections of size 3 in ∆ with ω i . The remaining part of ∆ is 0.

Ñ Scenario 2 (q " 2). We draw ω " ˘1 2 and one randomly selected row of ∆ is filled with ω while the other is identically 0.

Ñ Scenario 3 (q " 3). We draw ω i " ˘1 2 and we fill a randomly selected section of size 30 on the i-th row of ∆ with ω i , for i " 1, 2, 3. The remaining part of ∆ is 0.

The row structure is promoted by a normalized first finite difference operator

L " 1 2
¨1 ´1 0 . . . 0 ´1 2 . . . . . . . . . 0 . . . . . . . . . 0 . . . . . . . . . 2 ´1 0 . . . 0 ´1 1 ‹ ‹ ‹ ‹ ‹ ‹ ‹ ‹ ' (30)
which, through ∆ L ∆ t , tends to penalize the difference between two consecutive values on a same row (as does Fused-Lasso with ℓ 1 penalty). Yet, the Fused-Lasso is not a suitable alternative to GLas and Las in this precise context because B " ´∆ t Ω ´1 y is not supposed to have a row structure even if ∆ has one. For this choice of L, one can note that, in the particular case where R " diagpσ 2 1 , . . . , σ 2 q q, xxL, ∆ t Ω ´1 y ∆yy β "

˜q ÿ i"1 σ 2 i p ÿ j"2 pω i,j ´ωi,j´1 q 2 ¸β ě q ÿ i"1 σ 2β i p ÿ j"2 |ω i,j ´ωi,j´1 | 2β
where ω i,j is the pi, jq-th element of ∆, so we may fairly expect that β ě 1 is going to strengthen the smoothness of the estimation and to enforce all the more the structuring.

Remark 2.5.1 (Validity of the hypotheses). We could as well add a small diagonal element in the matrix L defined above, positive semi-definite but not invertible. The resulting effect would be a negligible ridge-like penalization on the elements of ∆. This is not required for the estimation procedure but useful for Theorem 2.3.1 to hold (see e.g. (H 2.1 )). Likewise, it seemed interesting to test some settings with β ă 1 even if the theory developped in the chapter does not give any guarantee for them, as a basis for comparison. First of all, one can observe that Las and GLas are left behind in all our simulations. This is not surprising since the covariance between the outputs cannot be recovered with the standard Lasso, at least for q ě 2. Generally, GLas remains more robust compared to Las, probably due to the high level of sparsity in ∆ approximately passed to B (provided that the covariances in R are small enough), and exploited by the grouping effect. In the weakly structured setting (Scenario 1), we also observe that, as expected, all PGGM procedures perform almost identically, with obviously an advantage for Or (although small, illustrating the accuracy of the estimation). In the strongly structured settings (Scenarios 2 and 3), Gm gives results below the expected level, because it is not designed to promote such layouts. On the contrary, thanks to this choice of L showing here great efficiency, GenGm and Spr are doing pretty well. Note that, in this context, GenGm with β " 1 is almost the same as Spr since, q being small, λ does not play a crucial role. However, some empirical facts draw our attention: the prediction error decreases with β to some extent, but the most interesting fact seems to be the simultaneous decrease of its variance. It is likely that the increasing pressure exerted by β on the estimation procedure leads to a higher homogeneity in the numerical results, despite the repetitions of random experiments under random settings. In other words, the structuring seems to be strengthened and we also observe that the convergence of the algorithm is faster, which logically follows from the latter remarks (especially clear when we compare β " 0.25 and β " 2). On the other hand, for the opposite reason, we notice that the predictions are hardly better than Gm (even worse in some cases), both on average and in terms of variability, for β ă 1, and these simulations tend to undermine such values of the hyperparameter. On the whole, GenGm with β ą 1 might be a sound approach for practitioners who place a high priority on structuring the estimations, even if Remark 2.5. for λ " 0 and β " 1, but unfortunately cannot be implemented in the general setting. As a result, computational times remain an issue that should be paid attention to.

Remark 2.5.3 (Oracle-type errors). The mean value of the estimation errors } p ∆´∆} 2 F leads to the same kind of observations for the models being compared in the simulations. But the minimal prediction error does not always coincide with an optimal support recovery due to the shrinkage effect on the estimation of ∆, when the coefficients or the covariates are not very contrasting. The so-called F -score is given by F " 2 p r r e p r `re where p r " TP TP `FP and r e " TP TP `FN are the precision and the recall, respectively, and where T/F and P/N stand for true/false and positive/negative. In the strongly structured scenarios, F is generally located between 0.60 and 0.65, and a deeper analysis shows that a proportion of more than 0.99 of true non-zero values are recovered (that is, the part of the true active set S related to ∆). If the models are not calibrated to reach the best prediction error but the best F -score, F regularly exceeds 0.90, at least for the structured procedures.

Nevertheless, Scenarios 2 and 3 are very strongly structured, more than one would expect from an unknown underlying generating process, and the real dataset of the next section is going to highlight the fact that the improvement may be hardly noticeable with respect to β. But we will see that β can still be useful for variable selection.

A real dataset

The dataset available as CanadianWeather in the R package fda contains daily temperature and precipitation at 35 different locations in Canada, averaged over annual reports starting in 1960 and ending in 1994 (see e.g. [START_REF] Ramsay | Functional Data Analysis[END_REF]). We intend to look at the direct links between the minimal and maximal rainfall (on the log 10 scale) and the temperature pattern in the 35 weather stations, so as to identify the times of the year that have a strong effect on rainfall (positive as well as negative). In this context, n " 35, q " 2 and p " 365. Figure 2.6 shows temperature and log-precipitation measured over a year in Montreal, chosen as an example, together with the empirical distribution of the minimal and maximal log-precipitation for the 35 weather stations. We can note that, since the data are averaged over numerous years, outliers are unlikely even for the extremes (min and max). Some authors (see e.g. [START_REF] Slawski | The structured elastic net for quantile regression and support vector classification[END_REF]) have already highlighted the pertinence of using the matrix L defined in (30) in this dataset, because the predictors are ordered temporally so that the selection of isolated days instead of relevant sequences of days seems an unreliable procedure for statistical interpretation. To assess the models, we repeat N " 100 times the following experiment:

1. we randomly select n t " 25 observations which constitute the training set on which we first perform a 2-fold cross-validation for parameter calibration, followed by the model estimation ;

2. the remaining n v " 10 observations then constitute the validation set, and are used to compute the MSPE (29) related to the prediction of the minimum (min p ) and maximum (max p ) precipitation.

We can see on Figure 2.7 that all structured PGGM perform almost identically, with the phenomenon described in the previous section still visible but to a lesser extent. We can even notice that structuring is hardly beneficial for this dataset, from a purely numerical point of view. This conclusion can also be found in [START_REF] Slawski | The structured elastic net for quantile regression and support vector classification[END_REF], where the author compares the structured Elastic-Net with unstructured alternatives to predict the 0.25-, 0.50-and 0.75-quantiles of the log-precipitation, through independent regressions. The median values of the estimated direct links between the temperature of the days and the pair pmin p , max p q are represented on Figure 2.10 together with the estimated regression coefficients, for β " 2. We recall that the relation B " ´∆ t Ω ´1 y simply lead to

p B " ´p ∆ t p Ω ´1 y .
We detect sequences of influent days in November, December, January and February, especially related to min p , positively at the end of the year and negatively at the beggining. This is broadly consistent with the analysis of [START_REF] Slawski | The structured elastic net for quantile regression and support vector classification[END_REF] -even if the responses are not extremes but quantiles in it -with however two differences: the regression coefficients associated with max p are much lower compared to min p whereas it is not that clear in the reference, and an activity is also detected between July and August. The main explanation, at least for the first of them, probably lies in the use of graphical models that take into account the correlation between responses. Indeed, as can be seen on Figure 2.11 which gives an overview of the estimation of R obtained from the repeated experiments, a non-zero correlation is detected between the responses (« 0.32). The influence of November and December on all quantiles and that of January and February on the 0.75-quantile in [START_REF] Slawski | The structured elastic net for quantile regression and support vector classification[END_REF] might actually be an artificial effect of the correlation with the 0.25-quantile. This is what our study suggests by highlighting min p compared to max p : the 'real' effect appears to be on min p whereas max p seems to react only through a phenomenon of correlation with min p . From this point of view, the interest of graphical models instead of independent regressions is particularly obvious. Let us also mention that, interestingly enough, we notice that the role of η tends to depreciate for the large values of β. For example, for the same regularization parameters pλ, µq " p0, 0.05q and β " 2, the difference between the estimated active sets for η " 0.1 and η " 1 is almost negligible (depending on the experiments, between 1 and 3 days are concerned, on average). Based on these studies and observations, we might conclude that β is insignificant when we are interested in the best prediction error on a validation set (even counterproductive with respect to computational times, e.g. compared to Spr), whereas it seems to have a substancial role to play when focusing on selection, by accelerating the discrimination of variables.

In the first case, η has to be carefully adjusted while in the second case, β will quickly help to reach the desired sparsity.

Remark 2.5.4 (Structure matrix). For the simulations and the real dataset, we have used the popular first finite difference operator given in (30). Other examples can be found in the literature, like the promotion of a genetic distance for genomic selection in Brassica napus [START_REF] Chiquet | Structured regularization for conditional Gaussian graphical models[END_REF] or the bidimensional

Conclusion

In conclusion, our work is a generalization of [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF], using the same technical tools to establish an upper bound on the estimation error when a prior on the direct links generates an additional structural penalty in the objective, provided that the model is suitably regularized. Our work is also an improvement of [START_REF] Chiquet | Structured regularization for conditional Gaussian graphical models[END_REF] since, while being inspired by the methodology of the authors, we generalize the prior and give some theoretical guarantees. The empirical study shows that the hyperparametrization in the prior, although more expensive in adjusting the parameters, is likely to refine the selection results but clearly, this does not appear as a crucial improvement compared to the two previous points. Let us conclude the chapter by highlighting two weaknesses that might be trails for future studies. On the one hand, the Laplace distribution is often used as a prior in the Bayesian Lasso (see e.g. Sec. 6.1 of [START_REF] Hastie | Statistical Learning with Sparsity: The Lasso and Generalizations[END_REF]). However, our reasonings do not allow β " 1{2, which may correspond to a multivariate Laplace distribution on the direct links. Combined with the first finite difference operator L, the choice β " 1{2 could generate a Fused-Lasso-type penalty. In this regard, it would be challenging and interesting to obtain some theoretical guarantees for β ě 1{2 and not only for β ě 1, even if our probably too brief simulation study does not encourage the choice of β ă 1. On the other hand, λ " 0 is a natural choice when q is small (this is in particular the configuration of [START_REF] Chiquet | Structured regularization for conditional Gaussian graphical models[END_REF]), not to mention that it is computionally faster. But, the proof of our theorem needs λ ą c λ h a ą 0 to hold. We think that a reasoning enabling to deal with λ " 0 should also be beneficial to the study. More generally, it would be instructive to consider a very high-dimensional setting (p " n and not only p " 10 2 although always larger than n, as in our experiments). Such studies should follow with omic data.

Chapter 3

A BAYESIAN APPROACH FOR PARTIAL GAUSSIAN GRAPHICAL MODELS WITH SPARSITY G
aussian graphical model is a widely studied topic both in the frequentist framework (as discussed above), and in the Bayesian one. For this second type of inference, we refer the reader for example to Chapter 10 of Maathuis et al. [START_REF] Maathuis | Handbook of Graphical Models[END_REF] where various Wishart-type priors are considered for Ω, see also Li et al. [START_REF] Li | Bayesian joint spike-and-slab graphical Lasso[END_REF] or Gan et al. [START_REF] Gan | Bayesian joint estimation of multiple graphical models[END_REF] for spike-and-slab approaches and all references within. However, to the best of our knowledge, the Bayesian approach for partial Gaussian graphical models is a new research topic.

In this chapter we present the results of a collaboration with Frédéric Proïa and Pascal Jézéquel, which has been published in Bayesian Analysis in 2022 [OOJP22]1 . We explore various Bayesian approaches to estimate PGGM, and propose hierarchical structures enable to deal with single-output as well as multipleoutput linear regressions, in small or high dimension, enforcing either no sparsity, sparsity, group sparsity or even sparse-group sparsity for a bi-level selection through partial correlations (direct links) between predictors and responses. Our work is inspired by the ideas of Xu and Ghosh [START_REF] Xu | Bayesian variable selection and estimation for Group Lasso[END_REF] for the single-output setting (q " 1), and by the ones of Liquet et al. [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF] for the multiple-output setting (q ą 1). Taking advantage of the relations (1), we consider that a Gaussian prior for B must remain Gaussian for ∆ (with a correctly updated variance), and that an inverse Wishart prior for R merely becomes a Wishart one for Ω y . Yet, despite these seemingly small changes in the design of the priors, we will see that the resulting distributions are completely different. The hierarchical models that we are going to study all come from this working base, but let us point out that a wide variety of refinements exist in the recent literature for Bayesian sparsity, like the grouped 'horseshoe' of Xu et `20], all related to the regression setting but that might also be investigated for PGGMs. To enforce various types of sparsity in ∆ for high-dimensional problems, we decided to make use of spike-and-slab priors, with a spike probability guided by a conjugate Beta distribution.

This chapter is organized as follows. Sections 3.2, 3.3 and 3.4 are dedicated to the study of our hierarchical models enforcing either no sparsity, sparsity, group sparsity or sparse-group sparsity in the direct links, respectively, according to the terminology of Section 2.1 of Giraud [START_REF] Giraud | Introduction to High-Dimensional Statistics[END_REF]. In particular, we will see that our bi-level selection clearly diverges from the strategy of Liquet et al. [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF]. We also adapt the reasoning of Yang and Narisetty [START_REF] Yang | Consistent group selection with bayesian high dimensional modeling[END_REF] to establish group selection consistency under some technical assumptions and an appropriate amount of sparsity. Section 3.5 is devoted to the conditional posterior distributions of the parameters in order to implement Gibbs samplers that are tested in Section 3.6. This empirical section is focused on a simulation study first, to evaluate and compare the efficiency of the models, then a real dataset is treated, and a short conclusion ends the paper. But, firstly, let us give some examples of what exactly we mean by 'sparse', 'group-sparse' and 'sparse-group-sparse' settings, and let us summarize the definitions that we have chosen to retain for the well-known distributions as well as for the less usual ones, in order to avoid any misinterpretation of our results and proofs.

Example 3.0.1. To explain a set of phenotypic traits, suppose that we investigate a large collection of genetic markers spread over twenty chromosomes. For coordinate sparsity ('sparse' setting), only a few markers are active. For group sparsity ('group-sparse' setting), the markers are clustered into groups (formed by chromosomes) and only a few of them are active. For sparse-group sparsity ('sparse-groupsparse' setting), only a few chromosomes are active and they are sparse, the result is a bi-level selection (chromosomes and markers). This will be the context of our example on real data (Section 3.6.2).

Introduction

Suppose now that we observe q-dimensional matrix of responses Y P R nˆq where the k-th row is Y t i , and the p-dimensional matrix of predictors X P R nˆp where the k-th row is X t i . We are dealing with a multivariate linear regression of the form

Y " XB `E
where B P R pˆq contains the regression coefficients and E P R nˆq is a matrix-variate Gaussian noise. We are in the PGGM setting discussed in Section 1.5. In short, assuming that the couple pY i , X i q P R q`p is jointly normally distributed with zero mean, covariance Σ and precision Ω, we can see that the blockwise decomposition given by

Ω " ˜Ωy ∆ ∆ t Ω x
with Ω y P S q ``, ∆ P R qˆp and Ω x P S p ``leads to Y i | X i " N q p´Ω ´1 y ∆ X i , Ω ´1 y q. This is a crucial remark because one can see that the multiple-output regression Y i " B t X i `Ei with Gaussian noise E i " N q p0, Rq may be reparametrized with

B " ´∆t Ω ´1 y and R " Ω ´1 y . ( 1
)
Before going deeper into the subject, let us summarize the definitions that we have chosen to retain for the well-known distributions as well as for the less usual ones, in order to avoid any misinterpretation of our results and proofs.

Definition 3.1.1 (Gaussian). The density of X P R d1ˆd2 following the matrix normal distribution where a ą 0 and b ą 0. The exponential distribution Epℓq is then defined as the Γp1, ℓq distribution, for ℓ ą 0.

MN d1ˆd2 pM, Σ 1 , Σ 2 q is given by ppXq " 1 p2πq d 1 d 2 2 |Σ 1 | d 2 2 |Σ 2 | d 1 2 exp ˆ´1 2 tr `Σ´1 2 pX ´M q t Σ ´1 1 pX ´M q ˘ẇhere M P R d1ˆd2 ,
Definition 3.1.4 (Beta). The density of X P r0, 1s following the Beta distribution βpa, bq is given by ppXq " X a´1 p1 ´Xq b´1 βpa, bq

1 t0 ď X ď 1u
where a ą 0, b ą 0 and β is the Beta function.

In all the paper, data and parameters are gathered in Θ " tY, X, ∆, Ω y , ν, λ, πu and, to standardize, for any e P Θ, we note Θ e " Θzteu.

The sparse setting

In this section, λ i P R is the i-th component of λ P R p , ∆ i P R q is the i-th column of ∆ and X i P R n stand for the i-th column of X (1 ď i ď p). Let us consider the hierarchical Bayesian model, where the columns of ∆ are assumed to be independent, given by

$ ' ' ' ' ' ' & ' ' ' ' ' ' % Y | X, ∆, Ω y " MN nˆq p´X ∆ t Ω ´1 y , I n , Ω ´1 y q ∆ i | Ω y , λ i , π K K " p1 ´πq N q p0, λ i Ω y q `π δ 0 λ i K K " Γpα, ℓ i q Ω y " W q pu, V q π " βpa, bq (2)
for i P 1, p , with hyperparameters α " 1 2 pq `1q, ℓ i ą 0, u ą q ´1, V P S q ``, a ą 0 and b ą 0. A general ungrouped sparsity is promoted in the columns of ∆ through the spike-and-slab prior. In this mixture model, π is the prior spike probability and λ is an adaptative shrinkage factor acting at the predictor scale (λ i is associated with the direct links between predictor i and all the responses). When ℓ i " ℓ for all i, we will rather speak of global shrinkage. The degree of sparsity will be characterized by the number N 0 of zero columns of ∆, that is

N 0 " Cardpi, ∆ i " 0q " p ÿ i"1 1 t∆i " 0u .
(3) To implement a Gibbs sampler from the full posterior distribution stemming from (2), we may use the conditional distributions given in the proposition below. Proposition 3.2.1. In the hierarchical model (2), the conditional posterior distributions are as follows.

´The parameter ∆ satisfies, for i P 1, p ,

∆ i | Θ ∆i " p1 ´pi q N q p´s i H i , s i Ω y q `pi δ 0 where H i " Ω y Y t X i `ÿ j ‰ i xX i , X j y ∆ j , s i " λ i 1 `λi }X i } 2 and p i " π π `p1 ´πq p1 `λi }X i } 2 q ´q 2 exp ´si H t i Ω ´1 y Hi 2 ¯.
54

The sparse setting

´The parameter Ω y satisfies

Ω y | Θ Ωy " MGIG q ˆn ´p `N0 `u 2 , ∆ pX t X `D´1 λ q ∆ t , Y t Y `V ´1ẇ
here D λ " diagpλ 1 , . . . , λ p q.

´The parameter λ satisfies, for i P 1, p ,

λ i | Θ λi " 1 t∆i ‰ 0u GIG ˆ1 2 , ∆ t i Ω ´1 y ∆ i , 2 ℓ i ˙`1 t∆i " 0u Γpα, ℓ i q.
´The parameter π satisfies π | Θ π " β `N0 `a, p ´N0 `b˘.

Proof. See Section 3.5.1.

Remark 3.2.2. The Bayesian Lasso, as introduced e.g. in Section 6.1 of [START_REF] Hastie | Statistical Learning with Sparsity: The Lasso and Generalizations[END_REF] or in [START_REF] Park | The Bayesian Lasso[END_REF], assumes a prior Laplace distribution for the regression coefficients conditional on the noise variance. In our case, ∆ i | Ω y , π is still a multivariate spike-and-slab (after integrating over λ i ), with a slab following a socalled multivariate K-distribution (see [START_REF] Eltoft | Multivariate scale mixture of Gaussians modeling[END_REF]), which is a generalization of the multivariate Laplace distribution. See e.g. Section 2.1 of [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF]. From this point of view, our study is in line with the usual Bayesian regression schemes. Perhaps even more interesting, going on with the idea of the authors, suppose that, for all 1 ď i ď p, ∆ i " b i ∆ i where ∆ i follows the multivariate K-distribution described above and b i | π " Bp1´πq is independent of ∆ i . Now, the sparsity in ∆ is not induced by a spike-and-slab strategy anymore but, equivalently, by multiplying the slab part by an independent Bernoulli variable being 0 with probability π. Then, it is possible to show that the negative log-likelihood of this alternative hierarchical model is given, up to an additive constant that does not depend on ∆, by

1 2 › › ›pY `X ∆ t Ω ´1 y q Ω 1 2 y › › › 2 F `p ÿ i"1 c i › › ›Ω ´1 2 y ∆ i › › › F `ln ˆ1 ´π π ˙p ÿ i"1 b i
where c i ą 0. We first recognize an ℓ 2 -type penalty but also an ℓ 0 -type penalty on ∆ (provided that π ă 1 2 ) since summing the b i amounts to counting the number of non-zero columns in ∆. Consequently, there is a close connection between our hierarchical Bayesian model and the regressions penalized by ℓ 2 and ℓ 0 norms, problems that are known to be very hard to solve due to combinatorial optimization.

The particular case q " 1 is a very useful corollary of the proposition. Here, the direct links form a row vector such that ∆ t P R p with components ∆ i P R (1 ď i ď p), and the precision matrix of the responses reduces to ω y ą 0. According to the parametrization of the distributions (see Section 3.1), the corresponding prior distribution of ω y is Γp u 2 , 1 2 v q for u, v ą 0 and the one of λ i is Epℓ i q for ℓ i ą 0. The other priors are unchanged.

Corollary 3.2.3. In the hierarchical model (2) with q " 1, the conditional posterior distributions are as follows.

´The parameter ∆ satisfies, for i P 1, p ,

∆ i | Θ ∆i " p1 ´pi q N p´s i h i , s i ω y q `pi δ 0 where h i " ω y xX i , Y y `ÿ j ‰ i xX i , X j y ∆ j , s i " λ i 1 `λi }X i } 2 and p i " π π `p1 ´πq p1 `λi }X i } 2 q ´1 2 exp ´si h 2 i 2 ωy ¯.
´The parameter ω y satisfies

ω y | Θ ωy " GIG ˆn ´p `N0 `u 2 , ∆ pX t X `D´1 λ q ∆ t , }Y } 2 `1 v ẇhere D λ " diagpλ 1 , . . . , λ p q.
´The parameter λ satisfies, for i P 1, p ,

λ i | Θ λi " 1 t∆i ‰ 0u GIG ˆ1 2 , ∆ 2 i ω y , 2 ℓ i ˙`1 t∆i " 0u Epℓ i q.
´The parameter π satisfies π | Θ π " β `N0 `a, p ´N0 `b˘.

Proof. This is a consequence of Proposition 3.2.1.

Note that we can also easily derive the Bayesian counterpart of the standard PGGM adapted to the small-dimensional case, with no sparsity, by taking π " 0.

Corollary 3.2.4. In the hierarchical model (2) with π " 0, the conditional posterior distributions are as follows.

´The parameter ∆ satisfies, for i P 1, p ,

∆ i | Θ ∆i " N q p´s i H i , s i Ω y q
where

H i " Ω y Y t X i `ÿ j ‰ i xX i , X j y ∆ j and s i " λ i 1 `λi }X i } 2 .
´The parameter Ω y satisfies

Ω y | Θ Ωy " MGIG q ˆn ´p `u 2 , ∆ pX t X `D´1 λ q ∆ t , Y t Y `V ´1ẇ
here D λ " diagpλ 1 , . . . , λ p q.

´The parameter λ satisfies, for i P 1, p ,

λ i | Θ λi " GIG ˆ1 2 , ∆ t i Ω ´1 y ∆ i , 2 ℓ i ˙.
Proof. This is a consequence of Proposition 3.2.1.

In the simulation study of Section 3.6.1, Scen. 0, 1 and 2 are dedicated to the sparse setting. The next section discusses the group sparsity in ∆.

The group-sparse setting

The predictors are now ordered in m groups of sizes κ 1 `. . . `κm " p. For the g-th group (1 ď g ď m), λ g P R is the g-th component of λ P R m , the covariate submatrix is X g P R nˆκg and the corresponding slice of ∆ is ∆ g P R qˆκg . Let us consider the hierarchical Bayesian model, where the columns of ∆ are assumed to be independent both within and between the groups, given by

$ ' ' ' ' ' ' & ' ' ' ' ' ' % Y | X, ∆, Ω y " MN nˆq p´X ∆ t Ω ´1 y , I n , Ω ´1 y q ∆ g | Ω y , λ g , π K K " p1 ´πq MN qˆκg p0, λ g Ω y , I κg q `π δ 0 λ g K K " Γpα g , ℓ g q Ω y " W q pu, V q π " βpa, bq (4)
for g P 1, m , with hyperparameters α g " 1 2 pq κ g `1q, ℓ g ą 0, u ą q ´1, V P S q ``, a ą 0 and b ą 0. A general group sparsity is promoted in the columns of ∆ through the spike-and-slab prior at the group level. In this mixture model, π is the prior spike probability and λ is an adaptative shrinkage factor acting at the group scale (λ g is associated with the direct links between the predictors of group g and all the responses). Likewise, when ℓ g " ℓ for all g, we will rather speak of global shrinkage. Now, the degree of sparsity will be characterized by N 0 given in (3), but also by the number G 0 of zero groups of ∆, that is

G 0 " Cardpg, ∆ g " 0q " m ÿ g"1 1 t∆g " 0u .
(5)

To implement a Gibbs sampler from the full posterior distribution stemming from (4), we may use the conditional distributions given in the proposition below. Proposition 3.3.1. In the hierarchical model (4), the conditional posterior distributions are as follows.

´The parameter ∆ satisfies, for g P 1, m , ∆ g | Θ ∆g " p1 ´pg q MN qˆκg p´H g S g , Ω y , S g q `pg δ 0 where

H g " Ω y Y t X g `ÿ j ‰ g ∆ j X t j X g , S g " λ g `Iκg `λg X t g X g ˘´1 and p g " π π `p1 ´πq |I κg `λg X t g X g | ´q 2 exp ´trpH t g Ω ´1 y Hg Sgq 2 ¯.
´The parameter Ω y satisfies

Ω y | Θ Ωy " MGIG q ˆn ´p `N0 `u 2 , ∆ pX t X `D´1 λ q ∆ t , Y t Y `V ´1ẇ
here D λ " diagpλ 1 , . . . , λ 1 , . . . , λ m , . . . , λ m q with each λ g duplicated κ g times.

´The parameter λ satisfies, for g P 1, m ,

λ g | Θ λg " 1 t∆g ‰ 0u GIG ˆ1 2 , trp∆ t g Ω ´1 y ∆ g q, 2 ℓ g ˙`1 t∆g " 0u Γpα g , ℓ g q. ´The parameter π satisfies π | Θ π " β `G0 `a, m ´G0 `b˘.
Proof. See Section 3.5.2.

Note that Remark 3.2.2 still applies to this configuration, after some adjustments (the ℓ 0 -like penalty is on the number of non-zero groups). Here again, the particular case q " 1 is a very useful corollary. The direct links form a row vector such that ∆ t P R p with groups ∆ t g P R κg (1 ď g ď m), the precision matrix of the responses reduces to ω y ą 0. According to the parametrization of the distributions (see Section 3.1), the corresponding prior distribution of ω y is Γp u 2 , 1 2 v q for u, v ą 0, like in the ungrouped setting. The other priors are unchanged.

Corollary 3.3.2. In the hierarchical model (4) with q " 1, the conditional posterior distributions are as follows.

´The parameter ∆ satisfies, for g P 1, m , ∆ t g | Θ ∆g " p1 ´pg q N κg p´S g H g , ω y S g q `pg δ 0 where

H g " ω y X t g Y `ÿ j ‰ g X t g X j ∆ t j , S g " λ g `Iκg `λg X t g X g ˘´1 and p g " π π `p1 ´πq |I κg `λg X t g X g | ´1 2 exp ´H t g Sg Hg 2 ωy ¯.
´The parameter ω y satisfies

ω y | Θ ωy " GIG ˆn ´p `N0 `u 2 , ∆ pX t X `D´1 λ q ∆ t , }Y } 2 `1 v ẇhere D λ " diagpλ 1 , .
. . , λ 1 , . . . , λ m , . . . , λ m q with each λ g duplicated κ g times.

´The parameter λ satisfies, for g P 1, m ,

λ g | Θ λg " 1 t∆g ‰ 0u GIG ˆ1 2 , }∆ g } 2 ω y , 2 ℓ g ˙`1 t∆g " 0u Γpα g , ℓ g q. ´The parameter π satisfies π | Θ π " β `G0 `a, m ´G0 `b˘.
Proof. This is a consequence of Proposition 3.3.1.

In the simulation study of Section 3.6.1, Scen. 3 and 4 are dedicated to the group-sparse setting.

To conclude this section, a theoretical guarantee is provided (given Ω y and with λ " λ n and π " π n depending on n). It is possible to obtain a model selection consistency property for this approach when both the number of observations n and the number of groups m " m n tend to infinity, by adapting the reasoning of [YN20] dedicated to the linear regression (with q " 1). Indeed, when Ω y is known, ∆ reduces to a linear transformation of B. Thus, it is not surprising that a similar result follows under the same kind of hypotheses. In the sequel, we denote by X pkq P R nˆ|k| the design matrix of rank r k corresponding to the submodel indexed by the binary vector k P t0, 1u m having |k| non-zero values (k g " 1 means that the g-th group is included in the model), and by Π pkq P R nˆn the projection matrix onto the column-space of X pkq . Similarly, ∆ restricted to k is ∆ pkq P R qˆ|k| . The true model is called t and t ˘g are submodels of t that contain only the g-th group or that are deprived of it, respectively. Let

δ 1 " inf 1 ď g ď |t| › › pI n ´Πpt ´g q q X pt `g q ∆ t pt `g q Ω ´1 2 y › › 2 F
and, for some K ą 0,

δ K 2 " inf k P Ei › › pI n ´Πpkq q X ptq ∆ t ptq Ω ´1 2 y › › 2
F with E i " tk such that t Ć k and r k ď Kr t u. Let also,

µ K n, min " inf k P F K µ `ˆX t pkq X pkq n ˙and μn " inf k P F µ ˚ˆX t
pkq pI n ´Πpk X tq q X pkq n ẇith F K " tk such that t Ă k and r k ď pK `1q r t u and F " tk such that |kzt| ą 0u, and where, for a square matrix A, µ `pAq is the minimum non-zero eigenvalue of A and µ ˚pAq is the geometric mean of the non-zero eigenvalues of A. The hypotheses are those of [YN20] that we have to slightly adapt. By f n -g n we mean that there is a constant c ‰ 0 such that f n {g n Ñ c as n tends to infinity.

(H.1) There exists a rate such that m n " e vn with v n Ñ `8 and v n " opnq. n " max i λ n, i .

(H.4) There exists ϵ 1 ą 0 such that δ 1 ą p1 `ϵ1 q r t rp4 `ηq ln m n ´η ln μn s.

(H.5) There exists ϵ 2 ą 0 such that δ K 2 ą p1 `ϵ2 q r t rp4 `ηq ln m n ´η ln μn s for some K ą maxp8{η 1, η{pη ´1qq.

We refer the reader to p. 917 of [START_REF] Yang | Consistent group selection with bayesian high dimensional modeling[END_REF] where the authors give very clarifying comments on the interpretation to be given to these technical assumptions. In particular, while (H.1), (H.2) and (H.3) control the behavior of m n , π n and λ n as n tends to infinity, (H.4) and (H.5) are related to sensitivity and specificity and are therefore in connection with the true model t.

Proposition 3.3.3. Suppose that (H.1)-(H.5) are satisfied. Then, as n tends to infinity,

PpT | Y, X, Ω y q P ÝÑ 1
where T " tt is selectedu and t is the true model.

Proof. The result is obtained by following the same lines as the proof of Theorem 2.1 of [START_REF] Yang | Consistent group selection with bayesian high dimensional modeling[END_REF]. One just has to clarify a few points to solve the issues arising from q ě 1 and from the adaptative shrinkage, which is done in Section 3.5.4. Remark 3.3.4. Obviously, Proposition 3.3.3 also holds for the sparse setting (with m " p) and in that case, it is instructive to draw the parallel with Theorem 1 of [RSZZ15] even if the estimation procedure is very different. The authors show that, to obtain a ? n-consistent estimation of the precision matrix Ω in a GGM, Ω must contain at most -? n{ ln p non-zero columns. In the Gibbs sampler (see Proposition 3.2.1), the slab probability 1 ´π is generated according to a distribution that satisfies

Er1 ´π | Θ π s " p ´N0 `b p `a `b
and Vp1 ´π | Θ π q " pN 0 `aqpp ´N0 `bq pp `a `bq 2 pp `a `b `1q .

Thus, if the model selects -? n{ ln p predictors, it follows that the posterior expectation of 1 ´π is -? n{pp ln pq " 1{p when p " e ? n . In that case, the posterior variance of 1 ´π is -1{p 2 . To sum up, in a model with -? n{ ln p predictors selected, the posterior distribution of 1 ´π is very concentrated around 1{p which conforms to (H.1) and (H.2). This is not directly comparable due to the different procedures, but it seems interesting to observe that the same orders of magnitude are involved to reach theoretical guarantees for the estimation of ∆.

In the next section, an approach is suggested to deal with sparse-group sparsity in ∆, for a bi-level selection.

The sparse-group-sparse setting

To produce a sparse model both at the variable level (for variable selection) and at the group level (for group selection), it seems natural to carry on with our strategy by introducing another spike-and-slab effect into the first one. The predictors are still ordered in m groups of sizes κ 1 `. . . `κm " p. For the g-th group (1 ď g ď m), λ g P R is the g-th component of λ P R m and, for the i-th predictor of this group (1 ď i ď κ g ), ν gi P R is the i-th component of ν g P R κg . The i-th column of the covariate submatrix X g is X gi P R n and the corresponding slice of ∆ g is ∆ gi P R q while ∆ gzi P R qˆpκg´1q is ∆ g deprived Proposition 3.4.1. In the hierarchical model (6), the conditional posterior distributions are as follows.

´The parameter ∆ gi satisfies, for g P 1, m and i P 1, κ g , ∆ gi | Θ ∆gi " p1 ´pgi q N q p´s gi H gi , s gi Ω y q `pgi δ 0 where

H gi " Ω y Y t X gi `ÿ h,j ‰ g,i xX gi , X hj y ∆ hj , s gi " ν gi λ g 1 `νgi λ g }X gi } 2 and p gi "
ρ gi ρ gi `p1 ´π1 q p1 ´π2 q p1 `νgi λ g }X gi } 2 q ´q 2 exp ´sgi H t gi Ω ´1 y Hgi 2 īn which ρ gi " p1 ´π1 q π 2 1 t∆ gzi ‰ 0u `π1 1 t∆ gzi " 0u .

´The parameter Ω y satisfies

Ω y | Θ Ωy " MGIG q ˆn ´p `N0 `u 2 , ∆ pX t X `D´1 λν q ∆ t , Y t Y `V ´1ẇ here D λν " diagpν 11 λ 1 , . . . , ν 1κ1 λ 1 , . . . , ν m1 λ m , . . . , ν mκm λ m q.
´The parameter ν satisfies, for g P 1, m and i P 1, κ g ,

ν gi | Θ νgi " 1 t∆gi ‰ 0u GIG ˜1 2 , ∆ t gi Ω ´1 y ∆ gi λ g , 2 ℓ gi ¸`1 t∆gi " 0u Γpα, ℓ gi q.
´The parameter λ satisfies, for g P 1, m ,

λ g | Θ λg " 1 t∆g ‰ 0u GIG ˆqN 0g `1 2
, trpD ´1 νg ∆ t g Ω ´1 y ∆ g q, 2 γ g ˙`1 t∆g " 0u Γpα g , γ g q where D νg " diagpν g1 , . . . , ν gκg q.

´The parameter π satisfies, for j P 1, 2 ,

π j | Θ πj " β `Aj `aj , B j `bj ˘.
where A 1 " G 0 , B 1 " m ´G0 , A 2 " J 0 and B 2 " p ´N0 .

Proof. See Section 3.5.3.

It only remains to give the explicit results for the particular case q " 1. The direct links form a row vector such that ∆ t P R p with groups ∆ t g P R κg (1 ď g ď m) containing predictors ∆ gi P R (1 ď i ď κ g ), and the precision matrix of the responses reduces to ω y ą 0. According to the parametrization of the distributions (see Section 3.1), the corresponding prior distribution of ω y is Γp u 2 , 1 2 v q for u, v ą 0, like in the other settings, and the one of ν gi is Epℓ gi q for ℓ gi ą 0. The other priors are unchanged.

Corollary 3.4.2. In the hierarchical model (6) with q " 1, the conditional posterior distributions are as follows.

´The parameter ∆ gi satisfies, for g P 1, m and i P 1, κ g , ∆ gi | Θ ∆gi " p1 ´pgi q N p´s gi h gi , s gi ω y q `pgi δ 0 where

h gi " ω y xX gi , Y y `ÿ h,j ‰ g,i xX gi , X hj y ∆ hj , s gi " ν gi λ g 1 `νgi λ g }X gi } 2 and p gi "
ρ gi ρ gi `p1 ´π1 q p1 ´π2 q p1 `νgi λ g }X gi } 2 q ´1 2 exp ´sgi h 2 gi 2 ωy īn which ρ gi " p1 ´π1 q π 2 1 t∆ gzi ‰ 0u `π1 1 t∆ gzi " 0u .

´The parameter ω y satisfies

ω y | Θ ωy " GIG ˆn ´p `N0 `u 2 , ∆ pX t X `D´1 λν q ∆ t , Y t Y `1 v ẇhere D λν " diagpν 11 λ 1 , . . . , ν 1κ1 λ 1 , . . . , ν m1 λ m , . . . , ν mκm λ m q.
´The parameter ν satisfies, for g P 1, m and i P 1, κ g ,

ν gi | Θ νgi " 1 t∆gi ‰ 0u GIG ˜1 2 , ∆ 2
gi λ g ω y , 2 ℓ gi ¸`1 t∆gi " 0u Epℓ gi q.

´The parameter λ satisfies, for g P 1, m ,

λ g | Θ λg " 1 t∆g ‰ 0u GIG ˜N0g `1 2 , ∆ g D ´1 νg ∆ t g ω y , 2 γ g ¸`1 t∆g " 0u Γpα g , γ g q
where D νg " diagpν g1 , . . . , ν gκg q.

´The parameter π satisfies, for j P 1, 2 ,

π j | Θ πj " β `Aj `aj , B j `bj ˘.
where

A 1 " G 0 , B 1 " m ´G0 , A 2 " J 0 and B 2 " p ´N0 .
Proof. This is a consequence of Proposition 3.4.1.

In the simulation study of Section 3.6.1, Scen. 5 and 6 are dedicated to the sparse-group-sparse setting. Now, let us prove our assertions by a few computational steps.

Conditional posterior distributions

This section is devoted to the proofs of our assertions for the different settings.

The sparse setting: proof of Proposition 3.2.1

First of all, the full posterior distribution of the parameters conditional on X and Y satisfies

pp∆, Ω y , λ, π | Y, Xq 9 ppY | X, ∆, Ω y q pp∆ | Ω y , λ, πq ppλq ppΩ y q ppπq 9 |Ω y | n 2 exp ˆ´1 2 › › ›pY `X ∆ t Ω ´1 y q Ω 1 2 y › › › 2 F ṗ ź i"1 « 1 ´π a λ q i |Ω y | exp ˜´∆ t i Ω ´1 y ∆ i 2 λ i ¸1t∆i ‰ 0u `π 1 t∆i " 0u ff λ 1 2 pq`1q´1 i e ´ℓi λi ˆ|Ω y | u´q´1 2 exp ˆ´trpV ´1 Ω y q 2 ˙π a´1 p1 ´πq b´1 . ( 9
)
On the one hand, exploiting the cyclic property of the trace, a tedious calculation shows that, for all

1 ď i ď p, › › ›pY `X ∆ t Ω ´1 y q Ω 1 2 y › › › 2 F " trpY t Y Ω y q `2 trpX t Y ∆q `trpX t X ∆ t Ω ´1 y ∆q " }X i } 2 ∆ t i Ω ´1 y ∆ i `2 ÿ j ‰ i xX i , X j y ∆ t j Ω ´1 y ∆ i `2 X t i Y ∆ i `T‰ i (10)
where the term T ‰ i does not depend on ∆ i . Thus,

pp∆ i | Θ ∆i q 9 exp ˜´1 2 }X i } 2 ∆ t i Ω ´1 y ∆ i ´ÿ j ‰ i xX i , X j y ∆ t j Ω ´1 y ∆ i ´X t i Y ∆ i « 1 ´π a λ q i |Ω y | exp ˜´∆ t i Ω ´1 y ∆ i 2 λ i ¸1t∆i ‰ 0u `π 1 t∆i " 0u ff " exp ˆ´1 2 p∆ i `si H i q t ps i Ω y q ´1 p∆ i `si H i q ėxp ˜si H t i Ω ´1 y H i 2 ¸1 ´π a λ q i |Ω y | 1 t∆i ‰ 0u `π 1 t∆i " 0u (11)
for all 1 ď i ď p, where

H i " Ω y Y t X i `ÿ j ‰ i xX i , X j y ∆ j and s i " λ i 1 `λi }X i } 2 .
This is still a multivariate Gaussian spike-and-slab distribution such that, by renormalizing, the spike has probability

p i " Pp∆ i " 0 | Θ ∆i q " π π `p1 ´πq p1 `λi }X i } 2 q ´q 2 exp ´si H t i Ω ´1 y Hi 2 ¯.
On the other hand, coming back to (10), we can also write

› › ›pY `X ∆ t Ω ´1 y q Ω 1 2 y › › › 2 F " trpY t Y Ω y q `trp∆ X t X ∆ t Ω ´1 y q `T‰ y
where T ‰ y does not depend on Ω y . That leads, via (9), to

ppΩ y | Θ Ωy q 9 |Ω y | n´p`N 0 `u´q´1 2 exp ˜´1 2 trppY t Y `V ´1q Ω y q ´1 2 ˜trp∆ X t X ∆ t Ω ´1 y q `ÿ ∆i ‰ 0 ∆ t i Ω ´1 y ∆ i λ i ¸" |Ω y | n´p`N 0 `u´q´1 2 exp ˆ´1 2 tr `pY t Y `V ´1q Ω y `∆ pX t X `D´1 λ q ∆ t Ω ´1 y ˘˙(12)
where N 0 is given in (3) and D λ " diagpλ 1 , . . . , λ p q. Finally, it is easy to see that, for all 1 ď i ď p,

ppλ i | Θ λi q 9 1 ? λ i exp ˜´∆ t i Ω ´1 y ∆ i 2 λ i ´ℓi λ i ¸1t∆i ‰ 0u `λ 1 2 pq`1q´1 i e ´ℓi λi 1 t∆i " 0u (13)
whereas

ppπ | Θ π q 9 π N0`a´1 p1 ´πq p´N0`b´1 . ( 14
)
We recognize in (11), ( 12), ( 13) and ( 14) the announced conditional posterior distributions, which concludes the proof.

The group-sparse setting: proof of Proposition 3.3.1

The full posterior distribution of the parameters conditional on X and Y satisfies

pp∆, Ω y , λ, π | Y, Xq 9 ppY | X, ∆, Ω y q pp∆ | Ω y , λ, πq ppλq ppΩ y q ppπq 9 |Ω y | n 2 exp ˆ´1 2 › › ›pY `X ∆ t Ω ´1 y q Ω 1 2 y › › › 2 F ṁ ź g"1 « 1 ´π b λ q κg g |Ω y | κg exp ˜´trp∆ t g Ω ´1 y ∆ g q 2 λ g ¸1t∆g ‰ 0u `π 1 t∆g " 0u ff λ 1 2 pq κg`1q´1 g e ´ℓg λg ˆ|Ω y | u´q´1 2 exp ˆ´trpV ´1 Ω y q 2 ˙π a´1 p1 ´πq b´1 . ( 15
)
Like in the previous proof, a first important step is to note that, for all 1 ď g ď m,

› › ›pY `X ∆ t Ω ´1 y q Ω 1 2 y › › › 2 F " › › › › › Y Ω 1 2 y `m ÿ j"1 X j ∆ t j Ω ´1 2 y › › › › › 2 F " }X g ∆ t g Ω ´1 2 y } 2 F `2 ÿ j ‰ g trp∆ j X t j X g ∆ t g Ω ´1 y q `2 trpX t g Y ∆ g q `T‰ g (16)
where the term T ‰ g does not depend on ∆ g . Thus, after a tedious calculation exploiting the cyclic property of the trace, one can obtain the factorization

pp∆ g | Θ ∆g q 9 exp ˜´1 2 }X g ∆ t g Ω ´1 2 y } 2 F ´ÿ j ‰ g trp∆ j X t j X g ∆ t g Ω ´1 y q ´trpX t g Y ∆ g q » - 1 ´π b λ q κg g |Ω y | κg exp ˜´trp∆ t g Ω ´1 y ∆ g q 2 λ g ¸1t∆g ‰ 0u `π 1 t∆g " 0u fi fl " exp ˆ´1 2 tr `S´1 g p∆ g `Hg S g q t Ω ´1 y p∆ g `Hg S g q ˘ėxp ˜trpH t g Ω ´1 y H g S g q 2 ¸1 ´π b λ q κg g |Ω y | κg 1 t∆g ‰ 0u `π 1 t∆g " 0u (17)
for all 1 ď g ď m, where

H g " Ω y Y t X g `ÿ j ‰ g ∆ j X t j X g and S g " λ g `Iκg `λg X t g X g ˘´1 .
We recognize the announced Gaussian spike-and-slab distribution, and the probability of the spike is given, after renormalization, by

p g " Pp∆ g " 0 | Θ ∆g q " π π `p1 ´πq |I κg `λg X t g X g | ´q 2 exp ´trpH t g Ω ´1 y Hg Sgq 2 ¯.
Following the same lines as the ones used to establish (12), we obtain from (15) the conditional distribution

ppΩ y | Θ Ωy q 9 |Ω y | n´p`N 0 `u´q´1 2 exp ˜´1 2 trppY t Y `V ´1q Ω y q ´1 2 ˜trp∆ X t X ∆ t Ω ´1 y q `ÿ ∆g ‰ 0 trp∆ t g Ω ´1 y ∆ g q λ g ¸" |Ω y | n´p`N 0 `u´q´1 2 exp ˆ´1 2 tr `pY t Y `V ´1q Ω y `∆ pX t X `D´1 λ q ∆ t Ω ´1 y ˘˙( 18
)
where D λ " diagpλ 1 , . . . , λ 1 , . . . , λ m , . . . , λ m q with each λ g duplicated κ g times, and since we can note that, due to the continuous nature of ∆ |t∆ ‰ 0u,

m ÿ g"1 κ g 1 t∆g ‰ 0u " p ´N0
for N 0 given in (3). Next, we obtain in a simpler way that, for all 1 ď g ď m,

ppλ g | Θ λg q 9 1 a λ g exp ˜´trp∆ t g Ω ´1 y ∆ g q 2 λ g ´ℓg λ g ¸1t∆g ‰ 0u `λ 1 2 pq κg`1q´1 g e ´ℓg λg 1 t∆g " 0u . (19)
Finally,

ppπ | Θ π q 9 π G0`a´1 p1 ´πq m´G0`b´1 (20)
where G 0 is defined in (5). We can check that the conditional distributions ( 17), ( 18), ( 19) and ( 20) correspond to the ones announced in the proposition, which concludes the proof.

The sparse-group-sparse setting: proof of Proposition 3.4.1

The full posterior distribution of the parameters conditional on X and Y satisfies pp∆, Ω y , ν, λ, π | Y, Xq 9 ppY | X, ∆, Ω y q pp∆ | Ω y , ν, λ, πq ppνq ppλq ppΩ y q ppπq

9 |Ω y | n 2 exp ˆ´1 2 › › ›pY `X ∆ t Ω ´1 y q Ω 1 2 y › › › 2 F ṁ ź g"1 « `p1 ´π1 q P g 1 t∆g ‰ 0u `π1 1 t∆g " 0u λ 1 2 pq κg`1q´1 g e ´γg λg κg ź i"1 ν 1 2 pq`1q´1 gi e ´ℓgi νgi ff ˆ|Ω y | u´q´1 2 exp ˆ´trpV ´1 Ω y q 2 ˙2 ź j"1 π aj ´1 j p1 ´πj q bj ´1 (21)
where, for 1 ď g ď m,

P g " κg ź i"1 « 1 ´π2 a pν gi λ g q q |Ω y | exp ˜´∆ t gi Ω ´1 y ∆ gi 2 ν gi λ g ¸1t∆gi ‰ 0u `π2 1 t∆gi " 0u ff .
Using the same decompositions as (10) or ( 16), the full posterior distribution given above leads to

pp∆ gi | Θ ∆gi q 9 exp ˜´1 2 }X gi } 2 ∆ t gi Ω ´1 y ∆ gi ´ÿ h,j ‰ g,i xX gi , X hj y ∆ t hj Ω ´1 y ∆ gi ´X t gi Y ∆ gi «p1 ´π1 q « 1 ´π2 a pν gi λ g q q |Ω y | exp ˜´∆ t gi Ω ´1 y ∆ gi 2 ν gi λ g ¸1t∆gi ‰ 0u `π2 1 t∆gi " 0u ff 1 t∆g ‰ 0u `π1 1 t∆g " 0u ff " exp ˆ´1 2 p∆ gi `sgi H gi q t ps gi Ω y q ´1 p∆ gi `sgi H gi q ėxp ˜sgi H t gi Ω ´1 y H gi 2 ¸p1 ´π1 q p1 ´π2 q a pν gi λ g q q |Ω y | 1 t∆gi ‰ 0u ``p1 ´π1 q π 2 1 t∆ gzi ‰ 0u `π1 1 t∆ gzi " 0u ˘1t∆gi " 0u (22)
for 1 ď g ď m and 1 ď i ď κ g , where ∆ gzi is ∆ g deprived of ∆ gi ,

H gi " Ω y Y t X gi `ÿ h,j ‰ g,i
xX gi , X hj y ∆ hj and s gi "

ν gi λ g 1 `νgi λ g }X gi } 2 .
Here, we used the binary equalities stemming from t∆ gi ‰ 0u X t∆ g ‰ 0u " t∆ gi ‰ 0u, t∆ gi " 0u X t∆ g ‰ 0u " t∆ gi " 0u X t∆ gzi ‰ 0u and t∆ gi " 0u X t∆ g " 0u " t∆ gi " 0u X t∆ gzi " 0u, which turn out to be very useful to separate ∆ gi and Θ ∆gi . This is characteristic of a multivariate Gaussian spike-and-slab distribution. By renormalizing, one can see that the spike has probability

p gi " Pp∆ gi " 0 | Θ ∆gi q " ρ gi ρ gi `p1 ´π1 q p1 ´π2 q p1 `νgi λ g }X gi } 2 q ´q 2 exp ´sgi H t gi Ω ´1 y Hgi 2 with ρ gi " p1 ´π1 q π 2 1 t∆ gzi ‰ 0u `π1 1 t∆ gzi " 0u .
Next, following (21) and the reasoning used to establish (12), we may also write

ppΩ y | Θ Ωy q 9 |Ω y | n´p`N 0 `u´q´1 2 exp ˜´1 2 trppY t Y `V ´1q Ω y q ´1 2 ˜trp∆ X t X ∆ t Ω ´1 y q `ÿ ∆gi ‰ 0 ∆ t gi Ω ´1 y ∆ gi ν gi λ g ¸" |Ω y | n´p`N 0 `u´q´1 2 exp ˆ´1 2 tr `pY t Y `V ´1q Ω y `∆ pX t X `D´1 λν q ∆ t Ω ´1 y ˘˙(23)
where N 0 is given in (3) and D λν " diagpν 11 λ 1 , . . . , ν 1κ1 λ 1 , . . . , ν m1 λ m , . . . , ν mκm λ m q. The shrinkage parameters ν and λ are easier to handle. For 1 ď g ď m and 1 ď i ď κ g ,

ppν gi | Θ νgi q 9 1 ? ν gi exp ˜´∆ t gi Ω ´1 y ∆ gi 2 ν gi λ g ´ℓgi ν gi ¸1t∆gi ‰ 0u `ν 1 2 pq`1q´1 gi e ´ℓgi νgi 1 t∆gi " 0u (24)
whereas

ppλ g | Θ λg q 9 λ qN 0g ´1 2 g exp ˜´trpD ´1 νg ∆ t g Ω ´1 y ∆ g q 2 λ g ´γg λ g ¸1t∆g ‰ 0u `λ 1 2 pq κg`1q´1 g e ´γg λg 1 t∆g " 0u (25)
where N 0g is defined in (7) and D νg " diagpν g1 , . . . , ν gκg q. Finally,

ppπ 1 | Θ π1 q 9 π G0`a1´1 1 p1 ´π1 q m´G0`b1´1 (26)
and

ppπ 2 | Θ π2 q 9 π J0`a2´1 2 p1 ´π2 q p´N0`b2´1 (27)
where G 0 and J 0 are given in ( 5) and ( 8), respectively. For the latter result, we used the fact that the number of non-zero columns in the non-zero groups must coincide with the number of non-zero columns of ∆, that is p ´N0 . Like in the previous proofs, we recognize the announced conditional distributions in ( 22), ( 23), ( 24), ( 25), ( 26) and ( 27). That concludes these tedious calculations.

Proof of Proposition 3.3.3

The result is obtained by following the steps of the proof of Theorem 2.1 in [YN20] but, beforehand, we need to clarify a few points to extend the reasoning of the authors from q " 1 to q ě 1 and take into account the adaptative shrinkage. For any model k, let K " tk is selectedu so that K " T when the true model t is considered. First, recall that λ and π are fixed and rewrite (15) like

P ∆ pK | Y, X, Ω y q 9 exp ˆ´1 2 › › ›pY `Xpkq ∆ t pkq Ω ´1 y q Ω 1 2 y › › › 2 F ṗ1 ´πq |k| π |k| a |Λ k | q |Ω y | kr exp ˜´trp∆ t pkq Ω ´1 y ∆ pkq D ´1 k q 2 9 p1 ´πq |k| π |k| a |Λ k | q |Ω y | kr exp ˜trp r ∆ pkq F k r ∆ t pkq Ω ´1 y q 2 ȩxp ˆ´1
2 tr ´p∆ pkq ´r ∆ pkq q F k p∆ pkq ´r ∆ pkq q t Ω ´1 y ¯˙(28)

where

F k " D ´1 k `Xt pkq X pkq , D k " diagppλ ℓ , . . . , λ ℓ q ℓ P k q with each λ ℓ duplicated κ ℓ times, k r " }pκ ℓ q ℓ P k } 1 , Λ k " diagppλ κ ℓ ℓ q ℓ P k q and r ∆ pkq " ´Ωy Y t X pkq F ´1 k .
Then, integrating over ∆ pkq , it follows (see Def. 3.1.1 with Σ 1 " Ω y and Σ 2 "

F ´1 k ) that PpK | Y, X, Ω y q " ˆRqˆκ k P ∆ pK | Y, X, Ω y , λ, πq d∆ pkq 9 p1 ´πq |k| π |k| a |Λ k | q |F k | q exp ˜trp r ∆ pkq F k r ∆ t pkq Ω ´1 y q 2 9 ˆ1 ´π π ˙|k| |Λ k | ´q 2 |F k | ´q 2 exp ˆ´1 2 tr `Y ˚t pI n ´Xpkq F ´1 k X t pkq q Y ˚˘" ˆ1 ´π π ˙|k| |Λ k | ´q 2 |F k | ´q 2 exp ˆ´1 2 ´RSS k p r ∆ pkq q `› › r ∆ pkq D ´1 2 k › › 2 F ¯ẇhere Y ˚" Y Ω 1 2 y , r ∆ pkq " Ω ´1 2 y r ∆ pkq and RSS k : H P R qˆκ k Þ Ñ }Y ˚´X pkq H t } 2
F is the residual sum of squares function in the renormalized linear model indexed by k, that is

Y ˚" ´Xpkq ∆ t pkq Ω ´1 2 y `Eẘ ith E ˚" E Ω 1 2
y " MN nˆq p0, I n , I q q. Thus, the so-called posterior ratio between any false model k and t is given by

PRpk, tq " PpK | Y, X, Ω y q PpT | Y, X, Ω y q " Q k Q t ˆ1 ´π π ˙|k|´|t| e ´1 2 p r R k ´r Rtq with Q k " |Λ k | ´q 2 |F k | ´q 2 and r R k " RSS k p r ∆ pkq q `} r ∆ pkq D ´1 2 k } 2 F
, using the notation of [START_REF] Yang | Consistent group selection with bayesian high dimensional modeling[END_REF]. In particular, due to the generalized ridge penalty, r ∆ pkq " arg min

H ´RSS k pHq `› › HD ´1 2 k › › 2 F ¯(29) so that for nested models k 1 and k 2 (with k 1 Ď k 2 ), we must have r R k2 ď r R k1 . Let also R k " }pI n Πpkq q Y ˚}2 F " }pI q b pI n ´Πpkq qq vecpY ˚q} 2 2 .
Cochran's theorem entails the chi-squared distributions R t " χ 2 pq pn ´rt qq and R t ´Rk " χ 2 pq pr k ´rt qq for any 'bigger' model k Ą t and q ě 1. Combining all these preliminary considerations, the strategy of [START_REF] Yang | Consistent group selection with bayesian high dimensional modeling[END_REF] now applies and leads, under our revised

hypotheses, to 1 ´PpT | Y, X, Ω y q PpT | Y, X, Ω y q " ÿ k ‰ t PRpk, tq P ÝÑ 0.
Matsumoto-Yor property (see Theorem 3.1 of [START_REF] Massam | The Matsumoto-Yor property and the structure of the Wishart distribution[END_REF]) to get a MGIG d sample from the very standard GIG and W d distributions, is unfortunately inapplicable in our context. Indeed, for example in the sparse setting, that would require finding z P R q such that Y t Y `V ´1 " b zz t for some b ą 0, which is clearly impossible since Y t Y `V ´1 has full rank. In [START_REF] Fazayeli | The Matrix Generalized Inverse Gaussian distribution: properties and applications[END_REF], the authors show that MGIG d pν, A, Bq is a unimodal distribution of which mode M P S d ``is the unique solution of the algebraic Riccati equation pd `1 ´2 νq M `M BM " A, and a standard importance sampling approach follows for the mean of the distribution. Our fallback solution is to solve this Riccati equation at each step and to replace all MGIG d random variables by the (unique) mode of the consecutive distributions. To assess the credibility of this ad hoc sampling, the 'oracle' models in which Ω y and the shrinkage parameters are known are added to the simulations. We will see that, despite an unavoidable loss, the results remain pretty consistent. In particular, the support recovery does not appear to be impacted.

A simulation study

In this empirical section, the matrix of order d ě 1 given by C d " `ρ|i´j| ˘1 ď i,j ď d will be used as a typical covariance structure, for some 0 ď ρ ă 1. Thus, the precision matrices will be chosen as a multiple of C ´1 d to keep the same guideline in our simulations. The responses Y i " B t X i `Ei are generated through relations (1) where, for all 1 ď k ď n, E i " N p0, Rq. Because our models assume prior independence (or group-independence) in the columns of ∆, it seems necessary to look at the influence of correlation among the predictors. So the standard choice X k " N p0, I p q is first considered, but in some cases we will also test X k " N p0, C p q for ρ " 0.5 and ρ " 0.9 to introduce a significant correlation between close predictors (see Figure 3.1). For each experiment, the support recovery of ∆ is evaluated thanks to the so-called F -score given by F " 2 p r r e p r `re where p r " TP TP `FP and r e " TP TP `FN are the precision and the recall, respectively, and where T/F and P/N stand for true/false and positive/negative. To assess prediction skills, n e randomly chosen observations are used for estimation (for different n e ) and the remaining n v " n ´ne " 100 independent observations serve to compute the mean squared prediction error (MSPE). The results are compared to the ones obtained via the penalized maximum of likelihood (PML) approach of [START_REF] Yuan | Partial Gaussian graphical model estimation[END_REF] thanks to the correctly adapted implementations of [START_REF] Chiquet | Structured regularization for conditional Gaussian graphical models[END_REF] and [START_REF] Okome | A partial graphical model with a structural prior on the direct links between predictors and responses[END_REF], with a cross-validated tuning parameter. In addition, we compute the sparse precision matrix estimations given by the graphical Lasso (GLasso) of [START_REF] Friedman | Sparse inverse covariance estimation with the graphical Lasso[END_REF], and by the CLIME algorithm of [START_REF] Cai | A constrained ℓ 1 minimization approach to sparse precision matrix estimation[END_REF], using the R packages glasso and fastclime, respectively. Note that we always keep a small value for q, so ∆ is penalized but not Ω y when possible (PML and GLasso). Finally, the recent approach of [START_REF] Ren | Asymptotic normality and optimalities in estimation of large Gaussian graphical models[END_REF], called ANT and based on the individual estimations of the partial correlations, is also implemented. Unlike PML, GLasso and CLIME, sparsity is not the result of penalizations for ANT but, instead, a threshold is deduced from the asymptotic normality of the estimates to decide which are significant and which can be set to zero. Let us add some preliminary comments about the methods compared in these simulations, all related to high-dimensional precision matrix estimation.

´There is a important advantage in favor of our Bayesian approaches, PML and ANT because they do not need the estimation of Ω x P S p ``. Indeed, extracting the estimation of ∆ P R qˆp and Ω y P S q `from that of the full precision matrix Ω P S q`p ``may generate a drastic bias when p " q, and that explains in particular why GLasso and CLIME give pretty bad results in what follows.

´In its standard version, ANT is not designed to produce column-sparsity or group-sparsity in ∆. So, by considering multiple testing at the column or even group level, we allow groups of coefficients to be zeroed simultaneously. We have observed that this modified ANT method (called ANT* in the simulations) loses a bit in prediction quality but is greatly improved for support recovery.

´Unfortunately, this is not appropriate for PML, GLasso and CLIME. It is therefore not surprising that they are largely outperformed by our Bayesian models and ANT* for (gs) and (sgs). Using grouppenalties, which to the best of our knowledge still does not exist, should improve the results of these methods to some extent.

The seven scenarios below, from Scen. 0 to Scen. 6, as heterogeneous as possible, represent the diversity of the situations (high-dimensionality, kind of sparsity, dimension of the responses, coefficients hard to detect, etc.). We repeat each one N " 100 or N " 50 times, depending on the computation times involved, and the numerical results for n e " 400 and uncorrelated predictors are summarized in Table 3.1. In addition, the evolution of MSPE is represented on Figure 3.1 for Scen. 1, 3 and 5, when n e grows from 100 to 500, both for uncorrelated and correlated predictors. The three configurations (s), (gs) and

(sgs) are tested on the grouped scenarios (from Scen. 3 to Scen. 6) with the adaptative shrinkage.

´Scenario 0 (small dimension, no sparsity). Let q " 1, p " 5 and set ω y " 1. We fill ∆ with N p0, 2 ω y q coefficients.

´Scenario 1 (sparse direct links, univariate responses). Let q " 1, p " 50 and set ω y " 1. We randomly choose 10 locations of ∆ filled with N p0, ω y q coefficients while the others are zero.

´Scenario 2 (sparse direct links, multivariate responses). Let q " 2, p " 80 and set Ω y " 2 C ´1 2 with ρ " 0.5. We randomly choose 10 columns of ∆ filled with N 2 p0, Ω y q coefficients while the others are zero.

´Scenario 3 (group-sparse direct links, univariate responses). Let q " 1, p " 320 and set ω y " 1. We consider m " 5 groups of size 100, 10, 100, 10 and 100. The two groups of size 10 are filled with N p0, 0.5 ω y q and N p0, ω y q coefficients, respectively, while the other groups are zero.

´Scenario 4 (group-sparse direct links, multivariate responses). Let q " 3, p " 500 and set Ω y " 3 C ´1 3 with ρ " 0.5. We divide the columns of ∆ into m " 25 groups of size 20. We randomly choose 3 groups filled with N 3 p0, 0.5 Ω y q, N 3 p0, Ω y q and N 3 p0, 1.5 Ω y q coefficients, respectively, while the other groups are zero.

´Scenario 5 (sparse-group-sparse direct links, univariate responses). Let q " 1, p " 150 and set ω y " 1. We consider m " 3 groups of size 50. Only the second group is non-zero, into which we randomly fill 10 locations with N p0, ω y q coefficients. ´Scenario 6 (sparse-group-sparse direct links, multivariate responses). Let q " 5, p " 1000 and set Ω y " 5 C ´1 5 with ρ " 0.5. We divide the columns of ∆ into m " 20 groups of size 50, and a randomly chosen one is half filled with N 5 p0, Ω y q coefficients. The others columns of ∆ are zero. Now, let us try to summarize our observations. In terms of support recovery, the Bayesian spikeand-slab framework and the modified ANT* method give results incomparably better than the sparsityinducing penalized approaches (PML, GLasso and CLIME). As suggested in Remark 3.3 of [START_REF] Okome | A partial graphical model with a structural prior on the direct links between predictors and responses[END_REF],

Mod

this may be a consequence of the fact that the cross-validation steps calibrate the models to reach the best prediction error, sometimes at the cost of support recovery by picking a small penalty level. The superiority of ANT over GLasso and CLIME is recognized and discussed in [START_REF] Ren | Asymptotic normality and optimalities in estimation of large Gaussian graphical models[END_REF], but this also highlights the ability of our Bayesian models to reach good results both in prediction and in support recovery. It can also be seen that (s) gives weaker results than (sgs) in the grouped scenarios, probably Hyperparam. p50, 1q -p3, 1q -p3, 1, 50, 1q p100, 1q -p20, 1q -p20, 1, 50, 1q

Mod

Table 3.1 : Medians of the mean squared prediction errors (with standard deviations), F -scores, precisions and recalls after N " 100 repetitions of Scen. 0 to Scen. 6 (N " 50 for Scen. 4 and Scen. 6), with n e " 400 and uncorrelated predictors. The suffix -or is used to denote 'oracle' settings. The hyperparameters chosen for the prior spike probability are indicated in the last row of each table, from left to right: pa, bq for (s) and (gs), pa 1 , b 1 , a 2 , b 2 q for (sgs).

due to the fact that it does not take into account the group structure, but still better than the penalized methods. However, the computational times involved (see remarks below) make (s) less relevant than (sgs) in these situations, even if the results are not drastically different. Unsurprisingly, (gs) is not suitable in the sparse-group-sparse settings in terms of support recovery. Our experiments show that it is able to identify influential groups without being mistaken but, even though the resulting estimates are small where they should be zero, it is not designed to be used for bi-level selection. Figure 3.1 shows that to 500. The black curves correspond to uncorrelated predictors (ρ " 0) while the blue and red curves correspond to correlated predictors (ρ " 0.5 and ρ " 0.9, respectively).

the results are pretty stable from n e " 200 observations in the learning set: for n e ă 200 the MSPEs are rather chaotic before stabilizing. The same figure also highlights that the presence of correlation in the predictors do not seem to have a significant effect on the estimation procedure, except for small size samples and high correlation where the degradation is noticeable. Overall, the real strength of the Bayesian spike-and-slab approach is clearly the support recovery of the direct links between predictors and responses but it seems that one can hardly expect to deal with very high-dimensional studies as long as we do not impose a group structure or a huge degree of sparsity. The highly competitive MSPEs obtained confirm the relevance of Bayesian PGGMs not only for variable selection but also for prediction purposes in the context of high-dimensional regressions.

Identification of a sparse set of predictors in a real dataset

Let us now study the Hopx dataset, fully described in [PBL `10]. It contains p " 770 genetic markers spread over m " 20 chromosomes from n " 29 inbred rats. It also contains the corresponding measured gene expression levels of q " 4 tissues (adrenal gland, fat, heart and kidney). The goal is to identify a sparse set of predictors that jointly explain the outcomes, with the natural group structure formed by chromosomes (see Table 3.2). This dataset has already been analyzed in [LBC `16], using a Bayesian regression without group structure, and later in [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF] including group and sparse-group structures. So the PGGM is supposed to bring new perspectives about relationships in terms of partial correlations. A particularity of this dataset is that the responses are very correlated, so we should expect an estimation of Ω ´1 y with significant non-diagonal elements and a clear advantage in using PGGMs. Indeed, a predictor considered to be influencing all the outcomes could be the result of a direct relation to one tissue propagated to the others by an artificial correlation effect. As can be seen on Figure 3.2, the predictors are also highly correlated with their neighbors (for the sake of readability, we only represent the correlogram of predictors located on chromosomes 8, 9 and 10).

Chr. The small sample size relative to the number of covariates (29{770) weakens the study. To strengthen our conclusions, we decided to run N " 100 experiments based on 25 randomly chosen observations and to aggregate the results. We first investigate the selection of predictors at the chromosomes scale, i.e. we run (gs) according to the previous protocol with an adaptative shrinkage and we choose pa, bq " p1, 20q in the prior probability π. The empirical distribution of the posterior probability of inclusion for each chromosome is represented on the left of Figure 3.3. The selection procedure focuses on chromosomes 14, 15 and 17 (and not just on chromosomes 2 and 3 as in [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF]) but the estimation process gives an overwhelming advantage to chromosome 14, far ahead of its neighbors. This is undoubtedly the influence of D14Mit3, a marker located on chromosome 14 and known to have a very significant effect on this dataset.

The main conclusion to be drawn at this stage is that chromosome 14 has a positive effect on Fat and a negative effect on Heart, as can also be seen on the right of Figure 3.3. Therefore, it is likely that the overall positive influence of D14Mit3 identified by previous authors is due to the combination of a direct positive link with Fat, a direct negative link with Heart and a correlation effect from the outcomes. This hypothesis is given additional credibility by the numerical results: from (gs), the corresponding column of ∆ is approximately p0.00, 0.04, ´0.09, 0.00q which, through relations (1), leads to p0.15, 0.25, 0.34, 0.21q as estimated regression coefficients. This roughly corresponds to the values indicated in Table 2 of [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF],

at least for the main effect on Heart. Thus for chromosome 14, the numerical results coincide but the interpretations are clearly different. Of course, similar reasonings can be carried out for the less influent chromosomes. It is perhaps more interesting to look for a bi-level selection in order to identify a sparse set of markers and not only chromosomes. In this regard, (sgs) is launched using the same statistical protocol, adaptative shrinkage and hardly informative hypermarameters a 1 " 3, b 1 " 1, a 2 " 1 and b 2 " 1 which happen to be sufficient to generate a huge degree of sparsity. While many chromosomes are excluded from the model given by (gs), with (sgs) we see some contributions localized in certain chromosomes having little influence when taken as a whole. At the markers scale, the randomness of the sampler and the high level of correlation between close predictors probably explain the presence of artifacts which sometimes make it difficult to distinguish the real contributions from the background noise. We therefore use the N " 100 experiments to build 95% confidence intervals and keep only significant estimates. By way of example, Figure 3.4 displays the results obtained on chromosomes 7, 8 and 14. The main markers standing out are summarized in Table 3.3 together with the kind of direct influences detected. Markers already highlighted in [LBC `16] or [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF] are also indicated. One can see that most of our conclusions coincide, but new markers are suggested (especially on chromosome 8) and others have disappeared. Overall, the more stringent statistical protocol that we used led to the retention of fewer predictors with more guarantee.

An important consequence of this study is the new interpretations in terms of direct influences allowed by PGGMs. Especially as the residual correlations, hidden in the estimation of R " Ω ´1 y and closely related to the correlations between the responses, are very high (greater than 0.7), as we suspected from Figure ced MCMC methods may also be useful or novel computational strategies like the 'shotgun' stochastic algorithm of [START_REF] Yang | Consistent group selection with bayesian high dimensional modeling[END_REF]. On the other hand, the procedures are obviously very sensitive to the initialization of the sampler, especially when p " n. For example, the term |I κg `λg X t g X g | is likely to explode when κ g is large and λ g ą 1, that is why λ g has to be carefully controlled via an accurate initial choice of ℓ g . Our heuristic approach is to initialize ℓ g such that Erλ g s ă 1 to control the behavior of |I κg `λg X t g X g | during the first iterations. This works pretty well in practice, but needs to be done on a case-by-case basis, which could be improved. From a theoretical point of view, we should obviously enhance the estimation procedure by sampling from the MGIG q distribution for q ą 1, and not using the mode. Our fallback solution gives satisfactory but not completely rigorous results. In addition, it could be interesting to generalize the support recovery guarantee of Proposition 3.3.3 to (sgs), which is certainly possible at the cost of a few additional developments. Overall, our study shows that for the moderate values of p (up to 10 3 or 10 4 ), the Bayesian approach of the partial Gaussian graphical models is a very serious alternative to the frequentist penalized estimations, for prediction but also and especially for support recovery.

INTRODUCTION TO SURVIVAL ANALYSIS

S

urvival analysis is a branch of Data Science that studies the time elapsed until the occurrence of a binary event, over a fixed observation period. Due to the convenience of this method to address an issue present in various research areas, it is used in several disciplines. In economics the event of interest could be the bursting of a speculative bubble, in industry the occurrence of a machine breakdown, and in sociology the obtaining of the first post-graduation job. Although for this same type of analysis the names differ according to the field (failure time analysis, reliability analysis, duration modeling, etc.), the questions treated and the statistical tools used are similar.

In the medical field, survival analysis exploits data from cohort surveys or clinical trials in order to extract information likely to help in designing effective therapeutic follow-up of patients. For example, it will be possible to study the impact of a treatment on the remission time of individuals, to extract the variables that may play an important role in the diagnosis, or to classify individuals according to their level of risk of recurrence [Chr87][ZCa16].

In this chapter, we provide an introduction to survival analysis in the context of modeling Triple Negative Breast Cancer (TNBC). The notions presented here will serve as a support for understanding Chapter 5, which deals with the issue of feature selection in survival analysis. We will rely mainly on the books by Kleinbaum and Klein [KK10], Lee and Wang [START_REF] Lee | Statistical methods for survival data analysis[END_REF], and Quigley [O'Q08], as well as various articles that we will present in due course.

Background and terminology

We are in a medical context where the data processed come from cohort surveys. A cohort survey can be divided into two phases. A first phase, known as the inclusion phase, during which patients who will serve as the basis of the study are recruited ; and a second phase, known as the follow-up phase, consisting of the therapeutic follow-up of patients and the collection of survey data until the event of interest occurs or the project ends. This type of survey produces very atypical data, in terms of survival time. First of all, the follow-up start date differs from patient to patient since it depends on the beginning of their exposure to risk (e.g., date of screening, start or end of treatment, etc.). In addition, since the disease progresses differently in each individual, the follow-up end date will also be specific to the patient. As shown in Figure 4.1, we can therefore observe three different situations.

1. The patient experienced the event during the project. This is the case for patients 3 and 5, their survival times are then observed.

2. The patient did not experience the event until the study ends. This is the case for patients 1 and 4, their survival times are not observed.

3. The patient's follow-up was interrupted during the study without them experiencing the event.

This is the case of patient 2, they represent individuals who leave the project for personal reasons (e.g. moving, voluntary discontinuation of treatment, death not related to the disease), they are considered lost to follow-up and their survival times are not observed. In this chapter, we present tools that can exploit these heterogeneous data to derive information that may help in therapeutic decision-making. Before going any further, let us review and define the vocabulary specific to cohort studies that we will be using in this introduction.

Time

Start of inclusion

End of inclusion

´Event or death: the terms death and event are synonymous in survival analysis and simply refer to the phenomenon of interest.

´Start of inclusion: refers to the beginning of the study project ; the date from which the patients are recruited.

´End of inclusion: refers to the date on which recruitment of new patients ends. The individuals who joined the project during the inclusion period constitute the final cohort sample.

´End of study: refers to the closing date of the trial, and therefore the end of patient follow-up. Any event occurring after this date will not be observed.

´Entry date: corresponds to the follow-up start date of an individual, in other words their T 0 . This date must correspond to the beginning of the individual's exposure, it is important to determine the event with which it is associated in order to have a good interpretation of the study results.

In the case of breast cancer, this may be the date of cancer screening, a medical examination, the start of treatment or a surgical operation.

´Date of event: corresponds to the date on which the event occurred.

´Date of last follow-up: corresponds to the date of the last news of an individual lost to follow-up, i.e. who left the clinical trial before the event occurs.

´Survival time: corresponds to the time elapsed between the entry date of an individual and their date of event.

Survival model

The main idea of survival analysis is to determine the impact of individual characteristics on survival time.

In the context of breast cancer, this would allow us to determine, for example, which patient profiles have a high probability of remission, and conversely which patient profiles will be more exposed to recurrence or death, and why. We might think of a standard linear regression problem defined as follows

T i " β t X i `ϵi ,
where T i corresponds to the survival time of individual i, X i P R p the values by individual i on the p predictor variables, β P R p the vector of regression coefficients, and ϵ i a noise term. However, we have seen previously that cohort survey data may include individuals who did not experience the event during their follow-up. Since their survival time is not observed, this creates a problem of missing informative data on the variable T . In this case, the use of standard linear regression is therefore excluded. An intuitive idea to get around this problem would be to create the regression model using only the individuals whose survival time is reported. This idea cannot be retained because it would introduce a significant bias in the study results. First, it means working under the assumption that the event will be observed for all individuals in the population, but it also implies a loss of information about the omitted individuals.

Indeed, although the latter did not observe the event during the survey, we know that they survived at least up to some observed time. This is important information that survival analysis models can exploit by introducing the notion of censoring.

Censoring

Censoring occurs when we have some information on the survival of an individual, but the date of death is not explicitly known. The idea is to observe not only the realizations of the survival time variable T , but also the quadruplet pY, C, T, δq, where for each individual i, we will denote Y i their observed survival time (follow-up time), C i their censoring time, T i their real survival time, and δ i their event status.

The definition of the variables Y and C depends on the type of censoring used. We present the three most widespread, their formulation is conditioned by the information previously available on individuals' survival time.

Left censoring

Left censoring applies when the patient's actual survival time is less than their observed survival time.

This phenomenon occurs in particular when the start date of exposure is not observable, and in studies where patients may observe the event outside the observation period. An example that illustrates this first case is a screening test. The patient contracts the disease before the test results are available.

i Context study subjects: patients with initially non-metastatic cancer, event (e): occurrence of first metastasis.

C: metastases detection e ?

i: patient who experienced the event before the test for metastases. The variables Y and δ are then defined as follows:

Y " maxpT, Cq ; δ " 1 tT ěCu .

Right censoring

Right censoring applies when the patient did not observe the event during their follow-up period. We then know that their actual survival time is greater than their observed survival time. This form of censoring is very common in analyses with cohort survey data, because even when the protocol provides for continuous monitoring, this is limited in time and some individuals leave the project before the event occurs. There are three subtypes of right censoring, in this report we will only discuss type 3 which is very common in clinical research. The variables Y and δ are then defined as follows:

Y " minpT, Cq ; δ " 1 tT ěCu .

Interval censoring

Interval censoring applies when the date of the event falls between two known dates. Like left censoring, it is found in studies where the monitoring is discontinuous ; patients can then observe the event between two observation periods. Interval censoring can be seen as a generalization of left and right censoring. It i Context study subjects: patients with initially non-metastatic cancer, event (e): occurrence of first metastasis.

metastases detection e ?

i: patient who experienced the event between test 2 and 3.

no metastasis also occurs in many clinical studies, but is ignored for convenience, assuming that the date of observation coincides with the date of the event [START_REF] Rodrigues | Use of interval-censored survival data as an alternative to kaplan-meier survival curves: studies of oral lesion occurrence in liver transplants and cancer recurrence[END_REF]. Otherwise Y is defined as follows:

Y " # sC l ; C r s if δ " 1, sC l ; 8r if δ " 0.

Survival time distribution

In survival analysis, estimating the probability distribution of the variable T is a crucial point. This one is characterized and can be defined by different functions (i.e survival function, density function, cumulative distribution function...), but we will see later on that the survival and hazard functions are the most manipulated in practice. From now on, we will assume that T is a continuous non-negative variable, and denote f ptq its density function and F ptq its cumulative distribution function.

The survival function

The survival function Sptq models the probability of an individual to survive beyond a fixed time t. In other words, it returns the probability that the patient has not yet observed the event by this instant t.

The survival function is defined by

Sptq " PpT ą tq, for t ą 0.

(

) 1
Sptq is in theory a continuous and decreasing function decreasing from 1 to 0 when t goes from 0 to `8. However, in practice its estimate is discretized according to the training data, resulting in a step function.

The hazard function

The hazard function hptq, also known as instantaneous risk rate, returns an indication of the probability of experiencing the event within a short time interval after t, given that the patient has survived up until t. In this sense, it makes it possible to determine the periods during which individuals are most exposed to the instantaneous risk of death, hence its name. The hazard function is defined as follows:

hptq " lim hÑ0 Ppt ă T ď t `h|T ą tq h " f ptq Sptq " ´lnpSptqq 1 , for t ą 0. (2)
Like with the survival curves, it is recommended to illustrate individuals' hazard functions graphically, to get a better interpretation of their instantaneous risk evolution over time. Figure 4.6 illustrates examples of hazard curves. We will see later that the pattern of these curves can give us some indications on the model distribution family, and thus conversely, it will be possible to make assumptions on the model law from prior information about the evolution of the risk over time.

The cumulative hazard function

The cumulative hazard function represents, by definition, the quantity of risk accumulated over a given period of time. It is written as We will now present some basic estimation methods, which are essential for this introductory chapter.

Hptq " ˆt 0 hpuqdu " ´lnpSptqq, for t ą 0. (3)
For this purpose, let us consider now and in all the chapter, the sample of n independent observations pY, δ, Xq, where for each patient i we have the triplet pY i , δ i , X i q. Y i P R `is the observed survival time (i.e. real or censoring time), δ i P t0, 1u is the event status, and X i P R p stands for the individual characteristics on the p predictor variables.

Non-parametric estimation -the Kaplan-Meier estimator

The Kaplan-Meier model, also called product limit estimator, provides a non-parametric estimate of the survival function Sptq . It is based on the fact that a patient will have a survival time greater than a fixed time t, if they were always at risk before t and did not experience the event at t. Thus, considering two times t i and t i´1 such that t i ą t i´1 and t i´1 ě 0, we can see that

PpT ą t i q " PpT ą t i , T ą t i´1 q " PpT ą t i |T ą t i´1 qPpT ą t i´1 q.
Suppose we have a set of distinct ordered times of death pt i q, such that for i " 1, ..., n, t i´1 ă t i and t 0 " 0, by induction we have

PpT ą t k q " k ź i"1 PpT ą t i |T ą t i´1 q, with k P 1, n . (4)
The Kaplan-Meier approach estimates the conditional probability of surviving at time Y i by the proportion of patients still at risk at Y i and who did not experience the event at that time. Let n i be the number of individuals at risk at time Y i (i.e. the number of patients present in the study at Y i ), and e i the number of events at Y i , we have

PKM pT ą Y i |T ą Y i´1 q " 1 ´ei n i .
Thus, under the assumption of distinct times Y i , the Kaplan-Meier estimator is given by ŜKM ptq "

ź i:Yiět ˆ1 ´δi n i ˙" ź i:Yiět ˆn ´i n ´i `1 ˙δi . ( 5
)
Under the assumption of independence between survival and censoring times, it is possible to show that ŜKM ptq is a consistent maximum likelihood estimator with negligible bias [START_REF] Kaplan | Nonparametric estimation from incomplete observations[END_REF].

The Kaplan-Meier estimator does not incorporate individual characteristics in its estimation procedure, which limits its scope of application. In practice, this method is used more for descriptive statistics purposes than for fitting a predictive model. In particular, Kaplan-Meier curves can be used to obtain a general idea of the evolution of the survival probability, or to compare survival among independent groups distinguished by a specific characteristic (e.g. the treatment received).

Semi-parametric estimation -Cox model

Semi-parametric estimation aims to build a reliable model for which a minimum of assumptions are made. In survival analysis and especially in medical field, the Cox proportional hazard model [START_REF] David R Cox | Partial likelihood[END_REF] is by far the most widely used. Its popularity stems from its robustness which allows it to approximate the results of a properly constructed parametric model. We will see, however, that this flexibility requires compromises.

Proportional hazard models

Proportional hazard models express a multiplicative effect of predictors on the hazard function. The latter is defined as the product of a baseline hazard function that depends only on time, and a positive function which represents the impact of individual characteristics. It is given by

hpt|Xq " h 0 ptq Rpβ, Xq, @ t ą 0, (6)
where β is the risk parameter to be estimated. Note that it is important to ensure the positivity of the function Rpβ, Xq over the range of possible values for β and X, in order to maintain the interpretability of the hazard function discussed in section 4.4.3. The hazard function thus defined presents a strong assumption that should be evaluated, namely the effect of explanatory variables does not vary over time.

Thus, for any pair of individuals i and j, who take the values X i and X j on the predictors, the ratio of their hazard functions is constant over time. Equivalently, we may say that the risk associated with individual i is proportional to the risk associated with individual j, and the proportionality constant does not depend on time.

HR ij " hpt|X i q hpt|X j q " Rpβ, X i q Rpβ, X j q ðñ hpt|X i q " Rpβ, X i q Rpβ, X j q hpt|X j q, @ t ą 0, @ i, j.

These models take their name from this property. Furthermore, the hazard ratio is a measure of the predictor effects. It has a similar interpretation to the odds ratio in logistic regression. Let be a patient i, a reference patient j, and their estimated hazard ratio y HR ij , ´if y HR ij " 1, there were no significant differences between the two patients, ´si y HR ij ą 1, the risk for patient i is greater than for patient j, ´si y HR ij ă 1, the risk for patient i is lower than for patient j.

Similarly, it enables to compare the risk between different groups of patients.

The Cox proportional hazards model

The Cox model is defined as follows hpt|Xq " h 0 ptq exppβ t Xq, @ t ą 0.

Here the relative risk function Rpβ, Xq is written as the exponential of a linear expression between predictors and regression coefficients, and the baseline hazard function describes the risk for individuals with X 0 " 0 P R p , who can serve as a reference. The hazard ratio for two individuals i and j is given by hpt|X i q hpt|X j q " exppβ t pX i ´Xj qq, @t ą 0.

Although it assumes a parametric form for the relative risk, the Cox model imposes no assumptions on the baseline, and focus on inference that allowed h 0 ptq to remain arbitrary. Therefore, its estimation is based on a semi-parametric procedure.

The partial likelihood

The likelihood function for right-censored data is built from two types of contribution. Suppose an individual i is observed for a time Y i . If the individual i observed the event, i.e Y i " t i and δ i " 1, their contribution to the likelihood is the density at Y i , given by f pY i |X i , βq " hpY i |X i , βqSpY i |X i , βq.

However, if the individual i is censored, i.e Y i " C i and δ i " 0, all we know is that they have survived beyond the observed time ; their contribution is then the probability of this event, which is SpY i |X i , βq.

Thus, the likelihood associated with a survival model is given by Lpβq "

n ź i"1 f pY i |X i , βq δi SpY i |X i , βq 1´δi " n ź i"1 hpY i |X i , βq δi SpY i |X i , βq, (9)
since f pY i |X i , βq " hpY i |X i , βqSpY i |X i , βq (see ( 2)). To apply the standard maximum likelihood method, and estimate the regression coefficients β in the context of proportional hazard models, we would need prior knowledge on the baseline hazard function. To overcome this restriction, Cox proposed a partial likelihood, based on the conditional probabilities of death, which eliminates the term h 0 . The idea is to recover the order of event occurrences rather than their distribution. Thus the partial likelihood will be the product on the observed data, that the individual i knows the event at time Y i , among the individuals still at risk at Y i , and given that there was an event at Y i . Which give

L p pβq " n ź i"1 ˜exppβX i q ř jPRpYiq exppβX j q ¸δi " ź i:δi"1 exppβX i q ř jPRpYiq exppβX j q , ( 10
)
where RpY i q " tj : Y j ě Y i u is the set of individuals still at risk at time Y i . The partial likelihood only partially considers censored individuals. In this sense, it cannot be considered as a genuine likelihood, however it behaves as such and has good computational capacities [START_REF] Kleinbaum | Survival analysis[END_REF]. The estimation of the coefficients β is generally obtained by maximizing the partial log-likelihood using a gradient descent algorithm.

Parametric estimation -some usual distributions

The Cox model semi-parametic estimation allows to recover the effects of predictor variables, without having to estimate the baseline function. Nevertheless, this flexibility comes with an inability for the method to recover the model survival distribution. Parametric models approximate the complete survival distribution while incorporating the effects of predictor variables in the fitted model. To do this, these methods rely on the strong assumption that the survival function follows a known distribution, the aim being to estimate the parameters of this distribution. Here we will present some well-known distribution families and formalize their estimation in the framework of proportional hazards models.

Some distribution families

The first step in the estimation procedure is based on the choice of the model distribution family. When the practitioner has no prior idea of the distribution of the data, the law of the model can be presumed from the form of one of the five functions which define the distribution of T , i.e. S, f , F , h and H. For example, it will be possible to choose a distribution family, according to the form of the survival function estimated by a non-parametric method, or even according to the assumptions made about the behavior of the risk over time. The Table 4. We can see that an exponential-type model is characterized by a constant risk over time and a clear decrease in the survival function. These assumptions may hold for example when studying the lifetime of some kinds of electronic components, but are generally unrealistic in biomedical field. A Weibull model is, on the other hand, characterized by a monotonic failure rate and is therefore suitable for many problems.

In the absence of explanatory variables, parametric models can be estimated by maximum likelihood.

Following the same reasoning as in Section 4.4.3, this likelihood is given by

L n pθq " n ź i"1 f pY i q δi SpY i q 1´δi , ( 11
)
where θ is the parameters characterising the distribution of T . The estimation θ of the parameters θ is usually obtained using iterative optimisation procedures such as the Newton-Raphson algorithm.

Proportional hazard models

In order to incorporate individual characteristics into the estimation procedure, a natural approach is to express the parameters of the desired distribution in terms of the covariates. There is three popular methods to modelling survival data with covariates: the Proportional Hazard (PH) model, Accelerated

Failure Time (AFT) model and the Proportional Odds (PO) model. For the sake of brevity, we will only present the parametric PH model. We direct the interested reader to the books by Kleinbaum and Klein [START_REF] Kleinbaum | Survival analysis[END_REF], and Lee and Wang [START_REF] Lee | Statistical methods for survival data analysis[END_REF], which present the other two approaches.

The parametric version of PH models is obtained by assuming a parametric form on the baseline hazard h 0 ptq of the hazard function (7). Among the distribution families presented in

By setting λ ‹ " λ expp βX k q, one can show that we obtain again a Weibull distribution with parameters pλ ‹ , kq. Thus, the Weibull family is closed under the proportional hazard assumption. Obviously, this reasoning holds for the exponential family, since it is a special case of Weibull where k " 1. The estimation of the parameters of these models is obtained by maximizing the likelihood using iterative optimisation procedures such as the Newton-Raphson method.

Estimation of penalized models

Although the Cox model is highly appreciated in the literature, it is not the best suited to high-dimensional problems, where the sample size is much smaller than the number of explanatory variables, and for which there is a non-negligible amount of censored observations. To avoid falling into overfitting, researchers have introduced penalized models. For most of the methods encountered in the literature, the principle is basically the same as the penalized methods presented earlier in this manuscript. Starting from the initial optimization problem, we incorporate a penalty term to the objective function, inducing sparsity and shrinkage in the coefficient vector. The survival Lasso of Tibshirani [START_REF] Tibshirani | The lasso method for variable selection in the cox model[END_REF] is again the best known method. It consists in penalizing the partial log likelihood of the Cox model with an ℓ 1 -norm penalty.

The new optimisation problem is given by β " arg min

β `´ℓℓ p pβq `λ |β| 1 ˘. ( 14
)
Most of the penalized estimation methods in survival analysis are based on the Cox model, and differ in the form of penalization proposed. The literature includes a wide variety of them adapting to different constraints of high dimensional survival analysis. In Chapter 5, we will present some of them.

Proportional hazards Cure model

The standard survival analysis is based on the strong assumption that all individuals will eventually observe the event of interest if the follow-up time is long enough. However, depending on the problem studied, it is likely that a part of the population is no longer at risk and will never experience the event. In the context of breast cancer, for example, taking an effective treatment can spare some patients from the recurrence event. These phenomena are modeled using Cure models, which assume that a fraction of the censored observations are in fact "cured" to the event. In other words, we will assume that lim tÑ`8 Sptq ą 0, which is characterized by a survival curve that reaches a plateau due to the absence of new events when there is only the cured population left to follow. Thus, a survival curve, estimated by a nonparametric method, that shows a long and stable plateau with a strong censoring at the tail (as in Figure 4.8), can be considered as an empirical evidence of cured individuals in the population.

Many parametric, non-parametric and semi-parametric methods of Cure models have been proposed in the literature [START_REF] Goldman | Survivorship analysis when cure is a possibility: a monte carlo study[END_REF][START_REF] Peng | A nonparametric mixture model for cure rate estimation[END_REF][START_REF] Vern | The use of mixture models for the analysis of survival data with long-term survivors[END_REF]. We focus here on the Proportional Hazards Cure model.

Model formulation

An intuitive way to formulate a Cure model is to use a mixture modeling approach, where the study population is assumed to be a mixture of cured and exposed individuals. Let L be the dummy variable that indicates whether an individual will eventually experience the event (L " 1) or not (L " 0), and define πpXq " PpL " 1 | Xq the fraction uncured of the population having characteristics X. Under the mixture modeling approach, the survival function now satisfies

S (c) pt | Xq " p1 ´πpXqq `πpXq Spt | L " 1, Xq, ( 15
)
where Spt | L " 1, Xq denotes the survival function for the exposed group. Thus, to study the predictor effects on the cure rate and survival distribution, we only need to model the distribution of L and the conditional distribution T |L " 1. In this vein Farewell proposed a parametric approach [START_REF] Vern | The use of mixture models for the analysis of survival data with long-term survivors[END_REF]. For the incidence part, they applied a logistic form to the distribution of L, i.e.

πpXq " PpL " 1 | X, bq " exppb t Xq 1 `exppb t Xq , (16) where b P R p is the vector of coefficients for this part. Then for the latency, they assumed the survival time for the exposed fraction to have a Weibull distribution. Kuk and Chen subsequently proposed a generalization of the Farewell model, which consists in estimating the conditionnal distribution T |L " 1 using a Cox model [START_REF] Anthony | A mixture model combining logistic regression with proportional hazards regression[END_REF]. The associated conditional hazard and survival functions are given by

hpt | L " 1, Xq " h 0 pt | L " 1q exppβXq, ( 17
)
Spt | L " 1, Xq " S 0 pt | L " 1q exppβXq, (18)
where S 0 pt | L " 1q " PpT ą t | L " 1, X " 0q is the baseline conditional survival function and β P R p the vector of coefficients for the latency part.

Maximum likelihood estimation

Building a PH mixture Cure model relies on the estimation of the parameters b and β, which is usually done by likelihood maximization. Suppose an individual i is observed for a time Y i . If the individual i observed the event, i.e Y i " t i and δ i " 1, we know they are not in the cured fraction and therefore L i " 1. Their contribution to the likelihood is given by πpX i q f pY i | L i " 1, X i q. However, if the individual i is censored, i.e Y i " C i and δ i " 0, we cannot tell if they are exposed and will experience the event at some future time after their follow-up, or if they are cured. Their contribution is the expression of the cure survival function at Y i , which is πpX i q `p1 ´πpX i qq SpY i | L i " 1, X i q. Thus, the observed full likelihood is given by Lpb, βq "

n ź i"1 " πpX i q f pY i | L i " 1, X i , βq ı δi " πpX i q `p1 ´πpX i qq SpY i | L i " 1, X i , βq ı 1´δi , " n ź i"1 " πpX i q hpY i | L i " 1, X i , βq SpY i | L i " 1, X i , βq ı δi " πpX i q `p1 ´πpX i qq SpY i | L i " 1, X i , βq ı 1´δi . ( 19
)
When the cured fraction satisfies 1 ´πpX i q " 0 for all combinations of characteristics X i , we recover the Cox model and the coefficients β can be estimated by maximizing the associated partial log-likelihood.

However, in the presence of a cured fraction, we cannot isolate the conditional survival function as performed in the Cox likelihood. In any case, it is not recommended to remove S 0 pt | L i " 1q from the optimization problem, at the risk of losing information about b [START_REF] Judy | Estimation in a cox proportional hazards cure model[END_REF]. Many estimation methods have been proposed to address this problem [ST00, PD00, KC92, Lu08]. We can cite as an example the nonparametric estimation approach of Peng and Dear [START_REF] Peng | A nonparametric mixture model for cure rate estimation[END_REF] and Sy and Taylor [START_REF] Judy | Estimation in a cox proportional hazards cure model[END_REF]. The idea is to express the complete likelihood with the latent variable L. Since L i is unobserved when δ i " 0, there are two possibilities. Either the subject is in the cured fraction and contribution to the likelihood is the probability to be cured, either they are exposed and their contribution is given by the conditional survival function. Thus, the complete and unobserved likelihood is given by Lpb, βq "

n ź i"1 " πpX i q hpY i |L i " 1, X i , βqSpY i |L i " 1, X i , βq ı δiLi ˆn ź i"1 " πpX i q SpY i |L i " 1, X i , βq ı p1´δiqLi ˆn ź i"1 " 1 ´πpX i q ı p1´δiqp1´Liq , (20)
"

n ź i"1 πpX i q Li p1 ´πpX i qq p1´Liq ˆn ź i"1 " hpY i |L i " 1, X i , βqSpY i |L i " 1, X i , βq ı δiLi " SpY i |L i " 1, X i , βq ı p1´δiqLi . ( 21
)
The complete-data full likelihood is expressed as the product of two elements, each including one of the two parameters of the mixture model. The estimation procedure is performed with an EM algorithm, first by computing the expected complete-data likelihood with respect to L, then by maximazing the likelihood obtained.

Chapter 5

STEPWISE VARIABLE SELECTION FOR

SURVIVAL ANALYSIS

M

ulti-omics analysis involves very high-dimensional data, where the number of individuals is considerably smaller than the number of explanatory variables. In particular, depending on the omics measures considered, the number of predictors can reach an order of magnitude of 10 5 , whereas cohort surveys generally contain only a few hundred individuals. This second point can be explained by the various constraints related to the recruitment of individuals within the surveys, but also by the cost and time constraints required for the collection of information, which are proportional to the number of omics approaches considered and to the quantity of individuals selected. In this sense, variable selection is a central issue in clinical research. On the one hand, for statistical purposes, to overcome the curse of dimensionality ; and on the other hand, for practical purposes, to build transportable models, that could be applicable in a hospital environment. The literature gathers numerous methods of variable selection to address this issue. Without claiming to be exhaustive, we can cite to that extent the survival Lasso of Tibshirani [START_REF] Tibshirani | The lasso method for variable selection in the cox model[END_REF], the priority-Lasso of Klau et al. [KJH `18] which integrates a priori information on the relevance of the different omics measures, or the boosting approaches of Buehlmann [START_REF] Buehlmann | Boosting for high-dimensional linear models[END_REF], Tutz and Binder [START_REF] Tutz | Generalized additive modeling with implicit variable selection by likelihood-based boosting[END_REF]. However, due to the very high-dimensionality of the processed data, methods based mainly on the minimization of the fitting error will not always lead to the selection of the real model support. Indeed, the higher the number of predictors, the higher is the probability to find a random combination of variables minimizing the fitting error. Moreover, each of the existing methods deals with different approaches, and lead to different selections, sometimes leaving the practitioner undecided.

In this chapter, we propose a stepwise variable selection algorithm (SVSSA), suitable for multi-omics analysis. The idea is to take advantage of the performances of variable selection methods praised by the literature, by offering a consensus between the different approaches. As can be seen in Figure 5.1, our procedure is decomposed into five steps.

Step 0 is optional and consists in removing outliers form the the following steps. In step 2, we exploit the correlation structures within the omics matrices by selecting groups of correlated variables. First, we build the different groups 1 with a clustering method, then we perform again a scoring at the group level.

Step 3, in some way, consists in selecting the most relevant variables in each group, but indirectly. To do this, we perform a scoring at the variable level, then select the variables being at the same time the most relevant, but also the least correlated between them.

Thus, we are seeking for a significant selection, while limiting information redundancy. Finally, in step 4 we make the final selection of at most P ‹ variables. This step combines a scoring procedure with a forward-backward selection procedure. The main idea of this whole process is to gradually reduce the size of the data, ensuring that at each step a consensus is reached on the choice of the selected variables.

Once the selection by SVSSA is done, the practitioner can perform predictions by building the model of their choice with the selected variables.

The following sections present the different steps of SVSSA. We will not go into the details of how the different methods used work, but we refer the interested reader to the Appendix A, where we provide a concise presentation of the tools, scoring procedures and algorithms used in SVSSA, and for more details 1. or rather subgroups since the omics matrices themselves are considered as the main groups please refer to the seminal articles. Moreover, note that this procedure does not aim to build a model but to select variables. Therefore, in some borrowed selection algorithms, we set aside the search for optimal penalization parameters, and set default parameters inducing sparsity. These default parameters can be modified by the practitioner. We make available the R codes of the SVSSA algorithm on Github

end if

Step 1 : run Algorithm 3 to perform a marginal selection on the omics matrices

Step 2 : run Algorithm 4 to first build subgroups of variables, then perform group selection on the omics matrices

Step 3 : run Algorithm 5 to perform a variable selection on the omics matrices

Step 4 : run Algorithm 6 for the final selection on clinical data and the remaining omics variables Now and for the rest of the chapter, consider that we have a sample of n independent observations pY i , δ i , X i q, where for an individual i, Y i corresponds to the observed survival time, δ i to the censoring indicator, and X i to the p-dimensional vector of the values taken on the predictors. Moreover, since the predictors are ordered in m groups of sizes κ 1 `... `κm " p, X i admits the decomposition pX ¯1 i , ..., X ¯m i q.

The observed data are in X P R nˆp where the i-th row is X t i , δ P R n and Y P R n , and we note pX pdq , Y pdq q the subsample containing only the individuals who observed the event. Note in particular that in all estimation phases where the target variable is only the survival time Y , the estimation is based on the subsample pX pdq , Y pdq q.

Preliminary step: Removing outliers

Outlier detection is an important step prior to building any learning model. In survival analysis, it enables the detection of extreme individuals with a survival time that is too long or too short given their individual characteristics, and who may therefore degrade the model estimation. This preliminary step is optional, but highly recommended. It can be done manually by the practitioner or automatically when launching SVSSA. The procedure we present is based on the paper of Carrasquinha et al. [START_REF] Carrasquinha | Consensus outlier detection in survival analysis using the rank product test[END_REF]. The authors proposed an anomaly scoring algorithm that performs a consensus between ten outlier detection methods on each individual of the dataset. We use their tools here, but we limit our procedure to four methods for computation time matter. The tools used here can be grouped into two approaches. First, the residual approach which represents one of the fundamental procedures in outlier detection. It is generally described matrices in the dataset. We can see that the size of the clinical data is derisory compared to that of the other measures (0.02 % of the dataset), while by contrast the CNA matrix is predominant (85.87 % of the dataset). This distribution illustrates the need for late selection within the clinical data so that their under-representation does not lead to an underestimation of their prognostic relevance. Before performing the analysis, pre-processing of the dataset is necessary due to the specificities of clinical data. Firstly, in cohort data, there is generally a lot of missing data, particularly because the individuals recruited do not wish to carry out all the planned tests. Furthermore, the variables considered in the clinical matrices are presented in different formats. In order for the methods selected in this analysis to work properly, it is necessary to remove the missing information from the dataset and to convert the variables that are not already in numerical format.

Step

The treatment of missing data must be done with caution so as not to lose too much clinical information. In our dataset, we have 15 variables and 233 observations with at least one missing information.

Since no observation in the sample is complete, it is necessary to study the a priori relevance of each variable, in order to find a balance in the removal of both variables and individuals, and thus maintain a suitable amount of information. We therefore first removed 18 irrelevant variables, some of which contained missing data. Table 5.2 lists these variables and the reasons for their removal. We also removed patient FUSCCTNBC389 because she had zero follow-up time, and therefore represents an outlier. At this stage of the study, the clinical matrix has 232 observations and 32 variables, for which we have 122 observations and 11 variables with at least one missing value. The boxplots on the left side of Figure 5.3 show the distribution of missing data within these incomplete variables and observations. Since the median of the variable was 25, we decided to remove the variables with at least 25 missing data so as not to further reduce the sample size. After that, only three observations had missing data, we removed them from the dataset as well. To summarize, in addition to the variables in Table 5.2 and patient FUSCCTNBC389, we removed the variables SNF_Subtype, Mutation_Subtype, BRCA1.2.MUT.20160718, sTILs, iTILs, Grade and the patients FUSCCTNBC134, FUSCCTNBC170, FUSCCTNBC245.

Variables

Reason for removal

Exome_sequencing indicates whether the patient has received exome sequencing, i.e. whether we observe her values in the Mutation matrix. This is the case for all patients in our dataset, so this variable is useless for model building.

RNA_sequencing

indicates whether the patient has received RNA sequencing, i.e. whether we observe her values in the RNAseq matrix. This is the case for all patients in our dataset, so this variable is useless for model building.

OncoScan_Array indicates whether the patient has been subjected to whole genome copy number assay, i.e. whether we observe her values in the CNA matrix. This is the case for all patients in our dataset, so this variable is useless for model building. This variable provides redundant information with the variables related to the three scores.

Sex

Histology_extended

contains 218 missing values.

RFS_time_Days

indicates the observed survival time of the patient in days. This variable provides redundant information with the variable RFS_time_Months which indicates the observed survival time in months.

Followup_Month

indicates the time the patient has spent in the project in months. It was calculated as the difference between the date of last follow-up and the date of surgery (which is equivalent to date of inclusion). This variable provides redundant information with the variable RFS_time_Months. In particular, they have the same value for censored observations.

Date_of_surgery

Presentation of the algorithms

In this study, we will compare the performance of our selection algorithm with nine methods that have been proven to be effective in the literature. However, since the SVSSA method has mainly a selection objective and not a prediction one. We will focus on the performance in terms of variable selection, in order to compare our approach to other methods. To do so, we use each method to perform a selection of variables assumed to be significant, and then we apply a Cox model with the selected variables to perform predictions. The choice of methods considered is mainly based on the benchmark study by Herrmann et al. [HPH `21] on survival predictions in multi-omics analysis. One can also notice that most of these comparative methods play a role in the SVSSA approach. This will allow to juxtapose the respective performances of these methods with their joint performances. Each of these methods performs selection either at the variables level. The tuning of the parameters is carried out according to the desired degree of sparsity, when this option is proposed, otherwise it is done by cross-validation. As previously discussed, a limited number of variables is strongly recommended to ensure the transportability of the models obtained in a clinical setting. Therefore, we have set the maximum number of variables to be selected at 100. Finally, we include in the comparative study a Cox model built with only clinical data. This will serve as a baseline model to evaluate the contribution of omics data in the context of our study.

Variable selection with the SVSSA method

In this section, we present the results obtained by SVSSA on the whole dataset. As mentioned above, the maximum number of predictors to be reached is 100. The input parameters are the default parameters unless otherwise specified.

Preliminary step: Removing outliers

We start with the preliminary step of outlier detection. During this step, the algorithm designated 10 individuals as outliers. were considered as significantly atypical by SVSSA. Moreover, the bar chart shows that SVSSA aligned mainly with the martingale residuals approach since all the detected individuals were also considered as the most atypical by this method. The DBHT method on the other hand showed less impact on this example. This last point can also be explained by the fact that it is the only approach based on predictive performance, and therefore it offers less common results with the other methods. Step

1: Marginal selection

After removing the outliers from the dataset, we proceed to the variable selection. For the first step of marginal selection, we set the selection rates to 10% for the RNAseq and mutation matrices, and 1% for the CNA matrix. By applying a stricter selection on the CNA matrix, we return to comparable scales for the three matrices, and guard against the bias that would have been caused by the overrepresentation of the CNA matrix in the models. Thus, by SVSSA, we selected 2 421 RNAseq variables, 1 354 mutation variables and 2276 CNA variables ; for a total of 6 112 explanatory variables (including clinical ones). Figure 5.5 summarizes the results obtained in this step. The graph on the left illustrates the intersections between the methods for the same selection rate. We can notice that 155 variables were selected unanimously by the four methods, and 1 169 variables were selected by three of them. The bar chart on the right shows that the methods based on the c-index, the Brier score, and the Wilcoxon test had the most impact for this selection step. Indeed, the measure of correlation with survival time resulted in more isolated choices -4898 variables were selected by this method alone -which were not considered as relevant by SVSSA.

Step 2: Selection of correlated groups

The dataset now consists of a reasonable amount of explanatory variables (6 112). Figure 5.7 summarizes the results obtained in this step. We notice that only 2 groups were selected by the four methods, while 24 were selected by three of them. Finally, the bar chart shows that the Block Forest, Bayesian PGGM and group Lasso methods had an equivalent impact for this selection step, while the IPF-Lasso is slightly behind.

Step 3: Selection of decorrelated variables Now we want to select a set of relevant variables that are as decorrelated as possible to minimize information redundancy. For this step we set the selection rate at 10%, that lead to a selection of 189 variables, for a total of 250 explanatory variables (including clinical ones). Figure 5.8 summarizes the results obtained in this step. The bmlasso method did not select any of the candidate variables, so it is excluded from the intersection plot, and naturally it is associated with a null proportion on the bar chart.

Among the selected variables, 9 have been detected by the three remaining methods, and we also notice that most of the variables considered significant by SVSSA come from the gradient boosting selection.

Step 4: Final selection

For this last step, we include clinical data in the selection procedure, as the size of the omics matrices is reasonable enough not to worry about underestimating the clinical information. The maximum number of variables to select was set at 100. This setting lead to a selection of 19 explanatory variables. Figure 5.9

Performance comparison

As mentioned above, we want to compare the performance of SVSSA in terms of variable selection with nine other methods. Each of them is used only in the selection phase, the prediction phase being carried out by a Cox model. For all the comparative methods considered, we used the dataset resulting from the removal of outliers ; the initial dataset led to many model fitting failures. Three measures are considered: the Uno's concordance index and the AUC score for the discrimination power, and the iBrier score for the predictive power ; each of them was evaluated using the package survAUC. Finally, two types of accuracy are presented ; the training accuracy and the validation accuracy.

The training accuracy is evaluated through a 10-fold stratified cross-validation on the same dataset used for the selection procedure. For all the methods considered, we first perform a variable selection phase using the whole dataset, then we perform the cross-validation on Cox models built with the selected variables. Table 5.4 shows that SVSSA offers good performances in terms of discrimination power, due to its high c-index and AUC score. Similarly, if we look at the standard deviation values for these measures, we see that the results of SVSSA are quite homogeneous on the different cross-validation folds.

Nevertheless, although SVSSA also has a very good predictive power, the priority-Lasso outperforms it on this criterion with a very low iBrier, associated with a low standard deviation as well. It is rare that a method offers the best performances over all criteria [HPH `21], hence the interest in measuring different ones to get an overview of its capabilities. This can be seen, for example, with the lasso and the bmlasso which minimize the iBrier quite well, but do not offer interesting results on the c-index and the AUC. Since SVSSA performs well on all three measures, this first analysis argues for a consistency of our procedure. However, it is necessary to verify this premature observation on a validation set independent The dataset is divided into two independent subsets, the training set represents 80% of the initial dataset, and the test set 20%. For each method, the selection phase is performed on the training set, and the Cox models are also built on this set restricted to the selected variables. The predictions are then applied on the validation set. The results obtained are reported in Table 5.5. While the ipf-Lasso, the priority-Lasso and SVSSA offer results close to those reported in Table 5.4, this is not the case for most methods.

This observation argues for the existence of an overfitting bias when evaluating the training accuracy of these methods. Moreover, the observations previously made on SVSSA and the Lasso priority are again observed here. The performances of the two methods are quite close, and SVSSA outperforms the priority-Lasso on two of the three measures. Finally, we see that SVSSA offers consistent results on this dataset.

Computation time

Another important criterion for this comparative analysis is the computation time. SVSSA offers good performance in terms of variable selection, but this procedure is based on a superposition of several models, which makes it by construction slower than the comparative methods. We can see in Table 5.6 that SVSSA requires about 9 hours of additional compilation time to the priority-Lasso which also offers very good results. Note, however, that the computation time associated with SVSSA varies greatly depending on the settings of the optional parameters. A parameter that can be modified easily without really damaging the results is the number of iterations of step 2 (see nIt in Algorithm 4). By lowering it to 1 instead of 5 we can save a few hours of computation in step 2. is still significant. However, we must consider that this approach is intended for the analysis of clinical data which take several years to collect, and only require a variable selection phase to be exploited in a hospital environment. The contribution in performance and the serenity offered by a consensus selection compared to the depreciation in terms of compilation time is at the practitioner's discretion. lasso ipfLasso priorLasso SGL gboost cboost ranger bmlasso SVSSA

Cure models

Due to their convenience, we performed the performance analysis on Cox models. However, after a variable selection, the practitioner must build a predictive model with the estimation method that suits best their problem. In the context of our study, where the event of interest includes relapse, disease progression and death, the hypothesis of the existence of a cured fraction in the population studied is obvious. Figure 5.11 also illustrates this point, although the follow-up time for some patients is too short to draw reliable conclusions. Cure models appear to be an obvious choice. We will not go into a deep analysis of TNBC breast cancer by Cure models here, however, we offer in Figure 5.12 an overview of analyses that would be interesting to do next. In particular, it will be possible to compare the survival curves of individuals according to one or more variables, and so define the characteristics of the cured individuals, notably in terms of treatment received. As an example, we can see in Figure 5.12 that patients who have undergone a mastectomy will have a much higher probability of survival than patients who have not gone through this surgical intervention, all other things being equal.

Conclusion and perspectives

The SVSSA selection procedure aims at automating the different steps of variable selection in survival analysis. Based on a consensus between recognized methods, it is statistically consistent and offers good predictive performances, however its computation time limits its application for daily use. This shortcoming is obviously a prospect for improvement. One perspective would be to implement it in Python, but we would also have to think about other estimation methods available in this language. Moreover, the methods used are mostly built on the basis of a Cox model, alternatives based on Cure models could be considered to fit the context of clinical studies. Finally, we would like to specify that while our performance study was based on the dataset resulting from the second selection phase of step 4, we highly recommend to retrieve the list of the P ‹ most significant variables for an in-depth study of their relevance with TNBC specialists. These studies (selection with SVSSA and in-depth study of the selected variables)

should also be performed on other omics datasets to evaluate the consistency of our algorithm. rMi " δ i ´Ĥ 0 pY i q expp βX i q.

(1)

The residual rMi can be interpreted as the difference between the number of deaths observed over r0, Y i s by the individual i and the number expected given the model. Thus, a censored individual who, from the model's point of view, would have accumulated a significant risk over this period will be assigned a lower residual than if he had observed the event. In this sense, martingale residuals reveal individuals with different survival patterns from those with the same individual characteristics. However, taking its values in s ´8, 1s, this specification has a marked skewness issue, therefore causing difficulty in anylizing its impact. where β psq refers to the saturated model, which is a free model where each individual has its own vector of coefficients β¨i ; there are therefore no random error term. After an adaptation to survival models, and in particular to Cox model, the authors define the deviance residual of an individual i by rDi " signpr M i q b ´2 `r M i `δi lnpδ i ´r M i q ˘,

Deviance residuals

where rMi corresponds to the martingale residual of this individual. We thus obtain a measure more centered around 0 with the same sign as the martingale residual. The residual rDi can be interpreted as the difference in log-likelihood between an overfitted model and the retained model. We can deduce that individuals with extreme values on the residuals had a particular influence during the estimation of the fitted model ; they are potential outliers.

Censored quantile regression residuals

Q T pτ | Xq " inf ␣ t : F T pt | Xq ě τ ( " βpτ q X, (3)
where βpτ q is the vector of regression coefficients at the quantile τ Ps0, 1r. The authors define the deviance residual of an individual i by rQi " Y i ´Qp0.5 | X i q.

(4)

A.1.2 Predictive performance analysis

In survival analysis, the Concordance Index (C-index) is one of the most commonly used metrics to measure model performance. It quantifies the rank correlation between observed survival times and the risk levels defined by the fitted model. There are several algorithms for estimating the C-index, we will only present the most widespread one, that of Harrell et al. [HCP `82]. Specifically, the authors assess the model's ability to provide a reliable ranking of the survival times, by estimating the fraction of pairs of individuals correctly ordered out of all available comparable pairs. Let η i denote the risk score that the individual i is assigned by the model. In the presence of censoring, we distinguish four cases:

´i and j have experienced the event: the pair pi, jq is correctly ranked if η i ă η j and Y i ą Y j , ´i and j are censored: no information on the quality of the ranking can be provided, ´i is censored and j has experienced the event so that Y i ą Y j : the pair pi, jq is correctly ranked if η i ă η j , ´i is censored and j has experienced the event so that Y i ă Y j : no information on the quality of the ranking can be provided.

The concordance index is then defined by

C index " ř i‰j 1 ηiăηj 1 YiąYj δ j ř i‰j 1 YiąYj δ j . ( 5
)
It can be interpreted as the probability that an individual with a short survival time is considered by the model to be more at risk than an individual who has lived longer. We use this metric to determine the impact of each individual on the model predictive performances. i. An outlier will therefore be characterized by a low p-value. However, the authors identified a limitation within this method. In particular, when the amount of available data is low, the bootstrap samples do

Dual Bootstraps Hypothesis

A.1.3 Scoring

We follow the scoring procedure described by Carrasquinha et al. [START_REF] Carrasquinha | Consensus outlier detection in survival analysis using the rank product test[END_REF], and based on the Rank Product test. In order to overcome the variability of the results offered by the various outlier detection methods, the authors propose to perform a consensus which takes the form of a scoring. For this, they perform the following rank product on each individual,

RP i " k ź j"1 rankpZ ij q, (6)
where k corresponds to the number of selected methods and Z ij to the measure of outlyingness of individual i by method j. Note however that before applying the rank function, we harmonize the measures so that a low value of Z ij corresponds to a high level of outlyingness for the method j. Thus, individuals with low ranks, and by continuity a low RP i , would a priori be more outlier. The algorithm then performs a significance test of this ranking based on H 0 : the ranking is random. The anomaly score is then represented by this test's p-value, the lower it will be, the more the individual will be considered as an outlier. In other words, if an individual is systematically defined as an outlier by the selected methods, it is likely that this classification is correct.

where ℓℓ p pβq is the partial likelihood of the Cox model. This method adapts perfectly to linear and logistic regressions, by modifying the likelihood term. The resulting model is a selection at variable level but conditional on the group of membership. In our context of four subgroups, we build the model using the cross-validation algorithm proposed by the authors. Once the model is obtained, we consider a subgroup as significant if none of its variables has been penalized.

Block Forest [HW19]

The Block Forest method proposed by Hornung and Wright is a variant from the well-known Random Forest by Breiman [START_REF] Breiman | Random forests[END_REF]. Inspired by the multi-omics data problem, this method considers variables group structure at the level of the split point selection. Recall that a random forest model is a set of decision trees built from bootstrap samples of the training data. Each decision tree is a series of binary tests that aim to divide the individuals according to a criterion of homogeneity in regard to the output. The point of interest here is the construction of these binary tests, or more precisely the split point selection procedure. In the Random Forest algorithm, this is done by a random selection of nvar predictors on which the algorithm selects the division that optimizes the considered split criterion. Hornung and Wright pointed out that treating each variable uniformly, when in a group structuring, and especially in the multi-omics framework, large groups will be overrepresented in the model regardless of their relevance as predictors. To overcome this shortcoming, the authors propose a weight-based splitting procedure. First, in order to give each group the possibility to be represented within the split point selection, the algorithm draws each group with a probability of 1 2 , i.e. all groups are selected with a probability `1 2 ˘m, and the draw is rerun if no group is selected. Then, for each admitted group g of size p m , the algorithm randomly selects ? p g predictors to be candidates for the split, so nvar " ř g admitted ? p g . Finally, the split criterion is weighted by the weights w g associated to each group in order to privilege variables coming from groups supposed to be more influential. The weights pw 1 , ..., w m q are defined by the algorithm through an optimization process. The optimal weights associated with a model provide an indication of the relative importance of the different groups for prediction.

This method adapts to all variants of the Random Forest, including Ishwaran's random survival Forest et al. [START_REF] Hemant Ishwaran | Random survival forests[END_REF] that we are interested in here. Since we wish to achieve a group selection, once our model is built from the four predictor subgroups, we consider a subgroup significant if its optimized weight is greater than a threshold set at 0.75 by default, but configurable by the practitioner.

A.3.5 Scoring

The global scoring consists simply in counting the number of times that a given subgroup has been selected by the different methods. We then select the 10% most significant ones. Since they have different sizes, this is not equivalent to the 10% most significant variables.

are interested here in the gradient-based method within Cox model framework. In this context, we seek to estimate the regression coefficients β by minimizing the negative partial log-likelihood which constitutes our loss function. The algorithm is initialized with β " 0 p,1 , then for each iteration, it computes the negative gradient vector of the loss function with respect to ηpX, βq " X t β. For each individual i, it is given by U i " B BηpX ¨i, βq ℓℓ p pβqq ˇˇˇη pX,βq"ηpX, βq " δ i ´ÿ lPRpyiq exppX t ¨l βq ř kPRpy l q exppX t ¨k βq

. ( 16
)
In order to maximize the partial likelihood (i.e. minimize the loss function) in the direction most correlated with the gradient, the algorithm estimates a linear model fitting the gradient U to each input X j , that is bj " arg min

βj 1 n |U ´Xj¨βj | 2 2 . ( 17
)
The weak model associated with the current iteration is then denoted by the coefficient bj ‹ for which j ‹ " arg min j

1 n |U ´Xj¨bj | 2 2 .
Finally, the algorithm updates the estimator βj ‹ " βj ‹ `ν bj ‹ where ν controls the learning rate. This phase of estimating weak models is iterated It max times. This number can be used as a regularization parameter to perform variable selection. In our procedure, we use the default parameters defined in R which are: ν " 0.1 and It max " 100. This hierarchical model is based on a spike-and-slab double-exponential prior (i.e a Laplace distribution), and is defined as follows $ ' & ' % β j |π j , s 0 , s 1 " p1 ´πj q Lpβ j |0, s 0 q `πj Lpβ j |0, s 1 q

Spike-and-Slab Lasso Cox

π j |θ " Bpθq θ " Up0, 1q (18)
where π j is the prior spike probability, and s 0 and s 1 are predefined scale parameters such as 0 ă s 0 ă s 1 .

s 0 is the spike scale parameter chosen small to induce strong shrinkage on β j , and s 1 is the slab scale parameter chosen large to instead induce weak or no shrinkage on the coefficient. The sparse model is

where ℓℓ p pβq is the Cox model partial log-likelihood, S j " p1 ´πj qs 0 `πj s 1 , and the term ř p j"1 1 Sj |β j | can be likened to a Lasso-type penalty. In our procedure we set by default s 0 " 0.2 and s 1 " 0.5. Résumé : Cette thèse s'articule autour de deux axes. Le premier constitue une contribution à l'étude des modèles graphiques gaussiens partiels (PGGM) dans le cadre de l'apprentissage en grande dimension. Plus précisément, nous nous intéressons à la modélisation à sorties multiples, où nous souhaitons estimer d'une part la matrice ∆ des liens directs entre les prédicteurs et les réponses, et d'autre part la matrice de précision conditionnelle des réponses Ωy. Nous débutons avec une approche fréquentiste par maximum de vraisemblance pénalisée, où nous proposons un PGGM muni de deux formes de pénalisation : une pénalisation ℓ1 induisant de la sparsité sur ∆ et Ωy, et une pénalisation structurante reflétant un a priori gaussien généralisé sur les liens directs. Nous montrons que, lorsqu'il est convenablement régularisé, ce modèle est agrémenté d'une garantie théorique prenant la forme d'une borne supérieure sur l'erreur d'estimation. Enfin, nous clôturons cette première réflexion par des études empiriques mettant en avant le caractère structurant de cette procédure d'estimation, et sa pertinence sur un jeu de données réelles.

Nous poursuivons par l'étude de la contrepartie bayésienne, jusqu'alors inexplorée dans la littérature. En suivant une stratégie spike and slab, nous offrons plusieurs structures hiérarchiques imposant soit une configuration saturée, sparse, group-sparse ou encore sparse-groupsparse de la matrice ∆. Nous obtenons une garantie théorique pour les configurations sparse et group-sparse, et illustrons les résultats compétitifs de ces modèles sur une étude de simulation et un jeu de données réels, menés avec des échantillonneurs de Gibbs. Le deuxième axe de la thèse est, quant à lui, entièrement dévolu à la sélection de variables pronostiques en analyse de suvie multi-omique. Nous y proposons un algorithme de sélection de variables descendante offrant un consensus entre différentes méthodes de régularisation, notamment celles présentées dans le premier axe. L'efficacité de cette approche est enfin étudiée sur des données relatives au cancer du sein triple négatif, en prenant le soin de répondre aux contraintes identifiées par les oncologues. Tous nos codes sont rendus disponibles à la communauté.

Title: Contribution to the study of graphical models and high-dimensional statistics applied to the modeling of Triple-Negative Breast Cancer. Keywords: high-dimensional linear regression, partial graphical model, partial correlation, structural penalization, sparsity, Bayesian approach, spike-and-slab, Gibbs sampler, variable selection.

Abstract: This thesis is articulated around two axes. The first one is a contribution to the study of partial Gaussian graphical models (PGGM) in highdimensional learning. Precisely, we are interested in the multiple-output modeling, where we aim at estimating, on the one hand the matrix ∆ of direct links between predictors and responses, and on the other hand the conditional precision matrix Ωy of responses. We start with a frequentist approach by penalized maximum likelihood, where we propose a PGGM with two forms of penalization: a ℓ1 penalty inducing sparsity on ∆ and Ωy, and a structural penalty reflecting a generalized Gaussian prior on the direct links. We show that, when properly regularized, this model comes with a theoretical guarantee taking the form of an upper bound on the estimation error. Finally, we close this first reflection with empirical studies highlighting the structuring property of this estimation procedure, and its relevance on a real dataset. We continue with the study of the Bayesian counterpart, previously unexplored in the literature. Following a spike and slab strategy, we offer several hierarchical structures imposing either a saturated, sparse, group-sparse or sparse-group-sparse configuration of the matrix ∆. We obtain a theoretical guarantee for the sparse and group-sparse configurations, and illustrate the competitive results of these models on a simulation study and a real dataset, conducted with Gibbs samplers. The second part of the thesis is entirely devoted to the selection of prognostic variables in multi-omics survival analysis. We propose a stepwise variable selection algorithm offering a consensus between different regularization methods, including those presented in the first axis. The efficiency of this approach is finally studied on a dataset relating to triple-negative breast cancer, while taking care to meet the constraints identified by oncologists. All our codes are made available to the community.

3. 2

 2 Correlogram of responses (left) and correlogram of predictors located on chromosomes 8, 9 and 10 (right). The colormap associates red with negative correlations and blue with positive correlations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Empirical distribution of the posterior probability of inclusion estimated by (gs) for each chromosome (left). Aggregated (gs) estimation of ∆ on chromosome 14 with D14Mit3 hilighted (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . XV 3.4 Aggregated (sgs) estimation of ∆ on chromosomes 7, 8 and 14, from left to right. The hilighted markers are D7Cebr205s3, D7Mit6, D7Rat19, Myc and D7Rat17 for chromosome 7, D8Mgh4, D8Rat135 and Rbp2 for chromosome 8 and D14Rat8 and D14Mit3 for chromosome 14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Illustration of survival data in a study-time scale. Observed survival times are indicated by solid diamonds, and the others are censored observations. . . . . . . . . . . . . . . . . 4.2 Example of left censoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Example of right censoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Example of interval censoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Example of survival curves: (left) the theoretical curve -(right) the estimated curve. . . . 4.6 Example of hazard curves. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.7 Representation of the hazard and survival functions of the usual distribution families according to shape, rate and scale parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . 4.8 Example of an estimated survival curve adapted to cure models. . . . . . . . . . . . . . . 5.1 Flowchart of the variable selection process using the SVSSA algorithm. . . . . . . . . . . . 5.2 Data distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Distribution of missing data restricted to variables and observations with at least one missing data after the first removal (left), and distribution of different variable formats in the clinical matrix after all missing data has been removed (right). . . . . . . . . . . . . . 5.4 Step 0: intersections between the four methods used for sets of 20 most atypical individuals according to each measure (left), proportion of outliers designated by SVSSA in each list of 20 most atypical individuals (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Step 1: intersections of the sets of variables selected by each of the four methods (left), proportion of variables selected by SVSSA in the sets of variables selected by each individual method (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6 Correlation plot of the first 100 variables for each comedy matrix . . . . . . . . . . . . . . 5.7 Step 2: intersections of the sets of groups selected by each of the four methods (left), proportion of groups selected by SVSSA in the sets of groups selected by each individual method (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

  proportion of variables selected by SVSSA in the sets of groups selected by each individual method (right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.10 Evolution of the size of the matrices after each selection step. . . . . . . . . . . . . . . . . 5.11 Kaplan-Meier curve estimated with the whole dataset. . . . . . . . . . . . . . . . . . . . . XVI 5.12 (a) Survival curves of an individual who underwent a mastectomy (dashed line) and an individual who did not through this surgery (solid line); (b) Survival curves of an individual belonging to the iC4 subtype (dashed line) and an individual belonging to another subtype (solid line); (c) and (d) survival curves of different individuals for multivariate models. . . 120 XVII NOTATIONS Notation Meaning 0 n,m null matrix of dimension n ˆm detpAq determinant of the square matrix A λ i pAq the i-th eigenvalues of A sppAq spectrum of matrix A taken in decreasing order (from λ 1 pAq " λ max pAq to λ d pAq " λ min pAqq S d `cone of symmetric positive semi-definite matrices of dimension d S d ``cone of symmetric positive definite matrices of dimension d vecpAq the vectorization of A into a column vector |A| ˚" }vecpAq} ˚the elementwise ℓ ˚norm of A |A| ´|A| ˚deprived of the diagonal terms of A }A} F " |A| 2 the Frobenius norm of A }A} 2 the spectral norm of A xxA, Byy " xvecpAq, vecpBqy " trpA t Bq the Frobenius inner product between any matrices A and B of same dimensions xu, vy " u t v the inner product of the Euclidean real space |u| 0 the number of non-zero values in the vector u A zj the square matrix A with row and column j removed A j the column j of the square matrix A without the diagonal element A jj rAs C the matrix A whose elements outside of the set of coordinates C are set to zero

Figure 1 . 1 :

 11 Figure 1.1 : Examples of undirected (top) and directed (bottom) graphs with their adjacency matrices.

Figure 1 . 2 :

 12 Figure 1.2 : Example of a directed graph with its associated moral graph.

  g.Maathuis et al., 2018, Chap. 10), the Bayesian counterpart of PGGM, in contrast, to the best of our knowledge, the Bayesian counterpart of PGGM has not yet been developed. This approach will be the subject of our

  et al.. Inspired by the structural penalizations proposed by Slawski [SZCT10] and Li and Li [LL10] in their structured versions of the Elastic Net estimator, and exploiting the partial reparametrization of the Gaussian graphical models of Sohn and Kim [SK12] and Yuan and Zhang [YZ14], they propose the Spring method, which consists of a PGGM with an additional structural penalty on direct links. In this chapter we present the results of a collaboration with Frédéric Proïa and Pascal Jézéquel, which has been published in ESAIM Probability and Statistics in 2021 [OOJP21] 1 . Our work is articulated on two axes. The first axis consists in the development of an estimation procedure for a penalized PGGM, named GenGm, which is based on a combination of the approaches of Yuan and Zhang [YZ14] and Chiquet et al .

β Ñ ` 8 .

 8 The marginal shapes (d " 1 and V " 1) of the distribution are represented on Figure2.1, depending on whether β ă 1, β " 1 or β ą 1.

Figure 2 . 1 :

 21 Figure 2.1 : Marginal shape of the generalized Gaussian distribution (d " 1 and V " 1) for some β ă 1 (dotted red), β " 1 (black) and some β ą 1 (dotted blue). The noteworthy cases β " 1{2 (Laplace), β " 1 (Gaussian) and β " `8 (uniform) are highlighted.

Lemma 2 . 4 . 2 .

 242 Let A P S d ``and B P S d `.

Lemma 2 .

 2 4.3. Let A P S d `and B P S d `. Then, λ min pAq trpBq ď trpABq ď λ max pAq trpBq. Proof. Since A ´λmin pAqI d P S d `and B P S d `,

y

  and C B " 2 Ω ˚´1 y ∆ ˚L, and considering s L given in (11), we obtain a compact form of ϕ 1 p0q ϕ 1 p0q " xxA n `ηβ s β´1 L C A , δϑ y yy `xxB n `ηβ s β´1 L C B , δϑ yx yy.

Figure 2 . 2 :

 22 Figure 2.2 : Mean squared prediction error for N " 500 repetitions of the weakly structured Scenario 1.

Figure 2 . 3 :

 23 Figure 2.3 : Mean squared prediction error for N " 500 repetitions of the strongly structured Scenario 2.

Figure 2 . 4 :

 24 Figure 2.4 : Mean squared prediction error for N " 500 repetitions of the strongly structured Scenario 3.

  2 below should probably temper this statement. To conclude, let us consider the strongly structured scenarios with L " I p (without structuring) in the Oracle setting with β " 2, and let us compare the results with those of Figures 2.3 and 2.4, obtained with the correct version of L given in (30). The results are displayed on Figure2.5 where we can see that the benefit of structuring is manifest. Unsurprisingly, the results without structuring are close to those of Gm since L " I p only strengthens the shrinkage effect with ridge-like additional penalties.

Figure 2 . 5 :

 25 Figure 2.5 : Mean squared prediction error for N " 500 repetitions of the strongly structured Scenario 2 (left) and Scenario 3 (right) for Or, Gm and the unstructured Or (L " I p ), with β " 2.

Figure 2 . 6 :

 26 Figure 2.6 : Temperature and log-precipitation measured over a year in Montreal (left). Empirical distribution of the minimal and maximal log-precipitation for the 35 weather stations (right).

Figure 2 .

 2 Figure 2.8 : Variable selection for min p by GenGm with pλ, µ, ηq " p0, 0.05, 1q and, from top to bottom, β P t0.5, 1, 1.5, 2u.

Figure 2 . 9 :

 29 Figure 2.9 : Variable selection for max p by GenGm with pλ, µ, ηq " p0, 0.05, 1q and, from top to bottom, β P t0.5, 1, 1.5, 2u.

Figure 2 . 10 :

 210 Figure 2.10 : Estimated direct links (top) and regression coefficients (bottom) for the pair pmin p , max p q by GenGm with pλ, µ, ηq " p0, 0.05, 1q and β " 2, after the N " 100 experiments. Dotted lines divide the panel into months.

Figure 2 . 11 :

 211 Figure 2.11 : Estimated correlation between min p and max p by GenGm with pλ, µ, ηq " p0, 0.05, 1q and β " 2, after the N " 100 experiments. The off-diagonal entry is approximately 0.32.

  al. [XSM `16], the 'aggressive' multivariate Dirichlet-Laplace prior of Wei et al. [WRHG20], the theoretical results for group selection consistency of Yang and Narisetty [YN20] or even the extension of the Bayesian spike-and-slab group selection to generalized additive models of Bai et al. [BMA

(H. 2 )

 2 The prior slab probability satisfies 1 ´πn -1{m n . (H.3) The shrinkage factors satisfy nλ 7 n -m 2`η n μ´η n and µ K n, min nλ 7 n Ñ `8 for some η ą 0, where λ 7

Figure 3 . 1 :

 31 Figure 3.1 : Medians of the mean squared prediction errors obtained after N " 100 repetitions of Scen. 1 (top), Scen. 3 (middle) and Scen. 5 (bottom) with ˘1 standard deviation and n e growing from 100to 500. The black curves correspond to uncorrelated predictors (ρ " 0) while the blue and red curves correspond to correlated predictors (ρ " 0.5 and ρ " 0.9, respectively).

Figure 3 . 2 :

 32 Figure 3.2 : Correlogram of responses (left) and correlogram of predictors located on chromosomes 8, 9 and 10 (right). The colormap associates red with negative correlations and blue with positive correlations.

Figure 3 . 3 :

 33 Figure 3.3 : Empirical distribution of the posterior probability of inclusion estimated by (gs) for each chromosome (left). Aggregated (gs) estimation of ∆ on chromosome 14 with D14Mit3 hilighted (right).

3. 2 .

 2

Figure 3 . 4 :

 34 Figure 3.4 : Aggregated (sgs) estimation of ∆ on chromosomes 7, 8 and 14, from left to right. The hilighted markers are D7Cebr205s3, D7Mit6, D7Rat19, Myc and D7Rat17 for chromosome 7, D8Mgh4, D8Rat135 and Rbp2 for chromosome 8 and D14Rat8 and D14Mit3 for chromosome 14.

Figure 4 . 1 :

 41 Figure 4.1 : Illustration of survival data in a study-time scale. Observed survival times are indicated by solid diamonds, and the others are censored observations.

Figure 4 . 2 :

 42 Figure 4.2 : Example of left censoring.

  who did not experience the event until the end of the study, j: patient lost to follow-up before knowing the event.

Figure 4 . 3 :

 43 Figure 4.3 : Example of right censoring.

Figure 4 . 4 :

 44 Figure 4.4 : Example of interval censoring.

Figure 4 .

 4 Figure 4.5 illustrates graphical representations of a function Sptq and the empirical survival function Ŝptq, commonly called survival curves.

Figure 4 . 5 :

 45 Figure 4.5 : Example of survival curves: (left) the theoretical curve -(right) the estimated curve.

Figure 4 . 6 :

 46 Figure 4.6 : Example of hazard curves.

Distribution Density f ptq Survival Sptq Hazard hptq Parameters Exponential λe ´λt e ´λt λ rate: λ ą 0 Weibull kλ ptλq k´1 e ´ptλq k e ´ptλq k kλ ptλq k´1 shape:

 k´1 1 represents the density, survival and hazard functions of the most widespread distribution families in the literature, and the Figure 4.7 illustrates the hazard and survival curves associated with them according to the values taken by their parameters. k ą 0 ; scale: λ ´1 ą 0 Gamma θ α Γpαq t α´1 e ´θt 1 ´γpα,θtq Γpαq f ptq Sptq shape: α ą 0 ; rate: θ ą 0 Log-logistic pθ{αqpt{αq θ´1 p1`pt{αq θ q 2 1 1`pt{αq θ pθ{αqpt{αq θ´1 1`pt{αq θ shape: θ ą 0 ; scale: α ą 0 Table 4.1 : Characteristic functions of the survival time for the usual distribution families.

Figure 4 . 7 :

 47 Figure 4.7 : Representation of the hazard and survival functions of the usual distribution families according to shape, rate and scale parameters.

Ŝptq 8 Figure 4 . 8 :

 848 Figure 4.8 : Example of an estimated survival curve adapted to cure models.

dataset. Steps 1

 1 to 3, focus on reducing the size of the omics matrices, to bring them back to an order of magnitude close to that of the clinical data. It is only in step 4 that clinical data enters the selection process. Algorithm 1 and Figure 5.1 provide an overview of the selection procedure. Each step of the process involves scoring based on four methods. Step 0 of outliers detection assigns an atypicality score to each individual in the dataset. For this preliminary step, only clinical information is taken into account to calculate the scores. Step 1 performs a large marginal selection within the omics matrices. The idea is to reduce, in the first instance, the size of the matrices to a reasonable level, to facilitate the execution of

Figure 5 . 1 :

 51 Figure 5.1 : Flowchart of the variable selection process using the SVSSA algorithm.

1 :

 1 Summary of the methods used in SVSSA.

Figure 5 . 2 :

 52 Figure 5.2 : Distribution of omics matrices in the dataset.

Figure 5 . 3 :

 53 Figure 5.3 : Distribution of missing data restricted to variables and observations with at least one missing data after the first removal (left), and distribution of different variable formats in the clinical matrix after all missing data has been removed (right).

  Figure 5.4 illustrates, on the one hand the intersections between the different methods used for sets of 20 most atypical individuals according to each measure ; and on the other hand the proportion, for each method, of individuals designated by SVSSA present in its list of the 20 most atypical. First notice on the left graph that only one individual is considered as outlier by the four methods, and five other individuals are considered as outliers by three of them. These first 6 individuals

Figure 5 . 4 :

 54 Figure 5.4 : Step 0: intersections between the four methods used for sets of 20 most atypical individuals according to each measure (left), proportion of outliers designated by SVSSA in each list of 20 most atypical individuals (right).

  Figure 5.6 represents a portion of the correlation plots of the different omics matrices. They clearly show the existence of group

Figure 5 . 5 :

 55 Figure 5.5 : Step 1: intersections of the sets of variables selected by each of the four methods (left), proportion of variables selected by SVSSA in the sets of variables selected by each individual method (right).

Figure 5 . 6 :

 56 Figure 5.6 : Correlation plot of the first 100 variables for each comedy matrix

Figure 5 . 7 :

 57 Figure 5.7 : Step 2: intersections of the sets of groups selected by each of the four methods (left), proportion of groups selected by SVSSA in the sets of groups selected by each individual method (right).

Figure 5 . 8 :

 58 Figure 5.8 : Step 3: intersections of the sets of variables selected by each of three methods (left), bmlasso did not select any of the candidate variables ; proportion of groups selected by SVSSA in the sets of groups selected by each individual method (right).

Figure 5 . 10 :

 510 Figure 5.10 : Evolution of the size of the matrices after each selection step.

Figure 5 . 11 :

 511 Figure 5.11 : Kaplan-Meier curve estimated with the whole dataset.

Figure 5 .

 5 Figure 5.12 : (a) Survival curves of an individual who underwent a mastectomy (dashed line) and an individual who did not through this surgery (solid line) ; (b) Survival curves of an individual belonging to the iC4 subtype (dashed line) and an individual belonging to another subtype (solid line) ; (c) and (d) survival curves of different individuals for multivariate models.

2 "

 2 [START_REF] Terry M Therneau | Martingale-based residuals for survival models[END_REF]. Deviance residuals have also been introduced by Therneau et al. in order to handle the skewness present in the martingale residuals distribution. More precisely, they propose a transformation of this measure in order to get as close as possible to a Gaussian distribution. To do so, the authors start from the definition of deviance in the general framework of linear regression models given by D " ℓℓpβ psq q ´ℓℓp βq ı ,

  [EHC14]. Eo et al. introduced the censored quantile regression residuals, adapting the Nardi and Schemper algorithm originally based on the Cox model to quantile regression for censored data. Consider F T ptq " PpT ď tq the distribution function of the survival time ; the τ th quantile of T is defined as the inverse function

  Testing (DBHT) [PCV15a] Pinto et al. first introduced the Bootstraps Hypothesis Testing (BHT) method which assigns an anomaly score to each individual of the dataset, depending on whether they increase or decrease the model C-index [PCV15b]. Let C zi be the C-index associated with the model built without the individual i, and C full the one obtained with the complete dataset, the algorithm tests H 0 : C zi ď C full on the basis of bootstrap samples from the dataset deprive of

  not have enough information to lead to a relevant model, which could negatively impact the individuals' anomaly score. To overcome this constraint, the authors proposed the DBHT method which performs the comparison test on the basis of samples of the same size. Concretely for each individual i, the algorithm generates a set B poison of bootstrap samples of the dataset that includes i in each bootstrap sample, and a set B antidote that, conversely, is deprived of i. Next, the algorithm tests H 0 : ErC antidote s ą ErC poison s from the concordance histograms of the two sets. An outlier is again characterized by a low p-value.

[ TSZY17 ]

 TSZY17 Proposed by Tang et al., the Spike-and-Slab Lasso Cox is a Bayesian variables selection method offering an adaptive shrinkage on Cox model regression coefficients.

  then estimated by searching for the posterior modes of the parameters, in particular by maximizing the log joint posterior density which verifies ln ppβ, π, θ | Y, δ, Xq 9 ℓℓ p pβq ´p ÿ lnp1 ´θq `πj lnpθq,

Titre:

  Contribution à l'étude des modèles graphiques et statistique en grande dimension appliquée à la modélisation du cancer du sein triple négatif.Mot clés : régression linéaire en grande dimension, modèle graphique partiel, corrélation partielle, pénalisation structurante, sparsité, approche bayésienne, spike-and-slab, échantillonneur de Gibbs, sélection de variables.

  L e cancer du sein est la forme de cancer la plus répandue et la plus meurtrière chez les femmes [FCS `21]. Depuis les années 1990, de nombreux travaux de recherche ont permis l'amélioration des techniques de dépistage et le développement de thérapies plus efficaces. La mortalité du cancer du sein a ainsi été endiguée, jusqu'à diminuer progressivement, mais le taux d'incidence poursuit son augmentation globale. C'est ainsi qu'en 2018, nous comptions en France métropolitaine, quatrième pays le plus touché par cette pathologie, 58 500 nouveaux cas et 12 146 décès [dC, Int]. La lutte contre le cancer du sein demeure donc un enjeu majeur de santé publique.

	INTRODUCTION (FRANÇAIS)

Il existe plusieurs types de cancers du sein définis par des facteurs histopronostiques qui influent sur le choix du traitement et le pronostic de la maladie [dCdl]. Le cancer du sein triple négatif (TNBC) est une forme tumorale particulièrement éprouvante en oncologie. Il s'agit d'une des formes les plus agressives du cancer du sein, que l'on retrouve dans environ 15% des cas, dont majoritairement des femmes non ménopausées. Elle se caractérise par un développement et une propagation rapide de la tumeur, ainsi qu'un risque de rechute des plus élevés donnant souvent lieu à des métastases. Ses particularités font du TNBC un type de cancer difficile à pronostiquer [Soc]. Afin d'améliorer la planification de la prévention et la conception de nouveaux traitements individualisés, une meilleure compréhension des mécanismes pathologiques de l'apparition et de la progression du TNBC est nécessaire. A cet effet, de nombreuses études sont effectuées afin de déterminer les facteurs biologiques ayant une réelle influence sur le pronostic du TNBC [QXH `16, TCJL `10, DTP `07].

  Nous commençons par définir les concepts élémentaires de la théorie des graphes, et mettons en lumière la relation entre la structure d'indépendances conditionnelles des variables et la matrice de précision du modèle graphique. Nous poursuivons par une présentation des modèles graphiques gaussiens et leurs avantages dans la récupération des liens directs entre prédicteurs et les réponses par rapport aux modèles de régression linéaire standards. Ensuite, nous présentons le modèle graphique gaussien partiel pénalisé introduit par Sohn et Kim [SK12] et Yuan et Zhang [YZ14], ainsi que ses avantages en termes de précision et de temps de calcul. Enfin, nous discutons de la littérature bayésienne des modèles graphiques gaussiens en grande dimension. (2) Le chapitre 2 présente les résultats obtenus dans le premier article de cette thèse, ayant fait l'objet d'une publication dans le journal ESAIM: Probability and Statistics [OOJP21]. Il est consacré à reast cancer is the most common and deadliest form of cancer in women [FCS `21]. Since the 1990s, numerous research studies have led to improved screening techniques and the development of more effective therapies. Mortality from breast cancer has been contained and is gradually decreasing, but the overall incidence rate continues to rise. France ranks fourth worldwide by incidence of breast cancer.

1) Le chapitre 1 offre une introduction succincte à la théorie des graphes, et en particulier aux modèles XXX graphiques gaussiens. l'estimation d'un modèle graphique gaussien partiel avec une pénalisation structurelle sur les liens directs. Après une présentation de la littérature qui aborde cette problématique, nous formalisons l'écriture de notre pénalisation gaussienne généralisée et exposons la nouvelle fonction objectif de notre PGGM. Nous poursuivons sur les garanties théoriques de notre estimateur, et détaillons la preuve du théorème principal. Enfin, nous testons les performances de notre méthode d'abord par des études empiriques sur des données synthétiques, puis en traitant un jeu de données réel, avant de conclure sur les perspectives d'évolution.

(3) Le chapitre 3 présente les résultats obtenus dans le deuxième article de cette thèse, ayant fait l'objet d'une publication dans le journal Bayesian Analysis

[START_REF] Okome | A bayesian approach for partial gaussian graphical models with sparsity[END_REF]

. Après une présentation des travaux qui nous ont précédés dans la littérature, nous formalisons l'écriture de nous modèles hiérarchiques.

Nous poursuivons sur les garanties théoriques associées aux configuration sparse et group-sparse.

Puis nous testons nos méthodes sur des échantillonneurs de Gibbs et un jeu de données réel, avant de conclure.

INTRODUCTION

B

In 2018, 58 500 new cases and 12 146 deaths were recorded in Metropolitan France alone

[dC, Int]

. The fight against breast cancer therefore remains a major public health issue.

There are several types of breast cancer defined by histopronostic factors that influence the choice of treatment and the prognosis of the disease [dCdl]. Triple-negative breast cancer (TNBC) is a particularly challenging tumor form in oncology. It is one of the most aggressive forms of breast cancer, occurring in approximately 15% of cases, mostly in premenopausal women. It is characterized by a rapid development and spread of the tumor, as well as a high risk of relapse, often resulting in metastasis. Its particularities make TNBC a type of cancer difficult to prognose [Soc]. In order to improve prevention planning and the design of new individualized treatments, a better understanding of pathological mechanisms of TNBC onset and progression is needed. To this end, numerous studies are being performed to determine the biological factors that actually influence the TNBC prognosis [QXH `16, TCJL `10, DTP `07]. Scientific and technological advances over the last two decades, in terms of data collection and processing, have greatly impacted the methods of research of prognostic factors. The various research units are no longer limited to data from their respective departments, they now have a large panel of biological data, of various types, formats and sources, offering new perspectives for clinical research. This dynamic has led to the development of multi-omics studies, where practitioners exploit several types of high-dimensional molecular data (such as genomic data for DNA, transcriptomic

Definition 1.2.3. Let

  be a random vector X and an undirected graph G. The vector X satisfies the global Markov property if for any subset of disjoint nodes tA, B, Cu in G, such that in G, A is separated from B given C (i.e. any path in G from a node in A to a node in B contains a node in C),

Definition 1.2.6. Let

  be a random vector X and a directed acyclic graph G. The vector X satisfies the global Markov property if for any subset of disjoint nodes tA, B, Cu in G, such that in G, A is d-separated from B given C (i.e. in the moral graph

The block coordinate descent of Banerjee et al. [BEGD08]. In

			their paper, Banerjee et al.
	consider the penalty |Ω| 1 , and treat the problem (23) via its dual form below, which estimates the
	covariance matrix instead of its inverse.
	Σ " arg max	tln detpW q : |W ´S| 8 ď λu,	(24)
	W
	with W " S `U , and U a symmetric matrix. They present a block descent algorithm for the problem (24)

Banerjee et al. [BEGD08]. In

  " tΩ : aI ĺ Ω ĺ bIu, and Q 2 " tu : }u} 8 ď λu. We will note f the function to minimize in (26).

	avec Q 1
				the same paper, Banerjee et
	al. also presented an estimation algorithm based on the Nesterov's first order method. The goal is not
	really to provide another algorithm, but to go through Nesterov's smooth minimization method in order
	to obtain a more rigorous and accurate complexity estimation than the block descent one. First, the
	problem (23) had to be reformulated as a non-smooth optimization problem adapted to the Nesterov
	method. For this purpose, the authors have shown that when we impose a bound on the eigenvalues of
	Ω, the problem (23) can be expressed by
	min ΩPQ1	´ln detpΩq `xxS, Ωyy `max uPQ2	xΩ, uy 2 ,	(26)

Friedman et al. [FHT08]. In

  their paper, Friedman et al. consider the |Ω| 1 penalty and present a blockwise coordinate descent algorithm that relies on the same principle as that of Banerjee et al., but with some variants that make it computationally favorable. In particular,

	they exploit a point raised by the first authors, namely that solving the problem (25) is equivalent to
	solving the dual problem

  Then, λ max ppABq ´1q ď λ max pA ´1q λ max pB ´1q ðñ λ min pABq ě λ min pAq λ min pBq.However, if B is not invertible, the relation trivially holds since we still have λ min pABq ě 0 from Lemma 2.4.2.

1, thus trpABq ě λ min pAq trpBq. The other inequality is obtained through λ max pAqI d Á P S d `. Lemma 2.4.4. Let A P S d ``and B P S d `. Then, λ min pAq λ min pBq ď λ min pABq and λ max pABq ď λ max pAq λ max pBq. Proof. On the one hand, λ max pABq ď }AB} 2 ď }A} 2 }B} 2 " λ max pAq λ max pBq, since A and B are symmetric and since, from Lemma 2.4.2 and by hypothesis, all eigenvalues appearing in the relation are non-negative. Suppose now that B is invertible so that both A ´1 and B ´1 belong to S d ``. Lemma 2.4.5. Let A P S d `and U P R dˆℓ . Then,

  R n pθ ˚`t pθ ´θ ˚qq " R n pθ ˚q `t xx∇R n pθ ˚q, θ ´θ ˚yy `et pθ, θ ˚q (19) for some second-order error term e t pθ, θ ˚q. Consider the reparametrization ϕptq " R n pθ ˚`t pθ ´θ ˚qq (20) so that ϕ 1 p0q " xx∇R n pθ ˚q, θ ´θ ˚yy. Let δθ y " Ω y ´Ωẙ and δθ yx " ∆ ´∆˚, let also δθ " θ ´θ ˚in a compact form. The estimation error is denoted

	δϑ " p θ ´θ ˚" p p Ω y	´Ωẙ , p ∆ ´∆˚q " pδϑ

n pθq " ´ln detpΩ y q `xxS y , Ω y yy `2 xxS yx , ∆yy

`xxS x , ∆ t Ω ´1 y ∆yy `η xxL, ∆ t Ω ´1 y ∆yy β . (

18

)

For any θ P Θ and t P R, by a Taylor expansion, y , δϑ yx q.

  Taking t " 1 in the Taylor expansion (19) with θ " p θ and considering the definition of ϕ in (20), by convexity, R n p p θq ´Rn pθ ˚q ě ϕ 1 p0q " xx∇R n pθ ˚q, θ ´θ ˚yy.

  d 2 " 2 ´d2 . Here we exploited the previous inequality twice, u L ě 0 and β ě 1. Moreover, note that from the Lemma 2.4.1 δθ y Ω ´1 y δθ y P S q `, and exploiting also the Lemma 2.4.2 we have that Ω ´1 y δθ y Ω ´1 y δθ y P S q `. From Lemmas 2.4.3, and 2.4.9, we obtain xxL, ∆ t Ω ´1 y δθ y Ω ´1 y δθ y Ω ´1 y ∆yy " xxΩ ´1 y , ∆ L ∆ t Ω ´1 y δθ y Ω ´1 y δθ y yy ď λ max pΩ ´1 y ∆ L ∆ t q xxΩ ´1 y , δθ y Ω ´1 y δθ y yy ď ω L xxΩ ´1 y , δθ y Ω ´1 y δθ y yy,where ω L is defined in (9). Replacing L by S x and ω L by ω S , a similar bound obviously holds using Lemma 2.4.8. Suppose that c and d are chosen so that c 1 ą 0, d 1 ą 0, c 2 ă 0 and d 2 ă 0. Then, from

	Lemma 2.4.9 we have,

ϕ 2 ptq ě xxΩ ´1 y , δθ y Ω ´1 y δθ y yy " 1 ´|c 2 | ω S ´ηβ u β´1 L |d 2 | ω L ‰ `c1 xxΩ ´1 y , δθ yx S x δθ t yx yy `ηβ u β´1 L d 1 xxΩ ´1 y , δθ yx L δθ t yx yy ě xxΩ ´1 y , δθ y Ω ´1 y δθ y yy " 1 ´|c 2 | ω S ´ηβ pp ω L q β´1 |d 2 | ω L ‰ `c1 xxΩ ´1 y , δθ yx S x δθ t yx yy `ηβ pp ω L q β´1 d 1 xxΩ ´1 y , δθ yx L δθ t yx yy. Now choose ϵ S ą 0 and ϵ L ą 0 small enough so that ϵ S ω S `ηβ p β´1 ω β L ϵ L ă 1 and fix c " ? 2 `ϵS and d " ? 2 `ϵL . We finally obtain ϕ 2 ptq ě a 1 xxΩ ´1 y , δθ y Ω ´1 y δθ y yy `a2 xxΩ ´1 y , δθ yx S x δθ t yx yy `a3 xxΩ ´1 y , δθ yx L δθ t yx yy

  Cauchy-Schwarz inequality |r¨s S | 2 1 ď |S| }r¨s S } 2 F , we finally obtain 0 ě ´cλ,µ maxth a , h b u a |S| }t δϑ} F `γr,η,β,p }t δϑ} 2

	F

  Σ 1 P S d1 ``and Σ 2 P S d2 ``. When d 2 " 1, this is a multivariate normal distribution N d pµ, Σq with d " d 1 , µ " M and Σ " Σ ´1 2 Σ 1 , having density

	ppXq "	p2πq	d 2 |Σ| 1	1 2	exp ˆ´1 2	pX ´µq t Σ ´1pX ´µq	ẇhere
	µ P R ppXq "	X ν´1 `a 2 ˘ν B ν `a 2 , b 2	˘e´a 2X	´b X 2 1 tX ą 0u
	where ν P R, a ą 0 and b ą 0.
	Definition 3.1.3 (Wishart/Gamma/Exponential). The density of X P S d ``following the matrix Wishart
	distribution W d pu, V q is given by
	ppXq "	2	u´d´1 2 2 Γ d `u 2 ˘|V | |X| d u	u 2	ˆ´1 exp 2	tr `V ´1X ˘˙1 tX P S d ``u
								is a
	Gamma distribution Γpa, bq with a " u 2 and 1 b " 2 V , having density
			ppXq "	b a X a´1 Γpaq	e ´b X 1 tX ą 0u

d and Σ P S d ``. Definition 3.1.2 (Generalized Inverse Gaussian). The density of X P S d ``following the matrix generalized inverse Gaussian distribution MGIG d pν, A, Bq is given by

ppXq " |X| ν´d `1 2 ˇˇA 2 ˇˇν B ν `A 2 , B 2 ˘exp ˆ´1 2 tr `A X ´1 `B X ˘˙1 tX P S d

``u where ν P R, A P S d ``, B P S d ``and B ν is a Bessel-type function of order ν. When d " 1, this is a generalized inverse Gaussian distribution GIGpν, a, bq with a " A and b " B, having density where u ą d ´1, V P S d ``and Γ d is the multivariate Gamma function of order d. When d " 1, this

Table 3 .

 3

		1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20
	Nb.	74	67	63	60	39	45	52	43	31	51	21	26	33	22	15	27	18	30	34	19

2 : Number of markers on each chromosome, which correspond to the sizes κ g of each group for 1 ď g ď 20 when running (gs) and (sgs).

Table 4

 4

	.1, only the

  2 . Dataset Data " pY, δ, Xq, group index GId " pId 1 , ..., Id m q where Id 1 are the indexes of the clinical variables if included, ultimate number of variables to select P ‹ , Optional: indicate whether the step 0 should be launched Step 0 " true, all optional inputs of Algorithms 3 to 6. Final dataset Data final , variables selected Selected_vars.if Step 0 " true thenStep 0 : run Algorithm 2 to remove outliers

	Algorithm 1 SVSSA
	Input: Output:

  indicates the sex of the patient. Since all patients are women, this variable is useless for model building. DNA_QC_Failed indicates whether the DNA quality control has failed. This variable takes the value FALSE for all patients, and is useless for model building. HRD indicates the estimation of homologous recombination deficiency score. It was calculated as the sum of three scores: telomeric allelic imbalance (NtAI), loss of heterozygosity (LOH) and large scale transition (LST) [JMS `19].

	indicates whether the patient underwent a BRCA1 mutation prior to 7/18/2016.This variable provides redundant information with	the variable BRCA1.2.MUT.20160718 which indicates whether the patient underwent a BRCA1 or BRCA2 mutation prior to	7/18/2016. a	same reason as BRCA1.MUT.20160718.
	BRCA1.MUT.20160718			BRCA2.MUT.20160718

  same reason as Followup_Month. No_Chemotherapy indicates whether the patient has not received chemotherapy. This variable provides redundant information with the variable Chemotherapy which indicates whether the patient has received chemotherapy.ER_IHC_scoreindicates the result of the immunohistochemistry test that determines the presence of estrogen receptor in the patient's cancer cells.This variable takes the value negative for all patients since it is a particularity of the TNBC b. It is therefore useless for model building.

	same reason as Followup_Month.	same reason as ER_IHC_score but with progesterone receptor.	same reason as ER_IHC_score but with HER2 protein.	contains 168 missing values.
	Date_of_last_followup	PR_IHC_score	ERBB2_IHC_score	ERBB2_FISH

Table 5 .

 5 2 : Presentation of the first clinical variables removed and the reasons behind these choices. The pathogenic mutation of the BRCA1 and BRCA2 genes affects a minority of patients with breast cancer, but its impact is major. It is one of the most powerful risk factors for this pathology, since 65% and 45% of the women affected, respectively, develop the disease [APN`03].b. The term triple-negative breast cancer refers to the fact that cancer cells test presence-negative for estrogen receptors, progesterone receptors and the HER2 protein [Soc].

a.

Table 5

 5

	.3 provides an overview of the

`18]

, GRridge [VDWLV

`16]

, sparse-group Lasso

[START_REF] Simon | A sparse-group lasso[END_REF]

,

´the boosting approach: gradient boosting [Bue06, DB16], Likelihood-based boosting [TB06, BASB09, DB16],

Table 5 .

 5 3 : Summary of the methods used in the comparative study, the symbol * indicates the methods incorporating the group structure in their estimation procedure.

Table 5

 5 of the training set. The evaluation of the validation accuracy is a standard training-validation procedure.

	Learner	c-index Mean sd	iBrier Mean sd	AUC score Mean sd	selected
	lasso	0.797 0.204 0.091 0.034 0.814	0.212	5
	ipfLasso	0.848 0.120 0.125 0.095 0.869	0.115	13
	priorLasso	0.913 0.058 0.083 0.037 0.932	0.042	21
	SGL	0.709 0.229 0.131 0.070 0.734	0.188	5
	gboost	0.929 0.051 0.134 0.200 0.936	0.051	18
	cboost	0.773 0.233 0.137 0.099 0.802	0.262	70
	ranger	0.594 0.251 0.245 0.213 0.551	0.239	100
	bmlasso	0.766 0.208 0.090 0.039 0.822	0.196	15
	clinicalCox 0.683 0.238 0.202 0.196 0.727	0.187	-
	SVSSA	0.938 0.066 0.095 0.103 0.943 0.058	19

.4 : Mean and standard deviation of performance measures obtained by 10-fold cross-validation on the training set.

  Table5.7 show the results of the two configurations in terms of training accuracy and computation time. The shorter configuration had better results, both compared to the initial SVSSA and to the priority-Lasso, but the computation time Table5.5 : Mean of performance measures obtained on the validation set.

	Learner	c-index iBrier AUC score
	lasso	0.576	0.094	0.655
	ipfLasso	0.818	0.075	0.865
	priorLasso	0.971	0.046	0.975
	SGL	0.681	0.087	0.760
	gboost	0.406	0.109	0.270
	cboost	0.763	0.074	0.761
	ranger	0.469	0.101	0.514
	bmlasso	0.337	0.154	0.301
	clinicalCox	0.885	0.051	0.914
	SVSSA	0.986	0.050	0.994

  Table 5.6 : Compilation time in hours when selecting variables on the whole dataset. Table 5.7 : Mean and standard deviation of performance measures obtained by 10-fold cross-validation on the training set for two configurations of SVSSA.

	0.038	1.224	2.448	0.086	0.005	0.143	0.699	0.006	11.579
	Learner		c-index Mean sd		iBrier Mean sd	AUC score Mean sd	selected Time
	SVSSA with nIt " 5 0.938 0.066 0.095 0.103 0.943	0.058	19	11.579
	SVSSA with nIt " 1 0.973 0.059 0.046 0.041 0.976 0.051	28	8.774

(4) Le chapitre 4 fait office d'introduction à l'analyse de survie. Nous commençons par définir le cadre et les problématiques spécifiques aux modèles de survie, et poursuivons sur les principales méthodes d'estimation de tels modèles dans le cadre non paramétrique, semi-paramétrique et paramétrique. Nous abordons ensuite la question de la sélection de variables en survie, au travers des méthodes d'estimation pénalisées, et concluons sur les modèles de Cure et leurs spécificités pour la modélisation d'évènements cliniques.(5) Le chapitre 5 présente les derniers résultats de cette thèse, à savoir une méthode de sélection de variables descendante adaptée aux études multi-omiques. La première partie du chapitre s'attelle à détailler le fonctionnement de notre algorithme, en apportant notamment une description succincte des différentes méthodes qui le composent. La deuxième partie est une mise en pratique sur des données réelles portant sur le cancer du sein triple négatif, dans laquelle nous comparons les résultats de notre approche avec des méthodes de régularisation bien établies, selon la pertinence de la sélection, les performances prédictives, et le temps de calcul. XXXI

The codes and the dataset are available at https://github.com/EuniceOkome/StructPGGM

We consider here the centered case because the data are assumed to be normalized.

The codes and the dataset are available at https://github.com/EuniceOkome/BayesPGGM

https://github.com/EuniceOkome/SVSSA

But we will see that, in terms of variable selection and statistical interpretation, L and β still have a substancial role to play. The point is that we have observed that the best prediction error does not usually coincide with a sparse solution (see Remark 2.5.3 above) when the coefficients or the covariates are not very contrasting.

In particular, this was the case of our simulation study with ˘1 2 coefficients and N p0, 1q covariates. So, just as they look at the Lasso's regularization paths, practitioners may choose the desired degree of sparsity, depending on p{n, by playing with the hyperparameters. Here, on the basis of the MSPE, most of the time we must retain µ ! 10 ´2 and only a few direct links are set to zero. To look for sequences of days directly related to min p and max p , we decided to constraint µ ě 10 ´2 and focus on variable selection. The active set of ∆ is evaluated on the basis of n t " 25 randomly chosen observations. The experiment is repeated N " 100 times, and the locations having a frequency of occurrence that exceeds 0.5 are retained (or, equivalently, those whose estimates have a non-zero median). This can be seen as a measure of variable importance. The results are given on Figures 2.8 and 2.9 below for min p and max p , respectively, with a fixed set of regularization parameters and increasing values of β. The objective is to show the influence of the latter, all other things being equal. The colored areas highlight the days having a frequency of occurrence, represented by gray crosses, that exceeds 0.5 in the N " 100 repetitions of the experiment. Note that, since we retain λ " 0 in these experiments, GenGm for β " 1 coincides with Spr. We can see that the increasing pressure exerted by β on the estimation procedure tends to refine the selection by giving priority to the most important variables and by dropping the others much more easily, at the cost of prediction results: we are undoubtedly in a selection process. The sequence of inclusions p S β2 Ă p S β1 for β 1 ă β 2 that we observe for the estimated active sets is clearly a guarantee of quality for the selected variables.

of ∆ gi . Here our approach diverges from [START_REF] Xu | Bayesian variable selection and estimation for Group Lasso[END_REF] and [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF]. The bi-level selection of the authors is made through spike-and-slab effects both at the group scale and on the individual variances, considered as truncated Gaussians, generating zero groups and (almost surely) zero coefficients within the groups.

Let us suggest instead the Bayesian hierarchical model given by

" p1 ´π1 q " p1 ´π2 q N q p0, λ g ν gi Ω y q `π2 δ 0 ‰ b κg `π1 δ 0 ν gi K K " Γpα, ℓ gi q λ g K K " Γpα g , γ g q Ω y " W q pu, V q π j K K " βpa j , b j q (6) for g P 1, m , i P 1, κ g and j P 1, 2 , with hyperparameters α " 1 2 pq `1q, α g " 1 2 pq κ g `1q, ℓ gi ą 0, γ g ą 0, u ą q ´1, V P S q ``, a j ą 0, and b j ą 0. In this mixture model, π 1 is the prior spike probability on the groups whereas π 2 is the prior spike probability within the non-zero groups, for a bi-level selection. In terms of cumulative shrinkage effects, λ is an adaptative shrinkage factor acting at the group scale and ν is an adaptative shrinkage factor acting at the predictor scale (λ g is associated with the direct links between the predictors of group g and all the responses whereas ν gi is associated with the direct links between predictor i of group g and all the responses). In this way, (6) opens up many perspectives for dealing with bi-level shrinkage. We can set γ g " γ for all g, for a global shrinkage at the group scale. At the predictor scale, when ℓ gi " ℓ g for all i, this is a global shrinkage in the g-th group but we might even consider a full global shrinkage ℓ gi " ℓ. However, an identifiability issue may result from the product λ g ν gi between group and within-group effects. Even if the posterior distributions depend on different levels of data that shall resolve it, one can for example fix λ g " 1 (for adaptative) or ν gi " 1 (for global) and let the shrinkage entirely rely on the other parameter. Although it achieves the same objectives as those of [START_REF] Xu | Bayesian variable selection and estimation for Group Lasso[END_REF] and [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF], this hierarchy seems more consistent with our previous sections (take π 2 " 0 and ν gi " 1 to remove the within-group effect and recover the group-sparse setting of Section 3.3, take π 1 " 0 and λ g " 1 to remove the group effect and recover the sparse setting of Section 3.2). In this context, the degree of sparsity is still characterized by N 0 given in (3) for the predictor scale, by G 0 given in (5) for the group scale, but also, for the within-group scale, by the number N 0g of zero columns in each particular group g, that is, for all 1 ď g ď m,

We also need to define the number J 0 of zero columns in the non-zero groups, that is

To implement a Gibbs sampler from the full posterior distribution stemming from (6), we may use the conditional distributions given in the proposition below.

Empirical results

In this section, let us call (s), (gs) and (sgs) the related settings, and let us denote by (ad) the adaptative shrinkage and by (gl) the global shrinkage. First of all, these models contain many hyperparameters that have to be carefully tuned. Our experiments showed that, unsurprinsingly, the results are strongly impacted by the prior amount of shrinkage on ∆, driven by ℓ and even by γ for (sgs). Apart from the usual cross-validation procedures, we could stay in line with our Bayesian approach and suggest conjugate Gamma hyperpriors. This is very easy to implement, but the hyperparameters are now replaced by other hyperparameters and the same questions arise. Instead, like in [START_REF] Xu | Bayesian variable selection and estimation for Group Lasso[END_REF] and [START_REF] Liquet | Bayesian variable selection regression of multivariate responses for group data[END_REF], we follow the idea of [START_REF] Park | The Bayesian Lasso[END_REF] and we use a Monte-Carlo EM algorithm. By way of example, from the full posterior probability (9) and since λ i " Γpα, ℓ i q for all i, it is not hard to see that, with (s),

where the term T ‰ ℓ does not depend on ℓ. Thus, the k-th iteration of the EM algorithm should lead to

for the adaptative shrinkage and the global shrinkage (λ i " λ), respectively. The intractable conditional expectations are then estimated with the help of the Gibbs samples. For (gs), the results are mainly the same as above (replace q `1 by qκ g `1 in the first case, ppq `1q by qp `m in the second case and consider 1 ď g ď m instead of 1 ď i ď p), and similar results also follow with (sgs). Recall that our definitions of the adaptative and global shrinkages are given in the corresponding sections, in the description of the hierarchical models. The tuning of u and V (or v) is actually trickier. Because ErΩ y s " uV , we set V " 1 u I q and u is conveniently chosen to be the smallest integer such that Ω y is (almost surely) invertible, that is u " q (see e.g. [START_REF] Brown | Multivariate Bayesian variable selection and prediction[END_REF]). This is particularly adapted when the dataset is standardized. Finally, a and b reflect the degree of sparsity to introduce in the direct links. We can set a " b to promote sparse settings, which is potentially interesting when p " n, but a " b " 1 is a standard non-informative choice and a ă b may also be useful for variable selection (see e.g. the real dataset of Section 3.6.2). They can be chosen from a cross-validation step (for prediction purposes) or to enforce some degree of sparsity (for selection purposes), just like a practitioner manages the tuning parameter of the Lasso. The posterior median is used to estimate ∆ and get sparsity whereas the posterior mean is used to estimate Ω y . Indeed, we don't want to impose any sparsity on Ω y (q is small), so we decided to retain this standard choice.

But the concern is much greater for ∆ because some coordinates must be exactly zero. This is the reason why the posterior median seemed a more appropriate choice (in particular, it suffices for the sampler to generate zeros more than half the time for the empirical posterior median to be zero). Due to the huge amount of calculations in the simulations, the estimations are made on the basis of 3000 iterations of the sampler in which the first 2000 are burn-ins. This is revised upwards for the real data (10000 iterations with 5000 burn-ins).

Remark 3.6.1. To the best of our knowledge, there is no simple way to sample from the MGIG d distribution as soon as d ą 1. The recent method described in Section 3.3.

Discussion and Conclusion

To conclude, we would like to draw the attention of the reader to some weaknesses of the study, still under investigation. On the one hand, as soon as p is large (say, p ě 500), the Bayesian studies should be conducted with a group structure or by promoting very sparse settings because due to the outline of the sampler, looping over each column of ∆ may quickly become intractable. A group structure limits the number of loops (only m ! p per sampler iteration), although each loop may require the generation of large Gaussian vectors (up to pq ˆκg q-dimensional), so compromises are needed. Subdividing the dataset is natural when it is intrinsically equipped with a group structure (e.g. that of the previous section), we could suggest otherwise a clustering of the set of predictors to gather similar entries and control the size of the groups. At this stage, our procedures cannot compete with the Lasso-type algorithms (GLasso, CLIME or even ANT) in terms of computational times. This is an issue on which future studies should focus (ongoing works are devoted to translating the samplers into more efficient environments), enhan-as the study of differences between the observed values of a variable of interest and the values predicted by a given regression model ; outliers are then represented by significant differences. As it stands, this definition is not perfectly applicable to survival models because it does not take censoring into account. We use three procedures that have been able to adapt to this constraint, thus improving the interpretability of the residual approach in the context of our study. The second approach is the predictive performance approach. The simple and intuitive idea is to penalize individuals who reveal a negative impact on the model predictive performances. This step relays only on the clinical data ; Algorithm 2 illustrates its flow.

The methods used to measure the outlierness are the martingale residuals of Therneau et al.

Step 1: Marginal selection

The marginal selection step consists in an aggregation of four correlation analysis methods. We exploit the idea that a variable marginally correlated with the target variables is likely to have a significant impact on the final model. To compare the degrees of marginal correlations, we assign a significance score to each variable according to its level of association with the variables Y , δ, and the pair pY, δq. The Algorithm 3 illustrates the flow of this selection step.

As discussed in the introduction section, this step does not include clinical predictors. The methods used to measure the significance of the correlation are the Spearman correlation test [START_REF] Spearman | The proof and measurement of association between two things[END_REF], the Wilcoxon-Mann-Whitney test [START_REF] Wilcoxon | Individual comparisons by ranking methods[END_REF][START_REF] Henry | On a test of whether one of two random variables is stochastically larger than the other[END_REF], the Uno's concordance index (c-index) [UCP `11] and the integrated Brier score (iBrier) [START_REF] Graf | Assessment and comparison of prognostic classification schemes for survival data[END_REF]. The scoring procedure is again based on the Rank Product test.

Moreover, in order to exploit the multi-omics aspect of the data in the following steps, we want to ensure that each group has a nonzero size after the current selection step. To do so, we perform the scoring procedure within each omic group g and select the min " p g , 0.1 min gPt2,...,mu pp g q `P ‹ ‰ most significant variables. This default setting aim to reduce all omics matrices to the same size, but the practitioner can assign different selection rates to each matrix.

Algorithm 3 Marginal selection

Input: Dataset Data " pY, δ, Xq, group index GId " pId 1 , ..., Id m q where Id 1 are the indexes of the clinical variables if included, ultimate number of variables to select P ‹ , Optional: number of variables to be select in each omic group P

Step 2: Selection of correlated groups

We have previously discussed the existence of correlation structures within omics data. In particular, within each grouping by omic type, we can observe more or less strong correlations between predictors.

Co-regulated gene modules and gene regulatory networks included in RNA-sequencing data are a good illustration of this point. In this step, we want to exploit the fact that some variables have a joint expression on the model, and offer better results when their correlation structure is taking into account.

To do so, we perform a selection by groups of correlated variables, segmented in two phases. First we build the groups using hierarchical clustering, then we aggregate four selection methods per group in order to assign a significant score to each of them. As in step 1, this procedure does not include the clinical data, and the selection models will be fit on the variables Y , δ and the pair pY, δq. For a better understanding, let's break down the Algorithm 4 which summarizes the flow of this selection step.

Recall that the input dataset results from step 1. The clustering procedure is applied to each distinct omic group, and the number of clusters considered is proportional to the size of the initial group. This arbitrary parameterization is set to ceilingpp g {50q, where p g is the number of variables in the group g. The idea is to create clusters of reasonable sizes to allow the estimation algorithms to run smoothly. Furthermore, note that the initial clustering may lead to the isolation of some variables. In this case, the algorithm will progressively reduce the number of clusters until we obtain subgroups of two predictors minimum.

For the sake of clarity, we now call groups the omic groups and subgroups the clusters resulting from the hierarchical clustering. In the estimation phase, the algorithm applies four group selection methods on three subgroups of variables. Each subset of subgroups is constructed randomly, so that any subgroup is included in a single subset. In order to test different combinations of subgroups and obtain relevant results, we iterate this estimation phase five times. Once the estimations are completed, each subgroup is assigned a significant score. The most significant are then selected so as to keep approximately minp 0.1p `P ‹ , p q variables. The method used in the estimation procedure are our Bayesian PGGM (see Chapter 3), the logistic group Lasso of Meier et al. [START_REF] Meier | The group lasso for logistic regression[END_REF], the IPF-Lasso of Boulesteix et al. [START_REF] Boulesteix | Ipf-lasso: integrative-penalized regression with penalty factors for prediction based on multi-omics data[END_REF]

Step 3: Selection of decorrelated variables

The main goal of step 2 was not only the selection of correlated groups, but also a selection of relevant variables based on random patterns favorable to their expression, i.e. combined with variables that should highlight their importance through joint actions. With the resulting data as support, we now want to recover variables that are both significant and as uncorrelated as possible, in order to limit information redundancy. To do so, we follow a reasoning similar to De Jay et al. in their mRMRe method [DJPCO `13],

but adapting it to our consensual context. For a better understanding, let's break down the Algorithm 5 which summarizes the flow of this third selection step.

First, let's denote p poq the number of non-clinical predictors in Data s3 , and let nvar " p poq {10 and nrep " p poq be the default parameters. This step is divided into two phases, an estimation phase and a selection phase. In the first one, the algorithm begins by constructing subsets of Data s3 each containing nvar of randomly drawn non-clinical predictors, so that any variable is included in a single subset. Then, the estimation phase goes on to apply four variable selection methods on these samples. In order to test different combinations of variables and obtain a relevant selection, this phase is iterated nrep times.

The method used in the estimation procedure are our Structural PGGM (see Chapter 2), the logistic Lasso [Lok99, SK03], the Gradient boosting with component-wise for Cox model [START_REF] Buehlmann | Boosting for high-dimensional linear models[END_REF][START_REF] De | Boosting in cox regression: a comparison between the likelihood-based and the model-based approaches with focus on the r-packages coxboost and mboost[END_REF], and the Spike-and-Slab Lasso Cox of Tang et al. [START_REF] Tang | The spike-and-slab lasso cox model for survival prediction and associated genes detection[END_REF]. As in the previous step, the results of this estimation phase provide a significance score for each variable, expressed as the number of times it was selected by the different methods. However, we will not only select the variables with the highest scores, but those Compute a bmlasso and give `1 to the score of variables selected Compute a glmboost and give `1 to the score of variables selected end for end for nsel " ceilingpτ p poq q Selected_vars " the variable j with the highest score while number of predictor selected ă nsel do score ppq j " score j penalized by the correlation with the selected predictors Add to Selected_vars the variable j with the highest penalized score end while that also have a low level of correlation with the other selected variables.

Step 4: Final selection

Throughout the previous steps, we refined the non-clinical dataset to keep only the most relevant variables.

We can now integrate the clinical data into the selection process, without concern about being unfairly underestimated due to their small size. This last step focuses on the estimation of survival models, and is composed of three phases ; an estimation phase, a first selection phase by scoring, and a second selection phase with forward-backward approach. The Algorithm 6 summarizes its flow.

Adding clinical data forces us to reflect on an additional problem of clinical analysis ; in particular, according to practitioners, we often observe a poison effect of certain omics variables on clinical variables.

In other words, the presence of certain variables in a model can negatively impact the evaluation of some clinical variables. To limit this inconvenience, we segment this last estimation phase in three parts.

First, we build the models using each individual group separately, then by pairs of groups, and finally we consider the interaction of all groups together. In this way, we can assess the relevance of the variables in different contexts. The method used in the estimation procedure are the sparge-group Lasso of Simon et al. [START_REF] Simon | A sparse-group lasso[END_REF], the priority-Lasso of Klau et al. [KJH `18], the random survival Forest of Ishwaran et al. for each group, each pair of groups and all groups together do Compute a sparse-group Lasso and give `1 to the score of variables selected Compute a priority-Lasso favoring clinical data and give `1 to the score of variables selected Compute a survival Forest and give `1 to the score of variables selected Compute a Coxboost and give `1 to the score of variables selected end for Select the P ‹ variables with the highest score Select the best combination of variables using a forward-backward approach that minimize the iBrier score on the learning dataset we perform a forward-backward selection procedure in order to minimize the cross-validation iBrier score.

Summary

We end this presentation of the SVSSA method with the Table 5.1 that provides a summary of the methods and algorithms used.

Application on real data

In this section, we explore the problem that motivated this thesis, the selection of relevant features for the diagnosis of TNBC. As presented in the Introduction, TNBC is a particularly challenging tumor form in oncology. Because of the difficulties encountered by oncologists when evaluating the diagnosis, there is a significant relapse rate in patients and a rapid progression of the disease to a nearly incurable metastatic stage. The SVSSA method seeks to overcome these limitations by providing a reasonable amount of significant prognostic factors, thus impacting the therapeutic follow-up of patients. In order to evaluate its relevance in this context, we will compare its predictive performances with those of other variable selection methods, on a real dataset.

Presentation of the dataset

Our study is based on data from the multi-omics TNBC cohort at Fudan University Shanghai Cancer Center (FUSCC), available on the National Omics Data Encyclopedia (NODE) platform 3 . This cohort included 465 TNBC patients, for whom we have one to three matrices of omic data. In order to perform a multi-omics analysis, we restricted our study to patients displaying the three omic measures. This left us with 233 patients and 253 411 explanatory variables distributed in four matrices: clinical, RNA sequencing (RNAseq), mutation and copy-number alteration (CNA). We are clearly in a high dimensional framework and more precisely in multi-omics data analysis. Figure 5.2 shows the distribution of the omics 3. NODE: https://www.biosino.org/node/project/detail/OEP000155

A.2 Step 1: Marginal selection

A.2.1 Correlation with survival time

To avoid any bias induced by censoring, the correlation analysis with survival time is only based on individuals who observed the event, i.e. pX pdq , Y pdq q.

Spearman correlation [Spe04]

The Spearman rank correlation coefficient is a nonparametric measurement of dependence based on the rank statistic. Let X j be a quantitative predictor and T the actual survival time variable, for which we have the sample pX pdq j , Y pdq q. The Spearman correlation coefficient between X pdq j and Y pdq is given by ρ j " S `rankpX pdq j q , rankpY pdq q bS `rankpX

where Sp.q and Sp. , .q correspond respectively to the empirical variance and covariance functions. RS j indicates the degree of monotonic dependence between the two variables ; the closer it is to 1 in absolute value, the higher the dependence. However, we are not interested in the correlation coefficient itself but in its significance. We then proceed with a bilateral test verifying H 0 : RS j " 0. A low p-value indicates the existence of a potential monotonic dependence between X j and T . This measure constitutes our significance score for this method.

A.2.2 Correlation with survival status

Here we analyze the correlation with survival status δ from the sample pδ, Xq.

Wilcoxon-Mann-Whitney test [Wil45, MW47]

The W-M-W test is a nonparametric statistical hypothesis test based on the rank statistic. It verifies the existence of significant differences between two groups of individuals on the basis of a quantitative variable. Consider a quantitative predictor X j for which we have independent and identically distributed observations, distributed within two samples X pdq j and X pcq j . Let F d and F c be the distribution functions corresponding to the two samples. The bilateral W-M-W test verifies H 0 : F d ptq " F c ptq @t. A low p-value implies a significant difference between the two groups, and therefore the existence of a potential dependence between X j and δ. This measure constitutes our significance score for this method.

A.2.3 Correlation with the survival pair

The last method of this step studies the correlation between each variable and the pair pY, δq. To this end, we use two measures of model performance, the Uno's concordance index (c-index) and the integrated Brier score (iBrier) ; while the c-index only assesses the model discrimination power, the iBrier evaluates also its accuracy. By using these two measures, we want to exploit the idea that a significant variable will result in a well-performing model. `11]. Uno et al. shown that the Harrell index presented in A.1.2 is biased when the number of censored data is high, and proposed an adaptation that overcomes this shortcoming. To assess the relationship between a variable j and the survival pair, we compute the c-index of the simple cox model (X j is the only predictor) by 10-fold cross-validation. This measure constitutes our significance score for this method.

Uno's concordance index [UCP

Integrated Brier score [GSSS99].

The Brier score was initially proposed for uncensored data. In this setting, it represents the mean squared deviation between the observed survival status and the survival probability estimated by the model. Graf et al. have later adapted it to survival problems with right censoring by weighting the squared deviations by the inverse probability of censoring. Let Gptq " PpC ą tq be the survival function of the censoring times, i.e the probability of not being censored until t. The Brier score at a given time t is defined by

with Ĝptq the Kaplan-Meier estimator of the censoring distribution. The integrated form provides an overall measure and is given by

A.2.4 Scoring

The global scoring is based on the Rank Product test procedure described in Section A.1.3. After harmonizing the scores, so that a low score refers to a high significance of the variable, the scoring function returns the p-value of associated with its ranking. For each omic group g, we select the P p1q g variables with the lowest p-values.

A.3 Step 2: Selection of correlated groups

A.3.1 Hierarchical clustering

We aim to group together the correlated variables within the estimation procedures, while limiting the size of the clusters to spare the borrowed alrogithms. We chose to perform a hierarchical clustering based on the proximity distance provided by unsupervised Random Forest. In our algorithm, we use this method to obtain the proximity matrix P g of the predictors of each initial group g. We then consider the transpose of the dataset restricted to the predictors of g, so that these represent the individuals to be classified in unsupervised trees.

Unsupervised Random Forest

Ward hierarchical clustering [WJ63]

Hierarchical ascending clustering (HAC) is an iterative grouping method based on a dissimilarity measure. We will use Ward's HAC which is the most widespread.

The algorithm is initialized with as many clusters as there are individuals, and merges at each step the two closest clusters in order to minimize the intra cluster variance, which is defined using a distance matrix between individuals. Let P g be the proximity matrix of the predictors of group g. Let Pg " 1 ´Pg be the distance matrix of the said predictors (which are our individuals here). The CAH optimization problem is given by arg min

where C actual is the set of current clusters, n X the number of elements in cluster X, and Pg px, yq the distance between x and y. The algorithm stops when there is only one cluster left. Once the maximal tree is obtained, we split it in order to recover the desired partition in m g clusters.

A.3.2 Estimation with survival time

As before, the estimation with the survival time will be based on the sample pY pdq , X pdq q to avoid the bias induced by censoring. We use here our Bayesian approach of the PGGM under the group-sparse setting.

Bayesian PGGM (see Chapter 3) For each set of four predictor subgroups, we apply the Bayesian PGGM algorithm with a default parameterization inducing sparsity, which is: a " 100, b " 1, type = group-sparse, and shrinkage = adaptive. We have chosen not to launch a parameter refinement process in order not to increase the computation time. However, we leave to the practitioner the possibility to fill in the parameters that suit him. The resulting model returns the list of subgroups selected from the four candidates. The latter then gain a point of significance.

A.3.3 Estimation with survival status

We continue with a classification-type variable selection method based on the sample pδ, Xq. The method borrowed here is the standard logistic group Lasso.

Logistic Group Lasso[MVDGB08]

The group Lasso of Yuan and Lin is an extension of the Lasso that performs variable selection at the group level in linear regression models. Meier et al. proposed an adaptation of this method for logistic regression. Let X " pX t 1 , ..., X t m q t P R p the vector of predictors divided into m groups, let p g be the size of X g the vector of group g. The logistic model is defined by

where β " pβ 0 , β t 1 , ..., β t m q t P R p`1 is the vector of regression coefficients. The group Lasso estimation of β is based on the following optimization problem βλ " arg min

where ℓpβq is the log-likelihood of the logistic model. Regarding to the penalty function, spp g q is a weight function which allows to rescale the penalty with respect to the size of the group. The default parameter is spp g q " ? p g , but this function can be modulated, for example it can be set to 0 for a group that we do not want to penalize. In our context, we consider four subgroups on which we apply the logistic group Lasso algorithm with a default parameterization inducing sparsity, i.e. λ " 10 ´2. Here again, we do not run a cross-validation procedure to adjust the parameter in order not to increase the computation time. The practitioner can of course change the default parameter. Just like for the Bayesian PGGM, the created model returns the list of the selected subgroups, those ones get a significance point.

A.3.4 Estimation with the survival pair

For this substep, we will use two methods specific to survival analysis that incorporate group structures.

The support sample is pY, δ, Xq.

IPF-Lasso [BDBJF17]

The IPF-Lasso method has been proposed by Boulesteix et al. as an improvement of Lasso in the context of multi-omics survival analysis. The authors explain that the use of several types of high-dimensional data requires the implementation of more adaptive selection methods.

Depending on the case, clinical data, which is both very small in size and very significant for prognosis, should not be penalized in the same way as very large omics data such as copy numbers. Therefore, their method allows to assign different penalty factors to groups of variables. The optimization problem follows that of the Cox Lasso, incorporating the group structure and different penalty terms β " arg min ´ℓℓ p pβq `m ÿ g"1

A.4 Step 3: Selection of decorrelated variables

A.5 Estimation with survival time

As before, the estimation with the survival time will be based on the sample pY pdq , X pdq q to avoid the bias induced by censoring. We use here our structuring approach of the PGGM.

Structural PGGM (see Chapter 2)

For each subset of data, we apply the GenGm algorithm with a default parameterization inducing sparsity, which is: λ " 0.001, µ " 0.05, η " 0.001, and β " 1.25. The matrix L " C ´1 associated with the structuring penalty is constructed from the subgroups obtained by clustering in step 2. For each pair of inputs pi, jq we have

if i and j are in the same subgroup, 0 otherwise.

(

The resulting model returns the list of inputs selected from the nvar candidates, these gain a point of significance.

A.5.1 Estimation with survival status

In the continuity of the group Lasso of step 2, we now borrow the well-known Lasso method on the sample pδ, Xq.

Logistic Lasso [Lok99, SK03] The Lasso method offers a sparse estimation of regression models, through a penalty ℓ 1 on the vector of coefficients. In the logistic framework, it is based on the following optimization problem βλ " arg min

where ℓℓpβq is the log-likelihood of the logistic model, and λ to the regularization parameter. In our procedure, we set this parameter by default to 10 ´2. At each iteration the model obtained returns the list of selected inputs, these gain one point of significance.

A.5.2 Estimation with the survival pair

For this substep, we will use two methods designed for variable selection, a boosting and a bayesian approaches. The support sample is pY, δ, Xq.

Gradient boosting with component-wise for Cox model [Bue06, DB16]

Boosting methods are based on an iterative estimation of model parameters, in order to gradually reduce its loss function.

More precisely, they aim to build a robust model from a set of weak models. There are several variants categorized according to the weak model estimation procedures and the final model construction rules. We

A.5.3 Scoring

The scoring procedure for this step is based on the minimum redundancy maximum relevance approach of De Jay et al. [DJPCO `13]. It consists in an iterative update of candidate variable importance scores according to their degree of association with the previously selected ones. Let's start by assigning each variable j a basic score of significance, which corresponds to the number of times it was designated significant during the estimation phase. Let us denote this score R

where ρ S pX j¨, X ¨lq denotes the Spearman coefficient between inputs j and l. The algorithm stops when the size of S reaches the desired number of selected feautures.

A.6 Step 4: Final selection A.6.1 Estimation with the survival pair

For this estimation phase, we use four variable selection methods, a group structure approach, an approach favoring clinical variables, a nonlinear approach by decision trees, and a boosting approach. The support sample is pY, δ, Xq.

Sparse-Group Lasso [SFHT13]

This method proposed by Simon et al. is an extension of the Lasso which performs a variable selection both on group and within group level. It is based on the combination of two forms of penalties ; a ℓ 2 -norm penalty to induce groupwise sparsity (like the Group-Lasso), and a ℓ 1 -norm penalty to induce intra-group sparsity within non-zero groups (like a Lasso per group). In the Cox model framework, the estimation is based on the following optimization problem βλ " arg min

In our procedure we tune the penalty parameter λ by an 10-fold cross-validation, and set by default α " 0.95 to encourage grouping as recommended by the authors.

priority-Lasso [KJH `18] Klau et al. emphasize the interest for practitioners in having high-dimensional estimation methods that incorporate variable group structure and exploit a prior knowledge on their practical usability. Thus, in the context of omics data, for approximately the same degree of precision, practitioners will choose a model built from variables already included in routine diagnostics such as clinical data, than variables with a high acquisition cost. To handle these two points, the authors proposed the priority-Lasso method. This is a hierarchical regression method that takes group structures into account, which are ordered according to a prior knowledge of their priority. The idea is to update the estimator an iterative way by building, at each step, a predictor explaining the current estimator residuals with the next priority group. Thus, groups with low priority variables will only enter the model if they explain a variability of the outcome that is not explainable by groups with higher priority. Let us place ourselves in the Cox model framework, and let π " pπ 1 , ..., π m q be the permutation of the groups according to their order of priority (i.e. π 1 is the group with the highest priority). The first step of the estimation procedure consists in extracting the information available in the highest priority group. To do this, the algorithm builds a Lasso predictor on the output with variables from π 1 , by solving βpπ1q " arg min

where ℓℓ p pβ pπ1q |Y, δ, X pπ1q q corresponds to the log-likelihood of the Cox model restricted to variables from π 1 . Let h pπ1q be the predictor associated with βpπ1q and r pπ1q its observed residuals. The following steps seek to gradually explain the remaining information on the output variability using increasingly lower priority groups. At each step g is then constructed a Lasso predictor on the current estimator residuals according to the next priority group, by solving βpπgq " arg min

The new estimator h pπgq is associated with the coefficient vector p βpπ1q , ..., βpπgq q. The final estimator is obtained when the m groups have been considered. Thus constructed, a non-priority group will only be integrated into the model if it offers non-redundant information with the previous groups. However, the authors raised a shortcoming about this procedure. Since the residuals are computed on the data that were used to train the predictors, the model accuracy may be overestimated. To overcome this, they propose to estimate each predictor by cross-validation. We follow this recommendation in our procedure and also determine the penalty parameters λ pπgq by cross-validation.

Random Survival Forest [IKBL08]

In the paragraph on the Block Forest method, we have slightly discussed the construction of random survival Forest. This method, proposed by Ishwaran et al., is an adaptation of Breiman's Random Forest [START_REF] Breiman | Random forests[END_REF] to survival data. Like the reference method, it is based on the construction of B survival trees using bootstrap samples from the original data. Each tree is a set of binary tests dividing the population into nodes more and more homogeneous, until they reach terminal nodes. At each node split, the algorithm randomly draws nvar candidate variables, then selects among them the split that maximizes survival difference between daughter nodes. The growth of a tree stops when it reaches a terminal node containing a homogeneous population, or a minimum of d 0 ą 0 deaths.

where pY phq , δ phq q is the sample of observations in h and RpY phq ¨i q the number of individuals, within h, still at risk at time Y

In our procedure, we use the default parameters nvar, B and d 0 provided in the R algorithm, and for variable selection purposes, we retrieve the variable importance matrix from the model. The P ‹ most significant variables are considered significant for this substep.

Likelihood-based boosting for Cox model [TB06, BASB09, DB16]

Tutz and Binder proposed a boosting method suitable for any generalized linear model. The estimation procedure is based on a e componentwise approach, where at each step one coefficient of the estimator is updated by a weak model.

In the Cox model framework, the loss function to be minimized is defined by the ℓ 2 -norm penalized negative partial log-likelihood, in which the results of the previous boosting steps are incorporated as an offset. Thus at step k, the weak model of a variable j results from the following loss function

where β is the estimator updated at step k ´1, and η " X t β the offset term from this step. The algorithm is initialized with β " 0 p,1 and η " 0 n,1 . Then, for each iteration, it first computes bj the first-order approximation around 0 of (26) for each variable j. This is given by bj " B Bβ j ℓℓ pen p0| βq

The weak model associated with the current iteration is then designated by the coefficient bj ‹ for which j ‹ " arg min j ℓℓ pen pβ j | βq. Finally, the algorithm updates the estimator βj ‹ " βj ‹ `b j ‹ . This phase of estimating weak models is iterated It max times. This number is used as a regularization parameter to perform variable selection. In our procedure, we use the default parameter It max " 2P ‹ .

A.6.2 Selection

As previously mentioned, the selection phase of this final step is divided into two parts. The first part is based on a scoring which consists in counting the number of times each variable has been selected during the estimation phase. The P ‹ variables with the highest score are selected. The second phase of selection consists in searching for the best performing combination, among the most significant variables. First, we perform a bottom-up selection procedure ; at each step we add a variable to the model if it reduces the iBrier. Then we perform a top-down selection on the obtained model ; at each step we remove a variable from the model if this removal reduces the iBrier. The iBrier scores are obtained by cross-validation on the training data. Moreover, even if at the end of the procedure we obtain a combination of performing variables, we nevertheless recommend that practitioners also study the list of P ‹ variables selected by scoring.

Annexe