Marie Pierre

Séverine Affeldt

Priya Param

Singh

Se ´verine Affeldt

Ilaria Cascone

Rasim Selimoglu

Jacques Camonis

Herve ´isambert
email: herve.isambert@curie.fr

Frédéric P M Langevin

Juan I Garaycoechea

Gerry P Crossan

Ketan J Patel

Enigma machine settings Ref Wheels Ring Ground Plugs C II VIII IV XAQ RFV DV FE GP HO JW RM SZ TN UI XL

Keywords: skeleton, simulated datasets from simulated networks, 7, 3 Using simulated datasets from undirected networks, 7, 3, 1 Generating datasets from undirected networks, 7, 3, 2 Reconstructing simulated undirected networks, 11, 2, 3 Temporal cascade patterns of advantageous mutations, 12 Conclusions

The inference of causality is an everyday life question that spans a broad range of domains for which interventions or time-series acquisition may turn out to be impracticable if not unethical. Yet, elucidating causal relationships within real-life complex systems can be rather convoluted when relying exclusively on observational data. In this dissertation, I report a novel network reconstruction method, which combines constraint-based and Bayesian frameworks to reliably reconstruct graphical models despite inherent sampling noise in finite observational datasets.

The approach is based on an information theory result tracing back the existence of colliders in graphical models to negative conditional 3-point information between observed variables. In turn, this provides a confident assessment of structural independencies in causal graphs, based on the ranking of their most likely contributing nodes with (significantly) positive conditional 3-point information.

Starting from a complete undirected graph, dispensible edges are progressively pruned by iteratively taking off the most likely positive conditional 3-point information from the 2-point (mutual) information between each pair of nodes. The resulting network skeleton is then partially directed by orienting and propagating edge directions, based on the sign and magnitude of the conditional 3-point information of unshielded triples. This 3off2 network reconstruction approach is shown to outperform constraint-based, search-and-score and earlier hybrid methods on a range of benchmark networks. In addition, the 3off2 method provides promising predictions when applied to the reconstruction of complex biological systems, such as hematopoietic regulatory subnetworks, zebrafish neural networks, mutational pathways driving tumour progression or the interplay of genomic properties on the evolution of vertebrates.

I would like to thank many friends outside the Institute for their everyday support and affection. Thanks a lot Yoriko, Fanny, Emilie, Clémentine and Julien for all these nice moments that repeatedly helped me to put any bad day into perspective, and especially Patrick for all these years of kind patience and support. These acknowledgements would be incomplete without mentioning my family. I am infinitely indebted to my parents who showed me how important and rewarding it

I would like to thank all the people who have contributed to turn this Ph.D. project into a unique human and scientific adventure.

First and foremost, I am deeply grateful to my supervisor, Hervé Isambert, whose unwavering support and trust have continuously guided me on the somewhat convoluted path of science. Thank you so much for your permanent availability and precious involvement in this ambitious and increasingly promising project that we have begun few years ago! I would like to express my sincere gratitude to Florence Jaffrezic and Philippe Leray, for their precious time reading my manuscript and their constructive and expert comments.

I would like to extend my gratitude to Christophe Gonzales for having given me the honour to chair my thesis committee. I also wish to thank Loïc Paulevé and Patrick Meyer for triggering interesting discussions.

I am also grateful to Sylvie Coscoy, Jacques Camonis and Martin Weigt for accepting to be my tutors during these four years of thesis. Thank you for your scientific insights and advice. I would also like to thank the professors from UPMC, Nizar Ouarti, Alessandra Carbone, Béatrice Bérard and Cécile Le Page, for offering me the exciting opportunity to teach during my Ph.D. project, and all the UPMC students of these courses for their steady dynamic involvement. I am also thankful to my former colleagues at Dassault Systèmes, and in particular Sylvain Huzé and Michel Latappy for their encouraging enthusiasm when I shared with them my plan of getting back to school. It would have not been possible to pursue such a wide range of captivating bioinformatic projects without the help of many master trainees. Thank you Samya, Maria, Francesca, Jatin, Wenjun and Fazal for your invaluable contribution! I am also very grateful to Sasha, Atoussa, Sébastien and Mica for their friendly past and ongoing collaborations.

Workdays would have been far less inspiring without my dear colleagues and friends, especially Giulia and Param, with whom I have shared most of my stay at the Institute.

Thanks a lot for the thrilling discussions and, most importantly, for participating in building such a nice team spirit. I would also like to thank Louis for his numerous biological inputs, and above all, for his kindness. Many thanks also go to the people with whom I had the chance to share my office, Démosthène, Nada, Grégory, Vasilica, Marzuk, Senthil, Simon, Aurore, Sabrina, Vasily, Daniel, Adrien and Guillaume. I would like to address many thanks to the efficient administrative team of the Institute, and in particular Agnès for being so generous with her time when it comes to helping others.

Chapter 1 Introduction

When I joined the Physical-Chemistry Curie Laboratory in 2011, the Isambert's research group was deeply involved in the understanding of a surprising observation. 'Dangerous' gene families, defined as prone to dominant (gain-of-function) mutations 1 , have been greatly expanded in the course of vertebrate evolution by contrast to gene families not implicated in diseases or prone to recessive (loss-offunction) mutations 2 . From an evolutionary perspective, this expansion of 'dangerous' gene families, implicated in cancer and other severe genetic diseases in human, is undoubtedly puzzling as it enhances the susceptibility of vertebrates to dominant genetic diseases. A first insight into the underlying evolutionary process came with the realization that the expansion of dangerous gene families resulted from very particular and dramatic accidents whereby the whole genome of a species is duplicated at once. Two rounds of such whole genome duplication (WGD) occurred at the origin of vertebrates some 500MY ago [Ohno, 1970, Putnam et al., 2008]. But could gene susceptibility to dominant deleterious mutations be somehow responsible for the striking evolutionary retention of 'dangerous' gene families after WGD? Dropped from a computational background, but truly motivated by issues related to evolution, I was immediately thrilled by this biological problem that highlights a surprising connection between two fairly different time scales. On the one hand, some genes, once mutated, play a crucial role in the development of complex diseases and hence, have a severe impact at the level of one individual's life span.

On the other hand, the reason why such genes remain in our genome, despite their potential harmfulness, could be found in the course of vertebrate evolution that begins hundreds of millions years ago.

The first part of my thesis was thus dedicated to the establishment of a novel evolutionary model suggesting that this somewhat counterintuitive expansion of 'dangerous' gene families is in fact a consequence of their susceptibility to deleterious mutations and purifying selection in polyploid species that arose from two rounds of whole genome duplication (WGD) events, dating back from the onset of jawed vertebrates, some 500MY ago [Ohno, 1970, Putnam et al., 2008]. In a nutshell, the evolutionary mechanism we have proposed relies on the fact that all the WGD-duplicates are initially acquired through speciation without the need to provide evolutionary benefit to be fixed in post-WGD species (section 1.1 & Chapter 8). This part of my research project, which has been the initial motivation for the second and main part of my PhD, has led to three publications (appended at the end of Chapter 1).

The main statistical approach I have used to analyse the retention of WGDduplicates is the Mediation analysis, formalized by Pearl [Pearl, 2001[START_REF] Pearl | The causal mediation formula-a guide to the assessment of pathways and mechanisms[END_REF]. It enabled me to disentangle and quantify the direct and indirect causal effects between the susceptibility to dominant deleterious mutations, the retention of WGDduplicates, so-called 'ohnologs' 3 , and a third genomic property chosen among the ones proposed so far to explain the retention of ohnologs (section 1.1 & Chapter 8). Yet, the Mediation analysis requires an oriented graphical model as input and does not directly uncover causal insights from observational data. Thus, I turned to the field of artificial intelligence and uncertainty in order to infer causal graphical models that could answer complex biological problems, such as the retention of ohnologs. In particular, beyond the susceptibility to dominant deleterious mutations, many other genomic properties, such as gene essentiality 4 , functional ontology, expression level, divergence rates, etc, that are all correlated to some extent with the fate of gene duplicates throughout vertebrate evolution, needed also to be taken into account.

This led me to the second part of my research project, which is the bulk of my thesis, namely the reconstruction of reliable causal graphical models from finite 3 In his seminal book, Susumu Ohno laid down the theoretical framework of the evolution after gene duplication [Ohno, 1970]. He proposed that genome duplications are a significant mechanism of evolution, even in the animal genomes [Ohno, 1970, Ohno et al., 1968]. The copies of genes arising from a whole genome duplication are now coined 'ohnologs', after his name. 4 Essential genes are genes for which the silencing of the two alleles lead to lethality or sterility. The essential genes are typically prone to recessive mutations. observational data, an utterly important issue in bioinformatics and other computational fields. In this part, I have developed a robust approach to reconstruct graphical models from finite datasets that combines the advantages of the stateof-the-art constraint-based (Chapter 3) and Bayesian (Chapter 4) methods. This new hybrid approach, named 3off2, is based on an information theoretic framework (Chapter 5) to confidently ascertain structural independencies in causal graphs and skeleton orientations (Chapter 6) with clear improvements over both constraintbased and Bayesian methods on benchmark networks (Chapter 7).

In the following section, I will briefly summarize the first part of my thesis (see reprints of papers at the end of this chapter for details) and also introduce one of the first applications for which the 3off2 method has brought valuable insights (Chapter 8). Yet, as shown later in this manuscript, the 3off2 method can be applied to a wide range of complex biological problems, such as the reconstruction of neural networks (Chapter 9), the discovery of transcription factors regulation networks (Chapter 10) or mutational pathways during tumour progression (Chapter 11). In the remaining of this chapter, I will introduce two main methods for causal graphical model reconstruction. Further details are provided in Chapters 3, 4 & 5. Finally, I will briefly outline how the 3off2 approach combines the state-of-the-art methods and provides a more robust scheme to reconstruct causal graphical models from finite observational datasets. The complete description of the 3off2 approach is detailed in Chapter 6.

Expansion of 'dangerous' gene families in vertebrates

Whole genome duplication (WGD) events have now been established in all major eukaryotes kingdoms. Two rounds of WGD in early vertebrates are frequently credited with creating the condition for the evolution of vertebrate complexity.

In the first part of my thesis, I have shown that the two rounds of WGD in vertebrates have led to the preferential expansion of 'dangerous' gene families, defined as prone to dominant deleterious mutations. The details of this work are left in the reprints of the publications appended at the end of this Chapter. These results suggest that the striking expansion of gene families implicated in cancer and other severe genetic diseases is in fact a consequence of their susceptibility to deleterious mutations and the ensuing purifying selection in post-WGD species.

Our data mining analysis, based on the 20, 506 human protein coding genes, first revealed a strong correlation between the retention of WGD duplicates, and their susceptibility to dominant deleterious mutations in human [START_REF] Affeldt | Évolution et cancer -expansion des familles de gènes dangereux par duplication du génome [evolution and cancer: expansion of dangerous gene repertoire by whole genome duplications] (article in french)[END_REF], Singh et al., 2012]. It appears that the human genes associated with the occurrence of cancer and other genetic diseases [START_REF] Huminiecki | 2R and remodeling of vertebrate signal transduction engine[END_REF]095) have retained significantly more ohnologs than expected by chance (48% versus 35%; 48% : 3, 844/8, 095; P = 1.3 × 10 -128 , χ 2 ). I have also found that the retention of ohnolog is more strongly related to their 'dangerousness' than their 'essentiality' [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF][START_REF] Singh | Human dominant disease genes are enriched in paralogs originating from whole genome duplication[END_REF] (Chapter 8).

To go beyond mere correlations, I have then performed Mediation Analyses, following the approach of Pearl [Pearl, 2001[START_REF] Pearl | The causal mediation formula-a guide to the assessment of pathways and mechanisms[END_REF], and quantified the direct and indirect effects of many genomic properties, such as essentiality, expression levels or divergence rates, on the retention of ohnologs. This enabled the investigation of an alternative hypothesis frequently invoked to account for the biased retention of ohnologs, namely the 'dosage-balance' hypothesis [Makino and McLysaght, 2010].

While this hypothesis posits that the ohnologs are retained because their interactions with protein partners require to maintain balanced expression levels throughout evolution, it appeared that the ohnologs have in fact been eliminated from permanent complexes in human (7.5% versus 35%; 7.5% : 18/239; P = 1.2 × 10 -18 , χ 2 ). The Mediation Analyses also showed (Fig. 1.1) that the gene susceptibility to deleterious mutations is more relevant than dosage-balance for the retention of ohnologs in more transient complexes [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF] (Chapter 8).

These results suggest that the retention of human ohnologs is primarily caused by their susceptibility to deleterious mutations. They further establish that the retention of many ohnologs suspected to be dosage balanced is in fact mediated by their susceptibility to dominant deleterious mutations. All in all, this supports a new evolutionary model relying on a non-adaptive mechanism that hinges on (i) the speciation event concomitant to WGD, and (ii) the dominance of deleterious mutations leading to purifying selection in post-WGD species (see appended publications & Chapter 8). These results exemplify the role of non-adaptive forces on the emergence of eukaryotes complexity.

Learning causal graphical models

The prospect of learning the direction of causal dependencies from mere correlations in observational data has long defied practical implementations [START_REF] Reichenbach | The Direction of Time[END_REF]. The fact that causal relationships can, to some extent, be inferred from nontemporal statistical data is now known to hinge on the unique statistical A Mediation Analysis using all human protein coding genes [START_REF] Conant | Duplicate genes and robustness to transient gene knock-downs in Caenorhabditis elegans[END_REF]506) B Mediation Analysis using human protein coding genes without SSD nor CNV [START_REF] Huminiecki | 2R and remodeling of vertebrate signal transduction engine[END_REF]215) Dosage Bal. Ohnolog Figure 1.1 Quantitative Mediation analysis of direct versus indirect effects of deleterious mutations and dosage balance on the retention of human ohnologs using (A) all human protein coding genes [START_REF] Conant | Duplicate genes and robustness to transient gene knock-downs in Caenorhabditis elegans[END_REF]506) or (B) human protein coding genes without small scale duplicates (SSD) nor copy number variants (CNV) [START_REF] Huminiecki | 2R and remodeling of vertebrate signal transduction engine[END_REF]215). The thickness of the arrows outlines the relative importance of the corresponding direct or indirect effects. Dir. < 0 or Ind. < 0 corresponds to an anticorrelated direct or indirect effect, respectively. Gene prone to deleterious mutations and/or to dosage balance (including haploinsufficient genes and genes involved in multiprotein complexes) are taken from [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF].

imprint of colliders in causal graphical models, provided that certain assumptions are made about the underlying process of data generation, such as its faithfulness to a tree structure [START_REF] Rebane | The recovery of causal poly-trees from statistical data[END_REF] or a directed acyclic graph model [START_REF] Pearl | Causality: models, reasoning and inference[END_REF][START_REF] Spirtes | Causation, Prediction, and Search[END_REF] (see Chapter 2). These early findings led to the developments of two types of network reconstruction approaches that have been applied to a variety of experimental datasets. These methods are either based on

Bayesian scores [START_REF] Cooper | A bayesian method for the induction of probabilistic networks from data[END_REF]Herskovits, 1992, Heckerman et al., 1995] or rely on the identification of structural independencies, which correspond to missing edges in the underlying network [START_REF] Pearl | A theory of inferred causation[END_REF][START_REF] Spirtes | Causation, Prediction, and Search[END_REF], 2000].

Bayesian inference methods typically require heuristic search strategies, such as hill-climbing algorithms, to sample the super-exponential space of possible networks. By contrast, constraint-based methods are expected to run in polynomial time on sparse underlying graphs, provided that a correct list of conditional independencies is available. Yet, in practice, conditional independencies need to be ascertained from the available observational data, based on adjustable statistical significance levels, and are not robust to sampling noise from finite datasets.

1.2.1

The state-of-the-art approaches Structure learning algorithms based on the identification of structural constraints, so-called constraint-based approaches such as the PC [START_REF] Spirtes | An algorithm for fast recovery of sparse causal graphs[END_REF] and IC [START_REF] Pearl | A theory of inferred causation[END_REF] algorithms, do not score and compare alternative networks. Instead they aim at ascertaining conditional independencies between variables to directly infer the Markov equivalent class of all causal graphs compatible with the available observational data. Yet, these methods are not robust to sampling noise in finite datasets as early errors in removing edges from the complete graph typically trigger the accumulation of compensatory errors later on in the pruning process. This cascading effect makes the constraint-based approaches sensitive to the adjustable significance level α, required for the conditional independence tests. In addition, traditional constraint-based methods are not robust to the order in which the conditional independence tests are processed, which prompted recent algorithmic improvements intending to achieve order-independence [START_REF] Colombo | Order-independent constraint-based causal structure learning[END_REF] (see Chapter 3).

Search-and-score methods

By contrast, search-and-score methods [START_REF] Chickering | Learning equivalence classes of bayesian-network structures[END_REF][START_REF] Cooper | A bayesian method for the induction of probabilistic networks from data[END_REF][START_REF] Friedman | Being bayesian about network structure. A bayesian approach to structure discovery in bayesian networks[END_REF], Heckerman et al., 1995] have the advantage of allowing for quantitative comparisons between alternative networks through their Bayesian scores but they are limited to rather small causal graphs due to the super-exponential space of possible directed graphs to sample. Hence, searchand-score approaches typically require either suitable prior restrictions on the structures [START_REF] Koivisto | Exact bayesian structure discovery in bayesian networks[END_REF]Sood, 2004, Silander and[START_REF] Silander | A simple approach for finding the globally optimal bayesian network structure[END_REF] or heuristic strategies such as hill-climbing algorithms to sample network space [START_REF] Bouckaert | Properties of bayesian belief network learning algorithms[END_REF], Chickering et al., 1995[START_REF] Friedman | Learning bayesian network structure from massive datasets: The 'sparse candidate' algorithm[END_REF] (see Chapter 4).

3off2: a novel network reconstruction method

In the second and main part of my thesis, I have developed a more robust approach to reconstruct graphical models from finite datasets that combines the main advantages of constraint-based and search-and-score methods to reliably reconstruct graphical models despite inherent sampling noise in finite observational datasets.

This new hybrid method, embedded in an information theoretic framework (Chapter 5), confidently ascertain structural independencies in causal graphs based on the ranking of their most likely contributing nodes. In a nutshell, this local optimization scheme and corresponding 3off2 algorithm iteratively take off the most likely conditional 3-point information from the 2-point (mutual) information between each pair of nodes. Conditional independencies are thus derived by progressively collecting the most significant indirect contributions to all pairwise mutual information (Chapter 6, section 6.3.1). The resulting network skeleton is then partially directed by orienting and propagating edge directions, based on the sign and magnitude of the conditional 3-point information of unshielded triples (Chapter 6, section 6.3.2.2). The approach is shown to outperform both constraint-based and Bayesian inference methods on a range of benchmark networks ( Chapter 7) and to retrieve more experimentally ascertained results than with other available methods in the field of systems biology .

INTRODUCTION

Just as some genes happen to be more ''essential,'' owing to their deleterious loss-of-function or null mutations, some genes can be classified as more ''dangerous,'' due to their propensity to acquire deleterious gain-of-function mutations. This is, in particular, the case for oncogenes and genes with autoinhibitory protein folds, whose mutations typically lead to constitutively active mutants with dominant deleterious phenotypes [START_REF] Pufall | Autoinhibitory domains: modular effectors of cellular regulation[END_REF]. Dominant deleterious mutations, that are lethal or drastically reduce fitness over the lifespan of organisms, must have also impacted their long term evolution on timescales relevant for genome evolution (e.g., >10-100 million years [MY]). In fact, dominant disease genes in humans have been shown to be under strong purifying selection [START_REF] Furney | Differences in the evolutionary history of disease genes affected by dominant or recessive mutations[END_REF]Blekhman et al., 2008;[START_REF] Cai | Similarly strong purifying selection acts on human disease genes of all evolutionary ages[END_REF]). Yet, ''dangerous'' gene families implicated in cancer and severe genetic diseases have also been greatly expanded by duplication in the course of vertebrate evolution. For example, the single orthologous locus, Ras85D in flies and Let-60 in nematodes, has been duplicated into three RAS loci in typical vertebrates, KRAS, HRAS, and NRAS, that present permanently activating mutations in 20%-25% of all human tumors, even though HRAS and NRAS have also been shown to be dispensable for mouse growth and development [START_REF] Ise | Targeted deletion of the H-ras gene decreases tumor formation in mouse skin carcinogenesis[END_REF][START_REF] Esteban | Targeted genomic disruption of H-ras and Nras, individually or in combination, reveals the dispensability of both loci for mouse growth and development[END_REF].

While the maintenance of essential genes is ensured by their lethal null mutations, the expansion of dangerous gene families remains an evolutionary puzzle from a natural selection perspective. Indeed, considering that many vertebrate disease genes are phylogenetically ancient [START_REF] Domazet-Loso | An ancient evolutionary origin of genes associated with human genetic diseases[END_REF][START_REF] Cai | Similarly strong purifying selection acts on human disease genes of all evolutionary ages[END_REF]Dickerson and Robertson, 2012), and that their orthologs also cause severe genetic disorders in extant invertebrates [START_REF] Berry | Germ-line tumor formation caused by activation of glp-1, a Caenorhabditis elegans member of the Notch family of receptors[END_REF][START_REF] Ciocan | The role of ras gene in the development of haemic neoplasia in Mytilus trossulus[END_REF][START_REF] Robert | Comparative study of tumorigenesis and tumor immunity in invertebrates and nonmammalian vertebrates[END_REF], it is surprising that dangerous gene families have been duplicated more than other vertebrate genes without known dominant deleterious mutations. While gene duplicates can confer mutational robustness against loss-of-function mutations, multiple copies of genes prone to gain-of-function mutations are expected to lead to an overall aggravation of a species' susceptibility to genetic diseases and thus be opposed by purifying selection.

Two alternative hypotheses can be put forward to account for the surprising expansion of dangerous gene families. Either, the propensity of certain genes to acquire dominant deleterious mutations could be a mere by-product of their presumed advantageous functions. In that case, only the overall benefit of gene family expansion should matter, irrespective of the mechanism of gene duplication. Alternatively, gene susceptibility to dominant deleterious mutations could have played a driving role in the striking expansion of dangerous gene families. But what could have been the selection mechanism?

In this article, we report converging evidences supporting the latter hypothesis and propose a simple evolutionary model to explain the expansion of such dangerous gene families. It is based on the observation that the majority of human genes prone to dominant deleterious mutations, such as oncogenes and genes with autoinhibitory protein folds, have not been duplicated through small scale duplication (SSD). Instead, the expansion of these dangerous gene families can be traced back to two rounds of whole-genome duplication (WGD), that occurred at the onset of jawed vertebrates, some 500 MY ago (Ohno, 1970;Putnam et al., 2008).

These two rounds of WGD in the early vertebrate lineage are frequently credited with creating the conditions for the evolution of vertebrate complexity. Indeed, WGD-duplicated genes, socalled ''ohnologs'' in honor of Susumu Ohno (Ohno, 1970;[START_REF] Wolfe | Robustness-it's not where you think it is[END_REF], have been preferentially retained in specific gene classes associated with organismal complexity, such as signal transduction pathways, transcription networks, and developmental genes [START_REF] Maere | Modeling gene and genome duplications in eukaryotes[END_REF][START_REF] Blomme | The gain and loss of genes during 600 million years of vertebrate evolution[END_REF][START_REF] Freeling | Gene-balanced duplications, like tetraploidy, provide predictable drive to increase morphological complexity[END_REF][START_REF] Se ´mon | Consequences of genome duplication[END_REF]Makino and McLysaght, 2010;Huminiecki and Heldin, 2010). By contrast, gene duplicates coming from SSD are strongly biased toward different functional categories, such as antigen processing, immune response, and metabolism (Huminiecki and Heldin, 2010). SSD paralogs and WGD ohnologs also differ in their gene expression and protein network properties [START_REF] Hakes | All duplicates are not equal: the difference between small-scale and genome duplication[END_REF][START_REF] Guan | Functional analysis of gene duplications in Saccharomyces cerevisiae[END_REF]. Furthermore, recent genome-wide analysis have shown that ohnologs in the human genome have experienced fewer SSD than ''nonohnolog'' genes and tend to be refractory to copy number variation (CNV) caused by polymorphism of small segmental duplications in human populations (Makino and McLysaght, 2010). These antagonist retention patterns of WGD and SSD/CNV gene duplicates in the human genome have been suggested to result from dosage balance constraints (Makino and McLysaght, 2010) on the relative expressions of multiple protein partners [START_REF] Veitia | Exploring the etiology of haploinsufficiency[END_REF], as proposed earlier for other organisms like yeast [START_REF] Papp | Dosage sensitivity and the evolution of gene families in yeast[END_REF] and the paramecium [START_REF] Aury | Global trends of wholegenome duplications revealed by the ciliate Paramecium tetraurelia[END_REF].

In this article, we investigate the evolutionary causes responsible for the expansion of gene families prone to deleterious mutations in vertebrates and propose a simple evolutionary model accounting for their antagonistic retention pattern after WGD and SSD events. The retention of ohnologs in the human genome is shown to be more strongly associated with their susceptibility to deleterious mutations, than their functional importance or ''essentiality.'' We also demonstrate using a causal inference analysis, that the retention of many ohnologs suspected to be dosage balanced is in fact an indirect effect of their higher susceptibility to deleterious mutations. We argue that the enhanced retention of dangerous ohnologs is a somewhat counterintuitive yet simple consequence of the speciation event triggered by WGD and the ensuing purifying selection in post-WGD species.

These findings rationalize, from an evolutionary perspective, the WGD expansion of gene families frequently implicated in genetic disorders, such as cancer, and highlight the importance of nonadaptive selection on the emergence of vertebrate complexity.

RESULTS

Genes Prone to Deleterious Mutations Retain More Ohnologs

We first analyzed a possible association between the susceptibility of human genes to deleterious mutations and their retention of ohnologs, as proposed in [START_REF] Gibson | Genetic redundancy in vertebrates: polyploidy and persistence of genes encoding multidomain proteins[END_REF] for multidomain proteins. To this end, we considered multiple classes of genes susceptible to deleterious mutations from experimentally verified databases and literature. These classes include cancer genes (from multiple sources including COSMIC [Forbes et al., 2011] and CancerGenes [START_REF] Higgins | CancerGenes: a gene selection resource for cancer genome projects[END_REF]), genes mutated in other genetic disorders, dominant negative genes from OMIM, and genes with autoinhibitory protein folds (Experimental Procedures). We looked at the relative contributions of WGD and SSD in the expansion of these ''dangerous'' gene classes.

The results, depicted in Figures 1 andS1, demonstrate indeed a strong association between the retention of human ohnologs from vertebrate WGD and their reported susceptibility to deleterious mutations, as compared to nonohnologs, whereas an (A and B) Prevalence of retained ohnologs either ''w/ SSD or CNV'' or ''w/o SSD & CNV'' for all 20,506 human protein-coding genes (A), and gene classes susceptible to deleterious mutations (B). Note that gene classes with higher susceptibility to deleterious mutations retained more ohnologs.

(C) Ohnolog retention in gene classes susceptible to dosage balance constraints. Fold changes in ohnolog/nonohnolog ratios are given relative to the reference from all human genes in (A). See also Figure S1.

opposite pattern is found for SSD/CNV gene duplicates. Overall, the 8,095 human genes associated with the occurrence of cancer and other genetic diseases have retained significantly more ohnologs than expected by chance, 48% versus 35% (48%; 3,844/8,095; p = 1.3 3 10 À129 , c 2 test). Furthermore, these associations, which do not take into account the actual severity of the gene mutations, are clearly enhanced when the analysis is restricted to genes with direct experimental evidence of dominant deleterious mutations, such as dominant disease genes (59%; 261/440; p = 1.7 3 10 À27 , c 2 test), dominant negative mutants (61%; 292/477; p = 3.9 3 10 À34 , c 2 test), oncogenes (61%; 493/813; p = 1.4 3 10 À54 , c 2 test), or genes exhibiting autoinhibitory constraints (76%; 350/461; p = 2.7 3 10 À77 , c 2 test). The biased retention of ohnologs is even stronger for genes combining several factors associated with an enhanced susceptibility to deleterious mutations, such as cancer genes with autoinhibitory folds, (80%; 294/369; p = 1.0 3 10 À73 , c 2 test), or oncogenes with autoinhibitory folds, (91%; 104/114; p = 6.9 3 10 À37 , c 2 test). This retention of dangerous ohnologs is illustrated on Table 1 that presents an up-to-date list of 76 hand-curated gene families of up to four ohnologs, exhibiting both autoinhibitory folds and oncogenic properties (see Table S1 for oncogenic and autoinhibitory details and references). These dangerous ohnologs are typically found along signal transduction cascades, from receptor tyrosine kinases and cytoplasmic or nuclear kinases to guanine exchange factors (GEF), GTPase activating proteins (GAP), and transcription factors (Table 1, gene classes A-E). In addition, autoinhibited oncogenes are also found in other ohnolog families with diverse functions (Table 1, gene class F). By contrast, we obtained a hand-curated list of only ten nonohnolog genes exhibiting both autoinhibitory and oncogenic properties, Table 1, gene class G (see Table S2 for oncogenic and autoinhibitory details and references). Interestingly, half of them (4/10) can be traced back to SSD events, which occurred after or at the same period of the two WGD in early vertebrate lineages (Table S2). All in all, this implies that >90% of known oncogenes with autoinhibitory folds have retained at least one ohnolog pair in the human genome (as well as, possibly, a few additional duplicates from more recent SSD events).

Ohnologs Are Conserved but More ''Dangerous'' than ''Essential'' We then investigated whether the susceptibility of ohnologs to deleterious mutations could be directly quantified through comparative sequence analysis. We used Ka/Ks ratio estimates, which measure the proportion of nonsynonymous substitutions (Ka) to the proportion of synonymous substitutions (Ks) (Extended Results and Table S3). Ohnologs exhibit statistically lower Ka/Ks ratios, Figures 2,S2,and S3, which provides direct evidence of strong conservation, consistent with a higher susceptibility of ohnologs to deleterious mutations. Similar trends have also been reported for ohnologs specific to teleost fishes [START_REF] Brunet | Gene loss and evolutionary rates following whole-genome duplication in teleost fishes[END_REF] or to the more recent WGD in Xenopus laevis lineage (Se ´mon and [START_REF] Se ´mon | Preferential subfunctionalization of slowevolving genes after allopolyploidization in Xenopus laevis[END_REF]. Note, however, that the functional consequences of such deleterious mutations, leading either to a gain or a loss of function, cannot be directly inferred from Ka/Ks distributions. Yet, as outlined below, we found marked differences in the retention of ''dangerous'' ohnologs prone to dominant gain-of-function mutations and ''essential'' ohnologs exhibiting lethal loss-of-function or null mutations.

While autosomal-dominant disease genes exhibit a strong ohnolog retention bias (Figure 1B), 59% versus 35% (59%; 261/440; p = 1.7 3 10 À27 , c 2 test), autosomal-recessive disease genes are not significantly enriched in ohnologs 37% versus 35% (37%; 221/598; p = 0.24, c 2 test). Similarly, human orthologs of mouse genes, reported as being ''essential'' genes from large-scale null mutant studies in mouse, are not strongly enriched in ohnologs 56% versus 54% (56%; 1,537/2,729; p = 3.8 3 10 À3 , c 2 test), where 54% = 3,190/5,956 is the global proportion of ohnologs among the 5,956 genes tested for null mutation in mouse (Experimental Procedures). In fact, this small enrichment becomes even nonsignificant once genes with dominant allelic mutants are removed from the list of 5,956 genes tested for essentiality in mouse, i.e., 50% versus 48% (50%; 760/1,525; p = 0.09, c 2 test), where 48% = 1,782/3,739 is the global proportion of ohnologs among the 3,739 genes tested for essentiality in mouse, after removing dominant disease genes, oncogenes, and genes with dominant negative mutations or autoinhibitory folds.

All in all, this shows that the retention of ohnologs has been most enhanced for genes prone to autosomal-dominant deleterious mutations and not autosomal-recessive deleterious mutations. This suggests that the retention of ohnologs is more strongly related to their ''dangerousness,'' as defined by their high susceptibility to dominant deleterious mutations, than their functional importance or ''essentiality,'' as identified through large-scale null mutation studies in mouse.

Ultimately, we will argue that the ''dangerousness'' of ohnologs effectively controls their individual retention in the genomes of post-WGD species, as will be shown below in the section Model for the Retention of Dangerous Ohnologs.

Mixed Susceptibility of Human Ohnologs to Dosage Balance An alternative hypothesis, focusing instead on the collective retention of interacting ohnologs, has been frequently invoked to account for the biased retention of ohnologs in unicellular organisms like yeast [START_REF] Papp | Dosage sensitivity and the evolution of gene families in yeast[END_REF] or the paramecium [START_REF] Aury | Global trends of wholegenome duplications revealed by the ciliate Paramecium tetraurelia[END_REF] and in higher eukaryotes [START_REF] Birchler | Dosagedependent gene regulation in multicellular eukaryotes: implications for dosage compensation, aneuploid syndromes, and quantitative traits[END_REF]Makino and McLysaght, 2010). This ''dosage balance'' hypothesis posits that interacting protein partners tend to maintain balanced expression levels in the course of evolution, in particular, for protein subunits of conserved complexes [START_REF] Birchler | Dosagedependent gene regulation in multicellular eukaryotes: implications for dosage compensation, aneuploid syndromes, and quantitative traits[END_REF][START_REF] Veitia | Exploring the etiology of haploinsufficiency[END_REF][START_REF] Papp | Dosage sensitivity and the evolution of gene families in yeast[END_REF][START_REF] Veitia | A generalized model of gene dosage and dominant negative effects in macromolecular complexes[END_REF]Makino and McLysaght, 2010). Thus, SSD of dosage balanced genes are thought to be generally detrimental through the dosage imbalance they induce, thereby raising the odds for their rapid nonfunctionalization [START_REF] Papp | Dosage sensitivity and the evolution of gene families in yeast[END_REF][START_REF] Maere | Modeling gene and genome duplications in eukaryotes[END_REF]. By contrast, rapid nonfunctionalization of ohnologs after WGD has been suggested to be opposed by dosage effect, in particular, for highly expressed genes and genes involved in protein complexes or metabolic pathways [START_REF] Aury | Global trends of wholegenome duplications revealed by the ciliate Paramecium tetraurelia[END_REF][START_REF] Evlampiev | Modeling protein network evolution under genome duplication and domain shuffling[END_REF][START_REF] Gout | The relationship among gene expression, the evolution of gene dosage, and the rate of protein evolution[END_REF]Makino and McLysaght, 2010). This is because WGD initially preserves correct relative dosage between

ABL1 ABL2 PKN1 PKN2 PKN3 ARAF BRAF RAF1 PRKAA1 PRKAA2 AKT1 AKT2 AKT3 PRKCA PRKCB PRKCG CAMK1 CAMK1D CAMK1G PNCK PRKCE PRKCH CAMKK1 CAMKK2 PRKCI PRKCZ CSNK1D CSNK1E PRKD1 PRKD2 PRKD3 GSK3A GSK3B PRKG1 PRKG2 GRK4 GRK5 GRK6 PTK2 PTK2B JAK1 JAK2 JAK3 TYK2 RSK1 RSK2 RSK3 RSK4 SRC FGR FYN YES1 MSK1 MSK2 HCK LCK BLK LYN NDR1 NDR2 MKNK1 MKNK2 SYK ZAP70 NEK6 NEK7 C. Ohnolog GEF ARHGEF3 NET1 RALGDS RGL1 RGL2 RGL3 ARHGEF6 COOL1 SOS1 SOS2 DBL DBS MCF2L2 TIAM1 TIAM2 FGD1 FGD2 FGD3 FGD4 TIM WGEF SGEF NGEF PDZ-RHOGEF LSC LARG VAV1 VAV2 VAV3 P114-RHOGEF GEF-H1 D. Ohnolog GAP ASAP1 ASAP2 ASAP3 PLXNA1 PLXNA2 PLXNA3 PLXNA4 IQGAP1 IQGAP2 IQGAP3 PLXNB1 PLXNB2 PLXNB3 PLXND1
E. Ohnolog DNA Binding and Transcription Factors

CEBPA CEBPB CEBPE IRF4 IRF8 IRF9 CUX1 CUX2 MEIS1 MEIS2 MEIS3 ELK1 ELK3 ELK4 p53 p63 p73 ETS1 ETS2 RUNX1 RUNX2 RUNX3 ETV1 ETV4 ETV5 SOX1 SOX2 SOX3 ETV6 ETV7
F. Other Ohnolog Genes with Both Autoinhibitory and Oncogenic Properties

ANP32A ANP32B ANP32E nNOS eNOS ATP2B1 ATP2B2 ATP2B3 ATP2B4 NOTCH1 NOTCH2 NOTCH3 cIAP1j2 XIAP PLCB1 PLCB2 PLCB3 CCNT1 CCNT2 PLCD1 PLCD3 PLCD4 FLNA FLNB FLNC PLCG1 PLCG2 FURIN PCSK4 PTPN1 PTPN2 KPNA2 KPNA7 SMURF1 SMURF2 NEDD4 NEDD4L TRPV1j3 TRPV2 TRPV4 TRPV5j6 NOXA1 NOXA2
G. Nonohnolog Genes with Both Autoinhibitory and Oncogenic Properties

CAMK4 ELF3 MELK MOS PDPK1 BRK PTPN11 RET RPS6KB1 TTN
GEF, guanine exchange factors; GAP, GTPase activating proteins. See also Tables S1 andS2.

expressed genes, whereas subsequent random nonfunctionalization of individual ohnologs disrupts this initial dosage balance.

For instance, yeast Saccharomyces cerevisiae has retained 76% of its ribosomal gene ohnologs from a 150 MY old WGD [START_REF] Kellis | Proof and evolutionary analysis of ancient genome duplication in the yeast Saccharomyces cerevisiae[END_REF][START_REF] Lin | Protein complexity, gene duplicability and gene dispensability in the yeast genome[END_REF], although the maintenance of these ohnologs has been suggested to require frequent gene conversion events [START_REF] Kellis | Proof and evolutionary analysis of ancient genome duplication in the yeast Saccharomyces cerevisiae[END_REF][START_REF] Evangelisti | Nonrandom survival of gene conversions among yeast ribosomal proteins duplicated through genome doubling[END_REF] as well as fine-tuned dosage compensation to ensure a balanced expression with the remaining 24% ribosomal genes having lost their ohnologs [START_REF] Zeevi | Compensation for differences in gene copy number among yeast ribosomal proteins is encoded within their promoters[END_REF]. Following on this dosage balance hypothesis, we performed statistical analysis on multiprotein complexes from HPRD (Keshava [START_REF] Prasad | Human Protein Reference Database-2009 update[END_REF] and CORUM [START_REF] Ruepp | CORUM: the comprehensive resource of mammalian protein complexes-2009[END_REF] databases and a hand-curated list of permanent complexes [START_REF] Zanivan | A new computational approach to analyze human protein complexes and predict novel protein interactions[END_REF] (Experimental Procedures) to investigate for a possible association between the retention of human ohnologs and their susceptibility to dosage balance constraints.

The results depicted in Figure 1C demonstrate, in agreement with (Makino and McLysaght, 2010), that genes implicated in multiprotein complexes have retained significantly more ohnologs than expected by chance, 41% versus 35% (41%; 1,567/ 3,814; p = 8.7 3 10 À17 , c 2 test). This trend is also enhanced when focusing on haploinsufficient genes, that are known for their actual sensitivity to dosage balance constraints [START_REF] Qian | Gene dosage and gene duplicability[END_REF] (54%; 179/330; p = 8.0 3 10 À14 , c 2 test).

Yet, surprisingly, an opposite trend corresponding to the elimination of ohnologs is observed for genes implicated in permanent complexes, that are presumably strongly sensitive to dosage balance constraints (7.5%; 18/239; p = 1.2 3 10 À18 , c 2 test) (Figure 1C). In fact, looking more closely at the few human ohnologs, that have not been eliminated from permanent complexes (Table 2), we found that they are likely under less stringent dosage balance constraints than most proteins in permanent complexes, as they typically coassociate with mitochondrial proteins or form large multimeric subcomplexes with intrinsic stoichiometry disequilibrium.

This suggests that the elimination of most ohnologs from permanent complexes is, in fact, strongly favored under dosage imbalance and becomes likely inevitable once a few of those ohnologs have been accidentally lost following WGD. Indeed, the uneven elimination of ohnologs in permanent complexes is expected to lead to the assembly of nonfunctional, partially formed complexes detrimental to the cell, unless dosage compensation mechanisms effectively re-establish proper dosage balance at the level of gene regulation [START_REF] Birchler | Dosagedependent gene regulation in multicellular eukaryotes: implications for dosage compensation, aneuploid syndromes, and quantitative traits[END_REF], as for yeast ribosomal proteins [START_REF] Zeevi | Compensation for differences in gene copy number among yeast ribosomal proteins is encoded within their promoters[END_REF]. By contrast, transient complexes, which are typically more modular than permanent complexes, are expected to accommodate such dosage changes more easily, as they do not usually require the same strict balance in the expression levels of their protein partners.

These findings on the differences in retention of human ohnologs between permanent and more transient complexes suggest the relevance of different underlying causes. Although dosage balance presumably remains the primary evolutionary constraint in permanent complexes (<2% of human genes), which lead to the elimination of ohnologs in permanent complexes in See also the Extended Results, Figures S2 andS3, and Table S3 for statistical significance and comparison with other invertebrate outgroups.

vertebrate genomes, gene susceptibility to deleterious mutations may be more relevant for the retention of ohnologs within the 17% of human genes participating in more transient complexes. For instance, transient complexes involved in phosphorylation cascades or GTPase signaling pathways are known to be more sensitive to the level of activation of their protein partners than to their total expression levels. Thus, although the active forms of multistate proteins typically amount to a small fraction of their total expression level, hence providing a large dynamic range for signal transduction, it also makes them particularly susceptible to gain-of-function mutations. Such mutations can shift protein activation levels 10-to 100-fold without changes in expression levels and likely underlie stronger evolutionary constraints than the 2-fold dosage imbalance caused by gene duplication.

Indirect Cause of Ohnolog Retention in Protein Complex

To further investigate the relative effects of dosage balance and gene susceptibility to deleterious mutations, we analyzed whether the overall enhanced retention of ohnologs within multiprotein complexes (Figure 1C) could indirectly result from an enhanced susceptibility to deleterious mutations. Indeed, as outlined in Figure 3A, cancer and disease genes are more prevalent within complexes than expected by chance, 29% versus 19% (29%; 2,362/8,095; p = 3.7 3 10 À132 , c 2 test) and this trend is enhanced for genes with stronger susceptibility to deleterious mutations, such as oncogenes (39%; 320/813; p = 2.9 3 10 À52 , c 2 test) or oncogenes with autoinhibitory folds (59%; 67/114; p = 2.9 3 10 À28 , c 2 test). By contrast, ohnologs are only slightly, although significantly, more prevalent in complexes than expected by chance, 22% versus 19% (22%; 1,567/7,110; p = 9.0 3 10 À14 , c 2 test), whereas the proportion implicated in cancer or disease genes is clearly enhanced 54% versus 39% (54%; 3,844/7,110; p = 9.5 3 10 À140 , c 2 test).

To go beyond these simple statistical associations and quantify the direct versus indirect effects of deleterious mutations and dosage balance constraints on the biased retention of human ohnologs, we have performed a Mediation analysis following the approach of Pearl (Pearl, 2001[START_REF] Pearl | The Mediation Formula: A guide to the assessment of causal pathways in nonlinear models[END_REF]. The Mediation frame-work, developed in the context of causal inference analysis, aims at uncovering, beyond statistical correlations, causal pathways along which changes in multivariate properties are transmitted from a cause, X, to an effect, Y. More specifically, a Mediation analysis assesses the importance of a mediator, M, in transmitting the indirect effect of X on the response Y h Y(x,m(x)) (Figure 3B).

Mediation analyses have been typically used in social sciences research [START_REF] Baron | The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations[END_REF] as, for instance, in the context of legal disputes over alleged discriminatory hiring. In such cases, the problem is to establish that gender or race (X) have directly influenced hiring (Y) and not simply indirectly through differences in qualification or experience (M). Mediation analyses have also been used in epidemiology, as in a formal study [START_REF] Robins | Identifiability and exchangeability for direct and indirect effects[END_REF] that establishes the direct effect of smoking (X) on the incidence of cardiovascular diseases (Y), while taking into account the indirect effect of other aggravating factors, such as hyperlipidemia (M).

In this report, we have applied the Mediation analysis to genomic data to discriminate between direct effect (DE) and indirect effect (IE) of deleterious mutations (X or M) and dosage balance constraints (M or X) on the biased retention of human ohnologs (Y). The results, derived in Extended Experimental Procedures (Table S4) and summarized in Figure 3C and Table S5, demonstrate that the retention of ohnologs in the human genome is more directly caused by their susceptibility to deleterious mutations than their interactions within multiprotein complexes.

Indeed, the direct causal effect of a change from ''noncomplex'' to ''complex'' proteins only accounts for 23% of a small total effect (TE) of complex on the retention of ohnologs (DE/TE = 23% with TE = 0.079), whereas 82% of this small total effect is indirectly mediated by their susceptibility to deleterious mutations (IE/TE = 82% with 5% nonlinear coupling between direct and indirect effects) (Extended Results). By contrast, the alternative hypothesis, assuming a direct effect of deleterious mutations, accounts for 99% of a three times larger total effect on ohnolog retention (DE/TE = 99% with TE = 0.23), whereas the ''complex'' versus ''noncomplex'' status of human genes has a negligible indirect effect on ohnolog retention in this case (IE/TE = 2%) (Extended Results). These trends are also further enhanced when the analysis is restricted to the 40% of human genes [START_REF] Huminiecki | 2R and remodeling of vertebrate signal transduction engine[END_REF]215) without SSD and CNV duplicates (Figure 3D; Table S5; Extended Results). In fact, the direct effect of multiprotein complexes then tends to oppose the retention of ohnologs (DE/TE = À33% with TE = 0.064), as in the case of permanent complexes detailed above, but on an increased sample size of 8,215 genes without SSD or CNV duplicates (i.e., more than a third of human genes) in place of 239 genes from permanent complexes. By contrast, there is a five times larger total effect due to the direct effect of deleterious mutations on the retention of ohnologs (DE/TE = 101% with TE = 0.32), Figure 3D. This is an instance of Simpson's paradox, where two effects oppose each other, thereby, revealing the existence of conflicting underlying causes, namely, a strong positive effect of deleterious mutations and a small negative effect of dosage balance constraints on the retention of human ohnologs without SSD and CNV duplicates.

We have also examined the effects of other alternative properties on the retention of ohnologs (Extended Results; Table S5). In particular, we have found that gene expression levels and Ka/Ks ratios do not significantly mediate the effect of deleterious mutations on the retention of ohnologs. In fact, gene expression levels (Extended Experimental Procedures) have a negligible total effect on the retention of human ohnologs (TE = 0.003), by contrast to what has been reported for the paramecium [START_REF] Gout | Differential retention of metabolic genes following whole-genome duplication[END_REF]. The total effects of Ka/Ks on ohnolog retention are also lower than the total effects of deleterious mutations, as TEs from deleterious mutations are $2to 3-fold stronger than TEs from Ka/Ks and become >10-fold stronger for genes without SSD and CNV (Extended Results).

In addition, we have performed a complementary systematic study of all these genomics properties using partial correlation analysis, which aims at ''removing'' the effect of a third property (Z) on the standard pair correlations between two variables (X) and (Y). The results detailed in Extended Results and Table S6 are entirely consistent with those obtained through mediation analysis, although the two approaches are not equivalent. Indeed, although mediation effects require partial correlation, partial correlation does not imply mediation, in general (Extended Results).

All in all, these results support the fact that the retention of ohnologs in the human genome is more strongly associated with their ''dangerousness'' (i.e., susceptibility to dominant deleterious mutations) than with their functional importance (''essentiality''), sensitivity to dosage balance, absolute expression levels or sequence conservation (i.e., Ka/Ks).

Model for the Retention of ''Dangerous'' Ohnologs

As demonstrated above, human genes with a documented sensitivity to dominant deleterious mutations have retained statistically more ohnologs from the two WGD events at the [START_REF] Pearl | The Mediation Formula: A guide to the assessment of causal pathways in nonlinear models[END_REF]. See also Extended Experimental Procedures. (C and D) Quantitative Mediation analysis of direct versus indirect effects of deleterious mutations and dosage balance on the retention of human ohnologs using (C) all human genes [START_REF] Conant | Duplicate genes and robustness to transient gene knock-downs in Caenorhabditis elegans[END_REF]506) or (D) all human genes without SSD nor CNV [START_REF] Huminiecki | 2R and remodeling of vertebrate signal transduction engine[END_REF]215). The thickness of the arrows outlines the relative importance of the corresponding direct or indirect effects. These results are consistent with those obtained from partial correlation analysis. See also the main text, Extended Results, and Tables S4,S5, and S6.

onset of jawed vertebrates. This suggests that ohnologs have been retained in vertebrate genomes, not because they initially brought selective advantages following WGD, but because their mutations were more likely detrimental or lethal than nonfunctional, thereby preventing their rapid elimination from the genomes of surviving individuals following WGD transitions, as outlined in the evolutionary model depicted in Figure 4.

For completeness and clarity, Figure 4 examines all possible evolutionary scenarios following either a SSD or a WGD duplication event in the genome of one or a few individuals in an initial population. The first and critical difference between SSD and WGD duplication events occurs at the population genetics level with an obligate speciation following WGD event, owing to the difference in ploidy between pre-and post-WGD individuals. As a result, all individuals in the post-WGD population carry twice as many genes as their pre-WGD relatives, whereas only a few individuals in the post-SSD population carry a single small duplicated region. Figure 4 then outlines the three mutation/selection scenarios focusing on a single gene duplicate in the genomes of post-SSD or post-WGD populations: (A) Beneficial mutations after SSD or WGD are expected to spread and become eventually fixed in the new populations, although the bottleneck in population size following WGD limits in practice the efficacy of adaptation in post-WGD species. (B) Neutral or nearly neutral mutations mainly lead to the random nonfunctionalization of one copy of most redundant gene duplicates and, therefore, to their elimination following both SSD and WGD events. In post-WGD populations, this results in the ''reciprocal gene loss'' of most gene duplicates, which is also known to lead to further speciations in post-WGD species, owing to the interbreeding incompatibility between post-WGD individuals with different ''reciprocal gene loss'' pattern (Lynch and Force, 2000a). Alternatively, neutral or nearly neutral mutations can also result in the eventual retention of both duplicate copies through subfunctionalization [START_REF] Hughes | The evolution of functionally novel proteins after gene duplication[END_REF][START_REF] Lynch | The probability of duplicate gene preservation by subfunctionalization[END_REF], that is, by rendering each duplicate copy unable to perform all the functions of their ancestral gene (see Discussion). (or their descendants) harboring them through purifying selection. However, this typically leads to opposite outcomes in post-SSD and post-WGD populations. In post-SSD populations, dominant deleterious mutations will tend to eliminate SSD duplicates before they have the time to reach fixation (see below). By contrast, in post-WGD populations, where all ohnologs have been initially fixed through WGD-induced speciation, purifying selection will effectively favor the retention of dangerous ohnologs, as all surviving individuals still present (nondeleterious) functional copies of these dangerous genes. Note, in particular, that this somewhat counterintuitive evolutionary model for the retention of ''dangerous'' ohnologs hinges on two unique features:

(1) It requires an autosomal dominance of deleterious mutations, in agreement with our observation, above, that retained ohnologs are more ''dangerous'' than ''essential.'' (2) It relies on the fact that successful WGD events start with a concomitant speciation event, which immediately fixes all ohnolog duplicates in the initial post-WGD population (Figure 4).

Note, also, that the same evolutionary trend is expected for dangerous SSD duplicates that would have the time (t) to become fixed through genetic drift in a population of size N before deleterious mutations can arise at a rate K, i.e., t = 4N < 1/K. This corresponds to a population bottleneck effect with N < 1/(4K) z5,000-10,000 for typical vertebrates.

DISCUSSION

Beyond human and vertebrate genomes, WGD events have now been established in all major eukaryote kingdoms (Se ´mon and [START_REF] Se ´mon | Consequences of genome duplication[END_REF][START_REF] Evlampiev | Modeling protein network evolution under genome duplication and domain shuffling[END_REF]. Unlike SSD events, WGD transitions provide a unique evolutionary mechanism, enabling the simultaneous duplication of entire genetic pathways and multiprotein complexes, followed by long periods of functional divergence and extensive loss of ohnologs [START_REF] Aury | Global trends of wholegenome duplications revealed by the ciliate Paramecium tetraurelia[END_REF]. Moreover, although both WGD and SSD events have expanded the gene repertoires and resulting protein networks [START_REF] Evlampiev | Modeling protein network evolution under genome duplication and domain shuffling[END_REF][START_REF] Evlampiev | Conservation and topology of protein interaction networks under duplication-divergence evolution[END_REF] of eukaryotes, it has become increasingly clear that WGD and SSD events actually lead to the expansion of different gene classes in the course of evolution, [START_REF] Maere | Modeling gene and genome duplications in eukaryotes[END_REF][START_REF] Aury | Global trends of wholegenome duplications revealed by the ciliate Paramecium tetraurelia[END_REF][START_REF] Se ´mon | Consequences of genome duplication[END_REF]Makino and McLysaght, 2010;Huminiecki and Heldin, 2010;and this study).

In this article, we report that WGD have effectively favored the expansion of gene families prone to deleterious mutations in the human genome, such as genes implicated in cancer and genes with autoinhibitory interactions. In particular, we found that the retention of many ohnologs suspected to be dosage balanced is in fact indirectly mediated by their susceptibility to deleterious mutations.

From a broader perspective, a number of studies have now shown that many genomic properties, such as gene essentiality, duplicability, functional ontology, network connectivity, expression level, mutational robustness, divergence rates, etc., all appear to be correlated to some extent. In the light of the present study, we expect that many of these statistically significant correlations mainly result from indirect rather than direct associations, which may even frequently oppose each other. This highlights the need to rely on more advanced inference methods to analyze the multiple, direct, and indirect causes underlying the evolution of specific gene repertoires.

In the present study, we have quantitatively analyzed the direct versus indirect effects of the susceptibility of human genes to deleterious mutation and dosage balance constraints on the retention of ohnologs and proposed a simple evolutionary mechanism to account for the initial retention of ''dangerous'' ohnologs after WGD (Figure 4). On longer timescales, we expect that this initial retention bias of ''dangerous'' ohnologs effectively promote a prolonged genetic drift and, thus, a progressive functional divergence between ohnolog pairs. This eventually favors the subfunctionalization [START_REF] Hughes | The evolution of functionally novel proteins after gene duplication[END_REF][START_REF] Lynch | The probability of duplicate gene preservation by subfunctionalization[END_REF] of ancestral functions between ohnolog pairs, which ultimately warrants their long-term maintenance following WGD events.

Note, however, that this subfunctionalization process requires that the expression of ohnologs is not rapidly suppressed by large-scale deletion or silencing mutations in regulatory regions. As ohnolog pairs are not arranged in tandem, large-scale deletions through unequal crossing-over cannot typically remove entire ohnolog duplicates while preserving the integrity of nearby genes. Furthermore, as the size of promoter or enhancer regions is typically much smaller than UTRs and coding regions, one expects that the rate of transcriptional silencing does not exceed the rates of functional silencing and divergence in UTRs and coding regions. In fact, early estimates [START_REF] Nadeau | Comparable rates of gene loss and functional divergence after genome duplications early in vertebrate evolution[END_REF] showed that gene loss and functional divergence after genome duplications in early vertebrates occurred at comparable rates in gene families including at least two ohnologs. This is also directly evidenced by pseudotetraploid species like the vertebrate Xenopus laevis, which still retains $40% of its initial ohnologs from a 30-million-year-old WGD (Se ´mon and [START_REF] Se ´mon | Preferential subfunctionalization of slowevolving genes after allopolyploidization in Xenopus laevis[END_REF]. All in all, this suggests that ohnologs prone to dominant deleterious mutations have at least a few million years to diverge and become nonredundant genes before they have a chance to be deleted or transcriptionally silenced.

Yet, we found that the retention of these dangerous ohnologs remains intrinsically stochastic by nature as many of them have also been eliminated following WGD events. This presumably occurred through loss-of-function mutations, transcriptional silencing, or large-scale deletion before ohnolog pairs could diverge and become nonredundant genes. More quantitatively, a simple theoretical estimate, derived from the long-term retention statistics of Figure 1, shows that only 6%-10% of the initial ohnolog duplicates have been retained on average at each round of WGD, Figure 5 (see Extended Results for details). By comparison, $23%-30% of the initial ohnologs prone to gain-of-function mutations have been retained on average at each WGD. This implies that genes susceptible to deleterious mutations are two to five times more likely to retain ohnologs on long evolutionary timescales. Moreover, genes combining several factors associated with enhanced susceptibility to autosomal-dominant deleterious mutations are shown to be more than ten times more likely to retain ohnologs than genes lacking gain-of-function mutations (Figure 5), as illustrated on the examples of oncogenes with autoinhibitory folds (Table 1).

In turn, the elimination of ohnologs has been shown to drive further speciation events within post-WGD (sub)populations, due to the emergence of recombination barriers from the accumulation of differences in ohnolog deletion patterns between post-WGD individuals (Lynch and Force, 2000a). The resulting fragmentation of post-WGD subpopulations is then expected to sustain negative selection pressure that favors the retention of the remaining ohnolog pairs prone to deleterious mutations, as outlined in Figure 4. Hence, although most WGDs are unlikely to bring much fitness benefit on short evolutionary timescales (if only due to the population bottlenecks associated with WGDinduced speciations; Figure 4), they provide a unique evolutionary mechanism to experiment virtually unlimited combinations of regulation/deletion patterns from redundant ohnolog genes. Over long timescales (>100-500 MY), such trial and error combinations have visibly led to the evolutionary success and radiation of WGD species.

In summary, we present evidence supporting an evolutionary link between the susceptibility of human genes to dominant deleterious mutations and the documented expansion of these ''dangerous'' gene families by two WGD events at the onset of jawed vertebrates. We propose that deleterious mutations, responsible for many cancers and other severe genetic diseases on the lifespan of human individuals, have also underlain purifying selection over long evolutionary timescales, which effectively favored the retention of vertebrate ohnologs prone to dominant deleterious mutations, as outlined in Figure 4. From a population genetics perspective, we argue that this counterintuitive retention of dangerous ohnologs hinges in fact on WGD-induced speciation events, which are largely credited for the genetic complexity and successful radiation of vertebrate species.

These findings highlight the importance of purifying selection from WGD events on the evolution of vertebrates and, beyond, exemplify the role of nonadaptive forces on the emergence of eukaryote complexity [START_REF] Ferna ´ndez | Non-adaptive origins of interactome complexity[END_REF].

EXPERIMENTAL PROCEDURES WGD Duplicated Genes or ''Ohnologs'' Human ohnolog genes were obtained from (Makino and McLysaght, 2010). Makino and McLysaght compared different vertebrate and six nonvertebrate outgroup genomes to identify ohnologs in the human genome. The final data set consists of 8,653 ohnolog pairs and 7,110 unique ohnologs. We further divided ohnologs into well supported [START_REF] Dickerson | On the origins of Mendelian disease genes in man: the impact of gene duplication[END_REF]963), plausible (894), and more uncertain [START_REF] Makino | Ohnologs in the human genome are dosage balanced and frequently associated with disease[END_REF]253) ohnologs (see Extended Experimental Procedures).

SSD Duplicated Genes

We identified paralogous genes within the human genome from sequence similarity search. We obtained a total of 11,185 SSD genes. In particular, paralogs that were not annotated as ohnologs were taken to be SSD genes (see Extended Experimental Procedures).

Genes with CNV

CNV regions were obtained from Database of Genomic Variants [START_REF] Zhang | Development of bioinformatics resources for display and analysis of copy number and other structural variants in the human genome[END_REF]. A total of 5,709 genes were identified to be CNV genes as their entire coding sequence fell within one of the CNV regions.

Cancer and Disease Genes

We obtained cancer genes from multiple databases, including COSMIC (Forbes et al., 2011) and CancerGenes [START_REF] Higgins | CancerGenes: a gene selection resource for cancer genome projects[END_REF], listed in Table S7. The detailed list of 6,917 cancer genes is given in Table S8 with a hand-curated list of 813 verified or predicted [START_REF] Bozic | Accumulation of driver and passenger mutations during tumor progression[END_REF] oncogenes (see Extended Experimental Procedures). We obtained 2,580 disease genes from the ''Morbidmap'' database of OMIM and hand curated subsets of 440 autosomaldominant and 598 autosomal-recessive disease genes from Blekhman et al. (2008).

Genes with Autoinhibitory Folds

To obtain genes coding for proteins with autoinhibitory folds we searched PubMed with keyword ''autoinhibitory domain'' and retrieved relevant autoinhibitory genes and domains manually. Further gene candidates with autoinhibitory folds were obtained from databases, OMIM, SwissProt, NCBI Gene, and GeneCards using the parsing terms: auto/self-inhibit*. Careful manual curation of this list of gene candidates with the available literature finally yielded a total of 461 genes with autoinhibitory folds (94% of initial candidates).

Essential Genes

Mouse essential genes were obtained from Mouse Genome Informatics database. Essential genes were defined as genes having lethal or infertility phenotypes on loss-of-function or knockout mutations (2,729 genes) (see Extended Experimental Procedures).

Genes in Complexes and Permanent Complexes

Protein complexes were obtained from Human Protein Reference Database (HPRD) [START_REF] Prasad | Human Protein Reference Database-2009 update[END_REF] and CORUM database [START_REF] Ruepp | CORUM: the comprehensive resource of mammalian protein complexes-2009[END_REF]. In addition, a manually curated data set of permanent complexes (239 genes) was obtained from [START_REF] Zanivan | A new computational approach to analyze human protein complexes and predict novel protein interactions[END_REF]. The final data set consists of 3,814 protein complex genes (see Extended Experimental Procedures). Estimates of ohnolog retention rates p s in early vertebrates from the observed fraction f s of ohnologs in the human genome for gene classes, s, with increasing susceptibility to deleterious mutations. The theoretical estimate (red curve) is obtained assuming that the retentions of ohnologs were comparable for each of the two WGD at the onset of vertebrates, and reads

Haploinsufficient and Dominant Negative Genes

P s = 2=f s À 1 À ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi ð2=f s À 1Þ 2 À 1 q
as detailed in the Extended Results and Tables S7 andS8. PLOS Computational Biology recently published an article by Chen, Zhao, van Noort, and Bork [START_REF] Chen | Human Monogenic Disease Genes Have Frequently Functionally Redundant Paralogs[END_REF] reporting that, in contrast to duplicated nondisease genes, human monogenic disease (MD) genes are [START_REF] Chen | Human Monogenic Disease Genes Have Frequently Functionally Redundant Paralogs[END_REF] enriched in duplicates (in agreement with earlier reports [START_REF] Makino | Ohnologs in the human genome are dosage balanced and frequently associated with disease[END_REF][START_REF] Dickerson | On the origins of Mendelian disease genes in man: the impact of gene duplication[END_REF][START_REF] Tinti | Evolution of signal multiplexing by 14-3-3-binding 2R-ohnologue protein families in the vertebrates[END_REF][START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF]) and ( 2) more functionally similar to their closest paralogs based on sequence conservation and expression profile similarity. Chen et al. then proposed that human MD genes frequently have functionally redundant paralogs that can mask the phenotypic effects of deleterious mutations.

We would like to point out here two lines of evidence that appear more relevant to the explanation of this surprising enrichment of human disease genes in duplicates. The first line of evidence indicates that human gene duplicates should be distinguished depending on whether they originate from smallscale duplication (SSD) or from the two rounds of whole genome duplication (WGD) that occurred in early vertebrates some 500 million years ago. In fact, as shown quantitatively below using Chen et al.'s dataset, human MD genes are actually depleted, not enriched, in SSD duplicates, whereas they are clearly enriched in WGD duplicates when compared to nondisease genes. This opposite retention pattern cannot be explained by a selection mechanism independent of the SSD or WGD origin of MD gene duplicates. The second line of evidence concerns the mode of inheritance of human MDs, which provides a more stringent criterion than sequence conservation or coexpression profile to assess the likelihood of functional compensation by paralogs of MD genes. In particular, the recessiveness of a human disease is expected to be a prerequisite for functional compensation by a paralog gene. Indeed, autosomal dominant MDs are unlikely to experience significant functional compensation from a different locus, since even a perfectly functional allele is unable to mask the deleterious phenotypic effects of a dominant allelic mutant on the same heterozygote locus.

We first address the difference between SSD duplicates and WGD duplicates, also called ''ohnologs'' after Susumu Ohno's early ''2R hypothesis'' [START_REF] Ohno | Evolution from fish to mammals by gene duplication[END_REF], which has now been firmly established [START_REF] Putnam | The amphioxus genome and the evolution of the chordate karyotype[END_REF]. The importance of distinguishing between SSD and WGD duplicates in the human genome has already been reported in a number of papers [START_REF] Makino | Ohnologs in the human genome are dosage balanced and frequently associated with disease[END_REF][START_REF] Dickerson | On the origins of Mendelian disease genes in man: the impact of gene duplication[END_REF][START_REF] Tinti | Evolution of signal multiplexing by 14-3-3-binding 2R-ohnologue protein families in the vertebrates[END_REF][START_REF] Huminiecki | 2R and remodeling of vertebrate signal transduction engine[END_REF], including our own [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF][START_REF] Malaguti | On the retention of gene duplicates prone to dominant deleterious mutations[END_REF]. As shown in Figure 1A, human genes tend to partition into three main gene categories with respect to duplicates: those with WGD but no SSD duplicates (about 28%), those with SSD but no WGD duplicates (about 41%), and singletons without WGD or SSD duplicates (about 24%), while human genes with both WGD and SSD duplicates are relatively rare (about 7%). Gene families enriched either in WGD or SSD duplicates also correspond to distinct functional classes [START_REF] Makino | Ohnologs in the human genome are dosage balanced and frequently associated with disease[END_REF][START_REF] Huminiecki | 2R and remodeling of vertebrate signal transduction engine[END_REF], with WGD genes frequently involved in signaling, regulation, and development, whereas SSD genes are typically implicated in different functions such as antigen processing, immune response, and metabolism.

In addition, human disease genes have been shown to be significantly enriched in WGD duplicates, while they are rather depleted in SSD duplicates [START_REF] Makino | Ohnologs in the human genome are dosage balanced and frequently associated with disease[END_REF][START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF][START_REF] Huminiecki | 2R and remodeling of vertebrate signal transduction engine[END_REF][START_REF] Malaguti | On the retention of gene duplicates prone to dominant deleterious mutations[END_REF]. This could not be seen with Chen et al.'s dataset, which lumps together all gene duplicates irrespective of their WGD or SSD origin. In fact, using the same monogenic disease (MD) dataset, we could readily extend these earlier results, as depicted in Figure 1B. MD 1B), these duplicates are primarily enriched in ohnologs and not SSD copies, as compared to the relative WGD and SSD content of the entire human genome (Figure 1A, Dataset S1).

To explain the global enrichment in MD gene duplicates, Chen et al. noticed that coexpressions between MDs and their closest paralogs are in general higher than that of nondisease genes (p = 0.00298, Figure 2B in [START_REF] Chen | Human Monogenic Disease Genes Have Frequently Functionally Redundant Paralogs[END_REF]), which they interpret as evidence that ''functional compensation by duplication of genes masks the phenotypic effects of deleterious mutations and reduces the probability of purging the defective genes from the human population.'' In particular, the retention of MD gene duplicates should be favored by the higher functional redundancy of recent, less-diverged duplicates. However, investigating the age of SSD duplicates from MD genes suggests rather the opposite, as MD genes tend to have fewer recent SSD than old SSD duplicates, as compared to nondisease (ND) genes (Figures 1A andB). In particular, focusing on genes with SSD but no ohnolog, we found that 9.2% [respectively 17%] of MD genes have SSD that are more recent [respectively ancient] than the two rounds of whole- Funding: PPS acknowledges a PhD fellowship from Erasmus Mundus (Universite Ṕierre et Marie Curie) and La Ligue Contre Le Cancer; SA and GM acknowledge a PhD fellowship from Ministry of Higher Education and Research. HI acknowledges funding from Foundation Pierre-Gilles de Gennes. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing Interests: The authors have declared that no competing interests exist.

genome duplication, while the overall genome exhibits 17.3% [respectively 15.1%] instead (p = 4.5610 234 ; FE test). This suggests that the functional compensation, which can occur between functionally redundant duplicates, leads to a depletion (not an enrichment) of MD genes with recent SSD, in agreement with an earlier report [START_REF] Hsiao | Role of duplicate genes in robustness against deleterious human mutations[END_REF].

In addition, we note that, while recent gene duplicates might be able to mask the phenotypic effect of recessive (e.g., loss-offunction) mutations, dominant (e.g., gain-of-function or dominant negative) mutations should typically lead to deleterious phenotypic effects regardless of the presence of any functionally redundant paralog at a different locus on the human genome.

In order to assess the extent of possible functional compensation on the retention of MD gene duplicates, we have thus investigated the mode of inheritance of human MDs. To this end, we retrieved the available information on the dominance and recessiveness of MDs from Online Mendelian Inheritance in Man (OMIM) [11] and Blekhman et al. [12]. Manual curation yielded 620 autosomal dominant and 838 autosomal recessive MD genes after excluding sex-linked genes and MD genes documented as both dominant and recessive (Dataset S1).

Using Chen et al.'s dataset and analysis, we then found that autosomal recessive MD gene duplicates (with possible functional compensation) do not exhibit significantly more correlated expression profiles than ND genes (p = 0.426, Wilcoxon Rank . Distributions of WGD, SSD, and singletons for human orthologs of mouse genes (A) tested for essentiality in mouse [START_REF] Eppig | The Mouse Genome Database (MGD): comprehensive resource for genetics and genomics of the laboratory mouse[END_REF], (B) found to be essential in mouse, and (C and D) after removing dominant disease genes, oncogenes, and genes with dominant negative mutations or autoinhibitory folds [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF]. (*) corresponds to small deviations (10 23 ,p,0.05, FE test) from the references in (A). Note that human orthologs of essential genes in mouse do not show any significant deviations in WGD, SSD, or singleton contents (p.0.05, FE test) once dominant disease genes, oncogenes, and genes with dominant negative mutations or autoinhibitory folds have been removed. Yet, taking into account the age of SSD duplicates reveals a relative lack of recent SSD genes in essential genes (see text). doi:10.1371/journal.pcbi.1003754.g002 Sum Test, as compared to p = 0.00298 for all MD genes in Figure 2B in [START_REF] Chen | Human Monogenic Disease Genes Have Frequently Functionally Redundant Paralogs[END_REF]), whereas autosomal dominant MD gene duplicates (with unlikely functional compensation) in fact exhibit significant expression profile correlations (p = 0.00028).

Moreover, looking for duplication biases of recessive versus dominant MDs confirmed that recessive MDs, which could in principle provide functional compensation, have not retained significantly more duplicates. Indeed, Figure 1C shows that recessive MDs do not present any biased retention of ohnologs, 28.3% versus 27.7% (p = 0.79; FE test); SSD duplicates, 38.8% versus 41.6% (p = 0.31; FE test); or singletons, 25.3% versus 23.7% (p = 0.42; FE test), as compared to their respective prevalence in the entire human genome (Figure 1A). These observations clearly show that the maintenance of recessive MD genes is largely independent of their WGD, SSD, or singleton status, suggesting limited effects of functional compensation by paralogs on the retention of gene duplicates associated to recessive MDs in human. By contrast, we observed (Figure 1D) that dominant MDs exhibit a strong enrichment in ohnologs, 45% versus 27.7% (p = 1.8610 210 ; FE test), with concomitant depletions in both SSD, 30.1% versus 41.6% (p = 0.0001; FE test), and singletons, 13.2% versus 23.7% (p = 1.59610 27 ; FE test). The same trend is observed for haploinsufficient and dominant negative genes [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF]. This is unlikely to result from a functional compensation by paralogs because of the molecular genetics of dominance, as discussed above.

Finally, we investigated the enrichment in WGD and SSD duplicates of essential genes for which functional compensation could in principle be advantageous owing to the lethality of their double mutants. However, we found that human orthologs of mouse genes, reported as being ''essential'' genes from large-scale null mutant studies in mouse [START_REF] Eppig | The Mouse Genome Database (MGD): comprehensive resource for genetics and genomics of the laboratory mouse[END_REF], are only slightly enriched in ohnologs, 48.9% versus 44.6% (p = 0.02, FE test), and hardly depleted in SSD, 30.9% versus 33% (p = 0.14, FE test), in which 44.6% and 33% are, respectively, the global proportions of ohnologs and SSD among the 6,436 genes tested for null mutation in mouse (Figures 2A and2B). In fact, these small deviations, consistent with earlier findings [START_REF] Makino | The complex relationship of gene duplication and essentiality[END_REF], even become nonsignificant once genes with dominant allelic mutants are removed from the list of 6,436 genes tested for essentiality in mouse (Figures 2C and 2D), i.e., 41.6% versus 37.8% for ohnologs (p = 0.1, FE test) and 34.1% versus 36.2% for SSD (p = 0.34, FE test), in which 37.8% and 36.2% are, respectively, the global proportions of ohnologs and SSD among the 3,493 genes tested for null mutation in mouse after removing dominant disease genes, oncogenes, and genes with dominant negative mutations or autoinhibitory folds (Figures 2C and2D) [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF]. Hence, we could not find any significant enrichment in duplicates in support of possible functional compensation for essential genes, in broad agreement with early reports [START_REF] Liao | Mouse duplicate genes are as essential as singletons[END_REF][START_REF] Liang | Gene essentiality, gene duplicability and protein connectivity in human and mouse[END_REF]. Moreover, taking into account the age distribution of SSD duplicates (including corrections for the visible age bias of genes tested for essentiality in mouse [Figures 1A and2A]) [START_REF] Su | Predicting the proportion of essential genes in mouse duplicates based on biased mouse knockout genes[END_REF][START_REF] Chen | Younger genes are less likely to be essential than older genes, and duplicates are less likely to be essential than singletons of the same age[END_REF], we actually found a relative lack of recent SSD of essential genes (Figure 2B and 2D) as observed for MD genes (Figure 1B-1D). This suggests evidence against functional redundancy in vertebrate essential genes, in agreement with an earlier report [START_REF] Su | Predicting the proportion of essential genes in mouse duplicates based on biased mouse knockout genes[END_REF] and similar observations in yeast [START_REF] Gu | Role of duplicate genes in genetic robustness against null mutations[END_REF] and nematodes [START_REF] Conant | Duplicate genes and robustness to transient gene knock-downs in Caenorhabditis elegans[END_REF].

So, what could be the evolutionary mechanism behind the enhanced retention of WGD duplicates and relative depletion of SSD duplicates and singletons associated to MDs in humans (Figure 1B)? In other works [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF][START_REF] Malaguti | On the retention of gene duplicates prone to dominant deleterious mutations[END_REF], we proposed a population genetics model based on the observation that a major difference between SSD and WGD scenarios concerns the timing of fixation of gene duplicates. It is well-known that the SSD scenario starts with a gene duplication in the genome of a single individual, which subsequently needs to spread through the entire population to reach fixation. By contrast, the WGD scenario entails an initial fixation of duplicated gene pairs in the genome of all individuals in the small population, arising through WGD. This is because WGD typically induces a speciation event due to the ploidy incompatibility of the post-WGD individuals with the rest of the pre-WGD population. This population genetics model [START_REF] Malaguti | On the retention of gene duplicates prone to dominant deleterious mutations[END_REF] for the fixation of SSD versus WGD duplicates then predicts that the enhanced retention of ''dangerous'' ohnologs prone to dominant deleterious mutations (as depicted in Figure 1D) is a direct consequence of purifying selection in post-WGD population, as most surviving individuals retain (nondeleterious) functional copies of their ohnologs that are prone to dominant deleterious mutations. By contrast, ohnologs prone to recessive deleterious mutations are more readily eliminated through loss-of-function mutations and are not expected to exhibit significant ohnolog retention bias (in agreement with Figure 1C). As for SSD duplicates, they are expected to be retained either from adaptive selection in large populations (N.10 5 ) or from purifying selection in small populations (N,10 4 ), in which SSD duplicates typically reach fixation by drift before their mutations actually occur, hence resembling the WGD scenario with an initial fixation of ohnologs through speciation in that case. This leads in principle to a complex retention pattern of SSD duplicates across evolutionary ages, in particular around WGD-induced population bottlenecks. Yet, overall it appears that genes with recent SSD duplicates are less likely to be MD genes than genes with WGD-old or older SSD duplicates (Figure 1).

All in all, we found that MD genes have preferentially retained WGD rather than SSD duplicates, as compared to nondisease genes. Yet, only dominant MD genes exhibit a clear enrichment in WGD duplicates, while the retention of duplicates of recessive MDs or essential genes, which might in principle experience functional compensation from paralogs, is in fact largely independent of their WGD, SSD, or singleton status. These results cannot be explained by the functional compensation hypothesis proposed in Chen et al. [START_REF] Chen | Human Monogenic Disease Genes Have Frequently Functionally Redundant Paralogs[END_REF]. They are, however, consistent with a population genetics model taking into account the initial fixation of ohnologs through WGD-induced speciation and the ensuing purifying selection in post-WGD populations [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF][START_REF] Malaguti | On the retention of gene duplicates prone to dominant deleterious mutations[END_REF].

Materials and Methods

We obtained 20,415 protein coding genes in the human genome from Ensembl version 70 (Dataset S1). Ohnologs (7,075 genes) were obtained from [START_REF] Tinti | Evolution of signal multiplexing by 14-3-3-binding 2R-ohnologue protein families in the vertebrates[END_REF], and SSDs (9,916 genes) were obtained by running an all-against-all BLASTp using the human proteins (see [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF] for details). The genes which could not be classified as ohnologs or SSDs were taken to be the singleton genes (4,846 genes). The duplication timing of SSD genes was obtained from Ensembl compara [START_REF] Vilella | Ensembl Compara GeneTrees: Complete, duplication-aware phylogenetic trees in vertebrates[END_REF] using BioMart.

MD genes were taken from Chen et al. [START_REF] Chen | Human Monogenic Disease Genes Have Frequently Functionally Redundant Paralogs[END_REF]. We could map 5,030 of 5,134 MD genes on our dataset using BioMart. Inheritance status of the MD genes were obtained either from the inheritance section from OMIM entries [11] or from Blekhman et al. [12]. After careful manual curation, we could obtain 1,458 MD genes in which the inheritance pattern was unambiguously described as either autosomal dominant (620 genes) or autosomal recessive (838 genes).

Expression profile correlation between autosomal recessive genes and ND genes, and autosomal dominant genes and ND genes was performed using the R scripts provided by Chen et al. [START_REF] Chen | Human Monogenic Disease Genes Have Frequently Functionally Redundant Paralogs[END_REF].

We obtained 6,436 mouse genes tested for null/knock-out mutations from the Mouse Genome Database (MGD) [START_REF] Eppig | The Mouse Genome Database (MGD): comprehensive resource for genetics and genomics of the laboratory mouse[END_REF], as described in [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF]. 2,938 of these 6,436 genes had lethal or infertile phenotypes and were classified as ''essential.'' Human one-to-one orthologs of these genes were obtained using Ensembl BioMart. We also investigated the enrichment of essential genes in duplicates after removing dominant disease genes. To this end we considered multiple classes of genes susceptible to dominant mutations, including 620 dominant disease genes (from this report), 5,996 oncogenes [START_REF] Forbes | COSMIC: mining complete cancer genomes in the Catalogue of Somatic Mutations in Cancer[END_REF], 566 dominant negative genes, and 461 genes having autoinhibitory folds [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF].

Chapter 2

Graphical Notation and Terminology

This chapter gives the general terminology and concepts on graphical models and causality that will be used throughout this dissertation.

Directed Acyclic Graphs

Graphical model terminology

A graphical model G = (V, E) is a representation of the relationships between random variables represented by the set of vertices V = {X 1 , X 2 , ..., X q }. Each edge in E denotes a relation between a pair of variables. If all the pairs of variables are connected by an edge, the graph is said to be complete. The edges can either be directed (→) or undirected ( ). When all edges are directed, the graph is also said directed. Graphical models that include both directed and undirected edges are named partially directed. When replacing all directed edges of a (partially) directed graph, one obtains a fully undirected network called skeleton.

Each vertex X i can be assigned an adjacency set, adj G (X i ), corresponding to the set of vertices that are connected by an edge to X i in G. If there exists a directed relation between X i and X j , such that X j → X i , then X j is called a parent of X i . A chain of connected vertices constitutes a path. If all edges included in a path are oriented in the same direction (e.g. ←←), the path is said to be directed.

Whenever the path is directed from X i to X j and there exists a directed edge X j → X i , then the path forms a directed cycle. If a (partially) directed graphical model does not contain any directed cycle, it is called a (Partially) Directed Acyclic Graph ((P )DAG).

In (P)DAG, some triples X i , X k , X j can be unshielded, meaning that only two pairs of nodes are adjacent, e.g. X The three possible non v-structures of the unshielded triples, Xi, X k , Xj X i X j , entail the same conditional independence, i.e. Xi ⊥ ⊥ Xj|X k .

i X k and X k X j while X i X j . If an unshielded triple, X i , X k , X j X i X j , is oriented as X i → X k ← X j , it
As detailed in the following sections, the three possible non-v-structures of an unshielded triple X i , X k , X j X i X j entail the same conditional independence information (i.e. X i ⊥ ⊥ X j |X k ), and thus cannot be differentiated from one another, based solely on observational data.

Interpretation of a DAG

The set of conditional independence relationships among the variables V, entailed by a DAG G, can be deduced by applying a graphical criterion called the d-separation [START_REF] Pearl | Causality: Models, Reasoning, and Inference[END_REF] (see section 2.1.2.1). If two sets of vertices (X i , X j )

are not adjacent in a DAG G, they are d-separated by a subset S of the remaining variables in G. It follows that X i and X j are independent conditional on S, i.e., X i ⊥ ⊥ X j |S, in any distribution P over V that factorizes 1 according to G. In fact, when assuming that the DAG G has the local Markov property 2 , we assume that the conditional independence relationships obtained via the d-separation criterion at least holds in the distribution P over V. Conversely, if the conditional independencies in P correspond exactly 3 to the ones that can be read off from G via the d-separation criterion, P is said to be faithful to G or stable.

1 P factorizes according to a DAG G = (V, E) if the joint density distribution of V can be written as p(X1, X2, ..., Xq) = q i=1 p(Xi|pa(G, Xi)), where pa(G, Xi) are the parents of Xi according to G. 2 Each vertex in the DAG G is independent of its non-descendants conditional on its parents. 3 i.e. no additional independences can be found from the population

The d-separation criterion

This d-separation criterion is tied to the notion of blocking a path in a causal graphical model, i.e. stopping the flow of information between two related variables. In particular, if three nodes (X i , X k , X j ) are related through a non-v-structure (either as a chain X i → X k → X j or a fork X i ← X k → X j ), the two extreme nodes are marginally dependent. However, when one knows the value of X k , learning something about X i has no effect on the probability of X j . Thus, once the value of X k is known, X i and X j become independent. In other words, X k blocks the path between X i and X j .

On the other hand, if (X i , X k , X j ) are related through a v-structure (a collider X i → X k ← X j ), the two extreme variables are, a priori, marginally independent.

Yet, X i and X j become dependent conditioned on the value of X k . As an example, we could consider that X i represents the possibility of an earthquake, X j denotes the possibility of being burgled and X k stands for the possible activation of an house alarm (due to an earthquake or a burglary). A priori, no relation exists between the occurrence of an earthquake (X i ) and the fact of being robbed (X j ) (assuming that the integrity of the house is preserved). However, let's suppose that the activation of the house alarm is acknowledged (i.e., the value of X k is known). If the occurrence of an earthquake is then ascertained, one can draw the conclusion that the house is (likely) safe and, rather than a burglar, the earthquake has induced the activation of the alarm. This effect is also known as the explaining away effect4 .

A definition of the d-separation can be taken from [Pearl, 2000, Definition 1.2.3],

Definition 2.1.1. d-Separation A path p is said to be d-separated (or blocked ) by a set of nodes Z if and only if 1. p contains a chain i → m → j or a fork i ← m → j such that the middle node m is in Z, or 2.
p contains an inverted fork (or collider ) i → m ← j such that the middle node m is not in Z and such that no descendant of m is in Z.

A set Z is said to d-separate X from Y if and only if Z blocks every path from a node in X to a node in Y.

Markov equivalence

Several causal DAGs can entail the same conditional independence relationships, as introduced in section 2.1.1 for the three possible non-v-structures of an unshielded triple (Fig. 2.1). These DAGs are said to be Markov equivalent and the class they populate is called a Markov equivalence class. More specifically, two DAGs are equivalent if and only if they have the same skeleton and the same v-structures [START_REF] Verma | Equivalence and synthesis of causal models[END_REF].

A Markov equivalence class can be uniquely represented by a Completed Partially Directed Acyclic Graph (CPDAG) [START_REF] Andersson | A characterization of markov equivalence classes for acyclic digraphs[END_REF][START_REF] Chickering | Learning equivalence classes of bayesian-network structures[END_REF] that has the following properties:

Ancestral Graphs

As introduced in section 2.1, a CPDAG can be learned from the conditional independence information for systems satisfying the causal sufficiency assumption and the faithfulness assumption. Yet, in practice, the causal sufficiency assumption is rarely satisfied and a CPDAG may not properly represent the underlying causal structure among the observed variables, unless possible hidden common causes, or latent variables, are explicitly considered. However, explicitly introducing all possible latent variables, without any constraints on the network topology or on the number of hidden common causes, creates an infinite search space of DAGs [Zhang, 2008a]. To circumvent this intractable problem, the class of ancestral graph models has been introduced [START_REF] Richardson | Ancestral graph markov models[END_REF]. As detailed below, this class of graphical models preserves the ancestral relationships encoded in the true underlying DAG, despite the presence of latent or selection variables.

The section 2.2.1 briefly describes common complications due to unobserved variables, while the section 2.2.2 gives the graphical terminology of ancestral graphs.

Complications from latent and selection variables

Two impediments can lead to the inference of spurious relationships when reconstructing a causal structure: the hidden (or latent) variables and the selection variables. The first ones correspond to unmeasured variables that contribute to observed associations. The second ones correspond to unobserved variables that characterize the subpopulation related to the samples drawn from the population of interest. As exemplified in section 2.2.1.1 by the variable Sel, the selection bias corresponds to a by design issue. Thus, any observed independence or association should always be regarded as conditional on the selection variables5 .

Spurious correlations

The following example, cited in [START_REF] Richardson | Learning in Graphical Models, chapter Chain Graphs and Symmetric Associations[END_REF] and attributed to Chris Meek, describes how the presence of latent confounders and selection variables can induce 'spurious correlations' among the observed variables.

Randomized trial of an ineffective drug with unpleasant side-effects

'The graph 2.2a represents a randomized trial of an ineffective drug with unpleasant side-effects. Patients are randomly assigned a treatment or control group (A). Those in the treatment group suffer from unpleasant side-effects (Ef ), the severity of which is influenced by the patient's general level of health (H ), with sicker patients suffering worse side-effects. Those patients who suffer sufficiently severe side-effects are likely to drop out of the study. The selection variable (Sel ) records whether or not a patient remains in the study, thus for all those remaining in the study Sel = StayIn. Since unhealthy patients who are taking the drug are more likely to drop out, those patients in the treatment group who remain in the study tend to be healthier than those in the control group. Finally health status (H ) influences how rapidly the patient recovers (R)'.

In Fig. 2.2, H is a hidden common cause responsible for the spurious correlation between Ef and R (blue edge). The selection variable Sel implies a correlation between A and R as the relationship between these two observed variables is in fact considered condition on Sel (Sel = StayIn).

Spurious causal effects

A related issue is the discovery of 'spurious causal effects'. In the DAG of Fig. 2.3, X i ⊥ ⊥ X j is the only independence among the observed variables, leading to believe that the underlying causal structure over the observed variables O is the v-structure X i → X k ← X j . This CPDAG suggests that X i and X j are causes of X k , while in fact no directed path exists between them. As detailed in section 2.2.2, the class of ancestral graphs would give a better representation of the relationships among the observed variables with the graphical model X i ←→ X k ←→ X j , which is a maximal ancestral graph (MAG) that uniquely represents the underlying DAG of the Fig. 2.3 over the observed variables while taking into account the hidden variables. In this representation, the '>' mark should be read as 'not an ancestor of'. Thus, the '>' at X i or X j indicate that X i or X j are not the cause of X k or of a selection variable. Similarly, the '>' around X k indicates that X k is not the cause of X i , X j or of a selection variable.

Graphical model terminology

The following sections give further details on a particular subclass of the class of mixed graphs, namely the class of ancestral graphs, introduced in section 2.2.1.2.

A mixed graph has possibly two kinds of endpoints, arrowhead (>) and tail (-), and can thus contain three types of edge, directed (←), bi-directed (←→) and undirected ( ). The symbol * is used to represent any of the endpoint marks. In a mixed graph, two vertices X i and X j linked by any kind of edge are adjacent;

X i is a parent of X j if X j ← X i ; X i is a spouse of X j if X j ←→ X i and X i is a neighbour of X j if X j X i . A chain of adjacent vertices is a path. A path from a vertex X 0 to the vertex X p is said directed if ∀ 0 ≤ i < p, X i is a parent of X i+1 . If X p is a parent of X 0 , the path forms a directed cycle. If the directed cycle contains a bi-directed edge, it is an almost directed cycle. X i is an ancestor of X j , an G (X j ), if there is a directed path from X i to X j . X j is then called a descendant of X i , de G (X i ).
As will be seen in section 2.2.3.1, if we assume that the data have been generated by a DAG, and if we assume further the presence of latent and selection variables, then one can uniquely represent the conditional independencies and the causal relations entailed by the underlying DAG with a maximal ancestral graph (MAG) over the observed variables [Zhang, 2008a[Zhang, , page 1874]].

The definition 2.2.1 gives the conditions for a mixed graph to be an ancestral graph.

Definition 2.2.1. Ancestral graph

A mixed graph G is an ancestral graph if:

1. G has no directed cycle.

2. G has no almost directed cycle.

3. For any undirected edge X i X j , X i and X j have no parents or spouses.

The first condition can directly be related to the obligate absence of directed cycle in a DAG. The second condition, coupled with the first one, emphasizes the broader interpretation of the arrowhead mark in an ancestral graph, i.e. the arrowhead represents a non-cause. For instance, in the ancestral graph X i → X j , the arrowhead at X j indicates that X j 'is not the cause' of X i . By contrast, that tail at X i gives a 'causal' information as it represents the fact that X i 'is the cause' of X j or of some selection variable. Hence, the tail is less informative than the arrowhead in the presence of selection bias. The third condition allows a proper representation of the selection effect while simplifying the fitting of ancestral graphs [START_REF] Richardson | Ancestral graph markov models[END_REF]. In a nutshell, an ancestral graph preserves, among the observed variables, the ancestral relationships encoded in the underlying DAG.

Interpretation of a MAG

As introduced earlier, when assuming the presence of hidden variables, a maximal ancestral graph (MAG) can uniquely entail a set of conditional independencies over the observed variables that can be deduced by applying the m-separation graphical criterion which generalizes the d-separation criterion defined in section 2.1 for DAGs.

The m-separation criterion

The m-separation criterion can be defined as follows:

Definition 2.2.

m-Separation

A path p in an ancestral graph is said to be m-separated (or blocked ) by a set of nodes Z if and only if 1. p contains a subpath X i , X j , X k such that the middle vertex X j is a noncollider on this path and

X j ∈ Z, or 2. p contains a collider 6 X i * → X j ← * X k such that X j / ∈ Z and no descendant of X j is in Z. A set Z is said to m-separate X from Y if and only if Z blocks every path from a node in X to a node in Y. If all pairs of vertices (X i , X k ) ∈ X × Y are m-separated
by Z, the sets of vertices X and Y are said m-separated by Z.

Similarly as for DAGs, the Markov property states that if X i and X j are mseparated by X k , then X i and X j are probabilistically independent conditional on X k , and thus, X i X i in G. However, by contrast with the DAGs, the reverse is not always true for an ancestral graph. In fact, unless an ancestral graph has the maximality property, it misses the so-called pairwise Markov property 7 , necessary to infer the skeleton. [START_REF] Richardson | Ancestral graph markov models[END_REF], such that:

∀ disjoint sets of variables X, Y, Z, X, Y, Z ⊆ O, X ⊥ ⊥ Y|Z ∪ S in G ⇐⇒ X ⊥ ⊥ Y|Z in M G .
Such MAG M G encodes the following causal information:

• X → Y, i.e. X ∈ an M G (Y ∪ S) and Y / ∈ an M G (X ∪ S)
X is a cause of Y or of some selection variable (tail at X), but Y is not a cause of X or of any selection variable (arrowhead at Y)

• X ↔ Y, i.e. X / ∈ an M G (Y ∪ S) and Y / ∈ an M G (X ∪ S)
X is not a cause of Y or of any selection variable (arrowhead at X), and Y is not a cause of X or of any selection variable (arrowhead at Y), i.e. there is a common cause to X and Y

• X -Y, i.e. X ∈ an M G (Y ∪ S) and Y ∈ an M G (X ∪ S)
X is a cause of Y or of some selection variable (tail at X), and Y is a cause of X or of some selection variable (tail at Y), i.e. because of the acyclicity assumption, X is a cause of some selection variable, and Y is a cause of some selection variable

MAGs M G 1 , M G 2 are Markov equivalent if ∀ disjoint sets X, Y, Z ⊆ O, X and Y are m-separated by Z in M G 1 ⇐⇒ X and Y are m-separated by Z in M G 2 .
Hence, all the members of the Markov equivalence class of a given MAG have the same adjacencies and the same invariant endpoints (arrowhead (>) or tail (-)). Indeed, within the Markov equivalent class of MAGs, one or both endpoints of some edges may be different depending on the MAG member. The mark '•'

indicates such variant endpoints.

The invariant endpoints and the edges learned from the conditional independen- Let M be a mixed graph over O. M is said to be a PAG that represents G if and only if, for any distribution P over V that is faithful to G, the following four conditions hold:

(i) X i X j in M ⇒ ∃ Y ⊆ O \ {X i , X j } such that X i ⊥ ⊥ X j |Y ∪ S in P; (ii) X i X j in M ⇒ ∀ Y ⊆ O \ {X i , X j }, X i ⊥ ⊥ X j |Y ∪ S in P; (iii) X i •→ X j in M ⇒ then X j / ∈ an M (X i ∪ S) (iv) X i • X j in M ⇒ then X j ∈ an M (X i ∪ S)
As an example, the graphical model indicates that X k is not the cause of X i , X j or a selection variable, and the '•' mark at X i or X j indicates uncertainty about the possibility of X i and X j to be a cause of X k .

X i •→ X k ←•X j is

Finding adjacencies in a PAG

As introduced in the section 2.2.1.1, the hidden variables prevent from inferring conditional independencies among the observed variables. The example of Fig. 2.4, taken from [START_REF] Spirtes | An algorithm for fast recovery of sparse causal graphs[END_REF], illustrates a spurious causal dependence due to the presence of latent variables. The latent variable L prevents from inferring the independence between X i and X j conditional on X k over the observed variables O, and hence from the removal of the edge between X i and X j , although

X i has no direct influence on X j in the DAG G over V = O ∪ L ∪ S.
Yet, following the definition of a PAG M (see definition 2.2.5), one can still decide for the adjacency between two observed variables when accounting for hidden variables, as:

X i / ∈ an M (X j ∪ S)
and,

X i ⊥ ⊥ X j |Y ∪ S        ⇒ X i ⊥ ⊥ X j |Y ∪ S with, -Y ⊆ O \ {X i , X j } -Y ⊂ D-SEP{X i , X j } or ⊂ D-SEP{X j , X i }
Hence, to decide whether X i and X j are adjacent in the PAG M, it is sufficient to find a separation set Y from a given D-SEP of {X i , X j } rather than trying all possible subsets in Y ⊆ O \ {X i , X j }. However, the D-SEP sets -which definition is closely related to the notion of inducing path -cannot be deduced from the observed conditional independencies [START_REF] Spirtes | An anytime algorithm for causal inference[END_REF]Glymour, 1991, Spirtes et al., 1999]. Hence, a superset containing the D-SEP, called Possible-D-SEP, has been proposed by [START_REF] Spirtes | Causation, Prediction, and Search[END_REF].

A Possible-D-SEP set is defined over the observed variables O as, Definition 2.2.6. Possible-D-SEP Let the distribution P over V = O∪L∪S be faithful to a DAG G. Let C be a PAG that represents G. A vertex X k belongs to Possible-D-SEP(X i , X j ), pds C (X i , X j ), if and only if there is a path π between X i and X j in C such that for every subpath

X m , X l , X h of π, X l is a collider on the subpath in C or X m , X l , X h is a triangle in C.
As detailed in the section 3. Hybrid approaches, based on a combination of these two types of methods are also discussed in the following chapters, as well as our new hybrid causal discovery method, namely 3off2, detailed in Chapter 6.

Chapter 3

Constraint-based Methods

Constraint-based approaches, such as the PC ( [START_REF] Spirtes | An anytime algorithm for causal inference[END_REF]Glymour, 1991, Spirtes et al., 2000]) and IC ( [START_REF] Pearl | A theory of inferred causation[END_REF]) algorithms, learn causal graphical models from observational data by searching for conditional independencies among variables. When assuming that the data have been generated by a DAG model, these algorithms return a Complete Partially Directed Acyclic Graph (CPDAG) that represents the Markov equivalent class of the underlying causal structure under the faithfulness1 and causal sufficiency 2 assumptions [Pearl andVerma, 1991, Spirtes et al., 2000]. To ascertain conditional independences between variables, these methods usually rely on statistical tests that depend on a significance level α. These approaches based on the identification of structural constraints typically run in polynomial time on sparse underlying graphs and are computationally feasible for large graphical models, up to thousands of variables [START_REF] Kalisch | Estimating high-dimensional directed acyclic graphs with the pc-algorithm[END_REF]. This chapter gives an overview of the constraint-based approach principles.

The novel approach and corresponding 3off2 algorithm, that will be detailed in Chapter 6, is directly inspired from the constraint-based methods. However, it relies on a quantitative information theoretic framework that allows for a robust identification of conditional independencies.

The PC algorithm

To reconstruct the underlying causal graphical model G = (O, E), where O is the set of observed variables and E the set of edges, constraint-based methods, and in particular the widely used PC ( [START_REF] Spirtes | An anytime algorithm for causal inference[END_REF]Glymour, 1991, Spirtes et al., 2000]) algorithm, typically proceed in three steps:

1) learning unnecessary edges and separation sets to obtain an undirected skeleton (Algorithm 1, lines [7 -17])

2) orienting unshielded triples as v-structures if their middle node is not in the separation set (Algorithm 1, rule R 0 , lines [19 -21])

3) propagating as many orientations as possible following propagation rules (R 1-3 ), which prevent the orientation of additional v-structures (R 3 ) and

directed cycles (R 2-3 ) (Algorithm 1, lines [23 -27])
The following sections give further details on each of these steps.

Learning the skeleton and sepsets

In the first step of the PC algorithm (Algorithm 1, (lines [7 -17])), the complete undirected graphical model taken as input is progressively thinned out based on the discovery of conditional independence relationships.

The efficiency of the PC algorithm lies in the organization of the conditional independence queries, Indep(X; Y |{U i }) (Algorithm 1, (line [START_REF] Hsiao | Role of duplicate genes in robustness against deleterious human mutations[END_REF])). Instead of checking all the conditional independencies, as the SGS algorithm does [START_REF] Spirtes | Causation, Prediction, and Search[END_REF], the PC algorithm is searching for the separation sets among the remaining adjacent vertices during the skeleton thinning out process, which can lead to a polynomial time execution in graphs of finite degree. First, starting with an empty separation set = 0, all pairs of variables, (X, Y ), are tested for marginal independence. Whenever such independence is found, X ⊥ ⊥ Y , the corresponding edge is removed, X Y , and the empty set is stored as separation set, Sep XY = Sep Y X = ∅. The value of is then incremented, and each remaining edge having vertices with an adjacency set of size at least is tested for conditional independence based on subset

{U i } ⊆ adj G (X) \ {Y } (or {U i } ⊆ adj G (Y ) \ {X}).
The PC algorithm iterates until all adjacency sets are smaller than . This first step returns the skeleton of the underlying graphical model and a separation set (possibly empty) for each missing edge.

As will be shown later, a major shortcoming of the PC algorithm is that it is not robust to sampling noise in finite dataset, which leads to the accumulation of errors during the edges pruning procedure.

By contrast, as will be shown in Chapter 6, the 3off2 learning method uncovers the conditional independencies among the observed variables following a robust iterative approach in order to build the skeleton of the underlying graphical model (section 6.3). In a nutshell, the 3off2 process iteratively builds the separation sets guided by a score based on 2-point and 3-point information statistics (Eq. 6.32) until no variable belonging to the separation sets can be found.

Orientating the skeleton

Once the skeleton and the separation sets are known, the v-structures can be identified (Algorithm 1, lines [19 -21]). As introduced in the subsection 2.1.2.1,

if the unshielded triple X i , X k , X j X i X j is a v-structure, knowing the value of X k creates a dependence between X i and X j , while if X i , X k , X j X i X j is a non- v-structure, knowing the value of X k induces a conditional independence between
X i and X j . Thus, if a separation set between X i and X j has been found, and if X k does not belong to it, then under the faithfulness and the causal sufficiency assumptions, one can conclude that X i , X k , X j X i X j is a v-structure and orient the two edges accordingly.

By contrast, as will be shown in section section 5.1.3, the 3off2 approach relies on the sign and magnitude of the 3-point information in order to decide whether a unshielded triple could be oriented as a collider (section 6.3.2.2).

Propagating the orientations

The last step of the PC algorithm consists in propagating the orientations to the remaining undirected edges following propagation rules (Algorithm 1, lines [23][24][25][26][27]) that guarantee that no directed cycles or new v-structures will be created.

It has been shown that repeated application of the three rules R 1-3 is sufficient to recover the CPDAG of the graphical model [Meek, 1995].

By contrast, in the 3off2 algorithm, the propagation and the orientation steps are simultaneously performed, following the rank of each unshielded triple (section 6.3.2.2).

∃{U i } ⊆ adj G (X)\{Y }, not yet considered with |{U i }| = do if Indep(X; Y |{U i }) at conf. level α then XY link is non-essential and removed separation set of XY : Sep XY = {U i } end end end Set = + 1 until ∀X ∈ G, |adj G (X)| ; 2. Orientation forall the unshielded triples do R 0 : {X -Z -Y & X Y & Z / ∈ Sep XY } ⇒ {X → Z ← Y } end 3. Propagation repeat R 1 : {X → Z -Y & X Y } ⇒ {Z → Y } R 2 : {X → Y → Z & X -Z} ⇒ {X → Z} R 3 : {X -Y → Z & X -T → Z & Y T } ⇒ {X → Z}
until no further orientation can be propagated ;

Statistical tests for conditional independence

The conditional independence tests performed during the pruning procedure of constraint-based methods, and in particular during the iteration step of the PC algorithm (Algorithm 1, line [START_REF] Hsiao | Role of duplicate genes in robustness against deleterious human mutations[END_REF]), usually rely on χ 2 or G 2 independence tests.

The decision of removing the edge between two variables X i and X j conditioned on a set of variables X k is made through the confrontation of the following hypothesis:

H 0 The variables X i and X j are independent conditioned on X k H a The variables X i and X j are not independent conditioned on X k

Chi-square conditional independence test

A χ 2 independence test can be used to accept or reject the null hypothesis, i.e. the independence between two discrete variables, X i and X j , conditioned on a set of discrete variables, X k (where X i has r i levels, X j has r j levels, and there exists q k combinations of levels for the set X k ).

The test confronts the following models:

• the independence model:

p i = P (X i |X k )P (X j |X k )
• the observed model:

p o = P (X i , X j |X k )
The model p o is represented by N ijk , the number of data points with

{X i = x i ,X j = x j and X k = x k } (with x k the kth state of the set X k ). The model p i is represented by N ik ×N jk N k
, where N ik corresponds to the number of data points with {X i = x i and X k = x k }, N jk corresponds to the number of data points with {X j = x j and X k = x k }, and N k corresponds to the number of data points with

{X k = x k }.
The test statistics is a χ 2 random variable defined as follow:

χ 2 statistics = r i i,j,l r j j=1 r k k=1 (N ijk -N ik ×N jk N k ) 2 N ik ×N jk N k (3.1)
Then, one can compute a p-value corresponding to the probability of observing a sample statistics as extreme as the test statistics, χ 2 statistics . The independence between X i and X j condition on X k is rejected (i.e. the edge is not removed) for a significance level α if and only if:

P χ 2 statistics < χ 2 df < α (3.2)
where df = (r i -1)(r j -1)q k is the the degree of freedom that also follows a χ 2 with a degree of freedom df = (r i -1)(r j -1)q k :

G 2 = 2 r i i=1 r j j=1 r k k=1 N ijk ln N ijk N ik ×N jk N k (3.3)
Similarly as for the χ 2 statistics, the G 2 likelihood ratio can the used to accept or reject the null hypothesis at a significance level α.

The significance level α may be thought as an adjustable parameter to improve the discovery of the underlying causal structure. A smaller α value tends to favour the independence model and thus to return a less connected graph. Conversely, a larger α tends to favour more complex causal graphical models, as can be seen in In practice, the 3off2 structure learning approach relies instead on the minimization of a 2-point information term, and the simultaneous maximization of a 3-point information term in order to estimate the likelihood that the edge X -Y should be removed (section 6.2.3, Eqs. 6.4 & 6.5). This strategy is shown to be more robust to sampling noise in finite datasets for benchmark networks (Chapter 7).

Variations on the PC algorithm

When an exact list of conditional independence relationships is given as input, the PC or the IC algorithms are guaranteed to learn the CPDAG of the underlying graphical model. However in practice, constraint-based methods should rely on statistical independence tests (subsection 3.1.4), to ascertain the conditional independencies from observational data.

As previously stated [Colombo andMaathuis, 2014, Dash and[START_REF] Dash | A hybrid anytime algorithm for the construction of causal models from sparse data[END_REF], the use of a statistical test on finite datasets makes the constraint-based methods not robust to sampling noise in finite datasets. Indeed, early errors in removing edges from the complete graph typically trigger the accumulation of compensatory errors later on in the prunning process and this cascading effect makes the constraintbased approaches sensitive to the adjustable significance level α required for the conditional independence tests. In addition, traditional constraint-based methods are not robust to the order in which the conditional independence tests are processed, possibly leading to a wide range of different results, in particular in highdimensional settings. This prompted recent algorithmic improvements intending to achieve order-independence, as the PC-stable algorithm proposed by [START_REF] Colombo | Order-independent constraint-based causal structure learning[END_REF], and tested on experimental observational data from yeast gene expression.

Besides, the PC algorithm, as well as other structure discovery approaches, requires specific assumptions to guarantee that the output would be the representative of the markov equivalent class of the underlying graphical model. Among these assumptions, the causal sufficiency plays an important role as it assumes the absence of latent and selection variables. Yet in practice, the system under observation typically contains unmeasured common causes and selection variables (section 2.2.1).

The FCI ('Fast Causal Inference') algorithm [START_REF] Spirtes | An algorithm for causal inference in the presence of latent variables and selection bias[END_REF][START_REF] Spirtes | Causation, Prediction, and Search[END_REF], and the more efficient RFCI ('Really Fast Causal Inference') algorithm [START_REF] Colombo | Learning highdimensional directed acyclic graphs with latent and selection variables[END_REF], have been proposed to enable the discovery of the Partial Ancestral Graph (PAG) (section 2.2.3.2), representative of the Markov equivalence class of DAGs when accounting for hidden and selection variables. The following subsections detail and discuss the PC-stable, the FCI and the RFCI algorithms.

PC-stable: an order-independent constraint-based approach

The dependence on the order in which the variables are given impedes the skeleton learning and the edge orientation steps, in particular with high dimensional settings. [START_REF] Colombo | Learning highdimensional directed acyclic graphs with latent and selection variables[END_REF] proposed a modification of the PC algorithm that removes the order-dependence on the input variables while discovering the skeleton. [START_REF] Ramsey | Adjacency-faithfulness and conservative causal inference[END_REF] and [START_REF] Colombo | Learning highdimensional directed acyclic graphs with latent and selection variables[END_REF]] also proposed complementary orientation procedures to reach order-independence during the orientation/propagation steps.

Order-independent skeleton learning

The order-dependence of the PC algorithm is related to the removal of the nonessential edge (Algorithm 1, lines [11]) within one level (Algorithm 1, lines [8 -16]), where corresponds to the size of the separation set. Indeed, removing an edge at a given level possibly modifies the adjacency sets of the remaining

edges not yet tested for conditional independence, and thus influences the outcome of the following statistical tests at this level. In practice, mistakes can occur when checking for conditional independence using a statistical test and lead to erroneous skeleton and separation sets. As the order in which the input variables are considered can imply different errors, a wide range of skeletons can be expected, in particular for high dimensional settings as shown in [START_REF] Colombo | Order-independent constraint-based causal structure learning[END_REF].

[ [START_REF] Colombo | Order-independent constraint-based causal structure learning[END_REF] proposed an order-independent version of the skeleton discovery step for the PC algorithm, called PC-stable, where the edge removal is not done within a specific level . Instead, the PC-stable algorithm stores all the possible separation sets found for a given level and removes the edges corresponding to the conditional independence only before reaching the next level (or eventually before ending the algorithm). This postponed edge removal between successive levels, rather than within one level, allows for an order-independent discovery of the skeleton, as the edges under statistical test for a level do not influence each others adjacency sets when removed.

As detailed in section 6.3.2, the 3off2 inference approach proceeds on a list of triples ordered by a rank (Eq. 6.32) that optimizes the likelihood that a variable Z is contributing to the relationship between two other variables, X and Y . As the 3off2 inference procedure relies on this rank, the skeleton produced is not dependent on the order in which the input variables are processed.

Order-independent orientation steps

As exemplified in [START_REF] Colombo | Learning highdimensional directed acyclic graphs with latent and selection variables[END_REF], the information contained in the separation sets are also order-dependent, as some separating sets may hold by mistake at some level α. When used for the orientation of the v-structures, these spurious separation sets may lead to incorrect orientations depending on the order in which the given variables are processed.

[ [START_REF] Ramsey | Adjacency-faithfulness and conservative causal inference[END_REF] proposed a conservative rule to remove the order-dependence of this orientationstep. [START_REF] Colombo | Order-independent constraint-based causal structure learning[END_REF] proposed a similar but less conservative approach. Both methods rely on the identification of unambiguous triples, and differ in the definition of the unambiguity. These approaches first collect, for each unshielded triples X i , X k , X j X i X j in the reconstructed skeleton C, all the separation sets {Y} among the variables belonging to adj C (X i ) and

adj C (X j ), such that X i ⊥ ⊥ X j |Y.
For the conservative approach [START_REF] Ramsey | Adjacency-faithfulness and conservative causal inference[END_REF], X i , X k , X j X i X j is said unambiguous if at least one separation set Y can be found, and X k belongs to all or none of the separation sets {Y}. When the triple is unambiguous, it is oriented as a v-structure if and only if X k is in none of the separation sets {Y}. By contrast, [START_REF] Colombo | Order-independent constraint-based causal structure learning[END_REF] rely on a majority rule to decide for unambiguity. X i , X k , X j X i X j is said unambiguous if at least one separation set Y can be found, and X k is not in exactly 50% of the {Y}.

When the triple is unambiguous, it is oriented as a v-structure if and only if X k is in less than half of the separation sets {Y}. For both methods, only the unambiguous triples are considered in the orientation propagation step. The algorithms resulting from the association of the PC-stable step and the conservative or the majority rule orientation step are called respectively CPC-stable and MPC-stable.

Finally, even when applying the CPC-stable or the MPC-stable algorithms, some orientation conflicts may occur among the unambiguous v-structures, for instance when X 1 → X 2 ← X 3 and X 2 → X 3 ← X 4 have both been learned. To avoid order-dependence, one can resolve the orientation conflict by setting a bidirectional edge between X 2 and X 33 . The algorithms resulting from the association of the PC-stable step, the conservative or the majority rule orientation step, and the orientation conflict feature are called respectively LCPC-stable and LMPC-stable.

As detailed in the section 6.3.2.2, the 3off2 orientation/propagation step ranks the unshielded triples by the absolute value of their 3-point information. Then, depending on the sign of the 3-point information, the unshielded triples are successively oriented as a v-structure or allow for a propagation (Algorithm 4, step 2. and Algorithm 5). This procedure based, solely on statistically significant 3point information, provides a robust orientation/propagation step, independent of the order in which the variables are processed.

Causal Inference with latent and selection variables

The FCI algorithm

As introduced in the section 2.2.1, unmeasured variables such as latent or selection variables, cause complications when learning a causal graphical model from the observed variables. The FCI ('Fast Causal Inference') algorithm [START_REF] Spirtes | An algorithm for causal inference in the presence of latent variables and selection bias[END_REF][START_REF] Spirtes | Causation, Prediction, and Search[END_REF] to R 10 established in [Zhang, 2008a]. The returned graphical model C, akin to a PAG [START_REF] Zhang | Causal reasoning with ancestral graphs[END_REF], is said maximally informative.

The Fig. As can be seen, the edge X 1 •-• X 5 has not been remove after the two first steps of the FCI algorithm as the vertex X 3 , required for the conditional independence between X 1 and X 5 , does not belong to adj C 1 (X 1 ) nor adj C 1 (X 5 ). However, in the following third step, the FCI algorithm searches more distant vertices as it relies on Possible-D-SEP to find the separation sets. Since pds [Zhang, 2008a] until no further orientation can be propagated ; {X 2 , X 3 , X 4 } and pds C 2 (X 5 , X 1 ) \ {X 1 , X 5 } = {X 2 , X 3 , X 4 , X 6 }, the FCI process can identify the conditional independence.

C 2 (X 1 , X 5 ) \ {X 1 , X 5 } =
R 0 : {X * -• Z •- * Y & X * * Y & Z / ∈ Sep XY } ⇒ {X * → Z ← * Y } end 3. Final skeleton, separation sets, out: C 3 for all vertices X ∈ C 2 do Compute pds C 2 (X, •) for all vertices Y ∈ adj C 2 (X) do Let = 0 while XY ∈ adj C 2 with |pds C 2 (X, •) \ {Y }| do while ∃ Y ⊂ pds C 2 (X, •) \ {Y } not
R 0 : {X * -• Z •- * Y & X * * Y & Z / ∈ Sep XY } ⇒ {X * → Z ← * Y } end 5. Propagation, out: C repeat Rules R 1 to R 10 from
The FCI algorithm appears rather convoluted and is known to be time consuming, as the Possible-D-SEP can get very large, even when considering sparse graphs.

By contrast, the 3off2 algorithm can easily deal with the assumption of latent variables by allowing the inclusion of observed variables that are not direct neighbours of the pair of nodes for which the conditional independence is tested.

As detailed in section 6.4.2, the 3off2 requires only simple modifications that still guarantee the discovery of a causal graphical model in a reasonable amount of time on sparse graphs.

Improvements of the FCI algorithm

The RFCI [START_REF] Colombo | Learning highdimensional directed acyclic graphs with latent and selection variables[END_REF] and the AnytimeFCI [START_REF] Spirtes | An anytime algorithm for causal inference[END_REF] algorithms improve the speed of the FCI algorithm that suffer from an exponential time complexity due to the potentially large size of the Possible-D-SEP sets of variables. Specifically, the RFCI approach follows the same three steps as the PC algorithm, but with modifications in the steps 2 and 3 to take into account the hidden variables. In fact, in the step 2, the RFCI algorithm remove extra edges before orienting the v-structures ( [START_REF] Colombo | Learning highdimensional directed acyclic graphs with latent and selection variables[END_REF], Lemma 3.1) and then used a modified version of the rule R 4 for the orientation of the discriminating paths [START_REF] Colombo | Learning highdimensional directed acyclic graphs with latent and selection variables[END_REF], Lemma 3.2). Avoiding to compute all the subsets of the Possible-D-SEP reduce the time complexity, yet the RFCI algorithm produces a less informative output than with the FCI approach. By contrast, the AnytimeFCI requires an additional adjustable parameter to perform fewer conditional tests as the FCI procedure by limiting the size of the conditioning set of variables.

As stipulated earlier, the 3off2 method combines constraint-based and Bayesian in-

Bayesian methods

The Bayesian methods [START_REF] Cooper | A bayesian method for the induction of probabilistic networks from data[END_REF] step through a space of possible causal structures, usually the space of DAGs, following a search-and-score procedure that aims at optimizing a scoring function. This score provides an indicator of the fitness between the model and the data and thus explores each graphical model in a quantitative manner. There exist various search strategies (e.g. simulated annealing [START_REF] Chickering | Learning Bayesian networks: Search methods and experimental results[END_REF], Markov Chain Monte Carlo [START_REF] Kocka | Improved learning of bayesian networks[END_REF], ant colony optimization [START_REF] De Campos | Technologies for constructing intelligent systems. chapter Stochastic Algorithms for Searching Causal Orderings in Bayesian Networks[END_REF]) and types of score (e.g. entropy-based [START_REF] Chow | Approximating Discrete Probability Distributions With Dependence Trees[END_REF]Liu, 1968, de Campos, 2006], Minimum Description Length [START_REF] Bouckaert | Probabilistic network construction using the minimum description length principle[END_REF][START_REF] Rissanen | Modeling by shortest data description[END_REF], Bayesian Dirichlet score with score equivalence [START_REF] Heckerman | Learning Bayesian Networks: The Combination of Knowledge and Statistical Data[END_REF]).

Although the constraint-based methods are based on rigorous theoretical founding (Chapter 3), the irreversible pruning process may lead to unreliable results when applied to finite datasets. By contrast, the Bayesian approaches are more robust as search-and-score methods are typically reversible and thus do not trigger cascading errors in the following steps of the exploration procedure. However, the application of these score-based approaches to the discovery of causal graphical models remains difficult when assuming the presence of latent variables. Yet, strategies such as STRUCTURAL EM 1 -based algorithms have been proposed [START_REF] Friedman | The bayesian structural em algorithm[END_REF][START_REF] Mitchell | Machine Learning[END_REF]. Finally, background knowledge of Bayesian methods can be set, either for the causal structure discovery or for the parameters inference, in the shape of prior functions.

The 3off2 learning approach combines constraint-based and Bayesian frameworks to reconstruct graphical models in a robust manner despite inherent sampling noise 1 Expectation Maximization in finite observational datasets. This chapter provides an overview of some of the well-known scoring functions used by the Bayesian structure discovery approaches.

Learning a Bayesian Network

A Bayesian network G defines a joint probability distribution of a m-dimensional multivariate data vector X = (X 1 , ..., X m ) [START_REF] Pearl | Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference[END_REF] and possesses two components: (i) a causal structure that entails the (in)dependence relationships between the observed variables, and (ii) a set of probability distributions that quantify these relationships. We are in particular interested in learning the first component, that can be represented by a DAG (section 2.1). Learning the structure of a Bayesian network consists in finding the DAG that best fits the data according to where P (G) corresponds to a prior on the graphical model G, and P (D) is a prior on the data D, also called evidence.

When solving the problem of learning a Bayesian network, the evidence being the same for all possible graphical models, maximizing the posterior probability P (G|D) is equivalent to maximize the joint probability P (G, D). Besides, when assuming a uniform prior P (G), maximizing the posterior probability P (G|D) turns out to be also equivalent to maximizing the likelihood P (D|G). In the following section, a uniform prior P (G) is assumed and the likelihood P (D|G) is used as a score.

The search-and-score procedures are typically based on a score that is decomposable, to allow a computationally feasible process. Some scores are also Markov equivalent, ensuring that they return the same score for networks in the same equivalence class.

Decomposability Searching through the space of structures in an efficient manner requires locally decomposable scoring functions. After a modification on a structure G 1 (e.g. an edge removal), leading to a new structure G 2 , a decomposable scoring function allows the evaluation of G 2 without recomputing the whole score but only by taking into account the score variation due to the modification.

In other words, a decomposable scoring function can be written as a sum of local scores (in the logarithmic space) that depend only on each node and its parents.

Score equivalence When searching through the space of equivalence classes of DAGs, it is preferable that the scoring function attributes the same weight to causal structures belonging to the same equivalence class. For instance, one would expect the three possible non-v-structures of an unshielded triple X i , X k , X j X i X j (Fig. 2.1) to receive the same score, but a different score than the v-structure [START_REF] Chickering | Learning Bayesian networks: Search methods and experimental results[END_REF] shown that the MDL (Eq. 4.8) score is score equivalent. By contrast, the K2, BD or BDe scores are not score equivalent.

X i → X k ← X j .
In the sections 4.

Scoring funtions 4.2.1 Bayesian scores

The K2 is one of the first Bayesian scoring function that has been proposed [START_REF] Cooper | A bayesian method for the induction of probabilistic networks from data[END_REF]. Under several assumptions, this scoring function can be expressed as,

Score K2 (D|G) = n i=1 q i j=1 log (r i -1)! (N ij + r i -1)! + r i k=1 log(N ijk !) (4.2)
Then, [START_REF] Heckerman | Learning Bayesian Networks: The Combination of Knowledge and Statistical Data[END_REF] proposed the Bayesian Dirichlet (BD) score, a generalization of the K2 score,

Score BD (D|G) = n i=1 q i j=1 log Γ(α ij ) Γ(N ij + α ij ) + r i k=1 log Γ(N ijk + α ijk ) Γ(α ijk ) (4.3)
where the α ijk , so-called hyperparameters, correspond to a a priori distribution of the parameters. In practice, these hyperparameters are not easily defined. [START_REF] Heckerman | Learning Bayesian Networks: The Combination of Knowledge and Statistical Data[END_REF] established a scoring function, the Bayesian Dirichlet equivalent, by assuming the likelihood equivalence. This allows a simple computation of the α ijk by introducing the imaginary (or equivalent) sample size N ,

α ijk = N × P (X i = x k , pa(X i = x j )|G c ) (4.4)
The imaginary sample size N defines the weight that should be assigned to the Finally, [START_REF] Buntine | Theory refinement on bayesian networks[END_REF] proposed a Bayesian scoring function that only depends on N by assuming a uniform probability for each configuration

{X i } ∪ pa G(X i ) ,
i.e. P (X i = x k , pa(X i = x j )|G c ) = 1 r i q i , which became a common assumption. The resulting score is the Bayesian Dirichlet equivalent uniform score,

Score BDeu (D|G) = n i=1 q i j=1 log Γ( N q i ) Γ(N ij + N q i ) + r i k=1 log Γ(N ijk + N r i q i ) Γ( N r i q i ) (4.5)
As can be seen from Eq. 4.5, this Bayesian score is decomposable, as well as score equivalent.

Although the 3off2 approach does not implement the BDeu criterion, its results have been compared with the networks learnt with a hill-climbing heuristic method relying on this criterion (Chapter 7, and Appendices A & B).

Information-theoretic scores

The idea behind the information-theoretic scoring functions is related to the principle of compression that can be obtained over the data D with an optimal code represented by a Bayesian network G. In fact, one can define the information content of D by G, i.e. the size of an optimal code induced by the distribution G encoding D, given D. This information content turns out to be equal to the negative of the log-likelihood function of the data D given G, i.e. log(P (D|G)), by using the Huffman code. Thus, minimizing the information content amounts to maximize the log-likelihood log(P (D|G)), and thus equivalently to maximize the posterior probability P (G|D).

The LL scoring function

The previous considerations lead to the definition of the LL scoring function. Let us assume that each observed variable X i has r i levels, and that the set of parents for each variable X i , pa G (X i ), has q i = Xp∈pa G (Xp) r p values. The likelihood of the dataset D given a network G can be computed by taking the product of the row probabilities,

P (D|G) = m i=1 q i j=1 r i k=1 θ N ijk ijk (4.6)
and maximizing the likelihood with the normalization constraint

k θ ijk = 1 leads to θ ijk = θ ijk = N ijk N ij .
By taking the log of the previous maximum likelihood, this lead to the LL scoring function, defined as,

LL(D|G) = m i=1 q i j=1 r i k=1 N ijk log N ijk N ij (4.7)
However, the LL score alone tends to overfit the data by favoring the reconstruction of complete networks. A common way of limiting the overfitting phenomenon is to penalize this score by taking into account the complexity of the graphical models, as detailed in the following section.

The MDL/BIC criterion

The Minimum Description Length (MDL) (or the Bayesian Information Criterion (BIC)) [Hansen andYu, 2001, Rissanen, 1978] As can be understood from Eq. 4.8, the MDL scoring function is locally decomposable,

MDL(D|G) = N m i=1 q i j=1 r i k=1 N ijk N log N ijk N ij - m i=1 1 2 log(N )(r i -1)q i (4.9)
In the context of the information-theoretic scoring functions, the quality formula of a Bayesian network can be generalized by φ(D|G) = LL(D|G) -f (N )|G|, where f (N ) is a non negative penalization. The AIC (Akaike Information Criterion) corresponds to the case of f (N ) = 1.

The MDL criterion has been implemented in our hybrid approach 3off2 in order to score and compare network candidates while removing extra edges. However, the 3off2 algorithm slightly reformulated the MDL criterion as the parent nodes are replaced by an unknown separation set U i of upstream nodes to be learnt simultaneously with the missing edge candidate (see section 6.2.2 for further details on the MDL criterion).

The NML criterion

As detailed in section 6.2.2, the MDL criterion tends to overestimate the encoding of parameters given the limited amount of data, and hence to increase the number of false negative edges, in particular when the number of levels r i is high. The Normalized Maximum Likelihodd (NML) [START_REF] Rissanen | Modeling by shortest data description[END_REF]Tabus, 2005, Shtarkov, 1987] criterion, based on the MDL principle, reduces this bias. Instead of overestimating the encoding of the required parameters, the NML uses the actual combinations of values of variables that exist in the available dataset D. For a network G, the normalized maximum likelihood score is defined as,

NML(D|G) = LL(D|G) -C N (G) (4.10)
where C N (G) involves an exponential sum over all possible datasets of size N that can be approximated by the following formula [Roos et al., 2008b],

C N (G) f C N (G) = n i=1 q i j=1 log(C r i N ij ) (4.11)
[ [START_REF] Kontkanen | A linear-time algorithm for computing the multinomial stochastic complexity[END_REF] proposed a linear-time algorithm for computing the log(C r i N ij ) complexity in the case of N ij observations of a single multinomial random variables (Algorithm 3).

Algorithm 3: Multinomial parametric complexity Compute log(C r i N ij ) where C l m ≡ C(l, m
) is obtained by the following recurrence:

1. C(1, m) = 1 2. C(2, m) = h 1 +h 2 =m m! h 1 !h 2 ! h 1 m h 1 h 2 m h 2 3. if (l > 2) then C(l, m) = C(l -1, m) + m l-2 C(l -2, m)
Similarly as for the MDL criterion, the factorized NML scoring function is decomposable as,

f NML(D|G) = m i=1 q i j=1 r i k=1 N ijk log N ijk N ij ) -log(C r i N ij ) (4.
12)

The factorized NML criterion has also been implemented in the 3off2 hybrid approach in order to score and compare network candidates before and after removing one extra edge. This criterion typically gives better results than the MDL criterion on benchmark networks for small datasets, i.e. when the bias due to the overestimation of the encoding of parameters is stronger (Chapter 7). Similarly as for the MDL criterion, the NML implemented in the 3off2 algorithm replaces the parent nodes with an unknown separation set {U i } of upstream nodes to be learnt simultaneously with the missing edge candidate (see section 6.2.2 for further details on the MDL criterion).

4.3

The search-and-score paradigm

The exact discovery

A simple idea when searching through the space of possible DAGs is to compute the score of all possible structures to find the Bayesian network that best fits the data. Yet, this is computationally infeasible for large networks as the number of causal graphical models grows superexponentially with the number of variables.

Even when the maximum in-degree is limited a priori, finding an optimal Bayesian network remains NP-hard [START_REF] Chickering | Learning Bayesian networks: Search methods and experimental results[END_REF].

Early developments led to algorithms that can solve the structure learning problem exactly for restricted classes of Bayesian networks, for instance if an ordering of the variables is given as input [START_REF] Cooper | A bayesian method for the induction of probabilistic networks from data[END_REF]. More recent studies have proposed algorithms that compute the exact posterior probability of network structures for a moderate number of variables (less than 30) when additional constraints are enforced on the space of structures [START_REF] Koivisto | Exact bayesian structure discovery in bayesian networks[END_REF][START_REF] Malone | Memory-efficient dynamic programming for learning optimal bayesian networks[END_REF][START_REF] Silander | A simple approach for finding the globally optimal bayesian network structure[END_REF].

Still, the limitations of the possible improvements of exact methods have prompted the developments of a variety of heuristic approaches.

The heuristic search approaches

Heuristic methods step through the space of possible DAGs, but do not explore the whole space exhaustively. The main differences between these searching methods stem from the local operators that progressively modify the causal structure to optimize the global score (e.g. edge addition, removal or inversion [START_REF] Chickering | Learning Bayesian networks: Search methods and experimental results[END_REF]). These greedy search approaches are computationally feasible thanks to the decomposability of the scoring function.

However, the greedy search procedure is known to lead to local optima. The hillclimbing algorithm with random restarts limits the risk of being trapped in a single local optimum by performing several greedy searches from different seed networks.

Other possible approaches are the simulated annealing procedures [START_REF] Kirkpatrick | Optimization by simulated annealing[END_REF], the genetic algorithms [START_REF] Larrañaga | Learning bayesian network structures by searching for the best ordering with genetic algorithms[END_REF]. As exemplified in the following section, it is possible to combine approaches in order to decide for the initial network.

The 3off2 reconstruction method uses instead a best-first search approach in order to identify the most likely dispensible edges. The iterative identification of these candidate edges is done thanks to a rank based on 2-point and 3-point information terms, allowing for a statistically robust search strategy.

Hybrid approaches

Hybrid methods have attempted to exploit the best of the constraint-based (Chapter 3) and Bayesian methods by combining the robustness of Bayesian scores with the attractive conceptual features of constraint-based approaches [START_REF] Cano | A score based ranking of the edges for the pc algorithm[END_REF][START_REF] Claassen | A bayesian approach to constraint based causal inference[END_REF][START_REF] Dash | A hybrid anytime algorithm for the construction of causal models from sparse data[END_REF][START_REF] Singh | An algorithm for the construction of bayesian network structures from data[END_REF][START_REF] Tsamardinos | The Max-Min Hill-Climbing Bayesian Network Structure Learning Algorithm[END_REF]. In particular, [START_REF] Dash | A hybrid anytime algorithm for the construction of causal models from sparse data[END_REF] have proposed to exploit an intrinsic weakness of the PC algorithm, its sensitivity to the order in which conditional independencies are tested on finite data (section 3.2.1),

to rank these different order-dependent PC predictions with Bayesian scores.

In fact, many of the hybrid approaches proceed in two steps: the first step relies on a constraint-based procedure that produces a graphical model, which is then given as input to the second step that relies on a Bayesian method. For instance, [START_REF] Singh | An algorithm for the construction of bayesian network structures from data[END_REF] developed an algorithm that first gets an absolute temporal ordering on the variables and then uses this ordering as input to the K2 algorithm [START_REF] Cooper | A bayesian method for the induction of probabilistic networks from data[END_REF]. The Max-Min Hill-Climbing (MMHC) algorithm [START_REF] Tsamardinos | The Max-Min Hill-Climbing Bayesian Network Structure Learning Algorithm[END_REF] is a prominent example of hybrid method. The main idea is to restrict the search space using first the Max-Min

Parents and Children (MMPC) constraint-based algorithm [START_REF] Tsamardinos | Time and sample efficient discovery of markov blankets and direct causal relations[END_REF]. The MMPC step provides for each variable X the set of its parents and children, PC X . Then, the MMHC method follows a greedy hill-climbing strategy with operators add-edge, delete-edge and reverse-edge. The add-edge operator is only considered when it is consistent with the previously discovered parent and children set (i.e., try Y → X if Y ∈ PC X ). The use of the operator deleteedge within the hill-climbing search allows for the removal of edges during the orientation phase of MMHC.

More recently [START_REF] Claassen | A bayesian approach to constraint based causal inference[END_REF] have also combined constraint-based and Bayesian approaches to improve the reliability of conditional independencies by summing the likelihoods over compatible graphs.

By contrast, the 3off2 algorithm uses Bayesian scores to progressively uncover the best supported conditional independencies, by iteratively "taking off" the most likely indirect contributions of conditional 3-point information from every 2-point (mutual) information of the causal graph (section 6.3). In addition, using likelihood ratios (Eqs. 6.17 & 6.18) instead of likelihood sums [START_REF] Claassen | A bayesian approach to constraint based causal inference[END_REF] circumvents the need to score conditional independencies over a potentially untractable number of compatible graphs.

Chapter 5

Information-theoretic methods

The information-theoretic methods typically decide for the statistical independence between two variables using information-theoretic measures [START_REF] Butte | Mutual information relevance networks: Functional genomic clustering using pairwise entropy measurements[END_REF][START_REF] Chow | Approximating Discrete Probability Distributions With Dependence Trees[END_REF][START_REF] Margolin | ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context[END_REF]]. These methods have already given promising results, in particular in the field of systems biology, for large networks up to thousands variables. An important concept of information theory [START_REF] Shannon | A mathematical theory of communication[END_REF] is the 2-point (mutual) information that provides a measure of dependencies between variables. In particular, while the correlation coefficients only measure linear (Pearson) or monotonic (Spearman) dependencies, the 2-point (mutual) information is also sensitive to diverse types of non-linear relationships.

Hence, the 2-point information can be interpreted as a general measure of dependencies, which may be of great interest in the context of complex biological systems [START_REF] Steuer | The mutual information: Detecting and evaluating dependencies between variables[END_REF].

In this chapter, we first introduce the entropy, and the 2-point and 3-point information quantities. These information-theoretic measures are at the root of our new causal structure learning method, namely 3off2, detailed in Chapter 6. The following sections are dedicated to state-of-art information-theoretic methods and hybrid methods relying on information-theoretic scores.

Information-theoretic measures

5.1.1 The Shannon entropy [START_REF] Fraser | Independent coordinates for strange attractors from mutual information[END_REF] defined the entropy in a general way as the 'quantity of surprise you should feel upon reading the result of a measurement'. Formally, the entropy H(X) of a random discrete variable X with r x levels is defined as, [START_REF] Cover | Elements of Information Theory[END_REF]Thomas, 2006, Shannon, 1948]:

H(X) = - rx i=1 p(x i ) log(p(x i )) (5.1)
where x i is one of the state of the variable X, and p(x i ) is the probability of observing x i . Thus, when the state of the variable X is fully determined, the entropy is null, while it is maximal when all the states have the same probability of occurrence (in the following, the log in Eq. 5.1 refers to the natural logarithm).

Similarly, the joint entropy for two discrete variables, X and Y , is defined as,

H(X, Y ) = - rx i=1 ry j=1 p(x i , y j ) log(p(x i , y j )) (5.2)
where Y has r y levels, y j is one of the state of the variable Y and p(x i , y j ) is the joint probability that X takes the value x i and Y takes the value y j .

A general way to decompose the entropy uses the conditional entropy,

H(X, Y ) = H(X|Y ) + H(Y ) (5.3)
where,

H(X|Y ) = -
I(X; Y ) = H(X) + H(Y ) -H(X, Y ) ≥ 0 (5.5)
Similarly, one can define the conditional 2-point information as,

I(X; Y |Z) = H(X|Z) + H(Y |Z) -H(X, Y |Z) ≥ 0 (5.6)

The data processing inequality

The Data Processing Inequality is a well-known inequality of the information theory. It can be intuitively understood as the fact that no information processing by a downstream variable Z can enhance or restore information imperfectly transmitted from X to Y . This inequality can be formally defined as, In other words, the variable Z cannot have more information about the variable X than the variable Y has about the variable X.

As detailed in the section 5.2.3, the ARACNe reconstruction method [START_REF] Margolin | ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context[END_REF] relies on the contrapositive of the DPI in order to remove superfluous edges during the structure discovery.

The 3off2 algorithm also includes in its edge removal procedure a rank that is consistent with the DPI (Eq. 6.32).

The multivariate information

When taking into account more than two variables, it is also possible to define a multivariate information quantity [START_REF] Mcgill | Multivariate information transmission[END_REF]. In particular, we are interested in the 3-point information, which can be defined by the difference between the 2point and the conditional 2-point information when introducing a third variable Z,

I(X;Y;Z) = I(X;Y ) -I(X;Y |Z) (5.7)
or, more generally,

I(X;Y;Z|{U i }) = I(X;Y |{U i }) -I(X;Y |{U i }, Z) (5.8)
with {U i }, a set of variables that does not contain X, Y nor Z.

The conditional 3-point information I(X; Y ;

Z|{U i }) is in fact symmetric in X, Y and Z, I(X; Y ; Z|{U i }) = I(X; Y |{U i }) -I(X; Y |{U i }, Z) = I(X; Z|{U i }) -I(X; Z|{U i }, Y ) = I(Y ; Z|{U i }) -I(Y ; Z|{U i }, X)
This symmetry can be shown when decomposing the 3-point information using entropy terms,

I(X; Y ; Z) = H(X) + H(Y ) + H(Z) -H(X, Y ) -H(X, Z) -H(Y, Z) + H(X, Y, Z)
It should be noted that, as opposed to the 2-point information, which is always positive, the 3-point information can also be negative. In the context of network discovery, as Eq. 5.8 is always valid, one can understand that no assumption on the underlying graphical model or on the amount of data available to estimate conditional 2-point and 3-point information terms is necessary.

Chapter 6 details how the 3off2 algorithm is taking advantage of Eq. 5.8, which is used to prove Lemmas 1, 3 & 4, as well as Proposition 5 (section 6. 1.3). This allows to trace back the origin of necessary causal relationships in a graphical model to the existence of a negative conditional 3-point information between three variables {X, Y, Z}, I(X; Y ; Z|{U i }) < 0, where {U i } accounts for a structural independency between two of them, e.g. I(X; Y |{U i }) = 0 (Lemma 4, section 6.1.3).

The cross-entropy of a Bayesian network

The maximum likelihood L D|G of a graphical model G over a set of variables {X i }

(section 4.1) is related to the cross-entropy H(G, D) = -{X i } p({X i }) log(q({X i }))
between the "true" probability distribution p({X i }) from the data D and the ap-

proximate probability distribution q({X i }) = i p(X i |{P a X i }) generated by the
Bayesian network G with specific parent nodes {P a X i } for each node X i , leading to [START_REF] Sanov | On the probability of large deviations of random variables[END_REF],

L D|G = e -N H(G,D) = e -N i H(X i |{P a X i })
(5.9) where i H(X i |{P a X i }) is the (conditional) entropy of the underlying causal graph.

This enables to score and compare alternative models through their maximum likelihood ratio as,

L D|G L D|G = e -N i (H(Xi|{Pa X i })-H(X i |{P a X i })) (5.10)
Note, in particular, that the significance level of the Maximum likelihood approach is set by the number N of independent observational data points.

5.2 State-of-the-art of information-theoretic methods

The Chow & Liu approach

The Chow & Liu approach [START_REF] Chow | Approximating Discrete Probability Distributions With Dependence Trees[END_REF] has been the first structure learning method relying on the 2-point information measure. The output is a Maximum Spanning Tree (MST), obtained with the Kruskal's algorithm, where each edge is weighted by a 2-point information measure. The main drawback of this method is that it outputs a sparse model although the underlying networks might not be sparse.

Relevance Network

The Relevance Network (RELNET) approach [START_REF] Butte | Mutual information relevance networks: Functional genomic clustering using pairwise entropy measurements[END_REF], primarily designed for clustering, computes the 2-point information of each pair of variables, and removes the edges of the pairs that have a mutual information lower than a given threshold. This method is computational efficiency, however its main limitation resides in the choice of the threshold, as a high threshold increases the number of false negative edges, while a low threshold increases the number of false positive edges, and hence impedes the precision. In fact, the 2-point information between two variables X and Y may result from a common cause Z, or from background correlations (e.g. uneven samples, normalization failure), and thus indicates a dependence although no direct relationship actually exists between X and Y .

The Context Likelihood of Relatedness (CLR) algorithm [START_REF] Faith | Large-scale mapping and validation of escherichia coli transcriptional regulation from a compendium of expression profiles[END_REF] is an extension of the RELNET approach that takes into account the background correlations by comparing the mutual information of each pair of variables to the background distribution of mutual informations of all other possible pairs between X or Y and the remaining variables. The CLR score between two variables X and Y corresponds to w xy = z 2 x + z 2 y , where 5.11) with µ x and σ x the mean and standard deviation of the empirical distribution of the mutual information values I(X, X k ) of X with all the remaining variables.

z x = max 0, I(X; Y ) -µ x σ x (
By contrast, the 3off2 approach relies on the most significantly positive conditional 3-point information measures (section 5.1.3) to identify the most likely contributors to observed relationships and decides for the removal of edges based on a criterion related to the complexity of the graphical model under reconstruction (section 6.2.2).

In [START_REF] Watkinson | Inference of Regulatory Gene Interactions from Expression Data Using Three-Way Mutual Information[END_REF], the authors substitute the mutual information I(i; j)

with the sum [I(G i ; G j ) + S(i; j)], where S(i; j) is the synergistic regulation index. This index measures the degree of confidence that a variable G i cooperatively regulates another variable G j with a third variable G k , and is defined as

S(i, j) = max k (-I(G i ; G j ; G k )), where (k = i), (k = j), I(G i ; G k ) < I(G i ; G j ) and I(G i ; G k ) < I(G k ; G j ).
The resulting Synergy-Augmented CLR (SA-CLR)

algorithm [START_REF] Watkinson | Inference of Regulatory Gene Interactions from Expression Data Using Three-Way Mutual Information[END_REF] aims at ascertaining the existence of an interaction between G i and G j by identifying a synergistic partner G k to G i in the regulation of G j .

While the SA-CLR approach aims at maximizing the 2-point information and minimizing the synergy interaction (i.e. I(G i ; G j ; G k ) most negative), the 3off2 method relies on a score that maximizes the conditional 3-point information

I(X; Y ; Z|{U i }) (likely non v-structure indicating the presence of a contributor),
and minimizes the conditional 2-point information I(X; Y |{U i } ∪ {Z}) (most likely base-edge of the non v-structure), where {U i } is a set of variables that possibly separates the pair (X, Y ) at end.

ARACNe

The By contrast, as will be shown in section 6.1.2, the 3off2 algorithm is using the 3-point information in order to decide whether a non-v-structure is more likely than a v-structure. Then, the decision of removing an edge only takes place after a full decomposition of the 2-point information decomposition has been uncovered in terms of 3-point information contributions (section 6.3.1), allowing for a robust discovery of the causal structure.

Minimum Redundancy Networks

The Minimum Redundancy Networks (MRNET) approach [START_REF] Meyer | Information-theoretic inference of large transcriptional regulatory networks[END_REF] is based on the Maximum Relevance Minimum Redundancy (MRMR) feature selection procedure [START_REF] Peng | Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy[END_REF]. For a given target variable X j , the MRMR approach ranks a set of selected feature variables X s j ⊆ X \ X j according to a score that is the difference between:

• the maximum relevance, represented by the 2-point information between X j and the feature candidate X i

• the minimum redundancy, represented by the average 2-point information between the feature candidate X i and the already selected features in X s j

This score is expected to rank well the feature variable having direct interactions with X j . The MRNET approach repeats the MRMR greedy search, taking successively all the variables as target X j . Formally, given a set S j of already selected features for the variable X j , S j is updated by choosing X i that maximizes the score s i :

s i = I(X i ; X j ) - 1 |S j | k =i I(X i ; X k ) (5.12)
Hence, for each X j , the approach iteratively builds the set X s j . This iterative process stops when 1

|S j | k =i I(X i ; X k ) > I(X i ; X j ).
Finally, for each pair of variables (X i , X j ), the MRNET approach computes two scores, s i and s j (Eq. 5.12).

If the maximum of these two scores is below a given threshold, the edge X i -X j is removed.

The limitation of this efficient method is related to an issue shared by the forward selection approaches, namely the strong dependence of the quality of the set X s j to the choice of the first selected feature variable, as an erroneous first feature variable impedes the following redundancy minimization process. [START_REF] Meyer | Information-theoretic inference of gene networks using backward elimination[END_REF] proposed the MRNET Backward (MRNETB) method to overcome this limitation.

As can be understood from Eq. 5.12, the set X s j maximizes the difference between the relevance (Eq. 5.14) of all selected variables and their redundancy (Eq. 5.15) as,

X s j = argmax S j (u r ) (5.13) u = 1 |S j | i∈S j I(X i ; X j ) (5.14) r = 2 |S j |(|S j | -1) i,k>i∈S j I(X i ; X k ) (5.15)
The MRNETB method uses a background removal procedure that starts from sets X s j containing all the variables and that iteratively removes the variables X i which induce the highest increase of the X s j score (Eq. 5.13).

By contrast, the 3off2 reconstruction method selects the variables belonging to the set of upstream nodes {U i } following a rank that indicates the most likely contributors to the observed relationships (section 6.32). This rank, based on 2point and 3-point information terms, allows for a robust discovery of conditional independencies and leads to the progressive recovery of the separation set of each pair of variables. Besides, similarly as for ARACNe, the MRNET and MRNETB approaches relies on the symmetric 2-point information measure, and thus cannot learn oriented networks. By associated the 3-point information to the 2-point information, the 3off2 algorithm can by contrast discover an oriented causal structure (see Lemma 4. in section 6.1.3 and Eq. 6.32).

In [START_REF] Bontempi | Causal filter selection in microarray data[END_REF], the authors use the synergy measure (section 5.2.2) in a feature selection procedure named min-Interaction Max-Relevancy (mIMR)

to learn graphical models. The forward step of this approach consists in selecting the variables x k that maximize the 2-point information with the target y, I(x k ; y) (Max-Relevance), and minimize the multivariate information, I(X s ; x k ; y)

(min-Interaction), where X s is the set of already selected variables. Hence, the incremental step of mIMR follows x * d+1 = argmax x k ∈X\Xs (I(x k ; y) -I(X s ; x k ; y)), where x * d+1 is the optimal variable to be inserted in the set X s containing already d selected variables.

Instead of following a feature selection approach, the 3off2 method relies on a constraint-based procedure. Indeed, 3off2 identifies conditional independences between pairs of variables by iteratively collecting the most likely contributors, {U i }, to an edge, X -Y . This identification relies on a score that maximizes the conditional 3-point information I(X; Y ; Z|{U i }) (identification of most likely non v-structures), and minimizes the conditional 2-point information I(X; Y |{U i } ∪ {Z}) (identification of the most likely base-edge), where {U i } ∪ {Z} is a set of variables that possibly separates the pair (X, Y ) at end.

The MI-CMI algorithm

To learn the structure of genetic regulatory networks, [START_REF] Liang | Gene regulatory network reconstruction using conditional mutual information[END_REF] have proposed to rely on both the 2-point and conditional 2-point information in order to evaluate the dependence between variables. The corresponding Mutual Information-Conditional Mutual Information (MI-CMI) algorithm first uses the mutual information to find most of the direct relationships among the variables.

The removal of edges relies on a given mutual information threshold, I th . Then, the conditional mutual information is computed either for fully connected or fully unconnected triples in order to detect indirect regulations, or interactive regulations. Given the threshold I th and the conditional mutual information value, edges can then be added or deleted (see [START_REF] Liang | Gene regulatory network reconstruction using conditional mutual information[END_REF], Algorithm 3).

In [START_REF] Zhao | Inferring connectivity of genetic regulatory networks using information-theoretic criteria[END_REF], the authors have proposed the Direct Connectivity Metric (DCM) [START_REF] Zhao | Inferring connectivity of genetic regulatory networks using information-theoretic criteria[END_REF], based on the 2-point information, I(X; Y ), and the conditional 2-point information, I(X; Y |Z), given a third variable Z. More precisely, the DCM corresponds to a score that simultaneously maximizes the 2-point and the least conditional 2-point information, as,

DCM (X; Y ) = η(X; Y ) = g I(X; Y ), min Z∈V\{X,Y } I(X; Y |Z) (5.16)
Thus, higher DCM values indicate that two variables are more likely to be directly related. Based on this metric, Zhao et al. devised an algorithm (see [START_REF] Zhao | Inferring connectivity of genetic regulatory networks using information-theoretic criteria[END_REF], Simplified Algorithm) that ranks all pairs of variables using the DCM and,

given a specified number of edges e, outputs an undirected network.

By contrast with these two methods, the 3off2 approach successively recomputes the conditional 2-point information of each pair of variables (X, Y ) by iteratively building a set of variables {U i } that possibly separates (X, Y ). This iterative scheme relies on the simultaneous maximization of the 3-point information conditioned on the {U i } set of variables and the minimization of the 2-point information also conditioned on the {U i } set. This scheme allows for a robust discovery of the underlying causal graphical model, without the need for input parameters, such as the number of expected edges or (conditional) 2-point information thresholds.

Hybrid approaches using Mutual Information Tests

This section presents a scoring function for learning Bayesian networks through a search-and-score process that performs statistical independence tests based on the χ 2 distribution in association with the 2-point information [START_REF] De Campos | A scoring function for learning bayesian networks based on mutual information and conditional independence tests[END_REF]].

[ [START_REF] De Campos | A scoring function for learning bayesian networks based on mutual information and conditional independence tests[END_REF] proposed a scoring function based on the (mutual) 2-point information, called the Mutual Information Tests score, defined as:

M IT (G|D) = i=1 pa G (X i ) =∅ 2N I(X i ; pa G (X i )) - s i j=1 χ α,l iσ i (j)
(5.17)

where I(X i ; pa G (X i )) is the 2-point information between each node X i and its parents. The second term of this scoring function is a penalization of the strength of the relationship between each variable and its parents to avoid adding variables that in fact do not belong to the parents set but may still increase the mutual information term.

The penalization term is related to the χ 2 test of independence, with α being the significance level and σ i = {σ i (1), ..., σ i (s i )} being any permutation of the index set (1, ..., s i ) of the parents pa G (X i ) = {X i1 , ..., X is i }. As stated in [START_REF] De Campos | A scoring function for learning bayesian networks based on mutual information and conditional independence tests[END_REF]], the decomposition of the 2-point information I(X i ; pa G (X i )), when including the parents one by one is not unique and this order impacts the value of the penalizing term. However, if the permutation of pa G (X i ) satisfies

r iσ i (1) ≥ r iσ i (2) ≥ ... ≥ r iσ i (s i ) 1
, where r ij represents the number of possible configurations when the parent set of X i is restricted only to X j , then it is guaranteed to produce the maximum penalization (see Theorem 2 [de Campos, 2006]). In this way, the s i ! permutations are no longer necessary. The MIT score can be used within a searchand-score procedure, although it does not evolve in the space of equivalent DAGs, but in a reduced search space of Restricted Acyclic Partially Directed Graphs (RPDAGs) [Acid and [START_REF] Bibliography | Searching for bayesian network structures in the space of restricted acyclic partially directed graphs[END_REF]].

As detailed in Chapter 6, the 3off2 algorithm is not trying to directly identifying the parents of each node by quantifying the strength of the relationships between each node and its parents candidate. Instead, it follows a constraint-based procedure and progressively builds a set of variables {U i } that separates pairs of variables because of their contribution to the observed relationship. This identification of non-essential edge relies on the 2-point and the 3-point information, and on previously introduced network complexity criterion, such as the MDL and the NML scores (Eq. 4.8 & 4.10).

Chapter 6

3off2: a hybrid method based on information statistics

This chapter presents a new hybrid method and its corresponding 3off2 algorithm [Affeldt and[START_REF] Affeldt | Robust reconstruction of causal graphical models based on conditional 2-point and 3-point information[END_REF]. It is directly inspired by the PC anc IC algorithms but relies on a quantitative information theoretic framework to reliably uncover conditional independencies in finite datasets and subsequently orient and propagate edge directions between connected variables.

As detailed in Chapter 3, constraint-based approaches, such as the PC [START_REF] Spirtes | An algorithm for fast recovery of sparse causal graphs[END_REF] and IC [START_REF] Pearl | A theory of inferred causation[END_REF] algorithms, infer causal graphs from observational data, by searching for conditional independencies among variables. Under the Faithfulness and causal sufficiency assumptions, these algorithms return a Complete Partially Directed Acyclic Graph (CPDAG) that represents the Markov equivalent class of the underlying causal structure [START_REF] Pearl | Causality: models, reasoning and inference[END_REF][START_REF] Spirtes | Causation, Prediction, and Search[END_REF]] (section 2.1.2.2). However, as previously stated, the sensitivity of the constraint-based methods to the adjustable significance level α used for the conditional independence tests and to the order in which the variables are processed (Algorithm 1, step 1) favours the accumulation of errors when the search procedure relies on finite observational data.

The 3off2 algorithm aims at improving constraint-based methods by uncovering the most reliable conditional independencies supported by the (finite) available data, based on a quantitative information theoretic framework.

Information theoretic framework

As detailed in section 5.1.4, the maximum likelihood of a graphical model G is related to the cross entropy, H(G, D) = -{X i } p({X i }) log(q({X i })), between the "true" probability distribution p({x i }) from the data D and the approximate probability distribution q({x i }) = i p(x i |{P a x i }) generated by the Bayesian network G. This relation can be used to distinguish v-structure graphical models from non-v-structure graphical models either in isolation (section 6.1.1) or embedded in a larger graph (section 6.1.2).

Inferring isolated v-structures vs non-v-structures

Applying the likelihood definition of a graphical model, Eq. 5.9, to isolated vstructures (Figure 6.1A) and Markov equivalent non-v-structures (Figure 6.1B-D), one obtains,

L v (XY ) = e -N [H(Z|X,Y )+H(X)+H(Y )] = e -N [H(X,Y,Z)+I(X;Y )] (6.1)
where I(X; Y ) = H(X) + H(Y ) -H(X, Y ) is the 2-point mutual information between X and Y (Eq. 5.5), and,

L nv (XY ) = e -N [H(X|Z)+H(Y |Z)+H(Z)] = e -N [H(X,Y,Z)+I(X;Y |Z)]
where I(X; Y |Z) = H(X|Z) + H(Y |Z) -H(X, Y |Z) is the conditional mutual information between X and Y given Z (Eq. 5.6). Hence, one obtains the likelihood ratio,

L v (XY ) L nv (XY ) = e -N [I(X;Y )-I(X;Y |Z)] = e -N I(X;Y ;Z) (6.2)
involving the 3-point information, I(X; Y ; Z) = I(X; Y ) -I(X; Y |Z), Eq. 5.7.

As long recognized in the field [START_REF] Han | Multiple mutual informations and multiple interactions in frequency data[END_REF][START_REF] Mcgill | Multivariate information transmission[END_REF]], 3-point information, I(X; Y ; Z), can be positive or negative (if I(X; Y ) < I(X; Y |Z)), unlike 2-point mutual information, which are always positive, I(X; Y ) 0.

More precisely, Eq. 6.2 demonstrates that the sign and magnitude of 3-point information provide a quantitative estimate of the relative likelihoods of isolated v-structures versus non-v-structures, which are in fact independent of their actual non-connected bases XY , XZ or Y Z,

L v (XY ) L nv (XY ) = L v (XZ) L nv (XZ) = L v (Y Z) L nv (Y Z) = e -N I(X;Y ;Z) (6.3)
Hence, a significantly negative 3-point information, I(X; Y ; Z) < 0, implies that a v-structure is more likely than a non-v-structure given the observed correlation data. Conversely, a significantly positive 3-point information, I(X; Y ; Z) > 0, implies that a non-v-structure model is more likely than a v-structure model.

Yet, Eq. 6.3 shows that, as the 3-point information, I(X; Y ; Z), is symmetric by construction, it cannot indicate how to orient v-structures or non-v-structures over the XY Z triple. To this end, it is however straightforward to show that the most likely base (XY , XZ or Y Z) of the local v-structure or non-v-structure corresponds to the pair with lowest mutual information, e.g., I(X; Y ) = min XY Z I(S; T ) , as shown by the likelihood ratios,

L v (XY ) L v (ST ) = L nv (XY ) L nv (ST ) = e -N I(X;Y ) e -N I(S;T )
Note, in particular, that choosing the base with the lowest mutual information is consistent with the Data Processing Inequality (Theorem 5.1.1) expected for non-v-structures, Figures 6.1B-D.

Hence, combining 3-point and 2-point information allows to determine the likelihood and the base of isolated v-structures versus non-v-structures. But how to extend such simple results to identify local v-structures and non-v-structures embedded within an entire graph G ?

Inferring embedded v-structures vs non-v-structures

To go from isolated to embedded v-structures and non-v-structures within a DAG G, we will consider the markov equivalent CPDAG of G (section 2.1.2.2) and introduce generalized v-structures and non-v-structures, Figures 6.1E-H. We will demonstrate that their relative likelihood, given the available observational data, can be estimated from the sign and magnitude of a conditional 3-point information, I(X; Y ; Z|{U i }), Eq. 6.4. This will extend our initial result valid for isolated vstructures and non-v-structures, Eq. 6.3.

Let's consider a pair of non-neighbor nodes X, Y with a set of upstream nodes {U i } n , where each node U i has at least one direct connection to X (U i → X) or Y

(U i → Y ) or to another upstream node U j ∈ {U i } n (U i → U j ) or only undirected links to these nodes (U i -X, U i -Y or U i -U j ).
Thus, given X, Y and a set of upstream nodes {U i } n , any additional node Z can either be:

• i) at the apex of a generalized v-structure, if all existing connections between X, Y , {U i } n and Z are directed and point towards Z, Figure 6.1E, or else,

• ii) Z has at least one undirected link with X, Y or one of the upstream nodes U i or at least one directed link pointing towards these nodes (Z → X, Then, similarly to the case of an isolated v-structure (Eq. 6.1), the maximum likelihood L v (XY ) of a generalized v-structure pointing towards Z from a base XY with upstream nodes {U i } n can be expressed as,

Z → Y or Z → U i ), Figures 6.1F
L v (XY ) = e -N [H(Z|X,Y,{U i })+H(X|{U i })+H(Y |{U i })+H({U i })] = e -N [H(X,Y,Z,{U i })+I(X;Y |{U i })]
where I(X; Y |{U i }) is the conditional mutual information between X and Y given

{U i }, I(X; Y |{U i }) = H(X|{U i }) + H(Y |{U i }) -H(X, Y |{U i }) -H({U i }).
Likewise, the maximum likelihood L nv (XY ) of a generalized non-v-structure of base xy with upstream nodes {U i } n and Z can be expressed as,

L nv (XY ) = e -N [H(X|Z,{U i })+H(Y |Z,{U i })+H(Z,{U i })] = e -N [H(X,Y,Z,{U i })+I(X;Y |Z,{U i })]
where

I(X; Y |Z, {U i }) = H(X|Z, {U i })+H(Y |Z, {U i })-H(X, Y |Z, {U i })-H(Z, {U i })
is the conditional mutual information between X and Y given Z and {U i }. Hence,

L v (XY ) L nv (XY ) = e -N I(X;Y ;Z|{U i }) (6.4)
in which the conditional 3-point information, I(X; Y ;

Z|{U i }) = I(X; Y |{U i }) - I(X; Y |Z, {U i}), is involved (section 5.1.3).
Hence, a significantly negative conditional 3-point information, I(X; Y ; Z|{U i }) < 0, implies that a generalized v-structure is more likely than a generalized non-vstructure given the available observational data. Conversely, a significantly positive conditional 3-point information, I(X; Y ; Z|{U i }) > 0, implies that a generalized non-v-structure model is more likely than a generalized v-structure model.

Yet, as the conditional 3-point information, I(X; Y ; Z|{U i }), is in fact invariant upon permutations between X, Y and Z (section 5.

L v (XY ) L v (ST ) = L nv (XY ) L nv (ST ) = e -N I(X;Y |{U i }) e -N I(S;T |{U i }) (6.
I(X; Y |Z, {U i }) I(X; Z|{U i }, Y ) I(X; Z|{U i }, Y ) + I(X; {U i }|Y ) I(X; Z, {U i }|Y ) I(X; Y ) I(X; Z, {U i })
where we have used the chain rule, I(X; Z,

{U i }|Y ) = I(X; Z|{U i }, Y )+I(X; {U i }|Y ),

Uncovering causality from a stable/faithful distribution

This section formalizes the previous results on isolated and embedded v-structures and non-v-structures by establishing how the causality can be uncovered from a stable distribution, in the framework of information theory.

Negative 3-point information as evidence of causality

Consider a network G = (V, E) and a stable (or faithful) distribution P (X) over

V , implying that each structural independency (i.e. missing edge XY in G) corresponds to a vanishing conditional 2-point (mutual) information and reciprocally as,

(X ⊥ ⊥ Y |{U i }) G ⇐⇒ (X ⊥ ⊥ Y |{U i }) P (6.6) ⇐⇒ I(X; Y |{U i }) = 0 (6.7)
Eq.6.6 assumes, in particular, that P (X) is a theoretical distribution, defined by a formal expression of its variables X = {X, Y, U 1 , U 2 , . . .}. Note, however, that no such expression is known a priori, in general, and P (X) must typically be estimated from the available data. In principle, an infinite amount of data would be necessary to infer an 'exact' stable distribution P (X) consistent with Eq.6.6.

In the following, we will first assume that such an infinite amount of data is available and distributed as a stable P (X) to establish how causality can be inferred statistically from conditional 2-point and 3-point information. We will then consider the more realistic situation for which P (X) is not known exactly and must be estimated from a finite amount of data.

The definition of the conditional 3-point information given by Eq. 5.8 provides also a generic decomposition of a conditional 2-point (X, Y ) information, I(X; Y |{U i }), by the introduction of a third node Z,

I(X;Y |{U i }) = I(X;Y;Z|{U i }) + I(X;Y |{U i }, Z) (6.8)
As already stipulated in section 5.1.3, Eq.6.8 is always valid, regardless of any assumption on the underlying graphical model and of the amount of data available to estimate conditional 2-point and 3-point information terms. Eq.6.8 will be used to prove the following lemmas and propositions, which trace back the origin of necessary causal relationships in a graphical model to the existence of a negative conditional 3-point information between three variables {X, Y, Z}, I(X; Y ; Z|{U i }) < 0, where {U i } accounts for a structural independency between two of them, e.g.

I(X; Y |{U

i }) = 0 (see Theorem 4). Lemma 1. Given a stable distribution P (X) on V , ∀X, Y ∈ V not adjacent in G, ∃{U i } ⊆ V \{X,Y } s.t. I(X; Y |{U i }) = 0 and ∀Z = X, Y, {U i }, I(X; Y ; Z|{U i }) 0. Proof. If X, Y ∈ V are not adjacent in G, this corresponds to a structural inde- pendency in G, i.e. ∃{U i } ⊆ V \{X,Y } s.t. I(X; Y |{U i }) = 0. Then ∀Z = X, Y, {U i } Eq.6.8 implies I(X; Y ; Z|{U i }) = -I(X; Y |{U i }, Z) 0, as (conditional) mutual
information is always positive. Corollary 2, which is a direct consequence of Equation 6.8 and the positivity of mutual information, will be the basis of the 3off2 causal network reconstruction algorithm, which iteratively "takes off" 3-point information from 2-point information, as I(X; Y |{U i }) -I(X; Y ; Z|{U i }) = I(X; Y |{U i }, Z), and update {U i } ← {U i } + Z as long as there remains some Z ∈ V with (significantly) positive conditional 3-point information I(X; Y ; Z|{U i }) > 0.

Lemma 3 (vanishing conditional 2-point and 3-point information in undirected networks). If G is an undirected (Markov) network, ∀X, Y ∈ V and

∀{U i } ⊆ V \{X,Y } s.t. I(X; Y |{U i }) = 0, then ∀Z = X, Y, {U i }, I(X; Y ; Z|{U i }) = 0. Proof. If G is a Markov network, ∀X, Y ∈ V and ∀{U i } ⊆ V \{X,Y } s.t. I(X; Y |{U i }) = 0, then ∀Z = X, Y, {U i }, I(X; Y |{U i }, Z) = 0 as conditioning observation cannot
induce correlations in Markov networks [START_REF] Koller | Probabilistic Graphical Models: Principles and Techniques[END_REF]. This implies that I(X; Y ; Z|{U i }) = 0 through Eq.6.8.

Note, however, that the converse of Lemma 3 is not true. Namely, (partially)

directed networks can also have vanishing conditional 3-point information associated to all their structural independencies. In particular, tree-like bayesian networks without colliders (i.e. without v-structures, X → Z ← Y ) present only vanishing 3-point information associated to their structural independencies, i.e. I(X; Y ;

Z|{U i }) = 0, ∀X, Y, Z, {U i } ∈ V s.t. I(X; Y |{U i }) = 0. However, such
a directed network must be Markov equivalent to an undirected network corresponding to the same structural independencies but lacking any trace of causal relationships (i.e. no directed edges). The probability distributions faithful to such directed networks do not contain evidence of obligate causality; i.e no directed edges can be unambiguously oriented.

The following Theorem 4 establishes the existence of negative conditional 3-point information as statistical evidence of obligate causality in graphical models. For the purpose of generality in this section, we do not exclude the possibility that unobserved 'latent' variables might mediate the causal relationships among observed variables. However, this requires dissociating the labelling of the two endpoints of each edges. Let us first recall the three different endpoint marks associated to such edges in mixed graphs: they are the tail (-), the head (>) and the unspecified (•) endpoint marks. In addition, we will use the asterisk symbol ( * ) as a wild card denoting any of the three marks (see section 2.2 for further details on mixed and ancestral graphs). Proof. Theorem 4 is the contrapositive of Lemma 3, with the additional use of Lemma 1.

Proposition 5 (origin of causality at unshielded triples with negative conditional 3-point information). for all unshielded triple, X * -

• Z •- * Y , ∃{U i } ⊆ V \{X,Y } s.t. I(X; Y |{U i }) = 0, if Z / ∈ {U i } then I(X; Y ; Z|{U i }) < 0 and
the unshielded triple should be oriented as Following the rationale of constraint-based approaches, it is then possible to 'propagate' further the orientations downstream of colliders, through positive (conditional) 3-point information, if one assumes that the underlying distribution P (X) is faithful to an ancestral graph G on V . An ancestral graph is a mixed graph, that is, with three types of edges, undirected (-), directed (← or →) or bidirectional (↔), but with i.) no directed cycle, ii.) no almost directed cycle (including one bidirectional edge) and iii.) no undirected edge with incoming arrowhead (such as X * → Z -Y ). In particular, Directed Acyclic Graphs (DAG) are subclasses of ancestral graphs (i.e. without undirected nor bidirectional edges).

X * → Z ← * Y Proof. if I(X; Y |{U i }) = 0 with Z / ∈ {U i },
Proposition 6 ('propagation' of causality at unshielded triples with positive conditional 3-pt information). Given a distribution P (X) faithful to an ancestral graph G on V , for all unshielded triple with already one converging ori-

entation, X * → Z •- * Y , ∃{U i } ⊂ V \{X,Y } s.t. I(X; Y |{U i }, Z) = 0, if Z ∈ {U i }
then I(X; Y ; Z|{U i }) > 0 and the first orientation should be 'propagated' to the second edge as X * → Z → Y .

Proof. If I(X; Y |{U i }) = 0 with Z ∈ {U i }, the unshielded triple cannot be a collider and, since G is assumed to be an ancestral graph, the edge Z -Y cannot be an undirected edge either. Hence, it has to be a directed edge, Z → Y and I(X; Y ; Z|{U i } \Z ) > 0 by faithfulness and Eq. 6.8

Note that the propagation rule of Proposition 5 (i.) can be applied iteratively to successive unshielded triples corresponding to positive conditional 3-point in-

formation. Yet, all arrowhead orientations can be ultimately traced back to a negative conditional 3-point information, Theorem 4 and Proposition 5.

6.2 Robust reconstruction of causal graphs from finite datasets

Finite size corrections of maximum likelihood

We now turn to the more practically relevant situation of finite datasets consisting of N independent data points. The associated sampling noise will instrinsically limit the accuracy of causal network reconstruction. In particular, conditional independencies cannot be exactly achieved (I(X; Y |{U i }) = 0) but can be reliably established using statistical criteria that depend on the number of data points N .

Given N independent datapoints from the available data D, let us recall the maximum likelihood, L D|G , that they might have been generated by the graphical model G [START_REF] Sanov | On the probability of large deviations of random variables[END_REF],

L D|G = e -N H(G,D) Z(G, D) = e N {x i } p({x i }) log(q({x i })) Z(G, D) (6.9)
where H(G, D) = -{x i } p({x i }) log(q({x i })) is the cross entropy between the "true" probability distribution p({x i }) of the data D and the theoretical probability distribution q({x i }) of the model G and Z(G, D) is a data-and model-dependent factor ensuring proper normalization condition. The structural constraints of the model G can be included a priori in the factorization form of the theoretical probability distribution, q({x i }).

In particular, if we assume a Bayesian network as underlying graphical model, q({x i }) factorizes as q({x i }) = i p(x i |{pa x i }), where {pa x i } denote the values of the parents of node X i , {Pa X i }, and leads to the following maximum likelihood expression,

L D|G = e -N i H(X i |{Pa X i }) Z(G, D) (6.10)
The model G can then be compared to the alternative model G \X→Y with one additional missing edge X → Y using the maximum likelihood ratio,

L D|G \X→Y L D|G = e -N I(X;Y |{Pa Y } \X ) Z(G, D) Z(G \X→Y , D) (6.11) where I(X; Y |{Pa Y } \X ) = H(Y |{Pa Y } \X ) -H(Y |{Pa Y })
. However, Eq.6.11 cannot be used as such to learn the underlying graphical model, as it assumes that the order between the nodes and their parents is already known (see however [START_REF] De Campos | A scoring function for learning bayesian networks based on mutual information and conditional independence tests[END_REF]). Yet, following the rationale of constraint-based approaches, Eq.6.11 can be reformulated by replacing the parent nodes with an unknown separation set {U i } to be learnt simultaneously with the missing edge candidate XY ,

L G \XY |{U i } L G = e -N I(X;Y |{U i })+k X;Y |{U i } (6.12) k X;Y |{U i } = log Z(G, D)/Z(G \XY |{U i } , D)
where the factor k X;Y |{U i } > 0 tends to limit the complexity of the models by favoring fewer edges. Namely, the condition,

I(X; Y |{U i }) < k X;Y |{U i } /N , implies
that simpler models compatible with the structural independency,

X ⊥ ⊥ Y |{U i },
are more likely than model G, given the finite available dataset. This replaces the 'perfect' conditional independency condition, I(X; Y |{U i }) = 0, valid in the limit of an infinite dataset, N → ∞.

A common complexity criterion in model selection is the Bayesian Information Criterion (BIC) or Minimal Description Length (MDL) criterion [Hansen andYu, 2001, Rissanen, 1978],

k MDL X;Y |{U i } = 1 2 (r x -1)(r y -1) i r u i log N (6.13)
where r x , r y and r u i are the number of levels of the corresponding variables (see also section 4.2.2.2). The MDL complexity, Eq.6.13, is simply related to the normalisation constant reached in the asymptotic limit of a large dataset N → ∞ (Laplace approximation). However, this limit distribution is only reached for very large datasets in practice.

Alternatively, the normalisation of the maximum likelihood can also be done over all possible datasets including the same number of data points to yield a (universal)

Normalized Maximum Likelihood (NML) criterion [START_REF] Rissanen | Modeling by shortest data description[END_REF]Tabus, 2005, Shtarkov, 1987] (see also section 4.2.2.2) and its decomposable [START_REF] Kontkanen | Computationally efficient methods for mdl-optimal density estimation and data clustering[END_REF]Myllymäki, 2007, Roos et al., 2008a] and

XY -symmetric version, k NML X;Y |{U i } , defined in section 6.2.2.
Then, instead of exploring the combinatorics of sepset composition {U i } for each missing edge candidate XY as in traditional constraint-based approaches, we propose that Eq.6.12 can be used to iteratively extend a likely sepset using the maximum likelihood ratios between two successive sepset candidates, i.e. between the already ascertained {U i } and the possible extended {U i } + Z, as, 6.14) using Eq.6.8 for I(X; Y ; Z|{U i }) and introducing a similar 3-point complexity conditioned on {U i } as,

L D|G \XY |{U i },Z L D|G \XY |{U i } = e N I(X;Y ;Z|{U i })+k X;Y ;Z|{U i } (
k X;Y ;Z|{U i } = k X;Y |{U i },Z -k X;Y |{U i }
where k X;Y ;Z|{U i } 0, unlike 3-point information, I(X; Y ; Z|{U i }) which can be positive or negative.

Introducing also the shifted 2-point and 3-point information for finite datasets as, 6.16) leads to maximum likelihood ratios equivalent to Eqs.6.12 and 6.14,

I (X; Y |{U i }) = I(X; Y |{U i }) - k X;Y |{U i } N (6.15) I (X; Y ; Z|{U i }) = I(X; Y ; Z|{U i }) + k X;Y ;Z|{U i } N (
L D|G \XY |{U i } L D|G = e -N I (X;Y |{U i }) (6.17) L D|G \XY |{U i },Z L D|G \XY |{U i } = e N I (X;Y ;Z|{U i }) (6.18)
As will become apparent in the following discussion, learning, iteratively, the most likely edge to be removed XY and its corresponding separation set {U i } will imply to simultaneously minimize 2-point information (Eq.6.17) while maximizing 3-point information (Eq.6.18).

Complexity of graphical models

This section gives further details on the complexities used by the 3off2 structure discovery approach and introduced in the section 4.2.2.

As previously stated (Eq.6.10 & 6.12), the complexity k G,D of a graphical model is related to the normalization constant Z(G, D) of its maximum likelihood as

k G,D = log Z(G, D), L G = e -N H(G,D) Z(G, D) = e -N H(G,D)-k G,D
For Bayesian networks with decomposable entropy, i.e.

H(G, D) = i H(X i |{Pa X i }), it is convenient to use decomposable complexities, k G,D = i k X i |{Pa X i } , L G = e -N i H(X i |{Pa X i })-i k X i |{Pa X i }
such that the comparison between alternative models G and G \X→Y (i.e. G with one missing edge X → Y ) leads to a simple local increment of the score,

L G \X→Y L G = e -N I(X;Y |{Pa Y } \X )+∆k Y |{Pa Y } \X (6.19) I(X; Y |{Pa Y } \X ) = H(Y |{Pa Y } \X ) -H(Y |{Pa Y }) 0 ∆k Y |{Pa Y } \X = k Y |{Pa Y } -k Y |{Pa Y } \X 0
As introduced in section 4.2.2, a common complexity criterion in model selection is the Bayesian Information Criterion (BIC) or Minimal Description Length (MDL)

criterion [Hansen andYu, 2001, Rissanen, 1978],

k MDL Y |{Pa Y } = 1 2 (r y -1) Pa Y j r j log N (6.20) ∆k MDL Y |{Pa Y } \X = 1 2 (r x -1)(r y -1)
Pay \X j r j log N (6.21)

where r x , r y and r j are the number of levels of each variable, x, y and j. The MDL complexity, Eq.6.20, is simply related to the normalisation constant reached in the asymptotic limit of a large dataset N → ∞ (Laplace approximation). The MDL complexity can also be derived from the Stirling approximation on the Bayesian measure [START_REF] Bouckaert | Probabilistic network construction using the minimum description length principle[END_REF][START_REF] Schwarz | Estimating the dimension of a model[END_REF]]. Yet, in practice, this limit distribution is only reached for very large datasets, as some of the least-likely (r y -1) j r j combinations of states of variables are in fact rarely (if ever) sampled in typical finite datasets. As a result, the MDL complexity criterion tends to underestimate the relevance of edges connecting variables with many levels, r i , leading to the removal of false negative edges.

To avoid such biases with finite datasets, the normalisation of the maximum likelihood can be done over all possible datasets with the same number N of data points.

This corresponds to the (universal) Normalized Maximum Likelihood (NML, introduced in section 4.2.2) criterion [START_REF] Kontkanen | A linear-time algorithm for computing the multinomial stochastic complexity[END_REF][START_REF] Rissanen | Kolmogorov's structure function in mdl theory and lossy data compression[END_REF], Roos et al., 2008a[START_REF] Shtarkov | Universal sequential coding of single messages[END_REF],

L G = e -N H(G,D) |D |=N e -N H(G,D ) = e -N H(G,D)-k NML G,D (6.22)
We introduce here the factorized version of the NML criterion [START_REF] Kontkanen | Computationally efficient methods for mdl-optimal density estimation and data clustering[END_REF]Myllymäki, 2007, Roos et al., 2008a] which corresponds to a decomposable NML score, where N yj is the number of data points corresponding to the jth state of the parents of Y , {Pa Y }, and N yj the number of data points corresponding to the j th state of the parents of Y , excluding X, {Pa Y } \X . Hence, the factorized NML score for each node X i corresponds to a separate normalisation for each state j = 1, ..., q i of its parents and involving exactly N ij data points of the finite dataset,

k NML G,D = X i k NML X i |{Pa X i } ,
L G = e -N i H(X i |{Pa X i })-i q i j log C r i N ij = e N i q i j r i k N ijk N log N ijk N ij -i q i j log C r i N ij = i q i j r i k N ijk N ij N ijk C r i N ij
where N ijk corresponds to the number of data points for which the ith node is in its kth state and its parents in their jth state, with

N ij = r i k N ijk . The universal normalization constant C r
n is then obtained by averaging over all possible partitions of the n data points into a maximum of r subsets,

1 + 2 +• • •+ r = n with k 0, C r n = 1 + 2 +•••+ r =n n! 1 ! 2 ! • • • r ! r k=1 k n k (6.25)
which can in fact be computed in linear-time using the following recursion [START_REF] Kontkanen | A linear-time algorithm for computing the multinomial stochastic complexity[END_REF],

C r n = C r-1 n + n r -2 C r-2 n (6.26)
with C r 0 = 1 for all r, C 1 n = 1 for all n and applying the general formula Eq.6.25 for r = 2, 6.27) or its Szpankowski approximation for large n (needed for n > 1000 in practice) [START_REF] Kontkanen | Computationally efficient methods for mdl-optimal density estimation and data clustering[END_REF][START_REF] Kontkanen | Efficient computation of stochastic complexity[END_REF][START_REF] Szpankowski | TCGA. Comprehensive molecular portraits of human breast tumours[END_REF],

C 2 n = n h=0 n h h n h n -h n n-h (
C 2 n = nπ 2 1 + 2 3 2 nπ + 1 12n + O 1 n 3/2 (6.28) nπ 2 exp 8 9nπ
+ 3π -16 36nπ (6.29)

Probability estimate of indirect contributions

The previous results enable us to estimate the probability of a node Z to contribute to the conditional mutual information I(X; Y |{U i }), by combining the probability, P nv (XY Z|{U i }), that the triple XY Z is a generalized non-v-structure conditioned on {U i } and the probability, P b (XY |{U i }), that its base is XY , where,

P nv (XY Z|{U i }) = L nv (XY ) L nv (XY ) + L v (XY ) P b (XY |{U i }) = L nv (XY ) L nv (XY ) + L nv (XZ) + L nv (Y Z)
that is, using Eqs. 6.4 & 6.5 and including finite size corrections of the maximum likelihoods (i.e. shifted information from Eqs. 6.16 & 6.15),

P nv (XY Z|{U i }) = 1 1 + e -N I (X;Y ;Z|{U i }) (6.30) P b (XY |{U i }) = 1 1 + e -N I (X;Z|{U i }) e -N I (X;Y |{U i }) + e -N I (Y ;Z|{U i }) e -N I (X;Y |{U i }) (6.
i }) = min P nv (XY Z|{U i }), P b (XY |{U i })
The pair of nodes XY with the most likely contribution from a third node Z can then be ordered according to their rank R(XY ; Z|{U i }) defined as,

R(XY ; Z|{U i }) = max Z P lb (Z; XY |{U i }) (6.32)
and Z can be iteratively added to the set of contributing nodes (i.e. {U i } ← {U i } + Z) of the top link XY = argmax XY R(XY ; Z|{U i }) to progressively recover the most significant indirect contributions to all pairwise mutual information in a causal graph, as outlined below.

6.3 The 3off2 scheme

Robust inference of conditional independencies

The previous results can be used to provide a robust inference method to identify conditional independencies and, hence, reconstruct the skeleton of underlying causal graphs from finite available observational data. The approach follows the spirit of constraint-based methods, such as the PC or IC algorithms, but recovers conditional independencies following an evolving ranking of the network edges, R(XY ; Z|{U i }), defined in Eq. 6.32.

Then, Z can be iteratively added to the set of contributing nodes (i.e. {U i } ← {U i }+Z) of the top edge XY = argmax XY R(XY ; Z|{U i }) to progressively recover the most significant indirect contributions to all pairwise mutual information in a causal graph.

Implementing this local optimization scheme, the 3off2 algorithm eventually learns the network skeleton by collecting the nodes of the separation sets one-by-one, instead of exploring the full combinatorics of sepset composition without any likelihood guidance. Indeed, the 3off2 scheme amounts to identify {U i } by "taking off" iteratively the "most likely" conditional 3-point information from each 2-point information as,

I(X; Y |{U i } n ) = I(X; Y ) -I(X; Y ; U 1 ) -I(X; Y ; U 2 |U 1 ) -• • • -I(X; Y ; U n |{U i } n-1 )
or equivalently between the shifted 2-point and 3-point information terms, defined in Eqs. 6.15 and 6.16 respectively, 6.33) This leads to the Algorithm 4 for the reconstruction of the graph skeleton using the 3off2 scheme. Note, in particular, that the 3off2 scheme to reconstruct graph skeleton is solely based on identifying structural independencies, which can also be applied to graphical models for undirected Markov networks.

I (X; Y |{U i } n ) = I (X; Y ) -I (X; Y ; U 1 ) -I (X; Y ; U 2 |U 1 ) -• • • -I (X; Y ; U n |{U i } n-1 ) (
i }) do expand contributing set {U i } ← {U i } + Z if I(X; Y |{U i }) < k X;Y |{U i } /N i.e. I (X; Y |{U i }) < 0 then XY link is non-essential
L c = { X, Z, Y X Y }, in decreasing order of |I (X; Y ; Z|{U i })| repeat Take X, Z, Y X Y ∈ L c with highest |I (X; Y ; Z|{U i })| on which R 0 or R 1 orientation rule can be applied if I X; Y ; Z|{U i }) < 0 then if X, Z, Y X Y has no diverging orientation, apply R 0 : {X - * Z * -Y & X Y & Z / ∈ Sep XY } ⇒ {X → Z ← Y } else if X, Z, Y X Y has one converging orientation, apply R 1 : {X → Z -Y & X Y } ⇒ {Z → Y } end Apply new orientation(s) to all other X , Z , Y X Y ∈ L c
until no additional orientation can be obtained ;

The 3off2 algorithm

The 3off2 scheme can be used to devise a two-step algorithm (see Algorithm ( 4)), inspired by constraint-based approaches, to first reconstruct network skeleton (Algorithm ( 4), step 1) before combining orientation and propagation of edges in a single step based on likelihood ratios (Algorithm (4), step 2).

Reconstruction of network skeleton

The 3off2 scheme will first be applied to iteratively remove edges with maximum positive contributions, I (X; Y ; U k |{U i } k-1 ) > 0, corresponding to the most likely generalized non-v-structures (Eq. 6.4), while minimizing simultaneously the remaining 2-point information, I (X; Y |{U i } k ) (Eq. 6.5), consistently with the data processing inequality.

Such 3off2 scheme (Algorithm ( 4), step 1) will therefore progressively lower the conditional 2-point information terms,

I (X; Y ) > • • • > I (X; Y |{U i } k-1 ) > I (X; Y |{U i } k )
and might ultimately result in the removal of the corresponding edge, XY , but only when a structural independency is actually found, i.e. I (X; Y |{U i } n ) < 0, as in constraint-based algorithms for a given significance level α. Yet, the skeleton obtained with the 3off2 scoring approach is expected to be more robust to finite observational data than the skeleton obtained with PC or IC algorithms, as the former results only from statistically significant 3-point contribu-

tions, I (X; Y ; U k |{U i } k-1 ) > 0, based on their quantitative 3off2 ranks, R(XY ; U k |{U i } k-1 ).
The best results on benchmark networks using these quantitative 3off2 ranks are obtained with the NML score. The MDL score leads to equivalent results, as expected, in the limit of very large datasets (Chapter 7). However, with smaller datasets, the most reliable results with the MDL score are obtained using non-shifted instead of shifted 2-point and 3-point information terms in the 3off2 rank of individual edges, Eq.6.32. This is because the MDL complexity tends to underestimate the importance of edges between nodes with many levels (section 6.2.2). For finite datasets, it easily leads to spurious conditional independencies, I (X; Y |U i) < 0, when using shifted 2-point and 3-point information, Eq.6.33, whereas using non-shifted information in the 3off2 ranks (Eq.6.32) tends to limit the number of false negatives as early errors in {U i } can only increase I(X; Y |U i) 0, in the end.

Orientation of network skeleton

The skeleton and the separation sets resulting from the 3off2 iteration step (Algorithm ( 4), step 1) can then be used to orient the edges and to propagate orientations to the unshielded triples. However, while the constraint-based methods distinguish the v-structures orientation step (Algorithm (1), step 2) from the propagation procedure (Algorithm 1, step 3), the 3off2 algorithm intertwines these two steps based on the respective likelihood scores of individual v-structures and nonv-structures (Algorithm ( 4), step 2).

As stated earlier, the magnitude and sign of the conditional 3-point information, I(X; Y ; Z|{U i }) (or equivalently the shifted 3-point information, Eq. 6.4), indicate if a non v-structure is more likely than a v-structure. Hence, all the unshielded triples can be ranked by the absolute value of their conditional 3-point information, that is, in decreasing order of their likelihood of being either a v-structure or a non-v-structure.

As detailed in the step 2 of Algorithm ( 4 Note that we only implement the R 0 and R 1 propagation rules, which are applied in decreasing order of likelihood. In particular, we do not consider propagation rules R 2 and R 3 which are not associated to likelihood scores but enforce the hypothesis of acyclic constraint.

As for the 3off2 skeleton reconstruction, the orientation/propagation step of 3off2 allows for a robust discovery of orientations from finite observational data as it relies on statistically significant conditional 3-point information, I (X; Y ; Z|{U i }).

During this step, 3off2 recovers and propagates as many orientations as possible in an iterative procedure following the quantitative ranks of the unshielded triples based on their conditional 3-point information.

6.4 Extension of the 3off2 algorithm

G on V , ∀X, Y, Z ∈ V and ∀{U i } ⊆ V s.t. I (X; Y |{U i }) < 0 (i.e. no XY edge) and I (X; Y ; Z|{U i }) < 0 then, i. if X, Y, Z form an unshielded triple, X •-• Z • -• Y , then it should be oriented as X → Z ← Y , with probabilities, P • X→Z = P • Y →Z = 1 + e N I (X;Y ;Z|{U i }) 1 + 3e N I (X;Y ;Z|{U i })
ii. similarly, if X, Y, Z form an unshielded triple, with one already known converging arrow, X → Z •-• Y , with probability P X→Z > P • X→Z , then the second edge should be oriented to form a v-structure, X → Z ← Y , with probability,

P Y →Z = P X→Z 1 1 + e N I (X;Y ;Z|{U i }) - 1 2 + 1 2
Proof. The implications (i.) and (ii.) rely on Eq.6.30 to estimate the probability that the two edges form a collider (P v (XY Z|{U i }) = 1-P nv (XY Z|{U i })). We start proving (ii.) using the probability decomposition formula:

P Y →Z = P X→Z P X→Z←Y P X→Z←Y + P X→Z→Y + (1 -P X→Z ) P X←Z←Y P X←Z←Y + P X←Z→Y = P X→Z 1 1 + e N I (X;Y ;Z|{U i }) - 1 2 + 1 2
which also leads to (i.) if one assumes P X→Z = P Y →Z by symmetry in absence of prior information on these orientations.

Following the rationale of constraint-based approaches, it is then possible to 'propagate' further the orientations downstream of colliders, using Eq.6.30 for positive (conditional) 3-point information. For simplicity and consistency, we only implement the propagation of orientation based on likelihood ratios, which can be quantified for finite datasets as proposed in the following Proposition 8. In particular, we do not extend the propagation rules [Meek, 1995] to inforce acyclic constraints that are necessary to have a complete reconstruction of the Markov equivalent class of the underlying DAG model.

Proposition 8 (robust 'propagation' of causality at unshielded triples with significantly positive conditional 3-pt information).

Assuming that the underlying graphical model is a DAG G on V , ∀X, Y, Z ∈ V and

∀{U i } ⊆ V \{X,
P Z→Y = P X→Z 1 1 + e -N I (X;Y ;Z|{U i }) - 1 2 + 1 2
Proof. This results is shown using the probability decomposition formula,

P Z→Y = P X→Z P X→Z→Y P X→Z←Y + P X→Z→Y + (1 -P X→Z ) P X←Z→Y P X←Z←Y + P X←Z→Y = P X→Z 1 1 + e -N I (X;Y ;Z|{U i }) - 1 2 + 1 2
Proposition 7 and Proposition 8 lead to the following Algorithm 5 for the orientation of unshielded triples of the graph skeleton obtained from Algorithm 4.

Algorithm 5: 3off2 probabilistic Orientation / Propagation Step

In: Graph skeleton from Algorithm 4 (step 1) and corresponding conditional 3-point information I (X; Y ; Z|{U i }).

Out: Partially oriented causal graph G with edge orientation probabilities.

3off2 probabilistic Orientation / Propagation Step sort list of unshielded triples, L c = { X, Z, Y X Y }, in decreasing order of their orientation/propagation probability initialized at 1/2 and computed from: until no additional orient./propa. probability > 1/2;

-(i.) Proposition 7, if I (X; Y ; Z|{U i }) < 0, or -(ii.) Proposition 8, if I (X; Y ; Z|{U i }) > 0 repeat Take X, Z, Y X Y ∈ L c with

Allowing for latent variables

Although constraint-based methods have benefited from the recent improvements of the PC algorithm (see PC-stable version in section 3.2.1), learning causal graphical models from observational data generated by complex biological systems frequently requires to allow for the presence of unobserved latent variables. As introduced in section 3.2.2, several constraint-based methods, such as the FCI ('Fast Causal Inference') [START_REF] Spirtes | An algorithm for causal inference in the presence of latent variables and selection bias[END_REF][START_REF] Spirtes | Causation, Prediction, and Search[END_REF], the RFCI ('Really Fast Causal Inference') [START_REF] Colombo | Learning highdimensional directed acyclic graphs with latent and selection variables[END_REF] or the AnytimeFCI [START_REF] Spirtes | An anytime algorithm for causal inference[END_REF] algorithms can learn a PAG (Partial Ancestral Graph, see definition 2.2.5) over the observed variables while taking into account the presence of latent variables. As exemplified in Fig. 3.1 section 3.2.2.1, when the causal sufficiency assumption does not hold, one should also consider nodes that do not belong to the neighbourhood of X or Y as potential members of the separation set, Sep XY , of this two variables. Indeed, in the example given in Fig. 3.1, the presence of the two latent variables, L 1 and L 2 , implies that X 3 should be added to the separation set of the pair (X 1 , X 5 ) in order to enable the removal of the edge X 1 -X 5 , as done in the PAG of the true underlying causal network.

By contrast, the 3off2 algorithm enables a simple extension of the search of the possible contributors to a pair of variables (X, Y ) including variables that do not belong to the neighbourhood of X or Y . This is done through a simple modification of lines 11 and 22 in the Algorithm 4, by setting the 3off2 approach to allow the search of Z beyond the neighbourhood of X or Y . This modification implies to check a greater number of nodes when progressively building the separation set.

Yet, as no combinatorial search of contributing nodes is done in the 3off2 approach, the iterative search of the most likely contributing node can be done beyond the first neighbours without significant computational burden (and thus avoiding the multiple steps of the FCI or related algorithms and the use of 'inducing paths' to uncover likely non neighbours contributing nodes).

As shown in Chapter 8, the 3off2 approach could be used to reconstruct networks of genomic properties suspected to have played a role in the evolution of the human genome and its enhanced susceptibility to genetic disease and cancer. In particular, the networks learnt with the 3off2 method support our non-adaptive evolutionary model which posits that the retention of copies of genes arising from whole-genome duplication events is primarily the consequence of their susceptibility to dominant deleterious mutations [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF]. However, many other properties may have participated in the fate of duplicated genes, and some of them are still unknown. For instance, Fig. 8.4 exhibits a bi-directed edge between the nodes labelled 'Essential' and 'Ka.Ks', which suggests that there remains hidden latent cause(s) beyond those included in the model (disease, SSD...) to relate a gene essentiality to its tendency to be (strongly) conserved. Another example is given in Chapter 9, where the 3off2 method is used to reconstruct networks corresponding to 2D layers of a zebrafish brain, corresponding to different depths of the brain. As neural regions usually span over the three dimensions, using the 3off2 method while allowing for the presence of latent variables, i.e. neurons belonging to the upper or lower layers, can prevent us from drawing spurious cause-effect relationships within single layers. Finally, Chapter 11, dedicated to the reconstruction of mutational pathways in breast cancer, also shows the importance of assuming the presence of latent variables. Indeed, the restricted number of samples does not allow to take into account thousands of genes in the network reconstructions. Thus, based on specific biological parameters, one has to define a set of genes suspected to participate in the tumour progression, which leads to ignore genes that may play a role in the emergence and proliferation of malignant cells.

In particular, as shown in Fig. 11.1, the relationship between the genes CDH1 and ERBB2 may not be a cause-effect interaction as suspected in recent studies, but the consequence of hidden common cause(s).

Allowing for missing data

Researchers need frequently to deal with missing values when using data generated from real-life systems. Missing data can have diverse sources such as mistakes during the records or missing answer to a survey. Methods such as Bayesian search-and-score approaches which require a global score need to rely on complete information without missing data. When it is not the case, the missing data need to be somehow generated from the available data with obvious statistical issues.

Filling the missing information could be done for instance using a simple mean value over the available data, or a more complex statistical model. However, if there is no bias in the missing data, i.e. if the data are missing completely at random (MCAR, [START_REF] Rubin | Inference and missing data[END_REF]), one can assume that the available information is still representative of the population. In this case, the counterpart of relying only on the available information is that all samples will not contribute equally to the model with concomitant variations in statistical precision of different subnetworks reconstructed from the available data.

The 3off2 approach provides a way to rely only on the available information and estimate local confidence of the reconstructed network. As the 3off2 method only requires the calculation of I(X; Y |{U i }) and I(X; Y ; Z|{U i }) at each iteration, the (sub)dataset considered needs to be complete solely for the variables X, Y, Z, {U i } involved in the calculation of these terms.

As exemplified in Chapters 8 and 11, this possibility to deal with missing data with the 3off2 method is shown to be crucial in the network reconstruction of genomic properties and mutational pathways. For the application detailed in Chapter 11, the gene network reconstructions rely on online databases such as COSMIC (Catalogue Of Somatic Mutations In Cancer, [START_REF] Forbes | The catalogue of somatic mutations in cancer (COSMIC)[END_REF]). Although the COSMIC database contains over a million of samples, only about 1% of these samples correspond to whole-exome sequencing. Indeed, most samples were obtained before the recent advent of NGS technologies. Besides, for a specific cancer type, the number of samples can drop to only a few tens. As illustrated in Table 6.1, if one decide to rely solely on complete information, the only sample available would be samp 4 (i.e. 1% of the dataset). However, genes g 3 and g 7 have in fact been simultaneously sequenced in 3 samples (samp 1 , samp 3 and samp 4 ), and genes g 2 and g 7 in 2 samples (samp 1 and samp 4 ). The 3off2 approach is taking the most out of incomplete datasets by allowing the computations to be performed only on the partially complete samples relative to the restricted set of variables considered (i.e. X, Y, Z and {U i }).

6.5 Implemented pipelines

The 3off2 pipeline

The 3off2 reconstruction method has been implemented mainly in R, and partly in C. Fig. 6.2 gives an overview of the pipeline through which evaluations of the 3off2 approach have been performed. This pipeline takes as compulsory inputs (i) a dataset containing N samples of p variables (discretized data) and (ii) an output directory path. The complexity can follow an MDL criterion, rather than the default NML criterion (see section 6.2.2 for details on MDL and NML graph complexities). If the samples of the input dataset are correlated, and if the number of independent samples is known (effective sample size, N ef f ), this information can also be provided to the pipeline (see section 7.3, related to the reconstruction of undirected networks, that gives further details and examples for this input argument). The parameter η is an integer (default value 1) that modifies the severity of the removal decision (higher values inducing more strongly biased decision towards removal). This amounts to shift the likelihood ratio Eq. 6.12 as, 6.34) which corresponds to redefining the two-point complexities as, As detailed in section 6.3.2, the 3off2 method first learns the skeleton of the graphical model ('3off2 skeleton' module), and then orients the edges ('3off2 orientation' module). These two steps provide the list of the discovered edges, as well as their conditional mutual information (I(X;

L G \XY |{U i } L G = ηe -N I(X;Y |{U i })+k X;Y |{U i } (
k X;Y |{U i } = k X;Y |{U i } + ln η
Y |{U i })), their set of contributors ({U i }),
the local complexity (associated with {X, Y, {U i }}), the number of samples on which relies the conditional mutual information (I(X; Y |{U i })), the orientation of the edge and the sign of the associated effect.

When the true underlying graphical model is known, it can be provided to the 'Graph summary' module that computes the number of true positive (TP), false positive (FP) and false negative (FN) edges (possibly adjusted with the number of edges wrongly oriented). The 'Graph summary' module also provides the associated Precision (Prec.), Recall (Rec.) and F-score (Fscore) measures. Finally, the 'Graph plot(s)' module, that can also take as input the nodes layout, generates several color coded plots. The edge color could either represent the confidence for this edge (i.e. the difference between its conditional mutual information and the local complexity), or the strength and sign of the total effect associated with this relationship (see Chapters 8, 9 or 11 for color coded plot examples).

The discoNet pipeline

The 3off2 reconstruction method has been evaluated against alternative approaches, namely the PC-stable algorithm [Colombo andMaathuis, 2014, Spirtes and[START_REF] Spirtes | An algorithm for fast recovery of sparse causal graphs[END_REF]), a Bayesian hill-climbing heuristics with random restarts [START_REF] Chickering | Learning Bayesian networks: Search methods and experimental results[END_REF] and Aracne, an information-based reconstruction method [START_REF] Margolin | ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context[END_REF] (evaluation results are given in Chapter 7 and Appendices A & B).

To facilitate batch evaluations, all these alternative methods and the 3off2 reconstruction approach have been encapsulated in the discoNet pipeline (Fig. 6.3). Parameters related to each method can be given as input arguments. For the PC algorithm, one can choose to run either the original [START_REF] Spirtes | An algorithm for fast recovery of sparse causal graphs[END_REF] or the stable version of the algorithm [START_REF] Colombo | Order-independent constraint-based causal structure learning[END_REF]. The significance level α can also be provided, as well as the orientation rule (i.e. conservative [START_REF] Ramsey | Adjacency-faithfulness and conservative causal inference[END_REF] or majority [START_REF] Colombo | Order-independent constraint-based causal structure learning[END_REF]). The implementation of the PC algorithm corresponds to the pcalg R package [Kalisch andBühlmann, 2008, Kalisch et al., 2012]. When using the hill-climbing approach, one can decide for the scoring function (BIC, Bayesian Information Criterion; BDe, Bayesian Dirichtlet equivalent; AIC, Akaike Information Criterion) and the number of random restarts. The implementation of the Bayesian hill-climbing heuristics in the discoNet pipeline corresponds to the bnlearn R package [START_REF] Scutari | Learning Bayesian Networks with the bnlearn R Package[END_REF]. Finally, the Aracne method can take into account a parameter to relax the removal of edges. The implementation of the Aracne method corresponds to the minet R package [START_REF] Meyer | minet: A R/Bioconductor package for inferring large transcriptional networks using mutual information[END_REF]. The output files of these methods can then be given to the shared modules 'Graph summary' and 'Graph plot(s)', that provide comparable results.

Chapter 7

Evaluation on Benchmark Networks

This chapter gives the results of the 3off2 learning approach on different types of benchmarks networks. In section 7.1, the benchmark networks are real-life causal graphical models containing 20 to 70 nodes. The datasets have been simulated from known parameters. By contrast, in the section 7.2, the benchmark networks and their corresponding datasets have been generated using the causal modeling tool Tetrad IV.

The results are evaluated against other methods in terms of Precision (or positive predictive value), P rec = T P/(T P + F P ), Recall or Sensitivity (true positive rate), Rec = T P/(T P + F N ), as well as F-score = 2 × P rec × Rec/(P rec + Rec)

for increasing sample size N =10 to 50,000 data points. We also define additional Precision, Recall and F-scores taking into account the edge orientations of the predicted networks against the corresponding CPDAG of the benchmark networks.

This amounts to label as false positives, all true positive edges of the skeleton with different orientation/non-orientation status as the CPDAG reference, T P misorient , leading to the orientation-dependent definitions T P = T P -T P misorient and F P = F P + T P misorient with the corresponding CPDAG Precision, Recall and F-scores taking into account edge orientations.

Using simulated datasets from real-life networks

We have tested the 3off2 network reconstruction approach to learn benchmark causal graphs containing 20 to 70 nodes, . The alternative inference methods used for comparison with 3off2 are the PC algorithm [START_REF] Spirtes | An algorithm for fast recovery of sparse causal graphs[END_REF] implemented in the pcalg package [Kalisch andBühlmann, 2008, Kalisch et al., 2012] and Bayesian inference using the hill-climbing heuristics implemented in the bnlearn package [START_REF] Scutari | Learning Bayesian Networks with the bnlearn R Package[END_REF]. In addition, we also compare the skeleton of 3off2 to the unoriented output of Aracne [START_REF] Margolin | ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context[END_REF]],

an information-based inference approach, which iteratively prunes links with the weakest mutual information based on the Data Processing Inequality. We have used the Aracne implementation of the minet package [START_REF] Meyer | minet: A R/Bioconductor package for inferring large transcriptional networks using mutual information[END_REF].

For each sample size, 3off2, Aracne, PC and the Bayesian inference methods have been tested on 50 replicates. give the average results over these multiple replicates when comparing the CPDAG (filled lines) of the reconstructed network (or its skeleton, dashed lined) to the CPDAG (or the skeleton) of the benchmark network. For each method, the plots presented in Figures 7.1 Precision, Recall and F-score for skeletons (dashed lines) and CPDAGs (filled lines).

The results are given for Aracne (black), PC (blue), Bayesian Hill-Climbing (green) and 3off2 (red). Recall and F-score for skeletons (dashed lines) and CPDAGs (filled lines). The results are given for Aracne (black), PC (blue), Bayesian Hill-Climbing (green) and 3off2 (red).

Finally, the best 3off2 network reconstructions are obtained using NML scores with shifted 2-point and 3-point information terms in the rank of individual edges, see Methods. Using MDL scores, instead, leads to equivalent results, as expected, in the limit of very large datasets (see section 6.2.2). However, with smaller datasets, the most reliable results with MDL scores are obtained using non-shifted instead of shifted 2-point and 3-point information terms in the 3off2 rank of individual edges, as discussed in Methods (see .

All in all, we found that the 3off2 inference approach typically reaches better or equivalent F-scores for all dataset sizes as compared to all other tested methods, (filled lines). The results are given for Aracne (black), PC (blue), Bayesian Hill-Climbing (green) and 3off2 (red).

Using simulated datasets from simulated networks

We have tested the 3off2 method on a range of benchmark networks of 50 nodes with up to 160 edges generated with the causal modeling tool Tetrad IV1 . The average connectivity k of these benchmark networks ranges between 1.6 to 6.4, and the average maximal in/out-degree between 3.2 to 8.8 (see Table B.1 for a detailed description).

The alternative methods used for comparison with 3off2 are the PC algorithm implemented in the pcalg package [Kalisch andBühlmann, 2008, Kalisch et al., 2012] and the hybrid method MMHC combining contraint-based skeleton and Bayesian orientation [START_REF] Tsamardinos | The Max-Min Hill-Climbing Bayesian Network Structure Learning Algorithm[END_REF], implemented in the bnlearn package [START_REF] Scutari | Learning Bayesian Networks with the bnlearn R Package[END_REF]. give the average CPDAG comparison results over 100 dataset replicates from 5 different benchmark networks (Table S1). The causal graphical models predicted by the 3off2 method are obtained using either the MDL/BIC or the NML complexities (see section 6. (Figures B. 7-B.18). All in all, we found that the 3off2 method outperforms both PC-stable and MMHC methods on all tested datasets, .

Additional comparisons were obtained with Bayesian inference implemented in the bnlearn package [START_REF] Scutari | Learning Bayesian Networks with the bnlearn R Package[END_REF] On denser networks ( k ≥ 5-6), Bayesian inference exhibits better F-scores than 3off2, in particular with AIC score, Figure B.24. However, the good performance with AIC strongly relies on its high Recall (but low Precision), due to its very small penalty term on large datasets, which makes it favor more complex networks (Figures B.24) but perform very poorly on sparse graphs . By contrast, the reconstruction of dense networks is impeded by the 3off2 scheme, as it is not always possible to uncover structural independencies, I(X; Y |{U i } n ) 0, in dense graphs through an ordered set {U i } n with only positive conditional 3-point Indeed in complex graphs, there are typically many indirect paths X → U j → Y between unconnected node pairs (X, Y ). At the beginning of the pruning process, this is prone to suggest likely v-structures X → Y ← U j , instead of the correct non-v-structures, X → U j → Y (for instance if I(X; U j ) I(X; Y ), I(X; U j ) I(U j ; Y ) and I(X; U j )-I(X; U j |Y ) = I(X; Y ; U j ) < 0, for all j). Such elimination of F N edge X → U j and conservation of F P X → Y tend to decrease both Precision and Recall, although 3off2 remains significantly more robust than PC and MMHC, Figure 7.10. Besides, for most practical applications on real life data, interpretable causal models should remain relatively sparse and avoid to display multiple indirected paths between unconnected nodes. algorithm to an Ising model where the spins correspond to the nodes of an undirected network, and the pairwise interactions between the spins correspond to the edges of this network. Each edge i j is associated to a weight J ij , a non-zero element of an adjacency matrix J that describes the direct pairwise coupling between the spins. Successive spin configurations are obtained by flipping each spin of the current configuration with a given probability p and by accepting or rejecting the resulting proposed configuration according to its Boltzmann probability. This provides a mean to generate large datasets corresponding to undirected networks with skeleton corresponding to the non zero elements of the matrix J.

information, I (X; Y ; U k |{U i } k-1 ) > 0.
One caveat of this data generating process is that the samples, corresponding to the successive configurations of the system, are correlated by construction (to obtain a significant acceptance rate in the Metropolis algorithm). As the 3off2 reconstruction method relies on the number of independent samples N to learn the skeleton and the orientations, it is necessary to implement a method to adjust the number N of correlated configurations to a smaller number N eff , i.e. effective N , corresponding to the estimated number of independent samples.

Hence, defining the fluctuations of each variable k around its mean (over the whole dataset) as,

σ i k = s i k -s i k i σ i+n k = s i+n k -s i+n k i
We then introduce the autocorrelation function C exp (n) defined as,

C exp (n) = 1 N -n N -n i=1 1 K K k=1 σ i k σ i+n k 1 N N i=1 1 K K k=1 σ i k 2
and compare its decay with an exponential function exp(-n/R). This enables us to estimate an effective number N eff of independent configurations as N eff = N/R.

Reconstructing simulated undirected networks

The skeletons of the ALARM and INSURANCE networks have been chosen as models to set the non zero weights of the adjacency matrix J. In order to evaluate the 3off2 reconstruction method on the discovery of the absence of causality, without being impaired by errors in the learning of the skeleton, the weights of the adjacency matrix J have been set to the (conditional) mutual informations computed with the 3off2 method when reconstructing the oriented ALARM or INSURANCE networks from a large dataset. These conditional mutual informations indeed amount to the remaining direct interactions between pairs of variables once the indirect effects of all positively contributing variables have been removed

from the mutual informations.

To reconstruct the undirected-ALARM and undirected-INSURANCE networks, datasets of size N = 10 6 have been generated. Tables 7.1 &7.2 give the true positive (TP), false positive (FP) and false negative (FN) counts for the reconstructions obtained with the 3off2 method. When assuming that all the samples are independent (N = 10 6 ), the 3off2 method orients all the discovered edges.

However, when adjusting for the number of samples, the 3off2 method rapidly indicates the absence of causality, and thus of orientations. Following the estimation method given in section 7.3.1, the estimated N eff for undirected-ALARM and undirected-INSURANCE networks is approximately 130, 000 (Figs. 7.11 & 7.12).

From N ef f = 100, 000 to N ef f = 150, 000, the 3off2 method gives one of the best F-score for both undirected networks, and without any orientated edge. In fact, the 3off2 approach already performs well at N eff 800, 000 for undirected-ALARM and N eff 700, 000 for undirected-INSURANCE (Tables 7.1 & The table gives the number of true positive (TP), false positive (FP) and false negative (FN) edges obtained when reconstructed the undirected-ALARM network with the 3off2 method (NML complexity). The precision (Prec.), recall (Rec.) and f-score (Fs.) evaluates the performance of the skeleton discovery. The %ort. gives the proportion of oriented edges within the learnt interactions. N eff TP FP FN Prec. Rec. Fs. %ort.

N eff .
The table gives the number of true positive (TP), false positive (FP) and false negative (FN) edges obtained when reconstructed the undirected-ALARM network with the 3off2 method (NML complexity). The precision (Prec.), recall (Rec.) and f-score (Fs.) evaluates the performance of the skeleton discovery. The %ort. gives the proportion of oriented edges within the learnt interactions. By contrast, as shown in Tables 7.3 & 7.4, the PC algorithm (PC-stable version using either the majority or the conservative orientation rule [Colombo andMaathuis, 2014, Ramsey et al., 2006]), requires an unusually small significance level to provide fully undirected networks (Tables 7.3 ) and f-score (Fs.) evaluates the performance of the skeleton discovery. The %ort. gives the proportion of oriented edges within the learnt interactions. The proportion of oriented edges when using either the majority or the conservative rule for the orientation step is given by %ort.maj. and %ort.csv. respectively.

These reconstructions of undirected networks performed by the 3off2 method support its ability to uncover the absence of causality from observational datasets.

The PC algorithm can also discover, for these benchmark networks, the absence of orientations. Yet, the PC algorithm requires a smaller significance level as compared to the values typically used in the literature, making it difficult to decide whether there is actually an absence of orientations, or a non detection of orientations.

Applications & Perspectives

Chapter 8

Interplay between genomic properties on the fate of gene duplicates with the 3off2 method

The mechanism underlying the emergence and the expansion of 'dangerous' gene families is an evolutionary oddity from a natural selection perspective. While the maintenance of 'essential' genes1 is ensured by their lethal null mutations, the expansion of 'dangerous' gene families implicated in cancer and other severe genetic diseases remains puzzling. These genes, albeit critical to normal cellular functions, can be referred to as 'dangerous' owing to their susceptibility to dominant deleterious mutations leading to diseases. Surprisingly, 'dangerous' genes have been duplicated more than others in the course of vertebrate evolution. What could have been the evolutionary causes responsible for such striking expansion? Have the gene susceptibility to dominant deleterious mutations played a driving role?

In this chapter, I present converging evidence supporting the hypothesis that 'dangerous' genes in the human genome have been preferentially retained after two rounds of whole-genome duplication (WGD) dating back from the onset of jawed vertebrates, some 500MY ago [Ohno, 1970, Putnam et al., 2008]. I further demonstrate that the retention of many WGD-duplicated genes, so-called 'ohnologs' 2 , suspected to be dosage balanced 3 , is in fact indirectly mediated by their susceptibility to deleterious mutations. In addition I introduce a somewhat counterintuitive yet simple evolutionary model accounting for this enhanced retention of 'dangerous' ohnologs. Finally, I discuss the networks of genomic properties reconstructed using the 3off2 method that support the proposed evolutionary model. We performed a data mining analysis that revealed a strong correlation between the retention of human ohnologs and their reported susceptibility to deleterious mutations [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF]. The statistics computed from this data mining analysis were based on the 20, 506 protein coding genes of the human genome (Ensembl release 61). 35% of these genes can be traced back to the two WGD events at the onset of jawed vertebrates [Makino and McLysaght, 2010]. We obtained, in particular, cancer gene datasets from multiple databases, including COSMIC [START_REF] Forbes | The catalogue of somatic mutations in cancer (COSMIC)[END_REF] and CancerGenes [START_REF] Higgins | CancerGenes: a gene selection resource for cancer genome projects[END_REF], other genetic disease genes from OMIM and dominant negative4 genes and genes with autoinhibitory protein folds5 from literature search.

It appears that the human genes associated with the occurence of cancer and other genetic diseases [START_REF] Huminiecki | 2R and remodeling of vertebrate signal transduction engine[END_REF]095) have retained significantly more ohnologs than expected by chance, 48% versus 35% (48%; 3, 844/8, 095; P = 1.3 × 10 -129 , χ 2 ). Furthermore, correlations are clearly enhanced when the analysis is restricted to genes with direct experimental evidence of dominant deleterious mutations, such as for genes with dominant negative mutants (61%; 292/477; P = 3.9 × 10 -34 , χ 2 ).

As shown later in section 8.4, networks of genomic properties reconstructed with the 3off2 method infer a direct causal effect between the properties 'Cancer' (i.e.

gene prone to mutations involved in cancer) or 'Dom Neg' (i.e. gene prone to dominant negative mutations) and 'Ohnolog' (i.e. gene having retained a copy from a whole genome duplication), indicating that a gene prone to dominant negative mutations or participating in a cancer is more likely to retain copies from WGD events (Figs. 8.3 & 8.4).

Retained ohnologs are more 'dangerous' than dosage balanced

We have also found [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF][START_REF] Singh | Human dominant disease genes are enriched in paralogs originating from whole genome duplication[END_REF] that the reported autosomal dominant disease genes (440) have experienced stronger retention biases in ohnologs (59%; 261/440; P = 1.7×10 -27 , χ 2 ) than the reported autosomal recessive diseases genes (598) (37%; 221/598; P = 0.24, χ 2 ). Besides, the human orthologs of mouse genes reported as being 'essential' genes (2729) from large scale null mutant studies in mouse (5956 genes tested) exhibit a small enrichment of ohnologs, 56 vs 54% (56%;

1537/2729; P = 3.8 × 10 -3 , χ 2 ), where 54% (3190/5656) is the global proportion of ohnologs among the genes tested in mouse for null mutation.

These results, in association with complementary statistics not shown here [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF], suggest that the retention of ohnologs is more strongly related to their 'dangerousness' than their 'essentiality' or 'dosage-balance' 6 . As shown in section 8.4, the networks of genomic properties obtained from these observational data with the 3off2 method infer no direct relationship between the properties 'Essential' or 'Complex' and 'Ohnolog'.

Going beyond simple correlations in genomic data

A number of studies have shown that many genomic properties are, to some extent, correlated. Among them, gene essentiality, functional ontology, expression level, divergence rates, etc, have been widely considered. Yet, many of these correlations, suggesting direct statistical association, could in fact be mediated through the indirect effect of third properties and only provide partial insights on complex biological systems. To go beyond such simple statistical associations and quantify the direct and indirect effects of genomic properties, I have performed Mediation analyses, following the approach of Pearl [Pearl, 2001[START_REF] Pearl | The causal mediation formula-a guide to the assessment of pathways and mechanisms[END_REF]. In particular, I aimed at disentangling the relative effects of deleterious mutations and dosage balance constraints on the retention of ohnologs.

The Mediation framework in the context of causal inference

The Mediation analysis, typically used in social sciences and epidemiology, aims at uncovering causal pathways along which changes in multivariates properties are transmitted from a cause, X, to an effect, Y . In other words, the Mediation analysis assesses the importance of a mediator, M , in transmitting the indirect effect of X on the response Y . In the Mediation framework, the average direct effect, DE xx , is defined as the change that Y would experience if x could be changed to x while keeping M fixed. Similarly, the average indirect effect, IE xx , is defined as the change that Y would experience if the mediator could be changed from m(x) to m(x ) while keeping X to its original value x (Fig. 8.1 & Table 8.1, A.) [START_REF] Pearl | The causal mediation formula-a guide to the assessment of pathways and mechanisms[END_REF]. In fact, the counterfactual expressions used to quantify these effects formally decouple the direct and indirect conditions on X, seen by the outcome Y .

m(x) In the framework of Bayesian statistics, the Mediation formulae can be expressed as in [START_REF] Pearl | The causal mediation formula-a guide to the assessment of pathways and mechanisms[END_REF]. n 000 +n 001 +n 010 +n 011 .

IE xx $ $ X > > DE xx / / Y (x, m(x))
Effects A. Shorthand notation B. Bayesian Framework DE xx Y (x , m(x)) -Y (x, m(x)) m [E(Y |x , m) -E(Y |x, m)]P (M |x) IE xx Y (x, m(x )) -Y (x, m(x)) m E(Y |x, m)[P (M |x ) -P (M |x)] T E xx Y (x , m(x )) -Y (x, m(x)) E(Y |x ) -E(Y |x)

The Mediation analysis applied to genomic data

I have applied the causal inference methods to the 20, 506 protein-coding genes in human. In particular, I focused on the disentanglement of direct and indirect effects favoring the retention of duplicated genes after the two rounds of wholegenome duplication (WGD) events that occured at the onset of jawed vertebrates, some 500MY ago [Ohno, 1970, Putnam et al., 2008]. More precisely, I quantified the effect of the dosage balance constraints ('dosage.bal.', as X or M ) and the effect of the gene susceptibility to deleterious mutations ('delet.mut.', as X or M ) on the retention of ohnologs ('ohno.', as Y ).

With respect to our data mining analysis [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF] In fact, as will be shown in section 8.4, the networks of genomic properties obtained with the 3off2 method infer no direct relationship between the properties 'Complex' and 'Ohnolog', suggesting that the propensity of a gene to retain its copies from WGD events is not a direct consequence if its involvement in multi-protein complexes (Figs. 8.3 & 8.4). From [START_REF] Affeldt | Évolution et cancer -expansion des familles de gènes dangereux par duplication du génome [evolution and cancer: expansion of dangerous gene repertoire by whole genome duplications] (article in french)[END_REF] Note, in particular, that this counterintuitive model for the retention of 'dangerous' ohnologs, hinges on (i) the speciation event concomitant to WGD, owing to the difference in ploidy between pre-and post-WGD individuals and (ii) autosomal dominance of deleterious mutations leading to purifying selection (C). This model is also supported by deterministic and stochastic simulations done in the Isambert's group [START_REF] Malaguti | On the retention of gene duplicates prone to dominant deleterious mutations[END_REF].

Although the retention of ohnologs remains stochastic, the susceptibility of 'dangerous' genes to dominant deleterious mutations significantly increases the odds of ohnolog retention through path (C), Fig. 8.2. By contrast, the bottleneck due to small WGD-population size limits the efficacy of adaptation that could lead eventually to fixation of beneficial mutations through path (A), Fig. 8.2. Therefore, as suggested by our data mining analysis [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF], most 'dangerous' ohnologs are retained by purifying selection (i.e. path (C)) due to their susceptibility to dominant deleterious mutations. In other words, purifying selection has minimized the rapid loss of 'dangerous' genes in post WGD population (Fig. 8.2)

and the ensuing sub-functionalization has ensured their long term survival in post-WGD species [START_REF] Malaguti | On the retention of gene duplicates prone to dominant deleterious mutations[END_REF]. 1) to ( 9) listed in Table 8.3. These properties also include SSD in order to uncover the antagonist pattern retention between SSD and WGD duplicates. Then, other properties, such as the copy number variation, the expression level or the degree of conservation of the protein sequence, have been taken into account, leading to twelve properties, Table 8.3.

As direct information of human essential gene is not available, the 'essential' data correspond either to essentiality in mouse orthologous 14 genes (Mouse Genome Informatics (MGI) database [START_REF] Eppig | The Mouse Genome Database (MGD): comprehensive resource for genetics and genomics of the laboratory mouse[END_REF]). The essentiality of human protein coding genes could be assessed for only 6, 436 genes out of 20, 415. Yet, as previously detailed in section 6.4.3, the 3off2 approach can deal with missing data by computing the 2-point and 3-point information terms solely on the observed variables, without impairing the other calculations.

The Ka.Ks property indicates the conservation of the gene sequence. Indeed, the Ka/Ks ratio measures the proportion of nonsynonymous substitutions (Ka) to the proportion of synonymous substitutions (Ks). I have shown in [START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF] that ohnologs exhibit statistically lower Ka/Ks ratios, which provided direct evidence of strong conservation, consistent with a higher susceptibility of ohnologs to Finally, gene expression values were downloaded from BioGPS [START_REF] Wu | BioGPS: an extensible and customizable portal for querying and organizing gene annotation resources[END_REF] and correspond to the median values of expression across 78 tissues/cell types [START_REF] Eppig | The Mouse Genome Database (MGD): comprehensive resource for genetics and genomics of the laboratory mouse[END_REF]372 genes in total). One of the first conclusions that can be drawn from Fig. 8.3 is that the 3off2 method does not recover any direct relationship between 'Complex' and 'Ohnolog'.

The former indicating the involvement of the gene in multi-protein complexes, it embodies dosage-balance constraints. Thus, this graphical model confirms that the retention of ohnolog does not primarily rely on this alternative hypothesis by [Makino and McLysaght, 2010]. A second interesting result is that 'Cancer', 'Dom Neg' and 'Autoinhib' properties directly point toward the 'Ohnolog' node, supporting our hypothesis that the susceptibility to (dominant) deleterious mutations participates to the retention of copies from the 2 rounds of WGD events that occurred at the origin of vertebrates. One can also recover from Fig. 8.3 the antagonist retention pattern between genes prone to SSD duplicates and genes prone to WGD duplicates. Indeed, the edge linking 'Ohnnolog' and 'recentSSD' displays an anti-correlation effect, with the edge direction consistent with the temporal retention of 'recentSSD' after the 2R-WGD in early vertebrates.

The graphical model given in Fig. 8.4 corresponds to the reconstructed causal graphical model when adding the properties ( 10) to ( 12) from Table 8.3. Note that the previous conclusions from Fig. 8.3 still hold, highlighting the robustness of our approach to the inclusion of additional properties (although the orientation between 'Ohnolog' and 'Autoinhib' is inverted). Interestingly, the Ka.Ks property is negatively pointed by 'Ohnolog', confirming our previous comparative genomic results on the conservation of ohnolog sequences. Besides, the cause-effect relationship between 'Ka.Ks' and 'recentSSD' is positive, supporting widely acknowledged results on the necessary positive selection (i.e. with higher Ka/Ks ratios) of SSD copies. We also note that genes with recent SSD tend to be less prone to be essential and implicated in disease as their SSD copy can buffer to some extent the deleterious effects of loss-of-function mutations.

spontaneous brain activity of zebrafish larvae brain [START_REF] Panier | Fast functional imaging of multiple brain regions in intact zebrafish larvae using selective plane illumination microscopy[END_REF]. As shown in the section 9.2, the 3off2 reconstruction method could confirm already known communication patterns between large areas corresponding either to sensory or motor structures.

9.1 The zebrafish calcium functional imaging

The calcium functional imaging

Micro-arrays of electrodes can be used to measure the action potentials of a few hundred neurons at the same time with a high temporal resolution. Yet, this method only provides information on small areas, and cannot capture long range communications throughout the whole brain. By contrast, calcium imaging is a non-invasive technique that can optically monitor the fluctuations of Ca 2+ concentration in a greater number of cells [START_REF] Yuste | Control of postsynaptic ca2+ influx in developing neocortex by excitatory and inhibitory neurotransmitters[END_REF]. These Ca 2+ fluctuations reflect the action potentials that affect the cells during the spiking process as the sudden depolarization during a spike corresponds to the opening of charged ion channels within the cellular membrane. If the cells contain a calcium sensitive probe, such as GCaMP3 (a GFP protein associated with a calcium sensitive Calmoduline), a fluorescent signal can be optically recorded [START_REF] Tian | Imaging neural activity in worms, flies and mice with improved GCaMP calcium indicators[END_REF][START_REF] Tian | Imaging neuronal activity with genetically encoded calcium indicators[END_REF].

The zebrafish as vertebrate model

The zebrafish is a small vertebrate (typically 200×500×1000µm) that is well suited for functional calcium imaging. Its brain, which already contains tens of thousands of neurons in a larvae of a few days, is able to respond to several stimuli, like light or scent, at an early developmental stage when the whole body is transparent. In particular, the Debrégeas group has applied calcium functional imaging to observe neuronal cells of genetically modified zebrafish expressing GCaMP3 [START_REF] Tian | Imaging neural activity in worms, flies and mice with improved GCaMP calcium indicators[END_REF][START_REF] Tian | Imaging neuronal activity with genetically encoded calcium indicators[END_REF]. This calcium indicator provides a fluorescent signal related to neural spikes.

Fig. 9.1, adapted from [START_REF] Portugues | Whole-brain activity maps reveal stereotyped, distributed networks for visuomotor behavior[END_REF], highlights known functional areas in the larval zebrafish brain. As reported in previous studies, the regions of interest (ROIs) within the zebrafish brain display a clear symmetry, except for the habenula (Hab) region, which is known to have pronounced anatomical asymmetry and have predominant responses on the left side [Bianco andWilson, 2009, Portugues et al., 2014].

Previous studies have shown that the activity in regions involved in sensory processing (e.g. tectal neuropil, pretectum (Pt) or inferior olive; Fig. The Selective-Plane Illumination Microscopy (SPIM) [START_REF] Holekamp | Fast three-dimensional fluorescence imaging of activity in neural populations by objective-coupled planar illumination microscopy[END_REF][START_REF] Keller | Fast, high-contrast imaging of animal development with scanned light sheet-based structured-illumination microscopy[END_REF][START_REF] Panier | Fast functional imaging of multiple brain regions in intact zebrafish larvae using selective plane illumination microscopy[END_REF] is a scanning laser sheet technology that performs an optical sectioning and collect images with a camera whose optical axis is orthogonal to the illumination plane. This technology allows the simultaneous observation of more neurons than with classical confocal or two photon microscopy, as different areas within the same z-plane are exposed for a much longer time on average. It has been shown [START_REF] Panier | Fast functional imaging of multiple brain regions in intact zebrafish larvae using selective plane illumination microscopy[END_REF] that the SPIM imaging technology reaches about 20-folds increase in the speed of acquisition, i.e. number of recorded neurons times acquisition rate, as compared with classical point-scanning imaging techniques (PSM). Specifically, PSM systems can acquire approximately one thousand neurons at 5Hz, while SPIM imaging technology reaches about 5, 000 at 20Hz. The figure 9.2, taken from [START_REF] Panier | Fast functional imaging of multiple brain regions in intact zebrafish larvae using selective plane illumination microscopy[END_REF], gives an overview of the experimental settings available at the Jean Perrin laboratory. As can be seen in The remainig step is to extract the action potentials from the ∆F/F quantity.

This extraction is a difficult task as it has to cope with many parameters such as the noise within the signal, the temporal resolution of the imaging system or the characteristics of the calcium indicators. A very simple extraction would consist in applying a threshold to the ∆F/F fluorescent signal, however more efficient methods that are based on deconvolution (i.e. Wiener filter) have also been proposed to extract the trains of action potentials [START_REF] Holekamp | Fast three-dimensional fluorescence imaging of activity in neural populations by objective-coupled planar illumination microscopy[END_REF]].

Yet, the peaks of activity in the filtered signal still remain large, and hence the temporal resolution is low (Fig. 9.4, green curve). Thus, the group of G. Debrégeas developed another filtering method, inspired from [START_REF] Ramdya | Reverse correlation of rapid calcium signals in the zebrafish optic tectum in vivo[END_REF], and based on the specific shape of the fluorescent signal, i.e. a rapid increase followed by a slow decrease (Fig. 9.4, blue curve). This processing, that relies on a Wiener filter and on a recursive procedure, produces good results on a range of tests performed in the Jean-Perrin laboratory. As can be seen in Fig. 9. Groups of neurons involved in similar functionalities within a specific region typically undergo similar bursts when the electrical signals are transmitted throughout the brain. Several studies have shown that the brain of zebrafish larvae can be partitioned into distinct neural areas [START_REF] Bianco | Visuomotor transformations underlying hunting behavior in zebrafish[END_REF][START_REF] Kubo | Functional architecture of an optic flow-responsive area that drives horizontal eye movements in zebrafish[END_REF][START_REF] Portugues | Whole-brain activity maps reveal stereotyped, distributed networks for visuomotor behavior[END_REF]. Thus, in order to avoid redundant edges corresponding to correlations within individual functional module, we have clustered the neurons that display correlated bursts within the same neighbourhood using a hierarchical studied in the zebrafish larva, relying on two-photon technology. In this section, the network reconstructions of 3off2 are compared with the temporal patterns uncovered in these experiments. These results are still exploratory and the offsets of various parameters remain to be studied.

Alternating neural activity in optokinetic reponse

The optokinetic response (OKR) is an innate reflexive behaviour that tends to reduce the motion of the image on the retina with an horizontal rotational movement of the eyes. In [START_REF] Portugues | Whole-brain activity maps reveal stereotyped, distributed networks for visuomotor behavior[END_REF], the OKR has been induced in zebrafish larva using sinusoidally rotating whole-field stimuli around the animal. The neural responses of the larval zebrafish, genetically modified to encode a calcium indicator GCaMP5G, have been recorded with a two-photon imaging system. The 2D layers of the brain analysed in [START_REF] Portugues | Whole-brain activity maps reveal stereotyped, distributed networks for visuomotor behavior[END_REF] revealed alternating temporal patterns. Specifically, (a) a columnar activity has been observed in the dorsal medulla, with an antiphase bursting pattern between left and right sides, and (b) left/right-alternating activity in the neuropil has also been observed (Fig. 9.9).

Besides, (c) sensory areas, such as tectal neuropil, pretectum and inferior olive have been reported to typically precede the activity in motor structures in the hindbrain. The causal networks reconstructed with the 3off2 method report these alternating patterns. Similar results can be seen in the 3off2 reconstructed network from the layer 6. In Fig. 9.13, the left/right alternating pattern in the medulla can be seen in the directed path (Cl 9 → Cl 5 → Cl 4 → Cl 3 → Cl 2 → Cl 1 ). Similarly as for layer 5, the same alternating pattern cannot be inferred when the reconstruction is based on 15 clusters, except with the oriented edge Cl 3 → Cl 2 .

Specific temporal patterns, from sensory to motor structures have also been reported in [START_REF] Portugues | Whole-brain activity maps reveal stereotyped, distributed networks for visuomotor behavior[END_REF]. The reconstruction of the layer 11 by the 3off2 method, using 15 and 30 clusters, shows relationships between clusters related to the inferior olive and caudal hindbrain (Fig. 9.14 & 9.15). However, while the re- ). This could be explained by the split of the bursting information between neighbour areas when using too many clusters, which in turn produces spurious (weak) orientations. It should be noted that, when reaching much deeper layers, 3off2 identified more relationships resulting from a common cause (bi-directed gray edges, Fig. 9.15), possibly indicated the influence of upper neurons not included in the 2D reconstructed network.

[ [START_REF] Portugues | Whole-brain activity maps reveal stereotyped, distributed networks for visuomotor behavior[END_REF]] also reported a left/right alternating activity in neuropil in the vicinity of the interpeduncular nucleus. Interestingly, the networks reconstructed with the 3off2 method display directed cycles between the areas closely related to the neuropil. Specifically, Fig. et al., 2014].

Hence, the 3off2 reconstruction method could display similar alternating temporal pattern as reported in [START_REF] Portugues | Whole-brain activity maps reveal stereotyped, distributed networks for visuomotor behavior[END_REF]]. Yet, while the resulting patterns in [START_REF] Portugues | Whole-brain activity maps reveal stereotyped, distributed networks for visuomotor behavior[END_REF] have been uncovered with temporal data, it should be emphasized that the 3off2 method does not require temporal information to infer the direction of interaction from observational data.

Neural activity related to hunting behaviour

During prey catching, the zebrafish undergoes a specific behaviour involving eye convergence with saccades that increase when approaching the target. This specialized oculomotor behaviour is suspected to favour the distance estimation to the prey and has been shown to imply the recruitment of assemblies of neurons in the optic tectum (OTc). Using two photon calcium imaging, [START_REF] Bianco | Visuomotor transformations underlying hunting behavior in zebrafish[END_REF] showed that the bursts produced by these assemblies precede and control the triggering of hunting responses and propose a neural model circuit underlying the hunting behaviour of the zebrafish (Fig. 9.16, taken from [START_REF] Bianco | Visuomotor transformations underlying hunting behavior in zebrafish[END_REF]).

Specifically, the visual information is transmitted to the retinal ganglion cell (see [START_REF] Makino | Ohnologs in the human genome are dosage balanced and frequently associated with disease[END_REF], Fig. Further investigation in terms of parameters should still be done in order to define the optimal number of clusters, to decide for the most accurate clustering method and to identify the mostly significant active neurons. Yet, these preliminary 2D network reconstructions could partially recover the model circuit deduced from experiments relying on stimuli [Bianco andEngert, 2015, Portugues et al., 2014] and the analysis of the spontaneous activity of the zebrafish brain based on the 3off2 method. This support the prevailing hypothesis of an underlying brain network revealed by spontaneous activity or stimulations. In the future, we would like to implement clustering of the signals in 3D and subsequent 3D network reconstruction. As several 2D reconstructed networks displayed in this chapter show edges suggesting the existence of latent variables (bi-directed gray edges), 3D clustering could possibly identify neurons in upper or deeper layers participating in these correlations.

Chapter 10

Reconstruction of the hematopoiesis regulation network with the 3off2 method

In this chapter, the 3off2 approach has been applied on a real biological dataset related to hematopoiesis. The hematopoiesis is a process that controls the emergence and the replacement of blood cells. In adults, these cells, originated from bone marrow, differentiate in distinct categories, among which the erythrocytes, the leukocytes and the thrombocytes, that ensure specific functions. For instance, while the erythrocytes (red blood cells) guarantee the oxygen transportation, the thrombocytes (platelets) are involved in the hemostasis and the leukocytes (white blood cells) play an important role in the immune system. During the hematopoiesis, transcription factors1 trigger and guide the differentiation process from Hematopoietic Stem Cells (HSC) to mature cells.

This chapter presents and discusses the reconstruction of a regulatory subnetwork of the hematopoiesis performed with 3off2 reconstruction method. This part of my project has been developed in collaboration with a PhD student in bioinformatics in the Isambert's group, Louis Verny [Affeldt and[START_REF] Affeldt | Robust reconstruction of causal graphical models based on conditional 2-point and 3-point information[END_REF].

The hematopoiesis regulation network

Transcription factors play a central role in hematopoiesis, from which derive the blood cell lineages. As suggested in previous studies, changes in the regulatory interactions among transcription factors [START_REF] Oram | A previously unrecognized promoter of lmo2 forms part of a transcriptional regulatory circuit mediating lmo2 expression in a subset of t-acute lymphoblastic leukaemia patients[END_REF] or their overexpression [START_REF] Cleveland | Lmo2 induces hematopoietic stem cell like features in T-cell progenitor cells prior to leukemia[END_REF] might be involved in the development of T-acute lymphoblastic leukaemia (T-ALL). The key role of the hematopoiesis and the potentially serious consequences of its disregulations emphasize the need to accurately establish the complex interactions between the transcription factors involved in this critical biological process.

10.2 Regulation network reconstruction with the 3off2 method 10.

Dataset and interactions of interest

The dataset we have used for this analysis consists of the single cell expressions 2 of 18 transcription factors [START_REF] Moignard | Characterization of transcriptional networks in blood stem and progenitor cells using high-throughput single-cell gene expression analysis[END_REF], known for their role in hematopoiesis. 597 single cells representing 5 different types of hematopoietic progenitors have been included in the analysis (N = 597). We reconstructed the corresponding network with the 3off2 method (Figure 10.1, Table 10.1) and other available approaches (Table 10.1). In particular, we focused on 11 regulatory interactions for which specific experimental evidence have been reported in the literature (Table 10.1).

The interactions recovered by 3off2 and alternative methods

The corresponding network has been reconstructed with the 3off2 method and four other available approaches, namely, PC [START_REF] Spirtes | An algorithm for fast recovery of sparse causal graphs[END_REF] implemented in the pcalg package [Kalisch andBühlmann, 2008, Kalisch et al., 2012], Bayesian inference using hill-climbing heuristics as well as the Max-Min Hill-Climbing (MMHC) hybrid method [START_REF] Tsamardinos | The Max-Min Hill-Climbing Bayesian Network Structure Learning Algorithm[END_REF], both implemented in the bnlearn package [START_REF] Scutari | Learning Bayesian Networks with the bnlearn R Package[END_REF], and finally, Aracne [START_REF] Margolin | ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context[END_REF] implemented in the minet package [START_REF] Meyer | minet: A R/Bioconductor package for inferring large transcriptional networks using mutual information[END_REF]] (Tables 10.1 and 10.2).

3off2 uncovers all 11 interactions for which specific experimental evidence has been reported in the literature (Figure 10.1, red links: known activations; blue links: known repressions) as well as 30 additional links (Figure 10.1, grey links: unknown regulatory interactions). By contrast, randomization of the actual data across samples for each TF leads to only 5.25 spurious interactions on average between the 18 TFs, instead of the 41 inferred edges from the actual data, and 1.62 spurious interactions on average, instead of the 16 interactions predicted among the 10 TFs involved in known regulatory interactions, Figure 10.1. This suggests that around 10-30% of the predicted edges might be spurious, due to inevitable sampling noise in the finite dataset.

In particular, the 3off2 inference approach successfully recovers the relationships of the regulatory triad between Gata2, Gfi1b and Gfi1 as described in [START_REF] Moignard | Characterization of transcriptional networks in blood stem and progenitor cells using high-throughput single-cell gene expression analysis[END_REF] and reports correct orientations for the edges involving Gata2 (Gfi1b and Gfi1 crossregulate in fact one another [START_REF] Moignard | Characterization of transcriptional networks in blood stem and progenitor cells using high-throughput single-cell gene expression analysis[END_REF], Table 10.1).

known Regulatory

3off2 PC PC MMHC MMHC Bayes hc Bayes hc Aracne

interactions NML α = 10 -1 α = 10 -2 BDe BIC BDe BIC = 0 Gata2 → Gfi1b [Moignard et al., 2013] → → Gfi1 → Gata2 [Moignard et al., 2013] → → → → Gfi1b
Gfi1 [START_REF] Moignard | Characterization of transcriptional networks in blood stem and progenitor cells using high-throughput single-cell gene expression analysis[END_REF] Gfi1 → PU.1 [START_REF] Spooner | A recurrent network involving the transcription factors pu.1 and gfi1 orchestrates innate and adaptive immune cell fates[END_REF] → → → → Lyl1 → Gfi1 [START_REF] Zohren | The transcription factor lyl-1 regulates lymphoid specification and the maintenance of early t lineage progenitors[END_REF] → → Ldb1 → Meis1 [START_REF] Li | Nuclear adaptor ldb1 regulates a transcriptional program essential for the maintenance of hematopoietic stem cells[END_REF]] → Gata2 → Scl [START_REF] Göttgens | Establishing the transcriptional programme for blood: the scl stem cell enhancer is regulated by a multiprotein complex containing ets and gata factors[END_REF] → → → → → Gfi1b → Meis1 [START_REF] Chowdhury | Differential transcriptional regulation of meis1 by gfi1b and its co-factors lsd1 and corest[END_REF] → → → → Ldb1 → Lyl1 [START_REF] Li | Nuclear adaptor ldb1 regulates a transcriptional program essential for the maintenance of hematopoietic stem cells[END_REF] → Erg → Lyl1 [START_REF] Chan | The paralogous hematopoietic regulators lyl1 and scl are coregulated by ets and gata factors, but lyl1 cannot rescue the early scl-/-phenotype[END_REF] → → → → Gata1 → Gata2 [START_REF] Grass | GATA-1-dependent transcriptional repression of GATA-2 via disruption of positive autoregulation and domain-wide chromatin remodeling[END_REF] ). corresponds to a successfully recovered interaction, however, with an opposite direction as reported in the literature.

stipulates that no direct regulatory interaction has been inferred, while corresponds to an undirected link. Red/blue edges correspond to experimentally proven activating/repressing interactions. Note in particular that Aracne does not infer edge direction.

The network reconstructed by 3off2 also correctly infers the influence of Gfi1 on PU.1 [START_REF] Spooner | A recurrent network involving the transcription factors pu.1 and gfi1 orchestrates innate and adaptive immune cell fates[END_REF], the regulation of Gfi1 by Lyl1 [START_REF] Zohren | The transcription factor lyl-1 regulates lymphoid specification and the maintenance of early t lineage progenitors[END_REF], and the regulatory effects of Ldb1 on Meis1 and Lyl1 [START_REF] Li | Nuclear adaptor ldb1 regulates a transcriptional program essential for the maintenance of hematopoietic stem cells[END_REF]. Finally, the interactions (Gata1 -Gata2 ), (Erg -Lyl1 ), (Gata2 -SCL) and (Meis1 -Gfi1b) are correctly inferred, however, with opposite directions as reported in the literature. Yet, overall 3off2 outperforms most of the other methods tested for the reconstruction of hematopoietic regulatory networks (Tables 10.1 and 10.2). Only the Bayesian hill-climbing method using a BDe score leads to comparable results by retrieving 10 out of 11 interactions and correctly orienting 8 of them.

These encouraging results from the 3off2 reconstruction method on experimentally proven regulatory interactions (red/blue edges in Figure 10.1) could motivate further investigations on novel regulatory interactions awaiting to be tested for their possible role in hematopoeisis (e.g. grey edges in Figure 10.1). 10.1). These calculations were made assuming that the 11 experimentally proven interactions correspond to the true regulatory network (in practice, we expect that some of the inferred edges counted as false positives might in fact turn out to be correctly predicted links). u indicates that the calculations were made on the undirected network, whereas d means directed network. The "Distance" measure corresponds to the number of "errors" (including orientations for the directed network comparison) over the total number of possible edges, i.e. Distance=2(F P + F N )/n(n -1), where n = 10 is the number of nodes in the subnetwork including the 11 known regulatory interactions (Figure 10.1).

Chapter 11

Reconstruction of mutational pathways involved in tumours progression with the 3off2 method Many biomolecular networks sustain the life processes of an organism. Mutations of genes belonging to these functional networks may lead to complex diseases, such as cancer. An increasing number of studies are trying to identify genes susceptible to mutations that favour the development of genetic diseases [START_REF] Dees | MuSiC: identifying mutational significance in cancer genomes[END_REF][START_REF] Mermel | GISTIC2.0 facilitates sensitive and confident localization of the targets of focal somatic copy-number alteration in human cancers[END_REF][START_REF] Marx | Cancer genomes: discerning drivers from passengers[END_REF]. In particular, genes involved at upstream positions in these biomolecular networks are of great interest, as they may be interesting therapeutic drug targets. However, there exists a significant heterogeneity in the mutations between individuals developing the same type of cancer, and this heterogeneity can also be observed in the tumour of the same patient [START_REF]Mutational heterogeneity in cancer and the search for new cancer-associated genes[END_REF]. In fact, the deregulations that drive tumour growth may originate from distinct mutated signalling pathways or arise through mutations at different levels of the same signalling pathway.

Despite this intra-and inter-patients heterogeneity, there still exist typical fences that tumoral cells should overcome to initiate and sustain an uncontrolled proliferation [START_REF] Vogelstein | Cancer genome landscapes[END_REF]. For instance, it is 'beneficial' for the (future) tumour cells to first lose their ability to differentiate, as this enhances their division rate and thus their proliferation. The breast cancer is the most common cancer in women. The malignant tissue typically develops from somatic1 mutations, although inherited mutations may favour the emergence of breast cancer tumours.

Previous studies have confirmed the existence of four main breast cancer molecular classes, namely the luminal A, the luminal B, the triple negative (or basal-like)

and the HER2 types [TCGA, 2012]. The luminal A and B are the most common subtypes (about 40% and 20% of the cases) and are associated with a deregulation of the oestrogen and progesterone receptors. Thus, they can be treated with an hormonal therapy. As compared to the luminal A type, the luminal B has a poorer prognosis. The luminal A tumour cells are characterized by a slow growth, while the luminal B subtype is more aggressive, proliferate faster, and is prone to spread through blood vessels.

The HER2 and the triple negative classes are less common (about 15% and 20% of the cases). The HER2 subtype is prone to early and frequent metastasis. This subtype typically corresponds to an over-expression of the HER2 gene, resulting in an over-production of growth-enhancing proteins, and is usually associated with a poor prognosis. The triple negative class produces fast-growing aggressive tumours. As this subtype lacks oestrogen and progesterone receptors, patients cannot benefit from an endocrine therapy.

The issue of missing values

The COSMIC database A single study is typically associated with a few hundred patients and thus provides a few hundred samples. The sequencing of these samples can either be wholegenome/exome or restricted to specific portions of the DNA. Yet, a few hundred samples, even when issued from whole-genome sequencing, do not provide enough information to allow a robust reconstruction of mutational pathways as will be shown below. This can be overcome by gathering distinct studies, although it usually implies to rely on an incomplete dataset. As detailed in section 6.4.3, the 3off2 reconstruction method can learn networks from datasets with missing data. We thus combined tumour samples obtained from different studies on breast cancer available in the COSMIC database and could benefit from a larger number of samples for our analysis.

The data used in this chapter contains 26, 494 samples that were downloaded from the COSMIC database using the web service COSMICMart5 . For each sample, the status of each gene is either 'mutated', 'non-mutated' or 'not sequenced', the latter corresponding to a missing value. The genes were ordered by their decreasing frequency of test, i.e. in increasing order of their proportion of missing values.

The reconstructions discussed in section 11. gives the average number of edges obtained when reconstructing mutational pathways with the 3off2 method from the 1, 000 shuffled datasets, while the column 'non-random' provides the number of edges based on the original (non-shuffled) datasets.

By contrast, the datasets of 26, 494 tumour samples containing missing values should provides more robust networks. Indeed, as given is Table 11.2, (B), the number of inferred edges when applying the 3off2 method on randomized incomplete datasets ranges from 4.8 (η = 1) to 0.8 (η > 8). This suggests lower probabilities of inferring edges that correspond to spurious associations (e.g. only 12%

for η = 3).

3off2 cascades of mutations in breast cancer

The Fig. 11.1 gives the reconstructed network obtained with the 3off2 method, when setting the η parameter to 3. As this reconstruction has been done over a restricted number of genes, the network has been inferred assuming the presence of hidden variables. The color of edges indicates the likely direction of the 'advantageous' cascade of mutations for tumour progression, with warmer colors representing positively correlated mutations and cold colors representing negatively correlated mutations (red and dark blue colors highlighting a stronger association). As can be seen from the Fig. 11.1, the number of samples on which the edges have been inferred ranges from 206 to 2007, with the lower bound being greater than the number of samples of the dataset without missing values. This highlights the advantage of the 3off2 method when inferring causal networks from incomplete datasets. More importantly, the network reconstructed with the 3off2 method suggest likely temporal information from observational data. Indeed, the samples correspond to 'snapshots' of the tumour mutational landscapes over distinct individuals, and no information of the cancer stage have been included in our analysis. The section 11.2.3 further details some series of mutations found in the 3off2 reconstructed network that are suspected to preferentially drive the tumour progression.

Temporal cascade patterns of advantageous mutations

Among the cascades of mutations shown in Fig. 11.1, the mutational pathway (PIK3R1 → TP53 → RB1) is an interesting example of a series of fences that the cells should overcome in order to initiate and sustain tumour growth. PIK3R1 is a gene that encodes one of the two subunits of the heterodimer PI3K (the other subunit being encoded by PIK3CA) that plays an important role in cellular processes, in particular cell growth and proliferation. It has been shown that alterations of PIK3R1 (or PIK3CA) result in PI3K pathway activation, and thus in an enhancement of cell proliferation [START_REF] Cheung | High frequency of PIK3R1 and PIK3R2 mutations in endometrial cancer elucidates a novel mechanism for regulation of pten protein stability[END_REF][START_REF] Cizkova | Pik3r1 underexpression is an independent prognostic marker in breast cancer[END_REF].

Mutations in TP53 are frequently found in breast cancer and induce the production of a non-functional protein, that can no longer ensure the control of errors in the genome or regulate the cell division process [START_REF] Gasco | The p53 pathway in breast cancer[END_REF]. This enables the accumulation of mutations in the proliferating cells. Finally, mutated RB1 gene is often found in the triple-negative breast cancer subtype [Robinson et al., 2013].

This gene plays an important role in the regulation of DNA replication before the division of the cell. It has been found that alterations of RB1 may induce genomic instabilities that trigger chromosomal aberrations, which might favour fixations of subsequent mutations.

Likewise, the series of mutations (PIK3CA → CDH1 → VHL) also provides an interesting cascade of mutations. As previously introduced, mutations in PIK3CA favour abnormal cell growth and proliferation. Then, mutations in CDH1, which produces E-cadherin,a major cell-cell adhesion molecule, typically leads to the loss of structural cohesion between cells, hence enabling the dissemination of tumour cells in the rest of the organism. Indeen, this gene has been found frequently mutated in invasive breast carcinoma [START_REF] Lei | CDH1 mutations are present in both ductal and lobular breast cancer, but promoter allelic variants show no detectable breast cancer risk[END_REF], in particular in the invasive lobular carcinoma (ILC) where early loss of CDH1 function is not rare. ILC cells usually invade surrounding areas through a specific (non-cohesive) infiltrating behaviour, also called 'Indian-file' [START_REF] Berx | E-cadherin is a tumour/invasion suppressor gene mutated in human lobular breast cancers[END_REF]. Finally, the VHL gene is known to be involved in the degradation of the transcription factor HIF1 which promotes angiogenesis and tumour cell migration. Hence, once mutated, the VHL gene may thus permit the migration of breast cancer cells and the development of surrounding vessels. This provides the migrating tumour cells with an increased glucose intake, which further sustains the primary tumour as well as the formation of metastasis [START_REF] Zia | The expression of the von hippel-lindau gene product and its impact on invasiveness of human breast cancer cells[END_REF].

In addition, the network in Fig. 11.1 shows two distinct subnetworks that either involve the genes PTEN/MAP2K4 or the genes PIK3CA/AKT1. As can be seen in Fig. 11.2 [START_REF]The landscape of cancer genes and mutational processes in breast cancer[END_REF], PTEN/MAP2K4 or PIK3CA/AKT1 play a role in Chapter 12

Conclusions

This dissertation presents a novel hybrid method, 3off2, for the reconstruction of causal graphical models from finite observational datasets. As shown in the evaluations of Chapter 7, this new approach appears to be more robust than the state-of-the-art methods, namely the constraint-based and Bayesian searchand-score algorithms, when applied to finite observational datasets with inherent sampling noise. As detailed in Chapter 6, the 3off2 hybrid approach benefits from the main advantages of the constraint-based and Bayesian reconstruction methods, and demonstrates a greater robustness than the existing baseline methods, such as the PC-stable algorithm [Colombo andMaathuis, 2014, Spirtes et al., 1993] or the hill-climbing search-and-score heuristics.

Specifically, the 3off2 method combines the Bayesian and constraint-based approaches within a rigorous information-theoretic framework to confidently ascertain structural independencies in causal graphical models underlying real-life complex systems. The keystone of the 3off2 approach is the iterative procedure that takes off the most likely conditional 3-point information from the 2-point (mutual) information between each pair of nodes to progressively uncover the separation sets and hence learn the direct relationships among the variables (section 6.3.2). The resulting skeleton of this robust iterative process is followed by an orientation/propagation step that relies on the most significant 3-point information terms to eventually provide the direct causal effects of graphical models (sections 6.3.2.2 & 6.4.1).

Earlier hybrid methods have also attempted to improve network reconstruction by combining the concepts of constraint-based approaches with the robustness of Bayesian scores [START_REF] Cano | A score based ranking of the edges for the pc algorithm[END_REF][START_REF] Claassen | A bayesian approach to constraint based causal inference[END_REF][START_REF] Dash | A hybrid anytime algorithm for the construction of causal models from sparse data[END_REF][START_REF] Tsamardinos | The Max-Min Hill-Climbing Bayesian Network Structure Learning Algorithm[END_REF]. In particular, [START_REF] Dash | A hybrid anytime algorithm for the construction of causal models from sparse data[END_REF] have proposed to exploit an intrinsic weakness of the PC algorithm, its sensitivity to the order in which conditional independencies are tested on finite data, to rank these different order-dependent PC predictions with Bayesian scores.

More recently, [START_REF] Tsamardinos | The Max-Min Hill-Climbing Bayesian Network Structure Learning Algorithm[END_REF] have also combined constraint-based and Bayesian approaches by first reconstructing an undirected skeleton through the identification of both parents and children of each nodes of the underlying graphical model and then orienting the edges (and possibly deleting some of them) through a Bayesian hill-climbing heuristics approach restricted to the skeleton.

This Max-Min Hill-Climbing (MMHC) approach tends to have a high precision in terms of skeleton but a rather limited sensibility, leading overall to lower skeleton and CPDAG F-scores than 3off2 and Bayesian hill climbing methods on the same benchmark networks (Chapter 7). Interestingly, however, the MMHC approach is among the fastest network reconstruction approaches allowing for scalability to large network sizes [START_REF] Tsamardinos | The Max-Min Hill-Climbing Bayesian Network Structure Learning Algorithm[END_REF].

The 3off2 algorithm is expected to run in polynomial time on typical sparse causal networks with low in-degree, just like constraint-based algorithms. However, in practice and despite the additional computation of conditional 2-point and 3-point information terms, we found that the 3off2 algorithm runs typically faster than constraint-based algorithms for large enough samples, due to a cascading accumulation of errors that inflate the combinatorial search of conditional independencies.

Instead, we found that 3off2 running time displays a similar trend as Bayesian hillclimbing heuristic methods, (Chapter 7 and Appendices A & B).

As exemplified through Chapters 8 to 11, the 3off2 method can be applied to various biological complex systems, ranging from the transcription factors interplay within a cell (Chapter 10) to the neural activity of the whole brain of a zebrafish larvae (Chapter 9). This could be achieved thanks to the robustness of the 3off2 method that can confidently identify direct causal effects from observational data despite the sampling noise inherent to finite biological datasets. In particular, the 3off2 method can uncover causal graphical models from datasets with missing values, which is a widely encountered issue not only in the field of systems biology but also in epidemiology or social sciences. As introduced in section 6.4.2, the 3off2 method can also allow for the presence of latent variables, an important issue when reconstructing causal graphical models, as one is not typically aware of the whole set of factors or parameters playing a role in the complex process under scrutiny.

The 3off2 method has been extensively evaluated on simulated and real-life-like datasets against state-of-the-art methods (Chapter 7) and complementary tests have also been performed in order to evaluate the 3off2 hybrid approach against earlier hybrid methods [ [START_REF] Affeldt | Robust reconstruction of causal graphical models based on conditional 2-point and 3-point information[END_REF]. However, the ability of the ALARM 37 nodes, 46 links, 509 parameters, mean degree 2.49, maximum in-degree 4

BARLEY 48 nodes, 84 links, 114,005 parameters, mean degree 3.5, maximum in-degree 4

HEPAR II 70 nodes, 123 links, 1,453 parameters, mean degree 3.51, maximum in-degree 6

A.1 Evaluation of the PC method by significance level

In this section, the results of the PC inference method [START_REF] Spirtes | An algorithm for fast recovery of sparse causal graphs[END_REF],

as implemented in the pcalg package [Kalisch andBühlmann, 2008, Kalisch et al., 2012], are evaluated using the following parameters values:

• a significance level α = 0.001 (PC 1e-03)

• a significance level α = 0.01 (PC 1e-02)

• a significance level α = 0.1 (PC 1e-01)

In particular, the stable implementation of the PC algorithm has been used, as well as the majority rule for the orientation and propagation steps [START_REF] Colombo | Order-independent constraint-based causal structure learning[END_REF]. As shown in Figures A.1

A.3 Evaluation of the Bayesian methods by score

In this section, the results of the Bayesian inference method using a hill-climbing (HC) heuristics with 100 random restarts [START_REF] Chickering | Learning Bayesian networks: Search methods and experimental results[END_REF], as implemented in the bnlearn package [START_REF] Scutari | Learning Bayesian Networks with the bnlearn R Package[END_REF], are evaluated using the following parameters values:

• Bayesian Dirichlet equivalent (BDe) score (HC BDe)

• Akaike Information Criterion (AIC) (HC AIC)

• Bayesian Information Criterion (BIC) score (HC BIC)

As shown in Figures A.11 A.11 ALARM network [37 nodes,46 links,509 parameters,Average degree 2.49,. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the Bayesian inference approach with BDe, AIC or BIC scores. The Bayesian inference method using the hill climbing heuristics and the BDe score didn't converge for 4 datasets out of 50 at sample size N = 10. For these non-converging reconstructions, the execution time has been set to 3, 600 seconds. A.12 BARLEY network [48 nodes,84 links,114,005 parameters,Average degree 3.5,. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the Bayesian inference approach with BDe, AIC or BIC scores. The Bayesian inference method using the hill climbing heuristics and the BDe score didn't converge for (i) 41 datasets out of 50 at sample size N = 10, (ii) 36 datasets out of 50 at sample size N = 50, (iii) 13 datasets out of 50 at sample size N = 90 and (iv) 1 dataset out of 50 at sample size N = 110. For these non-converging reconstructions, the execution time has been set to 3, 600 seconds. . Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the Bayesian inference approach with BDe, AIC or BIC scores. The Bayesian inference method using the hill climbing heuristics and the BDe score didn't converge for 5 datasets out of 50 at sample size N = 10. For these non-converging reconstructions, the execution time has been set to 3, 600 seconds. • 3off2 MDL(rank I): 3off2 inference approach using Minimum Description Length (MDL) criterion and non-shifted 2-point and 3-point information terms in the rank of individual edges As discussed in section 6.3.2, the MDL complexity using shifted 2-point and 3-point information terms leads the 3off2 inference approach to cumulate false negative edges with smaller datasets. Yet, the 3off2 algorithm gives good results when using the MDL criterion with non-shifted 2-point and 3-point information terms in the rank of individual edges. As shown in Figures A. 21-A.25, CPDAG F-scores of the 3off2 reconstruction method are typically better or comparable to the Bayesian hill-climbing heuristics using the BIC score, except for the CHILD benchmark network, although 3off2 eventually reaches better results than the Bayesian inference appraoch for very large datasets (N > 20, 000).

A.6 Execution time comparisons

This section is comparing the execution time of the different inference methods when reconstructing the CPDAG of the causal benchmark network that has generated the datasets. The methods and parameters values are:

• 3off2 MDL(rank I): 3off2 inference approach using Minimum Description Length (MDL) criterion and non-shifted 2-point and 3-point information terms in the rank of individual edges • 3off2 NML(rank I'): 3off2 inference approach using Normalized Maximum Likelihood (NML) criterion and shifted 2-point and 3-point information terms in the rank of individual edges

• PC 1e-01: PC inference method [START_REF] Spirtes | An algorithm for fast recovery of sparse causal graphs[END_REF] implemented in the pcalg package [Kalisch andBühlmann, 2008, Kalisch et al., 2012] with a significance level α = 0.1

• HC BIC: Bayesian inference using Bayesian Information Criterion (BIC) score and hill-climbing (HC) heuristics with 100 random restarts [START_REF] Chickering | Learning Bayesian networks: Search methods and experimental results[END_REF] implemented in the bnlearn package [START_REF] Scutari | Learning Bayesian Networks with the bnlearn R Package[END_REF] • MMHC BIC(α = 1e -1 ): Hybrid approach [START_REF] Tsamardinos | The Max-Min Hill-Climbing Bayesian Network Structure Learning Algorithm[END_REF] reconstructing first an undirected skeleton through the identification of the parents and children of each node of the underlying graph and then orienting the edges of the skeleton through a Bayesian hill climbing heuristics (possibly removing edges) using the BIC criterion with significance parameter α = 0.1 (using BDe criterion and the range α = 0.001 -0.1 gives very similar results for all tested benchmarks, not shown). The MMHC method is implemented in the bnlearn package [START_REF] Scutari | Learning Bayesian Networks with the bnlearn R Package[END_REF].

As shown in Figure A.26, the 3off2 reconstruction method is typically fast for the five benchmark networks. In particular, although the PC algorithm is faster at small sample sizes, the 3off2 inference approach becomes rapidly faster on larger datasets. This could be explain by the accumulation of errors during the reconstruction of the PC inference approach, leading to perform too many statistical tests on false positive edges.

Figure 1 .

 1 Figure 1. Prevalence of Retained Ohnologs in the Human Genome within Different Gene Classes

Figure 2 .

 2 Figure 2. Ka/Ks Distributions for WGD and SSD or CNV Duplicates in the Human Genome (A-D) Ka/Ks distributions for human-human (Hs-Hs) ohnolog pairs (A) and human-amphioxus (Hs-Bf) ortholog pairs (B) with different confidence status (see Extended Results). Ka/Ks distributions for human-amphioxus (Hs-Bf) ortholog pairs involving a human ohnolog (C) and for human-amphioxus (Hs-Bf) ortholog pairs exhibiting either SSD or CNV (D).See also the Extended Results, FiguresS2 and S3, and TableS3for statistical significance and comparison with other invertebrate outgroups.

Figure 4 .

 4 Figure 4. Evolutionary Trends of Duplicated Genes following SSD or WGD(A-C) Horizontal lines represent the genome of different individuals. Square blocks symbolize the genes, duplicated (SSD: blue; WGD: red) or not (black). Black crosses highlight the loss of one gene (small crosses) or the elimination of an individual (larger crosses), whereas bordered square blocks emphasize retained mutated copies. Evolutionary scenarios are depicted at the population genetics level following either a SSD (left panel) or a WGD (right panel) in one or a few individuals of an initial population. Unlike SSD, WGD is invariably coupled to a speciation event, owing to the difference in ploidy between pre-and post-WGD individuals. Three possible scenarios-beneficial (A), neutral or nearly neutral (B), or deleterious mutations (C) in one gene duplicate-are outlined in post-SSD and post-WGD populations. The main difference concerns the mutation/selection process of ''dangerous'' genes, i.e. genes prone to autosomal-dominant deleterious mutations (C). See main text for a detailed description.

Figure 5 .

 5 Figure 5. Estimates of Ohnolog Retention Rates

Figure 2

 2 Figure2. Distributions of WGD, SSD, and singletons for human orthologs of mouse genes (A) tested for essentiality in mouse[START_REF] Eppig | The Mouse Genome Database (MGD): comprehensive resource for genetics and genomics of the laboratory mouse[END_REF], (B) found to be essential in mouse, and (C and D) after removing dominant disease genes, oncogenes, and genes with dominant negative mutations or autoinhibitory folds[START_REF] Singh | On the expansion of "dangerous" gene repertoires by whole-genome duplications in early vertebrates[END_REF]. (*) corresponds to small deviations (10 23 ,p,0.05, FE test) from the references in (A). Note that human orthologs of essential genes in mouse do not show any significant deviations in WGD, SSD, or singleton contents (p.0.05, FE test) once dominant disease genes, oncogenes, and genes with dominant negative mutations or autoinhibitory folds have been removed. Yet, taking into account the age of SSD duplicates reveals a relative lack of recent SSD genes in essential genes (see text). doi:10.1371/journal.pcbi.1003754.g002

Figure 1 .

 1 Figure 1. Distributions of WGD, SSD, and singletons in (A) the whole human genome, (B) monogenic disease (MD) genes [1], (C) recessive MD genes, and (D) dominant MD genes. (***) corresponds to highly significant deviations (p,10 26 , FE test) and (**) to significant deviations (p,10 23 , FE test) from the references in (A). Note that recessive MD genes (C) do not show any significant deviations in WGD, SSD, or singleton contents (p.0.3, FE test), although taking into account the age of SSD duplicates reveals a relative lack of recent SSD genes in MD genes (see text). doi:10.1371/journal.pcbi.1003754.g001

  is said to be a v-structure. All the other possible orientations correspond to non-v-structures (Fig. 2.1).

Figure 2 . 1

 21 Figure 2.1 Possible non-v-structures for an unshielded triples.

  1. every edge of a CPDAG exists in each DAG of the Markov equivalence class 2. every undirected edge of a CPDAG is oriented as → in at least one DAG and as ← in at least one other DAG of the Markov equivalence class If one assumes faithfulness and causal sufficiency (i.e., no unmeasured common causes and no unmeasured selection variables, see section 2.2), a CPDAG C can be learned from the conditional independences between the observed random variables and used to represent the equivalence class of DAGs of the underlying causal graphical model.

Figure 2 . 2

 22 Figure 2.2 Spurious correlations due to latent confounders and selection variables. {A, Ef , R} are the observed variables, H is a latent variable and Sel is a selection variable. In Fig. 2.2b, the red edge indicates the 'spurious correlation' due to the selection variable Sel , while the blue edge indicates the 'spurious correlation' induced by the hidden common cause H .

Figure 2 . 3

 23 Figure 2.3 An example of DAG with latent variables. {Xi, X k , Xj} are observed variables, while {La, L b } are latent variables.

Definition 2 . 2 . 3 .

 223 MaximalityAn ancestral graph is said to be maximal if for any two non-adjacent vertices, there is a set of vertices that m-separates them. It can be shown that given any DAG G over V = O ∪ L ∪ S, where O is the set of observed variables, L is the set of latent variables, S is the set of selection variables and O, L, S are disjoint sets, there exists a unique maximal ancestral graph (MAG) M G over O alone that probabilistically represents G [Richardson

  cies can be represented by a Partial Ancestral Graph (PAG), having possibly six kinds of edges: , →, ←→, •, • • and •→. The bi-directed edges highlight the existence of hidden variables, and the undirected edges are the result of the selection variables. The following gives the definition of a PAG: Definition 2.2.5. Partial Ancestral Graph (PAG) Let G be a DAG over V = O ∪ L ∪ S -with O the set of observed variables, L the set of latent variables, S the set of selection variables and O, L, S disjoint sets-.

  the PAG over the observed variables O that represents the Markov equivalence class of MAGs of the underlying causal graphical model of Fig. 2.3. In this representation, the '>' around X k

Figure 2 . 4

 24 Figure 2.4 Example of a DAG G over V = O ∪ L ∪ S. {Xi, X k , Xj} ⊆ O where O is the set of observed variables and {L} ⊆ L where L is the set of latent variables.

  2.2.1, these Possible-D-SEP set of variables can be used to learn the representative of the Markov equivalent class of DAGs of the underlying causal graphical model when allowing for the presence of latent and selection variables. Given observational data, the problem of learning a DAG or a MAG (or, a representative of the corresponding Markov equivalence class, i.e. a CPDAG or a PAG, resp.) can be solved by two main types of methods relying either on a contraintbased approach (see Chapter 3), or on a search-and-score process (see Chapter 4).

Algorithm 1 :

 1 The PC algorithm In: observational data of variables O; an ordering order (O) on the variables; a confidence level α Out: CPDAG C 0. Initiation Start with a complete undirected graph G Let = 0 1. Iteration repeat while ∃ XY link with |adj G (X)\{Y }| do while XY ⊂ adj G and

Fig

  Fig. B.7 -B.12. Besides, as with large datasets the dependencies tend to have pvalues approaching 0 while the p-value of independencies are uniformly distributed from 0 to 1, one should also lower the significance level when the sample size increases, as can be seen in Fig. A.1 -A.5.

  allows the discovery of the representative of the Markov equivalence class of DAGs, from observed conditional independence information, while allowing for the presence of latent and selection variables. Algorithm 2 gives the details of the FCI algorithm. First, the discovery of the skeleton with the PC algorithm (Algorithm 2, step 1.) and the identification of the v-structures (Algorithm 2, step 2.) are performed. Then, complementary conditional independence tests, for which the conditioning set of variables is search only in the Possible-D-SEP (see section 2.2.3.3), are processed (Algorithm 2, step 3.). Before applying the final orientation rules (Algorithm 2, step 4.), the edges of the skeleton are all set to •-•. Finally, the propagation step follows the rules R 1

  3.1, taken from[START_REF] Colombo | Learning highdimensional directed acyclic graphs with latent and selection variables[END_REF] (section 3.3), illustrates the reconstruction of the PAG over O (Fig. 3.1, (c)) that represents the markov equivalence class of the DAG G over V = O ∪ L ∪ S (Fig. 3.1, (a)). The causal independence relationships have been deduced from the observed variables, {X i } (Fig. 3.1).

Algorithm 2 :

 2 The FCI algorithm In: observational data of variables O; an ordering order (O) on the variables; a confidence level α Out: (a mixed graph interpretable as) a PAG C [Zhang, 2008b] 0. Initiation Start with a complete undirected graph C with edges •-• 1. Initial skeleton, separation sets, out: C 1 Do PC algorithm to find an initial skeleton C 1 and the separation sets (Algorithm 1) 2. Initial orientations, out: C 2 forall the unshielded triples in C 1 do

  yet considered with |Y| = do if Indep(X; Y |{Y ∪ S}) at conf. level α then XY link is non-essential and removed separation set of XY : Sep XY = {U i } in C 2 as •-• 4. Final orientation, out: C 4 forall the unshielded triples in C 4 do

Figure 3 . 1

 31 Figure 3.1 DAG G over V = O ∪ L ∪ S and corresponding skeleton and PAG. {Xi} ⊆ O where O is the set of observed variables and {Li} ⊆ L where L is the set of latent variables. The PAG (c) corresponding to the underlying DAG (a) has been reconstructed using the FCI algorithm and the conditonal independence from observed variables (see main text).

  ference methods. Chapter 4 provides an overview on Bayesian structure discovery approaches that follow the search-and-score paradigm. Bayesian approaches have the advantage to not rely on statistical tests that are prone to erroneous removal on finite datasets. The search-and-score approaches also provide a global score for the learn causal graphical model, allowing the comparison between different structure candidates. Yet, classical Bayesian inference approaches need to exclude the possibility of hidden latent variables.Chapter 4

  a given scoring function, in other words, it consists in finding the graphical model G that optimizes the posterior probability P (G|D), where D is the dataset with N i.i.d. samples of the vector X. The posterior probability is related to the likelihood, P (D|G), of the data D given a graphical model G, and to the joint probability, P (G, D), through the Bayes Formula as,

  prior distribution as compared to the number of data points. Larger values of N tend to oversmooth the data and add two many edges to the structure. Typical values of N are 1, 5, 10, 20, 50. For our evaluations on benchmark networks (Chapter 7), the default value 10 has been chosen, as it corresponds to a standard value in the literature.

  scoring function limits the overfitting phenomenon by penalizing the LL score with the number of parameters required by the network under reconstruction, thus preferring simpler Bayesian networks. Using the MDL criterion can be interpreted has the minimization of (i) the coding length of the model, plus (ii) the coding length of the representation of the data when encoded by this model. The scoring function is thus defined as, MDL(D|G) = LL(D|G) --1)q i is the network complexity, i.e. the number of parameters needed for the model G to encode D. The first term of Eq. 4.8 stands for the number of bits needed to describe D while the second term measures the length of describing G, where each parameter of G counts for1 2 log(N ) bits. The MDL scoring function thus provides an appropriate trade-off between precision (maximization of the LL function) and complexity (minimization of the number of parameters).

  i , y j ) log(p(x i |y j ))(5.4) Eqs. 5.1, 5.4 and 5.3 lead to the 2-point information or mutual information, that can be interpreted as the reduction of the uncertainty on X and Y by the joint uncertainty of these two variables:

Theorem 5 . 1 . 1 .

 511 Data Processing Inequality Let's suppose that X, Y and Z are three variables related by the causal graphical model X → Y → Z, then I(X; Z) ≤ min(I(X; Y ), I(Y ; Z)).

  Algorithm for the Reconstruction of Accurate Cellular Network (ARACNe)[START_REF] Margolin | ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context[END_REF] has been widely applied in the field of reverse engineering of regulatory networks. It is an efficient method, taking advantage of the 2-point information (Eq. 5.5) and of the DPI (Eq.5.1.1). As a first step, the 2-point information, I(X; Y ), of all pairs of variables are computed and compared to a given threshold I 0 . Each pair having a 2-point information less than I 0 is removed. Then, relying on the DPI, the ARACNe method removes also the edge of all closed triples corresponding to the weakest 2-point information. Indeed, the ARACNe methods is assuming that each closed triples is embedding a non-v-structure and thus that the weakest edge indicates an indirect relationship that should be discarded from the graphical model.Although the ARACNe algorithm has shown satisfactory reconstructions on large networks, the systematic opening of the closed triples prevents from finding possible cycles and tends to increase the number of false negative edges. A parameter has been proposed to smooth the systematic removal of the weakest edge of certain closed triples. This corresponds to a percent tolerance threshold. Thus, if the weakest edge of the close triple X, Z, Y is X Z, it will be removed only if I(X; Z) ≤ I(X; Y )(1 -) and I(X; Z) ≤ I(Y ; Z)(1 -). One remaining issue is the absence of orientations as the ARACNe algorithm only provides a skeleton.

  -H. In such a case, Z might contribute to the mutual information I(X; Y ) and should be included in the set of upstream nodes {U i } n , thereby defining a generalized non-v-structure, Figures 6.1F-H.

1 . 3 )

 13 , it cannot indicate how to orient embedded v-structures or non-v-structures over the XY Z triple, as already noted in the case of isolated v-structures and non-v-structures, above. However, the most likely base (XY , XZ or Y Z) of the embedded v-structure or non-v-structure corresponds to the least correlated pair conditioned on {U i }, e.g., I(X; Y |{U i }) = min XY Z I(S; T |{U i }) , as shown with the following likelihood ratios,

5 )

 5 Note, in particular, that choosing the base with the lowest conditional mutual information, e.g., I(X; Y |{U i }) = min XY Z I(S; T |{U i }) , is consistent with the Data Processing Inequality expected for the generalized non-v-structure of Figures 6.1F-H, I(X; Y ) min I(X; Z, {U i }), I(Z, {U i }; Y ) , as shown below for I(X; Y ) and I(X; Z, {U i }), by subtracting I(X; Y ; Z|{U i }) on each side of the inequality I(X; Y |{U i }) I(X; Z|{U i }), leading to,

Figure 6 . 1

 61 Figure 6.1 Inference of v-structures versus non-v-structures by 3-point information from observational data.

Corollary 2 ( 3 -

 23 point contribution). ∀X, Y, Z ∈ V and ∀{U i } ⊆ V \{X,Y,Z} s.t. I(X; Y ; Z|{U i }) > 0, then I(X; Y |{U i }) > 0 (as well as I(X; Z|{U i }) > 0 and I(Y ; Z|{U i }) > 0 by symmetry of I(X; Y ; Z|{U i })).

Theorem 4 (

 4 negative conditional 3-point information as statistical evi-dence of causality). If ∃X, Y, Z ∈ V and {U i } ⊆ V \{X,Y,Z} s.t. I(X; Y |{U i }) = 0and I(X; Y ; Z|{U i }) < 0 then, G is (partially) directed, i.e. some variables in G are causally linked, either directly or indirectly through other variables, including possibly unknown, 'latent' variables unobserved in G.

2 .

 2 and removed separation set of XY : Sep XY = {U i } else find next most contributing node Z neighbor of X or Y and compute new 3off2 rank of XY : R(XY ; Z|{U i }) end sort the 3off2 rank list R(XY ; Z|{U i }) end end Orientation / Propagation Sort list of unshielded triples,

  ), the most likely v-structure is used to set the first orientations, following R 0 orientation rule. The possible propagations are then performed, following R 1 propagation rule, starting from the unshielded triple having the most positive conditional 3-point information. The following most likely v-structure is considered when no further propagation is possible on unshielded triples with greater absolute 3-point information. If conflicting orientations arise (such as a → b ← c & b → c ← d), the less likely v-structure and its possible propagations are ignored, unless latent variables are considered (section 6.4.2).

6. 4 . 1

 41 Probabilistic estimation of the edge endpointsGiven the skeleton obtained from Algorithm 4, Eqs.6.30 and 6.31 lead to the following Proposition 7 and Proposition 8 for the orientation and propagation rules of unshielded triples, which are equivalent to the Propostion 5 and Proposition 6 but for underlying DAG models (assuming no latent variables) and for finite datasets with the corresponding probabilities for the initiation/propagation of orientations.Proposition 7 (Significantly negative conditional 3-point information as robust statistical evidence of causality in finite datasets). Assuming that the underlying graphical model is a DAG

Figure 6 . 2

 62 Figure 6.2 The 3off2 reconstruction pipeline.

Figure 6 . 3

 63 Figure 6.3 The discoNet reconstruction pipeline.

Figure 7 . 1

 71 Figure 7.1 CHILD network [20 nodes, 25 links, 230 parameters, Average degree 2.5, Maximum in-degree 2]. Precision, Recall and F-score for skeletons (dashed lines) and CPDAGs (filled lines). The results are given for Aracne (black), PC (blue), Bayesian Hill-Climbing (green) and 3off2 (red).

Figure 7 . 2

 72 Figure 7.2 ALARM network [37 nodes, 46 links, 509 parameters, Average degree 2.49, Maximum in-degree 4]. Precision, Recall and F-score for skeletons (dashed lines) and CPDAGs (filled lines). The results are given for Aracne (black), PC (blue), Bayesian Hill-Climbing (green) and 3off2 (red).

Figure 7 . 3

 73 Figure 7.3 INSURANCE network [27 nodes, 52 links, 984 parameters, Average degree 3.85, Maximum in-degree 3].

Figure 7 . 4

 74 Figure 7.4 BARLEY network [48 nodes, 84 links, 114,005 parameters, Average degree 3.5, Maximum in-degree 4]. Precision,

Figure 7 . 5

 75 Figure 7.5 HEPAR II network [70 nodes, 123 links, 1,453 parameters, Average degree 3.51, Maximum in-degree 6]. Precision, Recall and F-score for skeletons (dashed lines) and CPDAGs

  2.2). Figures B.1-B.6 provide additional results on the prediction of the network skeletons and execution times. The alternative predictions are based on the stable implementation of the PC algorithm as well as the majority and conservative rules for the orientation and propagation steps [Colombo and Maathuis, 2014]. The PC and MMHC results are shown, Figures 7.6-7.10, for an independence test parameter α = 0.1, as reducing α tends to worsen the CPDAG F-score for benchmark networks with k ≥ 1.6 2

  , using AIC, BDe and BIC/MDL scores and hill-climbing heuristics with 30 to 100 random restarts, Figs B.19 -B.30. 3off2 reaches equivalent or significantly better F-scores than Bayesian hill-climbing for all dataset sizes on benchmark networks up to 120 edges ( k = 4.8). In particular, 3off2 with MDL scores reaches excellent F-scores on sparse networks (Figures B.19& B.20) and keeps one of the best F-scores over all sample sizes for less sparse networks when combined to NML complexity (Figures B.21-&-B.22). For somewhat denser networks ( k 5), the 3off2 F-score appears slightly lower than for Bayesian inference methods, Figure B.23, although it eventually becomes equivalent for large datasets (N 1000).

Figure 7 . 6 Figure 7 . 7

 7677 Figure 7.6 50 node, 40 edge benchmark networks generated using Tetrad. k = 1.6, k in max = 3.2, k out max = 3.6.

Figure 7 . 8 Figure 7 . 9

 7879 Figure 7.8 50 node, 80 edge benchmark networks generated using Tetrad. k = 3.2, k in max = 4.8, k out max = 5.6.

Figure 7 .

 7 Figure 7.10 50 node, 160 edge benchmark networks generated using Tetrad. k = 6.4, k in max = 8.6, k out max = 8.6.

Figure 7 .

 7 Figure 7.11 undirected-ALARM autocorrelation decay.

Figure 7 .

 7 Figure 7.12 undirected-INSURANCE autocorrelation decay.

8. 1

 1 Enhanced retention of ohnologs 8.1.1 Enhanced retention of 'dangerous' ohnologs

Figure 8 . 1

 81 Figure 8.1 Mediation diagram. This diagram shows the direct effect, DE xx , of changing X from x to x onto the outcome Y , and the indirect effect, IE xx , mediated by m(x).

  If X, M , and Y are binary variables, expectation values of Y , and values of M drawn from a conditional distribution P , can be estimated from the counts of the different possible triples of values for (X, M, Y ). Let's define n xmy as the number of (X, M, Y ) triples where (X, M, Y ) = (x, m, y). Then, E (Y |X = 0, M = 0) amounts to n 001 n 000 +n 001 . Similarly, P (M |X = 0) can be estimated by n 010 +n 011

Figure 8 . 2

 82 Figure 8.2 Evolutionary trends of duplicated genes following a WGD. Horizontal lines represent the genome of different individuals. Square blocks symbolize the genes, duplicated (grey) or not (black). Black crosses highlight the loss of one gene (small crosses) or the elimination of an individual (larger crosses), while bordered square blocks emphasize retained mutated copies. See main text for a detailed discussion of the three mutation/selection scenarios (A-C) and resulting evolutionary trends.From[START_REF] Affeldt | Évolution et cancer -expansion des familles de gènes dangereux par duplication du génome [evolution and cancer: expansion of dangerous gene repertoire by whole genome duplications] (article in french)[END_REF]

8. 4

 4 Reconstructing the interplay of genomic properties on the retention of ohnologs8.4.1 Genomic properties of interestThe 3off2 reconstruction method has been used to learn causal graphical models among several genomic properties associated with human protein coding genes and for which information has been computed in our group or collected from the literature. As I was primarily interested in disentangling the direct and indirect effect of the susceptibility to dominant deleterious mutations, and dosage balance constraints on the retention of ohnologs, I have first considered the genomic properties from (

  deleterious mutations. Genes for which Ka/Ks ratio could not be calculated were discarded from the analysis, leading to 15, 508 datapoints for this property. The remaining continuous Ka/Ks values have been binarized 0/1 with the threshold at 75% quantile across all human protein coding genes. Thus, the Ka.Ks property used in the 3off2 reconstructions (Fig.8.3 & 8.4) corresponds to 'high divergence of the sequence' (0[no]/1[yes]).

8. 4 . 2

 42 Non-adaptive retention mechanism supported by the 3off2 causal models The interactions between the first 9 genomic properties, as recovered by the 3off2 method, are given in Fig. 8.3. The colors indicate the strength of the relationships (from 'weak' [light blue/yellow/light grey] to strong [dark blue/red/dark grey]).The arrowheads indicate the direction of the effect, and the colors provide also the sign, with warm colors indicating a positive effect and cold colors indicating a negative effect. Finally, the non oriented edges suggest an absence of cause-effect mechanism, while the bi-directed edges indicate a common unobserved cause from a latent variable (section 6.4.2).

Figure 8 . 3

 83 Figure 8.3 Causal network of 9 genomic properties reconstructed with the 3off2 method. The colors indicate the strength of the relationships (from 'weak' [light blue/yellow/light grey] to strong [dark blue/red/dark grey]) and provide also the sign, with warm colors indicating a positive effect and cold colors indicating a negative effect. The arrowhead gives the direction of the effect. The non oriented edges suggest an absence of cause-effect mechanism, while the bi-directed edges indicate a common unobserved cause. See main text for detailed on the properties.

Figure 8 . 4

 84 Figure 8.4 Causal network of 12 genomic properties reconstructed with the 3off2 method.The colors indicate the strength of the relationships (from 'weak' [light blue/yellow/light grey] to strong [dark blue/red/dark grey]) and provide also the sign, with warm colors indicating a positive effect and cold colors indicating a negative effect. The arrowhead give the direction of the effect. The non oriented edges suggest an absence of cause-effect mechanism, while the bi-directed edges indicate a common unobserved cause. See main text for detailed on the properties.

Figure 9 . 1

 91 Figure 9.1 Scheme of main brain areas in zebrafish larvae. This scheme provides a dorsal view of zebrafish larvae brain. OT, optic tectum; OT spv, optic tectum stratum periventriculare; IO, inferior olive; Pt, pretectal area; Hab, habenula; Rhdb, rostral hindbrain; Chdb, caudal hindbrain; Dhdb, dorsal hinbrain. The gray dashed regions indicate more ventrally located areas.

9 . 1 )

 91 tend to temporally precede the activity in the areas related to motor structures (e.g. rostral hindbrain (Rhdb) in the cerebellum region, ventral/dorsal hinbrain (Chdb/Dhdb) in the medulla region; Fig.9.1)[START_REF] Portugues | Whole-brain activity maps reveal stereotyped, distributed networks for visuomotor behavior[END_REF]. This temporal patterns are further discussed in section 9.2, in relation with the 3off2 network reconstructions.

Figure 9 . 2

 92 Figure 9.2 Experimental setup of the SPIM-based functional imaging system (Jean Perrin laboratory, CNRS-UPMC). Schema of the experimental SPIM setup at the Jean Perrin laboratory, G. Debregeas & R. Candelier team. [Panier et al., 2013].

Fig. 9 . 2 ,

 92 the zebrafish larvae is kept paralysed in a low-melting temperature agarose solution within a tank that can be vertically translated via a piezo-positioner, allowing observations at different depths of the brain. The laser light sheet enables the capture of fluorescent signals emitted by the neurons and provides an image which is formed on the sCMOS sensor. This image undergoes a series of manual and automated filtering to discard rare remaining movements of the paralysed larvae, remove high frequency noise and most importantly, correctly identify the activity of each individual neuron. 9.1.5 From fluorescent signal to neuron activity The fluorescent signal formed on the sCMOS camera of the SPIM imaging system (Fig. 9.2), and the actual electrical signal differ by their amplitude, duration and shape. While the amplitude of the electrical signal is typically of about 100 mV, all fluorescent signals do not share the same amplitude. Besides, a spike lasts about 1ms while the relaxation of the fluorescent signal may reach up to 2s. Finally, instead of a spike, the fluorescent signal follows a double exponential law, f (t) ∝ e λr +e -λ d , where λ r and λ d are two characteristic times that depend on the calcium indicator (Fig. 9.4, blue curve; Results provided by Sébastien Wolf, Debrégeas research group). For each neuron, the fluorescence signal is not directly considered, but instead compared to the cell baseline fluorescence, b slow . Hence, the considered signal is ∆F/F = (F (t) -b slow )/b slow .

Figure 9 . 3

 93 Figure 9.3 Activity of a zebrafish neuron (blue), and the corresponding filtered signal using a Wiener filter (green).

Figure 9 . 4

 94 Figure 9.4 Magnification of figure 9.3

Figure 9 . 5

 95 Figure 9.5 Action potential train of a zebrafish neuron (green), and the corresponding fluorescent signal (red).

Figure 9 . 6

 96 Figure 9.6 Succession of two temporally separated action potentials of a zebrafish neuron (green), and the corresponding fluorescent signal (red).

  5 (results provided by Sébastien Wolf, Debrégeas research group), the filter is able to recover individual peaks from a fluorescent signal, even when action potentials from a same train get superimposed. The method proposed by G. Debrégeas correctly extracts temporally closed peaks, as shown in the Fig. 9.7 (results provided by Sébastien Wolf, Debrégeas research group).

Figure 9 . 7

 97 Figure 9.7 Fluorescent signal of a zebrafish action potential train (red) and the corresponding filtered signal (black) when the filter from the Jean-Perrin Laboratory is applied.

Figure 9 . 8

 98 Figure 9.8 Fluorescent signal of two temporally separated action potentials of a zebrafish neuron (red) and the corresponding filtered signal (black) when the filter from the Jean-Perrin Laboratory is applied. In section 9.2, the datasets used by 3off2 correspond to a binary version of the (downward) peak of Fig. 9.7 & 9.8 (Results provided by Sébastien Wolf, Debrégeas research group). Above a given threshold, the neuron will have the value 1, otherwise 0.

Figure 9 . 9

 99 Figure 9.9 Active areas during OKR behaviour. Color-coded activity phase of regions segmented from volumetric imaging data across three fish. The different colors highlight the temporal alternating patterns. The medulla (caudal part of the brain) shows a columnar alternating activity. Two areas in the pretectum (arrowheads) also display an alternating pattern. (Image taken from [Portugues et al., 2014])

Fig. 9 .

 9 10 & 9.11 show the reconstructed networks from the binary data corresponding to the neural activity in layer 5, when considering 15 (Fig.9.10) and 30 (Fig.9.11) clusters. The discovered edges are all associated with activation, and the color gives the strength of the relationship, from light yellow to red, red being the strongest. Grey edges are the non-oriented relationships. In Fig. 9.11, the clusters within the medulla area are related by a directed path displaying a possible left/right temporally alternating pattern, (Cl 10 → Cl 5 → Cl 7 → Cl 3 → Cl 4 → Cl 2 ). In the reconstructed network based on 15 clusters, this temporal pattern can not be seen, except within the oriented edges (Cl 2 → Cl 3 → Cl 1 ) (Fig. 9.10).

Figure 9 .

 9 Figure 9.10 3off2 reconstructed network [Layer 5, z = z0 +28µm, 15 clusters].

Figure 9 .

 9 Figure 9.11 3off2 reconstructed network [Layer 5, z = z0 +28µm, 30 clusters].

Fig. 9 .

 9 Fig. 9.12 & 9.13 show the reconstructed network from the binary data corresponding to the neural activity in layer 6, when considering 15 (Fig.9.12) and 30 (Fig.9.13) clusters.

Figure 9 .

 9 Figure 9.12 3off2 reconstructed network [Layer 6, z = z0 +28µm, 15 clusters].

Figure 9 .

 9 Figure 9.13 3off2 reconstructed network [Layer 6, z = z0 +28µm, 30 clusters].

Cl 2 →

 2 construction based on 15 clusters displays the expected temporal pattern, i.e. from sensory (inferior olive) to motor (caudal hindbrain) structures (edges Cl 1 → Cl 3 & Cl 4 ), the reconstruction relying on 30 clusters display an opposite (weak) temporal pattern (edges Cl 3 → Cl 1 & Cl 7 → Cl 2

Figure 9 .

 9 Figure 9.14 3off2 reconstructed network [Layer 11, z = z0 + 70µm, 15 clusters].

Figure 9 .

 9 Figure 9.15 3off2 reconstructed network [Layer 11, z = z0 + 70µm, 30 clusters].

  9.16) of the contralateral hemisphere. Neurons characterizing the stimulus are activated in the rostral tectum and recruit assemblies of tectal neurons ([START_REF] Dickerson | On the origins of Mendelian disease genes in man: the impact of gene duplication[END_REF] &[START_REF] Tinti | Evolution of signal multiplexing by 14-3-3-binding 2R-ohnologue protein families in the vertebrates[END_REF]

Figure 9 .Fig. 9 .

 99 Figure 9.16 Model circuit for visual prey recognition and release of hunting reponses. MR, medial rectus; OTc, optic tectum; MRF, mesencephalic reticular formation; EOMN, extra-ocular medial rectus motoneurons; RS, reticulospinal. This scheme provides a model circuit emerging from a prey-like vosual stimulus when observed by the zebrafish from the right temporal retina. See main text for details.

Figure 9 .

 9 Figure 9.173off2 reconstructed network [Layer 2, z = z0 + 7µm, 30 clusters].

Figure 9 .

 9 Figure 9.18 3off2 reconstructed network [Layer 4, z = z0 +21µm, 30 clusters].

Figure 9 .

 9 Figure 9.19 3off2 reconstructed network [Layer 1, z = z0, 30 clusters].

Figure 9 .

 9 Figure 9.20 3off2 reconstructed network [Layer 9, z = z0 +56µm, 30 clusters].

Figure 10 . 1

 101 Figure 10.1 Hematopoietic subnetwork reconstructed by 3off2The dataset[START_REF] Moignard | Characterization of transcriptional networks in blood stem and progenitor cells using high-throughput single-cell gene expression analysis[END_REF] concerns 18 transcription factors, 597 single cells, 5 different hematopoietic progenitor types. Red and blue edges correspond to experimentally proven activations and repressions, respectively as reported in the literature (Table10.1), while grey links indicate regulatory interactions for which no clear evidence has been established so far. Thinner arrows underline 3off2 misorientations.

Figure 11 . 1

 111 Figure 11.1 Mutational pathways reconstruction with the 3off2 method. The network reconstruction has been performed on 26, 494 tumour samples containing information (with missing values) on 23 genes, and assuming the presence of latent variables. The oriented edges indicate the series of mutations that appears to be the most 'beneficial' for the tumour growth. The color of edges gives the direction of the 'advantageous' cascade of mutations for the tumour progression, with warmer colors representing positively correlated mutations and cold colors representing negatively correlated mutations (red and dark blue colors highlighting a stronger association). The bi-directed grey edges corresponds to associations due to one or several unmeasured latent variable(s). The count associated to each edge indicates the number of samples on which the final 2-point and 3-point information terms have been computed.

Figure A. 1

 1 Figure A.1 ALARM network [37 nodes, 46 links, 509 parameters, Average degree 2.49, Maximum in-degree 4]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the PC inference approach.

Figure A. 2

 2 Figure A.2 BARLEY network [48 nodes, 84 links, 114,005 parameters, Average degree 3.5, Maximum in-degree 4]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the PC inference approach.

Figure A. 3

 3 Figure A.3 CHILD network [20 nodes, 25 links, 230 parameters, Average degree 2.5, Maximum in-degree 2]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the PC inference approach.

Figure A. 4

 4 Figure A.4 HEPAR II network [70 nodes, 123 links, 1,453 parameters, Average degree 3.51, Maximum in-degree 6]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the PC inference approach.

Figure A. 5

 5 Figure A.5 INSURANCE network [27 nodes, 52 links, 984 parameters, Average degree 3.85, Maximum in-degree 3]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the PC inference approach.

Figure A. 7 Figure A. 8

 78 Figure A.7 BARLEY network [48 nodes, 84 links, 114,005 parameters, Average degree 3.5, Maximum in-degree 4]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) using the Aracne inference approach.

Figure A. 9 Figure

 9 Figure A.9 HEPAR II network [70 nodes, 123 links, 1,453 parameters, Average degree 3.51, Maximum in-degree 6]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) using the Aracne inference approach.

  -A.[START_REF] Liao | Mouse duplicate genes are as essential as singletons[END_REF], depending on the underlying causal network, one has to choose the most suitable score for the hill-climbing heuristic approach to output its best reconstruction. The AIC score should be preferred for INSUR-ANCE(Figure A.15), the BIC score for the ALARM and HEPAR II(Figures A.11

  & A.14) and the BDe score for the CHILD and BARLEY networks(Figures A.13

Figure

  Figure A.11 ALARM network [37 nodes, 46 links, 509 parameters, Average degree 2.49,

Figure

  Figure A.12 BARLEY network [48 nodes, 84 links, 114,005 parameters, Average degree 3.5,

Figure A. 13

 13 Figure A.13 CHILD network [20 nodes, 25 links, 230 parameters, Average degree 2.5, Maximum in-degree 2]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the Bayesian inference approach with BDe, AIC or BIC scores.

Figure A. 14

 14 Figure A.14 HEPAR II network [70 nodes, 123 links, 1,453 parameters, Average degree 3.51, Maximum in-degree 6]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the Bayesian inference approach with BDe, AIC or BIC scores. The Bayesian inference method using the hill climbing heuristics and the BDe score didn't converge for 5 datasets out of 50 at sample size N = 10. For these non-converging reconstructions, the execution time has been set to 3, 600 seconds.

Figure A. 15

 15 Figure A.15 INSURANCE network [27 nodes, 52 links, 984 parameters, Average degree 3.85, Maximum in-degree 3]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the Bayesian inference approach with BDe, AIC or BIC scores.

Figure A. 16

 16 Figure A.16 ALARM network [37 nodes, 46 links, 509 parameters, Average degree 2.49, Maximum in-degree 4]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the 3off2 inference approach.

  Figure A.17 BARLEY network [48 nodes, 84 links, 114,005 parameters, Average degree 3.5,

  Figure A.17 BARLEY network [48 nodes, 84 links, 114,005 parameters, Average degree 3.5, Maximum in-degree 4]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the 3off2 inference approach.

Figure A. 18

 18 Figure A.18 CHILD network [20 nodes, 25 links, 230 parameters, Average degree 2.5, Maximum in-degree 2]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the 3off2 inference approach.

Figure

  Figure A.19 HEPAR II network [70 nodes, 123 links, 1,453 parameters, Average degree 3.51, Maximum in-degree 6]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using the 3off2 inference approach.

Figure A. 21

 21 Figure A.21 ALARM network [37 nodes, 46 links, 509 parameters, Average degree 2.49, Maximum in-degree 4]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using 3off2, Bayesian hill-climbing and Max-Min Hill-Climbing approaches.

Figure

  Figure A.22 BARLEY network [48 nodes, 84 links, 114,005 parameters, Average degree 3.5, Maximum in-degree 4]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using 3off2, Bayesian hill-climbing and Max-Min Hill-Climbing approaches.

Figure

  Figure A.23 CHILD network [20 nodes, 25 links, 230 parameters, Average degree 2.5, Maximum in-degree 2]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using 3off2, Bayesian hill-climbing and Max-Min Hill-Climbing approaches.

Figure

  Figure A.24 HEPAR II network [70 nodes, 123 links, 1,453 parameters, Average degree 3.51, Maximum in-degree 6]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using 3off2, Bayesian hill-climbing and Max-Min Hill-Climbing approaches.

Figure

  Figure A.25 INSURANCE network [27 nodes, 52 links, 984 parameters, Average degree 3.85, Maximum in-degree 3]. Precision, Recall, F-score and execution time for the reconstruction of the skeletons (dashed lines) and CPDAGs (filled lines) using 3off2, Bayesian hill-climbing and Max-Min Hill-Climbing approaches.

  Figure A.26 Network Reconstruction Execution time Execution time in seconds for the reconstruction of the five studied benchmark networks.

Table 1 .

 1 Ohnolog Families with Both Autoinhibitory and Oncogenic Properties A. Ohnolog Receptor Tyrosine Kinases and Other Receptor Kinases

	ALK	LTK

Table 2 .

 2 Low Retention of Ohnologs in Permanent Complexes

	Permanent Complexes a	Number of Ohnologs	Intrinsic Stoichiometry Disequilibrium of Ohnologs in Permanent Complexes
	ATP F0	3/12	the 3 ohnologs ATP5G1-3 form the 10-mer C-ring of the F-type ATP synthase
	ATP F1	0/5
	COX	2/11	the 2 ohnologs COX4I1,2 coassemble with 3 mitochondrial encoded genes
	SRS	2/32	Ohnologs are X-linked RPS4X (with no X-inactivation) and Y-linked RPS4Y1
	Mitochondrial SRS	0/30
	LRS	2/50	RPL3 and RPL39 have ohnologs RPL3L and RPL39L with unknown functions
	Mitochondrial LRS	0/48
	Proteasome	2/31	ohnologs PSMA7 or PSMA7L are included in the 2 rings of 7 a subunits
	Pyruvate dehydrogenase	0/5
	RNA Pol II	0/12
	RNA Pol III	0/9

COX, cytochrome c oxidase; LRS, large ribosomal subunit; SRS, small ribosomal subunit. a

[START_REF] Zanivan | A new computational approach to analyze human protein complexes and predict novel protein interactions[END_REF]

  Haploinsufficient and dominant negative candidate genes were obtained from parsing OMIM text files with Perl regular expressions. The resulting gene lists were manually curated with the available literature, yielding a total of

		1 1
	p
			autoinhibition & oncogene w/o SSD
		0.8 0.8
	Ohnolog Retention Rates	0.2 0.4 0.6 0.2 0.4 0.6	autoinhibition oncogenes autoinhibition & oncogene autoinhibition & cancer
			all genes
		0 0	0 0	0.2 0.2	0.4 0.4	0.6 0.6	0.8 0.8	1 1
			Observed fractions of ohnologs in gene classes

technical breakthrough for understanding segmentation clock dynamics and synchrony LIENS D'INTÉRÊT

	la sensibilité des techniques d'imagerie, permettront de continuer à développer de nouveaux rapporteurs. En outre, les outils de détection et de suivi des cel-lules, ainsi que l'analyse des oscillations permettront de développer le même type d'approche chez d'autres vertébrés. ‡ A Les auteurs déclarent n'avoir aucun lien d'intérêt	RÉFÉRENCES 1. Pourquié O. Vertebrate segmentation: from cyclic gene networks to scoliosis. Cell 2011 ; 145 : 650-63. 2. Soroldoni D, Oates AC. Live transgenic reporters of the vertebrate embryo's segmentation clock. Curr Opin Genet Dev 2011 ; 21 : 600-5. 3. Masamizu Y, Ohtsuka T, Takashima, Y, et al. Real-time imaging of the somite segmentation clock: revelation of unstable oscillators in the individual presomitic mesoderm cells. Proc Natl Acad Sci USA 2006 ; 103 : 1313-8. 4. Aulehla A, Wiegraebe W, Baubet V, et al. A beta-catenin gradient links the clock and wavefront systems in mouse embryo segmentation. Nat Cell Biol 2008 ; 10 : 186-93. 5. Takashima Y, Ohtsuka T, Gonzalez A, et al. Intronic	6. Delaune EA, François P, Shih NP, Amacher SL. Single-cell-resolution imaging of the impact of Notch signaling and mitosis on segmentation clock dynamics. Dev Cell 2012 ; 23 : 995-1005. 7. Gajewski M, Sieger D, Alt B, et al. Anterior and posterior waves of cyclic her1 gene expression are differentially regulated in the presomitic mesoderm of zebrafish. Development 2003 ; 130 : 4269-78. 8. Jiang YJ, Aerne BL, Smithers L, et al. Notch signalling and the synchronization of the somite segmentation clock. Nature 2000 ; 408 : 475-9. 9. Horikawa K, Ishimatsu K, Yoshimoto E, et al. Noise-resistant and synchronized oscillation of the segmentation clock. Nature 2006 ; 441 : 719-23.
	concernant les données publiées dans cet article.	delay is essential for oscillatory expression in the segmentation clock. Proc Natl Acad Sci USA 2011 ;
		108 : 3300-5.

NOUVELLE

Évolution et cancer Expansion des familles de gènes dangereux par duplication du génome

	Séverine Affeldt 1* , Param Priya Singh 1* , Ilaria Cascone 2 ,
	Rasim Selimoglu 2 , Jacques Camonis 2 , Hervé Isambert 1
	> Si la conservation des gènes essentiels
	à la vie des organismes se conçoit intui-
	tivement bien, à l'inverse, l'étonnante
	expansion des familles de gènes à l'ori-
	gine des cancers ou d'autres maladies
	génétiques chez les vertébrés pose ques-
	tion. Alors qu'on pourrait supposer que
	la multiplication de ces gènes « dan-
	gereux » confère malgré tout un avan-
	tage sélectif, nos travaux récents [1]
	suggèrent en fait que ces gènes ont été
	multipliés et conservés en raison de leur
	dangerosité à la suite de deux accidents
	génétiques majeurs correspondant à des
	duplications globales de génome.

De l'expansion des familles de gènes dangereux chez les vertébrés

	Au delà de ces exemples, nous avons	test  2 ) ou des gènes dont la protéine	Le mécanisme de conservation
	effectué une étude d'exploration de données à grande échelle sur l'en-	est auto-inhibée (76 % ; 350/461 ; p = 2,7 x 10 -77 , test  2 ). De même,	des ohnologues Pourquoi les multiples copies ohnologues
	semble des gènes humains, à partir des	lorsque l'on considère des classes de	de ces gènes dangereux, à l'origine de
	bases de données accessibles en ligne	gènes combinant plusieurs facteurs de	nombreux cancers et maladies géné-
	(comme COSMIC [8], CancerGenes [9])	dangerosité, telles que la classe des	tiques sévères, n'ont-elles pas été éli-
	et de publications. Ceci nous a per-	oncogènes dont la protéine est auto-	minées chez les vertébrés ? Pour le com-
	mis de mettre en évidence une forte	inhibée, ces biais de rétention dépassent	prendre, il faut avoir en tête deux points
	association entre la dangerosité des ohnologues et leur rétention dans le	mais qui a joué un rôle essentiel au les 90 % (91 % ; 104/114 ; p = 6,9 x 10 -37 , test  2 ). À l'inverse, nous avons	de gènes [4-6]. Cependant, de façon essentiels (Figure 2) : (1) la duplication globale du génome, lorsqu'elle n'est pas
	génome humain. Dans ces analyses, nous	cours de l'évolution, notre lignée encore montré que les gènes associés à des	surprenante, on constate que les copies létale, implique nécessairement l'appa-
	avons considéré différentes classes de	invertébrée a entièrement dupliqué son mutations délétères récessives chez	ohnologues retenues dans le génome rition d'une nouvelle espèce composée
	gènes susceptibles d'être affectés par	génome deux fois de suite et a sur-l'homme, ainsi que la plupart des gènes	humain comportent un nombre élevé de d'individus possèdant tous initialement
	des mutations délétères. Ces classes	vécu à ces deux accidents génétiques essentiels connus chez la souris, n'ont	gènes dangereux, c'est-à-dire présen-leurs gènes en double ; et (2) l'inégalité
	comprennent notamment des gènes	majeurs. Ces deux duplications globales pas conservé un excès d'ohnologues.	tant une forte susceptibilité aux muta-des gènes face aux mutations. Alors que
	dont l'implication dans les cancers est	du génome ont conduit à l'émergence et Il apparaît donc que la rétention des	tions délétères dominantes comme les la plupart des gènes tendent à perdre
	connue et des gènes dont les mutations	à la complexification des vertébrés dont gènes ohnologues serait directement	oncogènes notamment. Certains de ces leur fonction par mutation, les gènes
	induisent typiquement des phénotypes	certains gènes ont conservés jusqu'à liée à leur susceptibilité aux mutations	gènes dangereux ont même gardé leurs dangereux se caractérisent par le fait
	délétères dominants, par exemple via	quatre copies. Au total, un quart à un délétères dominantes et non pas à leur	quatre copies depuis l'origine des ver-que leurs mutations entraînent fréquem-
	une perte d'interaction d'auto-inhibition	tiers de nos gènes serait directement nature fondamentale pour l'organisme.	tébrés ! C'est le cas par exemple des ment une suractivation, c'est-à-dire un
	entraînant un gain de fonction perma-	issu de ces deux duplications de génome Nous avons en fait démontré [1], au	gènes RalGEF (Ral guanine-nucleotide gain de fonction, plutôt qu'une perte de
	nent pour le gène mutant.	à l'origine des vertébrés [2]. Ces gènes moyen d'analyses statistiques d'infé-	exchange factor) (Figure 1) qui activent fonction. En général, la perte de fonction
	Les résultats obtenus [1] indiquent	sont appelés gènes ohnologues en l'hon-rences bayésiennes, telles que l'analyse	les voies Ras-Ral impliquées dans la d'un ohnologue ne pose pas de problème
	que les 8 095 gènes impliqués dans des	neur du généticien Susumu Ohno qui fut de Médiation [10], que la dangero-	migration et la prolifération cellulaires tant qu'il reste une copie fonctionnelle
	cancers ou dans des maladies géné-	le premier à avancer l'hypothèse de ces sité est bien la cause principale de la	dans des tumeurs [7]. De même, le gène de ce gène, ce qui conduit à l'élimina-
	tiques comportent significativement	duplications globales du génome chez les rétention des gènes ohnologues chez	Ras, qu'on retrouve en un seul exemplaire tion progressive d'une des copies de la
	plus d'ohnologues que l'ensemble des	vertébrés [3]. l'homme. En particulier, les équilibres	chez les invertébrés comme la drosophile plupart des ohnologues non dangereux.
	Pour comprendre l'origine de l'expan-20 506 gènes codant pour les protéines	Après une duplication globale du entre les niveaux d'expression géné-	(Figure 1), a conservé chez la plupart des En revanche, la survenue de mutations
	sion des familles de gènes dangereux chez les vertébrés, il faut remonter à humaines, soit 48 % contre 35 % (48 % ; 3 844/8 095 ; p = 1,3 x 10 -129 , test  2 ).	génome, les organismes perdent géné-ralement près de 80 à 90 % des gènes tique, fréquemment proposés comme la principale cause des biais de rétention	vertébrés trois ohnologues proto-onco-géniques (KRas, HRas et NRas) (Figure 1) conduisant à des gains de fonction ou, plus généralement, à des phénotypes
	l'ancêtre commun de tous les vertébrés, En outre, ce biais de rétention aug-	dupliqués, ce qui entraîne une expansion des ohnologues, semblent en fait résul-	qui présentent des mutations constituti-dominants délétères qui caractérisent
	il y a quelque 500 millions d'années. Par mente fortement dans le cas des onco-	hétérogène de leurs voies de signali-ter indirectement des effets des muta-	vement actives dans plus de 25 % des cas les ohnologues dangereux, va entraîner
	un mécanisme presque toujours létal, gènes (61 % ; 493/813 ; p = 1,4 x 10 -54 ,	sation (Figure 1) et de leurs réseaux tions délétères dominantes [1].	de cancer chez l'homme. des pathologies du développement ou
	m/s n° 4, vol. 29, avril 2013
	DOI : 10.1051/medsci/2013294008

Figure 1. Expansion des voies de signalisation Ras-Ral. Voies

	x 2	x 2
		x 2
	?
		D. melanogaster B. lanceolatum H. sapiens
		x 2 x
		de signalisation Ras-Ral chez la drosophile (A) et chez l'homme (B). Après les deux
	duplications globales de génomes chez l'ancêtre des vertébrés, chez l'homme, le gène RGL a conservé ses quatre copies ohnologues (RalGDS, RGL1,
	RGL2, RGL3), le gène Ras a conservé trois copies ohnologues (Kras, Hras, Nras) et le gène Ral a conservé deux copies ohnologues (RalA, RalB) [7].
	Cet exemple illustre la nécessaire réorganisation des voies de signalisation et des réseaux d'interactions protéine-protéine après un évènement de
	duplication globale du génome, et l'élimination par divergence d'une partie des gènes dupliqués [4-6].

2 Vertébrés H. sapiens D. melanogaster Effecteurs Ral Ras Ral RalGEF RalBP1 RGL4 RalGPS1 RGL3 RalB RalGDS RalA RGL1 RGL2 RalGPS2 RGL CG5522 Ral RalBP1 Ras Duplication globale du génome Complexe exocyste Complexe exocyste Kras Nras Hras Nras Hras Kras Nras Hras Kras Nras Hras Kras Nras Hras Kras

	se passe comme si les individus de ces
	populations n'avaient pas pu se débar-
	rasser de nombreux gènes dangereux
	redondants, directement fixés par spé-
	ciation dans leur génome dupliqué et
	non pas fixés progressivement grâce à un
	avantage sélectif comme pour la plupart
	des gènes dupliqués individuellement.
	Ensuite, les gènes ohnologues conservés
	se différencient et deviennent souvent
	des acteurs majeurs du développement,
	de la signalisation et de la régulation
	cellulaires. Par exemple, les cadhérines,
	sorte de colle qui lient les cellules entre
	elles, ont conservé de multiples ohno-
	logues exprimés dans différents tissus,
	comme la E-cadhérine qui lie les cellules
	épithéliales entre elles, ou la N-cadhérine
	exprimée dans le tube neural et les neu-
	rones. Mais les mutations des cadhérines
	qui entraînent la décohésion des cellules
	entre elles sont aussi impliquées dans la
	migration des cellules tumorales et leur
	dissémination vers d'autres organes.
	Conclusions
	Ces accidents génétiques majeurs de
	doublement du génome survenus il y a
	500 millions d'années dans l'évolution
	des vertébrés ont donc permis l'émer-
	gence d'organismes plus complexes,
	mais aussi la multiplication des gènes
	dangereux chez les vertébrés. Ces résul-
	tats éclairent d'un point de vue évolu-
	tif nouveau l'expansion des familles de
	gènes fréquemment impliquées dans des
	maladies génétiques et de nombreux
	cancers. ‡

Evolution and cancer: expansion of dangerous gene repertoire by whole genome duplications LIENS D'INTÉRÊT

	des tumeurs. Celles-ci pénalisent les	distingue du concept d'avantage sélec-
	organismes atteints et, plus ou moins	tif généralement associé à l'évolution
	directement, leur descendance qui finira	(sélection naturelle ou adaptative).
	par s'interrompre. Pour autant, les gènes dangereux impliqués ne sont pas élimi-	La multiplication des gènes dangereux chez les vertébrés est donc liée à ce	Les auteurs déclarent n'avoir aucun lien d'intérêt concernant les données publiées dans cet article.
	nés mais au contraire conservés dans	phénomène spécifique et très rare de	RÉFÉRENCES
	la population, puisqu'ils sont encore	duplication globale du génome qui existe
	présents sous une forme non délétère dans le reste de la population issue de	en fait dans la plupart des branches eucaryotes. Sa compréhension doit se	1. Singh PP, Affeldt S, Cascone I, et al. On the expansion of dangerous gene repertoires by whole-genome duplications in early vertebrates. Cell Rep 2012 ; 2 :
	la duplication du génome (Figure 2). Ce	faire au niveau de la génétique des popu-	1387-98.
	processus évolutif par élimination de	lations au sein des nouvelles espèces	2. Makino T, McLysaght A. Ohnologs in the human genome are dosage balanced and frequently associated with
	mutants (sélection de purification) se	issues d'une duplication de génome. Tout	disease. Proc Natl Acad Sci USA 2010 ; 107 : 9270-74.

Figure 2. Évolution des gènes dupliqués après duplicaton globale du génome. A-C. Les

	lignes
	horizontales représentent le génome de différents individus. Les carrés symbolisent les gènes
	dupliqués (rouges) ou non dupliqués (noirs). Les croix noires représentent la perte d'un seul gène
	(petites croix) ou l'élimination d'un individu (grande croix), tandis que les carrés avec bordures
	indiquent les copies mutées. Les scénarios d'évolution sont décrits comme il se doit au niveau
	d'une population d'individus. Lorsqu'elle n'est pas létale, une duplication globale de génome
	chez un ou plusieurs individus de la population initiale (deux lignes horizontales rouges pour un
	individu avec duplication complète) implique nécessairement l'émergence d'une nouvelle espèce
	dont tous les gènes ont été dupliqués. La sélection de purification favorise alors indirectement
	la conservation des ohnologues dangereux non mutés dans le génome des individus qui survivent.
	Adapté de [1].

Sélection Mutation = population post-DGG Principale tendance évolutive Duplication globale du génome (DGG) Spéciation Adaptation Population initiale DGG Population post-DGG

	Un / quelque(s) individu(s)
	Bénéfique	Neutre	Délétère
	Nouvelle population

Rétention ohnologues « dangereux » Élimination ohnologues redondants Rétention ohnologues bénéfiques Génomes Nouvelle espèce Dérive Population initiale

	+	=	-
		Purification
	> La survie d'un organisme multicellu-	réparation de l'ADN est donc essentielle
	laire est étroitement liée au maintien	au maintien des populations de cellules
	de l'homéostasie dans l'ensemble des	souches [1]. Une étude récente de notre
	tissus constituant cet organisme. Cette ou sous-fonctionnalité groupe décrit le rôle d'un système de Perte de gènes capacité à maintenir cet équilibre vital réparation de l'ADN et la nature géno-
	A est assurée par des populations de cel-lules souches, via trois caractéristiques	B toxique de métabolites endogènes dans C les CSH de souris [2].
	principales : (1) la possibilité de se divi-
	ser et de renouveler ce compartiment en
	générant de nouvelles cellules souches ;
	(2) l'absence de fonctions tissulaires
	spécifiques ; et (3) la capacité de se
	différencier en cellules spécialisées. La
	moelle osseuse, où résident les cellules
	souches hématopoïétiques (CSH) res-
	ponsables de la production des cellules
	du sang, constitue un tissu modèle pour
	l'étude des cellules souches somatiques.
	En effet, l'identification de multiples
	marqueurs de surface et le dévelop-
	pement d'anticorps dirigés contre ces
	marqueurs et utilisables en cytométrie
	de flux (FACS, fluorescent activated cell
	sorter), permettent de discriminer et
	d'isoler les divers précurseurs des cel-
	lules sanguines.
	Les cellules souches sont sensibles,
	comme toute cellule, aux conditions
	de stress, en particulier aux dommages
	de l'ADN. Ainsi, au cours du temps,
	l'accumulation de mutations contribue
	au déclin des capacités des cellules
	souches. L'existence de mécanismes de

L'anémie de Fanconi, une pathologie s'accompagnant d'un défaut de réparation de l'ADN

		chromosomiques » [4]. En effet, en liant
		de façon covalente deux bases oppo-
		sées (interbrins d'ADN) ou adjacentes
		(intrabrin d'ADN), ces drogues induisent
		des lésions (ponts inter-ou intrabrins,
		DNA crosslinks) de l'ADN qui bloquent
		la réplication et la transcription. Ces
		ponts interbrins ont pour conséquence
		une augmentation des cassures chromo-
		somiques. Dans les cellules de patients
		atteints d'anémie de Fanconi, ceci est
	La thématique de recherche principale	dû à un défaut de réparation de l'ADN.
	de notre équipe concerne l'étude des	À ce jour, une quinzaine de gènes ont
	mécanismes de réparation de l'ADN et,	été identifiés, codant pour des protéines
	en particulier, celle des gènes impli-	impliquées dans cette voie de réparation
	qués dans l'anémie de Fanconi. Notre	des lésions pontantes de l'ADN.
	approche est essentiellement génétique
	et se base sur des modèles cellulaires et
	murins (souris génétiquement modifiées
	ou knockout [KO]). Du nom du pédiatre
	suisse Guido Fanconi qui décrivit cette
	maladie génétique rare au début du
	XX e siècle [3], l'anémie de Fanconi se
	manifeste dès l'enfance par des malfor-
	mations squelettiques, une forte prédis-
	position à certains types de cancers (en
	particulier des leucémies myéloïdes) et
	une aplasie médullaire progressive. Les
	cellules de ces patients sont très sen-
	sibles à des drogues anticancéreuses,
	telles que la cisplatine ou la mitomy-
	cine C. Cette sensibilité constitue la base
	du test de diagnostic dit de « cassures

Les aldéhydes, source de dommages à l'ADN

	Formal Comment
	Human Dominant Disease Genes Are Enriched in
	Paralogs Originating from Whole Genome Duplication
	Param Priya Singh ¤ , Se ´verine Affeldt, Giulia Malaguti, Herve ´Isambert*
	CNRS-UMR168, UPMC, Institut Curie, Research Center, Paris, France
	Les cassures chromosomiques se mani-
	festent spontanément dans les cellules
	de patients, en dehors de la présence de
	drogues comme la cisplatine, ce qui sug-
	gère l'existence d'agents génotoxiques
	présents de façon naturelle dans les
	cellules de ces patients. Notre groupe a
	confirmé cette hypothèse en montrant
	en 2011 que les aldéhydes produits par
	le métabolisme cellulaire (en particu-
	lier l'acétaldéhyde) étaient à l'origine
	de dommages à l'ADN dans les cellules
	de patients atteints d'anémie de Fan-
	coni [5]. Des souris double KO pour les
	m/s n° 4, vol. 29, avril 2013
	DOI : 10.1051/medsci/2013294009

  genes are significantly enriched in ohnologs, 38.3% versus 27.7% (p = 1.58610 225 ; Fisher's Exact [FE] test), while showing at the same time a significant depletion in both singletons, 16.5% versus 23.7% (p = 7.67610 220 ; FE test), and SSD, 36.1% versus 41.6% (p = 2.75610 26 ; FE test). MD genes are more specifically depleted in recent SSD, 9.2% versus 17.3% (p = 4.1610 250 ; FE test), while WGD-old and older SSD of MD genes are not significantly biased, i.e., 9.9% versus 9.2% (p = 0.12; FE test) and 17% versus 15.5% (p = 0.001; FE test), respectively (see below). These results demonstrate that, although MD genes retain significantly more duplicates than singletons (Figure

  Instead of the χ 2 statistics, Spirtes et al. proposed to use the G 2 likelihood ratio,

	3.1.4.2 G-square conditional independence test

  highest orientation / propagation probability > 1/2.

if I (X; Y ; Z|{U i }) < 0 then Orient/propagate edge direction(s) to form a v-structure X → Z ← Y with probabilities P X→Z and P Y →Z given by Proposition 7. else Propagate second edge direction to form a non-v-structure X → Z → Y assigning probability P Z→Y from Proposition 8. end Apply new orientation(s) and sort remaining list of unshielded triples L c ← L c \ X, Z, Y X Y after updating propagation probabilities.

Table 6 . 1

 61 g 1 g 2 g 3 g 4 g 5 g 6 g 7 Example of a dataset with missing information. The missing values can drastically reduce the number of samples if a globally complete dataset is required.

	samp 1 n		y			n
	samp 2	y				y
	samp 3		n			n
	samp 4 n n	y	n n	y	n ← 1%
	samp 5	n		n

Table 7 . 1

 71 7.2). Evaluation of the 3off2 reconstructions of undirected-ALARM networks by N eff .

	N eff TP FP FN Prec. Rec. Fs. %ort.
	1, 000, 000 43 79	3	0.35 0.93 0.51 100%
	900, 000 43 74	3	0.37 0.93 0.53 100%
	800, 000 43 70	3	0.38 0.93 0.54 100%
	700, 000 43 57	3	0.43 0.93 0.59 100%
	500, 000 42 36	4	0.54 0.92 0.68 100%
	250, 000 38	8	8	0.83 0.83 0.83	0%
	200, 000 38	3	8	0.93 0.83 0.87	0%
	150, 000 38	0	8	1.00 0.83 0.90	0%
	125, 000 37	0	9	1.00 0.80 0.89	0%
	100, 000 37	0	9	1.00 0.80 0.89	0%
	75, 000 36	0	10 1.00 0.78 0.88	0%
	50, 000 36	0	10 1.00 0.78 0.88	0%
	25, 000 34	0	12 1.00 0.74 0.85	0%
	10, 000 34	0	12 1.00 0.74 0.85	0%

Table 7 . 2

 72 Evaluation of the 3off2 reconstructions of undirected-INSURANCE networks by

	1, 000, 000 42 25 10 0.63 0.81 0.70 100%
	900, 000 42 23 10 0.65 0.81 0.72 100%
	800, 000 42 18 10 0.70 0.81 0.75 100%
	700, 000 41 15 11 0.73 0.79 0.76 100%
	500, 000 40	9	12 0.82 0.71 0.79 100%
	250, 000 37	2	15 0.95 0.71 0.81	0%
	200, 000 37	1	15 0.97 0.71 0.82	0%
	150, 000 37	0	15 1.00 0.71 0.83	0%
	125, 000 36	0	16 1.00 0.69 0.82	0%
	100, 000 35	0	17 1.00 0.67 0.80	0%
	75, 000 35	0	17 1.00 0.67 0.80	0%
	50, 000 35	0	17 1.00 0.67 0.80	0%
	25, 000 34	0	18 1.00 0.65 0.79	0%
	10, 000 30	0	22 1.00 0.58 0.73	0%

  & 7.4, α 10 -20 for undirected-ALARM and α 10 -7 for undirected-INSURANCE).

	α	TP FP FN Prec. Rec. Fs. %ort.maj. %ort.csv.
	1e -2	44 142 2	0.24 0.96 0.38	95%	45%
	1e -3	43 96	3	0.31 0.93 0.46	81%	45%
	1e -5	43 42	3	0.51 0.93 0.66	69%	43%
	1e -6 42 27	4	0.61 0.91 0.73	35%	0%
	1e -7	42 17	4	0.71 0.91 0.80	7%	3%
	1e -10 40	5	6	0.89 0.87 0.88	4%	4%
	1e -20 37	0	9	1.00 0.80 0.89	0%	0%
	1e -30 36	0	10 1.00 0.78 0.88	0%	0%
	1e -40 36	0	10 1.00 0.78 0.88	0%	0%

Table 7 . 3

 73 Evaluation of the PC-stable reconstructions of undirected-ALARM networks by N eff . The table gives the number of true positive (TP), false positive (FP) and false negative (FN) edges obtained when reconstructing the undirected-ALARM network with the PC-stable algorithm. The precision (Prec.), recall (Rec.) and f-score (Fs.) evaluates the performance of the skeleton discovery. The %ort. gives the proportion of oriented edges within the learnt interactions. The proportion of oriented edges when using either the majority or the conservative rule for the orientation step is given by %ort.maj. and %ort.csv. respectively.

	1e -2	47 64	5	0.42 0.90 0.58	97%	54%
	1e -3	43 33	9	0.57 0.83 0.68	89%	62%
	1e -5	41 16 11 0.72 0.79 0.75	75%	58%
	1e -6	41	9	11 0.82 0.79 0.80	82%	68%
	1e -7 40	8	12 0.83 0.77 0.80	75%	0%
	1e -10 39	1	13 0.98 0.75 0.85	0%	0%
	1e -20 37	0	15 1.00 0.71 0.83	0%	0%
	1e -30 36	0	16 1.00 0.69 0.82	0%	0%
	1e -40 35	0	17 1.00 0.67 0.80	0%	0%

α

TP FP FN Prec. Rec. Fs. %ort.maj. %ort.csv.

Table 7 . 4

 74 Evaluation of the PC-stable reconstructions of undirected-INSURANCE networks by N eff . The table gives the number of true positive (TP), false positive (FP) and false negative (FN) edges obtained when reconstructing the undirected-INSURANCE network with the PC-stable algorithm. The precision (Prec.), recall (Rec.

Table 8 .

 8 1, B., where E (Y |x, m) denotes the expectation value of Y given X and M , and P (M |x) denotes a value of M drawn from a distribution P conditionally on X. Owing to non-linear couplings, DE xx and IE xx usually do not sum to the total effect T E xx . However, the proportions DE xx /T E xx and IE xx /T E xx can be interpreted in terms of sufficient versus necessary contributions to T E xx

Table 8 . 1

 81 Mediation analysis formulae. This table gives the formulae corresponding to the direct effect, DE xx , and the indirect effect, IE xx , mediated my m(x).

Table 8 . 2

 82 Mediation analysis results. This table gives direct and indirect estimations between three genomic properties, namely the dosage-balance constraint, 'dosage.bal.', the susceptibility to deleterious mutations, 'delet.mut.' and the retention of ohnologs, 'ohno.'.

Table 8 . 3

 83 Genomic properties for the reconstruction of causal graphical models in the fate of gene duplicates. Twelve genomic properties participating in the fate of genome duplicates in the course of vertebrates evolution. The status of each property has been identified for all human protein coding genes, except for 'Essential', 'Ka.Ks' and 'ExpressionLevel'. See main text for complementary information.

  9.10 displays the directed cycles (Cl 9 → Cl 13 → Cl 10 → Cl 12 → Cl 11 → Cl 9 ) and (Cl 9 → Cl 13 → Cl 14 → Cl 11 → Cl 9 ), where Cl 9 & Cl 11 belong to the optic tectum, Cl 10 & Cl 12 belong to the pretectum and Cl 13 & Cl 14 correspond to retinal ganglion cells. Similarly, Fig. 9.12 display the directed cycle (Cl 9 → Cl 14 → Cl 12 → Cl 10 → Cl 9 ). Cl 21 & Cl 24 belonging to the neuropil, and the clusters Cl 27 & Cl 28 belonging to the pretectum through the neural circuit (Cl 21 → Cl 27 → Cl 28 ← Cl 24 → Cl 21 ). Similarly, the reconstructed layer 6 in Fig. 9.13 links the clusters Cl 23 & Cl 25 belonging to the neuropil, and the clusters Cl 27 & Cl 28 belonging to the pretectum through the neural circuit (Cl 23 → Cl 27 ← Cl 28 ← Cl 25 ← Cl 23 ). Interestingly, Fig. 9.11 & 9.13 both show directed edges from the retinal ganglion cells to the neuropil area (Cl 25 → Cl 21 & Cl 26 → Cl 23 → Cl 24 , Fig. 9.11; Cl 24 → Cl 23 & Cl 26 → Cl 25 , Fig. 9.13), as reported in the experimental observations [Portugues

	Although the reconstructed networks relying on 30 clusters do not show directed
	cycles between these areas, they display orientations that could be interpreted as
	alternating patterns. Indeed, the reconstructed layer 5 in Fig. 9.11 links the clus-
	ters

  ]

					→	→	→	→
	Correct edges (out of 11) (→/ / ) 11 9	7	6	6	10	8	8
	-Correct orientations	(→) 7	3	0	5	4	8	4	0
	-Mis/non-orientations	( / ) 4	6	7	1	2	2	4	8
	Missing links	( ) 0	2	4	5	5	1	3	3

Table 10 . 1

 101 Interactions reconstructed by 3off2 and alternative methods for a subnetwork of hematopoiesis regulation. → indicates a successfully recovered interaction including its direction as reported in the literature (see references in the table

Table 10 . 2

 102 Interactions reconstructed by 3off2 and four alternative methods on the subnetwork of 11 known regulatory interactions in hematopoiesis (Figure

	Statistical 3off2	PC	PC	MMHC MMHC Bayes hc Bayes hc Aracne
	Measure NML α = 10 -1 α = 10 -2	BDe	BIC	BDe	BIC	= 0
	Recall u	1	0.82	0.64	0.55	0.55	0.91	0.73	0.73
	Precision u 0.65	0.64	0.7	0.55	0.55	0.56	0.67	0.89
	Fscore u	0.79	0.72	0.67	0.55	0.55	0.69	0.70	0.80
	H u	0.13	0.18	0.16	0.22	0.22	0.22	0.13	0.09
	HIM u	0.16	0.17	0.13	0.19	0.19	0.22	0.10	0.08
	Distance u 0.13	0.16	0.16	0.22	0.22	0.18	0.16	0.11
	Recall d	1	0.6	0.2	0.5	0.44	0.89	0.57	NA
	Precision d 0.41	0.21	0.1	0.45	0.36	0.47	0.33	NA
	Fscore d	0.58	0.31	0.13	0.47	0.4	0.62	0.42	NA
	H d	0.14	0.21	0.17	0.12	0.14	0.14	0.14	NA
	HIM d	0.13	0.16	0.15	0.09	0.11	0.14	0.11	NA
	Distance d 0.22	0.29	0.29	0.24	0.27	0.22	0.24	NA

  of the malignant tissue. Finally, the metastasis stage can be reached by acquiring mutations that enable cells to escape from the primary tumour tissue.

	In this chapter, I present results related to the reconstruction of such mutational
	pathways, i.e. cascades of mutations that trigger and/or sustain the tumour devel-
	opment, in breast cancer tumours. These results rely on data made available by
	the COSMIC (Catalog Of Somatic Mutations In Cancer) database [Forbes et al.,
	2008], a web resource developed and maintained by the Wellcome Trust Sanger
	Institute. This part of my project has been developed in collaboration with a
	Master 2 student in bioinformatics in the Isambert's group, Wei Wenjun.
	11.1 The breast cancer dataset
	11.1.1 A brief overview of breast cancer

Then, mutations preventing DNA error checks and/or repair of these errors also favour the tumour development by offering new potentially 'advantageous' mutations. Subsequently, any mutation sustaining a significant angiogenesis and providing anti-growth factor resistance ensures further development

  2 contains information on tumour samples associated with many cancers. These samples have been collected and curated from the scientific literature and large-scale experimental screens from the Cancer Genome Project at the Sanger Institute. The version 69 of COSMIC, on which rely our dataset, In particular, the number of samples for which the whole-genome or whole-exome 4 has been sequenced is rapidly increasing (+33% between the version 69 and 70, table 11.1). However these whole-genome/exome data still represent a small portion of the COSMIC database (about 1%), as most studies have relied so far on the sequencing of a few genes of interest, i.e. frequently mutated in a

	provides 999, 872 samples that contain 1, 808, 915 non-synonymous mutations 3 for
	27, 829 genes. These data have been obtained from 9, 424 genomes and 19, 031
	publications.
	The growing number of studies in cancer, and the advances in next-generation
	sequencing (NGS) technologies enabled regular updates of the COSMIC resources
	since 2004. specific cancer.
		V69	V70 V70-V69	%
	Genes	27,829	28,735	906	+3%
	Samples	999,872 1,029,547	29,675	+3%
	Coding Mutations 1,808,915	2,002,811	193,896 +10%
	Papers	19,031	19,703	672	+3%
	Whole Genome/Exome	9,424	12,542	3,118 +33%

Table 11 . 1

 111 Evolution of tumour sample resources in the COSMIC database.

Table 11 . 2

 112 Evaluation of the 3off2 method when reconstructing networks from randomized and non-randomized datasets. The dataset of (A) 196 tumour samples without missing values and (B) 26, 494 tumour samples with missing values were shuffled 1, 000 times, for an η parameter from 1 to 10. The column 'random'

  B.1 Description of the benchmark networks nodes edges k Model k max k in Table B.1 Description summary of the 30 benchmark networks used to evaluate the reconstruction methods. The 30 benchmark networks of 50 nodes, and 20 to 160 edges, have been instantiated with the causal modeling tool Tetrad IV (http://www.phil.cmu.edu/tetrad/). For each model, 20 dataset replicates of size ranging between 50 and 50,000 were generated with Tetrad IV.

						max	k out max	N	Replicates
	50	20	0.8	1	4	2	3	[50 -50, 000]
				2	4	2	2	[50 -50, 000]
				3	3	3	2	[50 -50, 000]
				4	3	3	2	[50 -50, 000]
				5	3	2	2	[50 -50, 000]
				Avg.	3.4	2.4	2.2
	50	40	1.6	1	5	3	5	[50 -50, 000]
				2	6	3	3	[50 -50, 000]
				3	5	3	3	[50 -50, 000]
				4	4	4	4	[50 -50, 000]
				5	5	3	3	[50 -50, 000]
				Avg.	5	3.2	3.6
	50	60	2.4	1	7	5	3	[50 -50, 000]
				2	6	6	3	[50 -50, 000]
				3	6	4	4	[50 -50, 000]
				4	6	5	3	[50 -50, 000]
				5	7	3	5	[50 -50, 000]
				Avg.	6.4	4.6	3.6
	50	80	3.2	1	7	5	7	[50 -50, 000]
				2	7	5	5	[50 -50, 000]
				3	6	5	5	[50 -50, 000]
				4	6	5	6	[50 -50, 000]
				5	6	4	5	[50 -50, 000]
				Avg.	6.4	4.8	5.6
	50	120 4.8	1	10	10	7	[50 -50, 000]
				2	13	10	7	[50 -50, 000]
				3	9	6	8	[50 -50, 000]
				4	13	9	7	[50 -50, 000]
				5	12	9	7	[50 -50, 000]
				Avg.	11.4	8.8	7.2
	50	160 6.4	1	12	10	9	[50 -50, 000]
				2	13	9	9	[50 -50, 000]
				3	14	7	9	[50 -50, 000]
				4	11	7	8	[50 -50, 000]
				5	11	10	8	[50 -50, 000]
				Avg.	12.2	8.6	8.6

Dominant mutations lead to constitutively active mutant genes with dominant phenotypes typically detrimental for the organism. In particular, the oncogenes are prone to dominant deleterious mutations.

Recessive mutations are mutations that lead to a loss of function. In a diploid organism, both alleles need to be mutated in order to express detrimental phenotypes, as the (functional) non mutated allele can usually maintain the warrant functionality.

Cell Reports 2, 1387-1398, November 29, 2012 ª2012 The Authors

CNRS-UPMC UMR168,

Inserm U830, Institut Curie, Centre de recherche, 26, rue d'Ulm, 75248 Paris, France. *Contribution égale des auteurs herve.isambert@curie.fr

NouvellesAvril2013.indd 359 12/04/2013 10:24:47 m/s n°4, vol. 29, avril 2013

PLOSComputational Biology | www.ploscompbiol.org

July 2014 | Volume 10 | Issue 7 | e1003754

This example has been taken from[Naïm et al., 2011, Chapter 1]

Although the effect of a selection variable results in a genuine probabilistic association, the induced association will be qualified as 'spurious' in the following, as for the association resulting from a hidden variable.

A vertex Xj is called a collider if the two endpoint marks at this vertex are arrowheads, otherwise it is called a non-collider. In the mix graph Xi ←→ Xj ←→ X k , Xj is a collider. However, in the mix graph Xi ←→ Xj → X k , Xj is a non-collider.

The independence relations that are read off the DAG G using the d-separation graphical criterion hold in the probability distribution over O = {X1, X2, ..., Xq} and reciprocally (see also subsec-

tion 2.1.2.1)[START_REF] Makino | Ohnologs in the human genome are dosage balanced and frequently associated with disease[END_REF] The set of measured or observed variables O contains all the common causes of pairs of variables, i.e. there are no hidden confounders and no hidden selection variables

As these bi-directed edges only represent conflicting information,[START_REF] Colombo | Order-independent constraint-based causal structure learning[END_REF] stipulate that they cannot be interpreted causally (page 3756)

In other words, the first variable has the greatest number of levels, the second variable has the second largest number of levels, and so on.

http://www.phil.cmu.edu/tetrad

PC-stable benchmarks were tested up to N = 10, 000 due to their sharp increase in execution time, see

Refer to Chapter 1, footnote[START_REF] Tinti | Evolution of signal multiplexing by 14-3-3-binding 2R-ohnologue protein families in the vertebrates[END_REF]

Refer to Chapter 1, footnote[START_REF] Dickerson | On the origins of Mendelian disease genes in man: the impact of gene duplication[END_REF]

The 'dosage-balance' hypothesis posits that the ohnologs are retained because their interactions with protein partners require to maintain balanced expression levels throughout evolution.

Dominant negative phenomena occur in diploid organism when a mutated allele adversely interferes with the functional allele.

Autoinhibitory proteins are multidomain proteins where the domain having the functional property is inhibited by interaction with another part of the protein through folding. Conformational changes can for instance be triggered by interactions with other proteins.

Genes that participate in multi-protein complexes.

Dominant disease genes are genes prone to dominant deleterious mutations.

A transcription factor is a protein that controls the rate of transcription of a gene by binding to the DNA sequence.

Somatic mutations are the mutations acquired during the life of an individual. By contrast with germline mutations, they are not inherited from the parents.

http://cancer.sanger.ac.uk/cosmic

A non-synonymous mutation is a nucleotide mutation that modifies the amino acid sequence of the corresponding protein.

Whole-exome sequencing methods concern the subset of DNA that encodes proteins, also named exons.

http://cancer.sanger.ac.uk/biomart/martview

Acknowledgements

A. acknowledges a PhD fellowship from Ministry of Higher Education and Research, France; I.C. acknowledges postdoctoral support from ANR (grant ANR-08-BLAN-0290); R.S. acknowledges PhD fellowships from INCa and ARC; H.I. and J.C. acknowledge funding from Foundation Pierre-Gilles de Gennes.

Supporting Information

Dataset S1 Dataset used for the analysis. Gene IDs and symbols are from Ensembl v70. Values in the columns correspond to one of the following descriptions: N = no; Y = yes; Onc = oncogene; TS = tumor suppressor; O = others; AD = autosomal dominant; AR = autosomal recessive; XL = X-linked; and YL = Ylinked. (XLSX)

Author Contributions

Conceived and designed the experiments: PPS HI. Performed the experiments: PPS SA GM HI. Analyzed the data: PPS SA GM HI. Wrote the paper: PPS SA HI.

Recent genome wide analyses have shown that ohnologs, and SSD 9 (Small Scale Duplication) or CNV 10 (Copy Number Variation) exhibit antagonist retention patterns. Besides, ohnologs in the human genome have experienced fewer SSD than 'non-ohnolog' [Makino and McLysaght, 2010]. As the SSD constraint could biased our analysis, I have also considered only the 8, 215 human genes without SSD nor CNV duplicates. The direct effect of 'dosage.bal.' tends then to oppose the retention of ohnologs (-33.4%), while the indirect effect, mediated by 'delet.mut.', is even more pronounced (130.5%).

Interestingly, we will show in section 8.4 that the 3off2 method recovers this antagonist patterns, as can be seen in Figs. 8.3 & 8.4, that exhibit a negative causal effect from 'Ohnolog' towards 'recentSSD'.

Retention of 'dangerous' ohnologs through a counterintuitive mechanism

These findings have led the Isambert's group to a new evolutionary model describing a non-adaptive mechanism through which 'dangerous' ohnologs could have been retained during the course of evolution. Fig. 8.2 examines all possible evolutionary scenarios following a WGD event in the genome of one or a few individuals in an initial population [START_REF] Affeldt | Évolution et cancer -expansion des familles de gènes dangereux par duplication du génome [evolution and cancer: expansion of dangerous gene repertoire by whole genome duplications] (article in french)[END_REF], Singh et al., 2012].

A critical feature of WGD events compared to SSD events is their coupling to subsequent speciation events. Following the 'duplication' and the WGD-induced 'speciation', there could be three possible scenarios under 'mutation/selection' process, leading to fixation (Fig. 8.2) or loss of individual ohnologs:

(A) Retention of ohnologs by spread and fixation of beneficial mutations in the post-WGD population by positive selection (B) Loss of one copy by fixation of neutral non-functionalizing mutations by drift, also leading to further speciation in the post-WGD population (C) Elimination of individual(s) harboring dominant deleterious mutations, leading to a post-WGD population composed of surviving individuals still holding the (non-deleterious) functional copies of the 'dangerous' ohnologs 9 Small scale duplicates correspond to duplicated regions ranging from a few base pairs to a large genomic segment. 10 Copy number variations are a form of small scale duplications, and correspond to variants within a population where individual human can have different (number of) copies of a part of their genome.

All in all, these results obtained with the 3off2 reconstruction approach support the non-adaptive retention mechanism of ohnologs that have been proposed during the first part of my thesis [START_REF] Affeldt | Évolution et cancer -expansion des familles de gènes dangereux par duplication du génome [evolution and cancer: expansion of dangerous gene repertoire by whole genome duplications] (article in french)[END_REF], Malaguti et al., 2014, Singh et al., 2012]. This global overview provides valuable insights on the interplay between the genomic properties that influence directly or indirectly the fate of duplicates during the evolution enables. It also provides the causal graphical model that is needed to perform Mediation Analyses and quantify the causal relationships between these genomic properties.

Chapter 9

Reconstruction of zebrafish larvae neural networks from brain-wide in-vivo functional imaging Common neural processes can span over several brain regions and involve large scale ensemble of neurones. Getting a comprehensive understanding of the brain functional networks thus requires the ability of acquiring and processing datasets corresponding to large fractions of the brain. Several imaging techniques have been proposed and widely used, such as two-photon or confocal microscopy, however their acquisition speed limitation has only allowed for the exploration of a small part of the brain so far. Recently, the Selective-plane Illumination Microscopy (SPIM) has provided images, at the level of the individual neuron and for large brain areas, allowing for such wide-brain analysis.

In these chapter, I report and discuss some network reconstructions obtained with the 3off2 method when applied to biological datasets generated from single-cell resolution images acquired with the advanced SPIM imaging technology. These datasets correspond to 2D layers of a zebrafish larvae brain at different depths (z-plane images), and have been provided by the research group of Dr. Georges Debrégeas and Dr. Raphaël Candelier (Zebrafish behavior and calcium imagery research group, Jean Perrin laboratory, UPMC, Paris VI). Sébastien Wolf performed the experiments using a SPIM imaging system and processed the acquired images of the spontaneous brain activity of zebrafish in their larval stage. The binary datasets obtained from this single-cell resolution imaging system were used as input for the 3off2 method to reconstruct functional networks corresponding to the 9.2) or 30 (Table 9.3) clusters.

(1)

(2) (1)

(2)

Reconstruction of temporal activity patterns

In previous studies, neural circuits involved in innate reflexes [START_REF] Kubo | Functional architecture of an optic flow-responsive area that drives horizontal eye movements in zebrafish[END_REF][START_REF] Portugues | Whole-brain activity maps reveal stereotyped, distributed networks for visuomotor behavior[END_REF] or hunting behaviour [START_REF] Bianco | Visuomotor transformations underlying hunting behavior in zebrafish[END_REF] have been the JUN kinase signalling pathways, located downstream to MAPK8/MAPK9.

Figure 11.2 JUN kinase signalling pathway. This schema, taken from [START_REF]The landscape of cancer genes and mutational processes in breast cancer[END_REF],

represents the regulation of JUN kinases MAP2K7 and MAP2K8. Genes in green are activated by mutations, while genes in red are inactivated.

The JUN kinases are involved in the cell apoptosis and proliferation processes.

The Fig. 11.2 highlights the two following interactions pathways, Following these interactions, previous studies [START_REF]The landscape of cancer genes and mutational processes in breast cancer[END_REF] have predicted that,

• mutations in PIK3CA and/or AKT may induce a constitutive activation that reduce the activity of the JUN pathway

• mutations in PTEN and/or MAP2K4 may induce an inhibition of these genes that also impairs the JUN pathway Although the reduction of the JUN pathway activity is likely to have complex consequences, it may include the disruption of the cell apoptosis process. The 3off2 reconstruction seems to suggest that impairing either interactions in p A or alternatively in p B is sufficient to trigger and/or sustain the tumour progression by inhibiting the cell apoptosis.

All in all, the 3off2 method could be used to infer temporal cascades of mutations from incomplete observational datasets. Although this study of mutational pathways in breast cancer is still at its early stage, the 3off2 reconstructions already show interesting cascades of alterations. In the future, the inclusion of gene expression levels should also be considered. Indeed, beyond the mutations of the genes themselves, perturbations of signalling pathways through the disruption of regulatory sequences have also a significant impact on the tumour growth that should be incorporated in our analyses.

network still remains to be extensively evaluated. This will be done by adapting the simulated networks and datasets generated with the causal modeling tool Tetrad IV (section 7.2).

The Chapters 8 to 11 are ongoing projects that are being further investigated in our team. In particular, in order to provide a broader overview of the whole neural activity of the zebrafish larvae brain, we are now designing a 3D clustering method to allow for a 3D causal model reconstruction with the 3off2 approach.

This will also require further development in terms of graphical model display within the discoNet pipeline (section 6.5.2). As these 3D reconstructions will clarify the influence of brain areas across several 2D layers at different depths of the brain, this is a nice opportunity to assess the 3off2 method for the assumption of the presence of latent variables, as comparisons will be made possible between the 2D z-plane reconstructions and the 3D global neural network.

Furthermore, our reconstruction of the hematopoietic subnetwork (Chapter 10) relied so far on a dataset that combines single cell-expression data at different stages of the differentiation process. Our team is now exploring the temporal modifications of the hematopoietic subnetwork throughout these distinct stages. The first preliminary results on developmental stage specific datasets already highlight wellknown hubs among the transcription factors playing a role in the differentiation of the stem cells. Besides, the 3off2 approach also indicates sensible inhibitions and activations that are turned on and off along this complex biological process.

Importantly, the 3off2 method can be used for multivariate analysis that integrate heterogeneous datasets. Hence, as introduced in Chapter 11, complementary factors will be included in the analysis of tumour progression to provide a broader understanding of the tumour emergence and spreading process. Indeed, beyond single point mutations, we are now considering the role of the copy number variations and the expression levels of major drivers identified in several cancer types.

Whenever available, we will also considered complementary factors, such as clinical information (e.g. survival time, patient age, tumour stage).

Finally, a long-term development goal is to guarantee that this novel 3off2 approach for complex (biological) network reconstructions can be scaled up to thousands of variables. These future developments, that require several adaptations of the 3off2 robust scheme, are now under study and design.

Appendix A

Complementary evaluations on real-life networks

In this appendix, the results of the considered network reconstruction methods are evaluated using different parameters values and against each others in terms of Precision (or positive predictive value), P rec = T P/(T P + F P ), Recall or Sensitivity (true positive rate), Rec = T P/(T P + F N ), as well as F-score = 2 × P rec × Rec/(P rec + Rec) and execution time when comparing the CPDAG (filled lines) of the reconstructed network (or its skeleton (dashed line)) to the CPDAG (or the skeleton) of the benchmark network. The alternative methods are:

• the stable version of the PC algorithm [Colombo andMaathuis, 2014, Spirtes and[START_REF] Spirtes | An algorithm for fast recovery of sparse causal graphs[END_REF] implemented in the pcalg package [Kalisch andBühlmann, 2008, Kalisch et al., 2012] • the Bayesian inference method using the hill-climbing heuristics implemented in the bnlearn package [START_REF] Scutari | Learning Bayesian Networks with the bnlearn R Package[END_REF] • Aracne [START_REF] Margolin | ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context[END_REF]], an information-based inference approach, which iteratively prunes links with the weakest mutual information based on the Data Processing Inequality. We have used the Aracne implementation of the minet package [START_REF] Meyer | minet: A R/Bioconductor package for inferring large transcriptional networks using mutual information[END_REF] For sample sizes from N = 10 to 50, 000 data points, the methods have been tested on 50 replicates and the Figures A. 1-A.26 give the average results over these multiple replicates. The five following benchmark networks have been considered

(refer to the bnlearn package [START_REF] Scutari | Learning Bayesian Networks with the bnlearn R Package[END_REF] for more details on these networks):

CHILD 20 nodes, 25 links, 230 parameters, mean degree 2.5, maximum in-degree 2

INSURANCE 27 nodes, 52 links, 984 parameters, mean degree 3.85, maximum in-degree 3

A.2 Evaluation of the Aracne reconstruction method

In this section, the results of the Aracne inference method, as implemented in the minet package [START_REF] Meyer | minet: A R/Bioconductor package for inferring large transcriptional networks using mutual information[END_REF], are evaluated using the following parameters values:

• a threshold for the minimum difference in mutual information set to = 1/N (Aracne Eps 1/N)

• a threshold for the minimum difference in mutual information set to = 0 (Aracne Eps 0)

As shown in Figures A. 6-A.10, small positive values for the threshold parameters for minimum difference in mutual information typically worsen F-scores. A.4 Evaluation of 3off2 by score

In this section, the results of the 3off2 inference approach are evaluated using the following parameters values:

• 3off2 MDL(rank I): 3off2 inference approach using Minimum Description As discussed in section 6.3.2, the Figures A. 16-A.20 show that the MDL complexity using shifted 2-point and 3-point information terms leads the 3off2 inference approach to cumulate false negative edges with smaller datasets. Instead, the 3off2 inference approach using the NML criterion with shifted 2-point and 3-point information demonstrates very good results for all sample sizes, and for the five benchmark networks.

A.5 Evaluation of 3off2 against Bayesian and MMHC methods

In this section, the results of the 3off2 inference approach are evaluated against the Bayesian inference methods and a hybrid method using the following parameters values:

• HC BIC: Bayesian inference using Bayesian Information Criterion (BIC) score and hill-climbing (HC) heuristics with 100 random restarts [START_REF] Chickering | Learning Bayesian networks: Search methods and experimental results[END_REF] as implemented in the bnlearn package [START_REF] Scutari | Learning Bayesian Networks with the bnlearn R Package[END_REF] • MMHC BIC(α = 1e -1 ): Hybrid approach [START_REF] Tsamardinos | The Max-Min Hill-Climbing Bayesian Network Structure Learning Algorithm[END_REF] reconstructing first an undirected skeleton through the identification of the parents and children of each node of the underlying graph and then orienting the edges of the skeleton through a Bayesian hill climbing heuristics (possibly removing edges) using the BIC criterion with significance parameter α = 0.1 (using BDe criterion and the range α = 0.001 -0.1 gives very similar results for all tested benchmarks, not shown). The MMHC method is implemented in the bnlearn package [START_REF] Scutari | Learning Bayesian Networks with the bnlearn R Package[END_REF].