Maximilian H Fitz-James

Gonzalo Sabarís

Peter Sarkies

Frédéric Bantignies

Giacomo Cavalli
email: giacomo.cavalli@igh.cnrs.fr

Interchromosomal contacts between regulatory regions trigger stable transgenerational epigenetic inheritance in Drosophila

Keywords: Transgenerational Epigenetic Inheritance, Chromatin Contacts, Genome Organisation, Epimutation, Polycomb, GAGA-Factor, Fab-7

Non-genetic information can be inherited across generations by the process of Transgenerational Epigenetic Inheritance (TEI). TEI can be established by various triggering events, including transient genetic perturbations. In Drosophila, hemizygosity of the Fab-7 regulatory element triggers inheritance of the histone mark H3K27me3 at a homologous locus on another chromosome, resulting in heritable epigenetic differences in eye colour. By mutating transcription factor binding sites within the Fab-7 element, we demonstrate the importance of two proteins in the establishment and maintenance of TEI: GAGA-factor and Pleiohomeotic. We show that these proteins function by recruiting the Polycomb Repressive Complex 2 and by mediating interchromosomal chromatin contacts between Fab-7 and its homologous locus.

Finally, using an in vivo synthetic biology system to induce them, we show that chromatin contacts alone can establish TEI, providing a mechanism by which hemizygosity of one locus can establish epigenetic memory at another in trans through long-distance chromatin contacts.

Introduction

Epigenetic information has long been known to be a major factor in the regulation of gene expression [START_REF] Allis | The molecular hallmarks of epigenetic control[END_REF] . Whether such information can be transmitted across generations in various organisms has been a more elusive question, made difficult by the potential for genetic factors to confound experiments on heredity [START_REF] Cavalli | Advances in epigenetics link genetics to the environment and disease[END_REF][START_REF] Bohacek | A guide to designing germline-dependent epigenetic inheritance experiments in mammals[END_REF] . Through careful experimentation in model organisms, recent work has now demonstrated that such transgenerational epigenetic inheritance (TEI) does occur in a variety of organisms [START_REF] Liberman | Transgenerational Epigenetic Inheritance: From Phenomena to Molecular Mechanisms[END_REF][START_REF] Bošković | Transgenerational epigenetic inheritance[END_REF] . In addition, although much more work is required, the molecular mechanisms underlying these instances of inheritance have begun to be described [START_REF] Fitz-James | Molecular mechanisms of transgenerational epigenetic inheritance[END_REF] .

Many of the most well-known epigenetic regulators of gene expression have been implicated in different cases of TEI, including non-coding RNAs [START_REF] Rechavi | Starvation-induced transgenerational inheritance of small RNAs in C. elegans[END_REF][START_REF] Schott | Natural RNA interference directs a heritable response to the environment[END_REF][START_REF] Teixeira | PiRNA-mediated regulation of transposon alternative splicing in the soma and germ line[END_REF][START_REF] Duempelmann | Small RNAs in the Transgenerational Inheritance of Epigenetic Information[END_REF][START_REF] Cecere | Small RNAs in epigenetic inheritance: from mechanisms to trait transmission[END_REF][START_REF] Yu | Epigenetic inheritance mediated by coupling of RNAi and histone H3K9 methylation[END_REF] , DNA methylation [START_REF] Furci | Identification and characterisation of hypomethylated DNA loci controlling quantitative resistance in Arabidopsis[END_REF][START_REF] Silveira | Extensive Natural Epigenetic Variation at a De Novo Originated Gene[END_REF][START_REF] Takahashi | Transgenerational inheritance of acquired epigenetic signatures at CpG islands in mice[END_REF] and histone modifications [START_REF] Yu | Epigenetic inheritance mediated by coupling of RNAi and histone H3K9 methylation[END_REF][START_REF] Greer | Transgenerational epigenetic inheritance of longevity in Caenorhabditis elegans[END_REF][START_REF] Torres-Garcia | Epigenetic gene silencing by heterochromatin primes fungal resistance[END_REF][START_REF] Seong | Inheritance of stressinduced, ATF-2-dependent epigenetic change[END_REF][START_REF] Ciabrelli | Stable Polycomb-dependent transgenerational inheritance of chromatin states in Drosophila[END_REF] , while less typical sources of non-genetic information, such as 3D chromatin organisation [START_REF] Ciabrelli | Stable Polycomb-dependent transgenerational inheritance of chromatin states in Drosophila[END_REF] and transcription factor binding [START_REF] Jung | Recruitment of CTCF to an Fto enhancer is responsible for transgenerational inheritance of BPA-induced obesity[END_REF] , have been suggested as secondary signals in some cases.

Just as genes and their allelic variants are the basis of genetic variation, so are "epialleles" the basic units of heritable epigenetic change. Similarly, while mutation is the means by which genetic variation arises, "epimutation" describes the appearance of a heritable change in epigenetic information that gives rise to an epiallele. Epimutation provides an alternative source of heritable variation which differs from genetic mutation in that it has the potential to be more rapid, targeted and reversible, allowing for fast adaptation to a fluctuating environment [START_REF] Sabarís | Epigenetic inheritance in adaptive evolution[END_REF] .

Nonetheless, the underlying causes of epimutation and the mechanism by which they arise remain unclear and likely vary between organisms.

Given the complexity of mechanisms involved in TEI in higher eukaryotes, model systems that can be easily manipulated and allow one to both effectively track heritable phenotypes and analyse the underlying molecular events at play are critically needed in the field. One clear example of TEI occurs in Drosophila melanogaster, in a transgenic line called "Fab2L", involving a transgene that drives the expression of the eye pigmentation gene mini-white [START_REF] Bantignies | Inheritance of polycomb-dependent chromosomal interactions in Drosophila[END_REF] . Flies of the Fab2L line exhibit a stochastic phenotype, manifesting as mosaicism of pigmentation in the adult eye. A memory of this phenotype can be established by a transient genetic perturbation [START_REF] Ciabrelli | Stable Polycomb-dependent transgenerational inheritance of chromatin states in Drosophila[END_REF] which can then be maintained epigenetically for countless generations following the initial trigger. While this represents a clear case of TEI, its molecular mechanisms remain mysterious, making it a valuable model system to study the means by which heritable epigenetic variability arises. Here, we investigate the mechanistic basis for the establishment of transgenerational epigenetic inheritance at the Fab2L locus. We show that it is mediated by two key regulatory segments, and a Polycomb Response Element (PRE), which recruits either Polycomb Group (PcG) or Trithorax Group (TrxG) proteins to maintain the pattern of Abd-B expression established early in development. Both of these regions contain several consensus sequence motifs for the DNA-binding proteins GAF and Pho (Figure 1B). Among other functions, both of these proteins are known to be recruiters of PRC2, strongly suggesting their potential involvement in the variable H3K27me3 levels, and thus the eye colour phenotype, in Fab2L.

We generated three transgenic lines mutating all GAF and Pho sequence motifs within either the insulator (Fab2L-INS), the PRE (Fab2L-PRE) or both (Fab2L-INS-PRE) in the Fab2L transgene (Figure 1B). Fab2L-INS showed a significant decrease in GAF binding to the insulator, although Pho binding was little affected as was GAF binding to the PRE. Interestingly, Fab2L-PRE had the opposite effect, with GAF binding at both sites unaffected, while Pho binding was significantly decreased at the PRE only (Figure 1C,D). These results extended to the recruitment of PRC2 to Fab2L, which was largely unaffected in Fab2L-INS but significantly decreased in Fab2L-PRE, not only at the PRE itself but also exterior to the Fab-7 at the downstream LacZ region (Figure 1E). Mutation of both regions together shows clear additive effects. Indeed, in the Fab2L-INS-PRE line binding of both GAF and Pho are decreased at both the insulator and PRE, and in most cases to a significantly greater extent that in either of the single mutants (Figure 1C,D), although PRC2 binding was not significantly different from Fab2L-PRE (Figure 1E). Taken together, these results suggest that the insulator and PRE regions of Fab-7 act cooperatively to recruit GAF and Pho to the Fab2L transgene, although the PRE may play the greater role in the subsequent recruitment of PRC2.

We then analysed the downstream effects of this altered GAF and Pho recruitment on the chromatin and phenotype of the mutant lines. All three mutant lines had significantly decreased levels of H3K27me3 across the transgene, with the decrease being much more pronounced in the double mutant Fab2L-INS-PRE line (Figure 1F). These changes in chromatin translated to phenotypic effects on the eye colour of the adult flies. Indeed, all three mutant lines displayed shifts towards red eye colour compared to naïve Fab2L (Figures 1G-J andS1). While it remained considerable, the shift in Fab2L-INS was milder than for the other two lines, with approximately 16% of females and 61% of males exhibiting fully red eyes (Figures 1H andS1C,D). In contrast, almost all Fab2L-PRE and Fab2L-INS-PRE flies of both sexes exhibited fully red eyes. It is interesting to note, however, that while the shift towards red was complete for Fab2L-INS-PRE, with all individuals having uniform red eyes (Figures 1J andS1G,H), Fab2L-PRE retained some stochasticity, with around 7% of females possessing at least some white ommatidia (Figures 1I andS1E). These results reinforce the idea that the insulator and PRE together are responsible for the epigenetic and phenotypic variability of the Fab2L fly line.

The insulator and PRE regions of Fab-7 are individually sufficient to mediate epigenetic inheritance

In WT flies carrying the Fab2L transgene, epigenetic differences in expression between individuals are not inherited transgenerationally under normal conditions. . Indeed, when we applied repeated selection and crossing of the most extreme individuals in the population of an unmanipulated Fab2L line over ten generations, we did not obtain any significant differences in eye colour across the population (Figure S2A,B). However, crossing this "naïve" Fab2L line with another Fab2L line bearing a homozygous deletion of the endogenous Fab-7 gives F1 individuals carrying the transgene in a homozygous state, but the endogenous Fab-7 in a hemizygous state (Figure S2C). This hemizygosity establishes transgenerational epigenetic memory at the transgene, such that reconstituting the Fab2L genotype in the F2 results in a line which is genetically identical to the P0 Fab2L, but in which TEI is now possible [START_REF] Ciabrelli | Stable Polycomb-dependent transgenerational inheritance of chromatin states in Drosophila[END_REF] . Indeed, selection of this line over 10 generations resulted in either red or white "epilines": populations of flies with a significant proportion of individuals with monochrome eye colour, i.e. with 100% of their ommatidia either pigmented or unpigmented (Figure S2D). We also found that other crossing schemes which induced Fab-7 hemizygosity for one or two generations, were able to trigger TEI in Fab2L (Figure S2E-J).

Given the altered phenotype of the mutant Fab2L transgenic lines, we asked whether these mutations also interfered with the ability of the Fab2L transgene to maintain a memory of its epigenetic state across generations by TEI. Due to the shift towards red eyes in the naïve Fab2L-INS, Fab2L-PRE and Fab2L-INS-PRE lines, selection towards red eyes would not be informative. We therefore performed a transgenerational epigenetic selection experiment to determine if these mutant lines could be selected towards a more repressed, white-eyed phenotype than the naïve population. Just as with wild-type Fab2L (Figure S2C), we introduced a single generation of Fab-7 hemizygosity while leaving the mutant versions of the Fab2L transgene unmanipulated (Figure 2A). We then reconstituted the parental genotype, homozygous for the endogenous Fab-7, and selectively bred the most white-eyed individuals over subsequent generations. As a control, we used a wild-type Fab2L line which had previously been selected for a red-eyed phenotype. The Fab2L, Fab2L-INS and Fab2L-PRE, which all showed a greater or lesser degree of variability in their starting populations, were receptive to selection, showing a clear and gradual shift towards whiter eyes in both females and males over the generations (Figures 2B-D andS3A-C). In the case of Fab2L-INS, the appearance of some individuals with fully white eyes was even observed (Figures 2C andS3B), consistent with the less extreme de-repression observed in this line compared to the other mutants. In contrast, the Fab2L-INS-PRE line displayed no variation at any point during the experiment, with all individuals of both sexes maintaining a uniform red eye colour (Figures 2D andS3C). As expected, some pre-existing degree of Polycomb binding and some form of epigenetic variation at the Fab2L transgene is therefore prerequisite for TEI. However, these results also demonstrate that the insulator or PRE regions alone are still able to mediate TEI, indicating that they act together to maintain an epigenetic memory at the transgenic Fab-7 element in the Fab2L line.

The insulator and PRE regions of Fab-7 are required for horizontal transmission of a repressed epigenetic state through paramutation

The Fab2L transgene is not only able to acquire an altered epigenetic state by selection over generations, but can do so in a single generation by the process of "paramutation" [START_REF] Ciabrelli | Stable Polycomb-dependent transgenerational inheritance of chromatin states in Drosophila[END_REF] .

Paramutation denotes the horizontal transfer of an epigenetic state in trans between two homologous alleles, and has been described in many organisms including Drosophila [START_REF] Hollick | Paramutation and related phenomena in diverse species[END_REF][START_REF] Dorer | Transgene repeat arrays interact with distant heterochromatin and cause silencing in cis and trans[END_REF][START_REF] De Vanssay | Paramutation in Drosophila linked to emergence of a piRNA-producing locus[END_REF][START_REF] Capovilla | A new paramutation-like example at the Delta gene of Drosophila[END_REF] . In the Fab2L line, crossing a naïve Fab2L with an established Fab2L epiline (either white or redeyed) results in the acquisition by the naïve allele of the altered epigenetic state of the epiline allele. This phenomenon can be tracked by the use of a black [1] marker allele, closely linked to the Fab2L transgene, such that F2 individuals that have inherited both copies of Fab2L from the naïve parent can be determined with high probability (Figures 3A andS4A). Although these F2 flies possess the genetic material of the naïve P0 population, their epigenetic state resembles that of the epiline with which it was crossed, attesting to the acquisition over this genomic region of a new epigenetic state (Figures 3B andS4B).

To determine if mutation of the transgene interfered with horizontal transfer of epigenetic state, we crossed the Fab2L-INS, Fab2L-PRE and Fab2L-INS-PRE lines with a white-eyed Fab2L epiline in order to see if these mutant versions of the Fab2L transgene could acquire a repressed epigenetic state by paramutation in addition to, or instead of, selection over generations. Again, as control we used a wild-type Fab2L epiline which had previously acquired a de-repressed, red-eyed epigenetic state. Just as with the selection, Fab2L, Fab2L-INS and Fab2L-PRE were able to acquire a more white-eyed phenotype than the naïve parental lines (Figure 3C-E). Conversely, all Fab2L-INS-PRE individuals maintained their uniform red coloration of the eyes even after exposure in the F1 of the cross to a repressed epiallele (Figure 3F). While lines bearing a wild-type version of the insulator or PRE of Fab-7 alone were therefore able to acquire an altered epigenetic state by both selection over generations and paramutation, mutation of both regions together completely prevents acquisition of a repressed epigenetic state by either method. Taken together ,these results therefore suggest that the insulator and PRE work together not only to mediate epigenetic variation at the transgenic Fab-7, but also to maintain an epigenetic memory across generations.

GAF mediates long-range chromatin contacts through the insulator region of Fab-7

Our results point to PRC2 and GAF as key factors mediating the epigenetic variability, and its inheritance across generations, at the Fab2L transgene. PRC2 has a direct role in regulating the expression of the Fab2L transgene, as the differences in mini-white expression and phenotype of the Fab2L epilines correlate with differences in PRC2-deposited H3K27me3 across the transgene [START_REF] Ciabrelli | Stable Polycomb-dependent transgenerational inheritance of chromatin states in Drosophila[END_REF] . The role of GAF is less clear as mutation of the GAF sites in the insulator ultimately had little effect on PRC2 binding (Figure 1E), suggesting that GAF's primary role at the Fab-7 element may not be the recruitment of PRC2. Intriguingly, among its many other functions, GAF has been shown to mediate long-range chromatin contacts between its target genes [START_REF] Ogiyama | Polycomb-Dependent Chromatin Looping Contributes to Gene Silencing during Drosophila Development[END_REF][START_REF] Mahmoudi | GAGA can mediate enhancer function in trans by linking two separate DNA molecules[END_REF][START_REF] Chetverina | GAGA factor -a multifunctional pioneering chromatin protein[END_REF] . The three-dimensional organization of chromatin is a major factor in the regulation of gene expression, and polycomb target genes in particular are frequently found to colocalize in the nucleus at so-called "Polycomb bodies" [START_REF] Entrevan | Regulation of Genome Architecture and Function by Polycomb Proteins[END_REF] . Moreover, the transgenic and endogenous copies of Fab-7 were previously found to form chromatin contacts in the Fab2L line [START_REF] Ciabrelli | Stable Polycomb-dependent transgenerational inheritance of chromatin states in Drosophila[END_REF] .

To quantify these chromatin contacts, we performed Fluorescence in situ hybridisation (FISH) to visualise the regions surrounding the Fab-7 elements in Fab2L embryos carrying different copy numbers of the endogenous Fab-7 (Figure 4A,B). These regions showed significant colocalization in the nuclei of Fab2L embryos, but not in Fab2L ; Fab7 [1] embryos which lack the endogenous Fab-7. This shows that chromatin contacts do occur between these loci dependent on the presence of both Fab-7 elements. Intriguingly, the Fab2L-Fab-7 chromatin contacts observed in Fab2L increase even further in a Fab7 [1] / + genetic background in which only one copy of the endogenous Fab-7 is present.

To investigate the effect of Fab2L mutation on these chromatin contacts, we extended the FISH analysis to our mutated transgenic lines (Figures 4C-G andS5). In both a homozygous and hemizygous Fab-7 background, chromatin contacts between Fab2L and Fab-7 were comparable between Fab2L-PRE and wild-type Fab2L. This is measured both in a similar average distance between the two loci (Figure 4D,F) and in the proportion of nuclei in which the loci are in close proximity (Figure 4E,G). Conversely, contacts are significantly decreased in Fab2L-INS compared to wild-type Fab2L, suggesting a primary role for the insulator, and GAF, in mediating these chromatin contacts. It is worth noting, however, that mutation of the insulator does not fully abrogate chromatin contacts to the extent seen in Fab2L ; Fab7 [1], whereas mutation of both the insulator and PRE does. Thus, while Fab2L-PRE is not significantly different from Fab2L, and Fab2L-INS-PRE is not significantly different from Fab2L ; Fab7 [1],

Fab2L-INS displays an intermediate phenotype, indicating significant, but not complete, loss of contacts between the two loci (Figure 4D-G, andS5C,G). Taken together, these results strongly suggest that contacts between the Fab-7 elements are primarily mediated through the insulator region, but that the PRE may also play a stabilizing role.

Artificially-induced chromatin contacts are sufficient to induce TEI at the Fab2L locus

We then sought to investigate whether chromatin contacts may play a causal role in the establishment of TEI at Fab2L. Inter-loci Fab2L-Fab-7 distance does not differ significantly between white-or red-eyed epilines of Fab2L [START_REF] Ciabrelli | Stable Polycomb-dependent transgenerational inheritance of chromatin states in Drosophila[END_REF] , arguing against a contribution of chromatin contacts to the maintenance of epigenetic differences between these epilines. We also found that while these contacts are robust in late-stage embryos (stage 14-15) they are not observable in early stages (stage 4-5) (Figure S6), indicating that chromatin contacts themselves are unlikely to be the transgenerationally inherited signal of TEI. However, the increase in Fab2L-Fab-7 chromatin contacts in Fab2L ; Fab7 [1]/+ individuals hemizygous for the endogenous Fab-7 (Figure 4A), correlating with the triggering of TEI in the genetic crosses previously discussed (Figure S2), makes the gain of chromatin contacts a prime candidate for the molecular trigger that establishes TEI. We therefore wished to explore this correlation in greater detail.

To directly investigate the role of chromatin contacts in the triggering of TEI, we therefore sought to induce contacts rather than abrogate them. To achieve this, we developed an in vivo system to induce interchromosomal contacts between the two regions of interest without recourse to any genetic perturbation. We dubbed this system "Three-Dimensional Contact Induction System" or "3D-CIS", for its ability to bring two distant loci in proximity (Figures 5A andS7). To create this system, we inserted arrays of Lac or Tet operators adjacent to the transgenic and endogenous Fab-7 elements, respectively. Aside from the addition of these arrays, this line has the same genotype as the Fab2L line, and has a similar average distance and contact frequency between the two loci (Figures 5B,D,E andS8A). However, activation of the system by introducing a TetR-LacI fusion protein which binds to both arrays (Figure S7E,F) results in anchoring of the two Fab-7 elements to each other (Figure 5C). This anchoring leads to a decrease in the average Fab2L-Fab-7 distance and increase in the frequency of close contacts, both to levels comparable to the hemizygous Fab2L ; Fab7 [1] / + in which TEI is established (Figures 5D,E andS8B). Importantly, at no point are either the transgenic or endogenous Fab-7 in a hemizygous state (Figure S7A,B). The 3D-CIS system therefore allows us to investigate the effect of increasing chromatin contacts between the two Fab-7 elements in the absence of any genetic perturbation.

Just as with Fab2L in the absence of genetic perturbation (Figure S2A,B), selection of the 3D-CIS line in the "OFF" state over several generations did not result in any change in eye colour across the population, towards either white or red eyes (Figure 5B,F). After ten generations of selection, these lines also exhibited no difference in H3K27me3 levels between each other (Figure 5I). However, activation of the 3D-CIS system, by introduction of the TetR-LacI fusion protein and thus increase in contacts between the Fab2L transgene and endogenous Fab-7, was able to establish TEI, such that selection over subsequent generations resulted in both white and red epilines (Figure 5C,G). These epilines also had significant differences in H3K27me3 levels between them (Figure 5J), reminiscent of the differences between Fab2L epilines obtained by selection after transient hemizygosity of Fab-7 (Figure 5H). These results demonstrate that chromatin contacts alone, in the absence of any genetic perturbation, are sufficient to induce TEI at the Fab2L transgene. As further controls, we generated two more lines expressing either a LacI-LacI or a TetR-TetR fusion protein as part of the 3D-CIS system (Figure S7C,D). These lines were also unable to trigger TEI (Figure S9), showing that triggering is not due to expression of a fusion protein or its binding to either array singly, but conclusively results from the binding of the fusion protein to both arrays in tandem.

The Fab-7 element is required for stable chromatin contacts

The ability of the 3D-CIS system to induce TEI in Fab2L suggests that the primary role of the Fab-7 element in the establishment of TEI is to mediate long-range chromatin contacts between the transgenic and endogenous Fab-7 elements. To determine whether induced chromatin contacts can trigger TEI at the Fab2L transgene even in the absence of the Fab-7 element, we generated a new version of the 3D-CIS system in which the transgenic Fab-7 was deleted (Figure S10A). As expected, phenotypically, this line resembled Fab2L-INS-PRE, with all individuals possessing uniform red eyes (Figures 1J andS10B). Similarly, this LacO-Fab2L-Fab7Δ line was unable to acquire a repressed epigenetic state by either selection or paramutation (Figure S10C-E). Activation of the 3D-CIS system by introduction of the LacI-TetR fusion protein was also unable to trigger TEI in this line (Figure S10F-I). However, FISH analysis revealed that chromatin contacts between the transgene and the endogenous Fab-7 were not increased in this line. Indeed in both the "OFF" and "ON" state, 3D-CIS-Fab7Δ flies did not show any significant contacts between the two loci, comparable to Fab2L ; Fab7 [1] (Figure S10J). This suggests that 3D-CIS is insufficient to mediate long-range chromatin contacts on its own, but rather acts to stabilise or reinforce contacts already established between the two Fab-7 elements.

Altered epigenetic states remain stable in the absence of artificially-induced chromatin contacts

These results demonstrate a clear role for chromatin contacts in the initial triggering of TEI in Fab2L. However, due to experimental constraints, the 3D-CIS system remains active throughout the selection towards epilines. To determine whether the altered epigenetic states triggered by the 3D-CIS system can be maintained even in the absence of induced chromatin contacts, we crossed the 3D-CIS epilines with a naïve LacO-Fab2L. In the F2, flies lacking the TetR-LacI fusion protein (as determined by a GFP marker, see Methods) were selected and counted. Even in the absence of the TetR-LacI inducing chromatin contacts, this F2 generation had a majority of individuals with primarily white eyes, in the case of the white epiline (Figure 5K), or primarily red eyes, in the case of the red epilines (Figure S11A). The distribution of the population was comparable with those derived from a cross with Fab2L epilines triggered by transient hemizygosity rather than 3D-CIS (Figure S11B,C). The LacO-Fab2L was therefore able to maintain the memory of its altered epigenetic state, even in the absence of artificially-induced chromatin contacts with the endogenous Fab-7, demonstrating that enhancement of chromatin contacts is required to establish, but not maintain, transgenerational epigenetic inheritance.

Discussion

GAF-mediated chromatin contacts and PRC2-mediated epigenetic variability together account for TEI at the Fab2L transgene

Our results highlight the crucial role played by two subdomains of the Fab-7 element in the establishment of epigenetic variation at the Fab2L transgene, and the maintenance of its memory across generations. These subdomains are an insulator and a PRE, which act through the recruitment of the transcription factors GAF and Pho. Alone, one of these regions remains sufficient to maintain a certain degree of variation and epigenetic memory at the Fab2L transgene, albeit in a manner skewed towards de-repression. Mutation of all GAF and Pho sites across both regions, however, completely abrogates all variation, demonstrating that at least some binding of these proteins is essential (Figures 12).

Our findings suggest that the insulator and PRE cooperate to control epigenetic regulation of the Fab-7 element in two ways. The first is the recruitment of PRC2, which deposits H3K27me3 in a stochastic manner, leading to the observed variable eye colour phenotype. The second is to mediate long-range chromatin contacts between the two distant Fab-7 elements in the genome.

However, mutation of these subdomains suggests that the PRE is the comparatively more important of the two regions for PRC2 recruitment (and thus epigenetic variability) (Figure 1E), while the insulator is much more involved in the establishment of chromatin contacts (Figure 4).

Nevertheless, both elements contribute to some extent to both aspects of Fab2L regulation.

The distribution of GAF and Pho bindings sites between these subdomains suggests differing roles for these proteins, in agreement with what is known of their function. Indeed, the majority of GAF binding sites (6 out of 9) are located in the insulator, while the majority of Pho sites (3 out of 4) are in the PRE (Figure 1B). Based on these observations, we propose that GAF is the primary mediator of chromatin contacts, whereas Pho is the primary recruiter of PRC2, and thus responsible for the epigenetic variability at the Fab2L transgene. Together, these proteins thus mediate the dual functions of the Fab-7 element, both of which are essential to TEI in this model system.

PRC2-dependent epigenetic memory at the Fab2L locus

Our results clearly point to a central role for chromatin contacts in the establishment of TEI at Fab2L, but not in the inheritance of the alternative gene expression at Fab2L, as contacts are not present in early development (Figure S6). PRC2-deposited H3K27me3 is thus the primary epigenetic signal underpinning the variability at the Fab2L transgene and must be inherited by another mechanism independent of chromatin contacts, either by direct inheritance or by reconstruction in each generation based on an epigenetic memory maintained by some other signal [START_REF] Fitz-James | Molecular mechanisms of transgenerational epigenetic inheritance[END_REF] .

Alternatively, it is worth considering that a combination of direct and indirect inheritance mechanisms could serve to reinforce each other, providing a more stable epigenetic memory that either pathway alone. Previous work in Drosophila has provided evidence of germline inheritance of H3K27me3, arguing for its ability to be transmitted through gametogenesis at least [START_REF] Zenk | Germ line-inherited H3K27me3 restricts enhancer function during maternal-to-zygotic transition[END_REF] . Moreover, a recent study found that a DNA-binding protein, in this case CTCF, can remain bound to chromatin through development and maintain an epigenetic memory through this association [START_REF] Jung | Recruitment of CTCF to an Fto enhancer is responsible for transgenerational inheritance of BPA-induced obesity[END_REF] . That this mechanism could extend to a complex like PRC2, which both binds to and deposits H3K27me3, is an interesting prospect, as it has the potential to provide a positive feedback loop to stabilise transgenerational H3K27me3. In this way, inheritance of H3K27me3 both directly, by transmission through the germline, and indirectly, through a memory of PRC2 binding, could provide redundancy and reinforcement to ensure a more reliable inheritance of epigenetic memory. While our study is primarily concerned with explaining how TEI is triggered at the Fab2L locus, future investigations into the mechanism by which this epigenetic state is transmitted across generations will provide a more complete picture of this instance of TEI.

Chromatin contacts trigger PRC2-dependent TEI at the Fab2L locus

Our results indicate that the primary mechanism for the triggering of TEI at the Fab2L transgene is the promotion of physical contact within the nucleus between Fab2L and the endogenous Fab-7. Increased contact frequency can either be induced by hemizygosity (Figure S2)probably because the remaining copy of Fab-7 forms contacts more efficiently with its distant homolog at another locus in the absence of its homologous allele -or in a synthetic manner, such as in our transgenic 3D-CIS system (Figure 5). This demonstrates that chromatin contacts can establish transgenerational epigenetic memory in the absence of any genetic perturbation.

Nevertheless, we note that this system is unable to mimic the effects of hemizygosity in the absence of an adjacent Fab-7 element (Figure S10). This is explained by the fact that Fab2L-Fab-7 contacts are already observed to a lesser extent in Fab2L ; + individuals, but are increased in Fab2L ; Fab [7]/+ hemizygotes (Figure 4A). Thus, 3D-CIS acts to increase or stabilise the contacts already occurring between the two loci, rather than driving the contacts de novo.

As this trigger can lead to inheritance of epigenetic state in both directions (repression and derepression), any mechanism explaining how this trigger occurs must take into account this plasticity. We therefore propose a model whereby stabilisation of Fab2L-Fab-7 chromatin contacts allows for the exchange of PRC2 between the endogenous and transgenic Fab-7 elements, thereby triggering an epigenetic memory that can be selected towards extremes over generations (Figure 6). In this model, naïve Fab2L flies have stochastic recruitment of PRC2 to the transgene by Pho, leading to a random mosaic eye colour pattern (Figure 6A). Chromatin contacts are mediated by GAF, but in the absence of manipulation these contacts primarily occur between homologous alleles, i.e. Fab2L to Fab2L or endogenous Fab-7 to endogenous Fab-7 (Figure 6B). While interchromosomal contacts between Fab2L and Fab-7 do occur, they are transient and outcompeted by the preferential interaction between homologous alleles that is common in dipteran species [START_REF] Mckee | Homologous pairing and chromosome dynamics in meiosis and mitosis[END_REF] (Figure 6C). Stabilisation of these contacts is achieved upon Fab-7 hemizygosity (Figure 6D), because the remaining endogenous Fab-7, having lost its preferred interaction partner, is free to form more stable contacts with its imperfect transgenic partner without being outcompeted by its homologous allele. This situation can be mimicked in a homozygous state, and in the absence of genetic perturbation, thanks to the 3D-CIS transgenic system, which artificially stimulates chromatin contacts between Fab2L and Fab-7, making them interact preferentially with each other rather than with their homologous alleles (Figure 6E). When these trans interactions are sufficiently stable, exchange of PRC2 can occur between the PREs of the Fab-7 elements. Pho sites within the PRE can act as either donors or acceptors of PRC2, leading to small changes in expression of the transgenic mini-white reporter. The memory of this expression is maintained across generations, meaning that over time, these differences can be selected to extremes, leading to either fully repressed or fully de-repressed transgene expression, and monochrome eye colour (Figure 6F).

A few aspects of this model are worth highlighting. First, it would predict that a Fab2L/+ ; Fab7 [1]/+ double hemizygote, in which both loci have lost their preferential interaction partner on the homologue, should be an even more effective trigger of TEI than single hemizygosity.

While we have not examined this in detail, our different TEI triggering crosses support this, as epilines derived from Fab2L/+ ; Fab7 [1]/+ double hemizygotes tend to reach fixation faster than those derived from Fab2L ; Fab7 [1]/+ single hemizygotes during selection (Figure S2). Second, exchange of PRC2 binding between Fab2L alleles, rather than between Fab2L and Fab-7, could also explain how paramutation is able to transfer epigenetic state, albeit imperfectly, between homologous alleles in this line. Our model thus accounts for both the triggering of epiallelic identity, and its horizontal transfer by paramutation.

A broader role for hemizygosity and chromatin contacts in triggering TEI

One major question in the field of epigenetic inheritance is how heritable epigenetic variability, or epimutation, arises in the first place. Studies in plants suggest that heritable changes in DNA methylation can occur apparently spontaneously in these organisms, leading to long-term epigenetic differences between lines [START_REF] Johannes | Spontaneous epimutations in plants[END_REF][START_REF] Johannes | Assessing the Impact of Transgenerational Epigenetic Variation on Complex Traits[END_REF][START_REF] Kawakatsu | Epigenomic Diversity in a Global Collection of Arabidopsis thaliana Accessions[END_REF] . Recent studies in Caenorhabditis elegans have extended these observations to metazoans and to RNA and chromatin-based epigenetic changes [START_REF] Beltran | Epimutations driven by small RNAs arise frequently but most have limited duration in Caenorhabditis elegans[END_REF][START_REF] Wilson | Fluctuations in chromatin state at regulatory loci occur spontaneously under relaxed selection and are associated with epigenetically inherited variation in C. elegans gene expression 10[END_REF] . Other studies have sought to identify environmental triggers for TEI, directly linking epigenetic variation to an external stress to which it is intended to respond [START_REF] Cavalli | Advances in epigenetics link genetics to the environment and disease[END_REF][START_REF] Klosin | Transgenerational transmission of environmental information in C. elegans[END_REF][START_REF] Casier | Environmentally-Induced Transgenerational Epigenetic Inheritance: Implication of PIWI Interacting RNAs[END_REF] . The final prominent candidate for sources of epigenetic variation is genetic perturbation, different types of which have been shown to trigger TEI in a variety of organisms [START_REF] Takahashi | Transgenerational inheritance of acquired epigenetic signatures at CpG islands in mice[END_REF][START_REF] Greer | Transgenerational epigenetic inheritance of longevity in Caenorhabditis elegans[END_REF][START_REF] Reinders | Saccharomyces cerevisiae cAMP-dependent protein kinase controls entry into stationary phase through the Rim15p protein kinase[END_REF][START_REF] Rassoulzadegan | RNA-mediated non-mendelian inheritance of an epigenetic change in the mouse[END_REF] .

In the Drosophila Fab2L line, this genetic perturbation takes the form of transient hemizygosity of the endogenous Fab-7 region for at least one generation. It is interesting to note that unlike some cases of genetically-triggered TEI, in Fab2L the epigenetic memory is triggered and maintained not at the locus which is perturbed (the endogenous Fab-7) but elsewhere in the genome (the Fab2L transgene), testifying to the ability of trans-interactions to induce TEI. Fab-7 has been found to have a similar effect on other loci. Indeed, hemizygosity of Fab-7 has been shown to affect the expression of another PRE-containing gene in the distant Antennapedia cluster, with a phenotype that persisted for several generations after restoration of Fab-7 homozygosity [START_REF] Ciabrelli | Stable Polycomb-dependent transgenerational inheritance of chromatin states in Drosophila[END_REF] . This raises the question of whether similar mechanisms could be acting to trigger TEI at other loci in natural populations.

Recent sequencing of wild Drosophila melanogaster lines has revealed the incredible genomic variation between populations of this single species. This includes numerous and large-scale deletions, duplications and translocations across the genome [START_REF] Mackay | The Drosophila melanogaster Genetic Reference Panel[END_REF] . Mixing of two such genomically disparate populations would lead to a number of hemizygosity or heterozygosity events, as well as homology between very distant loci reminiscent of what is observed in Fab2L. Breeding in the wild thus has the potential to lead to many instances of naturally occurring genetic perturbation as potential triggers for TEI. Just as in Fab2L, it could be that establishment of TEI might be possible only between certain regions which are already prone to contact each other.

In this respect, PRC2 targets may be particularly interesting candidates for naturally occurring TEI. Indeed, as previously mentioned, many PRC2 targets are frequently clustered within the nucleus in polycomb bodies, forming a large domain of silenced chromatin [START_REF] Entrevan | Regulation of Genome Architecture and Function by Polycomb Proteins[END_REF] . Interestingly, genes regulated in this manner are more likely to possess both an insulator region and a PRE, just like Fab-7. Our study provides insight into the mechanisms by which this type of epimutation could occur, but it is only by extending this insight to a broader context that we will be able to determine the role of TEI in the phenotypic variation, and thus potentially adaptation, of natural populations. (C) When more than one copy of Fab-7 is present in the genome contacts can be initiated between distant Fab-7 elements, but these contacts are outcompeted by inter-allelic contacts and remain transient.

Figure Titles & Legends

(D-E) Stabilization of these contacts can be achieved through hemizygosity of one Fab-7 copy (D) or through synthetic biology tools (E). This stabilisation leads to exchange of PRC2 between the PREs, resulting in either increased PRC2 association and silencing, or decreased PRC2 association and de-repression, and triggering an epigenetic memory of this altered association.

(F) Over generations these slight differences can be selected to extremes, resulting in either very strong or very weak repression and strikingly different phenotypes.

Methods

Fly stocks and culture

Flies were raised in standard cornmeal yeast extract media. Standard temperature was 21°C, with the exception of P0 and F1 crosses in the experiments of Fab2L epiallele establishment by hemizygosity or 3D-CIS (Figures 2; 5F,G; S2; S7A-D, S9D,E and S10D-I), for which temperature was 18°C. The Fab2L and Fab2L ; Fab7 [1] To create the 3D-CIS system, arrays of 7 Tet operators and 21 Lac operators were inserted adjacent to the endogenous or transgenic Fab-7, respectively, using a single guide RNA to cut immediately to their 3'. Cassettes encoding recombinant proteins combining the Lac and/or Tet repressors with a GFP marker (TetR-GFP-LacI, TetR-GFP-TetR and LacI-GFP-LacI) under expression of an Actin-5C promoter were inserted into chromosome arm 3L separately by Phi recombination into an established AttP containing line (Bloomington 24480). The transgenes encoding these proteins were then recombined with the TetO-Fab-7, and introduced into a Fab2L background, ready to be crossed with the LacO-Fab2L as described in Supplementary information, Figure S7. All injections for these lines were provided by BestGene Inc. The Fab2L-Fab-7Δ line was derived from the LacO-Fab2L line by CRISPR-Cas9 targeted deletion of the transgenic Fab-7, designed and implemented by Rainbowgene Transgenic Flies Inc. [1], Fab2L-PRE, black [1], Fab2L-INS-PRE, black [1] and Fab2L-Fab7Δ, black [1] were generated by recombining the Fab2L transgene with the black [1] allele from the w[1118];

Fab2L-INS, black

black [1] line (Bloomington Drosophila Stock Center).

Chromatin Immunoprecipitation and antibodies

0 to 16 hour old embryos were collected in Embryo Wash Buffer (0.03% Triton X-100, 140mM NaCl) and dechorionated with bleach. Samples were crosslinked in 1 ml A1 buffer (60 mM KCl, 15 mM NaCl, 15 mM HEPES [pH 7.6], 4 mM MgCl2, 0.5% Triton X-100, 0.5 mM dithiothreitol (DTT), 10 mM sodium butyrate and complete EDTA-free protease inhibitor cocktail [Roche]), in the presence of 1.8% formaldehyde. Samples were homogenized with a micropestle and incubated for a total time of 15 minutes at room temperature. Crosslinking was stopped by adding 350 mM glycine followed by incubation for 5 min. The homogenate was transferred to a 2 ml tube and centrifuged for 5 minutes, 4,000g at 4°C. The supernatant was discarded, and the nuclear pellet was washed three times in 2 ml A1 buffer and once in 2 ml of Lysis buffer (140 mM NaCl, 15 mM HEPES [pH 7.6], 1 mM EDTA, 0.5mM EGTA, 1%Triton X-100, 0.5mMDTT, 0.1% sodium deoxycholate, 10 mM sodium butyrate and complete EDTA-free protease inhibitor cocktail [Roche]) at 4°C. Nuclei were than resuspended in 1.5 ml Lysis buffer in the presence of 0.1% SDS and 0.5% N-Laurosylsarcosine, transferred to a 15 ml falcon tube and incubated for 2 hours with agitation at 4°C. Samples were adjusted to 3 ml and chromatin was sonicated using a Q700 sonicator with microtip (QSonica) for a total of 6 minutes and 30 seconds at amplitude 50 (settings: 30 s on, 1min 30 s off x 13 cycles) in an ice bucket. Sheared chromatin had size range of 100 to 300 base pairs. After sonication and 5 minutes high-speed centrifugation at 4°C, fragmented chromatin was recovered in the supernatant and aliquoted in 5 μg (for H3K27me3 ChIP) or 20 μg (for non-histone protein ChIP) aliquots adjusted to a volume of 500 μl in Lysis Buffer with 0.1% SDS and 0.5% N-Laurosylsarcosine for storage at -20°C.

To perform the ChIP, samples were thawed on ice and chromatin was precleared by addition of 15 μl of Protein A Dynabeads (Invitrogen 10002D) followed by incubation for at least 1 hour at 4°C. Dynabeads were removed on a magnetic rack and antibodies were added at a dilution of 1:100 (a mock control in the presence of rabbit IgG was performed at the same time, while an input of the same size was set aside). Samples were incubated for overnight at 4°C on a rotating wheel. 30 μl of Protein A Dynabeads were added and incubation was continued for at least 2 hours at 4°C. Antibody-protein complexes bound to beads were washed 4 times in Lysis Buffer with 0.05% SDS and twice in TE Buffer (0.1 mM EDTA, 10 mM Tris (pH 8)) in 1 ml each time. Antibodies used in this study were as follows: anti-GAF polyclonal antibody [START_REF] Schuettengruber | Functional anatomy of polycomb and trithorax chromatin landscapes in Drosophila embryos[END_REF] ; anti-Pho polyclonal antibody [START_REF] Schuettengruber | Functional anatomy of polycomb and trithorax chromatin landscapes in Drosophila embryos[END_REF][START_REF] Brown | The Drosophila pho-like gene encodes a YY1-related DNA binding protein that is redundant with pleiohomeotic in homeotic gene silencing[END_REF] ; anti-E(z) polyclonal antibody [START_REF] Ogiyama | Polycomb-Dependent Chromatin Looping Contributes to Gene Silencing during Drosophila Development[END_REF] ; anti-H3K27me3 polyclonal antibody (Active Motif 39155), anti-GFP polyclonal antibody (Abcam ab290), normal rabbit IgG (Cell Signalling 2729).

Fluorescence in situ hybridization

Two-color 3D FISH was performed as previously described [START_REF] Bantignies | Polycomb-dependent regulatory contacts between[END_REF] . For a detailed protocol, see Bantignies and Cavalli, 2014 51 . Briefly, embryos were dechorionated with bleach and fixed in buffer A (60 mM KCl; 15 mM NaCl; 0.5 mM spermidine; 0.15 mM spermine; 2 mM EDTA; 0.5 mM EGTA; 15 mM PIPES, pH 7.4) with 4% paraformaldehyde for 25 min in the presence of heptane. Embryos were then devitellinized by adding methanol to the heptane phase, extracted and washed three times in methanol. Embryos were kept for a maximum of 4 months in methanol at 4C before proceeding to FISH. Fixed embryos were sequentially re-hydrated in PBT (PBS, 0.1% Tween 20) before being treated with 100-200 μg/ml RNaseA in PBT for 2 hours at room temperature. Embryos were then sequentially transferred into a pre-Hybridization Mixture (pHM: 50% formamide; 4XSSC; 100 mM NaH2PO4, pH 7.0; 0.1% Tween 20). Embryonic DNA was denatured in pHM at 80°C for 15 minutes. The pHM was removed, and denatured probes diluted in the FISH Hybridization Buffer (FHB: 10% dextransulfat; 50% deionized formamide; 2XSSC; 0.5 mg/ml Salmon Sperm DNA) were added to the tissues without prior cooling.

Hybridization was performed at 37°C overnight with gentle agitation. Post-hybridization washes were performed, starting with 50% formamide, 2XSSC, 0.3% CHAPS and sequentially returning to PBT. After an additional wash in PBS-Tr, DNA was counterstained with DAPI (at a final concentration of 0.1 ng/μl) in PBT and embryos were mounted with ProLong Gold Antifade (Invitrogen).

FISH probes for the 37B and 89E regions were made from a previous design described in Ciabrelli et al. 2017 [START_REF] Ciabrelli | Stable Polycomb-dependent transgenerational inheritance of chromatin states in Drosophila[END_REF] . For each region, 6 non-overlapping probes of between 1.2 and 1.7kb covering an area of approximately 12kb were generated using the FISH Tag DNA kit with Alexa Fluor 555 or Alexa Fluor 647 dyes (Invitrogen Life Technologies). 100ng of each probe were added to the 30µL of FHB for hybridization.

Microscopy and image analysis

For the FISH, the 3D distances between 37B and 89E loci were acquired and measured as follows: due to somatic pairing of homologous chromosomes in Drosophila, the majority of the nuclei in embryos show a single FISH spot for each probe. In the cases of non-overlap FISH signals between homologues, the closest distance between the centres of the two probes was considered. To measure distances, 3D stacks were collected from 3-5 different embryos.

Optical sections were collected at 0.5 μm intervals along Z-axis using a Leica SP8-UV microscope, Montpellier Resources Imaging (MRI) facility. Relative 3D distances between FISH signals were analyzed in approximately 80 to 120 nuclei per 3D stack using the Imaris software (Oxford Instruments). The distance distribution between the two probes was obtained by pooling replicates for each condition. (which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission.

Generations of Selection

Generations of Selection

Stable contacts lead to triggering of epigenetic memory Unstable interchromosomal contacts

Generations of Selection

Figure 1 .

 1 Figure 1. Mutation of GAF and Pho binding sites decreases epigenetic variability of the Fab2L transgene.

Figure 2 .

 2 Figure 2. Epigenetic inheritance of eye colour is abrogated in the absence of GAF andPho binding to the Fab2L transgene.

Figure 3 .

 3 Figure 3. The insulator and PRE regions are required for Fab2L to acquire a repressed epigenetic state through paramutation.

Figure 4 .

 4 Figure 4. Long-distance chromatin contacts between Fab2L and Fab-7 depend on the functionality of the insulator region.

Figure 5 .

 5 Figure 5. An in vivo synthetic biology system promotes interchromosomal contacts between Fab-7 elements and is sufficient to induce TEI without genetic perturbation.

Figure 6 .

 6 Figure 6. Model: Stabilisation of interchromosomal contacts triggers an epigenetic memory of PRC2 binding.

  lines were described inBantignies et al., 2003 [START_REF] Bantignies | Inheritance of polycomb-dependent chromosomal interactions in Drosophila[END_REF] . The Fab2L, black[1] line and pre-established Fab2L epilines (Fab2L-R* and Fab2L-W*) were described in Ciabrelli et al., 2017 19 . For generation of the Fab2L mutant lines, a transgenic line was made containing an AttP insertion site at cytogenetic position 37B, by CRISPR-Cas9 of a w[1118] line (Bloomington Drosophila Stock Center) to cut at the exact site of Fab2L transgene insertion in the Fab2L line. Fab2L-INS was then generated by Phi-recombination of an AttB-containing plasmid containing the entirety of the Fab2L transgene, with directed mutations of the insulator GAF and Pho sites, into this 37B-AttP line. Injection services for these two lines were provided by BestGene Inc..Fab2L-PRE and Fab2L-INS-PRE were then generated by CRISPR-Cas9 editing of Fab2L-INS, with a two-guide RNA strategy designed and implemented by Rainbowgene Transgenic Flies Inc.

  Chromatin was eluted from beads in 100 μl of 10 mM EDTA, 1% SDS,50 mM Tris (pH 8) at 65°C for 15 minutes and eluted again in 150 μl of 10 mM EDTA, 0.67% SDS, 50 mM Tris (pH 8) at 65°C for 15 minutes, with the eluate collected on a magnetic rack each time. The 250 μl eluates and 250 μl of the Input DNA samples (1:2 input) were incubated overnight at 65°C to reverse crosslinks and treated with Proteinase K for 3 hours at 56°C. DNA was isolated by addition of an equal volume of phenol-chloroform, supernatants collected and then ethanol precipitated for 2 hours at -20°C in the presence of 20 μg glycogen by addition of 25 μl 3M sodium acetate and 625 μl ethanol. Samples were centrifuged at high speed for 1 hour and washed in 500 μl of 70% ethanol before resuspension in 200 μl H2O. Immunoprecipitated DNA was used to analyze the enrichment of specific DNA fragments by real-time PCR (qPCR), using a Roche Light Cycler 480 and the Light Cycler 480 SYBR green I Master mix. For each amplicon, IP DNA was normalized to Input DNA. The ChIP/Input ratio was further normalized to a positive control region (engrailed). ChIP amplicons for the insulator or PRE regions were specific to either the WT or mutated transgenic sequence, depending on the genotype analysed.

Figure S4 .

 S4 Figure S4. Fab2L can acquire a derepressed epigenetic state in trans by paramutation.

Figure S5 .

 S5 Figure S5. Long-distance chromatin contacts between Fab2L and Fab-7 increase uponFab-7 hemizygosity, but not in insulator mutants.

Figure S6 .

 S6 Figure S6. Chromatin contacts between Fab-7 elements are not present in the early embryo.

Figure S7 .

 S7 Figure S7. Synthetic constructs of the 3D-CIS system and its variants bind to their target sequences.

Figure S8 .

 S8 Figure S8. Activation of the 3D-CIS system induces chromatin contacts between Fab2Land Fab-7.

Figure S9 .

 S9 Figure S9. Promotion of chromatin contacts between homologous Fab-7 alleles, ratherthan interchromosomal loci, using the 3D-CIS system does not lead to TEI establishment.

Figure S10 .

 S10 Figure S10. Artificial promotion of chromatin contacts is unable to significantly decrease inter-loci distance or trigger TEI in the absence of the Fab-7 element.

Figure S11 .

 S11 Figure S11. Altered epigenetic states remain stable in the absence of artificially-induced chromatin contacts

Fab

  Figure 1

Figure 4 %

 4 Figure 4

  Figure 6

  Figure S2

  Figure S3

Fab2L

  Figure S5

Figure S6 A(

 S6 Figure S6

Fab2L(

  Figure S9

  distant Hox loci in Drosophila. Cell 144, 214-226. 10.1016/j.cell.2010.12.026. 51. Bantignies, F., and Cavalli, G. (2014). Topological Organization of Drosophila Hox Genes Using DNA Fluorescent In Situ Hybridization. Methods Mol. Biol. 1196, 103-120. which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission.

	bioRxiv preprint	doi:	; https://doi.org/10.1101/2023.07.13.548806	this version posted September 5, 2023.	The copyright holder for this preprint
	10.1007/978-1-4939-1242-1_7.

(

% Female Population A % Female Population % Female Population % Female Population (

  which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission.

	bioRxiv preprint	doi:	; https://doi.org/10.1101/2023.07.13.548806	this version posted September 5, 2023.	The copyright holder for this preprint
				Fab2L ; +				X	Fab2L, black[1] ; +
			X
				White Epiline
					Fab2L / Fab2L, black[1] ; +
	B			C			Fab2L, black[1] ; +
										Fab2L-INS, black[1] ; +
			0.7%
			12.1%	16%	8%					10.9%
	24.6%		23.1%		24%					19.3%	24.6%
	75.4%		64.1%	52%
										45.2%
	D			E
			Fab2L-PRE, black[1] ; +						Fab2L-INS-PRE, black[1] ; +
				75
				50
				25
				0 0	1	2	3	4	5	6	7	8
	7%
			21.2%
			39.4%
	93%			100%					100%
			39.4%
	Figure 2

Red Epiline Paramutation: Transfer of epigenetic state between alleles

  which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission.

		bioRxiv preprint	doi:	; https://doi.org/10.1101/2023.07.13.548806	this version posted September 5, 2023.	The copyright holder for this preprint
	A	Fab2L White Epiline	Chr 2			Fab2L, black[1] Naive Line		B
			P0									1.1%	6.6%	3.9%	1.7% 7.6%
												30.3%	23.7%
												100%	98.9%	59.2%	22.9%	44.1%
				F1	*
												8.8%
												25.7%	9.6%
												45.6%
				F2							74.3%	36%
		C	Fab2L ; +		X	Fab2L, black[1] ; +	D	Fab2L ; +	X
				1.1%					3.6%		1.1%	3.9% 3.9%	1.7% 1.7%
								13.9%	17.8%
			100%	98.9%			98.9%	98.9%	75.6%		100%	98.9%	44.1%	49.1%	68.3%	28.3%
								98.9%
			White Epiline								White Epiline
				Fab2L / Fab2L, black[1] ; +			Fab2L / Fab2L-INS, black[1] ; +
						Fab2L, black[1] ; +				Fab2L-INS, black[1] ; +
						8.6%	3.4%
						71.4% 20%	20.7% 17.2%	44.8% 13.8%				8.6% 8.6% 8.6% 46.6% 27.6%	39.7%	31% 12.1% 17.2%
		E	Fab2L ; +		X					F	Fab2L ; +	X
				1.1%					2%		1.1%
								9.4%
			100%	98.9%			90.6%	98%		100%	98.9%	100%	100%
			White Epiline								White Epiline
				Fab2L / Fab2L-PRE, black[1] ; +			Fab2L / Fab2L-INS-PRE, black[1] ; +
						Fab2L-PRE, black[1] ; +			Fab2L-INS-PRE, black[1] ; +
						12.9%	9.5%
						41.9%	12.9%
						32.3%	90.5%				100%	100%
	Figure 3

(

INS Fab2L-PRE Fab2L-INS-PRE

  which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission. not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission.

		Chr 2 Chr 3
	The copyright holder for this preprint (which was The copyright holder for this preprint this version posted September 5, 2023. ; https://doi.org/10.1101/2023.07.13.548806 doi: bioRxiv preprint this version posted September 5, 2023. ; https://doi.org/10.1101/2023.07.13.548806 doi: bioRxiv preprint The copyright holder for this preprint this version posted September 5, 2023. ; https://doi.org/10.1101/2023.07.13.548806 doi: bioRxiv preprint Fab2L ; + Fab2L ; Fab7[1] X Fab2L ; + Fab-7 P0 Fab-7
				F1		Fab2L ; Fab7[1] / +
	A	Fab2L ; +	B	F2		Fab2L ; +
		1.8%
		12.2%				22.2% 17.8%
			Stochastic silencing		13.3%
		41.7%	44.3%
						46.7%
		4%			2.1%
	23%	9.5%			42.6% 13.8%	41.5%	Fab2L
		63.5%
			0.9%			3.1%
		10.4%
		41.5%	47.2%			26.1%	70.8%
	C		D
		P0	+ ; +	X	Fab2L ; Fab7[1]
	Chr. II Fab2L / + ; Fab7[1] / + Chr. III PRE mini-white F H Fab2L ; Fab7[1] / + Chr. II Fab2L ; + F2 F3 Fab2L / + ; Fab7[1] / + Fab2L ; Fab7[1] / + Fab2L ; + F2 F3 Fab2L ; + + ; Fab7[1] X 67.4% 41.9% 3.6% 3.6% 63.7% 5.5% 30.8% 22.2% 43.2% 34.6% 10.2% 15.2% 1.7% 72.9% 0.8% 2.5% 20.7% 18.4% 52.9% 8% Fab2L-Figure S1 Chr. III PRE E G F1 Fab2L ; + F2 F1 P0 Fab2L ; + F2 18.6% 37.3% 44.1% 45.2% 100% 1% 53.8% mini-white 23.9% 37% 39.1% 2.5% 93.1% 6.9% 28% 18.8% 12.5%
			39.8%
			28.8%			66.2%

((which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission.

  which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission. which was not certified by peer review) is the author/funder. All rights reserved. No reuse allowed without permission.The copyright holder for this preprint this version posted September 5, 2023. ; https://doi.org/10.1101/2023.07.13.548806 doi: bioRxiv preprint

	bioRxiv preprint	doi:	; https://doi.org/10.1101/2023.07.13.548806	this version posted September 5, 2023.	The copyright holder for this preprint
	A								B
	Fab2L Red Epiline A	P0 % Male Population	F1 Chr 2		*	Fab2L, black[1] Naive Line	57.1% B	% Male Population Fab2L / Fab2L, black[1] ; + Red Epiline 41.1% 98.4% 30.3% 59.2% Fab2L ; + Fab2L, black[1] ; + 44.1% 22.9% 23.7% X 1.8% 1.6% 1.7% 3.9% 6.6% 7.6%
											1.4%	6.3%
		93.2% 6.8%	F2			42.6% 6.6%	23.7%		60.7%	39.3% 40.8%	28.2% 19.7% 9.9%	75.9%	8% 9.8%	1.4% 23.6% 14.1%
								27.1%			10.6%	50.1%
	Figure S4
		C								D
		% Male Population								% Male Population
							1.8%
		100%								100%	100%

(X Fab2L, black

[1]

; + (

Acknowledgements

We thank Montpellier Resources Imaging facility as well as the Drosophila facility (both affiliated to BioCampus University of Montpellier, CNRS, INSERM, Montpellier, France). We thank Yuki Ogiyama for assistance with the cloning of the plasmids for Drosophila injection and Judith Kassis for the gift of the anti-Pho antibody. M.F-J. and G.S. were supported by the European Research Council Advanced Grant 3DEpi with additional support for M. F.-J from the MSD Avenir Foundation Grant GENE-IGH. F.B. and G.C. were supported by CNRS. Research in the P.S. lab was supported by the John Fell Fund (Grant No 0011417). Research in the G.C. laboratory was supported by grants from the European Research Council (Advanced Grant 3DEpi), the European CHROMDESIGN ITN project (Marie Skłodowska-Curie grant agreement No 813327), the European E-RARE NEURO DISEASES grant "IMPACT", by the Agence Nationale de la Recherche (PLASMADIFF3D, grant N. ANR-18-CE15-0010, LIUVCHROM, grant N. ANR-21-CE45-0011), by the Fondation ARC (EpiMM3D), by the MSD Avenir Foundation ((Project GENE-IGH), and by the French National Cancer Institute (INCa, PIT-MM grant N. INCA-PLBIO18-362).

peer review) is the author/funder. All

Author contributions

M. F-J. and G.C. conceived of and led the project. M. F-J. designed and performed the experiments. M. F-J. and G.C. interpreted the data. M. F-J., G.S. and F.B. performed the Drosophila transgenerational selection experiments. M. F-J. composed the manuscript with editorial input from G.C. and P.S. All authors reviewed and commented on the manuscript.

Declaration of Interests

The authors declare no competing interests.

Supplemental Information Titles & Legends


Cédric Gernigon
email: cedric.gernigon@inria.fr

Silviu-Ioan Filip

Olivier Sentieys
email: olivier.sentieys@inria.fr

Clément Coggiola
email: clement.coggiola@cnes.fr

Mickaël Bruno
email: mickael.bruno@cnes.fr

Low-Precision Floating-Point for Efficient On-Board Deep Neural Network Processing

Keywords: Deep Neural Networks (DNN), Reduced Precision, Quantization-Aware Training (QAT), Floating-Point, Ship Detection, Semantic Segmentation

One of the major bottlenecks in high-resolution Earth Observation (EO) space systems is the downlink between the satellite and the ground. Due to hardware limitations, onboard power limitations or ground-station operation costs, there is a strong need to reduce the amount of data transmitted. Various processing methods can be used to compress the data. One of them is the use of on-board deep learning to extract relevant information in the data. However, most ground-based deep neural network parameters and computations are performed using single-precision floating-point arithmetic, which is not adapted to the context of on-board processing. We propose to rely on quantized neural networks and study how to combine low precision (mini) floating-point arithmetic with a Quantization-Aware Training methodology. We evaluate our approach with a semantic segmentation task for ship detection using satellite images from the Airbus Ship dataset. Our results show that 6-bit floating-point quantization for both weights and activations can compete with single-precision without significant accuracy degradation. Using a Thin U-Net 32 model, only a 0.3% accuracy degradation is observed with 6-bit minifloat quantization (a 6-bit equivalent integer-based approach leads to a 0.5% degradation). An initial hardware study also confirms the potential impact of such lowprecision floating-point designs, but further investigation at the scale of a full inference accelerator is needed before concluding whether they are relevant in a practical on-board scenario.

I. INTRODUCTION

Earth Observation (EO) provides an effective way of exploring the physical, chemical, and biological information related to the Earth. This information collected by EO satellites is widely used in various research fields, especially in relation to the environment, where the measurements made by EO satellites are indispensable. Moreover, these new space applications related to Earth observation produce a huge volume of data extracted from various image and radar sensors. Transmitting all this data is possible through communication between satellites and ground stations. However, EO systems are limited by these downlink communications, due to hardware limitations, onboard power constraints or ground-station operation cost, for example. Thus, there is a need to reduce the amount of data to be transmitted through the downlink. While data compression is widely used for size reduction, the idea of transmitting only relevant data through on-board processing has only recently started gaining interest.

Artificial Intelligence (AI), and in particular Deep Learning (DL), is starting to be successfully applied in space applications. However, the inference computation of many models is still mainly performed on ground platforms due to their memory footprint and computational intensity [START_REF] Furano | Towards the Use of Artificial Intelligence on the Edge in Space Systems: Challenges and Opportunities[END_REF]. To mitigate this computational burden and the space-to-ground communication bottleneck, recent research has focused on neural network compression [START_REF] Choudhary | A comprehensive survey on model compression and acceleration[END_REF]. Moreover, various techniques such as pruning [START_REF] Han | Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding[END_REF], weight sharing [START_REF] Dupuis | On the Automatic Exploration of Weight Sharing for Deep Neural Network Compression[END_REF], distillation [START_REF] De Vieilleville | Towards Distillation of Deep Neural Networks for Satellite On-Board Image Segmentation[END_REF] and quantization [START_REF] Hubara | Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations[END_REF], can be used to reduce the computational intensity to make it compatible with on-board processing.

Quantization deals with how (what number format(s) and bit widths to use) model parameters (such as weights and bias terms) and activation signals (inputs and outputs to layers in a model) are computed and stored at inference time. Furthermore, reducing precision and adapting the number representation make quantization a particularly effective and practical technique.

In this paper, we investigate how to efficiently use quantization to accelerate Deep Neural Network (DNN) inference for space applications, with a strong focus on semantic segmentation tasks. Towards this goal, we present our ongoing work on the efficient use of low-precision floating-point quantization (so-called minifloats). Our method consists in adapting a quantized DNN training approach that has so far been mostly used for integer/fixed-point-based quantization.

II. BACKGROUND AND RELATED WORK

Deep neural network parameters and computations have been traditionally stored and computed using 32-bit single-

A. DNN Quantization

Much work in recent years has focused on reducing the bit-width of data and arithmetic in DNN models without impacting task accuracy. Quantization to 8-bit [START_REF] Jacob | Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference[END_REF] and sub 8-bit [START_REF] Zhou | DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients[END_REF] integer formats has been shown to match singleprecision baselines on convolutional networks for several computer vision tasks. In certain cases, it is even possible to go down to extremely low precisions, such as binary [START_REF] Courbariaux | Binarized Neural Networks: Training Deep Neural Networks[END_REF] and ternary [START_REF] Li | Ternary Weight Networks[END_REF] DNNs.

On the FP arithmetic side, 16-bit formats such as half precision FP16 (5-bit exponent) and bfloat16 (8-bit exponent) have found success in both inference acceleration and mixedprecision DNN training [START_REF] Kalamkar | A Study of BFLOAT16 for Deep Learning Training[END_REF]. More recently, 8-bit floatingpoint formats have also been explored in inference/training scenarios [START_REF] Kuzmin | FP8 Quantization: The Power of the Exponent[END_REF], [START_REF] Micikevicius | FP8 Formats for Deep Learning[END_REF], [START_REF] Wu | Phoenix: A Low-Precision Floating-Point Quantization Oriented Architecture for Convolutional Neural Networks[END_REF] and are starting to get hardware support (e.g. the NVIDIA Hopper architecture is an example). Sub 8-bit FP custom formats are also being studied in the literature in the context of model compression and inference acceleration [START_REF] Tambe | Algorithm-Hardware Co-Design of Adaptive Floating-Point Encodings for Resilient Deep Learning Inference[END_REF].

To apply such quantization schemes on a DNN model, it is necessary to adjust the parameters according to the target format. There are two main approaches to do this in practice. The first is Post-Training Quantization (PTQ), in which quantization is applied after training. While fast, it can lead to non-negligible loss in accuracy for extremely low bit width formats [START_REF] Banner | Post training 4-bit quantization of convolutional networks for rapid-deployment[END_REF]. The other approach is to iteratively quantize the model during training. This process is known as Quantization-Aware Training (QAT). While much slower, it generally leads to better quantization results. For an in-depth overview of quantization in the context of DNNs the reader can consult [START_REF] Gholami | A Survey of Quantization Methods for Efficient Neural Network Inference[END_REF].

B. Quantization-Aware Training

The emergence of QAT methods can be traced back to the pioneering work of [START_REF] Courbariaux | BinaryConnect: Training Deep Neural Networks with binary weights during propagations[END_REF], [START_REF] Courbariaux | Binarized Neural Networks: Training Deep Neural Networks[END_REF] on binary quantization of DNNs. The overall approach consists of using the quantized version of the network during forward and backward computations done throughout training, while performing updates on a full precision copy of the parameters (e.g. weights and biases). A critical ingredient of these methods is the use of a so-called Straight-Through Estimator (STE) [START_REF] Bengio | Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation[END_REF] that allows the backpropagation of gradients with respect to quantized binary variables (weights and activation signals). Subsequent work [START_REF] Zhou | DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients[END_REF] extends these ideas to larger bit widths and tackles quantization of gradient signals as well. Other methods propose learning parameters that bound the range of activation signals inside the network [START_REF] Choi | PACT: Parameterized Clipping Activation for Quantized Neural Networks[END_REF] and new gradient estimates that allow learning of appropriate scaling factors for integer-based quantization of both weights and activations [START_REF] Esser | Learned Step Size Quantization[END_REF].

III. METHODOLOGY

A. Floating-point Encoding

The floating-point formats we use, denoted as EeMm, are based on the IEEE-754 standard [START_REF]IEEE Standard for Floating-Point Arithmetic. IEEE Std[END_REF]. A floating-point value X with m bits of mantissa and e bits of exponent is represented (in binary notation) as

X = (-1) s × 1. x 1 . . . x m M X ×2 E X -E B ,
where

s is the sign bit, M X is the m-bit fractional mantissa (M X ∈ [0, 1)) and E X ∈ [0, 2 e -1] is the integer exponent. E B is the exponent bias term, which in the case of IEEE-like encodings is E B = 2 e-1 -1.
With respect to a fully IEEEcompliant format, we do not support ±∞, NaN encodings, and subnormal values. This leads to more representable values and simpler hardware (see [START_REF] Metzgen | Higher Performance Neural Networks with Small Floating Point[END_REF]). Values that overflow are saturated to the maximal representable number, whereas zeros are represented by E X = 0 and M X = 0. The values that would have been treated as subnormals (E X = 0, M X > 0 and X = (-1)

s × 0.M x × 2 -E B ) are instead viewed as one extra binade of normal values. The actual exponent E X -E B ∈ [-2 e-1 + 1, 2 e-1 ]
, as described in the IEEE-754 standard, covers an almost symmetric range of positive and negative values. If the dynamic range of the data to quantize is known, E B can be adapted to better match it. The PTQ AdaptivFloat [START_REF] Tambe | Algorithm-Hardware Co-Design of Adaptive Floating-Point Encodings for Resilient Deep Learning Inference[END_REF] method does this by examining the maximum magnitude of the weight tensors in each layer of the network. In a QAT setting, E B can be better chosen by learning it using a STE [START_REF] Bengio | Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation[END_REF] approach. Just as for the scaling factor of an integer-based format, a similar method [START_REF] Kuzmin | FP8 Quantization: The Power of the Exponent[END_REF] can be used to learn a real bias exponent in the floating-point case. In order to achieve a more hardwarefriendly minifloat format, we propose to further quantize this learned real exponent bias to an integer. We explore this in Sec. III-B.

B. Quantization Scheme

Our work can be seen as an extension of [START_REF] Kuzmin | FP8 Quantization: The Power of the Exponent[END_REF]. While initially considered for FP8 quantization with subnormals and a real exponent bias, we investigate its use in a sub 8-bit FP context without subnormals and an integer exponent bias. Subnormal support adds some hardware overhead, but for minifloat formats with small mantissa (2 or 3 bits), using the subnormal range as normal values can still lead to good results with a much smaller overhead (see [START_REF] Tatsumi | Mixing Low-Precision Formats in Multiply-Accumulate Units for DNN Training[END_REF]).

We quantize weights and activations using an STE-based approach as follows:

Forward: X q = quantize (X, E 0 ) Backward: ∂ℓ ∂X = ∂ℓ ∂X q ∂X q ∂X
where X is an unquantized weight or activation signal, ℓ is the loss function, and X q is the EeMm quantized version of X computed using Algorithm 1. The straight-through moniker comes from the fact that we take ∂X q /∂X = 1.

We propose to learn the exponent biases of each layer's weights and activations during training. The learned real values are rounded up (line 1 of Algorithm 1) to scale the network signals (weights & activations) by powers of two. The real values are stored for future use in the update phase with a SGD-type procedure (going from an iteration t to t + 1) as summarized in Algorithm 2 (lines 18 and 19). The g quantities in lines 8-13 represent the gradients of the loss ℓ with respect to the activation, weight and exponents, respectively. Their computation (the backward function calls) is handled through the PyTorch autograd engine. A schematic view of the entire QAT iteration is also given in Figure 1.

IV. EXPERIMENTS

We apply our approach on a image classification problem on the CIFAR-10 dataset [START_REF] Krizhevsky | Learning Multiple Layers of Features from Tiny Images[END_REF] and on a satellite image segmentation task using a lightweight U-Net model (adapted from [START_REF] Vaze | Low-Memory CNNs Enabling Real-Time Ultrasound Segmentation Towards Mobile Deployment[END_REF]).

A. CIFAR-10 Dataset and Implementation Details

The CIFAR-10 dataset consists of 32 × 32 pixels RGB labeled images divided into 10 categories. The dataset is composed of 50, 000 training images and 10, 000 test images. We trained a ResNet-20 model [START_REF] He | Deep Residual Learning for Image Recognition[END_REF] from scratch for 300 epochs using a cross entropy loss function. We apply the data augmentation operations proposed in [START_REF] Lee | Deeply-Supervised Nets[END_REF] over the training set and we use an SGD optimizer with 128 batch size, weight Algorithm 1 quantize: minifloat quantization algorithm Require: real-valued tensor X, floating-point format EeMm, real learned exponent bias E0. Ensure: quantized tensor Xq in the EeMm format.

1: EB = ⌈E0⌉ 2: xmin = 2 -E B • (1 + 2 -m ) 3: xmax = 2 2 e -1-E B • (2 -2 -m ) 4: Xc = clamp (X, -xmax, xmax) 5: Xscale = 2 ⌊log 2 (Xc)⌋-
W t q,k ← quantize W t k , E t 0,w,k 4: Ỹt k ← forward Y t q,k-1 , W t q,k 5: Y t k ← quantize Ỹt k , E t 0,a 6
: end for 7: 2. Backward propagation:

8: g Y t L = ∂ℓ Y t L , T t ∂Y t L 9: for k = L down to 1 do 10: g Y t k-1 ← backward_activ g Y t q,k , W t q,k 11: g W t k ← backward_weight g Y t q,k , Y t q,k 12: g E t 0,a,k ← backward_exp_bias_a g Y t q,k , Y t q,k 13: g E t 0,w,k ← backward_exp_bias_w g Y t q,k , Y t q,k 14
W t+1 k ← update W t k , g W t k , η
18:

E t+1 0,w,k ← update E t 0,w,k , g E t 0,w,k
, η

19:

E t+1 0,a,k ← update E t 0,a,k , g E t 0,a,k
, η 20: end for decay set to 0.0001, and momentum to 0.9. Weights are initialized using the Kaiming method [START_REF] He | Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification[END_REF] and the learning rate is scheduled using cosine annealing with starting rate set to 0.1.

B. Airbus Ship Dataset and Implementation Details

The Airbus Ship Dataset1 consists of 768 × 768 pixels RGB satellite images of ships with sea and harbor in the back- We use a Thin U-Net [START_REF] Vaze | Low-Memory CNNs Enabling Real-Time Ultrasound Segmentation Towards Mobile Deployment[END_REF] model (see Fig. 2), a smaller version of the larger and more widely known U-Net [START_REF] Ronneberger | U-Net: Convolutional Networks for Biomedical Image Segmentation[END_REF] architecture. More specifically, we consider the Thin U-Net 32 model, where 32 refers to the number of channels for each convolution. Its memory size is smaller by a factor of 290 compared to a standard U-Net architecture (going down from 288.08Mb to 0.99Mb), with little impact on accuracy. Similar to the standard U-Net, this model consists of two distinct blocks, an encoder and a decoder, interconnected by skip connections. The encoder is composed of 5 blocks containing two groups of 3×3 convolution + batch normalisation + ReLU with 2 × 2 maxpooling to down sample the feature maps. Followed by the decoder, it also has 5 blocks containing a 2 × 2 bilinear up-sampling layer and two groups of 3 × 3 convolution + batch normalisation + ReLU. Prediction masks are generated by a final 1 × 1 convolution without padding.

We apply random horizontal and vertical flip, random crop to resize images to 256 × 256 pixels, random brightness and random contrast as data augmentations on the training set. We use ADAM as the optimizer with a batch sizes of 32 and an aggregated global loss consisting of a Jaccard loss combined with binary cross entropy with logits, which according to [START_REF] Wang | Hard Attention Net for Automatic Retinal Vessel Segmentation[END_REF] gives the best results in practice. The learning rate is initially set to 0.001 and we use a multi step scheduler in order to reduce the learning rate by 0.5 every 200 epochs. We train the network for 600 epochs, using the Kaiming weight initialization method [START_REF] He | Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification[END_REF]. For computing the quantized model, we used the QAT approach from Algorithm 2 to fine tune the pretrained single-precision model for 50 epochs. Just like with an integer scaling factor [START_REF] Bhalgat | LSQ+: Improving Low-Bit Quantization Through Learnable Offsets and Better Initialization[END_REF], a good initialization of the exponent bias parameter is key to convergence with good accuracy. We have found that in practice good initial estimates can be determined by first training the network for a small number of iterations without optimizing the exponent bias (we did this for 200 iterations in our tests) and then picking it based on the values with maximum magnitude in the weight and activation tensors seen during this process, leading to

E 0 = 2 e-1 -log 2 max |X| 2 -2 -m .

C. Results

We use PyTorch 1.13 on a cluster of eight NVIDIA V100 GPUs to perform our experiments. Unlike the majority of integer-based quantization methods described in the literature which quantize the first and last layers to 8 bits (which is

63.4 E3M2 E3M2 2 2 e-1 E X = 0 64.8 E3M2 E3M2 2 2 e-1 M X = 0 and E X = 0 70.1 E3M3 E3M3 learn E X = 0 70.0 E3M2 E3M2 learn E X = 0 71.4 E4M2 E4M2 learn M X = 0 and E X = 0 70.9 E3M3 E3M3 learn M X = 0 and E X = 0 70.7 E3M2 E3M2 learn M X = 0 and E X = 0 68.1 E2M2 E2M2 learn M X = 0 and E X = 0
usually a larger word length than that for the other layers), we use the same small length format for all layers. This can lead to slightly better compression ratios, without affecting accuracy.

We have compared our custom floating-point formats with fixed-point and integer arithmetic alternatives, two quantization formats commonly used to accelerate inference. Results on the CIFAR-10 dataset are given in Table I. While fixedpoint quantization seems to degrade model accuracy significantly, low-precision floating-point variants are competitive with integer-based alternatives for very low quantization levels.

The results of applying minifloat quantization to ship detection are summarized in Table II. On this more complex task, 6-bit floating-point formats are necessary to match single precision accuracy, which is evaluated using an Intersection over Union (IoU) metric.

We note that a visual analysis of prediction outputs for the minifloat quantized model shows results that are relatively close to the single-precision baseline. When single ships are present in the image, our quantized model usually does a good job of detecting them (Fig. 3). Detecting small ships, as well as side by side ships and inshore ships is more challenging than detecting single large ones, leading to poorer predictions even with single-precision models (Fig. 4 and Fig. 5). Inshore [START_REF] Nie | Inshore Ship Detection Based on Mask R-CNN[END_REF] and small ship [START_REF] Zhang | R-CNN-Based Ship Detection from High Resolution Remote Sensing Imagery[END_REF] detection are challenging topics, both subject to active research.

D. Hardware Implementation Aspects

The results we have shown so far suggest that minifloat quantization is potentially a good choice for low-precision inference acceleration. However, floating-point addition is in general more resource-intensive than its integer/fixed-point counterpart.

This is somewhat counterbalanced by the multiplier, which in case of minifloats can be implemented efficiently using just look-up tables (LUTs), as opposed to an 8-bit integer variant that would require DSP blocks, which are much lower in number than LUTs on modern FPGAs (e.g. in a AMD-Xilinx UltraScale+ VUP13 FPGA, for every DSP block there are 140 6-input LUTs [START_REF] Metzgen | Higher Performance Neural Networks with Small Floating Point[END_REF]). DSPs can then be configured to implement adder trees for the accumulation part of multiply-accumulate (MAC) units, improving overall logic density. For instance, the results presented by AMD-Xilinx in [START_REF] Metzgen | Higher Performance Neural Networks with Small Floating Point[END_REF] claim 60% higher performance and 12.5% memory traffic reduction (leading also to lower power usage) when using a minifloat E3M3 format as opposed to INT8 in a ResNet-50 accelerator. To achieve this performance, they implemented as basic building block a hybrid MAC operator that combines the best of both worlds, a LUT-based minifloat multiplier and a simpler fixed-point adder. Compared to our minifloat formats where zeros are encoded with M X = 0 and E X = 0, the formats from [START_REF] Metzgen | Higher Performance Neural Networks with Small Floating Point[END_REF] are slightly different, opting for a larger range of zero code words corresponding to E X = 0. While the AMD-Xilinx choice leads to a slightly more efficient minifloat multiplier design (see Fig. 6 for a schematic view of the multiplier and an explanation), our synthesis results using Verilog and Vivado 2022.1 with a Zynq UltraScale+ ZCU102 as target show that the differences in LUT count between the two zero encoding choices are modest (see Fig. 7). The extra encoding space saved by our choice (M X ̸ = 0) has a positive impact on accuracy, as can be seen Table II for the E3M2 and E3M3 formats. We believe that these properties make the M X = 0 and E X = 0 encoding a better choice in practice.

E 1 M 1 E 1 M 2 E 1 M 3 E 2 M 1 E 2 M 2 E 2 M 3 E 3 M 1 E 3 M 2 E 3 M 3 E 4 M 1 E 4 M 2 E 4 M
The AMD-Xilinx ResNet-50 implementation uses a realvalued scaling factor. With an integer exponent bias like we propose, the logic needed to handle its propagation (e.g. in convolution operations) would amount to a simpler integer exponent shift. In the case of a hybrid MAC design, this could also potentially lead to simpler logic when converting from integers (accumulator outputs) to minifloat (multiplier operands). We leave the testing of these statements as future work.

V. CONCLUSION & ON-GOING WORK

Data transmission from satellites remains challenging due to the many limitations of downlink communication. While DNNs are successfully applied to spatial data processing, deep learning algorithms are now being considered as an onboard alternative to extract the relevant data to be sent to ground stations. However, strict hardware limitations of onboard systems make the use of vanilla DNN models in 32-bit floating-point arithmetic unrealistic.

We propose a QAT algorithm for learning compressed lowprecision floating-point DNN models. In addition, we learn the exponent biases of each layer for both weights and activations. Fig. 6: Implementation of a minifloat EeMm multiplier (adapted from [START_REF] Metzgen | Higher Performance Neural Networks with Small Floating Point[END_REF]). The minifloat multiplier preserves exponent accuracy by using a (e + 1)-bit exponent output and truncates the mantissa output to m + 1 bits, reducing the size of the logic needed to convert minifloat to fixed point in a hybrid minifloat fixed-point MAC design. To implement the E X = 0, M X = 0 zero encoding at the hardware level, we add the input mantissas as inputs to the block Detect Zero (dashed lines). This slightly complicates the zero detection logic compared to the E X = 0 design from [START_REF] Metzgen | Higher Performance Neural Networks with Small Floating Point[END_REF].

Our experiments on the CIFAR-10 and Airbus Ship datasets show good results, with low-precision floating-point models being competitive with single precision baselines.

To show the potential impact of using floating-point data formats, we have also suggested an implementation of a minifloat-enabled multiplier based on [START_REF] Metzgen | Higher Performance Neural Networks with Small Floating Point[END_REF] that can be used as a basis for a full DNN inference accelerator for our models. The next step will be to design, test and deploy such an accelerator for the full quantized Thin U-Net 32 on FPGA targets to better gauge the feasibility of using deep learning models with lowprecision FP data in an on-board space context.

Fig. 1 :

 1 Fig. 1: An overview of the QAT scheme we use in the training of minifloat quantized DNNs. The distinguishing feature of a QAT flow compared to a standard training procedure is the presence of quantizer blocks in each layer and activation function that use the STE mechanism to differentiate with respect to the quantized signals.

  : end for 15: 3. Parameter update: 16: for k = 1 to L do 17:

Fig. 2 :

 2 Fig. 2: Thin U-Net 32 architecture. It consists of a 5-stage encoder, followed by a 5-stage decoder, with skip connections between each corresponding stage pairs in the encoder-decoder blocks.

EFig. 7 :

 7 Fig. 7: LUT consumption when implementing a minifloat multiplier with two different zero encodings.

Fig. 3 :Fig. 4 :Fig. 5 :

 345 Fig. 3: Prediction of a single ship with original image and its associated masks.

  Xq = Xq • I |Xq|≥x min 8: return Xq Algorithm 2 FP QAT algorithm for training a L-layer model Require: a minibatch of inputs Y t 0 and targets T t , weights W t ∈ R, activation exponent biases E t 0,a , weight exponent biases E t 0,w , learning rate η > 0, loss function ℓ. Ensure: updated parameters W t+1 , E t+1 0,w and E t+1

			m
	6: Xq =	Xc Xscale	• Xscale
	7: 0,a

1: 1. Forward propagation: 2: for k = 1 to L do 3:

TABLE I :

 I Comparison of prediction accuracy for CIFAR-10 with different arithmetic formats and bit-widths for weights (W) and activations (A).

	Arithmetic Format Top-1 W bit-width A bit-width
	Single precision	92.5	M23E8	M23E8
	Fixed-point	90.2	3	3
	Integer	91.6	3	3
		91.3	M1E1	M2E2
	Minifloat	91.3	M1E2	M1E2
		90.8	M1E1	M1E2

TABLE II :

 II Comparison of prediction accuracy for the Thin U-Net 32 model on the Airbus Ship dataset with different arithmetic formats and bit-widths.

	Format	mean IoU W bit-width A bit-width	scaling factor	zero encoding
	Single-precision	71.0	M23E8	M23E8	/	M X = 0 and E X = 0
	Fixed-point	44.5	6	6	2 ⌈log 2 (max |X|)⌉	Zero point = 0
	Integer	70.5 68.3	6 5	5 4	learn learn	Zero point = 0 Zero point = 0
	Minifloat

Kaggle Airbus Ship Detection Challenge (July

2018): https://www.kaggle. com/c/airbus-ship-detection

ACKNOWLEDGMENT

We would like to thank Stéphane May for his help and advice. This work was performed using HPC resources from GENCI-IDRIS (Grant 2023-AD011013080R1).

This work is partly supported by the French Space Agency (CNES), Institut National de Recherche en Informatique et en Automatique (INRIA), and the COMINLabs (10-LABX-0007) LeanAI project.