Astrid Lancrey

Alexandra Joubert

Evelyne Duvernois-Berthet

Etienne Routhier

Saurabh Raj

Agnès Thierry

Marta Sigarteu

Loic Ponger

Vincent Croquette

Julien Mozziconacci
email: julien.mozziconacci@mnhn.fr

Jean-Baptiste Boulé
email: jean-baptiste.boule@mnhn.fr

Nucleosome positioning on large tandem DNA repeats of the '601' sequence engineered in Saccharomyces cerevisiae

The artificial 601 DNA sequence is often used to constrain the position of nucleosomes on a DNA molecule in vitro. Although the ability of the 147 base pair sequence to precisely position a nucleosome in vitro is well documented, application of this property in vivo has been explored only in a few studies and yielded contradictory conclusions. Our goal in the present study was to test the ability of the 601 sequence to dictate nucleosome positioning in Saccharomyces cerevisiae in the context of a long tandem repeat array inserted in a yeast chromosome. We engineered such arrays with three different repeat size, namely 167, 197 and 237 base pairs. Although our arrays are able to position nucleosomes in vitro, analysis of nucleosome occupancy in vivo revealed that nucleosomes are not preferentially positioned as expected on the 601-core sequence along the repeats and that the measured nucleosome repeat length does not correspond to the one expected by design. Altogether our results demonstrate that the rules defining nucleosome positions on this DNA sequence in vitro are not valid in vivo, at least in this chromosomal context, questioning the relevance of using the 601 sequence in vivo to achieve precise nucleosome positioning on designer synthetic DNA sequences.

Introduction

The nucleosomes are more than structural proteins that enable folding of a meter-long molecule into a micrometer-diameter nucleus, and are now recognized as master players in the many facets of genome metabolism.

The deposition of post-translational modifications on nucleosomes along genomes are tightly correlated with the regulation of gene expression [START_REF] Kornberg | Primary Role of the Nucleosome[END_REF]. The first experimental evidence for the existence of nucleosomes came from their regular, periodic assembly along the DNA molecule [START_REF] Olins | Spheroid chromatin units (v bodies)[END_REF][START_REF] Kornberg | Chromatin structure: A repeating unit of histones and DNA[END_REF]. In the following decades, the nucleosomal periodicity was measured in various contexts and the Nucleosomal Repeat Length (NRL) was shown to vary in different species, different cell types and even in different regions of chromosomes [START_REF] Kornberg | Structure of chromatin[END_REF]. As soon as the first eukaryotic DNA sequences were available, a link between the DNA helical pitch and the folding of the molecular into chromatin has been noted [START_REF] Trifonov | The pitch of chromatin DNA is reflected in its nucleotide sequence[END_REF]. As more NRL measures became available, Jonathan Widom noticed that NRL took preferential values of 167 + n * 10 base pairs (bp) and linked this observation to the value of the DNA helical pitch [START_REF] Widom | A relationship between the helical twist of DNA and the ordered positioning of nucleosomes in all eukaryotic cells[END_REF]. As the periodic position of nucleosomes became of high mechanistic interest to explain chromatin function, nucleosomal array reconstruction protocols for in vitro studies were developed. In order to test the physical and structural properties of DNA wrapping around an histone octamer and the role of the underlying DNA sequence, Widom and colleagues isolated, using a SELEX approach, a set of DNA sequences that could precisely position nucleosomes in vitro [START_REF] Lowary | New DNA sequence rules for high affinity binding to histone octamer and sequence-directed nucleosome positioning[END_REF]. Among the DNA sequences obtained, the "601" sequence was retained to be the one forming the most stable nucleosomal complex in vitro, together with an histone octamere. This "601" sequence has been successfully used in many in vitro studies to reconstruct regular arrays of nucleosomes for electron microscopy [START_REF] Robinson | Structure of the '30 nm' chromatin fibre: a key role for the linker histone[END_REF][START_REF] Correll | Short nucleosome repeats impose rotational modulations on chromatin fibre folding[END_REF] and single molecule manipulations [START_REF] Bancaud | Structural plasticity of single chromatin fibers revealed by torsional manipulation[END_REF][START_REF] Bancaud | Nucleosome chiral transition under positive torsional stress in single chromatin fibers[END_REF][START_REF] Meng | Quantitative analysis of single-molecule force spectroscopy on folded chromatin fibers[END_REF]. Structures of nucleosome core particles containing the 601 positioning sequence were also obtained by X-ray christallography, revealing in this case 145 bp of the 601 sequence wraped around the nucleosome [START_REF] Vasudevan | Crystal Structures of Nucleosome Core Particles Containing the '601' Strong Positioning Sequence[END_REF]. Regardless of the exact size of the 601 nucleosome, these studies confirmed that the 167 + n * 10 bp NRL rule was required to form regular arrays and revealed both their ambivalent ability to be highly resilient to supercoiling in their extended form [START_REF] Bancaud | Structural plasticity of single chromatin fibers revealed by torsional manipulation[END_REF][START_REF] Kaczmarczyk | Single-molecule force spectroscopy on histone H4 tail-cross-linked chromatin reveals fiber folding[END_REF] and to fold into very dense and compact fibers under specific buffer conditions [START_REF] Routh | Nucleosome repeat length and linker histone stoichiometry determine chromatin fiber structure[END_REF]. The detailed biophysical modeling of nucleosomal array structures showed that the observed quantized behavior of the NRL is primarily due to the combination of the nucleosome/nucleosome steric hindrance and the 10 bp helical pitch of the DNA linker [START_REF] Barbi | How the chromatin fiber deals with topological constraints[END_REF][START_REF] Wong | An all-atom model of the chromatin fiber containing linker histones reveals a versatile structure tuned by the nucleosomal repeat length[END_REF].

The potential use of the 601 sequence for in vivo studies has early been envisioned by Lowary and Widom themselves. However, only a handful of studies have been carried so far in this direction and the ability of one inserted sequence to position a nucleosome has led to divergent conclusions that pointed to the dependence on the context in which the 601 sequence was used (episomal vs genomically inserted) [START_REF] Gracey | An in vitro-identified high-affinity nucleosome-positioning signal is capable of transiently positioning a nucleosome in vivo[END_REF][START_REF] Subtil-Rodríguez | BRG1 helps RNA polymerase II to overcome a nucleosomal barrier during elongation, in vivo[END_REF][START_REF] Perales | Histone occupancy in vivo at the 601 nucleosome binding element is determined by transcriptional history[END_REF][START_REF] Cole | Perfect and imperfect nucleosome positioning in yeast[END_REF]. In yeast, single copies of the 601 sequence inserted genomically in an open reading frame (ORF) were shown to not promote strong positioning [START_REF] Perales | Histone occupancy in vivo at the 601 nucleosome binding element is determined by transcriptional history[END_REF]. Regular arrays of repeats of various sizes have also been used in the context of extrachromosomal plasmids. Although the 601 repeats affected nearby gene expression, the positioning of nucleosomes on the repeated sequences was not directly assessed, leaving open the possibility for indirect effects [START_REF] Chakraborty | Nucleosome-Positioning Sequence Repeats Impact Chromatin Silencing in Yeast Minichromosomes[END_REF]. Our goal here was to assess the ability of 601 sequence to position nucleosomes in vivo on long tandem arrays using three different linker lengths of 20, 50 and 90 bp. These lengths would respectively constrain the NRL respectively to 167, 197 or 237 bp. Using a genome editing approach that we recently developed to engineer long DNA tandem repeats in yeast chromosomal DNA [START_REF] Lancrey | Locus specific engineering of tandem DNA repeats in the genome of Saccharomyces cerevisiae using CRISPR/Cas9 and overlapping oligonucleotides[END_REF], we assembled tens of 601 repeats and analyzed the statistical positioning of nucleosomes on these arrays. Our results show that, in vivo, the 601 sequence is not able to position the nucleosomes on the repeated 601 cores, yielding NRL which are different from the ones we designed, except for the control 167 bp tandem array, which approximates the naturally observed yeast NRL.

Results

In vivo assembly of 601 tandem DNA repeats in the yeast genome

We used a strategy that we developed recently [START_REF] Lancrey | Locus specific engineering of tandem DNA repeats in the genome of Saccharomyces cerevisiae using CRISPR/Cas9 and overlapping oligonucleotides[END_REF] to assemble tandem DNA repeats containing the 601 sequence in the genome of Saccharomyces cerevisiae. Specifically, we engineered three designs of overlapping oligonucleotides to partially replace the non-essential YMR262W gene and its promoter, located in chromosome XIII, to give rise to arrays of tandem repeats of 167, 197 or 237 bp long monomers (Figure 1 and supplementary Figure S1). Integration at the YMR262W locus resulted in the removal of 129 bp upstream the start codon and 231 bp downstream the start codon [START_REF] Lancrey | Locus specific engineering of tandem DNA repeats in the genome of Saccharomyces cerevisiae using CRISPR/Cas9 and overlapping oligonucleotides[END_REF]. Sizes of arrays obtained by this approach ranged from one repeat to arrays longer than 15 kb, as estimated by gel electrophoresis, corresponding to more than 50 repeats (Figure 2). Given the high efficiency of this assembly method, screening of only a few dozen of clones were needed to find clones containing large repeats. Correct assembly was verified by sequencing of the assembly junctions (Supplementary Figure S2) as well as the homogeneity of the tandem repeats (Supplementary Figure S3). For each of the three designs, the clone with the largest array was kept for chromatin studies. Finally, since tandem repeats may be susceptible to genomic instability, we measured the stability of the arrays upon mitotic division. After around 25 generations, the percentage of rearranged clones was respectively 1, 2 and 4%, demonstrating that large tandem repeats of the 601 sequences are easily maintained in the yeast genome (Supplementary Figure

The NRL measured on 601 repeats are different in vitro and in vivo

In order to verify that our design of the DNA linkers 20, 50 and 90 bp did not affect the ability of the 601 sequence to position nucleosomes in vitro, we assembled chromatin on plasmids containing respectively 20, 14 and 14 repeats of the 167, 197 and 237 bp monomers (supplementary Figure S5A-C). In vitro assembled chromatin was digested by Micrococcal Nuclease (MNase), separated on gel and probed by Southern Blotting using a 601 probe. The results confirmed that nucleosomes were positionned by the 601 core on all three 601 repeats (supplementary Figure S5D,E). However with the 601-237 design we observed two additional shorter ladders reflecting that some nucleosomes are positioned according to a second NRL when the designed NRL is equal to 237 bp.

To get insights into nucleosome organization in the 601 repeats in vivo we first performed MNase-gel experiments on the wild type YPH499 strain and the three yeast strains harboring the repeats. MNase-gel experiments consisted on partially digesting the chromatin of cells harboring 601 repeats with MNase, followed by gel analysis of NRL over the whole genome or specifically within the repeats (Figure 3, see Material and Methods for details). We measured a NRL of 163 ±1 bp for the whole genome, similar to the values expected for S. cerevisiae [START_REF] Hughes | A Functional Evolutionary Approach to Identify Determinants of Nucleosome Positioning : A Unifying Model for Establishing the Genome-wide Pattern[END_REF]. The values measured after Southern blotting within the 601 arrays revealed that nucleosomes are spaced in average according to a 170±4 bp NRL, a 176±7 bp NRL and a 171±9 bp NRL respectively for the 167 bp, 197 bp and 237 bp repeats (Figure 3C,D). The NRL measured for the 601-167 repeats is close to the one expected, however nucleosomes did not appear preferentially spaced according to the pre-defined 197 and 237 bp NRL and are therefore not phased with the repeats. organization in all libraries [START_REF] Mavrich | A barrier nucleosome model for statistical positioning of nucleosomes throughout the yeast genome[END_REF], regardless of the two extent of digestion by MNase (Supplementary Figure S8). For each the three 601 arrays tested, the pattern of positioning of the nucleosome on the 601 repeat is very reproducible between libraries and does not depend strongly on their digestion level. For the 167 bp repeats most of the nucleosomes are positioned around the middle of the second half of the 601 core sequence (Figure 4A). For the 197 bp repeats, the positioning is more uniform, with a higher density at the boundary between the 601-core end and the linker (Figure 4B). For the 237 bp repeats, we detected three preferential positions centered respectively around 20, 90 and 160 bp (Figure 4C). In summary, nucleosomes are periodically positioned according to the pre-defined NRL only for the 601-167 repeats. However they are not preferentially found on the 601 core sequence. In the two other 601 designs we did not observe an exclusive nucleosome spacing according to the designed NRL, a results which confirms our MNase-gel results (Figure 3C,D).

In vivo assembled 601 repeats are weakly transcribed

Since the transcription machinery is strongly associated with chromatin remodeling complexes that could have a predominant role in nucleosome occupancy, we then checked the transcription activity of our synthetic 601 arrays.

To see whether deleting the YMR262W promoter upon repeat assembly was sufficient to abolish polymerase II transcription through the assembled array and the remaining of the YMR262W gene, we measured genome wide transcription in exponentially growing cells by RNA-seq of polyadenylated or total RNAs, and assessed the transcription level of every gene in the three 601 strains and in the wild-type strain. To quantify how many transcripts carry the 601 sequence, we compared the expression level in the 601 array with the rest of the coding genes by ordering every transcript according to its counts-per-million (cpm) value. When considering poly adenylated transcripts only, respectively 4.28, 1.94 and 12.7 per million transcripts were identified in the 167, 197 and 237 bp NRL strains. When ordered from high to low expression, transcripts containing the 601 sequence ranked respectively at position 5325, 5522 and 4789, meaning that in each strain more than 80% of the genes expressed are detected at a higher level. If we take into consideration the repeated nature of the array, the number of transcripts should also be divided by the approximate number of repeat within each array (∼50). For total RNAs, more 601 sequences were detected in the 167 and 237 bp arrays, hinting at some transcription in the arrays of non polyadenylated RNA species. Once normalized by the number of repeats, however, this still amounted to a low steady state level compared to most genes (Table 1), supplementary Tables S4,S5).

Discussion

The in vivo spacing between nucleosomes has been proposed to play an important role in many chromosome functions ranging from the definition of chromatin states and gene activity [START_REF] Valouev | Determinants of nucleosome organization in primary human cells[END_REF] to homologous pairing [START_REF] Hedley | Nucleosome induced homology recognition in chromatin[END_REF]. In this study we built synthetic DNA arrays in the genome of S. cerevisiae, as a first step to address experimentally the biological effects of unnatural nucleosome spacing. Based on the extensive published work on nucleosome positioning in vitro, we chose the 601 DNA sequence as a candidate sequence to drive nucleosome positioning over large arrays of tandem DNA, with three monomer length of respectively 167, 197 and 237 bp. First, we show that using our genome editing approach, we can easily assemble more than 15 kb-long homogeneous arrays of the 601 sequence for the three monomer tested. As expected, we show that identical arrays can position nucleosomes in vitro, at least to some extent (Supplementary Figure S5). However, the 601 sequence does not appear to force nucleosome positioning in vivo on any of the large tandem repeats. On the contrary, results of the MNase-gel and MNase-seq approaches indicate that in the yeast genome, synthetic arrays of 601 DNA do not result in a chromatin fragment with nucleosome position centered on the 601 core, nor phased accordingly to the repeat length of 197 or 237 bp. Nucleosomes are not randomly organized on these synthetic arrays (Figure 4A,B and C), but as general rule they tend to be positioned away from the 601 sequence and overlap more on the linker sequence. Given the average NRLs measured in the MNase-gel assay (171 bp vs 163 bp measured genome wide) and the peaked profiles observed in the MNase-seq analysis, our results suggest an effect of the repeat sequence (our results are not compatible with a 163 bp NRL regardless of the repeat sequence) but not a precise positioning as expected from design. There are several possible explanation for the lack of positioning observed on the 601 core in vivo. First, we tried to evaluate a possible role for transcription in overriding nucleosome positioning on the 601 core, in particular for the 197 bp and 237 bp spacing, which represent an unnatural NRL in S. cerevisiae. RNA sequencing of polyA enriched or total RNAs indicate a low level of transcription of RNAs containing the 601 sequence in the three experimental strains. This suggests that transcription does not play a significant role here.

A definitive argument against a role for transcription would however require to study the synthesis and decay rates of these RNAs, as the measurement of RNA steady states alone does not assess well the interplay between these two kinetically coupled processes [START_REF] Wichtowska | An interplay between transcription, processing, and degradation determines tRNA levels in yeast[END_REF][START_REF] Neymotin | Determination of in vivo RNA kinetics using RATE-seq[END_REF].In addition, further studies could be carried out to identify the polymerase(s) responsible for the low level transcription of the 601 repeats and its/their occupancy level on the arrays. Another possible caveat of our attempt to force unnatural nucleosome positioning in S. cerevisiae is our choice of monomer length. As shown in [START_REF] Wong | An all-atom model of the chromatin fiber containing linker histones reveals a versatile structure tuned by the nucleosomal repeat length[END_REF], the helicoidal nucleosomal arrangement of chromatin fibers reconstituted in vitro can be either right or left handed, for respectively NRL value of 167+10*n or 172+10*n (with n=1,2 or 3). Single-linker measurements proposed quantized linkers of 10n+5 bp, which would suggest a nucleosome spacing of 162 or 172 bp rather than the 167, 197 or 237 like we tested in this work [START_REF] Widom | A relationship between the helical twist of DNA and the ordered positioning of nucleosomes in all eukaryotic cells[END_REF][START_REF] Wang | Preferentially quantized linker DNA lengths in Saccharomyces cerevisiae[END_REF][START_REF] Brogaard | A map of nucleosome positions in yeast at base-pair resolution[END_REF]. Based on modeling [START_REF] Wong | An all-atom model of the chromatin fiber containing linker histones reveals a versatile structure tuned by the nucleosomal repeat length[END_REF] and structural work [START_REF] Routh | Nucleosome repeat length and linker histone stoichiometry determine chromatin fiber structure[END_REF][START_REF] Robinson | EM measurements define the dimensions of the "30-nm" chromatin fiber: evidence for a compact, interdigitated structure[END_REF], we chose 167, 197 bp and 237 bp as repeat length to test in our assay since they were proven to allow regular right handed chromatin folding in vitro. We cannot exclude however that in yeast, chromatin assembly factors would only favor left handed stacks of nucleosomes, which would therefore be allowed to form regularly on repeats of sizes belonging to the series 162, 172, 182 bp.

Therefore, we cannot rule out that constraints on chromatin folding in vivo in yeast supersedes sequence driven positioning by the 601 sequence. Our system however opens avenues to explore further such caveat.

Our results therefore emphasize the difference between in vitro and in vivo nucleosome positioning rules. The 601 sequence has been a workhorse for chromatin folding studies in vitro, and this current work serves as an additional cautionary tale in assuming the ability of the 601 sequence to impose nucleosome positioning beyond in 7/15 vitro conditions. Wu and Travers already reported that under certain in vitro conditions, the 601 sequence does not position nucleosomes well. These conditions might be in fact more relevant to physiological conditions [START_REF] Wu | Relative affinities of DNA sequences for the histone octamer depend strongly upon both the temperature and octamer concentration[END_REF].

In addition, our results are generally in line with conclusions drawn by Cole et al. when analyzing published studies of nucleosome occupancy on a single 601 sequence (or the closely related 603 sequence) in vivo, in yeast and human [START_REF] Cole | Perfect and imperfect nucleosome positioning in yeast[END_REF]. For example, Perales et al. showed that a single 601 element inserted in a yeast coding sequence was largely depleted of nucleosomes, and that presence of neighboring nucleosomes were influenced by transcriptional history [START_REF] Perales | Histone occupancy in vivo at the 601 nucleosome binding element is determined by transcriptional history[END_REF]. When present episomally in the promoter of a transgene expressed in the mouse liver, strong positioning was only observed early in development when the transgene is active [START_REF] Gracey | An in vitro-identified high-affinity nucleosome-positioning signal is capable of transiently positioning a nucleosome in vivo[END_REF]. Therefore, both these studies hinted at the possibility that the transcriptional history or status of the region in vicinity of the inserted nucleosome positioning sequence could override the intrinsic affinity of the 601 core to a nucleosome.

Our work is rather in favor of a weak affinity of the sequence in vivo alltogether.

Our results are also consistent with our recent deep learning-based mutational screen of the S. cerevisiae genome, which did not point at any DNA motifs that would attract and position nucleosomes in vivo [START_REF] Routhier | Genome-wide prediction of DNA mutation effect on nucleosome positions for yeast synthetic genomics[END_REF].

Secondly, the main driver of nucleosome spacing is not the DNA sequence but the yeast chromatin remodeling factors, which yield a consistent spacing of 163 bp (Figure 3 B). On our synthetic arrays, the observed spacing is slightly larger, ranging from 166 to 183 (Figure 3 D) but is still much smaller than the spacing that would be dictated by the DNA sequence alone for the 197 and 237 bp arrays. Based on these two principles, we can explain that in the case of the 167 bp array, since the periodicity of the sequence is close to the natural periodicity of nucleosome positioning in yeast, most nucleosomes are phased with the sequence and we observe a preferential positioning of the nucleosome centers on the monomer sequence (Figure 5 top). For the 197 bp constructs on the other hand, the spacing imposed by the yeast chromatin remodeling machinery is not in phase with the DNA array and we observe a more uniform distribution of nucleosomal centers, although there is still a slight preference for positioning outside of the 601 core (Figure 5, middle). Finally, in the case of the 237 bp repeat, the impose spacing is such that 3 nucleosomes would occupy approximately 2 DNA repeats, resulting in three fuzzy peaks in the distribution of the nucleosome centers (Figure 5, bottom).

The thermodynamics of the nucleosome positioning process is based on the sum of three different contributions: the energy of formation of a nucleosome on DNA, the DNA sequence affinity for the nucleosome and the long range interaction energy between nucleosomes [START_REF] Riposo | Nucleosome positioning and nucleosome stacking: two faces of the same coin[END_REF][START_REF] Beshnova | Regulation of the nucleosome repeat length in vivo by the DNA sequence, protein concentrations and long-range interactions[END_REF]. It appears from the present study that even for strong in vitro positioning sequences such as the 601, the in vivo energy of formation and long range interactions between nucleosomes are the most important terms. This does not exclude the possibility to design new DNA sequences that would regularly position nucleosomes within a non-physiological spacing, but these sequences need to be optimized in the context of the in vivo energy terms. To conclude, our study emphasizes that the 601 sequence should not be assumed to maintain nucleosome positioning when inserted in genomic loci in vivo, and new approaches will be needed to design in vivo nucleosome positioning sequences for the engineering of designer chromatin.

Materials and Methods

Strains, plasmids, reagents and media

In vivo genomic assembly of 601 DNA repeats was achieved in the yeast strain YPH499 [START_REF] Sikorski | A system of shuttle vectors and yeast host strains designed for efficient manipulation of DNA in Saccharomyces cerevisiae[END_REF]. Strains derived from this study are listed in Supplementary Table S1. Plasmids used for the in vivo expression of spCas9 and the guide RNA targeting the YMR262W gene have been previously described [START_REF] Lancrey | Locus specific engineering of tandem DNA repeats in the genome of Saccharomyces cerevisiae using CRISPR/Cas9 and overlapping oligonucleotides[END_REF] and are listed in Supplementary Table S2. Strains were grown at 30 • C in yeast extract Peptone Adenine Dextrose 2% media (YPAD) or in the appropriate synthetic complete Dextrose 2% media (SCD) media minus relevant amino acids necessary to maintain plasmid borne auxotrophic markers. All media reagents were purchased from Formedium and used as recommended. Oligonucleotides used in this study were synthetized by Eurogentec. Enzymes for nucleic acids modification, including MNase, were purchased from New England Biolabs. Zymolyase 20T was purchased from Amsbio.

Assembly of 601 tandem DNA repeats in the yeast genome

Assembly of 601 DNA repeats on an episomal vector in vivo was achieved using a method adapted from the Transformation Associated Recombination method [START_REF] Kouprina | Cloning of human centromeres by transformation-associated recombination in yeast and generation of functional human artificial chromosomes[END_REF]. hl2 µ plasmids carrying 601 repeats assembled with this technique were used for in vitro chromatin reconstitution and single molecule fingerprinting using magnetic tweezers. The full methods relating the in vitro reconstitution and fingerprinting can be found in the Supplementary Material and Methods. Direct in vivo assembly using CRISPR/Cas9 and overlapping oligonucleotides in the Chromosome XIII of S. cerevisiae strain YPH499 was performed as previously described [START_REF] Lancrey | Locus specific engineering of tandem DNA repeats in the genome of Saccharomyces cerevisiae using CRISPR/Cas9 and overlapping oligonucleotides[END_REF]. Briefly, donor DNA containing the left (YMR/601) and the right (601/YMR) genomic junction were amplified by PCR using primer couples AL-O-14/15 and AL-O-16/17 respectively. These donor DNA results, upon recombinational assembly, in the deletion of the region -129 to 232 bp of the YMR262W gene. All oligonucleotides used for in vivo repeat assembly are listed in Supplementary Table S3. YPH499 was first transformed with the CAS9 expressing plasmid p414-TEF1p-Cas9-CYC1t. This plasmid was a gift from George Church (Addgene plasmid ♯ 43802) [START_REF] Dicarlo | Genome engineering in Saccharomyces cerevisiae using CRISPR-Cas systems[END_REF]. The resulting strain was transformed using the LiAc technique [START_REF] Hill | DMSO-enhanced whole cell yeast transformation[END_REF] with 1 µg of gRNA expressing plasmid targeting YMR262W (pAL31), 100 pmol of each of the four or six appropriate 601-oligonucleotides (all oligonucleotides and plasmids are listed in supplementary Table S2 andS3), and 10 pmol of both YMR/601 and 601/YMR junction PCR. After transformation cells were plated on SCD-His-Ura to select for cells carrying both CAS9 and gRNA expressing plasmids. To test for correct in vivo assembly of 601-tandem repeated arrays, each genomic junction between the assembled repeated array and genomic DNA were analyzed by Sanger sequencing with primers AL-O-02/22 and AL-O-01/23, to amplify respectively the left YMR/601 junction or the right 601/YMR junction (see Supplementary Figure S2). To confirm assembly at the correct locus and to estimate the size of the assembled arrays, recombinant clones were also analyzed by southern-blotting. Genomic DNA from recombinant strains were digested with BamhI and DraI, which cut at each side of the insertion locus. Digested DNA was electrophoresed in 1% agarose and transferred by capillarity onto a nylon membrane (Hybond N+, GE healthcare). Membranes were hybridized in Church buffer at 68 • C with two different 32 P-radiolabeled probes (Church and Gilbert, 1984). The 601 probe was made of the 147 bp long 601-core sequence. The genomic probe was a 1 kb DNA fragment amplified by PCR from genomic DNA using primers AL-O-24/25 (see Table S3). Probes were labeled with the prime-it II random primer labelling kit (Agilent) using α 32P -dCTP. For each 167, 197 and 237 design the clone carrying the longest 601 tandem repeats was named respectively ALY1, ALY2 and ALY3 and selected for further analysis (Figure2). Membranes were scanned using a FLA 9500 GE healthcare Scanner at 200 µm resolution.

Chromatin digestion by Micrococcocal Nuclease (MNase)

Each strain was grown to an OD 600 of 0.8 in 250 mL Synthetic Complete Dextrose 2 % media at 30 • C with shaking at 200 rpm. Cultures were treated with a final concentration of 1.85 % formaldehyde for 30 min at 30 • C. Cross-linking was stopped by addition of 105 mM Glycine (final concentration) and collected by centrifugation (6500g, 10 min). Cell pellets were washed and resuspended in 50 mL of 1 M Sorbitol, 10 mM Tris pH 7.5 supplemented with 10 mM β-mercaptoethanol and 15 mg of Zymolyase 20T. Cells were further incubated at 30 • C with 1h shaking at 100 rpm. Spheroplasts were pelleted (6500g, 10 min), and resuspended in 2.4 mL solution containing 1M Sorbitol, 50 mM NaCl, 10 mM Tris pH 7.5, 5 mM MgCl 2 , 1 mM CaCl 2 and 0.75 % Igepal Ca630 freshly supplemented with 1 mM β-mercaptoethanol, 500 µM spermidine and various amounts of MNase (between 6000 and 30000 units), to allow choosing samples with the appropriate digestion pattern for further processing. The spheroplasts/MNase mixture was then incubated at 37 • C for precisely 30 min, allowing breaking of the spheroplasts and digestion of chromatin. The reaction was stopped by adding 600 µL of 1% SDS, 10 mM EDTA. Reversal of crosslink and protein removal was achieved by adding 0.6 mg of Proteinase K (Invitrogen) and overnight incubation at 65 • C. Samples were extracted with phenol/chloroform, and DNA was ethanol precipitated treated with DNase-free RNase.

MNase-gel analysis

The MNase-gel analysis was performed as follows. Mildly digested MNase resistant nucleosomal DNA was separated by electrophoresis in a 1.3 % agarose gel. After migration, the gel was colored with Ethidium Bromide (0.5µg/mL) and imaged in a Gbox bioImager (Syngene UK). The resulting image was analysed using a simple image analysis routine developed in-house in Matlab to calculate the size of digested nucleosomal DNA relative to Molecular Weight standards (1 kb+ DNA (Thermo Fischer Scientific)). Following gel imaging, the DNA was transferred on a nitrocellulose membrane by capillarity using the Southern blot protocol described above.

Membranes were hybridized with a 147 bp α 32P -labelled probed and exposed against a phosphor plate for 24-48 hours. Phosphor plates were scanned using a Typhoon FLA 9500 scanner at 200 µm resolution (GE Healthcare). The image obtained was analyzed with the same image analysis routines applied to gels colored with Ethidium Bromide.

Preparation of DNA libraries for MNase-seq analysis

For the preparation of fragment libraries, 500 ng of gel purified mononucleosomal DNA were repaired using the PreCR Repair Mix Kit (New England Biolabs) with 100 µM dNTPs in a 50 µL reaction, as recommended by the manufacturer. The reaction was incubated for 30 minutes at 37 • C, purified using the QIAquick PCR Purification Kit (Qiagen) and eluted in 80 µL H 2 O. Nucleosomal DNA was 5' phosphorylated by adding 333 µM dNTPs, 50 units of T4 polynucleotide kinase (NEB), 15 units of T4 DNA polymerase (NEB) and 5 units of Klenow DNA polymerase (NEB) to 80 µL of repaired mononucleosomal DNA to a final reaction volume of 120 µL. The reaction was incubated 30 minutes at room temperature, purified using the QIAquick PCR Purification Kit and eluted in 30 µL H2O. To add a 3'-dA to the nucleosomal DNA the 30 µL eluate was completed to 50 µL with 0.2 mM dATP and 15 units Klenow DNA polymerase (3'-5' exo-). The reaction was incubated 30 minutes at 37 • C and inactivated 10 minutes at 65 • C. Reaction was purified using the QIAquick PCR Purification Kit, and eluted in 20 µL. Adapters were then ligated to the nucleosomal DNA by completing the reaction to 30 µL with 10 mM adapters and 1200 units of T4 DNA ligase. The reaction was incubated at 16 • C overnight, followed by inactivation for 20 minutes at 65 • C. Ligated DNA was purified by gel electrophoresis. The adapter-ligated nucleosomal DNA was amplified by PCR prior to sequencing, using Illumina PE1.0 and PE2.0 primers. We used only 8 to 15 cycles to minimize possible bias due to PCR amplification. PCR were as follows: 3 µL of adapter-ligated nucleosomal DNA were used in a 50 µL reaction volume containing 5 µL of each Illumina paired-end PCR primers at 2 µM, 1 µL of 10mM dNTPmix and 2 µL of Phusion polymerase (NEB). PCR products were purified using the QIAquick PCR purification column and eluted with 30 µL Qiagen Elution Buffer. DNA concentration was determined using a Qubit Fluorometer. Libraries were multiplexed and then sequenced on a HiSeq 2000 device.

MNase-seq analysis

For each 601-containing strains four individual libraries were sequenced, yielding 5 to 28 Million reads for each library. For the three 601 designs we constructed a reference genome carrying two full repeats plus 65 bp of a third one. Indeed, the construction of the chimeric genome is required to take into account all possible alignments of the reads. Reads for which one mate overlaps the linker and the beginning of the following 601-core will align on a second repeat in our reference genome. Moreover, extreme cases where the 2 mates of a read overlap two linkers will align on the beginning of a third repeat (65 bp were chosen as the length of our read -1 bp) (see Supplementary Figure S1). After removing of barcodes, paired-end reads of 65 nt each end were mapped against the appropriate 601 (167, 197 or 237) reference genome using Bowtie2 (version 2.3.5.1) [START_REF] Langmead | Fast gapped-read alignment with Bowtie 2[END_REF][START_REF] Langmead | Ultrafast and memory-efficient alignment of short DNA sequences to the human genome[END_REF] allowing at most 2 alignments per read (-k 2) and fragments length ranging from 100 to 200 bp for 167 and 197 strains, and from 120 to 250 bp for 237 strains (-I and -X parameters). By choosing a reference genome with two repeats and 65 bp of an additional 601, a large proportion of the reads of interest should be able to map twice. For this reason we allowed 2 hits per read using Bowtie2. Concordant reads were specifically selected using samtools (version 1.10). Reads presenting multiple hits on the repeat region were filtered to select the most 5 ′ alignment using a python script. Coverage per base-pair was determined in both the whole-genome (WG) and the 601-repeats in each individual library, to calculate the enrichment factor in the repeats (see Supplementary Figure S6). For all the fragments overlapping the 601 area their midpoint positions were localized on the repeats using a python script. Finally, these midpoint coordinates were merged on a unique repeat to represent the averaged nucleosomal dyad density at each position of a single repeat using a custom R script.

RNA libraries preparation

Total RNA was extracted by starting from 25 mL culture of each strain grown to an OD 600 of 0.5 in SCD media at 30 • C with shaking at 200 rpm. Cells pellets were resuspended in 500 µL of Nucleazol solution (Macherey Nagel, 740404.200) followed by 20 minutes agitation in MN Bead Tubes Type C (Macherey Nagel, 740813.50).

Total RNA was then purified using the NucleoSpin® RNA Set for NucleoZOL (Macherey Nagel, 740406). RNA concentration and quality was determined using respectively Qubit™ RNA HS assay kit and Qubit™ RNA IQ Assay kit with a Qubit® fluorometer. The first RNA sequencing was performed using purified poly-A mRNA fragmented, reverse-transcribed into 300 bp cDNA before single-end Illumina adaptor ligation. Libraries were sequenced on a HiSeq 4000 (1x50bp) SR device by the Eurofins sequencing platform (INVIEW Transcriptome Explore). Another round of sequencing was performed using total RNAs without poly-A enrichment. After agmentation, RNAs were reverse-transcribed into 300 bp cDNA before single-end Illumina adaptor ligation.

cDNA Libraries were then sequenced on a NovaSeq 6000 S4 PE150 XP (2x150bp) by the Eurofins sequencing platform (NGSelect RNA on Illumina).

RNA-seq analysis

We sequenced RNA from the three 601-strain and the wild-type stain during exponential growth phase in YPAD media. Reads were pseudoaligned to the reference protein-coding cDNA collection of the S. cerevisiae strain S288C via Kallisto [START_REF] Bray | Near-optimal probabilistic RNA-seq quantification[END_REF] (version 0.46.2) in the single-end mode. The YMR262W cDNA sequence of the fasta files used to create index was replaced by the sequence of four 601 repeats of the appropriate design when aligning the three 601 strains. For each of the four biological replicates cpm (counts-per-million) values for every gene were calculated with the edgeR (version 3.30.3) package of Bioconductor [START_REF] Robinson | edgeR: A Bioconductor package for differential expression analysis of digital gene expression data[END_REF]. cpm values were normalized using edgeR to correct for sample-specific variation typically introduced by differences in library size. To verify that 601 repeats do not disturb the global transcriptome we calculated the log2(cpm) value of every gene (except the ones for which all samples displayed less than 1 cpm ) to determine expression fold change between the wild-type strain and the three 601 strains (see Supplementary Table S4 (polyA-RNA) and Supplementary Table S5 (total RNAs)). To estimate the 601 transcription rate we deduced in each 601-strain from the cpm value in the repeats the rank of expression of the 601 region over all genes analyzed in each library (Table 1).

Figure 1 .

 1 Figure 1. In vivo assembly of '601' tandem DNA repeats. (A) Schematic representation of the CRISPR/Cas9-assisted genomic integration of overlapping oligonucleotides. The method is represented for integration of 601-167 repeats in the YMR262W gene of S. cerevisiae. (B) Design of the 601-167 repeats: four 50 nt oligonucleotides f1-r2-f3-r4 overlapping on 20 nt were used to construct the 601-167 array. The 147 bp core sequence (grey) corresponds to the DNA portion theoretically wrapped around one nucleosome, followed by a 20 bp linker (blue). The f4 oligo overlaps with f1 so that several monomers can assemble into a repeated array. (C) Design of the 601-197 and 601-237 arrays, using six overlapping oligonucleotides.

Figure 2 .Figure 3 .

 23 Figure 2. Overlapping oligonucleotides assemble to form long repeated arrays at the YMR262W locus. (A) Diagram of the assembly region before and after CAS9-assisted recombinational assembly. Position of the genomic probe is represented by a dark line. Localization of the genomic-601 junction primers are also indicated. (B) Southern Blot analysis of yeast recombinant strains. Genomic DNA from strains containing 601 repeats were digested with BamhI and DraI. Samples were electrophoresed, blotted and hybridized with a 601 probe containing a 601-147 nt repeat unit. The first two lanes (WT) correspond respectively to the wild-type and CAS9 expressing strains. Five clones are shown for respectively the 167, the 197 and the 237 design; the strains containing arrays largers than 15 kb (marked with color arrows) were retained for further manipulations. (C) The membrane shown in B was stripped and rehybridized using the genomic probe shown in A. The star marks a non specific band present on both Southern blots and present in the WT controls

Figure 4 .

 4 Figure 4. Nucleosomal fragment midpoints are not localized at the middle of 601 cores in the repeats in vivo. Midpoints Kernel density curves of each individual library are displayed on the same graphic for each 601-strain: the 601-167 strain (A), the 601-197 strain (B) and the 601-237 strain (C). The orange vertical line marks the 601-core middle and the two other colored vertical lines delimit the linker portion of the corresponding 601 design. The expected distribution is schematized on top of each diagram. The fragment length distribution histograms of the four corresponding libraries are displayed for the 601-167 strain (D), the 601-197 strain (E) and the 601-237 strain (F).

Figure 5 .

 5 Figure 5. A model of nucleosome positioning on each of the 601 arrays in vivo. Proposition of nucleosome organization over 5-8 repeats (left panels based on both our NRL measures on gel and on the experimental distributions of nucleosomal fragment midpoints on each 601 repeats (right panels).

Table 1 .

 1 601 repeats inserted at the YMR262W locus are weakly transcribed.

	RNA sample 601 array type cpm	rank / total genes rank 601 monomer/ total genes
				which cpm is > 0 which cpm is > 0
	polyA	167	4.28	5325 / 6068	5915 / 6068
	polyA	197	1.94	5522 / 6031	5928 / 6031
	polyA	237	12.7	4789 / 6014	5711 / 6014
	total	167	19.56 1316 / 6612	5558 / 6612
	total	197	2.1	4830 / 6612	5555 / 6612
	total	237	38	637 / 6612	5534 / 6612

Acknowledgments

The authors thank the members of the Genome Structure and Instability unit for discussions and Dr Romain Koszul for providing assistance with sequencing. The authors also thank the four anonymous reviewers for their insightful comments on the original manuscript. This work was supported by a grant from the french national research agency (MuSeq, ANR-15-CE12-0015), Sorbonne Université (CONVERGENCE -CVG1110) and by core funding from CNRS, INSERM and MNHN to the Structure and Genome instability unit. French Museum of National History to the Genome Structure and Instability unit. AL was supported by a doctoral fellowship from the National Museum of Natural History. Work in the group of VC is part of "Institut Pierre-Gilles de Gennes" ("Investissements d'Avenir" program ANR-10-IDEX-0001-02 PSL and ANR-10-LABX-31) and the Qlife Institute of Convergence (PSL Univesité).

Data availability

Sequencing data are available here: https://www.ncbi.nlm.nih.gov/sra/PRJNA700072

Appendix A. Supplementary Data

Supplementary Methods and Figures are available online at XXX.

Conflict of interest statement.

The authors declare no conflict of interest.


M Chambe

E Kijak

Z Miklos

O Le Meur

R Cozot

K Bouatouch

HDR-LFNet: Inverse Tone Mapping using Fusion Network

Keywords: CCS Concepts, Computing methodologies → Computational photography, Image processing, Supervised learning

To capture the real-world luminance values, High Dynamic Range (HDR) image processing has been developed. HDR images have a richer content than the widely-used Standard Dynamic Range (SDR) images, and are used in a number of situations, e.g. in film industry. As HDR displays are more and more commercially available, we need to be able to process HDR images as well as SDR ones (for example, devising denoising algorithms, inpaiting or anti-aliasing). The most powerful methods to process images are now deep neural networks. However, the training of such networks calls for a lot of images, and HDR images datasets are relatively small. One way to generate HDR images is inverse tone mapping operators (iTMOs). They are algorithms that expand the dynamic range of SDR images. In this paper, we propose HDR-LFNet, a novel iTMO, and its HDR training dataset. Our method merges several existing handcrafted iTMOs, combined in a supervised neural network to produce an HDR output. Our lightweight network requires less training images than state-of-the-art methods, and has faster training phase. Besides, the quality of the generated images is similar to the state-of-the-art. We present the architecture as well as the subjective and experimental evaluations of our method.

Introduction

High Dynamic Range (HDR) images consist of BitMap arrays which contain, for each pixel, the raw luminance captured by the sensor at this point. These values are, in theory, unbounded and not quantized. This is a much richer content than Standard Dynamic Range (SDR) images, which correspond to the common widespread images. HDR-compatible hardware, such as monitors or cameras, are more and more available to the general public. This creates a need for image processing algorithms adapted to HDR content, making the HDR imaging a trending research domain.

HDR photographs are usually taken using the exposure fusion technique. We take several photographs of the same scene with different exposure times. The longer the exposure time, the more light is coming through the camera, and the more information we can obtain in dark parts. On the other hand, short exposure time pictures provide information in the brightest areas of the scene. This allows to gather information in every area of the scene. The different photographs are then merged together to yield a single HDR image. This method is easily achievable using the bracketing function of the camera, that is why it is so popular.

As most of the image datasets available are composed of SDR images, it is useful to devise algorithms to recover lost information in SDR images. From a single SDR image, we must use approxi-mations if we want to extend the dynamic range. Algorithms that extend dynamic range of SDR images are called inverse tone mapping operators (iTMO). Contrary to the classic method of exposure fusion, iTMOs only take as input a single SDR image and yield an HDR image. The result is an approximation of the truth, as SDR images do not contain as much information as HDR ones. However, iTMOs manage to avoid some artifacts caused by the exposure fusion, such as ghosting (due to movement in the scene), motion blur or Moiré effect (usually presents in high exposure pictures). While at first, iTMOs were based on content-based assumptions and were using photographic rules to extend the dynamic range, neural networks are now used to this end.

Inspired by other computer vision domains (such as saliency [START_REF] Mai | Saliency aggregation: A data-driven approach[END_REF] or denoising [START_REF] Kervrann | PEWA: Patch-based exponentially weighted aggregation for image denoising[END_REF]), we devise the HDR Light Fusion Network (HDR-LFNet), a new iTMO aggregating several existing iTMOs that are not based on learning algorithms. Hopefully, the fusion of all methods performs better than each input method individually. Our approach uses a supervised neural network with several orders of magnitude fewer parameters to learn compared to existing networks, which is achieved thanks to some pre-processing. This pre-processing lowers as well the number of training images needed. This is a real improvement, as our network needs HDR training images and HDR images are not as easy to gather as SDR images.

However, as we need fewer images to train, each image is increasingly important and needs to carry a lot of information to effectively train the network. We consider that high resolution images have a higher probability to contain the qualities that we need (light sources, dark areas, smooth gradient and high frequency areas). The HDR datasets that already exist usually contain a few hundreds images at most, with size around (1920 × 1080). To train our network, we then collected high resolution HDR images, and we compiled them in a new dataset.

We tested our method against state-of-the-art using four metrics on HDR images: HDR-VDP2, Harmonic HDR-IQA, PU-PSNR and PU-SSIM. Our method yields results similar to the state-of-theart, but runs with fewer parameters. We also conducted a user experiment to compare HDR-LFNet to state-of-the-art methods. Results show that HDR-LFNet is preferred to others in the user study. Along with the short training time required by our network, this user study shows that our method is usable and effective in a wider range of applications.

Our contributions are:

1. devising a novel inverse tone mapping light architecture that merges several existing iTMOs to get a more powerful one; 2. proposing a new dataset of high definition HDR images composed of 496 pairs of middle exposure SDR and the corresponding HDR; 3. evaluating our work and other state-of-the-art methods using objective metrics and a user experiment.

The rest of this article is divided as follows. Section 2 presents the recent work about HDR and inverse tone mapping. We explain our approach in section 3 and then present the evaluation of our method in section 4. Finally, section 5 concludes the paper.

Related work

Hardware-based HDR generation

Intuitively, the best way to create HDR content is to devise specific devices. One method is to use other kinds of sensors along with classic CCD sensors to get more information. For example, Han et al. [HZD*20] propose a new method to fuse an SDR image with an intensity map provided by an event camera (also called neuromorphic camera) using deep learning.

Instead of adding new information (such as intensity maps) of the same scene, another method is to modify an existing camera to better reconstruct the HDR afterwards. Some articles [MIPW20; STF*20] combine the design of a new lens -which point spread function is thus known and optimized for HDR content -and a neural network to reconstruct HDR content from the image taken by this modified lens.

Software-based HDR generation

Multi-exposure fusion. Using only a classic camera, the most popular way to create HDR content is by fusing multiple images of the same scene with different exposure times. Several fusion algorithms exist [START_REF] Mertens | Exposure Fusion: A Simple and Practical Alternative to High Dynamic Range Photography[END_REF][START_REF] Debevec | Recovering high dynamic range radiance maps from photographs[END_REF]. The fusion methods are known to yield HDR images of very good quality, but the process of taking the image is more difficult than other methods: both the camera and the scene must stand still for several seconds, the time to take several images with different exposure times. If the camera, or objects in the scene, move between the different shots, some ghosting artifacts will appear on the merged HDR image.

Single-exposure non-deep fusion. One way to reduce ghosting artifacts from HDR images is to use only one image and expand its dynamic range using iTMO. By doing so, HDR generation becomes an ill-posed problem as we do not have all the information needed to reconstruct faithfully the HDR image. In this situation we need to make some assumptions about the images we have. For example, we can assume that only the high luminance areas are lost. In that case, any pixel in the low or medium ranges would not be modified by the iTMO. Many algorithms were devised to expand the dynamic range from a single image. One idea is to use a nonlinear function to modify the luminance of the image differently based on the pixel value of the SDR version. We can cite several methods, such as Akyuz [AFR*07], Kovaleski & Oliveira [START_REF] Kovaleski | Highquality reverse tone mapping for a wide range of exposures[END_REF], or Landis [START_REF] Landis | Production-ready global illumination[END_REF]. All of them differ from one another by the nonlinear function they use to improve the dynamic range. The modification of the luminance value is solely based on the SDR pixel value, and so the context is not taken into account. Later works try to consider the semantics of each pixel to better improve the output quality using neural networks.

Deep neural networks for single image HDR reconstruction.

The latest and more powerful algorithms for single image HDR reconstruction are based on deep-learning algorithms, and more specifically supervised learning methods. These algorithms allow for tuning several millions of parameters using big datasets of images as ground truth. The first widely recognized deep CNN for single image HDR reconstruction is HDRCNN [START_REF] Eilertsen | HDR image reconstruction from a single exposure using deep CNNs[END_REF]. It uses a really deep network of about 30 million parameters to enhance the brightest part of the SDR picture in input. The output of the network is then combined with an augmented version of the SDR (obtained with an average inverse camera response function estimated over a dataset) using a mask to use the network output in bright zones and the augmented SDR in the other areas. The network has been pre-trained on simulated HDR data (using a simple iTMO on a large image dataset) and fine-tuned with true HDR images. Other papers [SRK20; LLC*20] improve on HDRCNN by using inpainting-like tasks in the network -either as pre-trained weights or as another module. The idea is that such a network must reconstruct the over-exposed areas of the images, as this information is lost in the SDR image. Usually, HDR imaging focuses on high lights, and therefore the proposed methods work on improving the over-exposed areas. However, the same tricks can be used to improve the quality of under-exposed areas as well. The network of Marnerides et al. [START_REF] Marnerides | Deep HDR Hallucination for Inverse Tone Mapping[END_REF] uses generative adversarial networks and inpainting to improve the dynamic range in both lowly and highly lit parts of the image.

In our approach, we use preprocessing to reduce significantly the number of parameters of our network, and postprocessing to correct submitted to COMPUTER GRAPHICS Forum (3/2022).

the network output as best as possible. This allows us to improve lowly and highly lit parts of images.

Our fusion network

Our goal is to expand the dynamic range of images. To achieve this goal, we use supervised learning algorithms to train a neural network. In this section we present the network architecture and the training process along with the training dataset. As we decided to use expanded version of images through iTMOs as input of our algorithm, we also present how we choose those operators. Our method is represented in Figure 1.

Overview of 3D convolutions

In this section we present some general characteristics of 3D convolutions, that we extensively use in our network.

An image can be represented as a tensor of size (H,W,C) with H the height, W the width and C the number of channels. A channel is a scalar array of size (H,W ) containing information about a specific characteristic. For example, in traditional coloured images, C = 3 (each channel contain information about a specific colour component for example in RGB values), or if we only use gray level images, C = 1. Convolutional neural networks work by learning the weights of convolution filters. At each level (that we call layer) in the encoder section, the number of channel increases, to learn more and more structured information. It is not rare to see tensors with size (H,W, 64) in the middle of a CNN. Tensors which are not the input or the output tensors of a neural network are called feature maps.

A classic 2D convolution in a CNN layer is characterized by its kernel size, denoted by (kx, ky). A 2D convolution between a layer with C channels and another one with C ′ channels will have kx × ky × C × C ′ parameters to learn. All scalar values in a (kx, ky,C) voxel of the input tensor are multiplied term by term with the kernel weights, and then added together to yield a single scalar value. This process is repeated C ′ times with C ′ different sets of weights, to get C ′ values. Starting with a tensor of size (H,W,C), this convolution yields a tensor of size (Hkx + 1,Wky + 1,C ′ ).

In our case, we use 3D convolutions, which work on volumes of images of size (H,W, D,C). The convolution kernels also have one more dimension (kx, ky, kz), but they are similar to 2D convolution: they have kx × ky × kz × C × C ′ parameters to learn, and starting with a tensor of size (H,W, D,C), this convolution yields a tensor of size (Hkx + 1,Wky + 1, Dkz + 1,C ′ ). Besides, as 2D convolutions, they consider all channels when computing the convolution sum. The advantage of 3D convolutions in our case is to specify the depth dimension: this allows for more control over the training of the network, and its behaviour.

2D convolutions and 3D convolutions are related: if we consider tensors with size (H,W,C) as (H,W, 1,C), 2D convolutions (kx, ky) are the same as 3D convolutions (kx, ky, 1). On the other hand, we can view tensors (H,W, D,C) as (H,W, DC) and convolutions (kx, ky, D) are the same as (kx, ky), but convolutions (kx, ky, kz) with kz < D are not translatable to 2D convolutions, so 3D convolutions are strictly more expressive than 2D ones.

Architecture

As represented on Figure 1, our method uses a neural network at its core. The architecture of our network is represented on Figure 2. We adopt an encoder-decoder shape to reduce the size of the images during the forward pass, and thus reducing the time and memory needed for training. Many different characteristics of our network are explained in this section.

Our network only processes the luminance of images -the L * component in the L * u * v * colour space -, and thus the input has one channel. The L * channel is linear, so we need to apply a gamma correction to the network output, as well as adding back colour to yield a final HDR output. These postprocessing operations are presented in Section 3.4. The activation function is based on ReLU. As HDR images usually do not contain values of 0, we define a new activation function called Nonzero-Relu as

ReLUa(x) = a if x ≤ a x else (1)
with a value of a = 10 -12 .

While designing our architecture, we follow some common guidelines to avoid well known problems. To avoid artifacts usually caused by deconvolutions, we instead upsample the feature maps, and then apply a classic convolution in the decoder part. Besides, along with maxpooling to reduce the dimension of our network, we use dropout layers to stabilize the training and dodge local minima of the loss function.

Inspired by other neural networks [LLC*20; KSL*16], we decided to lighten our network -in order to improve its performances -by using an architecture specific to our problem. For this purpose we have provided the network with two new characteristics: (1) the input of our network are images expanded with existing iTMOs, (2) 3D convolution layers are used to force the network to learn the added value of each pair of expanded images. These two characteristics are detailed in this section.

By using already inverse tone-mapped images as input of the network, this latter must learn an easier transformation from HDR to HDR rather than from SDR to HDR. These images are concatenated on the depth dimension of the tensor. We manage to drastically reduce the number of parameters of our network. By using three expanded versions of the same image (obtained with three different iTMOs) as input, we are able to reduce the number of parameters to approximately 2 × 10 5 , against the 10 6 to 10 8 parameters of state-of-the-art networks. The choice of iTMOs we use as input of our network is discussed in Section 3.6. As the number of trainable parameters is quite low compared to other networks, it should be harder to train using small crops of images as other methods tend to do. This assumption is verified in Section 4.2 by training our architecture with the HDR-Real dataset.

As we have several versions of the same image with only one channel, we can induce the network to learn how each iTMO interacts with the others. To this end, we use 3D convolutions instead of the classic 2D ones to focus the training.

To ensure that each pair of iTMO is considered, we need to input redundant information in our network. Our input tensor is (1,1,1)

16 (3,3,1) 16 16
(1,1,1) the concatenation (I 1 , I 2 , I 3 , I 1 ) in the depth dimension with I i the expanded image generated by the i-th iTMO algorithm. This allows 3D-convolution with depth kz = 2 to process all pairs (I 1 , I 2 ); (I 1 , I 3 ) and (I 2 , I 3 ) as explained in Section 3.1. This convolution is done at the heart of the network (represented by the red arrow on Figure 2). After this convolution, we just need to upsample the feature maps to get back the original resolution. The final convolutions are done using maximum depth 3D convolutions to simulate 2D convolutions, as the depth dimension is not useful anymore. We compare this method to the classic 2D Convolution in the ablation study (Section 4.3). Therefore, the input tensor of the network is of size (Height, Width, Depth = 4, Channels = 1). These considerations determine the depths of all feature maps of the networks, and therefore the depth values kz of all 3D convolutions. In the follow-ing, the depth values of 3D convolutions which are irrelevant (because they are already fixed by our previous construction choices) are denoted by a question mark ?.

1 (1,1,1) 1 1 (1,1,1)
Finally, we know that using convolution filters usually averages neighbour pixels, and therefore degrades the edges. First, to keep high frequencies as much as possible, we use skip connections. However, the feature maps before and after the fusion convolution have different depths, so we average the first feature maps over the depth dimension before merging them with the second ones. The aggregation of the feature maps is done by concatenating both feature maps, and then running a convolution with a kernel of (1, 1, ?). Moreover, we modify the kernel sizes of the convolutions of the decoder part in a coarse-to-fine manner. Indeed, we decrease the kernel size from (5, 5, ?) to (1, 1, ?). This allows the final convolusubmitted to COMPUTER GRAPHICS Forum (3/2022).

tion to be a (1, 1, ?) convolution, which better preserves the edges in the images. The impact of the kernel size decrease in the decoder part is discussed in the ablation study (Section 4.3).

Loss function

Our loss function is designed for comparing HDR images. We denote by L(I HDR ) the luminance of the ground truth HDR image, and by L the output of the network. The loss function used for training consists of four parts: (i) a mean absolute error (MAE) Yc = L 1 (L(I HDR ), L) for the actual values, (ii) a gradient-based error (gMAE) Yg = L 1 (g(L(I HDR )), g( L)) -with g the gradient computation using Scharr filters -to emphasize the shapes, (iii) a perceptual loss Yp to be more accurate on areas that are sensitive, and (iv) a dynamic range error Y d .

The perceptual loss is based on VGG16 [START_REF] Simonyan | Very Deep Convolutional Networks for Large-Scale Image Recognition[END_REF]. The idea is to compute visible errors at different scales, and to that end we use activation maps from an already trained VGG network. We compute the mean absolute error between deep features of the target and the output images at the first four layers. The computation is done using a L 1 difference.

Finally, we add a dynamic range error Y d :

Y d = D(L(I HDR )) -D( L) with D(X) = max(log X)-min(log X)
This dynamic range error is to ensure the network produces an output with a dynamic range similar to that of the ground truth. The actual loss function is then a combination of those four components:

Y = αYc + βYg + δYp + εY d (2)
Using the validation set, we found that the values α = 1, β = 0.3, δ = 0.15 and ε = 1 work best.

Post-processing

As our network only processes the luminance of images, we need some post-processing to at least add colour to the output image. Besides, as we input linear luminance images, we add a gamma correction to the output of the network to better match the ground truth image. We explain this process in this section.

In the following, we denote by I the coloured SDR input (and L(I) its luminance); L the output luminance of the network; and Î the HDR output recoloured with our method. To recolor our HDR images, we use the luminance preserving formula proposed by Mantiuk et al. [START_REF] Mantiuk | Color correction for tone mapping[END_REF]:

Î(γ, s) = Lexp( L, γ) I L(I) + 10 -5 -1 × s + 1 (3)
with γ the gamma factor, s ∈ [0; 1] the saturation factor and Lexp( L, γ) the expected HDR luminance. Because this formula was designed to preserve luminance, we ensure that L( Î(γ, s)) = Lexp( L, γ). We add the value 10 -5 to avoid problems with luminances of zero. We correct the output luminance computed by the network using a gamma transformation Lexp( L, γ) = L γ . The work of Mantiuk et al. also contains a method for automatic colour correction, however, their studies show that this method is not applicable to HDR images.

The saturation factor s and the gamma factor γ can be modified to yield different results. To get the maximum of correlation between the ground truth and our modified output, we choose both factors by minimizing the mean square error between the images of our training dataset and the computed image from our network. As human beings are more sensitive to order of magnitude of luminance rather than absolute values, we compute those difference in the log-domain. Let T be the set of training images. This amounts to:

γ * = arg min γ ∑ I∈T log 10 (L(I HDR )) -γ log 10 ( L) 2 2 (4) s * = arg min s ∑ I∈T I HDR -Î(γ * , s) 2 2
(5) After optimizing these formulas, we consider the parameters γ * = 2.659 and s * = 1.

Training dataset

Because we use supervised learning methods, we need an image dataset with HDR ground truth and SDR input images to train our network. We present our new training dataset in this section.

The training dataset is an essential component of a neural network. It is mandatory that we carefully select the right images with regards to the architecture and to our needs. The network we devise includes fewer parameters than the state-of-the-art networks. To train such a network, we need a sufficient number of imageswhich is less than other state-of-the-art networks -, but each image must contain as much information as possible. To do so, we need very high resolution images. These images may be found in currently available datasets, but they are not easily recoverable (see Table 1 for a comparison of the different existing HDR datasets). Therefore, we collect a new HDR image dataset. This dataset is mandatory to train our network, but is an addition to other datasets: the same data augmentation techniques can be applied to yield several thousands of smaller images. As such, our dataset can be used for the same purposes as other datasets.

We have taken photographs with a Sony Alpha 7 III camera. We submitted to COMPUTER GRAPHICS Forum (3/2022). use the bracketting mode to take 3 exposures at -3, 0 and +3 exposure values. The ground truth HDR image is obtained through exposure fusion using Photoshop algorithm, as it provides images with less artifacts. For the SDR input, we choose the middle-exposed photograph as it contains information balanced between dark areas and bright areas. Besides, colours are usually more saturated in low light environments, and less saturated in high light environments. The middle-exposed shot provides the best colour quality among the three exposure photographs. Each element of our dataset is then composed of the fused HDR image and the middle-exposed image as the SDR image input. Some examples of images are shown on Figure 3. We have managed to take 496 HDR photographs. We have done no photometric calibration, so all the images are provided in relative luminance values. We then have to normalize all images to use them as training images: we divide all image RGB values by the maximal RGB value in the dataset. As all images were taken with the same camera, this ensures that every HDR image has values in [0; 1] while maintaining homogeneity.

For training our model, we split this dataset in three parts: 80 images for testing the model; 56 images for validation; and the remaining 360 images for training. The 360 training images are then flipped horizontally and vertically to yield an effective training database of 1,440 images. This dataset is available online † .

For testing purposes and comparison with other methods, we use the HDR-Real test dataset proposed by Liu et al. [LLC*20]. It contains more than 8,000 pairs of SDR/HDR images and is widely used in the state-of-the-art as a test set thanks to its number of images. These images were obtained from 480 original HDR images using augmentation techniques, namely cropping (with a crop window of 512 × 512) and varying exposure times and CRFs to create SDR from HDR.

Choice of input iTMO

Method

As we want to use already inverse tone-mapped images as input of our network, we now need to select the iTMOs that we can use. We start from a set of inverse tone mapping operators: the five operators implemented in the Matlab HDR toolbox [START_REF] Banterle | Advanced High Dynamic Range Imaging (2nd Edition)[END_REF] and the style-aware tone expansion [START_REF] Bist | Style Aware Tone Expansion for HDR Displays[END_REF]. Those iTMOs are not based on learning algorithms, but rather handcrafted using photographic rules and common assumptions. Setting up the input tensor † ftp://ftp.irisa.fr/local/percept/public/hdrlfnet/ so that each pair of images is processed by the 3D convolution is feasible with three operators (as shown in Section 3.2). Among the six available iTMOs, we want to select three operators that are quite different, such that we have maximum performances with minimal network input size. The selection method is presented in this section.

First, we suppose that all of our operators are of similar output quality (with different strengths and weaknesses), and we use quality metrics to differentiate the operators on a chosen set of N images. We use our training dataset to do this, so we have N = 360. We process each inverse tone-mapped images of our set with six different metrics: HDR-VDP2, FSIM, MCS5, SI, PU-PSNR, PU-SSIM. These metrics are the ones used by Harmonic HDR-IQA [START_REF] Rousselot | Quality metric aggregation for HDR/WCG images[END_REF], and PU-PSNR and PU-SSIM [START_REF] Aydin | Extending Quality Metrics to Full Dynamic Range Images[END_REF]. This yields N × 6 points in a 6D space. Then, we project those points in a 2D space using the t-SNE visualization algorithm, to make the analysis easier. From this, we compute the coverage of the space of each set of 3 iTMOs among the 6. To do so, we compute the convex hull which encloses all points from 3 of the iTMOs, and we finally choose the 3 iTMOs which have a convex hull of maximum area. The t-SNE plot, along with the convex hull with largest area, is represented in Figure 4. The three chosen iTMOs are Akyuz [AFR*07], Landis [START_REF] Landis | Production-ready global illumination[END_REF] and Kovaleski and Oliveira [START_REF] Kovaleski | Highquality reverse tone mapping for a wide range of exposures[END_REF].

Discussion

All of these operators have different strengths and weaknesses. Akyuz operator is a simple algorithm that sets the maximum luminance value to a constant. While it usually burns the high luminance areas, low-and middle-exposed areas of the HDR output are quite faithful to the SDR version.

Landis operator uses a power function to improve the luminance values of pixels above a certain threshold. This yields high-exposed pixels with appropriate value, but may introduce artifacts: as the SDR image is quantized, some blocks or bands may appear on smooth gradient areas.

Finally, the operator from Kovaleski & Oliveira uses joint bilateral filters to smooth out the areas to expand. This method reduces submitted to COMPUTER GRAPHICS Forum (3/2022). the pixel values, but produces HDR images with less artifacts than Landis.

Note that the t-SNE algorithm which projects high dimension data points in smaller spaces is not very stable: small variations in the input data could modify the projection, and thus the chosen operators. We can however see on the Figure 4 that several iTMOs are very close to each other. As the areas of the bounding boxes are not very different to each others, this choice should not have a huge impact on the performances of our model.

Results

Implementation details

The network is written using the PyTorch framework and is available online ‡ . Using the dataset presented in Section 3.5, we train the network for 15 epochs, while reducing the learning rate each time the validation error increases. The number of epoch is quite low compared to other networks due to the small size of the network. Due to memory restrictions, we use a batchsize of 1.

Besides, each epoch runs for about 40 minutes, for a total training time of approximately 10 hours. This is a much faster training than state-of-the-art training, which ranges from a few days to a full week.

State-of-the-art comparison

In this section, we compare our method to other state-of-the art ones using objective metrics. The iTMOs we consider are HDR-CNN [START_REF] Eilertsen | HDR image reconstruction from a single exposure using deep CNNs[END_REF], ExpandNet [START_REF] Marnerides | Expandnet: A deep convolutional neural network for high dynamic range expansion from low dynamic range content[END_REF], the Single Image Network [LLC*20] and HDRUNet [CLZ*21]. We show some examples of images obtained with our method and with existing models on Figures 5 and6.

HDRCNN and the Single Image Network are presented in Section 2.2. We remind that HDRCNN contain about 30 × 10 6 trainable parameters, and the Single Image Network improves on HDR-CNN by using inpainting-like tasks during the training. It contains about 2 × 10 6 parameters. HDRUNet contains 1.6 × 10 6 parameters. The main idea behind HDRUNet is to split the network into three modules: a base network that performs most of the work, a condition network that computes spatially-variant transformations used to modify the deep features of the base network, and a weighting network that detects over-exposed areas to improve the reconstruction in those areas. All these networks are trained using their novel tanh L1 loss function. ExpandNet is a much lighter network with around 5 × 10 5 parameters. It also proposes a network with different modules, with each module working on a different scale: a global branch, a local branch and a dilation branch for mid-scale features. Each of those branches adds new information, which allows for a more faithful HDR reconstruction.

All of the networks are fully convolutional neural networks, meaning that the input can theoretically be of any size. However, for HDRCNN, due to how the deconvolutions are used, the input ‡ ftp://ftp.irisa.fr/local/percept/public/hdrlfnet/ must have its height and width multiple of 32. This means that some of the images are either cropped or resized to fit this requirement.

The metric used in the state of the art for comparing models is HDR-VDP2 [START_REF] Mantiuk | HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions[END_REF]. It is a metric with reference, and it takes as supplementary argument the angular resolution (measured in pixel per degree (ppd)). The angular resolution depends on the distance between the observer and the screen, and the resolution of the screen. This metric allows for a faithful simulation of the viewing experience on a specific screen. We represent on Figure 7 the impact of the angular resolution on the score. We notice that, although the actual scores change, the ranking of the model do not. For the rest of the evaluation, we use an angular resolution of 30 ppd.

As HDR-VDP2 only works on luminances, we use Harmonic HDR-IQA [START_REF] Rousselot | Quality metric aggregation for HDR/WCG images[END_REF], which is sensitive to colours, as well as PU-PSNR and PU-SSIM [START_REF] Aydin | Extending Quality Metrics to Full Dynamic Range Images[END_REF].

For this experiment, we test our network using the HDR-Real test dataset proposed by Liu et al. [LLC*20]. We train three versions of our network: (i) using the HDR-Real train, (ii) using our training dataset, and (iii) using our training dataset fine-tuned on HDR-Real train dataset. For fairness of comparison, we train these three versions for the same amount of time (10 hours). HDRCNN, HDRUNet and the Single Image Network are trained on the HDR-Real train set, while ExpandNet is used with the weights provided by the authors. We present the results of our evaluation in Table 2. We find out that, although our method do not perform the best, we manage to get second best on most of the metrics, except on Harmonic IQA. As Harmonic IQA assess the differences in colour between HDR images, this shows that our method does not reproduce colours as well as the other methods. However, the user study presented in Section 4.4 reveals that our method is preferred by observers. These results mean that although we are less faithful to the colours of the original image, our methods produces a more appealing picture than the state-of-the-art methods.

Note that our method performs best on HDR-Real test set when it is trained on our dataset, and not on HDR-Real train set. This comes from the nature of the datasets and our architecture: we managed to drastically reduce the number of trainable parameters. As explained in Section 3.5, the low number of parameters calls for fewer train images, but with richer content. HDR-Real images being only 512 × 512, our dataset is better suited to train our network than HDR-Real. We present on Figure 8 the mean HDR-VDP2 score as a function of the number of trainable parameters for our network, and networks of the state of the art. From this point of view, our model actually performs better than the state of the art.

Ablation study

In this section, we present the study conducted to assess the quality of the different components: the composed loss function and the usage of 3D convolutions, skip connections, and convolution kernel with varying sizes.

Each subsequent section presents a modified variant of our network. It is trained on our dataset and tested on HDR-Real. The average HDR-VDP2 score is presented on Table 3 for all variants, as well as for our proposed method (called final model in this section).

Composed loss function

Our loss function is composed of four different components. We train the same architecture with loss functions composed of only some of the original components detailed in Section 3.3. The mean absolute error (MAE) keeps the overall structure better among the four components, so we train networks with MAE and gradient loss (gMAE); MAE and dynamic range loss; and MAE and perceptual VGG loss. We also train a network with only MAE to compute the added value of each component. We use the same weighting as given in Equation (2) when training the different networks: for example, the version MAE + gMAE was trained using the loss function Y = Yc + 0.3Yg. Surprisingly, our method performs better when trained with only MAE rather than MAE + gMAE or MAE + Perceptual loss. This can be explained as follows. In the final model, each weighting coefficient assigned to each component of the loss function has been carefully tuned with regards to the others. When using only two components of the loss function, the weighting coefficients should be different. Therefore, we can assume that the scores given in Table 3 are not optimal, except when training only with MAE (as no tuning is necessary because there is only one component). However, due to the large difference in score between the final model and the different loss function versions, we assume that our full loss submitted to COMPUTER GRAPHICS Forum (3/2022).

Model

Mean HDR-VDP2 (std) ↑ Mean Harmonic IQA (std) ↑ PU-PSNR (std) ↑ PU-SSIM (std) ↑ HDRCNN [START_REF] Eilertsen | HDR image reconstruction from a single exposure using deep CNNs[END_REF] 49 function improves the output quality compared to the other tested loss functions. Besides, we notice that the loss is the most important component of the loss, as it effectively improves the quality according to the HDR-VDP2 scores. This is reflected in the relative weights of the loss: the weighting coefficient of the dynamic range loss component is much higher than the other weighting coefficients.

2-by-2 processing

As explained in section 3.2, we use 3D convolution layers in our network. To assess the usefulness of the 3D convolution layers, we train the same architecture with 2D convolution layers only. The related HDR-VDP2 scores in Table 3 show that 3D convolutions contribute positively to the quality of the result. Visually, the output of the 2D Convolution network is similar to the output of our network, but we notice some discrepancies, especially in lowly lit areas. This may come from the fact that highly lit areas are represented by high values in the tensor, and those values overpower low light levels during the backpropagation. This effect is mitigated through the use of 3D Convolution, thanks to the specialization of the network.

Varying kernel size

To further improve the reconstruction of details, we use, in the second half of the network, convolutions with decreasing size from (5, 5, ?) to (1, 1, ?). We train the same architecture with fixed-sized convolution kernel of (3, 3, ?). We notice some blur on those images, that comes from the convolution. Indeed, (3, 3, ?) averages the values of the pixels in the neighbourhood, which leads to faded edges on the image, and to a worse HDR-VDP2 score.

Subjective user study

We present in this section the user study we have conducted using our HDR SIM2 screen, to effectively compare the performance of our method with state-of-the art ones. We decided to perform a Two-Alternative Forced Choice (2AFC) experiment setting. To do so, we need to define the images pairs to be displayed on the HDR screen. For each participant, we randomly chose 10 images among our 80 test images and 5 versions of each of them: our method, HDRCNN, HDRUNet, ExpandNet and Single Image Network. The participants were presented with all possible image pairs created from the 10 test images and the 5 methods, for a total of 100 image pairs observed per session. In the following, we denote by ⟨I M , I M ′ ⟩ or ⟨I M ′ , I M ⟩ an image pair (with I X half the image generated by the method X): one half is generated by the method M from the SDR image I SDR and the other half by the method M ′ from the same SDR image I SDR (see Figure 9). We then asked the participants to choose their preferred method among the two methods shown in the image pair displayed on the HDR screen. To respect the aspect ratio of the images, we randomly chose if we display the left sides or the right sides of each image in the pair. The displayed image pair is then composed of twice the same side of one image, as shown in Figure 9.

As it is difficult to train state-of-the-art networks with our dataset, we decided to use the pre-trained networks provided by the authors of the state-of-the-art methods. Therefore, our study submitted to COMPUTER GRAPHICS Forum (3/2022). compares not only the architecture of these networks, but also their respective training datasets.

Each participant can attend multiple sessions (a session corresponds to the evaluation of 100 images pairs). If a participant attends another session, we use 10 images that the participant has not seen yet. The user study involved 29 participants (23M, 6F; age: avg=36.3 ± 12.7, min = 23, max = 71), and 30 sessions, for a total of 3,000 image pairs observed. Among the 29 participants, all of them reported to have normal or corrected-to-normal vision and 10 of them reported to have experienced HDR imaging before. During the experiment, each participant was asked to choose the method they preferred in the image pair. We have collected for each participant 100 answers (left or right of the image pair), each of them corresponds to a tested iTMO. For each participant p, we compute the preference scores xp(M 1 ), . The method M performs well according to the participant p if xp(M) ≥ 0.5. These preference scores are represented in Figure 10.

We can see on the Figure 10 that our method is largely preferred on average to all other tested methods. To further study these preferences, we performed a one-way ANOVA test after asserting that our data (the computed xp(M)) come from a normal distribution (using a Shapiro-Wilk test). We obtain a p-value p ≪ 0.05, meaning that the average values are significantly different. To further discriminate the methods, we perform a post-hoc test using a Tukey HSD test. This statistical test allows to compare the mean of every group (in our case, the groups are the different iTMOs) two-by-two. The results of the Tukey HSD test provides, for each pair of methods, the probability p that the mean preference scores of the two considered methods are the same. We present the results of the Tukey HSD test in Figure 10 by grouping together the methods with close probabilities p.

Using the post hoc test, we notice that the participants found on average no significant differences between the images processed by HDRCNN and by ExpandNet (Average value of preference score of x = 0.57). Using the same test, our method is preferred on average to all other methods (Average value of preference score of x = 0.83). Our method, HDRCNN and ExpandNet all have an average preference score of above 0.5, meaning that those three methods are most of the time preferred by the participants. This study shows that our method performs well for human observers on our test dataset.

Conclusion

We have presented a new inverse tone mapping operator, called HDR-LFNet, along with its training dataset. Our architecture is lighter than the networks of the state-of-the-art thanks to methods aggregation, a technique inspired by other computer vision domains. To be fully effective, this lightweight architecture requires high resolution images to train. As there is no existing training HDR dataset with sufficient resolution, we also release a new dataset of high resolution HDR images that can be used by the community in complement or in place of existing datasets.

Objectives metrics show that our method is on-par with other methods, but the conducted user study shows that our method is preferred by observers. Along with the lower number of parameters, our method is a real improvement over the state-of-the-art. Our HDR-LFNet can be used in several applications, where resources for training and storage of the model are limited. This also should allow to transform an SDR image dataset with annotations (quality score, aesthetics score, saliency data) to an HDR image dataset with corrected annotations.

Figure 1 :

 1 Figure 1: Our method HDR-LFNet uses a neural network to fuse several expanded versions of the input. This allows for faster training and lighter network. As we process linear luminance values, we use a gamma correction and colourization as post-processing.

Figure 2 :

 2 Figure2: Architecture of our network, which uses 3D-convolutions. The network processes volumes of images (H,W, D,C) (represented in yellow), where depth D is represented by magenta lines. The number below each feature map corresponds to the number of channels C (not graphically represented on the figure). Green arrows represent 3D convolution of kernel size (kx, ky, kz) followed by the activation function ReLUa, blue arrows represent skip connections (which are detailed in Section 3.2). The red arrow indicates the 3D convolution that effectively fuse the input images. Downsampling and maxpooling are represented by red-orange layers, and upsampling are represented by blue layers.

Figure 3 :

 3 Figure 3: Examples of SDR images from the proposed dataset.

Figure 4 :

 4 Figure 4: 2D projection of the scores of images obtained with the t-SNE algorithm. Each colour corresponds to a different iTMO. The red line is the maximum area convex hull, which encompasses points from Akyuz, Landis and Kovaleski & Oliveira iTMOs.

Figure 5 :Figure 6 :

 56 Figure 5: Example of an image from HDR-Real dataset processed by different models. For ease of view, HDR images have been tonemapped using the Drago algorithm [DMAC03].

Figure 7 :

 7 Figure 7: HDR-VDP2 score variation depending on the angular resolution on the HDR-Real dataset for four models.

Figure 8 :

 8 Figure 8: Mean HDR-VDP2 score against the number of trainable parameters of the tested models.

Figure 9 :

 9 Figure 9: Example of an image pair shown to one participant of the user study. Here, the left half was generated with HDRUNet and the right half with HDRCNN (all images have been tone-mapped using the Drago algorithm).

Figure 10 :

 10 Figure 10: Preference of all participants for each method. The red line corresponds to a preference of 50%. The stars are attributed according to the p-values of the Tukey HSD test: nothing for p > 0.05, * for 0.05 ≥ p > 0.005, ** for 0.005 ≥ p > 0.0005, *** for 0.0005 ≥ p.

  . . , xp(M 5 ) for each method M i . If we denote by IP(M) the set of image pairs that contains an image generated with the method M, we can compute xp(M) with xp(M) = card({⟨I M , I M ′ ⟩ ∈ IP(M) s.t. p preferred I M over I M ′ }) card(IP(M)) .

Table 1 :

 1 Characteristics of different HDR datasets. ( * This is the number of original HDR images in the training set, but this dataset is composed of more than 19,000 crops of size (512 × 512))

	Dataset	Number of HDR images	Images size
	HDR-Eye [NKHE15]	46	(1920 × 1080)
	DEIMOS [KFP*11]	79	(4300 × 2900)
	pfstools [MKMS07]	8	Variable
	HDRPS [Fai07]	105	(4300 × 2900)
	HDR-Real [LLC*20]	480 *	Variable
	Our dataset	496	∼ (6000 × 4000)

Table 2 :

 2 Mean and standard deviation of HDR-VDP2 and Harmonic IQA for several models on the HDR-Real [LLC*20] test set. Scores in bold are the best; Scores underlined are the second best.

		.0745 (7.6484)	0.3554 (0.0997)	20.1330 (8.7622)	0.4325 (0.4499)
	ExpandNet [MBHD18]	47.8268 (6.8777)	0.3685 (0.1003)	19.9602 (8.1942)	0.4043 (0.4643)
	Single Image [LLC*20]	51.0739 (6.9557)	0.3798 (0.1136)	26.4531 (8.8608)	0.5861 (0.4353)
	HDRUNet [CLZ*21]	48.1709 (6.1163)	0.4174 (0.0480)	18.0796 (7.3235)	0.3270 (0.4764)
	Our Method trained on HDR-Real	28.3378 (5.8721)	0.2764 (0.1386)	12.4031 (6.4205)	0.0771 (0.3641)
	Our Method fine-tuned on HDR-Real	41.3083 (8.7949)	0.3555 (0.1002)	19.7721 (8.9024)	0.3749 (0.4699)
	Proposed Method	49.8686 (8.4689)	0.3597 (0.1053)	21.0415 (8.8968)	0.4647 (0.4407)
	Model	Score
	With 2DConv	42.6345
	Without kernel size change 38.9088
	Only MAE	36.3503
	MAE + gMAE	34.9649
	MAE + dynLoss	39.9745
	MAE + VGGLoss	35.4325
	Final model	49.8686

Table 3 :

 3 Mean HDR-VDP2 score for different variants of the model.

submitted to COMPUTER GRAPHICS Forum (3/2022).