Etienne P Hessou

Michael Badawi

Laetitia Valentin

Guy Atohoun

Stanislaw Dzwigaj

Monica Calatayud

Frederik Tielens
email: frederik.tielens@vub.be

Elucidation of the IR of Cu and Mn substituted intraframework SiBEA zeolites

Keywords: DFT, IR, CuSiBEA, MnSiBEA, Zeolites

come   L'archive ouverte pluridisciplinaire

Introduction

Transition metal ionic centers play in important role in catalysis, and especially in heterogenous catalysis [START_REF] Notari | Microporous Crystalline Titanium Silicates[END_REF][START_REF] Corma | From microporous to mesoporous molecular sieve materials and their use in catalysis[END_REF]. The incorporation of nano sized transition metals in oxides, and in silica based materials, has shown to enable the fine tuning and control of the specific reactivity of such materials [START_REF] Tielens | Group V metal substitution in silicate model zeolites: In search for the active site[END_REF][START_REF] Tielens | What Do the Niobium Framework Sites Look Like in Redox Zeolites? A Combined Theoretical and Experimental Investigation[END_REF][START_REF] Tielens | Probing acid-base sites in vanadium redox zeolites by DFT calculation and compared with FTIR results[END_REF][START_REF] Wojtaszek | Comparison of competition between T=O and T-OH groups in vanadium, niobium, tantalum BEA zeolite and SOD based zeolites[END_REF][START_REF] Tielens | Chromium sites in zeolite framework: Chromyl or chromium hydroxyl groups?[END_REF][START_REF] Handzlik | Structure of monomeric chromium (VI) oxide species supported on silica: periodic and cluster DFT studies[END_REF][START_REF] Handzlik | Isolated chromium (VI) oxide species supported on Almodified silica: a molecular description[END_REF][START_REF] Weckhuysen | Olefin polymerization over supported chromium oxide catalysts[END_REF][START_REF] Chen | Kinetics and Mechanism of Oxidative Dehydrogenation of Propane on Vanadium, Molybdenum, and Tungsten Oxides[END_REF][START_REF] Bañares | Supported metal oxide and other catalysts for ethane conversion: a review[END_REF][START_REF] Deo | Reactivity of supported vanadium oxide catalysts: The partial oxidation of methanol[END_REF][START_REF] Gueddida | Interaction between transition metals (Co, Ni, and Cu) systems and amorphous silica surfaces: A DFT investigation[END_REF].

For decades transition metal oxides have been studied in and on different solid materials, and particularly in zeolites [START_REF] Tielens | Group V metal substitution in silicate model zeolites: In search for the active site[END_REF][START_REF] Boroń | Influence of iron state and acidity of zeolites on the catalytic activity of FeHBEA, FeHZSM-5 and FeHMOR in SCR of NO with NH3 and N2O decomposition[END_REF][START_REF] Dzwigaj | The similarities and differences in structural characteristics and physico-chemical properties of AgAlBEA and AgSiBEA zeolites[END_REF][START_REF] Kyriienko | High selectivity of TaSiBEA zeolite catalysts in 1, 3-butadiene production from ethanol and acetaldehyde mixture[END_REF][START_REF] Kyriienko | Ethanol conversion into 1, 3-butadiene by the Lebedev method over MTaSiBEA zeolites (M= Ag, Cu, Zn)[END_REF][START_REF] Kyriienko | Effect of the niobium state on the properties of NbSiBEA as bifunctional catalysts for gas-and liquid-phase tandem processes[END_REF][START_REF] Popovych | Design of Bifunctional Catalysts Based on Bea Zeolites for Tandem Processes with Participation of Ethanol[END_REF][START_REF] Trejda | Methanol oxidation on VSiBEA zeolites: Influence of V content on the catalytic properties[END_REF]. The presence of these transition metal sites in the zeolites has shown to introduce both acid/base as well as redox properties in the catalysts, together with the intrinsic size selectivity associated to the open structure containing channels and cages.

The possibility to obtain extra and intra-framework transition metal sites makes zeolites extremely versatile heterogenous catalysts [START_REF] Grundner | Single-site trinuclear copper oxygen clusters in mordenite for selective conversion of methane to methanol[END_REF][START_REF] Grundner | Synthesis of single-site copper catalysts for methane partial oxidation[END_REF][START_REF] Vanelderen | Spectroscopic Definition of the Copper Active Sites in Mordenite: Selective Methane Oxidation[END_REF]. In order to understand their reactivity, in-depth characterization is needed, and the combination of experimental and theoretical methods provide highly valuable information on the atomic level [START_REF] Qin | Understanding the Fundamentals of Microporosity Upgrading in Zeolites: Increasing Diffusion and Catalytic Performances[END_REF][START_REF] Komaty | Incorporation of trivalent cations in NaX zeolite nanocrystals for the adsorption of O 2 in the presence of CO 2[END_REF][START_REF] Khalil | Selective capture of phenol from biofuel using protonated faujasite zeolites with different Si/Al ratios[END_REF].

Recently, an increased interest in copper-and manganese-containing zeolite catalysts aroused, due to the high efficiency in processes important for environment protection such as selective catalytic reduction of NO with ammonia [START_REF] Lin | Selective catalytic reduction of NO with NH3 over Mn-Fe/USY under lean burn conditions[END_REF][START_REF] Sultana | Influence of support on the activity of Mn supported catalysts for SCR of NO with ammonia[END_REF][START_REF] Stanciulescu | Lowtemperature selective catalytic reduction of NOx with NH3 over Mn-containing catalysts[END_REF][START_REF] Komatsu | Kinetic Studies of Reduction of Nitric Oxide with Ammonia on Cu2+-Exchanged Zeolites[END_REF][START_REF] Colombo | A comparative study of the NH3-SCR reactions over a Cuzeolite and a Fe-zeolite catalyst[END_REF] and oxidation of volatile organic compounds (VOC) [START_REF] Meng | One-step hydrothermal synthesis of manganese-containing MFI-type zeolite, Mn-ZSM-5, characterization, and catalytic oxidation of hydrocarbons[END_REF][START_REF] Einaga | Catalytic oxidation of benzene with ozone over Mn ion-exchanged zeolites[END_REF][START_REF] Zhan | Catalytic oxidation of n-hexane on Mn-exchanged zeolites: Turnover rates, regioselectivity, and spatial constraints[END_REF].

The preparation of Cu-and Mn-containing zeolites is commonly achieved by conventional techniques of postsynthesis modification such as wet impregnation and ion-exchange or direct synthesis in hydrothermal conditions. However, copper and manganese in zeolites obtained by these conventional preparation procedures may be present at various oxidation states and chemical environments because these preparation methods do not allow controlling state of copper and manganese in the zeolite structure. As a result, Cu-and Mn containing zeolite materials with different kinds of copper and manganese species could be formed.

The presence of different types of active sites might lead to poor selectivity in many reactions, it is thus desirable to obtain a stable and high selective catalyst during the synthesis to hinder side reactions and to avoid fast deactivation. Thus, the optimization of current synthesis methods as well as the development of new methods of zeolite catalyst preparation are highly needed.

In this work we investigate and discuss Mn 2+ and Cu 2+ substitutions or insertions in the BEA zeolite framework, geometrically, energetically and spectroscopically (IR) in other to elucidate and localize active sites in these materials for applications in catalysis. The materials are synthesized and characterized by experimental IR techniques, combined with theoretical ab initio approaches, so as to determine both structure and stability around the metal sites. Thus, materials are first characterized by periodic Density Functional Theory (DFT) to determine the energetics and to simulate the IR spectroscopic properties. A set of 6 possible metal intraframework sites for Cu and Mn with neighboring hydroxyl groups is considered. The procedure is similar to previous studies where transition metal/Si substitution is investigated in simple sodalite (SOD) structure [START_REF] Tielens | Group V metal substitution in silicate model zeolites: In search for the active site[END_REF][START_REF] Tielens | What Do the Niobium Framework Sites Look Like in Redox Zeolites? A Combined Theoretical and Experimental Investigation[END_REF][START_REF] Tielens | Probing acid-base sites in vanadium redox zeolites by DFT calculation and compared with FTIR results[END_REF][START_REF] Dzwigaj | Ta (V)-single site BEA zeolite by two-step postsynthesis method: preparation and characterization[END_REF][START_REF] Tielens | What do vanadium framework sites look like in redox model silicate zeolites?[END_REF] and in BEA zeolite [START_REF] Tielens | Group V metal substitution in silicate model zeolites: In search for the active site[END_REF][START_REF] Tielens | What Do the Niobium Framework Sites Look Like in Redox Zeolites? A Combined Theoretical and Experimental Investigation[END_REF][START_REF] Tielens | Probing acid-base sites in vanadium redox zeolites by DFT calculation and compared with FTIR results[END_REF][START_REF] Wojtaszek | Comparison of competition between T=O and T-OH groups in vanadium, niobium, tantalum BEA zeolite and SOD based zeolites[END_REF][START_REF] Tielens | Chromium sites in zeolite framework: Chromyl or chromium hydroxyl groups?[END_REF][START_REF] Handzlik | Structure of monomeric chromium (VI) oxide species supported on silica: periodic and cluster DFT studies[END_REF][START_REF] Tielens | What do vanadium framework sites look like in redox model silicate zeolites?[END_REF][START_REF] Tranca | Methanol Oxidation to Formaldehyde on VSiBEA Zeolite: A Combined DFT/vdW/Transition Path Sampling and Experimental Study[END_REF][START_REF] Tielens | What do tantalum framework sites look like in zeolites? A combined theoretical and experimental investigation[END_REF]. As a result, accurate information about the thermodynamic stability of the Cu-and Mn-sites is provided, together with in silico IR spectra that guide the comparison with experimental results.

In the experimental counterpart, the modified zeolites are synthesized using a two-step postsynthesis method, composed of dealumination and impregnation steps, developed earlier by Dzwigaj et al. [START_REF] Dzwigaj | Incorporation of vanadium species in a dealuminated β zeolite[END_REF][START_REF] Dzwigaj | Probing different kinds of vanadium species in the VSiβ zeolite by diffuse reflectance UV-visible and photoluminescence spectroscopies[END_REF][START_REF] Dzwigaj | Role of Silanol Groups in the Incorporation of V in β Zeolite[END_REF][START_REF] Dzwigaj | Recent advances in the incorporation and identification of vanadium species in microporous materials[END_REF][START_REF] Dzwigaj | Remarkable effect of the preparation method on the state of vanadium in BEA zeolite: Lattice and extralattice V species[END_REF] to obtain Cu and Mn containing SiBEA zeolite with isolated, mononuclear Cu and Mn species. The CuSiBEA and MnSiBEA prepared by this postsynthesis method contains Cu(II) and Mn(II) species incorporated into the intraframework positions. The incorporation of copper and manganese by two-step postsynthesis method as Cu(II) and Mn(II) species has been proved in our previous works by DR UV-vis, EPR, FTIR-CO, XPS, EXAFS and XANES investigations and published in [START_REF] Dzwigaj | Do Cu(II) ions need Al atoms in their environment to make CuSiBEA active in the SCR of NO by ethanol or propane? A spectroscopy and catalysis study[END_REF][START_REF] Baran | Effect of postsynthesis preparation procedure on the state of copper in CuBEA zeolites and its catalytic properties in SCR of NO with NH3[END_REF] The presence of Cu(II) species in Cu2.0SiBEA sample has been earlier proved [START_REF] Dzwigaj | Do Cu(II) ions need Al atoms in their environment to make CuSiBEA active in the SCR of NO by ethanol or propane? A spectroscopy and catalysis study[END_REF] by DR UVvis, EPR and XPS spectra of Cu2.0SiBEA sample. The DR UV-vis spectrum showed a broad band at 835 nm characteristic of d-d transitions and an intense charge-transfer (CT) band at 280 nm. These bands could be assigned to the d-d transition of Cu 2+ (3d 9 ) and CT O 2-ª Cu 2+ transition of isolated mononuclear Cu(II) being in coordination with lattice oxygen, respectively [START_REF] Moretti | A comparison between Cu-ZSM-5, Cu-S-1 and Cu-mesoporous-silica-alumina as catalysts for NO decomposition[END_REF][START_REF] Groothaert | Selective Oxidation of Methane by the Bis(μ-oxo)dicopper Core Stabilized on ZSM-5 and Mordenite Zeolites[END_REF][START_REF] Praliaud | Surface and bulk properties of Cu-ZSM-5 and Cu/Al2O3 solids during redox treatments. Correlation with the selective reduction of nitric oxide by hydrocarbons[END_REF][START_REF] Lezcano-Gonzalez | Chemical deactivation of Cu-SSZ-13 ammonia selective catalytic reduction (NH3-SCR) systems[END_REF]. The EPR spectrum of Cu2.0SiBEA showed at 298 K isotopic signal (giso = 2.19) typical of octahedral Cu(II) species [START_REF] Godiksen | Coordination Environment of Copper Sites in Cu-CHA Zeolite Investigated by Electron Paramagnetic Resonance[END_REF][START_REF] Soria | Influence of the Preparation Method, Outgassing Treatment, and Adsorption of NO and/or O2 on the Cu2+ Species in Cu-ZSM-5: An EPR Study[END_REF][START_REF] Larsen | Electron Paramagnetic Resonance Studies of Copper Ion-Exchanged ZSM-5[END_REF][START_REF] Sass | Comparative electron spin echo study of aluminum modulation associated with copper (2+) in Ca-ZSM-5 and K-ZSM-5 zeolites[END_REF][START_REF] Carl | Variable-temperature electron paramagnetic resonance studies of copper-exchanged zeolites[END_REF]. Moreover, our earlier FTIR investigation [START_REF] Baran | Effect of postsynthesis preparation procedure on the state of copper in CuBEA zeolites and its catalytic properties in SCR of NO with NH3[END_REF] using CO as a probe molecule have confirmed that in Cu2.0SiBEA sample, prepared by two_step postsynthesis method, cobalt is present as isolated mononuclear copper(II) species. This is proved by intense band at 2202 cm -1 atributted to Cu(II)-CO adducts.

The presence of Cu(II) species in CuSiBEA samples has been confirmed in our earlier investigation using XPS studies [START_REF] Janas | Effect of Co content on the catalytic activity of CoSiBEA zeolite in the selective catalytic reduction of NO with ethanol: Nature of the cobalt species[END_REF]. Because Cu 2p3/2 core excitation considered alone does not allow determining the oxidation state and coordination of copper without doubt, the CuSiBEA samples are characterized by using both X-ray excited Cu 2p3/2 and Cu L3VV transitions. Therefore, a shake-up satellites associated with Cu 2p3/2 peak and a modified Auger parameter were employed for Cu oxidation state determination. In addition, the shake-up satellite associated with Cu 2p3/2 peak appearing at about 945 eV for CuSiBEA, confirms that copper is mainly present in (II) oxidation state. The low intensity of the satellite for all CuSiBEA is probably due to high copper dispersion in the zeolite structure as isolated tetracoordinated Cu(II) [START_REF] Corma | Determining the Nature of the Active Sites of Cu-Beta Zeolites for the Selective Catalytic Reduction (SCR) of NOxby Using a Coupled Reaction-XAES/XPS Study[END_REF]. The presence of second component (B)

for Cu 2p3/2 and Cu 2p1/2 at higher BE of about 936-937 eV and 956-957 eV, respectively, reveals that probably some amount of octacoordinated Cu(II) species is also present, which is in line with similar studies on Cu/Y zeolite reported by Narayana et al. [START_REF] Narayana | X-ray photoelectron and electron spin resonance spectroscopic studies of CuNaY zeolites[END_REF].

The presence of Cu(II) species in CuSiBEA, prepared by two-step postsynthesis method has been evidenced by the Fourier transform of k3 -weighted Cu K-edge EXAFS and XANES spectra [START_REF] Dzwigaj | Do Cu(II) ions need Al atoms in their environment to make CuSiBEA active in the SCR of NO by ethanol or propane? A spectroscopy and catalysis study[END_REF]. The Fourier transform of CuSiBEA shows only one single and intense peak at 1.6-1.7 A˚ due to the presence in the first shell of Cu(II) of four oxygen neighbours [START_REF] Wells | The lanthanides and actinides[END_REF]. For CuSiBEA, a small preedge peak at 8975 eV is observed in XANES spectrum due to the 1s --> 3d transition characteristic of Cu 2+ [START_REF] Bart | Near-edge X-ray absorption spectroscopy in catalysis[END_REF].

In conclusion, the combined DR UV-vis-NIR, EPR, FTIR-CO, XPS, EXAFS and XANES investigations suggest that copper in CuSiBEA zeolite is mainly present as isolated mononuclear Cu(II) species.

The presence of Mn(II) species in MnSiBEA sample has been earlier evidenced by XPS with the binding energy of Mn2p3/2 in the range of 640.6 and 642.0 eV [START_REF] Baran | Incorporation of Mn into the vacant T-atom sites of a BEA zeolite as isolated, mononuclear Mn: FTIR, XPS, EPR and DR UV-Vis studies[END_REF] and by XANES spectroscopy [START_REF] Baran | Influence of the nature and environment of manganese in Mn-BEA zeolites on NO conversion in selective catalytic reduction with ammonia[END_REF].

The samples are then characterized by IR spectroscopy.

Computational details

The theoretical calculations have been performed using the Vienna Ab Initio Simulation Package (VASP) [START_REF] Kresse | Ab initio molecular dynamics for liquid metals[END_REF]. Density functional theory was used applying the augmented plane wave method (PAW) [START_REF] Kresse | From ultrasoft pseudopotentials to the projector augmented-wave method[END_REF] to describe the electron-ion interactions with a cut-off energy of 500 eV [START_REF] Wojtaszek | Comparison of competition between T=O and T-OH groups in vanadium, niobium, tantalum BEA zeolite and SOD based zeolites[END_REF][START_REF] Kulkarni | Influence of plane wave cut-off on structural and electronic properties in Sn-BEA and Ti-BEA zeolite water molecule interaction[END_REF]. The Perdew Burke Ernzerhof functional (PBE) [START_REF] Perdew | Generalized gradient approximation made simple[END_REF] was employed, and the Kohn-Sham spin-polarized equations were solved self-consistently until the energy difference of the cycles is smaller than 10 -6 eV. The atomic positions were fully optimized until all forces were smaller than 0.01 eV/Å per atom. All computations were performed only at the Γ-point due to the large size of the unit cell.

To accurately describe the zeolite structure, van der Waals (vdW) interactions should be included [START_REF] Chibani | A DFT investigation of the adsorption of iodine compounds and water in H-, Na-, Ag-, and Cu-mordenite[END_REF][START_REF] Hessou | Adsorption of NO, NO2, CO, H2O and CO2 over isolated monovalent cations in faujasite zeolite: a periodic DFT investigation[END_REF][START_REF] Göltl | Van der Waals interactions between hydrocarbon molecules and zeolites: Periodic calculations at different levels of theory, from density functional theory to the random phase approximation and Møller-Plesset perturbation theory[END_REF][START_REF] Ramalho | Accounting for van der Waals interactions between adsorbates and surfaces in density functional theory based calculations: selected examples[END_REF]. In this work, the Grimme D3-correction method by introducing the Becke-Johnson damping potential [START_REF] Grimme | A consistent and accurate ab initio parametrization of density functional dispersion correction (DFT-D) for the 94 elements H-Pu[END_REF][START_REF] Grimme | Effect of the damping function in dispersion corrected density functional theory[END_REF] was used to compute the substitution energy of each M-substituted BEA zeolite, as described below.

To simulate the vibrational frequencies and intensities of IR spectra, the Density Functional Perturbation Theory (DFPT) [START_REF] Baroni | Phonons and related crystal properties from density-functional perturbation theory[END_REF][START_REF] Karhánek | A density-functional study of the adsorption of methane-thiol on the (111) surfaces of the Ni-group metals: II. Vibrational spectroscopy[END_REF][START_REF] Khalil | Biofuel purification: Coupling experimental and theoretical investigations for efficient separation of phenol from aromatics by zeolites[END_REF][START_REF] Foucaud | Adsorption mechanisms of fatty acids on fluorite unraveled by infrared spectroscopy and first-principles calculations[END_REF] was employed on our relaxed structures. In the dipolar approximation, the intensity of ith normal mode of vibration at a frequency ωi, is proportional to the square of the change of dipole moment associated with the atomic motion along the eigenvector ei of that mode:

Ii ∝ = ∑ ∑ ∑ * ( ) , ( )
Where, , ( ) is the displacement of ith-atom in the eigenvector of the ith normal mode and * ( ) = ( ) is the Born charge of the ith atom, as implemented in VASP [START_REF] Gajdoš | Linear optical properties in the projector-augmented wave methodology[END_REF].

To build the MBEA zeolite model, the whole orthorhombic unit cell with dimensions (12.6 Å × 12.6 Å × 26.2 Å) of siliceous BEA is used (Figure 1) [START_REF] Wojtaszek | Comparison of competition between T=O and T-OH groups in vanadium, niobium, tantalum BEA zeolite and SOD based zeolites[END_REF][START_REF] Shetty | A comparative study of structural, acidic and hydrophilic properties of Sn-BEA with Ti-BEA using periodic density functional theory[END_REF]. According to crystallographic data, zeolite

Experimental Details

Copper-and Manganese-containing BEA zeolites were prepared from parent TEABEA zeolite by a two-step postsynthesis method [START_REF] Dzwigaj | Incorporation of vanadium species in a dealuminated β zeolite[END_REF][START_REF] Dzwigaj | Probing different kinds of vanadium species in the VSiβ zeolite by diffuse reflectance UV-visible and photoluminescence spectroscopies[END_REF][START_REF] Dzwigaj | Role of Silanol Groups in the Incorporation of V in β Zeolite[END_REF][START_REF] Dzwigaj | Recent advances in the incorporation and identification of vanadium species in microporous materials[END_REF][START_REF] Dzwigaj | Remarkable effect of the preparation method on the state of vanadium in BEA zeolite: Lattice and extralattice V species[END_REF]. In the first step, TEABEA zeolite (Si/Al = 17) was dealuminated by a treatment with nitric acid solution of C = 13 mol L -1 at 353 K for 4 h, to obtain aluminum free SiBEA zeolite, and then it was washed several times with distilled water and dried at 368 K overnight.

In the second step SiBEA zeolites were put in contact with Cu(NO3)2 or Mn(NO3)2 aqueous solution with appropriate concentration and stirred for 24 h at 298 K. Then, the suspensions were stirred in evaporator under vacuum of a water pump for 2 h at 333 K until the water was evaporated. The obtained materials with Cu and Mn atoms in the framework position were denoted as Cu2.0SiBEA or Mn2.0SiBEA where number next to element symbol refers to Cu or Mn wt % content.

Results and discussion

Geometric and energetic analysis

The Cu and Mn atoms were substituted for the Si in the BEA framework at the T1 site thus leading to CuSiBEA and MnSiBEA. The number of substitutions in m% is 1.64 and 1.42, respectively, for CuSiBEA and MnSiBEA and the Si/M ratio is equal to 63 for both. The difference between these rates is due to the difference of molar mass of both atoms. The introduction of a M 2+ ion in a zeolite framework causes a charge deficit that must be compensated for the neutrality of the system, achieved by incorporating 2 H + . The protons may bind to lattice oxygen sites forming of hydroxyl groups, in contrast with M ions with higher oxidation states for which oxo-groups can be introduced (See refs [START_REF] Tielens | What Do the Niobium Framework Sites Look Like in Redox Zeolites? A Combined Theoretical and Experimental Investigation[END_REF][START_REF] Wojtaszek | Comparison of competition between T=O and T-OH groups in vanadium, niobium, tantalum BEA zeolite and SOD based zeolites[END_REF][START_REF] Tielens | What do vanadium framework sites look like in redox model silicate zeolites?[END_REF][START_REF] Tranca | Methanol Oxidation to Formaldehyde on VSiBEA Zeolite: A Combined DFT/vdW/Transition Path Sampling and Experimental Study[END_REF][START_REF] Tielens | What do tantalum framework sites look like in zeolites? A combined theoretical and experimental investigation[END_REF]). The hydroxyl groups surrounding the M center can be organized as shown in Figure 2, leading to 6 isomers that have been considered in the present work. The Si atoms of the two OSi groups which are linked to the transition metal do not have the same chemical environment (T2, T3, T7 and T8) and it is the same for two silanol groups. Accordingly, we choose to name the different isomers by the site of the two silanol groups as follow: T2-T3, T2-T7, T2-T8, T3-T7 and T7-T8 i.e. T2-T3 represents the configuration in which OH groups are linked to the silicon atoms located in site T2 and T3. For each isomer, we have investigated several orientations to the 2 H + of the 2 OH groups and only the most stable one is reported.

The optimized structures of the six CuSiBEA isomers are displayed in Figure 3 while Table 1 shows the geometric parameters and the relative total energies calculated for CuSiBEA. Copper fits into the framework with a coordination number II making two Cu-OSi bonds. The first varies between 1.78 and 1.81 Å while the second between 1.81 and 1.88 Å. These two bonds are on average 1.82 Å. They exceed the classical Si-O bonds in zeolites whose average value is 1.64 Å. This difference can be explained by the difference in size of these two elements whose ionic radii are 40 and 78 pm, respectively, for Si 4+ and Cu 2+ . The M-OSi distances are then proportional to the size of the metals.

From the orientation of the surrounding silanol groups one can conclude that copper also interacts weakly with them. The calculated Cu-OH distances vary between 2.0 and 2.87 Å with an average value of 2.25 Å. The O-H distances of the two hydroxyl groups vary very weakly between 0.97 and The calculated relative energy for CuSiBEA is in range of 0 to 0.21 eV. This parameter is used to classify the six configurations in order of decreasing stability as follows: T3-T7 > T7-T8 > T3-T8 > T2-T7 > T2-T8 > T2-T3. Here the most stable configuration is the T3-T7 unlike Mn-BEA where it is rather the most stable T7-T8. However, the energy difference between T3-T7 and T7-T8 is quite low (0.02 eV) for Cu-BEA then one can consider T3-T7 ≈ T7-T8. The geometric parameters and the relative energies calculated for MnSiBEA are reported in Table 2. By substituting the silicon with the manganese, two Mn-O(Si) bonds are formed with lengths varying between 1.89 and 1.98 Å with an average value of 1.93 Å. The two other oxygen atoms located in the vicinity of the metal (Mn) are now involved in two silanol obtained by adding two H + protons. These oxygen atoms are less close to the metal (Mn) compared to the two previous ones with Mn-O(H) distances varying in range of 2.16 to 2.29 Å (an average of 2.21 Å). We notice that Mn-OSi are slightly larger than Cu-OSi and this even though the ionic radius of Mn 2+ (67 pm) is smaller than that of Cu 2+ (73 pm) Cu. Regarding the O-H distances, they vary weakly between 0.97 and 1 Å with an average value of 0.99 Å. Si-OH distances are quite similar for all configurations with an average value of 1.67 Å. The calculated OMnO angles vary widely between 124 and 153° with an average value of 143°.

The relative energies calculated for MnSiBEA vary between 0 and 0.29 eV. With these relative energies, the six configurations can be classified in terms of stability as follow: T7-T8 > T3-T7 > T3-T8 > T2-T7 > T2-T8 > T2-T3. The most stable configuration (T7-T8) displays 145 ° for OMnO angle while the least stable configuration (T2-T3) displays 124 ° for the OMnO angle. The total substitution reaction energy is estimated following the reaction:

! + #$ → # !$ +
with MH2 and Si in the gas phase. The reaction energy was calculated to be equal to 6.9 and 5.5 eV, for Cu 2+ and Mn 2+ respectively. These values, calculated at 0 K, show that the substitution is an endothermic reaction. The substitution energy of Cu 2+ is 0.6 eV higher than that of Mn 2+ .

Accordingly, one can conclude that the substitution should be easiest for Mn 2+ than Cu 2+ .

Frequencies analysis

Calculated vibrational frequencies of CuSiBEA for all configurations were reported in Table 3. The accuracy of the calculated frequencies was found to be within 75 cm -1 for the high wave number OH vibrations, and a few cm -1 for the Metal-O vibrations around 1000 cm -1 [START_REF] Gierada | Hydration in silica based mesoporous materials: a DFT model[END_REF][START_REF] Gierada | Understanding the Acidic Properties of the Amorphous Hydroxylated Silica Surface[END_REF][START_REF] Siodla | Theoretical and experimental insight into zinc loading on mesoporous silica[END_REF]. The analysis of each spectrum obtained shows two peaks corresponding to OH stretching ranging from 3215 to 3725 cm -1 .

The shift between these two peaks varies from 40 to 395 cm -1 . The smallest deviation is obtained with the T7-T8 configuration while the largest deviation (395 cm -1 ) is observed with the T7-T8 configuration. The most stable configuration (T3-T7) has 140 cm -1 between both peaks.

The peaks that can be attributed to the vibrations of SiO (H) elongations are observed around 800-900 cm -1 for the six configurations. Regarding CuO elongation vibrations, they are between 500 and 700 cm -1 . The calculated IR spectra for the most stable configuration is displayed in Figure 5. Assignments in Table 3.

We have reported in Table 4 the calculated vibrational frequencies of MnSiBEA for the six configurations. All the calculated IR spectra show two peaks between 3740 and 3150 cm -1 . These two peaks correspond to the vibrations of elongations of the two hydroxyl groups. They are separated by a difference ranging from 113 to 250 cm -1 , which makes it possible to confirm without ambiguity the presence of the two hydroxyl groups in the framework. Of the two OH peaks, the peak with the greater frequency corresponds to the hydroxyl group bonded to the T (Si) atom of the higher site. This suggests that the frequency of vibration of OH elongation strongly depends on the site T where this group is located. Peaks located between 3200 and 3550 cm -1 can be associated with hydroxyl groups involved in hydrogen bonds [START_REF] Dzwigaj | Incorporation of vanadium species in a dealuminated β zeolite[END_REF][START_REF] Kondo | Acid property of silanol groups on zeolites assessed by reaction probe IR study[END_REF][START_REF] Zecchina | Silicalite characterization. 2. IR spectroscopy of the interaction of carbon monoxide with internal and external hydroxyl groups[END_REF]. The most stable configuration (T7-T8) predicted by relative energies displays these two peaks at frequencies 3735 and 3623 cm -1 while the least stable configuration (T2-T3) displays its two OH peaks at frequencies 3412 and 3163 cm -1 . A large range without any vibration is present on all configurations. It ranges from 1195 to 3160 cm -1 for the least stable configuration and from 1175 to 3620 cm -1 for the most stable configuration. The area from 1000 to 1190 cm -1 corresponds to the vibrations of SiO elongations. The vibrations of deformations of OH groups are between 800 and 1000 cm -1 . The vibrations of SiO elongations of the silanols are also observed in this range. Similar values are reported in Table 3 for CuSiBEA. The peaks corresponding to MnO vibrations are located at 580 and 660 cm -1 . The vibration frequencies and vibration modes calculated for both MnSiBEA and CuSiBEA are quite similar despite the significant difference in size between Mn and Cu. Figure 6 displays the calculated IR spectrum of MnSiBEA. Assignments in Table 4.

4000 3900 3800 3700 3600 3500 3400 3300 3200 The Figure 7 shows the experimental IR spectra of Cu2.0SiBEA, Mn2.0SiBEA and SiBEA in the range of OH groups (4000-3200 cm -1 ) (A) and structural vibration of BEA zeolite (1000 -500 cm - 1 ) (B). Both spectra (Cu2.0SiBEA and Mn2.0SiBEA) are similar, proof that the incorporation of Cu 2+ and Mn 2+ into the framework of the zeolite succeed without any modification of the framework. This is also supported by the IR spectrum of pure SiBEA (Figure7). that can be found as Supporting Information. Stretching vibrations of OH appear at 3540 and 3733 cm -1 supporting the presence of hydroxyl groups. By comparing experimental and theoretical spectra, in the range of 4000-3200 cm -1 , a very nice agreement is observed. Experimental peak at 968 cm -1 for CuSiBEA and MnSiBEA can be attributed to Si-O stretching. Si-O(H) vibrations are around 810 cm -1 while Cu-O an Mn-O peaks are in a range of 500-650 cm -1 . Difference seen in this range between CuSiBEA and MnSiBEA can be due to the formation of Cu-O and Mn-O bonds. However, considering the complexity of the system and the calculation level, theoretical results are found to be in very good agreement with experiment, allowing an ab initio assignment of the bands observed.

Conclusions

In this study, we describe the incorporation of copper (II) and manganese (II) in vacant T atom sites of SiBEA zeolite. This process leads to the formation of 6 possible isomers, involving the formation of hydroxyl groups in different lattice positions. An energetic analysis made it possible to identify the most stable isomer in each of the two models. The IR spectra are calculated and compared with those determined experimentally. The spectra of the M-modified zeolites are very similar to the pure SiBEA system, indicating that the incorporation reaction was reached without modification of the framework.

Experimental spectra are resolved and associated with molecular vibrations and site geometry. No experimental vibrations were detected between 1100 and 3100 cm -1 . OH vibrations are obtained around 3500 and 3700 cm -1 . CuO(Si) vibrations are found around 600 cm -1 as well as MnO(Si) vibrations with a slight difference. Both CuO(H) and MnO(H) overlap with SiO(H) bands around 1000 cm -1 . Theoretical results are in nice agreement with experimental ones, providing the localization and speciation of the potential catalytic sites generated by the incorporation of Cu 2+ and Mn 2+ in the zeolite framework. Finally, it is also shown that the accurate molecular description of the active sites is important for understanding the catalysis going on and for developing novel catalytic processes.

Supporting information

IR spectra of the pure and M-modified SiBEA materials

  BEA contains nine different tetrahedral-sites (T-sites)[START_REF] Wojtaszek | Comparison of competition between T=O and T-OH groups in vanadium, niobium, tantalum BEA zeolite and SOD based zeolites[END_REF][START_REF] Newsam | Structural Characterization of Zeolite Beta[END_REF]. There are basically 192 atoms in the unit cell with 64 Si atoms and 128 O atoms. It is known that BEA has 9 distinct crystallographic T-sites. It was chosen to substitute one T site on position 1 because this position has been identified as most stable for incorporation of some metals[START_REF] Wojtaszek | Comparison of competition between T=O and T-OH groups in vanadium, niobium, tantalum BEA zeolite and SOD based zeolites[END_REF][START_REF] Shetty | A comparative study of structural, acidic and hydrophilic properties of Sn-BEA with Ti-BEA using periodic density functional theory[END_REF]. The substitution of Si 4+ by M 2+ leads to a structure with 194 atoms, including 2H + ions to keep electroneutrality. The final substituted T-site contains an M ion surrounded by two Si-OH groups and anchored via 2 Si-O-M linkages, as can be seen in Figure2and Figure4.

Figure 1 .

 1 Figure 1. BEA zeolite unit cell used in the calculation. Blue: Si, Red: O and Green: position of the M ion corresponding to the crystallographic site T1.

Figure 2 .

 2 Figure 2. Different isomers of MSiBEA (M = Cu and Mn) with surrounded groups OSi (in red) and OH (in green).

  1.0 Å. The O-Cu-O bridge varies widely depending on the configurations between 130 and 160 °. The average value calculated over all the configurations is 147°. This value is well above the average of the calculated O-Si-O angle (110°).

Figure 3 .

 3 Figure 3. Optimized geometries of CuSiBEA (Cu atom in gold). Cu-O and H-bond distances are displayed.

Figure 4 .

 4 Figure 4. Optimized geometries of MnSiBEA (Mn atom in purple). Mn-O and H-bond distances are displayed.

Figure 5 .

 5 Figure 5. Calculated IR spectrum for the CuSiBEA zeolite obtained in the range 500 -4000 cm -1 .Assignments in Table3.

Figure 6 .

 6 Figure 6. Calculated IR spectrum for the MnSiBEA zeolite obtained in the range 500 -4000 cm -1 . Assignments in Table4.

Figure 7 .

 7 Figure 7. Experimental IR spectra of Mn2.0SiBEA (A), Cu2.0SiBEA and SiBEA in the range of OH groups (4000-3200) (A) and M-O groups (1000-500 cm-1) (B) is shown.

Table 1 .

 1 Geometric parameters and relative energies of the different Cu(II)SiBEA configurations studied. When two values are available, they are presented by v1/v2 where v1 is the first value and v2 the second. Distances in Å, Angles in ° and Energies in eV.

	Cu(II)SiBEA	Cu-OSi	Cu-OH	Si-OH	O-H	ΔErel
	T2-T3	1.777/1.870 1.991/2.601 1.641/1.673 0.982/0.998 130.2 0.21
	T2-T7	1.814/1.819 2.009/2.761 1.634/1.685 0.984/0.992 155.6 0.11
	T2-T8	1.809/1.810 2.046/2.279 1.646/1.669 0.980/0.998 152.5 0.19
	T3-T7	1.812/1.824 2.046/2.774 1.651/1.663 0.979/0.983 159.5 0.00
	T3-T8	1.797/1.848 2.135/2.264 1.658/1.665 0.977/0.985 148.2 0.08
	T7-T8	1.799/1.878 2.008/2.358 1.640/1.672 0.975/0.976 138.3 0.02

Concerning MnSiBEA, we have considered the same configuration as those used for CuSiBEA. The optimized geometries for MnSiBEA are shown in Figure

4

.

Table 2 .

 2 Geometric parameters and relative energies of the different Mn(II)SiBEA configurations studied. When two values are available, they are presented by v1/v2 where v1 is the first value and v2 the second. Distances in Å, Angles in ° and Energies in eV.

	Mn(II)SiBEA	Mn-OSi	Mn-OH	Si-OH	O-H		ΔErel
	T2-T3	1.896/1.969 2.182/2.204 1.661/1.662 0.992/1.001	124	0.29
	T2-T7	1.893/1.934 2.233/2.288 1.657/1.665 0.983/0.993 153.2	0.09
	T2-T8	1.893/1.897 2.176/2.231 1.661/1.663 0.989/0.994 149.8	0.23
	T3-T7	1.918/1.946 2.188/2.206 1.670/1.672 0.977/0.986 145.9	0.02
	T3-T8	1.913/1.948 2.164/2.241 1.665/1.676 0.981/0.985 137.8	0.06
	T7-T8	1.923/1.978 2.179/2.193 1.663/1.673 0.975/0.980 145.0	0.00

Table 3 .

 3 Calculated vibrational frequencies of CuSiBEA with possible assignment. Frequencies in

	cm -1 . str=stretching, rock=rocking modes
	T2-T3 T2-T7 T2-T8 T3-T7	T3-T8	T7-T8	Experiment	Assignment
	3536.6 3547.6 3609.0 3627.3	3678.8	3722.6	3733	OH str
	3300	3303.9 3214.1 3547.6	3536.4	3683.1	3540	OH str
	1172.1 1190.7 1183.9 1179.4	1171.9	1175		SiO str
	1157.9 1177.2 1162.9 1154.4	1167.3	1161.5	968	SiO str
	1052.7 1036.2	989.0	984	974.5	914.1	-	OH rock
	1028.9	983.8	512.8	944.6	905.2	-	-	OH rock
	990.9	963.9	1048.8	922.2	1012.8	1002.5	-	SiO(Cu) str
	959.1	940.4	1037.3	838.4	964.7	957.3	-	SiO(Cu) str
	887.7	888.5	881.6	916.1	863.2	825.1	810	SiO(H) str
	835.3	808.6	839.3	818.8	831.8	786.4	744	SiO(H) str
	608.8	665.5	613.8	659.3	689.7	665	580	CuO str
	484.6	435.4	520.4	546.5	543.1	502.1	525	CuO str

Table 4 .

 4 Calculated vibrational frequencies of MnSiBEA with possible assignment. Frequencies in

	cm -1 . str=stretching, rock=rocking modes
	T2-T3 T2-T7 T2-T8 T3-T7	T3-T8	T7-T8	Experiment	Assignment
	3412.6 3558.6 3422.8 3687.3	3619.9	3735	3733	OH str
	3163.1	3351	3290.2 3512.2	3501.1	3622.7	3540	OH str
	1191	1176.8 1179.6 1158.5	1167.2	1170.7	-	SiO str
	1166.8 1162.1	1166	1150.1	1165.3	1163.6	968	SiO str
	996.7	967.8	806.4	1000.9	1004.7	957.9	-	OH rock
	804.2	930.9	594.7	969.1	930.4	911	-	OH rock
	868.5	867.2	868.8	847.1	853.8	855.8	810	SiO(H) str
	857.3	850.8	853.2	833.4	829.5	831.4	744	SiO(H) str
	657.7	659.9	-	-	668.3	665.2	625	MnO str
	620.7	647.9	-	-	620.7	636.8	580	MnO str

Acknowledgements

E.H. is grateful to the VUB-ALGC group and to the Global Minds project (Benin -VUB) for financial support. This work was granted access to the HPC resources of TGCC under the allocations 2020-A0080810433 and 2021-A0100810433 by GENCI-EDARI project. Computational resources and services were also provided by the Shared ICT Services Centre funded by the Vrije Universiteit Brussel, the Flemish Supercomputer Center (VSC) and FWO. FT wishes to acknowledge the VUB for support, among other through a Strategic Research Program awarded to his group. F.T. is also grateful to the Université de Lorraine for a one-month visiting professorship in 2019 and 2020.


Valéry Dewil

Arnaud Barral

Gabriele Facciolo

Pablo Arias

Self-supervision versus synthetic datasets: which is the lesser evil in the context of video denoising?

Supervised training has led to state-of-the-art results in image and video denoising. However, its application to real data is limited since it requires large datasets of noisy-clean pairs that are difficult to obtain. For this reason, networks are often trained on realistic synthetic data. More recently, some self-supervised frameworks have been proposed for training such denoising networks directly on the noisy data without requiring ground truth. On synthetic denoising problems supervised training outperforms self-supervised approaches, however in recent years the gap has become narrower, especially for video. In this paper, we propose a study aiming to determine which is the best approach to train denoising networks for real raw videos: supervision on synthetic realistic data or self-supervision on real data. A complete study with quantitative results in case of natural videos with real motion is impossible since no dataset with clean-noisy pairs exists. We address this issue by considering three independent experiments in which we compare the two frameworks. We found that self-supervision on the real data outperforms supervision on synthetic data, and that in normal illumination conditions the drop in performance is due to the synthetic ground truth generation, not the noise model.

Introduction

For both images and videos, denoising is still an active research subject. All the more so in the case of real noise, where the real distribution of the noise may be unknown or at least hard to model. In recent years, data-driven methods based on training convolutional neural networks (CNNs) have taken over the state of the art in several image and video restoration tasks. In addition to their superior performance, CNNs offer a greater flexibility as they can be trained to denoise potentially any type of noise [7,8,22,[START_REF] Wang | Sar image despeckling using a convolutional neural network[END_REF]. In contrast, traditional model-based approaches require a tractable model of the noise, and specific algorithms for each type of noise, e.g. [START_REF] Boulanger | Patchbased nonlocal functional for denoising fluorescence microscopy image sequences[END_REF][START_REF] Coupé | Nonlocal means-based speckle filtering for ultrasound images[END_REF][START_REF] Chang | Two-stage convolutional neural network for medical noise removal via image decomposition[END_REF][START_REF] Lebrun | The noise clinic: a blind image denoising algorithm[END_REF][START_REF] Maggioni | Joint removal of random and fixed-pattern noise through spatiotemporal video filtering[END_REF][START_REF] Salmon | Poisson noise reduction with nonlocal pca[END_REF][START_REF] Zhao | Ratio-based multitemporal sar images denoising: Rabasar[END_REF]. This makes learning-based approaches an ideal candidate for restoration of real videos. Yet, this is still rather unexplored, with most of the research that considers real data focusing on single image denoising. The main reason for this is the lack of available training datasets for video denoising.

The standard approach to data-driven methods is via supervised learning, for which a dataset of pairs of input and expected output is used to train the network. Network architectures trained in a supervised manner yield state-ofthe-art results. However, supervised training requires large datasets with pairs of clean-noisy signals, which are very hard to obtain in the case of real images [START_REF] Abdelhamed | A high-quality denoising dataset for smartphone cameras[END_REF]8,37], and even more so for real videos. The classical solution is to train networks on synthetic datasets where a clean video is artificially degraded. Nonetheless, CNNs are very sensitive to mismatches between the data distributions at training and testing times [37]. Addressing this issue is currently one of the most important problems in learning-based image restoration, and has recently attracted a lot of attention.

One research trend focuses on generating realistic synthetic datasets for supervised training. The noise in the raw sensor is commonly approximated as an additive heteroscedastic Gaussian with a signal dependent variance [START_REF] Foi | Practical poissonian-gaussian noise modeling and fitting for single-image raw-data[END_REF]. A more accurate model is the Poisson-Gaussian model [START_REF] Foi | Practical poissonian-gaussian noise modeling and fitting for single-image raw-data[END_REF], which still has some limitations as it does not take into account non-linear behavior of the sensor (e.g. clipping), dead pixels, heavy tails of read noise, etc. It has been shown that more comprehensive models of the noise yield better results [START_REF] Wei | Physics-based noise modeling for extreme low-light photography[END_REF]. Other works rely on data driven generative approaches to synthesize noise [START_REF] Abdelhamed | Noise flow: Noise modeling with conditional normalizing flows[END_REF][START_REF] Chang | Learning camera-aware noise models[END_REF][START_REF] Chen | Image blind denoising with generative adversarial network based noise modeling[END_REF][START_REF] Kim | Grdn: Grouped residual dense network for real image denoising and gan-based real-world noise modeling[END_REF][START_REF] Wolf | Deflow: Learning complex image degradations from unpaired data with conditional flows[END_REF][START_REF] Yue | Dual adversarial network: Toward real-world noise removal and noise generation[END_REF]. Creating synthetic datasets requires synthesizing the clean data too. This is straightforward for RGB denoising, but it is far from trivial for raw denoising [5, [START_REF] Wang | Practical deep raw image denoising on mobile devices[END_REF]50] or for other imaging modalities.

Another research trend is based on developing selfsupervised approaches that do not require any ground truth, i.e. they can be trained directly on the degraded data. An additional advantage is that, in principle, no complex noise modeling is required in order to apply these methods. The All network architectures are based on UDVD [START_REF] Dev | Unsupervised deep video denoising[END_REF], MF2F (c) uses the self-supervised framework of [START_REF] Dewil | Self-supervised training for blind multi-frame video denoising[END_REF] and blind-spot (d) uses [START_REF] Dev | Unsupervised deep video denoising[END_REF].

(top-brick wall ISO 3200) Self-supervised networks recover more details. (middle top-bench ISO12800) Natural texture of the stones and the granularity of the ground are removed by the supervised network. (middle bottom-trees ISO 3200) Self-supervised networks have a better reconstruction of the texture of the trees (bottom-wire-grid ISO12800) The structure of the wire grid is better reconstructed with the self-supervised networks.

main principle of these techniques is to exploit the signal regularity, and train the network to predict one part of the signal from the rest. Self-supervised methods exist for both images [START_REF] Batson | Noise2Self: Blind denoising by self-supervision[END_REF][START_REF] Dalsasso | Exploiting multi-temporal information for improved speckle reduction of sentinel-1 sar images by deep learning[END_REF]15,[START_REF] Krull | Noise2void-learning denoising from single noisy images[END_REF][START_REF] Moran | Noisier2Noise: Learning to Denoise from Unpaired Noisy Data[END_REF][START_REF] Quan | Self2Self With Dropout: Learning Self-Supervised Denoising From Single Image[END_REF] and videos [START_REF] Dewil | Self-supervised training for blind multi-frame video denoising[END_REF][START_REF] Dev | Unsupervised deep video denoising[END_REF] denoising, demosaicing [START_REF] Ehret | Joint demosaicing and denoising by overfitting of bursts of raw images[END_REF] and super-resolution [34,35]. On artificial datasets, supervised training outperforms self-supervised approaches. However, recent self-supervised methods have shown competitive results, specially in video denoising.

The natural question is then, what is the best approach to train denoising neural networks for real videos? Is it better to train with ground truth supervision on realistic synthetic datasets, or should one train directly on the real data with a self-supervised approach? The former suffers from the generalization gap between simulated and real data, while the latter pays the price of not having supervision from a clean ground truth. The question is which is the lesser evil.

Contribution.

In this paper, we study the question of which training framework has to be used for video denoising networks: supervised on synthetic data or selfsupervised on real data. This requires to compare quantitatively and fairly both approaches in a controlled setting.

Ideally, this should be done by testing them on evaluation datasets of real natural videos with ground-truth. However, there are no such datasets due to the inherent complexity of simultaneously acquiring noisy and noiseless videos for natural dynamic scenes. We circumvent this problem by considering two surrogates for real data: 1) a synthetic raw dataset with a comprehensive noise model, and 2) a real dataset of static scenes for which ground truth can be estimated via frame averaging. Finally, we evaluate both approaches on real natural videos visually. In all cases, we apply a rigorous training methodology to make sure that we compare fairly the training approaches.

The next section reviews the related work. In Section 3, we present the architecture used in this study as well as a description of the self-supervised trainings. The overall protocol of the study (including datasets and training strategies) is detailed in Section 4. Experiments details and results are presented in Section 5.

Related work

Self-supervised training methods are often compared to supervised training on synthetic datasets [START_REF] Batson | Noise2Self: Blind denoising by self-supervision[END_REF][START_REF] Dewil | Self-supervised training for blind multi-frame video denoising[END_REF]15,[START_REF] Krull | Noise2void-learning denoising from single noisy images[END_REF][START_REF] Dev | Unsupervised deep video denoising[END_REF]. In this setting, supervised training is optimal (e.g. with respect to the MSE) and the goal is to achieve the same performance with self-supervision. Our situation is different, since we are interested on the performance on real data of a supervised network trained on synthetic data.

In the case of still images, it is possible to acquire real datasets with ground truth. The ground truth can be either estimated by acquiring a burst of images of static scenes and averaging them [START_REF] Abdelhamed | A high-quality denoising dataset for smartphone cameras[END_REF]8] or using long exposure times [37]. In such datasets it is possible to train with supervision on real data, and it has been observed that training a network with unrealistic simulated data leads to worse results [START_REF] Abdelhamed | Noise flow: Noise modeling with conditional normalizing flows[END_REF][START_REF] Wei | Physics-based noise modeling for extreme low-light photography[END_REF].

This motivated research into how to better simulate real noise. The simplest noise model for raw images is the heteroscedastic Gaussian noise model [29] which supposes the noise to be additive, zero-mean and with a variance as a affine function of the intensity. This corresponds to the sum of two noise sources: the shot noise modeling photons arriving at the sensor and the readout noise introduced by the electronics. In spite of its known limitations, this model is widely used [5,18,[START_REF] Jaroensri | Generating training data for denoising real rgb images via camera pipeline simulation[END_REF][START_REF] Plötz | Neural nearest neighbors networks[END_REF][START_REF] Wang | Practical deep raw image denoising on mobile devices[END_REF]50]. In [START_REF] Claus | Videnn: Deep blind video denoising[END_REF] the authors use an additive and zero-mean heteroscedastic Gaussian noise but the variance does not follow an affine model. In [START_REF] Zhao | Robust principal component analysis with complex noise[END_REF]53] a Gaussian mixture model is used. In [START_REF] Wei | Physics-based noise modeling for extreme low-light photography[END_REF], the shot noise is considered Poissonian and a Tukey-Lambda distribution is used to model heavy tails in the readout noise. Additionally, other noise sources are also modeled like the banding pattern noise (e.g. row noise) or quantization noise.

Other approaches for simulating real noisy sequences use data-driven generative methods, such as adversarial generative models [START_REF] Chen | Image blind denoising with generative adversarial network based noise modeling[END_REF][START_REF] Kim | Grdn: Grouped residual dense network for real image denoising and gan-based real-world noise modeling[END_REF][START_REF] Yue | Dual adversarial network: Toward real-world noise removal and noise generation[END_REF]. In these works, a genera-tive network is trained to generate a noise close to the real one while a discriminative network is trained to determine whether a noise sample is real or has been generated. In [START_REF] Abdelhamed | Noise flow: Noise modeling with conditional normalizing flows[END_REF] a neural network entirely composed of invertible layers is used to simulate realistic noise from clean data. It was trained on the SIDD dataset [START_REF] Abdelhamed | A high-quality denoising dataset for smartphone cameras[END_REF] and can reproduce the realistic noise of the five cameras with a smaller KL-divergence with respect to the real noise than the heteroscedastic Gaussian noise. Similarly in [START_REF] Wolf | Deflow: Learning complex image degradations from unpaired data with conditional flows[END_REF], a CNN is trained to generate realistic degraded data from clean ones.

For raw denoising, it is important also to simulate the raw clean ground truth, a problem that has received less attention. The standard approach is to use sRGB images and apply a simple inverse camera pipeline to generate the raw [5,18]. In [50] the inverse pipeline is implemented by a network that is learned from real data.

Self-supervised training methods

Self-supervised learning methods learn directly from the noisy data by exploiting differences in the correlation structure of signal and noise. A part of the input is withheld from the network, which is trained to predict the withheld part. If the noise of the withheld part is independent from the one given to the network, then the network can only minimize the loss by predicting the clean signal.

The state of the art in self-supervised video denoising is represented by Multi Frame-to-Frame (MF2F) [START_REF] Dewil | Self-supervised training for blind multi-frame video denoising[END_REF] and Unsupervised Deep Video Denoising (UDVD) [START_REF] Dev | Unsupervised deep video denoising[END_REF]. Both techniques were found to be efficient on the real raw video denoising task, and thus we are going to include both of them in our comparison. We will describe them below.

UDVD: blind-spot network for video denoising

The UDVD method relies on the blind-spot technique recently introduced in [START_REF] Batson | Noise2Self: Blind denoising by self-supervision[END_REF][START_REF] Krull | Noise2void-learning denoising from single noisy images[END_REF][START_REF] Krull | Probabilistic noise2void: Unsupervised content-aware denoising[END_REF][START_REF] Laine | High-quality self-supervised deep image denoising[END_REF]: a special convolutional network architecture is used, which has a blind-spot at the center of its receptive field. The network is trained to predict the value of this pixel in the noisy video. It is predicted from the surrounding neighbors (both spatial and temporal), exploiting the spatio-temporal regularity of the clean video. The blind-spot technique decreases the denoising performance compared with a non blind-spot (normal) network, as many details are lost. This gap is significant in the case of image denoising, but it is less discernible for video.

Network architecture. For our experiments, we use the architecture of UDVD [START_REF] Dev | Unsupervised deep video denoising[END_REF]. This video denoising network takes a stack of five frames as input. It consists of two cascaded Unets (as in [START_REF] Tassano | Fastdvdnet: Towards real-time deep video denoising without flow estimation[END_REF]). The first Unet is applied three times on each group of three contiguous frames. This produces three outputs that are fused by a second Unet into the final denoised output. The input stack is rotated by the four The input stack is rotated by multiples of 90°( 1)

(2) (2) (2) (2) (3) (3) (3) (3) (4)
Figure 2. Architecture of the network introduced in [START_REF] Dev | Unsupervised deep video denoising[END_REF]. The input stack is firstly rotated by multiples of 90° [START_REF] Abdelhamed | Noise flow: Noise modeling with conditional normalizing flows[END_REF]. Each four rotated stacks is processed by the cascaded Unets (2), producing four outputs (3) which are combined together after the rotations are inverted and a 1 × 1 convolution (4). multiples of 90°and denoised by the network. The four outputs are finally combined by a 1 × 1 convolution. The architecture is shown in Figure 2.

To generate the blind-spot, the first Unet uses asymmetric convolutional filters that are vertically causal so that the four outputs only depend on the pixels above. In this way, the receptive field does not contain the central pixel. The blind-spot can be removed by shifting the input data one pixel down after the rotation. The UDVD architecture is also bias-free [START_REF] Mohan | Robust and interpretable blind image denoising via bias-free convolutional neural networks[END_REF], which has proved to generalize better to unseen noise level at test time.

UDVD training. The self-supervised UDVD blind-spot network is trained by minimizing the L2 loss between the output of the network and the corresponding noisy input frame.

MF2F training

In MF2F, the weights θ of a network F θ are updated by minimizing the loss ∥κ t • (W t,t-1 F θ (S t ) -f t-1 ) ∥ 1 , where κ t is an occlusion mask, W t,t-1 is a warping operator from frame at time t to t -1 (based on an estimated optical flow), S t is the stack of frames [f t-4 , f t-2 , f t , f t-2 , f t+4 ] and f t-1 is the first past frame serving as target (to prevent the trivial identity mapping, it is out of the input stack). The alignment requires a high quality optical flow plus a mask for alignment errors, which are estimated on the noisy data. The MF2F results strongly depends on the optical flow accuracy, which is computed using the TV-L1 method [START_REF] Sánchez Pérez | Tv-l1 optical flow estimation[END_REF][START_REF] Zach | A duality based approach for realtime tv-l 1 optical flow[END_REF].

The application of the warping operator W t,t-1 requires interpolating the network output at subpixel positions. Interpolating the raw image is problematic. A naive approach would be to pack the raw as a 4 channels image of half the resolution and warp each channel. However, these low resolution channels are heavily aliased. We found better results applying a demosaicing D to the network output, warping on the RGB domain, and re-mosaicing it afterwards. That is, our warping operator can be expressed as

W raw t,t-1 = M W rgb t,t-1 D
, where M is the remosaicing operator. For the demosaicing we use the Hamilton-Adams method [START_REF] Hamilton | Adaptive color plane interpolation in single sensor color electronic camera[END_REF][START_REF] Jin | A mathematical analysis and implementation of residual interpolation demosaicking algorithms[END_REF].

Methodology

Our goal is to compare two strategies for training a denoising network for raw real videos: supervised training on realistic synthetic data, or self-supervised training directly on the real data. To that aim we need a dataset of synthetic noisy videos and one of real natural videos for evaluation. In the following, we describe our evaluation protocol (see Figure 3).

Dataset of real videos. Since there are no datasets of real natural videos with ground truth, we will consider two surrogates: (1) synthetic videos with a comprehensive noise model, and (2) static real videos with ground truth generated by frame averaging. The first will allow us to measure the effect of an oversimplified noise model in the synthetic dataset of dynamic scenes with natural motion. The second is static, but will be useful to have a quantitative evaluation on real data. Lastly, we will consider a dataset of real natural videos for visual evaluation. More details about these datasets will be given respectively in Sections 5.1, 5.2 and 5.3. For simplicity we will talk about the surrogate real dataset (abridged to surrogate dataset) in the following, even though it might not be actually real data, but our proxy for real data. The surrogate real dataset is represented in green in the diagrams of Figure 3.

Dataset of synthetic videos. For each surrogate dataset we generate a synthetic realistic dataset with clean-noisy pairs for supervised training (represented in red in Figure 3). We use the REDS 120 dataset [33] which consists of 270 dynamic sequences (split in 240 training and 30 validation sequences) of outdoors scenes taken in daylight conditions, with frame rate 120 fps and of size 1280 × 720. We temporally downsampled each sequence by taking one frame over three, resulting sequences with 166 frames at 40 fps. Note that this makes the task more complicated for MF2F whose results highly depend on the optical flow estimation accuracy, and therefore on the amount of motion.

These clean sRGB sequences are unprocessed back to the raw domain following [5], which we adapted to our specific case. First, we use a fix color correction matrix throughout all the dataset. We sample random white balance coefficients for each sequence, and use the same coefficients for all frames in the sequence. In the supplementary material, we give more details about our unprocessing procedure steps. This gives us a dataset of clean raw video sequences.

Finally, we add realistic noise to the unprocessed ground-truth for simulating real noisy sequences from the clean ones. For that purpose, a heteroscedastic Gaussian noise model is estimated from the surrogate dataset [29]. More details about the noise estimation can be found in the supplementary material.

Note that the synthetic dataset is tailored to model the surrogate dataset: we use the same Bayer pattern, the ranges of both datasets are matched and the parameters of the synthetic noise model are fitted to approximate the noise in the surrogate dataset.

Networks

We will use for our experiments the UDVD architecture described in the previous section. This network is computationally costly and has a significant memory footprint. In this paper, we do not focus on achieving the state of the art and reduce this architecture by a factor 4 by using 1/4 of the channels in all layers. This architecture can be used with or without the blind-spot.

Due to the small size of the surrogate dataset, we followed [START_REF] Dewil | Self-supervised training for blind multi-frame video denoising[END_REF][START_REF] Yue | Supervised raw video denoising with a benchmark dataset on dynamic scenes[END_REF] and pre-trained the network with supervision on the bigger synthetic dataset.

We pre-trained this architecture with a blind-spot as well as without the blind-spot (denoted as normal). The reason is that we do not need a blind-spot network for applying MF2F as well as for other supervised training strategies discussed later; while the self-supervised UDVD requires a blind-spot.

For comparing the supervised and self-supervised frameworks, we consider different training strategies. Figure 3 summarizes them. Note that once trained, the evaluation of networks trained with or without supervision requires the same amount of time and computational resources. We now describe the different networks and how we trained them.

Model-supervised net This network is trained with supervision on the synthetic dataset. We train two versions of this network: normal and with the blind-spot. The latter will be used as the pre-trained network for the self-supervised blind-spot fine-tuning, while the former is the supervised network trained on synthetic data that we wish to compare with the self-supervised approaches.

Gold standard net

The gold standard solution for such training is to train with supervision directly on the real data. Although this is not possible in practice because it requires to have access to a large dataset with clean-noisy pairs, it is possible here to fine-tune the normal model-supervised net on the surrogate dataset. This will give us a reference of the best training that could be achieved to situate the performance of the other trainings.

Noise-ablation net Two kinds of modeling were used for the supervised trainings: the unprocessing of sRGB to generate synthetic raw clean videos and the noise. When possible, we fine-tune an noise-ablation net that will allow us to differentiate the impact of the noise model from that of the generation of the clean data by eliminating the unprocessing step. To this aim, we generate noisy images by adding synthetic heteroscedastic Gaussian noise to the clean ground truth of the surrogate dataset. We fine-tune the normal model-supervised net on this data with supervision from the real ground truth.

Self-Supervised blind-spot We fine-tune the pre-trained UDVD architecture with blind-spot on the surrogate dataset with self-supervision following [START_REF] Dev | Unsupervised deep video denoising[END_REF].

MF2F net A second self-supervised network is trained following the MF2F framework as explained in the Section 3.2. Given that MF2F does not requires the network to have a blind-spot, we use the weights of the pre-trained normal model-supervised net as starting point of the finetuning.

Experiments

In this section, we first describe the setting of each of the three experiments together with the obtained results for both approaches. For better visualization, the video frames displayed in this section have been gamma corrected (with γ = 2.2), demosaicked with [START_REF] Kiku | Minimized-laplacian residual interpolation for color image demosaicking[END_REF] and white-balanced.

Exp I: dynamic scenes with simulated noise

In this experiment we use the REDS 120 dataset to generate the clean raw data for both the surrogate dataset and the synthetic dataset. The difference lies on the noise model: we use the Poisson-Tukey lambda distribution of [START_REF] Wei | Physics-based noise modeling for extreme low-light photography[END_REF] as noise model for the surrogate dataset. This noise models extreme low-light conditions. In [START_REF] Wei | Physics-based noise modeling for extreme low-light photography[END_REF] the authors provide parameters for three cameras. We use the noise parameters estimated for the Nikon D850. The noise in the synthetic dataset is the heteroscedastic Gaussian with parameters set to approximate the Poisson-Tukey lambda noise of the surrogate dataset. All networks are pre-trained on the training split of the synthetic dataset, and the finetunings are performed using the training split of the surrogate dataset.

Results. The first row of Table 1 summarizes the average PSNR on our surrogate validation set for the different training strategies. The results show that the self-supervised approaches outperform the supervised training in the synthetic dataset: the self-supervised blind-spot network surpasses the model-supervised network by almost 0.7dB and has a much higher SSIM value. The results of the MF2F network have a PSNR similar to the model-supervised, but has a higher SSIM.

From Figure 4, we notice that both self-supervised networks recover more details and have a better reconstruction of the textures. The self-supervised blind-spot is even close to the gold standard. The result of MF2F has a small color shift, which is why it has a lower PSNR. As the heteroscedastic Gaussian noise model does not fully approximate the noise of the surrogate real test set, the modelsupervised net results contain denoising artifacts which decrease its performance. On the contrary the self-supervised networks learn the actual noise of this simulated camera and produce results which compete with the gold standard.

Exp II: real static videos as surrogate data

In the previous experiment, we use artificial ground truth in the surrogate dataset. In this section, we are interested in the comparison between model-supervised and self-supervised on real data. To provide quantitative results, we use the Smartphone Image Denoising Dataset (SIDD) [START_REF] Abdelhamed | A high-quality denoising dataset for smartphone cameras[END_REF], as it has ground truth. It provides images of ten static scenes, taken by five real cameras with different ISO levels, shutter speeds or illuminations levels. For each, The model-supervised with blind-spot serves as a pre-trained for the self-supervised UDVD. In Experience I, the surrogate dataset is synthetic. Thus we cannot derive an noise-ablation net for this experiment as it would be identical to the model-supervised normal. Table 2. Evaluation on the indoor CRVD dataset [START_REF] Yue | Supervised raw video denoising with a benchmark dataset on dynamic scenes[END_REF]. No network was trained on this dataset (even the self-supervised ones).

the authors give an estimated ground-truth image obtained by averaging a burst of frames. We generate a ground truth constant video from the ground truth image, as there is no motion in the scene. We use eight static sequences of about 150 frames each obtained with the Google Pixel camera for ISO level 800. This surrogate dataset is split into six sequences as a fine-tuning pool and two as a testing/validation pool. For both supervised and self-supervised networks, quantitative results are evaluated on the testing pool of this surrogate dataset.

Results. The average PSNR and SSIM on the real validation set are presented in Table 1. As for Experiment I, the trainings with self-supervision lead to a better performance than the trainings done in a supervised setting. On average, the self-supervised blind-spot outperforms the model-supervised by 1.5dB. In Figure 5, the results with the selfsupervised networks are sharper and have more details. Figure 6 shows that the model-supervised network creates also artifacts (see near the text).

In this experiment, the synthetic dataset differs from the surrogate both in the noise model and the ground truth. In order to differentiate this effects, we look at the results of the noise-ablation network, which is trained using clean real data as ground truth, with the simulated heteroscedastic noise. It is remarkable that the result of the noise-ablation network matches exactly with the one of the gold standard (both in PSNR and SSIM). Visual inspection confirms that both results are indeed very similar. We deduce from this that in this case, the heteroscedastic noise model is a good approximation of the real noise, and therefore the problem of the model-supervised network is most likely due to the unprocessed synthetic clean data. On the contrary, in Experiment I, the clean data was the same for both the surrogate and synthetic datasets, thus the failure of model-supervised net was caused by a bad noise modeling.

Exp III: real dynamic scenes

In our final experiment, we will use the dataset introduced in [START_REF] Yue | Supervised raw video denoising with a benchmark dataset on dynamic scenes[END_REF] for a visual comparison. It consists of real noisy raw videos of 10 outdoor dynamic scenes acquired with a surveillance camera for five ISO levels. For such real data, we do not have ground-truth. As before we pre-train the networks on the synthetic REDS 120 dataset with heteroscedastic noise. We considered two ISO levels: 3200 and 12800, and fit the parameters of the noise model to approximate the real noise for each ISO level.

Results. Visual results are shown in Figure 1. In this setting as well, the self-supervised training yields more details leading to a better global reconstruction of the objects.

In [START_REF] Yue | Supervised raw video denoising with a benchmark dataset on dynamic scenes[END_REF], the authors also acquire a dataset of videos with ground-truth of indoor scenes taken with the same camera (denoted CRVD). To simulate motion the authors produced stop-motion videos: the camera is fixed on a tripod and several images are taken for ground-truth estimation via averaging. Then, objects in the scene are slightly moved and the process is repeated to acquire frames. This results in an unnatural motion. As an additional study, we evaluated the previous networks (trained on either the synthetic dataset or the real outdoor data with real motion) on this indoor dataset. No fine-tunings were done to the indoor dataset as it is very small (10 sequences of only 7 frames each). In particular, the self-supervised networks were trained for the CRVD outdoor dataset and all the networks were trained for real motion. This study is another illustration of the network behavior in case of dataset bias. The quantitative results for two ISO levels 3200 and 12800 are gathered in Table 2. For both ISOs, self-supervised outperforms the supervised network.

Conclusion

In this work we propose a protocol to compare in fair conditions two training approaches for denoising real raw videos: supervised training on synthetic data and selfsupervised training on the real data. The difficulty of acquiring real videos with ground truth prevents us for doing a simple comparison. To address this issue, we set three experiments covering different use cases such as low light conditions, real motion, real noise at different ISO levels. In all cases, the self-supervised approaches outperformed the supervised one. Among self-supervised techniques, the blind-spot approach UDVD gave better results than MF2F. The main caveat of UDVD is that blind-spot networks tend to be costlier. MF2F can be used to train any multi-frame network architecture. Our experiments also shed light on how to improve the supervised approach. For normal illumination conditions (such as in the SIDD dataset) the main cause of the generalization gap of supervised training on synthetic data, is not necessarily the simple heteroscedastic Gaussian noise, indicating that more effort needs to be put in better modeling of the clean raw data.

Figure 1 .

 1 Figure 1. Comparison of video denoising networks trained with supervision on synthetic data (b) or self-supervision on real data (c-d).All network architectures are based on UDVD[START_REF] Dev | Unsupervised deep video denoising[END_REF], MF2F (c) uses the self-supervised framework of[START_REF] Dewil | Self-supervised training for blind multi-frame video denoising[END_REF] and blind-spot (d) uses[START_REF] Dev | Unsupervised deep video denoising[END_REF]. (top-brick wall ISO 3200) Self-supervised networks recover more details. (middle top-bench ISO12800) Natural texture of the stones and the granularity of the ground are removed by the supervised network. (middle bottom-trees ISO 3200) Self-supervised networks have a better reconstruction of the texture of the trees (bottom-wire-grid ISO12800) The structure of the wire grid is better reconstructed with the self-supervised networks.

Figure 3 .

 3 Figure 3. The surrogate real dataset is the green cylinder, the synthetic dataset is in red. (a) The model-supervised is trained with supervision on the synthetic dataset with synthetic noise (either with or without blind-spot). (b) The previous model-supervised are finetuned on the surrogate dataset. The steps are (1) fine-tune on real data (2) (when possible) fine-tune on real clean data but with synthetic noise (3) Self-supervised fine-tuning directly on noisy data (UDVD and MF2F).

Figure 4 .

 4 Figure 4. Comparison of the different training strategies for the Exp I.

  Figure 5. Comparison of the different training strategies for the Experience II: the normal network trained with supervision (e) recovers less details than networks trained with self-supervision (see the red rectangles) and does not preserve correctly the colors (colors are washed in the orange and red pens). Furthermore, the model-supervised leaves some residual noise (see the blue rectangles).

Figure 6 .

 6 Figure 6. Comparison of the different training strategies for the Experience II: the model-supervised normal leaves some residual noise (see around the letters).

Table 1 .

 1 Average PSNR and SSIM over all the sequences of the surrogate real test dataset. The model-supervised normal do not have a blind-spot.

			Supervised networks	Self-supervised networks
	Exp	Gold standard Noise-ablation net net	Model-supervised normal blind-spot	MF2F	blind-spot
	I	29.90 / .8393	N/A	28.77 / .7886 27.89 / .8862	28.76 / .8153 29.46 / .9325
	II	50.03 / .9913	50.03 / .9913	47.22 / .9847 46.55 / .9938	47.42 / .9849 48.71 / .9965

Acknowledgments. Work supported by a grant from MENRT. It was also partly financed by Office of Naval research grant N00014-17-1-2552. This work was performed using HPC resources from GENCI-IDRIS (grants 2022-AD011012453R1 and 2022-AD011012458R1) and from the "Mésocentre" computing center of CentraleSupélec and ENS Paris-Saclay supported by CNRS and Région Île-de-France (http://mesocentre.centralesupelec.fr/).