M Orio

J K Bindra

J Van Tol

M Giorgi

N Dalal

S Bertaina

J Vantol

Quantum dynamics of Mn 2+ in dimethylammonium magnesium formate

HAL is

I. INTRODUCTION

Metal-organic frameworks (MOFs) are constituted of two main building units: the framework consisting of metal centers connected to each other by organic linker molecules and the cation molecule in the cavity the dynamics of which are responsible for the dielectric transition. Previous works report hybrid organic-inorganic metal-organic frameworks (MOF), such as [(CH 3 ) 2 NH 2 ][M(HCOO) 3 ] (dimethylammonium metal formate or DMAMF, M is divalent transition metal ions [START_REF] Jain | Order-Disorder Antiferroelectric Phase Transition in a Hybrid Inorganic-Organic Framework with the Perovskite Architecture[END_REF][START_REF] Besara | Mechanism of the order-disorder phase transition, and glassy behavior in the metal-organic framework [(CH3)2NH2]Zn(HCOO)3[END_REF][START_REF] Jain | Multiferroic Behavior Associated with an Order-Disorder Hydrogen Bonding Transition in Metal-Organic Frameworks (MOFs) with the Perovskite ABX 3 Architecture[END_REF][START_REF] Abhyankar | On Mn 2+ EPR Probing of the Ferroelectric Transition and Absence of Magnetoelectric Coupling in Dimethylammonium Manganese Formate (CH 3 ) 2 NH 2 Mn(HCOO) 3 , a Metal-Organic Complex with the Pb-Free Perovskite Framework[END_REF][5][START_REF] Wang | Dimethylammonium copper formate [(CH3)2NH 2]Cu(HCOO)3: A metal-organic framework with quasi-one-dimensional antiferromagnetism and magnetostriction[END_REF][START_REF] Sánchez-Andújar | First-order structural transition in the multiferroic perovskite-like formate [(CH3)2NH2][Mn(HCOO)3][END_REF] . Heat capacity and dielectric measurements of DMZnF indicated a phase transition at approximately 160 K 2 . The order-disorder phase transition is a common property in these materials. At higher temperatures in the disordered phase the (CH 3 ) 2 NH + 2 (DMA + , dimethylammonium) cation that is trapped within the cage is disordered, which means that the nitrogen from the amine group can occupy three locally equivalent positions by forming hydrogen bonds with oxygen atoms from the formate linkers. As the temperature is decreased, the long-range order is established due to the cooperative ordering of the cations at T c . In the low temperature phase the nitrogen atoms freezes in to a single position in the cavity, while the metal-formate framework becomes more distorted [START_REF] Besara | Mechanism of the order-disorder phase transition, and glassy behavior in the metal-organic framework [(CH3)2NH2]Zn(HCOO)3[END_REF] . The Mg analogue (DMMgF), is known to exhibit dielectric transition at exceptionally high T c of approximately 270 K [START_REF] Pato-Doldán | Near room temperature dielectric transition in the perovskite formate framework [(CH3)2NH2][Mg(HCOO)3][END_REF] . The single crystal X-ray diffraction (XRD) studies reveal that the high-and low-temperature phases of these compounds belong to the trigonal, R 3c (centrosymmetric) and the monoclinic, Cc (non-centrosymmetric) space a) Electronic mail: maylis.orio@univ-amu.fr. b) Electronic mail: sylvain.bertaina@im2np.fr. groups, respectively [START_REF] Pato-Doldán | Near room temperature dielectric transition in the perovskite formate framework [(CH3)2NH2][Mg(HCOO)3][END_REF][START_REF] Asaji | Phase transition and cationic motion in the perovskite formate framework [(CH3)2NH2][Mg(HCOO)3][END_REF] . The metal-formate frameworks of these materials consist of pseudo-cuboid cavities, each containing a single DMA+ cation as shwo in Fig. 1. DMMgF has been intensively investigated using heat capacity, dielectric, and XRD measurements [START_REF] Pato-Doldán | Near room temperature dielectric transition in the perovskite formate framework [(CH3)2NH2][Mg(HCOO)3][END_REF][START_REF] Asaji | Phase transition and cationic motion in the perovskite formate framework [(CH3)2NH2][Mg(HCOO)3][END_REF] . However, despite this huge effort the precise nature of the phase transition in DMMgF is still obscured. Among many other experimental methods, electron paramagnetic resonance (EPR) spectroscopy is well-suited to study structural phase transitions [START_REF] Abhyankar | On Mn 2+ EPR Probing of the Ferroelectric Transition and Absence of Magnetoelectric Coupling in Dimethylammonium Manganese Formate (CH 3 ) 2 NH 2 Mn(HCOO) 3 , a Metal-Organic Complex with the Pb-Free Perovskite Framework[END_REF]5,[START_REF] Abhyankar | Magnetic resonance probing of ferroelectricity and magnetism in metalorganic frameworks[END_REF][11][START_REF] Šimėnas | Spectroscopic Study of [(CH3)2NH2][Zn(HCOO)3] Hybrid Perovskite Containing Different Nitrogen Isotopes[END_REF][START_REF] Šimėnas | Electron paramagnetic resonance and electric characterization of a [CH3NH2NH 2][Zn(HCOO)3] perovskite metal formate framework[END_REF] . It used to detect the local environment of a paramagnetic center (e.g., local order parameter such as electric polarization) that can be influenced by the structural transformations. Although most of the MOFs do not contain any intrinsic paramagnetic center, they can be doped with a small amount of paramagnetic transition metal ions (e.g., Mn 2+ ) which act as local probes in the structure. In our previous studies we have employed continuous wave (CW) EPR spectroscopy to successfully investigate the phase transition in undoped DMAMnF and DMAZnF:Mn 2+ MOFs [START_REF] Abhyankar | On Mn 2+ EPR Probing of the Ferroelectric Transition and Absence of Magnetoelectric Coupling in Dimethylammonium Manganese Formate (CH 3 ) 2 NH 2 Mn(HCOO) 3 , a Metal-Organic Complex with the Pb-Free Perovskite Framework[END_REF]5 . However, due to the strong magnetic dipolar and exchange interactions between the Mn 2+ centers the EPR spectrum of DMAMnF consists of a single broad line which is barely sensitive to the ordering of the DMA + cations 4 . Also, slow dynamics of DMA + cation around the phase transitions in DMAZnF:Mn 2+ was investigated using S-Band (4 GHz) EPR. S-Band EPR spectra yielded clear signatures of the slow motion of both the formate and DMA + groups 5 . EPR methods have also been used to characterize the low temperature phases and dynamics in Mn 2+ and Cu

2+ doped niccolite [NH 3 (CH 2 ) 4 NH 3 ][Zn(HCOO) 3 ] 2 12
. Mn 2+ doping in such systems is the probe of choice for the local properties (crystal fields, motion). The large number of EPR transitions as well as the long coherence time allows us to have access to the quantum dynamics of the spins. Large spin ions like Mn 2+ (s = 5/2) is used for their quantum coherence properties as a potential qubit [START_REF] Loss | Quantum computation with quantum dots[END_REF][START_REF] Leuenberger | Quantum computing in molecular magnets[END_REF] . In weak crystal field it was shown that multiple quantum coherence can be induced and controlled [START_REF] Bertaina | Multiphoton Coherent Manipulation in Large-Spin Qubits[END_REF][START_REF] Bertaina | Resonant single-photon and multiphoton coherent transitions in a detuned regime[END_REF][START_REF] Bertaina | Multi-photon Rabi oscillations in high spin paramagnetic impurity[END_REF] . In this paper, by mean of incoherent (continous wave) and coherent (pulsed wave) EPR, we describe the quantum dynamics of DMAMgF:Mn 2+ . We show how the complex quantum dynamics of Mn 2+ in moderate crystal fields can be resolved using a kernel density estimation.

II. MATERIALS AND METHODS

A. Sample preparation

Sample preparation The DMAMgF:Mn2+samples were prepared using the method similar to the one described in detail in our earlier paper with some minor modifications. A 23 mL solution of 50 vol % dimethylformamide (DMF) in Nanopure water into which 85.0 mM MgCl 2 and 2 µM MnCl 2 (0.002%) were dissolved was sealed in a 35 mL pressure vessel. The pressure vessel was then heated to 140 c ircC for 2 days, then allowed to cool to room temperature. Once cool, the supernatant was decanted. Single crystals were obtained by slowly evaporating the solution in a 10 dram glass vial with a 1 in. diameter. We assumed a cuboid shape of the crystal, and the axes of the Cartesian reference frame were chosen such that they coincide with the edges of the crystal. (102) parallel to H and the angular dependent single crystal HF-EPR measurements were performed by rotating the sample about (112) plane.

B. Single crystal X-ray diffraction

A suitable crystal for compound DMAMgF:Mn 2+ was measured on a Rigaku Oxford Diffraction SuperNova diffractometer at 220 K (< T c =260K) at the CuKα radiation (λ=1.54184 Å). Data collection reduction and multiscan ABSPACK correction were performed with CrysAl-isPro (Rigaku Oxford Diffraction). Using Olex2 19 the structures were solved by intrinsic phasing methods with SHELXT [START_REF] Sheldrick | SHELXT -Integrated space-group and crystal-structure determination[END_REF] and SHELXL [START_REF] Sheldrick | Crystal structure refinement with SHELXL[END_REF] was used for full matrix least squares refinement. All H-atoms were found experimentally and their coordinates and Uiso parameters were constraint to 1.5Ueq (parent atoms) for the methyls and to 1.2Ueq (parent atom) for the other carbons.

C. EPR spectroscopy

X-band (about 9.6GHz) EPR measurements were performed using two conventional Bruker spectrometers operating in continuous wave (cw) mode X-band (EMX -9.6 GHz) and pulse mode X-band (E680 9.6 GHz). The cw spectrometer uses a standard 4102ST resonator (TE 102 ) installed in an oxford cryostat ESR900. Low temperature measurements were performed using a cryogen free Bruker Stinger cryocooler allowing the temperature to reach 7K. The angular dependence was measured using an automatic goniometer. Magnetic field modulation (f m = 100 kHz) associated with lock-in detection was employed, resulting in the derivative of the signal. The amplitude of the modulation can be set up to 10 G and was carefully chosen to be below any linewidth to avoid overmodulation effect.

The pulse spectrometer was used to performed Rabi oscillations measurements on Mn 2+ ions 18 using the sequence P Rτ 1π/2τ 2πecho where P R is Rabi pulse which controls the coherent rotation of the spin and τ 1 is a wait time longer than the coherence time in order to destroy the transverse magnetization T 2 but shorter than the relaxation time T 1 maintaining the longitudinal magnetization S z . The later is then recorded by the standard Hahn echo. Du to the large distribution of transitions, the π/2 pulse is selective in DMAMgF:Mn 2+ . However, it is more convenient in field sweep Rabi oscillation sequence since one want to excite just a small quantity of spin (the ones actually in resonance) in the spin packet. The measurements were performed at 7K using the Bruker MD5 dielectric resonator overcoupled. The microwave field h mw was calibrated by measuring the nutation of a S=1/2 standard.

High-field/high-frequency EPR (HF-EPR) experiments have been carried out using a homemade quasioptical superheterodyn setup developed at NHMFL [START_REF] Van Tol | A quasioptical transient electron spin resonance spectrometer operating at 120 and 240 GHz[END_REF] . The spectrometer operates at 240 GHz and at temperature from RT down to 5 K. Angular dependence with respect of magnetic field direction is achieved using a manual goniometer every 18 •

D. DFT

All theoretical calculations were based on the Density Functional Theory (DFT) and were performed with the ORCA program package [START_REF] Neese | Software update: The ORCA program system, version 4.0[END_REF] . To facilitate comparisons between theory and experiments, all DFT models were obtained from the experimental X-ray crystal structures and were optimized while constraining the positions of all heavy atoms to their experimentally derived coordinates. Only the positions of the hydrogen atoms were relaxed because these are not reliably determined from the X-ray structure. Geometry optimizations were undertaken using the GGA functional BP86 [START_REF] Perdew | Density-functional approximation for the correlation energy of the inhomogeneous electron gas[END_REF][START_REF] Perdew | Erratum: Density-Functional Approximation for the Correlation Energy of the Inhomogeneous Electron Gas[END_REF][START_REF] Becke | Density-functional exchange-energy approximation with correct asymptotic behavior[END_REF] in combination with the TZV/P 27 basis set for all atoms and by exploiting the resolution of the identity (RI) approximation in the Split-RI-J variant [START_REF] Neese | Metal and ligand hyperfine couplings in transition metal complexes: The effect of spin-orbit coupling as studied by coupled perturbed Kohn-Sham theory[END_REF] with the appropriate Coulomb fitting sets [START_REF] Weigend | Accurate Coulomb-fitting basis sets for H to Rn[END_REF] . Increased integration grids (Grid4 and GridX4 in ORCA convention) and tight SCF convergence criteria were used. The zero-field splitting parameters were obtained from additional single-point calculations using the BP functional. Scalar relativistic effects were included with ZORA paired with the SARC def2-TZVP(-f) ba- sis sets [START_REF] Pantazis | All-Electron Scalar Relativistic Basis Sets for Third-Row Transition Metal Atoms[END_REF][START_REF] Pantazis | All-Electron Scalar Relativistic Basis Sets for the Lanthanides[END_REF] and the decontracted def2-TZVP/J Coulomb fitting basis sets for all atoms. The spin-spin contribution to the zfs was calculated on the basis of the UNO determinant [START_REF] Sinnecker | Spin-Spin Contributions to the Zero-Field Splitting Tensor in Organic Triplets, Carbenes and Birad-icalsA Density Functional and Ab Initio Study[END_REF] .

III. RABI DISTRIBUTION CALCULATION

To simulate the EPR spectra, we used the following Hamiltonian:

H = µ b H[g] S + S[A] I + k q B q k Ôq k ( S) (1)
Here, the first term represents the Zeeman interaction with [g] the g tensor and µ b is the Bohr magneton, the second represents the hyperfine interaction with the hyperfine constant tensor [A] considered isotropic, while the last term represents the crystal-field interaction in the formalism of the extended Stevens operators [START_REF] Abragam | Paramagnetic Resonance and Hyperfine Structure in the Iron Transition Group[END_REF][START_REF] Ryabov | On the Generation of Operator Equivalents and the Calculation of Their Matrix Elements[END_REF] Ôq k ( S) (k=2, 4, 6 and q = -k, . . . , k ). The B q k are real coefficients with the relations : 3B 0 2 = D the axial anisotropy, B [START_REF] Besara | Mechanism of the order-disorder phase transition, and glassy behavior in the metal-organic framework [(CH3)2NH2]Zn(HCOO)3[END_REF] 2 = E the rhombic anisotropy, 24B 4 4 = a the cubic contribution and F = 180B 0 4 -36B 4 4 the fourth order contribution. k = 6 terms were considered small enough to be neglected. [g] and [A] are considered as scalar (isotropic) for Mn 2+ .

The EPR simulations were performed using a hybrid method. The conventional cw EPR spectra were simulated using the Matlab package Easyspin v5.2.28 [START_REF] Stoll | EasySpin, a comprehensive software package for spectral simulation and analysis in EPR[END_REF] .

The Rabi mode distributions were computed using a database approach. Due to the large anisotropy and the disoriented nature of DMAMgF:Mn 2+ at low temperature, the full dynamical density matrix model developed in ref. [36] for nphoton transitions should have been too heavy to implement and unnecessary. Due to the large zfs expected in this family of compounds 5, [START_REF] Šimėnas | Spectroscopic Study of [(CH3)2NH2][Zn(HCOO)3] Hybrid Perovskite Containing Different Nitrogen Isotopes[END_REF] , only the 1-photon transitions are expected to occur(nphoton transitions are expected when D ∼ h mw [START_REF] Bertaina | Tunable multiphoton Rabi oscillations in an electronic spin system[END_REF] with h mw the microwave field ). Using the first order Fermi golden rule, the Rabi frequency of a transition m to n is

| < m|S + |n > |.
The database of the Rabi frequencies was constructed as the following. The static field orientation is set first, then, after diagonalization of the Hamiltonian, all transitions fields are computed (regardless of the intensity), the Rabi frequency of each transition is calculated by the Fermi's golden rule and the intensity of the transition simply equals the square of the Rabi frequencies. Orientations, resonance fields, transitions and Rabi frequencies are collected in the database. The treatment of the data is then realized using Pandas module of Python 3.8. We use the kernel density estimation (KDE) [START_REF] Rosenblatt | Remarks on Some Nonparametric Estimates of a Density Function[END_REF] to reconstruct the Rabi oscillation distribution:

fh (x) = 1 n n i=1 K h (x -x i ) ( 2
)
where K is the kernel function, h the bandwidth and n the number of samples. This method can be seen as an extension of the histogram method which counts the number of occurrences around a value. In our analysis we used the Gaussian kernel function . Since we have access to the transitions, field resonances and orientations, we can identify the nature and distribution of all transitions.

IV. RESULTS AND DISCUSSION

A. HT Phase

Continuous wave EPR studies have been previously reported for the LT phase of the Zn compound, DMAZnF:Mn 2+11,12 and a particular focus has been made on the dynamics in the HT phase 5 . Here we are interested in determining focus on how the change from Zn to Mg, which highly affects the structural phase transition temperature, can also have effects on the dynamics probed by EPR. Figure 3(a) shows the EPR signal of DMAMgF:Mn 2+ for H [102] at room temperature. The signal is composed of 6 lines corresponding to the m I nuclear spin projection of 55 Mn. The separation between lines is 93.6 G which corresponds to 262.3 MHz or 87.5×10 -4 cm -1 and is a classical value reported for Mn 2+ ions [START_REF] Low | Paramagnetic Resonance Spectrum of Manganese in Cubic MgO and CaF2[END_REF] . In solid-state EPR and for single crystal measurements, the intensity and linewidth of the 6 line pattern is expected to be comparable while in our case the lines at lowest and highest field (corresponding to m I = ±5/2 ) are clearly sharper than the onespresent in the intermediate field region. Moreover no forbidden transition [START_REF] Smith | Angular Dependence of the Intensities of "Forbidden" Transitions of Mn2+ in MgO[END_REF] is resolved while it is usually observed in Mn 2+ ions in anisotropic crystal field. This behavior was also observed 5 in DMAZnF:Mn 2+ and was attributed to the slow motion regime [START_REF] Mclachlan | Line Widths of Electron Resonance Spectra in Solution[END_REF][START_REF] Goldammer | Electron-Paramagnetic-Resonance Study of Manganese Ions Bound to Concanavalin A[END_REF][START_REF] Goldammer | Electron paramagnetic relaxation and EPR-line shapes of manganese ions in a sulfonated polystyrene ion-exchange resin at various degrees of hydration[END_REF] the DMA + movement around the Mn 2+ ion. Using this model [START_REF] Mclachlan | Line Widths of Electron Resonance Spectra in Solution[END_REF] we managed to simulate the experimental data within DMAZnF:Mn 2+ . Indeed, by decreasing the temperature, the DMA + slow down and when its correlation time τ c is longer than the time scale of the measurement (i.e. > 1/f mw ), the system is considered frozen and can be described by the rigid model. However, in DMAMgF:Mn 2+ , T c (263 K) is higher than in the Zn counterparts (173 K) and the structure of the system changes before reaching the frozen regime. To observe the frozen regime we have to increase significantly the frequency. Fig. 3(b) shows the EPR signal of DMAMgF:Mn 2+ recorded at f mw =240 GHz and at room temperature. Contrary to the X-band measurements, all of the six nuclear isotope lines for transitions m S = -1/2 ↔ 1/2 have the same intensity suggesting that we have reached to rigid limit of EPR. However, whatever the orientation of crystal is, no satellite lines corresponding to m S = ±5/2 ↔ ±3/2 and m S = ±3/2 ↔ ±1/2 were resolved indicating a large distribution of the crystal field parameters. To simulate the spectrum of Fig. 3(b) we used the crystal field parameters of the DMAZnF:Mn 2+ compound 5 D = B 2 0 /3=250 MHz but we have to set a distribution of the crystal field parameter ∆D=260 MHz which seems inconsistent since ∆D is usually less than 10% of D. To explain this large value of ∆D it is worth to mention that, in the HT phase, the system is locally highly disordered. Indeed, the Mn 2+ ion is surrounded by 8 DMA + cations which all have 3 different positions giving rise to 3 8 = 6561 configurations of the crystal field which are responsible for the large zfs distribution.

To support and rationalise the experimental findings about ∆D, DFT calculations were conducted. To do so, we employed a methodology similar to that from our previous study on DMAZnF:Mn 2+44 and worked with a minimal model consisting in one Mn 2+ ion bound to 6 formate anions and surrounded by 8 DMA + cations 45 . The resulting metal cluster displays a quasi-octahedral coordination geometry. Based on the high temperature single crystal XRD structure of DMMgF that identified three equivalent positions of the nitrogen in each DMA+, we have considered several configurations in which the Mn-N distances for 6 DMA + can take values of 4.495, 5.121 and 5.679 Å while the 2 remaining DMA + display Mn-N distances of 5.698 Å. This provides a random sampling of the different situations and allows to determine the distribution of the zfs parameter, which was found to be ∆D ≈ 234 MHz. The computed value is in fair agreement with the experimentally estimated value of 260 MHz and our calculations adequately reproduce the increased value for the zfs distribution when comparing DMAMgF:Mn 2+ to DMAZnF:Mn 2+ (∆D DF T ≈ 125 MHz and ∆D exp ≈ 150 MHz). While there is no clear evidence for an effect from the Mg, our results confirm the influence from the DMA + cations on the crystal-field effect on the Mn 2+ as observed in the case of DMAZnF:Mn 2+ . Fig. 4 shows the temperature dependence of the peak-peak linewidth ∆H pp of the smallest field line (m I = -5/2) for DMAMgF:Mn 2+ (this work) and DMAZnF:Mn 2+ (from Ref.

[5]) in the HT phase.

In DMAZnF:Mn 2+ , T c =170 K while in DMAMgF:Mn 2+ T c =260 K that's why the temperature range of the HT phase in DMAMgF:Mn 2+ is more limited. In X-band and in slow regime, the temperature dependence of ∆H pp is the barely the same in the two compounds showing that the correlation time τ c of the DMA + cation is weakly affected by the nature of the metal, Zn or Mg, as pointed out in Ref.

[? ]. In the rigid limit the linewidth of the m S = -1/2 ↔ 1/2 transition in DMAMgF:Mn 2+ is about two times smaller than in DMAZnF:Mn 2+ at high frequencies although the working frequency used to investigate DMAMgF:Mn 2+ was 240 GHz while it was 34 GHz for DMAZnF:Mn 2+ . The distribution of the crystal field does not affect this transition, and a distribution of g-factor should have as effect to increase the linewidth when the frequency is increased.

B. LT Phase

FIG. 5.

First derivative if the EPR spectrum of DMAMgF:Mn 2+ recorded on a powder sample, at T = 100 K and fmw = 9.6 GHz (red line). The best simulation obtained using the parameters reported in the main text is presented in blue.

When the temperature is below T c =263 K, the system undergoes a structural phase transition from disordered trigonal R 3c to ordered monoclinic Cc. Fig. 5 shows the EPR spectrum of DMAMgF:Mn 2+ powder at T=100 K. The signal is now more resolved than in the HT phase due to an apparent decreasing of the ∆D value. Using eq.( 1) we simulate the spectrum with the following parameters : B [START_REF] Abhyankar | On Mn 2+ EPR Probing of the Ferroelectric Transition and Absence of Magnetoelectric Coupling in Dimethylammonium Manganese Formate (CH 3 ) 2 NH 2 Mn(HCOO) 3 , a Metal-Organic Complex with the Pb-Free Perovskite Framework[END_REF] 4 =0.5 MHz and A=264 MHz. While second order crystal field parameters are provided with a good approximation, the fourth order ones have to be taken with caution since the transitions m S = ±5/2 ↔ ±3/2 and m S = ±3/2 ↔ ±1/2 are not perfectly resolved. Nevertheless, the crystal field parameters are in the range of what is expected for this family of material. [START_REF] Šimėnas | Spectroscopic Study of [(CH3)2NH2][Zn(HCOO)3] Hybrid Perovskite Containing Different Nitrogen Isotopes[END_REF] The crystal field distribution ∆D is about 50 MHz, much less than the one needed to simulate the HT phase. In the LT phase, the DMA + cations now have only one possible orientation induced by the ferroelastic phase. The residual ∆D is thus due to local inhomogeneity. Fig. 6 shows the angular dependence of the X-band EPR signal of DMAMgF:Mn 2+ recorded at T =100 K with a resolution of 1 • . Most of the transitions are between 3000 G and 3500 G and are therefor impossible to resolve. However, the highest anisotropic transitions, m S = ±5/2 ↔ ±3/2, are visible on the edges of the spectra. The triangles point to the maximum of resonance fields and are found at 18 • and 108 • for the blue ones and at 55 • and 145 • for the red ones. The angular separation between these two magnetic substructures is 53 • which corresponds to the angle between the two MnO 6 orientation resolved by XRD (see Fig. S1 45 ). It is worth noticing that, in DMAZnF:Mn 2+ , six substructures was necessary to describe the angular dependence of the EPR [START_REF] Šimėnas | Electron paramagnetic resonance and electric characterization of a [CH3NH2NH 2][Zn(HCOO)3] perovskite metal formate framework[END_REF] . Single crystal XRD data obtained at 100 K reveals the presence of a two component twin with a minor domain of only 5% weight. (Fig. S2 45 ).

B 0 2 =110 MHz, B 2 2 =10 MHz, B 0 4 =0.5 MHz, B 2 4 =0.7 MHz,
To confirm that DMAMgF:Mn 2+ has mostly a single ferroelastic domain, we performed high frequency/field EPR (240 GHz ≡ 8.87 T) at T =5 K. At this temperature and for this field, the Boltzmann statistic populates mostly the lowest energy levels (m S = -5/2). The transitions m S = -5/2 ↔ -3/2 are mainly visible, while the others are either weak or absent. Thus, the spectra are "cleaned" and easier to read. Fig 7 shows the angular dependence of DMAMgF:Mn 2+ at f = 240 GHz. Blue and red triangles represent the resonance fields and are useful to follow the angular variation of the resonance field of the two magnetic substructures. We can clearly distinguish the 2 sub-magnetic systems. The small signals at 90 • and 108 • are due to thermal population of the m S = -3/2 state which is caused by temperature instability in the cryostat.

While EPR studies [START_REF] Šimenas | Single Crystal Electron Paramagnetic Resonance of Dimethylammonium and Ammonium Hybrid Formate Frameworks: Influence of External Electric Field[END_REF] has clearly shown the presence of three crystallographic twins of two magnetic domains in DMAZnF:Mn 2+ , we observe only a single domain of two magnetic structures. If one consider that changing the metal ion in DMAXF from Zn to Mg modify the elastic property of the MOF, the same arguiment can be used to explain the dramatic increase of T c in the DMAMgF compared to DMAZnF.

Pulsed EPR

Despite our efforts to resolve all EPR lines in DMAMgF:Mn 2+ using low and high frequency EPR, the large number of lines in Mn 2+ makes it difficult to identify them completely. By means of pulsed EPR, we measured the field sweep Rabi oscillations. The Rabi spectroscopy adds another dimension to the EPR spectrum. While for a fixed field, the EPR intensity might contain many transitions with unresolved contribution, the Rabi frequency of each transition is often unique. On Fig. 8(a) we show the 1D echo field and on Fig. 8(b) we show the contour plot of the field sweep fast Fourier transform (FFT) of Rabi oscillations recorded on a single crystal of DMAMgF:Mn 2+ at T = 7 K with h mw = 4.8 G. The red dashed line corresponds to the frequency expected for a S = 1/2 spin. Clearly, the 2D Rabi field sweep helps to resolve many more transitions. The distribution in the frequency dimension is due to the damping of the oscillation while the distribution in the field dimension is due to the ∆D. Qualitatively, the broadly distributed frequencies are related to the m S = ±5/2 ↔ ±3/2 and m S = ±3/2 ↔ ±1/2 transitions which are sensitive to ∆D while the narrow distributed ones are related to m S = -1/2 ↔ 1/2 which are insensitive to the crystal field. Moreover, in first approximation, the Rabi frequency in a large spin system is given by the adapted Fermi Golden rules [START_REF] Schweiger | Principles of Pulse Electron Paramagnetic Resonance[END_REF] :

F R (S, m S ) = S(S + 1) -m S (m S + 1) × F R (S = 1/2) (3) such as : F R (S = 5/2, m S = -5/2) = √ 5 × F R (S = 1/2) = 30.2 MHz, F R (S = 5/2, m S = -3/2) = √ 8 × F R (S = 1/2) = 38.8 MHz and F R (S = 5/2, m S = -1/2) = √ 9 × F R (S = 1/2) = 40.5 MHz
While the agreement with this model is fair (but not exact) for m S = ±5/2 ↔ ±3/2 and m S = ±3/2 ↔ ±1/2 it fails for m S = -1/2 ↔ +1/2. To understand the distribution of Rabi frequencies in DMAMgF:Mn 2+ we developed a kernel density estimation of all transitions and all orientations in the crystals. Using the crystal field parameters found in the previous section, we computed the fields of resonance for each (allowed and forbidden) transitions accessible from H =260 mT to H =430 mT. For each pair of resonance field/transition we computed the amplitude of transition probability (A tr ) in the direction perpendicular to the field orientation that is imposed by the experimental condition since the cavity force the microwave polarization to be perpendicular to the static field. The Rabi frequency is then F R = gµ b h mw h A tr , with g is the g-factor (close to 2), µ b the Bohr magneton, h the Planck constant and h mw the microwave field. This series of Rabi frequencies is then computed for all orientations. We discretize the space using an icosphere to avoid overdensity of orientations by using a simple equally spaced Euler's angles. The calculation included 20609 orientations and using a threshold of 10 -2 to suppress the far too low frequency Rabi oscillations we obtained 1 346 898 sets of data containing resonance fields, probability amplitude, transition identification and Euler angles.

We first describe the Rabi and field distribution for the "allowed" transitions (∆m S = ±1 and ∆m I = 0 ). Fig. 9 shows the Rabi frequencies and magnetic field distributions for the 30 allowed transitions. First, the field distribution follows what we expected for a Mn 2+ ion in a moderate crystal field : m S = ±5/2 ↔ ±3/2 are spread on large distribution of the field, m S = ±3/2 ↔ ±1/2 are slightly less distributed while m S = -1/2 ↔ +1/2 are fairly insensitive to the crystal field and induce the narrow lines observed on Fig. 8. More surprisingly, the Rabi frequency distribution is less intuitive. The transitions m S = ±5/2 ↔ ±3/2 are weakly distributed around 2.2 × F R (S = 1/2) whatever m I is, confirming the valid- FIG. 9. Rabi frequency (left) and resonance field (right) distributions calculated in DMAMgF:Mn 2+ using the crystal field parameters extracted from CW measurements for all allowed transitions (∆m S = ±1 and ∆m I = 0). The frequency distribution is presented vertically to help the comparison with the experimental data. The unit is set to be proportional to the Rabi frequency of the S = 1/2 isotropic caliber. ity of Eq. ( 3) while the m S = ±3/2 ↔ ±1/2 transitions are different. m I = ±5/2 also show a weak frequency distribution around 2.8 × F R (S = 1/2) but m I = ±3/2 and m I = ±1/2 are much more distributed. The transitions m S = -1/2 ↔ +1/2 are even more sensitive to the orientation. In this case the Rabi frequency distribution is very broad and dependent to the m I value while the resonance field distribution is essentially independent to the orientation. This explains why the m S = -1/2 ↔ +1/2 transition is narrow in the field dimension but distributed in a large range of Rabi frequency (see Fig. 8) . Now we consider the case of Rabi frequencies and resonant field distributions of the "forbidden" transi- tions. The calculation method is based on the first order Fermi's golden rule and so only one photon is involved in the resonance mechanism. The multiple photon transitions [START_REF] Bertaina | Multiphoton Coherent Manipulation in Large-Spin Qubits[END_REF][START_REF] Bertaina | Resonant single-photon and multiphoton coherent transitions in a detuned regime[END_REF][START_REF] Sorokin | Multiple Quantum Transitions in Paramagnetic Resonance[END_REF] ∆m S > 1 are not taken into account. However such transitions are induced only when h mw ∼ D which is far from being the case here. The "forbidden" transitions that we are considering are thus ∆m S = ±1 and ∆m I = 0 . It appears that the transitions m S = -1/2 ↔ 1/2 and m I = -1/2 ↔ 1/2 (see SI) are weakly distributed in field, which is expected for such transition but we also observe that the Rabi frequency distribution is centered at 1.6 ×F R (S = 1/2) making these transitions highly probable.

To confront the model, we have measured the field sweep Rabi oscillations of a powder of DMAMgF:Mn 2+ . After some signal processing (baseline correction, hamming, zero filing) the FFT is presented in Fig. 10 (a). We then simulated the distribution of both frequencies and resonance fields of a Mn 2+ ion with the crystal field parameters extracted from CW-EPR of DMAMgF:Mn 2+ and a microwave field of h mw =1.8 G. Our simulation describes rather well the experimental data. We should note that the apparent slope in the Rabi frequencies around 15 MHz in Fig. 10(a) is du to the nuclear zeeman of protons ω N (42.57 MHz/T ) which induce an amplification of the Rabi intensity in case of Hartman-Hahn conditions [START_REF] Hartmann | Nuclear Double Resonance in the Rotating Frame[END_REF] (ω N ∼ Ω R ). Without fitting parameters we can describe the experimental data but the strength of this method is to help of the identification of transitions. Fig. S6 and S7 show the frequency and field distribution of all principal transitions.

To simulate the single crystal field sweep Rabi frequency distribution presented in Fig. 8 we just replaced the Euler angles sweep by the crystal field parameter D distribution in order to simulate the broadening. We used a Gaussian distribution with width of 40 MHz corresponding to the strain of D extracted from EPR measurements. We then calculate the field sweep Rabi frequency distribution for two orientations separated by 58 • which correspond to the angle between the 2 magnetic domains observed in Fig. 6 and presented in Fig. S8. The agreement between theory and experiment (Fig. 8) is fairly good thus confirming the presence of two magnetic structures that are disoriented by about 60 • . We also note that very low modes at about 2.5 MHz are visible experimentally but not displayed in our model. We believe these modes are related to the recently observed quantum rotor tunneling of methyl group in DMAZnF [START_REF] Šimėnas | Magnetic excitation and readout of methyl group tunnel coherence[END_REF] which has not been taken into account in our model.

V. CONCLUSION

We employed the electron spin resonance technique to investigate the dynamics of the electron spin of a Mn 2+ ion used as a probe in the multiferroic compound DMAMgF. In the high temperature phase, the X-band cw-EPR study demonstrated that the correlation time describing the motion of DMA + in DMAMgF:Mn 2+ is similar to that observed in the Zn analogue. The high frequency EPR data revealed a large zfs distribution in DMAMgF:Mn 2+ . Using a wide range of configurations and DFT computations, we were able to support this finding and fairly estimate this distribution. In the low temperature phase, we observed a single elastic phase containing two magnetic structures in agreement with single crystal XRD analysis. The complex EPR structure observed in the LT phase was then solved using pulse EPR combined to the field sweep Rabi oscillations method. A model based on the crystal field and the kernel density estimation of all possible transitions and orientations finally provided an accurate description of the complicated EPR structure of DMAMgF:Mn 2+ .

AUTHOR'S CONTRIBUTIONS

SB and ND designed and directed the study. JKB synthetize the samples. JKB and JVT conducted the HF-EPR measurements. SB performed the X-band EPR measurements. MG performed the XRD measurements. MO carried out the DFT calculations. MO and SB conducted the theoretical analysis and wrote the paper with input from all authors. All authors contributed to the implementation of the research and to the analysis of the results.

FIG. 1 .

 1 FIG. 1. Crystal structure of the DMAMgF framework at T > Tc (left, HT phase -R 3c) and T < Tc (right, LT phase -Cc). The Mg (in blue) and Mn probe (in pink) are in the center of the oxygen octahedron. The DMA + are located in the formate cages and have three equivalent positions in the HT phase, represented by 3 nitrogens (green) while in the LT phase only one position is expected. For clarity the hydrogen atoms are not shown. The figures are realized with Blender a using the structure investigated in this work. a 3D open source software : https://www.blender.org/

FIG. 3 .

 3 FIG. 3. (color online) (a)First derivative with respect to the field of the EPR signal of a single crystal of DMAMgF:Mn 2+ recorded at X-band and room temperature (red). The blue line below represents the simulation of the spectrum in the slow motion regime with a correlation time τc = 2 × 10 -9 s. (b) EPR signal recorded on the same crystal at fmw=240 GHz and at room temperature. The blue line is the rigid limit spectrum simulation using a distribution of axial anisotropy ∆D.

FIG. 4 .

 4 FIG.4. Peak-to-peak linewidth of the rightmost EPR line in DMAMgF:Mn 2+ , recorded between 300 and 260 K at X-band and 240 GHz. The data for DMAZnF:Mn 2+ are extracted from Ref.[5]. The vertical dashed line is the structural transition temperature.

FIG. 6 .

 6 FIG. 6. Angular dependence of the EPR signal of a single crystal of DMAMgF:Mn 2+ recorded at T = 100 K and fmw = 9.6 GHz. The blue and red triangles show the extreme resonance field of the m S = ±5/2 ↔ ±3/2 transitions (the only well resolved) and help to identify the two magnetic structures.

FIG. 7 .

 7 FIG. 7. (top) Angular dependence of the HF-EPR spectra of a single crystal of DMAMgF:Mn 2+ recorded at T = 5 K. Blue and red triangles indicates the resonance field. Temperature stability was not perfect and satellite lines from m S = -3/2 ↔ -1/2 can be seen at 90 • and 108 • . (bottom) Angular dependence of the resonance fields. The squares represent the resonance fields and the lines correspond the simulations of the resonance field using Eq. (1) .

FIG. 8 .

 8 FIG. 8. (a). Echo field sweep of a single crystal of DMAMgF:Mn 2+ at T = 7 K. (b) Fast Fourier Transform of the field sweep Rabi oscillations for the same orientation with a pumping pulse of strength hmw =4.8 G. Red (Blue) is the presence (absence) of frequency density. The dashed red line represents the frequency expected for S = 1/2 at the same pulse amplitude.

FIG. 10

 10 FIG. 10. (a) Fast Fourier Transform of the field sweep Rabi oscillations for powder sample of DMAMgF:Mn 2+ . A pumping pulse of strength hmw =1.8 G was used . Red (Blue) is the presence (absence) of frequency density. The dashed red line represents the frequency expected for S = 1/2 at the same pulse amplitude. (b) Simulation of Rabi oscillations in a powder using the crystal field parameters of DMAMgF:Mn 2+ .

ACKNOWLEDGMENTS

Pulsed ESR measurements were supported by the Centre National de la Recherche Scientifique (CNRS) research infrastructure RENARD (Grant No. IR-RPE CNRS 3443). We thank the international research program of CNRS -PICS SomeTIME. The NHMFL is supported by the NSF Cooperative Agreement Grant No. DMR-1157490 and the State of Florida.

DATASETS

The data that support the findings of this study are openly available in Zanodo at http://doi.org/10.5281/zenodo.4521882, reference number 4521882.

SUPPLEMENTARY INFORMATION

See supplementary information for XRD data and structure description, detail about DFT minimal models and simulation of rabi field sweep distributions.


Tangui Le Gléau
email: tangui.legleau@orange.com

Xavier Marjou
email: xavier.marjou@orange.com

Tayeb Lemlouma
email: tayeb.lemlouma@irisa.fr

Benoit Radier
email: benoit.radier@orange.com

A Multi-agent OpenAI Gym Environment for Telecom Providers Cooperation

Keywords: Multi-agent system, simulation frameworks, RAN sharing, Game Theory

The ever-increasing use of the Internet (streaming, Internet of things, etc.) constantly demands more connectivity, which incentivises telecommunications providers to collaborate by sharing resources to collectively increase the quality of service without deploying more infrastructure. However, to the best of our knowledge, there is no tool for testing and evaluating participation strategies in such collaborations. This article presents a new adaptable framework, based on the OpenAI Gym toolkit, allowing to generate customisable environments for cooperating on radio resources. This framework facilitates the development and comparison of agents (such as reinforcement learning agents) in a generic way. The main goal of the paper is to detail the available functionalities of our framework. We then focus on game theory aspects as multi-player games induced by these environments can be considered as sequential social dilemmas. We show in particular that although each agent has no incentive to remain cooperative at each step of such iterated games, a mutual cooperation provides better outcomes (in other words, Nash Equilibrium is non optimal)

I. INTRODUCTION

More and more connectivity is necessary to respond to the needs due to the explosion of Internet uses (streaming, Internet of Things, autonomous vehicles, telemedicine etc). One major challenge of connectivity providers is to cope with the increase in data traffic given that deploying new infrastructure leads to important financial and environmental costs. In cellular networks, an attractive solution is to share resources between Mobile Network Operator (MNO), for example using roaming, radio access network sharing or radio access network slice [START_REF] Turner | Inception: Digital twins for 5g network infrastructuresharing[END_REF]. More recently, marketplaces of connectivity have also been considered, involving financial transactions such as trading mechanisms or auctions whereas other solutions may rely on dynamic exchanges and fair cooperation between providers. In mobile cellular networks, two major use-cases for cooperation are generally considered: network coverage extension and network capacity extension. In the former case, some users may have no connectivity due to a lack of coverage of their home provider's infrastructure. In the latter case, there may be too many users with respect to the network cell capacity of their home provider. To overcome these problems, a provider may want to collaborate with one or more other providers to benefit from their resources resulting in closer or less loaded cells for the users in difficulty as in [START_REF] Tavares | Beyond mobile-network sharing: Regulatory challenges in dense urban areas[END_REF]. In some situations as in Figure 1, the providers may even envisage scenarios based on reciprocity and fair share of resources, so that each provider equally benefits from the collaboration.

Before collaboration After collaboration

MNO A MNO B

Fig. 1: Principle of multi-MNO collaboration: by mutually taking charge of their partner's user, each MNO decreases its cell load and optimises the connectivity.

To evaluate and identify a participation strategy in such collaborating scenarios, multi-agent learning offers compelling perspectives as it computes the optimal transactions between players. In particular, the recent emergence of Multi-Agent Reinforcement Learning (MARL) in non-cooperative games is very promising [START_REF] Lerer | Maintaining cooperation in complex social dilemmas using deep reinforcement learning[END_REF]- [START_REF] Lowe | Multi-agent actor-critic for mixed cooperative-competitive environments[END_REF]. However, the study of Reinforcement Learning (RL) requires an environment for each specific task. To compensate the absence of such an environment for our use-case, we introduce a novel framework to create some fully adaptable RL environments for the simulation of multi-provider cooperation. We decided to adopt the OpenAI Gym toolkit which is a reference in the study of RL agents [START_REF] Brockman | Openai gym[END_REF].

The paper is structured as follows. First, we explain in section II some useful concepts for a better understanding. Then in section III, we introduce our framework and describe in details its adaptable parameters. In section IV, we focus briefly on multi-agent interaction, in particular showing that this kind of cooperation falls into what is called a social dilemma. We conclude in section V in particular about further works.

978-1-7281-7705-2/21/$31.00 ©2021 IEEE

II. BACKGROUND

In this section, we explain some specific notions: Telecom vocabulary, Markov processes and the specifications of the Gym Framework.

A. Vocabulary of Telecom providers 1) MNO: In our framework, an agent (or player) represents a Telecom Provider called a Mobile Network Operator (MNO).

2) Sites: Each MNO owns a set of base stations also called cell sites. Moreover, we consider that each site is divided into a fixed number of sectors which delimit the so-called cells. We use the term cell for these sectors (see Figure 2). Usually, the number of cells is fixed to 3 but we will see later that it can be modified.

3) Users: In the environment, some users (or userequipments) stand around the sites. Each user is a customer of one and only one provider (MNO) and is covered by the closest site owned by its provider.

4) Radio resources:

For each site, and for each cell, the MNO can share a part of its resources. Red, Green and Blue. There are one site (owned by Red) and nine users (three per MNO). The colour of a dot identifies the user's home MNO (i.e. the MNO to which the user is subscribed) ; the colour of a dotted line identifies the user's visited MNO (i.e. the MNO providing the radio resource)

B. Partially Observable Markov Decision Processes

In the literature, Reinforcement Learning environments are commonly formalised as Partially Observable Markov Decision Processes (POMDP) which are a generalisation of Markov Decision Processes (MDP) [START_REF] Bellman | A markovian decision process[END_REF]. POMDP are usually defined as a 6-tuple (S, A, T, R, ⌦, O) where:

• S is a set of states • A is a set of actions • T : S ⇥ A ! [0, 1] is a (stochastic) transition function • R : S ⇥ A ! R is a reward function • ⌦ is a set of observations • O : S ⇥ A ⇥ ⌦ ! [0, 1] is an (stochastic) observation function
The goal is to find a policy ⇡ : s 7 ! a so that the cumulated reward R t is maximised.

POMDP can be solved by exact methods like value-iteration algorithms when the model is known thanks to the Bellman equations. When the model is unknown and when the spaces are not too high, it is necessary to use RL algorithms such as the classical Q-Learning or REINFORCE [START_REF] Richard S Sutton | Introduction to reinforcement learning[END_REF]. Finally, when states and/or actions sets are very high or even continuous, an approximation function can be used, in particular using neural networks which made the emergence of Deep Reinforcement Learning [START_REF] Mnih | Human-level control through deep reinforcement learning[END_REF].

C. Gym environments

OpenAI Gym is a toolkit [START_REF] Brockman | Openai gym[END_REF] which allows hosting RL environments and interacting with them through Reinforcement Learning strategies. Therefore Gym is practical to evaluate and compare Reinforcement Learning agents in a generic way. A Gym environment is synthesised by four main application programming interface (API) functions:

1) env.init(): the environment is set with the initial set (which can be randomised) 2) env.reset(): the same than the previous function but after initialling at least once the environment 3) env.step(state, action) = [new_state, reward, done, info] : the most important function among the four, it allows to execute an action to move to another state. Doing a step, the agent receives its reward and an "end of episode" Boolean (done) and some optional information. 4) env.render(): allows to see a visual of current state of environment

III. OUR MULTI-AGENT FRAMEWORK

In this section, we present our Telecom Providers Cooperation environment 1 .

To be integrated as an OpenAI Gym environment, and therefore benefit from Gym's flexibility, our environment implements the four API functions mentioned above. In addition, we also describe its configurable features as well as the major provided functions.

A. Environmental features

In our framework, some preset environments are available which are described in III-C but one can also create specific one in which we can modify the following parameters :

1) Number of agents: n_agents : In the context, the number of agents (or players) is the number of MNOs.

2) Positions of sites: positions_sites is a list of n_agents lists containing the (x, y) fixed positions of sites 3) Number of cells: n_cells determines the number of cells a site can provide in a given frequency band. In most deployed networks, 3 is a typical value, but next generation networks like 5G networks may review this significantly.

4) Positions of users: User equipments can be considered fixed or mobile (see III-A5). In all cases, the number and initial positions of users have to be defined. (x, y) positions have to be defined in the list positions_users. A function called random_users is also available to define random positions. At last, each user has a fixed affiliation to one and only one player (provider). This is defined in the list clients containing the MNO identifier for each user.

5) Mobility of users:

Users can be defined motionless or mobile. For the latter case, we created a class Kinematics which is used to define a profile of movement for users. The preset type called 'line' is a simple line movement with constant speed 2 .

6) Discretization of observation:

A vector-type observation can be useful for some agent algorithms like those using Deep RL. Therefore an option of the environment allows to convert observation in multi-layer grid-world which fits to many Deep RL (such as CNN) algorithms (Also see III-B2)

B. Gym Features

In this section, we describe the POMDP features (section II-B) of our Gym toolkit (II-C).

1) Actions: At each step, each agent (MNO) plays an action which is the partition of its resources for each cell of each site. Formally, an action is an numpy array a of shape (n_sites, n_cells, n_agents) where:

8i S , 8k C , N X iP =1 a[i S , k C , i P ] = 1 (1)
with N the number of MNOs (players) and i S , k C , i P respectively the indices of sites, cells and players.

For example, Figure 3 displays four examples of environment env_3A_3S_9U-v2, where the action performed by the first agent (Red agent) was respectively: 2 2 An example of mobility for the environment env_3A_5S_30U-v0 can be viewed here : https://www.youtube.com/watch?v=ZfCgvPOmUoc with the cell's index incrementing in anticlockwise direction beginning by the right one.

2) Observations:

The observation state is subject to discussion. A multi-agent with partial observation is considered as non-Markovian which often requires to create a state with the history of actions. Such a state is created within algorithms [START_REF] Hausknecht | Deep recurrent q-learning for partially observable mdps[END_REF], [START_REF] Lin | Memory approaches to reinforcement learning in non-Markovian domains[END_REF], that is why environments only need to provide current observation. We then propose a state available in two versions: total and partial observations. It provides a bitmap showing the positions of the sites and users. Such a bitmap is suited for most of Deep RL algorithms using Convolutional Neural Networks [START_REF] Mnih | Human-level control through deep reinforcement learning[END_REF], [START_REF] Mnih | Playing atari with deep reinforcement learning[END_REF]. In the figure 4, we show the two kinds of observations in an example. Fig. 4: Total and partial observations of example environment env_3A_3S_9U-v1 (shown in Figure 3)

3) Transition function: The users are connected to sites/cells according to the shares that players allocate to each other. To compute the link, we decided to adopt a simple greedy algorithm. The links are decided in an increasing order of distances of all tuples user-sites (u, s) when the proper cell of s allocates enough resources to u's provider. Note that, although this allocation is not optimal, it is rather realistic to model such a cooperative environment. In addition, a more optimal solution would be NP-hard and an optimisation with good computational performance is out of the scope of the paper. Figure 3 shows links in the same environment with different allocations.

4) Rewards:

For the reward of our Gym environment, we propose to use a reward (or utility) function R P for each player P (provider) inspired from [START_REF] Mohammad T Kawser | Downlink snr to cqi mapping for different multipleantenna techniques in lte[END_REF].

R P = X u2CP u m u exp( D u D max ) (2)
where :

• C P is the set of customers of P

• u is the share of radio resources allocated for provider of u (P ) in the cell • m u is the number of customers of P standing in the cell of u • D u is the distance between u and its connecting antenna, D max is a scaling constant

C. Examples of predefined environments

Our environment can be instantiated with any combination of players (MNOs), sites and users. For example, Figure 5 shows four simple canonical environments with two or three players, one or two sites per player and one to ten users. These environments are directly available in the framework.

IV. GAME THEORY ANALYSIS Though implementing agents is out of the scope of the paper, we found interesting to focus on Game Theory aspects. After explaining some definitions, we will show that multiplayer games in our environments can be considered as a social dilemmas.

A. Game Theory and Stochastic Games

Multi-agent interaction leads to game theory issues of which we explain here some basics.

1) Strategy and Utility:

A strategy (also called policy) is basically a function that maps personal states (s (i) 2 S) to actions (a (i) 2 A) : ⇡ i : s (i) 7 ! a (i) . In a game theory approach, each agent (or player) is driven by a utility function which measures the satisfaction of a strategy. A given strategy can be based on the gain, money, popularity, number of points, quality of service (for telecom), etc.

2) Nash Equilibrium: A major notion of Game Theory is the Nash Equilibrium which can be simplified as a situation where every agent has no incentive to change alone its strategy. Formally, the Nash Equilibrium is defined as follows: if

G (i) (⇡ i ) is the payoff of agent A i , a joint strategy (⇡ ⇤ i ) i2I is said to be a Nash Equilibrium if [14]: 8i 2 I, 8⇡ i , G (i) (⇡ ⇤ i , ⇡ ⇤ i ) G (i) (⇡ i , ⇡ ⇤ i ) with ⇡ i = [⇡ 0 , ..., ⇡ i 1 , ⇡ i+1 , ..., ⇡ N 1 ] (3)
3) Stochastic games: Stochastic games are an extension of POMDP (II-B) with multiple decision makers [START_REF] Lloyd | Stochastic games[END_REF]. N agents I = {1, ..., N} play in the same POMDP game where S and A are extended to product sets ⇥ i2I S i and ⇥ i2I A i . Each agent i 2 I receives a distinct reward r (i) that it wants to independently maximise.

B. Social Dilemmas

As seen in previous works [START_REF] Tangui | Game theory approach in multi-agent resources sharing[END_REF] for a similar use-case, we show here that the games we consider in the paper can be considered as sequential social dilemmas [START_REF] Joel Z Leibo | Multi-agent reinforcement learning in sequential social dilemmas[END_REF]. We explain in this section its principle. Let us consider the case with N = 2 players (providers) and with simply two possible actions: cooperate and defect. The figure 6 shows the four possible payoffs. Mutual defection is the Nash Equilibrium while the optimal outcome is the mutual cooperation. For further details, see [START_REF] Joel Z Leibo | Multi-agent reinforcement learning in sequential social dilemmas[END_REF] Such a game is called social dilemma if the following inequalities are verified:

• R > P (1) • R > S (2)
• at least one of these two inequalities:

-T > R: greed (3a) -P > S: fear (3b) (1) means that mutual cooperation is better than mutual defection and (2) that mutual cooperation is better than being exploited. Formally, a social dilemma is a game which admits at least one non-optimal Nash equilibrium in particular (Defect, Defect) in Prisoner's Dilemma (where greed (3a) and fear (3b) are verified). Social dilemmas exist with more than two players. In the figure 7b, we represent a three-player game modelled by the environment env_3A_3S_18U-v0 (an available predefined environment) with the reward function detailed in III-B4. We emphasis the dilemma with a Schelling diagram since payoffs tables are illegible with more than two players. Diagram (for a better understanding, see [START_REF] Joel Z Leibo | Multi-agent reinforcement learning in sequential social dilemmas[END_REF] or [START_REF] Thomas | Micromotives and macrobehavior[END_REF]). Red and Blue are cooperating with each other and Green cooperates only with Red.

V. CONCLUSION

Providers may consider cooperating to meet exploding connectivity needs. To do so, they might adopt a simple game theory approach without financial transactions where players (providers) make transactions so that they all increase their quality of service. These assumptions then lead among other things to the study of agents behaviour in a context of telecom resources sharing. To anticipate the lack of some realistic environment where agents can learn to optimally interact, we implemented a framework able to generate some customizable RL environments in the OpenAI Gym toolkit. The main goal of this article is to introduce the provided features of our framework. We additionally presented a brief study of game theory situation of the problem. We showed in particular that our considered exchange example can be view as a so-called social dilemma, a situation where players have no incentive to collaborate alone despite the optimal mutual cooperation. In further works, we plan to generate environments derived from our framework to study algorithms known to be robust for multi-players social dilemmas.

Fig. 2 :

 2 Fig.2: Example of environment with three players (MNOs): Red, Green and Blue. There are one site (owned by Red) and nine users (three per MNO). The colour of a dot identifies the user's home MNO (i.e. the MNO to which the user is subscribed) ; the colour of a dotted line identifies the user's visited MNO (i.e. the MNO providing the radio resource)

Fig. 3 :

 3 Fig. 3: Four instances of a simple env_3A_3S_9U environment (3 Red/Green/Blue players each owning 1 site and 3 users). Fig 5b is one optimal cooperation. Fig 5d shows a situation where only one player (red one) agree to cooperate

Fig. 6 :

 6 Fig.6: A two-player stochastic game in situation of Sequential Social Dilemma : symmetric payoffs verify S < P < R < T . Mutual defection is the Nash Equilibrium while the optimal outcome is the mutual cooperation. For further details, see[START_REF] Joel Z Leibo | Multi-agent reinforcement learning in sequential social dilemmas[END_REF]

Fig. 5 :

 5 Fig. 5: Some examples of environments already available : a very simple two-players prisoner dilemma case (a), some three-players ones (b) and (c) with circular and reciprocal cooperation, and finally a random three-players one (d)

Fig. 7 :

 7 Fig.7: A three-players social dilemma shown by a Schelling Diagram (for a better understanding, see[START_REF] Joel Z Leibo | Multi-agent reinforcement learning in sequential social dilemmas[END_REF] or[START_REF] Thomas | Micromotives and macrobehavior[END_REF]). Red and Blue are cooperating with each other and Green cooperates only with Red.

The source code of our framework is available on GitHub: https://github.com/tlgleo/gym-MNOs-cooperation