Dr Virgile Trannoy

Albano N Carneiro Neto

Carlos D S Brites

Luís D Carlos
email: lcarlos@ua.pt

Dr Hélène Serier-Brault

Dr A N Carneiro

Dr C D S Neto

Pr L D Brites

Phantom- Carlos

Engineering of mixed Eu 3+ /Tb 3+ metal-organic frameworks luminescent thermometers with tunable sensitivity

Keywords: metal-organic frameworks, luminescent thermometers, europium, terbium, Tb-to-Eu energy transfer

In the last decade, numerous Ln-bearing metal-organic frameworks are reported for luminescence thermometry applications. Although its Ln 3+ composition is always thoroughly determined, this parameter is never optimized to improve thermometric performances. Here, we tackle the optimization of thermometric performances of luminescent probes by reporting a series of mixed Eu 3+ -Tb 3+ metal-organic frameworks. The thermometric performances are accessed as a function of the Eu 3+ content yielding to a maximum relative sensitivity between 0.19 and 0.44 %•K -1 registered at temperatures between 340 and 240 K, respectively. A meticulous theoretical investigation of the Tb 3+ -to-Eu 3+ energy transfer in the series of mixed Eu 3+ -Tb 3+ metal-organic frameworks is also performed to determine the predominant pathway of the energy transfer. For the first time, we present clear evidence the significant influence of the Eu/Tb ratio on the energy transfer between Ln 3+ emitting centers that definitively determines the operating temperature range and the maximum relative sensitivity of the luminescent thermal probes.

Introduction

Metal-Organic Frameworks (MOFs) are crystalline porous materials built up from metallic nodes and organic linkers that have attracted great interest due to their wide range of possible applications. [1][2][3][4][5] Effectively, the wide variety of chemical composition, structural diversity, conjugated with its intrinsic adjustable porosity makes these materials promising candidates not only for purely sorption-related applications but also in other areas where their tunable optical or electronic properties can be exploited for diverse applications. [6][7][8][9] The lanthanide-bearing MOFs (Ln-MOFs) is a sub-class of MOFs whose optical properties may arise either from the organic or from the inorganic part, have emerged as unique and promising luminescent thermometers. [10][11][12][13][START_REF] Brites | Handb. Phys. Chem. Rare Earths[END_REF][START_REF] Rocha | [END_REF][16][17] Indeed, compared to conventional contact thermometry, luminescent thermometry have unique and distinct advantages of fast response, high accuracy, being noninvasive and presenting high spatial resolution (typically submicron scale) where traditional methods are ineffective. [18][START_REF] Savchuk | Sensors Actuators[END_REF][20] Among the diverse aspects to Ln 3+ -based luminescent thermometry, one of the most robust methodologies relies on the measurement of the intensity of two transitions of distinct Ln 3+ emitting centers. For mixed-metal Ln-MOFs, the temperature is most often determined from the ratio between the intensity of the 5 D4→ 7 F5 and of the 5 D0→ 7 F2 transitions of Tb 3+ and Eu 3+ , respectively. [START_REF] Rocha | [END_REF]21,22] Since the first report of the ratiometric Eu-Tb-mixed MOF luminescent thermometer, [21] the number of materials displaying potential for luminescent thermometry increased drastically, notably because of the inexhaustible selection of suitable organic ligands. However, not many works have been devoted to identifying the structural parameters which can govern the finetuning of the thermometric performance parameters, namely the operating temperature range and the relative thermal sensitivity. Recently we investigated the thermometric properties of an Eu-Tb mixed MOF built upon the bidentate linker 1,3-benzene-dicarboxylic acid (1,3-H2bdc, also called isophthalic acid), namely [Tb0.87Eu0. 13(1,3-bdc)3(H2O)2], presenting the maximum relative thermal sensitivity in the cryogenic range (T<100K). [23] The 1,3-H2bdc linker is a bidentate ligand with different geometric effects to connect metal ions into multidimensional structures via numerous coordination modes. Consequently, several MOFs based on this ligand have been already reported with different structuration, despite their thermometric performance have not always been studied. [24][25][26][27][28] Herein we reported the study of another Eu-Tb mixed isophthalate MOF, [Tb1-xEux(CH3COO)(1,3-bdc)(H2O)], using isophthalic acid as the linker, which contains Ln2 SBUs as inorganic nodes in a 3D framework and with an ancillary ligand. [24][25][26] Aiming at the rational design of the operating temperature range and relative thermal sensitivity of isophthalate MOFs, we investigated the luminescence properties of a series of six distinct Eu-Tb compositions (the Eu 3+ content varying from 3 to 15 mol.%). To gain some rational on the design of these intriguing samples, we use theoretical simulations of the Tb 3+ -to-Eu 3+ energy transfer for correlating the chemical composition with the thermometric properties and determine what is the main Tb 3+ -to-Eu 3+ energy transfer pathway which is determinant for the rational engineering of the compounds looking forward to fine-tuning of the maximum relative sensitivity and the corresponding temperature at which it occurs.

Structural and chemical characterization

Three single-doped compounds, Ln-CP (Ln=Eu, Tb, or Gd) were obtained in moderate yields as a white microcrystalline powder using hydrothermal conditions. The powder X-ray diffraction (PXRD) analysis (Figure S1) and FTIR measurements (Figure S2) reveal all coordination polymers are isostructural with the Ln(Ac)(ip)(H2O)20.5H2O compounds (Ln=Gd, Sm, Nd, Sm, Eu, La) reported in the literature. [24][25][26] The asymmetric unit is composed of symmetric dimers of Eu 3+ cations, six 1,3-bdc 2-ligands, two acetate ligands, and four coordinated water molecules (Figure 1a,b). Each Ln 3+ is 9-coordinated in tricapped trigonal prismatic geometry formed by nine O-atoms from three different 1,3-bdc 2-ligands, two distinct acetate ligands, and two water molecules. Both [LnO9] polyhedra are bridged by two acetate O-atoms to generate the Ln2 SBU of an edge-sharing polyhedron, where the Ln-Ln distance is 4.35 Å in a dimer. Each dimer is also connected to six 1,3-bdc 2-ligands, playing the role of the organic spacer with other neighboring dimers to generate a three-dimensional coordination polymer, the smallest Ln-Ln distance between two Ln2 SBU units is 5.78 Å. The thermal (Figure S3) and elemental analyses confirmed the chemical composition of single doped compounds.

The mixed compounds denoted by Tb1-xEux (x = 0.030, 0.050, 0.075, 0.100, 0.125, and 0.150, the obtained molar ratio was determined by ICP-AES, Table 1S) were obtained pure in moderate yields as a white microcrystalline powder. Both PXRD analysis (Figure 1c) and FT-IR spectroscopy (Figure S4) confirm that the mixed compounds are isostructural with the pure compounds.

Photophysical properties

The luminescence properties were investigated in the solid-state at room temperature. Upon excitation at 289 nm, the emission spectra of Tb-CP (Figure S5) and Eu-CP (Figure S6) display the characteristic green and red luminescence of Tb 3+ and Eu 3+ ions, respectively, exhibiting typical lines at 488, 542, 585 and 619 nm, for Tb-CP, and at 582, 616, 650 and 692 nm, for Eu-CP. The emission lines are attributed to the Tb 3+ 5 D4→ 7 F6-3 transitions and the Eu 3+ 5 D0→ 7 F1-4 transitions, respectively. Furthermore, the emission bands of the 1,3-bdc 2-ligand are completely absent in the luminescent spectra of both compounds, evidencing an efficient sensitization of both Tb 3+ and Eu 3+ ions by the organic ligand. The phosphorescence spectrum at 77 K (Figure S7) of Gd-CP was recorded to identify the lowest-lying triplet energy state positioning of the organic ligands. Then, the triplet level is estimated to be 26040 cm -1 from the shortest-wavelength, which is in a suitable energy range to sensitize both Eu 3+ and Tb 3+ according to the Latva's empirical rule. [29] The emission spectra of the mixed compounds exhibit the intra-4f transitions of the Tb 3+ and Eu 3+ ions (Figure S8). Increasing the Eu 3+ content the emission color shifts for x = 0.030, pure green emission (similar to that of the Tb-CP compound), to essentially pure red when x = 0.150 (as registered for the Eu-CP), (Figure S9 and Table S2). This shift in the color coordinates is due to the Tb 3+ -to-Eu 3+ energy transfer, which was evidenced by the presence of 7 F6→ 5 D4 Tb 3+ transition (at 483 nm) within the 5 D0→ 7 F2 Eu 3+ transition in the excitation spectra of a mixed compound (Figure S10). The 5 D4 (monitored at 545 nm) and 5 D0 (monitored at 615 nm) decay curves of single-doped and codoped samples are well described by single exponential functions, (Figure S11 to S13).

In the mixed compounds, the Tb 3+ -to-Eu 3+ energy transfer rate (Wexp) and the transfer efficiency (ηET) were determined from Equation 1 and 2, respectively: [30][31][32][33] 𝑊𝑊 exp = 1 𝜏𝜏 -1 𝜏𝜏 0

( 1 )

𝜂𝜂 𝐸𝐸𝐸𝐸 = 1 - 𝜏𝜏 𝜏𝜏 0 ( 2 )
where 𝜏𝜏 and 𝜏𝜏 0 correspond to the lifetimes of the donor (Tb 3+ ) in the presence and absence of the acceptor (Eu 3+ ), respectively (Table S3 to S5 and Table 1). The obtained values are reported in Table 1 and Figure S14. Then, as suggested by the rapid evolution of CIE coordinates with the increase of Eu 3+ (Figure S9), the Tb 3+ -to-Eu 3+ energy transfer is relatively high and varies from 77%, for x = 0.030, to 93%, for x = 0.150 (Figure S15).

Luminescent thermometry

The temperature-induced change on the emission spectrum motivated the application of the mixed compounds for luminescent thermometry. As an illustrative example, Figure 2a presents the temperature-dependent emission spectra of the compound with x = 0.150 in the 150-350 K range (similar data of all the other mixed compounds are reported in Figure S16 to S20). The integrated areas of the 5 D4→ 7 F5 (Tb 3+ ), I1, and 5 D0→ 7 F2 (Eu 3+ ), I2, transitions were calculated in the 530-560 nm and 603-633 nm ranges, respectively. As five consecutive emission spectra were recorded for each sample and each temperature, the mean values of the areas ± SD were reported. While I1 corresponds to only the 5 D4→ 7 F5 transition (the 5 D1 Eu 3+ emission is not discerned, Figure S6 and S21), the I2 integration range includes a small (ca. 6%) contribution from the Tb 3+ 5 D4→ 7 F3 transition (Figure S21). The contribution of the Eu 3+ emission to I2 is, thus, highly dominant (>93%) and both I2(T) and the thermometer performance is unaffected by the presence of the 5 D4→ 7 F3 transition (Figure S22). This is valid for all the Tb 3+ contents studied in this work and, therefore, in what follows we consider the commonly assumed labelling of I2 as being only due to the 5 D0→ 7 F2 contribution.

The integrated areas of the transitions originated in both Ln 3+ centers present a clear thermal quenching (Figure 2b) compared to the thermal quenching present in the Tb-CP and Eu-CP (Figure S11 and S12). Whereas I2 decreases about 10% of its initial value, I1 decreases almost 50% upon temperature increase (the same is valid for all the other samples). As the temperature dependence of the I1 is much higher than that of I2, we consider that the temperature dependence of the I1/I2 ratio is determined essentially by the thermal quenching of the Tb 3+ transition (I1).

We use the phenomenological Mott-Seitz expression for a single deactivation channel to describe the I1(T) dependence: [34,35] 𝐼𝐼 1 (𝑇𝑇) =

𝐼𝐼 01 1 + 𝛼𝛼 exp (-∆𝐸𝐸 𝑘𝑘 𝐵𝐵 𝑇𝑇) ⁄ ( 3
)
where I01 is the integrated area in the limit of T→0 K, α is the ratio between the nonradiative and radiative decay rates in the limit of T→0 K, and ∆E is the energy difference between the emitting level ( 5 D4) and the level responsible for its nonradiative deactivation. The ratio between the integrated areas of the I1 and I2 transitions is thus described by: [START_REF] Brites | Handb. Phys. Chem. Rare Earths[END_REF] Δ ≡ 𝐼𝐼

1 𝐼𝐼 2 ≈ Δ 0 1 + 𝛼𝛼 exp (-∆𝐸𝐸 𝑘𝑘 𝐵𝐵 𝑇𝑇) ⁄ ( 4
)
where ∆0 is the I1/I2 ratio in the limit T→0 (∆0=I1/I02). Fitting Equation 4 to the experimental ∆(T) curves we find excellent correlation coefficients attesting the adequacy of this simple model to get the calibration curve of the mixed compounds (Figure S23). The fitting parameters for the Tb1-xEux compounds are listed in Table 2 and show a decrease of ∆E and an increase of α as the Eu content increases (Figure 2d). As the Tb 3+ -to-Eu 3+ energy transfer efficiency is independent of the temperature (Figure S14), we suggest that the quenching of the Tb 3+ and Eu 3+ emission intensities as temperature increases (Figure 2b) is due to the activation of nonradiative channels. In the former case, Tb 3+ -to-Eu 3+ energy transfer is responsible for the observed temperature dependence, as addressed in the next section, whereas the small temperature dependence of the 5 D0→ 7 F2 intensity should be related to the activation of Eu 3+to-ligand states back transfer. This aspect, however, is not discussed in this manuscript.

To evaluate the thermal performance of the mixed compounds, we calculate the relative thermal sensitivity using: [START_REF] Brites | Handb. Phys. Chem. Rare Earths[END_REF] 𝑆𝑆

r = Δ𝐸𝐸 𝑘𝑘 𝐵𝐵 𝑇𝑇 2 � 𝛼𝛼 𝛼𝛼 + exp(Δ𝐸𝐸/𝑘𝑘 𝐵𝐵 𝑇𝑇) � . ( 5 )
The corresponding uncertainty (∆Sr) is calculated using home-made MatLab® routines by propagating the uncertainty in the α and ∆E fitting parameters. Intriguingly, the increase of the Eu content on the mixed compounds influences its thermometric performance. By increasing the amount of Eu 3+ from x = 0.030 to 0.150, a shift of 100 K towards low temperatures is observed on Tm, together with an increase in Sm from 0.19±0.02 to 0.44±0.02 %⋅K -1 . We estimate the uncertainty in Tm determining the temperature range for which Sr>Sm-∆Sr. This variation on the relative thermal sensitivity is explained by the increase of the α and the decrease of ∆E with the increase of the Eu 3+ content (Table 2), which means that the nonradiative rate increases relative to the radiative one, and consequently the temperature-dependence is more pronounced. These sensitivity values are comparable to the few examples existing in the literature of luminescent ratiometric MOF-based thermometers operating in the biological temperature range. [36][37][38][39][40][41][42] The temperature uncertainty of the thermometers is calculated using: [START_REF] Brites | Handb. Phys. Chem. Rare Earths[END_REF] 𝛿𝛿𝑇𝑇 = 1

𝑆𝑆 𝑟𝑟 𝛿𝛿𝛿𝛿 𝛿𝛿 = 𝑘𝑘 𝐵𝐵 𝑇𝑇 2 Δ𝐸𝐸 � 𝛼𝛼 + exp(Δ𝐸𝐸/𝑘𝑘 𝐵𝐵 𝑇𝑇) 𝛼𝛼 � 𝛿𝛿𝛿𝛿 𝛿𝛿 ( 6 )
where δ∆/∆ is the relative uncertainty on the thermometric parameter, assumed as 0.1% (the minimum relative uncertainty in ∆ recorded in all the samples). [START_REF] Brites | Handb. Phys. Chem. Rare Earths[END_REF] The temperature cycling between 150 and 300 K (Figure S24) demonstrated a repeatability value higher than 99.4% at 300 K. As we will show in the next section, the dependence of Sm with the Eu 3+ amount is well described by Tb 3+ -to-Eu 3+ non-radiative energy transfer processes.

Theoretical Tb-Eu energy transfer

In this section, it is presented a theoretical procedure to estimate the Tb-to-Eu energy transfer rates based on crystallographic structure and the theory of nonradiative energy transfer between lanthanide ions. [43,44] From the crystallographic data, one can obtain the arrangement of the host sites that can be occupied by Eu 3+ (acceptor) or Tb 3+ (donor) ions, as shown in Figure 3a-c.

The energy transfer rates between lanthanide ions were calculated taking into account the dipole-dipole (𝑊𝑊 𝑑𝑑-𝑑𝑑 ), dipole-quadrupole (𝑊𝑊 𝑑𝑑-𝑞𝑞 ), quadrupole-quadrupole (𝑊𝑊 𝑞𝑞-𝑞𝑞 ), exchange (𝑊𝑊 𝑒𝑒𝑒𝑒 ), and magnetic dipole-magnetic dipole (𝑊𝑊 𝑚𝑚𝑑𝑑-𝑚𝑚𝑑𝑑 ) mechanisms, [31] Equation 7 to 11, respectively:

𝑊𝑊 𝑑𝑑-𝑑𝑑 = (1 -𝜎𝜎 1 𝐷𝐷 ) 2 (1 -𝜎𝜎 1 𝐴𝐴 ) 2 (2𝐽𝐽 𝐷𝐷 * + 1)(2𝐽𝐽 𝐴𝐴 + 1) 4𝜋𝜋 3ℏ 𝑒𝑒 4 𝑅𝑅 𝐿𝐿 6 �� 𝛺𝛺 𝐾𝐾 𝐷𝐷 𝐾𝐾 ⟨𝜓𝜓 𝐷𝐷 𝐽𝐽 𝐷𝐷 �𝑈𝑈 (𝐾𝐾) �𝜓𝜓 𝐷𝐷 * 𝐽𝐽 𝐷𝐷 * ⟩ 2 � × �� 𝛺𝛺 𝐾𝐾 𝐴𝐴 𝐾𝐾 ⟨𝜓𝜓 𝐴𝐴 * 𝐽𝐽 𝐴𝐴 * �𝑈𝑈 (𝐾𝐾) �𝜓𝜓 𝐴𝐴 𝐽𝐽 𝐴𝐴 ⟩ 2 � 𝐹𝐹 ( 7 ) 𝑊𝑊 𝑑𝑑-𝑞𝑞, 𝑞𝑞-𝑑𝑑 = �1 -𝜎𝜎 1 𝐷𝐷,𝐴𝐴 � 2 �1 -𝜎𝜎 2 𝐴𝐴,𝐷𝐷 � 2 (2𝐽𝐽 𝐷𝐷 * + 1)(2𝐽𝐽 𝐴𝐴 + 1) 𝜋𝜋 ℏ 𝑒𝑒 4 𝑅𝑅 𝐿𝐿 8 ⟨𝑓𝑓�𝐶𝐶 (2) �𝑓𝑓⟩ 2 × ��� 𝛺𝛺 𝐾𝐾 𝐷𝐷 𝐾𝐾 ⟨𝜓𝜓 𝐷𝐷 𝐽𝐽 𝐷𝐷 �𝑈𝑈 (𝐾𝐾) �𝜓𝜓 𝐷𝐷 * 𝐽𝐽 𝐷𝐷 * ⟩ 2 � 〈𝑟𝑟 2 〉 𝐴𝐴 2 ⟨𝜓𝜓 𝐴𝐴 * 𝐽𝐽 𝐴𝐴 * �𝑈𝑈 (2) �𝜓𝜓 𝐴𝐴 𝐽𝐽 𝐴𝐴 ⟩ 2 + �� 𝛺𝛺 𝐾𝐾 𝐴𝐴 𝐾𝐾 ⟨𝜓𝜓 𝐴𝐴 𝐽𝐽 𝐴𝐴 �𝑈𝑈 (𝐾𝐾) �𝜓𝜓 𝐴𝐴 * 𝐽𝐽 𝐴𝐴 * ⟩ 2 � 〈𝑟𝑟 2 〉 𝐷𝐷 2 ⟨𝜓𝜓 𝐷𝐷 * 𝐽𝐽 𝐷𝐷 * �𝑈𝑈 (2) �𝜓𝜓 𝐷𝐷 𝐽𝐽 𝐷𝐷 ⟩ 2 � 𝐹𝐹 ( 8 )
𝑊𝑊 𝑞𝑞-𝑞𝑞 = (1 -𝜎𝜎 2 𝐷𝐷 ) 2 (1 -𝜎𝜎 2 𝐴𝐴 ) 2 (2𝐽𝐽 𝐷𝐷 * + 1)(2𝐽𝐽 𝐴𝐴 + 1) 28𝜋𝜋 5ℏ 𝑒𝑒 4 𝑅𝑅 𝐿𝐿 10 〈𝑟𝑟 2 〉 𝐷𝐷 2 〈𝑟𝑟 2 〉 𝐴𝐴 2 ⟨𝑓𝑓�𝐶𝐶 (2) �𝑓𝑓⟩ 4 × ⟨𝜓𝜓 𝐷𝐷 𝐽𝐽 𝐷𝐷 �𝑈𝑈 (2) �𝜓𝜓 𝐷𝐷 * 𝐽𝐽 𝐷𝐷 * ⟩ 2 ⟨𝜓𝜓 𝐴𝐴 * 𝐽𝐽 𝐴𝐴 * �𝑈𝑈 (2) �𝜓𝜓 𝐴𝐴 𝐽𝐽 𝐴𝐴 ⟩ 2 𝐹𝐹 ( 9 )
𝑊𝑊 𝑒𝑒𝑒𝑒 = 2𝜋𝜋 ℏ �� 𝑒𝑒 2 𝑅𝑅 𝐿𝐿 � 𝜌𝜌 𝑓𝑓-𝑓𝑓 2 � 2 𝐹𝐹 ( 10 )
𝑊𝑊 𝑚𝑚𝑑𝑑-𝑚𝑚𝑑𝑑 = (1 -𝜎𝜎 1 𝐷𝐷 ) 2 (1 -𝜎𝜎 1 𝐴𝐴 ) 2 (2𝐽𝐽 𝐷𝐷 * + 1)(2𝐽𝐽 𝐴𝐴 + 1) 4𝜋𝜋 3ℏ 𝜇𝜇 𝐵𝐵 4 𝑅𝑅 𝐿𝐿 6 × ⟨𝜓𝜓 𝐷𝐷 𝐽𝐽 𝐷𝐷 ‖𝐿𝐿 + 2𝑆𝑆‖𝜓𝜓 𝐷𝐷 * 𝐽𝐽 𝐷𝐷 * ⟩ 2 ⟨𝜓𝜓 𝐴𝐴 * 𝐽𝐽 𝐴𝐴 * ‖𝐿𝐿 + 2𝑆𝑆‖𝜓𝜓 𝐴𝐴 𝐽𝐽 𝐴𝐴 ⟩ 2 𝐹𝐹 ( 11 )
where the intensity parameters 𝛺𝛺 𝐾𝐾 are obtained from Equation S1 using only the forced electric dipole (FED) mechanism (Equation S2), as given in the Judd-Ofelt theory, [45,46] and discussed in the references [43,47] . The sets of 𝛺𝛺 𝐾𝐾 (FED) values obtained, in units of 10 In Equation 10, 𝜌𝜌 𝑓𝑓-𝑓𝑓 is the overlap integral between 4f subshells of the donor and acceptor lanthanide ions. The values of 𝜌𝜌 𝑓𝑓-𝑓𝑓 (Table S6) as a function of the Tb-Eu distance (RL) were calculated using the parametric function 𝜌𝜌 𝑓𝑓-𝑓𝑓 (𝑅𝑅 𝐿𝐿 ) = 𝑒𝑒𝑒𝑒𝑒𝑒(𝑎𝑎 + 𝑏𝑏𝑅𝑅 𝐿𝐿 + 𝑐𝑐𝑅𝑅 𝐿𝐿 2 ), being 𝑎𝑎 = -0.032, 𝑏𝑏 = -0.261, and 𝑐𝑐 = -0.34. [48] The 𝜌𝜌 𝑓𝑓-𝑓𝑓 decays very fast to zero with the increase of donoracceptor distance RL, as presented in Reference [31] for the case of Tb-Eu. This is the reason why the 𝑊𝑊 𝑒𝑒𝑒𝑒 frequently can be neglected in the Ln-Ln energy transfer processes, where the donoracceptor distances are, commonly, higher than 4 Å, [44] in contrast to the case of intramolecular energy transfer processes in lanthanide chelates. [44,47] The nonradiative energy transfer involving magnetic dipole interactions in inorganic solids containing donor and acceptor ions has been treated in the early 1950s by Dexter [49] and Galanin [50] , in the 1970s by Ermolaev [51,52] , and in the 1990s by Tanner and collaborators. [53][54][55] The formulation of 𝑊𝑊 𝑚𝑚𝑑𝑑-𝑚𝑚𝑑𝑑 , as presented in Equation 11, is found in the Reference [31].

The ⟨𝜓𝜓𝐽𝐽‖ ‖𝜓𝜓 * 𝐽𝐽 * ⟩ 2 are doubly reduced matrix elements: the ⟨𝜓𝜓𝐽𝐽‖𝐿𝐿 + 2𝑆𝑆‖𝜓𝜓 * 𝐽𝐽 * ⟩ 2 were calculated (see Table S8) using the Ofelt's eigenfunctions in the intermediate coupling scheme, [56] and the ⟨𝜓𝜓𝐽𝐽�𝑈𝑈 (𝐾𝐾) �𝜓𝜓 * 𝐽𝐽 * ⟩ 2 values were obtained from Reference [START_REF] Carnall | Energy Level Structure and Transition Probabilities in the Spectra of the Trivalent Lanthanides in LaF₃[END_REF] 〈𝑟𝑟 𝐾𝐾 〉 are 4f radial integrals. [START_REF] Edvardsson | [END_REF] The shielding factors (1 -𝜎𝜎 𝑘𝑘 , with 𝑘𝑘 = 1 and 2) were calculated according to: [59] �1 -𝜎𝜎 𝑘𝑘 𝐷𝐷,𝐴𝐴 � = 𝜌𝜌(2𝛽𝛽) 𝑘𝑘+1

where 𝜌𝜌 and 𝛽𝛽 = (1 + 𝜌𝜌) -1 (values of 𝜌𝜌 in Table S6) are quantities related to the Ln-Ligand chemical bond [47,[60][61][62] and were estimated using the approach described in Reference [48].

𝜇𝜇 𝐵𝐵 �= 𝑒𝑒ℏ 2𝑚𝑚 𝑒𝑒 𝑐𝑐
� in Equation 11is the Bohr magneton in Gaussian-CGS units.

In all equations regarding the Ln-Ln energy transfer mechanisms, the spectral overlap factor (𝐹𝐹) appears. This quantity is related to the energy mismatch conditions between the donor and acceptor states and the following expression for 𝐹𝐹 has been used: [44] 𝐹𝐹 = ln(2)

√𝜋𝜋 1 ℏ 2 𝛾𝛾 𝐸𝐸𝑇𝑇 𝛾𝛾 𝐸𝐸𝐸𝐸 ��� 1 ℏ𝛾𝛾 𝐸𝐸𝑇𝑇 � 2 + � 1 ℏ𝛾𝛾 𝐸𝐸𝐸𝐸 � 2 � ln(2)� - 1 2 × exp ⎣ ⎢ ⎢ ⎡ 1 4 � 2 𝛿𝛿𝐸𝐸 (ℏ𝛾𝛾 𝐸𝐸𝑇𝑇 ) 2 ln2� 2 �� 1 ℏ𝛾𝛾 𝐸𝐸𝐸𝐸 � 2 + � 1 ℏ𝛾𝛾 𝐸𝐸𝑇𝑇 � 2 � ln2 -� 𝛿𝛿𝐸𝐸 ℏ𝛾𝛾 𝐸𝐸𝑇𝑇 � 2 ln(2) ⎦ ⎥ ⎥ ⎤ ( 13 )
where ℏ𝛾𝛾 𝐸𝐸𝑇𝑇 and ℏ𝛾𝛾 𝐸𝐸𝐸𝐸 correspond to the bandwidths at half-height of the Tb 3+ (donor) and Eu 3+

(acceptor) transitions, respectively. ΔE is the energy difference between donor and acceptor transitions (𝛿𝛿𝐸𝐸 = 𝐸𝐸 𝐷𝐷 -𝐸𝐸 𝐴𝐴 ). Due to the strong shielding effect over the 4f electrons, the energy levels of the Ln 3+ do not present significant shift with the changing of the environment, hence, the energy levels for LnF3 can be employed in the calculus without any discrepancy (Table S7 for the comparison between transitions energies for Eu 3+ and Tb 3+ in LnF3 and aqueous medium). In the present work, it will be considered the ℏ𝛾𝛾 𝐸𝐸𝑇𝑇 = ℏ𝛾𝛾 𝐸𝐸𝐸𝐸 = 250 𝑐𝑐𝑐𝑐 -1 , a value that is acceptable concerning the nature of 4f transitions.

The energy transfer pathways were chosen with the combination of four donor states (Tb 3+ : 5 D4→ 7 F3-6) and nine acceptor states (Eu 3+ : 7 F0,1,2→ 5 D0,1,2), each pathway was labeled as

shown Table 4. The selection rules on the 𝐽𝐽 quantum numbers were also taken into account:

|𝐽𝐽 -𝐽𝐽′| ≤ 𝐾𝐾 ≤ 𝐽𝐽 + 𝐽𝐽′ for multipolar mechanisms (𝑊𝑊 𝑑𝑑-𝑑𝑑 , 𝑊𝑊 𝑑𝑑-𝑞𝑞 and 𝑊𝑊 𝑞𝑞-𝑞𝑞 ); ∆𝐽𝐽 = 0, ±1 for the magnetic dipole-magnetic dipole mechanism (𝑊𝑊 𝑚𝑚𝑑𝑑-𝑚𝑚𝑑𝑑 ). The J-mixing effect will also be considered (see assumption 1 below), allowing the case of 𝐽𝐽 = 𝐽𝐽' = 0. No defined selection rules on J appear for the exchange mechanism (𝑊𝑊 𝑒𝑒𝑒𝑒 ).

For each pathway, the energy transfer rates were calculated by the sum over Equation 7to 11 �𝜔𝜔 = 𝑊𝑊 𝑑𝑑-𝑑𝑑 + 𝑊𝑊 𝑑𝑑-𝑞𝑞 + 𝑊𝑊 𝑞𝑞-𝑞𝑞 + 𝑊𝑊 𝑒𝑒𝑒𝑒 + 𝑊𝑊 𝑚𝑚𝑑𝑑-𝑚𝑚𝑑𝑑 � and, when appropriate, the following points should be taken into account: [31] 1) In the case of the acceptor Eu 3+ ion, the levels 7 F0, 7 F1, and 7 F2 are thermally dependent according to the Boltzmann distribution (0.65, 0.33, and 0.02 at 300 K, respectively [63] ). In the case of the pathway involving the 7 F0 5 D0 transition, the J-mixing between the 7 F0 and 7 F2 levels (of the order of 5%) was considered.

2) When ΔE is negative (the donor state lies below the acceptor level), the energy transfer rates were multiplied by the barrier factor exp(ΔE/kBT), where kB is Boltzmann's constant and T is the temperature. The case of pathway 9 (energy from 5 D4→ 7 F5 to 7 F1→ 5 D1, see Table 4) is an exception due to the resonance between the electronic Stark levels, as considered in References. [31,64] 3) 𝑊𝑊 (= ∑ 𝜔𝜔) is the total rate for a specific donor-acceptor distance and it is multiplied by the occurrence 𝑂𝑂 𝑖𝑖 (𝑒𝑒) of exist a pair donor-acceptor at 𝑅𝑅 𝐿𝐿 , which decreases when 𝑒𝑒 (Eu 3+ content) increases.

The average donor-acceptor energy transfer rate 〈𝑊𝑊〉 is given by: [31] 〈𝑊𝑊〉

= �〈𝑊𝑊〉 𝑖𝑖 𝑖𝑖 = (1 -𝑒𝑒) • 𝑒𝑒 • �� 𝑂𝑂 𝑖𝑖 (𝑒𝑒) • 𝑊𝑊 𝑖𝑖 𝑖𝑖 � ( 14 )
where 𝑊𝑊 𝑖𝑖 is the sum over all pathways for a specific 𝑅𝑅 𝐿𝐿 (assumption 3). The Eu 3+ ions concentration (𝑒𝑒), as well as the Tb 3+ ones (1 -𝑒𝑒), are considered explicitly. The coefficients 𝑂𝑂 𝑖𝑖 and 𝑐𝑐 𝑖𝑖 (from Reference [31]) have the same meaning. However, the latter case is obtained by a weighted-average of donor-acceptor while the coefficients in Equation 14 consists of a direct number of occurrences per Eu 3+ of existing a pair Tb-Eu at the distance 𝑅𝑅 𝐿𝐿 (𝑖𝑖). Thus, the 𝑂𝑂 𝑖𝑖 can be obtained as:

𝑂𝑂 𝑖𝑖 = 𝒩𝒩(𝑖𝑖) 𝑠𝑠 • 𝑒𝑒 ( 15
)
where 𝒩𝒩 is the average counting number of donor-acceptor pair present at distance 𝑅𝑅 𝐿𝐿 (𝑖𝑖), 𝑠𝑠 is the number of host sites (Tb 3+ ions) available for substitution by the Eu 3+ one (𝑒𝑒 in fraction).

The 𝑂𝑂 𝑖𝑖 also carries a dependence with the Eu 3+ concentrations. For example, in an extremely diluted system with a few Eu 3+ ions, the condition 𝑙𝑙𝑖𝑖𝑐𝑐 𝑒𝑒→0% 𝑂𝑂 𝑖𝑖 (𝑒𝑒) = 1 must be satisfied and, in the opposite situation 𝑙𝑙𝑖𝑖𝑐𝑐 𝑒𝑒→100% 𝑂𝑂 𝑖𝑖 (𝑒𝑒) = 0. These limits are valid when the host and the donor ions are the same (Tb 3+ ions), justifying the use of the pre-factor (1 -𝑒𝑒) • 𝑒𝑒, otherwise, when 𝑒𝑒 → 0% (system without Eu 3+ ions), the 〈𝑊𝑊〉 do not converge to zero.

The coefficients 𝑂𝑂 𝑖𝑖 (𝑒𝑒) were obtained adapting a computer program [65] that simulates the doping process and extracts specifics Tb-Eu distances in a doped supercell (5×5×5, containing 𝑠𝑠 = 500 Tb 3+ sites) previously constructed from the crystallographic unit cell. The values of 𝑂𝑂 𝑖𝑖 (𝑒𝑒) are presented in Table S9.

Table 4 shows the rates of each pathway for the shortest distance RL=4.35 Å, the most important for the overall energy transfer process. The energy transfer is dominated by far by the 𝑊𝑊 𝑞𝑞-𝑞𝑞 mechanism (see pathway 9). This result agrees with the conclusions in References [66,67] . The exchange mechanism becomes more dominant (in the case of Tb-Eu energy transfer) when RL is shorter than 4.12 Å, as in the case of Ba3Tb0.90Eu0.10(PO4)3 eulytite. [31] The W1 (Table 4) denotes the sum over all individual pathways considering assumptions 1 and 2. Pathway 9 (highlighted in bold) is the dominant one and does not include the barrier factor due to the resonance condition of the Stark levels. Beyond the first distance, the next two neighbor sites are located at 5.78 and 8.27 Å, although each of them has twice chance to form a Tb-Eu pair than the shortest one (Figure 3c), their contributions are expressively lower in one and three orders of magnitude (Table S9 andS10), respectively. It is worth to highlight that pathway 9 represents 89.36 (Table 4), 99.44 (Table S10), and 99.07% (Table S11) of the Tbto-Eu total energy transfer. For this reason, from now it will be considered only the ( 5 D4 7 F5)( 7 F1 5 D1) energy transfer.

Applying the values of 〈𝑊𝑊〉 of Equation (1) and using the 5 D4 lifetime in the absence of Eu 3+ ion (τ0=1.040×10 -3 s, Table 1), one can estimate also the dependence of the 5 D4 lifetime 𝜏𝜏 with the Eu 3+ content using:

𝜏𝜏 = 1 �〈𝑊𝑊〉 + 1 𝜏𝜏 0 � ( 16 )
A good agreement between theoretical and experimental values of the energy transfer rates and 5 D4 lifetimes is depicted in Figure 3d.

When temperature increases, the population of 7 F1 also increases from 7 F0 obeying the Boltzmann's distribution that depends on the gap between the ground 7 F0 and first excited 7 F1 levels (~377 cm -1 , Supplementary Figure 25). Since the most relevant energy transfer pathway involves the 7 F1 level, the major energy transfer rate is blocked when T→0 K and starts to grow when the temperature rises.

Figure 3f shows both temperature and Eu 3+ content effects upon the Tb-to-Eu energy transfer.

The back energy transfer is not operative since the multiphonon relaxation from 5 D1 to 5 D0 is very high -in the order of 10 4 s -1 in NaGdTiO4:Eu 3+ at 4.2 K [68] and more than 10 5 s -1 in YVO4:Eu 3+ for temperature in the range of 100 < T < 600 K [69] -digesting a fast depopulation of the 5 D1 level. The surface presented in Figure 3f was generated using 892 calculations of 〈𝑊𝑊〉. The data were statistically treated by the weighted least squares method. [START_REF] Wasserman | All of Nonparametric Statistics[END_REF] Figure 4 presents the dependence of the performance parameters Sm and Tm with the 〈𝑊𝑊〉 and with the Eu 3+ content. The increase observed in the 〈𝑊𝑊〉 with the Eu 3+ content is also observed in the maximum relative thermal sensitivity (Sm) value suggesting a strong linear correlation between both quantities. Furthermore, for the temperature in which it occurs (Tm) we perceive the exact opposite trend (Figure 4).

From the qualitative point of view, the increase in the Tb-to-Eu energy transfer rate favors the depopulation of the 5 D4 level of Tb 3+ with the temperature increase, and thus a higher slope in the ∆(T) curve is expected at its inflection point (in which the maximum sensitivity occurs),

that is recorded at lower temperature values. In other words, our results point out that the higher the 〈𝑊𝑊〉 value, the higher the Sm, favoring the performance of the luminescent thermometer.

The complete modelling of the thermal response of these mixed compounds that will certainly be useful for a detailed and quantitative optimization analysis is now underway and will be reported in future work.

Conclusion

The versatility of the tailored design of the MOFs has attracted attention for its potential application in luminescence thermometry in the last decade. Despite many works are studying the thermometric performance of a single sample or a series of samples, rare are the reports connecting the design and composition of the framework with the desired thermometric performance. In this work presented a series of Tb1-xEux (x=0.030-0.150) MOF luminescent thermometers built upon isophtalic and acetic acids. The crystal structure is composed by Ln2

SBUs connected by organic linkers to form a 3D network. We evidenced that this dimer topology promotes the presence of an efficient Tb 3+ -to-Eu 3+ energy transfer which varies from 77 to 93% when the Eu 3+ content increases from 0.030 to 0.150. By increasing the Eu 3+ content we observe a shift of 100 K towards low temperatures on Tm, connected to an increase in Sm from 0.19 to 0.44 %⋅K -1 . This variation on the Sm is partially explained by the increase of the nonradiative decay rate of the Tb 3+ 5 D4→ 7 F5 transition (relatively to the radiative one) with the increase of the Eu 3+ doping. A detailed experimental and theoretical study on Tb-to-Eu energy transfer processes was very successful evidencing a notorious agreement between both. Using computer simulations we estimated the average (or effective) energy transfer rates using doping simulations and obtained the probability of occurrence of the donors-acceptor pairs, concluding that this energy transfer is dominated by the 5 D4→ 7 F5 (Tb 3+ ) to 7 F1→ 5 D1 (Eu 3+ ) pathway.

Moreover, in line with the conclusions of recently published works, [31,66,67] we found out that the most relevant energy transfer mechanism is the quadrupole-quadrupole one, due to the shortest Tb-Eu distance that is 4.35 Å.

Comparing the energy barrier value extracted from the numerical simulations (260 cm -1 for the dominant pathway, Table 4) with those resulting from the Mott-Seitz analysis (∼500-600 cm -1 , Table 2) we notice a factor of ∼2 between them. Although the values are in the same order of magnitude, the discrepancy is supported by i) the assumption that the positions of the energy levels in the MOF were based on those reported by Carnall et al. [START_REF] Carnall | Energy Level Structure and Transition Probabilities in the Spectra of the Trivalent Lanthanides in LaF₃[END_REF] for the Ln 3+ -doped lanthanum trifluoride; ii) the well-known fact that the Mott-Seitz model generally overestimates the energy barrier value; [START_REF] Blasse | Nonradiative Processes in Luminescent Materials: A Materials Scientists View[END_REF] and ii) the fitted ∆𝐸𝐸 value in Equation 4 is an effective energy gap involving the depopulation of the 5 D4 Tb 3+ level by the 5 D1 Eu 3+ one and the depopulation of the 5 D0 Eu 3+ level by the triplet ligand level (explaining the temperature dependence of the 5 D0→ 7 F2 transition).

We also perceive an increase in the Tb 3+ -to-Eu 3+ energy transfer rate with the temperature increase, what is strongly correlated to the enhancement of the maximum relative thermal sensitivity of the luminescent thermometer and with the concomitant decrease of the temperature at which the maximum thermal sensitivity is recorded.

The unique experimental and theoretical blended methodology presented here enabled us to rationally engineer the composition of the Tb1-xEux mixed compound, looking forward to tunable thermal performance, a driving force paving the road for the future tailored-designing of molecular thermometers.

Experimental Section

Reagents and Chemicals: Tb(NO3)36H2O (99.99%), Eu(NO3)36H2O (99.99%), Gd(NO3)36H2O (99.99%), and isophtalic acid (99%) were purchased from Alfa Aesar while acid acetic acid (99.8%) was purchased from Acros Organics. All chemicals were used without further purification. Eu(NO3)36H2O in the appropriate ratio was dissolved in 2 mL of deionized water containing glacial acetic acid (100 µL, 1.7 mmol). The two solutions were mixed and a white precipitate was formed. The pH of the mixture was adjusted at 5 by addition of a small amount of a NaOH solution (1.5 M). Then, the above mixture was placed in a sealed 21 ml Teflon lined stainless vessel, which was heated at 150°C for 72 h under autogenous pressure and cooled down to room temperature. The as-obtained white power was finally recovered by filtration and washed with EtOH. For the whole series, the chemical yields were around 40%. The targeted compositions for Tb1-xEux was x = 0.030, 0.050, 0.075, 0.100, 0.125 and 0.150. The Tb/Eu molar ratio was checked by IPC-AES, the experimental compositions are reported in Table S1. Tables Table 1. The 5 D4 (monitoring at 545 nm) and 5 D0 level (monitoring at 615 nm) lifetimes and Tb 3+ -to-Eu 3+ energy transfer efficiency (ηET) in the single-doped and mixed Tb1-xEux samples at 300 K. The excitation wavelength is 289 nm.

Sample

x 5 D4 (10 -3 s) * values recalculated in reference [41]

  Structural and Photophysical Characterization: Powder X-ray Diffraction spectra were monitored using a D8 Bruker diffractometer in the Bragg-Brentano geometry, equipped with a front germanium monochromator, a copper anode (CuK-L3 radiation λ=1.540598 Å) and a LynxEye PSD detector. The simulated pattern was obtained by the PowderCell 2.4 software from the cif file of the Eu-CP compound (CCDC 251041). Thermogravimetric analyses (TGA) were performed by flowing dry air with a heating and cooling rate of 5°C/min on a SETARAM TG-DSC 111 between 20 and 800°C. Fourier transform infrared (FT-IR) spectra were recorded in the 4000-400 cm -1 range on a Bruker Vertex FTIR spectrometer equipped with a computer control using the OPUS software. For ICP-AES analyses, samples were dissolved in a 10% HNO3 solution and analyzed by ICP-AES. The calibration curve was established from the analysis of five standard solutions containing Eu and Tb in various contents to cover a range in accordance with targeted concentrations. The standard solutions were prepared with high purity monoelemental solutions in acidic solution. Photoluminescence spectra were recorded on a Jobin-Yvon Fluorolog 3 fluorometer equipped with a photomultiplier (excitation source: 450 W Xe arc lamp) using the front face acquisition mode. The emission spectra were corrected for detection and optical spectral response of the spectrofluorimeter and the excitation spectra were weighed for the spectral distribution of the lamp intensity using a photodiode reference detector.The temperature-dependent photoluminescence measurements were recorded on the same spectrometer controlling the temperature by a cryostat coupled with the Fluorolog cooled by liquid nitrogen. Lifetimes measurements were measured with the same equipment but the excitation was performed by a UV Xenon flash tube while the time-dependence of the emission was recorded with the photomultiplier.Synthesis of [Gd(CH3COO)(1,3-bdc)(H2O)2]• 0.5H2O (Gd-CP):1,3-H2BDC (115 mg, 0.69 mmol) was dissolved in 5 mL of a NaOH solution (0.6 M) while Gd(NO3)36H2O (258 mg, 0.69 mmol) was dissolved in 2 mL of deionized water containing glacial acetic acid (100 µL, 1.7 mmol). The two solutions were mixed and a white precipitate was formed. The pH of the mixture was adjusted at 5 by addition of a small amount of a NaOH solution (1.5 M). Then, the above mixture was placed in a sealed 21 ml Teflon lined stainless vessel, which was heated at 150°C for 72 h under autogenous pressure and cooled down to room temperature. The asobtained white power was finally recovered by filtration and washed with EtOH. Yield: 0.218 g (37%

  Synthesis of [Tb(CH3COO)(1,3-bdc)(H2O)2]0.5H2O (Tb-CP):1,3-H2BDC (115 mg, 0.69 mmol) was dissolved in 5 mL of a NaOH solution (0.6 M) while Tb(NO3)36H2O (290 mg, 0.69 mmol) was dissolved in 2 mL of deionized water containing glacial acetic acid (100 µL, 1.7 mmol). The two solutions were mixed and a white precipitate was formed. The pH of the mixture was adjusted at 5 by addition of a small amount of a NaOH solution (1.5 M). Then, the above mixture was placed in a sealed 21 ml Teflon lined stainless vessel, which was heated at 150°C for 72 h under autogenous pressure and cooled down to room temperature. The asobtained white power was finally recovered by filtration and washed with EtOH. Yield: 0.242 g (41%

  719 (m), 689 (m), 613 (m), 575 (m), 525 (m). Synthesis of [Tb1-xEux(CH3COO)(1,3-bdc)(H2O)2]0.5H2O (Tb1-xEux): 1,3-H2BDC (115 mg, 0.69 mmol) was dissolved in 5 mL of a NaOH solution (0.6 M) while Tb(NO3)36H2O and

FiguresFigure 1 .

 1 Figures

  Figure 1. (a) Representation of Ln2SBU formed by two symmetric [LnO9] polyhedron linked by an edge formed by two acetate ligand (color code: blue for Ln 3+ , grey for carbon, red for oxygen, yellow for the acetate ligands and white for water hydrogens). H atoms of carboxylate ligands are omitted for more clarity. (b) Representation of the crystal structure along the a-axis. (c) X-ray diffraction patterns of Tb1-xEux compounds. The bottom line corresponds to the simulated profile of the Eu-CP.

Figure 2 (

 2 Figure 2 (a) Temperature-dependent emission spectra in the 150-350 K range of Tb0.950Eu0.050 upon 289 nm excitation. (b) Corresponding dependence of the normalized integrated areas of I1 (Tb 3+ , 5 D4→ 7 F5) and I2 (Eu 3+ , 5 D0→ 7 F2). (c) Relative thermal sensitivity for the mixed compounds and (d) temperature uncertainty in the 150-350 K range. The dots mark the Sm values and the minimum δT values, respectively in panels c and d, and the interrupted lines are guides for the eyes.

Figure 3 .

 3 Figure 3. (a) Representation of a 3×3×3 cell of the doped Tb-CP. Scheme of (b) the isolation of the neighbors' ions and (c) the extraction of the donor-acceptors distances. (d) Tb 3+ -to-Eu 3+ nonradiative energy transfer rates obtained using Equation 1 (points) and Equation 14 (line). (e) 5 D4 lifetime values (Equation16and Table1) as a function of the Eu 3+ content (x). (f) Tb 3+to-Eu 3+ energy transfer rate 〈𝑾𝑾〉 as a function of the Eu 3+ content and of the temperature for Tb1-xEux compounds.

Figure 4 .

 4 Figure 4. (a) Maximum relative thermal sensitivity and (b) temperature value at which it occurs against the calculated Tb 3+ -to-Eu 3+ energy transfer rate 〈𝑾𝑾〉 for the Tb1-xEux compounds. The lines are guides for the eyes.

4 -

 4 benzenedicarboxylic acid, BPA = biphenyl-3,5-dicarboxylic acid, bpydc = 2,2'bipyridine-5,5'-dicarboxylic acid, ad = adipic acid, phth = phthalic acid, L =1,3bis(carboxymethyl-)-imidazolium, ox = oxalic acid, PDC = pyridine-3,5-dicarboxylic acid.

  -48 9.9⨯10 -45 3.4⨯10 -41 6.8⨯10 -41 2.0⨯10-42 2.2⨯10 -51𝑊𝑊 1 = � 𝜔𝜔 = 9.9⨯10 4

  -20 cm 2 , are: Tb 3+ [Ω 2 = 0.03; Ω 4 = 0.19; Ω 6 = 0.36] and Eu 3+ [Ω 2 = 0.05; Ω 4 = 0.36; Ω 6 = 0.71] . The 𝑊𝑊 𝑞𝑞-𝑞𝑞 , 𝑊𝑊 𝑒𝑒𝑒𝑒 and 𝑊𝑊 𝑚𝑚𝑑𝑑-𝑚𝑚𝑑𝑑 mechanisms are independent of the Ω 𝐾𝐾 parameters.

5 D0 (10 -3 s)

					ηET (%)
	Eu-CP	-	-	0.39 ± 0.05	-
	Tb-CP	-	1.19 ± 0.08	-	-
		0.030	0.28 ± 0.04	0.49 ± 0.05	76 ± 4
		0.050	0.17 ± 0.04	0.45 ± 0.05	84 ± 3
		0.075	0.12 ± 0.04	0.42 ± 0.04	89 ± 3
	Tb1-xEux
		0.100	0.09 ± 0.04	0.42 ± 0.04	91 ± 3
		0.125	0.08 ± 0.04	0.40 ± 0.03	92 ± 3
		0.150	0.08± 0.04	0.48 ± 0.05	92 ± 3

Table 2 .

 2 Fitting parameters of Equation 4 to the experimental ∆(T) for the Tb1-xEux compounds.

	x	∆0	α	∆E (cm -1 )	r 2
	0.030	0.67±0.01	4.46±0.53	620±26	0.9999
	0.050	0.67±0.01	4.65±0.50	614±31	0.9999
	0.075	0.19±0.01	5.90±0.40	600±22	0.9999
	0.100	0.27±0.01	7.94±0.93	570±30	0.9998
	0.125	0.14±0.01	10.00±0.91	522±10	0.9999
	0.150	0.41±0.01	10.87±0.40	500±18	0.9999

Table 3 .

 3 Thermometric performance of the mixed compounds reported here compared with illustrative examples of the literature.

	Material

Eu content Sm (%•K -1 ) δTm (K) Tm (K) Reference

		x=0.030 0.19±0.02 0.53 338±41
		x=0.050 0.20±0.02 0.51 333±40
		x=0.075 0.24±0.01 0.42 314±29
	Tb1-xEux				This work
		x=0.100 0.31±0.03 0.32 284±32
		x=0.125 0.40±0.02 0.25 251±22
		x=0.150 0.44±0.02 0.23 236±16
	Tb0.99Eu0.01(bdc)1.5(H2O)2 x=0.01	0.37	318	[36]
	Tb0.80Eu0.20BPA	x=0.20	1.19	313	[38]
	Eu 3+ @UiO-bpydc	x=0.001	2.19 *	293	[37]
	Tb0.995Eu0.005 @In(OH)-bpydc	x=0.0005	4.47 *	333	[37]
	Tb1-xEux(ad)0.5 (phth)(H2O)2	x=0.001 x=0.005 x=0.003	0.42 1.21 0.59	303 303 303	[40]
	Tb0.97Eu0.03(L)(ox)(H2O) x=0.03	1.38	340	[41]
	Tb1.95Eu0.05-PDC	x=0.05	0.64	318

Table 4 .

 4 Pairwise nonradiative energy transfer rates (s -1 ) from Tb 3+ to Eu 3+ at the shortest distance (RL=4.35 Å).

	Pathway Label	Tb 3+	Eu 3+	∆𝐄𝐄	𝐖𝐖 𝐝𝐝-𝐝𝐝	𝐖𝐖 𝐝𝐝-𝐪𝐪	𝐖𝐖 𝐪𝐪-𝐪𝐪	𝐖𝐖 𝐞𝐞𝐞𝐞	𝐖𝐖 𝐦𝐦𝐝𝐝-𝐦𝐦𝐝𝐝	𝛚𝛚
	1	5 D4→ 7 F6	7

Acknowledgements

This work was developed within the scope of the project CICECO -Aveiro Institute of Materials, UIDB/50011/2020, financed by Portuguese funds through the FCT/MEC and when appropriate co-financed by FEDER under the PT2020 Partnership Agreement. Financial support from NanoHeatControl project (POCI-01-0145-FEDER-031469), funded by FEDER, through POCI and by Portuguese funds (OE), through FCT/MCTES is also acknowledged. V.T. and A.N.C.N. thanks la Fondation de la Maison de la Chimie and the SusPhotoSolutions project, CENTRO-01-0145-FEDER-000005 Portugal, respectively, for their postdoctoral fellowships.

Received: ((will be filled in by the editorial staff))

Revised: ((will be filled in by the editorial staff)) Published online: ((will be filled in by the editorial staff))

Supporting Information

Supporting Information is available from the Wiley Online Library or from the author.

Table of content

Here is reported the tuning of the relative thermal sensitivity in a series of Eu-Tb mixed metalorganic framework luminescent thermometers. The sensitivity and the temperature range are modulated by an accurate control of the Eu/Tb ratio. Theorerical calculations highlight the chemical composition directly impacts the Tb-to-Eu energy transfer rate.


Chenyang Zhao
email: c.zhao@ed.ac.uk

Timothy M Hospedales
email: t.hospedales@ed.ac.uk

Freek Stulp
email: freek.stulp@dlr.de

Olivier Sigaud
email: olivier.sigaud@isir.upmc.fr

Tensor based knowledge transfer across skill categories for robot control

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Advances in robot hardware, policy representations and policy-search based reinforcement learning have led to a growing number of successful demonstrations of robot control involving challenging dynamics, including baseball [START_REF] Peters | Reinforcement learning of motor skills with policy gradients[END_REF], ball-in-cup [START_REF] Stulp | [END_REF], pancake flipping [START_REF] Kormushev | [END_REF], table tennis [Kober et al., 2010] and tetherball [START_REF] Kober | Policy search for motor primitives in robotics[END_REF]. These tasks involve non-linear policies and non-convex reward functions. Thus, direct application of reinforcement learning has met with limited success as their non-linearity and large search space would require a prohibitive number of trials. A widely used strategy has therefore been to obtain an initial manual demonstration of the skill [START_REF] Argall | [END_REF], followed by imitation learning a suitable policy representation such as Dynamic Movement Primitives (DMPs) [START_REF] Schaal | [END_REF]. Based on this good initial model, policy-search based reinforcement learning is used to fine-tune the resulting control policy [Kober et al., 2010]. Nevertheless, we would like robots to learn skills autonomously, because manually demonstrating each task (or possibly even variant thereof) is labor intensive, and because humans may not even know the solution to some control tasks we wish a robot to solve.

The desire to increase autonomy in this way has motivated extensive work into generalising skills. For example multitask learning addresses sharing information across multiple skills [START_REF] Deisenroth | [END_REF][START_REF] Parisotto | [END_REF] and contextual (or parameterised) policies build skills that generalise across variants [Stulp et al., 2013;[START_REF] Kupcsik | [END_REF] within one family of tasks. Most of these studies however, address generalisation of skills that are relatively simple parametric variants of each other. For example throwing objects of different weights [START_REF] Stulp | [END_REF], throwing to different target locations [START_REF] Kupcsik | [END_REF], or going via different waypoints to avoid various obstacles [Stulp et al., 2013].

In this paper, we are inspired by the vision of lifelong learning [START_REF] Thrun | Is learning the n-th thing any easier than learning the first?[END_REF]Ruvolo and Eaton, 2013b]. That is, the idea that new tasks should get progressively easier to learn as a wider and deeper set of prior tasks are mastered by a learner with the ability to extract task-agnostic generalisations from experience. With this in mind, we go beyond existing contextual policy work and explore transfer learning across a heterogeneous set of dynamic control tasks that do not lie in a simple parameterised family. In our setting, a robot starts with knowledge of a set of source task variants (e.g., throwing type tasks). The aim is then for it to master a different category of target task (e.g., catching type task) autonomously through reinforcement learning (RL). We explore four task families: reaching to a target position, throwing at a target [START_REF] Deisenroth | [END_REF]Kober et al., 2010], casting at a target [START_REF] Kober | Policy search for motor primitives in robotics[END_REF], and ball-in-cup [START_REF] Stulp | [END_REF]. Some of these are typically prohibitively difficult to learn directly with RL, so to learn one task autonomously based only on past experience of another, the robot must abstract and transfer task-agnostic generalisations. To achieve this, we define a class of neural network controllers loosely inspired by the DMPs [START_REF] Schaal | [END_REF] commonly used to solve these dynamic tasks. We first multi-task learn a set of multiple source tasks from a given family (e.g., throwing objects of various weights to various locations), and the controllers for these correspond to a stack of such neural networks. Then we factorise this set of tasks to obtain transferrable latent skills. Finally, by using these latent skills as a basis to construct a policy network, we are able to learn a set of target tasks (such as ball-in-cup with various string lengths) autonomously, without demonstration. Uniquely our tensor-based transfer framework enables simultaneous discovery and sharing of latent skills across both task categories and actuators [START_REF] Luck | [END_REF].

Our contributions are two-fold: (i) We introduce a DMPinspired neural network that can represent a variety of dynamic skills, and show how multiple networks can be factored to obtain transferrable latent skills, (ii) We evaluate this idea with four challenging dynamic tasks, and show how transferring latent skills can dramatically speed up learning target tasks, ultimately allowing challenging new skills to be mastered autonomously by policy-search reinforcement learning.

Related Work

Learning challenging robot control skills is usually achieved in practice by supervised learning from demonstration. Nevertheless, the vision of learning-based robot control is for robots to learn new skills autonomously. Direct application of RL however requires prohibitive amounts of training, risking physical damage to the robot. This has motivated a fruitful line of research into contextual (parameterised) policies and options where a robot learns to solve a related family of tasks, such as throwing objects of different weights [START_REF] Stulp | [END_REF]. By contrast, we aim to achieve autonomous learning of a new task category through transfer learning.

Transfer & Multi-task Learning Transfer (TL) and multitask (MTL) learning aim to improve and accelerate learning by sharing knowledge across different tasks, in a uniand multi-directional way respectively. These are well studied topics in supervised machine learning. One of the earliest applications to RL showed that for classic pole balance problems, it was faster to learn a policy for a novel system with knowledge transferred from other similar systems [START_REF] Selfridge | [END_REF]. TL has since been widely applied to accelerate reinforcement learning [START_REF] Taylor | [END_REF]. MTL has been used to jointly optimise multiple RL tasks including inverted pendulum problems with various mass or length [START_REF] Lazaric | Bayesian multi-task reinforcement learning[END_REF] and stacking various numbers of blocks [START_REF] Deisenroth | [END_REF]. However, in our terminology these are members of task family, rather than distinct task categories. Very recently, multi-task RL has been applied in learning to play multiple video games [START_REF] Parisotto | [END_REF].

A related work to ours is PG-ELLA [Ammar et al., 2014], which adapts the successful GO-MTL algorithm [START_REF] Kumar | Abhishek Kumar and Hal Daume III. Learning task grouping and overlap in multitask learning[END_REF] from supervised learning to the policy gradient based RL setting. Their idea is to apply low-rank matrix factorisation to a stack of linear models and share information through the resulting subspace, and PG-ELLA applies this to control tasks such as pole-balancing. We go significantly beyond this: PG-ELLA/GO-MTL deal with linear policies only. This policy representation means they can only address simple linear control tasks, and not the dynamic tasks we study. These require dynamic trajectory planning that is highly nonlinear in time. Secondly PG-ELLA shares latent knowledge across tasks only. Our tensor-based TL framework represents policy for each actuator as a slice of tensor and therefore, discovers and shares latent skill across both tasks [Ammar et al., 2014] and actuators [START_REF] Luck | [END_REF] simultaneously.

Lifelong & Curriculum Learning Lifelong learning takes TL/MTL ideas further with the vision that as more tasks are learned, better task-agnostic abstract knowledge can be extracted. The resulting more humanlike "learning to learn" should make each new task easier to progressively easier to master [START_REF] Thrun | Is learning the n-th thing any easier than learning the first?[END_REF]. These ideas have also been studied in robot control, for example by treating different environments [Ring, 1998], or robot hardware platforms [START_REF] Isele | [END_REF] as multiple tasks to be mastered in a lifelong learning manner. The choice of task sequence is important to the outcome in lifelong learning [Bengio et al., 2009], but despite some work [Ruvolo and Eaton, 2013a], how to predict a good curriculum in advance is still an open question. In this paper we demonstrate that with an intuitive choice of curriculum, our framework can ultimately learn a challenging dynamic control skill autonomously through RL.

Low-Rank Tensors and Factorisation Low-rank tensor models are often studied for missing data imputation, and compressing/speeding up large neural networks [START_REF] Lebedev | [END_REF]. A few MTL studies used low-rank tensors for knowledge sharing across tasks with a structured description, rather than a simple index [Wimalawarne et al., 2014]. However, in these studies the underlying per-task representation in each case is still a linear model (single vector) that predicts a single output. We address tensor factorisation based transfer to share knowledge across both tasks and multiple outputs (actuators) with non-linear neural network-based models.

Methodology

Policy Representation

The challenging dynamic control tasks we address in this paper are commonly solved by DMP-based approaches. DMPs combines a non-linear open-loop term (which depends on time) with a linear closed-loop term (which depends on state). Inspired by this idea, we design a policy network a t,i = π(t, x o t,i ) which produces the ith actuator's force a t,i given the current time t and observed state x o t,i . The network has inputs for t and x o t,i , and includes two learnable layers. The first is a radial-basis function (RBF) layer that functions as a trajectory planner, allowing temporally extended and highly non-linear movements to be generated. The second is a linear fully-connected (FC) layer that encodes learned controller parameters that will ensure the trajectory is followed. The roles of these roughly correspond to the two components in a DMP.

⇥ i i +1 +1 -1 -1
Figure 1: Network defining our policy ai = π(x o t,i , t) from timestep t and observed state x o t,1 to action at,i.

Specifically, for actuator i the policy is parameterised as:

φ n (t; µ, Σ) = 1 √ 2πσ n exp (t -µ n ) 2 2σ 2 n
(1)

x d t,i = Θ i Φ(t; µ, Σ) (2)
ẋ d t,i = 1 2∆t (x d t+∆t,i -x d t-∆t,i ) (3)
a t,i = Ψ i (x d t,i -x o t,i ) (4)
Here the learnable parameters are the matrices Ψ i that implement the controller, and Θ i containing the weights of N basis functions describing the non-linear trajectory. Fig. 1 illustrates the network for the ith actuator. The dash lines indicate that desired velocity is not generated from the feed forward network directly, but via the Euler method as in Eq. ( 3).

In contrast to the conventional DMP-based pipeline, our approach jointly learns the inter-related problems of trajectory planning and PD controller.

Single Task Learning The network can be trained in a supervised way via learning from demonstration, or by RL via policy-search. For RL, we use Covariance Matrix Adaptation -Evolutionary Strategy (CMA-ES) [START_REF] Kern | [END_REF], a direct-policy search method for optimisation of non-linear non-convex functions in continuous space. CMA-ES has connections to policy-gradient and has been shown to outperform other RL algorithms for our type of policy [START_REF] Stulp | [END_REF]. For an actuator i, both layers' parameters can be summarised as a vector w i containing both vectorised RBF weights Θ i ∈ R N and controller parameters x) . Thus, for a given task, the parameters are represented as a matrix W ∈ R D×A where D = (N +dim(x)) and A = dim(a), where dim(x) = 2 in the case of a PD controller.

Ψ i ∈ R dim(

Low-Rank Tensor Factorisation: Latent Skills

Given the above policy representation, we aim to discover latent skills that can be shared across tasks and also actuators.

As summarised above, the policy for each task is represented as a matrix, so multiple tasks stack into a 3-way tensor. We will achieve knowledge sharing through low-rank modelling of this tensor. Unlike for matrices, there are many definitions of low-rank tensor factorisation, and we use one of the most general, Tucker decomposition [START_REF] Tucker | [END_REF]. Tucker decomposition factors an N -way tensor into a lower-rank Nway core tensor and N matrices along each mode. A 3-way tensor X ∈ R D1×D2×D3 is assumed to be composed as:

X = G × 1 U 1 × 2 U 2 × 3 U 3 (5)
where G ∈ R K1×K2×K3 is the lower-rank core tensor, and U n ∈ R Kn×Dn are the factor matrices and can be regarded as the principal components in each mode, and K n ≤ D n . This can be efficiently solved [START_REF] Lathauwer | [END_REF] as a higher-order singular value decomposition problem and obtaining U n as the U matrix from SVD of mode-n flattening of X , after which the core tensor is obtained by

G = X × 1 U T 1 × 2 U T 2 × 3 U T 3 . (6)

Multi-Task and Transfer Learning Strategy

Given a set of tasks/skills to learn, we aim to use the above tensor factorisation strategy to extract task-agnostic information to share between them (multi-task learning), and to transfer to benefit the learning of new tasks (transfer learning). A schematic overview of the procedure is given in Fig. 2. Formalisation We assume that we have P source tasks where W p represents the policy parameter matrix for the pth task's network (as described in Sec. 3.1). The policy parameters for all source tasks can be stacked as slices of a 3-way tensor W of size D × A × P . The latent task assumption is to factorise the weight tensor as per Eq. ( 5). In this case G is the core tensor of size K 1 × K 2 × K 3 that contains knowledge abstracting both skills and actuators. U 2 ∈ R K2×A contains actuator specific knowledge, and U 3 ∈ R K3×P encodes task specific knowledge. Based on this decomposition, we consider both multi-task and transfer learning.

Multi-task Learning To jointly learn several tasks, we exploit 'constructive' multi-task learning [Yang and [START_REF] Yang | A unified perspective on multi-domain and multi-task learning[END_REF][START_REF] Yang | Deep multi-task representation learning: A tensor factorisation approach[END_REF]. That is, the parameters we actually train with CMA-ES are the factors G and U 1,...,3 , which are then multiplied out (Eq. 5) to obtain the tensor parameterising the policies for all tasks (neural network RBF and FC parameters) in order to perform a rollout of a given task.

For regularisation we place a L 1 -norm on U 3 and L 2 -norm on the others. Simultaneously training the parameters in this way shares knowledge across actuators and across tasks.

Learning a Target Task We next consider transferring knowledge to a target task, given a set of source tasks (modelled by G and U 1,...,3 above). To achieve this, we gather the task-agnostic knowledge (latent skills) by contracting the task independent factors into a tensor L ∈ R D×A×K as

L = G × 1 U 1 × 2 U 2 .
The tensor L is then transferred to the target task. This provides a good initial subspace, so that a challenging target task in a different category can in practice now be learned autonomously with RL. The policy parameters for each target task are initialised as a random point in the transferred subspace W target = L × 3 s target by initialising the weight vector s target randomly. The learner then searches for the policy W target by direct policy search in the space of s target and L with a L 2 -norm regulariser on output weights. Algorithm 1 summarises the overall procedure.

Discussion In contrast to some other one-to-one transfer learning approaches [START_REF] Taylor | [END_REF], an important difference is that our algorithm exploits transfer from multiple source tasks in order to extract task-agnostic information that is likely to be transferable. The subspace/latent task set L is updated when learning the target, so we can see that transfer learning here is about providing a good initial condition for the subspace before performing RL.

Algorithm 1 Tensor-based Transfer Learning {Source Task Learning} for p = 1 to P source tasks do Learn source task policy Θ p and Ψ p . end for Initialise tensor W containing weights as slices. MTL initial condition:

W → G × 1 U 1 × 2 U 2 × 3 U 3 .
Multi-task learn source tasks. {Target Task Policy Search} Task-agnostic knowledge tensor:

W = G × 1 U 1 × 2 U 2 .
Initialise L with slices of tensor W . Initialise s as one random source task. while not converged do Fix L and update s with CMA-ES Fix s and update L with CMA-ES end while 4 Experiments

Environment and Tasks

We exploit a simulated robot arm whose end-effector operates in a 2-dimensional space. In the dynamical simulator, we assume an ideal inverse kinematic system which is able to convert the movement of end-effector into angular movement of each joint. Our policy network outputs the acceleration of end-effector in both horizontal and vertical directions at each time step t. In detail, the agent observes the positions and velocities, and outputs the accelerations for two actuators (A = 2). Each actuator trajectory is modelled by N = 26 weighted RBFs (centers between 0 and 5, with variances σ = 0.6) in the first layer of network and each actuator controller is modelled with 2 PD controlling parameters.

For most experiments, we assume P = 16 task instances per category, and thus the policy tensor is W ∈ R 28×2×16 . For transfer learning, we assume K = 8 latent tasks throughout. The task categories used in our experiments are as follows:

Target Reaching (Reach) The agent learns to move the end-effector, reaching specific target position with time discounted reward for each time step and penalties on large velocities and accelerations. Tasks within the category are to reach different positions, so task parameters are the 2dimensional positions of goal points. Object Throwing (Throw) The learner aims to throw a ball into a basket by holding it with the end-effector and then releasing it during dynamic arm motion [Kober et al., 2010]. We fix the ball release time. The task parameters only includes the position of target basket in 2-dimensional space. Ball-In-Cup (BIC) In this task [START_REF] Stulp | [END_REF][START_REF] Kober | Policy search for motor primitives in robotics[END_REF], the robot holds a cup in its end-effector and the cup has a string attached, from which a ball hangs. Initially, the ball is hanging at rest vertically below the cup. The task is to induce motion in the ball through the string: swinging the ball up and catching the ball with the cup. This is illustrated in Figure 3, the red line shows the position of the ball when the string is taunt and the green lines show the ball position when the string is loose. The task parameters include the length of the string and the mass of the ball. Casting (Cast) In casting [START_REF] Kober | Policy search for motor primitives in robotics[END_REF], a ball is attached to the end-effector by a string. The task is to get the ball into a small cup, placed in front of the robot. Because the end-effector controls the ball only indirectly via the string (which may change between loose and taunt depending on the dynamics), the dynamics are very different to throwing. We fixed the position of target cup and the task parameters include the length of the string and the mass of the ball only. Cost Function The cost for reaching tasks is defined with action penalties and a time discounted linear summation of quadratic functions with respective to the distance to target state

J = T t=0 [γ t ((x -x target ) 2 -b) + α a t ],
where constant b is a pre-defined baseline. The other three tasks use a cost function of the same parametric form. The cost (cf. reward) J of each episode is based on the difference between horizontal position of the ball x b and the cup x c at t m when they are at the same vertical height. The cost is assigned as 0 if the ball fails to reach the same height as the cup. L 2 -norms on actions are added to penalise extreme accelerations. For our experiments, we choose γ = 0.99, α = 0.001.

J = T t=0 α a t if vertical heights never match γ tm min(b 1 (x b -x c ) 2 -b, 0) + T t=0 α a t otherwise

Transfer Learning

Setting In the first experiment we investigate autonomously learning a new category of target task with RL, given a set of known source tasks from a different category. We use CMA-ES as base learner with the initial parameter σ = 0.01. Our focus is on comparing the impact of different types of knowledge transfer, assuming the source tasks are well learned. For simplicity, we therefore learn the source with supervised demonstration followed by MTL RL refinement. There are multiple ways to evaluate transfer learning performance [START_REF] Taylor | [END_REF]. We report (i) the total reward/cost during learning and (ii) target task success rate -the percentage of experiments that the robot successfully completes (e.g., gets the ball in the cup) when learning terminates. For parsimony, we adopt an experimental design where each task category is considered in turn as both a source and a target. Therefore 4 task categories entail 16 transfer experiments. Each experiment considers families of 16 source and 50 target tasks. Alternatives We compare our tensor-based transfer approach with: Scratch: learning the target from scratch. Direct: A simple direct transfer learning baseline of initialising the target task to that of a (randomly chosen) learned source task. This is a common strategy of transfer by 'warm start' followed by fine-tuning [START_REF] Taylor | [END_REF]. Matrix: Matrix-based transfer approach, where the policy parameters for all tasks are structured as a matrix of size DA×P , knowledge sharing is achieved by SVD -thus only transferring knowledge across tasks and not also across actuators. These transfer approaches are thus about providing a good initial condition for the policy-search based RL of the target task, but ours transfers the latent tasks in a tensor structure. Results Fig. 4a shows illustrative learning curves of 4 of the 16 experiments. Note that only successful learned experiments are counted for the learning curves. Fig. 4b summarises learning success rate and total cost for all 16 experiments. We observe that: (i) Learning from scratch is feasible (it achieves success) although slow/costly for Reaching and Throwing. However it mostly fails to succeed at the harder BIC and Casting tasks. (ii) Among the transfer learning approaches, we see that our tensor-based approach is best in terms of total cost/learning speed, followed by matrix-based transfer, and direct's warm-start approach. (iii) Our tensorbased transfer is the only one to solve (high success rate) most tasks given most sources. Overall the results show that with a suitably designed policy and TL strategy, it is possible to learn challenging dynamic control tasks autonomously, even when transferring from very different and much easier source tasks (e.g., Reach→BIC). Previous solutions to BIC have required demonstration [START_REF] Stulp | [END_REF].

Comparative Analysis

Competitors We next compare our framework against three competitors with alternative policy representations, and transfer strategies. ELLA: Our implementation of [Ruvolo and Eaton, 2013b], a state of the art framework for matrixbased transfer of linear models. We adapt to our purpose by providing it the inputs [x t , t] our model uses. RBF-ELLA: As ELLA is designed for linear models, we generalise it for non-linear tasks through RBF-tiling the input space. FCNN: A fully connected multi-layer NN policy mapping [x t , t] to actions via one RELU hidden layer of 8 units is a conventional alternative to our DMP-inspired policy.

Results

We perform the same experiment as the previous section, considering 16 task pairs for transfer. Fig. 5 shows an illustrative example of the learning curve of each method along with summary statistics (i) Average learning cost, and (ii) % of total cost 'wins' (which method had the smallest cost in a given experiment). Fig. 5a illustrates our tensortransfer approach starting off better and converging quicker and to better asymptote than alternatives. Fig. 5b shows that we achieve much better total cost on average, and considering each experiment achieve the best total cost in most cases.

Discussion Why does our method perform so much better than ELLA which has previously shown convincing results on benchmarks like cart-pole, quadrotor, etc? The linear policy of ELLA can solve these classic benchmarks as they have a specific target state (e.g., balance/hover position) and are linearly controllable closed loop systems [START_REF] Mokhtari | [END_REF]] that do not require extended dynamic trajectory planning. However in our dynamic manipulation tasks, the objective is not a simple target state: it is an extended movement trajectory. Our policy class including a non-linear open loop layer as trajectory planner, and closed-loop controller can address this. ELLA can be seen as implementing only the linear feedback controller in the final layer of our network. RBF-ELLA is much more flexible and could potentially solve our tasks. But it suffers from the needing to tile RBFs in 3/5D (P/PD control), which means either an under-fitting policy, or very many parameters to train (we used 5 × 5 × 5 tiling to give it a similar number of parameters to ours).

Curriculum Learning

The previous experiments showed that with pre-learned source tasks, we can autonomously learn a new task category with transfer and RL. In this section we explore whether it is possible to learn dynamic manipulation tasks autonomously with no supervision anywhere in the pipeline, by constructing an appropriate training curriculum. The procedure starts with learning the easiest task-category from scratch with multitask RL, and the task-agnostic tensor is extracted and transferred to bootstrap learning the next task-category with transfer multi-task RL. We consider the task category curriculum Reach-Throw-BIC. We compare following this curriculum (CL) using tensor transfer against no curriculum (NC). We control for the total cost in rollouts giving both conditions 6000 trials of pre-training. In NC these iterations are used on MTL learning of BIC family tasks. In CL, they are spent on MTL learning of reaching then throwing task categories.

Based on these initial conditions, Fig. 5c, 5d shows the average cost and success rate of the following 4000 trials of MTL training on BIC for both approaches. The periodic bumps are caused by the alternating optimisation of L and s. The results show that the challenging BIC task can be au-tonomously solved by knowledge transfer from a curriculum of autonomously learned source tasks, thus achieving BIC without any demonstrating supervision.

Conclusions

We explored transfer learning to enable autonomous reinforcement learning of non-linear dynamic control tasks. Through our effective policy network representation and tensor based transfer of the latent task subspace, the speed and asymptotic success rate of autonomous RL of target tasks is significantly improved. With curriculum transfer learning, we were ultimately able to learn hard tasks such as BIC from scratch autonomously without any demonstration. In future work we will explore lifelong MTL within and across task families, and extend our framework to model tasks within families as contextual policies rather than discrete tasks.

Figure 2 :

 2 Figure 2: Transfer Learning Schematic. A set of P source tasks are learned and the matrices representing each source policy are stacked in to a tensor W. Factorising W separates task-specific, actuator-specific and shared knowledge. Task-independent latent skills are gathered in tensor L. For the target task, the agent learns the weights s that reconstructs L into policy parameters.

Figure 3 :

 3 Figure 3: Successful Movement Trajectory of Ball In Cup. The arm starts from static state and moves rightward firstly, generating the ball a momentum toward right, then moves leftward fast, generating the ball a large momentum toward upper left. After the ball is pulled back through the string, the arm moves the cup towards the ball's dropping point and catches the ball.

  Figure 4: Learning curves (a) and summary statistics (b) of different transfer strategies.

  Figure 5: (a,b) Comparison vs State of the Art. (c,d) Autonomously learning ball-in-cup from scratch via curriculum.

Acknowledgements

This research was supported by the European Union's Horizon 2020 research and innovation program under grant agreement No 640891.