Pauline Bouhier

Céline Cannes

David Lambertin

Christian Grisolia

Davide Rodrigues

Sylvie Delpech

Evaluation of several conditioning matrices for the management of radioactive metal beryllium wastes

Keywords: Beryllium, corrosion, conditioning matrices, thermodynamic data, electrochemical measurements

Introduction

Beryllium (Be) presents interesting properties, such as a low density, a relatively high melting point (1287 °C), a low neutron-capture cross section and a high neutron scattering cross section, a high specific heat and thermal conductivity, good mechanical properties at elevated temperatures and high oxygen gettering characteristics [START_REF] Walsh | Beryllium chemistry and processing[END_REF]. It is then employed in a wide range of applications and especially in nuclear industry. Be metal is currently employed in thermal reactors as a moderator, a reflector or fuel cladding [START_REF] Pih | A survey of beryllium technology and nuclear applications[END_REF][START_REF] Beeston | [END_REF][4][5]. It is also under study as interesting material for the future fusion power reactors. In the International Thermonuclear Experimental Reactor, Be constitutes the main plasma facing for the first wall of the tokamak and the neutron multiplier in the breeding blanket [6][7][8][9][START_REF] Pajuste | Jet Contributors[END_REF].

The use of beryllium in fission and fusion reactors will make it radioactive by neutron irradiation. Activated Be wastes will then be generated. Although some of the wastes can be cleared or recycled, it will remain large volume of metallic Be waste to consider. Depending on the dose rate, a near-surface or geological disposal can be proposed. Concrete encapsulation is one strategy to manage the low-and intermediate-level radioactive metallic wastes by isolating them from the environment [11][START_REF] Glasser | Application of inorganic cement to the conditioning and immobilisation of radioactive wastes, in Handbook of advanced radioactive wastes conditioning technologies[END_REF][START_REF] Milestone | [END_REF]. It consists on the immobilization of the metal in a cementitious matrix.

However, to ensure a safety storage, it is primordial to determine the reactivity of beryllium in the matrix and in the disposal environment. The literature on the radioactive beryllium waste management is really scarce. One reference considers four methods to condition the beryllium waste: cementation, bituminisation, vitrification and phosphatisation [14]. The authors concluded that phosphate and glass matrices would be the optimal solution to manage the beryllium waste.

However, for cementation, they have only considered ordinary Portland cement as conditioning matrix and they have assumed from literature data that the beryllium corrosion rate would be too high according to the pH of the pore water solution (higher than 12). However, recent experimental data have shown strong uptake of beryllium in ordinary Portland cement, low-pH cements and calcium-silicate hydrated cements, showing that these types of cement could be used as matrices for Be(II) conditioning [15,16]. This uptake is controlled by sorption processes in the cement solid phases and the solubility has been found not relevant in the studied conditions. However, the behavior of a metal in a conditioning matrix is mainly governed by its corrosion. Indeed, the major risk is the oxidation of the metal by the interstitial water, resulting in the hydrogen release. This can lead to gas driven transport of radionuclides out of the waste package.

The confinement is no longer ensured. Beryllium presents a serious drawback, a very high toxicity.

Beryllium in powder form is known as carcinogen and it can cause an incurable respiratory disease, the berylliosis [17,[START_REF] Dai | Proc. Natl. Acad. Sci[END_REF]. This may explain the few works on the chemistry of this element.

Corrosion studies are nevertheless reported in the literature [START_REF] Scott | [END_REF][20][21][22][23][24][25] which show that beryllium is highly corrosion resistant in water at low temperature. Moreover, studies have also shown that impurities in the surface can cause local corrosion [26] and the presence of aggressive anions in solution, such as chloride ions, can generate pitting corrosion [27][28][29][30]. The metal is protected by a layer of oxide BeO(s) or hydroxide solid phase Be(OH)2 (s) of 20 to 100 angstroms thickness, when exposed to air. The reactivity of beryllium is mainly related to the stability of this protective layer.

In acid solutions, the dissolution of the solid oxide layer is extensively proven, leading to a continuous corrosion of the metal. By contrast, in alkaline to hyperalkaline solutions, the studies on the solubility of the beryllium oxide or hydroxide solid phase are more controversial. While oldest studies based on thermodynamic calculation predict in alkaline solution the formation of anionic species, such as Be2O3 2- [START_REF] Pourbaix | Atlas d'équilibres électrochimiques[END_REF], more recent studies show that the predominant species in alkaline to hyperalkaline (NaOH 5 mol.L -1 ) solution are Be(OH)2, Be(OH) 3-and Be(OH)4 2- [START_REF] Çevirim-Papaioannou | [END_REF].

As anionic species have been formed in alkaline media, beryllium is not protected against the corrosion by a passivation phase. In such conditions, the corrosion is continuous. Although some studies have shown a strong uptake of beryllium species in low and high alkaline cementitious matrices, the high beryllium corrosion leads to a continuous production of hydrogen gas, which can compromise the nuclear waste confinement [15,16].

We have undertaken a study on the reactivity of metal beryllium in conditioning matrices to propose the safest storage of the radioactivepackages containing beryllium waste. This study is mainly based on (i) the exploitation of thermodynamic data reported in the literature, concerning the potential -pH diagram of beryllium in water to predict its corrosion behavior in cementitious matrices and (ii) the electrochemical measurements carried out on beryllium incorporated in a cement as a function of time and cement formulations. Five types of cement have been selected with pore water solution presenting different pH ranges: brushite cement, magnesium phosphate cement, calcium sulfoaluminate cement, ordinary Portland cement and activated slag. Two techniques have been selected: monitoring the open circuit potential (OCP) with time and electrochemical impedance spectroscopy (EIS) at OCP.

Experimental section

Chemical products and cement preparation

The activated slag (AS) was prepared by mixing 95 wt% of slag (from Ecocem) and 5 wt% of a 2.5 M NaOH solution.

The ordinary Portland cement (OPC) grade CEM I 52.5 PM-ES was selected. It is a low alkali cement with low tricalcium aluminate content.

The calcium sulfoaluminate cement (CSA) was prepared by mixing 80 wt% of calcium sulfoaluminate clinker with 20 wt% of anhydrite.

Magnesium phosphate cement (MPC) was prepared in a Turbula blender by mixing "hard-burnt" magnesium oxide, potassium dihydrogen phosphate, fly ash, and boric acid for 20 minutes in the following proportions (wt%): MgO: 7.62; KH2PO4: 25.71; fly ash: 33.33; sand: 33.33 H3BO3: 2.

The brushite cement was a commercial binder (FOTIMINE, from Sulitec) comprising a Fotimine powder A (wollastonite CaSiO3 (98.4%) and calcite CaCO3 (1.6%)) and a Fotimine solution B (phosphoric acid (9.3 ± 0.5 mol.L -1 , borax Na2B4O7 (0.15 ± 0.01 mol.L -1 ) to be mixed in the 1.25/1 mass ratio.

The characteristics of the types of cement are reported in the Table 1.

When cements consisted of a powder mixture, they were mixed in a Turbula blender for 20 minutes. Cement paste volumes of about 100 mL were prepared with a three bladed propeller stirrer according to the following protocol: 1) start with demineralized water, 2) add premixed powders while stirring at low speed (≈100 rpm) and 3) mix for 2 minutes at middle speed (≈600 rpm). 4) mix for 3 minutes at high speed (≈1200 rpm).

The paste water content was adjusted as necessary according to the types of cement; the cements differed by their fineness and chemical water demand (Table 2).

Cement pH measurements

The pH measurements were carried out with a Consort C3010 pH meter and a Mettler Toledo Inlab Expert Pt1000 pH probe. Calibration was performed with IUPAC pH standard solutions 1.679 -4.005 -6.865 -10.012 and 12.45 (± 0.01).

The pore fluid pH of cement was carried out according to the protocol set up by Alonso et al. [START_REF] Çevirim-Papaioannou | [END_REF]: 300g of cements were prepared and divided into samples of approximately 30 g in tightly closed 50 mL vials. Samples were stored at room temperature (21 °C ± 1 °C) over a period of 30 minutes to 28 days. At each measurement deadline, a sample was ground (Ø < 1 mm), weighed precisely and suspended in demineralized water with a mass ratio of cement/water of 1. The sample is then stirred for 5 minutes before measuring the pH of the suspension. For the activated slag cement, a mass dilution of 100 of the suspension in demineralized water was performed to measure a pH within the calibration range of the pH meter.

Electrochemical measurements

A three-electrode cell was used for the electrochemical measurements, with a working electrode (beryllium wire), a counter electrode (platinum wire) and a quasi-reference electrode (platinum wire). The electrochemical set-up is reported in the Figure 1. The potential of the Pt wire was measured in solution, against a Ag/AgCl reference electrode, and was calculated against a normal hydrogen electrode (NHE) as a function of the pH (Figure 2). All potentials presented in this manuscript were calculated with respect to the NHE.

The beryllium wire (1.2 mm diameter, Goodfellow, purity 99.7%) was cut at different lengths. The surface of the Be electrodes was 1.17 cm 2 in the brushite cement, 0.98 cm 2 in the magnesium phosphate cement, 1.13 cm 2 in the calcium-sulfo aluminate cement, 1.21 cm 2 in the ordinary Portland cement and 1.24 cm 2 in the activated slag. The Be wires were introduced in small pipette tip and maintained with a rapid glue (Araldite). The so-prepared Be electrodes were cleaned by rinsing them with ethanol and were pickled with a HCl (0.5 M) solution until a light grey appearance (around 1 min). Then they were rinsed thoroughly with demineralized water and dried with a tissue paper.

The Pt wire (diameter: 1 mm, Goodfellow, purity 99.9%) was cut to obtain in the electrochemical set-up an electrode surface of 1.26 cm 2 . The wire was cleaned with ethanol and heated until brightred color with a blowtorch.

As beryllium powder is highly toxic, the Be wire is stored and handled under a glove box, in case of formation of powder during its handling. The electrochemical experiments were carried out in this glove box. Just after their preparation, the cement pastes were transferred to plastic pots, which were immediately closed with the cap containing the electrodes. The five electrochemical cells were then stored in the glove box during all the measurement. The curing conditions are then: ambient temperature and under air. Previous experiments have shown that with this kind of setup, the diffusion of the ambient atmosphere is very limited. No carbonation of samples is expected.

Measurements were carried out using a potentiostat (AMETEK model VersaSTAT 4) piloted by the VersaStudio software. Electrochemical Impedance Spectroscopy (EIS) diagrams were

recorded at the open circuit potential with 10 mV amplitude. Frequency ranged between 10 5 and 10 -1 Hz, with 10 frequencies values per logarithmic decade.

Results and discussion

Thermodynamic data

The reactivity of an element can be evaluated by a thermodynamic approach. For example, the potential-pH diagrams of one element in a given media, calculated from the thermochemical data of all the different chemical forms of this element , are used to determine its speciation in this media. From these diagrams, in the case of a metallic element, three potential-acidity zones can be defined corresponding to the immunity, the passivation and the corrosion domain of the element.

The immunity zone corresponds to the stability domain of element in its metal state. In the corrosion zone, the metal is continuously oxidized to an ionic species. In the passivation zone, the metal is oxidized to a solid phase, which can protect the metal and limit the corrosion reaction depending on the characterization of this solid phase: adherence, compacity, porosity, solubility.

In general, in aqueous solution, the metal oxide or hydroxide phases allow a good protection against the corrosion for numerous metallic compounds.

Concerning beryllium, a first potential-pH diagram was calculated in water at 25°C by M. Pourbaix in 1963 [START_REF] Pourbaix | Atlas d'équilibres électrochimiques[END_REF] using the thermodynamic data of W.M. Latimer [START_REF] Latimer | The Oxidation States of the Elements and Their Potentials in Aqueous Solutions[END_REF] (Fig. 3). Pourbaix has considered the solid phases Be, Be(OH)2 amorphous, Be(OH)2  Be(OH)2  and BeO and the soluble species Be 2+ , Be2O 2+ , Be2O3 2-and BeO2 2-. The stability domain of the water is limited by its oxidation (O2/H2O) and its reduction (H2O/H2).

Figure 3 shows that Be is oxidized to Be(II) in water whatever the potential and the pH. In acidic media, for pH lower than 3, the stable form of beryllium is the soluble ionic species Be 2+ . In highly alkaline solutions, the predominant form is Be2O3 2-. The diagrams were calculated for several concentrations of Be(II), 10 -6 , 10 -4 and 10 -2 M.. In the case of beryllium, the passivation zone is expected to correspond to the stability domain of Be(OH)2 : pH from 3.9 to 10.7 for [Be(II)] = 10 - 6 M, pH from 2.9 to 11.7 for [Be(II)] = 10 -4 M and pH from 1.9 to 12.7 for [Be(II)] = 10 -2 M. In this pH range, the beryllium corrosion depends on the stability of the hydroxide layer. Pourbaix [START_REF] Pourbaix | Atlas d'équilibres électrochimiques[END_REF]The author has specified that the amorphous hydroxide is not stable in contact with air and water. It crystallizes in alpha and beta phase. The -phase is metastable and gradually turns into -phase, which is the most stable hydroxide phase considering the thermodynamic data.

The solubility of the hydroxide solid forms as a function of the pH was also calculated in the reference [START_REF] Pourbaix | Atlas d'équilibres électrochimiques[END_REF]. Differences between the solubility values calculated with the thermodynamic data and the values experimentally obtained were observed. The authors have attributed this discrepancy to an inaccuracy of the values of the free enthalpy of formation for the beryllium species.

However, no modification of the thermochemical data of Be(II) species is observed between 1965 and 1998, in the different reports from the National Institute of Standards and Technology [START_REF] Parker | NBS Technical Note 270-6 -Selected Values of Chemical Thermodynamic Properties, Tables for the Alkaline Earth Elements[END_REF][START_REF] Wagman | The NBS Tables of Chemical Thermodynamic Properties: Selected Values for Inorganic and C1 and C2 Organic Substances in SI Units[END_REF][START_REF] Chase | NIST-JANAF Thermochemical Tables, 4th Edition[END_REF]. The main difference between the thermochemical data in these reports and the ones given by Latimer is the absence of data for Be2O3 2-, which was the predominant species in the highest pH zone of the diagram calculated by Pourbaix. Studies on the hydrolysis of beryllium confirm the formation of monomeric species such as Be(OH)2  Be(OH)2 , Be(OH)2 aq and show the stability of polynuclear ones such as Be2(OH) 3+ , Be3(OH)3 3+ , Be5(OH)6 4+ and Be6(OH)8 4+ for concentrations of Be(II) in solution higher than 10 -4 mol.L -1 [38,[START_REF]Hydrolysis of Metal Ions[END_REF]. The associated constants of formation of these species have also been reported. In 2016, all the thermodynamic data on the speciation and hydrolysis of Be in aqueous solution have been published in the reference [START_REF]Hydrolysis of Metal Ions[END_REF] without considering Be(OH)4 2-. Very recently, an experimental study on the solubility of beryllium species has been carried out in acidic to hyperalkaline solutions to accumulate thermodynamic data on beryllium [START_REF] Çevirim-Papaioannou | [END_REF].

Based on the thermodynamic data reported in the literature [START_REF] Çevirim-Papaioannou | [END_REF][START_REF]Hydrolysis of Metal Ions[END_REF][START_REF]Thermodynamic Data for Fifty reference Elements[END_REF][START_REF] Shock | A Inorganic species in geologic fluids: Correlations among standard molal properties of aqueous ions and hydroxide complexes[END_REF], we have calculated the E-pH diagram of beryllium for a concentration of 10 -6 , 10 -4 and 10 -2 mol.L -1 at 25°C considering the following species: Be, Be 2+ , BeOH + , Be(OH)2, Be(OH)3 -, Be(OH)4 2-, Be2(OH) 3+ , Be3(OH)3 3+ , Be5(OH)6 4+ and Be6(OH)8 4+ . The hydroxide solid phase Be(OH)2  has been considered and not Be(OH)2  because the form α has been shown as the most experimentally stable phase at room temperature [START_REF] Çevirim-Papaioannou | [END_REF]. The diagrams calculated using the thermodynamic data of the references [START_REF] Çevirim-Papaioannou | [END_REF][START_REF]Hydrolysis of Metal Ions[END_REF][START_REF]Thermodynamic Data for Fifty reference Elements[END_REF][START_REF] Shock | A Inorganic species in geologic fluids: Correlations among standard molal properties of aqueous ions and hydroxide complexes[END_REF] are given Figure 4. The considered acido-basic reactions and the electrochemical reactions to draw the diagrams with the associated thermodynamic data are reported respectively in the Tables 3 and4. The diagram calculated at a beryllium concentration of 10 -6 M (Fig. 4) is very different to the one calculated by Pourbaix (Fig. 3). The cationic species Be(OH) + is the predominant species at pH from 5.4 to 7.2 and in the alkaline domain, two species are stable, Be(OH)3 -, from 11.7 to 13.3 and Be(OH)4 2-, from pH 13.3 to 16. The diagrams calculated at a concentration of 10 -4 and 10 -2 M present the same predominant species: Be 2+ , Be(OH)2 and Be(OH)4 2-. We can also observe a very short domain of stability for the species Be3(OH)3 3+ at 10 -2 M, from pH 4.28 to 4.32. At these concentrations, the diagrams calculated by Pourbaix and the ones calculated in this work differs mainly in the alkaline to hyper-alkaline region: the stable chemical form of beryllium is Be(OH)4 2-.instead of Be2O3 2--.

Moreover, the main difference of these diagrams compared with the diagram calculated by Pourbaix [START_REF] Pourbaix | Atlas d'équilibres électrochimiques[END_REF] is a shift of the passivation domain towards the highest pH values: 7.2-11.7 at 10 -6 M ; 5.3-13.5 at 10 -4 M ; 4.3-14.5 at 10 -2 M.. This pH shift is explained by (i) considering the α instead of  phase for Be(OH)2 and (ii) the absence of thermochemical data for Be2O3 2-in the recent databases.

Selection of conditioning matrices

The ordinary Portland cement (OPC) presents several intrinsic qualities: availability, low cost, ease of use, good mechanical strength and, in general, stability over time. The basicity of hydrated Portland cement is an advantage in the field of radioactive waste because this material is capable of insolubilizing a large number of radionuclides (generally conditioned at high oxidation states) [42,43]. Therefore, OPC is the mainly used matrix for conditioning low-or intermediate-level waste. However, this basicity can present an obstacle for conditioning reactive metals, which are oxidized in a soluble species in this alkaline pH domain with production of hydrogen gas. Five types of cement have then been selected with different pore solution pH, to determine the best matrix for beryllium waste conditioning: activated slag (AS), ordinary Portland cement (OPC), calcium-sulfoaluminate cement (CSA), magnesium phosphate cement (MPC) and brushite cement (BC).

The pH of these cements has been measured as a function of time (Figure 5). The reactivity of beryllium can be studied on the entire pH domain, since the brushite cement presents an acid pore solution (pH from 1.75 to 6.44), the magnesium phosphate cement a neutral pore solution (pH from 5.6 to 8.4), the calcium-sulfoaluminate has a basic interstitial solution (pH from 10.9 to 12.3), the Ordinary Portland cement is highly alkaline (pH from 12.5 and 12.9) and the pH values are the highest for the activated slag (from 12.9 to 13.8).

The beryllium corrosion depends strongly on the characteristic of the hydroxide solid phase: solubility, adherence, porosity, compacity. According to the thermodynamic data, two stability domains have been determined for this solid phase.. The pH domains of the selected cements have been superimposed to the stability domain of the hydroxide phase determined with the thermodynamic data at the three different concentrations. Two different chemical behavior are expected from the Figure 6.

The pH solution will also be modified by the corrosion reaction. When Be 2+ is oxidized by water, the reaction produces H2 but also OH -ions, which increases the pH of the pore solution. On the other hand, when anions are produced in high pH domain, the oxidation reaction consumes OH - , thus decreases the pH of the solution. In all cases, the tendency is to go to the stability domain of the beryllium hydroxide solid form. The influence of corrosion reaction on pH solution will depend on the amount of metal against the volume of matrix. In the frame of this study, the ratio (cement volume) / (metal volume) is close to 5000. Thus, we can consider that the variation of pH is only monitored by the cement setting.

If we consider the Pourbaix work, in the activated slag and the ordinary Portland cement, the hydroxide solid layer should be dissolved in the ionic species Be2O3 2-. A continuous aqueous corrosion of Be is therefore expected with production of hydrogen. In the calcium-sulfoaluminate cement, a corrosion should be observed during the first days of immersion and should progressively be diminished since the pH of this type of cement tends to 10.9, a value ranging in the pH domain of Be(OH)2 stabilityf. The corrosion of beryllium to the soluble Be 2+ species is expected in the brushite cement during the first day. However, the corrosion should be quickly limited by the formation of the hydroxide solid phase Be(OH)2. Finally, the magnesium phosphate cement is the only one with pore solution pH values in the domain of stability of the protective layer. The Be reactivity should be the lowest in this type of cement.

According to our calculations, three types of cement, OPC, CSA and MPC, present pore solution pH values ranging in the domain of the stability of the hydroxide solid Be(OH)2. Under these conditions, beryllium should be protected against the corrosion and these three types of cement could be selected as conditioning matrices for beryllium waste. Activated slag presents a mitigate case, as the pH values of its interstitial solution is too high during the first days. The predominant form of beryllium should be the soluble species Be(OH)3 -or Be(OH)4 2-. A corrosion should then take place in this type of cement during the first days of encapsulation. BC can definitively not be chosen for conditioning of beryllium wastes. According to the last thermodynamic data, the pH of this cement is too acid. Beryllium should be corroded in the Be 2+ soluble species with production of hydrogen during several days. This study put in evidence two different chemical behaviors of beryllium taking into account the thermodynamic data. It is then important to perform experimental measurements on the Be reactivity in the different types of cement to be able to propose a safe confinement for this nuclear waste.

Electrochemical measurements of Be in conditioning matrices

The reactivity of Be has been studied using the same electrochemical approach that the one adopted previously to study the reactivity of Al and U encapsulated in a conditioning matrix: open circuit potential and electrochemical impedance spectroscopy measurements [44][45][46].

OCP measurements

The open circuit potential (or the potential at zero current) characterizes the redox reaction involved at the electrode interface. It depends on the nature of the oxidizing agent and on the reaction kinetic. In the case of an aqueous corrosion, the potential is close to the reduction potential of water and if the metal is protected against the corrosion, the OCP tends to increase. The greater the difference between the OCP and the reduction potential of water, the less corroded the metal.

The OCP measurement can then indicate qualitatively the reactivity of the metal at any hydration time of the cement.

The variation of the beryllium OCP encapsulated in a matrix has been measured as a function of time (in day) for the five types of cement. As a usual reference electrode cannot be used in a cement, a platinum pseudo-reference has been selected for the electrochemical measurements. The calibration curve, potential of platinum against the normal hydrogen electrode (NHE) as a function of the pH, given in the experimental section (Fig. 2), has been used to calculate the potential of beryllium against the normal hydrogen electrode. Knowing the evolution of the pH of the interstitial solution of the cement, the Pt potential can be calculated for each measurement and the OCP measured on beryllium electrode can be given against the NHE reference electrode.

Figure 7 shows that for all the types of cement, OCP first increases and tends to a plateau. The OCP is the highest in the Portland cement, while it is the lowest in the activated slag during the first days of immersion. The increasing period of OCP and the constant values obtained at the end of the studied periods depend on the type of cement. After more than 100 days of immersion, the OCP is the highest in the magnesium phosphate cement and the lowest in the activated slag. The measurements on Be electrode in the brushite cement have been stopped after one month, because it was difficult to acquire electrochemical data with high quality. This has been attributed to the high production of hydrogen at the Be surface by aqueous corrosion in this acid cement. The electrode/medium is perturbed by the formation of this gas. Large bubbles have been observed in the vicinity of the beryllium electrode. Nevertheless, we cannot conclude on the reactivity of beryllium in the selected cements by comparing directly the variation of the OCP as a function of time for each type of cement. The reduction potential of water depends on the pH, which varies both as a function of the type of cement and time. In order to compare Be(II)/Be and H2O/H2 redox potentials, the theoretical potential of the redox couple H2O/H2 has been calculated by applying the Nernst equation at the measured pH of the cement at a given time (EH2O/H2 /ENH = 0 -0.059 pH). The values are reported Figure 7 and compared to the Be OCP ones for each type of cement.

The main observations are:

-At long term, in the Portland cement and in the calcium sulfoaluminate cement, the variations of Be OCP and H2O/H2 redox potential are similar. The difference between the two values is close to 0.8 V (Fig. 8). This result shows a very low corrosion of Be during all the immersion time. In these two types of cement, Be is in passivation conditions. The metal is protected by a stable layer formed on its surface. That agrees with our thermodynamic calculations presented in this paper.

-During the first days, Be seems more reactive in CSA than in OPC since the OCP is just 0.26 V higher than the H2O/H2 redox potential (Fig. 8). In CSA, hydrogen is produced at the beginning of the conditioning.

-In magnesium phosphate cement, the variation of the Be OCP is not similar to the one observed in the Portland and the calcium sulfoaluminate cements. It takes more than 40 days instead of 7 days to observe a plateau for the OCP values. At long term, the difference Be OCP -E (H2O/H2) reaches 0.7 V (Fig. 8), which is quite high. We suppose that after about 40 days of immersion, a protective layer is formed at the Be surface ensuring efficient passivation. By contrast, the first day, the OCP value is just 0.18 V higher than the redox potential of water. This result suggests that at the beginning, beryllium is corroded if it is immersed in a magnesium phosphate cement. That can be explained by the low pH of this cement at the early age. The protective layer seems not stable in these conditions.

-In the activated slag, a plateau is observed for the OCP variation with time from 25 days. The maximal difference measured between the Be OCP and the H2O/H2 redox potential is 0.59 V (Fig. 8), which is lower than the values measured in the OPC, CSA and MPC. Moreover, as this difference reaches values around 0.3 and 0.4 V during the first days, we can assume that Be is corroded in this type of cement during this period.

-The worst case is observed when beryllium is encapsulated in the brushite cement. During the entire studied period, the OCP values are comparable to the H2O/H2 redox potential (Fig. 8). We conclude that beryllium is highly corroded in the brushite cement with production of high amount of hydrogen. This production of gas has been confirmed by the formation of bubbles at the vicinity of the beryllium surface.

The OCP measurements are in good agreement with the results obtained by our thermodynamic calculation. Three types of cement can be retained for the Be waste conditioning: OPC, CSA and MPC, whereas the brushite cement is definitively not adequate. In BC, the Be corrosion is too high, leading to a too high production of hydrogen in the waste package. AS can also not be chosen for Be conditioning because of its aqueous corrosion during the first days of immersion.

EIS measurements

Electrochemical Impedance Spectroscopy is an interesting technique for understanding the reactivity of redox compounds and to access to data on the evolution of the interface metal/electrolyte with time. Moreover, if the EIS is recorded at the OCP, this technique presents the advantage not to alter the metal sample/medium interface. The reactivity of the metal can be examined in situ and during a long-time range. The electrochemical impedance spectra, in Nyquist and Bode representations, have been recorded on a beryllium electrode in the five types of cement.

We have previously shown that cement brings an electrical contribution to the impedance measurement [45]. In order to discriminate the impedance related to the cement from the one related to the beryllium reactivity, measurements on a platinum electrode, as an inert electrode, have been performed in parallel.

Impedance is generally modelled with an electrical circuit (Figure 9). Two parts of the circuit have to be distinguished. The first part corresponds to the impedance contribution to the cement matrix (Zmatrix). The details of this impedance are already described in the reference [45]. Each matrix has its own impedance contribution. The second part corresponds to the impedance related to the electrochemical process involving at the metal/matrix interface. This impedance is constituted of the double layer capacitance (Cdl) or a constant phase element (CPE) [START_REF] Bard | Electrochemical methods: Fundamentals and applications[END_REF] in parallel with the faradaic impedance (ZFAR), which is characteristic of the electrochemical reactions involving at the electrode/media interface. When a passivation process occurs at the metal/matrix interface, the faradaic impedance tends to a very high value and the current flows through the capacitive branch (or CPE). That is called "capacitance effect" and the shape of the Nyquist diagram is close to a capacitance response, which means a linear variation parallel to the Zi-axis.

-Cement contribution to EIS measurements

The EIS measurements realized on Pt electrode are given Figure 10. For all the matrices except CSA, the contribution of the cement involves at high frequencies between 10 5 and 10 3 Hz. In the case of CSA, the matrix contribution involves in the all frequency domain. That is particularly observed after 100 days with the apparition of a loop at high frequencies. This loop can be characteristic of (i) an electroactive species in the pore solution or (ii) an evolution of the porosity of the matrix.

-Be reactivity contribution to EIS measurements Figure 11 and 12 present the evolution of EIS diagrams recorded respectively in the brushite cement and the four other types of cement on Be electrode as a function of time. Be spectra recorded in BC (Fig. 11) are not exploitable probably because of H2 bubbles formation when Be is introduced in the matrix, which inhibits the electrical contact with the electrode and the potentiostat. The evolution with time of Be impedance recorded in OPC and MPC is low. In these two types of cement, a capacitive effect is observed on EIS spectra, corresponding to a passivation layer at the electrode surface. In AS, the impedance begins with a capacitive loop, which increases with time. That is explained by a passivation process, which occurs after 40 immersion days, probably in relation with the decrease of the cementitious matrix pH with time.

Measurements carried out on Be electrode in CSA present the same shape than the diagrams recorded on Pt electrode. It is not possible to separate the contribution of the matrix from the electrochemical process involved on Be electrode. No conclusion can be given from EIS spectra recorded in CSA. However, based on OCP measurements, it seems that Be is passivated in CSA during all the immersion period.

It is difficult to compare the reactivity of Be in OPC and MPC on the basis of a qualitative approach. The Bode diagrams present a frequency shift between these two types of cement. That can be due to the matrix contribution or to the double layer capacitance, which depends on the type of matrix. In both cases, a strong passivation process is observed. This passivation process is also observed in AS after 40 days. These observations are in agreement with the E-pH diagram calculated in this paper and with the evolution of the matrices pH with time.

Conclusion

Due to its specific properties, beryllium is a metal compound used in the nuclear industry. The management of Be nuclear waste has then to be considered. One way to manage the low-and intermediate-level waste is the encapsulation in a cement. To select the best matrices for beryllium waste conditioning, five types of cement, having pore solution with different pH, have been selected: activated slag, Portland cement, calcium sulfoaluminate cement, magnesium phosphate cement and brushite cement. The reactivity of beryllium has first been studied in water as a function of the pH, a key parameter, by using a theoretical approach. The beryllium data are relatively scarce. A potential-pH diagram has been established in 1963 by M. Pourbaix. According to these data, beryllium waste cannot be conditioned in highly alkaline solution, such as the pore solution of OPC, AS and CSA. If we consider only the pH parameter, a neutral medium such as the MPC seems the only cement capable of a safe conditioning of the beryllium waste. The evolution of the thermodynamic database and the recent published data have led us to re-calculate the E-pH diagram of Be which shows that Be is stable in neutral to highly alkaline solutions. Thus, the beryllium should be passivated by the hydroxide solid phase Be(OH)2 in OPC, CSA and MPC, while its corrosion is too high in brushite cement and activated slag to envisage them as conditioning matrices. To confirm these first results, an electrochemical study, by measuring the open circuit potential and the electrochemical impedance, has also been undertaken to follow the reactivity of beryllium in the different types of cement with time. This study is carried out in situ and offers the advantage of considering several parameters and not only the pH, such as the charge transfer kinetic and the level of passivation of the solid layer. The experimental results confirm the behavior of beryllium expected by our thermodynamic calculations. A summary of these results is given Table 5.

The beryllium nuclear waste can be conditioned in a Portland cement, a calcium sulfoaluminate cement or a magnesium phosphate cement. However, in MPC a corrosion is observed during the first days, while in OPC the beryllium is highly passivated during the entire studied time range. In the activated slag, the corrosion of beryllium is higher than in OPC and CSA during an immersion of 40 days. The production of hydrogen by aqueous corrosion should be too high to propose the AS as a conditioning matrix for Be waste. Moreover, the experimental data confirm that the brushite cement has to be excluded to encapsulate the beryllium. It would be interesting to confirm these results by measuring the volume of hydrogen gas produced in each type of cement by gas chromatography. This will be done in a future work. 4) with the values of the associated reaction enthalpy (G 0 ) from [START_REF] Çevirim-Papaioannou | [END_REF][START_REF]Hydrolysis of Metal Ions[END_REF][START_REF]Thermodynamic Data for Fifty reference Elements[END_REF][START_REF] Shock | A Inorganic species in geologic fluids: Correlations among standard molal properties of aqueous ions and hydroxide complexes[END_REF]

Figure 1 :

 1 Figure 1: Electrochemical set-up for open circuit potential and electrochemical impedance spectroscopy measurements.

Figure 3 :Figure 4 :

 34 Figure 3: Potential -pH diagram of beryllium for 10 -6 (--), 10 -4 (--)and 10 -2 (-•) mol.L -1 inwater at 25°C, calculated by M. Pourbaix[START_REF] Pourbaix | Atlas d'équilibres électrochimiques[END_REF]

Figure 5 :

 5 Figure 5: Variation with time (in day) of the pore solution pH of conditioning matrices, activated slag (AS), ordinary Portland cement (OPC), calcium sulfoaluminate cement (CSA), magnesium phosphate cement (MPC) and brushite cement (BC).

Figure 6 :

 6 Figure 6: Superimposition of the pH domains of the selected conditioning matrices and the pH domains of the Be(OH)2 stability determined in the reference [31] (grey lines) and in this work (black lines).

Figure 7 :

 7 Figure 7: Comparison of the variation of the beryllium Open Circuit Potential (OCP) and the potential of the H2O/H2 redox couple with time (in day) in the ordinary Portland cement (OPC), calcium sulfoaluminate cement (CSA), magnesium phosphate cement (MPC), activated slag (AS) and brushite cement (BC).

Figure 8 :

 8 Figure 8: Variation of the difference between the beryllium OCP and the H2O/H2 redox potential measured in the five types of cement as a function of time (in day).

Figure 10 :

 10 Figure 10: Electrochemical impedance diagrams (Nyquist (left) and Bode (right) representation) of a platinum electrode immersed during 7 and 101 days in ordinary Portland cement (OPC), calcium sulfoaluminate cement (CSA), magnesium phosphate cement (MPC) and activated slag (AS).

Figure 11 :

 11 Figure 11: Electrochemical impedance diagrams (Nyquist (left) and Bode (right) representations) recorded on a beryllium electrode at the OCP, with a 10 mV amplitude, in brushite cement (BC) for different immersion times (in day).

Figure 12 :

 12 Figure 12: Electrochemical impedance diagrams (Nyquist (left) and Bode (right) representation) of a beryllium electrode immersed during different times (in day) in ordinary Portland cement (OPC), magnesium phosphate cement (MPC), activated slag (AS) and calcium sulfoaluminate cement (CSA), recorded at the OCP with a 10 mV amplitude.

Table 2 :

 2 Water content of cement pastes

	Type of cement	Activated slag	Portland	Calcium sulfoaluminate cement	Magnesium phosphate cement	Brushite cement
	w/c	0.50	0.40	0.55 a	0.51 b	1.25 c
	a water / (calcium sulfoaluminate clinker + gypsum); b water / (MgO + KH2PO4); c solution/cement

Table 3 :

 3 Acido-basic reactions considered for the calculation of the E-pH diagram (presented Figure4) with the values of the associated reaction constant (K) from[START_REF] Çevirim-Papaioannou | [END_REF][START_REF]Hydrolysis of Metal Ions[END_REF][START_REF]Thermodynamic Data for Fifty reference Elements[END_REF][START_REF] Shock | A Inorganic species in geologic fluids: Correlations among standard molal properties of aqueous ions and hydroxide complexes[END_REF].

	Acido-basic equilibrium	log K
	Be 2+ + H2O  BeOH + + H +	-5.39
	Be 2+ + 2H2O  α-Be(OH)2 + 2H +	-6.6
	2Be 2+ + H2O  Be2OH 3+ + H +	-3.54
	3Be 2+ + 3H2O  Be3(OH)3 3+ + 3H +	-8.83
	5Be 2+ + 6H2O  Be5(OH)6 4+ + 6H +	-19.1
	6Be 2+ + 8H2O  Be6(OH)8 4+ + 8H +	-26.3
	Be 2+ + 3H2O  Be(OH)3 -+ 3H +	-24.3
	Be 2+ + 4H2O  Be(OH)4 2-+ 4H +	-37.6

Table 4 :

 4 Electrochemical reactions considered for the calculation of the E-pH diagram (presented Figure

Table 5 :

 5 and of the standard potential (E 0 ). Summary of the passivation of Be expected from thermodynamic calculations, OCP and

	Electrochemical reaction	Nernst Relation								G 0 (J.mol -1 )	E 0 (V/NHE)
	Be 2+ + 2e -→ Be	E E =  +	2.3RT 2F	2 log[Be ] +								378922	-1.96
	Be(OH) + + H + + 2e -→ Be + H2O	E	E =  +	2F 2.3RT	log[Be(OH) +	]	-	2F 2.3RT	pH	348207	-1.80
	Be(OH)2(α) + 2H + + 2e -→ Be + 2H2O	E	E =  -	F 2.3RT	pH								341312	-1.77
	Be(OH)3 -+ 3H + + 2e -→ Be + 3H2O	E	E =  +	2F 2.3RT	3 log[Be(OH) -	]		-	2F 2.3RT	3pH	240450	-1.25
	Be(OH)4 2-+ 4H + + 2e -→ Be + 4H2O	E	E =  +	2F 2.3RT	2 4 log[Be(OH)	-	]	-	F 2.3RT	2pH	164651	-0.85

Author information

Corresponding author: celine.cannes@ijclab.in2p3.fr / [START_REF] Alonso | Cement-Based Materials for Nuclear Waste Storage[END_REF]


Laurent Maurin

Hamid Makil

Jean-Michel Bourbotte

Hassen Hamrita

Gwenael Jouvin

Vladimir Kondrasovs

Stephane Rougeault

Guy Cheymol

Hicham Maskrot

Loic Barbot

J.-M Bourbotte

J.-F Villard

V Bouyer

A Denoix

C Journeau

J.-C Jaboulay

C Guénaut

M Brovchenko

I Duhamel

L Pichon

S Fourrez

Y Ouerdane

P Ferdinand

Long length SPNDs and Distributed Optical Fiber Sensors for Severe Accident remote monitoring & their contribution to Nuclear Safety in the post-Fukushima context

IRESNE Technical Meeting on Advanced Technologies and Systems for Containment Preservation in Accident Conditions

Rhodium Platinum Background Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit Measurement limit

Measurement limit Measurement limit Measurement limit

Distributed

  position candidates for SPND & Distributed OFS Presentation linked to SAMMI-2020 publication, available at: https://hal-cea.archives-ouvertes.fr/cea-03324457 Post-Fukushima project Total loss of electrical power sources following the tsunami, TEPCO NPP operator with almost no data about the accident status and its evolution  safety must always prevail Aimed at providing innovative solutions based on passive sensors operated from a remote and safe place, in order to: Monitor the normal operation of the reactor during 60 + years Monitor the Severe Accident to apply on time appropriate mitigation strategies & keep informed nuclear safety authorities about: Vessel breakthrough Corium pouring on the floor and its interaction with concrete until the prospective breakthrough Long term corium cooling once relocated in the core catcher (Gen III) 2 technologies: ex-core long length SPNDs-TC & Distributed OFS ex-vessel measurements designed for Gen II & Gen III (EPR) 7 partners (Public R&D, Academic R&D, Industrials) Ended in Dec. 2018 from IRSN, 2013 ex-vessel positions candidates (reactor pit, melt discharge channel & spreading compartment) for long length SPNDs-TC & OFS cables according to specifications Candidates for sensors locations Gen III (EPR) SPND -currents self-generated under radiations -battery operated capable monitoring system -long (several meters) lengths SPNDs to get the adequate sensitivity for ex-core detection -rely on the properties of the backscattered light in optical fibers and reflectometry -passive sensors with remote measurement systems up to several km away from the NPP area OFS Information remotely obtained from a safe place, even in case of total loss of power supplies: -to increase the nuclear safety, -to reduce operator exposition to ionizing radiations, -to avoid the breach of 3 rd containment barrier & environmental consequences with on-time & appropriate mitigation strategies SPND current modelling SPND current modelling using CEA MATiSSe toolbox • Input data: neutron & gamma energy spectra & fluxes • Accurate modelling of the SPND and its environment • { SPND instrumented pole, concrete, corium } whole set • MONTE CARLO MCNP6.1 code using JEFF3.2 & ENDF/B-VII.1 libraries • 3 main sources:  n: rate of 𝑛, 𝛽 -reactions & 𝑛, 𝛾 𝛾, 𝑒 contributions  e: 𝛽 -transport  ɣ: 𝛾, 𝑒 contributions • MATiSSe code qualified along with several experimental validations  L. Barbot et al., "Calculation to experiment comparison of SPND signals in various nuclear reactor environments", ANIMMA 2015, 20-24 April 2015, Lisbon, Portugal.  L. Barbot et al., "Experimental Validation of a Monte Carlo based Toolbox for SPND and PGD Simulation in the OSIRIS MTR", PHYSOR 2016, 1-5 May 2016, Sun Valley, USA. physical equation implemented in the CEA MATiSSe toolbox for SPND current modelling

 2 ••• 2 (

 22 From Normal Operation to Severe Accident with corium in the vessel (no relocation)  Vessel breakthrough with corium pouring on the floor • Dedicated amplification device [1 pA 100 nA] • Qualified with 60 Co source projector & in research reactor . Barbot et al., 2018 Results achieved in DISCOMS project prior to final MCCI VULCANO experiment OFS in the reactor pit makes sense • Modelling ɣ doses & fast neutrons (E > 1 MeV) fluence in the reactor pit concrete floor [M. Brovchenko, 2016]  Concrete floor shielding provides a good protection against ionizing radiations NO: Normal Operation / SA: Severe Accident  60 years NO is the most damaging process in comparison with 24 hours SA • Rad-Hard optical fibers selection & qualification in ɣ irradiator [G. Cheymol, 2020]  Possible to assess RIA < 50 dB/km at 2 MGy at both 1310 nm & 1550 nm wavelengths  100 m per cable in the reactor pit  5 dB optical budget for the optical fiber under test  Cable polymer compounds under ionizing radiations produce molecular H Protective carbon layer against diffusion of di-hydrogen H 2 in fiber core is mandatory • Industrial optical sensor cables incorporating Rad-Hard optical fibers • Compliance to EDF PMUC standard required in the call for tenders from surface to -50 cm fast neutrons (60 y NO) [n/cm 2 ] kerma gamma (60 y NO + 24 h SA) [MGy] Gen II from 2 × 10 16 to 2 × 10 14 ÷100 from 5.7 to 0Tests up to 750°C (circuit integrity)  Severe Accident scenario Optical Fibers Sensor (strain, temperature, security) cables qualification OFS cables qualification tubular oven with optical sensor cable in a pipe (to protect from fumes) interesting cables are: -the security cable  up to 750°C (90 + min) -the strain sensor cable  earlier break 8 security cable after 2 hours at 750°C CEA IAEA Technical Meeting on Advanced Technologies and Systems for Containment Preservation in Accident Conditions | 11-13 Oct. 2022 | L. Maurin et al. Photon-counting & telecom OTDRs  cables length • Brillouin B-OFDA (closed loop)  { ∆𝜀 & ∆𝑇 } • Rayleigh OFDR  high-resolution { ∆𝜀 & ∆𝑇 } • Raman DTS  { 𝑇 } SPNDs)-TC poles • Current measurement on the background cable • Temperature measurement with the TC in the instrumented poles• 50 kg prototypical UO 2 (no radiation)• Corium obtained with strong exothermic thermite reaction• 100 + thermocouples to monitor the temperature propagation front concrete

(Fukushima 2011 scenario)

Summary


Thomas Pellegrini
email: thomas.pellegrini@irit.fr

LANGUAGE-BASED AUDIO RETRIEVAL WITH TEXTUAL EMBEDDINGS OF TAG NAMES

Keywords: Language-based audio retrieval, Pre-trained representations, PaSST, MPnet, Transformers, AudioSet tags

published or not. The documents may come

INTRODUCTION

The ability to retrieve audio recordings from a text query can be very useful when searching for recordings in a large audio database. Language-based audio retrieval systems rank audio samples according to their match with a queried caption.

Recent systems are usually comprised of two neural networks: a text encoder, responsible for encoding a concise and dense representation of a textual query, and an audio encoder that encodes the audio content of the candidate recordings [START_REF] Xie | Unsupervised audio-caption aligning learns correspondences between individual sound events and textual phrases[END_REF][START_REF] Xie | Dcase 2022 challenge task 6b: Language-based audio retrieval[END_REF]. Both encoders are expected to provide representations embedded in the same subspace, where the textual and audio samples can be compared in terms of a distance.

In this paper, we present a baseline system that follows this architecture, based on two open-sourced pre-trained models to extract text and audio embeddings. Our main innovation is two-fold: i) we use logits as basic audio embeddings [START_REF] Tzinis | Improving universal sound separation using sound classification[END_REF], instead of the more usual 2-d feature maps extracted from a hidden layer of a pretrained deep neural network [START_REF] Xu | The SJTU System for DCASE2022 Challenge Task 6: Audio Captioning with Audio-Text Retrieval Pre-training[END_REF][START_REF] Mei | Language-based audio retrieval with pre-trained models[END_REF][START_REF] De Gail | Take it easy: Relaxing contrastive ranking loss with CIDEr[END_REF], ii) we improve our baseline system by incorporating information from tags predicted on the audio recordings. We propose to combine the basic audio logit embeddings with the textual embeddings of the tag names.

Thanks for funding by the French ANR agency (LUDAU, ANR-18-CE23-0005-01) and "Investing for the Future -PIA3" AI Interdisciplinary Institute ANITI (ANR-19-PI3A-0004). 1 PyTorch code available: https://github.com/topel/ my-audio-retrieval-dcase2022 After describing our system architectures and experimental setup, we report results obtained on Clotho v2 [START_REF] Drossos | Clotho: An audio captioning dataset[END_REF]. We then discuss our architecture design choices in depth.

SYSTEM DESCRIPTION

Our system architecture is shown in Fig. 1, with the audio encoder on the right part of the figure, and the caption encoder on the left part. We used two open-sourced pretrained transformers as encoders. Both were "frozen": the embeddings were extracted only once offline and used as input to our systems. The learnable part of the model corresponds to a single linear layer that projects the audio embeddings onto the caption embedding subspace.

Text encoder: sentence embeddings with MPnet

The textual queries (captions) are encoded as sentence embeddings, obtained by averaging the word embeddings outputted by a sentence transformer. The all-mpnet-base-v2 model [8], further referred to as MPnet, was used to extract these embeddings ⃗ es, which are 768dimensional ℓ2-normalized vectors. MPnet is a transformer of more than 109 M parameters, trained on over a billion pairs of sentences. It was chosen since it was reported as the best model for sentence embedding against a selected list of other models [9]. We apply Layer normalization [START_REF] Ba | Layer normalization[END_REF], followed by ℓ2-normalization is applied to the embeddings to obtain the final ones, named ⃗ eq: ⃗ eq = ℓ2-norm (LayerNorm (⃗ es)))

Layer normalization (LayerNorm, LN) is used to normalize the samples of a minibatch independently, with adaptive per-dimension bias and gain (scale). We found that using those led to slight overfitting, thus, we removed this option. Therefore, our text encoder does not have any learnable parameter.

Audio encoder: AudioSet logit embeddings with PaSST

In the proposed system, an audio recording is encoded as a single vector, a so-called scene embedding, using the Patchout transformer named PaSST [START_REF] Koutini | Efficient training of audio transformers with patchout[END_REF], pretrained on AudioSet [START_REF] Gemmeke | Audio set: An ontology and human-labeled dataset for audio events[END_REF]. We use the "logits" embedding outputted by PaSST, which is a 527-dimensional dense vector, comprised of the logits predicted for the 527 AudioSet event tag classes. We denote this embedding ⃗ e l .

Layer-norm is then used, in this case with adaptive bias and gain, followed by a linear layer and a final ℓ2-normalization. The final embedding is denoted ⃗ ea. We can summarize these steps in Eq. 2: We will, here-after, refer to our baseline system using PaSST and MPnet as PaSST-MPnet.

⃗ ea = ℓ2-norm (Linear (LayerNorm (⃗ e l ))) (2)

Augmented audio encoder: tag embeddings with MPnet

Adding information from tags and audio event predictions was shown to help audio captioning systems [START_REF] Ye | Improving the performance of automated audio captioning via integrating the acoustic and textual information[END_REF][START_REF] Gontier | Automated audio captioning by fine-tuning bart with audioset tags[END_REF][START_REF] Han | Automated audio captioning with weakly supervised pre-training and word selection methods[END_REF]. It should also help in our retrieval task. In this work, we tried to add such information within the audio encoder, by using MPnet to encode tag names as text embeddings. This tag name encoder is shown in Fig. 1, on the left of the audio encoding branch.

AudioSet includes 527 audio event classes, with names that can be single words, e.g., "Water", "Rain" or a sequence of words, such as "Child speech, kid speaking". We precomputed 527 tag name embeddings, extracted with MPnet.

For a given audio file, we use the PaSST logits ⃗ e l to weight each of the 527 embeddings, sum them to obtain a single tag embedding ⃗ et, according the following equations. First, the PaSST logit embedding is passed through a sigmoid function to obtain multilabel probabilities ⃗ p, then we compute the weighted sum of the precomputed tag embeddings:

⃗ p = sigmoid (⃗ e l ) (3)
⃗ et = ℓ2-norm     ⃗ p c pc Wt     (4)
where Wt ∈ R 527×d is the matrix containing the 527 embeddings of the AudioSet tag names. d is the dimension of the embeddings provided by MPnet (d = 768). c is an index over the 527 tag categories.

After an ℓ2-normalization, ⃗ et is combined to ⃗ ea with a convex linear summation:

⃗ eat = λ ⃗ ea + (1 -λ) ⃗ et (5)
where λ ∈ [0, 1] is a weight tuned on a validation subset. The final audio embedding ⃗ eat is ℓ2-normalized before being compared to the caption query embedding:

⃗ eat = ℓ2-norm (⃗ eat) (6)
The learnable parameters of the proposed system correspond to the audio encoder ones: in the linear layer and in the element-wise adaptive biases and gains of LN, with a total of 407 k parameters. If we take into account the large-scale transformers, this number increases up to 196M parameters, but as previously mentioned, we did not fine-tune the transformers.

Training objective

We used the same loss and similarity scoring functions than in the challenge baseline system [START_REF] Xie | Dcase 2022 challenge task 6b: Language-based audio retrieval[END_REF]. Similarity between audio and caption embeddings is estimated by taking the dot product between their embeddings, and all our systems were trained with a samplingbased triplet loss [START_REF] Harwath | Unsupervised learning of spoken language with visual context[END_REF]. We would like a model to provide a similarity score S p high for positive audio-query pairs, and low for either an audio embedding not paired with the caption query (audio impostor score S n a ), or for a caption not corresponding to the audio embedding (query impostor score S n q ). The audio and query impostors are randomly selected from the minibatch of samples being processed. The loss function as a function of the network learnable parameters θ is the sum of the contribution of these three scores:

Development-testing split

Evaluation dataset System # params mAP@10 R@1 R@5 R@10 mAP@10 R@1 R@5 R@10

L(θ) = 1 N batch max(0, S n a -S p +η)+max(0, S n q -S p +η) ( 7
)
where N is the number of samples in the minibatch, η a margin.

Evaluation

Once a system is trained, retrieval is performed by scoring the similarity between the caption embedding and the audio embeddings of all the candidate audio files. The same scoring function as during training is used: the dot product. Following the DCASE 2022 challenge requirements, our systems retrieve ten audio files for each queried caption, sorted according to their match with the query. Standard information retrieval metrics are used to evaluate the audio retrieval models: mean average precision at top-10 (mAP@10) [START_REF] Kaur | Comparative analysis on cross-modal information retrieval: A review[END_REF] and recall at k, the number of retrieved audio items considered for scoring (R@k with k ∈ {1, 5, 10}) [START_REF] Koepke | Audio retrieval with natural language queries: A benchmark study[END_REF]. To estimate these metrics, only a correct audio-caption pair is considered as positive, all the other audio recordings retrieved are considered negative. To rank the systems submitted to the DCASE 2022 challenge, mAP@10 was the main metric [START_REF] Xie | Dcase 2022 challenge task 6b: Language-based audio retrieval[END_REF]. It is a rank-aware metric. For a given queried caption, the precision values are averaged at the positions where the relevant items are in the retrieved rank list. The R@k metric is rank-unaware, and corresponds to the proportion of relevant items among the top-k retrieved results. For both mAP@10 and R@k, the higher the value, the better the system.

EXPERIMENTAL RESULTS

Datasets

We conducted our experiments on Clotho v2 [START_REF] Drossos | Clotho: An audio captioning dataset[END_REF], with the splits defined in the DCASE 2022 challenge [19]. The development set is comprised of 3839 audio clips with 19195 reference captions for training, and 1045 audio recordings with 5225 captions for testing. We refer to this test subset as development-testing, dev-test in short. We did not use the validation set. A Clotho evaluation set was provided in the challenge, used to rank the submitted systems. It contains 1000 audio files with 1000 associated captions [START_REF] Xie | Dcase 2022 challenge task 6b: Language-based audio retrieval[END_REF]. We also used for pretraining the training subset of AudioCaps [START_REF] Kim | Audiocaps: Generating captions for audios in the wild[END_REF], containing 46231 audio files with one reference caption per file.

Experimental setup

All our models were trained on Clotho v2 for 50 epochs with the same setup: minibatches of 128 samples, a 1e-3 initial learning rate, and the Adam optimizer. We used a reduce-on-plateau scheduler based on the Clotho validation split loss, with a 0.5 ratio and a 5epoch patience. Smaller and larger batch sizes were tested, but 128 was found to be the best one. We used η = 0.4, and λ = 0.8 when using the tags, as will be discussed here-after. When pretraining a model on AudioCaps, the number of epochs was 100 and no learning rate scheduler was used.

Results

Table 1 shows the results of our systems, together with the ones of the baseline proposed by the DCASE 2022 Task 6b challenge organizers (based on a convolutional recurrent neural network and pretrained word embeddings), and of a CNN-transformer [START_REF] Mei | Language-based audio retrieval with pre-trained models[END_REF], which reached the second rank out of ten teams in the challenge, with models comprised of a number of parameters similar to ours.

First, our systems based on pretrained transformers for both audio and text encoding largely outperformed the challenge baseline, which was trained from scratch on the Clotho v2 development dataset. Relying on pretrained models is efficient in this case, where labeled training data is scarce. Second, our results are worse by about 0.02 on the evaluation dataset (eval) than on the developmenttesting subset (dev-test). This is probably due to the fact that model selection was performed on this subset, so overfitting might have happened. It can also be that eval is more difficult than dev-test.

Compared to our baseline model PaSST-MPnet, adding information from the tag embeddings was beneficial: PaSST-MPnet-tags reached 0.234 mAP@10 compared to 0.229, on dev-test, and 0.214 compared to 0.212 on eval.

Pretraining on AudioCaps (PaSST-MPnet-tags-AC) brought a 0.06 absolute mAP@10 improvement on dev-test, compared to PaSST-MPnet-tags. Interestingly, this gain was not observed on eval, so we cannot conclude that pretraining was beneficial or not. PaSST-MPnet-tags models trained on AudioCaps, before finetuning on Clotho, reached about 0.18 mAP@10 on Clotho dev-test.

Finally, an ensemble of two PaSST-MPnet-tags-AC models trained with different seeds, led to our best results. By ensemble, we mean that the distance scores outputted by the two models were averaged. This setting allowed us to obtain the fourth place of the challenge [19].

ARCHITECTURE DESIGN CHOICES

In the following, all reported experiments were conducted with PaSST-MPnet-tags models.

Influence of LayerNorm in the audio encoder

As described in Section 2, in our models, we use LayerNorm (LN) with adaptive gain and bias applied to the PaSST embeddings in the First, if we do not use LN in the audio encoder, mAP@10 and the other metrics are much worse: 0.217 mAP@10, compared to 0.234 reported in Table 1.

Second, we tried to use batch normalization (BatchNorm, BN) in the audio encoder, instead of LN. Using BN and keeping all the rest of the architecture unchanged (and for a given random seed), led to a 0.229 mAP@10 value, again worse than 0.234 when using LN. By observing the learning curves, BN seems to overfit a little more than LN. When using adaptive bias and scale, the number of learnable parameters is the same for both types of normalization, i.e. twice the dimension of the embeddings. The only possible reason why BN overfits more is because of the running means and standard deviations estimated on the training subset of data, that may not generalize so well on the test subset. These are necessary for BN but not for LN, since LN performs a per sample normalization.

Impact of the margin η

In [START_REF] Harwath | Unsupervised learning of spoken language with visual context[END_REF], the authors used a default value of 1.0 for the margin η, used in the loss function, given in Eq. 7. In the DCASE challenge baseline, this default value was also used. In our case, the audio and queried caption embeddings are of unit norm. The results of the dot products, i.e., the similarity scores, are theoretically between -1.0 and 1.0. A margin of 1.0 might be to difficult to obtain when training a model. Indeed, η = 1.0 led to a 0.225 mAP@10 on dev-test, worse than 0.234 obtained with η = 0.4. We tried several values lower than 1.0, and η = 0.4 was the best value.

In Fig. 2, we plotted the score distributions of the positive pairs (Pos, a positive caption-audio pair) and negative pairs (A imp and Q imp for audio and query impostors, resp.), on the training subset. The distributions on the dev-test subset are very similar. The difference between the top and the bottom plots is the value of the margin: η = 0.4 (top figure) and η = 1.0 (bottom figure). As can be seen, the support interval of the scores is smaller with η = 0.4 than with η = 1.0: the positive pair scores reach about 0.62 maximum with η = 0.4, and about 0.8 with η = 1.0. Nevertheless, we can see more overlap with impostor scores in the case of η = 1.0, which mean that more mistakes are made by this model.

Influence of the audio/tag weight λ

Figure 3 shows the mAP@10 values on dev-test, when varying λ from Eq. 5. When λ = 0, the audio encoder provides as output the tag embeddings ⃗ et, and the model has no learnable parameters at all. In this setting, a 0.105 mAP@10 is obtained. This shows that using the tag name embeddings weighted with their probabilities already brings some information. When λ = 1, the audio encoder provides as output the projected PaSST embeddings ⃗ ea, and the model corresponds to PaSST-MPnet, which reached 0.229 mAP@10. In between, the score varies, and the best value was obtained with λ = 0.8.

CONCLUSION

We reported text-based audio retrieval experiments, with systems based on pretrained audio and sentence large scale transformers. In our baseline system, we used logits as audio embeddings, instead of the most commonly used 2-d feature maps extracted from earlier layers in a deep neural network. We improved this system by adding information from AudioSet tags, encoded as sentence embeddings. Finally, we discussed some of our architecture design choices: the use of layer normalization, a value smaller than 1.0 for the margin in the contrastive loss function for learning, and the 0.8 value for weighting the audio embeddings when adding the tag contribution within the audio encoder.

As a short-term perspective, we would like to use logit embeddings together with more standard 2-d feature maps in the audio encoder, to see whether those are complementary [START_REF] Mei | Language-based audio retrieval with pre-trained models[END_REF]. We also would like to investigate the use of external data, either for pretraining models or in a semi-supervised learning setting, where existing algorithms could be adapted to the present task [START_REF] Cances | Comparison of semi-supervised deep learning algorithms for audio classification[END_REF].

Figure 1 :

 1 Figure 1: System architecture. Left: audio encoder (with and without tag embeddings), right: text encoder. ⃗ e l : logit embedding, ⃗ ea: audio embedding, ⃗ et: weighted AudioSet tag embedding, ⃗ eat: combined audio and tag embedding, ⃗ es: sentence embedding, ⃗ eq: textual query embedding, Wt: matrix of the AudioSet tag name embeddings.

Figure 2 :

 2 Figure 2: Similarity score distributions on the training subset when using either a 0.4 (top) or a 1.0 (bottom) value for the margin η, when training a PaSST-MPnet-tags model. Pos: positive audiocaption pair, A imp: negative audio-query pair (audio impostor), Q imp: negative query-audio pair (caption impostor).

Figure 3 :

 3 Figure 3: mAP@10 values on dev-test according to the audio embedding weight λ.

Table 1 :

 1 Results on Clotho v2.

	Challenge baseline	732k	0.068	0.032 0.109	0.188	0.061	0.026 0.102	0.176
	CNN-transformer [5]	195M	0.260	0.150 0.400	0.530	0.251	0.153 0.387	0.504
	PaSST-MPnet	196M	0.229	0.134 0.355	0.482	0.212	0.124 0.319	0.448
	PaSST-MPnet-tags	196M	0.234	0.138 0.364	0.485	0.214	0.128 0.332	0.445
	PaSST-MPnet-tags-AC	196M	0.240	0.145 0.365	0.500	0.213	0.125 0.322	0.454
	Ensemble (2 models)	196M	0.243	0.148 0.369	0.498	0.216	0.127 0.321	0.463