est destinée au dépôt

  Non-typeable Haemophilus influenzae-Moraxella catarrhalis vaccine for the prevention of exacerbations in chronic obstructive pulmonary disease: a multicentre, randomised, placebo-controlled, observer-blinded, proof-of-concept, phase 2b trial Stefan Andreas, Marco Testa, Laurent Boyer, Guy Brusselle, Wim Janssens, Edward Kerwin, Alberto Papi, Bonavuth Pek, Luis Puente-Maestu, Dinesh Saralaya, et al. To cite this version: Stefan Andreas, Marco Testa, Laurent Boyer, Guy Brusselle, Wim Janssens, et al.. Non-typeable Haemophilus influenzae-Moraxella catarrhalis vaccine for the prevention of exacerbations in chronic obstructive pulmonary disease: a multicentre, randomised, placebo-controlled, observer-blinded, proofof-concept, phase 2b trial. The Lancet Respiratory Medicine, 2022, 10.1016/S2213-2600(21)00502-6. hal-03526684


Thomas Gautier
email: thomas.gautier@latmos.ipsl.fr

Nathalie Carrasco

Isabelle Schmitz-Afonso

David Touboul

Cyril Szopa

Arnaud Buch

Pascal Pernot

Nitrogen incorporation in Titan's tholins inferred by high resolution orbitrap mass spectrometry and gas chromatography-mass spectrometry

niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Titan, the biggest satellite of Saturn, presents a dense atmosphere mainly composed of molecular nitrogen and methane. Solar irradiation and electrically charged particles coming from Saturn's magnetosphere induce chemical reactions in the atmosphere of Titan that lead to the formation of an organic haze veil surrounding Titan.

These organic aerosols are of high interest due to both the deep impact of their presence on atmospheric processes [START_REF] Mckay | The greenhouse and antigreenhouse effects on Titan[END_REF] and their organic structure. Indeed, aerosols are composed of nitrogen, carbon and hydrogen, and thus probably include molecules of high astrobiological interest. For example, the presence of molecules such as adenine was reported in a recent study performed on aerosol analogues produced in the laboratory [START_REF] Hörst | Formation of amino acids and nucleotide bases in a Titan atmosphere simulation experiment[END_REF].

Even if the Cassini/Huygens (NASA/ESA) space mission provided information on these aerosols -either by in-situ measurements done with the Huygens descent probe or by spectroscopic probing with the Cassini spacecraft during the fly-by of the satellite -their physical and chemical nature remain mostly unknown [START_REF] Coll | Can laboratory tholins mimic the chemistry producing Titan's aerosols? A review in light of ACP experimental results[END_REF][START_REF] Israel | Complex organic matter in Titan's atmospheric aerosols from in situ pyrolysis and analysis[END_REF][START_REF] Waite | The process of tholin formation in Titan's upper atmosphere[END_REF][START_REF] West | Titan's Haze[END_REF].

In order to better understand and constrain their composition and formation pathways, it is of great interest to study laboratory analogues, the so called "tholins", to support observations on Titan's aerosols [START_REF] Cable | Titan tholins: simulating Titan organic chemistry in the Cassini-Huygens era[END_REF][START_REF] Coll | Can laboratory tholins mimic the chemistry producing Titan's aerosols? A review in light of ACP experimental results[END_REF].

Different laboratory experiments are devoted to producing Titan's tholins, including the PAMPRE device developed in LATMOS (France). It is a radiofrequency capacitively coupled plasma (RF-CCP) allowing the production of a large quantity of tholins grown in volume, due to the particular configuration of the plasma discharge [START_REF] Szopa | PAMPRE: a dusty plasma experiment for Titan's tholins production and study[END_REF]. These samples produced in volume in the plasma discharge will be called p-tholins in the present study.

Previous studies have been performed on chemical composition of tholins produced in plasma experiments and are comparable with our p-tholins. Elemental analysis [START_REF] Imanaka | Laboratory experiments of Titan tholin formed in cold plasma at various pressures: implications for nitrogen-containing polycyclic aromatic compounds in Titan haze[END_REF][START_REF] Sciamma-O'brien | Titan's atmosphere: an optimal gas mixture for aerosol production?[END_REF] provided a first chemical information on the C/N/H ratios in tholins. These analyses are not sufficient to determine their detailed molecular composition. Consequently, other types of chemical analyses have been used such as infrared spectroscopy [START_REF] Gautier | Mid-and far-infrared absorption spectroscopy of Titan's aerosols analogues[END_REF][START_REF] Imanaka | Optical constants of Titan tholins at mid-infrared wavelengths (2.5-25 µm) and the possible chemical nature of Titan's haze particles[END_REF], X-ray photoelectron spectroscopy [START_REF] Ruiz-Bermejo | CH 4 /N 2 /H 2 -spark hydrophobic tholins: a systematic approach to the characterisation of tholins[END_REF], 2008;[START_REF] Tran | The photochemical formation of a Titan haze analog. Structural analysis by X-ray photoelectron and infrared spectroscopy[END_REF], chromatographic techniques such as gas chromatography-mass spectrometry (GC-MS) [START_REF] He | Identification of nitrogenous organic species in Titan aerosols analogs: nitrogen fixation routes in early atmospheres[END_REF]; pyrolysis-GC-MS or high performance liquid chromatography [START_REF] Coll | Can laboratory tholins mimic the chemistry producing Titan's aerosols? A review in light of ACP experimental results[END_REF][START_REF] Ruiz-Bermejo | CH 4 /N 2 /H 2 spark hydrophilic tholins: as y s t e m a t i c approach to the characterization of tholins[END_REF], nuclear magnetic resonance spectroscopy [START_REF] Derenne | New insights into the structure and chemistry of Titan's tholins via 13 C and 15 N solid state nuclear magnetic resonance spectroscopy[END_REF]He et al., 2012aHe et al., , 2012b) ) and high resolution mass spectrometry, based on either orbitrap [START_REF] Hörst | Formation of amino acids and nucleotide bases in a Titan atmosphere simulation experiment[END_REF][START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF]o r Fourier transform ion cyclotron resonance [START_REF] Somogyi | Chemical ionization in the atmosphere? A model study on negatively charged "exotic" ions generated from Titan's tholins by ultrahigh resolution MS and MS/MS[END_REF].

These analyses confirmed that tholins are not only made of hydrocarbons but also include a large amount of nitrogen which contributes to chemical functional groups of interest for astrobiology, such as amines or nitriles. A polymeric growth of tholins has already been proposed [START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF][START_REF] Carrasco | Characterization of Titan's tholins: solubility, morphology and molecular structure revisited[END_REF]. This hypothesis is discussed in the recent review by [START_REF] Cable | Titan tholins: simulating Titan organic chemistry in the Cassini-Huygens era[END_REF].

In addition to p-tholins, we also studied an intermediate material, the so called c-tholins, also generated with the PAMPRE experiment, in a cryogenic trap. C-tholins are formed by the reaction of neutral gases condensed at the reactor outlet in a cold trap maintained approximately at the liquid nitrogen temperature [START_REF] Gautier | Nitrile gas chemistry in Titan's atmosphere[END_REF]. Discussed in details in [START_REF] Gautier | Nitrile gas chemistry in Titan's atmosphere[END_REF], the major gas products trapped were found to be qualitatively in agreement with the main species photoproduced in Titan's atmosphere. C-tholins are then formed in the cold trap when it is warmed up to the room temperature. It might be interesting to draw closer the c-tholins formation with the evolution of Titan's aerosols in the lower atmosphere. Indeed, at low altitude, some reactive gases such as HCN or CH 3 CN can condensate on the surface of aerosols [START_REF] Lavvas | Condensation in Titan's atmosphere at the Huygens landing site[END_REF][START_REF] Anderson | Titan's aerosol and stratospheric ice opacities between 18 and 500 µm: vertical and spectral characteristics from Cassini CIRS[END_REF]. As a result, chemical reactions can be induced in this condensed phase at the surface of aerosols. Reasonably, we can make the assumption that this chemistry would lead to the formation of an organic material possibly similar to the c-tholins.

In order to infer the influence of ion chemistry on tholins composition, we present here a comparative study of p-tholins (produced in the plasma, with ion chemistry) and c-tholins (produced outside a plasma). C-tholins are formed spontaneously inside the cryogenic trap isolated from the reactor, when it is warmed up to room temperature. The trapped species react then to form a brownish residue, the c-tholins. The investigations on the chemical patterns in p-tholins molecular structure are performed using high resolution mass spectrometry with the data analysis proposed by [START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF]. We also use GC-MS analyses to identify specific components of the unknown c-tholins material.

Experimental set-up and protocol

PAMPRE experiment

A detailed description of the experimental setup configuration used in this study can be found in previous publications [START_REF] Gautier | Nitrile gas chemistry in Titan's atmosphere[END_REF][START_REF] Szopa | PAMPRE: a dusty plasma experiment for Titan's tholins production and study[END_REF]. Here we briefly remind the main characteristics of the setup. The gaseous mixture used in the setup is a nitrogen-methane mixture, containing from 1 to 10% CH 4 , simulating Titan's main atmospheric composition [START_REF] Niemann | The abundances of constituents of Titan's atmosphere from the GCMS instrument on the Huygens probe[END_REF][START_REF] Waite | The process of tholin formation in Titan's upper atmosphere[END_REF]. The reactor is a cylindrical stainless steel chamber. The RF-CCP discharge is established thanks to an RF 13.56 MHz frequency generator. The gas mixture is injected as a continuous flow through the polarized electrode and extracted by a vacuum pump. This design allows the flow to be uniform in the reactor.

A cold trap, approximately brought to the liquid nitrogen temperature, can be set on the gas outlet of the reactor. Gas pumped from the reactor passes through this trap where products coming from the reactive mixture can condense, depending on the pressure and temperature. With this set-up (for c-tholins production experiments), the pressure in the PAMPRE reactor up to 1.7 ± 0.1 mbar instead of 0.9 ± 0.1m b a ri n standard conditions due to the reduced pumping efficiency induced by the cold trap. At the end of an experimental run, the trap is isolated from the reactor with a valve.

Samples produced in the plasma (p-tholins) and in the cryogenic trap (c-tholins)

The experiments were performed with two N 2 /CH 4 mixtures containing 1 and 5% CH 4 , respectively.

P-tholins samples were accumulated during two days in a glass vessel surrounding the reactive plasma and then collected into vials such as presented in [START_REF] Hadamcik | Optical properties of analogs of Titan's aerosols produced by dusty plasma[END_REF].

C-tholins are produced in the cryogenic trap where gas produced in the plasma discharge are trapped for 2 h. When the trap is warmed up to the room temperature, a part of the condensed gases evaporates whereas the other part is involved in chemical reactions which generate c-tholins at the bottom of the cold trap. These c-tholins are formed without involving ionic chemistry and are thus supposed to come from direct polymerization of the gaseous products in the cold trap, or by reactions in the condensed phase in the cold trap. Analysis of the gaseous phase has been reported in a previous study [START_REF] Gautier | Nitrile gas chemistry in Titan's atmosphere[END_REF], showing that it is mainly composed of nitriles and hydrocarbons.

Sample treatment for mass spectrometric analyzes

For mass spectrometric analysis, p-tholins were first dissolved in methanol (HPLC grade, Baker) at a concentration of 1m gm L -1 , then mixed thoroughly and the resulting solution was filtered through a PTFE 0.2 µm membrane. The choice of methanol as the solvent is consistent with a previous study on p-tholins solubility [START_REF] Carrasco | Characterization of Titan's tholins: solubility, morphology and molecular structure revisited[END_REF]. C-tholins were also dissolved in methanol for analysis. It can be noticed that p-tholins are poorly soluble in methanol (∼30% solubility) whereas the c-tholins are totally soluble. The samples were then infused in an ElectroSpray Ionization (ESI) Source in MeOH/water (50/50, V/V) at a flow rate of 10 µL min -1 . Water was chosen to improve ionization efficiency. Since tholins degradation in liquid environment occurs on time scales varying from a few hours to a few weeks [START_REF] Neish | Low temperature hydrolysis of laboratory tholins in ammonia-water solutions: implications for prebiotic chemistry on Titan[END_REF][START_REF] Poch | Production yields of organics of astrobiological interest from H 2 O-NH 3 hydrolysis of Titan's tholins[END_REF], samples were dissolved a few minutes before their analysis in order to prevent tholins structure alteration from interaction with the solvent.

The resulting ions were then analyzed using a high resolution mass spectrometer, with a hybrid linear trap/orbitrap analyzer (LTQ-Orbitrap, Thermo Scientific). Acquisition parameters in positive ion mode were: needle voltage 4.5 kV, capillary temperature 275 • C, capillary voltage 25 V, tube lens voltage 65 V and sheath gas flow rate 12 arbitrary unit (a.u.). In negative ion mode, parameters were: needle voltage 3k V , capillary temperature 275 • C, flow rate 12 a.u.

The mass spectrometer was externally calibrated using caffeine, MRFA (met-arg-phe-ala) peptide mix and Ultramark 1621 allowing a mass precision below 2 ppm. Data were acquired between m/z 50 and m/z 800 and integrated on several hundred scans (usually between 350 and 400). Data were acquired with a mass resolution fixed at 100 000 for an ion at m/z 450. The experimental resolution ( m/m) was determined to be ∼200 000 at m/z 150. As a first approach for data treatment, we only considered protonation ([M + H] + species, positive ion mode) and deprotonation ([M -H] - species, negative ion mode) reactions. Data were processed using XCalibur 2.0 software and its peak attribution function. In order to ensure unambiguous attribution of the species, the data treatment was limited up to m/z 400. Attribution parameters were: maximum authorized deviation of 5 ppm to ensure possible attribution for species of low masses (m/z < 120), maximum carbon atoms number of 60, maximum nitrogen atoms number of 60, maximum hydrogen atoms number of 100. The last three parameters were set to be sure to possibly attribute all existing peaks in the sample.

Nitrogen rule was used and charge was set to +1 (and -1f o r analysis in negative ion mode). In order to take into account possible oxygen contamination (in the reactor due to residual gases after pumping or when sample is exposed to air) two oxygen atoms were also allowed in the peak attribution. Similar to the study of [START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF], we focused this study on the non-oxygenated fraction of the sample. All oxygenated peaks were thus removed from the spectra before statistical analysis.

The c-tholins sample produced with 4% of methane was also analyzed using GC-MS. The first analysis is a direct injection into the chromatographer after dilution. A second analysis is performed on the c-tholins diluted in methanol and using derivatization to enhance the volatility of refractory compounds. The two derivatization agents used for this work are DMF-DMA (N,N-dimethylformamide dimethylacetal) and MTBSTFA (N-(t-butyldimethylsilyl)-N-methyltrifluoroacetamide). DMF-DMA is an alkylation agent generally used for derivatization of amines (primary and secondary) and carboxylic acids [START_REF] Freissinet | Search for evidence of life in space: analysis of enantiomeric organic molecules by N,N-dimethylformamide dimethylacetal derivative dependant gas chromatography-mass spectrometry[END_REF][START_REF] Meierhenrich | Pyrolytic methylation assisted enantioseparation of chiral hydroxycarboxylic acids[END_REF]. MTBSTFA is a silylation (addition of -Si(CH 3 ) 2 -C(CH 3 ) 3 fragment) agent commonly used for amino acid derivatization [START_REF] Buch | Development of a gas chromatography compatible Sample Processing System (SPS) for the in-situ analysis of refractory organic matter in Martian soil: preliminary results[END_REF].

For direct injection c-tholins are diluted in dichloromethane and are diluted in methanol for the derivatization process. C-tholins appear to be totally soluble in both solvents, which is not the case for p-tholins [START_REF] Carrasco | Characterization of Titan's tholins: solubility, morphology and molecular structure revisited[END_REF].

The column used for this analysis is a CP-Chirasil-Dex CB(Varian), L = 30 m, ID = 0.25 mm, df = 0.53 µm. The injector temperature is set at 270 • C, helium flow rate at 1m Lm i n -1 , split at 1/20. Temperature ramp is: 34 • C for 5m i n , then 5 • Cmin -1 up to 190 • C and then isotherm at 190 • C for 5m i n .

Results and discussion

Influence of methane percentage on p-tholins composition

In addition to the analysis of the data for p-tholins produced at 1% and 5% of methane in nitrogen, we validate our data treatment by reanalyzing data acquired for a previous study [START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF]o n p-tholins produced under the same experimental conditions with 2% and 10% of methane in nitrogen respectively.

Fig. 1 presents the Orbitrap mass spectra of four different ptholins samples produced with 1%, 2%, 5% and 10% of methane in the gas mixture. Two kinds of structures are visible on these spectra: first, intense and isolated peaks (mainly at m/z < 150); second, regularly spaced clusters generally observed when analyzing a polymeric material. Orbitrap mass spectra in positive ion mode of p-tholins samples produced with 1%, 2%, 5% and 10% of methane. Spectra at 2% and 10% are the same spectra that were analyzed in [START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF]. To enhance visibility of the spectra intensity is limited to 2e6, some peaks in the 2% and 5% spectra extend slightly above this limit.

The cluster structure is representative of almost all the material at high methane percentage, whereas the first structure is predominant in spectra of p-tholins produced with 1, 2 or 5% of methane.

These light species are mainly located in the right wing of the Van Krevelen diagram (cf. Fig. 3), which means compounds containing a large amount of nitrogen (N/C ratio higher than 1.5). These light but nitrogen-rich species are therefore less incorporated in samples produced at 10% of methane. The decrease of nitrogen content in p-tholins with increase of methane percentage, has also been observed at a global scale, by elemental analysis (Sciamma-O' Brien et al., 2010) and by infrared spectroscopy, studying the evolution of aliphatic carbon/amines bands ratio [START_REF] Gautier | Mid-and far-infrared absorption spectroscopy of Titan's aerosols analogues[END_REF]. As shown in Fig. 2, there is an inversion in the relative intensity of these two peaks when increasing the methane percentage. C 3 H 6 N 6 is more predominant for p-tholins produced with 1, 2 and 5% of methane whereas for 10%, C 5 H 10 N 4 is the major compound exhibiting less nitrogen and more carbon atoms. This is consistent with the increase of global nitrogen content in p-tholins when methane percentage decreases as observed in the elemental analyses [START_REF] Sciamma-O'brien | Titan's atmosphere: an optimal gas mixture for aerosol production?[END_REF]. At an intermediate percentage (5%), both peaks have similar intensities. This phenomenon can be observed in all predominant clusters between m/z 70 and m/z 300.

In order to compare the samples, Van Krevelen diagrams adapted for nitrogen bearing molecules can be plotted [START_REF] Imanaka | Role of photoionization in the formation of complex organic molecules in Titan's upper atmosphere[END_REF][START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF][START_REF] Somogyi | Chemical ionization in the atmosphere? A model study on negatively charged "exotic" ions generated from Titan's tholins by ultrahigh resolution MS and MS/MS[END_REF]. These diagrams are scatter plots of the H/C ratio as a function of N/C ratio for detected molecules, as shown in Fig. 3. In the 2D Van Krevelen diagrams, the visible linear alignments of structures cannot be directly linked to the polymer structure but only reflects the possible combinations for C X H Y N Z molecules. These combinations are limited to a compositional space defined by all combinations of carbon, nitrogen and hydrogen atoms in agreement with the nitrogen rule. The regular patterns observed in a standard 2D Van Krevelen diagram only reflect the intrinsic order of the compositional space [START_REF] Hertkorn | Natural organic matter and the event horizon of mass spectrometry[END_REF].

The first visible phenomenon in Fig. 3 is a decrease in the number of nitrogen bearing species of p-tholins observed when methane percentage increases. This is indicated by the depletion of the right "wing" (i.e. N/C > 1) of the diagram when the methane percentage increases, correlated with an increase of the density of the detected compounds with low N/C and H/C ratios (N/C < 0.5 and H/C < 2). The latter may be linked to an increase of unsaturated hydrocarbons content in p-tholins, recently observed by Nuclear Magnetic Resonance (NMR) [START_REF] Derenne | New insights into the structure and chemistry of Titan's tholins via 13 C and 15 N solid state nuclear magnetic resonance spectroscopy[END_REF].

A second effect visible in all p-tholins samples is that red and violet dots (high masses ions, m/z > 350) are much more concentrated than yellow and orange (low masses ions) which are dispersed all over the diagram. This means that the species distribution tends to concentrate around specific values at high masses.

Polymeric structure of p-tholins

A modified 3D Van Krevelen diagram, with addition of ln(m/z) as a third dimension [START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF], is visible in Fig. 4 for p-tholins produced with 5% of methane.

From Fig. 4, it is clear that a convergence of the distribution occurs when m/z increases. This would mean that after enough polymerization steps, the material composition tends toward average ratios of H/C = 1.5 and N/C = 0.5, i.e. an ideal co-polymer with an average formula of (C 2 H 3 N) n .

Fig. 4 also highlights that the molecules are organized among separated families distributed on different plans. In order to infer the most parsimonious polymeric representation of p-tholins, we extended the method proposed by [START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF], by testing a comprehensive set of monomeric bases. All detected molecules C x H y N z can hypothetically be formed by two fragments, for example CH 2 and HCN, plus a residue of carbon atoms: α. Then, each molecule detected in the mass spectrum can be defined as Cα-(CH 2 ) m -(HCN) n where:

n is the number of nitrogen atoms, systematically attributed to HCN units; The formulae for n, m and α has to be adapted to different base choices. Here we tested several alternative bases to determine which one provides the purest and most parsimonious representation of the material. Two bases, HCN/CH 2 and HC 3 N/C 2 H 2 were used to validate the process proposed in [START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF]. We also studied the following base pairs: HCN/C 2 H 2 ; HCN/C 2 H 4 ; HCN/C 2 H 6 ; CH 3 CN/CH 2 ; CH 3 CN/C 2 H 2 ; CH 3 CN/C 2 H 4 ; CH 3 CN/C 2 H 6 ; CH 2 NH/CH 2 ; CH 2 NH/C 2 H 2 and NH 3 /C 2 H 4 .

The distribution of the families, i.e. the number of families used to describe the whole material and the global intensity of the family (equal to the normalized sum of all peaks belonging to the family) was studied for each base. An example of the distribution for three different bases (HC 3 N/C 2 H 2 ; HCN/C 2 H 2 ; CH 3 CN/C 2 H 4 ) is given in Fig. 5.

We can see on Fig. 5 that the choice of the fragments influences the α distribution. It can be more or less centered on 0 and wide, defining two ranking criteria for the distribution:

-Based on the minimum of statistical entropy principal, we consider that a narrow distribution represents the sample better than a wide one.

-For two distribution of the same width, the one the most centered on α = 0 (i.e. which requires the less carbon atom compensation) is considered as the best.

The whole set of distributions represented by blue dots is summarized in Fig. 6. The size of the dot is proportional to the number of families used to describe the polymer, and the color of the dot is linked to the position of the distribution relatively to the α = 0 family.

In a general way, we prefer the narrowest distribution, as it provides the most parsimonious description of the sample (corresponding to the minimum of statistical entropy). For two distributions of the equal width, the one with the smallest absolute value of alpha is preferred, as it requires less carbon compensation.

For the hydrocarbons patterns, it seems that CH 2 and C 2 H 4 (which are statistically equivalent for this representation) present systematically better polymer reconstruction than C 2 H 2 and C 2 H 6 when used with nitriles as nitrogenous compounds. This could mean that the hydrocarbons needed for p-tholins initiation should not be highly unsaturated, as previously suggested [START_REF] Kaiser | On the formation of polyacetylenes and cyanopolyacetylenes in Titan's atmosphere and their role in astrobiology[END_REF]. Another possible explanation could be that tholins indeed grow through either unsaturated (such as C 2 H 2 ) or totally saturated (such as C 2 H 6 ) materials, and then either gain or loss hydrogen atoms by further chemistry in the reactive medium [START_REF] Sekine | The role of organic haze in Titan's atmospheric chemistry I. Laboratory investigation on heterogeneous reaction of atomic hydrogen with Titan tholin[END_REF][START_REF] Gautier | Methane conversion in a N 2 -CH 4 radiofrequency discharge[END_REF]. On the choice of the nitrogen bearing pattern, nitriles, and especially CH 3 CN and HCN, provide more parsimonious representations than ammonia, which is a better polymeric base than imines. Looking at distributions and considering stable neutral species, the best possible bases are HCN/C 2 H 4 and CH 3 CN/C 2 H 4 . This is fully in agreement with the main chemical species identified in the atmosphere of Titan. Although it was not considered as an exclusion criteria, it is interesting to note here that the convergence point observed in the 3D Van Krevelen diagram corresponds to a C:H:N ratio of 1:1.5:0.5, which is also the ratio found considering 2 HCN + C 2 H 4 or HCN + CH 2 .

Comparison of p-tholins mass spectra obtained in the positive and negative ionization modes

Recent work on chemical analyses of p-tholins showed the interest of using the negative ion mode [START_REF] Somogyi | Chemical ionization in the atmosphere? A model study on negatively charged "exotic" ions generated from Titan's tholins by ultrahigh resolution MS and MS/MS[END_REF]. The comparison between positive and negative ionization modes of p-tholins produced in PAMPRE with 5% of methane is given in Fig. 7.

As shown in Fig. 7, the molecules detected in the negative ion mode are less numerous (40% less than in positive ionization mode) and different from the molecules detected in the positive ion mode. The molecules detected in negative mode generally present a lower H/C ratio, leading to a higher Double Bond Equivalent (DBE). Positive ionization seems to emphasize the detection of hydrogen rich nitrogenous compounds such as amines, whereas negative ionization mode allows detection of compounds poorer in hydrogen -such as nitriles or cyclic compounds. Both amine and nitrile functions have been detected unambiguously in p-tholins by infrared spectroscopy [START_REF] Gautier | Mid-and far-infrared absorption spectroscopy of Titan's aerosols analogues[END_REF][START_REF] Imanaka | Laboratory experiments of Titan tholin formed in cold plasma at various pressures: implications for nitrogen-containing polycyclic aromatic compounds in Titan haze[END_REF][START_REF] Quirico | New experimental constraints on the composition and structure of tholins[END_REF]. The detection of fewer compounds in the negative ion mode may be connected with the infrared analyses where amine bands are much more predominant in p-tholins spectrum than nitrile bands. This is consistent with an important amine content in p-tholins. The enhancement of nitriles in the negative ionization mode might also be linked to their high electronic affinity [START_REF] Vuitton | Negative ion chemistry in Titan's upper atmosphere[END_REF].

Comparison of the composition of p-tholins versus c-tholins

The mass spectrum of the c-tholins obtained with 5% of methane is shown in Fig. 8 with the spectrum of p-tholins in the same condition as a reference point. Fig. 9 presents a close-up of Fig. 8. Fig. 9 shows that the distribution of peaks inside clusters is totally different between the two materials. The distribution of peaks in a cluster is systematically shifted toward low masses for the c-tholins compared with p-tholins. This observation is shown in Fig. 9 for the m/z range 120-130, but it is also observed for the entire spectrum.

It must be noted that all the peaks observed in p-tholins are systematically present in the c-tholins too. This means that Van Krevelen diagram are exactly the same for c-tholins and p-tholins since this representation does not include information on the peak intensities. As for the 2D diagrams, since c-tholins presents similar Van Krevelen diagram than p-tholins, the 3D Van Krevelen diagram of c-tholins also presents this convergence at high masses toward H/C = 1.5 and N/C = 0.5 point.

The dominant peaks in p-tholins are not the dominant ones in the c-tholins. Indeed, the shift of peaks toward low masses (in the same cluster) observed in the c-tholins (see Fig. 9) seems to be due to a larger incorporation of carbon compared to nitrogen in the c-tholins.

It should be noted that both samples were produced at different pressures: 0.9 mbar for p-tholins, 1.7 mbar for c-tholins. Elemental analysis has shown that the pressure effect on the sample chemical composition is weak [START_REF] Sciamma-O'brien | Titan's atmosphere: an optimal gas mixture for aerosol production?[END_REF].

It should also be noticed that a heterogeneous chemistry probably occurs in both cases. In the PAMPRE reactor, heterogeneous chemistry could occur on the surface of p-tholins growing in the gas phase (solid-gas process), whereas in the cold trap, the c-tholins are formed during the melting of condensed icy species, potentially leading to a solid(c-tholins)-liquid chemical process.

The major difference between the samples explaining such a nitrogen depletion in c-tholins is thus the chemical environment of the samples during their production. Indeed p-tholins are produced within a reactive plasma, which includes interaction with electrons and positive/negative ions. On the contrary, their lifetimes preclude ions to reach the cryogenic trap. C-tholins are thus produced in an only a neutral environment. The difference between the samples means that the ion chemistry plays a more important role in the formation of p-tholins, whereas the chemistry generating the c-tholins seems to mainly induce polymerization of carbon rich molecules.

C-tholins analysis by GC-MS

A chromatogram obtained by direct GC-MS analysis of c-tholins is given in Fig. SI 2 (a andb). As seen in this figure, many peaks in the chromatograms cannot be identified. This is mainly due to the nature of compounds present in tholins-kind materials (mainly nitriles here), which are underrepresented in the databases available for molecular identification.

The list of all compounds identified by direct and derivatization methods is given in Table 1.

Direct GC-MS analysis allowed identification of ten molecules within c-tholins, all of them being nitrogenized (either bearing amine functional groups or nitrile functional groups or both). However such direct analysis detects only the volatile compounds present in the mixture. After derivatization, the main difficulty for interpreting data comes from the lack of reference in the databases for such N-rich organic compounds. The few peaks identified are mostly highly nitrogenized compounds, mainly nitrile or amines. The detection of alanine and urea in derivatized tholins produced by plasma has also been reported [START_REF] Ramírez | The fate of aerosols on the surface of Titan[END_REF][START_REF] Hörst | Formation of amino acids and nucleotide bases in a Titan atmosphere simulation experiment[END_REF].

The molecules detected in c-tholins by GC-MS are consistent with analysis of p-tholins by GC-MS and pyr-GC-MS in the literature, including numerous nitrogen bearing molecules, especially with nitrile functions and nitrogen inclusion within aromatic cycles [START_REF] Coll | Experimental laboratory simulation of Titan's atmosphere: aerosols and gas phase[END_REF][START_REF] He | Identification of nitrogenous organic species in Titan aerosols analogs: nitrogen fixation routes in early atmospheres[END_REF][START_REF] Mcguigan | Analysis of Titan tholin pyrolysis products by comprehensive two-dimensional gas chromatography-timeof-flight mass spectrometry[END_REF].

C-tholins analysis is also in agreement with the molecules identified in the gas phase in the cryogenic trap connected to a GC-MS [START_REF] Gautier | Nitrile gas chemistry in Titan's atmosphere[END_REF]. The detection of numerous nitrogen bearing molecules including nitrile functional groups confirms the nitrile reactivity as suggested in [START_REF] Gautier | Nitrile gas chemistry in Titan's atmosphere[END_REF]. In the present study many molecules bearing amine functions were also detected. This kind of molecule was not observed in the gas phase coming from the cryogenic trap. A possible explanation for this difference is that the gas injected in the GC-MS was transferred from the cryogenic trap through stainless steel connections, where amines can easily adsorb.

The detected molecules present a quite low N/C ratio (compared to orbitrap analysis of c-tholins), but nitrogen atoms are detected within the structure and not only as terminal functional groups, as previously suggested for p-tholins with C-N-C alternation [START_REF] Carrasco | Characterization of Titan's tholins: solubility, morphology and molecular structure revisited[END_REF]. This emphasizes that nitrogen seems to be essential to initiate the molecular growth.

Table 1

List of molecules detected using direct injection and after derivatization, their identification in Fig. SI 2 and their formula. Molecules are identified by comparison of their mass spectra with the NIST database.

The presence of oxygenated species (whereas no oxygen is introduced in PAMPRE), including amino acids such as alanine is also reported. These oxygenated compounds may be formed by oxidation of c-tholins material, either when samples are harvested and exposed to air or dissolved in methanol before evaporation and derivatization. Even if these compounds are clearly not formed in the gas phase in the PAMPRE reactor, their presence is still interesting for the astrobiological aspect of tholins science. Indeed, even if unwanted, the formation of these compounds did not imply specific manipulations. It could then be possible that such a process occurs naturally on Titan. Studies in the literature show similar behavior for p-tholins in water-ammonia solutions [START_REF] Neish | Low temperature hydrolysis of laboratory tholins in ammonia-water solutions: implications for prebiotic chemistry on Titan[END_REF][START_REF] Poch | Production yields of organics of astrobiological interest from H 2 O-NH 3 hydrolysis of Titan's tholins[END_REF].

Finally, all molecules detected with this technique are also observed using ESI-orbitrap mass spectrometry. The identifica-tion by GC-MS of more than twenty molecules in c-tholins allows us to draw a first sketch of the growth processes -as it has been done previously with p-tholins [START_REF] Carrasco | Characterization of Titan's tholins: solubility, morphology and molecular structure revisited[END_REF][START_REF] Coll | Experimental laboratory simulation of Titan's atmosphere: aerosols and gas phase[END_REF][START_REF] He | Identification of nitrogenous organic species in Titan aerosols analogs: nitrogen fixation routes in early atmospheres[END_REF][START_REF] Mcguigan | Analysis of Titan tholin pyrolysis products by comprehensive two-dimensional gas chromatography-timeof-flight mass spectrometry[END_REF][START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF]-for the 50-150 Da mass range, a particularly important range since the initiation of the growth of the whole material starts there. These first trends only represent a small fraction of the material (∼20 molecules detected by GC-MS against more than 15 000 with orbitrap MS). From this point of view, the improvement of mass spectra databases, especially the inclusion of nitrogen-bearing molecules, would be a great help to complete the identification of compounds detected by GC-MS and thus the overall knowledge of c-tholins formation processes.

Conclusion

In the present work we study the impact of methane percentage in the gas phase on p-tholins high resolution mass spectra, as initiated in [START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF] with plasma-generated tholins produced in two different conditions (2% and 10% of methane). P-tholins spectra display polymeric structures which are confirmed by the identification of several polymer families in the material. This supports the fact that p-tholins are copolymer-like materials, probably not linear, but hyper branched copolymer [START_REF] Klee | Review: mass spectrometry of step-growth polymers[END_REF].

Nevertheless, it seems that at high masses (m/z > 300), p-tholins produced with 5% of methane could be approximated to an average ideal (C 2 H 3 N) n polymer as shown by the analysis of 2D and 3D Van Krevelen diagrams.

We also compare high resolution mass spectra of p-tholins and c-tholins. This provides some information on the reactive pathways to tholins formation, emphasizing differences between a material produced by polymerization of neutral gases and a material produced in a reactive plasma. The predominant molecules detected in p-tholins contain more nitrogen than the one detected in c-tholins. We can assume that the main difference between both materials is the ion chemistry involved in p-tholins formation but not in ctholins formation. This means that ion chemistry assists nitrogen incorporation into p-tholins, as opposed to neutral chemistry that leads to molecules with lower N/C ratio.

Finally, c-tholins are further analyzed by GC-MS, identifying the structure of about twenty molecules in the 50-150 m/z range, the key mass range for polymerization initiation. This primary study highlights some trends in c-tholins growth. Molecules are mainly linear chains, with an N/C ratio decreasing with the chain size. Nitrogen reactivity also seems to be essential for the radical growth since nitrogen is found within the structure and not only as terminal functional groups.

In conclusion, the study presented here helps us to understand the formation processes of Titan's tholins. The analytic and statistical methods we used here could also be of great interest for other highly complex organic compounds of cosmochemical interests such as kerogen or the soluble organic matter in meteorites.

  Fig. SI 1 (supplementary information) shows the formation of the c-tholins in the cold trap when warmed up to room temperature.

Fig

  Fig.1. Orbitrap mass spectra in positive ion mode of p-tholins samples produced with 1%, 2%, 5% and 10% of methane. Spectra at 2% and 10% are the same spectra that were analyzed in[START_REF] Pernot | Tholinomics-chemical analysis of nitrogen-rich polymers[END_REF]. To enhance visibility of the spectra intensity is limited to 2e6, some peaks in the 2% and 5% spectra extend slightly above this limit.

  Fig. 2 presents a close up of Fig. 1 centered on the peaks detected around m/z 127. Two major ion peaks are detected at m/z 127.0725 ( m =-1.31 ppm) and m/z 127.0977 ( m = -1.30 ppm), respectively attributed to C 3 H 6 N 6 (possibly melamine) and C 5 H 10 N 4 .

Fig. 2 .

 2 Fig. 2. Close-up of Fig. 1 around m/z 127 for each p-tholins sample.

Fig. 3 .

 3 Fig. 3. Van Krevelen representation of p-tholins formed with 1%, 2%, 5% and 10% of methane. Colors of the dots represent the mass of the peak.

Fig. 4 .

 4 Fig. 4. 3D Van Krevelen diagram of p-tholins 5%. Each color correspond to a plan defined by an α value within the HCN/CH 2 monomer basis (cf. part 3.2). The α =-5 central plan is in yellow and negative and positive α values are located symmetrically around this central plan. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.) m = (H-N)/2 considering that all H atoms but in (HCN) n belongs to CH 2 . H and N are respectively the number of hydrogen atoms and nitrogen atoms detected in a molecule; -α is the number of carbon atoms completing the pure copolymer, thus α = C-(H + N)/2. It is used to discriminate between families of compounds.

Fig. 5 .

 5 Fig. 5. Representation of the α families distribution in p-tholins produced with 5% CH 4 for three different monomers combination: (a) HC 3 2 2 (number of families: 26, distribution maximum: -13); (b) HCN/C 2 H 2 (number of families: 16, distribution center: -5); (c) CH 3 CN/C 2 H 4 (number of families: 8, distribution center: -1).

Fig. 6 .Fig. 7 .

 67 Fig. 6. Schematic representation of bases. The radii of the dots represent the size of the distribution, i.e. number of families needed to describe the polymer. The larger radius corresponds to 26 families whereas the smaller one corresponds to only 8 families. The colors of the dots are linked to the eccentricity of the distribution: the darker the dots are, the more the distribution is centered on 0. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 8 .

 8 Fig. 8. Orbitrap mass spectra between m/z 60 and m/z 300 in positive ionization mode of p-tholins and c-tholins samples produced with 5% of methane.

Fig. 9 .

 9 Fig. 9. Close-up of Fig. 9 on the 120-130 m/z range. Textbox indicates the predominant peaks in each cluster.

Acknowledgements

Authors would like to thank Dr. Heather Graham for her help proofreading this article. This work was financially supported by CNRS (PNP and ANR-09-704 JCJC-0038 contract).

Appendix A. Supplementary material

Supplementary material related to this article can be found online at http://dx.doi.org/10.1016/j.epsl.2014.07.011.


Karim Radouane
email: karim.radouane@mines-ales.fr

Andon Tchechmedjiev
email: andon.tchechmedjiev@mines-ales.fr

Binbin Xu
email: binbin.xu@mines-ales.fr

Sebastien Harispe
email: sebastien.harispe@mines-ales.fr

Sébastien Harispe

Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) Comparison of Deep Learning Approaches for Protective Behaviour Detection Under Class Imbalance from MoCap and EMG data

Keywords: AffectMove Task 1, Multimodal behaviour classification, Deep learning architectures

come   L'archive ouverte pluridisciplinaire

I. INTRODUCTION

The 2021 AffectMove challenge [START_REF] Olugbade | The Affectmove 2021 Challenge -Affect Recognition from Naturalistic Movement Data[END_REF] organised in the context of the EnTimeMent H2020 project and collocated, as a workshop, with the ACII 20201 conference, offered the opportunity of participating in three challenge tasks that involve the detection of characteristics of human motion in multimodal and unimodal settings:

1) Protective Behaviour Detection based on Multimodal

Body Movement Data. This first task provided participants with Motion capture (MoCap) + Electromyography (EMG) data from the EMOPAIN [START_REF] Aung | The Automatic Detection of Chronic Pain-Related Expression: Requirements, Challenges and the Multimodal EmoPain Dataset[END_REF] for protective behaviour detection (which is a response to pain while performing the actions) across several subjects and tasks, with fixed three-second recordings. across several subjects and tasks, with variable recording lengths and uneven sampling.

3) Detection of Lightness and Fragility in Dance Movement based on Multimodal Data. The third tasked involved the multimodal classification of movement from video and accelerometers to detect frailty in dance moves. Task 1 focuses mainly on protective behaviour detection in response to pain in subjects suffering from disorders due to chronic musculoskeletal pain. Such disorders, including the typical example of low-back-pain, although it may appear to be a mundane non-systemic disease of little concern, in reality it affects a significant part of the population. Chronic musculoskeletal pain causes significant disruptions of quality of life, but is also a leading cause of medical leave in professionals. Musculoskeltal disorder and the resulting pain are a significant burden on national healthcare systems. Although there are general principles in their treatment, few objective criteria exist to evaluate the recuperation of patients or their level of pain (strong psychosomatic component). Protective behaviour in response to pain in particular, hinders recovery by accentuating non-use, and also constitutes an objective marker in gauging the real level of pain (as opposed to self-reported scales). Being able to detect protective behaviour automatically from sensor data, can provide a valuable tool to reform clinical protocols in the treatment of chronic musculoskeletal disorders.

In the dataset of Task 1, each subject and experimental sequence pair represented one record and was materialized as a single file. The MoCap data was encoded in the first 51 columns of the data file (17 joints × 3 spatial coordinates) and the EMG data (two electrodes on the upper fibres of trapezius muscles, two on the lumbar paraspinal muscles) was encoded in the next 4 columns, followed by the action identifier. Each instance was composed of 180 frames (60fps) and the EMG signal was prepossessed (signal envelope), down-sampled and aligned with MoCap frames.

Predicting the protective behaviour requires devising systems that either rely on MoCap solely or multimodal systems considering both MoCap and EMG.

Possible machine learning approaches include those from conventional machine learning and deep learning. Relevant model architectures mainly stem from computer vision and particularly from skeleton-based action recognition systems. In this work, we choose to focus on deep learning approaches, 978-1-6654-0021-3/21/$31.00 ©2021 European Union building at first the baselines with simple methods (LSTM [START_REF] Egede | EMOPAIN challenge 2020: Multimodal pain evaluation from facial and bodily expressions[END_REF], [START_REF] Wang | Recurrent network based automatic detection of chronic pain protective behavior using mocap and semg data[END_REF] and Transformer [START_REF] Vaswani | Attention is all you need[END_REF]) and then exploring one state-ofthe-art (SOTA) system of action recognition based on graph convolutional networks [6], which leverage the specificity of skeleton data (skeleton topology, relative distances, velocities). Each of the systems was developed by a different member of the team independently and then integrated in a common evaluation setting, in order to perform comparable model selection.

In this paper, we give a synthetic account of related work, followed by a description of the three systems (design, model selection, evaluation) and then a meta-analysis of the results with regard to the overall objective and the final scores of our systems on the test set of the challenge.

II. RELATED WORK

The common baseline neural network model to treat time series type data is generally based on LSTM (Long short-term memory) architectures and skeleton-based action recognition is no exception. In the EMOPAIN Challenge 2020 [START_REF] Egede | EMOPAIN challenge 2020: Multimodal pain evaluation from facial and bodily expressions[END_REF], that is based on the same EMOPAIN dataset [START_REF] Aung | The Automatic Detection of Chronic Pain-Related Expression: Requirements, Challenges and the Multimodal EmoPain Dataset[END_REF] as the current AffectMove task (with different labels), participants (also in [START_REF] Wang | Recurrent network based automatic detection of chronic pain protective behavior using mocap and semg data[END_REF]) used a stacked LSTM model as baseline built with three LSTM layers (32 hidden units) followed by a dropout layer with probability of 0.5. This model gave an average accuracy of 0.828 on test set (MOCAP and EMG data combined).

More generally, LSTMs when applied to movement classification in larger benchmarks such as NTU-60 or NTU-120 [START_REF] Shahroudy | Ntu rgb+d: A large scale dataset for 3d human activity analysis[END_REF] have either, been augmented to take into account spatiotemporal information, showing moderate performance (STA-LSTM at 64% accuracy [START_REF] Liu | Spatio-temporal lstm with trust gates for 3d human action recognition[END_REF]), or with attention mechanisms, showing close to SOTA performance (AGC-LSTM at 95% accuracy [START_REF] Si | An attention enhanced graph convolutional lstm network for skeleton-based action recognition[END_REF]).

Besides approaches based on recurrent neural networks, Transformers have shown tremendous potential, initially in Natural Language Processing, but in the past year the model has been adapted and applied to other types of modalities, from speech, to computer vision, to multi-modal systems. There are now numerous Transformer adaptations on temporal, spatial or spatio-temporal data, including architectures specific to Motion Capture and/or pose estimation data. A few notable examples include:

• the Variational Autoencoder Transfomer (Transformer VAE [START_REF] Petrovich | Action-conditioned 3d human motion synthesis with transformer vae[END_REF]) that allows learning a representation of parameters of human motion (MoCap), pre-trained on an action-conditioned movement generation task; • the Spatial Temporal Transformer Network (ST-TA [START_REF] Plizzari | Skeleton-based action recognition via spatial and temporal transformer networks[END_REF]), is also a Transformer specifically adapted to MoCap timeseries, by a fusion of a temporal attention model (in the standard formulation of temporal self-attention) and of a spatial transformer that extends the more typical graph convolutional network approach with an attention mechanism. While Transformer VAE isn't technically an action recognition system, it can, and has been evaluated as such (on a small subset of NTU-13). Although it doesn't show SOTA performance on-par with systems specialized for action recognition, the architecture is extremely innovative and could be eventually harnessed to produce multi-task representations of movement similar to what has transformed Natural Language Processing since 2018. ST-TR on the other hand shows SOTA performance (96.1% accuracy on NTU-60), but the "temporal transformer" is actually implemented as a convolutional neural network with KQV (Key-Query-Value) attention on top, and the spatial transformer is in fact a Graph Convolutional Network with added attention. As such, ST-TR is much more similar to the more common GCN architectures than it is with the Transformer, besides the use of attention, which in itself isn't specific to transformers and has existed long before the inception of Transformers.

Speaking of GCNs, they constitute the other major family of approaches for skeleton-based action-recognition and the current SOTA system among GCNs, PA-ResGCN [6], is almost ex aequo with ST-TR at 96.1%, which probably doesn't constitute a significant difference. ResGCN combines the 3D coordinates with spatial skeleton structure (relative distances between joints) and with velocity gradients. Additionally, partwise attention is used to inform the contribution of 5 manually defined body parts (sub-graphs) to the overall classification. We shall not further describe the state of the art of GCNs for skeleton based action recognition, but Song et al. [6] already present a very comprehensive literature review and comparison. GCNs have already been applied to the EMOPAIN dataset as well [START_REF] Wang | Leveraging activity recognition to enable protective behavior detection in continuous data[END_REF].

Based on the observations, we chose to implement a vanilla Transformer baseline without adaptations to evaluate its potential for Task 1. We also set-out to adapt and apply the current non-Transformer SOTA system for skeleton-based action recognition, PA-ResGCN to gauge its performance in the real-life setting offered by Task 1.

This review doesn't address the multi-modal integration of EMG, although our Transformer baseline significantly benefited from EMG through the most naïve form of early fusion: concatenation of EMG features with the 3D joint coordinates. The PA-ResGCN system doesn't allow such a use of EMG, and the time allotted for the challenge didn't allow us to explore effective late fusion strategies either. We did make attempts on simple late fusion, through the concatenation of a fully connected layer from the PA-ResGCN output (-N51) with that of a simple CNN effective at EMG classification with a common training objective. However, these attempts were unsuccessful (lower performance than without fusion). However, attempts for multimodal fusion of EMG and MoCap on EMOPAIN, have shown some success, particularly with more sophisticated fusions mechanisms [START_REF] Wang | Recurrent network based automatic detection of chronic pain protective behavior using mocap and semg data[END_REF], [START_REF] Bao | Multimodal data fusion based on the global workspace theory[END_REF], indicating that this may be a path forward for further improvement. Another aspect of interest to exploiting the Task 1 data set, is the handling of significant class imbalance (Table I), we won't explore the literature exhaustively, but we employed a dynamic weighting scheme from the literature [START_REF] Cui | Class-balanced loss based on effective number of samples[END_REF] for the binary cross entropy loss, which proved capital in obtaining good convergence during training. We also experimented with using a Mathew's Correlation Coefficient loss [START_REF] Abhishek | Matthews correlation coefficient loss for deep convolutional networks: Application to skin lesion segmentation[END_REF], which is robust to class imbalance and matched the metric used in Task 1. Although it was also successful in countering the class imbalance, it's numerically less numerical stable.

III. METHODS

A. Dataset and Protocol

As described earlier, the data set provides a sequence of 180 3D skeleton joints (17 × 3) along with 4 channels of EMG signal downsampled and synchronized. The dataset is labeled with actions and protective behaviour annotations, but the labels are only provided to participants for the training and validation subsets. Table I presents some general statistics about training and validation data. One important characteristic is the strong class imbalance and the significantly smaller size of the validation set. Additionally, since there are several repetitions by several subjects (a joint inter/intra subjects acquisition design), it is important to keep subject sessions atomically linked together if the data set is shuffled prior to training a system, as otherwise significant bias would be induced. We also observed, in one of the baselines for the EMOPAIN dataset [START_REF] Egede | EMOPAIN challenge 2020: Multimodal pain evaluation from facial and bodily expressions[END_REF], that the scores obtained on the validation set are surprisingly low (accuracy 0.4636, and F 1 0.4811 [START_REF] Wang | Recurrent network based automatic detection of chronic pain protective behavior using mocap and semg data[END_REF]) for a binary classification problem, while the performance on the training set was higher, and comparable to that obtained on the test set. These discordant scores between validation and test sets suggest that these two datasets are hypothetically highly different in feature space (they represent different distributions of classes and subjects). It appears that in the AffectMove challenge in 2021, a similar data splitting rule was likely applied in preparing the data sets, given the observed distribution (Table I), as models with good performance on the training set systematically showed less interesting performance on the validation set in a consistent manner. This raises the question of how representative the validation set is compared to the final tests set, as tuning hyper parameters on a non-representative distribution would likely lead to a system that significantly under-performs on the test set.

In this work, we chose to use the validation set to fine-tune the systems and to produce the final prediction, although in hindsight, a retraining on both the training and validation sets to produce the final predictions could have been more appropriate for some of the models, particularly the LSTM which suffered greatly for significant overfitting on the validation set, as we will describe later.

B. Transformer-based architectures

Several simple Transformer models [START_REF] Vaswani | Attention is all you need[END_REF] have been tested in our experiments. These models rely on the very popular selfattention mechanism to construct an embedding representation of the input signal, that will next be used to perform the final binary classification. We introduce the general design approach considered as well as details about the Transformer architecture we have adopted.

1) General Architecture: We consider a general seq-to-one approach i.e. for a given sequence of input values indexed by the temporal dimension (MoCap and corresponding EMG data), the model produces a single 2D output corresponding to the probability distribution of observed protective behaviour (probability that a protective behaviour is observed or not).

The several steps considered are (i) iterative computation of timestep embeddings using transformer blocks based on self-attention, (ii) instance embedding averaging the timestep embeddings, (iii) linear transformation of the instance embedding into R 2 , (iv) softmax to obtain the final probability distribution over the two classes for protective behaviour.

In the following we index by t ∈ 1, 180 the different timesteps composing a given input to classify. 1 For a timestep t, we also denote respectively the 4×1D EMG input values and the 17×3D MoCap input values by x t EM G ∈ R 4 , and x t M OC ∈ R 51 respectively. In this section x t = (x t EM G , x t M OC ) is the concatenation of the two vectors x t EM G and x t M OC at time t (x t ∈ R 55 ). The input provided to the transformer model was X = [x (1) , . . . , x (180) ] ∈ R 55×180 . No additional efforts have been made to help the model (i) extract specific characteristics of the MOCAP and EMG data (e.g. by applying specific preprocessing steps), (ii) explicitly distinguish between the 17 3D coordinates corresponding to the MoCap data.

The general transformer architecture considered in our experiments can mainly be seen as an encoder stacking n transformer blocks that are trainable mappings, i.e. a transformer block i ∈ 1, n is a mapping B (i) : R d×180 → R d×180 , i.e. the stacking is the composition B (n) (. . . (B (1) (X))). A linear transformation is first applied element-wise to each of the timestep entries of X ∈ R 55×180 to obtain the R d×180 input that will be given to the first transformer block, i.e. the same linear transformation from R 55 to R d is used to project each x t , t ∈ 1, 180 into R d (d is the embedding size and an hyperparameter of our system).

Each transformer block applies standard treatments such as : (i) standard Multi-Head self-Attention (MHA), (ii) normalization prior and after MHA, (iii) a feedforward network applied element-wise using 2 fully connected layers based on ReLu activation function. The Query, Key, and Value linear transformations of each transformer block is a transformation from R d to R d with each of the h head processing an input in R d/h (with configurations defined such as d mod h = 0).

Since standard Transformers are permutation invariant, we implemented optional positional embeddings a trainable linear 1 i, j denotes the set of integer between i and j, both included. projection applied to each timestep (linear transformation from 1, n to R d ). When we activate positional embeddings, the embedding is summed to the inputs prior to applying the first transformer block. The final instance embedding has been defined as the average of the output of block B (n) which can be seen as the final timestep embeddings. A simple linear projection is then applied to obtain a R 2 projection on which a softmax is applied to obtain the probability distribution referring to the protective behaviour. Figure 1 illustrates the model.

2) Specific design choices: In order to constrain the architecture which relies on a large number of parameters, we have considered shared trainable parameters among the transformers blocks (i.e. same parameters among each B (i) , i ∈ 1, n ). Considering an architecture with n transformer blocks, the encoder part of the model in charge of the processing from the input data (optionally augmented by the positional embeddings) to the timesteps embeddings can therefore be seen has a repeated processing of the same transformer block. In addition, no residual connection (skipping connection) has been used.

3) Training:

We added several dropout layers during training to avoid overfitting. The loss function we consider is a weighted Binary Cross (weights computed at batch level), optimized with Adam.

We train several Transformer models based on the aforementioned architecture using different hyperparameters (number of blocks, number of heads, size of internal projections d, using positional embeddings or not, batch size, the learning rate). We then select 21 models that maximizing the sum of the F 1 measure obtained on both given training and validation sets considering all training epochs -these models have been trained up to 4k epochs. F 1 sums of the selected models range from 1.63 to 1.60. Figure 2 and3 show the results obtained in training two architecture configurations from which some of the 21 final models have been selected (smoothed with 10-point moving average). Note that several selected models share the same hyperparameter configurations but have been obtained at different training epochs.

4) Prediction:

We then use a voting strategy over the predictions of the 21 models. For a given test input, the prediction is output maximising the probability of the output class. The final class considered for a given test input is defined using majority voting of all 21 predictions made by the 21 models. Figure 4 presents the distribution of the percentage of votes for majority classes. We observe that the 21 selected models fully agree (same prediction) in 76.4% of the test cases. A majority agreement of less than 90% of the models can be observed in 11.3% of the test cases.

C. LSTM

Here, we used two bi-directional LSTM layers model with batch normalization and dropout. Considering that LSTM type models in the two works [START_REF] Egede | EMOPAIN challenge 2020: Multimodal pain evaluation from facial and bodily expressions[END_REF], [START_REF] Wang | Recurrent network based automatic detection of chronic pain protective behavior using mocap and semg data[END_REF] showed relatively poor Only the MoCap data is used in this baseline model as pretraining couldn't include EMG data. The input of the model is thus 51 × 180 (17 sequences of 3D-joints). All data is normalized with z-score in which the main mean and standard deviation used for the filtering are estimated from the pretraining set. Since the pre-training dataset is quite large and complex, the number of hidden units are set relatively high (512 units for the 1st layer, 256 units for the 2nd layer). The learning-rate is fixed at 0.01 in the whole training and final testing process. No voting is applied, only the model that gave the best performance on the validation set is chosen to predict with unlabeled test set. This means that the final model tends to be "biased" towards the validation set whether there's over-fitting or not, which creates the risk of a collapse of performance under the test set. The pre-training on NTU RGB+D reached a stable validation accuracy 70.69%. For the fine-tuning of the pre-trained model, the best model obtained showed an accuracy of 90%, F 1 at 0.6514 for protective behaviour class. On the validation set, it seemed that this model is interesting in that it appeared to outperform the more complex models (Transformer and GCN), however, as we'll see in the meta analysis, the phenomenon didn't persist in the final evaluation on the test set.

D. Graph Convolutional Network

Graph Convolutional Networks for skeleton-based action recognition take into account the spatial and temporal characteristics specific to human skeleton data. Indeed, the human skeleton can be seen as a natural graph, and if we also consider the evolution over time of 3D joint coordinates, skeleton-pose data can be modeled as a temporal graph. Such a representation is efficient with regard to skeletal pose data and also in preserving the most important information and topology. Architecture like ResGCN [6] also includes part-wise attention, which can be beneficial for protective behaviour detection, as some parts of the body are more significantly involved in the protective behaviour (the area impacted by pain, typically the lower neck or the lower back). 1) Architecture: For the GCN implementation we started from ResGCN in its PA-ResGCN variant [6], which encodes three main features extracted from skeleton poses: joints (like most other systems), velocities and bones. The use of these three features allows PA-ResGCN to reach strong SOTA results as highlighted in Section 2. We modify the official implementation of ResGCN 2 by adapting the data loader for the data format of AffectMove Task 1, notably the formulation of the graph for the GCN. The class imbalance in the dataset prevents PA-ResGCN from performing as well as the baseline models, which is why we propose a weighted reformulation of the binary cross-entropy loss that accounts for and compensates the imbalance during training.

Additionally, we propose a simple multimodal extension that performs a late fusion between PA-ResGCN(-N51) and a convolutional neural network for EMG classification.

For the latter, the signals are first filtered with an exponential moving average (α = 0.2). The CNN architecture consists of one layer normalization per Batch to adjust scale of input EMG data with learnable parameters, followed by one 2D convolutional layer, batch pooling and two linear layers. PReLU activation is used after the convolutional layers and Tanh in between the two linear layers. This architecture serves to extract 128 EMG features, which are then concatenated to the 256 features extracted by PA-ResGCN-N51. Figure 7 illustrates the full multimodal architecture, please refer to Song et al. [6] for the details of the PA-ResGCN-N51 architecture and part-wise attention.

2) Loss definition: We use Binary cross-entropy with logits along with an added weighing scheme to penalize the majority class. The weights are computed per batch like in Cui et al. [START_REF] Cui | Class-balanced loss based on effective number of samples[END_REF], where we consider β → 1, so that following the paper's notation, E n = n s,k . The loss l k for the k th batch is defined as :

l k = k w s [y s logσ(x s ) + (1 -y s ) • log(1 -σ(x s ))] (1)
w s = 1 n s,k n s,k = n -,k + , if y s = 0 n +,k + , otherwise (2)
Where B k is the set of training examples of the k th batch, w s is the weight per sample s where n (-,k) and n (+,k) are respectively the number of negative and positive samples present in the k th batch, was set to 10 -6 to avoid division by 0 errors.

3)

Training & Optimization schedule: First, we use only the 3D MoCap data to train the PA-ResGCN, particularly the PA-ReGCN-N51 variant where N51 means there are 51 convolutional or FC layers within the model and where PA stands for part Attention. We experiment with two types of optimisation approaches: (i) Optimizer based on stochastic gradient descent (SGD) with a Nesterov momentum of 0.9, weight decay of 10 -4 and with cosine scheduler (warm restarts [START_REF] Loshchilov | Sgdr: Stochastic gradient descent with warm restarts[END_REF]), similarly to Song et al. [6], (ii) an Adam optimizer with an initial learning rate of 10 -3 . The SGD optimizer with a cosine scheduler led to better results in comparison with the Adam optimizer (albeit slightly slower). Secondly, we jointly train the CNN for EMG measures and PA-ResGCN-N51 for 3D skeleton data for training the architecture to include information from EMG.

E. Model selection & validation

The model selection results are presented in Table III. We first evaluated the PA-ResGCN-N51 architecture at 0.77 million parameters in different Loss/optimizer settings, we then evaluated the best performing combination with the PA-ResGCN-B19 model with basic blocks (B19 stands for 19 basic blocks, 3.61 million parameters) that achieved SOTA on NTU-60 and NTU-120. Finally, we evaluate the inclusion of EMG to PA-ResGCN-N51 (best performing model). The PA-ResGCN-N51 model configuration with a weighted loss and SGD gives the best performance with an F 1 score 0.64 in comparison with the same model with SGD and an unweighted loss (0.54 F 1 , +10%). This improvement demonstrates the benefits introduced by the weighted loss. We can see that unweighted loss generally gives a slightly better precision (compared to the PA-ResGCN-B19 model), with a recall approximately similar to precision, while the weighted loss maximizes the recall and F 1 score. The Adam optimizer (on PA-ResGCN-N51) shows sub-par performance compared to SGD with the cosine scheduler. With the PA-ResGCN-B19 model for the best previous loss/optimizer combination, we observed lower overall performance. Contrarily to NTU-60 or NTU120, which are very large databases, our training data is comparatively smaller for Task 1 and we hypothesize that this model needs a lot more data to actually converge to something better. Likewise, the introduction of the EMG signals to the PA-ResGCN-N51 model degrades the results, as performance deteriorates by -6%. The reason for this decrease could be either a low signal-to-noise ratio in the filtered EMG signal envelope, or the cause of an ill-adapted fusion technique (single fully connected layer).

IV. RESULTS AND DISCUSSION

If we consider the final ranking of our submitted runs on the test set of Task 1, unsurprisingly, the best performing system was PA-ResGCN-N51 (w/o EMG) with a F 1 score of 53.36% on the test set for the minority class (MCC 0.4247, Table IV line 3). The Transformer baseline (MoCap + EMG) came second at 41.05% F 1 test performance (MCC 0.3523, Table IV line 2) and the LSTM baseline third at 31.16% F 1 (MCC 0.1763, Table IV, line 1). While the Transformer had a comparable behaviour on the validation set and on the test set, the LSTM was actually on-par with PA-ResGCN-N51 on the validation set. On the test set the performance of the LSTM collapsed with a F 1 of only 0.3116. This seems to comfort the hypothesis that, the distribution of subjects in the validation set is quite different from the training or test test sets. Although, the objective of the task was to provide a realistic setting, the careful selection of a good validation set is paramount to producing models that generalize well regardless of the non-competitive framing. Another reason

V. CONCLUSION

In this work, our team has explored two baseline strategies (Transformer and LSTM) and a state of the art approach (PA-ResGCN) for the skeleton + EMG based protective behaviour prediction for Task 1 of the AffectMove challenge. The SOTA architecture of PA-ResGCN did indeed achieve the best performance both on the validation set and test set of the challenge compared to the baseline approaches without using the EMG signal. Although the fusion strategy we implemented to add EMG to PA-ResGCN-N51 was unsuccessful, it was trivial in nature due to the time constraints. More state of the art multi-modal fusion strategies may prove successful at integrating EMG and leading to increased performance (as observed with the Transformer). The effectiveness of PA-ResGCN remains relative, as the amount of training data for task 1 is on the smaller end of the scale. A more thorough error-analysis will allow for specific adaptations to PA-ResGCN that would lead to better results.

2 )

 2 Detection of Reflective Thinking based on Body Movement Data. The second task provided participants with MoCap data only for reflective thinking classification This work was granted access to the HPC resources of IDRIS under the allocation 2021-AD011011309R1 made by GENCI.

Fig. 1 .

 1 Fig. 1. The transformer baseline architecture.

Fig. 2 .

 2 Fig. 2. Evolution of several performance metrics (F 1 , MCC, Accuracy and Balanced Accuracy) during training for a specific tested Transformer architecture (blue training set, orange validation set) -values have been smooth averaging the 100 surrounding values of each point. The architecture used all 55 features, 15 blocks, 4 heads, embedding sizes of 256, used positional embedding; training has been made using a batch size of 50 and a learning rate of 10 -4 .

Fig. 3 .

 3 Fig. 3. Similar to Figure 2 with different hyperparameters: 55 features, 3 blocks, 8 heads, embedding sizes of 256, used positional embedding; training has been made using a batch size of 50 and a learning rate of 10 -4 .

Fig. 4 .Fig. 5 .

 45 Fig. 4. Distribution of the percentages of votes given to the selected class (majority voting).

Fig. 6 .

 6 Fig. 6. Pre-training process on NTU RGB+D dataset. NTU RGB+D dataset is randomly split into training 90% and validation 10% sets in which the classes are stratified. The stopping validation accuracy at 57 epochs on the 60 classes is 70.69%.

Fig. 7 .

 7 Fig. 7. Architecture of the multimodal PA-ResGCN-N51 + EMG CNN system. The final run only includes the left branch without EMG.

TABLE I CHARACTERISTICS

 I AND DISTRIBUTION OF CLASSES ACROSS THE TRAIN AND VALIDATION SETS.

		Training set			Validation set
	Protective Count Percent Protective Count Percent
	0	4522	77.60%	0	1580	85.68%
	1	1305	22.40%	1	264	14.32%

TABLE II RE

 II -TRAINING RESULTS WITH THE PRE-TRAINED MODEL ON training SET, EVALUATED ON validation SET (1844 SAMPLES), EARLY STOPPED AT THE END OF 3 EPOCHS

		Metrics
		Prot. B. 0 Prot. B. 1
	Specificity	0.6552	0.9413
	Precision	0.9430	0.6477
	Recall	0.9413	0.6552
	F 1	0.9421	0.6514
	MCC	0.5936
	Accuracy	0.9008

TABLE III MODEL

 III SELECTION RESULTS FOR PA-RESGCN. THE FIRST THREE LINES AND THE LAST LINE ARE BASED ON THE PA-RESGCN-N51 MODEL, WHILE LINE FOUR IS BASED ON THE PA-RESGCN-B19.

	Model	Params. EMG	Weighted. Loss	Optimizer Class	F1	P	R	Avg. F1	Weighted Avg. F1	MCC Acc.	Balanced Acc.
	N51	0.73m		SGD	1 0	0.54 0.55 0.52 0.93 0.92 0.93	0.73	0.87	0.46	0.87	0.73
	N51	0.73m		SGD	1 0	0.64 0.56 0.74 0.93 0.95 0.90	0.78	0.89	0.58	0.88	0.82
	N51	0.73m		Adam	1 0	0.50 0.42 0.63 0.89 0.93 0.85	0.70	0.83	0.41	0.82	0.74
	B19	3.61m		SGD	1 0	0.56 0.47 0.69 0.91 0.94 0.87	0.73	0.86	0.48	0.84	0.78
	N51 + CNN	0.86m		SGD	1 0	0.60 0.93	0.56 0.64 0.94 0.91	0.75	0.88	0.53	0.88	0.78

TABLE IV OFFICIAL

 IV RESULTS ON TASK 1 TEST SET FOR THE THREE SUBMITTED RUNS the poor test performance of the LSTM could be that MoCap-only may be insufficient to predict if one behaviour is protective or not. Even though pre-training has been proved to be useful in many other application, since the test set labels are not disclosed yet, one cannot truly evaluate whether pretraining is beneficial or whether it hurts the generalizability of protective behaviour detection. Despite the poor behaviour of the LSTM architecture on the test corpus, pre-training did lead to a significant improvement on the validation set, and we can extrapolate that an architecture like ResNet in its larger form, could have benefited much more from pre-training on other datasets such as NTU, before fine-tuning on Task 1.

	System	F 1 C0	F 1 C1	MCC	Acc.
	1. LSTM Baseline	85.83	31.16	0.17628 76.49
	2. Transformer Baseline	89.96	41.05	0.35234 82.84
	3. PA-ResGCN-N51	89.07	53.36	0.42471 82.28

of