Cyril Garnero

Mathieu Lepesant

Cécile Marcelot

Yooleemi Shin

Christian Mény

Pierre Farger

Bénédicte Warot-Fonrose

Raul Arenal

Guillaume Viau

Aikaterini Soulantika

Cécile Garcia-Marcelot

Christian Meny

Katerina Soulantica

Pierre Fau

Patrick Poveda

Lise-Marie Lacroix
email: lmlacroi@insa-toulouse.fr

Bruno Chaudret

Chemical Ordering in Bimetallic FeCo Nanoparticles: From a Direct Chemical Synthesis to Application As Efficient High-Frequency Magnetic Material

Keywords: Magnetic nanoparticles, Zero field 58 Co FNR, Mössbauer spectroscopy, composite materials, inductor-based filter TOC :

  L'archive ouverte pluridisciplinaire

The needs for telecommunications have considerably increased in the past decade, requiring high performance, low power consumption and low cost radiofrequency circuits to support the huge expansion of local exchanges such as video on demand, interconnections with mobile phone applications or shared storage. Among the different passive components present in these circuits, integrated planar inductors play a key role to properly filter the signal of interest while rejecting the surrounding electromagnetic noise up to the GHz, which is the range of frequency commonly used for cellular phone networks worldwide. The efficiency of these inductors, evaluated by the value of their inductance, should be constantly increased. However, geometrical features, such as number of spires and diameter of the wires, can no longer be improved without leading to larger inductors. Thus, alternatives are actively sought, the most promising one consisting in improving the environment permeability by embedding the inductors in a soft ferromagnetic material. [1] An alternative to the magnetic thin films commonly used consists in preparing composite materials composed of optimized magnetic nanoparticles embedded in an insulating matrix. [2] Thanks to the development of liquid-phase synthesis during the past two decades, various magnetic nanoparticles (NPs) of controlled sizes and shapes were synthesized, [3] opening great perspectives for RF filters, as well as answering to the increasing demand in magnetic data storage [4], [5] and in chemical, [6] environmental [7] and biomedical applications. [8] Single crystalline Fe, Co or Ni NPs have been obtained with a nanometric size control, [9] and shapes ranging from spheres to cubes, rods or wires, allowing in turn a fine control of their physical properties. [3] Among the different magnetic materials, the FeCo alloy constitutes the material of primary choice for magnetic inductors since it combines the highest saturation magnetization (MS Fe50Co50 = 235 A.m².kg -1 ) with a low anisotropy constant (K Fe50Co50 = 1,5.10 4 J.m -3 ), [START_REF] Coey | Magnetism and Magnetic Materials[END_REF] allowing high frequency applications. However, such bimetallic alloy NPs are highly challenging to synthesize. Segregation issues are often encountered between the two components, leading to core-shell, onions or dumbbell NPs. [START_REF] Ferrando | [END_REF] Despite important efforts to develop liquid phase syntheses, including thermal decomposition [12] , polyol process [13][14] , hydrothermal synthesis [15][16][17] and sonochemical method [18,19] , crystalline FeCo nanoparticles (NPs) could rarely be achieved directly. Indeed, the difference of reactivity of the precursors and the mild temperature of reaction used lead to poorly crystallized nanoparticles with inhomogeneous chemical structure, which therefore exhibit a reduced saturation magnetization. A high temperature annealing process is then mandatory to achieve the desired bulk magnetic properties, but at the expense of the size and shape control of the nanoparticles due to sintering.

Our group has previously reported the synthesis of 15 nm FeCo NPs, which self-organize into millimeter supercrystals, by the co-decomposition of Fe(CO)5 and Co(η 3 -C8H13)(η 4 -C8H12) under H2. [20][21][22] The particles exhibited however, a magnetization limited to 160 A.m².kg -1 due to a chemically inhomogeneous structure and a partial carburation, which is likely resulting from the presence of CO in the iron precursor. [23] Thanks to a fine optimization of the organometallic precursors used, we report here the synthesis of highly crystalline FeCo nanoparticles with bulk magnetic properties. Depending on the nature of the ligands used, the structure of the NPs could be varied from a radial composition gradient configuration to the chemically ordered B2 phase, as revealed by the combined 57 Fe Mossbauer, zero field 59 Co Ferromagnetic Nuclear Resonance (FNR) and XRD studies. The efficiency of a highly magnetic composite material, prepared with optimized FeCo NPs immersed in epoxy resin, was finally tested onto commercial inductors, leading to a 27% improvement of the properties over a broad frequency range.

Results and discussion

FeCo nanoparticles (NPs) were obtained by reacting {Fe[N(SiMe3)2]2}2 (Me = CH3) and {Co[N(SiMe3)2]2THF} (THF = tetrahydrofuran) at 150°C with H2. Depending on the nature and the concentration of the added ligands, nanoparticles with a mean size varying from 1.5 to 90 nm were obtained (Figure S1). Figures 1a andS2 show transmission electron microscopy images (TEM) of nanocubes, with a mean size of 8.3 ± 0.9 nm, synthesized after 48h of reaction in presence of palmitic acid and hexadecylamine in a 3:4 ratio. These particles will be referred hereinafter as FeCo-PA NPs. Replacing palmitic acid by hexadecylammonium chloride leads to NPs with a mean diameter of 11.4 ± 0.7 nm, referred as FeCo-HDAHCl NPs (Figures 1b, S3-4). The reaction kinetic being fastened in presence of HDAHCl ligands, the reaction time could be shortened to 24h (Figures S5-6).

Both types of particles are highly crystalline and exhibit the expected bcc structure as revealed by high resolution transmission electron microscopy (HRTEM, Figures 1c-d) and Xray diffraction (XRD, Figure S7). The crystallite sizes could be extracted using Rietveld refinement and are respectively of 7.1 nm and 9.9 nm for FeCo-PA and FeCo-HDAHCl NPs, in good agreement with the TEM mean size. A thin oxide shell (ca. 1 nm) is clearly observed on the HRTEM and high angle annular dark field image scanning TEM (HAADF-STEM, Figure S8). This shell is formed due to air-exposure during the sample introduction through air in the TEM. The freshly prepared particles are purely metallic, as evidenced by the XRD patterns performed under Ar atmosphere as further confirmed by advanced spectroscopies (see below). Chemical analyses were performed on the final powder by inductively coupled plasma mass spectrometry (ICP-MS) revealing a global composition of Fe52Co48 for both type of NPs, very close to the equimolar ratio introduced. The local distribution of Fe and Co within the nanoparticles was further characterized by electron energy loss spectroscopy (EELS, Figures S8-9) and energy dispersive X-ray spectroscopy (EDX, Figure S10). STEM-EDX mapping revealed a local composition of Fe53Co47, in good agreement with the ICP-MS, revealing the homogeneity of the NP chemical composition. The distribution of the Fe and Co within individual NPs was further evidenced by probe-corrected STEM-EELS profile. While Co-rich core was always detected for FeCo-PA NPs (Figure S8), three different profiles were statistically identified in the FeCo-HDAHCl NPs, corresponding to Co-rich core, onion-like core and FeCo core (Figure S9).

The chemical order was studied at the atomic scale by combining 57 Fe Mössbauer spectroscopy and Zero field 59 Co ferromagnetic nuclear resonance (FNR i.e. nuclear magnetic resonance for ferromagnets) [24] to probe respectively the Fe and Co environments without facing the interference of any oxidation artefacts [25,26] . The Mössbauer signals recorded at 5K and 300K (see Figure S11) exhibit a sextet-like contribution, typical from FeCo alloys [25,27,28] . The absence of any large hyperfine field contribution confirmed that after synthesis, the obtained particles are purely metallic and do not present any oxide shell. The fitting of the experimental spectra, reported in Table S1, evidenced broad hyperfine field distributions both for HADHCl and PA ligands. An additional contribution at low field in the case of PA ligands could be attributed to magnetically coupled Fe(II) molecular species as previously reported for pure Fe NPs [9] , evidencing a slower reduction of the Fe species compared to the HDAHCl case.

The 59 Co FNR spectra clearly evidenced a major difference between the NPs obtained in the presence of PA and HDAHCl ligands. Indeed, a broad spin echo peak was observed in the frequency range from 180 to 320 MHz for FeCo-PA NPs. The fitting of this peak (see figure S12) reveals the presence of a continuous concentration gradient from pure Co towards pure Fe across the NP radius (corresponding to 26 atomic layers). In the case of HDAHCl-NPs three main distinct peaks were observed at 200, 224 and 290 MHz respectively and a shoulder at 211 MHz. The 200 MHz peak can be attributed to a bcc Co metastable phase. Such a phase has been reported in a bcc epitaxial Fe/Co thin film structure [29] , and could reveal the respectively. An original short range order favoring even numbers of Fe atoms around Co has already been reported in CoFe alloy films. [30] The simultaneous observation of the 200 and 224 MHz peaks (with a much smaller 211 MHz shoulder) might be the fingerprint of such unusual order. Finally, the major contribution in the spectra is at 290 MHz and is characteristic of the chemically-ordered local environment known as B2, with a CsCl structure (see inset figure 2d). [31] Since the FNR spectrum reveals Co rich contributions in the 200-224 MHz range, the local B2 order has to be slightly enriched in Fe in order to preserve the FeCo stoichiometry. This is confirmed by the observation of the shoulder at 310 MHz characteristic of a Fe enriched B2 order. To our knowledge, it is the first time that the shortrange B2 order is stabilized within NPs directly obtained after a liquid phase synthesis, thanks to the fine optimization of the precursor reactivity and the ligands used.

In-situ XRD study was performed during the thermal annealing of FeCo-HDAHCl and FeCo-PA NPs (Figure 3 and S13) under a reducing atmosphere (95% N2, 5% H2) following a multi-step temperature profile up to 800°C (Figure S14). The use of a Co-source avoided the Fe atoms fluorescence and allowed the detection of the three superstructure reflections (100), (111) and (210) observed in the long-range crystalline B2 phase. [32] The appearance of the B2 phase, which corresponds to a long-range chemical-ordered CsCl-type structure, was characterized by the temperature evolution of the order parameter S, defined by : [33,[START_REF] Warren | X-Ray Diffraction[END_REF] = -1 -

With rA the fraction of the site A occupied by the right atom A, xA the atomic fraction of A in the alloy (xCo = xFe = 0.5) and FA the fraction of the site A (FCo = FFe = 0.5).

For a Fe50Co50 alloy, S values vary between S=0 for a fully disordered alloy to S=1 for a fully ordered B2 alloy. In the case of FeCo-HDAHCl NPs, a shoulder at the (100) peak is already observed at 300°C, while the three characteristic peaks are clearly evidenced at 400°C, leading to an order parameter S = 0.7 (Figure 3c). The order parameter keeps on increasing, reaching S= 0.76 at 700°C. Above this temperature, the order vanishes, as expected from the bulk phase diagram of FeCo alloys. After cooling down to room temperature, the long-range B2 phase was retrieved as evidenced by the order parameter S = 0.76. In contrast for FeCo-PA NPs, the partial ordering only appears at 500°C and was characterized by an order parameter of 0.55 (Figure S13). Such thermal behavior is in agreement with the previous report on the thermal ordering of disordered 30 nm Fe50Co50 NPs prepared by polyol process. [32] The fact that FeCo-HDAHCl NPs exhibit the B2 structure at lower temperature and with a better ordering factor than previously reported NPs and FeCo-PA NPs results

from the presence of a short-range B2 order within as-prepared FeCo-HDAHCl NPs, as previously shown by Co FNR. The short-range B2 order region acted as seeds during the recrystallization process which was thermally activated, leading to a long-range chemical ordering under mild conditions (300-400°C). Both type of NPs exhibit a soft magnetic behavior at room temperature (Figure 4), with saturation magnetizations (MS) of 226 and 204 A.m².kg -1 FeCo for HDAHCl and PA NPs respectively, very close to the expected bulk value (MsFe50cCo50 = 235 A.m².kg -1 FeCo) [START_REF] Coey | Magnetism and Magnetic Materials[END_REF] . The low temperature curve (Figure S15) revealed no exchange bias indicating that the FeCo nanoparticles are not oxidized, in agreement with the Mössbauer spectroscopy. The organometallic synthesis of FeCo NPs starting from two amide precursors yielded for the first time highly crystalline NPs with tunable sizes. In the presence of palmitic acid, carboxylate-containing precursors of Co and Fe which have different stabilities, lead to NPs exhibiting a concentration gradient. [9], [START_REF] Liakakos | [END_REF] Thus, one can assume that the reduction followed a two-step process consisting in a first nucleation of Co nuclei, which then favored the decomposition of the more stable Fe-carboxylate precursor, leading to a fairly slow NP growth. On the contrary, in the presence of HDAHCl, the intermediate chloride species formed seem to have a comparable reactivity, preventing the formation of a pure Co core.

The concomitant co decomposition of the Fe and Co species lead for the first time to the synthesis of the chemically ordered short-range B2 structure. Therefore, a very fine tuning of the mild reaction conditions opens the route for the synthesis of magnetic alloys with controllable chemical structure and thus physical properties.

For the integration purposes, the net magnetic fraction into the composite materials being a key parameter, as-synthesized FeCo-HDAHCl NPs were preferred to FeCo-PA NPs since they exhibit a higher magnetization and a larger metallic content (80% and 50% respectively).

These NPs were mixed with epoxy monomers and let to polymerize at room temperature to form ca. 250 µm thick self-supporting composite pellets (Figures 5a inset andS16). The epoxy resin was chosen as a matrix for the FeCo composites since it advantageously combines electrical insulating characteristics, essential for high frequency applications, with an enhanced chemical resistance and efficient barrier properties to oxygen and moisture [36] .

The dispersion of the nanoparticles within the composite pellet cross section was characterized using SEM and TEM (Figures 5a-b). Thanks to a slow evaporation of the NP solvent, only few aggregates are observed, FeCo NPs being well dispersed along the whole thickness of the pellet. The loading of the composite could be reproducibly varied from 10 to 65% weight fraction of FeCo NPs, leading to a net magnetization under an applied field of 3T varying from 26 to 152 A.m².kg -1 (Figures 5c andS17, and Table S2). Assuming that the density of each components of the composite materials is similar to its bulk value, namely ρFeCo = 8.3, ρligands = 0.9 and ρepoxy = 1.1, net magnetic volume fractions varying from 2 to 19% were determined.

Figure 5: a) SEM image of the cross section of a 30% in weight FeCo-HDAHCl NP loaded pellet. Due to the use of backscattered electrons, the metallic NPs appear as white dots while the epoxy appears grey, inset: optical image of the 4mm squared pellet b) TEM image of an ultramicrotome slice of the epoxy composite. c) Evolution of the room temperature magnetization at 3T (M3T) of the composites as a function of the targeted FeCo loading. A linear fitting is superimposed as a guide for the eyes. d) Evolution of the relative 3T magnetization (solid scatters and lines) and the exchange bias (open scatters and dashed lines) as a function of air exposure for pure FeCo-HDAHCl NPs (black) and the epoxy composite loaded with 30% (circles) and 60% (triangles) in weight FeCo-HDAHCl NPs. e) Picture of commercial silicon chip containing copper micro-inductors topped with FeCo rich epoxy composites of this study, f) evolution of the inductance as a function of frequency for the commercial inductor (black line) and a similar inductor covered with epoxy composite loaded with 30% (red line) and 60% (blue line) in weight of FeCo-HDAHCl NPs.

The magnetic properties were assessed as a function of time under air exposure to probe the protection against the oxidation of the epoxy matrix. The magnetization of pure FeCo-HDAHCl NPs quickly diminished due to a strong oxidation once exposed to the air. The exchange bias, measured at low temperature increased up to 65 mT after only 5 days. In the case of the epoxy-FeCo composite, 80 to 90% of the magnetization was retained after a 65day air exposure for 60% and 30% loaded composite respectively. An exchange bias limited to 14 and 8 mT respectively confirmed that the FeCo NPs are fairly well protected against the oxidation, the epoxy resin acting as an efficient air barrier. The resulting composite was successfully integrated on a commercial inductor-based filter using a pick and place approach. Enhancement of 18% and 27% of the inductance value at 100 MHz were measured with the 30% and 60% loaded composite respectively compared to the air-core commercial inductor reference. Such increase represents an important step for the development of commercial planar inductors.

Conclusion

We report here a new chemical synthesis of monodisperse FeCo nanoparticles using the co- Experimental Section:

decomposition

General synthesis

Mesitylene (99%), toluene (99%), tetrahydrofuran (THF, 99%) and pentane (99%) were purchased from Merck, Acros, Carlo Erba and Fisher respectively. The solvents were purified on alumina desiccant (Pure Solv, Innovativ Technology) and degassed through three freezepump-thaw cycles. Hexadecylamine (HDA, 99%) was purchased from Sigma-Aldrich. The two metallic precursors (Fe{N[Si(CH3)3]2}2)2 (FeNSi, >99.9%) and Co(N(Si(CH3)3)2)2,THF (CoNSi, >99.9%) were purchased from Nanomeps.

Synthesis of HDAHCl.

Hexadecylamonium chloride (HDAHCl) was synthesized following an adapted procedure described elsewhere [37] . Briefly, a solution of 10 mmol of hexadecylamine (HDA) (2.415 g) in 400 mL of pentane was prepared in the glove box. An excess amount of HCl in diethylether (14 mL, 1.0 M) was added under Ar atmosphere. The solution turned milky white immediately. The solution was stirred for 30 min then allowed to decant. After the removal of the supernatant, the resulting white concentrated solution was washed 2 times with pentane. The final solution was evaporating and 2,5 g (90% yield) of white powder (HDA•HCl ) was obtained. Preparation of the magnetic composite.

Synthesis of

In a glove box, 1,5 mL of THF is added on 50 mg of powder. To ensure a proper dispersion, the solution was sonicated for 30 min. To prepare a composite loaded with 30% of metal in weight, 50 mg of the epoxy monomer (LY5052) and 19 mg of the hardener (Aradur 5052)

were then added. The resulting solution was further sonicated for 30 min and drop casted in a PDMS mold with 4 pitches of 4 x 4 x 0.5 mm 3 . 5 μL drops were deposited in each pitch and the THF allowed evaporating. To obtain a sufficient thickness, the process was repeated 13 times. After a 24h reticulation of the epoxy resin at room temperature, mechanically stable ~250 µm thick composite pellets were retrieved.

Structural and chemical characterizations

The chemical compositions were determined by chemical analysis using inductively coupled plasma mass spectrometry (ICP-MS, Pascher laboratory).

FeCo NPs were characterized by transmission electron microscopy (TEM), using a 100kV Jeol JEM 1011F and a JEM-ARM200F Cold FEG microscope with corrected probe. For sample preparation, the particles were redispersed in toluene and a drop of the dispersion was deposited on a carbon coated copper grid. The chemical profile measured by energy loss spectroscopy were acquired using the Scanning TEM mode with a probe size of 0,15 nm and steps of 0.477 nm. Energy X-ray dispersive analysis were conducted routinely on a Scanning Electron Microscopy (SEM) JEOL 2100F equipped with a SSD Brucker detector to determine the relative FeCo content.

Spatial-resolved electron energy loss spectroscopy (EELS) measurements were performed on probe-corrected scanning transmission electron microscope (STEM) FEI Titan Low-Base 60-300 operating at 300 keV (fitted with a X-FEG® gun and Cs-probe corrector (CESCOR from CEOS GmbH)). EEL spectra were recorded using the spectrum-imaging (SPIM in 2D or spectrum-line (SPLI) in 1D) mode [38] in a Gatan GIF Tridiem ESR 865 spectrometer. The convergent semi-angle was of 25 mrad, the collection semi-angle was of 80 mrad and the energy resolution ~ 1.2 eV.

XRD measurements were performed on a PANalytical Empyrean diffractometer using Co-Kα radiation and equipped with a linear Pixcel1D detector. In a glovebox, few mg of the powder was sealed between two kapton sheets to prevent any air exposure. The in-situ annealing measurements were performed in a sealed chamber XRK900 from Anton Paar under a reducing atmosphere (1 bar, 5% H2, 95% N2). The powder patterns were measured at room temperature, 100°C, 200°C, 250°C, 300°C, 400°C, 500°C, 600°C, 700°C, 800°C then 30°C after cooling down at a rate of ca. 17°C/min (Figure S12). For FeCO-PA NPs, the two first steps at 100°C and 200°C were skipped. At each step the sample was maintained for 1h prior to the XRD acquisitions consisting of a 35-110° acquisition in 20 minutes with 0.067°.s -1 as scan speed and a step of 0.052°, and three high-resolution acquisitions in the 35-38°, 64-67° and 88-91° region with an accumulation of 24 minutes.

The rietveld refinement for the determination of the order parameter S was performed using MAUD software. [37] the refinement involved the fit of the background polynomia, the scale factor, the profile parameters, the lattice parameters, one equivalent Debye-Waller factor and the occupancy rate of Co and Fe atoms on the special positions (000) and ( ).

In order to keep the stoichiometry, the atomic occupancy were restrained using the relations below : Mössbauer spectra were measured at 300 K and 5K using a 57 Co source (WISSEL). Samples were prepared in the glovebox, and extreme care was taken to avoid oxidation during transfer to the apparatus. The analysis of Mossbauer spectra was performed by fitting the data by Lorentzian shape peaks and applying a least-squares method using Recoil software.

The 59 Co FNR experiments (NMR in ferromagnets) were performed at 2K without any external static field in a home-made spectrometer. The integrated spin-echo intensity was recorded a broadband un-tuned (high-pass design) pulsed FNR spectrometer with phase-sensitive detection and automated frequency scanning (pulse length: 5 μs, delay: 3 µs). The frequency response of the setup is flat within 0,5 dB. In order to prevent oxidation the samples were prepared in the glove box and sealed in regular NMR tubes with vacuum grease and epoxy resin.

Physical characterizations

Magnetic measurements were performed using a Quantum Design Physical Property Measurement System (PPMS) in the Vibrating Sample Magnetometer (VSM) configuration.

For the FeCo NPs, 10 mg of powder were introduced in a Teflon cap sealed by vacuum grease. For the composite, the pellet was directly mounted on a quartz sample holder. 3T magnetization cycles were recorded at 300K and 5K after a 3T field cooling.

Figure 1 :

 1 Figure 1: a-b) Conventional transmission electron microscopy (TEM) micrographs. c-d) HRTEM images of FeCo nanoparticles prepared with PA (left column) or HDAHCl (right column).

Figure 2 :

 2 Figure 2: a-c) 5K 57 Fe Mossbauer spectrum and b-d) 59 Co Ferromagnetic Nuclear Resonance spectrum of the a-b) FeCo-PA and c-d) FeCo-HDAHCl nanoparticles. As inset, schematic view of the gradient profile determined for PA NPs and B2 order for the HDAHCl NPs (Fe : blue, Co : red) . Dashed lines are guides for the eyes for the different contributions on FeCo-HDAHCl FNR spectrum.

  presence of NPs exhibiting a pure Co part close to a FeCo alloy. The 211 MHz shoulder and the 224 MHz contribution peak evidenced a structure composed of Co-rich bcc FeCo phase with 1st shell coordination polyhedra containing 7 Co : 1 Fe atoms and 6 Co : 2 Fe atoms

Figure 3 :

 3 Figure 3: a) In-situ XRD recorded during thermal annealing of FeCo-HDAHCl NPs under reducing atmosphere. b) Enlarged view of the characteristic B2 superstructures peaks. c) Thermal evolution of the order parameter S determined for FeCo-HDAHCl (red dots) and FeCo-PA (black triangles) NPs. Red solid line, black dashed line and blue arrows are guide for the eyes following the temperature increase and the final cooling.

Figure 4 :

 4 Figure 4: Magnetic hysteresis loop at 300K of the as-synthesized FeCo NPs prepared in presence of PA (black line) or HDAHCl ligands (red line).

  of two metalloid amides under mild conditions without post treatment annealing. A fine tuning of the precursors reactivity was achieved thanks to the ligands used leading to two different NP type. For the first time the presence of the ordered B2 phase was evidenced in NPs prepared in presence of HDAHCl ligands, leading to magnetic properties comparable with the bulk ones (Ms = 226 A.m².kg -1 ). Optimized composite materials were prepared by mixing these NP with epoxy monomers. Composites with variable magnetic loadings were prepared and successfully integrated on inductor-based filters, leading to a 27% enhancement of the inductance value at 100 MHz. Although this enhancement remains modest this study evidences the possibility to use ex situ synthesized NPs for inductance improvement, opening the path for NP-based composites for advanced microelectronics applications after improvement of the magnetic volume fraction of the composite and of the device geometry by the full immersion of the inductors into the magnetic material.Acknowledgements:Lucien Datas and Jean-François Meunier are warmly thanked for high resolution TEM and Mössbauer spectroscopy measurements. This work was performed in the frame of TOURS 2015, project supported by the French « Programme de l'économie numérique des Investissements d'Avenir». We gratefully acknowledge the International Associated Laboratory (LIA)-M²OZART for financial support. Some of the HR-STEM and EELS studies were conducted at the Laboratorio de Microscopias Avanzadas, Instituto de Nanociencia de Aragon, Universidad de Zaragoza, Spain. R. A. gratefully acknowledges the support from the Spanish Ministry of Economy and Competitiveness (MINECO) through project MAT2016-79776-P (AEF/FEDER. UE). In IPCMS Strasbourg the work was supported by the CNRS LIA "NANOFUNC" and the LABEX NIE (No. ANR-11-LABX-0058_NIE).

  FeCo nanoparticles. * Preparation of the precursor solution In a glove box, 150 mg (0,4 mmol) of Fe(N(Si(CH3)3)2)2 and 180 mg (0,4 mmol) of Co(N(Si(CH3)3)2)2,THF were separately dissolved in 2,5 ml of mesitylene and then mixed leading to a green solution. * Preparation of FeCo-PA NPs 308 mg (1,2 mmol) of PA and 386 mg (1,6 mmol) of HDA were dissolved in 10 ml and 5 ml of mesitylene respectively. The PA solution was added onto the precursor mixture, which became blue. The resulting solution was magnetically stirred during 5 min. The HDA solution was added, leading to a violet coloration which quickly vanished, the solution returning to blue after few seconds. The final 20 mL solution was transferred into a Fisher Porter bottle and pressurized under 3 bars of H2, the solution became darker. The reaction was heated at 150°C and let to react 48 h to allow the reduction of the stable Fe-carboxylate species formed in-situ. * Preparation of FeCo-HDAHCl NPs 334 mg (1,2 mmol) of HDAHCl and 386 mg (1,6 mmol) of HDA were dispersed in 10 ml and 5 ml of mesitylene respectively. The precursor solution was added onto the HDAHCl one, which became brownish. The resulting solution was magnetically stirred during 5 min, the solution turning to dark blue. Finally, the HDA solution is added. The final 20 mL solution is transferred into a Fisher Porter bottle and placed in a pre-heated oil bath at 150 °C for 1 h under Ar blanket to ensure the complete dissolution and homogenization of the reactants.The solution was then allowed to cool down at room temperature and further pressurized under 3 bars of H2. The reaction was heated again at 150°C for 24 h to allow the decomposition of the precursors.* Purification of the NPsAt the end of both type of reaction, the H2 atmosphere was released, a black solution was observed. The NPs were recovered by magnetically assisted precipitation, washed twice with toluene and twice with THF and finally dried, leading to a black powder kept in the glove box. Reaction yields were calculated from the final metallic weight content of the powder over the initial Fe and Co weight introduced. Yields of 23 and 66% were obtained for FeCo-PA and FeCo-HDAHCl NPs, respectively, due to partial weight losses during the purification steps and a non complete reduction of the Fe-carboxylate species as previously discussed.

)

  corresponding to rA in eq(1).


Dynamic HAADF STEM observation of single atom chain as transient state of Au ultrathin nanowire breakdown.

Lise-Marie Lacroix, † * Raul Arenal, ‡+ * Guillaume Viau † . † LPCNO, Université de Toulouse, INSA, UPS, LPCNO (Laboratoire de Physique et Chimie des Nano-Objets), F-31077 Toulouse, France; CNRS; UMR 5215 ; LPCNO, F-31077 Toulouse, France ‡ Laboratorio de Microscopias Avanzadas (LMA), Instituto de Nanociencia de Aragon (INA), U. Zaragoza, C/ Mariano Esquillor s/n, 50018 Zaragoza (Spain). + Fundacion ARAID, 50004 Zaragoza, Spain.

Supporting Information Placeholder

ABSTRACT: Ultrathin chemically grown Au nanowires undergo irremediable structural modification under external stimuli. Thanks to dynamic high-angle annular dark field imaging, electron beam induced damaging was followed, revealing the formation of linear gold atom chains as long as reactive clusters on the side, opening fascinating perspectives for both catalysis and electronic transport applications.

Metallic nanowires represent ideal objects for fundamental studies as long as potential applications in sensing, 1 catalysis and electronic contacts. 2,3 Quantized conductance scaling with 2e 2 /h was observed in atomic wires obtained by mechanical deformation of point contact, 4,5,6 breaking junction, 7 electrochemical approaches. 8 Recently, ultrathin gold nanowires exhibiting a diameter of 1.7 nm and a micrometric length were synthesized by reduction of gold chloride salt in presence of oleylamine. 9,10,11,12 These highly crystalline wires, grown along the <111> direction of the fcc structure, opens great perspective for electronic transport studies. 13,14 Their small diameter leads to quantized phenomena while their extreme aspect ratio render physical contact by lithography feasible. 14,15 However, thin metal nanowires are prone to fragment into chain of spheres at high temperature, as described by the Plateau-Rayleigh instability. 16,17 The critical temperature at which this phenomenon occurs is strongly dependent on the wire surface energy and thus on the crystalline surface exposed and on the ligands adsorbed. 18 Other perturbation such as electron beam can also induce such instability. 19 Radiolysis, due to plasmon losses, and knock-on damage, resulting from high energy scattering, are the two most common types of primary damage that occur in irradiated samples, 20 depending on their electronic state. 21,22 We report here for the first time a dynamic study of the Au wire breaking by HAADF-STEM. Sub atomic resolution obtained with the aberration corrected microscope combined with the high diffusion of Au enabled to follow the trajectory of individual atoms, evidencing the formation of atom chains prior to the breaking and free clusters aside. 1.7 nm ultrathin Au nanowires were obtained in liquid phase following previously reported synthesis (Figure 1a). The Z-contrast nature of HAADF-STEM imaging makes this technique an ideal method for identifying heavy atoms (as gold, Z=79) on light supports (carbon, Z=6, in the present case) as evidenced in Figure 1b. Surprisingly, at 80 kV and under very low dose (0.0145 e -A -2 s -1

), the degradation of Au NWs was extremely fast (Video S1-S2). However, at 300 kV, the electron beam damaging could be dynamically followed using a 1.6 frame.s -1 scanning (Video S3-S9). It is well established that higher accelerating voltage reduces radiolysis damage due to lower inelastic scattering probability. However, opposite happens for knock-on damage, the threshold for gold clusters and particles being above 200 and 400 kV respectively. 24,25 As a consequence, though gold is a good conductor, the evolution of these Au NW seems to be governed by radiolysis at low voltage. At 300 kV, atomic columns as long as individual atoms on the side of the wire could be detected as evidenced in Figure 1b. The wire breaking occurred through a continual structural reorganization, with multiple twin boundaries, as long as a thickening of the initial diameter, leading to rough surfaces, as evidenced in Figure 2 and supplementary videos S1.

Contrarily to the theoretical model of Plateau Rayleigh instability, 26 the breakdown involves first an asymetric modification of the apexes. Moreover, prior to disruption, two types of chains were observed exhibiting 3-4 atoms (Video S3-S6 and Figure S1) or single-atom thickness (Video S3, S7-S8, Figure 2a-b andS2). The single-atom chains (SAC) results from the downsizing of the channel from 3-4 atoms down to a single one as evidenced by the successive snapshots in Figure 2a-b extracted from video S3. On the contrary, some multi-atom chains (MAC) exhibit a different behavior and break very suddenly (Figure S3). These two types were previously reported on gold nanowires obtained by HRTEM irradiation of a Au polycrystalline thin film. 27 Due to the constant rotation of the wire under the electron beam, the atomic positions could not always be resolved. We followed two cases of single atom chain (SAC) and multi atom chain (MAC) breaking, for which the wire stays in zone axis. In the case of SAC formation (Figure S4), the wire remained highly crystalline along the <111> direction, with a continuity of the dense plane across the junction. If twin planes are mobile, as previously reported, 28,29 the apexes only rotates of few degrees (~2°) respectively to each other (Figure S4). SAC exhibit an important flexibility as evidenced by the drastic modification of its inclination, likely due to the mobility of the anchoring atoms. 30 The interatomic distance within atom chains was found in the range 0.23 -0.31 nm, with a median at 0.29 ± 0.10 nm, in agreement with the nearest-neighbor distance in gold (0.29 nm). 30 The larger interatomic distance of 0.31 nm, may result from the carbon contamination as previously reported. 31 In the case of MAC (Figure S5), important recrystallization occurred. Contrarily to SAC, the dense planes are highly tilted, resulting in apex direction rotated of 23° respectively to the <111> direction. The position of atoms in the MAC, was analyzed and revealed unambiguously a portion of dense plane (60° angles) with slight positional disorder (Figure S5). The interatomic distance was found in the range 0.24 -0.29. Contrarily to the previous reports, the observed MAC is only one atom thick and do not exhibit helical multi-shell structure. 32 Concomitantly to the chain formation, individual Au atoms were steadily ejected and migrate from the shrinking channel. The trajectory of each atom could be followed, evidencing the formation of small clusters such as dimers or trimers. These clusters were fairly unstable and could dissociate after few seconds to yield isolated atoms. Such metallic clusters have been reported as highly reactive for catalysis, and in the case of gold, towards CO oxidation. 33,34 Combined with the presence of the highly twinned and rough Au wires, 35 enhanced catalytic activities are expected. Formation of quantized atomic channel along the wire length is clearly evidenced during the fragmentation process. These intermediate states were stable enough to be imaged at 300 kV for few seconds. Modification of the wire surface energy, through ligands exchange process, could enable to further stabilize the single-atom thick chains, 8 , 36 and thus open great perspective for the easy-made parallel quantized wires. Appearance of small and highly reactive clusters strengthens the potentiality of ultrathin Au nanowire for catalysis. Though studied under electron beam irradiation, similar breaking mechanism, involving channel shrinking and ejection of reactive atoms may occur under any external stimuli such as current flow or temperature rising, commonly used for Rayleigh instability studies.

ASSOCIATED CONTENT

Supporting Information

Experimental details and videos are available free of charge via the Internet at http://pubs.acs.org.

Table of Content

  23Briefly, a 10 mM HAuCl4 solution was reduced by triisopropylsilane in presence of oleylamine at 40°C for 1h30. The reducing agent and surfactant excess was removed by purification prior to the deposition of Au wires on the carbon coated Cu grid. Electron-beam shower, during 20-25 min, was performed in-situ to prevent contamination during STEM experiments. HAADF-STEM study was performed with an aberration corrected microscope working at 80 -300 kV. The aberration-free probe is taken to have a 25 mrad converge semi-angle, providing a resolution of ~1.3 Å (80 kV) and ~0.8 Å (300 kV). The inner and outer collection angles used in recording the HAADF images were 60 and 200 mrad, respectively. Low electron doses of 0.0145 e -300 kV) were employed.

Figure 1 .

 1 Figure 1. a) TEM and b) HAADF-STEM images of Au nanowires.

Figure 2 .

 2 Figure 2. Snapshots extracted from video S3 of E-beam induced wire breaking which proceeds through the decrease of the channel diameter from a) 3 atoms to b) single atom. c) resulting 7 nm rods surrounded by Au atoms and dimers. As insets, schematic 3D views of a) MATC, b) SATC and c) dimer and isolated atoms. Scale bar : 1 nm. White arrows indicate twin planes

  Rodrigues et al. invoked the difference of crystalline orientation of the atom chains obtained, [110] versus [111] or [100] respectively, to explain such discrepancy. Starting from single-crystalline Au nanowires grown along <111> direction, we did not expect such variety.

AUTHOR INFORMATION

Corresponding Author

Lise-Marie Lacroix. lmlacroi@insa-toulouse.fr. Tel : +33567048833.Fax : +33561559697 Raul Arenal, arenal@unizar.es, Tel : +34976762985

Author Contributions

All the authors contributed equally.

Notes

The authors declare no competing financial interests.

ACKNOWLEDGMENT

The authors acknowledge the financial support of the Labex NEXT, N° 11 LABX 075, and of the European Associated Laboratory LEA TALEM. The scanning transmission electron microscopy studies were conducted at the Laboratorio de MicroscopiasAvanzadas (LMA) at the Instituto de Nanociencia de Aragon (INA) -Universidad de Zaragoza (Spain). The TEM measurements were supported by the European Union Seventh Framework Program under Grant Agreement 312483 -ESTEEM2 (Integrated Infrastructure Initiative -I3).


Javad Amirian

Bingqing Zhang

Francisco Valente Castro

Juan José Baldelomar

Jean-Bernard Hayet

Julien Pettré

L-Cas Pets Vehicle

Kitti Kitti

OpenTraj: Assessing Prediction Complexity in Human Trajectories Datasets

Keywords: Human trajectory prediction, trajectory dataset, motion prediction, trajectory forecasting, dataset assessment, benchmarking Prediction Complexity Agents Intention Interactions Observation Noise Non-Linearity Multi-Modality Recording Platform Crowd Density Sensor Type Duration Scene Layout

Human Trajectory Prediction (HTP) has gained much momentum in the last years and many solutions have been proposed to solve it. Proper benchmarking being a key issue for comparing methods, this paper addresses the question of evaluating how complex is a given dataset with respect to the prediction problem. For assessing a dataset complexity, we define a series of indicators around three concepts: Trajectory predictability; Trajectory regularity; Context complexity. We compare the most common datasets used in HTP in the light of these indicators and discuss what this may imply on benchmarking of HTP algorithms. Our source code is released on Github 1 .

Introduction

Human trajectory prediction (HTP) is a crucial task for many applications, ranging from self-driving cars to social robots, etc. The communities of computer vision, mobile robotics, and crowd dynamics have been noticeably active on this topic. Many outstanding prediction algorithms have been proposed, from physics-based social force models [1][2][3] to data-driven models [START_REF] Alahi | Social lstm: Human trajectory prediction in crowded spaces[END_REF][START_REF] Gupta | Social gan: Socially acceptable trajectories with generative adversarial networks[END_REF][START_REF] Salzmann | Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data[END_REF].

In parallel, efforts have been made towards a proper benchmarking of the existing techniques. This has led to the creation of pedestrians trajectories datasets for this purpose, or to the re-use of datasets initially designed for other purposes, such as benchmarking Multiple Object Tracking algorithms. Most HTP works [3][START_REF] Alahi | Social lstm: Human trajectory prediction in crowded spaces[END_REF][START_REF] Gupta | Social gan: Socially acceptable trajectories with generative adversarial networks[END_REF][START_REF] Amirian | Social ways: Learning multi-modal distributions of pedestrian trajectories with GANs[END_REF] report performance on the sequences of two well-known HTP datasets: the ETH dataset [2] and the UCY dataset [START_REF] Lerner | Crowds by example[END_REF]. The metrics for comparing prediction performance involve the Average Displacement Error (ADE) and the Final Displacement Error (FDE) on standardized prediction tasks. Other datasets have been used in the same way, but performance comparisons are sometimes subject to controversy, and it remains hard to highlight how significant good performance on a particular sequence or dataset means about a prediction algorithm. In this paper, we address the following questions: (1) How to measure the complexity or difficulty of a particular dataset for the prediction task? (2) How do the currently used HTP datasets compare to each other? Can we draw conclusions about the strengths/weaknesses of state of the art algorithms?

Our contributions are two-fold: (1) We propose a series of meaningful and interpretable indicators to assess the complexity behind an HTP dataset, and (2) we analyze some of the most common datasets through these indicators.

In Section 3, we categorize datasets complexity along three axes, trajectories predictability, trajectories regularity, and context complexity. In Section 4, we define indicators quantifying the complexity factors. In Section 5, we apply these indicators on common HTP datasets and we discuss the results in Section 6.

Related work: HTP datasets

Due to the non-rigidness nature of the human body or occlusions, people tracking is a difficult problem and has attracted notable attention. Many video datasets have been designed as benchmarking tools for this purpose and used intensively in HTP. Following the recent progress in autonomous driving, other datasets have emerged, involving more complex scenarios. In this section, we propose a taxonomy of HTP datasets and review some of the most representative ones.

The zoo of HTP datasets: A brief taxonomy

Many intertwined factors explain how some trajectories or datasets are harder to predict than others for HTP algorithms. In Fig. 1, we summarize essential factors behind prediction complexity, as circles; we separate hidden (blue) and controlled (green) factors. Among hidden factors, we emphasize those related to the acquisition (noisy data), to the environment (multi-modality), or to crowdrelated factors (interactions complexity). Some factors can be controlled, such as the recording platform or the choice of the location. To illustrate the variety of setups, snapshots from common HTP datasets are given in Fig. 2. Raw data may be recorded by a single [2] or multiple [START_REF] Caesar | nuScenes: A multimodal dataset for autonomous driving[END_REF] sensors, ranging from monocular cameras [START_REF] Bock | The ind dataset: A drone dataset of naturalistic road user trajectories at german intersections[END_REF][START_REF] Robicquet | Learning social etiquette: Human trajectory understanding in crowded scenes[END_REF][START_REF] Benfold | Guiding visual surveillance by tracking human attention[END_REF] to stereo-cameras, RGB-D cameras, LiDAR, RADARs, or a mix [START_REF] Sun | Scalability in perception for autonomous driving: Waymo open dataset[END_REF][START_REF] Caesar | nuScenes: A multimodal dataset for autonomous driving[END_REF]. Sensors may provide 3D annotations, but most HTP algorithms run on 2D data (the ground plane), and we focus here on 2D analysis.

Annotation is either manual [2,[START_REF] Lerner | Crowds by example[END_REF][START_REF] Geiger | Vision meets robotics: the KITTI dataset[END_REF], semi-automatic [START_REF] Yang | Top-view trajectories: A pedestrian dataset of vehicle-crowd interaction from controlled experiments and crowded campus[END_REF], or fully automatic, using detection algorithms [START_REF] Bock | The ind dataset: A drone dataset of naturalistic road user trajectories at german intersections[END_REF]. In most datasets, the annotations provide the agents' positions in the image. Annotated positions can be projected from image coordinates to world coordinates, given homographies or camera projection matrices. For moving sensors (robots [START_REF] Ess | Depth and appearance for mobile scene analysis[END_REF] or cars [START_REF] Geiger | Vision meets robotics: the KITTI dataset[END_REF][START_REF] Caesar | nuScenes: A multimodal dataset for autonomous driving[END_REF][START_REF] Sun | Scalability in perception for autonomous driving: Waymo open dataset[END_REF]), the data are sensorcentered, but odometry data are provided to get all positions in a common frame.

A short review of common HTP datasets

HTP Datasets from static cameras and drones. The Performance Evaluation of Tracking and Surveillance (PETS) workshops have released several datasets for benchmarking Multiple Object Tracking [START_REF] Leal-Taixé | Motchallenge 2015: Towards a benchmark for multi-target tracking[END_REF] systems. In particular, the 11 sequences of the PETS'2009 dataset [START_REF] Ferryman | Pets2009: Dataset and challenge[END_REF], recorded through 8 monocular cameras, include data from acting pedestrians, with different levels of density, and have been used in HTP benchmarking [START_REF] Sadeghian | Trajnet: Towards a benchmark for human trajectory prediction[END_REF]. The Town-Centre dataset [START_REF] Benfold | Guiding visual surveillance by tracking human attention[END_REF] was also released for visual tracking purposes, with annotations of video footage monitoring a busy town center. It involves around two thousand walking pedestrians with well structured (motion along a street), natural behaviors. The Wild Track dataset [START_REF] Chavdarova | Wildtrack: A multi-camera hd dataset for dense unscripted pedestrian detection[END_REF] was designed for testing person detection in harsh situations (dense crowds) and provides 312 pedestrian trajectories in 400-frame sequences (from 7 views) at 2fps. The EIF dataset [START_REF] Majecka | Statistical models of pedestrian behaviour in the forum[END_REF] gives ∼90k trajectories of persons in a university courtyard, from an overhead camera. The BIWI pedestrian dataset [2] is composed of 2 scenes with hundreds of trajectories of pedestrians engaged in walking activities. The ATC [START_REF] Brscic | Person position and body direction tracking in large public spaces using 3d range sensors[END_REF] dataset contains annotations for 92 days of pedestrian trajectories in a shopping mall, acquired from 49 3D sensors.

The UCY dataset [START_REF] Lerner | Crowds by example[END_REF] provides three scenes with walking/standing activities. Developed for crowd simulation, it exhibits different crowd density levels and a clear flow structure. The Bottleneck dataset [START_REF] Seyfried | New insights into pedestrian flow through bottlenecks[END_REF] also arose from crowd simulation and involved crowd controlled experiments (e.g., through bottlenecks).

VIRAT [START_REF] Oh | A large-scale benchmark dataset for event recognition in surveillance video[END_REF] has been designed for activity recognition. It contains annotated trajectories on 11 distinct scenes, in diverse contexts (parking lot, university campus) and mostly natural behaviors. It generally involves one or two agents and objects. A particular case of activity recognition is the one of sports activities [START_REF] Harmon | Predicting shot making in basketball learnt from adversarial multiagent trajectories[END_REF], for which many data are available through players tracking technology.

The Stanford Drone Dataset (SDD) [START_REF] Robicquet | Learning social etiquette: Human trajectory understanding in crowded scenes[END_REF] is a large scale dataset with 60 sequences in eight scenes, filmed from a still drone. It provides trajectories of ∼19k moving agents in a university campus, with interactions between pedestrians, cyclists, skateboarders, cars, buses. DUT and CITR [START_REF] Yang | Top-view trajectories: A pedestrian dataset of vehicle-crowd interaction from controlled experiments and crowded campus[END_REF] datasets have also been acquired from hovering drones for evaluating inter-personal and carpedestrian interactions. They include, respectively, 1793 and 340 pedestrian trajectories. The inD dataset [START_REF] Bock | The ind dataset: A drone dataset of naturalistic road user trajectories at german intersections[END_REF], acquired with a static drone, contains more than 11K trajectories of road users, mostly motorized agents. The scenarios are oriented to urban mobility, with scenes at roundabouts or road intersections. Ko-PER [START_REF] Strigel | The ko-per intersection laserscanner and video dataset[END_REF] pursues a similar motivation of monitoring spaces shared between cars and non-motorized users. It provides trajectories of pedestrians and vehicles at one road intersection, acquired through laser scans and videos. Similarly, the VRU dataset [START_REF] Bieshaar | Cooperative starting movement detection of cyclists using convolutional neural networks and a boosted stacking ensemble[END_REF] features around 80 cyclists trajectories, recorded at an urban intersection using cameras and LiDARs. The Forking Paths Dataset [START_REF] Liang | The garden of forking paths: Towards multi-future trajectory prediction[END_REF] was created under the Carla 3D simulator, but it uses real trajectories, which are extrapolated by human annotators to simulate multi-modality with different latent goals.

AV datasets. Some datasets offer data collected for training/benchmarking algorithms for autonomous vehicles (AV). They may be more difficult because of the mobile data acquisition and because the trajectories are often shorter. LCAS [START_REF] Yan | Online learning for human classification in 3d lidar-based tracking[END_REF] was acquired from a LiDAR sensor on a mobile robot. KITTI [START_REF] Geiger | Vision meets robotics: the KITTI dataset[END_REF] has been a popular benchmarking source in computer vision and robotics. Its tracking sub-dataset provides 3D annotations (cars/pedestrians) for ∼20 LiDAR and video sequences in urban contexts. AV companies have recently released their datasets, as Waymo [START_REF] Sun | Scalability in perception for autonomous driving: Waymo open dataset[END_REF], with hours of high-resolution sensor data or Argo AI with its Argoverse [START_REF] Chang | Argoverse: 3d tracking and forecasting with rich maps[END_REF] dataset, featuring 3D tracking annotations for 11k tracked objects over 113 small sequences. Nutonomy disclosed its nuScenes dataset [START_REF] Caesar | nuScenes: A multimodal dataset for autonomous driving[END_REF] with 85 annotated scenes in the streets of Miami and Pittsburgh.

Benchmarking through meta-datasets. Meta-datasets have been designed for augmenting the variety of environments and testing the generalization capacities of HTP systems. TrajNet [START_REF] Sadeghian | Trajnet: Towards a benchmark for human trajectory prediction[END_REF] includes ETH, UCY, SDD and PETS; in [START_REF] Becker | An evaluation of trajectory prediction approaches and notes on the trajnet benchmark[END_REF], Becker et al. proposed a comprehensive study over the TrajNet training set, giving tips for designing a good predictor and comparing traditional regression baselines vs. neural-network schemes. Trajnet++ [START_REF] Kothari | Trajnet++: Large-scale trajectory forecasting benchmark[END_REF] proposes a hierarchy of categorization among trajectories to better understand trajectory distributions within datasets. By mid-2020, over 45 solutions have been submitted on Trajnet, with advanced prediction techniques [START_REF] Becker | An evaluation of trajectory prediction approaches and notes on the trajnet benchmark[END_REF][START_REF] Alahi | Social lstm: Human trajectory prediction in crowded spaces[END_REF][START_REF] Gupta | Social gan: Socially acceptable trajectories with generative adversarial networks[END_REF][START_REF] Ellis | Modelling pedestrian trajectory patterns with gaussian processes[END_REF][START_REF] Giuliari | Transformer networks for trajectory forecasting[END_REF], but also Social-Force-based models [1], and variants of linear predictors, that give accuracy levels of 94% of the best model [START_REF] Giuliari | Transformer networks for trajectory forecasting[END_REF]. In this work, we give tools to get a deeper understanding of the intrinsic complexities behind these datasets.

3 Problem description and formulation of needs in HTP

Notations and problem formulation

A trajectory dataset is referred to as X. We assume that it is made of N a trajectories of distinct agents. To be as fair as possible in our comparisons, we mainly reason in terms of absolute time-stamps, even though the acquisition frequency may vary. Within X, the full trajectory of the i-th agent (i ∈ [1, N a ]) is denoted by T i , its starting time as τ i , its duration as δ i . For t ∈ [τ i , τ i + δ i ], we refer to the state of agent i at t as x i t . We observe x i t only for a finite subset of timestamps (at camera acquisition times). The frames are defined as the set of observations at those times and are denoted by F t . Each frame contains K t agents samples.

The state x i t includes the 2D position p i t in a Cartesian system in meter. It is often obtained from images and mapped to a world frame; the velocity v i t , in m/s, can be estimated by finite differences or filtering.

To compare trajectories, following a common practice in HTP, we split all the original trajectories into N t trajlets with a common duration ∆ = 4.8s. HTP uses trajlets of ∆ obs seconds as observations and the next ∆ pred seconds as the prediction targets. Hereafter, the set of distinct trajectories of duration ∆ obtained this way are referred to as X k where k ∈ [1, N t ] covers the trajlets (with potentially repetitions of the same agent). Typically, N t ≫ N a . Each trajlet may be seen as an observed part and its corresponding target is referred to as X k + . In the following, we use functions operating at different levels, with different writing conventions. Trajectory-level functions F (X), with capital letters, act on trajlets X. Sometimes, we consider the values of F at specific time values t, at we denote the functions as F t (X). Frame-level functions F(F) act on frames F.

Datasets complexity

We define three families of indicators over trajectory datasets that allow us to compare them and identify what makes them more "difficult" than other.

Predictability. A dataset can be analyzed through how easily individual trajectories can be predicted given the rest of the dataset, independently from the predictor. Low predictability on the trajlet distribution p(X) makes forecasting systems struggle with multi-modal predictive distributions, e.g., at crossroads. In that case, stochastic forecasting methods may be better than deterministic ones, as the latter typically average over the outputs seen in the training data.

Trajectory (ir)regularity. Another dataset characterization is through geometrical and physical properties of the trajectories, to reflect irregularities or deviations to "simple" models. We will use speeds, accelerations for that purpose.

Context complexity. Some indicators evaluate the complexity of the context, i.e., external factors that influence the course of individual trajectories. Typically, crowd density has a strong impact on the difficulty of HTP.

These indicators operate at different levels and may be correlated. For example, complex scenes or high crowdedness levels may lead to geometric irregularities in the trajectories and to lower predictability levels. Finally, even though it is common to combine datasets, our analysis is focused on individual datasets.

Numerical Assessment of a HTP Dataset complexity

Based on the elements from Section 3, we propose several indicators for assessing a dataset difficulty, most of the kind F (X k ), defined at the level of trajlets X k .

Overall description of the set of trajlets

To explore the distribution p(T) in a dataset, we first consider the distributions of pedestrian positions at a timestep t. We parametrize each trajlet by fitting a cubic spline p k (t) with t ∈ [0, 4.8]. For t ∈ [0, 4.8], we get 50 time samples

S(t) = {p k (t), 1 ≤ k ≤ N t }

and analyze S(t) through clustering and entropy:

-Number of Clusters M t (X): We fit a Gaussian Mixture Model (GMM) to our sample set using Expectation Maximization and select the number of clusters with the Bayesian Information Criterion [START_REF] Claeskens | Cambridge Series in Statistical and Probabilistic Mathematics[END_REF]. -Entropy H t (X): We get a kernel density estimation of S(t) (see below in Section 4.2) and use the obtained probabilities to estimate the entropy.

High entropy means that many data points do not occur frequently, while low entropy means that most data points are "predictable". Similarly, a large number of clusters would require a more complex predictive model. Both indicators give us an understanding of how homogeneous through time are all the trajectories in the dataset.

Evaluating datasets trajlet-wise predictability

To quantify the trajectory predictability, we use the conditional entropy of the predicted part of the trajectory, given its observed part. Some authors [START_REF] Li | Assessing spatiotemporal predictability of lbsn: A case study of three foursquare datasets[END_REF] have used alternatively the maximum of the corresponding density. For a trajectory X k ∪ X k + , we define the conditional entropy conditioned to the observed X k as

H(X k ) = -E X+ [log p(X + |X k )]. (1)
We use kernel density estimation with the whole dataset X (N t trajectories) to estimate it. We have N obs observed points during the first ∆ obs seconds (trajlet X k ) and N pred points to predict during the last ∆ pred seconds (trajlet X + k ). We define a Gaussian kernel K h over the sum of Euclidean distances between the consecutive points along two trajectories X and X ′ with N points each (in R 2N ):

K h,N (X, X ′ ) = 1 (2πh 2 ) N exp(- 1 2h 2 X -X ′ 2 ), ( 2
)
where h is a common bandwidth factor for all the dimensions. We get an approximate conditional density as the ratio of the two kernel density estimates

p(X + |X k ) ≈ 1 Nt Nt l=1 K h,N obs +N pred (X ∪ X k + , X l ∪ X l + ) 1 Nt Nt l=1 K h,N obs (X k -, X l -) . ( 3
) Since K h,N obs +N pred (X k ∪X k + , X l ∪X l + ) = K h,N obs (X, X l )K h,N pred (X + , X l + )
, we can express the distribution of Eq. 3 as the following mixture of Gaussian:

p(X+|X k ) ≈ N t l=1 ω l (X k )K h,N pred (X+, X l + ) with ω l (X k ) = K h,N obs (X k , X l ) N t l=1 K h,N obs (X k , X l ) . ( 4
)
For a trajlet X k , we estimate H(X k ) by sampling M samples X

+ from Eq. 4:

H(X k ) ≈ - 1 M M m=1 log( Nt l=1 ω l (X k )K(X (m)
+ , X l + )).

(5)

Evaluating trajectories regularity

In this section, we define geometric and statistical indicators evaluating how regular individual trajectories X k in a dataset may be.

Motion properties.

A first series of indicators are obtained through speed distributions, where speed is defined as: s(x t ) = v t . At the level of a trajectory X k , we evaluate the mean and the largest deviation of speeds along the trajectory

S avg (X k ) = average t∈[τ k ,τ k +δ k ] (s(x t )) (6)
S rg (X k ) = max t∈[τ k ,τ k +δ k ] (s(x t )) - min t∈[τ k ,τ k +δ k ] (s(x t )). (7)
The higher the speed, the larger the displacements and the more uncertain the target whereabouts. Also, speed variations can reflect on high-level properties such as people activity in the environment or the complexity of this environment.

Regularity is evaluated through accelerations a(x t ) ≈ 1 dt [s(x t+dt ) -s(x t )]. It can reflect the interactions of an agent with its environment according to social-force model [1]: agents typically keep their preferred speed while there is no reason to change it. High accelerations appear when an agent avoids collision or joins a group. We consider the average and maximal accelerations along

X k A avg (X k ) = average t∈[τ k ,τ k +δ k ] (|a(x t )|); A max (X k ) = max t∈[τ k ,τ k +δ k ] (|a(x t )|). ( 8
)
Non-linearity of trajectories. Path efficiency is defined as the ratio of the distance between the trajectory endpoints over the trajectory length:

F (X k ) = p τ k +δ k -p τ k τ k +δ k t=τ k dl . (9)
The higher its value, the closer the path is to a straight line, so we would expect that the prediction task will be "easier" for high values of F (X k ).

Another indicator is the average angular deviation from a linear motion. To estimate it, we align all trajlets by translating them to the origin of coordinate system and rotating them such that the first velocity is aligned with the x axis:

Xk = R(-∡v k 0 ) -p k 0 X k 1 T . ( 10
)
Then the deviation of a trajectory X k at t and its average value are defined as:

D t (X k ) = ∡ Xk t and D(X k ) = average t∈[τ k ,τ k +δ k ] (D t (X k )). ( 11
)

Evaluating the context complexity

The data acquisition context may impact HTP in different ways. It may ease the prediction by introducing correlations: With groups, it can be easier to predict one's motion from the other group members. In general, social interactions result into adjustments that may be generate non-linearities (and lower predictability).

Collision avoidance is the most basic type of interaction. Higher density resulting into more interactions, this aspect is also evaluated by the density metrics below. However, high-density crowds may ease the prediction (e.g., laminar flow of people). To reflect the intensity of collision avoidance-based interactions, we use the distance of closest approach (DCA) [START_REF] Olivier | Collision avoidance between two walkers: Role-dependent strategies[END_REF] at t, for a pair of agents (i, j):

dca(t, i, j) = x i t -x j t 2 -(max(0, (v i t -v j t ) T (x i t -x j t ) v i t -v j t )) 2 , (12)
and for a trajlet X k (relative to an agent i k ), we consider the overall minimum

C(X k ) = min t∈[τ k ,τ k +δ k ] min j dca(t, i k , j). (13)
In [START_REF] Karamouzas | Universal power law governing pedestrian interactions[END_REF], the authors suggest that time-to-collision (TTC) is strongly correlated with trajectory adjustments. The TTC for a pair of agents i, j, modeled as disks of radius R, for which a collision will occur when keeping their velocity, is

τ (t, i, j) = 1 v i t -v j t 2 [δ ij t -(δ ij t ) 2 -v i t -v j t 2 ( x i t -x j t 2 -4R 2 )] ( 14
)
where

δ ij t = (v i t -v j t ) T (x i t -x j t )
. In [START_REF] Karamouzas | Universal power law governing pedestrian interactions[END_REF], the authors also proposed quantifying the interaction strength between pedestrians as an energy function of τ :

E(τ ) = k τ 2 e -τ τ + , (15)
with k a scaling factor and τ + an upper bound for TTC. Like [START_REF] Karamouzas | Universal power law governing pedestrian interactions[END_REF], we estimate the actual TTC probability density between pedestrians (from Eq. 14) over the probability density that would arise without interaction (using the timescrambling approach of [START_REF] Karamouzas | Universal power law governing pedestrian interactions[END_REF]). Then we estimate E(τ ) with Eq. 15. As the range of well-defined values for τ may be small, we group the data into 0.2s intervals and use t-tests to find out the lower bound τ -when two consecutive bins are significantly different (p < 0.05). The upper bound τ + is fixed as 3s. TTC and energy interaction are extended for trajlets (only if there exists future collision):

T (X k ) = min t∈[τ k ,τ k +δ k ] min j τ (t, i k , j) and E(X k ) = E(T (X k )). ( 16
)
Density & distance measures. For a frame F t , the Global Density is defined as the number of agents per unit area D(F t ) = Kt A(X) , with K t the number of agents present at t and A(X) the spatial extent of X, evaluated from the extreme x, y values. The Local Density measures the density in a neighborhood. Plaue et al. [START_REF] Plaue | Trajectory extraction and density analysis of intersecting pedestrian flows from video recordings[END_REF] infer it with a nearest-neighbour kernel estimator. For a point x t ,

ρ(x t ) = 1 2π Kt i=1 1 (λd i t ) 2 exp - x i t -x t 2 2(λd i t ) 2 , ( 17
)
with d i t = min j =i x i tx j t the distance from i to its nearest neighbor and λ > 0 a smoothing parameter. ρ is used to evaluate a trajlet-wise local density indicator

L(X k ) = max t∈[τ k ,τ k +δ k ] ρ(x i k t ). ( 18
)

Experiments

In this section, we analyze some common HTP datasets in the light of the indicators presented in the previous section. In Table 1, we give statistics (location, number of agents, duration. . . ) for the datasets we have chosen to evaluate. We gather the most commonly used in HTP evaluation (ETH, UCY, SDD in particular) and datasets coming from a variety of modalities (static cameras, drones, autonomous vehicles. . . ), to include different species from the zoo of Section 2.1.

For those including very distinct sub-sequences, e.g., ETH, UCY, SDD, inD, and Bottleneck (also denoted by BN in the figures), we split them into their constituting sequences. Also, note that we have focused only on pedestrians (no cyclist nor cars). We also ruled out any dataset containing less than 100 trajectories (e.g., UCY Arxiepiskopi or PETS).

To analyze a dataset X, we apply systematically the following preprocessing Table 1: General statistics of assessed datasets. The columns present the type of location where the data is collected, the acquisition means, number of annotated pedestrians, the -rounded-duration (in minute or hour), the total duration of all trajectories, number of trajlets, and percent of non-static trajlets, respectively. Fig. 3: Entropy H t (X) and number of clusters M t (X), as described in Section 4.1, at different progression rates t, for a dataset X. Each dot corresponds to one t.

Overall description of the set of trajlets. For the indicators of Section 4.2, we have chosen h = 0.5m for the Gaussian in the kernel-based density estimation; the number of samples used to evaluate the entropy is M = 30; the maximal number of clusters when clustering unconditional or conditional trajectories distributions is 21. In Fig. 3, we plot the distributions of the overall entropy and number of clusters, at different progression rates along the dataset trajectories. Without surprise, higher entropy values are observed for the less structured datasets (without main directed flows) such as SDD or inD. The number of clusters follows a similar trend, indicating possible multi-modality.

(

+ 7
X k ), S rg (X k ), A avg (X k ), A max (X k ).
From top to bottom: speed means and variations, mean and max. accelerations. distributions: large lower tails indicate high proportions of easy-to-predict trajlets, while large upper tails indicate high proportions of hard-to-predict trajlets.

Regularity indicators. In Fig. 5, we depict the distributions of the regularity indicators S avg (X k ), S rg (X k ), A avg (X k ), A max (X k ) from Eqs. 7 and 8. Speed averages are generally centered around 1 and 1.5m/s. Disparities among datasets appear with speed variations and average accelerations: ETH or UCY Zara sequences do not exhibit large speed variations, e.g. compared to Wild Track. In Fig. 6a, we depict the path efficiency F (X k ) fro Eq. 9, and we observe that ETH, UCY paths tend to be straighter. More complex paths appear in Bottleneck, due to the interactions within the crowd, or in SDD-deathCircle, EIF, due to the environment complexity. In Fig. 6b, deviations D t (X k ) are displayed for different progression rates along the trajectories, and reflect similar trends.

Context complexity indicators.

For estimating the TTC in Eq. 14, we set R = 0.3m, and for the interaction energy of Eq. 15, we set k = 1. The local density of Eq. 18 uses λ = 1. In Fig. 7, we display the collision avoidance-related indicators (TTC, DCA and interaction energy) described in Section 4.4, while in Fig. 8, we depict the density-related indicators. Most samples have low interaction energy, but interesting interaction levels are visible Zara, InD. The global density for most datasets stays less than 0.1 p/m 2 while in InD(1&2), Edinburgh and SDD (Coupa & Bookstore), it is even less than 0.02. Bottleneck (1d & 2d) are significantly high density scenarios. For this reason why we depict them separately. Most natural trajectory datasets have a local density about 0 -4p/m 2 while such number is higher (2 -4p/m 2 ) in Bottleneck. With both density indicators, a dataset such as WildTrack has a high global density and low local density, indicating a relatively sparse occupation. Conversely, low global density and high local density in Ind suggests the pedestrians are more clustered. This observation is also reflected in the interaction and entropy indicators as well.

Discussion

Among the findings from the previous Section, Fig. 4 shows that the predictability among most datasets varies in mostly the same ranges. Regarding the motion properties of the datasets (see Fig. 5), another finding is pedestrians' average speed, which, in most cases, varies from 1.0 to 1.5 m/s. However, this is not the case for Bottleneck dataset, because the high density of the crowd does not allow the pedestrians to move with a 'normal' speed. In the SDD dataset, we observe multiple pedestrians strolling the campus. As shown in Fig. 6b these low-speed motions are usually associated with high deviation from linear motion, though part of this effect is related to the complexity of the scene layout. Also, for most of the datasets, the speed variation of trajlets remains almost below 0.5. This is not a true hypothesis for LCas and WildTrack. As one would expect, the distribution of mean/max acceleration of trajlets is highly correlated with speed variations. In Fig. 6a we see that almost all values are bigger than 90%. For Bottleneck we see this phenomenon, where by increasing the crowd density and decreasing crowd speed, the paths become less efficient.

Conclusions & Future Work

We have presented in this work a series of indicators for gaining insight into the intrinsic complexity of Human Trajectory Prediction datasets. These indicators cover concepts such as trajectory predictability and regularity, and complexity in the level of inter-pedestrian interactions. In light of these indicators, datasets commonly used in HTP exhibit very different characteristics. In particular, it may explain why predictions techniques that do not use explicit modeling of social interactions, and consider trajectories as independent processes, may be rather successful on datasets where e.g., most trajectories have low collision energy; it may also indicate that some of the more recent datasets with higher levels of density and interaction between agents could provide more reliable information on the quality of the prediction algorithm. Finally, the trajlet-wise analysis presented here opens the door to some evolution in benchmarking processes, as we could evaluate scores by re-weighting the target trajlets in the function of the presented indicators.

Fig. 1 :

 1 Fig. 1: Taxonomy of trajectories datasets for Human Trajectory Prediction.

  (a) ETH-Univ (b) ETH-Hotel (c) UCY-Zara (d) UCY-Students (e) UCY-Arx.

Fig. 2 :

 2 Fig. 2: Sample snapshots from a few common HTP datasets.

  iv ET H-Ho te l UC Y-Za ra UC Y-Un iv SD Dco up a SD Dbo ok st or e SD Dde at hC irc le In D-1 In D-2 KI TT I LC as -M in er va W ild Tr ac k Ed in bu rg h BN -

Fig. 4 :Fig. 5 :

 45 Fig. 4: Conditional Entropies H(X k ).

Fig. 6 : 7 :

 67 Fig. 6: Regularity indicators: Path efficiency and deviation from linear motion.

Fig. 8 :

 8 Fig. 8: Density indicators: On the top, global density (one data point for each frame); on the bottom, local density (one data point for each trajlet).

https://github.com/crowdbotp/OpenTraj

Acknowledgements

This research is supported by the CrowdBot H2020 EU Project http://crowdbot.eu/ and by the Intel Probabilistic Computing initiative. The work done by Francisco Valente Castro was sponsored using an MSc Scholarship given by CONACYT with the following scholar registry number 1000188.