Robert Dominko

Alen Vizintin

Guiliana Aquilanti

Lorenzo Stievano

Maria Joseph Helen

Anji Reddy Munnangi

Maximilian Fichtner

Iztok Arcon

Polysulfides Formation in Different Electrolytes from the Perspective of X-ray Absorption Spectroscopy

Li-S batteries are promising energy storage technology for the future, however there two major problems remained which need to be solved before successful commercialization. Capacity fading due to polysulfide shuttle and corrosion of lithium metal are directly connected with the type and quantity of electrolyte used in the cells. Several recent works show dependence of the electrochemical behavior of Li-S batteries on type of the electrolyte. In this work we compare and discuss a discharge mechanism of sulfur conversion in three different electrolytes based on measurements with sulfur K-edge XAS. The sulfur conversion mechanism in the ether based electrolytes, the most studied type of solvents in the Li-S batteries, which are enabling high solubility of polysulfides are compared with the fluorinated ether based electrolytes with a reduced polysulfide solubility and in carbonate based electrolytes with the sulfur confined into a ultramicroporous carbon. In all three cases the sulfur reduction proceeds through polysulfide intermediate phases with a difference on the type polysulfides detected at different steps of discharge.

Electrification of road transport has increased the pressure on materials' scientists to improve performance of the current Li-ion batteries and to develop new advanced high energy battery technologies. 1 Among several post lithium-ion technologies, the lithium sulfur (Li-S) batteries are recognized as the most promising for commercialization in the near future. A combination of lithium and sulfur in the electrochemical cell corresponds to the theoretical energy density of 2600 Wh/kg, while the maximum practically accessible energy density is predicted to be close to 600 Wh/kg. 2,3 This is much higher compared to Li-ion batteries and besides that, sulfur is inexpensive and naturally abundant. Nevertheless, problems related to the solubility of polysulfides in the electrolyte and the related redox shuttle phenomena cause short cycle life, potential safety problems, poor cycling efficiency, and relative fast self-discharge. Additional problems of Li-S batteries are very low electronic conductivity of the both end members in the discharge and charge process (i.e. sulfur and Li 2 S) and extensive corrosion of the metal lithium anode.

Different directions how to improve Li-S battery cycle life have been explored, like synthesis of the optimized porous host matrices with active sites for polysulfide anchoring, 4,5 design and optimization of separators which can effectively suppress polysulfide cross communication between electrodes 6,7 and protection of lithium by artificial SEI 8 or by additives. 9 While most of the research was performed in the binary mixture of solvents using alkyl ethers (glymes) and heterocyclic acetyl (dioxolane), less attention has been paid to the development of new electrolytes for Li-S batteries. 10 Reasons for that are nested in the requirements which should be fulfilled in the development of new formulations. First of all, electrolytes used in the electrochemical cells must have good ionic conductivity and electrochemical stability in a wide potential window. It is desired that they are non-toxic and environmentally friendly, non-reactive with active species, non-corrosive with cell housing and they should enable the sufficient dissolution of the supporting salt.

z E-mail: Robert.Dominko@ki.si Electrolytes used in Li-S batteries typically enable the dissolution of polysulfides which act as redox mediator species in the conversion reaction and significantly improve the kinetics of the electrochemical reaction. With that, solubility of sulfur and polysulfides has a direct influence on the discharge mechanism due to differences in dissolution and solvation of polysulfides. It is well accepted that the electrochemical mechanism of sulfur reduction in Li-S batteries is a stepwise process where a different equilibrium states between sulfur, polysulfides and Li 2 S exists. 11 In general, the discharge mechanism can be described by using two to three equilibrium states i.e. equilibrium between sulfur and long chain polysulfides dissolved into the electrolyte, equilibrium between long and short chain polysulfides and equilibrium between polysulfides and precipitated Li 2 S (or even nonsoluble Li 2 S 2 ). 12 The potential of the high voltage plateau, typically found in the starting part of the electrochemical curve, is controlled by the dissolution of sulfur into the electrolyte. Thus, one can expect that the equilibrium potential during high voltage plateau will vary depends on the type of electrolyte used in the electrochemical characterization. The low voltage plateau corresponds to the formation of non-soluble Li 2 S, which starts to precipitates when the solubility of Li 2 S is exceeded. The understanding of the sulfur conversion mechanism is crucial to further improve electrochemical behavior of Li-S batteries.

Numerous analytical techniques have been applied for the characterization of Li-S batteries in order to better understand the mechanism of sulfur conversion within the Li-S battery. Different groups [13][14][15] showed that operando XRD can be used as an excellent tool to characterize the formation and consumption of a crystalline active species for instance Li 2 S and different allotropes of sulfur. Early studies were not concentrated on the detection of polysulfides, since they were considered as amorphous species detected in the background of the XRD spectra. Recently, J. Conder et al. 16 showed the direct observation of lithium polysulfides adsorbed on the surface of a glass fiber separator in the Li-S battery during operation by means of operando X-ray diffraction (XRD). Although synchrotron XRD enables detection of polysulfides, 16,17 much better understanding can be obtained by using sulfur K-edge X-ray absorption spectroscopy (XAS). With a difference to XRD, XAS is sensitive to coordination of the selected atom and can differentiate between different local coordination and oxidation states. This technique is currently used by several groups [18][19][20] and it provides the most comprehensive picture of reaction mechanism(s) inside the battery since it can precisely differentiate sulfur species in the cathode during discharge and charge process. In most of the reported works, analysis of the near edge structure of the X-ray absorption spectrum (XANES) is used for discussion of the mechanism and only a few papers discuss also extended X-ray absorption fine structure (EXAFS) part of the XAS spectra. The reason for that are sulfate groups in the salt (i.e. LiTFSI) or in the solvent (i.e. sulfonyl functional groups) composing the electrolyte. Nevertheless, different sulfur compounds (sulfur, polysulfides, sulfur radicals, Li 2 S, molecules with C-S bonds, sulfite, and sulfate) formed in the electrochemical cell can be identified by characteristic energy position of the sulfur edge and pre-edge resonances in the range between 2468 eV for sulfur radicals 21 to 2478 eV for sulfate 22 (corresponding to sulfur with the highest oxidation state). As a reference database for polysulfides, different stoichiometric mixtures between sulfur and lithium were synthesized and measured by XAS and 7 Li NMR. 19 Measured spectra of sodium and lithium polysulfides represent a reference database that can be used for analysis of spectra obtained during battery operation. All intermediate states of polysulfides can be identified by the ratio between the pre-peak at 2470.2 eV (characteristic pre-peak for polysulfides) and the sulfur peak at 2472 eV typical of elemental sulfur, since the relative intensity of the pre-peak is proportional to the relative amount of Li in Li 2 S y . Experimental results 19 show that obtained spectra can be in general described with a contribution from the long chain polysulfides (Li 2 S y with y ≥ 6) where the intensity of pre-peak is relatively weak compared to sulfur peak and with mid-to short chain polysulfides (Li 2 S y with y ≤ 4), where the ratio between pre-peak and sulfur peak is smaller but still not equal to that expected for Li 2 S 2 . Early theoretical work predicted slight variations in the position of the lithium polysulfide pre-edge peak in the adsorption spectra. 23 By using Resonant Inelastic X-ray Scattering (RIXS) no shifts were observed for the set of reference samples, neither within operando measurement. 24 Nevertheless, information gathered from operando XAS measurements can serve as a reliable indicator of changes within the Li-S battery upon operation.

Other types of spectroscopic analytical techniques are predominantly based on higher wavelengths such as Raman spectroscopy, 25 infrared spectroscopy 26 and UV-Vis spectroscopy. 27,28 The latter has demonstrated to be a useful technique for qualitative and quantitative detection of polysulfides in the separator (detection of polysulfides which diffuse/migrate out from the cathode composite). In several of our works 22,27,28 we highlighted the sensitivity of the UV-VIS analytical technique, showing that it offers hints about the polysulfide formation mechanism which occurs in the bulk of the cathode. However, this spectroscopic tool for the study of polysulfide mechanism is based on the prediction that polysulfides are soluble into the electrolyte. Besides that, typically a high ratio between electrolyte and sulfur is used in the experiment. This technique is less powerful in the current trends where electrolytes with a reduced solubility of polysulfides are predicted to be a key for successful commercialization of Li-S batteries. Similarly, reduced applicability of UV-Vis spectroscopy is also in the carbonate based electrolytes where sulfur has to be confined in narrow (nanometer size) pores, well passivated on the surface to protect direct reaction between carbonate solvents and polysulfides.

In this contribution we discuss Li-S battery mechanisms in different electrolyte systems by means of XAS. The cases of sulfur reduction in ether based electrolytes enabling high solubility of polysulfides, in fluorinated ether based electrolytes with a reduced polysulfide solubility and in carbonate based electrolytes with the sulfur confined into a ultramicroporous carbon to avoid the polysulfide solubility are discussed based on measurements with sulfur K-edge XAS.

Experimental

X-ray absorption spectra were recorded at the sulfur K-edge in fluorescence-detection mode at the XAFS beamline of synchrotron Elettra (Basovizza, Trieste). 29 A Si (111) double crystal monochromator was used with a resolution of about 0.4 eV at 2.5 keV. Higherorder harmonics were effectively eliminated by using a double flat silica mirror placed at a grazing angle of 8 mrad. The intensity of the monochromatic X-ray beam before the sample was measured by a 30 cm-long ionization chamber detector, filled with a mixture of 30 mbar of N 2 and 1970 mbar of He. The fluorescence signal was detected with a SDD detector. The spectra were measured within the interval of -150 to 730 eV relative to the S K-edge (2472 eV). In the XANES region, equidistant energy steps of 0.2 eV were used, whereas for the higher energy region, equidistant k steps of 0.03 Å -1 were adopted, with an integration time of 5 s per point, resulting in the total time for one spectrum of 65 min. The experimental setup of the beamline for the operando XAS experiments is described in details elsewhere. 30 The first spectrum was measured on the asprepared battery, and a series of 27 following spectra were collected in operando mode during battery discharge at C/30 rate. The exact energy calibration was established with an absorption measurement on the native sulfur, where the maximum of the pre-edge peak was set to 2472.0 eV. 19 In order to limit self-absorption effects in sulfur K-edge XANES spectra, the cathode was prepared in the initial state with a relatively low amount of sulfur. The setup of the measurement, cell and battery configuration were same as in our previous operando Li-S battery study, presented in more detail in the Supporting Information of the paper. 12 The analysis of the XANES spectra was performed with the IFEFFIT program package ATHENA. 31 No self-absorption correction was applied to the measured XAS spectra, because the composition of the cathode (ratio between sulfur, lithium, electrolyte, and CB) could not be determined precisely enough.

The electrodes for the operando XAS measurements were prepared by mixing the carbon sulfur composite with conductive additive and binder as explained in detail in our previous publications. 12,19 operando XANES measurements were performed in the two electrode set up with as prepared cathode and lithium metal (Aldrich) which were separated by glass fiber separator wetted with different types of electrolytes. Sulfur conversion in the mesoporous carbon was studied in the 1 M LiTDI TEGDME:DOL 12 electrolyte and in the 1 M LiTFSI TTFE:DOL (1,2-(1,1,2,2-tetrafluoroethoxy)ethane and 1,3-dioxolane) 32 electrolyte. Carbonate based electrolyte (1 M LiPF 6 in EC:DMC) were used in the combination with the ultramicroporous carbon. Galvanostatic cycling was performed in a modified Swagelok cell with thin beryllium window. 12 Cell were discharged at a current density of C/30 (55.7 mA g -1 ) to 1.5 V by using Biologic SP 200 galvanostat/potentiostat.

Results and Discussion

Sulfur reduction proceeds through disproportionation reactions of different polysulfide species which can be in the solid state form or dissolved in the electrolyte. Understanding their formation and conversion to Li 2 S requires studies under real-time conditions since with that we can avoid the artefacts from post treatment and we can get insight to the real working mechanism(s). 17 XAS provides details on oxidation state (XANES part of the spectrum) and atomic distribution (EXAFS part of the spectrum). Contributions of different sulfur compounds in the cathode (sulfur, Li-polysulfides with different chain lengths, Li 2 S) in the XANES spectra can be identified by characteristic energy positions and shape of the sulfur absorption edge and pre-edge resonances. 12 The energy position of the pre-edge resonance in elemental sulfur is at 2472 eV. Lithium polysulfides, containing a reduced form of sulfur with oxidation state S 2-, can be identified by a characteristic pre-peak at 2470.2 eV together with a sulfur peak at 2472 eV. The XANES spectrum of crystalline Li 2 S exhibits the first maximum at sulfur absorption edge at 2473 eV. An additional spectral signature was described in the literature 21 with pre-edge resonances at 2468 eV, corresponding to S 3 •-radical. Using a combination of spectral components shown in Figure 1 one can describe mechanism of sulfur conversion in different electrolytes by using a linear combination analysis (LCA). Some other contributions can be identified like formation of radicals, 21 formation of H 2 S or C-S bond due to nucleophilic reaction of polysulfides with solvent molecules.

Mechanism in the ether based electrolytes.-Alkyl ethers and heterocyclic acetyl represent a family of solvents stable towards polysulfides and with different polysulfide solubilities. This family of solvents is frequently applied in the battery research and it offers high conversion of sulfur into Li 2 S due to soluble intermediate species.

The solubility of polysulfides depends on the type of solvents as well as on the concentrations of salt(s) and additives. [33][34][35] It is well accepted that polysulfides act as redox mediator species improving rate capability of Li-S batteries. Improved sulfur conversion is related to the solubility of sulfur in the heterocyclic acetal. 34 However, high solubility of polysulfides leads to polysulfide shuttle phenomena due to diffusion/migration of polysulfides out from the cathode composite. Additionally, soluble polysulfides increase the viscosity of the electrolyte and reduce the lithium mobility in the electrolyte. More recently, fluorinated ether solvents showed improved Li-S battery behavior with more stable capacity retention. [36][37][38] This is attributed to the reduced solubility of polysulfides which also enables Li-S battery cycling with reduced amount of electrolyte (<5μL/mg S ) since the physicochemical properties of the electrolyte are stable during cycling.

At this step the open question is how different polysulfide solubility influences the reaction mechanism of sulfur conversion. We studied those mechanisms by different analytical techniques; however, in this work we will focus on the differences observed by using sulfur Kedge XAS. The mechanism of sulfur conversion in the mixture of alkyl ethers and heterocyclic acetyl (solvent mixture of TEGDME and DOL) was studied in the presence of 1 M LiTDI salt and such a system enabled us to measure and analyze complete XAS spectra (XANES and EXAFS part). 12 All sulfur K-edge XANES spectra measured in operando mode can be thoroughly described as a linear combination of three components (sulfur, polysulfides and Li 2 S). Two different rates of sulfur conversion into polysulfides were detected in the starting part of the electrochemical reduction, which can be correlated to the different voltage plateaus observed in the electrochemical curve (Figure 2a). At the beginning of the reduction process, during the high voltage plateau, we observed fast consumption of elemental sulfur and increase of polysulfide concentration. The conversion rate of sulfur changed once the potential started to decrease from high voltage plateau towards to the low voltage plateau. The maximum ratio of polysulfides within the cathode composite was detected at the beginning of the low voltage plateau, where precipitation of Li 2 S starts. During the low voltage plateau, three phases coexist, with slow reduction of the remaining elemental sulfur and with linear increase of Li 2 S quantity accompanied by the linear decrease of polysulfides quantity. At the end of the reduction process sulfur is completely consumed and we observed approximately 70% of Li 2 S in co-existence with polysulfides. Interestingly, all operando spectra could be fitted with only one polysulfide standard that suggests presence of an equilibrium state of polysulfides within the electrochemical cell.

A sulfur-free electrolyte enabled the possibility to study the transformation of the local environment of sulfur by quantitative EXAFS analysis. It was performed for the S-S first coordination shell and the variation of the average coordination number of S-S component is shown in Figure 2b. A detailed explanations of the procedures used for EXAFS analysis are given in our previous work. 12 The variation of the average coordination number of S-S component shows gradual decrease of coordination number during high voltage plateau. At the end of the high voltage plateau, the average coordination number is decreased from 2 to 1.6(2) which corresponds to the average coordination number typical of S 6 2-. At the beginning of the low voltage plateau, a steeper decrease of average coordination number is observed with a rapid decrease to values between S 4 2-and S 2 2-. EXAFS analysis has confirmed the gradual shortening of polysulfide chain length as the average coordination number of the S-S component decreases during the discharge process.

Sulfur and polysulfide solubility in the fluorinated based ethers are several orders of magnitude lower compared to linear glymes. 32 That can be explained by different potential of the high voltage plateau typically found at 2.2 V versus lithium reference and not at 2.4 V as observed in the electrolyte solutions with linear glymes. Nevertheless, a detailed analysis of Li-S battery discharge mechanism in 1 M LiTFSI TTFE:DOL electrolyte by using S K-edge XANES shows co-existence of polysulfides with elemental sulfur through entire discharge process. 32 Precipitation of Li 2 S starts at the beginning of the low voltage plateau and this equilibrium is not influenced by choice of electrolyte. 32 In the LCF analysis of the operando XANES spectra measured on batteries with ether electrolyte used, only a single reference polysulfide spectrum was sufficient to describe polysulfide contribution during the entire process of discharge. That is pointing out on the existence of equilibrium of different polysulfides during the reduction process. This equilibrium depends on the different parameters and for each working system, reference spectra for polysulfides needs to be extracted out from in the operando XANES spectra. Mechanism in the carbonate solvent based electrolytes.-Ultramicroporous carbons with a pore diameter smaller than 0.7 nm gained considerable interest as host matrix for sulfur in Li-S batteries since such sulfur confinement shows excellent electrochemical stability even in the carbonate based solvents. [39][40][41] It is well accepted that soluble polysulfides react with the carbonate based electrolytes via a nucleophilic addition or substitution reactions 42 leading to extensive degradation of electrolyte. In this work ultramicroporous carbon derived from coconut shell containing predominately micropores with a diameter of 0.5 nm has been used as host matrix for sulfur. In a previous work some of us proved excellent electrochemical properties in the 1 M LiPF 6 EC:DMC electrolyte. Cycling stability over 400 cycles was demonstrated with excellent Coulombic efficiency. That was possible due to the effective separation of sulfur confined in ultramicropores and due to formation of SEI on the surface of coconut shell derived carbon in the first cycle. A short plateau in the first cycle close to 2.4 V corresponds to the formation of SEI on the surface of carbon/sulfur composite which effectively separate electrolyte and carbon sulfur composite. Once a protective SEI is formed it enables lithium diffusion and prevents diffusion of polysulfides into the electrolyte; consequently no soluble polysulfides can be detected during cycling. The confined sulfur without a direct contact with the electrolyte shows a different reduction mechanism compared to that observed in ether based solvents, where the starting reduction potential depends on the sulfur solubility. In the previous work 39 where a detailed analysis performed by ex situ XPS sputter profiling on electrodes removed from batteries stopped at different states of charge showed the direct conversion of sulfur to Li 2 S 2 /Li 2 S. In this work we evaluate the reduction mechanism by operando sulfur K-edge XAS. operando XANES spectra measured during battery discharge are shown in Figure 3. Principle component analysis (PCA) 31 of the whole set of spectra shows that a linear combination of four different components is required to completely describe each XANES spectrum in the series. Components with a physical meaning are sulfur, long and short chain Li-polysulfides Li 2 S y and lithium sulfide Li 2 S (Figure 4). The reference Li 2 S XANES spectrum was obtained in our previous operando Li-S battery study, 12 while the remaining three reference XANES profiles are extracted directly from the set of operando XANES spectra of this battery. The elemental sulfur XANES spectrum is obtained from the spectrum of the as-prepared battery, which contained only elemental sulfur. No difference was observed between bulk sulfur and sulfur infiltrated into the ultramicropores of carbon derived from coconut shells. The two Lipolysulfides spectra, (Li 2 S y -long chain, and Li 2 S y -short chain) are representative spectra for a mixture of long chain polysulfides (S 8 2-, S 6

2-), and a mixture of short chain polysulfides (S 4 2-, S 2 2-), respectively. All operando XANES spectra measured in other intermediate states of the battery during discharge can be completely described as linear combination of these four reference spectra. The quality of the linear combination fits (LCF) is demonstrated in Figure 5. In this way the evolution of the relative amounts of the sulfur species (sulfur, two different types of Li-polysulfides Li 2 S y , and lithium sulfide Li 2 S) in the cathode during discharge are determined with high precision, with an uncertainty below 1% for all components. In contrast to our previous work, 12,19 two different polysulfide standards are needed in LCF analysis, as indicated by PCA analysis, to completely describe XANES spectra of all intermediate states of the battery. That points out a different mechanism of sulfur reduction in the ultramicropores.

End of discharge at 1.5 V contains 29% of Li 2 S, while the number of reacted Li suggests almost complete conversion of sulfur to Li 2 S. This difference is due to the irreversible consumption of Li for the formation of a protective SEI film on the surface of the ultramicroporous carbon. Efficient SEI formation is required to prevent formation of H 2 S due to the presence of HF in the electrolytes with LiPF 6 salt. Formation of SEI can be associated to charge consumed during plateau close to 2.4 V, namely XANES spectra during this period of charge are almost unchanged. Once when voltage drops to 1.9 V vs. lithium reference a fast increase of long chain-Li 2 S y was observed. A composition of Li 1.27 S represents a verge, where contribution from long chain-Li 2 S y starts to decrease accompanying with a formation of short chain-Li 2 S y and Li 2 S. After this point, the relative amount of Li 2 S is linearly increasing and it reaches 29% at the end of the discharge. Increase of short chain-Li 2 S y polysulfides is not linear which points out potential shortening of polysulfides, however the end of discharge composition of the electrode is a mixture between Li 2 S (29%) and Li 2 S 2 (71%) as it was already suggested by XPS. 39 This final composition determined from the XAS measurements was expected also from the charge consumed during the reduction, where a monotonous voltage drop from 1.9 to 1.5 V mainly corresponds to the reduction of sulfur to Li 2 S and Li 2 S 2 and only very minor part to additional passivation. A more complete conversion to Li 2 S is probably hindered by space confinement within the pores with a dimension less than 0.7 nm.

Mechanistic changes influenced by different electrolytes.-XAS

measurements confirmed the presence of polysulfides in all three studied electrolytes with some differences. Sulfur reduction proceeds in all three electrolytes through polysulfide intermediates with a difference that sulfur almost completely disappears in the first part of discharge in the 1 M LiTDI TEGDME:DOL electrolyte. On the contrary, in electrolytes characterized by a sparing solubility for polysulfides or in carbonate based electrolytes, only part of the sulfur is converted into polysulfides during high voltage plateau, the rest being continuously consumed by the end of discharge. Differences in the reaction mechanism in the first part of the reduction process are therefore strictly related to the solubility of sulfur and polysulfides in the electrolyte: with higher solubility, we observed faster sulfur conversion into polysulfides. Interestingly also higher solubility of sulfur and polysulfides determines the potential of the high voltage plateau, which is at 2.4 V in the TEGDME:DOL electrolyte and at 2.2 V in TTFE:DOL electrolyte, where solubility of sulfur and polysulfides is a few orders of magnitude lower. In carbonate based electrolytes due to the effective separation of sulfur confined in ultramicropores and due to protective nature of SEI, there is no solubility of sulfur and polysulfides in the electrolyte and the high voltage plateau is absent in the electrochemical reduction curve. Nevertheless, independent on whether the electrolyte composition enables the presence of soluble polysulfides or not, the Li-S battery discharge mechanism proceeds through polysulfides, which can be in the solid state form or dissolved. A conversion process of the polysulfides from long chain to short chain is observed in all tested electrolytes, even though in ether based electrolytes we were not able to clearly distinguish between different chain lengths. That was made possible by using ultramicroporous carbon, where the clear presence of long chain polysulfides at the beginning of discharge was observed, followed by the presence of short chain polysulfides (most likely Li 2 S 2 composition) in the second part of the discharge process. The equilibrium potential of Li 2 S formation is at similar voltage in both ether based electrolytes used in this study while in the confined space, Li 2 S formation starts at much lower voltage. That can be attributed to the limited space for volumetric changes that are required for formation of Li 2 S. The latter is also the most probable reason for relative low ratio of Li 2 S formed at the end of discharge compared to end of discharge in the TEGDME:DOL electrolyte.

Slight difference in the sulfur conversion mechanism between three investigated electrolytes can be used for better understanding of advantages and drawbacks of each electrolyte. Mixture of alkyl ethers and heterocyclic acetyl (TEGDME and DOL) enables high solubility of sulfur and polysulfides. That influences physical properties of electrolyte (i.e. viscosity, lithium mobility . . . ) and consequently higher amounts of electrolyte are required to achieve high capacity and long cycle life, but electrolyte quantity higher then 3 mL per 1 g of sulfur is not acceptable for commercial purposes.

Sulfur and polysulfide solubility in the sparingly soluble solvents (i.e. fluorinated ethers) is for several orders of magnitude lower compared to glymes, what allows decrease of electrolyte quantity and low solubility of polysulfides still has a function of redox mediators. The weakest point of using fluorinated ethers is increased resistance film on lithium surface, but due to the possibility of having working Li-S battery with very low amount of electrolyte this type of electrolyte is of particular interest for commercially available cells.

Carbonate solvents are known as much better passivation agents for lithium compared to ethers and since sulfur infiltrated ultra microporous carbons can be used in the combination carbonate based electrolyte, this direction offers new unexplored field of Li-S batteries. Major difficulties are lower voltage due to difference in the mechanism and lower capacity due to volumetric constrains inside pores, while benefits of that should be taken from Li-ion battery technology.

Conclusions

In the ether based electrolytes the sulfur conversion and formation of polysulfides occurs predominantly during the high voltage plateau. The maximum ratio of polysulfides is detected at the beginning of the low voltage plateau, where the precipitation of Li 2 S begins. The high voltage potential in the ether based electrolytes is determined by the solubility of sulfur and polysulfides. During the low voltage plateau a coexistence of three different oxidation states of sulfur is typically found, while the potential of low voltage plateau depends on the equilibrium between polysulfides and Li 2 S. End of discharge, sulfur is converted to polysulfides which are in the coexistence with Li 2 S as a major product of sulfur conversion. The fluorinated based electrolytes enable much lower solubility of sulfur and polysulfides in the electrolyte, the mechanism of sulfur conversion is not affected by decreased solubility. Although solubility of polysulfides is several orders of magnitude lower, polysulfides equilibrium with elemental sulfur is dominating high voltage plateau and similar as in the ether based electrolytes, the precipitation of Li 2 S starts with a low voltage plateau.

Reversible sulfur reduction into Li 2 S in the carbonate based electrolytes is possible only with confinement of sulfur within the ultra micropores. Nevertheless the conversion of sulfur proceeds through polysulfides, however a clear distinction between long chain polysulfides at the beginning of discharge and short chain polysulfides at the end of discharge have been observed. The end discharge product is a mixture of Li 2 S 2 and Li 2 S in the ratio 2:1. This is different compared to ether based electrolytes where a predominant end discharge phase is Li 2 S in coexistence with mixture of different polysulfides.

Figure 1 .

 1 Figure1. Sulfur K-edge XANES spectra of sulfur compounds present in the Li-S battery: sulfur, Li 2 S y polysulfides (long and short chains), and lithium sulfide Li 2 S. The two polysulfides spectra, (Li 2 S y -long chains, and Li 2 S y -short chains) are representative spectra for a mixture of long chains polysulfides, and a mixture of short chains polysulfides, respectively. The vertical dotted lines at 2470.2 and 2472.0 eV mark characteristic pre-peaks in the Li 2 S y and sulfur reference spectra.

Figure 2 .

 2 Figure 2. a) Electrochemical curve obtained during discharge in the 1 M LiTDI TEGDME:DOL electrolyte and the relative amounts of three sulfur compounds (sulfur, polysulfides and Li 2 S) determined with a linear combination fit of XANES spectra; b) variation of the average S coordination number during the first discharge as determined by analysis of the EXAFS part of spectra measured in the 1 M LiTDI TEGDME:DOL electrolyte. Reprinted with permission from Ref. 12. Copyright (2015) American Chemical Society.

Figure 3 .

 3 Figure 3. In operando sulfur K-edge XANES spectra measured on as prepared Li-S battery with sulfur infiltrated ultramicroporous carbon within 1 M LiPF 6 EC:DEC electrolyte at discharge C/30.

Figure 4 .

 4 Figure 4. Electrochemical curve obtained during discharging at a current rate of C/30 and the relative amount of the four sulfur compounds (sulfur, long and short chains Li 2 S y , and Li 2 S) during discharge, determined with a linear combination fit of S K-edge XANES spectra measured at different operando states of the battery using four reference XANES profiles (sulfur, Li 2 S y with long and with short chains, and Li 2 S).

Figure 5 .

 5 Figure 5. Sulfur K-edge XANES spectra of a Li-S battery measured in operando in two intermediate states during discharge: a) 556 min and b) 1371 min. Dots: experiment; dashed magenta line: best fit with linear combination four reference XANES profiles (sulfur, long chain polysulfides, short chain polysulfides, and Li 2 S), plotted below.

Acknowledgments

Access to synchrotron radiation facilities of Elettra (project 20140487) is acknowledged. We would like to thank Luca Olivi of Elettra for expert advice on beamline operation. This research has received funding from the Slovenian Research Agency (research core funding No. P1-0112 and P2-0393), and from the European Union's Horizon 2020 research and innovation program under Grant Agreement No. 666221 (HELIS).


Abdelghani Maddi
email: abdelghani.maddi@hceres.fr

David Sapinho
email: david.sapinho@hceres.fr

Does Open Access Really Increase Impact? A Large-Scale Randomized Analysis

Introduction

The Open Access Citation Advantage (OACA) has been a major topic of discussion in the literature over the past twenty years, generally assuming that a better accessibility fosters research impact. In fact, this belief is not so obvious, with contradictory findings in the studies dealing with the OACA, that have often proven tough to compare, depending on many factors such as disciplines, OA statuses taken into account, publications types as well as the method and database used. Overall, studies that found the existence of OACA were more common in social sciences [START_REF] Mikki | Filter bubbles in interdisciplinary research: a case study on climate and society[END_REF][START_REF] Abbasi | Citation indicators' comparison of LIS open access and subscription publications based on Scopus[END_REF][START_REF] Valderrama-Zurián | On the bibliometric nature of a foreseeable relationship: open access and education[END_REF], medical and health sciences [START_REF] Hudson | Citation rates for open-access versus pay-to-access articles in clinical journals specializing in paediatric medicine[END_REF][START_REF] O'kelly | Predatory publishing or a lack of peer review transparency?-a contemporary analysis of indexed open and non-open access articles in paediatric urology[END_REF][START_REF] Miller | Open access availability of anatomy papers presented at meetings of the American and British Associations of Clinical Anatomists[END_REF], and natural sciences [START_REF] Lin | Non-Open Access and Its Adverse Impact on International Journal of Molecular Sciences[END_REF][START_REF] Wang | The open access advantage considering citation, article usage and social media attention[END_REF][START_REF] Clements | Open access articles receive more citations in hybrid marine ecology journals[END_REF]. OACA is less important (and in several disciplines non-existent) in Physical Sciences and Engineering [START_REF] Archambault | Research impact of paywalled versus open access papers[END_REF]. In addition, there are some studies (less numerous) that concluded on the nonexistence of OACA in social sciences [START_REF] Zhang | The Effect of Open Access on Citation Impact: A Comparison Study Based on Web Citation Analysis[END_REF], medical and health sciences [START_REF] Tonia | If I tweet will you cite? The effect of social media exposure of articles on downloads and citations[END_REF][START_REF] Mimouni | Citation Rates of Open-Access and Non-Open Access Clinical Studies in Leading Ophthalmology Journals[END_REF][START_REF] Nazim | Do Open Access Journals Have Greater Citation Impact? A Study of Journals in Health and Medical Sciences in India[END_REF], and natural sciences [START_REF] Campos | Insularity and citation behavior of scientific articles in young fields: the case of ethnobiology[END_REF][START_REF] Peidu | An Empirical Examination of Citation in Life Science[END_REF]. On a large sample of the Web of Science database, (Dorta-González, González-Betancor and Dorta-González, 2017) concluded that there is no OACA in all disciplines. Recently, a review analysis on the topic identified 134 publications dealing with OACA (Langham-Putrow, Bakker and Riegelman, 2021). Applying the EBL critical appraisal method to analyze the risks of bias based on factors like sample size, data collection or study design, the authors emphasized that most of these studies (131about 98%) present a high risk. Two of the three publications with a low risk of bias relate to medical and natural science research [START_REF] Tonia | If I tweet will you cite? The effect of social media exposure of articles on downloads and citations[END_REF][START_REF] Nelson | Citations, mandates, and money: Author motivations to publish in chemistry hybrid open access journals[END_REF], and the third [START_REF] Sotudeh | The citation advantage of author-pays model: the case of Springer and Elsevier OA journals[END_REF] used relatively old data (2007)(2008)(2009)(2010)(2011). Moreover, none of these articles used randomisation techniques/control group. As several known confounding factors have limited the scope of the results, a key issue to address the question rigorously would be to figure out a way to "isolate" the OA effect. OA publications should thus be compared to a counterfactual sample of publications with the only difference is to be published in subscription-based journals. In this paper, we propose a method to constitute a control group to isolate the OACA effect. By definition, OACA analysis answers the following question: "what would be the impact of an OA publication if it had been published in a subscription-based journal?" The answer this question is only possible if this publication is compared to its counterparts, with respect to characteristics that can influence the impact, such as the discipline, the type of journals, or other known characteristics (e.g. the number of authors, whether it has received a specific funding or the number of funders). We limit ourselves to publications in gold OA, but distinguishing between publications in full OA journals and hybrid journals. The reason why we did not take into account the green and bronze routes is the difficulty of conducting an analysis of the OACA for this type of publications. Thus, for the green route, it is not possible to identify precisely the moment when each publication was dumped in an open archive when for the bronze route; the durability of the OA status is highly questionable.

Data

We extracted the publications data from the French OST in-house database. It includes five indexes of the WoS available from Clarivate Analytics (Science Citation Index Expanded (SCIE), Social Sciences Citation Index (SSCI), Arts & Humanities Citation Index (AHCI), Conference Proceedings Citation Index (CPCI-SSH) and Conference Proceedings Citation Index (CPCI-S)), and corresponds to WoS content indexed through the end of March 2021. The study focuses on three types of publications: articles, reviews and conference proceedings.

Method

Sample selection procedure and weighting

Figure 1 describes the sample selection procedure.

OA sample

We selected all the documents published with a Gold OA status from 2010 to 2020 by distinguishing those published in a full OA and those in hybrid journals, representing respectively 2,458,378 and 1,024,430 publications.

Control sample

We used the raking ratio method [START_REF] Deville | Calibration Estimators in Survey Sampling[END_REF][START_REF] Deville | Generalized raking procedures in survey sampling[END_REF][START_REF] Daza | Raking weights with R[END_REF] to ensure comparability between the two samples. The method comprises the construction of a counterfactual sample similar to the sample of interest, except for the analyzed parameter, which is in our case the OA status. The control sample is thus obtained by finding all the non-OA publications, qualified as doubles, that match with OA publications on a set of features identified in the literature as having an effect on the citation impact [START_REF] Judge | What Causes a Management Article to be Cited-Article, Author, or Journal?[END_REF][START_REF] Yan | The funding factor: a cross-disciplinary examination of the association between research funding and citation impact[END_REF][START_REF] Waltman | Use of the journal impact factor for assessing individual articles: Statistically flawed or not?[END_REF][START_REF] Maddi | Article processing charges, altmetrics and citation impact: Is there an economic rationale?[END_REF]. These features are characterised as follows:

-the publication year (11 classes : 2010 to 2020), -the discipline (OST classification in 27 ERC panels), -the journal impact (5 classes : <0.8, [0.8 , 1. 2[, [1.2 , 1.8[, [1.8 , 2.2[, >=2.2), for the calculation method see: [START_REF] Maddi | Article processing charges, altmetrics and citation impact: Is there an economic rationale?[END_REF]. -the number of countries of contributors, based on WoS addresses information On this basis, we categorized each OA publication in one among 242,924 different clusters. From the same clusters, we then identify 12,088,681 double candidates among which 10,310,342 and 11,533,001 are respectively eligible for full OA and hybrid OA publications.

Raking weights

We finally calculated the raking weights for the publications of the control sample in order to ensure that both samples have a comparable distribution with respect to the characteristics identified.

In our calculations, we distinguish publications in full OA journals and those in hybrid journals.

OACA calculation

We define the Open Access Citation Advantage (OACA) as the percentage of advantage in the citation score that could be attributed to Open Access (OA). We used the following formula:

𝑂𝐴𝐶𝐴 = 𝑀𝑁𝐶𝑆 𝑂𝐴 -𝑀𝑁𝐶𝑆 𝑁𝑜𝑛-𝑂𝐴 𝑀𝑁𝐶𝑆 𝑁𝑜𝑛-𝑂𝐴 * 100
Where MNCSOA is the Mean Normalized Citation Score [START_REF] Leydesdorff | Remaining problems with the "New Crown Indicator[END_REF] in the OA group, and MNCSNon-OA, the same score in the Non-OA counterfactual group.

Results

Figure 2 describes the evolution of OACA of publications between 2010 and 2020, respectively in fully OA journals and hybrid journals. The first finding is that OACA highly depends on the type of journal. Indeed, for publications in fully OA journals, there is rather a citation disadvantage, which average about -20 to -15% over the period 2010-2020, when compared to the uncontrolled group (all non-OA WoS publications). This disadvantage reduced significantly until 2016 to approach zero before decreasing to reach the value at the start of the period (i.e. -20%). Contrasting this are the results for OA publications in hybrid journals. Figure 2 shows a high OACA at the start of the period, which receive, on average, 60% more citations than non-OA publications to reach 70% in 2016. Since that year, however, the downward trend is comparable to that observed for full OA journals. Even if OACA persists, it steadily declines throughout the end of the period. This figure also highlights the effect of using a non-OA control group similar to that of OA publications represented by the blue curves. Whatever the type of journal, OACA tends to be lower with this group as reference, suggesting that the use of non-adjusted control groups overestimates OACA.

Figure 3 declines the OACA by discipline, comparing two periods, for each type of journal. In fully OA journals (Figure 3a), there is rather a citation disadvantage in most of disciplines, except for a few in Life Sciences. These are precisely "Immunity, Infection and Immunotherapy" (LS6), "Molecules of Life: Biological Mechanisms, Structures and Functions" (LS1), "Integrative Biology: From Genes and Genomes to Systems" (LS2) and "Cellular, Developmental and Regenerative Biology" (LS3). The OACA in these disciplines varies between 10 and 20% (after raking), in the second period . The comparison between the two periods shows, in general, no major variations. Some exceptions should be noted, like PE2 for which the OACA dropped significantly from 50% in 2010-12 to 5% in 2018-20. On the flipside, the OACA exists in the majority of disciplines in hybrid journals. LS6 is the discipline with the highest OACA (83% in the second period), followed by LS2 (73%). The OACA is however much lower (even negative in some cases) in Physical Sciences and Engineering. This is notably the case for Materials Engineering (PE11), Physical And Analytical Chemical Sciences (PE4) and Synthetic Chemistry And Materials (PE5) with OACA rates of -12, -13 and -14% respectively. -there are many disciplinary disparities concerning the evolution of the OACA over time, -the greatest falls in the OACA are observed in the PE disciplines, -on the contrary the OACA tends to increase in the SH disciplines, -the OACA is relatively stable in the LS disciplines, except for LS1 and LS2 where a significant decrease is observed (and L8 and LS9 to a lesser extent).

Conclusion

Through this paper, we have analysed the question of Open Access Citation Advantage (OACA) in the Web of Science database. To do so, we compared citation impact (MNCS) of 2,458,378 publications in fully OA journals to that (weighted MNCS) of a control group of non-OA publications (#10,310,842). Similarly, we did the same exercise for OA publications in hybrid journals (#1,024,430) and their control group (#11,533,001), over the period 2010-2020. The results showed that there is no OACA for publications in fully OA journals, and that there is rather a disadvantage. Conversely, the OACA seems to be a reality in hybrid journals, suggesting that a better accessibility in this context tends to improve the visibility of publications.

Another important finding of our study is that 2016 appears to be a turning point on how an easier accessibility to publications influences their visibility, with a strong and regular downward trend from this year onwards, regardless of the type of journal. Nevertheless, the analysis by discipline showed great discrepancies, both in the initial level of OACA and its evolution over the time. It could still be noticed that the most important falls are for disciplines where the initial level was the higher. Finally, the effect of using an adjusted control group is clearly demonstrated, even if it has no effect in the trends observed.

Discussion

The lack of OACA for publications in fully OA journals is to be expected, as a great proportion of OA journals are newly created and less attractive to high-impact senior researchers. However, the citation disadvantage of these publications has largely diminished over time to disappear in 2016, as these journals gain a readership over the years and become more visible. OACA of publications in hybrid journals is also to be expected. These journals are well established in the publishing market and highly visible to scientific communities. For this type of journal, the fact that an article is in OA has a positive impact on citations more than articles that are in the same journals but with limited access to subscribers (hence the OACA). These results therefore suggest that the OACA depends on the type of journal; i.e. to hope to benefit from an OACA, it would first be necessary to choose a journal visible to the community (highimpact journal). In this case, the OA statue will act as a catalyst that would increase the citations received.

Another striking result of this paper is the fall of the OACA from 2016. The citation advantage fell from 70% to 9% between 2016 and 2020 (for publications in hybrid journals). We wonder if this fall is linked to the increase in the notoriety of pirate sites (eg Sci-Hub) from 2016 (especially after the ranking in 2016 of Alexandra Elbakyan -creator of Sci-Hub -in the top 10 influential figures of Nature (Nature's 10, 2016, p. 10)). In other words, the democratization of pirate sites instantly cancels the positive effect of OA publication insofar as the question of access to scientific content no longer arises. This is a hypothesis that we will investigate in future studies.

  (5 classes : 1,2,3,4 and 5 or more) -the number of funding received, based on WoS acknowledgment information (5 classes : 1,2,3,4 and 5 or more) -the presence of an ERC funding (2 classes : Yes or No) -the presence of at least one European (UE27) address (2 classes : Yes or No) -the presence of a patent citation (2 classes : Yes or No)

Figure 1 :

 1 Figure 1: Method of constructing the counterfactual sample of OA publications.

Figure 2 :

 2 OACA all disciplines combined a) Fully OA journals b) Hybrid journals

Figure 3 :

 3 Figure 3: OACA (after raking) between 2010-12 and 2018-20, by discipline* a) Full OA journals


Jie Liu

Bogdan Nicolae
email: bnicolae@anl.gov

Dong Li

Justin M Wozniak

Tekin Bicer
email: tbicer@anl.gov

Zhengchun Liu
email: zhengchun.liu@anl.gov

Ian Foster
email: foster@anl.gov

Large Scale Caching and Streaming of Training Data for Online Deep Learning

Keywords: deep learning, distributed caching, data pipelines, reuse or training data

The training of deep neural network models on large data remains a difficult problem, despite progress towards scalable techniques. In particular, there is a mismatch between the random but predetermined order in which AI flows select training samples and the streaming I/O patterns for which traditional HPC data storage (e.g., parallel file systems) are designed. In addition, as more data are obtained, it is feasible neither simply to train learning models incrementally, due to catastrophic forgetting (i.e., bias towards new samples), nor to train frequently from scratch, due to prohibitive time and/or resource constraints. In this paper, we study data management techniques that combine caching and streaming with rehearsal support in order to enable efficient access to training samples in both offline training and continual learning. We revisit state-of-art streaming approaches based on data pipelines that transparently handle prefetching, caching, shuffling, and data augmentation, and discuss the challenges and opportunities that arise when combining these methods with data-parallel training techniques. We also report on preliminary experiments that evaluate the I/O overheads involved in accessing the training samples from a parallel file system (PFS) under several concurrency scenarios, highlighting the impact of the PFS on the design of the data pipelines.

Introduction

Deep learning (DL) applications are rapidly gaining traction both in industry and scientific computing, driven by the accumulation of massive data. In science, for example, instruments that collect data at GB/s and 100+ TB/day present a wide range of learning opportunities. We thus see significant interest in deploying DL on high-performance computing (HPC) systems in order to enable rapid learning in various scientific areas, such as fusion energy science, computational fluid dynamics, lattice quantum chromodynamics, virtual drug response prediction, and cancer research.

Various approaches for training DL models on massive data have been proposed: coarse-grain parallelization on multiple nodes using data-parallel [START_REF] Goodfellow | Deep Learning (Adaptive Computation and Machine Learning series)[END_REF], model-parallel [START_REF] Dean | Large scale distributed deep networks[END_REF], pipeline-parallel [START_REF] Narayanan | PipeDream: Generalized pipeline parallelism for DNN training[END_REF], and hybrid techniques; fine-grain parallelization on many-core architectures by constructing and scheduling execution graphs at the tensor level; and low-level optimizations of operators [START_REF] Ivanov | Data movement is all you need: A case study on optimizing transformers[END_REF] and communication primitives [START_REF] Chu | NV-group: Link-efficient reduction for distributed deep learning on modern dense GPU systems[END_REF]. Most such work is targeted at alleviating the computational overhead needed to perform the forward and backward passes in DL training, as well as the communication costs associated with synchronizing subtasks across devices and nodes.

However, as computation and communication become highly optimized, another bottleneck begins to emerge that has seen comparatively less attention: the I/O operations needed to read training data and feed them to the computational pipeline. This bottleneck is particularly prominent at scale in large data centers and supercomputers that typically feature a large number of compute nodes connected to a storage repository of relatively limited I/O bandwidth (e.g., a parallel file system). Indeed, the limited I/O bandwidth of parallel file systems is known to cause bottlenecks in general [START_REF] Reed | Exascale computing and big data[END_REF], especially under concurrent access.

This I/O bottleneck is further exacerbated in the case of DL applications by a mismatch between the preferred access pattern of parallel file systems (large non-overlapping reads from a few files) and the access pattern of DL training (small, pseudo-random reads from many files). In fact, popular reference datasets such as ImageNet [START_REF] You | ImageNet training in minutes[END_REF], used to evaluate image classification DL models, feature millions of small images, each of size 100 KB or less. Retrieving these files is expensive because each file access incurs a double latency cost, both at the level of metadata (to obtain the location of a dataset) and data (I/O at offset within dataset). Unfortunately, parallel file systems are primarily optimized to deliver high throughput at the cost of high latency, which explains the mismatch regarding the preferred access pattern.

Once extracted from a remote repository, the raw input data is only the beginning of a complex data ingestion pipeline that involves decoding of training samples as tensors, augmentations (e.g., stretching or shifting the color spectrum of images), pseudo-random shuffling, grouping of samples into batches, etc. The data ingestion pipeline is typically implemented in asynchronous fashion though producer-consumer buffers that involve caching at each stage. Using this approach, each DL training iteration overlaps with the data ingestion, thereby hiding the overhead of I/O operations and successive transformations in the pipeline, which reduces the overall runtime.

In addition to data ingestion pipelines, there is an increasing for other data abstractions. For example, continual learning [START_REF] Parisi | Continual lifelong learning with neural networks: A review[END_REF] involves updating a DL model in near-real time, by linking the data ingestion pipeline directly to a data stream rather than a storage repository. However, it is not possible to simply incrementally train the DL model with the new data, because this would lead to catastrophic forgetting: a bias of the DL models towards the most recent training samples at the expense of older ones, which effectively causes them to reinforce new patterns and forget old ones. A similar issue is also exhibited by reinforcement learning (RL) [START_REF] Lillicrap | Continuous control with deep reinforcement learning[END_REF] approaches, because the samples are generated by sequential exploration of the states of an environment, which may lead to an excessive reinforcement of recent states at the expense of older ones. A common approach to address the problem of catastrophic forgetting is rehearsal, i.e., mixing new training samples with previously encountered, representative training samples in order to enable incremental training without bias. Thus, there is a need to transparently store, retrieve and mix representative training samples with regular training samples in the data ingestion pipeline.

In this context, there is a need for flexible data runtimes that are capable of addressing a variety of data ingestion patterns (irregular fine-grain I/O from storage repositories, direct streaming from data sources, rehearsal, augmentations and other post-processing). In addition, since the data ingestion pipeline is asynchronous, flexibility is also needed with respect to resource allocation, such as to dynamically respond to changes in the requirements of computational and I/O resources. This applies both to the interference between the data ingestion pipeline and the training, as well as to the interference between the different stages of the data ingestion pipeline itself. In this paper, we study the challenges, trade-offs and opportunities in the design of flexible data runtimes for both offline and continual DL model training We summarize our contributions as follows: ples to DL models in several configurations (directly from a parallel file system or cached on local storage) using TensorFlow data pipelines for a representative DL application. Based on these preliminary results, we discuss the observed bottlenecks and their impact on the identified opportunities (Section 5).

Related work

The challenging nature of the pseudo-random I/O access patterns generated by DL training on small training samples is acknowledged by several studies [START_REF] Chowdhury | I/O characterization and performance evaluation of BeeGFS for deep learning[END_REF][START_REF] Murray | tf.data: A machine learning data processing framework[END_REF]. One approach to addressing this problem is to reorganize the training data. For example, the lightweight Lightning Memory-Mapped Database (LMDB) maps content directly into memory (thus taking advantage of OS-level I/O optimizations) and uses B+trees to index it (thus reducing metadata overheads). However, Pumma et al. [START_REF] Pumma | Scalable deep learning via I/O analysis and optimization[END_REF] have shown that this solution does not mitigate the problem sufficiently, as I/O overheads still dominate training (up to 90%) even for only a small degree of parallelism. Other approaches such as FanStore [START_REF] Zhang | Efficient I/O for neural network training with compressed data[END_REF] provide a global cache layer on node-local burst buffers in a compressed format, allowing POSIX-compliant file access to the compressed data in user space. Further optimizations explore prefetching with perfect knowledge of future I/O based on fixing the seeds of pseudo-random number generators [START_REF] Dryden | Clairvoyant prefetching for distributed machine learning i/o[END_REF]. Such approaches are limited in their applicability to accelerating low-level I/O operations only.

Overlapping computations with I/O operations by prefetching and caching training samples in the background is another popular direction [START_REF] Nicolae | DataStates: Towards lightweight data models for deep learning[END_REF][START_REF] Nicolae | Ve-loC: Towards high performance adaptive asynchronous checkpointing at large scale[END_REF]. However, generic I/O optimizations are not enough for this purpose, because specific operations such as shuffling the training samples with various ordering guarantees and/or complex transformations to augment the data are also needed before the data can be passed to training pipelines. To this end, abstractions such as data loader [START_REF] Yang | Accelerating data loading in deep neural network training[END_REF], data pipelines [START_REF] Murray | tf.data: A machine learning data processing framework[END_REF] and DALI [START_REF]NVIDIA Data Loading Library[END_REF] are becoming increasingly popular for both PyTorch and Ten-sorFlow ecosystems. However, current implementations of such approaches provide only limited support for multi-node parallelism.

Several approaches deal with catastrophic forgetting [START_REF] Parisi | Continual lifelong learning with neural networks: A review[END_REF]. Inspired by cognitive and neural science theories, rehearsal methods date back three decades [START_REF] Robins | Catastrophic forgetting in neural networks: The role of rehearsal mechanisms[END_REF]. The most straightforward approach is to augment each minibatch with previously streamed training samples, called exemplars, that were selected randomly to be persisted to the storage repository for the purpose of rehearsal. To improve I/O performance, classification models such as iCARL [START_REF] Rebuffi | iCaRL: Incremental classifier and representation learning[END_REF] employ a limited memory buffer that stores a fixed number of exemplars for each class. However, this approach may lead to a decrease of accuracy when the number of classes increases, because exemplars from each class need to be dropped in order to make them all fit into the memory buffer. Approaches such as Naive Incremental Learning (NIL) [START_REF] Muñoz | Incremental learning model inspired in rehearsal for deep convolutional networks[END_REF] simply use a random replacement policy without considering classes. An alternative to storing exemplars is to train a generative adversarial network (GAN) from the stream in order to produce fake exemplars on demand that capture the patterns needed for rehearsal [START_REF] Liu | Generative feature replay for class-incremental learning[END_REF]. However, such approaches suffer from high overheads to train and keep the GAN up to date. Other approaches are based on regularization (i.e., estimating the relevance of the DNN parameters and penalizing those that show significant change from one task to another) or architectural changes that grow subnetworks dynamically [START_REF] Rusu | Progressive neural networks[END_REF]. However, rehearsal remains one of the most widely used techniques for mitigating catastrophic forgetting, and thus the problem of caching and storage of exemplars remains important.

In summary, state-of-art approaches are not sufficiently optimized to stream efficiently from a parallel file system or directly from an external data source into the DL training pipelines at scale on multiple nodes, nor do they offer support for rehearsal. To the best of our knowledge, we are the first to consider this combined problem in a distributed setting.

Data Pipelines

Modern DL runtimes such as TensorFlow are beginning to acknowledge the importance of optimizing the entire input data lifecycle, from reading the training samples all the way to feeding them to the training pipeline. To this end, abstractions such as data pipelines [START_REF] Murray | tf.data: A machine learning data processing framework[END_REF] are becoming increasingly popular. In this section, we revisit data pipelines, which will form the basis of our study.

In a nutshell, data pipelines abstract input data as a potentially infinite sequence of elements that can be either tensors or composite types (tuples, nested datasets, etc.). The elements can be accessed by means of an iterator, which offers sequential access through a simple API call: get_next(). The state of an iterator can be checkpointed to a file by using save() and restore().

Since most DL training algorithms rely on mini-batch SGD (stochastic gradient descent), training samples are not accessed individually, but rather in groups called mini-batches that are returned by get_next(). The path from reading the input data all the way to generating the mini-batches creates a complex producer-consumer pipeline, as illustrated in Figure 1.

The entry point in the producer-consumer pipeline is a flexible record enumeration abstraction that is responsible for exposing new elements in a metadata queue. For example, in the case of a POSIX filesystem, the metadata queue will hold file names. These metadata are consumed by the readers, who are responsible for fetching the content of the elements. A framework like TensorFlow provides multiple types of readers, to support various data sources. The readers can optionally pass the content to a pre-processing task responsible for data augmentation, which can be defined by the user by using a map() transformation. From there, the elements are enqueued in a shuffle queue, from which the prefetcher extracts a pseudo-random subset of the elements that it then assembles into mini-batches. Finally, the mini-batches arrive in the batch queue, from where they are consumed by get_next().

Data pipelines are highly customizable via composability. A basic pipeline creates a one-to-one correspondence between the elements of the dataset and the iterator. This version can be extended by adding more intermediate stages, and parameterized through primitives that return a new data pipeline, which in turn can be further optimized. For example, to combine two different data sources (or to parallelize I/O), two data pipelines ds and other_ds can be interleaved with the statement: ds = ds.interleave(other_ds, num_parallel_calls=2). Similar primitives allow for the definition and parameterization (number of threads, buffer size, batch size, etc.) of map transformations, shuffling, prefetching, and batching.

This decoupled design allows data pipelines to achieve efficient overlapping of tasks, not only of a data pipeline with the training pipeline itself, but also of multiple intermediate stages within the data pipeline. However, such overlapping also leads to contention for resources (computational units, memory, network bandwidth), which introduces non-trivial trade-offs. For example, memory can be split between the data pipeline and training pipeline in different ways: metadata queue, shuffle queue, and prefetch queue. Computational units must similarly be shared between DL training and threads allocated to the data pipeline. Thus, fine-tuning a data pipeline is challenging.

To address this issue, data pipelines combine a series of optimizations that are applied both statically and dynamically. Static optimizations can benefit from the fact that composability allows the expression tree of a data pipeline to be inspected and updated at runtime to use more efficient methods where such exist. For example, if the iterator produces mini-batches, then it would be inefficient to apply a map() transformation individually to each training sample; instead, it can be applied in bulk after assembling an entire mini-batch. This is an example of a fusion operation.

Opportunities for dynamic arise because there are many variables that are known only at runtime: properties of the training samples (e.g., image size) and available resources (e.g., computational units, memory, network bandwidth). Thus, data pipelines implement an auto-tuning mechanism that redistributes resources among the intermediate stages so as to minimize the expected latency of producing an element that is obtained by get_next(). Specifically, the performance of each intermediate stage (obtained from the expression tree) is constantly monitored at runtime in order to build and update an analytical model for it. Analytical models are composed and optimized using gradient descent, in a manner similar to the actual DL training. Using this approach, the entire end-to-end pipeline can be optimized. This aspect is essential, because global resource contention inevitably leads to cases in which locally optimal decisions are suboptimal. For example, they may employ excessive parallelism and buffering, which in turn lead to inefficient thread scheduling, memory utilization, and caching locality. This effect is even more pronounced for ensemble learning, neural network architecture search and hyper-parameter optimization, where a large number of model variations need to be explored concurrently. In this case, the different data pipelines incur redundant overheads even during the same epoch, because the training data needed by one model variation may be cached by another model variation. An opportunity in this context is to allow the data pipelines to be aware of each other's producer-consumer queues and to share their content in order to optimize their performance and scalability as a group. Unlike state-of-art collaborative caching approaches, this approach would expose not only input data but also intermediate data (e.g., training samples after they were transformed by map()), which may further reduce redundant overheads. However, this approach also requires that data pipelines be able to discover new intermediate data dynamically as they become available, which introduces the need for advanced metadata management techniques.

Adaptive streaming with rehearsal support

When, as in continual learning, input data are streamed directly from a data source without being accumulated on a storage repository, it is not sufficient simply to minimize the time required to access data pipeline iterators. It is also important to adapt to the rate at which training samples arrive. Specifically, if the stream generates input data more slowly than are consumed by the iterator, then the overall training pipeline will be slowed down. On the other hand, if the stream generates data more rapidly than the can be consumed by the iterator, we encounter the difficult question of what to do with the excessive input data. Should we simply drop them at random, or alternatively buffer them under the assumption that the production/consumption rate will change in the future? If buffering is feasible, where can it be done? There are multiple choices, each with its own trade-offs: (1) a sufficiently large storage repository can be used as a buffer, with the caveat that subsequent reads will incur I/O overheads; (2) spare capacity in the queues of other data pipelines can be used for buffering, although this may interfere with their own optimizations.

A related problem is how to store the exemplars needed for rehearsal, as discussed in Section 1. If exemplars are persisted in the storage repository, then more can be accumulated for better diversity (especially when considering that the same exemplar can be augmented by the data pipeline in different ways). However, exemplars are then more costly to access and process. Another approach is to introduce an additional queue (e.g., rehearsal queue) to mitigate this problem, at the expense of complicating the data pipeline and introducing more resource contention.

Decoupled design

Efforts to build data pipelines are not limited to the Tensorflow ecosystem. For example, NVIDIA's DALI (The NVIDIA Data Loading Library) [START_REF]NVIDIA Data Loading Library[END_REF] aims to provide highly optimized building blocks for loading and processing image, video and audio data for DL applications. The key focus is on portability: it can be used as a drop-in replacement for built in data loaders and data iterators of popular DL frameworks, including Tensorflow data pipelines. DALI supports a wide range of data formats and augmentations, while taking advantage of vendor-specific optimizations, such as leveraging a direct data path between storage and GPU memory (using GPUDirect). As a consequence, the native data pipelines provided by the DL frameworks, while offering better integration with the data-flow graphs, may not always be the preferred choice of users. Thus, optimizations such as collaborative producer-consumer queues and streaming with rehearsal support should ideally be abstracted as a separate layer that can integrate with more than a single data pipeline effort.

Fortunately, efforts such as DALI rely on a similar architecture as the Tensorflow data pipelines, sharing the same principle of chaining producer-consumer queues. Thus, an opportunity in this context is to provide a unified API to manage generic objects that are cached collaboratively in the queues of multiple compute nodes. For example, given the pseudo-random nature of I/O accesses and other operations in general, the interactions with the queues and caches are deterministic. Therefore, given a fixed caching algorithm and a fixed seed used by the pseudo-random number generator of each compute node, the availability of data and even memory offset on a remote compute node can be determined without additional metadata that needs to be exchanged between the compute nodes. Such capabilities can be encapsulated and exposed as a separate layer for use with Tensorflow pipelines, DALI and potentially other frameworks.

Preliminary evaluation 5.1 Experimental Setup

Our experiments were performed on Argonne's ThetaGPU cluster, a testbed specifically optimized for training DNN models at scale. It comprises 24 NVIDIA DGX A100 nodes, each with eight NVIDIA A100 Tensor Core GPUs and two AMD Rome CPUs. Memory-wise, each node is equipped with 1 TB of DDR4 memory and 320 GB GPU memory, for a total of 24 TB DDR4 and 7.6 TB GPU memory. The nodes are interconnected using 20 Mellanox QM9700 HDR200 40-port switches wired in a fat-tree topology. External storage is provided by a Lustre parallel file system deployment with aggregate 250 GB/ bandwidth, mounted using POSIX (referred to as the PFS).

In terms of software, we use TensorFlow 2.5.0 and Horovod 0.20.3. Communication between nodes is facilitated by Open-MPI 4.1.1, which is used by Horovod.

Methodology

In order to quantify the limitations of the TensorFlow data pipelines, we study both the I/O overheads and the impact of those overheads on overall training duration under various data-parallel training scenarios. To enable data-parallelism, we rely on the Horovod [START_REF] Sergeev | Meet Horovod: Uber's open source distributed deep learning framework for tensorflow[END_REF] runtime library. Specifically, Horovod hides the details of parallelization by wrapping around the optimizer normally instantiated by a Keras DL application. Using this approach, Horovod augments the data-flow graph transparently to average the local gradients of all workers using an all-reduce collective communication pattern (e.g., as provided by MPI implementations) before proceeding with the weight updates.

To measure the I/O overheads, we use Darshan [START_REF] Snyder | Modular HPC I/O characterization with Darshan[END_REF], an I/O profiling tool that transparently intercepts all POSIX system calls in order to generate statistics (number of calls, durations, sizes, etc.). The impact on the training duration is non-trivial to measure, because the data pipelines overlap the sample pre-pocessing with the mini-batch training. Thus, we rely on an indirect method that compares two data storage strategies: (1) the dataset is shared by all model replicas of the data-parallel training via the PFS; (2) the dataset is cached in-memory on each compute node by using the /dev/shm mount point, which features a tmpfs filesystem. In the case of (2), the I/O overheads are negligible. Thus, by measuring the difference between the end-to-end runtimes of ( 1) and ( 2), we can infer by how much time the data pipelines stall the training process due to the I/O overheads of the PFS. As we shall see, we find that the I/O overheads of the PFS and the duration of the stalls are high, thereby justifying the opportunities mentioned in Section 4.

Application: ResNet-50. We use ResNet-50 [START_REF] He | Deep residual learning for image recognition[END_REF] as a representative DL model that is frequently used as a DL benchmark. Specifically, it is a family of DNN where the layers learn residual functions with reference to the input layers, instead of learning unreferenced functions. This allows ResNet to train extremely deep neural networks with 150+ layers. For the purpose of this work, we focus on the variant with 50 layers, which has a reference implementation for use with Keras and Horovod. Although ResNet-50 is deep, its overall size is relatively small (i.e., in the order of MiB). Therefore, the model and its intermediate computational states can be cached in the GPU memory and does cause any I/O related interference that may influence the study of the data pipelines.

Datasets: ImageNet and Tiny-ImageNet. As input to ResNet-50, we have chosen two standardized datasets. Ima-geNet is a widely used dataset in the evaluation of DL models for image classification. It comprises 1.28 million training images and 50,000 validation images, most with a size of between 10 KB and 100 KB, and each assigned to one of 1000 classes. Tiny-ImageNet is similar to ImageNet, but smaller, with just 200 image classes, a training dataset of 100,000 images, a validation dataset of 10,000 images, and a test dataset of 10,000 images. All images are of size 64×64.

Results

We first examine the I/O overheads captured by Darshan for the case when the dataset is shared via the PFS. We study strong scaling in two scenarios: (1) increasing number of GPUs per node (Figure 3a); and (2) increasing number of nodes with eight GPUs/node (Figure 3b).

Figure 2 confirms that DNN training is a read-intensive scenario in which each process predominantly performs independent read operations; the independent write operations incur only minimal overhead (cached Python JIT compiled code and temporary files). The figure also shows there are no shared read or write operations, which confirms the data pipelines do not produce intermediate temporary files that shared and potentially subject to concurrency control. Note that the read amount per process decreases with increasing number of GPUs, due to partitioning. There are several interesting observations with respect to the read overheads, depicted in Figure 3. For a single node, despite no concurrent access to the PFS, the read overheads are high. For a single GPU, they reach up to 25% of the total runtime for Tiny-ImageNet and 50% for ImageNet. The metadata overheads are also non-trivial, reaching up to 5% of total runtime. With an increasing number of GPUs per node (4 and 8), both the read and metadata overheads begin to decrease due to OS-level caching. However, this is only possible because our nodes are equipped with 1 TB of main memory and ResNet-50 is a relatively small model. With increasing model complexity, there will be less free memory available for OS-level caching and therefore diminished benefits. For an increasing number of nodes (more than eight GPUs), the read and metadata overheads keep decreasing for Tiny-ImageNet, which is expected because different nodes can read from different I/O servers in parallel, therefore achieving a higher aggregated I/O bandwidth. However, the decrease of the I/O overheads is much slower for ImageNet, which means the PFS is experiencing I/O bottlenecks under concurrency, even for a small number of nodes.

To explain these findings better, we depict in Figure 4 the number of read operations broken down by size ranges. We see that the total number of read operations in the 0-100 bytes range equals the number of files for each dataset. This is due to the fact that the data pipeline needs to read the header of each image to determine its size and allocate the corresponding tensors before following up with a subsequent read of the content. Since there are many files in each dataset, the resulting aggregate metadata overhead is high. Furthermore, since each file is small, most read operations involve less than 10 KB in the case of Tiny-ImageNet and are in the order of 100 KB in the case of ImageNet. The PFS was not optimized for this pattern, which leads to an overall throughput in the order of a few MB/s per node-two orders of magnitude less than the peak I/O throughput.

For completeness, we show in Tables 1 and2 the actual measurements of the end-to-end runtime and cumulative duration of read and metadata I/O operations. The cached runtime corresponds to the case when the full dataset is cached in memory on each compute node (on the /dev/shm).

Figure 5 depicts the percent increase in runtime due to reading from the PFS vs. the cached runtime. For Tiny-ImageNet, the increase in runtime is high when using a single GPU (up to 50%), but it drops sharply for an increasing number of GPUs. Thus, we can conclude that despite large read overheads, the data pipelines successfully mask them without causing stalls. However, the situation is reversed for ImageNet: the increase in runtime is much higher (up to 200%) and it only drops slightly from one GPU to four and eight GPUs, but then goes up again much faster. This confirm our hypothesis that a PFS is likely to become a significant bottleneck both at small scale and large scale.

Furthermore, another important observation can be made: the relative increase in runtime of PFS vs. cached is much higher than the PFS I/O overhead (reads and metadata operations) to PFS runtime ratio. Thus, we can conclude that the stalls caused by the slow PFS read operations are augmented further down the data pipelines, which means it will not suffice solely to make the read operations faster.

Conclusions

We have reviewed the challenges and opportunities of designing efficient data pipelines for data-parallel training of DNN models, which is essential in the design of flexible AI runtimes deployed on HPC systems. Based on a preliminary evaluation of Tensorflow data pipelines on several nodes and GPUs, we highlighted a significant I/O overhead due to both data ingestion and metadata operations that involve a parallel file system, which can cause up to 200% increase in execution time. Based on this preliminary evaluation, we plan to pursue future work that enables data pipelines to collaborate and cache training samples as a group, both with respect to raw input data and intermediate transformations.

Such an approach aims to minimize expensive I/O interactions with parallel file systems and other external repositories, thereby reducing observed I/O overheads. Furthermore, streaming training data for continual learning and/or reinforcement learning often involves rehearsal of representative historic training samples and/or learning patterns, which can be implemented in a distributed fashion by leveraging collaborative caching. Finally, the rising popularity of a variety of data loading and preprocessing frameworks makes it important to focus on a decoupled design that expose a unified API that can be leveraged in such frameworks.

Figure 1 .

 1 Figure 1. The data pipeline abstraction as implemented in TensorFlow. It exposes an iterator that returns the next mini-batch to be used by the training process with each invocation. The pipeline is implemented as an asynchronous producer-consumer workflow with intermediate buffers where the training samples are read from the repository, encoded as tensors, optionally transformed using custom augmentation functions, shuffled and finally assembled as mini-batches.

4

  Challenges and Opportunities4.1 Collaborative producer-consumer queues Data pipelines are designed to be used individually on each compute node. As a consequence, they are unaware of each other and may perform sub-optimally at large scale. For example, in a data-parallel training scenario in which many identical model replicas average their gradients, input data are typically shuffled and partitioned among multiple model replicas, each of which then visits a different set of training samples. To this end, each model replica could create a data pipeline for its assigned partition. However, after one epoch, a reshuffling and repartitioning is necessary in order to preserve the global randomness of sampling. Unaware that most input data are already cached, pre-processed, shuffled, and batched on other nodes, the data pipelines will start from scratch, thereby incurring redundant overheads (I/O contention to the parallel file system, resource contention due to overlapping with the training pipeline) that may slow DL training.

Figure 2 .

 2 Figure 2. Amount of I/O (independent reads, independent writes, shared reads, shared writes) per process during ResNet-50 training. The Y axis aggregates all I/O amounts performed by all I/O operations.

Figure 3 .

 3 Figure 3. Duration of PFS I/O operations (reads, writes, metadata queries) per process as a percentage of the total runtime for an increasing number of GPUs (up to eight per node).

Figure 4 .

 4 Figure 4. Bytes per read operation when training Resnet-50. Each label on the X axis represents a different size range, while the Y axis counts the number of read operations in each range.

Figure 5 .

 5 Figure 5. PFS impact measured as percent increase of end-to-end runtime over in-memory caching. The X-axis labels refer to the number of GPUs used for training with the tiny (T) or full (I) ImageNet dataset. Lower is better.

•

  We revisit state-of-art streaming techniques for feeding training samples to DL model training pipelines.

	We focus in particular on the data pipeline abstraction
	used by TensorFlow to enable transparent handling
	of prefetching, caching, shuffling, and data augmenta-
	tion. These methods overlap data transformations with
	DL training, which generates contention for resources
	(computational units, memory, network bandwidth)
	and this non-trivial trade-offs (Section 3).
	• We discuss several considerations that arise when data
	pipelines are used at scale in conjunction with data-
	parallel and/or ensemble training techniques. In this
	context, the data pipelines constructed on each com-
	pute node are not independent of each other, but rather
	share input data, intermediate data, and/or resources
	(e.g., the data repository). This observation leads us to
	identify several challenges and opportunities to lever-
	age synergies between the independent data pipelines
	for both offline training and continual learning; in the
	latter case, we advocate for extensions to facilitate
	transparent rehearsal (Sections 4).
	• We evaluate the I/O overhead of serving training sam-

Table 1 .

 1 End-to-end runtime and cumulative duration of read and metadata I/O operations, in seconds, per process when training Resnet-50 on Tiny-Imagenet.

				GPUs
		1	4	8	16	32	64
	PFS reads	322 52.7 24.5 12.4 4.93 3.3
	PFS metadata	121 31.6 15.6 10.5 6.8 5.0
	PFS runtime	1473 421 275 221 171 160
	Cached runtime 985 362 269 217 168 157

Table 2 .

 2 End-to-end runtime and cumulative duration of read and metadata I/O operations, in seconds, per process when training Resnet-50 on Tiny-Imagenet.

				GPUs
		1	4	8	16	32	64
	PFS reads	19473 3441 1502 859 449 217
	PFS metadata	1233 326.7 211.1 145.2 108.9 73.1
	PFS runtime	36247 8345 4298 2709 1744 1209
	Cached runtime 15263 3852 1975 1038 576 364

FlexScience '22, July 1, 2022, Minneapolis, MN, USA Liu et al.