Eloïse Teboul

Eliise Tammekivi

Magali Batteau

Christophe Geantet

Karine Faure
email: karine.faure@isa-lyon.fr

Off-line two-dimensional separation involving supercritical fluid chromatography for the characterization of the wastewater from algae hydrothermal liquefaction

Keywords: Two-dimensional liquid chromatography, supercritical fluid chromatography, Off-line LC x SFC separation, Microalgae, Hydrothermal liquefaction

Off-line two-dimensional separation involving supercritical fluid chromatography for the characterization of the wastewater from algae hydrothermal liquefaction

Eloïse Teboul, Eliise Tammekivi, Magali Batteau, Christophe Geantet, Karine Faure

To cite this version:

Eloïse Teboul, Eliise Tammekivi, Magali Batteau, Christophe Geantet, Karine Faure. Off-line two-dimensional separation involving supercritical fluid chromatography for the characterization of the wastewater from algae hydrothermal liquefaction. Journal of Chromatography A, 2023, 1694, pp.463907. 10.1016/j.chroma.2023.463907. hal-04037993

Introduction

Microalgae conversion is a promising technology for biofuel production. Among various thermochemical methods applied for converting algae biomass, hydrothermal liquefaction (HTL) is one possible conversion process that has the advantage of using wet raw material and moderate temperatures without the need for solvents. Therefore, it possesses a lower energy consumption than pyrolysis [START_REF] Barreiro | Hydrothermal liquefaction (HTL) of microalgae for biofuel production: State of the art review and future prospects[END_REF]. Among cultivated algae for biofuel production, Chlorellacea are very attractive species considering their resistance and fast-growing strains [START_REF] Krienitz | Phylogenetic relationship of Chlorella and Parachlorella gen. nov. (Chlorophyta, Trebouxiophyceae)[END_REF]. During the HTL process, the content of algae cells (lipids, proteins, and carbohydrates) is subjected to hydrolysis, deamination, and decarboxylation, leading to the production of compounds of interest for biofuel production. Furthermore, intermolecular reactions such as condensation and Maillard reaction lead to the formation of many polar compounds. The conversion results in two phases: a bio-oil that undergoes catalytic upgrading to produce paraffins, and an aqueous wastewater. This wastewater is rich in dissolved organic compounds, especially in N-containing molecules. It can be valorized further as a nutrient source for cultivating new algae [START_REF] Jena | Evaluation of microalgae cultivation using recovered aqueous co-product from thermochemical liquefaction of algal biomass[END_REF][START_REF] Hognon | Comparison of pyrolysis and hydrothermal liquefaction of Chlamydomonas reinhardtii[END_REF][START_REF] Leng | Use of microalgae to recycle nutrients in aqueous phase derived from hydrothermal liquefaction process[END_REF], which reduces the economic footprint of algae conversion. However, the utilization of HTL wastewater has a dual effect on algae. It may slow down the algae growth but simultaneously increase the lipid content [START_REF] Tsarpali | Beneficial use of the aqueous phase generated during hydrothermal carbonization of algae as nutrient source for algae cultivation[END_REF]. Chlorella sorokiniana showed the best adaptation to HTL wastewater as a growth medium [START_REF] Stablein | Microalgal photosynthetic inhibition and mixotrophic growth in Post Hydrothermal Liquefaction Wastewater (PHW)[END_REF]. The characterization of the HTL wastewater is of major importance to i) confirm the conversion pathways and optimize the HTL parameters, ii) select the algae strains and optimize their growth conditions, and iii) identify which compounds serve as nutrients and which compounds may have inhibitory effects on growth or the lipid yield.

The complex chemical composition of HTL wastewater has been demonstrated by detecting a large number of compounds with GC x GC-MS: cyclic nitrogen compounds such as pyrroles, indoles, pyrazines, and furans from amino acid condensation; phenols, ketones, and carboxylic acids from the decomposition of polysaccharides, piperazines and pyrrolidones from Maillard reaction between carbohydrates and basic amino acids [START_REF] Sundarrajan | Insights into valuing the aqueous phase derived from hydrothermal liquefaction[END_REF][START_REF] Maddi | Quantitative Characterization of Aqueous Byproducts from Hydrothermal Liquefaction of Municipal Wastes, Food Industry Wastes, and Biomass Grown on Waste[END_REF][START_REF] Pham | Chemical and Biological Characterization of Wastewater Generated from Hydrothermal Liquefaction of Spirulina[END_REF]. While most of these molecules can be identified by using the NIST library, only 30 molecules have been quantified so far using GC-FID. Besides, the polarity of the organic acids or N-containing compounds and the aqueous nature of the wastewater may orientate the analyst towards liquid chromatography analysis. Thus, Lazzari et al. [START_REF] Lazzari | Comprehensive two-dimensional liquid chromatography-based quali-quantitative screening of aqueous phases from pyrolysis bio-oils[END_REF][START_REF] Lazzari | Quantitative analysis of aqueous phases of bio-oils resulting from pyrolysis of different biomasses by two-dimensional comprehensive liquid chromatography[END_REF] used LC x LC-HRMS to analyze the aqueous phase obtained from the pyrolysis of different biomasses, illustrating the power of two-dimensional separation for this purpose. However, the RPLC x RPLC setup showed some limitations in the orthogonality it could provide. Besides, to the best of our knowledge, the use of twodimensional (2D) LC techniques has not yet been reported for the aqueous phase from HTL.

In this work, we evaluated the potential of using supercritical fluid chromatography (SFC) in combination with RPLC to analyze HTL wastewater from algae conversion. While RPLC in the first dimension is fully adapted to the separation of organic compounds dissolved in water, SFC on polar stationary phase shall provide an orthogonal separation mechanism, increasing the overall peak capacity compared to the RPLC x RPLC method.

The combination of LC and SFC has been scarcely implemented despite its high orthogonality, mainly because of the unavailability of commercial online instrumentation [START_REF] Kaplitz | Two-dimensional separation techniques using supercritical fluid chromatography[END_REF][START_REF] Burlet-Parendel | Opportunities and challenges of liquid chromatography coupled to supercritical fluid chromatography[END_REF]. Proof of concept online LCxSFC has been experienced in our group on the aqueous phase of pyrolytic oil [START_REF] Sarrut | Potential and limitations of on-line comprehensive reversed phase liquid chromatography x supercritical fluid chromatography for the separation of neutral compounds: An approach to separate an aqueous extract of bio-oil[END_REF] but it requires mobilizing simultaneously a LCxLC and a SFC instrument. On the other hand, off-line configuration can easily be implemented using two distinct instruments and providing a large set of information in just a few hours. The conditions used in both dimensions were optimized, using analysis time and effective peak capacity as criteria. Careful considerations were brought on the subject of injecting LC fractions into the SFC so that there would be no need to modify the fractions between the two dimensions. The developed off-line LC x SFC-HRMS was then applied to the aqueous wastewater of the HTL conversion of Chlorella sorokiniana.

Material and methods

Chemicals and sample preparation

Acetonitrile (ACN) of HPLC grade was purchased from Honeywell Riedel-de-Haën (Seelze, Germany), formic acid (FA) and methanol (MeOH) (HPLC grade) were purchased from Merck (Darmstadt, Germany), and ethanol (EtOH) of HPLC grade was obtained from Fisher Scientific (Loughborough, UK). Water was obtained from an Elga water purification system (Veolia water STI, Le Plessis Robinson, France). Pressurized liquid CO2 (99.995%) was obtained from Air Liquide (Pierre Bénite, France).

Chlorella sorokinania was grown under controlled conditions for 10 days and harvested by Dr. Delrue at CEA Cadarache (CTREG/DPACA, France). It was converted into bio-oil using the HTL process by Dr. Roubaud at CEA Grenoble (Biomass Technology Lab, LITEN, France). The aqueous phase was separated from the organic phase, then filtered through a Chromaphil Xtra PES 0.2 µm membrane filter, and stored at 5 °C.

Instruments

Agilent Infinity II 1260 LC system was equipped with a binary solvent delivery pump, an autosampler with a 900 µL loop, a column oven compatible with temperatures up to 90 °C, a diode array detector (DAD) with a 1 µL flow-cell, and a fraction collector with a capacity of 80 fractions. The maximum allowable flow rate was 4 mL/min and the maximum allowable pressure was 400 bar. Data acquisition was performed using Chemstation software (Agilent Technologies, Waldbronn, Germany). The extracolumn variance and dwell volume were determined experimentally and found to be 36 µL 2 and 2.26 mL, respectively. Agilent Infinity II 1260 SFC system was equipped with a binary solvent delivery pump, an Infinity II autosampler with a 100 µL loop, a column oven compatible with temperatures up to 85 °C, a DAD detector with a 2 µL flow-cell withstanding a pressure up to 400 bar, a back pressure regulator (BPR), and a make-up pump. The maximum allowable pressure was 400 bar. The dwell volume was measured in quasi-liquid conditions at 600 µL. Feed injection conditions were as follows: overfeed volume 4 µL, feed speed 1000 µL/min, and heptane as the feed solvent.

As the HRMS, an Agilent 6560 Ion-Mobility Quadrupole Time-of-Flight LC/MS equipped with a Jet Stream electrospray ionization (ESI) source (Agilent Technologies) was used.

Data acquisition was performed using Chemstation software (Agilent Technologies) and treated using MassHunter Qualitative 10.0. The 2D chromatograms were plotted using an in-house developed Matlab program from S. Heinisch and F. Rouvière.

Methods

The LC method was performed using a XSelect CSH Phenyl-Hexyl column (150 mm x 4.6 mm, 5 µm) on the Agilent Infinity II 1260 LC. The flow rate was 2.8 mL/min and the temperature was 30 °C. Final gradient conditions are displayed in Table 1. The injection volume was 500 µL. A set of 80 fractions were collected from the first dimension during the cycle time of 11.55 min, comprising gradient time, dwell time and dead time. Each fraction was collected every 0.14 min, providing a fraction volume of 392 µL, using the fraction collector on the Agilent Infinity II 1260 LC.

Four SFC columns purchased from Waters were screened: Torus 2-picolamine (2-PIC), 1aminoanthracene (1-AA), diol (DIOL), and diethylamine (DEA) (all with dimensions of 100 mm x 3 mm, 1.7 µm), using a generic gradient elution (2% -50% of methanol or ethanol as the co-solvent, normalized gradient slope of 5%). The temperature, the BPR, and the wavelengths were set respectively at 40 °C, 140 bar, 210 nm and 254 nm for all the experiments. The maximum flow rate allowed for all four columns was determined to be 1.1 mL/min when using 50 % co-solvent (generating around 400 bars). The final method was performed on a shorter DIOL column (50 mm x 3 mm, 1.7 µm) with a flow rate of 1.8 mL/min, as summarized in Table 1.

The 50-mm long SFC column was hyphenated to the MS using a "preBPR splitter with a sheath pump" configuration [START_REF] Guillarme | What are the current solutions for interfacing supercritical fluid chromatography and mass spectrometry[END_REF]. Briefly, the end of the SFC column was connected to a make-up pump delivering 2 mL/min of MeOH + 0.1% FA using a T-union. Then, the resulting flow was split using a second T-union. A low restriction tubing (170 µm, 90 cm) led most of the flow to the BPR, while a high restriction tubing (9 µm, 100 cm) decreased the flow rate entering the ESI source. The stainless steel tubing kit was provided by Agilent Technologies. In the final configuration, an additional pump was connected via a third T-union, delivering MeOH + 0.1 % FA to the MS at 200 µL/min. The ESI source settings were the following: drying gas temperature, 300 °C; drying gas flow rate, 11 L/min; nebulizer pressure, 40 psi; sheath gas temperature, 350 °C; sheath gas flow rate, 11 L/min; fragmentor voltage 185 V; capillary voltage, 3500 V. Mass spectra data were collected in full scan mode with a mass range of 100-1700 m/z (with 18920 FWHM resolution at m/z = 322) and an acquisition rate of 10 spectra/s. No collision energy was employed during the MS data collection. The accurate mass of the precursor ion was recorded with a mass tolerance of ± 2 ppm.

Results and discussion

The off-line configuration allowed the use of commercially available instruments and decoupled both separation times and mobile phase flow rates between the two dimensions. This means that the stationary and mobile phases of each dimension could be screened individually and that the geometry of the columns can also be very different. Hence, the first dimension should be set on a column with a large volume, facilitating the work of fraction collection systems while limiting the impact of the external variances of the instrument on the chromatographic performances. The second dimension geometry has to be much smaller so that the flow would be adjusted for the hyphenation with MS while the overall separation time is kept reasonable.

Development of the first dimension ( 1 D) separation method

Based on a previous study [START_REF] Sarrut | Potential and limitations of on-line comprehensive reversed phase liquid chromatography x supercritical fluid chromatography for the separation of neutral compounds: An approach to separate an aqueous extract of bio-oil[END_REF], the phenyl hexyl stationary phase was selected according to its theoretical orthogonality with SFC stationary phases investigated here [START_REF] West | An improved classification of stationary phases for ultra-high performance supercritical fluid chromatography[END_REF]. A scouting analysis of the sample was performed at 1% gradient steepness over the range 1%-80% ACN, on a 150 mm x 4.6 mm column to facilitate fraction collection. It was revealed that the detected compounds eluted in a range of 1% to 30% ACN, indicating that the molecules present in the aqueous phase of HTL exhibit medium to high polarity. The 1 D operating conditions were optimized so that the overall peak capacity of the 2D separation was maximized. According to the linear solvent strength theory [START_REF] Snyder | High-Performance Gradient Elution: The Practical Application of the Linear-Solvent-Strength Model[END_REF], the theoretical peak capacity in the first dimension 𝑛 𝑡ℎ can be estimated by

𝑛 𝑡ℎ = 1 + √𝑁 4 1 2.3 𝑠𝑆+1 ln ( 2.3 𝑠𝑆+1 2.3 𝑠𝑆 𝑒 2.3 𝑆∆𝐶 - 1 2.3 𝑠𝑆 ) ( 1
)
Where N is the column efficiency, S the slope of the retention model, ΔC the elution composition range, and s the 1 D gradient steepness.

However, the first dimension undergoes sampling after the elution, which has deleterious effects on the resolution of peaks. It is generally conceived that a peak should be sampled 3 to 5 times. But in most online LC x LC separations, this recommendation is usually not followed. Indeed, in online configuration the sampling time equals the second dimension duty cycle. Reducing the sampling period would induce a reduction in the second dimension gradient time that limits the peak capacity of the second dimension. In off-line configuration, there is no such constraint on sampling time as the two dimensions are decoupled in time and the number of fractions is only limited by the instrumental constraints. Using Monte Carlo simulation of Gaussian peaks, Davis et al. evaluated the impact of undersampling on the peak capacity obtained after collection (𝑛 𝑠 ) as Equation 2[19]

𝑛 𝑠 = 𝑛 𝑡ℎ √1+0.21( 𝑡 𝑠 𝜎 𝑡 )² (2)
With 𝜎 𝑡 the average peak standard deviation in time units and ts the sampling time, corresponding to the time required to collect a fraction in a vial.

When approximating that the peaks are homogeneously distributed along the gradient time tg, the theoretical peak capacity nth can be expressed as:

𝑛 𝑡ℎ = 𝑡 𝑔 4𝜎 𝑡 (3)
In a comprehensive mode, the collection has to be performed along the whole separation space, here during tg. The number of fractions 𝑛 𝑓 required in a comprehensive mode is therefore:

𝑛 𝑓 = 𝑡 𝑔 𝑡 𝑠 (4)
Combining equations ( 2), (3), and (4), the effective peak capacity after sampling in off-line configuration can be expressed as a function of the theoretical peak capacity and the number of fractions ( 5)

𝑛 𝑠 = 𝑛 𝑡ℎ √ 1+3.36( 𝑛 𝑡ℎ 𝑛 𝑓 )² (5)
By plotting the percentage of the remaining peak capacity

𝑛 𝑠 𝑛 𝑡ℎ
vs. the ratio of the number of fractions over the theoretical peak capacity 1a), it can be observed that a goal of 80% remaining peak capacity requires that the number of fractions exceeds the theoretical peak capacity by a factor 2.5. This objective is far from being reached, as off-line LC x SFC publications often report collection of only 40-50 fractions [START_REF] Stevenson | Comprehensive two-dimensional chromatography with coupling of reversed phase high performance liquid chromatography and supercritical fluid chromatography[END_REF][START_REF] Wei | A highefficiency strategy integrating offline two-dimensional separation and data post-processing with dereplication: Characterization of bufadienolides in Venenum Bufonis as a case study[END_REF], a number that is surprisingly low since fraction collection systems offer between 80 and 120 vials, depending on the manufacturer. The undersampling effect is often not taken into account for determining the effective peak capacity of the off-line 2D approach while it is systematically done in the optimization of online LC x LC [START_REF] Vivó-Truyols | Comprehensive Study on the Optimization of Online Two-Dimensional Liquid Chromatographic Systems Considering Losses in Theoretical Peak Capacity in First-and Second-Dimensions: A Pareto-Optimality Approach[END_REF][START_REF] Sarrut | Optimization of conditions in on-line comprehensive two-dimensional reversed phase liquid chromatography. Experimental comparison with onedimensional reversed phase liquid chromatography for the separation of peptides[END_REF].

𝑛 𝑓 𝑛 𝑡ℎ (Figure
The theoretical peak capacity on a typical HPLC column geometry of 4.6 x 150 mm; 5 µm particle size, can be estimated using equation 1 (Figure 1b). It relates both to the square root of efficiency and to the gradient steepness, and so depends on flow rate and gradient time. Increasing the flow rate from 0.7 mL/min (optimal flow rate) to 2.8 mL/min (maximal flow rate) on the LC column led to an expected loss of observed efficiency, from 10400 plates to 6600 plates, while for the same gradient time, increasing the flow rate decreased the gradient steepness. Theoretical peak capacity for the two flow rates was plotted as a function of gradient time (Figure 1b) and showed that using the 4.6 mm x 150 mm; 5 µm column geometry, it was more beneficial to work at large flow rate. Theoretical peak capacity was estimated between 50 and 140 for a gradient time between 3 min and 60 min (Figure 1b). Keeping 80% effective peak capacity (Equation 5) would require between 125 and 700 fractions, that would not only being difficult to handle technically, but also would result in an unreasonable 2 D analysis time. Figure 1b reports the effective peak capacity obtained after sampling when using a collection system with 80 fractions. The method development in online LC x LC tends to suggest that the 1 D gradient time should be quite long, usually 30 to 60 min, so that the peak broadening in the first dimension would allow a longer sampling time corresponding to the 2 D cycle time. On the contrary, in off-line mode, the maximal peak capacity obtained after sampling is reached for short gradient times (Figure 1b). In the present work, using a collection system of 80 fractions, a 10 min gradient performed at maximal flow rate was sufficient to reach the maximal peak capacity of 41 (Figure 1b). Using another collection system allowing 100 fractions would increase the peak capacity after undersampling to a value of 50. With these selected gradient time and flow rate, the gradient range was adjusted to 1%-40% ACN, so that according to the new normalized gradient slope, all the compounds eluted during the gradient. The composition at elution of the last peak was evaluated to be 33 % ACN in these conditions.

Thanks to the aqueous nature of the algal HTL waste and the use of an RPLC method in the first dimension, on-column focusing allowed large injection volumes, thus increasing sensitivity without compromising the chromatographic performances. Injection compression factor is related to the ratio between two solute retention factors: ks , obtained with the injection solvent, here 100 % water, and ke, obtained with the mobile phase at elution. [START_REF] Groskreutz | Quantitative evaluation of models for solvent-based, oncolumn focusing in liquid chromatography[END_REF]. With a sample in 100% water and a gradient starting at 1 % ACN, the compression factor is expected to be over 1 and increasing with the solute retention. Three distinct peaks distributed in the first half of the chromatogram were monitored using UV detection at 254 nm (Figure 2a). The peaks eluting before Peak 1 were not sufficiently resolved to measure their peak width. The peak capacity calculated for these three peaks when injecting increasing volumes was compared to the peak capacity obtained when injecting 100 µL, corresponding to 5.8% of the column volume (Figure 2b). Peak broadening when injecting over 200 µL resulted in a peak capacity decrease as expected. While Peak 1 (9% ACN at elution) and Peak 3 (17% ACN at elution) followed the same variation, with a loss of only 20% in peak capacity, Peak 2 (16% ACN at elution) exhibited a much larger loss. An injection volume of 500 µL (30% of the column dead volume) was selected, hence keeping more than 70% of the peak capacity for compounds eluting with over 9% ACN. The broadening of compounds with a composition at elution lower than 9% ACN could not be monitored using the 1 D LC-UV system because of co-elution, but the 2D chromatograms developed thereafter showed their peak width remained similar to more retained compounds.

The variability of the first dimension separation was assessed in the final conditions, stated in Table 1, by monitoring the peak width and retention time of 10 peaks spread over the gradient. A sequence of 5 injections was performed on three different days. The intra-day standard deviation of retention time was below 0.01 min for all 10 peaks. The inter-day standard deviation of retention time was below 0.05 min for all peaks, reaching only 0.01 min for 80% of the compounds.

The average peak width at 4σ was measured at a value of 0.11 min ± 0.02 min, so the experimental peak capacity before fraction collection was estimated using equation 3 and the experimental peak width, at a value of 67. Using an 80-vials fraction collector, the 1 D effective peak capacity corrected by the undersampling (Equation 5) was 37, quite close to the value of 41 calculated with theoretical performances (Figure 1b). The difference may be explained by the instrumental variance generated by the collector that was not taken into account in the theoretical calculation.

Development of the 2 D SFC method

The initial screening was performed on four 100 mm long columns with various stationary phases, namely 2-PIC, 1-AA, DIOL and DEA, using methanol or ethanol as co-solvent B, a generic gradient (2-50% B), and a normalized slope of 5% (Figure S1). The sample was initially diluted in 85:15 water/ACN (v/v) to simulate the composition of a fraction collected after the first dimension. The columns based on 1-aminoanthracene and diethylamine were found to be too retentive, leading to uneluted compounds that were removed from the column after flushing it with 100% organic solvent. We suppose that the non-eluting compounds are acidic, as an excessive retention on amino-based stationary phases has already been reported for this class of compounds [START_REF] Prothmann | Ultra-high-performance supercritical fluid chromatography with quadrupole-time-of-flight mass spectrometry (UHPSFC/QTOF-MS) for analysis of lignin-derived monomeric compounds in processed lignin samples[END_REF]. The chromatographic profile obtained using 2-PIC and DIOL columns were similar, with a slightly higher retention on DIOL column. Using a lower gradient steepness of 1% and an initial composition of 2 % organic solvent, the separation space offered by the DIOL column was investigated for various co-solvent compositions (Figure S2). Compounds were detected in the range of 2% -29% using MeOH as the co-solvent and 2% -37% using EtOH as the co-solvent (Figure S2a). The effect of additives in the EtOH co-solvent such as 0.1% formic acid or 2% water was found to have minor impact on the quality of the separation (Figure S2b). Therefore, ethanol was selected as the co-solvent.

The performances of the off-line 2D separation were simulated for various sets of LC and SFC conditions. The LC flow rate was 2.8 mL/min. The 1 D gradient time was allowed to vary between 3 min and 60 min. The calculations were performed for 60, 80 or 100 fractions. The 2 D column length was either 50 mm or 100 mm. The 2 D flow-rate was fixed at the maximal allowable column pressure drop (i.e. 400 bars) when reaching the maximal co-solvent content, corresponding to 1.8 mL/min on a 50 mm long column and 1.1 mL/min on a 100 mm long column. The 2 D gradient time was allowed to vary between 1 min and 10 min. Other conditions were as reported in Table 1.

The theoretical peak capacity of the 2D off-line separation n2D,theo and the overall analysis time t2D were the selected criteria.

The theoretical peak capacity n2D,theo is the result of the product of the theoretical peak capacities from each dimension, corrected by the undersampling after the 1 D collection (Equation 6). In a first approach, the injection effects and the instrumental variances were not included in the calculation. 𝑛 2𝐷,𝑡ℎ𝑒𝑜 = 𝑛 𝑠 * 𝑛 𝑆𝐹𝐶 [START_REF] Tsarpali | Beneficial use of the aqueous phase generated during hydrothermal carbonization of algae as nutrient source for algae cultivation[END_REF] The theoretical peak capacity of the LC separation, after fraction collection 𝑛 𝑠 , was calculated using equation 5. The theoretical peak capacity of the SFC separation nSFC is difficult to model as the density increase that undergoes the mobile phase during the gradient may impact the efficiency. For example, working at a flow rate of 1.8 mL/min on a 50 mm column, the column efficiency was measured at 7400 plates for 2% EtOH as co-solvent and at 4700 plates when the mobile phase contained 50% EtOH. For a first estimation, Equation 1 was used for calculation, with an average efficiency. Better approximation should require a deeper investigation on the modeling of kinetic performances of SFC in gradient mode.

The duration of the offline 2D analysis t2D is the result from the 1 D gradient time and the sum of the 2 D cycle times for analyzing the LC fractions in a comprehensive mode (Equation 7).

𝑡 2𝐷 = ( 𝑡 𝑑𝑤𝑒𝑙𝑙 1 + 𝑡 0 1 + 𝑡 𝑔 1 ) + 𝑛 𝑓 × ( 𝑡 𝑖𝑛𝑗 2 + 𝑡 𝑑𝑤𝑒𝑙𝑙 2 + 𝑡 0 2 + 𝑡 𝑔 2 ) (7)
With i tdwell , i t0 and i tg the dwell time, the dead time and the gradient time of dimension i, respectively. The injection duration in second dimension 2 tinj is a fixed value depending only on the SFC injection design. The Agilent SFC instrument incorporates a feed injector, aimed at pressurizing the sample before introduction along the mobile phase flow [START_REF] Vanderlinden | Effect of the feed injection method on band broadening in analytical supercritical fluid chromatography[END_REF]. The duration of the injection process 2 tinj using this instrument was estimated to be 0.56 min (average of 10 measurements).

The calculated performances of the offline LCxSFC separation are represented in Figure 3. As could be expected, an increase in theoretical peak capacity may be achieved at the expense of longer analysis time, whether increasing the first or the second dimension gradient time. However, the 1 D gradient time had a limited impact on the performances with an optimum at 10 min whatever the set of conditions, as observed previously, whereas the 2 D gradient time (colors in Figure 3) had a major impact on both peak capacity and analysis time. For example for 80 fractions, the optimized theoretical peak capacity was 3359 for 159 min ( 2 tg 1 min), 3866 for 234 min ( 2 tg 2 min) and 4994 for 479 min ( 2 tg 5 min). Secondly, the number of fractions collected had a major impact on both performances criteria. Finally, it appeared that the shortest column, i.e. 50 mm (circle symbols in Figure 3) provided better results than the longest one for short 2 tg. This may due to the beneficial impact of shallow gradients despite a loss in column efficiency. Nonetheless, the two trends where very similar for 2 tg equal 10 min. Using a longer SFC column should be advantageous when peak capacity higher than 5000 are required, but at an expense of analysis time over 16 hours.

In the light of the above, the LC gradient time was set to 10 min and the number of fractions to the maximum allowed by the available instrument. The second dimension was performed on a 50 mm long column at a maximal flow rate of 1.8 mL/min. For an analysis time of 200 min, the gradient time of the second dimension was set at 1.34 min (equation 7) considering an injection time of 0.56 min, a step to return to initial composition in 0.12 min (corresponding to one column dead time) and a hold time at initial composition for 0.33 min (corresponding to the instrument dwell time). In these conditions, the theoretical peak capacity was estimated at 3587.

The SFC instrument was then coupled with HRMS using a commercial pre-BPR splitter with a make-up pump interface that has been proven optimal for the ESI source [START_REF] -G. Perrenoud | Coupling state-of-the-art supercritical fluid chromatography and mass spectrometry: From hyphenation interface optimization to high-sensitivity analysis of pharmaceutical compounds[END_REF]. Methanol + 0.1% FA was added as the make-up solvent to further improve the MS signal. The solvent flow rate entering the MS source can be calculated using the equations described by Guillarme et al. [START_REF] Guillarme | What are the current solutions for interfacing supercritical fluid chromatography and mass spectrometry[END_REF], adapted in our work to the viscous binary mixture of ethanol and methanol. Simulations were performed for the initial mobile phase composition (2% EtOH in CO2) and for the final mobile phase composition (50% EtOH). When the make-up flow was below 600 µL/min, the flow was directed entirely to the MS and the BPR system experienced an issue with maintaining stable pressure. Increasing the make-up flow resulted in an increase in the split ratio. At initial composition, the flow of solvent entering the MS slightly increased with the make-up flow while the flow of CO2 drastically decreased (Figure S1). Using a make-up flow of 2000 µL/min resulted in an initial flow of solvent entering the MS at 760 µL/min. The overall flow, including SFC eluent and make-up flow was splitted by 2.7, from 3800 µL/min to 1419 µL/min into the MS. Unfortunately, the MS signal was not repeatable at the beginning of the gradient in these conditions, despite using a caloratherm at 60 °C to avoid freezing the tubing. Using an additional pump delivering 200 µL/min as proposed by Crepier [START_REF] Crepier | Ultra-high performance supercritical fluid chromatography hyphenated to atmospheric pressure chemical ionization high resolution mass spectrometry for the characterization of fast pyrolysis bio-oils[END_REF] prevented solute precipitation by reducing the CO2 content to 40%. At the final gradient composition, the viscosity of the eluent was much higher, leading to a more important split of flow, estimated at a value of 4.6 (Figure S3). The flow of CO2 entering the MS tubing was low, calculated at 196 µL/min and no freezing or precipitation was noted. With a final 2000 µL/min make-up flow and a 200 µL/min additional flow, the amount of organic solvent entering the MS source remains almost constant through the gradient (960 µL/min at initial gradient conditions and 850 µL/min at final gradient conditions). Meanwhile, the composition of the organic solvent entering the source evolved with the chromatographic gradient, with a ratio 4:96 EtOH/MeOH (v/v) at initial composition and 53:47 EtOH/MeOH (v/v) at final conditions, which had no major influence on the MS response.

The peaks were spread over the whole range of co-solvent (Figure S4) with very different profiles according to detection modes. The molecules responding in positive ESI-MS mode eluted between 0.6 min and 2 min, whereas peaks detected in ESI-MS negative mode eluted mostly between 0.4 min to 1.2 min, and peaks eluting between 0.2 min and 0.4 min were only detected in UV. This observation clearly illustrated the diversity of molecules and the need for three detection modes.

Because isolated peaks could not be distinguished using this short SFC separation, the maximal volume that could be injected into the SFC had to be determined using extracted ion chromatograms (EIC). The fractions collected from the first dimension were in a composition ranging from 99% to 60% water. As the sample was not soluble in aprotic solvents, it was decided to inject the fractions without any treatment. With the objective of increasing the sensitivity, increasing volumes of sample diluted in water/ACN with either 80% or 50% water were injected in the 2 D conditions. Two peaks were monitored, one eluting in the early part of the gradient, at 8% co-solvent (m/z 184) and the other one eluting at 18% co-solvent (m/z 151), after the water peak, which eluted at 15% co-solvent (1.01 min).

The retention time of the early eluting peak increased linearly with the injected water volume (Figures 4a and4b). Nonetheless, the peak width was not affected by the content of water and the peak capacity was found to be higher when the diluent contained more water (Figure 4e). This phenomenon resulted from the higher retention of water molecules than of early eluting compounds on the polar stationary phase of SFC [START_REF] Batteau | Effect of the injection of water-containing diluents on band broadening in analytical supercritical fluid chromatography[END_REF]. As expected, the compounds eluting after water molecules were much more affected by the content of water in the diluent. The content of 80% water in the diluent (corresponding to the fractions eluting at the beginning of the 1 D gradient) did not produce any retention shift of the highly retained compound. However, the increase in volume had a detrimental effect on peak width, with peaks broadening when 10 µL were injected and peaks splitting when 20 µL were injected (Figure 4c). With a lower water content in the diluent (50%, corresponding to the composition at the end of the 1 D gradient), peaks were broadening in a more typical way, without any splitting (Figure 4d). Therefore, to maintain 70% of the peak capacity in SFC, whatever the variation of diluent of the fractions eluting from the 1 D gradient, it was decided to limit the injection volume in the second dimension to 10 µL. Because the peak widths had to be measured on EIC, the experimental peak capacity took into account the dispersion generated by the electrospray source. Under these injection conditions, the average peak width at 4σ was measured 5 times for the peak eluting before water, i.e., before 1.0 min at a value of 0.039 min and for 5 peaks eluting after water at a value of 0.050 min. Hence, the experimental peak capacity reached between dead time and 1.0 min was estimated at 22, while the experimental peak capacity in the second part of the SFC chromatogram was estimated at 16.

LC x SFC-HRMS analysis of the HTL wastewater

Applying the final conditions (Table 1), the wastewater from the HTL of Chlorella sorokiniana was analysed by off-line LC x SFC-HRMS. The total time could be divided into a 10 min 1 D separation and 80 times 2.35 min ( 2 D cycle time), yielding an overall analysis time of 3.3 h. The 80 resulting chromatograms were compiled in an in-house developed Matlab program to plot a 2D chromatogram (Figure 5). The final chromatographic profiles that were obtained in the range 100-1700 m/z in negative ionization mode (Figure 5a) and positive ionization mode (Figure 5b) were quite dissimilar, suggesting the interest in keeping both ionization modes. This was expected given the wide diversity of molecular structures present in algae wastewater, with N-containing compounds and O-containing compounds having very different ionization responses in the ESI source.

The peak broadening due to injection effects during transfer was confirmed by monitoring the peak width (expressed here as FWHM) of 4 isolated peaks detected in negative mode, noted 1 to 4 in Figure 5a. Peaks eluting in the left part of the contour plot, i.e., with 2 D retention time below 1.2 min (Peaks 1 and 2), exhibited low peak width whatever their position in 1 D, measured at 0.022 min and 0.023 min for peak 1 and peak 2, respectively. Peaks eluting in the first part of the 1 D LC ( 1 tR below 5 min, i.e., fraction 30) and with 2 tR over 1.2 min (bottom-right in Figure 5) were very broad due to the transfer of diluent with high water content. For instance, peak 3 exhibited a width of 0.036 min. Finally, the molecules retained in both LC and SFC (top-right in Figure 5) were well resolved, with peak 4 displaying a width of 0.024 min. Overall, it can be concluded that when fractions contained high amounts of water, only one-quarter of the elution space (bottom-right in Figure 5) exhibited broader peaks, for which the impact on peak capacity was limited to around 30% of loss compared to the other peaks.

The resulting 2D plots show that the developed LC x SFC method has large orthogonality, with the peaks spread over the entire separation space. The high orthogonality is caused by highly different retention mechanisms, as LC retention is based on dispersive and π-π interactions while SFC retention is based on dipole-dipole interaction. In negative mode, a downward diagonal was observed (Figure 5a), implying that the retention mechanisms are of opposing forces on the compounds responding in this ionization mode. However, the molecules responding in positive ionization mode (Figure 5b) were distributed evenly over the separation space. The orthogonality performances were calculated using the asterisk method [START_REF] Camenzuli | A new measure of orthogonality for multi-dimensional chromatography[END_REF], taking into account all detection modes. The results in Table 2 show that each dimension was well used, with peak spread over 90%. As already observed in Figure 5, there was no direct correlation between the two dimensions, as demonstrated by the absence of clustering along the upward diagonal. On the other hand, a certain degree of clustering could be detected along the downward diagonal, but its extent seemed quite low, with a 75% peak spread around this vector. According to the asterisk method, the overall occupation rate 𝛾 was 75%, while a convex hull method [START_REF] Nowik | Assessment of Two-Dimensional Separative Systems Using Nearest-Neighbor Distances Approach. Part 1: Orthogonality Aspects[END_REF], much simpler to implement, slightly overestimated this value to be 78%.

The predicted effective peak capacity was estimated (equation 8) as the theoretical peak capacity corrected by i) the external variances due to the injection effects in LC, 𝛽 𝐿𝐶 , and in SFC, 𝛽 𝑆𝐹𝐶 , both values being set at 70%, and ii) the overall occupation rate 𝛾. The predictive effective peak capacity was calculated at 1318. 𝑛 2𝐷,𝑝𝑟𝑒𝑑. = 𝑛 2𝐷,𝑡ℎ𝑒𝑜 * 𝛽 𝐿𝐶 * 𝛽 𝑆𝐹𝐶 * 𝛾 [START_REF] Sundarrajan | Insights into valuing the aqueous phase derived from hydrothermal liquefaction[END_REF] Taking into account experimental peak widths in both dimension, undersampling with 80 fractions, and orthogonality, the experimental effective peak capacity of this method was estimated at around 1050 for an overall analysis time of 200 min (3.3 h). The discrepancy between prediction and experimental value may come from the effect of external variances and the bias from LSS hypothesis on SFC separation. This experimental value was compared to one-dimensional techniques, either LC-HRMS or SFC-HRMS or to the two-dimensional online LCxLC-HRMS, all in optimized conditions. Both one-dimensional separations were carried out on 100 mm long column at a 1% gradient steepness to maximize peak capacity while limiting dilution. The best LC separation was achieved in 15 min (Figure S5a) and provided 122 in peak capacity whereas the best SFC separation was achieved in 30 min (Figure S5b) and provided 250 as experimental effective peak capacity. Online LCxLC was considered in two different configurations. The first one consisted in the use of fluorophenyl stationary phase in the first dimension while the second configuration used phenylhexyl in the first dimension. Both configurations used ACN and formic acid as 1 D mobile phase. The 2 D column was a Cortecs C18 with MeOH and ammonium acetate as 2D mobile phase. While these configurations were selected to maximize orthogonality, there was still a large correlation of the retention behaviour in both dimensions (Figure S6) as expected from RPLC x RPLC combinations. Peaks responding in ESI + occupied at best 45% of the available separation space (calculated on phenylhexyl x C18 combination), very similar to the occupation rate found by Lazzari et al. [START_REF] Lazzari | Comprehensive two-dimensional liquid chromatography-based quali-quantitative screening of aqueous phases from pyrolysis bio-oils[END_REF] for the analysis of aqueous phases using other LCxLC combination. Nonetheless, the effective peak capacity generated by LCxLC reached 1190 in 30 min, but required a specific instrumentation.

The developed LC x SFC-HRMS technique could be used for a non-target analysis to compare the wastewater composition resulting from different HTL processes or from different algae strains. However, this kind of study would require a large number of replicates to get statistically representative results due to the versatility of plant demography, in which case a faster online strategy could be preferred. On the other hand, a two-dimensional separation hyphenated to high-resolution mass spectrometry is of interest for target analysis by reducing matrix effects and improving separation of isomers. An off-line configuration offers a quick access, without dedicated instrumentation, to a highly discriminative analysis. Once the position of the target is elucidated using a comprehensive analysis, the separation can be restricted to elution zones of interest, and fraction analysis can be completed a posteriori if necessary. The interest of the LC x SFC method over a one-dimensional method can be illustrated through the monitoring of molecules whose content in the wastewater used for algae cultivation is suspected to have an impact on either the algae's growth or lipid yield. For example, Figure 6 shows the extracted chromatograms of ion m/z 247.1447. This ion corresponds to the molecular ion of 3-benzyl-6-isopropyl-2,5-piperazinedione. Piperazinediones are generated from the recombination of amino acids present in the algae during the hydrothermal liquefaction. They represent a beneficial source of nitrogen for algae cultivation on wastewater. However, after reaching the optimal growth, a prolonged N-starvation induces the production of starch and lipids for energy storage in the cells. If the N-starvation is prolonged, then lipids accumulate while the starch content decreases [START_REF] Nordin | Mechanism of carbon partitioning towards starch and triacylglycerol in Chlorella vulgaris under nitrogen stress through whole-transcriptome analysis[END_REF]. Consequently, the intake of piperazinediones should be strongly controlled during Nstarvation. The one-dimensional LC-HRMS analysis exhibited a cluster and two distinct peaks with a signal-to-noise ratio above 10 (Figure 6a), while the SFC-HRMS analysis retrieved two main peaks in a much noisier baseline (Figure 6b), that would not allow any quantitative analysis. The two-dimensional LCxLC analysis (Figure 6c) exhibited 9 spots with a signal-to-noise ratio over 10, demonstrating twodimensional separation reduced matrix effects that strongly inhibited MS signal when only one separation dimension was used. Nonetheless, a cluster was still present (around 10 min in the first dimension) with a diagonalization suggesting that despite a larger peak capacity, the peaks with the same m/z ratio were not fully resolved. The 2D LCxSFC chromatogram with its higher orthogonality compared to LCxLC (Figure 6d) exhibited 14 spots with a signal-to-noise ratio over 10 at this specific m/z ratio, offering a better characterization of these potential isomers or compounds with similar fragments.

Conclusion

In this study, an off-line 2D RPLC x SFC-HRMS method was developed and proved to be a powerful tool for the characterization of HTL wastewater of algae.

The wastewater was directly introduced into the first RPLC dimension without any sample treatment. Injection volume was maximized up to 30% of the column dead volume. Repeatability and intermediate precision of the 1 D separation were assessed. The theoretical estimation of fraction collection highlighted the importance of the number of fractions while underlining the limitation in peak capacity performances of an off-line approach. The 2 D SFC separation was optimized in terms of stationary phase, mobile phase, and hyphenation to HRMS. The comprehensive LC x SFC-HRMS analysis of HTL wastewater from Chlorella sorokiniana illustrated the large orthogonality of the 2D method, with a large occupation of the separation space. Using 80 fractions, the overall peak capacity reached 1000.

The developed off-line LC x SFC-HRMS method could be used for non-target analysis but also to facilitate target analysis thanks to the ability to decrease matrix effects and separate the numerous isomers present in HTL wastewater. 5); b) plot of the peak capacity as a function of gradient time, for a column with geometry 4.6 x 150 mm, 5 µm, with a gradient running from 1 % to 40 % B, using a flow rate at its optimal value (0.7 mL/min, red marks) or maximal value (2.8 mL/min, black marks), with 80 fractions collected. Left axis is the theoretical peak capacity (dashed lines) and right axis is the effective peak capacity after sampling ns (plain lines).

Figure 1 .

 1 Figure 1. Theoretical performances of the LC dimension. a) plot of the remaining peak capacity vs. the ratio of the number of fractions over theoretical peak capacity 𝑛 𝑓 𝑛 𝑡ℎ (Equation 5); b) plot of the peak

Figure 2 .

 2 Figure 2. Effect of the injection volume on the RPLC separation of the aqueous phase (a) Overlaid chromatograms with injection volumes of 100 μL (red) and 500 μL (black). (b) Peak capacity loss as a function of injection volume for three different peaks (shown in the top figure). Peak width measured at half peak height. Flow rate: 2.8 mL/min; gradient time: 10 min. UV-detection at 254 nm. Other conditions given inTable 1 (1D conditions).

Figure 3 .Figure 4 .

 34 Figure 3. Theoretical performances of the LC x SFC separation, expressed by a plot of the total time as a function of the 2D theoretical peak capacity. LC flow rate 2.8 mL/min, LC gradient range 3 to 60 min. Fraction number: 60, 80 and 100 (white, light colored and deep colored marks, respectively). SFC gradient time 1 min (blue), 2 min (orange), 5 min (grey) and 10 min (green). SFC column length 50 mm (circle marks) and 100 mm (triangle marks)

Figure 5 .

 5 Figure 5. Base peak chromatograms from the LC x SFC-ESI-HRMS analysis of the wastewater sample, using ESI source in a) negative mode and b) positive mode. Peaks noted 1 to 4 are the monitored isolated peaks spread over the separation space. The experimental conditions are in Table1.

Figure 6 .

 6 Figure 6. Extracted ion chromatograms at m/z 247.1447 for the analysis of Chlorella sorokiniana HTL wastewater using a) LC-HRMS, b) SFC-HRMS, c) LCxLC-HRMS and d) LCxSFC -HRMS

Figure

  Figure S4 Base peak chromatogram of the aqueous phase of HTL from Sorokiniana Algae, SFC-HRMS obtained with an electrospray source in a) positive mode, b) negative mode, c) UV at 254 nm. Diol column (3 x 50 mm; 1.7 µm), gradient 2% to 50 % EtOH in 1.34 min, 1.8 mL/min, other conditions in Table 1

Figure

  Figure S5 Base peak chromatogram (ESI+) of the aqueous phase of HTL from Sorokiniana Algae using a) LC phenyl hexyl column, isocratic step at 1 % ACN + 0.1 % formic acid for 1.5 min, then gradient 1%-30% ACN + 0.1 % formic acid in 15 min, 0.5 mL/min, column geometry 2.1 x 100 mm; 1.7 µm, sample diluted by 10 in water, Vinj 10 µL. b) SFC Diol column gradient 2% to 50 % EtOH in 30 min, 1.0 mL/min, column geometry 3 x 100 mm; 1.7 µm, BPR 140 bars, 40°C. sample diluted by 3 in ACN/H2O 50/50. Vinj 7 µL. HRMS conditions as in Table1.

Figure S6 .

 S6 Figure S6. Online LCxLC -Base peak chromatogram of the aqueous phase of HTL from Chlorella sorokiniana. First dimension: column geometry 100 x 2.1 mm, 1.7 µm. 30°C, A: water and B: ACN, gradient 1% to 31 % B in 30 min, flow rate 150 µL/min, Vinj 5 µL. Second dimension: column geometry 30 x 2.1 mm, 1.6 µm. 40°C, A: water + 10 mM ammonium acetate and B: MeOH + 10 mM ammonium acetate, gradient 20% to 100% B in 0.21 min, flow rate 2600 µL/min, Vinj 16 µL. Split between first and second dimension by 3. Detection as in Table 1

Acknowledgments

This work was funded by the French national agency for research (RafBioAlg ANR-18-CE43-0009). The authors thank Adriana Ramirez and Dr. Florien Delrue from CEA Cadarache (CTREG/DPACA) for the cultivation of Chlorella sorokiniana algae and for interesting discussions about their growth behavior in HTL wastewater. The authors also thank Dr. Anne Roubaud and Lucie Matricon from CEA Grenoble (Biomass Technology Lab, LITEN) for providing wastewater from HTL-processed algae.


Quentin Guilloteau

Olivier Richard

Raphaël Bleuse

Eric Rutten

Folding a Cluster containing a Distributed File-System

The development, study, and experimental validation of cluster and grid middlewares are combersome, time and resource consuming. Researchers need to deploy their middlewares on multiple nodes to test and evaluate their modifications. But these middlewares are destine to be used in production with hundreds or thousands of machines, and deploying at this scale to perform tests and evaluations is too costly. In this paper, we investigate folding techniques to represent a full scale cluster on less physical machines. We evaluate the performance of a distributed I/O benchmark application with respect to the level of folding to determine guidelines for future folded experiments containing a distributed file-system.

Introduction

The increase of computing demands from scientists from all fields led to the development of computing cluster and grid architectures. And with these new architectures came new challenges. Due to their high prices, clusters are often shared among several research laboratories and teams. This sharing motivates the development of middlewares such as batch schedulers that are responsible to assign the user jobs to physical machines, manage the state of the resources, deal with reservation, etc. Such cluster, or grid, middlewares are complex applications of great research interest, and they must be tested before reaching production. However, they are usually destine to operate in an environment of hundreds or thousands of machines. Deploying a full scale environment is too costly to perform simple tests and very specific evaluations.

We thus want to answer the following question: Can we reduce the number of physical machines needed to perform a full scale experiment while keeping a similar behavior as the full scale system?

One solution to reduce the number of machines used for experiments would be to use simulation techniques. The system, the applications, and the middlewares are modeled and can then be executed on a single node. Beside reducing the number of machines used, simulators also reduce the execution time of the experiments. In the context of distributed systems and applications, projects such as Simgrid [START_REF] Casanova | Versatile, Scalable, and Accurate Simulation of Distributed Applications and Platforms[END_REF] and Batsim [START_REF] Dutot | Batsim: a Realistic Language-Independent Resources and Jobs Management Systems Simulator[END_REF] are leading the way. One drawback of simulation is that the real middleware is not being executed, or not fully executed, but instead, a partial or modelled version in a modelled environment. In the case of cluster and grid middlewares, the applications are often far too complex to model them fully correctly.

Another approach is to fold the experiment by deploying more "virtual" resources on physical machines. In the case of a Resources and Jobs Management Systems (RJMS), like Slurm [START_REF] Yoo | SLURM: Simple Linux Utility for Resource Management[END_REF] or OAR [START_REF] Capit | A batch scheduler with high level components[END_REF], it is possible to define several resources, from the point of view of the RJMS, on a single machine, which would be completely transparent for the users. For example, one can deploy a 1000-node virtual cluster on 10 physical nodes by defining 100 virtual resources on each node. The advantage is that the experiment takes place on the real system (CPU, network, disk, etc.) and with the real middleware code. The drawback is that this folding can introduce noise in the experiment and degrade performance.

In this paper we evaluate the performance of a distributed I/O benchmark when we fold a computing cluster onto itself, and quantify the impact of folding a computing cluster containing a file system. We then give a rule of thumb for choosing the amount of folding for distributed experiences containing a distributed file-system.

Section 2 presents a motivating example for this study. Section 3 defines notions and concepts. We present the experimental protocol in Section 4 and perform the evaluation in Section 5.

Motivating Example: the CiGri middleware

CiGri [START_REF] Georgiou | Evaluations of the lightweight grid cigri upon the grid5000 platform[END_REF] is a grid middleware in production at the French Gricad meso-center1 . The goal of CiGri is to use the idle resources of the meso-center. It interacts with several clusters managed by OAR [START_REF] Capit | A batch scheduler with high level components[END_REF] batch schedulers.

Users of CiGri submit Bag-of-Tasks applications to the middleware. Such applications are composed of thousands of short, independent and similar tasks are classified as embarrassingly parallel which make them a good candidate for "filling the holes" in the cluster schedules. Monte-Carlo simulations or parameter sweeps are examples of Bag-of-Tasks applications.

Once the set of tasks submitted to CiGri, the middleware will submit sub sets of jobs to the different schedulers of the grid. The jobs are submitted with the lowest priority in order to allow premium users of the clusters to get the resources used by CiGri jobs if needed.

Figure 1 depicts the interactions between CiGri and different clusters of the grid. One problem of CiGri is its submission algorithm. CiGri will submit a batch of jobs the one scheduler, and wait for the completion of the batch to submit again. This strategy can lead to an underutilization of the cluster resources. Moreover, one objective of CiGri is to harvest in a non-intrusive fashion. Meaning that the premium users of the different clusters must not notice the impact of the CiGri jobs on the platform. However, once executing, the CiGri jobs are using the shared resources of the cluster (file-system, network, etc.), which can have an impact on the performance of every user jobs.

In the context of research projects requiring to modify CiGri to control its job submission based on the cluster file-system load [START_REF] Guilloteau | Model-free control for resource harvesting in computing grids[END_REF][START_REF] Guilloteau | Collecte de ressources libres dans une grille en préservant le système de fichiers : une approche autonomique[END_REF][START_REF] Guilloteau | Controlling the Injection of Best-Effort Tasks to Harvest Idle Computing Grid Resources[END_REF][START_REF] Stahl | Towards a control-theory approach for minimizing unused grid resources[END_REF][START_REF] Gabriel | A control-theory approach for cluster autonomic management: maximizing usage while avoiding overload[END_REF], we must deploy and perform experimentation evaluation of a modified version of CiGri on a realistic environment. It is unreasonable to deploy on the entire Gricad meso-center, or using simulation due to the complex software stack (CiGri + OAR + users jobs). We are thus interested in folding strategies to perform the evaluation of our CiGri modifications.

A study of the CiGri jobs running on the Gricad platform [START_REF] Guilloteau | Étude des applications Bag-of-Tasks du méso-centre Gricad[END_REF] gives a statistical description of the execution times of those jobs. This allows us to use a sleep model to represent the execution times. As sleep calls are extremely lightweight for a CPU to deal with, we are able to fold several virtual resources onto one physical resource. However, as explained earlier, no realistic job only does computation without reading and/or writing data. We can extend the previous job model by adding I/O operations before or after the sleep with the dd command. This addition makes it less obvious how much we can afford to fold. The questions that we want to answer are thus the following:

1. What is the minimum number of machines we need to deploy to emulate a full scale cluster with this job model while keeping the same performance in I/O?

2. Is there a trade-off between the number of physical resources used and the overhead of performances due to the folding?

Definitions & Concepts

We define the folding of a deployment as the action of defining several "virtual resources" on a physical resource.

A physical resource is a node from a cluster, and a virtual resource represents a resource on the full scale system from the point of view of the RJMS. We also define the folding factor (f f old ) as the division of the number of physical resources in the deployment divided by the number of virtual resources. Intuitively, it represents the number of virtual resources for each physical resource:

f f old = #resource virtual #resource physical ∈ [1, +∞[ (1)
Figure 2 depicts an example of folding a deployment.

Methodology

We aim at evaluating the variations in performance of a distributed application using a distributed file-system for different value of folding factors (f f old ). As explained in Section 2, the job model that we are using is a sleep time to represent the CPU bound phase, and a dd operation to represent the I/O phase. The sleep operation allows us to fold the CPU bound phase on the same machine easily without noise. We thus focus on the performance of the I/O operations in a folded deployment.

Experimental Setup

The following experiments were carried on the gros cluster, located in Nancy, of the Grid'5000 [START_REF] Balouek | Adding Virtualization Capabilities to the Grid'5000 Testbed[END_REF] French test bed. The machines of this cluster have an Intel Xeon Gold 5220 CPU with 18 cores, 96 GiB of memory, a 2 x 25 Gbps (SR-IOV) network and a 480 GB SSD SATA Micron MTFDDAK480TDN disk. The reproducibility of the deployed environment was ensured by NixOS Compose [START_REF] Guilloteau | Painless Transposition of Reproducible Distributed Environments with NixOS Compose[END_REF].

Benchmark application

To evaluate the performance of the cluster file system, we chose to use the IOR [START_REF]IOR Benchmark[END_REF] benchmark. IOR is a MPI application to benchmark I/O performances. It is the most popular benchmark among research on I/O in HPC [START_REF] Zanon | A checkpoint of research on parallel i/o for high-performance computing[END_REF], and is also used in the context of the IO500 list [16]. IOR has a multitude of parameters, but give the main ones here. We used the POSIX protocol (api=POSIX), a transfer size (transferSize) of 1Mbytes and a segment count of 1 (segmentCount). We assigned a single file per IOR process (filePerProc), and checked the correct writing and reading afterwards (checkWrite and checkRead). The number of tasks (numTasks), which represent the number of IOR processes, and the block size (blockSize), which is the total size of the file to read/write in this case, are parameters of the following experiments. We used OpenMPI with the TCP backend.

Distributed File-Systems

For the chosen distributed file-systems, we used NFS and OrangeFS.

NFS (v4)

NFS [START_REF] Pawlowski | The NFS version 4 protocol[END_REF] is a popular distributed file-system for small clusters. There is only one server. Clients mount the filesystem and can perform POSIX operations. Figure 3a depicts the simplified architecture of a distributed file system like NFS. All the clients query the same I/O node for their files. NFS servers do have several workers that can manage the requests concurrently. The NFS export options used are: *(rw,no_subtree_check,fsid=0,no_root_squash). The NFS server runs under the default configuration (8 workers).

OrangeFS

OrangeFS [START_REF] Moore | OrangeFS: Advancing PVFS[END_REF] (or PVFS2) is a parallel file-system. This means that there are several servers (also called I/O nodes) to answer the requests of the clients. Figure 3b depicts the simplified architecture of a parallel file-system like OrangeFS. The clients query a I/O node of the file system. If one stripe asked by the client is not present on the query I/O nodes, the file system indicate on which I/O node to find it. We used the default configuration recommended by the OrangeFS installer (the I/O nodes host both the metadata and the storage). Moreover, in our system (Section 2), we do not consider the problem of metadata.

Number of I/O nodes

In the case of PFS, like OrangeFS, we did not find any methodology nor "rule of thumb" to define the number of I/O nodes for a computing cluster. In the following, we will consider OrangeFS file-systems with 1, 2, or 4 I/O nodes. Note that in the case of NFS there is only one I/O node.

I/O load

We consider 5 different sizes of I/O operations to perform, both in writing and reading: 1Mbytes, 10Mbytes, 100Mbytes, 500Mbytes, and 1Gbytes. These file sizes will be the values of the IOR blockSize option.

Number of CPU nodes

As a first step, and because we aim at emulating clusters from regional meso-centers, we will thus consider small clusters of 8, 16, 24, and 32 nodes. These number of nodes will be the values of the IOR numTasks option: one tasks per node of the cluster.

Protocol

Let us consider a system with N CPU nodes. We first deploy the system at full scale, i.e., N machines and the I/O nodes of the file system. As IOR is an MPI application, we compute the hostfile with one slots per compute node. We then start the IOR benchmark, that we repeat 5 times (the IOR repetitions option), and gather the performance reports. We remove one compute node from the hostfile and recompute the slots for the remaining nodes to keep the number of processes (numTasks) constant. This protocol is then repeated for all the different variations: number of CPU, size of I/O operations, type of file system, number of I/O nodes.

Figure 4 shows a visual representation of the experimental protocol for an experiment with 4 CPU nodes.

Evaluation

In this Section, we present the results of the experiments presented in the previous section. We first consider the file-system of the cluster to be NFS in Section 5.1, and then OrangeFS in Section 5.2.

Evaluation of NFS

Figure 5 shows the results of the experiments when using NFS. We also plot the 95% confidence intervals. Note that we did not remove the outliers. We notice that the writing performances (bottom row) does not seem to be affected by the folding of the deployment as it remains flat. However, for the reading performances (top row), we can see that the reading times increasing for higher folding factor. This means that the more we fold, the more we degrade the reading performances.

Take away # 1

Write operations on NFS do not seem to be affected by the folding of the deployment. On the other hand, read operation performances degrade the more the cluster is folded with a quadratic behavior.

We modeled the reading performances based on the size of the file to read and the number of CPU nodes involved. Figure 6 shows the results of a linear regression between the reading time and the folding ratio, file size and number of CPU nodes involved. We fitted a model with the following form:

t read (f f old , f size ) ≃ α + β 1 f f old 2 + β 2 f size + γf f old 2 f size (2)
with α, β 1 , β 2 , γ ∈ R the coefficients of the model. The R 2 of the fitting is 0.9982. Figure 6 shows the fitting of the model on the NFS data. The bottom row represents the same information as the top one, but in log scale. We can see that the model fits all the file sizes but for 1M. We believe that the variations in performance for the 1M files are due to noise.

We are interested in knowing the maximum folding factor (f f old ) possible for a desired file size (f size ). Let p > 1 be the percentage of increased reading time compared to the full scale deployment containing nb cpu CPU nodes. By using the definition of the model for t read in Equation 2, we get: We start with a full scale deployment, i.e., one MPI process (viewed as a virtual resource) per physical node (Figure 4a), and remove from the hostfile the physical nodes one by one while keeping the number of MPI processes (i.e., the number of virtual resources) constant. We recompute the number of slots per node to balance the processes (Figures 4b and4c). The experiment stops when there is no more node to remove (i.e., after Figure 4d). for experiments with different cluster size (i.e., number of CPU nodes) and different sizes of file to read and write (point shape). We observe that the writing performances are not affected by the folding, but that the reading ones are, and that the degradation has quadratic growth with respect to the folding factor. Model of the mean reading time for the folding factor and block size Figure 6: Linear regression modeling the reading times (y-axis) and the folding factor (x-axis), file size (point shape). The top row shows the fitting of the model on the data, and the bottom row the same data but in log scale. We can see that the model fits correctly the data for file sizes greater than 10M. 1M files does not seem to be affected by the folding, and their variation in performance seem to be due to noise. 7a shows the maximum folding factor to use to have a desired overhead on the reading times on NFS base on the file size. Figure 7b shows the distribution of the number of read requests per size on ANL-Theta. Evolution of the writing times based on the folding for expes with different numbers of IOR MPI processes

t read (f, f size ) t read (nb cpu , f size ) < p =⇒ f < p × t read (nb cpu , f size ) -(α + β 2 f size ) β 1 + γf size (3)
-1 0 0 1 0 0 -1 K 1 K -1 0 K 1 0 K -1 0 0 K 1 0 0 K -1 M 1 M -4 M 4 M -1 0 M 1 0 M -1 0 0 M 1 0 0 M -1 G 1 G -P

Take away # 2

For files of size 10M, folding 10 resources onto a single physical resource leads to a degradation of 5%. To reach the same degradation for file size of 100M or 1G, the maximum folding factor would be 5. (Figure 7a)

From the model defined previously, Figure 7a, and the Darshan [START_REF] Carns | Understanding and improving computational science storage access through continuous characterization[END_REF][START_REF] Carns | 24/7 characterization of petascale I/O workloads[END_REF] logs from ANL-Theta between 2019 and 2022 (Figure 7b)2 , we can have an estimation of the overhead if we decided to rerun these Darshan logs (on NFS and with the job model considered in Section 2) with different folding factors. For example, a folding factor of 10 would lead to an overhead of 64 hours over 4 years (i.e., an increase of 0.2%), while requiring 10 times fewer machines.

Evaluation of OrangeFS

In the case of OrangeFS, there is an extra dimension to explore: the number of I/O nodes in the file-system.

Figures 8 and9 show respectively the evolution of the performance in reading and writing time of the IOR benchmark for different number of CPU nodes in the cluster and I/O nodes in the file-system, as well as different sizes of file to read/write.

We can see on Figure 8 that contrary to NFS, there is a significant loss of performance for the write operations when increasing the folding factor. This loss of performance appears more significant when there are more I/O nodes in the file-system.

Concerning the reading performances (Figure 9), we observe the same behavior as for NFS. High folding factors lead to an increase of reading time. The increase appears greater when there are more I/O nodes in the file-system. The start of this increase seems to depend on the number of CPU nodes. The more CPU nodes, the later the increase starts. It is interesting to note that the variation when measuring the reading time during the experiments is smaller (thus more stable) than for NFS.

Take away # 3

The reading performance of a fully folded deployment does not depend on the number of I/O nodes in OrangeFS (Figure 9).

For both reading and writing performances there seem to be two behaviors. First a phase where the folding does not affect the performances, and then, from a given folding factor, the reading/writing times grow linearly. As we want to know the maximum folding factor until which the folded system behaves like a full scale system, we are now interested in finding this breakpoint where the behavior changes. Using segmented regression techniques [START_REF] Vito | Estimating regression models with unknown break-points[END_REF], we found a model that we simplified to make it a rule of thumb:

f break,r ≃ 1 + 0.3 × nb cpu -0.5 × nb io f break,w ≃ 2 + 0.3 × nb cpu -0.5 × nb io (4)
where, f break,r and f break,w are respectivelly the folding factor of the breakpoint for the reading and writing performances, nb cpu and nb io are the number of CPU and I/O nodes in the system.

Take away # 4

For writing and reading times on OrangeFS, there is a breakpoint in performance before which there is no degradation. We modeled rules of thumb to them in Equation 4. These rules of thumb are depicted in Figure 10.

As f break,w (Equation 4) will always be greater than f break,r , the overall breaking point in performance for OrangeFS is f break,r .

Conclusion

In this paper, we investigated the use of folding techniques to reduce the number of deployed machines for the large scale evaluation of applications such as grid and cluster middlewares. We have seen that folding requires a change in the job model We focused on the impact on the performances of the file-system. To do so, we took a distributed I/O benchmark, IOR, and ran it for several cluster sizes, I/O loads and distributed file-systems. We analyzed the results of the benchmark and reached to the following conclusions:

• Write operations on NFS are not subject to folding (Take away 1).

• The performance of read operations on NFS can be modeled with a quadratic relation, and this model can be used to determine the maximum folding for an accepted degradation (Take away 2).

• The performance of read operations in a fully folded cluster with OrangeFS do not depend on the number of I/O nodes (Take away 3).

• There are breaking point in reading and writing performance for OrangeFS when the fold the cluster. Equation 4gives rules of thumb to estimate these breakpoints (Take away 4).

This study presents some limitations. The studied file-systems are not among the most popular in large HPC centers. It would be interesting to consider file-systems such as Lustre [START_REF]Lustre File-System[END_REF] of BeeGFS [START_REF]BeeGFS File-System[END_REF]. The reasons of this loss of performance due to folding are still unclear. The main suspect is the network. We did observe a speed similar (23 Gbps) to the one advertised on the NIC (25 Gbps). We think that the overhead can be due to the TCP protocol. Investigating different network protocol like InfinyBand or OmniPath would be interesting.

Figure 1 :

 1 Figure 1: Interactions between CiGri and the schedulers OAR of the computing grid. CiGri users submit Bag-of-Tasks applications, whose jobs are then submitted to the different cluster schedulers of the computing grid. The computing clusters are shared with users with more priority, thus CiGri jobs must be killed if one premium user requires the resources.

  Folding with a factor of 1 (4 physical resources and 4 virtual resources).

  Folding with a factor of 2 (2 physical resources and 4 virtual resources).

  Folding with a factor of 4 (1 physical resource and 4 virtual resources).

Figure 2 :

 2 Figure 2: Example of folding a deployment for a system with 4 resources. Figure 2a depicts the system deployed at full scale. Figure 2c represent the system completely folded. And Figure 2b shows an intermediate folded deployment.

  Distributed File system like NFS. Multiple clients query a single I/O node that process all the requests. File System I/O ops. IO node IO node (b) Parallel File system like OrangeFS. Multiple clients query in parallel all the I/O nodes.

Figure 4 :

 4 Figure4: Graphical representation of the experimental protocol presented in Section 4. We start with a full scale deployment, i.e., one MPI process (viewed as a virtual resource) per physical node (Figure4a), and remove from the hostfile the physical nodes one by one while keeping the number of MPI processes (i.e., the number of virtual resources) constant. We recompute the number of slots per node to balance the processes (Figures4b and 4c). The experiment stops when there is no more node to remove (i.e., after Figure4d).

  r/w times based on the folding ratio for experiments with different number of CPU nodes

Figure 5 :

 5 Figure 5: Evolution of the reading (top row) and writing (bottom row) times based on the folding factor (x-axis)for experiments with different cluster size (i.e., number of CPU nodes) and different sizes of file to read and write (point shape). We observe that the writing performances are not affected by the folding, but that the reading ones are, and that the degradation has quadratic growth with respect to the folding factor.

  compared to the full scaled deployment Maximum number of processes per machine Size of the file to read 10M 100M 1G (a) Maximum folding factor (f f old ) based on the accepted degradation of the reading time compared to the full scale deployment and to the size of the file to read.

  in bytes Proportion of all Read operations (b) Proportion of the size of the read operations on ANL-Theta between 2019 and 2022. The majority is smaller than 100K. These values are extracted from Darshan[START_REF] Carns | Understanding and improving computational science storage access through continuous characterization[END_REF][START_REF] Carns | 24/7 characterization of petascale I/O workloads[END_REF] logs.

Figure 7 :

 7 Figure7: Figure7ashows the maximum folding factor to use to have a desired overhead on the reading times on NFS base on the file size. Figure7bshows the distribution of the number of read requests per size on ANL-Theta.

Figure 8 :

 8 Figure 8: Evolution of the writing times with OrangeFS based on the folding factor (f f old ) for experiments with different number of CPU and I/O nodes and different sizes of file to write.

  Figure Evolution of the reading times with OrangeFS based on the folding factor (f f old ) for experiments with different number of CPU and IO nodes and different sizes of file to read.

Figure 10 :

 10 Figure 10: Model of the breaking point in behavior of performance in reading (left) and writing (right) for 32 CPU nodes (nb cpu ) and 4 I/O nodes (nb io ). The model (dashed line) comes from Equation 4.

https://gricad.univ-grenoble-alpes.fr/index_en.html

This data was generated from resources of the Argonne Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC02-06CH11357.

Acknowledgments

Experiments presented in this paper were carried out using the Grid'5000 testbed, supported by a scientific interest group hosted by Inria and including CNRS, RENATER and several Universities as well as other organizations (see https://www.grid5000.fr).


PhD Mathieu Marillier
email: mmarillier@chu-grenoble.fr

Mathieu Gruet

Anne-Catherine Bernard

Samuel Verges

Onofre Moran-Mendoza

J Alberto Neder

The relationship between perceived and performance fatigability in severe fibrotic interstitial lung disease: a prospective, crosssectional study

Keywords: exercise test, hypoxia, muscle fatigue, oxygen, pulmonary fibrosis

Take-home message: Our study suggests that interventions geared to improve peripheral factors of performance fatigability during exercise in interstitial lung disease may prove valuable to decrease patients' perceived fatigability, since both seem closely related.

Dear Editor,

A sizeable fraction (~70%) of patients with fibrotic interstitial lung disease (f-ILD) suffer from and seek medical advice due to debilitating symptoms of fatigue. [START_REF] Guenther | The European IPF registry (eurIPFreg): baseline characteristics and survival of patients with idiopathic pulmonary fibrosis[END_REF] Broadly, fatigue can be defined as an unrelenting feeling of tiredness or lack of energy that is not improved by rest. Fatigue is, however, a complex symptom thus remaining poorly understood and appropriately treated in f-ILD. [START_REF] Kahlmann | Managing Fatigue in Patients With Interstitial Lung Disease[END_REF] Fatigue is thought to originate from predisposing, precipitating, and perpetuating factors that are not necessarily specific to f-ILD, although most factors may be applicable to the disease (as reviewed in [START_REF] Kahlmann | Managing Fatigue in Patients With Interstitial Lung Disease[END_REF]). In this letter, we reason using the recent and renowned framework for fatigue offered by Enoka and Duchateau [START_REF] Enoka | Translating Fatigue to Human Performance[END_REF] in which "perceived" and "performance" fatigability interact to limit one's physical (and cognitive) functioning.

Perceived fatigability refers to an individual's sensations about fatigue. [START_REF] Gruet | Fatigue in Chronic Respiratory Diseases: Theoretical Framework and Implications For Real-Life Performance and Rehabilitation[END_REF] Of note, it can be assessed as a trait characteristic (i.e. the amount of fatigue faced by patients over a prolonged period of time) or as a state level (i.e. the amount of fatigue faced at a specific moment in time, such as physical tasks). [START_REF] Gruet | Fatigue in Chronic Respiratory Diseases: Theoretical Framework and Implications For Real-Life Performance and Rehabilitation[END_REF] Performance fatigability corresponds to the objective changes in performance relative to the requirements brought by a specific task. This component depends on the ability of the neuromuscular system to meet such specific requirements and may be divided in central (e.g. voluntary activation) and peripheral (e.g. tissue O 2 delivery) factors. [START_REF] Gruet | Fatigue in Chronic Respiratory Diseases: Theoretical Framework and Implications For Real-Life Performance and Rehabilitation[END_REF] It is, however, unknown whether sensations of fatigue relate, at least in part, to alterations in neuromuscular mechanisms in patients with severe f-ILD. In this context, we aimed to offer a comprehensive assessment of fatigue in patients with f-ILD comprising both perceived (trait/state level) and performance fatigability and assess relationships between these two components.

Sixteen patients (13 men, 73±8 years, 10 with idiopathic pulmonary fibrosis, total lung capacity= 63±15% predicted, lung transfer for carbon monoxide= 42±10% predicted) were recruited from the ILD clinic (Division of Respirology, Kingston, Canada) between November 2018 and February 2020. They first performed an incremental (typically 5-15 W increment every 2 min [START_REF] Marillier | Quantifying leg muscle deoxygenation during incremental cycling in hypoxemic patients with fibrotic interstitial lung disease[END_REF][START_REF] Marillier | Influence of exertional hypoxemia on cerebral oxygenation in fibrotic interstitial lung disease[END_REF]) exercise test on a bicycle ergometer to determine peak work rate (WR peak ). All patients had severe exertional hypoxaemia (room air nadir of O 2 saturation by pulse oximetry <88% [START_REF] Plessis | Exertional hypoxemia is more severe in fibrotic interstitial lung disease than in COPD[END_REF]). On a different day, they performed a constant-load, bicycle exercise test (60% WR peak ) to symptom limitation. [START_REF] Marillier | Oxygen supplementation during exercise improves leg muscle fatigue in chronic fibrotic interstitial lung disease[END_REF] Magnetic femoral nerve stimulation assessed quadriceps neuromuscular fatigability. [START_REF] Marillier | Oxygen supplementation during exercise improves leg muscle fatigue in chronic fibrotic interstitial lung disease[END_REF] Peripheral and central components of performance fatigability were measured as the pre-to post-exercise reduction in the potentiated quadriceps twitch amplitude (∆Tw) and voluntary activation (∆VA, interpolated twitch technique [START_REF] Merton | Voluntary strength and fatigue[END_REF]), respectively. [START_REF] Marillier | Neuromuscular Dysfunction and Cortical Impairment in Sleep Apnea Syndrome[END_REF] ∆Tw and ∆VA were also expressed as a function of the external work (kJ= work rate (W) × exercise time (s)/1000) performed during constant-load exercise test. The state level of perceived fatigability was assessed using the Rating-of-Fatigue (ROF) scale, a valid tool to measure the intensity of fatigue in various contexts, such as whole-body exercise. [START_REF] Micklewright | Development and Validity of the Rating-of-Fatigue Scale[END_REF] Perceived fatigability as a trait was assessed with the Fatigue Severity Scale (FSS) [START_REF] Krupp | The fatigue severity scale. Application to patients with multiple sclerosis and systemic lupus erythematosus[END_REF], a tool validated in a large sample to assess and quantify this outcome for clinical and research purposes [START_REF] Valko | Validation of the fatigue severity scale in a Swiss cohort[END_REF] and previously used in f-ILD [START_REF] Keyser | Changes in fatigability following intense aerobic exercise training in patients with interstitial lung disease[END_REF].

Patients were then discriminated according to their FSS (trait) and ROF (state) scores as "less fatigued" or "more fatigued", using a cut-off of 38 points (FSS) and 0.14 units/kJ (ROF). These cut-off scores corresponded to the median sample score for both scales. This prospective study was approved by the Queen's University (Kingston, Canada)

Research Ethics Board (DMED-2150-18). Written informed consent was obtained from each participant prior to their participation.

Moderate and severe perceived fatigability as a trait (FSS) was reported by 12/16 (75%) and 6/16 (38%) of patients with f-ILD, respectively [START_REF] Lerdal | Fatigue Severity Scale. Encyclopedia of Quality of Life and Well-Being Research[END_REF]; specifically, less fatigued patients had FSS scores of 29±10 and more fatigued patients had a score of 48±5 points (p<0.001). On average, patients exercised at 43±13 W for 17.0±4.0 min, leading to 44.4±19.0 kJ of work (with no difference according to FSS subgroups). Indices of performance fatigability [Δ maximal voluntary contraction (MVC), ΔTw and ΔVA, both absolute and work corrected values] did not differ between FSS subgroups nor correlated with FFS scores (p>0.05). Similarly, perceived fatigability as a state (i.e. on exercise: ROF scores, both absolute and work corrected) did not differ between less vs more fatigued patients: 6.3±2.1 vs 6.0±1.7 ROF units, and 0.20±0.12 vs 0.14±0.06 ROF units/kJ at exercise cessation, respectively (both p>0.05); there was no correlation between trait and state levels of perceived fatigability (p>0.05).

Considering perceived fatigability as a state, less and more fatigued patients had ROF scores of 5.6±1.9 vs 6.6±1.8 ROF units (p>0.05) and 0.10±0.03 vs 0.24±0.08 ROF units/kJ (p=0.001) at exercise cessation, respectively. This was due to less external work performed in more fatigued patients: 29.7±12.1 vs 59.0±11.6 kJ (p<0.001). When ROF scores were work adjusted, Δ Tw did not differ between less and more fatigued patients: -21.3±10.2 vs -20.9±7.6% drop from baseline, respectively (p>0.05). However, ΔTw/work was larger in more fatigued patients: -0.72±0.13 vs -0.38±0.20%/kJ (p=0.001; Fatigue assessment in f-ILD has so far been restricted to a trait characteristic. [START_REF] Kahlmann | Managing Fatigue in Patients With Interstitial Lung Disease[END_REF] The present study is innovating because it offers a thorough assessment of fatigue, comprising both components of perceived (trait and state levels) fatigability and directly measured performance fatigability. We found that i) trait level of perceived fatigability was not linked to performance fatigability but, in contrast, ii) state level of perceived fatigability and performance fatigability elicited by exercise significantly correlated when corrected external work performed in f-ILD. Our approach seems particularly relevant when considering the multidimensional facets of a complex symptom, which can hardly be captured by a single measure.

Paneroni and colleagues recently found a similar mismatch between trait level of perceived fatigability and performance fatigability on exercise in severe to end-stage chronic obstructive pulmonary disease. [START_REF] Paneroni | Relationship between perceived and neuromuscular fatigue in COPD patients with chronic respiratory failure with long-term oxygen therapy: a cross-sectional study[END_REF] They, in fact, reported that FSS was inversely related to forced expiratory volume in 1 sec, that is, perceived fatigability was associated with the severity of airflow obstruction. Applying such reasoning, we found that FSS inversely correlated with forced vital capacity (r= -0.52; p=0.038), suggesting that greater perceived fatigability was associated with the severity of lung restriction in our sample. However, perceived fatigability did not correlate with other relevant variables such as exercise tolerance (peak O 2 uptake or work rate) or the extent of exertional hypoxemia (p>0.05). Owing to the fact that fatigue in f-ILD is multifactorial, its management may likely require a comprehensive approach comprising, for instance, optimised treatment of the underlying disease and other associated comorbidities (e.g. sleep apnea or obesity) or supportive measures (e.g. cognitive behavioral therapy) [START_REF] Kahlmann | Managing Fatigue in Patients With Interstitial Lung Disease[END_REF].

Importantly, perceived fatigability and the peripheral component of performance fatigability during exercise were strongly associated in f-ILD. In this context, acute and chronic interventions aiming at lessening peripheral impairments underlying fatigability might translate into beneficial effects on perceived fatigability during physical exertion in this population. Specifically, in such patients with severe exerciserelated hypoxemia, acute strategies targeting enhancement of muscle O 2 delivery (e.g. supplemental O 2 [START_REF] Marillier | Oxygen supplementation during exercise improves leg muscle fatigue in chronic fibrotic interstitial lung disease[END_REF] or nitrate intake [START_REF] Zamani | The benefits and risks of beetroot juice consumption: a systematic review[END_REF]) may be of strong interest. From a chronic standpoint, rehabilitative exercise training producing favourable changes in muscle structure and function has been related to improved muscle fatigability in other respiratory disorders [START_REF] Marillier | Locomotor Muscles in COPD: The Rationale for Rehabilitative Exercise Training[END_REF]. The corollary is that lessening exercise-related fatigability may help patients being more active [START_REF] Keyser | Changes in fatigability following intense aerobic exercise training in patients with interstitial lung disease[END_REF], physical inactivity being itself a perpetuating factor for fatigue in f-ILD [START_REF] Kahlmann | Managing Fatigue in Patients With Interstitial Lung Disease[END_REF]. Of note, we only enrolled a small sample of patients with severe f-ILD (mostly idiopathic pulmonary fibrosis). It is unknown whether similar results would be observed in milder form or other etiologies of f-ILD.

In conclusion, we found no association between trait level of perceived

Figure 1 ,

 1 Figure 1, panel A). In fact, ROF and ΔTw strongly correlated (r= -0.80; p<0.001) when adjusted for external work (panel B). Other indices of performance fatigability (ΔMVC and ΔVA, both absolute and work corrected) did not differ between ROF subgroups nor correlated with ROF scores (p>0.05; panels C-D).

Figure 1 .

 1 FIGURE LEGEND Figure 1. Comparison of performance fatigability indices according to patients' state level of perceived fatigability during exercise and their corresponding relationships in fibrotic interstitial lung disease.

Acknowledgements. The authors thank Angie Zapotichny, Filipa Saramago and Brandon Palmer (Kingston Health Sciences Centre) and the different members of the Respiratory Investigation Unit (Queen's University) for their continuous assistance.

We thus want to disclose that these data (i.e. peripheral muscle fatigability) have been published elsewhere [START_REF] Marillier | Oxygen supplementation during exercise improves leg muscle fatigue in chronic fibrotic interstitial lung disease[END_REF]. However, other data and the rationale/analyses we herein present are original and unpublished.