V Lerch

G Gary1

P Herve

Thermomechanical properties of polycarbonate under dynamic loading

In the present study, dynamic compression tests have been performed on polycarbonate (PC) specimen for strain rates in the range of 500 to 2ooos• 1 and on aluminum for a strain rate around 130os• 1 using a Split Hopkinson Pressure Bar (SHPB). Temperature measurement has been carried out using a fast response infrared optical pyrometer which is able to measure the surface temperature rise of a specimen during the tests. The temperature data shows that the mechanical work is not completely converted into heat. It implies the existence of a stored energy within the material and possibly a subsequent delay for the conversion of mechanical work into heat.

Introduction

Plastic deformations generate work which is converted into heat. The subsequent temperature rise can induce a modification of the properties of material and more often a thermal softening. The equation of heat conduction [l] which relates the mechanical state (strain and stress) to temperature is given by: apC P V''T-pCpt =-/Jo-•t P +kETatr(t• )/(1-2v)

(1)

where the "dots" indicate the differentiation with respect to time, a is the thermal diffusivity, p is the density, CE is the heat capacity, /3 is defined below, a is the stress, t P is the plastic strain rate, K is the coefficient of thermal expansion, E is Young's modulus, T0 is the initial or ambient temperature, t• is the elastic strain rate and v is Poisson's ratio. The first term on the right represents heating due to irreversible plastic deformation and the second term represents heating due to the reversible thermoelastic effect. If elasticity is neglected and adiabatic conditions prevail, then the heat equation takes a more simple form and the conversion rate of plastic work into heat is given by : /3 = pe t [START_REF] Taylor | Latent heat remaining metal after cold working[END_REF] (J"$P

The phenomena of mechanical dissipation of energy in metals have been known for a very long time. Taylor and Quinney [START_REF] Taylor | Latent heat remaining metal after cold working[END_REF] showed that approximately 90% of plastic work was transformed instantaneously into heat, the remainder being stored in the microstructure of the material. More recently, Hodowany et al. [START_REF] Hodowany | Partition of plastic work into heat and stored energy in metals[END_REF] showed that the fraction of plastic work converted into heat for an aluminum alloy Al2024-T3 varies from 60 to 30% at high strain rates (3000s-1 ). These phenomena have not been much studied in the case of polymers. In this study, one will observe and characterize the transformation of plastic work into heat for polycarbonate. Then the results of polycarbonate will be compared with the results of aluminum.

Experimental procedure 1.Measuring mechanical properties

The technique we use allows the measurement of the conversion rate under adiabatic conditions [1] when there are no thermal exchanges between the specimen and the surrounding devices. It is then possible to investigate the effects of some parameters among those governing the loading, like the strain rate. The SHPB allows to measure the forces and the displacements applied to both faces of the specimen. In order to make a dynamic compression test •with a SHPB, a short specimen is inserted between two bars with a higher yield stress than the tested material. Strain gauges are cemented to the bars. With the impact of a projectile (sticker), a longitudinal compressive elastic wave is induced in the incident bar (called the incident wave). A part of this wave is reflected at the bar-specimen interface (called the reflected wave), the other part is transmitted through the specimen and induces an elastic wave in the transmitted bar (called the transmitted wave). Such waves are shown in.figure 2.

The waves are known at points A and B, thanks to the recording of the corresponding longitudinal strain. They have then to be shifted to the bar ends in contact with the specimen.

In our processing technique, this shifting takes account of the wave dispersion correction and uses an assisted precise delay setting based on the simulation of the elastic transient response of the specimen, as described in [START_REF] Zhao | On the use of SHPB technique to determine the dynamic behavior of the materials in the range of the smalls strains[END_REF]. Let us call e;(t), er(t) and e,(t) the corresponding waves, after the shifting process. The velocities and forces at both specimen faces are given by the following formulas [START_REF] Zhao | On the use of SHPB technique to determine the dynamic behavior of the materials in the range of the smalls strains[END_REF] :

Finput (t) = SbEb(e;(t) +er (t)) V,nput( t) = -Cb(e ;(t )-er (t)) Foutput (t) = SbEbe,(t) V output (t) = -Cbe,(t) (3)
where Sb, Eb and Cb are the bar's cross-sectional area, Young's modulus and elastic wave velocity, respectively.

Assuming then that stresses and strains are homogenous in the specimen, the average behavior of the material tested can be found. This assumption is checked by comparing input and output forces before going on processing the test data. An example of the equilibrium is plotted in figure 3 in the case of a test for PC at 900s-1. However, the signal produced by the detector is directly dependent on the surface emissivity of the specimen. Its value is often unknown, even at the early stage of the test, and may also depend on temperature, strain, surface finish, etc ... Moreover, we have to take into account the fact that the temperature for the low strains is close to the ambient temperature.

Temperature measurement

The theoretical signal from an infrared source over the spectral band of the pyrometer can be calculated by integrating the Planck distribution so that the output signal of the pyrometer is

given by [7] :

(4

)
where A is a calibration constant, n is the solid angle between the imaging system and the detector, S d is the area of the detector, According to the bibliography, we can say that the emissivity of aluminum is equal to 0.25 and is almost independent of the temperature and the strain.

1.2.1. Experimental set-up l.

Detector

There exist several types of optical transducers available for the measurement of temperature. Important factors that influence the choice of a radiometer are time resolution, expected temperature range, materials involved in the experiments and size of the system. A single element, Indium-Antimonide (InSb), photovoltaic detector, with maximal responsivity in the 4 to 5.5µm wavelength region, was used in this study. The detector is mounted in a liquid nitrogen cooled dewar with a sapphire window. Since the detector is at 77K, the noise is significantly reduced. The detector area is a I mm diameter spot and its response time is in the order of one microsecond.

Imaging system

The pyrometer shown in figure 1 has a spectral sensitivity range between 1 and 5.5µm. The radiation of the transmitting surface is collected through a biconvex lens in fluorite (25.4mm in diameter and 25.4mm in focal distance). The distance between the detector and the lens and the distance between the lens and the emitting surface of the specimen are equal to twice the focal distance. This optical assembly optimizes the energy received by the detector from the lmm diameter spot of the specimen surface. The signal/noise ratio of the detector obtained \vith this optical assembly is maximum.

1.2.2. Experimental procedure 1.2.2.1. Alignment
In order to measure the surface temperature, it is necessary to carefully focus the optical system on the specified surface region of the specimen. Before each test, a thin metallic pointer temporarily mounted on the optical system is used to defi ne precisely the position of the focal point on the surface of the specimen placed between the bars.

Calibration

The calibration technique consists in heating the specimen up to a known temperature and measuring the pyrometer signal. After a series of SHPB tests, a black body ( e = 0.97) was placed between the SHPB at the same location than the specimen. Since the electronics are only sensitive to AC signals, a chopper wheel was used to produce a pseudo AC signal

(figure 1 calibration apparatus). We simultaneously record the signal amplitude from the detector and the temperature of the black body which is measured by a K type thermocouple. The corresponding curve is shown infigure 5 which allows to determine the calibration constant A . for PC and aluminum.

Results

Commercial polycarbonate (PC) cubes with section lOmm*lOmm and thickness of 6-Smm and a specimen of aluminum with section 8mm*8mm and thickness of 6mm were tested. The true compressive stress versus true strain curves are shown in figure 6 for PC and aluminum. This figure shows that the fl ow stress is dependent on the strain rate for PC. Two characteristics are observed : fi rstly, a rather large linear domain exists until a strain of 0.1. This domain is one order of magnitude greater than the typical linear elastic domain of metals. Secondly, following this linear domain, a pronounced softening domain is noticed up to a strain of about 0.3 beyond which the material shows strain hardening. The temperature does not rise significantly below 0.1 strain (figure 7) beyond which it rises steeply and levels off before dropping. This dropping is due to the fact that the specimen is not in the view fi eld of pyrometer. For the test at a strain rate 1800s-1 , the maximum of temperature corresponds to a true strain of 0.8. The curves showing the temperature rise versus plastic strain are shown in figure 8 for PC and aluminum. For PC, even at high strains, the temperature is lower than 75°C which is a smaller value than the melting temperature. This explains the lack of significant thermal softening for the test at 1800 s• 1 . In this case, the temperature rise does not seem dependant on the strain rate. This article presents a first evaluation the rate of conversion of plastic work into heat during a dynamic SHPB test on polycarbonate specimens. An infrared pyrometer has been developed which is capable of measuring specimen surface temperatures with a time resolution of about lµs. The single detector has been used to measure the temperature during compression loadings at strain rates up to 2000s-1 . The experimental setup was capable of repeatable measurement of mechanical and thermal quantities. The temperature seems to be independent of the strain rate. The observed instantaneous conversion factor of plastic work into heat varies between 0.4 and 1 and there was no significant observed dependence of p on the strain rate. By comparison with aluminum , the typical evolution of the conversion factor for PC shows a thermal behavior providing an extra information that will help for a better further definition of the constitutive law.

2 :

 2 figure 1 : Scheme of the split Hopkinson bar with optical pyrometer figure 2 : Raw data for a test on polycarbonate (a) incident and reflected waves, (b) transmitted wave and (c) temperature signal as functions of time

figure 3 :

 3 figure 3 : Ch ecking equilibrium for a polycarbonate specimen : input and output forces

figure 4 :

 4 figure 4 : Emissivity for polycarbonate as functions of temperature

figure 5 :

 5 figure 5 : Calibration curve

6 :

 6 figure 6 : True stress-true strain plot at different strain rates for polycarbonate and aluminum

figure 7 :figure 8 :

 78 figure 7 : True stress-true strain and temperature at a strain rate of 1800s-1 for polycarbonate

figure 9 :

 9 figure 9: Fraction of plastic work converted into heat as functions of plastic strain for polycarbonate and aluminum


Claire Ramus
email: claire.ramus@cea.fr

Agnès Hovasse
email: ahovasse@unistra.fr

Marlène Marcellin
email: marlene.marcellin@ipbs.fr

Anne-Marie Hesse
email: anne-marie.hesse@cea.fr

Emmanuelle Mouton-Barbosa
email: emmanuelle.mouton@ipbs.fr

David Bouyssié
email: bouyssie@ipbs.fr

Sebastian Vaca
email: sebastian.vaca@etu.unistra.fr

Christine Carapito
email: ccarapito@unistra.fr

Karima Chaoui
email: karima.chaoui@ipbs.fr

Christophe Bruley
email: christophe.bruley@cea.fr

Jérôme Garin
email: jerome.garin@cea.fr

Sarah Cianférani
email: sarah.cianferani@unistra.fr

Myriam Ferro
email: myriam.ferro@cea.fr

Alain Van Dorssaeler

Odile Burlet-Schiltz

Christine Schaeffer
email: christine.schaeffer@unistra.fr

Yohann Couté
email: yohann.coute@cea.fr

Anne Gonzalez De Peredo

Benchmarking quantitative label-free LC-MS data processing workflows using a complex spiked proteomic standard dataset

Keywords: proteomic standard, nanoLC-MS/MS, label-free quantification, computational proteomics, spectral counting, MS signal analysis

A C C E P T E D M A N U S C R I P T A C C E P T E D M A N U S C R I P T

expression in complex biological samples. However, the computational processing of the data for label-free quantification still remains a challenge. Here, we used a proteomic standard composed of an equimolar mixture of 48 human proteins (Sigma UPS1) spiked at different concentrations into a background of yeast cell lysate to benchmark several label-free quantitative workflows, involving different software packages developed in recent years.

This experimental design allowed to finely assess their performances in terms of sensitivity and false discovery rate, by measuring the number of true and false-positive (respectively UPS1 or yeast background proteins found as differential). The spiked standard dataset has been deposited to the ProteomeXchange repository with the identifier PXD001819 * and can be used to benchmark other label-free workflows, adjust software parameter settings, improve algorithms for extraction of the quantitative metrics from raw MS data, or evaluate downstream statistical methods.

Introduction

Label-free quantitative methods based on LC-MS/MS have become increasingly popular in proteomic studies, as an attractive and powerful way to analyze differential protein expression in complex biological samples [START_REF] Nahnsen | Tools for label-free peptide quantification[END_REF][START_REF] Neilson | Less label, more free: approaches in label-free quantitative mass spectrometry[END_REF][START_REF] Sandin | Data processing methods and quality control strategies for label-free LC-MS protein quantification[END_REF]. They can be based either on the measurement of the MS/MS sampling rate for a particular protein (spectral counting), or on the MS chromatographic peak area of its corresponding peptides in the survey MS scan (MS trace analysis), both values being directly related to protein abundance. Both approaches have benefited from tremendous improvements in instrumentation, namely increased sequencing speed for spectral counting approaches (up to 15-20Hz in recent orbitrap or Q-TOF mass spectrometers) and higher resolution allowing more accurate MS signal analysis and improved matching of complex LC-MS maps. These methods have concomitantly gained in analytical depth, and can now routinely be used to profile the expression of thousands of proteins from biological systems submitted to different conditions. An important point is however to be able to assess, minimize, and eventually correct the variability associated to the LC-MS/MS analytical workflow, to ensure sufficient repeatability of the measurements and provide robust relative quantification of proteins across samples. To this respect, the development of proteomic standards has proved to be essential to assess the performances of LC-MS platforms, provide a quality control of the system and identify potential sources of variability. Importantly, they are also needed to evaluate the downstream elements of the analytical pipeline, i.e. bioinformatics processing and statistical analysis, which represent critical steps to generate the final comparative results.

The yeast Saccharomyces cerevisiae proteome has been used in many studies as a test sample to illustrate the benefits of various technological optimizations in the LC-MS/MS workflow.

Due to its wide availability and relatively high complexity and dynamic range, it can be considered as a good surrogate to many real biological samples, both for method development

A C C E P T E D M A N U S C R I P T

and quality control. In previous studies, yeast samples have been used to establish and demonstrate the efficiency of a wide range of metrics to evaluate the LC-MS/MS performances [START_REF] Beasley-Green | A proteomics performance standard to support measurement quality in proteomics[END_REF][START_REF] Rudnick | Performance metrics for liquid chromatography-tandem mass spectrometry systems in proteomics analyses[END_REF]. These metrics were directly related to the LC system, the MS instrument (electrospray source, MS1 and MS2 intensities), the dynamic sampling, but also the first steps of data processing, i.e. peptide identification results. They were applied by Paulovich et al. for LC-MS benchmarking of several instrumental systems operated in different laboratories [START_REF] Paulovich | Interlaboratory study characterizing a yeast performance standard for benchmarking LC-MS platform performance[END_REF].

Instead of focusing on specific proteins or peptides, the monitoring proposed in this study allowed to give a global and very exhaustive view of the quality of the analysis through general metrics reflecting for example the median peak FWHM on the whole peptide population, the number of MS1 or MS2 scans triggered over various portions of the chromatogram, the level of TIC, the median MS1 signal for the population of identified precursors, or the number of peptides and proteins identified.

However, the final objective of most label-free studies is to measure quantitative levels, and detect variation of some proteins across samples. To evaluate the performances of a workflow in this respect, it is relevant to use a standard spiked with known amounts of some peptides or proteins, which can then be specifically monitored to assess the ability of the analysis in detecting relative quantitative changes. Controlled datasets based on spike-in experiments thus represent a useful tool to objectively assess the performances of quantitative methods for differential analysis. Paired comparison between spiked versus non-spiked samples can be performed to benchmark analytical and computational pipelines for biomarker discovery.

Such controlled datasets with known "ground truth" have been for example generated in the past in the field of microarray analysis, by spiking at different concentrations a panel of 100-200 specific RNAs into a well-defined, constant background of RNA species [START_REF] Choe | Preferred analysis methods for Affymetrix GeneChips revealed by a wholly defined control dataset[END_REF], and was then widely used as a gold standard to evaluate various data processing methods [START_REF] Dabney | A reanalysis of a published Affymetrix GeneChip control dataset[END_REF][START_REF] Hertogh | A benchmark for statistical microarray data analysis that preserves actual biological and technical variance[END_REF][START_REF] Irizarry | Feature-level exploration of a published Affymetrix GeneChip control dataset[END_REF][START_REF] Irizarry | Comparison of Affymetrix GeneChip expression measures[END_REF][START_REF] Pearson | A comprehensive re-analysis of the Golden Spike data: towards a benchmark for differential expression methods[END_REF]. In the proteomics field, spiked samples are also often used to evaluate MS methods or data

A C C E P T E D M A N U S C R I P T
processing tools, although generally the number of spiked proteins or peptides is relatively low [START_REF] Hoekman | msCompare: A Framework for Quantitative Analysis of Label-free LC-MS Data for Comparative Candidate Biomarker Studies[END_REF][START_REF] Tsou | IDEAL-Q, an automated tool for label-free quantitation analysis using an efficient peptide alignment approach and spectral data validation[END_REF][START_REF] Zhang | Evaluation for Computational Platforms of LC-MS Based Label-Free Quantitative Proteomics: A Global View[END_REF][START_REF] Christin | A critical assessment of feature selection methods for biomarker discovery in clinical proteomics[END_REF]. Interestingly, as exemplified in the report from Paulovich et al, the use of a more complex spiked material, such as the UPS1 standard containing 48 well-characterized purified proteins, allows to compute more extensively the exact proportion of false discoveries (number of yeast false positives relative to the total number of proteins declared as variant) and of true discoveries (number of true positives out of the 48 real variant UPS1 proteins). As a proof of concept of the kind of benchmarking that can be done with this spiked standard, they showed the performances of a spectral count approach (the SASPECT method)

for detection of biomarkers when comparing the spiked sample (simulating a case sample)

and the pure yeast reference sample (control sample).

In the present study, we wanted to extend this concept and use the yeast+UPS1 standard to benchmark several tools developed in recent years for relative quantification, including widely used software such as MaxQuant and Skyline. Indeed, while numerous software tools have been developed and are more and more routinely used for label-free quantitation, stringent and side-by-side evaluations have to be performed to prove the efficiency of the quantification. In addition, proper tuning and parameter settings in each of these software tools are also important for optimal downstream analysis. We thus generated a dataset from

Experimental procedures

Sample preparation. A yeast cell lysate was prepared in 8M urea / 0.1M ammonium bicarbonate buffer, protein concentration was adjusted at 8µg/µL after Bradford assay, and this lysate was used to resuspend and perform a serial dilution of the UPS1 standard mixture (Sigma). Twenty µL of each of the resulting samples, corresponding to 9 different spiked levels of UPS1 (respectively 0.05 -0.125 -0.250 -0.5 -2.5 -5 -12.5 -25 -50 fmol of UPS1 /µg of yeast lysate), were reduced with DTT and alkylated with iodoacetamide. The urea concentration was lowered to 1M by dilution, and proteins were digested in solution by addition of 2% of trypsin overnight. Enzymatic digestion was stopped by addition of TFA (0.5% final concentration).

NanoLC-MS/MS analysis. Samples (2µg of yeast cell lysate + different spiked level of UPS1) were analyzed in triplicate by nanoLC-MS/MS using a nanoRS UHPLC system (Dionex, Amsterdam, The Netherlands) coupled to an LTQ-Orbitrap Velos mass spectrometer (Thermo Fisher Scientific, Bremen, Germany). 2 µL of each sample were loaded on a C-18 precolumn (300 µm ID x 5 mm, Dionex) at 20 µL/min in 5% acetonitrile, 0.05% TFA. After 5 minutes desalting, the precolumn was switched online with the analytical C-18 column (75 µm ID x 15 cm, in-house packed with C18 Reprosil) equilibrated in 95% solvent A (5% acetonitrile, 0.2% formic acid) and 5% solvent B (80% acetonitrile, 0.2% formic acid).

Peptides were eluted using the following gradient of solvent B at 300 nL/min flow rate: 5 to 25% gradient during 75 min; 25 to 50% during 30min; 50 to 100% during 10min. The LTQ-

A C C E P T E D M A N U S C R I P T

Orbitrap Velos was operated in data-dependent acquisition mode with the XCalibur software.

Survey scan MS were acquired in the Orbitrap on the 300-2000 m/z range with the resolution set to a value of 60000. The 20 most intense ions per survey scan were selected for CID fragmentation and the resulting fragments were analyzed in the linear trap (LTQ). Dynamic exclusion was employed within 60 seconds to prevent repetitive selection of the same peptide.

MS data processing.

The dataset was processed according to different workflows listed in as the instrument, trypsin/P as the enzyme and 2 missed cleavages were allowed. Precursor and fragment mass error tolerances were set at 5 ppm and 0.8 Da, respectively. Peptide variable modifications allowed during the search were: acetyl (Protein N-ter), oxidation (M), whereas carbamidomethyl (C) was set as fixed modification. To calculate the false discovery rate (FDR), the search was performed using the "decoy" option in Mascot. Validation was performed with an in-house developed module associated to MFPaQ [START_REF] Bouyssie | Mascot file parsing and quantification (MFPaQ), a new software to parse, validate, and quantify proteomics data generated by ICAT and SILAC mass spectrometric analyses: application to the proteomics study of membrane proteins from primary human endothelial cells[END_REF] (http://mfpaq.sourceforge.net/), based on the target-decoy strategy, as described before [START_REF] Gautier | Label-free quantification and shotgun analysis of complex proteomes by one-dimensional SDS-PAGE/NanoLC-MS: evaluation for the large scale analysis of inflammatory human endothelial cells[END_REF].

A C C E P T E D M A N U S C R I P T

Briefly, FDR at peptide level was calculated as described in [START_REF] Navarro | A refined method to calculate false discovery rates for peptide identification using decoy databases[END_REF] and set at 5% by adjusting peptide p-value threshold. Validated peptides were assembled into protein groups following the principle of parsimony (Occam's razor) [START_REF] Zhang | Proteomic parsimony through bipartite graph analysis improves accuracy and transparency[END_REF]. Protein groups were then validated to obtain a FDR of 1% at the protein level, by adjusting the threshold on a protein group score defined as the sum of peptide score offsets (difference between each peptide Mascot score and its homology or identity threshold). The total spectral count metric was extracted for each protein group by MFPaQ in each analytical run.

Workflow 2: Andromeda / MaxQuant / Spectral Counting. Acquired MS data were processed using MaxQuant version 1.3.0.5 [START_REF] Cox | MaxQuant enables high peptide identification rates, individualized p.p.b.-range mass accuracies and proteome-wide protein quantification[END_REF]. Derived peak lists were submitted to the Andromeda search engine [START_REF] Cox | Andromeda: a peptide search engine integrated into the MaxQuant environment[END_REF]) (www.maxquant.org). For database searches, the precursor mass tolerance was set to 20 ppm for first searches and 6ppm for main Andromeda database searches. The fragment ion mass tolerance was set to 0.5 Da. Trypsin/P was chosen as the enzyme and 2 missed cleavages were allowed. Oxidation of methionine and protein N-terminal acetylation were defined as variable modifications, and carbamidomethylation of cysteine was defined as a fixed modification. Minimum peptide length was set to six amino acids. Minimum number of unique peptides was set to one. Maximum FDRcalculated by employing a reverse database strategywere set to 1% for peptides and proteins. Proteins identified as "reverse"

and "only identified by site" were discarded from the list of identified proteins. In this particular workflow, total spectral count for each validated protein group was computed from msms.txt table. ESI-TRAP was chosen as the instrument, trypsin/P as the enzyme and 2 missed cleavages were allowed. Precursor and fragment mass error tolerances were set at 5 ppm and 0.8 Da, respectively. Peptide variable modifications allowed during the search were: acetylation

A C C E P T E D M A N U S C R I P T

(Protein N-ter), oxidation (M), whereas carbamidomethyl (C) was set as fixed modification.

The IRMa software v1.31 [START_REF] Dupierris | A toolbox for validation of mass spectrometry peptides identification and generation of database: IRMa[END_REF] was used to filter the results. Filters used were : (1) peptides whose score ≥ query homology threshold (p<0. For the identification of proteins, a Mascot ion score had to be minimum 30 and above the 95% Mascot significance threshold ("Identity score"). The target-decoy database search allowed us to control and estimate the false positive identification rate of our study, and the final catalogue of proteins presented an estimated false discovery rate (FDR) below 5%. The spectral count metric used for quantitation corresponds to the Unweighted Spectrum Count values in Scaffold.

Workflow 5: ExtractMSn / Mascot / MFPaQ / MS Signal analysis. The first steps (peaklist creation, database search, validation) were the same than in workflow 1. Quantification of proteins was then performed using the label-free module implemented in the MFPaQ v4.0.0

A C C E P T E D M A N U S C R I P T

software, as previously described [START_REF] Gautier | Label-free quantification and shotgun analysis of complex proteomes by one-dimensional SDS-PAGE/NanoLC-MS: evaluation for the large scale analysis of inflammatory human endothelial cells[END_REF][START_REF] Mouton-Barbosa | In-depth exploration of cerebrospinal fluid by combining peptide ligand library treatment and label-free protein quantification[END_REF]. Briefly, the software uses the validated identification results and retrieves the XIC of the identified peptide ions in the corresponding raw nanoLC-MS files, based on their experimentally measured RT and monoisotopic m/z values. Peptide ions identified in all the samples to be compared are used to build a retention time matrix and re-align in time LC-MS runs. For peptides not identified by MS/MS in a particular run, this re-alignment matrix is used to perform cross-assignment and extract their XIC signal starting from a predicted RT. Normalization across conditions is performed based on the median of XIC area ratios for all the extracted peptide ions. Protein quantification is based on a protein abundance index calculated as the average of XIC area values for at most three intense reference tryptic peptides per protein.

Workflow 6 and 7: Andromeda / MaxQuant / MS Signal analysis. The first steps (database search with Andromeda and validation) were the same as in workflow 2. For quantification purposes, either Intensities (workflow 6) or LFQ [START_REF] Cox | Accurate proteome-wide label-free quantification by delayed normalization and maximal peptide ratio extraction, termed MaxLFQ[END_REF] (workflow 7) calculated by MaxQuant were used. The LFQ metric, as described in [START_REF] Cox | Accurate proteome-wide label-free quantification by delayed normalization and maximal peptide ratio extraction, termed MaxLFQ[END_REF], is derived from the raw intensities by the MaxLFQ algorithm, which uses a specific normalization procedure, as well as a particular aggregation method to calculate protein intensities, by taking into account, for each protein, all the peptide ratios measured in all pair-wise comparisons of the different quantified samples. "Match between run" time window was set to 2 minutes. For LFQ quantification, only protein ratios calculated from at least two unique peptides ratios (min LFQ ratio count=2) were considered for calculation of the LFQ protein intensity.

Results

Experimental design, sample preparation and analysis. In order to evaluate different quantitative workflows in their ability to correctly detect known variant proteins in complex samples, we prepared a series of 9 yeast lysate samples spiked with growing concentrations of the Sigma UPS1 standard composed of an equimolar mixture of 48 human proteins. To that aim, UPS1 lyophilized proteins were directly resuspended using the yeast lysate prepared in urea buffer, and a serial dilution of this initial mixture was then performed using the same yeast lysate, resulting in spiked UPS1 concentrations ranging from 50amol/µg up to 50fmol/µg of yeast lysate. Protein samples were digested with trypsin, and resulting peptides were analyzed by nanoLC-MS/MS on a LTQ Velos-Orbitrap instrument, using routine chromatographic conditions (15cm C18 reverse-phase column, 2 hours gradient) and datadependent acquisition MS parameters (resolution 60000 for MS survey scan, top 20 CID sequencing in the ion trap). Triplicate MS analysis was performed for each sample, resulting in 27 raw data files that were subsequently processed in different ways, using several computational workflows (Table 1). Two different softwares were used for protein identification (Mascot and Andromeda) and 5 solutions were employed for protein quantification (Scaffold, IRMa/hEIDI (Hesse et al, in preparation), MaxQuant [START_REF] Cox | MaxQuant enables high peptide identification rates, individualized p.p.b.-range mass accuracies and proteome-wide protein quantification[END_REF][START_REF] Cox | Andromeda: a peptide search engine integrated into the MaxQuant environment[END_REF][START_REF] Cox | A practical guide to the MaxQuant computational platform for SILAC-based quantitative proteomics[END_REF],

MFPaQ [START_REF] Bouyssie | Mascot file parsing and quantification (MFPaQ), a new software to parse, validate, and quantify proteomics data generated by ICAT and SILAC mass spectrometric analyses: application to the proteomics study of membrane proteins from primary human endothelial cells[END_REF][START_REF] Mouton-Barbosa | In-depth exploration of cerebrospinal fluid by combining peptide ligand library treatment and label-free protein quantification[END_REF] and Skyline [START_REF] Maclean | Skyline: an open source document editor for creating and analyzing targeted proteomics experiments[END_REF][START_REF] Schilling | Platformindependent and label-free quantitation of proteomic data using MS1 extracted ion chromatograms in skyline: application to protein acetylation and phosphorylation[END_REF]), some of them generating a unique quantitative output, either spectral counting or MS signal extraction data, and some of them generating both types

A C C E P T E D M A N U S C R I P T

of quantitative data. Finally, 8 different quantitative datasets were obtained, as indicated in Table 1 and described in details in the Experimental section.

We first evaluated the identification datasets in a qualitative way by simply reporting the number of identified and validated proteins for both the background (yeast proteins) and the spiked standard (UPS1 proteins) in each sample. Sup data 1 shows the number of proteins identified by MS/MS sequencing and validated by various bioinformatics workflows. As expected, the total number of proteins, reflecting mainly the constant yeast background proteome, was fairly reproducible across triplicate MS analysis and across the series of spiked samples, whereas the number of identified UPS1 proteins increased with the spiked amount.

While no UPS1 protein was correctly identified at a concentration of 500amol/µg (as none of the peptide sequence matches could be validated at this concentration), all 48 human proteins were sequenced and correctly identified at 50fmol/µg. From these results, we decided to select different concentration levels of UPS1 to perform pairwise quantitative comparisons of samples, trying to mimic distinct biochemical situations, as illustrated in Figure 1.

Comparison A (500amol/µg versus 50fmol/µg) should reflect a case were a protein is typically under the detection level of the instrument in one condition, and strongly expressed in the other condition with a fold change of 100. In comparison B (5fmol/µg versus 50fmol/µg), the protein may be in turn detectable in both conditions, and strongly upregulated with a fold change of 10. Finally, comparison C (12.5fmol/µg versus 25fmol/µg) should simulate a situation where the protein is detectable in both conditions, but only slightly up-regulated with a fold change of 2. Because "real-life" biological samples usually contain many proteins with a differential abundance, encompassing a wide range of absolute expression levels and fold change values, we tried to approximate such a situation by gathering together the quantitative data obtained for each binary comparison, after computational processing. Using this post-processing assembly of the 3 individual datasets, we composed a global quantitative dataset containing theoretically 144 variant proteins (UPS1 proteins issued from the 3 relative quantitative analyses, and thus expected to vary with a fold change of 100, 10 or 2), and a background of around 2500 non-variant yeast proteins (measured and quantified in the different pairwise comparisons) (see ref [START_REF] Ramus | Spiked proteomic standard dataset for testing label-free quantitative software and statistical methods[END_REF], Sup Table 1).

The generation of this synthetic dataset allowed us to illustrate, in a single representation, the performance of quantitative proteomic tools and methods, challenged with different situations.

The final aim of relative quantitative analysis in discovery proteomics is usually to identify differentially expressed proteins. Therefore, the tested informatics workflows were mainly evaluated in their ability to correctly detect the expected variants, rather than in the accuracy of the measured fold change. The experimental design and the spiked standard used here allowed us to unambiguously assess such performances by counting the number of truepositives (TP) and false-positives (FP), respectively UPS1 or yeast background proteins found to be differentially expressed. Clearly, the classification of proteins as variant (positive hits)

or non-variant (negative hits) both relies on the one hand, on the accuracy of the quantitative metrics generated by the bioinformatics software, and on the other hand, on the performance of the statistical test and criteria used to discriminate the positive and negative populations.

Here, we mainly tried to benchmark the former step of the workflow (extraction of quantitative metrics by informatics tools), and we didn't aim to evaluate statistical methods.

We thus used a common, simple statistical test for protein classification, based either on the beta-binomial method for spectral count datasets [START_REF] Pham | On the beta-binomial model for analysis of spectral count data in label-free tandem mass spectrometry-based proteomics[END_REF], or on a modified t-test for datasets containing peptide intensity-based values (see Experimental section and below). Proteins were classified as variant or non-variant by a combined filtering on the p-value of this statistical test and on the fold change value, as very often performed in "real life " biological studies [START_REF] Albrethsen | Proteomic profiling of Mycobacterium tuberculosis identifies nutrient-starvation-responsive toxin-antitoxin systems[END_REF][START_REF] Bell | Quantitative proteomics reveals the induction of mitophagy in tumor necrosis factor-alpha-activated (TNFalpha) macrophages[END_REF][START_REF] Ichikawa | Prognostic significance of promyelocytic leukemia expression in gastrointestinal stromal tumor; integrated proteomic and transcriptomic analysis[END_REF][START_REF] Zhang | Label-free proteomic analysis of PBMCs reveals gender differences in response to long-term antiretroviral therapy of HIV[END_REF]. Following such classification, the sensitivity of the workflows for the detection of

A C C E P T E D M A N U S C R I P T
variant proteins (number of true positive hits relative to the real total number of variant proteins, i.e. TP/144), and false discovery proportion (FDP, defined as the number of false positive hits relative to the total number of proteins found as variant, i.e. FP/(TP+FP)) could easily be computed.

Performances of spectral counting for discrimination of variant proteins. Figure 2A shows the volcano plots obtained by applying spectral counting quantification methods, in which the log 10 (p-value) (calculated from the results of the BetaBinomial R package) is plotted against the calculated protein log 2 (fold change). As illustrated on these graphs, the majority of UPS1 proteins from comparison A and B (green and red populations, theoretical fold changes of respectively 100 and 10) were easily discriminated from the background of yeast proteins (grey), by both their p-values and fold changes. This was particularly the case with software tools such as IRMa/hEIDI and Scaffold. These results indicated the ability of the spectral count-based quantitative approaches to confidently detect protein variations of high to medium amplitude while minimizing the level of false discoveries. However, it can be noted that the UPS1 proteins quantified in the comparison C (12.5fmol/µg versus 25fmol/µg, yellow dots) were not well segregated from the background independently of the software used.

Overall, these observations pointed out some limitations of quantification with spectral count data when dealing with low fold change variations or weakly concentrated proteins.

From these data, we tried to determine which criterion was best suited to retrieve significantly variant proteins. Sensitivity-FDP curves were plotted for the data obtained from the different workflows by using either the fold change or the p-value as a unique criterion to classify the proteins, and we further wanted to apply combinations of these filters to improve the Here again we plotted different sensitivity-FDP curves by classifying the proteins either on their absolute fold change, on their welch t-test p-value, or by a combination of these criteria (setting up a fixed threshold for one of them and varying the other) (Sup data2B). In the case of MS intensity values obtained in our dataset, the fold change appeared to be generally a more efficient filter to discriminate TP from background than a simple statistical test based on the variance of the protein intensities. Indeed, the modified Welch t-test may produce a high number of FP hits on this particular dataset containing only three analytical replicates, finally leading to a high FDP after multiple testing. For example, on the MaxQuant LFQ dataset (workflow 7), filtering the proteins at a 0.05 cutoff only on the Welch p-value allowed to efficiently retrieve almost all UPS1 variant proteins (134 out of 144, e.g. 94% sensitivity), but with as many as 387 FP yeast proteins declared as variant (i.e. a final FDP of 74%). On the other hand, correction of the p-values for multiple-testing with methods such as the Benjamini-Hochberg (BH) procedure can be used to limit the number of FP and control the final FDR, but at the cost of a much lower sensitivity. For example, applying this correction on the same dataset and filtering afterwards with a BH adjusted p-value cutoff of 0.05 led to only 3 FP yeast proteins, but the number of TP UPS1 proteins also dropped to 50 (i.e. a calculated final FDP of 6%, close to the desired theoretical value, but a sensitivity of only 35%, see ref [START_REF] Ramus | Spiked proteomic standard dataset for testing label-free quantitative software and statistical methods[END_REF], Sup Table 1). Finally, combining fold change and Welch t-test p-value criteria emerged as the most discriminant approach, and allowed to reach good sensitivity has to be noticed however that all workflows produced still relatively elevated FDP values, that may be related to signal extraction errors by the softwares. The best compromise between sensitivity and FDP was obtained using the LFQ metric from MaxQuant [START_REF] Cox | Accurate proteome-wide label-free quantification by delayed normalization and maximal peptide ratio extraction, termed MaxLFQ[END_REF] and the Top3 metric from MFPaQ [START_REF] Mouton-Barbosa | In-depth exploration of cerebrospinal fluid by combining peptide ligand library treatment and label-free protein quantification[END_REF].

Use of the spiked standard dataset to highlight data processing problems and optimize the workflows.

We next wanted to take advantage of this model dataset to identify quantification errors associated to the generation of false-negative (FN) and false-positive (FP) proteins, and illustrate a number of possible mistakes introduced by the different MS intensity based

A C C E P T E D M A N U S C R I P T

workflows. Protein quantification is a multi-step process, and possible errors associated to each of these steps may influence the final result. Obviously, processing steps based on peptide validation, grouping, and peptide-to-protein inference are important for final protein quantification. Sup data 3A illustrates a case where quantification based on non-specific peptides, shared between a stable yeast protein and a UPS1 variant protein (Ubiquitin-40S ribosomal protein S27a), compromised the result and led to classification of the spiked protein as a FN. Most of the time however, errors seem to take place at the signal extraction step itself. Sup data 3B shows a situation with overlapping isotopic patterns from several coeluting species, in which the MFPaQ software wrongly picked, in addition to the monoisopic peak of the correct peptide, the third and second isotope peaks from other species, as well as the monoisopic peak of a closely eluting isobaric peptide. Such errors could be avoided through a better recognition by the algorithms of peptide isotopic patterns. In addition, in the cases illustrated here, 16 peptides were correctly quantified for the protein, while signal extraction error occurred occasionally on a single peptide. Enabling the detection and elimination of outlier peptides with adequate testing procedures (option not enabled in that case) would alleviate such problems. Good alignment of LC-MS runs in retention time is also important for correct peak picking when cross assignment between runs is implemented.

Some errors in Skyline could be attributed to wrong selection of a particular peptide in one of the runs in which the peptide was not sequenced by MS/MS, and in which XIC extraction was thus performed based on the RT of the peptide in another run (not shown). It must be noticed that tracking and eventually correcting these signal extraction errors is quite dependent on the software interface. To this respect, a software like MFPaQ offers a visualization interface that enables a rapid inspection of the XICs extracted for each peptide in the different conditions, and possibly unselects some of them to eliminate these peptides from the final quantification of the protein. However, it does not allow going back to raw MS data and correct for example

A C C E P T E D M A N U S C R I P T

the selection of the integration area directly on the chromatogram. This in turn is possible in Skyline, which really offers an interactive interface to efficiently review the results and manually correct possible mistakes. We thus wanted to take advantage of this feature and evaluate whether manual validation of the entire dataset was practically possible and how efficient it could be to improve the quantitative results. It took around 15 hours to manually check all the peptide ions from the dataset and either validate or correct the integration of the corresponding XIC. Figure 4 shows the result of this exhaustive reviewing of the data on the accuracy of the quantitative result. While relatively time consuming, the manual correction clearly reduces the number of both false positive and false negative. The sensitivity was thus improved (from 88% using raw data to 97% after manual correction) and the FDP was false positive hits, 3 contained peptides that were clearly "contaminated" with UPS1 peptides, 4 had very low intensity signals, and one of them was detected as variant while the expression profile of the related peptides did not follow that of UPS1 peptides. Altogether, these residual mistakes remaining after in-depth manual validation may reflect the minimal margin of error of the label-free, MS intensity-based quantification process, which may be difficult to reduce even by improving the automatic signal extraction algorithms of the software.

Discussion

A C C E P T E D M A N U S C R I P T

In this study, we generated a complex, spiked proteomic standard dataset, in which the ground truth is well characterized, and showed its utility for benchmarking label-free relative quantification computational workflows. Different protein standards have been used in the past to measure the performances of such software and data processing methods, ranging from simple mixtures of recombinant proteins, to complex cellular extracts spiked with a known amount of exogenous proteins. In the design of such a standard, it is important to be able to easily differentiate the spiked proteins from the background after the database search and identification process, in order to perform a correct classification of spiked (TP) and background (TN) molecules. The most straightforward approaches are either to apply some isotopic labeling on the background or the spiked samples, or to use sets of proteins from different species. Ideally, the number of spiked molecules should be large enough to provide a relevant statistical estimation of the sensitivity and FDP of the quantitative methods.

Typically, samples can be spiked with recombinant purified proteins added in known quantities to the background, or with a much more complex sample, such as a biological extract from another species. In recent studies aiming at benchmarking software tools, such "double-proteome" samples have been used. For example, a mixture of lysates from human cells and from the Streptococcus pyrogenes bacterium at different ratios was used in a comparative study to show the performances of the OpenMS software [START_REF] Weisser | An automated pipeline for high-throughput label-free quantitative proteomics[END_REF]. Similarly, Cox et al used a complex digest of Hela cells, spiked with an E.coli digested cellular extract at two different amounts, creating a 3 fold variation of the E.coli proteins in the quantitative comparison [START_REF] Cox | MaxLFQ allows accurate proteome-wide label-free quantification by delayed normalization and maximal peptide ratio extraction[END_REF]. In that later case, the spiked population represents a significant portion of the total sample (about one third of the identified proteins). Such a dataset may simulate particular biological experiments where a stimulation could for example induce a massive variation of the proteome, or some interaction proteomics experiments where a control is compared to an affinity purified sample containing many up-regulated proteins. However,

A C C E P T E D M A N U S C R I P T

normalization of such datasets may be difficult, because the usual hypothesis underlying normalization procedures is that the major part of the protein population remains stable, and the median of the fold change distribution should be 1. On the other hand, spiking a proteome background with a calibrated set of recombinant purified proteins is statistically less representative, as the number of TP decreases, but allows to simulate easily a classical expression proteomics experiment, in which a very minor part of the proteome will undergo a fold change. The UPS1 commercial standard, containing an equimolar mixture of 48 purified human proteins, represents a convenient sample for a spiking scheme experiment, and offers already a significant number of TP that allows to get an estimation of the sensitivity and FDP of the computational methods.

As software tools are expected to perform unequally depending on the fold change and amount of the spiked proteins, producing signals that will be more or less difficult to extract from the raw data according to their intensities, it is important to challenge them with different simulated variations. In a previous study, Cox et al spiked UPS1 in combination with the UPS2 standard, which contains the same proteins than UPS1, but distributed into 6 groups of decreasing concentration, spanning 5 orders of dynamic range [START_REF] Cox | MaxLFQ allows accurate proteome-wide label-free quantification by delayed normalization and maximal peptide ratio extraction[END_REF]. By adding respectively these two standards into a background E.coli proteome, the authors simulated a situation where groups of proteins vary with different ratios, in a single pairwise comparison (6 analytical runs corresponding to 2 conditions with 3 technical replicates). However, in that case, only a small number of proteins are representative of each ratio, and many highly diluted UPS2 proteins are hardly detectable, creating a significant set of proteins which are differentially expressed but not really quantifiable.

In the present study, we chose to spike the UPS1 mixture at 9 different concentrations in a background yeast proteome, as described previously in Paulovich et al [START_REF] Paulovich | Interlaboratory study characterizing a yeast performance standard for benchmarking LC-MS platform performance[END_REF], and analyzed these samples in triplicate, resulting in a dataset of 27 runs. In order to artificially recreate a

A C C E P T E D M A N U S C R I P T

simulated dataset containing TP with different intensities and fold change values, we performed several pairwise comparisons by label-free quantification, and then combined the quantitative outputs.This approach has the benefit to illustrate the performances of the computational and statistical methods in a more comprehensive way. As a proof of principle, we show here the results obtained by simulating 3 kinds of variations (comparisons A, B and C: detection in only one condition; high fold change; moderate fold change). In principle, more comparisons could be performed and gathered to better approximate the inherent complexity of the variations that take place in a real biological experiment. For example, we didn't challenge here the software tools with comparisons involving only the more diluted spikes of the UPS1 concentration range, which would simulate variations of lower abundance proteins. Nevertheless, the different UPS1 spikes considered here could represent different types of biological samples, notably affinity purifications for large fold change analyses, or more classical proteome-wide analyses including moderate but significant expression fold change for some regulated proteins.

While label-free methods are more and more used for quantification of complex protein mixtures in biological studies, they are sometimes still considered as less accurate and reliable than label-based approaches. In addition, while many software tools for label-free quantification have been developed and are available, it may be difficult for an unexperienced user to choose a particular workflow. Finally, the quality of the results may be influenced by the parameter settings and the user's expertise with the programs. Consequently, test datasets are really needed to assess the performances of a given label-free workflow, adjust the parameters of a particular algorithm, and optimize post-processing methods such as missing value imputation, normalization, and statistical tests. The dataset presented here offers such possibilities, as illustrated on 8 different label-free pipelines which were objectively evaluated, and for which the number of FN and FP could be easily measured. The results

A C C E P T E D M A N U S C R I P T

On the other hand, our results indicated that workflows based on signal extraction clearly have the potential to be globally very sensitive, and are effective in detecting large variations as well as accurately measuring moderate fold changes. Sensitivity levels up to 90-100% could be reached by relaxing filtering criteria. Thus, when admitting FDP levels higher than 10%, such workflows outperformed spectral count methods for the classification of differentially expressed proteins in the dataset (Sup data 4B). They represent promising approaches to detect variations even on minor proteins expressed at low level in the sample, and/or showing subtle changes. However, it has to be noticed that at present, software tools based on MS intensity analysis still generate a significant number of FN and FP. The presence of false positive hits (type I error) associated to statistical tests in multiple comparisons is a well documented problem when using high-throughput analytical methods which enable the quantification of hundreds or thousands of species. When a large number of statistical tests are performed, the final proportion of false discoveries (FDP) is actually larger than the userspecified p-value cutoff used for each individual test. Multiple testing correction procedures are classically used to adust the individual p-values of each gene or protein, and to control the final FDR, such as the Benjamini-Hochberg method. Interestingly, spiked datasets, such as the yeast-UPS1 dataset provided here, allow to experimentally measure this FDP rate as well as the associated sensitivity, and could represent a useful tool for optimization of statistical processing steps for proteomic data. The Benjamini-Hochberg adjustment, while very effective for controlling the final FDR of the process, appeared to be very conservative and reduced strongly the sensitivity of the workflows. In our hands, empiric filtering based on the combination of p-value and fold change (or z-score) cutoffs offered a more efficient compromise to obtain good sensitivity with relatively low levels of experimentally measured FDP, although this FDP was not formally controlled through the statistical process. Clearly, further studies will be needed to implement statistical methods allowing to control the FDR

A C C E P T E D M A N U S C R I P T

rate when looking for differentially expressed proteins in proteomic experiments. For example, while we used here arbitrary, fixed fold-change and p-value cutoffs, other approaches have been described in which the fold-change cutoff can be modulated as a function of the t-test P-value, to increase sensitivity for a given FDR after Benjamini-Hochberg correction [START_REF] Carvalho | Improving the TFold test for differential shotgun proteomics[END_REF]. Additionnally, pre-filtering can also be implemented to eliminate lowly abundant proteins which tend to give artificially high fold change values after spectral count quantification, and create false positives [START_REF] Carvalho | Improving the TFold test for differential shotgun proteomics[END_REF]. Finally, other statistical methods have bee proposed previously for microarray data in order to take into account a fold-change threshold of interest in a formal hypothesis test with FDR control [START_REF] Dembele | Fold change rank ordering statistics: a new method for detecting differentially expressed genes[END_REF][START_REF] Mccarthy | Testing significance relative to a fold-change threshold is a TREAT[END_REF][START_REF] Vaes | Statistical analysis of differential gene expression relative to a fold change threshold on NanoString data of mouse odorant receptor genes[END_REF].

The occurrence of FP and FN hits is also a problem that has to be tackled upstream of statistical processing, at the level of quantitative analysis and raw data processing tools, as Although manual validation of the whole population of peptide ions, as performed in this study, is certainly overly long and impracticable in "real-life" biological studies, the ability to go back to the raw data for manual inspection of some specific proteins is probably an important feature for a label-free quantitative software. Indeed, the user can in this way really check the evidence for the differential expression of a protein, directly on the XIC and MS spectra of the different peptides. This manual verification can be performed on specific proteins that make biological sense (e.g. on some expected markers which would not be found as variants, due to signal extraction errors by the software, but also on new candidate proteins that will be subsequently selected for further validation studies, to ensure that these are not false positive).

In summary, our study on the presented standard dataset indicates that 1/ the number of falsepositive hits from label-free quantitative analysis is still significant, even with the best performing workflows, 2/ that manual verification by the expert allows to reduce it, illustrating that there is still some margin of improvement for the automatic signal extraction step by label-free software, and 3/ that a residual number of errors remain inherently difficult to avoid, independently of the quality of the signal extraction procedure, particularly in the case of co-elution and overlapping peptide features, which would in turn require better resolution of both chromatographic and MS instruments. Ideally, label-free software should offer good performances in order to keep this number of FP relatively low, but also offer a

A C C E P T E D M A N U S C R I P T

user-friendly interface allowing to efficiently going back to the raw data and check the MS signal extraction on all the peptides of a particular candidate protein.

Conclusion

As outlined in previous reports, benchmark datasets are really needed to evaluate software algorithms in mass spectrometry-based protein analysis, and should be made freely available [START_REF] Yates | Toward objective evaluation of proteomic algorithms[END_REF]. All raw MS data generated from the spiked standard presented here have been deposited

to the ProteomeXchange Consortium [START_REF] Vizcaino | ProteomeXchange provides globally coordinated proteomics data submission and dissemination[END_REF] via the PRIDE partner repository with the dataset identifier PXD001819, and quantitative outputs from the different workflows tested are given in ref [START_REF] Ramus | Spiked proteomic standard dataset for testing label-free quantitative software and statistical methods[END_REF], Sup Table 1. It must be noticed that all these results are dependent on the parameter settings used for each computational workflow, and to this respect, one main utility of this model dataset may be to help the users in optimizing the tuning and finding the best parameters for a particular tool. Additionally, we hope that such spiked datasets could be useful for developers in order to efficiently test algorithms and improve the extraction of intensity metrics for protein quantitation. Finally, post-processing steps such as possible normalization, imputation of missing values, and downstream statistical analysis will also strongly influence the results. The use of spiked datasets could be beneficial to objectively evaluate their performances and their ability to reduce the level of FP and correctly classify variant proteins in large-scale studies.

A C C E P T E D M A N U S C R I P T A C C E P T E D M A N U S C R I P T

  yeast samples spiked with 9 different concentrations of UPS1, analyzed in triplicate on an Orbitrap-Velos mass spectrometer. Starting from this dataset, different data processing workflows were implemented to perform relative quantification of proteins. Common statistical tests and fold-change criteria were used to identify differential peptides and proteins, for several theoretical fold variations of the spiked UPS1 standard. This experimental design allowed us to assess the performances of several workflows (4 based on spectral-count analysis and 4 based on MS signal analysis) in discovering true positive (UPS1 proteins successfully classified as variant) and avoiding false positive (yeast proteins A C C E P T E D M A N U S C R I P T erroneaously detected as variant). Overall, this study allowed to objectively evaluate labelfree quantitative methods and concretely illustrate what one can expect from these approaches in terms of false discovery proportion and sensitivity for the detection of variant proteins.

Workflow 3 :

 3 Mascot Distiller / Mascot / IRMa-hEIDI / Spectral Counting. Data were processed automatically using Mascot Distiller software (version 2.4.3.0, Matrix Science).

Workflow 8 :

 8 Mascot Distiller / Mascot / Skyline / MS Signal analysis. Peaklist creation was performed with Mascot Distiller as described in workflow 3, then database searches were performed with Mascot and validated with Scaffold as described for workflow 4. XIC signal corresponding to all validated peptides were extracted using the Skyline software[START_REF] Maclean | Skyline: an open source document editor for creating and analyzing targeted proteomics experiments[END_REF] A C C E P T E D M A N U S C R I P T [(Welch t-test difference) for all quantified proteins] } / Standard deviation [(Welch t-test difference) for all quantified proteins].

  classification. Resulting curves (Sup data 2A) show that the beta-binomial test was per se more efficient than a simple fold change to discriminate the TP from the TN. However, applying an additional fixed fold change cutoff improved significantly the results, as could beA C C E P T E DM A N U S C R I P T anticipated already from the volcano plots. On the dataset presented here, the best classification was obtained for all the workflows by applying this double-filtering approach with a threshold of 2 (or ½) on the fold change. Therefore sensitivity-FDP curves were plotted this way (variation of the p-value combined with a fixed threshold of 1 on the absolute log 2 (fold change)) for the different spectral count workflows as shown in Figure2B.Globally, the best results were obtained with workflow 3 (Mascot/IRMa-hEIDI) which allowed for example to obtain a reasonable sensitivity (62%) with a very low FDP (4%) when setting a stringent p-value threshold of 0.001. Leveraging the p-value threshold at 0.0025 led to a slightly better sensitivity (67%) at the cost of a FDP increase to about 10%. Interestingly, in the case of workflows 3 (Mascot/IRMa-hEIDI) and 4 (Mascot/Scaffold), it was possible to reach really low FDP values by increasing the stringency on the p-value, showing the efficiency of these data processing tools for the exclusion of FP. Altogether, it turns out that spectral count approaches were very efficient for detecting high levels of variations on relatively abundant proteins, but tends to fail to reach high sensitivity on the present dataset which includes a population with moderate fold change variations. Markedly, very low levels of FDP can be reached with appropriate filtering.Performances of MS intensity-based methods.Figure 3A shows the volcano plots from data obtained using different MS feature extraction tools (-log 10 (p-value) -calculated with the two-samples welch t-test from Perseus -plotted against the log 2 (fold change)). Conversely to what we observed with spectral-counting, the plots obtained with MS intensity-based techniques show that a large majority of UPS1 proteins quantified in the different pair-wise comparisons (green, red, and yellow populations) can be visually discriminated from the background of yeast proteins. While proteins with high signal levels and high theoretical fold changes were most often easily classified as variant (good p-values and high calculated fold A C C E P T E D M A N U S C R I P T changes), it can be noticed that even the UPS1 proteins quantified in the comparison C can be segregated from background, although with a partial overlap.

  with relatively low FDP. It has to be noticed that, unlike with the statistical t-test, setting a fold change threshold was quite sensitive to any shift in the population fold change distribution and to the optional normalization procedure applied in the workflows. Since some of the used methods contained a normalization step (e.g. MFPaQ or MaxQuant with the LFQ A C C E P T E D M A N U S C R I P T metric) and others not (e.g. Skyline or MaxQuant based on summed peptide intensity values), we used a z-score to avoid possible discrepancies between quantitative data depending on their origin. This z-score reflects, for each protein, the distance between the protein fold change and the mean of the population fold changes, relative to the standard deviation of this population (see Experimental procedures for calculation of the z-score). The combination of z-score and p-value criteria gave efficient discrimination results, as shown in Sup data 2B.For example, in the case of the MaxQuant LFQ workflow, we obtained a sensitivity of 94% and a calculated FDP of 8% when combining a |z-score| threshold of 1 and a Welch t-test pvalue threshold of 0.05.

Figure

  Figure3Bshows the sensitivity-FDP curves obtained for the MS intensity based workflows

  significantly reduced (from 22% to around 9%) (sensitivity and FDP values calculated by filtering proteins based on a welch t-test p-value<0.05 calculated with the two-samples test from Perseus and |z-score|>1). In addition, the calculated fold changes were closer to the expected theoretical values. It appeared that most of the extraction errors generating false positive hits were related to low intensity signals, as illustrated in Figure 4. Finally, after manual correction, no more than 8 yeast proteins were classified as variant. Out of these 8

  these false hits are very often associated to signal extraction or matching problems (Sup data 3). Indeed, extraction of peptide intensity values is a complex process based on MS peak picking, isotope pattern and chromatographic peak recognition, and association of peptide features with MS/MS identification results, which can be complicated by the frequent occurrence of overlapping peptides in the LC-MS space. In our comparison, the MaxQuant software performed the best when using the LFQ metric (Fig3B). In MaxQuant, the data analysis starts from the detection of features in the LC-MS map, based on recognition of elution peaks and peptide isotope profiles. In contrast, the processing in MFPaQ and Skyline is based on direct XICs extraction, using as a starting point m/z and RT coordinates derived from MS/MS identification results. Our study indicates that the later approach can however also produce good results, as illustrated by the good sensitivity and FDP obtained from MFPaQ quantification. A higher number of false positive hits were obtained with Skyline, which could be attributed in most instances to the absence of realignment procedure in the version of Skyline used for this study, and incorrect retrieving of peptide signals at deviatedA C C E P T E D M A N U S C R I P TRT in some of the conditions. On the other hand, the interactive interface of Skyline allowed to efficiently check the signal extraction, and enabled an in-depth manual verification which clearly improved the final quantitative results, and particularly allowed to reduce the number of false-positive. The reduction of false-positive is an important challenge in label-free based discovery proteomic approaches, as it will directly influence the success of further validation steps, based on the selection of protein candidates from the first quantitative analysis.

Figure 1 :

 1 Figure 1: Experimental design. A series of 9 yeast lysate samples spiked with growing

Figure 2 :

 2 Figure 2: Quantitative results obtained with spectral counting workflows.

Figure 3 :

 3 Figure 3: Quantitative results obtained with MS feature extraction workflows

Figure 4 :

 4 Figure 4: Manual feature-extraction correction in Skyline. The graphs illustrate the log 2 (fold

Figure 2 :

 2 Figure 2: Quantitative results obtained with spectral count workflows A

Figure 3 Figure 4 :

 34 Figure 3: Quantitative results obtained with software tools based on MS feature extraction A

Table

  1, consisting in the following steps: peaklist generation, database search, validation of the identified proteins and extraction of quantitative metric (spectral count or MS signal).

	According to the different tools used for each step, eight distinct workflows were evaluated.
	The same databases were used for peptide identifications: yeast database from UniprotKB
	(S_cerevisiae_ 20121108.fasta, 7798 sequences) and a compiled database containing the
	UPS1 human sequences (48 sequences).
	Workflow 1: ExtractMSn / Mascot / MFPaQ / Spectral Counting. The Mascot Daemon
	software (version 2.4; Matrix Science, London, UK) was used to perform database searches,
	using the Extract_msn.exe macro provided with Xcalibur (version 2.0 SR2; Thermo Fisher
	Scientific) to generate peaklists. Parameters used for creation of the peaklists were: parent
	ions in the mass range 400-4500, no grouping of MS/MS scans, and threshold at 1000.
	Peaklists were submitted to Mascot database searches (version 2.4.2). ESI-TRAP was chosen

Table 2 :

 2 FDP and TPR obtained on the spiked dataset for different quantitative workflows.Similar criteria were used for all workflows to classify proteins as variant (positive hits), i.e. |z-score|>1 and Welch t-test p-value<0.05. Human UPS1 proteins and yeast proteins verifying these criteria were counted respectively as True Positive and False Positive. False Discovery Proportion and True Positive Rate (sensitivity) were computed as described in the table.

			Maxquant	Maxquant
		MFPaQ	Intensity	LFQ	Skyline
		(workflow 5)	(workflow 6)	(workflow 7)	(workflow 8)
	True Positive	135	130	134	126
	False Positive	25	18	11	36
	FDP=FP/(FP+TP)*100	16%	12%	8%	22%
	TPR=TP/(TP+FN)*100	94%	90%	93%	88%

Acknowledgements:

This work was funded through the French National Agency for Research (ANR) (grant ANR-10-INBS-08; ProFI project, "Infrastructures Nationales en Biologie et Santé"; "Investissements d'Avenir" call).

(Skyline version v2.5, daily updates of April 2014, https://skyline.gs.washington.edu). This method was well described by Schilling et al (Schilling et al, MCP, 2012). Total areas, corresponding to the sum of the 3 extracted isotopes areas, were used for statistical analysis.

Statistical analysis.

For pairwise comparisons of samples spiked at different concentrations of UPS1, same statistical tests and fold-change criteria were applied to the quantitative data obtained from each workflow, as follows:

When working on spectral count metrics (workflows 1-2-3-4), a beta-binomial test was performed based on triplicate MS/MS analyses. p-values were calculated with the software package BetaBinomial_1.2 [START_REF] Pham | On the beta-binomial model for analysis of spectral count data in label-free tandem mass spectrometry-based proteomics[END_REF] implemented in R. Fold change was calculated as ratio of average spectral counts from both conditions. For proteins absent in all replicates of one specific condition, their spectral count values were modified by adding 1 spectrum to all 6 samples in order to be able to calculate a fold change for these particular proteins. To classify proteins as variant and non-variant and plot ROC curves, different combinations of criteria were tested (|log 2 fold change| > x, from 0.8 to 3 ; p-value < y, from 0.05 to 0.0001).

When working on MS signal intensity-based metrics (workflows 5-6-7-8), proteins were filtered out if they were not quantified in at least all replicates from one condition. Missing protein intensity values were replaced by a constant value calculated independently for each sample as the 5-percentile value of the total population. A welch t-test (two-tailed t-test, unequal variances) based on triplicate MS analyses was then performed on log 2 transformed values using the Perseus toolbox (version 1.4.0.11; http://141.61.102.17/perseus_doku).

Criteria used to classify the proteins were the Welch t-test difference calculated by Perseus (difference between the two compared conditions of the mean log 2 transformed value for triplicate MS/MS analyses), and the Welch t-test p-value. Results were filtered using different combinations of these criteria: |welch t-test difference| > x (from 0 to 7) and p-value < y (from 0.3 to 0.0001). z-score was also calculated as z-score= {(Welch t-test difference) -Median retrieving the spectral count metric could per se be seen as a basic process which is not errorprone, depending on the workflow used, some differences in FDP were observed at the same sensitivity levels. In fact, spectral count approaches are still dependent on the quality of peptide validation, selection and grouping, which may directly influence the performances of the different software tools tested here.

A C C E P T E D M A N U S C R I P T

Notes:

The authors declare no competing financial interest.

A C C E P T E D M A N U S C R I P T