Keywords: spectroscopy, signal processing, Cadzow denoising, Singular Value Decomposition (SVD), benchmarking graphical

Denoising applied to spectroscopies -part II: decreasing computation time

Spectroscopies are of fundamental importance but can suffer from low sensitivity.

Singular Value Decomposition (SVD) is a highly interesting mathematical tool, which can be conjugated with low-rank approximation to denoise spectra and increase sensitivity. SVD is also involved in data mining with Principal Component Analysis (PCA). In this paper, we focussed on the optimisation of SVD duration, which is a time-consuming computation. Both Intel processors (CPU) and Nvidia graphic cards (GPU) were benchmarked. A 100 times gain was achieved when combining divide and conquer algorithm, Intel Math Kernel Library (MKL), SSE3 (Streaming SIMD Extensions) hardware instructions and single precision. In such case, the CPU can outperform the GPU driven by CUDA technology. These results give a strong background to optimise SVD computation at the user scale.

Introduction

Spectrocopies are very efficient tools to help scientists in various domains: physics, chemistry, biology or medecine. However, the intrinsic sensitivity is highly dependant of the technique used. In particular, Nuclear Magnetic Resonance (NMR) [START_REF] Bonhomme | Recent NMR developments applied to organic-inorganic materials[END_REF] and Raman spectroscopy [START_REF] Das | Raman spectroscopy: Recent advancements, techniques and applications[END_REF] are poorly sensitive but are very precise local probes commonly used to study materials. While NMR can detect only one nucleus over 10 5 in usual conditions [START_REF] Levitt | Spin dynamics: basics of nuclear magnetic resonance[END_REF], Raman spectroscopy can only detect one photon over 10 6 [START_REF] Gautam | Raman and mid-infrared spectroscopic imaging: applications and advancements[END_REF]. Both NMR and Raman sensitivity were improved over the years, as was detailed in part (I) of this study [START_REF] Laurent | Denoising applied to spectroscopies -part I: concept and limits[END_REF], but still suffer from low signal-to-noise ratio (SNR), especially when studying amorphous or non-stable materials. Additionally, different physicochemical techniques can be hyphenated to obtain multiple signatures in a single experiment [START_REF] Tayler | Analysis of mass-limited mixtures using supercritical-fluid chromatography and microcoil NMR[END_REF][START_REF] Cardell | An overview of emerging hyphenated SEM-EDX and Raman spectroscopy systems: applications in life, environmental and materials sciences[END_REF]. This sensitivity gain entails an increasing amount of data to analyse, especially in the domain of metabolomics [START_REF] Ali | Simultaneous time-dependent surface-enhanced raman spectroscopy, metabolomics, and proteomics reveal cancer cell death mechanisms associated with gold nanorod photothermal therapy[END_REF][START_REF] Marchand | Multidimensional NMR approaches towards highly resolved, sensitive and high-throughput quantitative metabolomics[END_REF]. In such case, it is necessary to combine statistics and chemistry, what is called chemometrics [START_REF] Gemperline | Practical guide to chemometrics[END_REF][START_REF] Elmi Rayaleh | Extraction des connaissances en imagerie microspectrométrique par analyse chimiométrique : application à la caractérisation des constituants d'un calcul urinaire[END_REF]. A similar problem of data mining is present in social engineering [START_REF] Chen | Mining social media data for understanding students' learning experiences[END_REF] or with medical images dictionaries [START_REF] Wang | An improved coupled dictionary and multi-norm constraint fusion method for CT/MR medical images[END_REF]. Many tools are available to process these data: Principal Component Analysis (PCA) [START_REF] Wold | Principal component analysis[END_REF], Principal Component Regression (PCR) [START_REF] Wu | Fast regression methods in a Lanczos (or PLS-1) basis. Theory and applications[END_REF], Partial Least Squares (PLS) [START_REF] Geladi | Partial least-squares regression: a tutorial[END_REF], Discriminant Analysis (PLS-DA) [START_REF] Gromski | A tutorial review: metabolomics and partial least squaresdiscriminant analysis -a marriage of convenience or a shotgun wedding[END_REF], Independent Component Analysis (ICA) [START_REF] Comon | Independent component analysis, A new concept?[END_REF], or Non-negative Matrix Factorisation (NMF) [START_REF] Woelffel | Analysis of soda-lime glasses using non-negative matrix factor deconvolution of Raman spectra[END_REF]. The aim of these multivariate data analysis methods is to find relevant parameters in order to discriminate samples (ex wine from region A or B [START_REF] Monakhova | Independent components analysis to increase efficiency of discriminant analysis methods (FDA and LDA): application to NMR fingerprinting of wine[END_REF]). These tools are of paramount importance to apply data mining to spectroscopies [START_REF] Maione | Recent applications of multivariate data analysis methods in the authentication of rice and the most analyzed parameters: a review[END_REF]. For instance, PCA was used recently with Gas Chromatography / Quadrupole Time-of-Flight (GC/Q-ToF) mass spectrometry [START_REF] Wang | Application of GC/Q-TOF combined with advanced data mining and chemometric tools in the characterization and quality control of bay leaves[END_REF], Mid-InfraRed (MIR) spectroscopy [START_REF] Vimalajeewa | Evaluation of non-linearity in MIR spectroscopic data for compressed learning[END_REF] and Inductively Coupled Plasma Optical Emission Spectroscopy (ICP-OES) [START_REF] Othman | Assessing risk and sources of heavy metals in a tropical river basin: a case study of the Selangor river, Malaysia[END_REF]. An important step of PCA and relative techniques is Singular Value Decomposition (SVD) [START_REF] Orfanidis | SVD, PCA, KLT, CCA, and all that[END_REF][START_REF] Shlens | A tutorial on principal component analysis[END_REF].

Moreover, SVD was proposed as a method to denoise signals by Tufts et al [START_REF] Tufts | Data adaptive signal estimation by singular value decomposition of a data matrix[END_REF], and was generalised by Cadzow in 1988 [START_REF] Cadzow | Signal enhancement -a composite property mapping algorithm[END_REF]. In the context of low sensitivity and in the continuation of a previous communication [START_REF] Laurent | SVD Performances to denoise NMR and Raman spectra[END_REF], we thoroughly described SVD in part (I) of this work [START_REF] Laurent | Denoising applied to spectroscopies -part I: concept and limits[END_REF]. We first gave theoretical background on SVD, low-rank approximation [START_REF] Tufts | Estimation of a signal waveform from noisy data using low-rank approximation to a data matrix[END_REF], Hankel or Toeplitz matrices [START_REF] Heinig | Toeplitz and Hankel matrices[END_REF] and SNR definitions. SVD was applied to Raman and NMR spectra. We highlighted that best results were obtained with square matrices and data in time domain rather than in frequency domain. Automatic thresholding was applied thanks to Malinowki's significant level indicator [START_REF] Malinowski | Factor analysis in chemistry[END_REF].

6 × 7380 = 44280 denoised spectra with known noise were compared to their non-noisy counterparts. It was evidenced that the minimum peak SNR measured on maximum of noise (PSNR max ) needed to have reliable results was PSNR max = 2, leading to a gain on acquisition time of 2.3. Surprisingly, while Lorentzian peaks were correctly denoised, SVD transformed Gaussian peaks into intermediate Gaussian / Lorentzian ones, which overestimated their peak area by 20 %.

The main disadvantage of SVD is its long computation time, especially for big data sets. It is thus essential to optimise the computation procedure. Different approaches have been chosen in literature: specialised processors [START_REF] Cavallaro | CORDIC arithmetic for an SVD processor[END_REF], wavelet transformation before performing the SVD [START_REF] Vogt | Fast principal component analysis of large data sets[END_REF], divide and conquer method [START_REF] Gu | A Divide-and-Conquer Algorithm for the Bidiagonal SVD[END_REF] or sparse matrices [START_REF] Halko | Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions[END_REF][START_REF] Condat | Cadzow denoising upgraded: a new projection method for the recovery of Dirac pulses from noisy linear measurements[END_REF]. Another approach is General Purpose computing on Graphics Processing Unit (GPGPU) [START_REF] Bourgoin | High level data structures for GPGPU programming in a statically typed language[END_REF]. SVD has been applied multiple times using GPGPU [START_REF] Andrecut | Parallel GPU implementation of iterative PCA algorithms[END_REF][START_REF] Lahabar | Singular value decomposition on GPU using CUDA[END_REF][START_REF] Novakovic | A GPU-based hyperbolic SVD algorithm[END_REF][START_REF] Irofti | GPU parallel implementation of the approximate K-SVD algorithm using OpenCL[END_REF]. Two programming languages are available: CUDA (Compute Unified Device Architecture) for Nvidia graphic cards [START_REF] Nickolls | Scalable parallel programming with CUDA[END_REF] and OpenCL (Open Computing Language) for all graphic cards (Graphic Processing Unit, GPU) and processors (Central Processing Unit, CPU) [START_REF]OpenCL parallel computing for heterogeneous devices[END_REF]. In addition, Dongarra et al. developed very efficient algorithms combining CPU and GPU [START_REF] Dongarra | Accelerating numerical dense linear algebra calculations with GPUs[END_REF][START_REF] Gates | Accelerating the SVD two stage bidiagonal reduction and divide and conquer using GPUs[END_REF], what is called heterogeneous computing [START_REF] Xie | A Few of the Most Popular Models for Heterogeneous Parallel Programming[END_REF]. Recently Man et al proposed a Java implementation of SVD for NMR (48) using CPU [START_REF] Man | Java application for FID denoising with SVD[END_REF] or Nvidia GPU [START_REF] Man | Graphic processing unit (GPU), 2012 SVD Java 7 application for FID denoising[END_REF]. Pending questions can be raised on these Java applications: (i) how powerful does the computer need to be? (ii) how long does computation take? (iii) how large can be the data set? (iv) is the used algorithm efficient? (v) will other programming languages give better results?

In this second part (II), after providing some experimental details in section "Materials and methods", we benchmarked SVD using Java, on various CPU and Nvidia GPU ranging over 10 and 6 years, respectively. We then focussed on algorithms and precision under Matlab. In a further step, we tried to decorrelate software libraries from hardware capabilities (Single Instruction Multiple Data, SIMD) [START_REF] Hofmann | Comparing the performance of different x86 SIMD instruction sets for a medical imaging application on modern multi-and manycore chips[END_REF]. Finally, we compared Java, Matlab and Python to reach the fastest possible denoising computation.

Materials and methods

Solid-state NMR experiments

Two solid-state NMR spectra were used to benchmark SVD. The first one was a 29 Si spectrum with 4096 complex points, used for matrices up to 2015 × 2014 = 4.1e6. For matrices above this limit, a 87 Sr spectrum with 30504 complex points, was chosen. The noisy and denoised spectra for 29 Si and 87 Sr are presented on Figures 1a and1b, respectively. SVD was applied on Free Induction Decay (FID, time domain) after removal of the first 68 points corresponding to oversampled digitalisation. The sample analysed by 29 Si solid state NMR and the experiments were presented in part (I) of this study [START_REF] Laurent | Denoising applied to spectroscopies -part I: concept and limits[END_REF]. Briefly, it was representative of sol-gel chemistry, combining hydrophobicity and mechanical properties [START_REF] Faustini | Hydrophobic, antireflective, self-cleaning, and antifogging sol-gel coatings: an example of multifunctional nanostructured materials for photovoltaic cells[END_REF]. A 50:50 mix of Methyltriethoxysilane (MTEOS) : tetraethylorthosilicate (TEOS) was prepared by spray drying giving spherical micrometer silica particles. Carr-Purcell-Meiboom-Gill (CPMG) Magic Angle Spinning (MAS) experiments [START_REF] Malfait | Increased 29 Si NMR sensitivity in glasses with a Carr-Purcell-Meiboom-Gill echotrain[END_REF] were performed in 40 minutes on a Bruker Avance III spectrometer operating at 300.29 MHz for 1 H and 59.65 MHz for 29 Si.

The sample analysed by 87 Sr solid state NMR was a model of biocompatible material in relation with bone substitutions [START_REF] Laurencin | 87 Sr, 119 Sn, 127 I single and { 1 H/ 19 F}-double resonance solidstate NMR experiments: application to inorganic materials and nanobuilding blocks[END_REF]. Non-hydrated non-protonated Sr(PO 3 F) was studied on a Bruker Avance III spectrometer operating at 699.98 MHz for 1 H and 30.34 MHz for 87 Sr in a 5 mm static probe. In order to enhance sensitivity, DFS [START_REF] Iuga | Population and coherence transfer induced by double frequency sweeps in half-integer quadrupolar spin systems[END_REF], WURST [START_REF] Schurko | Ultra-wideline solid-state NMR spectroscopy[END_REF] and CPMG were used with 58,000 transients and a relaxation delay of 300 ms, leading to a total acquisition time of 5.5 h. 260 echoes were acquired with a full echo delay of 0.12 ms.

Measurement of SVD computation times

Special care has been taken to validate measured times: the computer was checked to be idle, without any update running and without an active internet browser window, which would use the graphic card through Adobe Flash module. Computations were systematically repeated and results were highly reproducible. Nvidia drivers ranging from 331.113 to 352.30 were used, either under Linux Ubuntu 14.04, Centos 5, Fedora 21, 22 or Windows XP, 7 SP1 or 8.1. Under Windows it was necessary to modify the TdrLevel registry key to 0 in order to avoid graphics driver failure [START_REF]TDR registry keys[END_REF]. Under Java (Oracle Corporation, Redwood Shores, CA, USA) and Matlab (The MathWorks, Inc., Natick, MA, USA), 29 Si FID was used up to 4028 points. Above this value, 87 Sr FID was used. The corresponding FID was truncated if needed to the desired data length. Under Python [START_REF] Van Rossum | Python Tutorial[END_REF], data set was a simple list of increasing values with the corresponding length. Unless otherwise stated, k = 25 singular values were kept for low-rank approximation. This value corresponded to the major spikelets observed on 29 Si spectrum (Figure 1a) and was not changed for coherence along the series. The measured delays are the sum of decomposition and low-rank approximation steps, including all processor to graphic card latencies, if relevant.

Java

Two applications are available online: one for CPU [START_REF] Man | Java application for FID denoising with SVD[END_REF] and the other one for Nvidia GPU [START_REF] Man | Denoising NMR timedomain signal by singular-value decomposition accelerated by graphics processing units[END_REF][START_REF] Man | Graphic processing unit (GPU), 2012 SVD Java 7 application for FID denoising[END_REF]. While CPU version calls JAMPACK library [START_REF] Stewart | Jampack: a Java matrix package[END_REF], GPU version calls CULA R15 [START_REF] Humphrey | CULA: hybrid GPU accelerated linear algebra routines[END_REF]. The CPU 32 bits version failed above a matrix size of 1025 × 1024 = 1.0e6 points, whatever the CPU used. The same problem was observed with GPU 32 bits version above a matrix size of 3005 × 1024 = 3.1e6 points. The reason is that Java heap space is limited to around 1.5 GB for 32 bits applications [START_REF] Javin | What is the maximum Heap Size of 32 bit or 64-bit JVM in Windows and Linux? Available at[END_REF], while it is 16 exabytes for 64 bits applications. This memory amount corresponds not only to the data, but also to the program and its libraries. 64 bits applications are not compatible with 32 bits Java runtime environment and 32 bits operating systems. As the source code was not accessible and no internal timer was implemented in these Java applications, computation times were measured with a handheld chronograph giving a time resolution of 1 s for both SVD step and low-rank approximation step. For a same GPU, no computation time difference was observed between Windows and Linux.

Matlab

Three versions were tested: R2010a, corresponding to the most recent compatible with CULA library free (in version R14) [START_REF] Laurent | SVD under Matlab with CULA link[END_REF]; R2014a, corresponding to the most recent for old graphic cards with Compute Capability (CC) less than 2.0, such as GTX 260; and R2015a, corresponding to a recent version. FID were imported thanks to matNMR [START_REF] Van Beek | matNMR: A flexible toolbox for processing, analyzing and visualizing magnetic resonance data in Matlab®[END_REF]. Under R2014a and R2015a, their respective Parallel Computing Toolbox was added to use gpuArray function. Computation times were measured with an internal timer. The source code is available in file Figure_II.4a_II.4b.m of [START_REF] Laurent | Denoising applied to spectroscopies: parts I and IIprograms and datas[END_REF].

Python

No significant time difference was observed between Python 2.7 and 3.5 versions, neither between 32 and 64 bits versions. The source code is compatible with all of these options and is available in file Figure_II.5.py of (64) (CPU and GPU). Different SVD implementations were tested with NumPy 1.10.1 (65), SciPy 0.16.1 [START_REF] Jones | SciPy: open source scientific tools for Python[END_REF] and Scikit-Cuda 0.5.0 [START_REF] Givon | scikit-cuda 0.5.1: a Python interface to GPU-powered libraries[END_REF]. The last one required PyCUDA 2015. 1.3 (68) and CULA R18 free [START_REF] Humphrey | CULA: hybrid GPU accelerated linear algebra routines[END_REF]. Computation times were measured with an internal timer as well.

Under Linux, Automatically Tuned Linear Algebra Software (ATLAS) [START_REF] Whaley | Automatically Tuned Linear Algebra Software[END_REF] and Open source Basic Linear Algebra Subprograms (OpenBlas) (70) development libraries, separately either one or the other, were installed as rpm packages. For each library, NumPy and SciPy were build with the pip mechanism 1 . Compiling these two latter packages with Intel Math Kernel Library (MKL) (71) was also tested. Under Windows, NumPy and SciPy superpack 32 bits with ATLAS library are available [START_REF][END_REF][START_REF]SciPy: Scientific library for Python[END_REF], and also pre-compiled packages with MKL library [START_REF] Gohlke | Python extension packages for windows[END_REF].

Results and discussion

As stated in the introduction, the main disadvantage of SVD is its long computation time. Of course the hardware itself is important but multiple steps are present between the human level function call and the hardware level implementation, namely the algorithm, the libraries and the use of hardware instructions (Figure 2a). Even on hardware, we can choose to compute either on CPU or on GPU. In the following, we checked the respective benefits of all these parameters.

1

Forcing reinstallation of a specific python package can be done with 'pip install --ignoreinstalled numpy==1.10.1'.

Influence of hardware under Java CPU and GPU applications

Java CPU and GPU benchmarks

In an attempt to characterise the hardware needed to compute SVD, we used CPU ranging over 10 years and GPU ranging over 6 years, for both desktop and laptop computers (Tables 1 and2 and Figure 2b). According to Figure 2a, we were changing hardware level, supposing that everything was optimised at software level and measuring durations at human level. Study has been restricted to Intel CPU and Nvidia GPU because no AMD CPU was available in the laboratory and AMD GPU are not compatible with CUDA. Results are presented on Figure 3a.

We noticed first that computation were much faster for GPU than for CPU, with 2 to 23 s for GPU (dashed lines) and 67 to 500 s for CPU (top right end of plain lines),

i.e. 22 to 34 times speed increase, for a quite small matrix of 1025 × 1024 = 1.0e6

points. This almost square shape was due to construction of the Hankel matrix with a dataset containing an even number of points. Indeed, in order to not overwrite the corner point, it is necessary to add one row over columns. No significant time difference was seen with a true square matrix and thus this small shape difference will be neglected in the following.

Surprisingly, even a low-end GPU of 2008 (8400 GS) was surpassing a middlerange CPU of 2013 (Core i5 4670K) (with 23 and 67 s, respectively). This behaviour was really intriguing, as we would expect at least similar computation performance (75).

However, when checking CPU activity, we observed that, with the CPU application, only one core was busy, what is called mono-threading. On the contrary, with the GPU application, not only GPU was fully busy, but also all the available CPU cores were used, what is called multi-threading. This difference between mono-and multithreading is explored in section"Influence of algorithm under Matlab". When increasing the square matrix size, a linear trend was visible in logarithmic scale, down-shifted for faster hardware (Figure 3a). However, computation time jumped when going from a rectangular matrix (diamond symbol) to a square one (square symbol), even if matrix size was not so different. This behaviour was observed for both CPU and GPU at 1537 × 512 = 7.9e5 vs. 1025 × 1024 = 1.0e6 points, and similarly for GPU at 3005 × 1024 = 3.1e6 vs. 2015 × 2014 = 4.1e6 points. This denoted that the number of mathematical operations dramatically increased for a square matrix. An explanation could be the use of reduced SVD for rectangular matrices, not computing the last rows and columns of U and V T unitary matrices (see part (I) of this work ( 5)).

This time jump is probably the reason why a rectangular matrix shape is chosen in most studies, despite a square matrix gives more precise singular values, i.e. more performant denoising.

Thanks to the speed up obtained on GPU, it was possible to compute a matrix of 4097 × 4096 = 1.7e7 points in 31 s on a mid-range GPU of 2012 (GTX 660). By extrapolating the curve in Figure 3a, it may take around 6000 s on a mid-range CPU of 2013 (Core i5 4670K). For square matrices, the slope seems similar between graphic cards. As underlined in section "Materials and methods", only 64 bits Java versions can handle matrix sizes above 3005 × 1024 = 3.1e6 points. However, all the processors and some graphic cards (NVS 160M, FX 770M and 820M) were benchmarked using the 32 bits Java applications, which explains the truncated curves for this hardware.

Java CPU performance indicator

To better characterise the hardware needed for Java CPU SVD, we looked for a performance indicator. One would expect CPU frequency to be a good one but it was clearly not the case as shown on Figure 3b. However, mono-thread performance evidenced a trend and was thus a usable parameter as shown on Figure 3d. This is coherent with our observation of only one active processor core under the Java CPU application.

Characteristic points are visible on these curves (Figures 3b and3d). Light blue arrows highlight Pentium M 745 and Pentium 4 530, which were two processors of 2004. Their frequencies were very different (1800 and 3000 Mhz, respectively) but their mono-thread performance were similar. Additionally, with a bigger matrix size (yellow line), the Pentium M 745 was faster than the Pentium 4 530, even if the latter had a higher frequency. The main difference between them was their cache size, of 2 and 1 MB, respectively. The CPU cache is the amount of quick memory directly available inside the processor. On the contrary, memory plugged into motherboard is at least 10 times slower. As SVD request many matrix-vector multiplication, memory access is limiting.

Light green arrows evidence Core 2 Duo E6400 and Core i3 4005U, which were two processors with a similar frequency but released in 2006 and 2013, respectively. No performance increase was observed using Java SVD CPU application between these two processors. That was also questioning as we would expect that some hardware optimisations happened in 7 years. These observations highlight that the best CPU for SVD under Java will not necessarily be recent or have a high frequency, but rather have a high memory cache and a high monothread performance. In other words, it is better to use an old high-or middle-range CPU than a new low-range one. This explains why Core 2 Duo T9600 was faster than Core i3 4005U. The former processor is a good candidate to denoise all over the night a matrix of 8193 × 8192 = 6.7e7 points with the 64 bits CPU Java application.

Java GPU performance indicator

Graphic card computing power is characterized by core frequency and number of cores.

Despite frequency was not so different along the series, number of cores was strongly increasing over the years and with them the Single Precision Floating Point performance (SP or FP32), expressed in Giga FLoating-point Operations Per Second (GFLOPS) [START_REF] Dolbeau | Theoretical peak FLOPS per instruction set: a tutorial[END_REF]. Only high-end professional cards have a high Double Precision

Floating Point performance (DP or FP64). General public GPU are more commonly devoted to games and lack DP. The precision is the number of bits used to store numbers, 32 and 64 for SP and DP, respectively [START_REF] Whitehead | Precision & performance: floating point and IEEE 754 compliance for NVIDIA GPUs[END_REF]. The higher the precision used, the lower the computed error. However, the errors initially present in the matrix can be larger than rounding errors [START_REF] Stewart | Perturbation theory for the singular value decomposition[END_REF]. Moreover, CULA free ( 60), the library implemented on Java GPU application could only use SP. It was thus useless to invest money in professional cards and we favoured general public GPU. For instance, a Nvidia Tesla P100 GPU costs around 8 k€. On Figure 3c, a time decreasing linear trend was obtained in logarithmic scale when increasing SP, which denoted a good indicator.

Another important parameter for SVD with Java GPU application, was the amount of memory available, both on GPU (device) and on motherboard (host).

Plassman stated that SVD needed up to 8 n² + 12 n work storage [START_REF] Plassman | A survey of singular value decomposition methods and performance comparison of some available serial codes[END_REF], for a matrix with n columns. This value had to be multiplied by 4 bytes for both floating and integer numbers to be stored in memory. Additionally, there was a 1.5-3 times transient overhead during low-rank approximation. Following this rule, the largest tractable matrix was 6657 × 6656 = 4.4e7 points on a GPU with 2 GB of memory.

In this section we have seen influence of hardware on SVD computation time under Java. As stated above, the time difference between CPU and GPU Java application was intriguing, especially when comparing the low-end graphic card 8400 GS to the middle-range processor Core i5 4670K. Additionally, the CPU and GPU application were mono-threaded and multi-threaded, respectively. This was typically an algorithm problem and we explored it using Matlab.

Influence of algorithm under Matlab

Algorithms are the mathematical operations involved and their informatics implementation to obtain the relevant function. Plassman compared the available SVD algorithms and their impact on ill-conditioned matrices [START_REF] Plassman | A survey of singular value decomposition methods and performance comparison of some available serial codes[END_REF]. The simplest computation method is to use eigendecomposition, but its lack of precision was demonstrated by

Läuchli [START_REF] Läuchli | Jordan-elimination und ausgleichung nach kleinsten quadraten[END_REF]. The classic complete SVD uses a three steps process:

(1) reduction to bidiagonal form, (2) computation of SVD on bidiagonal matrix, (3) obtention of singular vectors.

Step (1) involves Householder reflections and step (2) can either use QR iteration in Golub-Kahan-Reinsch (GKR) algorithm [START_REF] Golub | Calculating the singular values and pseudoinverse of a matrix[END_REF][START_REF] Golub | Singular value decomposition and least squares solutions[END_REF], divide-and-conquer method [START_REF] Gu | A Divide-and-Conquer Algorithm for the Bidiagonal SVD[END_REF][START_REF] Cuppen | A divide and conquer method for the symmetric tridiagonal eigenproblem[END_REF] or Multiple Relatively Robust Representations (MRRR) [START_REF] Willems | Computing the bidiagonal SVD using multiple relatively robust representations[END_REF]. An alternative SVD algorithm, combining steps (1) and ( 2), is to use Jacobi rotations and convergence criteria [START_REF] Demmel | Jacobi's method is more accurate than QR[END_REF].

To explore algorithm influence we focussed on two computers, one from 2008 with a Core 2 Quad Q8200 and a GTX 260 under Linux, and the other one from 2013 with a Core i5 4670K and a GTX 660 under Windows. Both were in the same price segment and reflected middle-range equipment available at those dates. The used Matlab versions were detailed in section "Materials and methods". According to Figure 2a, we fixed hardware level and observed software level influence.

Matlab R2010a

Under Matlab R2010a, DP computation times on CPU were already smaller than those with Java CPU application for a matrix of 1025 × 1024 = 1.0e6 points (plain dark blue line on Figures 4a and4b): 78.2 s instead of 149 s for Core 2 Quad Q8200, and 11.6 s instead of 67 s for Core i5 4670K. It should be noted that computation times can be divided by two when using single precision (plain red line), and by two again with noncomplex data (not shown). Java CPU application used complex numbers and accordingly to the above results, it presumably used DP. While Java used only monothreading, Matlab computation started with a multi-threaded step and kept on with a mono-threaded one. This already denoted a different algorithm between the two programming languages.

Under this Matlab version it was also possible to use GPU with CULA, which was the library implemented under Java GPU application. Results were similar between Java GPU and Matlab R2010a + CULA + SP GPU applications (dashed red line).

However, as we used the free version of CULA, DP computation was not allowed on GPU and R2010a + CULA + DP (plain black line) felt back on CPU with LAPACK library in multi-threading mode. As a consequence, strong improvement was observed against R2010a + DP (plain dark blue line). At this point, an order of magnitude on computation times has already been gained for CPU DP under Matlab.

Matlab R2014a

Further improvement on CPU was obtained with R2014a + DP (plain yellow line on Figures 4a and4b) being almost two times faster than R2010a + CULA + DP (plain black line), and 7-33 times faster than R2010a + DP (plain dark blue line), depending on matrix size. This was explained by the divide and conquer approach preferred for SVD starting from R2010b. Gu et al claimed that this algorithm was 9 times faster on bidiagonal matrices [START_REF] Gu | Efficient computation of the singular value decomposition with applications to least squares problems[END_REF], in agreement with our observations. The extra gain is due to the four cores simultaneously used on the CPU. Again SP (plain green line) was two times faster than DP. Small matrices, up to 1025 × 1024 = 1.0e6 points for Core 2 Quad Q8200, and up to 3073 × 3072 = 9.4e6 points for Core i5 4670K, were even computed faster on CPU with R2014a + SP than on GPU with R2010a + CULA + SP, as indicated by the green arrow. Unfortunately, CULA free was not compatible with R2014a, but it was nevertheless possible to use GPU thanks to the gpuArray Matlab function.

Surprisingly, worst results were obtained, with a GPU time longer than its corresponding CPU time. Moreover R2014a + gpuArray + DP times (dashed yellow line) were shorter or equal to R2014a + gpuArray + SP times (dashed green line), what is in contradiction with DP / SP ratio on GPU (1 / 8 and 1 / 24 for GTX 260 and GTX 660, respectively). This revealed that part of the computation was done in DP, despite SP was called. When checking CPU and GPU activity during SVD, it was observed that GPU was only used at the beginning and at the end of the processing. This denoted that SVD using gpuArray under Matlab R2014a was not an optimised algorithm and that this version should be avoided.

Matlab R2015a

In order to check if a new version of Matlab could further improve computation times, we used Matlab R2015a. A slight decrease was observed on CPU from 57.8 to 52.5 s and from 29.1 to 27.0 s for R2014a + DP (plain yellow line on Figures 4a and4b), R2015a + DP (plain brown line), R2014a + SP (plain green line) and R2015a + SP (plain light blue line), respectively, for a matrix of 4097 × 4096 = 1.7e7 points. Matlab R2015a was not compatible with GTX 260 GPU, due to its compute capability of 1.3. A algorithm used and on the relative float performance of CPU and GPU. Low-end GPU are thus not recommended as better results are obtained with CPU. In DP mode, despite the half computing power of a Core i5 4670K against SP, even a GTX 660 never overpassed it (not shown).

Similarly to Java, matrix size under Matlab is limited by host and device memory. Nevertheless, memory consumption under Matlab is improved over Java as no overhead is present during low-rank approximation, pushing away maximum matrix size. Additionally, host memory amount is considerably reduced, down to be almost identical to device memory one. A GPU with 2 GB of memory is thus limited to matrices of 7169 × 7168 = 5.1e7 points.

In this section, we highlighted that the divide and conquer algorithm decrease and4b. Matlab R2014a is not recommended neither for GPU. Next step was to focus on the libraries used and their call to hardware instructions, which we explored under Python.

Influence of libraries and hardware instructions under Python

According to Figure 2a, software level is divided in algorithms and libraries. After changing the algorithms, i.e. the involved mathematical functions, we were interested in the underlying libraries, i.e. the link between software level and hardware level. A library is a collection of functions that consists of pre-written optimised code

SSE and AVX hardware instructions

Despite MKL seemed very promising, it was not the only factor changing in the above experiment as the implemented hardware optimisations changed from SSE2 When moving from no SSE (left column) to SSE3 (middle column), computation times were divided by three with an additional gain for SP. Moreover, when using both MKL and AVX2, a huge performance was obtained, outperforming GPU computation with CULA + SP (hatched red line).

As underlined here, the time needed to perform SVD was impressively reduced by a factor of 38 on the same CPU under Python, by optimising the used libraries and their hardware calls. Indeed, decomposition of a matrix of size 1025 × 1024 = 1.0e6 points was done with SciPy in 7.6 s without optimisations and in 0.2 s using MKL library and AVX2 instructions.

Comparison of Java, Matlab and Python

Computation times

In previous sections, Java, Matlab and Python software were used for their specific testing capabilities. But how do they compare to each other? In order to answer this question, Figure 6 shows SVD computation times for a matrix of 1025 × 1024 = 1.0e6 Python, both with MKL library and all available SIMD instructions activated 6 . For this small matrix of 1025 × 1024 = 1.0e6 points, the CPU outperformed the GPU, due to data transfer delays limiting GPU efficiency. However, for larger matrices, computation was faster on GPU.

Comparing Java CPU and Python with MKL, there was a gain of 100 on the same CPU. This was explained as follows:

• a factor of 9 using the divide and conquer algorithm

• a factor of 3 using hardware instructions such as SSE3 or AVX2

• a factor of 2 using MKL library

• a factor of 2 using single precision instead of double precision

Maximum matrix size

This major time improvement raise the question of the absolute maximum matrix size that could be computed using SVD and low-rank approximation. As underlined in section "Java GPU performance indicator", the limiting parameter is memory, both on GPU device side and on CPU host side. The crucial point is to use 64 bits applications and a GPU with as much memory as possible. Nevertheless, this will depend on the way memory is allocated and released during SVD process. Our late investigations, on a GTX 1070 with 8 GB of memory and 7040 SP GFLOPS, gave the following maxima on our 87 Sr FID. The computation times were the sum of SVD and low-rank steps. To maximize the latter, the operation was performed with all singular values, that is to say without any denoising.

• Java GPU: 9217 × 9216 = 8.5e7 complex points in 150 + 32 = 182 s.

• Matlab R2018a + gpuArray + SP: 12289 × 12288 = 1.5e8 complex points in 188 + 4 = 192 s.

• Python + CULA + SP: 15219 × 15218 = 2.3e8 complex points in 574 + 6 = 600 s.

Under Python, it was thus possible to apply SVD on the full 87 Sr FID, without any truncation. For comparison, a Nvidia P100 GPU, with 4670 DP GFLOPS and 16 GB of memory, completed the full SVD of a 20000 × 20000 = 4.0e8 real matrix in 90 s, with a highly optimised CPU-GPU algorithm [START_REF] Gates | Accelerating the SVD two stage bidiagonal reduction and divide and conquer using GPUs[END_REF]. This result was really impressive as the authors obtained a faster computation on a much larger matrix with less computing power and double precision. There is thus plenty of place to improve SVD denoising.

Directly comparing Java, Matlab and Python was a difficult task as they were not optimised in the same way and it was hard to check what was hidden under the hood. However, Java GPU was less performant, both in speed and in matrix size. Better results may be obtained with optimised libraries. While Matlab was faster, the memory usage was limiting. Python computation was not as fast but could handle the biggest matrix. This time advantage for Matlab was explained by a better CPU usage during SVD on GPU. However memory was more finely managed under Python. Our results suggested that the key parameter was not the software and the programming language, but rather the used libraries and the calls to hardware instructions.

An additional advantage of Python is that it is free of charge and rather easy to program. In order to compute SVD in a minimum amount of time, we recommand to install a Python distribution with MKL library included, such as Anaconda [START_REF] Anaconda | Anaconda software distribution[END_REF], and to add the following libraries: SciPy (73), scikit-cuda [START_REF] Givon | scikit-cuda 0.5.1: a Python interface to GPU-powered libraries[END_REF], PyCUDA [START_REF] Klöckner | PyCUDA and PyOpenCL: A scripting-based approach to GPU run-time code generation[END_REF], and nmrglue [START_REF] Helmus | Nmrglue: an open source Python package for the analysis of multidimensional NMR data[END_REF] for NMR data. In addition, CULA [START_REF] Humphrey | CULA: hybrid GPU accelerated linear algebra routines[END_REF] and CUDA toolkit (43) packages are necessary. We provide in file svd_auto.py of (64) an optimised SVD function using either the CPU or the GPU, depending on the matrix size. Our tests suggested a minimum value of 4096 columns or rows to switch from the CPU to the GPU. This default value will depend on the hardware used and can be checked by running directly the program. The code is designed to be as simple as possible, with only one necessary parameter, namely the matrix two-dimensional array. Automatic thresholding is applied using Malinowski's significant level indicator [START_REF] Malinowski | Factor analysis in chemistry[END_REF]. This SVD function is also suitable to be used in PCA and related data mining techniques. In addition, we provide a second program (file denoise_nmr.py of (64)), in charge of importing and exporting Bruker NMR data and to prepare the matrix transferred to SVD program. Again, the only requested parameter is the data directory.

Conclusion

This article separated in two parts focussed on SVD, which is used both for spectra denoising and as part of PCA data mining. In the first part, we gave theoretical background and found the minimum experimental signal-to-noise ratio needed to have a correct denoised spectrum. We highlighted the overestimation of denoised Gaussian peaks. In this second part, we focussed on the computation time needed for SVD treatment. While our first attempts under Java CPU were extremely slow even with a recent processor, their counterparts with graphic cards were extraordinary fast. This

Figure 1 :

 1 Figure 1: Spectra used to benchmark SVD; top: noisy spectra; bottom: denoised spectra. a) 29 Si CPMG MAS solid-state NMR spectrum of a 50:50 MTEOS:TEOS sample; processing: Java GPU application with 2015 × 2014 = 4.1e6 points, k = 25 singular values, computation time of 6 s on a GTX 660, cosine apodisation. b) 87 Sr DFS-WURST-CPMG solid-state static NMR spectrum of non-hydrated non-protonated Sr(PO 3 F); processing: Java GPU application with 4097 × 4096 = 1.7e7 points, k = 25 singular values, computation time of 31 s on a GTX 660, cosine apodisation, magnitude calculation.

Figure 2 :

 2 Figure 2: a) SVD function call diagram on CPU and GPU. MKL: Intel Math Kernel Library; SSE: Streaming SIMD Extensions; AVX: Advanced Vector Extensions; FMA: Fused Multiply Add; DP: double precision; SP: single precision. b) Some Nvidia GPU used. From top to bottom: 8400 GS, GTX 260, GTX 660.

Figure 3 :

 3 Figure 3: Computation times for SVD under Java. a) Comparison of CPU and GPU times against matrix size; ◊: rectangular matrix; □: square matrix. b) CPU times against CPU frequency. c) GPU times against single precision float performance. d) CPU times against monothread performance; light blue and light green arrows evidence CPU cache and CPU release year influence, respectively. CPU and GPU times are drawn with plain lines and dashed lines, respectively.

Figure 4 :

 4 Figure 4: a) and b) Computation times for SVD under Matlab. Comparison of CPU and GPU times against matrix size for different Matlab versions. a) C2Q Q8200 + GTX 260 (2008). b) i5 4670K + GTX 660 (2013). c) SP performance against matrix size measured with GPUBench v1p7 under R2014a; the horizontal scale is larger than on a) and b). CPU and GPU times are drawn with plain and dashed lines, respectively. Red and green arrows indicate bad algorithm and CPU-GPU crossing, respectively.

  SVD computation time by a factor of nine. SP gives an additional factor of two in computation time on CPU, being faster than GPU for matrices smaller than 1025 × 1024 = 1.0e6 points (Matlab R2014a vs. R2010a). Despite the strong improvement for SVD on CPU, middle-range GPU remains relevant in SP mode for matrices above this size, up to the GPU memory limit (see previous paragraph). For legacy hardware dating from 2008, the best compromise is to use Matlab R2010a and CULA free R14 with SP. For hardware dating from 2013, the best choice is to use the most recent Matlab version with SP and gpuArray function. CPU computation should especially be avoided on Matlab R2010a as evidenced by the red arrows on Figures 4a

  version. Third, for all libraries tested, no performance increase was visible with NumPy when changing from DP (blue line) to SP (red line), which may indicate a bug of NumPy. On the contrary, SciPy + SP computation times (green line) were almost half DP ones (yellow line), as expected from DP / SP computing power ratio. Finally, for this small matrix of 1025 × 1024 = 1.0e6 points, and no matter if ATLAS, OpenBLAS or MKL library was installed, CULA + SP (hatched red line) was slower on GPU than MKL + SciPy + SP on CPU.

Figure 5 :

 5 Figure 5: Computation times for SVD under Python with influence of libraries and hardware optimisations for a matrix size of 1025 × 1024 = 1.0e6 points. a) C2Q Q8200 + GTX 260 (2008) under Linux. b) i5 4670K + GTX 660 (2013) under Windows.

Figure 6 :

 6 Figure 6: Comparison of Java, Matlab and Python SVD computation times for a matrix size of 1025 × 1024 = 1.0e6 points, in logarithmic scale. a) C2Q Q8200 + GTX 260 (2008) under Linux. b) i5 4670K + GTX 660 (2013) under Windows.

Table 1 .

 1 Properties of CPU used for SVD under Java. Gray rows indicate hardware used for SVD under Matlab and Python. a: ark.intel.com; b: CPU-Z 1.72; c: Performance Test 8.0.

	Central Processing Unit (CPU)	Type	Year a	Fabri-cation (nm) b	Number of cores b	Cache (MB) a	Core frequency (MHz) b	Memory frequency (MHz) b	Memory size (MB) b	CPU Mark c	Mono-thread (Mops/s) c	Matrices (millions/s) c
	Intel Pentium M 745	laptop	2004	90	1	2	1800	133	1024	444.8	577	1.41
	Intel Pentium 4 530	desktop 2004	90	2	1	3000	200	1024	335.2	726	0.29
	Intel Core 2 Duo E6400 desktop 2006	65	2	2	2130	333	2048	1451	849	3.72
	Intel Core 2 Quad Q8200 desktop 2008	45	4	4	2330	400	4096	2001	1004	5.8
	Intel Core 2 Duo T9600	laptop	2008	45	2	6	2800	400	4096	2190	1161	4.67
	Intel Core i3 4005U	laptop	2013	22	4	3	1700	800	4096	2551	1010	11.4
	Intel Core i5 4670K	desktop 2013	22	4	6	4200	1000	8192	8824	2519	31.6

Table 2

 2

	Graphics Processing Unit (GPU)	Type	Year a	Fabri-cation (nm) a	Number of cores a	Band-width (GB/s) a	Core frequency (MHz) a	Memory frequency (MHz) a	Memory size (MB) a	Single precision float (GFLOPS) b	Double precision float (GFLOPS) b	CUDA compute capability b
	Nvidia Quadro FX 570	desktop 2007	80	16	12.8	460	400	256	29	#N/A	1.1
	Nvidia GeForce 8400 GS desktop 2008	65	8	6.4	567	400	512	21	#N/A	1.1
	Nvidia Quadro NVS 160M laptop	2008	65	8	11.2	580	700	256	23	#N/A	1.1
	Nvidia Quadro FX 770M laptop	2008	65	32	25.6	500	800	512	79	#N/A	1.1
	Nvidia GeForce GTX 260 desktop 2008	65	216	111.9	576	1000	896	533	67	1.3
	Nvidia GeForce 820M	laptop	2012	28	96	14.4	625	900	2048	315	31	2.1
	Nvidia GeForce GTX 660 desktop 2012	28	960	144.2	1100	1500	2048	1707	88	3.0

. Properties of GPU used for SVD under Java. Gray rows indicate hardware used for SVD under Matlab and Python. a: GPU-Z 0.8; b: Cuda-Z 0.9; #N/A: not available.

  points, which is the maximum size for Java 32 bits CPU application. Similar conclusions were raised for our two reference computers (2008 and 2013). The measured computation times were grouped into three categories: unoptimised CPU (in plain blue), GPU (in hatched red) and optimised CPU (in plain red), from the slowest to the fastest. The first group consisted of Java CPU, Matlab R2010a and Python with default configuration. The second group contained Java GPU, Matlab R2010a with CULA or Matlab R2015a with gpuArray, depending on GPU generation, and of Python with CULA. The third group referred to a recent version of Matlab and to compiled

'version('-blas')' under Matlab gives MKL 11.0.5 for R2014a and MKL 11.1.1 for R2015a.

Acknowledgements

The French Région Ile de France -SESAME program is acknowledged for financial support (700 MHz spectrometer). Sylvie Masse and Cedric Lorthioir are thanked for fruitful discussions.

much stronger improvement was obtained for the above matrix size with GTX 660 GPU: from 69.1 to 41.4 s and from 52.7 to 12.9 s for R2014a + gpuArray + DP (dashed yellow line), R2015a + gpuArray + DP (dashed brown line), R2014a + gpuArray + SP (dashed green line) and R2015a + gpuArray + SP (dashed light blue line), respectively. The latter configuration outperformed R2010a + CULA + SP (27.5 s, dashed red line) owing to a more pronounced GPU utilisation during SVD. Despite SP was faster than DP, the DP / SP = 1 / 24 ratio was not respected. However, SVD algorithm was strongly optimised in R2015a + gpuArray against R2014a + gpuArray. While Core i5 4670K CPU remained more efficient for matrices up to 1025 × 1024 = 1.0e6 points, GTX 660 GPU outperformed it in SP mode for larger matrices. The obtained computation times under Matlab R2015a are thus very good, both on CPU and GPU and were better than under Java.

Matlab GPUBench

The cross in computation time between CPU and GPU was further investigated with GPUBench v1p7 [START_REF]MathWorks parallel computing toolbox team[END_REF]. This code compared CPU and GPU performance against matrix size for matrix-vector left division, which is a linear equations system solver. Such computation gives much less peak SP and DP float performance than reported in Table 2, and is rather compute-bond than memory-bound. This benchmark involves lots of matrix-vector operations as SVD does. For both 2008 and 2013 computers, a crossing was visible between CPU and GPU in SP mode (Figure 4c). This was explained by the time needed for data goings and comings between processor and graphic card and between graphic card core and its memory (Figure 2a). This is a hardware limitation.

Interestingly, the cross appeared in the same matrix size range (1e5 to 1e7) than the one observed for SVD (Figures 4a and4b). However, its position strongly depends on the unexpected difference led us to check if different Matlab versions could improve this situation. The divide and conquer algorithm was very helpful. Additional tests were undergone under Python to check the influence of software libraries and of SIMD hardware instructions call. Combining these optimisations, computation times on processor were even better than on graphic cards, being 100 times faster than our first tests under Java CPU, for a matrix of 1025 × 1024 = 1.0e6 points. Despite this approach is generalisable to any intensive computation, specific time gain will depend on the involved mathematical operations. The take-home message is thus to update software and to use optimised libraries and especially Intel MKL if available. This choice should be preferred against hardware updates. However, for matrix above 4097 × 4096 = 1.7e7 points and middle range hardware, GPU gave better results, up to GPU memory limit. We thus provided Python programs to apply SVD either on CPU or on GPU, and to denoise NMR FID. Further improvement could be obtained with mixed CPU/GPU optimised code, i.e. hybrid computing [START_REF] Liu | A divide-and-conquer approach for solving singular value decomposition on a heterogeneous system[END_REF]. However, such an approach is not suitable for non-computer-scientists people. Using clMAGMA library [START_REF] Du | From CUDA to OpenCL: Towards a performance-portable solution for multiplatform GPU programming[END_REF], combining divide and conquer on both CPU and GPU could be a good alternative [START_REF] Haidar | Unified development for mixed multi-GPU and multi-coprocessor environments using a lightweight runtime environment[END_REF]. In this case, it would be possible not only to use Nvidia GPU with CUDA but also AMD GPU with OpenCL.

This study has given strong background and optimisations for experiments involving SVD, either for denoising or for PCA. It may thus help scientists who want to use efficiently this technique, which is expected to be widely used in the forthcoming years.

Supplementary material

Programs source codes are available online in [START_REF] Laurent | Denoising applied to spectroscopies: parts I and IIprograms and datas[END_REF].


Carl Safi
email: carl.safi@ensiacet.fr

Alina Violeta Ursu

Céline Laroche

Bachar Zebib

Othmane Merah

Pierre-Yves Pontalier

Carlos Vaca-Garcia

Aqueous extraction of proteins from microalgae: Effect of different cell disruption methods

Keywords:

Manual grinding

Introduction

Microalgae were first exploited for their capacity to accumula te pro teins and, through time, interest in this biomass took a new course espe cially during the Iast two decades with increasing demand for sustainable energy. This biomass proved to be an important source of lipids suitable for biodiesel production. Hence, many of the studies were concentrated on lipid extraction for fuel purposes, neglecting the potential of microalgae to produce proteins and other high-value com ponents [1]. However, until now ail studies and estimates confirmed that costs of production of biodiesel from microalgae remain high [START_REF] Chisti | Biodiesel from microalgae[END_REF][START_REF] Rodolfi | Microalgae for oil: strain selection, induction oflipid synthesis and outdoor mass cultivation in a low-cost photobioreactor[END_REF] and far from being competitive with fossil fuel. Researchers are there fore turning towards valuing other components present in the microalgae such as proteins, pigments, dyes, sugars, etc.

Extracting the totality of a specific component from microalgae is often prevented by the intrinsic rigidity of its cell wall. To overcome this barrier, an initial operation unit of cell disruption is required to per mit complete access to the internai components and facilitate the ex traction process. Hence, many cell disruption techniques have been tested to break the cell wall of microalgae such as bead milling [START_REF] Doucha | Influence of processing parameters on disintegration of Chlore/la cells in various types of homogenizers[END_REF]5 ], ultrasonication [START_REF] Furuki | Rapid and selec tive extraction of phycocyanin from Spirulina platensis with ultrasonic cell disrup tion[END_REF][START_REF] Gerde | Evaluation ofmicroalgae cell disruption by ultrasonic treatrnent[END_REF][START_REF] Gouveia | Microalgae as a raw material for biofuels production[END_REF], microwave radiation [START_REF] Zheng | Disruption of Chlore/la vulgaris cells for the release of biodiesel-producing lipids: a comparison of grinding, ultrasonication, bead milling, enzymatic lysis, and microwaves[END_REF], enzymatic treatment [START_REF] Fleurence | The enzymatic degradation of algal cell walls: a useful approach for im proving protein accessibility?[END_REF][START_REF] Sari | Enzyme assisted protein extraction from rape seed, soybean, and microalgae meals[END_REF], cell homogenizer [START_REF] Mendes-Pinto | Evaluation of dif ferent cell disruption processes on encysted cells of Haematococcus pluvialis: effects on astaxanthin recovery and implications for bio-availability[END_REF] and high-pressure cell disruption [START_REF] Jubeau | Fleurence, High pressure dis ruption: a two-step treatrnent for selective extraction of intracellular components from the microalga Porphyridium cruentum[END_REF] to recover different components. The efficiency of cell disruption was usually evaluated by extracting a single component especially lipids be fore and after applying the treatment or by microscopie observation. To our knowledge, studies of microalgal proteins have been focused on: evaluating the nitrogen to protein conversion factor [START_REF] Gerde | Optimizing protein isola tion from defatted and non-defatted Nannochloropsis microalgae biomass[END_REF][START_REF] Lopez | Protein measurements of microalgal and cyanobacterial biomass[END_REF][16][17][START_REF] Safi | Influence of microalgae cell wall characteristics on protein extractability and determination of nitrogen-to-protein conversion factors[END_REF]; finding the best method to analyze proteins and differentiate between soluble and non-soluble proteins [START_REF] Barbarino ; 0 | An evaluation of methods for extraction and quantification ofprotein from marine macro-and microalgae[END_REF]; and analyzing the behavior ofproteins at the air/water interface [20].

Therefore, the present study focuses on evaluating the effect of dif ferent cell disruption techniques on protein extractability in water of five different microalgae having different cell wall macrostructures. Namely, the CyanobacteriumArthrospira platensis, which has a relative ly fragile cell wall, composed mainly ofmurein and no cellulose [21,[START_REF] Lu | Preventive effects of Spirulina platensis on skeletal muscle damage under exercise-induced oxidative stress[END_REF]. The Chlorophycean Chlorella vulgaris and the Eustigmatophyceae Nannochloropsis oculata, which have a cell wall mainly composed of cel lulose and hemicelluloses [START_REF] Payne | Evaluation of diets for culture of the calanoid copepod Gladioferens imparipes[END_REF]. Another Chlorophycean Haematococcus pluvialis has a thick trilaminar cell wall composed of cellulose and spo ropollenin [START_REF] Mendes-Pinto | Evaluation of dif ferent cell disruption processes on encysted cells of Haematococcus pluvialis: effects on astaxanthin recovery and implications for bio-availability[END_REF][START_REF] Aflalo | On the relative efficiency of two-vs. one-stage production of astaxanthin by the green alga Haematococcus pluvialis[END_REF][START_REF] Montsant | Presence of a nonhydrolyzable biopolymer in the cell wall of vegetative ce lis and astaxanthin-rich cysts of Haematococcus pluvialis (Chlorophyceae)[END_REF]. The composition of its cell wall, similar to that of spores, makes this microalga Jess permeable and extremely resistant to mechanical treatments [START_REF] Hagen | Ultrastructural and chemical changes in the cell wall of Haematococcus pluvialis (Volvocales, Chlorophyta) during aplanospore for mation[END_REF]. Finally, the Rodophythe Porphyridium cruentum, which Jacks a true cell wall, but instead is encapsulated by a layer of sulfurized polysaccharides [START_REF] Adda | Effect of nitrate on growth and production of cell-wall polysaccharide by the unicellular red alga Porphyridium[END_REF][START_REF] Arad | The potential of production of sulfated polysaccharides from Porphyridium[END_REF][START_REF] Arad | Effect ofnitrogen on polysaccharide production in a Porphyridium sp[END_REF][START_REF] Geresh | The extracellular polysaccharides of the red microalgae: chemis try and rheology[END_REF][START_REF] Geresh | Weinstein, Sulfation of extracellular polysaccharides of red microalgae: preparation, characterization and properties[END_REF][START_REF] Sobczuk | Effects of agitation on the microalgae Phaeodactylum tricomutum and Porphyridium cruentum[END_REF].

In addition, the microalgae selected in this study have a cytoplasm containing soluble proteins, and they ail have a chloroplast except for A. platensis, which instead has thylakoids bundles circling the peripheral part of the cytoplasm with their associated structures, the phycobilisomes ( containing the phycobiliproteins) present on the surface of the thyla koids like in the chloroplast of P. cruentum [21 ]. Furthermore, the chloro plast also contains soluble proteins and a central pyrenoid, which is a non membrane, bound organelle composed of RuBisCO.

In this study, proteins released in the aqueous media were evaluated and discussed considering the cell wall macrostructure of each microalga along with the effect of each cell disruption technique used.

Materials and methods

Microalgae

The microalgae selected are the Cyanobacteria Arthrospira platensis (strain PCC 8005), two different Chlorophyceaen Chlorella vulgaris (strain SAG 211-19), and Haematococcus pluvialis (unknown strain), one Rhodophyta Porphyridium cruentum (strain UTEX 161 ), and the Eustigmatophyceae Nannochloropsis oculata (unknown strain).

Each microalga was cultivated in a different culture medium. Hemerick medium was used for P. cruentum, Sueoka medium for C. vulgaris, Basal medium for H. pluvialis, Conway medium for N. oculata and Zarrouk medium for A. platensis. Ali strains were grown in batch mode in a 10 L indoor tubular air-lift photo-bioreactor (PBR at 25 °C [START_REF] Loubiere | Investigations in an external-loop air lift photobioreactor with annular light chambers and swirling flow[END_REF] inoculated from a prior culture in a fiat panel air-lift PBR (1 L). Cul ture mixing was achieved by sterile air injection from the bottom of the PBR. The pH and temperature were recorded by a pH/temperature probe (Mettler Toledo SG 3253 sensor), and monitored by the acquisi tion software LabVIEW. The pH was regulated at 7.5 with CO 2 bubbling. Microalgae were harvested during the exponential growth phase and concentrated by centrifugation, and then supplied as frozen paste from Alpha Biotech (Asserac, France). The biomass concentration of the paste was 20-24% dry weight.

Reagents

The Lowry kit ( a prepared mixture of Lowry reagent, BSA standards and 2 N Folin-Ciocalteu reagents) was from Thermo Scientific. NaOH granules and 37% HCl were purchased from Sigma Aldrich and used as received.

Microalgae pre-treatment 2.3.1. Freeze-drying

The frozen paste of crude microalga ( about 70 g) was directly intro duced to a Fisher Bioblock Scientific Alpha 2-4 LD Plus device (Illkirch, France). The pressure was reduced to 0.010 bar and the temperature was further decreased to -80 °C and freeze-drying was conducted under vacuum for 48 h. Dry biomass was stored under anhydrous con ditions. Before any disruption treatment, the cells were vigorously rehydrated in distilled water to ensure good homogeneity of the sample.

Microalgae treatments 2.4.1. Contrai

Cells (0.5 g) were dispersed for 2 h in 25 mL distilled water and the supernatant was recovered by centrifugation at 10,000 g for 10 min at 20 °C for protein analysis. This treatment was considered as a blank to compare with the other extraction treatments.

High-pressure cell disruptor

A TS Haiva series, 2.2-kW, disrupter from Constant Systems Limited (Northants, UK), was a pp lied, in two passes at a pressure of 2700 bar, to a biomass sample at a concentration of 2% dry weight (0.5 g of dry cells dispersed in 25 mL distilled water).

Ultrasonication

This treatment was carried out using a VC-750HV (20 kHz, 13 mm probe) ultrasonic processor on 0.5 g of dry cells dispersed in 25 mL dis tilled water. Total treatment time was 30 min in cycles of 5 s of ultrasonication and 15 s of resting time in order to prevent overheating the sample.

Manual grinding

Dry microalgae were manually ground using a mortar for 5 min, and then 0.5 g was dispersed in 25 mL distilled water for 2 h. Samples were taken for protein analysis.

Chemical treatment

Mother solutions were prepared with approximately 500 mL of dis tilled water and 2 N Na OH was added to adjust the solution to pH 12 for maximum protein solubility. A sample of 0.5 g of freeze-dried biomass was added to 25 mL of mother solution. The mixture was then stirred for 2 h at 40 °C. The separation of the supernatant from the pellet was conducted by centrifugation at 10,000 g for 10 min at 20 °C. The super natant was then adjusted to pH 3 with 0.1 M HCI in order to precipitate the proteins. The protein isolate was collected after centrifugation at 10,000 g for 10 min at 20 °C and the pellet was neutralized with 0.01 M NaOH [20]. Samples were taken for protein analysis.

Lowry method

After every disruption treatment, the liquid/solid separation was conducted by centrifugation at 10,000 g for 10 min at room tempera ture and the supematant was analyzed by the Lowry method. [START_REF] Lowry | Protein measurement with the Folin phenol reagent[END_REF] A calibration curve was prepared using bovine standard albumin at a concentration range of O to 1500 µg mL -1 . In order to measure the pro tein content, 0.2 mL of each standard or samples containing the crude protein extract was withdrawn and then 1 mL of modified Lowry re agent was added to each sample. Each sample was then vortexed and incubated for 10 min. After incubation, 100 µL of Folin-Ciocalteu Re agent (1 N) was added and again vortexed and incubated for 30 min. The blue color solution was then measured at 750 nm with a UV-1800 Shimadzu spectrophotometer.

Elemental analysis

Total nitrogen was evaluated by using a Perkin Elmer 2400 series II elemental analyzer. Microalgal samples (2 mg) were placed in tin cap sules and heated at 925 °C, using pure oxygen as the combustion gas and pure helium as the carrier gas, and the nitrogen concentration was evaluated. For ail the previous analyses, three experiments were conducted separately with ail the microalgae.

Confocal laser scanning microscopy

Cells were observed with an SP2-AOBS confocal laser-scanning mi croscope from Leica microsystems (Nanterre-France). The fluoro chrome calcofluor white that binds to the cell wall was added to the samples. When excited at 488 nm, the cells are identified as light blue colored.

Statistical analysis

Three experiments were conducted separately on ail microalgae and their protein extract. Statistical analyses were carried out on Microsoft Excel 2011 and Statgraphics Sigma Express. ANOVA test was carried out and measurements of three replicates for each sample were repro ducible for ± 5% of the respective mean values. The total protein in crude microalgae was determined by obtaining total nitrogen through elemental analysis and converting it into protein percentage using the conversion factor found for each crude microalga in the study conducted by Safi et al. [START_REF] Safi | Influence of microalgae cell wall characteristics on protein extractability and determination of nitrogen-to-protein conversion factors[END_REF]. In all cases, the total protein content was high, ranging from 46 to 57% dw (Table 1 ).

The fraction of soluble proteins released in water after each cell dis ruption technique is presented in Fig. 1. The fraction of soluble protein in the total protein present in the microalgae was also evaluated and all these results are given in Table 2.

In this work, four cell disruption techniques are compared, along with a control in distilled water, in order to evaluate protein exiting by diffusion through the membranes and walls into water media. The recovery yield in the control ranges from 6.5% dw with H. pluvialis to 25% dw with P. cruentum. The latter alga is considered as fragile and the former as resistant to disruption. Among the tested techniques, high-pressure cell disruption was the best technique for all the microalgae, with a recovery yield of 41% to 90% dw. Moreover, the lowest protein concentration for ail microalgae was obtained in the water con trol and through manual grinding, especially for rigid cell walled microalgae. A relative difference was noticed in the concentration of protein released between the microalgae with fragile and rigid cell walls. P. cruentum released the most compared to A. platensis. After ultrasonication a minor increase in protein concentration was notice able for the green microalgae, especially for C. vulgaris, and a more im portant increase was observed for the A. platensis and P. cruentum. Furthermore, the chemical treatment showed a significant increase of protein released (N. oculata and C. vulgaris statistically released the same protein concentration) (Fig. 1 ).

In order to better interpret these results, microscopie observation was carried out. The laser scanning confocal microscopie images pre sented in Fig. 2 showed that in the cases of P. cruentum and A. platensis a total disruption of the cell wall occurred after high pressure cell disruption. On the other hand, for C. vulgaris, N. oculata and H. pluvialis, only a majority of cells were completely disrupted, while a few cells maintained their globular form.

Diswssion

The goal of the present study was to highlight the release of protein into aqueous media after the application of different cell disruption techniques. The results do not only rely on the mechanical rigidity of the cell wall of each microalga but also on its chemical characteristics. Indeed, having a deep understanding of the macrostructure is necessary in order to evaluate the release of components after any treatment was conducted on the cells. This approach has been considered in a study conducted by Jubeau et al. in order to selectively extract intracellular components such as proteins and phycoerythrin after cell disruption of P. cruentum [START_REF] Jubeau | Fleurence, High pressure dis ruption: a two-step treatrnent for selective extraction of intracellular components from the microalga Porphyridium cruentum[END_REF]. Moreover, the freeze-drying process that conserves the samples well makes the protein extraction harder for some species [START_REF] Safi | Influence of microalgae cell wall characteristics on protein extractability and determination of nitrogen-to-protein conversion factors[END_REF]. In addition, after freeze-drying the cells are more aggregated, which lowers the contact surface with the extracting solvent, and also could affect the integrity of the cell wall in fragile species [START_REF] Cordero | Viability of mass algal cultures preserved by freezing and freeze-drying[END_REF][START_REF] Rossignol | Membrane technology for the continuous separation microalgae/culture medium: compared perfor mances of cross-flow microfiltration and ultrafiltration[END_REF].

Osmosis is the net movement of solvent (water) molecules through a partially permeable membrane into a region of higher solute concen tration. Water usually travels through the membrane, the vacuole, the chloroplast, and the mitochondria by diffusing across the phospholipid bilayer via water channels (aquaporins), which are proteins embedded in the cell membrane that regulate the flow of water. Hence, the water only treatment was not considered as a cell disruption technique, but it was carried out in this study as a reference control for the other tech niques. Surprisingly, the dispersion of microalgae in water released up to 19-25% of soluble proteins per dry weight (Table 2) fromA. platensis and P. cruentum, coloring the water light blue for the former and light red for the latter. This indicates that water penetrated the cell walls of both microalgae but also succeeded in penetrating the intra-thylakoids space of A. platensis and permeated the chloroplast of P. cruentum to slightly dis solve the phycobilisomes present on the thylakoid membranes. On the contrary, the osmosis phenomenon was not strongly effective for the green microalgae, which are known to have rigid cell walls that resist water permeating the structure and, thus, releasing only 6-10% proteins (Table 2).

Taking into account the standard deviation of three samples consid ered for the green microalgae ( C. vulgaris, N. oculata and H. pluvialis ), ail the values of released proteins after water treatment and manual grind ing shown in Fig. 1 are statistically equivalent, indicating again the resis tance of their cell walls after manual grinding. This was not the case for the A. platensis and P. cruentum with stronger coloration of water 53.4 ± 0.2 78.0 ± 2.8 P. cruentum 24.8 ± 0.3 49.5 ± 0.7 67.0 ± 0.9 73.5 ± 1.2 90.0 ± 2.4

Based on three replicates for three experiments ± SD (n = 9).

Protein yield was calculated according to the following equation: proportion of hydrosoluble protein in total protein = (llow.,, / N ea x NIP) x 100 (%).

N ea : total nitrogen in initial biomass (% dw) obtained by elemental analysis. NTP: nitrogen-to-protein conversion factor from Safi et al. [START_REF] Safi | Influence of microalgae cell wall characteristics on protein extractability and determination of nitrogen-to-protein conversion factors[END_REF]. Pww ry : water-soluble protein.

associated with increase in protein concentration, indicating that the in ternai structure ofboth microalgae is being further altered, and simulta neously facilitating the penetration of water to dissolve more proteins. Ultrasonication produces cavitation in cells and facilitates cell dis ruption; it did not cause any change for H. pluvialis, but showed a minor effect on the cell wall of N. oculata and C. vulgaris by possibly making it difficult for water to extract cytoplasmic proteins without al tering the structure of their chloroplast. The concentration of soluble proteins and coloration kept increasing for the fragile cell-walled microalgae by releasing 47-68% dw.

Chemical treatment was a key process that showed increases in pro tein concentration compared to the other treatrnents. P. cruentum lacks a well-defined cell wall. Since protein solubility is dependent on pH, cell wall characteristics and chemical composition [START_REF] Sari | Enzyme assisted protein extraction from rape seed, soybean, and microalgae meals[END_REF], the high pH easily solubilized proteins without any resistance from its pseudo-cell wall. But in the case of the green microalgae, the sodium hydroxide is able to perforrn a process similar to mercerization, by penetrating the micro crystalline structure of the cellulosic cell walls of the green microalgae [START_REF] Safi | Influence of microalgae cell wall characteristics on protein extractability and determination of nitrogen-to-protein conversion factors[END_REF]. The alkaline solution can also easily dissolve the hemicelluloses at tached to cellulose as it has been demonstrated during the refining of lignocellulosic substrates (straw, bran, and wood). In addition, it indi cates that this treatrnent gave more access to the cytoplasmic proteins, and recovered the same concentration of proteins from N. oculata and C. vulgaris (Fig. 1 ). However, the sporopollenin contained in the most rigid cell wall (H. pluvialis) is known to be extremely resistant to chem ical agents [START_REF] Hagen | Ultrastructural and chemical changes in the cell wall of Haematococcus pluvialis (Volvocales, Chlorophyta) during aplanospore for mation[END_REF], which explains the low recovery of soluble proteins. A. platensis has a cell wall rich in amino sugars cross-linked with oligopeptide chains. The former are labile in alkaline conditions by deamidation of the N-acetylglucosamine and the latter are soluble in al kaline conditions. Therefore, the cell wall becomes perrneable allowing the alkaline extraction ofproteins [START_REF] Safi | Influence of microalgae cell wall characteristics on protein extractability and determination of nitrogen-to-protein conversion factors[END_REF]. Hence, ail these results demon strate that the chemical action acts in synergy with the mechanical characteristics of the cell wall.

High-pressure cell disruption was the most efficient technique for ail microalgae; the concentration of proteins was statistically the same for the green microalgae, with evidence that the majority of the cells were broken white some ofthem remained intact (Fig. 2). The chloroplast of these species was also partially damaged, as it is revealed by the color ation in light green ( chlorophyll) of the aqueous extra et. Indeed, chloro phyll is a hydrophobie pigment; its presence in the aqueous phase indicates the formation of micellar structures and it points to a possible alteration of the chloroplast. The other indication is that some cell debris containing the green pigment were extremely reduced in size and did not precipitate in the pellet after centrifugation at 10000 g, leading to a greenish color of the supernatant as it occurred in previous work [START_REF] Gerde | Evaluation ofmicroalgae cell disruption by ultrasonic treatrnent[END_REF]. Hence, after two passes, water had access to cytoplasmic proteins and partially infiltrated the chloroplast. Therefore, this method released al most half of the proteins present inside the rigid cell-walled microalgae (Table 2), indicating again the resistance of their cell wall. On the other hand, as expected according to their fragile cell wall (Table 2), A. platensis and P. cruentum did not show much resistance, and the sol uble protein concentration of total protein was 78% dw for the former and 90% dw for the latter (Table 2). Moreover, an important coloration of the aqueous extract for both microalgae was observed with a pellet having lost its red coloration for P. cruentum. This was also supported by microscopie observation, showing that their structure was completely altered (Fig. 2).

The same order of rigidity was obtained in another study [START_REF] Safi | Influence of microalgae cell wall characteristics on protein extractability and determination of nitrogen-to-protein conversion factors[END_REF] that took into account the values of the nitrogen to protein conversion fac tors before and after protein extraction and then attributed them to the rigidity of the cell walls. This result shows that in order to compare the efficiency of cell disruption technology, it is more accurate to use fragile cell algae such as P. cruentum.

The present study shows additional insight into the understanding of the recovery of proteins after different cell disruptions. Hence, among ail the techniques used the high-pressure cell disruption was the most efficient but not sufficient to recover more than 50% of the pro teins for the green microalgae, indicating that more passes are required to completely disrupt their macrostructure, and thus more energy input will be necessary.

6 A

 6 Fig. 1. Soluble protein concentration released in water after each cell disruption technique (±5D for three replicates in three experiments (n = 9)).

Fig. 2 .

 2 Fig. 2. Confocal laser scanning microscopy offive microalgae before (right) and after (left) high-pressure cell disintegration. (A and B) C. vulgaris, (C and D) A. platensis, (E and F) P. cruentum, ( G and H) H. pluvialis, (I and]) N. oculata.

Table 1

 1 Protein and nitrogen content for each microalga based on three replicates for three experiments ± SD (n = 9).

	Microalga	Total nitrogen (%.dw-1 )	Total proteins (% dw-1 )
	H.pluvialis	8.27 ± 0.08	51.7 ± 0.43
	N. oculata	7.41 ± 0.40	46.5 ± 2.14
	C. vulgaris	7.81 ± 0.18	49.6 ± 1.04
	A. platensis	8.53 ± 0.20	53.5 ± 1.10
	P.cruentum	9.04 ± 0.70	57.3 ± 3.84
	3.Results