Sepehr Andreina Laera
email: andreina.laera@u-pem.fr

Sepehr Shakeri Yekta

Mattias Hedenström

Rémy Buzier

Gilles Guibaud

Mårten Dario

Giovanni Esposito

Eric D Van Hullebusch

Dario Mårten

A simultaneous assessment of organic matter and trace elements bio-accessibility in substrate and digestate from an anaerobic digestion plant

Keywords:

de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Availability of trace elements (TE) as micronutrients is essential for stable and efficient performance of anaerobic digestion processes [START_REF] Feng | Impact of trace element addition on biogas production from food industrial waste--linking process to microbial communities[END_REF][START_REF] Karlsson | Impact of trace element addition on degradation efficiency of volatile fatty acids, oleic acid and phenyl acetate and on microbial populations in a biogas digester[END_REF][START_REF] Lindorfer | Nutrient and trace element supply in anaerobic digestion plants and effect of trace element application[END_REF]. Due to a complex network of reactions, controlling the chemical speciation of TE, a small fraction of TE in the digesters environment is often available for microbial activities, while the majority of TE occur as precipitates, adsorbed or complexed species [START_REF] Aquino | Bioavailability and Toxicity of Metal Nutrients during Anaerobic Digestion[END_REF][START_REF] Maharaj | ADM1 based mathematical model of trace element complexation in anaerobic digestion processes[END_REF][START_REF] Maharaj | ADM1 based mathematical model of trace element precipitation/dissolution in anaerobic digestion processes[END_REF]. Several studies attempted to determine the fraction of TE available for uptake by microorganisms with sequential extraction procedures, such as the modified Tessier method [START_REF] Van Hullebusch | Comparison of three sequential extraction procedures to describe metal fractionation in anaerobic granular sludges[END_REF] and the Community Bureau of Reference (BCR) method [START_REF] Rauret | Improvement of BCR three step sequential extraction procedure to the certification of new sediment and soil reference materials[END_REF] to guarantee an adequate supply of TE in digesters [START_REF] Braga | Metal fractionation in sludge from sewage UASB treatment[END_REF][START_REF] Cao | The chemical and dynamic distribution characteristics of iron, cobalt and nickel in three different anaerobic digestates: effect of pH and trace elements dosage[END_REF][START_REF] Ortner | Can bioavailability of trace nutrients be measured in anaerobic digestion?[END_REF]. The procedure involves sequential treatment of the samples by chemical reagents such as neutral salts, weak and strong acids or reducing and oxidizing agents to selectively extract TE. Despite the limitations associated to the sequential extraction procedures including the poor selectivity of chemical reagents [START_REF] Bacon | Is there a future for sequential chemical extraction?[END_REF][START_REF] Filgueiras | Chemical sequential extraction for metal partitioning in environmental solid samples[END_REF], the lack of uniformity in the procedure and the complexity of chemical composition of anaerobic samples [START_REF] Thanh | Trace metal speciation and bioavailability in anaerobic digestion: A review[END_REF], there is a growing interest in using sequential extraction procedures to fractionate TE [START_REF] Ortner | Can bioavailability of trace nutrients be measured in anaerobic digestion?[END_REF][START_REF] Zhu | Sequential extraction of anaerobic digestate sludge for the determination of partitioning of heavy metals[END_REF]. Indeed, results obtained from sequential extraction methods are valuable for anaerobic digestion processes and for environmental risk assessments of digestate utilization as a soil fertilizer [START_REF] Ortner | Can bioavailability of trace nutrients be measured in anaerobic digestion?[END_REF][START_REF] Zhu | Sequential extraction of anaerobic digestate sludge for the determination of partitioning of heavy metals[END_REF], as it allows to quantitatively determine the TE fractions with different degree of solubility and reactivity. Moreover, sequential extraction methods are useful tools to assess mobility, accessibility, and potential bio-availability of TE [START_REF] Harmsen | Measuring Bioavailability: From a Scientific Approach to Standard Methods[END_REF]. [START_REF] Braga | Metal fractionation in sludge from sewage UASB treatment[END_REF] observed that micronutrient TE, including Se, Zn, Ni and Fe were mainly found in the organic matter (OM)/sulfide fraction of the modified Tessier method applied on sewage sludge samples. Similar results were also found by [START_REF] Zufiaurre | Speciation of metals in sewage sludge for agricultural uses[END_REF] in sewage sludge samples. Such fraction is considered less mobile than other TE fractions extracted by the sequential extraction procedure and less bio-accessible for microbial uptake [START_REF] Filgueiras | Chemical sequential extraction for metal partitioning in environmental solid samples[END_REF].

Different chemical interactions between TE and organic/inorganic compounds originating from the substrate and generated during the anaerobic digestion process will determine the chemical speciation of TE and consequently their bio-accessibility and bioavailability for anaerobic microorganisms in digesters and for plants and soil microorganisms when digestate is spread on lands [START_REF] Fermoso | Fate of Trace Metals in Anaerobic Digestion[END_REF][START_REF] Thanh | Trace metal speciation and bioavailability in anaerobic digestion: A review[END_REF]. Inorganic compounds such as sulfide (S 2-), phosphate (PO4 3-) and carbonate (CO3 2-) may precipitate TE in anaerobic digestion systems as simulated by [START_REF] Maharaj | ADM1 based mathematical model of trace element precipitation/dissolution in anaerobic digestion processes[END_REF] with a dynamic mathematical model based on anaerobic digestion model no.1 (ADM1). Such inorganic compounds would compromise the availability of TE for microbial uptake in anaerobic digesters [START_REF] Callander | Precipitation, chelation, and the availability of metals as nutrients in anaerobic digestion. I. Methodology[END_REF]). In addition, TE could be complexed with organic chelators becoming either more or less available for microbial uptake depending on the binding strength of metal-organic complexes. Organic compounds contain functional groups such as carboxyl, hydroxyl or amino groups with high affinity to complex with TE [START_REF] Callander | Precipitation, chelation, and the availability of metals as nutrients in anaerobic digestion. I. Methodology[END_REF]. Gonzalez-Gil et al. (2003) observed that amino acids in yeast extracts form soluble complexes with Ni and Co which prevent their precipitation with sulfide and consequently increased their availability for microorganisms. Moreover, the degradation process of bio-accessible OM deriving from substrate or digestate, can release TE in solution that becomes potential bio-accessible for up-take by anaerobic digester microorganisms or soil-dwelling organisms [START_REF] Knoop | Fate of nutrients and heavy metals during two-stage digestion and aerobic post-treatment of municipal organic waste[END_REF]. Accordingly, association of TE with OM play an important role in bioavailability of TE. [START_REF] Jimenez | Methane production and fertilizing value of organic waste: Organic matter characterization for a better prediction of valorization pathways[END_REF][START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF] assessed the bio-degradability and the bio-accessibility of OM in organic wastes using a physical-chemical sequential extraction procedure for a large number of samples, including municipal sludge samples, municipal solid wastes, digestate and compost. The sequential extraction procedure includes the following fractions: i) Extractable Soluble from Particulate Organic Matter (SPOM), which contains water-soluble proteins and sugars; ii) Readily Extractable Organic Matter (REOM), representing easily accessible proteins and lipids;

iii) Slowly Extractable Organic Matter (SEOM), containing humic-like and fulvic acid-like structures as well as complex proteins and certain lignocellulosic compounds and iv) Poorly Extractable Organic Matter (PEOM), targeting hemicellulose and cellulose [START_REF] Jimenez | Methane production and fertilizing value of organic waste: Organic matter characterization for a better prediction of valorization pathways[END_REF].

The development of OM fractionation methods allows assessing the accessibility of OM as carbon and energy sources for microorganisms in anaerobic digesters. However, to the best of our knowledge, no research work assessed simultaneous OM and TE extraction for assessment of the bio-accessibility of a combined source of carbon, energy and micronutrient TE. Thus, we aim to evaluate the application of a sequential extraction method adapted for OM fractionation by [START_REF] Jimenez | Methane production and fertilizing value of organic waste: Organic matter characterization for a better prediction of valorization pathways[END_REF][START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF] to determine simultaneously the accessibility of TE in substrate and digestate samples. For this purpose, substrate and digestate samples from a fullscale anaerobic digester, regularly supplied with TE supplements, were used to determine the TE concentrations in different OM fractions. The TE investigated in this study are Co, Fe, Ni, Mn, Mo, and Zn, which are important micronutrients for microbial activities [START_REF] Gustavsson | Bioavailability of cobalt and nickel during anaerobic digestion of sulfur-rich stillage for biogas formation[END_REF][START_REF] Gustavsson | The feasibility of trace element supplementation for stable operation of wheat stillage-fed biogas tank reactors[END_REF] and Al, As, Cd, Cr, Cu and Pb, which could be harmful to soil microorganisms and plant growth once digestate is applied as soil amendment [START_REF] Bajgiran | Influence of soil amendments and soil properties on macro-and micronutrient availability to microorganisms and plants[END_REF][START_REF] Kupper | Heavy metals in source-separated compost and digestates[END_REF][START_REF] Nkoa | Agricultural benefits and environmental risks of soil fertilization with anaerobic digestates: A review[END_REF]. Moreover, changes in structural characteristics of OM after each step of the extraction procedure were studied by nuclear magnetic resonance (NMR) spectroscopy in order to assess different organic groups in the samples, which were removed by extracting reagents during the 71 sequential extraction procedure. 72

Material and methods

Samples

Substrate and digestate were collected from a full-scale anaerobic co-digestion plant located in Linköping, Sweden. The anaerobic digestion plant has a capacity of 125 000 tons of waste per year and treats the organic fraction in household waste (50%), slaughterhouse waste (25%) and industrial waste (25%) at 42°C with an organic loading rate of 4,5 kgVS/L•d and a hydraulic retention time of 37 days. The substrate was collected from a tank after 1-hour pasteurization at 70°C and TE addition, whereas the digestate was collected from the main anaerobic digester sampling port. About 1 liter of each sample was collected in acid washed polypropylene (PP) bottles. The bottles were flushed with nitrogen (N2) prior to sampling and closed with a lid after collection to reduce sample exposure to air during sampling and transportation from the plant to the laboratory. Once in the laboratory, the samples were immediately treated in accordance to the sequential extraction procedure.

Sequential extractions procedure

The sequential extractions of dissolved organic matter (DOM), REOM and SEOM were carried out as described by [START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF], while SPOM and PEOM fractions were extracted according to [START_REF] Jimenez | Methane production and fertilizing value of organic waste: Organic matter characterization for a better prediction of valorization pathways[END_REF]. The latter modified protocol includes calcium chloride (CaCl2) reagent for SPOM extraction and sulfuric acid (H2SO4) for PEOM extraction compared to the procedure proposed by [START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF]. Moreover, we slightly modified the protocols to adapt the method for simultaneous extraction of OM and TE (Table 1). The main modifications involve the use of raw sample, rather than freeze dried sample, N2 flushing during operations to reduce sample oxidation and changes in TE speciation (e.g. formation of metal oxides [START_REF] Ortner | Can bioavailability of trace nutrients be measured in anaerobic digestion?[END_REF]) which would determine a change in the bio-accessibility pattern of trace elements. Moreover, the mass of the sample and the volume of reagents were decreased compared to the original procedures to adapt the method to facilities available in the laboratory such as the high-speed centrifuge (Beckman J2-21M, USA), which was used to separate the supernatants from the solids. In short, the first step of the procedure separates DOM from the solid residue. Approximately 300-600 ml of sample, with a total solids content of 4.8±0.2 wt%, and 14.6±0.1 wt% for digestate and substrate, respectively, was centrifuged at 18600×g for 30 min at 10°C. Then, the supernatant, containing DOM, was filtered through 0.45 µm polyethersulfone (PES) syringe filters (Pall Laboratory). The solid residue was flushed with N2, sealed and stored at 4°C in PP centrifuge tubes (Sarstedt) before performing the next extraction step. In the second step, SPOM was extracted according to the procedure by [START_REF] Jimenez | Methane production and fertilizing value of organic waste: Organic matter characterization for a better prediction of valorization pathways[END_REF]. Approximately 3 g of pellet were shaken in polypropylene copolymer tubes (Termo Scientific Nalgene) with 24 ml (mass ratio 1:8) of 10 mM CaCl2 (pH 8) at 200 rpm and at 30°C for 15 min. The suspension was then centrifuged at 18600×g for 30 min at 4°C and the supernatant containing SPOM was recovered and filtered through 0.45 µm PES syringe filters.

The residual solid was treated with the same reagent three more times. During extraction of SPOM, N2 was flushed in the tubes. Subsequently, the solid residue was rinsed four times with 24 ml of 10 mM NaCl and 10 mM NaOH (pH 11) [START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF]. The suspension was shaken, centrifuged and filtered to recover REOM fraction. Thereafter, the residual pellet was used to extract carbonate, sulfides and hydroxides (CSH) fraction by adding 24 ml of 0.1 M HCl for 1 h at 30°C and 200 rpm [START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF]. Unlike the original procedure [START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF], this fraction was recovered for further analyses. The resulting solid residue was washed with ultrapure water and neutralized to pH 7 with 0.1 M NaOH. Subsequently, the solid residue was suspended in 24 ml of 0.1 M NaOH (pH 12) and shaken at 200 rpm and at 30°C for 1 h to recover the SEOM fraction [START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF]. This step was repeated three more times.

Finally, the residual pellet was shaken two times with 24 ml of 72% (w:w) H2SO4 for 3 h at 30°C and 200 rpm for extraction of PEOM [START_REF] Jimenez | Methane production and fertilizing value of organic waste: Organic matter characterization for a better prediction of valorization pathways[END_REF]. The residual solid, which is the Non-Extractable Organic Matter (NEOM), was recovered and freeze-dried for further analyses. The sequential extraction was performed on triplicate samples. All reagents were prepared in acid washed glassware and with ultrapure deaerated water.

Table 1 shows the sequential extraction steps of the procedure.

(Table 1 here) 2.3. Analytical procedures

Chemical analysis

The pH of the samples was measured with a pH meter (InoLab 7310, WTW, Weilheim, Germany). Total solids (TS) and volatile solids (VS) content in the raw samples and pellet collected after DOM extraction were quantified in triplicates according to the Swedish Standard method (SS-028113; 25). Thus, an aliquot of sample was dried in porcelain crucibles at 105 °C for 20 h to measure TS content. Then, the dried samples were heated up to 550°C for 2 h in a muffle furnace to determine VS content. Dissolved organic carbon (C) was measured in the filtered supernatants by total organic carbon analyzer (TOC-VCHS, Shimadzu, Japan). During analysis, ultrapure water was analyzed after each set of triplicates to avoid cross contamination.

Total C and N content in the solid residue collected at each extraction step was determined by CHNS/O elemental analyzer (EA2400, Perkin Elmer, USA). Prior to analysis, the samples were freeze-dried and finely ground with a mortar and pestle.

The concentration of TE for each extracted faction was quantified in the filtered supernatants, whereas the total TE' content (i.e. Al, As, Cd, Co, Cr, Cu, Fe, Mn, Mo, Ni, Pb and Zn) was extracted after digestion of substrate and digestate samples according to the Swedish standard method (SS028311) using 7 M HNO3 in an autoclave at 120 °C for 30 min as described by Shakeri Yekta et al. (2014a). Trace element concentrations in the samples collected both after the acid digestion method and the sequential extraction procedure were quantified by inductively coupled plasma mass spectrometry (ICP-MS, Nexion 300D, Perkin Elmer, USA). Before ICP-MS analysis, if required, samples were acidified with concentrated HNO3 (1% of the sample volume) and stored at 4°C until analysis. To reduce possible chemical interference, some of the TE were analyzed by using kinetic energy discrimination (KED) or dynamic reaction cell (DRC) mode during ICP-MS analysis. A graphical scheme of the analyses performed on the collected samples is showed in Figure 1.

(Figure 1 here) 2.3.2. Nuclear magnetic resonance spectroscopy NMR analysis was performed on the solid residues recovered at each step of the extraction procedure. Solid residues were preferred to the liquid fractions to reduce possible interferences, generated by the chemical reagents, with the sample NMR signals.

About 0.4 g dry mass of sample was pre-treated with 2 M HCl for 1 h to remove the paramagnetic TE that would be detrimental to the quality of the NMR spectra according to Shakeri Yekta et al. (2018). The suspension was centrifuged and the supernatant was discarded, while the solid residue was recovered and freeze-dried. Approximately 80 mg of each sample was transferred to 4 mm ZrO2 rotors for solid state cross polarization magic angle spinning (CPMAS) 13 C NMR analysis and approximately 100 mg of sample was milled using a Fritsch Pulverisette 7 planetary ball-mill to prepare it for solution-state 1D 1 H and 2D 1 H- 13 C heteronuclear single quantum coherence (HSQC) NMR analysis. The protocol used for grinding consisted of 5 × 10 min milling with 5 min pauses in between to prevent overheating of the samples. 20 mg of milled sample were transferred to 5 mm NMR tubes and 600 µl of deuterated dimethyl sulfoxide (DMSO-d6) was added. The CPMAS 13 C analysis was performed in triplicate and these were later pooled to get sufficient material for liquid state NMR analysis. Solid state CPMAS 13 C NMR spectra and liquid state 1D 1 H and 2D HSQC 1 H- 13 C NMR spectra were acquired using a Bruker 500 MHz AVANCE III spectrometer equipped with a 4 mm MAS probe and a Bruker 600 MHz AVANCE III HD spectrometer equipped with a 5 mm cryoprobe, respectively. CPMAS 13 C NMR spectra were recorded using cp pulse sequence and 3500 scans.

The relaxation delay was 1 s and spin-rate was 10 kHz. 1D 1 H and 2D HSQC 1 H- 13 C NMR spectra were recorded using zg30 and hsqcetgpsisp2.2 pulse sequences and 8 and 16 scans, respectively. The relaxation delay was 1.5 s and 2 s for 1D 1 H and 2D HSQC 1 H- 13 C NMR, respectively. The spectra processing was performed in Topspin 3.5 (Bruker Biospin, Germany) and spectra were calibrated using adamantane as an external reference for CPMAS spectra or the residual DMSO peak (δH/C: 2.49/39.5 ppm) in the case of 2D 1 H-13 C HSQC spectra.

Analytical quality control

The methodological limit of detection (MLD) and quantification (MLQ) were calculated using the conservative formula as the average plus three or ten times the standard deviation of the blanks, respectively, to consider contamination from elements in the reagents. During the sequential extraction procedure, we used 36 and 96 procedural blanks for TE and dissolved organic C analysis, respectively. Quality controls at 20 and 50 mg/L dissolved organic C were analyzed for every 10 samples during dissolved organic C analysis. The recovery was equal or above 94% among all analyses. A reference material Sewage sludge CRM029-050 was acid digested in triplicate to evaluate the performance of the adopted total acid digestion method for TE' analysis. The accuracy of the method, calculated by dividing the mean observed concentration to the certified value (van Reeuwijk and Houba, 1998), was equal or higher than 90% for all analyzed elements.

In addition, spiked samples were analyzed to check possible matrix interferences on the measured concentrations of analytes due to the diversity of reagents used to perform the sequential extraction procedure. The method adopted to evaluate the matrix effect is standard addition method. We observed no significant matrix effect for dissolved organic C analysis (p≥0.04) in all OM fractions, whereas we observed matrix effect for TE analysis in PEOM fraction extraction and therefore results for this fraction are not presented.

Results and discussion

Organic matter composition of substrate and digestate

The TS content of substrate and digestate samples were 14.6±0.1 and 4.8±0.2 wt%, respectively, with the VS contents of 91.4±0.4 and 76.2±0.6 % of TS. The TS content after centrifugation and separation of the liquid fraction (i.e. solid pellets used for the sequential extraction procedure) were 33.1±0.4 wt% for substrate and 19.6±0.5 wt% for digestate samples. The pH of substrate and digestate was 4.9 and 8.1, respectively. We observed different distribution of C among the operationally defined organic fractions (i.e. DOM, SPOM, REOM, SEOM and PEOM) for substrate and digestate samples (Figures 2a and2b). A large proportion of organic C in the substrate was present as DOM (76% of extracted organic C), whereas PEOM had the highest proportion among the OM fractions of the digestate (47% of extracted organic C). The DOM fraction mainly contains water-soluble organic substances, whereas PEOM contains recalcitrant

Structural characteristics of sequentially extracted organic matter fractions

NMR spectroscopy analyses of the solid pellets after each step of the sequential extraction allowed to differentiate the major structural groups, which remain after application of chemical reagents used for fractionation of particulate OM (Table 1).

Additional results display that the solid residues recovered after DOM, SPOM, REOM and SEOM extractions have a comparable distribution of C among different organic groups in both substrate and digestate samples. Only a slight reduction of carbohydrates was observed in the SEOM solid residue in substrate. On the other hand, we observed a reduction of C in anomeric and carbohydrate organic groups in solid residues of substrate and digestate after the PEOM extraction, implying that application of 72% H2SO4 resulted in partial dissolution of cellulosic structures.

The 1 H- 13 C HSQC NMR spectra of digestate solid residues collected after extraction of DOM, SPOM, and REOM fractions were qualitatively similar. Thus, OM extraction by 10 mM CaCl2 and a mixture of 10 mM NaCl and 10 mM NaOH, used for extraction of SPOM and REOM, did not selectively remove OM groups from the particulate OM. However, the 1 H- 13 Thus, the OM extracted by 72% H2SO4 mainly originate from carbohydrate and partially from protein contents of the samples. These results support the findings in [START_REF] Jimenez | A new organic matter fractionation methodology for organic wastes: Bioaccessibility and complexity characterization for treatment optimization[END_REF], where the authors identified the biochemical nature of each extracted fractions by testing the sequential extraction protocol on several representative samples (e.g. lipid-rich agri-food waste, cardboard and crispbread). Based on percentage of chemical oxygen demand (COD) extracted in each fraction from the representative samples, the authors found that protein-like and lipid-like compounds were mainly extracted in SEOM fraction, whereas carbohydrates and holocelluloses in the PEOM fraction.

Similarly, 1 H- 13 C HSQC NMR spectra of substrate solid residues, collected after extraction of DOM, SPOM, and REOM were qualitatively similar, whereas peaks from CH(α) groups of the amino acids and peaks from aromatic amino acid side-chains were significantly reduced in solid residue collected after SEOM fraction. Moreover, signals related to cellulose and amino-acids were removed from the solid residue of substrate after the PEOM extraction. Furthermore, comparison of the amount of C extracted during the sequential extraction procedure and total C content of the samples demonstrated that more than 60% of the total initial C was still retained in both the substrate and digestate samples after sequential extraction. Nevertheless, the structural characterization of OM in our study revealed that NEOM, representing the fraction with low degree of bio-accessibility [START_REF] Jimenez | A new organic matter fractionation methodology for organic wastes: Bioaccessibility and complexity characterization for treatment optimization[END_REF], comprised mainly of aliphatic and aromatic CH groups of the protein/biomass fraction of the OM.

The major organic molecules targeted by the chemical reagents used for fractionation of particulate OM according to [START_REF] Jimenez | Methane production and fertilizing value of organic waste: Organic matter characterization for a better prediction of valorization pathways[END_REF][START_REF] Jimenez | A new organic matter fractionation methodology for organic wastes: Bioaccessibility and complexity characterization for treatment optimization[END_REF][START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF] and the findings of this study are summarized in Table 2. It should be highlighted that in this study, NMR spectroscopy was performed on the solid residues recovered at each step of the extraction procedure, whereas [START_REF] Jimenez | Methane production and fertilizing value of organic waste: Organic matter characterization for a better prediction of valorization pathways[END_REF][START_REF] Jimenez | A new organic matter fractionation methodology for organic wastes: Bioaccessibility and complexity characterization for treatment optimization[END_REF][START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF] identified the nature of the organic molecules in the liquid fractions extracted by the fractionation procedure. Similarities are only found in SEOM and PEOM fractions, whereas the organic molecules found in NEOM fraction reflects the nature of substrate analyzed.

(Table 2 here)

Simultaneous extraction of trace elements with organic matter fractions

Concentrations of TE extracted at each step during sequential extraction of OM are reported in Table 3. Element contents in CSH fraction are also reported as this fraction may include TE likely bound to inorganic ligands. Quantification of TE concentrations in PEOM fraction had a high degree of uncertainty due to analytical interferences, caused by reagent matrix and are omitted from Table 3. Overall, we observed a higher concentration of total TE in digestate than substrate (on TS basis), which is also a result of OM content reduction in digestate compared to substrate. Among elements, total concentrations of Fe, Al, Mn and Zn prevailed in digestate and substrate, whereas total concentration of As, Cd, Co, Cr, Mo, and Pb were lower than 11 µg/gTSin in both samples.

(Table 3 here) More than 60% of total Cd, Co, Fe, Mn, Ni and Zn were extracted together with DOM, SPOM, REOM, CSH and SEOM organic fractions of the digestate and substrate samples, whereas it is assumed that the remaining concentrations were in the residual pellet after SEOM fraction extraction. Additionally, sum of As concentrations in the fractions extracted along with sequential extraction of OM from substrate was higher than the As concentrations measured after total digestion of the samples by 7M HNO3 (Table 3). Molybdenum concentration in the samples was relatively low (2.3±0.1 µg/gTSin in digestate, 0.68±0.02 µg/gTSin in substrate) and only 13% of total Mo was recovered during the sequential extraction procedure, primarily in DOM and SPOM fraction. The recovery of Cr was also low, i.e. 18% and 29% for digestate and substrate, respectively, mainly found in DOM and CSH fractions, whereas only 4% of Cu was recovered in DOM fraction in digestate. In general, the highest concentration of all TE was found in DOM and CSH fractions of digestate and substrate. Low concentrations of Al, Fe, Mn, Mo and Ni were found in SPOM fraction of substrate and digestate, additionally Co and Zn were extracted from substrate in SPOM fraction. Notably, the concentration of five elements was below MLD and MLQ in digestate and substrate. Among quantified elements, Co, Fe, Mn and Mo were found in REOM fraction of both samples, additionally Al and Ni were extracted from digestate in this fraction. Finally, SEOM contained Al, Cr, Fe, Mn, Ni and Zn with relatively low concentrations extracted from both samples.

It should be emphasized that, except for the DOM fraction obtained by centrifugation, the other extraction steps involve reagents that may interact with TE species in the sample and promote the dissolution/precipitation of elements together with OM extraction. However, we do not exclude that TE, which were extracted together with the operationally defined fractions of OM, may originate from organically-bound and/or inorganic TE compounds (e.g. CSH fraction) in the samples. To further assess the origin of TE in OM fractions and assess the contribution of TE Indeed, we noticed that the concentration of elements found in the "shifted" CSH fraction is similar to the concentration of elements found in CSH fraction of the original fractionation procedure. Accordingly, the assessment of simultaneous extraction of OM and TE suggested that 31% to 98% and from 61% to 94% of total elements' content, depending on the specific element, are associated with the mineral fraction (i.e. CSH fraction) (or strongly bound to organic compounds) in substrate and digestate, respectively, whereas the remaining portion is likely associated with the extracted OM fractions.

Implications for simultaneous assessment of trace elements and organic matter bioaccessibility

Comparison of the OM fractionation of digestate and substrate demonstrated that the anaerobic digestion process resulted in a decrease of dissolved organic C in DOM fraction of the substrate, while the PEOM fraction was enriched in the digestate (Figure 2 a,b). The DOM fraction contains more bio-accessible organic substances compared to the other fractions, whereas PEOM represents the least bio-accessible fraction of the OM, which is mainly composed of hemicellulose, cellulose and starch based on 13 C CPMAS and 1 H- 13 C HSQC NMR spectroscopy in this study (Table 2).

A distribution of TE based on different degree of bio-accessibility is proposed in Figure 3. The proposed distribution is based on the knowledge of the leaching strength of the reagents used during the extraction procedure and the results obtained from the modified sequential extraction procedure (i.e. mineral fraction extracted at the beginning of the extraction procedure). Therefore, we suggest that TE found in DOM fraction are mobile in the digester environment and thus, more bio-accessible. The TE found in SPOM fraction are potentially bio-accessible, whereas TE found in REOM, CSH and SEOM fractions are considered poorly-bio-accessible. It is noteworthy that the CSH fraction contains metals bound to minerals with different solubility (e.g. metals bound to phosphate and carbonate minerals encompass a higher solubility and potential bioavailability in the digester environment than amorphous metal-sulfides). Moreover, part of TE extracted in CSH fraction could be associated to mineral particles present in extracellular polymeric substances (EPS) as observed by D'Abzac et al. (2010) using scanning electron microscopic analysis in anaerobic granular sludge. Therefore, association of metals in CSH to poorly-accessible fraction should be considered relative to the operationally-defined fractions of metals in this study, which is based on leaching strength of the reagent used. Finally, the elements not extracted by the sequential extraction procedure are likely not immediately bio-accessible, but they may be mobilized on the long term after degradation of the OM present in PEOM and NEOM fractions.

The high concentration of TE found in DOM fraction is related to the presence of dissolved metal species (free ions and complexes with inorganic and organic metal-binding ligands) as well as metal-containing colloids and particles (<0.45 µm). Organic macromolecules such as proteins may as well contain metals (e.g. Co-containing vitamin B12), which contribute to the pool of metals associated with DOM (Shakeri Yekta et al., 2014a; [START_REF] Zhu | Sequential extraction of anaerobic digestate sludge for the determination of partitioning of heavy metals[END_REF]. Therefore, we hypothesize that TE in DOM fraction are accessible for interaction with the biological interface.

Regarding TE found in SPOM fraction, obtained by washing the sample pellets with CaCl2 reagent, we assume that elements are potentially mobile and bio-accessible since they were likely released in solution by ion exchange mechanisms with Ca 2+ or Cl -, therefore TE bio-accessibility is related to availability of this fraction for taking part in ion-exchange reactions. Indeed, CaCl2 reagent is commonly used in soil analysis to extract the exchangeable fraction of TE which is also the most available fraction for plant uptake [START_REF] Filgueiras | Chemical sequential extraction for metal partitioning in environmental solid samples[END_REF][START_REF] Houba | State of the art and future developments in soil analysis for bioavailability assessment[END_REF]. The TE associated with CSH fraction is likely related to the TE as minerals, such as amorphous metal sulfide, metal carbonate and metal phosphate precipitates, which are dissolved under acidic conditions upon addition of HCl. Accordingly, the accessibility of TE in the form of inorganic precipitates in solid phase is probably limited, and the availability of TE bound to this fraction is largely dependent on the solubility of the metal-containing minerals. Chemical speciation analysis of TE in different anaerobic digesters suggested that sulfide is likely the major inorganic ligand, scavenging TE from aqueous phase in co-digesters and the TE in solid phase is dominated as TE-sulfide (Shakeri Yekta et al., 2014b). Due to poor solubility of TE-sulfide minerals as potential dominant species in the CSH fraction, the accessibility of metals in this fraction for microorganisms is likely constrained. REOM and SEOM fractions include elements extracted from the samples under alkaline condition (pH [START_REF] Fermoso | Fate of Trace Metals in Anaerobic Digestion[END_REF][START_REF] Filgueiras | Chemical sequential extraction for metal partitioning in environmental solid samples[END_REF]. Prevalence of such high pH is uncommon in anaerobic digesters and in environment. Furthermore, dissolution of metal species, which commonly occur at low pH, is unlikely to occur during extraction of REOM and SEOM fractions, implying that TE extracted might potentially originate from the simultaneously extracted OM. As this fraction of OM is considered less accessible and could be solubilized only after a pH increase, the bio-accessibility of TE associated with REOM and SEOM fractions may be limited.

In substrate, more than 50% of total As, Co, Mn and Ni are bio-accessible or potentially bioaccessible, whereas less than 40% of total Fe, Zn and Mo are bio-accessible or potentially bioaccessible. It is well reported that Co, Fe, Mn, Mo, Ni and Zn are important TE for optimal performance of the anaerobic digestion processes as demonstrated by Gustavsson et al. [START_REF] Gustavsson | Bioavailability of cobalt and nickel during anaerobic digestion of sulfur-rich stillage for biogas formation[END_REF][START_REF] Gustavsson | The feasibility of trace element supplementation for stable operation of wheat stillage-fed biogas tank reactors[END_REF] and reviewed by [START_REF] Zandvoort | Trace Metals in Anaerobic Granular Sludge Reactors: Bioavailability and Dosing Strategies[END_REF] and by [START_REF] Demirel | Trace element requirements of agricultural biogas digesters during biological conversion of renewable biomass to methane[END_REF]. Therefore, their bio-accessibility for anaerobic microorganisms is relevant. In digestate, except for As, all elements have poor or limited bio-accessibility, suggesting the formation of more stable forms of trace elements during anaerobic digestion process. This information is relevant to fulfill national or European requirements for application of digestate as soil amendment. Indeed, the current results show that less than 20% of total Cd, Cr, Cu, Ni, Pb and Zn, considered harmful elements for plant uptake when present at high concentrations [START_REF] Saveyn | End-of-waste criteria for biodegradable waste subjected to biological treatment (compost & digestate[END_REF], are immediately bio-accessible or potentially bio-accessible. However, the other elements which serve as nutrient for plants also have poor or no bio-accessibility.

(Figure 3 here)

Conclusions

More than 60% of total As, Cd, Co, Fe, Mn, Ni and Zn were extracted during the fractionation procedure mainly in DOM and CSH fractions, which were defined as the immediately and poorly bio-accessible fractions, respectively. Between 31% and 98% of total elements was likely associated to minerals (CSH fraction) in both substrate and digestate, whereas the remaining elements were associated with OM. We observed that SEOM fraction mainly contains proteins, whereas PEOM contains hemicellulose, cellulose, starch and certain proteins. However, no specific organic molecules were extracted along with SPOM and REOM fractions.

Tables and Figures

  and insoluble organic compounds according to[START_REF] Jimenez | Methane production and fertilizing value of organic waste: Organic matter characterization for a better prediction of valorization pathways[END_REF][START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF]. In substrate, only 9% of extracted organic C was contained in PEOM and 15% of extracted organic C was present in SPOM, REOM and SEOM, which mainly contain proteins and sugars, lipids, humic-like and fulvic acid-like structures based on fluorescence spectroscopic characterization of the OM extracted in the supernatant after each step by[START_REF] Jimenez | Methane production and fertilizing value of organic waste: Organic matter characterization for a better prediction of valorization pathways[END_REF]. In digestate, 28% of extracted organic C was as DOM, while 25% of extracted organic C was present in SPOM, REOM and SEOM.To support the OM characterization of the extracted fractions provided by[START_REF] Jimenez | Methane production and fertilizing value of organic waste: Organic matter characterization for a better prediction of valorization pathways[END_REF][START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF], NMR spectroscopy was applied to look into the structural compositions of the organic molecules in the solid residues recovered after each extraction step of the sequential extraction procedure. Moreover, NMR spectroscopy has shown to yield good resolutions of spectra in complex organic matrices such as substrate and digestate (Shakeri Yekta et al., 2018).Based on Kögel-Knabner (1997) and on[START_REF] Tambone | Assessing amendment properties of digestate by studying the organic matter composition and the degree of biological stability during the anaerobic digestion of the organic fraction of MSW[END_REF],13 C CPMAS NMR spectra, presented in Figures2c and 2d, were divided in five regions corresponding to different organic structures: aliphatic chain C (δC 0-47 ppm), carbohydrates (δC 47-90 ppm), anomeric C (δC 90-110 ppm), aromatic C (δC 110-160 ppm) and carbonyl C (δC 160-187 ppm). Figures2c and 2d

  show a higher contribution of aliphatic, aromatic and carbonyl C resonances in spectra of the digestate compared to the substrate, while carbohydrates signals had a higher contribution in the substrate spectra. Accordingly, aliphatic, aromatic and carbonyl C, mainly attributed to lipidand/or protein-like structures[START_REF] Keeler | Chemical-structural information from solidstate 13C NMR studies of a suite of humic materials from a lower montane forest soil, Colorado, USA[END_REF][START_REF] Kögel-Knabner | 13C and 15N NMR spectroscopy as a tool in soil organic matter studies[END_REF][START_REF] Simpson | NMR spectroscopy in environmental research: From molecular interactions to global processes[END_REF], were enriched in the solid phase of the digestate upon anaerobic digestion. These observations are in agreement with previously reported13 C CPMAS NMR results by Tambone et al. (2013), who showed a decrease of O-alkyl carbon signals (δC 47-113 ppm) attributed to carbohydrates in substrates, whereas aromatic-C (δC 113-160 ppm) and aliphatic chain C (δC 0-47 ppm) accumulated in digestate samples. Additionally, 2D 1 H-13 C HSQC NMR spectra of solid residue from substrate and digestate show that anomeric signals at δH 4.45-5.2 ppm and δC 99-102 ppm (Simpson et al., 2011) as well as Oalkyl signals at δH 3.4-3.8 ppm and δC 68-79 ppm (Soucémarianadin et al., 2017) from hemicellulose and starch were more readily degraded during the anaerobic digestion process compared to non-anomeric (δH 3.24-3.63 ppm and δC 60-72 ppm) (Soucémarianadin et al., 2017) and anomeric C signals (δH/C: 4.32/102.4 ppm) (Soucémarianadin et al., 2017) from cellulose, which was left in the digestate. Interestingly, the peak from unsaturated double bonds of aliphatic structures such as fatty acids (δH/C: 5.3/130.1 ppm) is not visible in the digestate sample, indicating that aliphatic double bonds in substrate OM were susceptible to degradation during the anaerobic digestion. (Figure 2 here)

  C HSQC NMR spectrum of pellet collected after extraction of SEOM fraction contained fewer cross peaks within the chemical shift regions assigned to CH(α) groups of amino acids in peptide chains and proteins (δH/C: 4.0-4.7/45-62 ppm) (Simpson et al., 2011). Furthermore, several peaks in the aliphatic region where signals from amino acid side-chains appear, e.g. the peak at δH/C: 2.0/15.1 ppm assigned to methionine CH3-groups (Shakeri Yekta et al., 2018), were absent or had reduced intensities in the spectra of pellets after SEOM extraction. The major peaks in the aromatic region, δH/C: 6.5-7.4/113-134 ppm, assigned to aromatic side-chains of amino acids (based on comparisons with reference spectra) also experienced a reduction in signal intensities after SEOM extraction. Accordingly, 0.1M NaOH reagent mainly extracted proteins in SEOM fraction. It is notable that the presence of protein-derived resonances in the spectra even after SEOM extraction indicates that the proteins were only partially extracted during this step. The 1 H-13 C HSQC NMR spectra of digestate proved that carbohydrate signals mainly from cellulose, δH/C: 4.1-5.2/94-106 ppm (anomeric) and δH/C: 2.9-4.11/59-84 ppm (O-alkyl), and signals related to amino acids were removed from the solid residue after the PEOM extraction.

  containing minerals compounds during the sequential extraction, we performed the CSH fraction extraction step between DOM and SPOM extraction steps. Addition of 0.1M HCl during CSH extraction results in dissolution of metals bound to minerals under acidic conditions e.g. metals bound to carbonate, phosphate and amorphous metal sulfide (Albacete et al., 2015; Filgueiras et al., 2002; Rickard and Morse, 2005). Thus, shifting the extraction of CSH fraction prior to sequential extraction of SPOM, REOM, and SEOM allows the removal of metals bound to minerals, whereas TE simultaneously extracted during the subsequent extraction steps represent the fractions most likely bound to OM. Thus, concentrations of TE in each fraction provide information on potential association of elements with operationally defined OM fractions.

Figure 1 .

 1 Figure 1. Analyses performed on the supernatants and solid residues collected during the sequential extraction procedure. DOC: dissolved organic carbon; TE: trace elements analysis; NMR: nuclear magnetic resonance spectroscopy; C/N: total carbon and nitrogen content; TS-VS: total and volatile solids content.

Figure 2 .

 2 Figure 2. Organic matter characterization of substrate and digestate according to the sequential extraction procedure and NMR spectra. a-b) Relative distribution of organic C extracted after each step of the sequential extraction procedure form substrate and digestate samples. c-d) 13 C CPMAS NMR spectra of DOM solid residues (n:3) from substrate and digestate. The integrated peak areas are expressed in % of total integral.

Figure 3 .

 3 Figure 3. Interpretation of metals fractionation in terms of potential bio accessibility in substrate and digestate.

  et al.(2017, 2015, 2014) and the one performed in this study. The nature of the organic molecules was identified by using reference samples and/or 3D fluorescence spectroscopy in the work of[START_REF] Jimenez | Methane production and fertilizing value of organic waste: Organic matter characterization for a better prediction of valorization pathways[END_REF][START_REF] Jimenez | A new organic matter fractionation methodology for organic wastes: Bioaccessibility and complexity characterization for treatment optimization[END_REF][START_REF] Jimenez | Prediction of anaerobic biodegradability and bioaccessibility of municipal sludge by coupling sequential extractions with fluorescence spectroscopy: Towards ADM1 variables characterization[END_REF], whereas NMR spectroscopy was used in this study.

Figure 1 .

 1 Figure 1.

28

 28

Figure 2 .

 2 Figure 2.

Figure 3

 3 Figure 3.

Table Captions Table 1 .

 Captions1 Sequential extraction procedure adapted from Jimenez et al. (2017, 2014) with some modifications highlighted in italic font. The extracted fractions are listed in order of decreasing bio-accessibility.

Table 2 .

 2 Target molecules extracted by the sequential extraction procedure adopted by Jimenez

Table 3 .

 3 Trace elements concentration found in each extracted fraction and total elements concentration in digestate (grey rows) and substrate (white rows). Except of DOM fraction extraction, results are mean of triplicate ± standard deviation.

Table 2 .

 2

	480
		Target molecules by Jimenez et al. (2017, 2015,
			Target molecules according to this study (NMR
	Fractions	2014) (Reference samples + 3D fluorescence
			spectroscopy)
		spectroscopy)
	SPOM	Water-soluble proteins and sugars	The reagent did not selectively remove OM
	REOM	Proteins and lipids	The reagent did not selectively remove OM
		Humic-like and fulvic like acids, complex	Certain proteins (CH(α) groups of the amino
	SEOM	proteins (i.e. glucolated proteins) and certain	acids; methionine CH3-groups; aromatic side-
		lignocellulosic compounds	chains of amino acids)
			Carbohydrate (e.g. hemicellulose, cellulose and
	PEOM	Hemicellulose and cellulose
			starch) and certain proteins
		Lignin-like compounds and non-extractable	Protein/biomass (aliphatic and aromatic CH
	NEOM
		humic-like acids (i.e. humin)	groups)

Table 3 .

 3

	482
		DOM	SPOM	REOM	CSH	SEOM	Total	% of total
		(µg/gTSin)	(µg/gTSin)	(µg/gTSin)	(µg/gTSin)	(µg/gTSin)	(µg/gTSin)	content
	Al	23.2	1.4±0.1	7.4±0.8	386.6±147.1	44.2±23.6	1359.4±33.7	34%
		7.7	0.9±0.2	<1.0#	318.0±8.6	10.3±1.0	722.1±59.7	47%
	As	0.6	<0.1*	<0.02#	<0.4*	<0.1#	1.5±0.1	42%
		0.4	<0.1*	<0.01#	<0.2#	<0.1#	0.22±0.02	180%
	Cd	0.01	<0.1#	<0.0003#	0.13±0.01	<0.004*	0.22±0.01	63%
		0.01	<0.04#	<0.01#	0.09±0.00	<0.001#	0.10±0.00	95%
	Co	1.1	<0.2*	0.05±0.00	6.9±0.4	0.28±0.02	10.5±0.1	79%
		2.7	0.35±0.01	0.01±0.00	0.32±0.02	<0.01*	4.04±0.04	83%
	Cr	0.3	<0.1#	<0.01#	0.7±0.1	0.08±0.00	5.9±0.2	18%
		0.2	<0.1#	<0.01*	0.32±0.01	0.05±0.00	2.0±0.1	29%
	Cu	1.4	<0.9#	<0.8#	<3.4*	<0.1*	40.4±6.4	4%
		0.2	<0.5#	<0.5#	6.2±0.1	<0.4#	<35.8 §	-
					8502.1±253		12623.1±22
	Fe	471.8	12.8±1.0	9.1±0.7		68.7±9.0		72%
					5.0		2.8
					2119.9±108.
		749.3	6.8±2.1	2.9±0.3		36.1±1.9	4393.0±71.0	66%
					3
	Mn	8.7	1.9±0.1	0.18±0.02	95.0±33.3	0.4±0.1	121.1±5.1	88%
		16.2	4.2±0.1	0.15±0.02	9.0±0.8	0.07±0.00	46.1±1.5	64%
	Mo	0.2	0.09±0.01	0.04±0.00	0.01±0.00	<0.1#	2.3±0.1	13%
		0.2	0.01±0.00	0.02±0.00	<0.02*	<0.1#	0.68±0.02	28%
	Ni	1.6	0.56±0.01	0.17±0.02	17.1±6.2	0.22±0.02	23.5±0.2	84%
		1.3	0.17±0.03	<0.01*	0.5±0.1	0.03±0.01	2.7±0.1	75%
	Pb	0.1	<0.1#	<0.04#	1.6±0.3	<0.1*	3.8±0.6	45%
		0.01	<0.05#	<0.02#	0.6±0.1	<0.1*	<1.8 §	-

Acknowledgements

The Biogas Research Center at Linköping University is acknowledged for providing materials and laboratories to perform analytical analyses. Lena Lundman is also acknowledged for laboratory assistance and the Knut and Alice Wallenberg foundation program NMR for Life (www.nmrforlife.se) for NMR spectroscopy support. Finally, members of COST Action ES1302 European Network on Ecological Functions of Trace Metals in Anaerobic Biotechnologies are acknowledged for providing insightful comments to this experimental work.

Funding sources

This work was supported by the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 643071. This work was also supported by the Swedish Research Council Formas grant No 2016-01054.

Appendix A. Supplementary data E-supplementary data for this work can be found in e-version of this paper online.

Conflict of interest

The authors declare no conflict of interest. The percentage of total extracted is the ratio of the sum of element's concentration in DOM, SPOM, REOM, CSH and SEOM over the total element' concentration.

*MLQ=average blanks ± 10*standard deviation blanks (n=36) expressed on the same concentration basis (µg/gTSin) as those for the samples using 25.6 gTSin/l and 41.6 gTSin/l as conversion factor for digestate and substrate respectively.

#MLD=average blanks ± 3*standard deviation blanks (n=36) § MLQ=average blanks ± 10*standard deviation blanks (n=3), using 10.5 gTSin/l as conversion factor.


Kevin Fauvel
email: kevin.fauvel@inria.fr

Véronique Masson

Elisa Fromont
email: elisa.fromont@irisa.fr

Élisa Fromont

A Performance-Explainability Framework to Benchmark Machine Learning Methods: Application to Multivariate Time Series Classifiers

published or not. The documents may come

Introduction

There has been an increasing trend in recent years to leverage machine learning methods to automate decision-making processes. However, for many applications, the adoption of such methods cannot rely solely on their prediction performance. For example, the European Union's General Data Protection Regulation, which became enforceable on 25 May 2018, introduces a right to explanation for all individuals so that they can obtain "meaningful explanations of the logic involved" when automated decision-making has "legal effects" on individuals or similarly "significantly affecting" them1 . Therefore, in addition to their prediction performance, machine learning methods have to be assessed on how they can supply their decisions with explanations.

The performance of a machine learning method can be assessed by the extent to which it correctly predicts unseen instances. A metric like the accuracy score commonly measures the performance of a classification model. However, there is no standard approach to assess explainability. First, there is no mathematical definition of explainability. A definition proposed by [Miller, 2019] states that the higher the explainability of a machine learning algorithm, the easier it is for someone to comprehend why certain decisions or predictions have been made. Second, there are several methods belonging to different categories (explainability-by-design, post-hoc model-specific explainability and post-hoc modelagnostic explainability) [START_REF] Du | [END_REF], which provide their own form of explanations.

The requirements for explainable machine learning methods are dependent upon the application and to whom the explanations are intended for [Tomsett et al., 2018;Bohlen-der and Kóhl, 2019]. In order to match these requirements and conduct experiments to validate the usefulness of the explanations by the end-users, there is a need to have a comprehensive assessment of the explainability of the existing methods. Doshi-Velez and Kim [2017] claim that creating a shared language is essential for the evaluation and comparison of machine learning methods, which is currently challenging without a set of explanation characteristics. As far as we have seen, there is no existing framework which defines a set of explanation characteristics that systematize the assessment of the explainability of existing machine learning methods.

Hence, in this paper, we propose a new framework to assess and benchmark the performance-explainability characteristics of machine learning methods. The framework hypothesizes a set of explanation characteristics, and as emphasized in [Wolf, 2019], focuses on what people might need to understand about machine learning methods in order to act in concert with the model outputs. The framework does not claim to be exhaustive and excludes application-specific implementation constraints like time, memory usage and privacy. It could be a basis for the development of a comprehensive assessment of the machine learning methods with regards to their performance and explainability and for the design of new machine learning methods. Due to space constraint, we limit the illustration of the use of the framework to one category of machine learning methods and we choose the Multivariate Time Series (MTS) classifiers. Multivariate data which integrates temporal evolution has received significant interests over the past decade, driven by automatic and high-resolution monitoring applications (e.g. healthcare [START_REF] Li | [END_REF], mobility [START_REF] Jiang | [END_REF], natural disasters [Fauvel et al., 2020a]). Moreover, the available explainability solutions to support the current state-of-the-art MTS classifiers remain limited, so this category of methods appears meaningful to assess for us.

The contributions of this paper are the following:

• We present a new performance-explainability analytical framework to assess and benchmark machine learning methods; • We detail a set of characteristics that systematize the performance-explainability assessment of existing machine learning methods; • We illustrate the use of the framework by benchmarking the current state-of-the-art MTS classifiers.

In this section, we first position this paper in the related work and introduce the different categories of explainability methods as a background to the notions that will be discussed in the framework. Then, we present the state-of-the-art machine learning methods that will be used to illustrate the framework, i.e. MTS classifiers.

Explainability

Multiple taxonomies of explainability methods have been derived from different frameworks [START_REF] Guidotti | [END_REF]Ventocilla et al., 2018;[START_REF] Du | [END_REF]. However, none of them defines a set of explanation characteristics that systematize the assessment of the explainability of existing machine learning methods. [START_REF] Guidotti | [END_REF] provides a classification of the main problems addressed in the literature with respect to the notion of explanation and the type of machine learning systems. [Ventocilla et al., 2018] proposes a high-level taxonomy of interpretable and interactive machine learning composed of six elements (Dataset, Optimizer, Model, Predictions, Evaluator and Goodness). And, [START_REF] Du | [END_REF] categorizes existing explainability methods of machine learning models into either by design or post-hoc explainability. As our framework aims to cover all types of methods, we do not present the frameworks focusing on a particular type of explainability methods (e.g. [START_REF] Lundberg | [END_REF]Ancona et al., 2018;[START_REF] Henin | [END_REF]).

A five-step method to understand the requirements for explainable AI systems has been published in [Hall et al., 2019]. The five steps are: explainee role definition, explanation characteristics identification, requirements collection, existing methods assessment and requirements/existing methods mapping. Our framework can be positioned as a further development of the fourth step of the method by detailing a set of explanations characteristics that systematize the assessment of existing methods. Our framework does not include application-specific implementation constraints like time, memory usage and privacy.

As a background to the notions that will be discussed in the framework, we introduce the three commonly recognized categories (explainability-by-design, post-hoc modelspecific explainability and post-hoc model-agnostic explainability) [START_REF] Du | [END_REF] to which all of the explainability methods are belonging to. First, some machine learning models provide explainability-by-design. These self-explanatory models incorporate explainability directly to their structures. This category includes, for example, decision trees, rulebased models and linear models. Next, post-hoc modelspecific explainability methods are specifically designed to extract explanations for a particular model. These methods usually derive explanations by examining internal model structures and parameters. For example, a method has been designed to measure the contribution of each feature in random forests [START_REF] Palczewska | [END_REF]; and another one has been designed to identify the regions of input data that are important for predictions in convolutional neural networks using the class-specific gradient information [START_REF] Selvaraju | [END_REF]. Finally, post-hoc model-agnostic explainability methods provide explanations from any machine learning model.

These methods treat the model as a black-box and does not inspect internal model parameters. For example, the permutation feature importance method [Altmann et al., 2010] and the methods using an explainable surrogate model [Lakkaraju et al., 2017;[START_REF] Lundberg | [END_REF][START_REF] Ribeiro | [END_REF][START_REF] Guidotti | [END_REF] belong to this category.

The explainability methods presented reflect the diversity of explanations generated to support model predictions, therefore the need for a framework in order to benchmark the machine learning methods explainability. The next section present the MTS classifiers that will be used to illustrate the framework.

Multivariate Time Series Classifiers

The state-of-the-art MTS classifiers consist of a diverse range of methods which can be categorized into three families: similarity-based, feature-based and deep learning methods.

Similarity-based methods make use of similarity measures to compare two MTS. Dynamic Time Warping (DTW) has been shown to be the best similarity measure to use along the k-Nearest Neighbors (k-NN) [Seto et al., 2015]. There are two versions of kNN-DTW for MTS: dependent (DTW D ) and independent (DTW I ). Neither dominates over the other [START_REF] Shokoohi-Yekta | [END_REF] from an accuracy perspective but DTW I allows the analysis of distance differences at feature level.

Next, feature-based methods include shapelets (gRSF [START_REF] Karlsson | Generalized Random Shapelet Forests[END_REF], UFS [START_REF] Wistuba | [END_REF]) and bag-of-words (LPS [Baydogan and Runger, 2016], mv-ARF [START_REF] Tuncel | [END_REF], SMTS [START_REF] Baydogan | [END_REF], WEASEL+MUSE [Schäfer and Leser, 2017]) models. WEASEL+MUSE shows better results compared to gRSF, LPS, mv-ARF, SMTS and UFS on average (20 MTS datasets). WEASEL+MUSE generates a bag-of-words representation by applying various sliding windows with different sizes on each discretized dimension (Symbolic Fourier Approximation) to capture features (unigrams, bigrams, dimension identification). Following a feature selection with chi-square test, it classifies the MTS based on a logistic regression classifier.

Then, deep learning methods use Long-Short Term Memory (LSTM) and/or Convolutional Neural Networks (CNN). According to the results published, the current state-of-theart model (MLSTM-FCN) is proposed in [START_REF] Karim | [END_REF] and consists of a LSTM layer and a stacked CNN layer along with Squeeze-and-Excitation blocks to generate latent features.

Therefore, we choose to benchmark the performanceexplainability of the best-in-class for each similaritybased, feature-based and deep learning category (DTW I , WEASEL+MUSE and MLSTM-FCN classifiers). The next section introduces the performance-explainability framework, which is illustrated with the benchmark of the bestin-class MTS classifiers in section 4.

Performance-Explainability Framework

The framework aims to respond to the different questions an end-user may ask to take an informed decision based on the predictions made by a machine learning model: What is the level of performance of the model? Is the model comprehensible? Is it possible to get an explanation for a particular instance? Which kind of information does the explanation provide? Can we trust the explanations? What is the target user category of the explanations? The performanceexplainability framework that we propose is composed of the following components, which will also be translated into terms specific to our application (MTS classifiers) whenever relevant:

Performance What is the level of performance of the model? The first component of the framework characterizes the performance of a machine learning model. Different methods (e.g. holdout, k-fold cross-validation) and metrics (e.g. accuracy, F-measure, Area Under the ROC Curve) exist to evaluate the performance of a machine learning model [START_REF] Witten | Data Mining: Practical Machine Learning Tools and Techniques[END_REF]. However, there is no consensus on an evaluation procedure to assess the performance of a machine learning model. Recent work suggests that the definition of such an evaluation procedure necessitates the development of a measurement theory for machine learning [Flach, 2019]. Many of the problems stem from a limited appreciation of the importance of the scale on which the evaluation measures are expressed.

Then, in current practices, the choice of a metric to evaluate the performance of a machine learning model depends on the application. According to the application, a metric aligned with the goal of the experiments is selected, which prevents the performance comparison of machine learning models across applications.

Therefore, the performance component in the framework is defined as a first step towards a standard procedure to assess the performance of machine learning models. It corresponds to the relative performance of a model on a particular application. More specifically, it indicates the relative performance of the models as compared to the state-of-the-art model on a particular application and an evaluation setting. This definition allows the categorization of the models' performance on an application and an evaluation setting. In the case of different applications with a similar machine learning task, the performance component can give the list of models which outperformed current state-of-the-art models on their respective application. Thus, it points to certain models that could be interesting to evaluate on a new application, without providing guarantee that these models would perform the same on this new application. We propose an assessment of the performance in three categories:

• Best: best performance. It corresponds to the performance of the first ranked model on the application following an evaluation setting (models, evaluation method, datasets); • Similar: performance similar to that of the state-of-theart models. Based on the same evaluation setting, it corresponds to all the models which do not show a statistically significant performance difference with the second ranked model. For example, the statistical comparison of multiple classifiers on multiple datasets is usually presented on a critical difference diagram [Demšar, 2006];

• Below: performance below that of the state-of-the-art models. It corresponds to the performance of the remaining models with the same evaluation setting.

Model Comprehensibility Is the model comprehensible?

The model comprehensibility corresponds to the ability for the user to understand how the model works and produces certain predictions. Comprehensibility is tightly linked to the model complexity; yet, there is no consensus on model complexity assessment [START_REF] Guidotti | [END_REF]. Currently, two categories of models are commonly recognized: "whitebox" models, i.e. easy-to-understand models, and "blackbox" models, i.e. complicated-to-understand models [Lipton, 2016]. For example, many rule-based models and decision trees are regarded as "white-box" models while ensemble methods and deep learning models are "black-box" models. Not all rule-based models or decision trees are "whitebox" models. Cognitive limitations of humans place restrictions on the complexity of the approximations that are understandable to humans. For example, a decision tree with a hundred levels cannot be considered as an easy-to-understand model [Lakkaraju et al., 2017].

Nevertheless, the distinction between "white-box" models and "black-box" models is clear among the machine learning methods of this paper. The state-of-the-art MTS classifiers are all "black-box" except one which is an easy-to-understand similarity-based approach. Therefore, due to space limitation, we propose a first assessment of the comprehensibility in two categories and we plan to further elaborate this component in future work:

• Black-Box: "black-box" model, i.e. complicated-tounderstand models; • White-Box: "white-box" model, i.e. easy-to-understand models.

Granularity of the Explanations Is it possible to get an explanation for a particular instance? The granularity indicates the level of possible explanations. Two levels are generally distinguished: global and local [START_REF] Du | [END_REF]. Global explainability means that explanations concern the overall behavior of the model across the full dataset, while local explainability informs the user about a particular prediction. Some methods can provide either global or local-only explainability while other methods can provide both (e.g. decision trees). Therefore, we propose an assessment of the granularity in three categories:

• Global: global explainability;

• Local: local explainability;

• Global & Local: both global and local explainability. Information Type Which kind of information does the explanation provide? The information type informs the user about the kind of information communicated. The most valuable information is close to the language of human reasoning, with causal and counterfactual rules [START_REF] Pearl | [END_REF]. Causal rules can tell the user that certain observed variables are the causes of specific model predictions. However, machine learning usually leverages statistical associations in the data and do not convey information about the causal relationships among the observed variables and the unobserved confounding variables. The usual statistical asso-ciations discovered by machine learning methods highly depend on the machine learning task. Therefore, we first give a generic high-level definition of the information type and then we detail and illustrate it for the application case of this paper (MTS classification). We propose a generic assessment of the information type in 3 categories from the least to the most informative:

• Importance: the explanations reveal the relative importance of each dataset variable on predictions. The importance indicates the statistical contribution of each variable to the underlying model when making decisions; • Patterns: the explanations provide the small conjunctions of symbols with a predefined semantic (patterns) associated with the predictions; • Causal: the most informative category corresponds to explanations under the form of causal rules;

In this paper, the issue of Multivariate Time Series (MTS) classification is addressed. A MTS M = {x 1 , ..., x d } ∈ R d * l is an ordered sequence of d ∈ N streams with x i = (x i,1 , ..., x i,l ), where l is the length of the time series and d is the number of multivariate dimensions. Thus, considering the MTS data type, the information can be structured around the features, i.e. the observed variables, and the time. We propose to decompose the 3 categories presented into 8 categories. In addition, we will illustrate each of these categories with an application in the medical field. Figure 1 shows the first MTS of the UEA Atrial Fibrilation [START_REF] Bagnall | [END_REF] test set that belongs to the class Non-Terminating Atrial Fibrilation. This MTS is composed of two dimensions (two channels ECG) with a length of 640 (5 second period with 128 samples per second). It is worth noting that the explanations provided to illustrate each category are assumptive rather than validated, they are given as illustrative in nature. • Features (Importance): the explanations reveal the relative importance of the features on predictions. For example, in order to support a model output from the MTS of the Figure 1, the explanations could tell the user that the channel 2 has a greater importance on the prediction than the channel 1;

• Features + Time (Importance): the explanations provide the relative importance of the features and timestamps on predictions. For example, in order to support a model output from the MTS of the Figure 1, the explanations could tell the user that the channel 2 has a greater importance on the prediction than the channel 1 and that the timestamps are in increasing order of importance on the prediction; • Features + Time + Values (Importance): in addition to the relative importance of the features and timestamps on predictions, the explanations indicate the discriminative values of a feature for each class. For example in Figure 1, the explanations could give the same explanations as the previous category, plus, it could tell the user that the timestamps with the highest importance are associated with high values (values above 0.15) on the channel 2; • Uni Itemsets (Patterns): the explanations provide patterns under the form of groups of values, also called itemsets, which occur per feature and are associated with the prediction. For example, in order to support a model output from the MTS of the Figure 1, the explanations could tell the user that the following itemsets are associated with the prediction: {channel 1: extremely high value (above 1); channel 1: low value (below -0.05)} and {channel 2: high value (above 0.15); channel 2: extremely low value (below -0.1)}. The first itemset can be read as: the prediction is associated with the occurence on the channel 1 of an extremely high value being above 1 and a low value being below -0.05 at another moment, without information on which one appears first; • Multi Itemsets (Patterns): the explanations provide patterns under the form of multidimensional itemsets, i.e. groups of values composed of different features, which are associated with the prediction. For example, in order to support a model output from the MTS of the Figure 1, the explanations could tell the user that the following itemset is associated with the prediction: {channel 1: extremely high value (above 1); channel 2: high value (above 0.15)}; • Uni Sequences (Patterns): the explanations provide patterns under the form of ordered groups of values, also called sequences, which occur per feature and are associated with the prediction. For example, in order to support a model output from the MTS of the Figure 1, the explanations could tell the user that the following sequences are associated with the prediction: <channel 1: extremely high value (above 1); channel 1: low value (below -0.05)> and <channel 2: high values (above 0.15) with an increase during 1 second>. The first sequence can be read as: the prediction is associated with the occurrence on the channel 1 of an extremely high value being above 1 followed by a low value being below -0.05; • Multi Sequences (Patterns): the explanations provide patterns under the form of multidimensional sequences, i.e. ordered groups of values composed of different features, which are associated with the prediction. For example, in order to support a model output from the MTS 1 Predefined train/test splits and an arithmetic mean of the accuracies on 35 public datasets [START_REF] Karim | [END_REF]. As presented in section 2.2, the models evaluated in the benchmark are: DTW D , DTW I , gRSF, LPS, MLSTM-FCN, mv-ARF, SMTS, UFS and WEASEL+MUSE.

of the Figure 1, the explanations could tell the user that the following sequence is associated with the prediction: <channel 1: extremely high value (above 1); channel 2: high values (above 0.15) with an increase during 1 second>; • Causal: the last category corresponds to explanations under the form of causal rules. For example, in order to support a model output from the MTS of the Figure 1, the explanations could tell the user that the following rule applies: if (channel 1: extremely high value (above 1)) & 2: high values (above 0.15) with an increase during 1 second), then the MTS belongs to the class Non-Terminating Atrial Fibrilation. Faithfulness Can we trust the explanations? The faithfulness corresponds to the level of trust an end-user can have in the explanations of model predictions, i.e. the level of relatedness of the explanations to what the model actually computes. An explanation extracted directly from the original model is faithful by definition. Some post-hoc explanation methods propose to approximate the behavior of the original "blackbox" model with an explainable surrogate model. The explanations from the surrogate models cannot be perfectly faithful with respect to the original model [Rudin, 2019]. The fidelity criteria is used to quantify the faithfulness by the extent to which the surrogate model imitates the prediction score of the original model [START_REF] Guidotti | [END_REF].

In this paper, two MTS classifiers use an explainable surrogate model among the three state-of-the-art methods presented in section 4. However, there is no need to distinguish between the degree of fidelity of the surrogate models for the purpose of the comparison in this paper. Therefore, due to space limitation, we propose a first assessment of the faithfulness in two categories and we plan to further elaborate this component in future work:

• Imperfect: imperfect faithfulness (use of an explainable surrogate model); • Perfect: perfect faithfulness. User category What is the target user category of the explanations? The user category indicates the audience to whom the explanations are accessible. The user's experience will affect what kind of cognitive chunks they have, that is, how they organize individual elements of information into collections [START_REF] Neath | Human Memory: An Introduction to Research, Data, and Theory[END_REF]]. Thus, it could be interesting to categorize the user types and associate with the model to whom the explanations will be accessible to. The broader the audience, the better are the explanations. Therefore, we propose an assessment in three categories:

• Machine Learning Expert;

• Domain Expert: domain experts (e.g. professionals, researchers); • Broad Audience: non-domain experts (e.g. policy makers). In order to compare the methods visually using the proposed framework, the different aspects can be represented on a parallel coordinates plot. A parallel coordinate plot allows a 2-dimensional visualization of a high dimensional dataset and is suited for the categorical data of this framework. The next section presents an example of parallel coordinates plots comparing the state-of-the-art MTS classifiers.

Application to Multivariate Time Series Classifiers

This section shows how the framework presented in the previous section can be used to assess and benchmark the stateof-the-art MTS classifiers. As introduced in section 2.2, the state-of-the-art MTS classifiers are: DTW I , MLSTM-FCN and WEASEL+MUSE. The results of the assessment are summarized in Table 1, illustrated in Figure 2 and detailed in the following paragraphs.

The first MTS classifier belongs to the similarity-based category and is the one-nearest neighbor MTS classifier with DTW distance (DTW I ). DTW I classifies MTS based on the label of the nearest sample and a similarity calculated as the cumulative distances of all dimensions independently measured under DTW. For each MTS, the explanation supporting the classification is the ranking of features and timestamps in decreasing order of their DTW distance with the nearest MTS. Based on predefined train/test splits and an arithmetic mean of the accuracies, DTW I underperforms the current state-of-the-art MTS classifiers on the 35 public datasets (Performance: Below). The results from [START_REF] Karim | [END_REF] shows that DTW I has a statistically significant lower performance than MLSTM-FCN and WEASEL+MUSE. Furthermore, DTW I supports its predictions with limited information (Information: Features+Time) that needs to be analyzed by a domain expert to ensure that it is relevant for the application (User: Domain Expert). However, DTW I is an easyto-understand model (Comprehensibility: White-Box) which provides faithful explanations (Faithfulness: Perfect) for each MTS (Granularity: Local).

Then, we can analyze MLSTM-FCN and WEASEL+ MUSE together. First, based on predefined train/test splits and an arithmetic mean of the accuracies, MLSTM-FCN exhibits the best performance on the 35 public datasets [START_REF] Karim | [END_REF]. As presented in section 2.2, the models evaluated in the benchmark are: DTWD, DTWI , gRSF, LPS, MLSTM-FCN, mv-ARF, SMTS, UFS and WEASEL+MUSE.

(Performance: Best) [START_REF] Karim | [END_REF], followed by WEASEL+MUSE (Performance: Similar). Second, both MLSTM-FCN and WEASEL+MUSE are "black-box" classifiers without being explainable-by-design or having a posthoc model-specific explainability method. Thus, the explainability characteristics of these models depend on the choice of the post-hoc model-agnostic explainability method. We have selected SHapley Additive exPlanations (SHAP) [START_REF] Lundberg | [END_REF], a state-of-the-art post-hoc model-agnostic explainability method offering explanations at all granularity levels. SHAP method measures how much each variable (Features+Time) impacts predictions and comes up with a ranking of the variables which could be exploited by domain experts. The combination of MLSTM-FCN and WEASEL+MUSE with SHAP enables them to outperform DTW I while reaching explanations with a similar level of information (Information: Features+Time, DTW I : Fea-tures+Time), in the meantime remaining accessible to the same user category (User: Domain Expert, DTW I : Domain Expert). However, as opposed to DTW I , SHAP relies on a surrogate model which cannot provide perfectly faithful explanations (Faithfulness: Imperfect, DTW I : Perfect).

Therefore, based on the performance-explainability framework introduced, if a "white-box" model and perfect faithfulness are not required, it would be preferable to choose MLSTM-FCN with SHAP instead of the other state-of-the-art MTS classifiers on average on the 35 public datasets. In addition to its better level of performance, MLSTM-FCN with SHAP provides the same level of information and at all granularity levels.

However, the imperfect faithfulness of the explanations could prevent the use of MLSTM-FCN with a surrogate explainable model on numerous applications. In addition, the level of information provided to support the predictions remains limited (Information: Features+Time). Therefore, based on the assessment of the current state-of-the-art MTS classifiers with the framework proposed, it would be valuable for instance to design some new high-performing MTS classifiers which provide faithful and more informative explanations. For example, it could be interesting to work in the direction proposed in [Fauvel et al., 2020b]. It presents a new MTS classifier (XEM) which reconciles performance (Performance: Best) and faithfulness while providing the time window used to classify the whole MTS (Information: Uni Sequences). XEM is based on a new hybrid ensemble method that combines an explicit approach to handle the biasvariance trade-off and an implicit approach to individualize classifier errors on different parts of the training data [Fauvel et al., 2019]. Nevertheless, the explanations provided by XEM are only available per MTS (Granularity: Local) and the level of information could be further improved. As suggested by the authors, it could be interesting to analyze the time windows identified for each class to determine if they contain some common multidimensional sequences (Information: Multi Sequences, Granularity: Both Global & Local). These patterns could also broaden the audience as they would summarize the key information in the discriminative time windows.

Conclusion

We have presented a new performance-explainability analytical framework to assess and benchmark the machine learning methods. The framework details a set of characteristics that systematize the performance-explainability assessment of machine learning methods. In addition, it can be employed to identify ways to improve current machine learning methods and to design new ones. Finally, we have illustrated the use of the framework by benchmarking the current state-ofthe-art MTS classifiers. With regards to future work, we plan to further elaborate the definition of the different components of the framework (especially the Model Comprehensibility, the Information Type and the Faithfulness) and evaluate the relevance of integrating new components. Then, we plan to apply the framework extensively to assess the different types of existing machine learning methods.

Figure 1 :

 1 Figure 1: The first MTS sample of the UEA Atrial Fibrilation test set. It belongs to the class Non-Terminating Atrial Fibrilation and is composed of two channels ECG on a 5 second period (128 samples per second).

Figure 2 :

 2 Figure2: Parallel coordinates plot of the state-of-the-art MTS classifiers. Performance evaluation method: predefined train/test splits and an arithmetic mean of the accuracies on 35 public datasets[START_REF] Karim | [END_REF]. As presented in section 2.2, the models evaluated in the benchmark are: DTWD, DTWI , gRSF, LPS, MLSTM-FCN, mv-ARF, SMTS, UFS and WEASEL+MUSE.

Table 1 :

 1 Summary of framework results of the state-of-the-art MTS classifiers.

		Similarity-Based	Feature-Based	Deep Learning
		DTW I	WEASEL+MUSE with SHAP	MLSTM-FCN with SHAP
	Performance	Below 1	Similar 1	Best 1
	Comprehensibility	White-Box	Black-Box	Black-Box
	Granularity	Local	Both Global & Local	Both Global & Local
	Information	Features+Time	Features+Time	Features+Time
	Faithfulness	Perfect	Imperfect	Imperfect
	User	Domain Expert	Domain Expert	Domain Expert

https://ec.europa.eu/info/law/law-topic/data-protection en

Acknowledgments

This work was supported by the French National Research Agency under the Investments for the Future Program (ANR-16-CONV-0004) and the Inria Project Lab "Hybrid Approaches for Interpretable AI" (HyAIAI).