Tiffany Antoine

Christèle Icard-Vernière

Giulia Scorrano

Amal Salhi

Charlotte Halimi

Stéphane Georgé

Frédéric Carrière

Claire Mouquet-Rivier

Emmanuelle Reboul
email: emmanuelle.reboul@univ-amu.fr

Evaluation of vitamin D bioaccessibility and mineral solubility from test meals containing meat and/or cereals and/or pulses using in vitro digestion

Keywords: cholecalciferol, iron, zinc, magnesium, bioavailability, chickpeas, semolina, beef

ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Rethinking food systems for sustainability, from production to consumption, to provide better nutritional value at lower environmental cost is a major challenge. Pulses might contribute to improve sustainability, due to restoration of soil nitrogen [START_REF] Gan | Diversifying crop rotations with pulses enhances system productivity[END_REF] and to their content in high-quality proteins, fibers and minerals [START_REF] Margier | Nutritional Composition and Bioactive Content of Legumes: Characterization of Pulses Frequently Consumed in France and Effect of the Cooking Method[END_REF]. Unfortunately, pulse consumption has declined in past decades in industrialized countries [START_REF] Oliveira | Increasing Pulse Consumption to Improve Human Health and Food Security and to Mitigate Climate Change[END_REF], despite the efforts by international authorities, such as the Food and Agriculture Organization (FAO), to promote their consumption. In some French traditional dishes (i.e. cassoulet) pulses are consumed asin a replacement of the meal starchy component.

From the viewpoint of diet sustainability, bringing pulses, and other protein-rich plant foods, back in our plates would be of interest for substituting resource-consuming animal proteins [START_REF] Salome | Plant-Protein Diversity Is Critical to Ensuring the Nutritional Adequacy of Diets When Replacing Animal With Plant Protein: Observed and Modeled Diets of French Adults (INCA3)[END_REF].

However, pulses may also present some nutritional disadvantages, depending on their amount in the diet. Indeed, they contain several bioactive compounds, such as phytates or tannins, that persist after cooking and that can negatively affect mineral absorption by quenching them during digestion [START_REF] Champ | Non-nutrient bioactive substances of pulses[END_REF]. Additionally, we experimentally demonstrated that phytates and tannins, as well as fibers and saponins, can negatively influence the bioavailability of fatsoluble vitamins [START_REF] Margier | The Presence of Pulses within a Meal can Alter Fat-Soluble Vitamin Bioavailability[END_REF], possibly due to modulation of lipolysis during digestion.

These observations raise several questions about the amount of pulses that might be included in the diet without reducing the meal overall nutritional quality, especially by decreasing micronutrient bioavailability.

In this work, we assessed for the first time the bioaccessibility/solubility of a fat-soluble vitamin (vitamin D3, i.e. cholecalciferol) and minerals (iron, magnesium and zinc) in test meals made with different combinations of meat (minced beef), cereals (semolina) and pulses

Materials and methods

Chemicals

Vitamin D3 (> 95% pure), retinyl acetate (> 95% pure), trioctanoin, sodium taurodeoxycholate, pancreatin (P7545; 8×USP), fibers (cellulose), phytates (phytic acid sodium salt hydrate), saponins (purified quillaia saponins), and tannins (grape seed oligomeric proanthocyanidins) were purchased from Sigma Aldrich (Saint-Quentin-Fallavier, France).

Porcine colipase was purified as previously described (Rugani et al., 1995) and stored as a stock solution at 1mg/mL. Refined olive oil was generously provided by Dr Marie-Josèphe Amiot (UMR Moisa, Montpellier, France). Minced beef (Casino brand, 5% of fat), semolina (Casino brand) and fresh potatoes in bulk were bought at a local supermarket (Casino, Marseille, France). Chickpeas (Cicer arietinum) were purchased from the CIACAM society (Vitrolles, France). The chickpeas used in this study were from the same lots than those fully characterized in a previous work [START_REF] Margier | Nutritional Composition and Bioactive Content of Legumes: Characterization of Pulses Frequently Consumed in France and Effect of the Cooking Method[END_REF].

Fiber, phytate, saponin and tannin measurements in pulses and pureed potatoes

Fibers, phytates, saponins and tannins were previously measured in different prepared pulses [START_REF] Margier | Nutritional Composition and Bioactive Content of Legumes: Characterization of Pulses Frequently Consumed in France and Effect of the Cooking Method[END_REF] and potatoes [START_REF] Margier | The Presence of Pulses within a Meal can Alter Fat-Soluble Vitamin Bioavailability[END_REF]. Fibers, phytates, saponins and tannins from semolina were measured with the same methods. Results are summarized in Table 1.

In vitro digestion

Minced beef was pan-cooked (no added fat), semolina was cooked in an equal volume of hot water for 10 min, and chickpeas were canned at Centre Technique de Conservation des Produits Agricoles (CTCPA, Avignon, France), according to a standard protocol (Margier et al., 2018).

The test meals used in the first set of in vitro digestion experiments contained 100 µL of refined olive oil supplemented with 100 ng of vitamin D, and 0.6 g of pan-cooked minced beef, 3.35 g of pureed potatoes (control) and/or 3.35 g of canned chickpeas. The proportions of the different foods were chosen according to a previous validated protocol [START_REF] Reboul | Bioaccessibility of carotenoids and vitamin E from their main dietary sources[END_REF].

The test meals used in the second set of experiments contained 3.35 g of potatoes and 200 µL of refined olive oil, and were supplemented with purified cellulose (0.35 mg), phytates (21.25 mg), saponins (3 mg) or tannins (0.2 mg). These amounts were chosen to mimic their concentration in the test meals with chickpeas [START_REF] Margier | Nutritional Composition and Bioactive Content of Legumes: Characterization of Pulses Frequently Consumed in France and Effect of the Cooking Method[END_REF]. Each test meal was supplemented with 100 ng of vitamin D dissolved in refined olive oil, and with 80 µg of iron in the form of ferrous sulfate monohydrate (American International Chemical, Westborough, MA).

The in vitro digestion experiments were carried out in quadruplicate as previously described, except that all the volumes of solutions were divided by two [START_REF] Malapert | beta-Cyclodextrin Does not Alter the Bioaccessibility and the Uptake by Caco-2 Cells of Olive By-Product Phenolic Compounds[END_REF][START_REF] Reboul | Bioaccessibility of carotenoids and vitamin E from their main dietary sources[END_REF]. The final mixture representing the digesta was centrifuged (2500 g, 10 °C, 1 h) to separate the aqueous phase containing the mixed micelles and the solubilized minerals. Aliquots of digesta and aqueous phases were stored at -80°C under nitrogen atmosphere until analysis.

Mineral analysis

Iron, zinc and magnesium were extracted using a closed-vessel microwave digestion system (Ethos-1, Milestone, Italy) from about 1 g of sample (raw food/ingredients, digesta, or aqueous phases) in a 0.25:1.75 nitric acid/hydrogen peroxide mixture. The closed vessels were placed in the microwave oven and digested at 1200 W for 30 min. After cooling, samples were diluted to 6 mL with ultrapure water. Iron, zinc and magnesium contents were analyzed with an Agilent 5100 VDV ICP-OES spectrometer (Agilent, Les Ulis, France).

Mineral solubility was calculated i) as percentage of the soluble mineral recovered in the aqueous phases relative to the total mineral recovered in the whole digesta, and ii) as percentage of the soluble mineral recovered in the aqueous phases relative to the initial amount added in the test meal at the beginning of digestion.

Vitamin D analysis

Vitamin D was extracted from 100 µL of digesta or 100µL of mixed micelles (aqueous phases) as follows. 400 µL distillated water was added to each sample to reach a final volume of 500µL. Retinyl acetate (internal standard) was added to the samples in 500µL ethanol.

Then, mixtures were extracted twice with two volumes of hexane. The hexane phase obtained after centrifugation (500g, 10min, 4°C) was evaporated to dryness under nitrogen, and the dried residue was dissolved in 200µL acetonitrile/methanol (60/40, v/v). A volume of 100-180µL was used for HPLC analysis. The HPLC system included a Shimadzu separation module (LC-20ADSP HPLC Pumps and SIL-20CHT Autosampler, Shimadzu, Marne-lavallée, France) and an SPD-M20A Shimadzu photodiode array detector (PDA, detection at 265nm for vitamin D, 325nm for retinyl acetate, spectral analysis between 190nm and 600nm). Vitamin D and retinyl acetate (Sigma Aldrich) were separated using a 250 × 4.6 nm, 5µm Zorbax Eclipse XDB-C18 column and a guard column (Agilent). The mobile phase was 60% acetonitrile, 38% methanol and 2% water. Flow rate was 1.5mL/min and the column was kept at a constant temperature (40°C). Vitamin D was identified by spectral analysis and/or on the basis of the retention time and co-injection compared with the pure standards.

Quantification was performed using the Chromeleon software (version 6.8, ThermoFisher Scientific, Villebon sur Yvette, France) by comparing the peak area with the standard reference curves. All HPLC-grade solvents were from Carlo-Erba (Peypin, France).

Vitamin D bioaccessibility was calculated as the percentage of vitamin D recovered in mixed micelles relative to the amount of vitamin recovered in the whole digesta at the end of the digestion. As vitamin D could be variably degraded during digestion, depending on the meal composition, vitamin D bioaccessibility was also calculated as the percentage of vitamin D recovered in mixed micelles relative to the initial amount of vitamin D added to the meal at the beginning of digestion.

Lipid extraction and analysis by thin-layer chromatography coupled with flame ionization detection

Lipolysis products were analyzed in the digesta of test meals made of semolina (control) or chickpeas according to Cavalier et al. [START_REF] Cavalier | Validation of lipolysis product extraction from aqueous/biological samples, separation and quantification by thin-layer chromatography with flame ionization detection analysis using O-cholesteryl ethylene glycol as a new internal standard[END_REF]. Briefly, lipids were extracted from 1 mL of digesta using 5 mL of a chloroform/methanol mixture (2:1, v/v) after acidification of the digesta with 200 µl of 1N HCl, followed by vortex homogenization and centrifugation. The lower organic phase was then recovered and dried by adding magnesium sulfate powder to each sample to remove the residual water that could interfere with lipid separation. Samples were again centrifuged to sediment magnesium sulfate, and the organic phase was removed for chromatographic analysis. The lipid extracts were spotted (1 µL) on SIII chromarods (Iatron laboratories, Tokyo, Japan) using a semi-automated sample spotter.

Elution was carried out with a heptane/ether/formic acid mixture (55:45:1, v/v/v) for approximately 20 min. The chromarods were then dried and solvent evaporated in an oven at 150 ° C for 15 min. The detection was carried out using a Iatroscan MK6 device (Iatron laboratories), and chromatograms were integrated using the Chromstar software. Mass detection data were converted in moles using calibration curves established with various amounts of oleic acid, monoolein, diolein and triolein, used as reference standards. The lipolysis level was expressed as the percentage of free fatty acids (FFA) and monoglycerides present at the digestion end relative to the total amount of acyl chains present in the test meal triglycerides.

Study of the effects of bioactive compounds on pancreatic lipase activity

The effects of the different bioactive compounds on pancreatic lipase activity were also studied. Enzymatic kinetics were recorded using a pHstat (Metrohm titrino 718, Villebon-sur-Yvette, France) and trioctanoin as substrate. The triglyceride substrate hydrolysis by the lipase was monitored by adding 0.1 N sodium hydroxide (NaOH) when the pH decreased from the set threshold (pH 8). It was considered that the amount of NaOH (μmol) added over time to maintain the pH constant was equivalent to the amount of FFA produced by the lipase activity, if the FFA were completely ionized (pKa < pH). The chosen pH threshold corresponded to the value at which the enzyme exhibits its maximum activity (pH = 8). The enzymatic reaction was performed using 250 µL of trioctanoin dispersed by mechanical stirring in 15 mL of a solution containing 0.3 mM Tris, 2 mM CaCl2, 150 mM NaCl and 0.5 mM or 4 mM sodium taurodeoxycholate. Several concentrations of the bioactive compounds were tested. A stock solution of pancreatin at 10 mg/mL was prepared in pure water, and 3 µL were used for each assay, with 5 μl of colipase (1 mg/ml) that is required to measure the full enzyme activity in the presence of bile salts. Purified bioactive compounds were added during the pHstat test. Three attempts were made to study the inhibitory effect of each compound, and different concentrations of each compound were tested. Their concentrations were chosen to represent those present in pulses, according to previous data [START_REF] Margier | Nutritional Composition and Bioactive Content of Legumes: Characterization of Pulses Frequently Consumed in France and Effect of the Cooking Method[END_REF].

Statistical analysis

Results are expressed as the mean ± SEM. Differences between more than two groups of unpaired data were tested using the non-parametric Kruskal-Wallis test. The non-parametric Mann-Whitney U test was used as a post-hoc test when the Kruskal-Wallis test showed significant differences among groups. P values ≤0.05 were considered significant. All statistical analyses were performed using the GraphPad Prism software, version 8.4.3 (GraphPad Software, San Diego, California, U.S.A.).

Results

Mineral solubility is decreased in meals containing pulses

Iron solubility, expressed as the percentage of soluble iron relative the iron recovered in the digesta, was significantly reduced in meals containing chickpeas compared with meals containing beef (-11.5 ± 1.8% in the chickpea meal, -24.3 ± 0.9% in the beef + chickpea meal and -32.2 ± 1.7% in the semolina + chickpea meal, p≤0.05) (Fig. 1A). Similar results were obtained concerning zinc and magnesium solubility (-17.0 ± 0.8% in the chickpea meal, -29.6 ± 4.1% in the beef + chickpea meal and -35.6 ± 2.7% in the semolina + chickpea meal; -13.0 ± 1.3% in the chickpea meal, -20.0 ± 0.9% in the beef + chickpea meal and -25.2 ± 3.2% in the semolina + chickpea meal, respectively, p≤0.05).

Solubility values were lower when solubility was expressed as the percentage of soluble mineral relative to the total mineral initially present in the different test meals, but the decrease observed for the three minerals in the presence of chickpeas remained significant (Fig. 1B).

Mineral solubility is only slightly decreased by isolated bioactive compounds

Analysis of the effects of isolated bioactive compounds on mineral solubility (Fig. 1C) showed that tannins and saponins significantly decreased iron solubility expressed as the percentage of mineral remaining in the digesta (-6.0 ± 0.5% and -9.1 ± 1.0%, respectively, p≤0.05). Zinc solubility also was significantly decreased in the presence of phytates, tannins and saponins (-16.5 ± 3.3%; -25.5 ± 6.8% and -24.4 ± 1.1%, respectively, p≤0.05).

Magnesium solubility was not modified. When solubility was expressed as the percentage relative to the mineral initial quantity in the different test meals, only tannins showed a negative impact on iron solubility (-35.6 ± 0.5%, p≤0.05) (Fig. 1D).

Vitamin D stability and bioaccessibility are strongly influenced by the meal composition

Analysis of vitamin D bioaccessibility in the different test meals (Fig. 2) showed that when bioaccessibility was expressed as the percentage of micellar vitamin D relative to the vitamin D present in the whole digesta, vitamin D transfer to micelles was decreased in the presence of chickpeas (-45.3 ± 3.8% in the beef + chickpea meal, and -44.5 ± 4.3% in the semolina + chickpea meal, p≤0.05). However, when bioaccessibility was expressed as the percentage of micellar vitamin D relative to the vitamin D amount initially present in the meals, the presence of semolina and/or chickpeas increased vitamin D bioaccessibility (+ 539.8 ± 110.3% for the semolina meal, + 519.3 ± 45.1% for the chickpea meal, + 180.9 ± 32.0% for the beef + chickpea meal, and + 522.0 ± 27.6% for the semolina + chickpea meal). This discrepancy was due to major differences in vitamin D stability during the in vitro digestion process (Table 2).

Vitamin D bioaccessibility is decreased by phytates and tannins, and increased by saponins

Finally, analysis of the effects of isolated bioactive compounds on vitamin D bioaccessibility showed that vitamin D bioaccessibility, expressed as the percentage of vitamin D recovered in the digesta, was decreased by phytates and tannins (-28.7 ± 1.4% and -18.2 ± 2.3%, respectively, p≤0.05) and was increased by saponins (+ 8.9 ± 4.5%, p>0.05). This was not modified when bioaccessibility was expressed as the percentage of the vitamin D initial amount in the test meal, possibly due to the minor modifications of vitamin D stability (i.e. same test meal) in the different conditions (Table 2).

Lipolysis is delayed in the presence of chickpeas, partly due to inhibition of the pancreatic lipase activity

Lipolysis rate reached 32.2 ± 3.5% at the end of the in vitro digestion of chickpeas, and 47.7 ± 6.5% at the end of the in vitro digestion of semolina (data not shown).

Pancreatic lipase activity was then inhibited by addition of different bioactive compounds during the lipolysis reaction (Fig. 3). First, the effects of these compounds were measured when the lipase was in the presence of colipase and 0.5 mM bile salts, the optimum concentration for measuring pancreatic lipase activity. Indeed, higher concentrations could have concealed the inhibitory effects of bioactive substances. Then, the inhibition assays were repeated in the presence of 4 mM bile salts, a micellar concentration that allows displacing surfactants from the oil-water interface. This procedure allows detecting non-specific inhibition due to the surfactant effects of the tested molecules. The results showed that tannins were the strongest inhibitors among the tested bioactive compounds. Indeed, the activity of pancreatic lipase present in pancreatin decreased in a dose-dependent manner upon addition of tannins (from 225 µg to 4 mg per assay; Fig. 3). Moreover, the tannin inhibitory effect observed at low bile salt concentration was still present when the bile salt concentration was increased to 4 mM. This suggested a specific interaction with pancreatic lipase. Lipase activity was reduced by 50% after addition of 2 mg of tannins, which corresponded to an activity of 53.3 U/mg versus 106.7 U/mg for the pancreatin control (1 U=1 µmole FFA released per min). Phytate inhibitory effect was less marked, with a residual activity of 68.8% after addition of 15 mg of phytates. Saponin inhibitory effect decreased when the bile salt concentration was increased to 4 mM (Fig. 3). This suggested that saponin inhibitory effects on pancreatic lipase were only due the surface activity of saponins and not to a specific interaction with the lipase. Finally, cellulose had an extremely weak inhibitory effect with a residual activity of 94%.

Discussion

The aim of this study was to assess the effect of the composition of complex test meals on vitamin D and mineral (iron, zinc and magnesium) bioaccessibility/solubility.

We particularly focused on vitamin D and iron because many people might be at high risk of deficiency across the world. In 2011, 29% of non-pregnant women, 38% of pregnant women, and 43% of children had anemia worldwide [START_REF] Stevens | Global, regional, and national trends in haemoglobin concentration and prevalence of total and severe anaemia in children and pregnant and non-pregnant women for 1995-2011: a systematic analysis of population-representative data[END_REF]. Moreover, it has been reported that iron deficiency accounts for 25% and close to 40% of all anemia cases in preschool children and non-pregnant women, respectively [START_REF] Petry | The Proportion of Anemia Associated with Iron Deficiency in Low, Medium, and High Human Development Index Countries: A Systematic Analysis of National Surveys[END_REF]. Even in high income countries, anemia prevalence was about 16% (11% and 22 % in preschool children and non-pregnant women, respectively) [START_REF] Stevens | Global, regional, and national trends in haemoglobin concentration and prevalence of total and severe anaemia in children and pregnant and non-pregnant women for 1995-2011: a systematic analysis of population-representative data[END_REF]. Similarly, vitamin D deficiency and insufficiency remain a public health issue in high and low income countries [START_REF] Holick | The vitamin D deficiency pandemic: Approaches for diagnosis, treatment and prevention[END_REF]. As pulses contain bioactive compounds that can negatively interact with these two families of micronutrients, it is important to investigate whether eating more pulses can exacerbate such insufficiencies.

To this aim, we used an in vitro digestion model to simultaneously assess vitamin D bioaccessibility and mineral solubility. In vitro models are useful tools to investigate nutrient, micronutrient and contaminant bioaccessibility (i.e. their transfer to the aqueous absorbable phase of the bolus) [START_REF] Brodkorb | INFOGEST static in vitro simulation of gastrointestinal food digestion[END_REF]. Our in vitro digestion model was previously fully validated for assessing fat-soluble micronutrient bioaccessibility from complex test meals [START_REF] Goncalves | Fatty acids affect micellar properties and modulate vitamin D uptake and basolateral efflux in Caco-2 cells[END_REF][START_REF] Reboul | Bioaccessibility of carotenoids and vitamin E from their main dietary sources[END_REF]. The advantage of this model is the easy solubilization of test meals in high amount of buffer at the beginning of the digestion.

Although less representative of the physiological conditions compared to the INFOGEST procedure [START_REF] Brodkorb | INFOGEST static in vitro simulation of gastrointestinal food digestion[END_REF], this does not affect the model ability to predict fat-soluble micronutrient bioavailability in a healthy human population [START_REF] Reboul | Bioaccessibility of carotenoids and vitamin E from their main dietary sources[END_REF]. Moreover, this model is easy to set up and was miniaturized in this study by dividing by two all the product quantities compared with the original protocol [START_REF] Goncalves | Fatty acids affect micellar properties and modulate vitamin D uptake and basolateral efflux in Caco-2 cells[END_REF].

Our findings first confirmed that the presence of chickpeas in a meal can alter iron, zinc and magnesium solubility. Solubility is required for the possible mineral intestinal absorption and is a criterion used to assess mineral availability [START_REF] Arpadjan | Bioaccessibility of Cd, Cu, Fe, Mn, Pb, and Zn in hazelnut and walnut kernels investigated by an enzymolysis approach[END_REF][START_REF] Engle-Stone | Meat and ascorbic acid can promote Fe availability from Fe-phytate but not from Fe-tannic acid complexes[END_REF]. The most important effects were observed in test meals containing only semolina and chickpeas.

Overall, this confirms that mineral bioavailability is reduced in plant-based meals [START_REF] Gibson | Is iron and zinc nutrition a concern for vegetarian infants and young children in industrialized countries?[END_REF], and indicates that the presence of pulses, even in a meal containing beef, is deleterious. Among the different bioactive substances present in pulses, tannins (at concentrations mimicking those found in chickpeas) were the most efficient mineral chelators, especially iron. This is in agreement with previous data in humans showing that in the absence of calcium, tannic acid but not phytic acid decreases the fasting non-heme iron bioavailability [START_REF] Jaramillo | Effect of phytic acid, tannic acid and pectin on fasting iron bioavailability both in the presence and absence of calcium[END_REF]. This is also consistent with the observation that tannic acid-iron complexes are less soluble and more stable than phytic acid-iron complexes [START_REF] Engle-Stone | Meat and ascorbic acid can promote Fe availability from Fe-phytate but not from Fe-tannic acid complexes[END_REF].

Our results also showed that when expressed as the percentage of vitamin D recovered in the digesta at the end of the digestion protocol, chickpeas in a meal can limit vitamin D transfer to the micellar phase of the bolus. These results confirm our previous work showing that fatsoluble vitamin bioaccessibility is significantly lower in test meals made of meat and pulses than in test meals made of meat and potatoes [START_REF] Margier | The Presence of Pulses within a Meal can Alter Fat-Soluble Vitamin Bioavailability[END_REF]. These data are supported also by the results of the lipolysis analysis. Indeed, we found that the lipolysis rate was lower in the meals containing chickpeas than in the meals with only semolina. The lower concentration of lipolysis products (FFA, monoglycerides) in the micellar phase obtained during the digestion of the chickpea meal could results in decreased vitamin D incorporation in mixed micelles, and consequently in reduced bioaccessibility. However, results were quite different when vitamin D bioaccessibility was expressed as the percentage of vitamin D incorporated in the test meal. In this case, vitamin D bioaccessibility was significantly lower in the test meal containing beef than in test meals containing plant-based food only. This was due to the important degradation of vitamin D during in vitro digestion of meals containing beef, and/or higher protection against degradation in plant-based meals. Indeed, it is acknowledge that meat is a prooxidant factor during digestion, and can lead to the generation of reactive oxygen species and lipid oxidation products (Van Hecke, [START_REF] Van Hecke | Oxidation During Digestion of Meat: Interactions with the Diet and Helicobacter pylori Gastritis, and Implications on Human Health[END_REF]. Conversely, pulses and cereals contain several antioxidants, such as phenolic compounds [START_REF] Durazzo | Evaluation of Antioxidant Properties in Cereals: Study of Some Traditional Italian Wheats[END_REF][START_REF] Margier | Nutritional Composition and Bioactive Content of Legumes: Characterization of Pulses Frequently Consumed in France and Effect of the Cooking Method[END_REF], that can protect vitamin D. However, vitamin D degradation during in vivo digestion is likely less important than during in vitro digestion where vitamin D is more exposed to UV and oxygen. Additional studies using in vivo approaches are needed to determine which factor (i.e.

increased transfer vs increased degradation in the presence of beef) is preponderant in modulating vitamin D bioaccessibility. In standardized test meals with only potatoes , in which vitamin D degradation was similar in the different conditions, vitamin D bioaccessibility was decreased by addition of purified phytates and tannins (two bioactive compounds normally found in pulses), in accordance with our previous results for vitamin K [START_REF] Margier | The Presence of Pulses within a Meal can Alter Fat-Soluble Vitamin Bioavailability[END_REF]. Conversely, and unlike vitamin K, vitamin D bioaccessibility was increased by saponins that have cyclic triterpenoid or steroid structures. Saponin surfactant properties [START_REF] Ruyssen | The measurement of surface tension and interfacial tension of saponin solutions[END_REF] may contribute to vitamin D solubilization. This discrepancy is probably related to the very different molecular structures of vitamin K (a naphthoquinone) and vitamin D (a seco-steroid). We might hypothesize that saponins compete with vitamin K for incorporation in the mixed micelles, while they may associate with vitamin D and thus increase its solubility in the aqueous phase. This is in line with a previous study showing vitamin D self-association and thus solubilization in the aqueous phase at high concentrations [START_REF] Desmarchelier | Molecular interactions governing the incorporation of cholecalciferol and retinyl-palmitate in mixed taurocholate-lipid micelles[END_REF].

Interestingly, the experiments with pancreatic lipase showed that when used at concentrations similar to those found in pulses, purified bioactive compounds can also display an inhibitory effect on lipase activity. This observation might explain the decrease in vitamin D bioaccessibility. Analysis of the effects of bioactive substances on the lipase activity in the presence of colipase and 0.5 mM bile salts and then 4 mM bile salts (a more physiological concentration but that could suppress non-specific inhibition by surfactant molecule) showed that tannins had the most important inhibitory effect. This could be due to the formation of complexes with pancreatic lipase, as demonstrated for other enzymes [START_REF] Rocchetti | Effect of dietary polyphenols on the in vitro starch digestibility of pigmented maize varieties under cooking conditions[END_REF]. It was previously reported that saponins bind specifically to pancreatic lipase and inhibit its activity [START_REF] Zhao | Antiobese and hypolipidemic effects of platycodin saponins in diet-induced obese rats: evidences for lipase inhibition and calorie intake restriction[END_REF]. Saponins displayed some inhibitory effect on pancreatic lipase activity in the experiments with low bile salt concentration, but this effect was decreased in the presence of 4 mM bile salts, suggesting that this inhibition is not the result of a specific interaction with the lipase. Phytates displayed a moderate inhibitory action compared with tannins. Previous studies showed that phytates interfere with the activity of the pancreatic lipase (REF), but the underlying mechanisms remain unclear. Finally, cellulose, used as a fiber model, had a minor inhibitory effect. We now need to evaluate the effects of the combination of these compounds because they may have additive or antagonistic effects [START_REF] Margier | The Presence of Pulses within a Meal can Alter Fat-Soluble Vitamin Bioavailability[END_REF].

Overall, we demonstrated that our in vitro digestion model is suitable for assessing the bioaccessibility/solubility of minerals and fat-soluble micronutrients. We found that complex meals made only of plant-based products may lead to reduced micronutrient bioaccessibility.

Therefore, this in vitro digestion model could be particularly useful for testing different recipes in order to improve micronutrient bioavailability by using other food products [START_REF] Platel | Bioavailability of Micronutrients from Plant Foods: An Update[END_REF]. However, additional studies are needed to validate these findings i) Lipase activity was measured in the presence of 0.5 mM or of 4 mM bile salts, and the residual activity after addition of the bioactive compounds was expressed as percentage of the activity measured with pancreatin alone. Various amounts of bioactive compounds were tested: tannins (225 µg, 400 µg, 2 mg, 4 mg), phytates (1.5 mg, 15 mg), saponins (1 mg, 2 mg), and cellulose (100 mg). Data are representative of at least 3 experiments.

Tables Table 1:

Figure 1 :

 1 Figure 1: Iron, zinc and magnesium solubility

Figure 2 :

 2 Figure 2: Vitamin D bioaccessibility

Figure 3 :

 3 Figure 3: Effects of bioactive compounds on pancreatic lipase activity.

  ± 0.2 a 518.0 ± 25.9 b 16.0 ± 1.6 b 7.45 ± 1.9 b Canned chickpeas 6.4 ± 2.1 b 526.4 ± 22.3 b 116.8 ± 0.2 c 13.7 ± 1.5 c Different superscript letters in the same column indicate a significant difference between groups (p≤0.05).

(chickpeas). We then investigated the specific effect of fibers, phytates, tannins and saponins on both vitamin D and mineral solubility. Our results indicate that this in vitro digestion model is suitable for assessing the bioaccessibility/solubility of fat-soluble micronutrients and minerals in complex test meals.

using an enterocyte culture model to assess the effect of the different test meals on absorption, and ii) using in vivo models to fully evaluate micronutrient bioavailability.

Once confirmed, these data will help to better integrate pulses in a balanced diet, for example by associating them with micronutrient-rich foods or with foods that promote micronutrient bioavailability. These findings mays also be used by manufacturers to optimize the nutritional quality of recipes containing pulses.

Acknowledgments:

This work was funded by a national Priority Action of the Human Nutrition department of the French National Institute for Agriculture, Food and Environment (INRAE).

T. Antoine received a doctoral fellowship from INRAE, CTCPA and C2VN.

Authors are grateful to Marielle Margier (C2VN, Marseille, France) for helpful technical help.

This work was partly presented as a poster at the 13 th European Nutrition Conference, October 15-18 2019, Dublin (Antoine et al., 2020).

Conflict of interest:

Other authors declare no conflicts of interest or financial interest.


Perceval Wajsbürt

Arnaud Sarfati

Xavier Tannier
email: xavier.tannier@sorbonne-universite.fr

Medical concept normalization in French using multilingual terminologies and contextual embeddings

Keywords: Natural Language Processing, Information Extraction, Medical concept normalization, Multilingual representation

Introduction: Concept normalization is the task of linking terms from textual medical documents to their concept in terminologies such as the UMLS®. Traditional approaches to this problem depend heavily on the coverage of available resources, which poses a problem for languages other than English.

Objective: We present a system for concept normalization in French. We consider textual mentions already extracted and labeled by a named entity recognition system, and we classify these mentions with a UMLS concept unique identifier. We take advantage of the multilingual nature of available terminologies and embedding models to improve concept normalization in French without translation nor direct supervision. Materials and Methods: We consider the task as a highly-multiclass classification problem. The terms are encoded with contextualized embeddings and classified via cosine similarity and softmax. A first step uses a subset of the terminology to finetune the embeddings and train the model. A second step adds the entire target terminology, and the model is trained further with hard negative selection and softmax sampling.

Results: On two corpora from the Quaero FrenchMed benchmark, we show that our approach can lead to good results even with no labeled data at all; and that it outperforms existing supervised methods with labeled data.

Discussion: Training the system with both French and English terms improves by a large margin the performance of the system on a French benchmark, regardless of the way the embeddings were pretrained (French, English, multilingual). Our distantly supervised method can be applied to any kind of documents or medical domain, as it does not require any concept-labeled documents.

Conclusion: These experiments pave the way for simpler and more effective multilingual approaches to processing medical texts in languages other than English.

INTRODUCTION

Medical concept recognition and normalization is a classical and essential task of natural language processing (NLP) for biomedical applications [START_REF] Raghavan | How essential are unstructured clinical narratives and information fusion to clinical trial recruitment?[END_REF]. The normalization of terms or phrases present in unstructured texts consists of linking these phrases to the concepts of a knowledge base or a biomedical terminology. This task, also called "entity linking," can be seen as a massively multiclass classification task, since reference terminologies can contain several million concepts.

This number of possible classes makes the problem difficult from many points of view. In particular, it makes a fully-supervised learning approach using labeled textual documents impossible since it is impossible to annotate a corpus containing all of the concepts.

For this reason, it is necessary to use the terminology itself, and the lists of terms associated with the concepts, to guide the system [START_REF] Savova | Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications[END_REF][START_REF] Aronson | An overview of MetaMap: historical perspective and recent advances[END_REF][START_REF] Cabot | Retrieving Clinical and Omic Data from Electronic Health Records[END_REF]. This then raises the problem of the availability of this terminological resource in the target language. In the UMLS metathesaurus [START_REF] Bodenreider | The Unified Medical Language System (UMLS): integrating biomedical terminology[END_REF], often used as the reference terminology, less than 4% of the concepts are associated with at least one term in French, yet one of the most represented languages after English. In the corpus Quaero FrenchMed [START_REF] Névéol | The QUAERO French Medical Corpus: A Ressource for Medical Entity Recognition and Normalization[END_REF], containing French biomedical texts with mentions labeled with UMLS-2014AB concepts, 27% of the concepts have no synonym in French in this version of the UMLS. Efforts have been made to improve this coverage by manual or automatic translation, or by mapping local terminologies, leading to more complete resources out of the official UMLS [START_REF] Deléger | A twofold strategy for translating a medical terminology into French[END_REF][START_REF] Zweigenbaum | UMLF: a unified medical lexicon for French[END_REF][START_REF] Marko | Towards a multilingual medical lexicon[END_REF][START_REF] Grosjean | Health multi-terminology portal: a semantic added-value for patient safety[END_REF][START_REF] Névéol | Language Resources for French in the Biomedical Domain[END_REF]. However, the gap is still significant, and this represents a real pitfall for the NLP systems in French, and more generally, in all languages other than English [START_REF] Névéol | Clinical Natural Language Processing in languages other than English: opportunities and challenges[END_REF].

Recent years have seen the advent in NLP of the use of models pretrained on large amounts of data, on tasks such as language modeling, for an application by transfer learning on different tasks with fewer data. In particular, these models make it possible to represent words or phrases in the vector space (embeddings) by preserving the similarities and the semantic relationships between these words. They have improved state of the art in many tasks, including in the biomedical domain [START_REF] Habibi | Deep learning with word embeddings improves biomedical named entity recognition[END_REF][START_REF] Pyysalo | Distributional Semantics Resources for Biomedical Text Processing[END_REF]. Some of these models have been pretrained on multilingual data [START_REF] Conneau | Unsupervised Cross-lingual Representation Learning at Scale[END_REF][START_REF] Feng | Language-agnostic BERT Sentence Embedding[END_REF][START_REF] Devlin | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[END_REF], without any alignment or translation mechanism, and showed performance gains for a wide range of crosslingual transfer tasks.

In this work, we present a contribution to the task of concept normalization in French. We focus on normalizing mentions that are already extracted and labeled (gold standard spans of text). We classify these mentions with a UMLS concept unique identifier. In the end-to-end task of extracting concepts from free, unlabeled text, obtaining these labeled mentions automatically could be achieved with a named entity recognition system trained on French medical data [START_REF] Lerner | Terminologies augmented recurrent neural network model for clinical named entity recognition[END_REF][START_REF] French | Participation de l'équipe du LIMICS à DEFT 2020[END_REF][START_REF] Tourille | Evaluation of a Sequence Tagging Tool for Biomedical Texts[END_REF].

We improve medical concept normalization from gold-standard named entities in French; we overcome the difficulty of the limited terminological resources in this language by taking advantage of English terminologies and the latest multilingual embedding models. We propose a hybrid approach using the multilingual nature of both external knowledge from the UMLS and data-driven, pretrained embedding models. We do not use any manually-enhanced and automatically-translated terminology and do not rely on any task-specific rule. We take advantage of the latest progress in neural network optimization, which allows us to cast the normalization task as a standard, highlymulticlass, classification problem. In contrast, other recent deep learning-based approaches work around the problem by converting the task into a nearest neighbor problem and must insert all the available synonyms in the search space. On two corpora from the Quaero FrenchMed benchmark, we show that our approach can lead to good results even with no labeled data at all (distant learning from the terminology only). It outperforms existing methods, including those using machine translation, with labeled data (distant + supervised learning). We also designed a set of experiments showing the contribution of multilingual models and resources compared to monolingual. 1

BACKGROUND

The clinical normalization problem has been extensively studied in clinical informatics literature.

Methods have shifted from rule-based and term indexer systems [START_REF] Afzal | Biomedical Concept Recognition in French Text Using Automatic Translation of English Terms[END_REF][START_REF] Souza | Sieve-based entity linking for the biomedical domain[END_REF][START_REF] Leaman | TaggerOne: Joint named entity recognition and normalization with semi-Markov Models[END_REF][START_REF] Cabot | Retrieving Clinical and Omic Data from Electronic Health Records[END_REF] to machine learning systems [START_REF] Leaman | DNorm: Disease name normalization with pairwise learning to rank[END_REF][START_REF] Li | CNN-based ranking for biomedical entity normalization[END_REF][START_REF] Roller | Cross-lingual Candidate Search for Biomedical Concept Normalization[END_REF][START_REF] Fakhraei | NSEEN: Neural Semantic Embedding for Entity Normalization[END_REF][START_REF] Sung | Biomedical Entity Representations with Synonym Marginalization[END_REF][START_REF] Phan | Robust representation learning of biomedical names[END_REF].

The normalization problem is also known in the general domain as entity linking [START_REF] Shen | Entity linking with a knowledge base: Issues, techniques, and solutions[END_REF][START_REF] Sevgili | Neural Entity Linking: A Survey of Models based on Deep Learning[END_REF], but differs by the fact that the general domain annotated corpora can leverage larger annotated corpora such as Wikipedia. These make it possible to perform a single supervised training and rely on entity frequencies. However, medical lexicons do not provide context nor accurate medical concept frequencies.

Deep synonym similarity approaches [START_REF] Leaman | DNorm: Disease name normalization with pairwise learning to rank[END_REF][START_REF] Li | CNN-based ranking for biomedical entity normalization[END_REF][START_REF] Fakhraei | NSEEN: Neural Semantic Embedding for Entity Normalization[END_REF][START_REF] Phan | Robust representation learning of biomedical names[END_REF][START_REF] Sung | Biomedical Entity Representations with Synonym Marginalization[END_REF][START_REF] Mondal | Medical Entity Linking using Triplet Network[END_REF] reformulate the normalization task as a nearest neighbor or ranking problem. However, all synonym representations must be stored or recomputed at inference time to perform lookup. Another approach is to subset the list of concepts using an efficient search engine and apply a neural similarity model to achieve a more refined classification amongst the candidates [START_REF] Zongcheng | Bert-based ranking for biomedical entity normalization[END_REF]. However, the search engine in such methods can fail for entities that share no lexical pattern with their concept synonyms, especially when normalizing in a language other than English.

We chose to consider the task as a standard classification task amongst concepts, meaning that we only encode concepts of the target terminology, rather than their synonyms, into fixed-length representations that can be stored and even indexed to accelerate lookup at inference time. This way of framing the problem as a standard classification is similar to [START_REF] Wright | NormCo: Deep Disease Normalization for Biomedical Knowledge Base Construction[END_REF]. However, in our case, we deal with a considerable number of classes (nearly a million in our experiments). We tackle this difficulty by making use of developments in neural network optimization over the last years, such as Adam [START_REF] Kingma | Adam: A method for stochastic optimization[END_REF], BatchNorm [START_REF] Ioffe | Batch normalization: Accelerating deep network training by reducing internal covariate shift[END_REF], or the recent advances on the face identification problem [START_REF] Deng | ArcFace: Additive Angular Margin Loss for Deep Face Recognition[END_REF][START_REF] Wang | NormFace: L2 hypersphere embedding for face verification[END_REF][START_REF] Wang | CosFace: Large Margin Cosine Loss for Deep Face Recognition[END_REF]. Indeed, face identification is similar to medical normalization in terms of the number of target classes and the low number of examples per class.

Medical named entity representation learning has closely followed the general domain NLP research by using pretrained models such as Word2Vec [START_REF] Mikolov | Efficient estimation of word representations in vector space[END_REF], Elmo [START_REF] Peters | Deep contextualized word representations[END_REF], BERT, and its multilingual variant [START_REF] Devlin | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding[END_REF]. The latter processes the input text, regardless of its language, to compute contextualized token representations.

The normalization of medical entities in languages other than English has so far relied mainly on the translation of English synonyms into the target language [START_REF] Afzal | Biomedical Concept Recognition in French Text Using Automatic Translation of English Terms[END_REF][START_REF] Cabot | Retrieving Clinical and Omic Data from Electronic Health Records[END_REF], or conversely, the translation of entities into English [START_REF] Chiaramello | Use of "off-theshelf" information extraction algorithms in clinical informatics: A feasibility study of MetaMap annotation of Italian medical notes[END_REF][START_REF] Roller | Cross-lingual Candidate Search for Biomedical Concept Normalization[END_REF]. These systems use processing pipelines that mix term search with software like MetaMap [START_REF] Aronson | Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program[END_REF] or Apache Solr, and web-service or local translation [START_REF] Jiang | WI-ENRE in CLEF eHealth Evaluation Lab 2015: Clinical named entity recognition based on CRF[END_REF]. We chose to design and evaluate an auto-sufficient deep neural network classifier with few to no preprocessing of the input named entities.

MATERIAL

UMLS

The Unified Medical Language System® (UMLS®) is a metathesaurus that unifies concepts from several dozen terminologies in the biomedical domain [START_REF] Bodenreider | The Unified Medical Language System (UMLS): integrating biomedical terminology[END_REF]. Each concept in the UMLS is assigned a Concept Unique Identifier (CUI), a set of terms (or synonyms), possibly in multiple languages, and a semantic type. UMLS semantic types are grouped in 15 semantic groups and each concept is associated with one semantic group, with very few exceptions [START_REF] Mccray | Aggregating UMLS semantic types for reducing conceptual complexity[END_REF]. For example, "Eicosapentanoic acid" (concept C0000545) is in the chemical (CHEM) group, while "Accountant" (concept C0000937) is in the living beings (LIVB) group. Table 1 shows statistics on the number of concepts and synonyms in English and French, for the versions 2014AB and 2019AB, both used in this work.

French is the 2nd (resp. 5th) most represented language in the 2014 (resp. 2019) version in the metathesaurus, but only 3.5% (resp. 3.6%) of the concepts have terms in French. In this article, we will call "English mirror" the synonyms in English for the concepts that also have synonyms in

French. We call "English 5 sources" (EN5) the UMLS concepts that have an English synonym and are either in the five CHV, SNOMEDCT_US, MTH, NCI, or MSH terminologies. We chose these terminologies because they cover 96% of the labels in the Quaero training corpus, without exceeding a million labels.

Quaero FrenchMed corpora

The Quaero FrenchMed corpus [START_REF] Névéol | Language Resources for French in the Biomedical Domain[END_REF] consists of two sets of textual documents in French, annotated with concept CUIs from the 2014AB version of the UMLS:

• Titles of research articles indexed in the MEDLINE database

• Information on marketed drugs from the European Medicines Agency (EMEA)

Unlike other normalization corpora such as NCBI [START_REF] Doğan | NCBI disease corpus: A resource for disease name recognition and concept normalization[END_REF] or BC5CDR [START_REF] Li | BioCreative V CDR task corpus: a resource for chemical disease relation extraction[END_REF], the annotated concepts were not limited to vocabularies such as SNOMED, MeSH, or OMIM. However, they were limited to 10 of the 15 UMLS semantic groups.

We Table 2 presents general corpus statistics including the number of annotated mentions (i.e., text spans linked to UMLS concepts within the documents), the number of unique mentions, the number of unique concept CUIs, as well as the rate of mentions in each corpus that are linked to a concept with at least one synonym in French in the terminology. Note that very few mentions are annotated with more than one CUI in the corpora.

To ensure a fair comparison with other systems published on this benchmark, we use the 2014AB version of the UMLS, unless specified otherwise.

METHODS

Problem definition

We cast the normalization problem as a classification task. C = {c} is the set of all concepts c (i.e., classes to predict) identified by their CUI. Each concept is associated with one semantic group [START_REF] Mccray | Aggregating UMLS semantic types for reducing conceptual complexity[END_REF]. We denote the set of all concepts in a semantic group as . A mention m is a phrase in a textual document referring to a concept. In this work, we consider these mentions to be already available and labeled with a semantic group . The set of synonyms that share a same concept is called a synset.

For example, the concept C0678222 contains the synonyms "breast cancer", "breast carcinoma", "carcinoma of the breast", is associated with the semantic type "Neoplastic Process" and is therefore in the semantic group "DISO" (Disorders). Given a french term "cancer du sein" extracted from a document and pre-labelled with the "DISO" semantic group, our goal will be to correctly map it to the C0678222 concept.

Given a dataset, , , . . . , our goal is to build a CUI classifier, i.e to learn a probability distribution to predict the concept of each mention ∈ : * | ; ;

where represents the parameters of the encoder (detailed below), which goal is to map a mention to a dense vector space, and represents the embeddings of the concepts in this space, that have the same semantic group as the mention. Our model is a classification model built on top of a sequence to sequence encoder, and we chose [START_REF] Raghavan | How essential are unstructured clinical narratives and information fusion to clinical trial recruitment?[END_REF] pretrained Transformer [START_REF] Vaswani | Attention is all you need[END_REF] models in our experiments. We call this model MLNorm (for multilingual normalization). The model is described in this section and illustrated in Figure 1.

Model

The mentions are first tokenized into wordpieces [START_REF] Wu | Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation[END_REF] and fed into a pretrained BERT encoder in order to build contextualized representations for each token.

BERT(m)

These contextualized token representations are then averaged across each mention as , without the first [CLS] and last [SEP] special tokens 2

1 ! -2 $ ∈ ,%&
We then perform a projection into a lower dimension embedding to reduce the model size, apply the rectified linear activation function and normalize the result with batch normalization. This leads to a mention embedding ℎ . Finally, we classify each mention by computing the cosine similarity between its representation and the embedding of the concepts in the semantic group of the mention. Following [START_REF] Wang | CosFace: Large Margin Cosine Loss for Deep Face Recognition[END_REF] we multiply the similarity by a hyperparameter 6. We obtain concept probabilities by applying the softmax function on these scores.

| ; ; -7 ⋅ 87 9 : ; ,< = ∑ ?∈@ A -7 ⋅ 87 9 : ;, < B where C6DE-ℎ ,

: ; ||: ; || ⋅ < = ||< = ||
is the embedding of the gold concept 2 These special tokens are mandatory in a BERT model but do not carry information about the mention.

(

) (3) 2
(5)

? is the embedding of the a concept in the semantic group of of c and {4, 5, 0, 3, YZ[\ }

Training

4.3.1

Step 1: Initial full model training.

The training occurs in two steps, illustrated in Figure 2. During step 1, we learn to represent medical entities by finetuning the pretrained transformer model and train a subset of the concepts embeddings to classify synonyms from a UMLS subset.

Since this part is time-consuming and computationally expensive, we subset the UMLS and only keep the concepts with at least one French synonym (UMLS FR + EN mirror) to focus our training on the multilingual capacity of the model. This leads to a system called S1, limited to predicting only concepts having French synonyms.

4.3.2

Step 2: Large-scale local concept embedding learning.

In the second phase (step 2), we freeze the transformer and projection parameters and train the representation of all the concepts (not only those having French synonyms) with a local-only learning approach. The intuition behind this is that enough synonyms were seen during step 1 so that the Transformer has an adequate representation capacity for medical mentions in general: we now just need to add the missing concepts to the model. Moreover, for each mention, most of the concepts have a near-zero probability and are not updated during the optimization (Figure 3).

For each newly-added concept, we initialize its embedding as the normalized sum of its synset's representations:

9^ $ ∈7_ 79` ℎ
and lookup for each synonym its a highest scoring concepts as predicted by S1, that we call top candidates.

Cb_ EdDd -6 Cb ? | , ,

We train the concept embeddings as in step 1, but we consider only the batch true concepts and top candidates to compute the loss. This relates to softmax sampling methods [START_REF] Jean | On using very large target vocabulary for neural machine translation[END_REF] with synonym (6) [START_REF] Deléger | A twofold strategy for translating a medical terminology into French[END_REF] dependent hard negatives [START_REF] Schroff | FaceNet: A unified embedding for face recognition and clustering[END_REF]. The encoder being frozen, the synonyms' embeddings stay the same during S2, and we can efficiently compute the indices of these top candidates before starting the gradient descent of step 2. Using this method, we only have to compute gradients for a relevant subset of the concept embeddings, thus enabling a faster and more memory-efficient training. We used two different learning rates, ! YZ[\ for the pretrained transformer, ! `^7? for the concept embeddings and projection layer. During the training, we vary the learning rates using two schedules. Following [START_REF] Sun | How to Fine-Tune BERT for Text Classification?[END_REF], we used a slanted triangular learning rate ! YZ[\ for BERT with a warmup phase of 10% of the total number of training steps. We keep the learning rate ! `^7? constant during the warm-up phase and linearly decay it for the rest of the training.

Prediction

At inference time, a mention is tokenized and passed into the encoder and the classifier. We apply a threshold to remove all the predictions that have a low probability.

Experimental setups

We performed two main sets of experiments with our model MLNorm, that we call "distantly supervised" and "fully-supervised."

In the first setup (systems S1 and S2 in Figure 2), we used only distant supervision from the UMLS, and no direct concept supervision from the available, labeled datasets from Medline and EMEA.

These systems then do not suffer any potential bias from the corpus specificities and do not benefit from the redundancy of mentions within the labeled datasets. In the second setup (S1sup and S2sup), we add mentions and labels from the Medline and EMEA training sets, thus enabling comparison with state-of-the-art fully-supervised approaches using these data.

Step 2 experiments use the English concepts from the five sources ("EN5") described in Section 3.1.

Comparisons and ablation studies

Baselines

We compare MLNorm to the following baselines:

-the top ranked systems of respectively CLEF 2015 [START_REF] Afzal | Biomedical Concept Recognition in French Text Using Automatic Translation of English Terms[END_REF] and CLEF 2016 [START_REF] Cabot | Retrieving Clinical and Omic Data from Electronic Health Records[END_REF], on the same exact task of normalization from gold-standard mentions. The CLEF 2015 winning team [START_REF] Afzal | Biomedical Concept Recognition in French Text Using Automatic Translation of English Terms[END_REF] first augments the French UMLS by translating a subset of the English UMLS concepts encountered in Medline abstracts, using Google Translate. This terminology is then queried by a rule-based text indexer. The CLEF 2016 winning team [START_REF] Cabot | Retrieving Clinical and Omic Data from Electronic Health Records[END_REF] relies on their ECMT indexer which performs bag of words concept matching at the sentence level and integrates up to 13 terminologies partially or totally translated into French.

-the best-performing system, to the best of our knowledge (Roller et al. [START_REF] Roller | Cross-lingual Candidate Search for Biomedical Concept Normalization[END_REF]). They first train a local LSTM-based French to English translator on synonym pairs from the UMLS and other general domain sources. The French and English terminologies are then indexed and searched using Apache Solr through exact and fuzzy matching rules.

-as the system from Roller et al. [START_REF] Roller | Cross-lingual Candidate Search for Biomedical Concept Normalization[END_REF], based on machine translation + English-only normalization, is quite different from our own system, we also experimented with a machine translation approach relying on our own model. This allows a fair comparison between our multilingual space learning approach and a translation-based approach. For this purpose, we translated all UMLS French terms with a state-of-the-art pretrained (opus-mt-fr-en)

translation system [START_REF] Tiedemann | OPUS-MT Building open translation services for the World[END_REF] built with MarianMT [START_REF] Junczys-Dowmunt | Marian: Fast Neural Machine Translation in C++[END_REF] and trained on the OPUS bitext repository corpus [START_REF] Tiedemann | Parallel data, tools and interfaces in OPUS[END_REF]. We then trained our model with all original-English and translated-English terms. We called this strong baseline BERT-MT (using the English BERT) and mBERT-MT (using the multilingual BERT).

We also performed a range of ablation studies and additional experiments on the distantly supervised setup, in order to estimate the impact of our different choices.

Impact of the data language

-FR/EN: the same system without step 2 (i.e. S1 only), with the French synonyms and their English mirror -FR-only: S1 with only the French terms -EN-only: S1 with only the English mirror terms Impact of the pretrained embeddings (our system vs. camemBERT and BERT): we compare our system with the same system using BERT embeddings trained on French data only (CamemBERT [START_REF] Martin | CamemBERT: a Tasty French Language Model[END_REF], model camembert-base-uncased) or using the English-only BERT (model bert-base-uncased), instead of the multilingual BERT, in order to evaluate the contribution of the multilingual embeddings.

Impact of more French terms (our system vs. UMLS2019): we present an experiment using the 2019AB version of the UMLS, containing 154k concepts with French synonyms instead of 89k in the 2014AB version. With this system (UMLS2019), we aim at showing the impact of adding new concepts to the terminology used for distant supervision.

Impact of the two-step architecture (our system vs. 1-step): finally, we trained S1 with all the synonyms (French and English from EN5), and did not perform any step 2 with frozen embeddings. This is a much more time-and memory-consuming experiment that will allow us to estimate the quality of the model pretraining and the possible trade-off between cost and quality.

Hyperparameter selection

For all these experiments, we trained on the training sets and evaluated on the test sets from Quaero FrenchMed datasets (Medline 2015 and 2016, EMEA 2015 and 2016). We chose the hyperparameters by selecting the best-performing values on the training set of Quaero in the distant supervision setting. We run our models on a 20 Go Tesla P40 GPU, except the 1-step experiment which required a 30Go Tesla V100 GPU.

RESULTS

As a result from the hyperparameter search described above, the token embeddings space of size 768 is projected into a space of size 350, the cosine similarity scaling parameter s is 20, both dropout rates for the transformer and the projection layer are set to 0.2. We set the batch size to 128, the maximum synonym wordpiece count to 100, and the maximum learning rates to ! YZ[\ 2-&e and ! 8 9f` ! fg8h 8-&j . We used Adam with parameters k 0.9 and k 0.999.

During step 2, we preselect the a 100 highest scoring concepts for each synonym. We perform the step 1 training for 15 epochs and the step 2 for 5 epochs for all models except S1FR and S1EN since they contain fewer synonyms for the same number of concepts and had to be trained longer for 30 and 20 epochs respectively. All S1 system predictions were filtered with a same threshold of 0.5 and S2 with a threshold of 0.1.

We finetuned the multilingual pretrained BERT model (bert-base-multilingual-uncased) in all our experiments, unless specified otherwise.

We report our main results on the test datasets from the challenges Quaero FrenchMed 2015 and 2016 (Table 3), as well as the results of our additional experiments described in previous Section (Table 4), using the traditional metrics precision, recall and F1-measure (harmonic mean of precision and recall). We also give some examples of the distantly supervised system's output (MLNorm S2), that highlight its multilingual capacities (Table 6).

DISCUSSION

Our distantly supervised system MLNorm S2 obtains very good results without concept-labeled training data (Table 3). It even reaches the same results as the best fully-supervised system published so far [START_REF] Roller | Cross-lingual Candidate Search for Biomedical Concept Normalization[END_REF] on the corpus MEDLINE 2015 (F1=0.737 vs. 0.736) and outperforms all participants of the 2016 edition. Note that CLEF campaigns provide scores on both end-to-end task (named entity + normalization) and normalization-only task; similarly to Roller et al. [START_REF] Roller | Cross-lingual Candidate Search for Biomedical Concept Normalization[END_REF], we compare to the latter. The much higher term redundancy can explain the better score of supervised systems on EMEA corpus (e.g., F1=0.835 and 0.734 on 2015 and 2016 for [START_REF] Roller | Cross-lingual Candidate Search for Biomedical Concept Normalization[END_REF] vs. resp. 0.765 and 0.727 for our system S2) between training and test set (see Table 2), which gives a free boost to supervised systems but is probably not very representative of reality.

We can also see that the system using only French synonyms (FR-only) performs much poorlier, with almost 20 points less in recall than S2, which we can attribute to the missing concepts in the French UMLS.

Our experiments with translated French terms (Table 4) show that even a good machine translation model can lower the accuracy of the final model. We experimented with both English BERT and multilingual BERT to account for the impact of the transformer pre-training language. We could argue that the off-the-shelf translation model could be improved by fine-tuning on UMLS synonyms like [START_REF] Roller | Cross-lingual Candidate Search for Biomedical Concept Normalization[END_REF]. However, we think that those results hint at the fact that translation and indexer pipeline may suffer from error cascade: being trained in an end-to-end fashion, our system does not suffer from this behavior. Table 6 shows that the ambiguity of some terms ("cave" can mean both "cellar" and "cava" in English) is lost during translation.

6.1

Impact of the supervision

Table 3 shows that our supervised system S2sup obtains a F1 gain of 5 and 9 points on resp.

MEDLINE and EMEA corpora, with an improvement of both precision and recall. It outperforms other systems by a large margin on MEDLINE. It also outperforms [START_REF] Roller | Cross-lingual Candidate Search for Biomedical Concept Normalization[END_REF] on EMEA 2015 and 2016, but not [START_REF] Afzal | Biomedical Concept Recognition in French Text Using Automatic Translation of English Terms[END_REF] that obtained a perfect precision on EMEA 2015, at the cost of many handcrafted rules and extra labeled data.

Impact of the data language

In Table 5, we compare the same model trained with either only the French synonyms of the UMLS 2014AB (FR-only), only the English mirror (synonyms of the same concepts, EN-only), or with both of these two sets of synonyms (FR/EN). FR/EN achieves a 8 points improvement over FRonly, despite having the same concepts coverage and the same pretrained embeddings. This indicates that a larger training set, even in a different language can help improve the system's performance by a significant margin. This improvement could be attributed to the lexical similarities between French and English languages. For example in Table 6, the only training French synonym of "vaccination par le b.c.g" is "immunisation contre la tuberculose" and shares no common word. The system can therefore benefit from the addition of similar terms, such as "bcg vaccination" even though they are in English.

Impact of the pretrained embeddings

Our experiments with French-only embeddings CamemBERT and English-only embeddings BERT, reported in Table 5, show that our hypothesis that multilingual embeddings improve the system's performance is not verified, with almost no difference between these three embeddings. French wordpieces and embeddings can handle medical terms in English, and vice versa. Even if this can be again explained in part by the proximity of the two languages concerned, the low results of ENonly, yet benefiting of much larger training data, suggest that it is not that obvious; besides, other papers in the literature suggest that multilingual embeddings are helpful even for such pairs of languages [START_REF] Pires | How multilingual is multilingual BERT?[END_REF][START_REF] Wu | Are All Languages Created Equal in Multilingual BERT?[END_REF]. This observation may also be due to the fact that medical synonym normalization data (short word sequences) is quite different from BERT pretraining data (full sentences), so it is harder for the model to re-use its multilingual knowledge. This aspect deserves more experiments, notably on other, non-European languages. Note that biomedical-specific embeddings such as Clinical BERT [START_REF] Alsentzer | Publicly Available Clinical BERT Embeddings[END_REF] are not yet available in French, which is why we did not consider them. Moreover, as illustrated in Table 6, we can see that the model correctly predicts concepts, even when no common wordpieces exist between the test term and the training synonyms of the target concept. Therefore, the proximity between French and English cannot be the only explanation to the model performance. To correctly classify the mention "cinquième métacarpien"

(fifth metacarpal bone) to its concept, without having the numeral "cinquième" in any of the training synonyms, the model must have learned to generalize from other concepts that contained both French "cinquième" and English "fifth" in their training synonyms.

We can also note that despite addressing out-of-vocabulary errors with wordpiece vocabularies, such errors still exist. For example in Table 6, "robusta" (single wordpiece "##robusta") and "rubusta" (two wordpieces, "##rubus" and "ta") are tokenized differently despite having almost identical characters.

Impact of the two-step training

Our experiment with one-step training procedure showed no improvement over the two-step training (Table 5, "1-step"), and took approximately 15 hours instead of 7 hours (5 hours for S1 and 2 hours for S2 with one million synonyms). Our two-step method can therefore effectively reduce training time without reducing accuracy by choosing an appropriate partition of the training data.

Our results even show a slight loss in performance for the one-step model. This could be explained by the regularization that occurs in the two-step training when we freeze the encoder during S2.

Indeed, since most of the data seen during S2 is English, unfreezing the encoder may encourage the model to forget its inner translation capabilities.

UMLS 2019

Finally, our experiment with UMLS 2019AB (UMLS2019, Table 5) leads to results a little below our system, despite the much higher number of concepts having French synonyms. The system has more French terms to train on, but the coverage in Quaero corpora is not far better.

CONCLUSION AND PERSPECTIVES

We built a model using multilingual terminologies and embeddings to normalize medical concepts for a language having much lower resources than English. We obtained very good results, even in a non-fully-supervised setting, which guarantees robustness. Another attractive property of this approach is that there is no need to retrain the model from scratch when new classes (concepts) are created, which is a common problem of traditional classification methods for dynamic sets of classes.

Future work will extend these experiments to other languages and other tasks such as named entity recognition.

  used two different versions of these corpora in our experiments. The first version, that we call EMEA 2015 and Medline 2015, was used for the CLEF eHealth evaluation lab in 2015, a challenge for named entity recognition and concept normalization. The organizers proposed a training set and a test set for this task. In 2016, a new challenge was organized; the 2015 test set was released as a development set, and a new test set was annotated, leading to a larger corpus containing the previous one.

Figure 1 .

 1 Figure 1. Model overview. This same model is used in both steps 1 and 2 of Figure 2. During step 1, candidate concepts (bottom right of the figure) are UMLS FR + EN mirror; during step 2, candidate concepts are gold + top candidates as described in section 4.3.2.

  ℎ () *,+ ,-./ 0 ⋅ + 3 where () *,+ is the batch normalization layer with mean 4 and variance 5, and W and b are the projection weights and bias respectively.

Figure 2 .

 2 Figure 2. Overview of our different experiments with MLNorm.

Figure 3 .

 3 Figure 3. Overview of the local concept embedding learning. For each synonym (green dot), we precompute its k closest concepts neighbors (squares in the blue disk). Only these concept neighbors will be updated (arrows) each time the synonym appears in a batch.

Table 1 .

 1 UMLS statistics. The English mirror is the set of concepts having synonyms in both English and French.

	Version	Language	#synonyms	#concepts	#synonyms / #concepts
	2014AB	English	5,772,518	2,528,878	2.28
		English 5	2,298 600	766,548	3.00
		sources
		French	179,992	88,985	2.02
		English mirror	544,383	88,911	6,12
	2019AB	English	9,187,793	4,258,236	2.16
		English 5	3,055,453	968,467	3.15
		sources
		French	374,144	154,362	2.42
		English mirror	903,098	154,307	5,85

Table 2 .

 2 Overview of the Quaero French Medical corpus. Note that 2015 and 2016 training sets are the same (*) and that the 2016 development set is the 2015 test set (**).

	Corpus		Mentions	Unique	Unique concepts	French coverage
				mentions
	EMEA	train*	2695	923	650	0.67
	2015	test**	2260	756	525	0.70
	Medline	train*	2994	2296	1860	0.77
	2015	test**	2977	2288	1847	0.76
	EMEA	train*	2695	923	650	0.67
	2016	dev**	2260	756	525	0.70
		test	2204	658	474	0.62
	Medline	train*	2994	2296	1860	0.77
	2016	dev**	2977	2288	1847	0.76
		test	3103	2390	1909	0.79

Table 3 .

 3 Main results for our system on 2015 and 2016 corpora, and comparison with existing systems.

	MEDLINE 2015		EMEA 2015
	Prec. Rec.	F1	Prec.	Rec.	F1

Table 4 .

 4 Comparison of our system with a comparable machine translation approach, using our classifier.

	Model	MEDLINE 2015	EMEA 2015
		Prec.	Rec.	F1	Prec. Rec.	F1
	MLNorm	0.756 0.719 0.737 0.797 0.736 0.765
	mBERT-MT (MLNorm with MT)	0.735 0.702 0.718 0.784 0.746 0.765
	BERT-MT (MLNorm with MT)	0.751 0.698 0.724 0.774 0.737 0.755

Table 5 .

 5 Additional results on the 2015 corpora, about the impact of the data language, of the pretrained embeddings, the size of the French terminology and the 2-step approach.

	Experiment	Model	MEDLINE 2015		EMEA 2015
			Prec. Rec.	F1	Prec.	Rec.	F1
	Impact of the	FR-only	0.738 0.528 0.615 0.824 0.528 0.644
	data language (step S1 only,	EN-only (EN mirror only, no FR syn)	0.797 0.451 0.575 0.843 0.410 0.551
	UMLS 2014)	FR/EN (FR + EN mirror)	0.783 0.621 0.693 0.827 0.574 0.678
	Impact of the	MLNorm	0.756 0.719 0.737 0.797 0.736 0.765
	pretrained
	embeddings	camemBERT	0.769 0.704 0.735 0.821 0.699 0.755
	(steps S1+ S2)	BERT (English base BERT)	0.759 0.716 0.737 0.805 0.734 0.768
		…instead of multilingual BERT
	Impact of more	MLNorm	0.756 0.719 0.737 0.797 0.736 0.765
	FR terms (steps S1+S2)	UMLS 2019 (instead of UMLS 2014)	0.753 0.710 0.731 0.795 0.728 0.760
	Impact of two	MLNorm	0.756 0.719 0.737 0.797 0.736 0.765
	step training
		1-step (S1 and S2 merged in 1 step, with S2	0.785 0.692 0.736 0.816 0.714 0.762
		training data)

Table 6 .

 6 Some predictions from our system. The last two columns contain the synonyms seen during training for the target concept and the predicted one, if different. Some long or similar synonyms have been removed to improve readability.

	System	Example mention	Expected concept and its synonyms from	Predicted concept if
			the train set	wrong, and its synonyms
				for the train set
	MLNorm (S2)	greffon renal	C1261317 [EN] transplanted kidney	✓
			[EN] kidney transplant
			[EN] structure of transplanted kidney
		cinquième métacarpien	C0730166 [EN] bone structure of fifth metacarpal	✓
			[EN] fifth metacarpal bone
			[EN] fifth metacarpal
		vaccination par le b.c.g	C0199804 [FR] immunisation contre la tuberculose	✓
			[EN] vaccination against tuberculosis
			[EN] bcg vaccination
			[EN] tuberculosis vaccination
			[EN] tuberculosis immunization
			[EN] administration of bcg vaccine
			… (other similar English synonyms)
		in vitro	C0681828	C3850137
			[EN] in vitro study [EN] studies vitro [EN] study vitro	[EN] in vitro techniques [EN] technique in vitro [EN] in vitro as topic
		coffea robusta	C0678439	C1138610
			[EN] coffea rubusta (food)	[EN] coffea arabica,
				unspecified
	mBERT-MT	cellar	C0042460	C0007634
		(translated from the French term "cave")	[EN] vena cava structure [EN] venae cavae [EN] vena cava [EN] cavae [EN] venae [EN] vena caval structure [EN] vena caval	[EN] cell [EN] cell structure [EN] cell type [EN] cells set [EN] cellula [EN] cellular
			[EN] vein [MT] veins cellars (from "veines caves") [MT] vein cellar (from "veine cave")	[EN] normal cell [EN] set of cells [MT] cells (from
				"cellules")
		be careful	C0004268	C3257858
		(translated from the French term "attention")	[EN] attention [EN] attentions	[EN] my thinking is usually careful and
				purposeful

The code for all experiments described in this paper is available at the following URL: https://github.com/percevalw/deep_multilingual_normalization/tree/master

ACKNOWLEDGMENTS

The authors thank the AP-HP health data warehouse for supporting this work.