Sarra Mezhoud
email: mezhoud.sarra@icmpe.cnrs.fr

Benjamin Le Droumaguet
email: ledroumaguet@icmpe.cnrs.fr

Patrick Aimedieu
email: patrick.aimedieu@enpc.fr

Vincent Monchiet
email: vincent.monchiet@univ-paris-est.fr

Michel Bornert
email: michel.bornert@enpc.fr

Dr Daniel Grande
email: grande@icmpe.cnrs.fr

Dr Benjamin Le Droumaguet

Investigation of morphology associated with biporous polymeric materials obtained by the double porogen templating approach

Keywords: doubly porous polymeric materials, porogenic agents, Spark Plasma Sintering, Xray computed microtomography

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Investigation of morphology associated with biporous polymeric materials obtained by the double porogen templating approach

Introduction

Porous materials with bimodal porosity have been the subject of intense research for many years, mostly because of their large range of applications. They are widely used as innovative materials in many areas, including civil engineering, separative sciences [START_REF] Lav | Porous polystyrene-based monolithic materials templated by semi-interpenetrating polymer networks for capillary electrochromatography[END_REF][START_REF] Lav | Porous Poly(styreneco-divinylbenzene) Neutral Monolith: From Design and Characterization to Reversed-Phase Capillary Electrochromatography Applications[END_REF][START_REF] Yu | Preparation of monolithic polymers with controlled porous properties for microfluidic chip applications using photoinitiated free-radical polymerization[END_REF], tissue engineering [START_REF] Langer | Tissue Engineering[END_REF] and drug delivery applications [START_REF] Nair | Biodegradable polymers as biomaterials[END_REF]. In the area of tissue engineering for instance, the first porosity level with pore sizes over 100 µm should allow for efficient seeding and proliferation of suitable cell lines through the material, while the second one with pore sizes lower than 1 µm may permit to improve the nutrient and waste flow within the material when the macropores are clogged at the last stage of the cell culture. In engineering applications, the larger porosity level would allow macromolecules flow through the material, while the smaller porosity level would be dedicated to the passage of smaller molecules or play an important role regarding filtration issues [START_REF] Nair | Biodegradable polymers as biomaterials[END_REF][START_REF] Auriault | Deformable porous-media with double porosity -Quasi-statics .1. Coupling effects[END_REF][START_REF] Boutin | Acoustic absorption of porous surfacing with dual porosity[END_REF].

In this context, different strategies have hitherto been developed for the development of polymeric materials exhibiting a double porosity. Temperature-induced phase separation (TIPS) has recently been reported by one of our groups for the design of poly(2-hydroxyethyl methacrylate) (PHEMA)-based scaffolds by using a co-solvent mixture constituted of water and 1,4-dioxane. Solidification process occurred by quenching the PHEMA/co-solvent mixture at a given temperature, and subsequent freeze-drying permitted to obtain biporous PHEMA frameworks [START_REF] Ly | Tailoring doubly porous poly(2hydroxyethyl methacrylate)-based materials via thermally induced phase separation[END_REF]. TIPS in conjunction with particle leaching has also been reported earlier for the synthesis of poly(L-lactide)-(PLLA) [START_REF] Santamaria | Influence of the macro and micro-porous structure on the mechanical behavior of poly(L-lactic acid) scaffolds[END_REF], gelatin- [START_REF] Liu | Phase separation, pore structure, and properties of nanofibrous gelatin scaffolds[END_REF] or poly(lactide-co-glycolide)-(PLGA) [START_REF] Yang | Formation of porous PLGA scaffolds by a combining method of thermally induced phase separation and porogen leaching[END_REF] based scaffolds. In these studies, poly(ethyl methacrylate) (PEMA), paraffin, and sucrose spheres (with different diameter ranges) were used as macroporogens, while dioxane, ethanol/water or chloroform allowed for the formation of a lower porosity level during the TIPS process. Furthermore, the High Internal Phase Emulsion (HIPE) technique was also used to obtain PHEMA-based materials with a hierarchical porosity [START_REF] Kovacic | Highly porous open-cellular monoliths from 2hydroxyethyl methacrylate based high internal phase emulsions (HIPEs): Preparation and void size tuning[END_REF][START_REF] Kulygin | Porous poly(2-hydroxyethyl methacrylate) hydrogels synthesized within high internal phase emulsions[END_REF][START_REF] Cameron | High internal phase emulsion templating as a route to well-defined porous polymers[END_REF][START_REF] Silverstein | Emulsion-templated porous polymers: A retrospective perspective[END_REF]. The first porosity level was generated by the removal of an organic porogenic solvent, while the second one resulted from the interconnections between adjacent macropores. The design of doubly porous polymeric materials was also investigated by a double porogen approach involving a macromolecular porogen, i.e. poly(ethylene glycol), in combination with NaCl particles to obtain PLLA scaffolds [START_REF] Ghosh | The double porogen approach as a new technique for the fabrication of interconnected poly(L-lactic acid) and starch based biodegradable scaffolds[END_REF]. Finally, in the past few years, doubly porous PHEMA materials were produced via a double porogen templating approach through the use of inorganic CaCO3 [START_REF] Droumaguet | Novel Polymeric Materials with Double Porosity: Synthesis and Characterization[END_REF][START_REF] Droumaguet | Engineering functional doubly porous PHEMA-based materials[END_REF] and NaCl [START_REF] Ly | Facile fabrication of doubly porous polymeric materials with controlled nano-and macro-porosity[END_REF][START_REF] Ly | Designing and modeling doubly porous polymeric materials[END_REF] particles or PMMA beads [START_REF] Droumaguet | Novel Polymeric Materials with Double Porosity: Synthesis and Characterization[END_REF][START_REF] Droumaguet | Engineering functional doubly porous PHEMA-based materials[END_REF] to generate the macroporosity.

The second porosity level was obtained by using either hydroxyapatite nanoparticles [START_REF] Droumaguet | Engineering functional doubly porous PHEMA-based materials[END_REF] or porogenic solvents, such as alcohols [START_REF] Droumaguet | Novel Polymeric Materials with Double Porosity: Synthesis and Characterization[END_REF][START_REF] Droumaguet | Engineering functional doubly porous PHEMA-based materials[END_REF][START_REF] Ly | Facile fabrication of doubly porous polymeric materials with controlled nano-and macro-porosity[END_REF].

In this study, we aim to get a thorough insight into the morphology of model biporous PHEMAbased materials obtained by the double porogen templating approach. In order to generate the macroporosity, either SPS-sintered NaCl particles or non-sintered ones are used as the macroporogenic template, while the second porosity level is generated by using a porogenic solvent through phase separation during the material formation by free-radical copolymerization. We focus our investigation on tuning the macroporosity, and more specifically on the effect of NaCl-template packing and particle size on porosity features associated with biporous frameworks. The porosity of the as-obtained porous materials is carefully characterized by means of mercury intrusion porosimetry (MIP), scanning electron microscopy (SEM), and nitrogen sorption porosimetry. Complementary X-ray computed microtomography (μCT) and synchrotron μCT analysis were also performed to investigate the 3-D microstructure and pore morphology of the biporous networks. Finally, swelling behavior of those materials was investigated so as to understand the contribution of each porosity level and the impact of interconnection in the higher porosity level.

Materials and methods

Materials

2-Hydroxyethyl methacrylate (HEMA, 97 %), ethylene glycol dimethacrylate (EGDMA, 98 %), and 2,2-dimethoxy-2-phenylacetophenone (DMPA, 99 %) were purchased from Aldrich.

Propan-2-ol (i-PrOH, for analysis ACS-Reag. Ph. Eur) was supplied by Carlo Erba. Sodium chloride (NaCl) particles with size ranging from 50 to 500 µm were purchased from Prolabo; they were then sieved to isolate the particle fractions with sizes ranging from 125 to 200 µm, from 200 to 250 µm, and from 250 to 400 µm that were stored in moisture-free conditions. All reagents and solvents were used without any further purification procedure.

Preparation of doubly porous PHEMA-based materials

Firstly, fractions of 125-200 µm, 200-250 µm and 250-400 µm NaCl particles were fused by Spark Plasma Sintering (SPS) procedure so as to obtain a 3-D NaCl particle template. A Sumitomo Dr. Sinter Lab 515S SPS machine from Fuji Electronic Industrial was used. For the sintering treatment, 2 g of NaCl particles were introduced into a cylindrical graphite die (diameter: 10 mm) and heated from room temperature to 100 °C at a heating rate of 40 °C min -1 .

A 3 kN force was applied onto the sample under argon. After 5 min at 100 °C, the fused template was cooled to room temperature at a rate of 40 °C min -1 . For the sake of comparison, 200-250 µm sieved NaCl particles were also used without any sintering procedure. 2 g of particles were weighed in a vial and then gently stirred under orbital shaker in order to ensure homogeneous particle packing. In a second step, HEMA and EGDMA were mixed with a molar ratio of 70/30 mol. % in the presence of DMPA (2 wt. % with respect to the comonomers), and propan-2-ol (80 vol. %). The mixture was added to the NaCl particle template (either sintered or not), and the polymerization was conducted in a UV oven for 4 h at 365 nm. Non-sintered NaCl particles were removed by extraction with deionized water for 3 days (water was changed once a day), while sintered NaCl particles were removed for one day. After this porogen extraction step, the samples were washed abundantly with water, and dried at room temperature under vacuum.

The samples were analyzed by gravimetry so as to confirm the total removal of the porogenic agents (NaCl macroparticles and solvent).

Morphology of (bi)porous PHEMA polymers

SEM observation of the materials was performed on a MERLIN microscope from Zeiss equipped with InLens and SE2 detectors using a low accelerating tension (2-3 kV) with a diaphragm aperture of 30 μm. The samples were first cryofractured and coated with a 4 nm layer of palladium/platinum alloy in a Cressington 208 HR sputter-coater.

Determination of porous features

Porosity ratios, pore volumes, and pore size distributions of the materials were determined by mercury intrusion porosimetry (MIP) using a Micromeritics AutoPore IV 9500 porosimeter.

The determination of the porosity features was based on the Washburn equation between the applied pressure (from 1.03 to 206.8 MPa) and the pore size into which mercury intruded. The nitrogen sorption measurements were carried out at 77 K with a Quantachrome Autosorb iQ analyzer. The specific surface area values for the investigated porous polymers were quantified using the BET method at relative pressure (P/P0) values ranging from 0.05 to 0.3.

Study of the microstructure of (bi)porous scaffolds by X-Ray microtomography

The computed X-ray microtomography investigations were performed using an Ultratom microtomograph system manufactured by RX Solutions. Two imaging setups were used, Science (pixel size 9 µm, Gadox scintillator) was used to investigate smaller samples at a higher resolution. 3-D images were reconstructed from the radiographic projections using the X-act software provided by the manufacturer. For the larger scale investigations, the experimental protocol consisted in sticking a cylindrical biporous PHEMA sample, about 1 cm in diameter, on top of a thin cylindrical holder, which was mounted on the rotating stage of the microtomograph. The micro-focus source was set at 25 kV tension and 400 μA intensity, and the geometric setup ensured a voxel size of 8.75 μm in the reconstructed images. 1440 radiographic projections were taken over a complete 360-degree rotation in about 10 h, and 3-D images of 1840 × 1840 × 1270 voxels were reconstructed. For the smaller scale investigation, the sample of about 3 mm in diameter was held in a plastic straw, which was stuck on the rotating stage. The nano-focus source was used at 50 kV tension and 100 μA beam intensity.

The camera was used in 4 × 4 binning mode in order to increase the signal to noise ratio. The 700 projections were recorded in about 2 h and the 3-D reconstructed images had a definition of 1000 × 1000 × 700 voxels, with a voxel size of 2.0 µm; scanning time was about 2 h. All the images shown in this study were analyzed with the Fiji ® Freeware. As the considered materials were almost transparent to X-rays, the obtained images were rather noisy. That is the reason why they had to be digitally filtered for some investigations; the "2-D anisotropic diffusion" filter available in Fiji has been selected to process the 2-D slices, for its ability to reduce noise while preserving interfaces.

Synchrotron X-ray tomography imaging

The synchrotron X-ray tomography investigations were performed at Synchrotron Soleil (Saint Aubin, France) on the ANATOMIX beamline with a parallel beam and a pink beam with an In the first one, the sample is scanned over 180° (4 times) to reconstruct an image of 2000  2000 pixels with low noise. The second one is a 360° scan with an off-center axis and 2 heights, to reconstruct 2 juxtaposed images of 4000  4000  2000 pixels with an overlap.

Water uptake determination

In a typical experiment, a known weight (Wd) of PHEMA (bi)porous polymer was immersed in milliQ deionized water. At specific times, the same sample was weighted to obtain Ww as a function of time. The water uptake of the polymer networks can be expressed following Eq.

(1):

𝑊𝑎𝑡𝑒𝑟 𝑈𝑝𝑡𝑎𝑘𝑒 = 𝑊 𝑤 -𝑊 𝑑 𝑊 𝑑 Equation 1
where Ww and Wd stand for the wet mass and the dry mass, respectively. The monitoring of water uptake as a function of time was stopped when a plateau was reached.

Results and discussion

As previously demonstrated in different studies, NaCl particles have been successfully used to generate macroporous [START_REF] Kim | Three-dimensional aqueous-derived biomaterial scaffolds from silk fibroin[END_REF] or biporous materials through particle templating combined to a porogenic solvent after a straightforward porogen removal [START_REF] Ghosh | The double porogen approach as a new technique for the fabrication of interconnected poly(L-lactic acid) and starch based biodegradable scaffolds[END_REF][START_REF] Gross | Thermally sprayed scaffolds for tissue engineering applications[END_REF][START_REF] Gross | Biodegradable composite scaffolds with an interconnected spherical network for bone tissue engineering[END_REF]. It should be stressed that previous reports based on the double porogen approach using NaCl as a macroporogen only concerned either non-sintered NaCl particles [START_REF] Ghosh | The double porogen approach as a new technique for the fabrication of interconnected poly(L-lactic acid) and starch based biodegradable scaffolds[END_REF] or sintered ones [START_REF] Gross | Thermally sprayed scaffolds for tissue engineering applications[END_REF][START_REF] Gross | Biodegradable composite scaffolds with an interconnected spherical network for bone tissue engineering[END_REF], and no thorough investigation of the influence of nanoporogen or even macroporogen features on pore morphology was provided. Recently, we have developed a somehow smarter strategy toward doubly porous PHEMA-based networks in which NaCl macroparticles sintered through the SPS technique are used to form a 3-D continuous macroporogenic template in conjunction with a porogenic agent consisting of a suitable solvent. In one such case, the 3-D continuous NaCl particle-based template leads to a macroporosity level, while the porogenic solvent ensures a lower porosity level through a phase separation process during the polymerization step [START_REF] Droumaguet | Engineering functional doubly porous PHEMA-based materials[END_REF][START_REF] Ly | Facile fabrication of doubly porous polymeric materials with controlled nano-and macro-porosity[END_REF][START_REF] Ly | Designing and modeling doubly porous polymeric materials[END_REF].

At this stage, it is worth mentioning that the porogenic solvent must obviously be miscible with the comonomers in order for the phase separation to occur during the copolymerization process.

Indeed, when the polymerization is at an advanced step, the incompatibility between the solvent and the polymeric network results in the expulsion of solvent molecules from the polymer, thus leading to pores whose sizes depend notably on the nature and concentration of the solvent as well as the crosslinker concentration [START_REF] Ly | Facile fabrication of doubly porous polymeric materials with controlled nano-and macro-porosity[END_REF]. The advantage of our approach, that is based on the concomitant use of two distinct porogenic agents, namely NaCl macroparticles and a porogenic solvent (i-PrOH in the present study), lies in the independent control of each porosity level. In our previous studies [START_REF] Ly | Facile fabrication of doubly porous polymeric materials with controlled nano-and macro-porosity[END_REF], we have thus shown that the use of more and more hydrophobic alcohols as porogens for the preparation of hydrophilic PHEMA-based crosslinked materials leads to increasing average pore sizes for the lower porosity level without any significant modification of the macroporosity. Likewise, an increasing proportion of the porogenic solvent also leads to increasing pore sizes related to the lower porosity level, while decreasing the upper to lower porosity ratio.

Effect of macroporogen packing on doubly porous materials

The present study is specifically focused on the macroporogen packing effect on biporous PHEMA-based networks by comparing sintered vs. non-sintered NaCl macroparticles with a size range comprised between 200 and 250 µm. The polymerization mixture was constituted of a fixed HEMA/EGDMA molar ratio of 70/30 mol. % in conjunction with 80 vol. % i-PrOH with respect to the comonomer volume. The MIP profiles and SEM micrographs of the resulting doubly porous PHEMA-based materials are respectively reported in Fig. 1 and Fig. 2, while their main porous features are gathered in Table 1. The biporous structure prepared from nonsintered NaCl particles (Fig. 1a) exhibited an ill-defined dual porosity along with pore sizes with a macroporosity centered around 200 µm and a lower porosity level centered around 6 µm.

In sharp contrast, doubly porous networks prepared from SPS-sintered NaCl particles presented a well-defined bimodal porosity with a first macropore size of about 60 µm and a second macropore size around 1.5 µm, as observed in Fig. 1b.

Porous characteristics Non-interconnected macropores

Interconnected macropores Larger macropore size (µm) a 200 60

Smaller macropore size (µm) a 6 1.5

Upper to lower porosity ratio (vol. %) a nd c 80/20

Total pore volume (mL g -1 ) a 6.5 6.0

Porosity ratio (%) a 80 87 Specific surface area (m 2 g -1 ) b 80 105 a Values as determined by MIP. b Values as determined by nitrogen sorption measurements (BET method). c This ratio could not be determined properly.

It is worth mentioning that the pore sizes obtained from MIP measurements are often underestimated when compared to SEM observations. This has already been observed notably with poly(HIPE) samples [START_REF] Audouin | Synthesis of polymer-silica hybrid polyHIPEs by double in situ polymerization of concentrated water in oil emulsion[END_REF] or when using fused PMMA beads as macroporogens [START_REF] Ly | Designing and modeling doubly porous polymeric materials[END_REF], and it could be explained by the threshold size theory that was reported in the literature concerning MIP measurements on cement-based materials [START_REF] Aligizaki | Pore structure of cement-based materials: testing, interpretation and requirements[END_REF]. This original biporous material presented a upper to lower porosity volume ratio equal to 80/20 vol. %. In this case, the resulting biporous material presented interconnected macropores and thus a higher porosity ratio, i.e. 87 % compared to 80 % for the corresponding analogue prepared from non-sintered NaCl particles, due to closer particle packing resulting from SPS-mediated NaCl sintering. As it can be seen in Fig. 2, SEM observations of the biporous material prepared from non-sintered macroporogenic agent clearly showed the occurrence of two porosity levels. Non-interconnected macropores having NaCl particle imprints could be detected (Fig. 2B), while globular polymeric structures were observed within the macropore walls delimiting an interconnected network with a lower porosity level (Fig. 2A), that is usually found when phase separation occurs in such polymerization mixtures constituted of at least one porogenic solvent. In the latter case, the interconnection between adjacent macropores appeared to be largely absent. Comparatively, SEM investigation of the biporous PHEMA framework obtained from sintered NaCl particles showed a high level of interconnections between adjacent macropores, while each level of porosity was preserved. One such sample exhibited a higher specific surface area, i.e. SBET = 105 m 2 g -1 , than that with non-interconnected macropores (SBET = 80 m 2 g -1 ), as measured from nitrogen sorption porosimetry.

μCT has recently been implemented for the characterization of macroporous polymer-based materials, and especially for biodegradable porous polymers dedicated to tissue engineering applications [START_REF] Cartmell | Quantitative microcomputed tomography analysis of mineralization within three-dimensional scaffolds in vitro[END_REF][START_REF] Darling | 3D microtomographic characterization of precision extruded poly-epsilon-caprolactone scaffolds[END_REF][START_REF] Lin | Microarchitectural and mechanical characterization of oriented porous polymer scaffolds[END_REF][START_REF] Van Lenthe | Nondestructive micro-computed tomography for biological imaging and quantification of scaffold-bone interaction in vivo[END_REF][START_REF] Jones | Analysis of 3D bone ingrowth into polymer scaffolds via micro-computed tomography imaging[END_REF][START_REF] Moore | Quantitative analysis of interconnectivity of porous biodegradable scaffolds with microcomputed tomography[END_REF]. This remarkable technique notably allowed for some 3-D volumic rendering of the porous materials or the corresponding macroporogenic templates [START_REF] Van Lenthe | Nondestructive micro-computed tomography for biological imaging and quantification of scaffold-bone interaction in vivo[END_REF], thus leading to significant information that could be complementary to more classical SEM and MIP techniques. A significant advantage of µCT with respect to SEM was its ability to investigate the bulk of the material in a non-destructive manner, with almost no constraints in terms of sample preparation. Unlike MIP, it provided also a direct, at least qualitative, information on pore size and geometry. On the other hand, spatial resolution of µCT was still limited and the low X-ray absorption contrast provided by highly porous polymeric materials led to noisy images.

μCT imaging was first performed on doubly porous PHEMA cylindrical samples 1 cm in diameter, and in this particular case, the NaCl macroporogens were not previously sieved (grain size between 50 and 500 µm). The microstructure of samples elaborated from non-sintered (Fig. 3a) and SPS-sintered (Fig. 3b) NaCl particles permitted to confirm the surface observations made by SEM regarding the macroporous network (i.e. upper porosity level). As shown in Fig. 3, the pore distribution was rather homogeneous in both samples, and they showed a macroporous network with pore imprints of sizes similar to those of NaCl particles. The exploration of the 3-D image of the sample prepared from non-sintered NaCl (cross-section presented in Fig. 3a) qualitatively confirmed that the macropores were mostly isolated within the polymer matrix. The macropore morphology presented various shapes, such as pseudocubes, spheres, and pyramids. The matrix exhibited some large areas without pores, a few hundred micrometers in size; some thin matrix walls separating pores, or even inside pores, with a thickness close to voxel size, could however also be observed. A few neighbor pores seemed to be connected; it was however not clear whether they were separated or not by a thin matrix wall which could not be resolved by the 3-D image of the whole samples. Such situations were however seldom.

The 3-D µCT image of the sample obtained from the SPS-fused particles, a cross-section of which was provided in Fig. 3b, showed a microstructure with mostly pseudo-spherical pores separated by much regular and thinner matrix walls (less than 50µm). However, the 3-D exploration of these data showed that most pores seemed to be connected to their neighbors. Similar connections could be found for almost all pores in other cross-sections. Strictly speaking, this did not prove that pores were really connected, as separating walls could have been too thin with respect to image resolution. These observations were however consistent with the expected features of the microstructure of this material. It is noteworthy that some bright area could be observed on the periphery of the sample reflecting strongly attenuating constituents, which might be residual NaCl particles potentially entrapped in the material and not removed during preparation. The major differences between both samples, as qualitatively demonstrated by the µCt images, laid in the interconnection between adjacent macropores, the pore shape and the matrix volume fraction. It is noteworthy that the polymer matrix proportion was obviously more important for the material displaying non-interconnected macropores. This could be explained as the consequence of the random packing of inorganic macroporogens used in the preparation of the NaCl template associated with the porous materials. This result was also confirmed more quantitatively by experimental measurements of the density of samples elaborated from nonsintered NaCl particles which was equal to 0.093 g.cm -3 while materials prepared with SPSsintered NaCl particles showed a 0.067 g.cm -3 density. Such a global measurement of density allows one to directly compare the macroporosity, if one assumes the lower porosity level to be the same in both samples, which sounds reasonable in a first approximation as both pore forming mechanisms are essentially independent.

Furthermore, the macroporosity of the sample obtained from non-sintered NaCl was quantitatively evaluated from its µCT image with the following methodology. The average grey level gav of a large region of interest of the sample, made of 200 slices, was compared to the average grey level of voids gv, and of microporous matrix gm, which could be determined by a manual selection within a 2-D cross section of sufficiently large domains strictly included in the pore or matrix phase. As grey levels in µCT images reflect average X-ray attenuation coefficients, which in case of two-phase materials are essentially proportional to the volume fraction of constituents within voxels, a simple rule of mixture can be used to evaluate the average macroporosity P from these quantities, [START_REF] Bruchon | Full 3D investigation and characterisation of capillary collapse of a loose unsaturated sand using X-ray CT[END_REF] according to gav = P × gv + (1-P) × gm.

When applied to non-filtered images (see Fig 4a for a typical cross-section), this procedure led to an evaluation of P of 64 %.

A similar procedure could however not be applied efficiently to the other sample obtained from fused NaCl because the value of gm was difficult to quantify precisely, for two reasons (see Fig. 4d): first, the matrix domains are much thinner so it is hard to select large domains made of matrix only, and second, the grey levels within the matrix exhibited larger fluctuations, possibly because of a larger variability of the lower porosity level, or the presence of residual traces of NaCl (as observed near the periphery of the sample). However, assuming a similar lower porosity level in both samples, the macroporosity of the second material could be deduced from that of the first one and from the ratios of the overall densities. In this way, a macroporosity of 74 % was found for SPS-sintered samples, i.e. a value very close to that found independently by means of MIP, which is 0.8 × 0.87 = 70 %.

Let us finally stress that a more conventional approach to evaluate volume fractions would have been based on a segmentation of images. It would however have been much less accurate and possibly biased, because of the high noise level of the images and the voxel size which is too large with respect to many small features of the microstructure, such as wall thicknesses. To illustrate this, a filtered region of interest made of 200 slices (see Fig. 4b for a representation of one of these) has been segmented making use of a threshold chosen such that the pore phase, defined as the voxels with a grey level below this value, represent 64 % of all voxels. The resulting image is illustrated in Fig. 4c which shows significant differences with Fig. 4a, especially regarding the polymer walls. This holds for images of the material obtained from non-sintered NaCl grains; the situation is even more critical for the other one, for which conventional thresholding would lead to meaningless results.

Effect of macroporogen size on doubly porous materials

Further, the macroporosity of biporous PHEMA-based samples was carefully investigated by varying the size of NaCl particles used to generate the 3-D inorganic template of the porous materials. The choice of the macroporogen size had no significant influence on the lower porosity level as the two pore-forming mechanisms are independent, as mentioned above. The amount of porogenic solvent was kept constant, while three different fractions of sieved NaCl particles, i.e. 125-200 µm, 200-250 µm, and 250-400 µm, were used to prepare biporous PHEMA-based materials that were then characterized finely in terms of porosity features. MIP profiles reported in Fig. 5 logically showed a correlation between an increase in the macroporogen size and an increase in the size of resulting macropores and pore volumes. For the lowest NaCl size (i.e., 125-200 µm), a macroporosity centered on 40 µm was observed with a 77 % porosity ratio. For the intermediate particle size (i.e., 200-250 µm), the macropore size was nearly centered on 60 µm with an 87 % porosity ratio. Finally, biporous materials designed with the highest macroporogen size (i.e., 250-400 µm) exhibited a macropore size of about 70 µm and a porosity ratio equal to 88 %. Again, it is noteworthy that the macroporogen size was higher than the corresponding macropore size. This is actually quite often observed with porous materials for which the pore size determined by MIP is generally underestimated when compared to SEM observation, likely due to the threshold size theory [START_REF] Audouin | Synthesis of polymer-silica hybrid polyHIPEs by double in situ polymerization of concentrated water in oil emulsion[END_REF][START_REF] Aligizaki | Pore structure of cement-based materials: testing, interpretation and requirements[END_REF]. Likewise, specific surface area values increased gradually with increasing macroporogen sizes, while the upper to lower porosity volume ratio was nearly constant (Table 2). Biporous networks prepared from 125-200 µm and 200-250 µm NaCl particles were analyzed via μCT imaging. As shown in Fig. 6 and Fig. 7, the macropore dispersion was homogeneous for both samples but some areas showed larger pores (approximate diameter: 500 µm) due to NaCl particles aggregation during sintering step. For both samples, porosity ratio was again difficult to estimate accurately using either of the above presented methods, because of high noise levels, low contrast difference between the polymer matrix and voids, and too coarse voxel size with respect to microstructural features. In order to investigate more precisely the morphology of these materials, a sample prepared with 200-250 µm sintered NaCl particles was observed via μCT measurements with higher spatial resolution, namely a voxel size of 2.0 µm instead of 8.75 µm. Despite the noise, Fig. 8 permitted to clearly observe a porous network (corresponding to the lower porosity level) within larger matrix domains, surrounded by uniform thin walls. This thickness of the latter was about 1 voxel in the images, indicating an actual physical width lower than the voxel size. This result was in good agreement with the previous SEM micrographs showing a porous structure surrounded by a homogenous polymeric shell. Furthermore, μCT image showed some isolated matrix filaments certainly formed in the thin areas near the necks between sintered grains, or in the cracks in the NaCl particles caused by the SPS sintering step. This very fine structure could not be resolved accurately within the images with a coarser voxel size, which explained the difficulty to segment the latter. Because of very high noise levels, it was also difficult to segment efficiently the higher resolution images. Further 3-D imaging attempts using devices with higher photon fluxes as those available on synchrotron facilities are planned in forthcoming work to investigate more quantitatively the microstructure of such materials. The use of synchrotron radiation as an X-ray source brought important improvements to the imaging. [START_REF] Flannery | Three-Dimensional X-ray Microtomography[END_REF][START_REF] Kinney | X-Ray Tomographic Microscopy (XTM) Using Synchrotron Radiation[END_REF] Synchrotron radiation results from the bending of a higher energy electron beam due to a magnetic field. The main advantage of this technique is that the high flux allows one to resolve very subtle variations in absorptivity and therefore in internal structure.

Additional advantages of synchrotron radiation include X-ray beam collimation, which simplifies the tomographic reconstruction algorithm, and the tunability of the X-ray energy to a narrow energy band. Moreover, the use of a monochromatic X-ray beam improves the accuracy of the reconstructed tomographic images by eliminating the issue of energy dependence on X-ray absorption. In our case, synchrotron imaging is expected to provide much more accurate, less ambiguous, more spatially resolved and much less noisy images, within more reasonable scanning times. It is important to mention that the experiments were carried out on a new and not yet optimized beamline (Anatomix, Synchrotron Soleil, Saint-Aubin, France) that did not yet include a monochromator. The experiments were therefore carried out using a wide X-ray spectrum determined by the gap of the magnets and a solid filter. The spectrum was thus reduced to approximately 10 keV providing more suitable radiation for polymer absorption compared to laboratory microtomography. Moreover, all the samples were observed directly after the elaboration without the incorporation of a contrast agent. The observation was performed on biporous samples prepared from sintered and non-sintered NaCl particles with sizes ranging from 125 to 400 μm (Fig. 9). Monoporous materials with large interconnected and non-interconnected pores (200-250 μm) were also analyzed in a second step (Fig. 10). X-ray synchrotron tomography permitted to considerably reduce the noise and the ring artifacts without image processing tools such as 2-D anisotropic filter. Moreover, the main advantage is that the use of a contrast agent was not necessary because the beam energy and the spatial resolution were more adapted. The reconstructed volumes were higher (2000 3 pixels and 4000 3 pixels vs. 1700 3 pixels for classical microtomography) as well as the number of radiographic projections (4000/8000 vs. 1440). This technique permitted also to reduce the scanning time from 9 h to few minutes. As observed in the Fig. 9, the conclusions that have been made for the images obtained with classical X-ray tomography are confirmed. Indeed, three different features could be observed: the large pores, the porous polymeric matrix and the thin polymeric walls. We could notice that the interconnected materials prepared with the lowest NaCl particles size (i.e. 125-200 μm) exhibited less polymeric matrix. Indeed, as the grains were smaller, these templates would have required more reactants (monomers) to form a sufficient amount of polymer which could fill all the template interstices. Furthermore, the omnipresence of the walls deformation probably induced by the drying is even more visible particularly for non-interconnected networks regardless the salt size used during the elaboration. Such observations also permitted to confirm the reliability of the use of Lugol solution, as the microstructures features are similar with and without contrast agent. PHEMA and EGDMA-crosslinked PHEMA (40 % and 42 %, respectively). The study was performed on interconnected and non-interconnected biporous networks prepared from NaCl particle size between 200 and 250 μm. Fig. 11 represents the water uptake of interconnected and non-interconnected biporous networks as a function of time. In both cases, two plateaux were observed that probably correspond to the water filling of each porosity level. Indeed, it is likely to happen that water enters the large and small pores before binding with the polymer. In the case of interconnected networks, the water filling began with the channels between larger pores and then larger pores during 20 min. Then, the water uptake reached a constant value equal to 1500 % during 30 min. In a second step, water drains for 1 h through smaller pores before reaching a second plateau. At the end of the experiment, the water uptake was approximately equal to 2400 %, and the mass of the wet material was almost 24 times higher than that of the dry sample. In the case of non-interconnected networks, the evolution of the swelling degree was similar excepted for the second plateau which was reached less rapidly than in the case of interconnected networks (180 min vs. 120 min, respectively). This difference is likely due to the lack of interconnections between larger pores which should facilitate the draining of water. The water uptake was then lower in the case of non-interconnected biporous networks. Indeed, the material incorporated 1200 % of water at the end of the experiment. Water uptake was also evaluated with monoporous PHEMA-based polymers. Some previous studies showed that a bulk PHEMA-based polymer (HEMA/EGDMA, 96/4 mol. %) incorporated 58 % of water after 2 days. [START_REF] Mabilleau | Effects of the length of crosslink chain on poly(2-hydroxyethyl methacrylate) (pHEMA) swelling and biomechanical properties[END_REF] Moreover, Lokman et. al found that PHEMA porous monoliths crosslinked with maleic anhydride and presenting 78 % porosity were able to incorporate 68 % of water. [START_REF] Uzun | Porous poly(hydroxyethyl methacrylate) based monolith as a new adsorbent for affinity chromatography[END_REF] Finally, Gomes et. al showed that the water uptake of scaffolds prepared with ethylene vinyl alcohol and cellulose acetate could reach values of around 30 %. [START_REF] Gomes | A new approach based on injection moulding to produce biodegradable starch-based polymeric scaffolds: morphology, mechanical and degradation behaviour[END_REF] For the sake of comparison, we performed this study on monoporous materials exhibiting large pores or small pores. The swelling degree as a function of time is reported in Fig. 12. As expected, a monoporous network incorporated less water than a biporous one. We could notice that the equilibrium swelling degree was reached after 10 min in the case of a material with small pores (350 % water uptake), 40 min for a material with large interconnected pores (275 % water uptake) and 180 minutes for large non-interconnected pores (50 % water uptake). It is important to mention the presence of one single plateau in the case of monoporous samples. This observation permitted to confirm the hypothesis in which the two porosity levels were filled gradually in the case of biporous networks. In the case of materials exhibiting only small pores and samples exhibiting interconnected large pores, a constant swelling degree was reached rapidly, whereas water had more difficulties to drain through the materials with noninterconnected large pores. We could explain this phenomenon by the lack of interconnections between the larger pores. Moreover, the absence of smaller pores reduced the path of the fluid through the material. Furthermore, the thin resin formed around the material during the polymerization (due to the absence of porogenic solvent) could also slow down the passage of water. As expected, the lowest swelling degree was obtained in the case of non-interconnected monoporous network with only 50% water uptake. This observation permits to explain the low permeability of these materials and their low water uptake compared to the other structures.

Finally, as expected, the swelling ratio of bulk PHEMA is very low, i.e. 6%, when compared to its (bi)porous counterparts.

Conclusions

This paper demonstrated the pivotal role of porogenic particle packing on the pore interconnectivity in biporous PHEMA-based materials obtained through the double porogen templating approach. SPS-mediated macroporogen sintering induced a higher compaction of NaCl particles, thus generating fewer interparticular voids, and eventually afforded a higher porosity ratio of the resulting porous polymeric materials with interconnected macropores. The NaCl macroporogen size was also determined to be a crucial parameter regarding the porosity features, as increasing macroporogen sizes were associated with increasing porosity ratios. A X-ray computed microtomography (μCT) analysis confirmed these results as the 3-D microstructure observed in the case of the material prepared from non-sintered NaCl particles pointed out some sparse interconnections between adjacent macropores, likely due to very few contact points between neighbor NaCl particles in the 3-D inorganic template of the porous material. This studies also highlighted the considerable capacity of biporous PHEMA materials to incorporate water compared to bulk and monoporous polymers. Indeed, these complex structures were able to incorporate until 2400 % of water in a few hours, while monoporous counterparts only incorporated 300 %. In the literature, it was demonstrated that the water uptake of scaffolds and porous monoliths did not exceed 68 %. Moreover, the pore interconnectivity increased greatly the water uptake (by a factor of nearly 5-6). As abovementioned, the synchrotron tomography permitted to observe the fine microstructure with 0.65 μm voxel size which is much lower than the scans made by CT (8 μm and 2 μm). This research permitted to investigate the 3-D microstructure of doubly porous polymeric networks by means of classical μCT and synchrotron μCT. The different experimental protocols and imaging devices configuration allowed a precise observation of the morphology of such materials as the pores connectivity and shape. Imaging provided also crucial information about the void/polymeric matrix interfaces which play an important role in transport properties. Three microstructural features were observed : the large pores having the NaCl particles shape, the polymeric matrix exhibiting the small pores, and thin polymeric walls. However, the quantitative analysis of the porosity was tricky because of the very thin polymeric walls and filaments which made difficult the segmentation step. Finally, some μCT and synchrotron images will be selected as input data for the computation of fluid flow through porous medium in forthcoming experiments. The interconnection in the higher porosity level was demonstrated to be crucial regarding transport properties in such biporous PHEMA-based materials as their water uptake behavior is much more important than that of their non-interconnected counterparts.

  Mezhoud, Benjamin Le Droumaguet, Patrick Aimedieu, Vincent Monchiet, Michel Bornert, Daniel Grande To cite this version: Sarra Mezhoud, Benjamin Le Droumaguet, Patrick Aimedieu, Vincent Monchiet, Michel Bornert, et al.. Investigation of morphology associated with biporous polymeric materials obtained by the double porogen templating approach. Colloid and Polymer Science, 2021, 299 (3), pp.537-550. 10.1007/s00396-020-04747-9. hal-03171995

providing 3 -

 3 D images with different spatial resolutions. In the first one, aimed at imaging the whole manufactured samples of 1 cm in diameter, a Hamamatsu L10801 micro-focus reflection X-ray source (max. 230 kV and 200 W, min. spot size 5 µm) was combined with a Varian 2520 flat panel detector (1920 × 1536 pixels, pixel size: 127 µm, CSI scintillator). The second configuration based on a Hamamatsu nano-focus transmission L10711-23 source (max. 160 kV and 15 W, min. spot size 0.25 µm) and a 4008 × 2672 pixels VHR camera from Photonic

energy range from 5

 5 to 25 keV, resulting from the use of a 12.5 mm gap on the ondulator and filtering by Au. Note that the Anatomix beamline is the latest beamline of Synchrotron, constructed in the context of the EquipEx project NanoImagesX. Our experiment run in April 2018 was the first official user-experiment on this line, which was still in a temporary, not optimized, configuration. A local tomography was performed with a 10  optics in absorption contrast mode and a Hammamatsu Orca Flash 4.0 V2 camera (2048  2048 pixels, 6.5 μm), associated with a scintillator 500 μm in thickness. The cylindrical material of about 1 cm in diameter was stuck on a PMMA sample holder inserted on the rotating stage. Radiographic scans using 2000 to 8000 projections were run together with the acquisition of "dark" (i.e. without beam) and "at" images (i.e. without sample), averaged over 50 frames. Because the setup was not yet optimized, only the central part of the radiographs were perfectly focused, on a height of about 1000 pixels. During these experiments, 2 acquisition modes were carried out.

Fig. 1

 1 Fig. 1 MIP profiles of biporous PHEMA materials prepared from 200-250 µm non-sintered (a) or SPS-sintered (b) NaCl particles.

Fig. 2

 2 Fig. 2 SEM micrographs of biporous PHEMA-based materials with non-interconnected macropores prepared by using non-sintered NaCl particles (A,B) and with interconnected macropores prepared by using SPS-sintered NaCl particles (C,D).

Fig. 3

 3 Fig. 3 μCT pictures of porous PHEMA-based networks exhibiting non-interconnected macropores (a) and interconnected macropores (b).

Fig. 4

 4 Fig. 4 μCT enlarged pictures of porous PHEMA-based frameworks: (i) exhibiting non interconnected macropores before (a) and after image processing (b) as well as corresponding area after segmentation (c) or (ii) exhibiting interconnected macropores (d). Scale bars represent 500 µm.

Fig. 5

 5 Fig. 5 (a) MIP pore size distribution profiles and (b) cumulative intrusion volumes of biporous PHEMA-based materials prepared from sintered NaCl particles in the presence of i-PrOH as a porogenic solvent. Three NaCl particle size fractions were used, i.e. 125-200 µm, 200-250 µm, and 250-400 µm.

Fig. 6

 6 Fig. 6 μCT pictures of biporous PHEMA-based networks prepared from 125-200 µm (a) and 200-250 µm (b) NaCl particles. Scale bars represent 500 µm.

Fig. 7

 7 Fig. 7 μCT enlarged pictures of porous PHEMA-based frameworks exhibiting interconnected macropores generated from 125-200 µm (a) and 200-250 µm (b) NaCl particles. Scale bars represent 500 µm.

Fig. 8

 8 Fig. 8 μCT picture and SEM micrograph of porous PHEMA-based framework exhibiting interconnected macropores generated from 200-250 µm NaCl particles.

Fig. 9

 9 Fig. 9 Synchrotron images of doubly porous networks prepared with 125-200 μm sintered (a) and non-sintered NaCl particles (b), prepared with 200-250 μm sintered (c) and non-sintered

Fig. 11

 11 Fig. 11 Water uptake of non-interconnected and interconnected biporous networks as a function of time (*P = porosity ratio).

Fig. 12

 12 Fig.[START_REF] Kovacic | Highly porous open-cellular monoliths from 2hydroxyethyl methacrylate based high internal phase emulsions (HIPEs): Preparation and void size tuning[END_REF] Water uptake of monoporous networks with interconnected and non-interconnected large pores and small pores as a function of time (*P = porosity ratio).

Table 1

 1 Porous features of biporous PHEMA-based materials with non-interconnected and interconnected macropores.

Table 2

 2 Porous features of biporous PHEMA-based materials with varying interconnected macropore sizes.

Acknowledgments

This work was supported by a French government grant from ANR within the frame of the national program Investments for the Future ANR-11-LABX-022-01 (LabEx MMCD project).

The authors are also indebted to B. Villeroy (ICMPE, CNRS) for his kind technical assistance in the SPS technique.

Compliance with ethical standards

Funding

This work has benefited from a French government grant by ANR within the framework of the national program Investments for the Future ANR-11-LABX-022-01 (LabEx MMCD project).

NaCl particles (d) and prepared with 250-400 μm sintered (e) and non-sintered NaCl particles (f).

The materials exhibiting only large pores were elaborated without the use of propan-2-ol as porogenic solvent . In Fig. 10a corresponding to an interconnected network with large pores, the non-porous polymeric matrix could be observed, and the large pores size was similar to the one obtained with biporous materials (Fig. 9). In the material exhibiting non-interconnected large pores (Fig. 10b), we could observe that the NaCl particles were not completely extracted from the material. This is probably due to the resin that has been formed during the polymerization.

Swelling properties of mono-and biporous PHEMA-based materials

Some of us recently demonstrated that the presence of distinct (non-)interconnected porosity levels within biporous polymeric scaffold has a crucial impact on their fluid flow transfer properties, especially on imbibition and drying. [START_REF] Lerouge | Drying of a Compressible Biporous Material[END_REF][START_REF] Lerouge | Synergistic actions of mixed small and large pores for capillary absorption through biporous polymeric materials[END_REF] Several studies have also been performed regarding the swelling behavior of PHEMA-based materials in the presence of water. [START_REF] Mabilleau | Effects of the length of crosslink chain on poly(2-hydroxyethyl methacrylate) (pHEMA) swelling and biomechanical properties[END_REF][START_REF] Refojo | Hydrophobic interaction in poly(2-hydroxyethyl methacrylate) homogeneous hydrogel[END_REF][START_REF] Uzun | Porous poly(hydroxyethyl methacrylate) based monolith as a new adsorbent for affinity chromatography[END_REF] For PHEMA-based biporous networks, it has been evaluated by measuring the evolution of the water uptake during 4 h. Previously, Mabilleau et al. have notably highlighted the effect of the crosslinker backbone length on bulk PHEMA water uptake and thus swelling.

[38] PHEMA crosslinked with polyethylene diacrylate exhibited the highest value of water uptake, i.e. 56 %, while the lowest values were obtained for divinylbenzene-crosslinked

Conflict of interest

The authors declare that they have no conflict of interest.


Studzinski Filipo

Perotto

Sattar Vakili
email: sattar.vakili@mtkresearch.com

Pratik Gajane
email: pratik.gajane@unileoben.ac.at

Yaser Faghan
email: yaser.faghan@cemapre.pt

Mathieu Bourgais
email: mathieu.bourgais@insa-rouen.fr

Gambler Bandits and the Regret of Being Ruined Extended Abstract

Keywords: MAB, Ruin, Risk-Averse Decision Making, Safe RL

In this paper we consider a particular class of problems called multiarmed gambler bandits (MAGB) which constitutes a modified version of the Bernoulli MAB problem where two new elements must be taken into account: the budget and the risk of ruin. The agent has an initial budget that evolves in time following the received rewards, which can be either +1 after a success or -1 after a failure. The problem can also be seen as a MAB version of the classic gambler's ruin game. The contribution of this paper is a preliminary analysis on the probability of being ruined given the current budget and observations, and the proposition of an alternative regret formulation, combining the classic regret notion with the expected loss due to the probability of being ruined. Finally, standard state-of-the-art methods are experimentally compared using the proposed metric.

MAB AND MAGB

Multiarmed bandits (MAB) constitute a framework to model online sequential decision-making while facing the exploration-exploitation dilemma [START_REF] Lattimore | Bandit algorithms[END_REF][START_REF] Slivkins | Introduction to Multi-Armed Bandits[END_REF][START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF]. A MAB is typically represented by an agent interacting with a discrete random process (or a "slot machine") by choosing, at each round 𝑡, some action 𝐴 𝑡 = 𝑖 to perform among 𝑘 possible actions (or "arms"), then receiving a corresponding reward 𝑅 𝑡 . Because the complete information about the reward functions is not available, the agent must estimate them by sampling (i.e. by pulling the arms and observing the received rewards). In the standard stochastic setting [START_REF] Auer | Finite-time Analysis of the Multiarmed Bandit Problem[END_REF], the rewards originated from the same arm are independent but identically distributed, and observing an arm does not give any information about other arms. The objective is to maximize the expected sum of rewards over a potentially infinite time-horizon, finding a strategy that minimizes the expected regret (i.e. the cumulated difference between the rewards that could be obtained by always pulling the arm with highest mean, and the rewards the agent expects to receive following the given strategy). A good policy should guarantee sub-linear regret for any configuration of arms, i.e. the expected average regret per round must tend to zero asymptotically as time tends to infinity [START_REF] Agrawal | Sample Mean Based Index Policies with 𝑂 (log 𝑛) Regret for the Multi-Armed Bandit Problem[END_REF][START_REF] Bubeck | Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems[END_REF][START_REF] Garivier | KL-UCBswitch: optimal regret bounds for stochastic bandits from both a distributiondependent and a distribution-free viewpoints[END_REF].

In this paper, we define a particular MAB variation called multiarmed gambler bandits (MAGB), which constitutes a subclass of survival MAB [START_REF] Studzinski Perotto | Open Problem: Risk of Ruin in Multiarmed Bandits[END_REF]. A MAGB can be formally defined as a random process that exposes 𝑘 ∈ N + arms to an agent having an initial budget 𝑏 0 ∈ N + , which evolves in time with the received rewards, so that 𝐵 ℎ = 𝑏 0 + ℎ 𝑡 =1 𝑅 𝑡 . Let P = {𝑝 1 , • • • , 𝑝 𝑘 } be the set of parameters that regulate the underlying Bernoulli distributions from which the rewards 𝑅 𝑡 ∈ {+1, -1} are drawn. It means that, at each round 𝑡 ∈ N + , the agent executes an action 𝑖, which either increases its budget 𝐵 𝑡 by 1 with stationary probability 𝑝 𝑖 ∈ [0, 1], or decreases it by 1 with probability 1 -𝑝 𝑖 . The game stops when 𝐵 𝑡 = 0 happens for the first time (the gambler is ruined), but it can be occasionally played forever if the initial conditions allow the budget to increase infinitely.

When taken separately, each arm within a MAGB can be seen as an instance of a gambler's ruin game played against an infinitely rich adversary [START_REF] Stewart | Gambler's Ruin[END_REF][START_REF] Feller | An Introduction to Probability Theory and Its Applications[END_REF][START_REF] Prakash Gorroochurn | Huygens and the Gambler's Ruin (1657)[END_REF][START_REF] Song | A note on the history of the gambler's ruin problem[END_REF]. For that reason, the probability of surviving, playing the game forever, and never being ruined, having a current budget 𝐵 𝑡 , and repeatedly pulling arm 𝑖, is:

lim ℎ→∞ 𝜔 ℎ,𝑖 =        1 - 1-𝑝 𝑖 𝑝 𝑖 𝐵 𝑡 if 𝑝 𝑖 > 0.5 , 0 if 𝑝 𝑖 ≤ 0.5 . (1)
In contrast to the standard MAB, solving a MAGB involves a multi-objective optimization: in addition to minimizing the expected regret generated by the rounds when the best arm is not played (classic regret), the agent must also minimize the expected regret generated by the probability of being ruined. To analyze that, we define the notion of expected normalized relative regret ℓ ∈ [0, 1]:

ℓ ℎ,𝜋 = 𝜔 ℎ,𝜋 𝜔 * ℎ • ∑ 𝑘 𝑖=1 𝑝 * -𝑝 𝑖 𝑝 * • E[𝑁 𝑖,ℎ ] ℎ normalized classic regret + 𝜔 * ℎ -𝜔 ℎ,𝜋 𝜔 * ℎ regret due to ruin , (2)
where ℎ is the considered (potentially infinite) time-horizon, 𝑝 * and 𝑝 𝑖 are, respectively, the underlying parameters of the optimal arm and of arm 𝑖, E[𝑁 𝑖,ℎ ] is the number of rounds arm 𝑖 is expected to be pulled, and 𝜔 ℎ,𝜋 and 𝜔 * ℎ are the probability of surviving, respectively, following a given strategy 𝜋, or always playing the best arm. In finite-horizon experimental scenarios, after several independent repetitions, the expected normalized relative regret can be approximated empirically by averaging the normalized difference between the obtained final budget and the potentially best budget:

lℎ,𝜋 = 1 -𝐵 ℎ,𝜋 /𝐵 * ℎ . (3)

RELATED WORKS ON SAFE BANDITS

The search for safety guarantees is receiving increased attention within the reinforcement learning community [5, 9, 12, 14-16, 19, 25, 26, 42, 43, 54] and in particular concerning multiarmed bandits [START_REF] Garcelon | Conservative Exploration in Reinforcement Learning[END_REF][START_REF] Garcelon | Improved Algorithms for Conservative Exploration in Bandits[END_REF]. In an alternative version of the problem called risk-averse MAB [START_REF] Cassel | A General Approach to Multi-Armed Bandits Under Risk Criteria[END_REF][START_REF] Galichet | Exploration vs Exploitation vs Safety: Risk-Aware Multi-Armed Bandits[END_REF][START_REF] Maillard | Robust Risk-Averse Stochastic Multi-armed Bandits[END_REF][START_REF] Sani | Risk-Aversion in Multiarmed Bandits[END_REF][START_REF] Vakili | Decision Variance in Risk-Averse Online Learning[END_REF][START_REF] Vakili | Risk-Averse Multi-Armed Bandit Problems Under Mean-Variance Measure[END_REF][START_REF] Zhu | Thompson Sampling Algorithms for Mean-Variance Bandits[END_REF], the agent must take into account the expected variability on the expected rewards in order to identify (and avoid) unstable (then considered risky) actions, but without worrying about ruin, since the notion of budget is not considered. In this sense, the risk-reward trade-off can be tackled by using some risk-aware metric, such as the mean-variance, or the conditional value at risk. A MAGB cannot be reduced to the risk-averse setting due to the absence of notion of ruin, which leads to a simplified interpretation of safety as a synonym of reward constancy. In addition, in the Bernoulli case, both mean and variance are directly dependent on 𝑝. In another variation of the problem called conservative bandits [START_REF] Garcelon | Improved Algorithms for Conservative Exploration in Bandits[END_REF][START_REF] Wu | Conservative Bandits[END_REF], the agent knows, a priori, a default action with its underlying reward mean, and it is constrained to respect a threshold in the ongoing relative regret compared to that action.

In another modified version of the problem called budgeted MAB [START_REF] Agrawal | An efficient algorithm for contextual bandits with knapsacks, and an extension to concave objectives[END_REF][START_REF] Kook | Graph Sparsification in the Semi-streaming Model[END_REF][START_REF] Badanidiyuru | Bandits with Knapsacks[END_REF][START_REF] Dekel | Bandits with switching costs: 𝑇 2/3 regret[END_REF][START_REF] Ding | Multi-Armed Bandit with Budget Constraint and Variable Costs[END_REF][START_REF] Koren | Bandits with Movement Costs and Adaptive Pricing[END_REF][START_REF] Koren | Multi-Armed Bandits with Metric Movement Costs[END_REF][START_REF] Luedtke | Asymptotically optimal algorithms for budgeted multiple play bandits[END_REF][START_REF] Tran-Thanh | Epsilon-First Policies for Budget-Limited Multi-Armed Bandits[END_REF][START_REF] Tran-Thanh | Knapsack Based Optimal Policies for Budget-Limited Multi-Armed Bandits[END_REF][START_REF] Xia | Finite budget analysis of multi-armed bandit problems[END_REF][START_REF] Datong | Budget-Constrained Multi-Armed Bandits With Multiple Plays[END_REF], the player receives a reward but needs to pay a cost after pulling an arm, which is taken from a given initial budget. The process stops when the budget is over. In this setting, reward and cost are independent functions associated to each arm. The goal is to maximize cumulated rewards, constrained by a budget that limits the cumulated costs. The arm with best estimated reward-to-cost ratio should be preferred. Alternatively, the budget can be imposed only on a preliminary exploration phase [START_REF] Audibert | Best Arm Identification in Multi-Armed Bandits[END_REF][START_REF] Bubeck | Pure Exploration in Multi-armed Bandits Problems[END_REF][START_REF] Guha | Approximation algorithms for budgeted learning problems[END_REF][START_REF] Madani | The Budgeted Multiarmed Bandit Problem[END_REF], and the question is how to spend the budget efficiently in order to identify the best arm. A MAGB cannot be reduced to any of those budgeted settings due to the explicit separation between rewards and costs, which does not exist in a MAGB.

FINDINGS AND CONCLUSIONS

In the experimental setting, a MAGB with 𝑘 = 10 arms is instantiated, each one with a different parameter 𝑝 𝑖 , linearly distributed between 0.45 and 0.55 (i.e. half positive and half negative mean rewarded arms). The initial budget is set to 𝑏 0 = 𝑘 = 10, and the results are averaged after 2000 repetitions and over time-horizon ℎ = 5000. The standard UCB1 method [START_REF] Auer | Finite-time Analysis of the Multiarmed Bandit Problem[END_REF][START_REF] Maillard | A Finite-Time Analysis of Multi-armed Bandits Problems with Kullback-Leibler Divergences[END_REF] is compared with other state-ofthe-art MAB algorithms, namely KL-UCB [START_REF] Garivier | The KL-UCB Algorithm for Bounded Stochastic Bandits and Beyond[END_REF], Bayes-UCB [START_REF] Kaufmann | On Bayesian Upper Confidence Bounds for Bandit Problems[END_REF], and Thompson-Sampling [START_REF] Agrawal | Near-Optimal Regret Bounds for Thompson Sampling[END_REF][START_REF] Kaufmann | Thompson Sampling: An Asymptotically Optimal Finite-Time Analysis[END_REF][START_REF] Korda | Thompson Sampling for 1-Dimensional Exponential Family Bandits[END_REF], which have proven to asymptotically achieve logarithmic regret for Bernoulli arms, matching the accepted theoretical lower bound [START_REF] Lai | Asymptotically Efficient Adaptive Allocation Rules[END_REF], but also with classic naive sub-optimal methods, namely Empirical-Means and 𝜀-greedy [START_REF] Lattimore | Bandit algorithms[END_REF][START_REF] Sutton | Reinforcement Learning: An Introduction[END_REF][START_REF] Vermorel | Multi-armed Bandit Algorithms and Empirical Evaluation[END_REF], and with an original simple heuristic called Empirical-Sum, which chooses, at each round, the arm with highest observed sum of rewards. Finally, some fixed arm policies are included, namely Best-Arm (always pull the arm with highest mean), Worst-Arm (the arm with lowest mean), Worst-Positive-Arm (the arm with lowest 𝑝 greater than 0.5), and Best-Negative-Arm (the arm with highest 𝑝 lower than 0.5).

The methods are compared considering their survival rate, defined by the proportion of episodes that run without ruin until the predefined time-horizon, and considering their empirical normalized relative regret, given by Eq. ( 3), as shown in the Figure 1. UCB1 presents a heavy regret due to its conservative behavior, which leads to intense exploration during the initial rounds, and often to ruin. The naive methods (Empirical-Means, Empirical-Sum, and 𝜀-Greedy), which are classically sub-optimal, present better survival rates against the classically optimal algorithms (Bayes-UCB, Thompson-Sampling, and KL-UCB), which finally allows them to present better relative regret. In conclusion, taking the overall performance together, mixing the regret caused by sub-optimal choices (i.e. the regret in classic terms) and the regret caused by ruin, upsets the standard insights and strategies concerning MAB. Intuitively, an algorithm for minimizing this alternative kind of regret must carefully coordinate the remaining budget with the confidence on the estimated distributions, seeking for minimizing the probability of ruin when the budget is relatively low, and gradually becoming classically optimal, as the budget increases. Future works must include a more comprehensive set of experimental scenarios, a theoretical analysis about the regret bounds of the selected algorithms, and the extension of this survival setting to Markovian decision processes.

Figure 1 :

 1 Figure 1: Survival rates and average empirical relative normalized regrets, 𝑛 = 2000 episodes, time-horizon ℎ = 5000.

Preprint -Proc. of the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2021), U. Endriss, A. Nowé, F. Dignum, A. Lomuscio (eds.), May 3-7, 2021, Online. © 2021 International Foundation for Autonomous Agents and Multiagent Systems (www.ifaamas.org). All rights reserved.


Evan Dufraisse
email: edufraisse@gmail.com

Philippe Leray
email: philippe.leray@ls2n.fr

Raphaël Nedellec
email: rnedellec@talend.com

Tarek Benkhelif
email: tbenkhelif@talend.com

INTERACTIVE ANOMALY DETECTION IN MIXED TABULAR DATA USING BAYESIAN NETWORKS Interactive Anomaly Detection in Mixed Tabular Data using Bayesian Networks

Keywords: Bayesian Networks, Anomaly Detection, Outlier Detection, Functional Dependencies, Mixed Data 1. https://data.world/tarek-benkhelif/nflplayers

The last decades improvements in processing abilities have quickly led to an increasing use of data analyses implying massive data-sets. To retrieve insightful information from any data driven approach, a pivotal aspect to ensure is good data quality. Manual correction of massive data-sets requires tremendous efforts, is prone to errors, and results being really costly. If knowledge in a specific field can often allow the development of efficient models for anomaly detection and data correction, this knowledge can sometimes be unavailable and a more generic approach should be found. This paper presents a novel approach to anomaly detection and correction in mixed tabular data using Bayesian Networks. We present an algorithm for detecting anomalies and offering correction hints based on Jensen scores computed within the Markov Blankets of considered variables. We also discuss the incremental corrections of detection model using user's feedback, as well as additional aspects related to discretization in mixed data and its effects on detection efficiency. Finally we also discuss how functional dependencies can be managed to detect errors while improving faithfulness and computation speed.

Introduction

When considering tabular data, anomalies (also refered as outliers in the literature) are often thought as odd and isolated points within a multi-dimensional space. This has led many anomaly detection algorithms to be designed using exclusively space-based approaches relying on distance and density measures within neighborhoods of data points. It has been shown by [START_REF] Lu | Effective outlier detection based on Bayesian network and proximity[END_REF] that the detection of probabilistic dependencies violations using Bayesian Networks was complementary to space-based approaches and could allow the detection of a broader range of anomalies when combined. The idea of combining anomaly detection algorithms isn't new and ensemble techniques are already used as an attempt to join strengths and reduce bias of different algorithms [START_REF] Aggarwal | Outlier Ensembles[END_REF]. In this respect, considering the core difference and complementarity between distance-based and probabilistic-based detection algorithms, it seems both natural and essential to combine these approaches.

Functional dependencies are not rare when dealing with tabular data. This kind of dependency can be useful for detecting anomalies, but also problematic if we try to learn probabilistic dependencies. Our objective is to interactively detect and correct anomalies by using both functional and probabilistic dependencies facing mixed data. Several approaches have been described for continuous or categorical data, but none to our knowledge tries to tackle the case of mixed data.

In former studies, various scoring methods have been employed to identify anomalies : direct likelihood ranking of individuals [START_REF] Cansado | Unsupervised anomaly detection using bayesian networks[END_REF], patterns recognition from manually defined thresholds [START_REF] Babbar | Mining causal outliers using Gaussian Bayesian networks[END_REF] and normalized likelihood-based scores [START_REF] Kirk | Anomaly detection and attribution using Bayesian networks[END_REF][START_REF] Rashidi | Anomaly detection in categorical datasets using bayesian networks[END_REF]. Manual threshold pattern recognition suffers from the need of defining rules on what is anomalous and direct likelihood estimation is prone to detect "rare" rather than "incoherent" individuals. We thus decided to use a normalized likelihood-based score similar to what [START_REF] Kirk | Anomaly detection and attribution using Bayesian networks[END_REF] and [START_REF] Rashidi | Anomaly detection in categorical datasets using bayesian networks[END_REF] mention, and often called Jensen score. We offer a new usage of this score by defining a different exploration strategy built around an extended definition of Markov blankets for a particular Bayesian Network construction. Our article also distinguishes itself by presenting a way of integrating functional dependencies and adding value through interactivity with an expert in the process of anomaly detection.

When used with Bayesian Networks, mixed data is generally subjected to a discretization step for which several uni-modal or multi-modal algorithms exist [START_REF] Boullé | MODL: A Bayes optimal discretization method for continuous attributes[END_REF][START_REF] Mabrouk | An efficient Bayesian network structure learning algorithm in the presence of deterministic relations[END_REF][START_REF] Chen | Learning discrete Bayesian networks from continuous data[END_REF]. Inspired by the work of [START_REF] Mabrouk | An efficient Bayesian network structure learning algorithm in the presence of deterministic relations[END_REF], we've designed an new uni-modal discretization process to be used for anomaly detection networks.

Motivated by the large presence of pairwise functional dependencies in today's databases, we mainly focused on the treatment of this type of dependence. For this aspect, we rely on the results of [START_REF] Rahier | Bayesian networks for static and temporal data fusion[END_REF], who described a way of learning a Bayesian network when facing data including functional dependencies, by first detecting these dependencies, and then by restricting a discrete Bayesian network structure learning algorithm to a subset of functionally independent variables to discover their probabilistic dependencies. Other publications focus on the discovery of functional dependencies involving more variables [START_REF] Caruccio | Relaxed functional dependencies-a survey of approaches[END_REF].

Section 2.2 offers a general overview of our interactive anomaly detection system dealing with mixed tabular data. Section 2.3 describes the way we decide to deal with mixed data by discretizing continuous variables. Section 2.4 is dedicated to functional dependencies detection and their use for identifying a first set of anomalies. Once functional dependencies have been processed, section 2.5 presents the hybrid Bayesian network we learn to capture the probabilistic dependencies with original variables. In section 2.6, we give details about our anomaly detection approach based on an adapted use of the Jensen score to detect anomalies in extended Markov blankets of our hybrid Bayesian network. Section 2.7 briefly deal with the model update when the expert decides to correct values in the data-set. Section 3 is dedicated to the empirical evaluation of our proposition. Section 4 concludes on the contribution of this paper and our perspectives.

Contribution

Notations

Let us introduce the notations used in the following sections. Our algorithm is dealing with n random variables, with l discrete variables {X 1 ,... , X l } and n -l continuous ones { Xl+1 ,... , Xn }. {X l+1 ,... , X n } will denote the discretized counterpart of these continuous variables.

xi will represent an instance of a continuous variable, when x i will represent its discretized value, or the value of a discrete variable. V al(X i ) will denote the set of possible values for the discrete variable X i .

Figure 1: Architecture of our interactive anomaly detection system dealing with mixed data. X = {X 1 ,... , X l , Xl+1 ,... , Xn } is the set of original variables, X = {X 1 ,... , X l , X l+1 ,... , X n } is the set of the discrete and discretized variables, and X = {X 1 ,... , X l , Xl+1 ,... , Xn , X l+1 ,... , X n } is the original set of variables extended with the discretized ones.

D and D respectively denote the original mixed data-set (over X) and the discretized one (over X), and N is the number of instances in the data-set.

X|s , X |s , X|s will correspond to subsets of variables, with the corresponding datasets D|s , D |s .

Overview

Figure 1 presents the general overview of our interactive anomaly detection system dealing with mixed tabular data. As explained in introduction, and represented in the step 0 in Figure 1, we assume that distancebased errors, which can't be detected using probabilistic-based approaches, have been detected and corrected as a pre-processing step. Removing non-informative variables prior to execution can also speed up the process depending on the implementation.

Our proposition really starts with the discretization of the considered mixed data-set in step 1. We proceed here to an unimodal discretization that will be described in section 2.3.

Once the data-set is discretized, we detect in step 2 functional dependencies which could exist within the data-set. As part of this study we are only considering functional dependencies between pairs of variables, as presented in section 2.4. At the end of this step, an outlier detection and correction can be performed based on functional dependencies only. Finally, one subset of "functionally independent" variables is selected.

In step 3, we then learn the structure of a Conditional Linear Gaussian Bayesian Network (CLGBN) capturing all the probabilistic dependencies between the previously selected variables. Section 2.5 gives more details about this step.

Step 4 is dedicated to the parameter learning of this Bayesian network, from the original dataset, or its update when the dataset will be corrected in step 6 (section 2.7).

The anomaly detection task (step 5) presented in section 2.6 uses probabilistic dependencies previously discovered to score and rank potential anomalies and offer hints of correction.

Step 6 is the interactive part of the process, where some detected and ranked anomalies are presented to one expert in charge of validating or not the proposition in order to correct the anomalies. Based on the expert's correction choices, probability tables can be updated in step 4 and the process can iterate until no more anomalies are validated by the expert.

Discretization

In the context of mixed data, discretization is in itself an important subject in Bayesian network learning and more generally in Machine Learning. Using an arbitrary discretization by frequency or by even intervals generally leads to sub-optimal networks. Even though this discretization arbitrariness can be reduced using multi-modal approaches [START_REF] Boullé | MODL: A Bayes optimal discretization method for continuous attributes[END_REF][START_REF] Mabrouk | Multivariate cluster-based discretization for Bayesian network structure learning[END_REF][START_REF] Chen | Learning discrete Bayesian networks from continuous data[END_REF], these often come at a considerable computing cost.

For this reason, in this first version of our anomaly detection system, we use a simple uni-modal discretization approach based on the Expectation-Maximization algorithm [START_REF] Dempster | Maximum likelihood from incomplete data via the EM algorithm[END_REF][START_REF] Mclachlan | The EM algorithm and extensions[END_REF]. For each continuous variable Xi ∈ { Xl+1 ,... , Xn } we learn a mixture of G i ≥ 2 Gaussians and discretize them by assigning the label of the most explaining Gaussian to entry i of every instances x.

In order to prevent over-fitting, the number of Gaussians G i for each mixture is determined by applying an usual stepwise greedy search to optimize the BIC criterion, but some other criteria can be used [START_REF] Celeux | Model Selection for Mixture Models-Perspectives and Strategies[END_REF].

The parameters of the Gaussian distributions have been initialized by using a classical grid initialization between the bounds of the considered variable for the mean and a constant standard deviation.

Functional Dependencies

A functional dependency between one variable X i and a set of variables W is usually defined by the fact that for every valid instance of W , that instance of W uniquely determines the value of X i . This can be seen as a special case of probabilistic dependency and be detected if the conditional entropy H(X i |W ) = 0.

Presence of functional dependencies are known to render conditional independence tests between variables unreliable, and thus would lead to unfaithful graph structures when learning our Bayesian network in the next step [START_REF] Luo | Learning bayesian networks in semi-deterministic systems[END_REF][START_REF] Rodrigues De Morais | Handling almost-deterministic relationships in constraint-based Bayesian network discovery : Application to cancer risk factor identification[END_REF][START_REF] Mabrouk | An efficient Bayesian network structure learning algorithm in the presence of deterministic relations[END_REF][START_REF] Rahier | Bayesian networks for static and temporal data fusion[END_REF]. For this reason, we offer to detect in this step functional dependencies and correct found functional violations.

Being interested in detecting functional dependencies in the context of anomaly detection, we need to expand our definition of functional dependencies to what is often defined as "quasideterminism" or "relaxed functional dependencies", so that potential errors don't prevent their detection. Relaxed functional dependencies are functional dependencies that almost hold except on a "small" subset of entries. Several algorithms have been published for relaxed dependencies mining [START_REF] Caruccio | Relaxed functional dependencies-a survey of approaches[END_REF][START_REF] Bleifuß | Approximate discovery of functional dependencies for large datasets[END_REF][START_REF] Caruccio | Mining relaxed functional dependencies from data[END_REF].

In our architecture, we propose to use the results of [START_REF] Rahier | Bayesian networks for static and temporal data fusion[END_REF] which offer a simple approach to detect and cluster pairwise quasi-deterministic dependencies between discrete variables by relying on conditional entropy computations. The algorithm essentially considers each pair of variables (X i , X j ) and check if the conditional entropy is inferior to a given . In short, if H(X i |X j ) < , X j is a "functional" parent candidate for X i . [START_REF] Rahier | Bayesian networks for static and temporal data fusion[END_REF] finally structures the functional dependencies discovered as a spanning forest F where each X i has at most one "functional" parent P a F (X i ). Variables without "functional" parents are called roots, and are functionally independent. We will denote later R this set of roots, X |R the subset of functionally independent discrete variables and X|R the corresponding subset in the original variables.

After their detection, we learn the CPD P (X i |P a F (X i )) of each functional dependency present in F and we create a Boolean function f i that allows the identification of values (x i , x j ) of (X i , P a F (X i )) that violate the quasi-deterministic functional dependency discovered.

f i (X i = x i , P a F (X i ) = x k ) = 1 -δ x i ,x k i (1)
where

x k i = argmax x i ∈V al(X i ) P (X i = x i |P a|F (X i ) = x k ) (2)
For each instance in D we then check the values of f i functions, whenever a f i returns true we report the error to the user along with the involved variables (X i = x i , P a F (X i ) = x k ) and ask for a correction of this anomaly.

CLGBN structure Learning

The objective of this section is to learn a Conditional Linear Gaussian Bayesian Network (CLGBN, Lauritzen and Wermuth 1989) that captures the probabilistic dependencies between variables X|R , i.e. once the functional dependencies have been processed in section 2.4. Restricting ourselves to this subset of variables ensures the absence of functional dependencies and allows to maintain the same amount of information while reducing the number of variables.

Rather than learning directly this CLGBN from the mixed data, we present a two-step alternative process inspired from the work of [START_REF] Mabrouk | An efficient Bayesian network structure learning algorithm in the presence of deterministic relations[END_REF] and relying on the Gaussian mixtures found during discretization step (section 2.3). First, a simple discrete Bayesian Network structure is learned on X |R , the discretized version of the variables This structure learning step is generic and can be made using any preferred algorithm.

Second step consists in creating the final CLGBN over variables X|R by augmenting the discrete Bayesian Network learned over X |R with the continuous variables present in X|R and adding a conditional probabilistic dependency P ( Xi |X i ) for each of these continuous variables. These conditional distribution will be defined with the Gaussian mixtures learned in section 2.3.

The obtained CLGBN contains both the probabilistic dependencies between the continuous variables and their discrete counterparts, and the probabilistic dependencies discovered between all the discrete or discretized variables.

Anomaly Detection and Correction

Anomaly detection using Bayesian Network whether on continuous/discrete data or temporal series has been subject of several publications [START_REF] Kirk | Anomaly detection and attribution using Bayesian networks[END_REF][START_REF] Das | Detecting anomalous records in categorical datasets[END_REF][START_REF] Rashidi | Anomaly detection in categorical datasets using bayesian networks[END_REF][START_REF] Babbar | Mining causal outliers using Gaussian Bayesian networks[END_REF]. Some approaches have been based solely on reporting individuals with lowest likelihoods of occurrence, others on defining manually patterns rules to identify anomalous individuals [START_REF] Babbar | Mining causal outliers using Gaussian Bayesian networks[END_REF]. Our detection approach distinguishes itself both from the use of the CLGBN learned in the previous section and from the application of an anomalousness scoring that considers Markov Blanket of variables.

MARKOV BLANKET BASED SCORING OF ANOMALOUSNESS

As part of this architecture, we use a scoring method defined in eqn 3 relying on a normalized likelihood score called the Jensen score [START_REF] Jensen | Bayesian networks and decision graphs[END_REF][START_REF] Kirk | Anomaly detection and attribution using Bayesian networks[END_REF][START_REF] Rashidi | Anomaly detection in categorical datasets using bayesian networks[END_REF].

J(x 1 , x 2 , ..., x n ) = log( P (x 1 )P (x 2 )...P (x n ) P (x 1 , x 2 , ..., x n ) ) (3)
The intuition behind the use of this score is the following. Given a conjunction of variables and their values, if the probability of occurrence of this conjunction is inferior to the product of their marginal probabilities of occurrence, we can flag the conjunction as incoherent and should report it as abnormal to the user. In other words, a conjunction of values must have a Jensen score below zero to be considered coherent.

In a Bayesian network defined over a set of variables X, the Markov Blanket of one node X i is by definition the set of variables M B(X i ) given which X i becomes conditionally independent from X \ M B(X i ). It means that when the values of X are all observed, the values of M B(X i ) are the only ones that are necessary to estimate the posterior probability of X i . A side effect is that we can estimate J(X i , M B(X i )) to determine if there is a probabilistic anomaly between the value observed for X i and the observation of its Markov blanket.

In our CLGBN, the discretized variables present in X|R are unobserved. So, we propose to modify this Markov blanket by replacing the discretized (and unobserved) nodes {X j } present in {M B(X i ) ∪ X i } by their continuous counterpart { Xj } in order to obtain an extended Markov blanket eM B(X i ).

In this definition, we are considering each information given by the observation of continuous variable Xj = xj as a soft evidence P (X j |x j ) sufficient to determine the posterior distribution of its discretized variable X j , whatever the values of other variables dependent with X j (as defined by [START_REF] Valtorta | Soft evidential update for probabilistic multiagent systems[END_REF], by opposition to the virtual evidence defined by Pearl (1988). The use of this soft evidence is also a way to reduce the arbitrariness of the previous discretization step by not using directly the discretized value as an evidence.

By this way, this extended Markov blanket eM B(X i ) contains all the necessary information to estimate the posterior distribution of X i .

With this reasoning, we propose to compute J(eM B(X i ) = emb i ) to determine if there is a probabilistic anomaly between the value observed for X i and the observation of its extended Markov blanket, where emb i is the observation of eM B(X i ).

DETECTION, RANKING AND CORRECTION PROPOSAL

For each variable X i in the original mixed variable set (restricted to the functionally independent variables) X|R the scoring of anomalousness J(eM B(X i ) = emb i ) for a observed configuration emb i is computed for all the N instances of the dataset and normalized (cf. eqn 4).

j(eM B(X

i ) = emb i ) = J(eM B(X i ) = emb i ) -E[J(eM B(X i ))] std(J(eM B(X i ))) (4)
As explained in section 2.6.1, only the instances mb i with a positive value of the Jensen score J(eM B(X i ) = mb i ) > 0 are considered and detected as anomalies. In order to rank these anomalies even if they are raised by different eM B(X i ), we propose to rank them with respect to their normalized scoring function j(eM B(X i ) = emb i ).

One anomaly can be detected in several eM B(X i ), but the more important is that one of them is proposed to the expert in order to correct it. This potential anomaly is then proposed to the expert as a set of variables and their observed values eM B(X i ) = emb i . The expert is then in charge of looking at all these variables and values and checking what pair of variable and value (X j , x j ) could be the anomaly (where X j belongs to eM B i ). We decide here to help the expert by proposing him the posterior distribution P (X j | {emb i \ x j }) of each X j candidate (or its discretized counterpart) without its potential abnormal value x i .

Model updating

With the list of information proposed to him in the previous section, the expert can decide to correct one variable X j with a new value x * j instead of the previous value x j . We then proceed with the change of our CLBGN parameters that are dependent with this variable by updating the sufficient statistics classically used and stored during the parameter estimation phase. We make here the assumption, that such a small change in the dataset (and the repetition of these small changes caused by anomaly correction) will not change the underlying structure of our model.

Experiments

Datasets and implementation

For these experiments, we used the NFL2016 dataset 1 previously pre-processed which contains here about 2330 NFL players entries. Information includes both continuous variables (Jersey number, Age, Height, Weight, ...) and discrete ones (Position, Position-Group, Side, ...).

We know that three functional dependencies are present in this data-set (Position → Position-Group, Years → Level and Position-Group → Side).

Even if we don't know exactly what entries in this data-set have anomalies, we have selected this data-set because of the existence of several external sources about NFL players that allows us to fact-check all the possible anomalies detected.

We are currently working with several other data-sets, but fact-checking takes a long time, so we decided to provide here only results for this NFL data-set. As we don't know exactly where are all the possible anomalies in this data-set, the only performance measurements we can use are True and False Positive (TP, FP), after our fact-checking step.

The implementation of our architecture has been performed with our C++ library dedicated to Probabilistic Graphical Models, PILGRIM2 , with the help of ProBT3 library for handling CLGBNs.

The BN structure learning used during step 3 is our implementation of the classical Greedy search algorithm initially proposed by [START_REF] Chickering | Learning Bayesian networks: Search methods and experimental results[END_REF], or an exact search with BIC scoring

Experimental design and results

PROBABILISTIC DETECTION OF ANOMALIES

In this experiment, we've decided to run our system without Step 2 (related to functional dependencies) in order to detect, rank and propose anomalies only with the help of our CLGBN.

Among the 2330 instances (NFL players), 364 anomalies have been detected (with some anomalies related to the same player) and reported to the expert. After fact-checking the information related to each anomaly detected, 131 detections are real anomalies, 20 detections are unsure ones, 140 are false detection, and 73 are considered as "surprising" instances, i.e. unusual values, but corresponding to the reality. In Figure 2, we can also see that we are able to detect real anomalies from both discrete (Position) and continuous (Jersey and Weight) variables.

Even if we know that this dataset includes some functional dependencies, we were able in this first experiment to illustrate the interest of the use of our CLGBN in order to detect anomalies in discrete or continuous framework.

INTEREST OF THE EXTENDED JENSEN SCORE

This second experiment is the same as the previous one but with a specific focus on the Jensen score values. The anomalies detected by our Jensen scores are filtered with respect to a given threshold.

Figure 3 shows us the evolution of True and False positive anomaly detection when focusing on the extended Markov blanket of Jersey variable. We can see that our scoring function is helpful for ranking anomalies. By increasing the detection threshold, only a fraction of the previous real anomalies (T P max) are detected but the quality of the detection (ratio T P/(T P + F P )) increases.

Figure 4 presents the distribution of the (un-normalized) Jensen score computed on the extended Markov blanket for two variables Height and Weight, for real and false detections. Confirming the previous results, this Jensen score helps to detect real anomalies (the higher this score is, the more True positives we have). But, by comparing distributions between both variables, we can see that our score distributions are not comparable between two different extended Markov blankets. This observation confirms the interest of using the normalized version described in eq. 4. In this third experiment, we run once again our anomaly detection system, by also taking into account functional dependency detection. As a first result, the three expected functional dependencies are detected in our dataset : Position → Position-Group → Side and Years → Level. No anomalies were detected during this phase, which is coherent with the results of our previous experiments where the 131 real anomalies discovered were not involving variables present in the existing functional dependencies.

The CLGBN obtained during step 3 is described in Figure 5 with or without taking into account the functional dependency detection. We can notice that both structures capture similar probabilistic dependencies, except the one between Level and Jersey. As partially observed in figure 2, some anomalies have been detected with the first CLGBN regarding Level Markov blanket. All these anomalies where False Positive. Our new CLGBN doesn't contain this dependency and those False Positive are no longer detected.

Without any filtering or action from the user, by using only our scoring function, 920 anomalies would be detected, which is about 3 times the number of detection made in the previous experiment.

We identified that 126 of the 131 FP were rightly identified, 14 of the 20 unsure and 192 of the previous 213 FP. But more than 480 detections have been made scoring the eM B(Height), whereas our first experiment only detected about 40 potential anomalies for it. This surprising result

Conclusion

We proposed in this paper an architecture dedicated to interactive anomaly detection system dealing with mixed tabular data, and taking into account both functional dependencies and probabilistic ones for anomaly detection. As functional dependencies are known to perturb Bayesian network structure learning algorithms, we first detect an correct them independently. We deal with mixed data by discretizing continuous variables. We then detect functional dependencies prior to BN structure learning, considering only discrete and discretized variables. We soften the effect of discretization boundaries for the anomaly detection phase by extending the learned BN with direct dependencies between discretized variables and their continuous counterparts. This Continuous Linear Gaussian BN is finally used for detecting probabilistic anomalies. For this task, we adapted existing Jensen score to focus on specific subsets of variables, taking into account Markov blankets both for detection and correction hints.

In our first experiments with NFL dataset, we were able to show that even the probabilistic part of our architecture is able to detect real anomalies in a dataset. The same experiments confirmed the interest of our extended Jensen score, and its normalized version. Another experiment highlighted the importance of functional dependencies, and the fact we have to resort to an improved Bayesian Network structure learning algorithm or an exact one when the number of variables is low, to ensure the quality of the Markov blanket.

This work is the first step of the development of our architecture, with several perspectives. In a very short term, we intend to perform larger experiments with other datasets in order to consolidate the interest of our proposition.

We are also planning to work on improvements of our discretization process, for instance with a multivariate process intertwined with the BN learning [START_REF] Mabrouk | Multivariate cluster-based discretization for Bayesian network structure learning[END_REF] or by using mixtures of truncated Gaussians [START_REF] Gonzales | Dealing with continuous variables in graphical models[END_REF] instead of Gaussians in order to avoid spurious discretization effects.

Detecting more complex functional dependencies could also be interesting, even though we know we will face complexity issues.

Finally, this architecture can also be improved by working on the interaction with the expert, improving his guidance and better using his feedback to score anomalies.

Figure 2 :

 2 Figure 2: Distribution of detection and real anomalies detected among the variables

Figure 3 :

 3 Figure 3: Evolution of detection performances related to Jersey variable when Jensen scores are filtered with respect to a given threshold.

Figure 4 :

 4 Figure 4: Comparison of (un-normalized) Scores J(eM B) distribution for variables Height and Weigth for real (TP) and false detections

Figure 5 :

 5 Figure 5: CLGBN obtained during step 3, without taking into account functional dependencies (left) and by taking into account the functional dependencies (right). The dotted dependencies added in the right figure correspond to the functional dependencies

http://pilgrim.univ-nantes.fr/

https://www.probayes.com/