Jérémy Chopin
email: jeremy.chopin@univ-angers.fr

Jean-Baptiste Fasquel
email: jean-baptiste.fasquel@univ-angers.fr

Harold Mouchère
email: harold.mouchere@univ-nantes.fr

Rozenn Dahyot
email: rozenn.dahyot@tcd.ie

Isabelle Bloch
email: isabelle.bloch@telecom-paris.fr

Semantic image segmentation based on spatial relationships and inexact graph matching

Keywords: Computer vision, Deep learning, Inexact graph matching, Quadratic assignment problem

We propose a method for semantic image segmentation, combining a deep neural network and spatial relationships between image regions, encoded in a graph representation of the scene. Our proposal is based on inexact graph matching, formulated as a quadratic assignment problem applied to the output of the neural network. The proposed method is evaluated on a public dataset used for segmentation of images of faces, and compared to the U-Net deep neural network that is widely used for semantic segmentation. Preliminary results show that our approach is promising. In terms of Intersection-over-Union of region bounding boxes, the improvement is of 2.4% in average, compared to U-Net, and up to 24.4% for some regions. Further improvements are observed when reducing the size of the training dataset (up to 8.5% in average).

I. INTRODUCTION

Deep learning has shown its efficiency in many fields [START_REF] Goodfellow | Deep Learning. Cambridge[END_REF], in particular for semantic segmentation of images in computer vision [START_REF] Garcia-Garcia | A survey on deep learning techniques for image and video semantic segmentation[END_REF]. One limitation of deep neural network approaches concerns the requirement of a large and representative training dataset, including its annotations, as recently highlighted in [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF]. Moreover, deep learning is intrinsically based on embedded information at pixel level, which is then processed and combined through different layers involving multiple parameters to be optimized.

This type of approach ignores the structural information that can be observed at a high level. This structural information can for example concern spatial relationships between different spatial entities, as illustrated in Figure 1 with the relative positions between the main regions of the face observed in the annotated image. This type of high-level structural information is often ignored in semantic image analysis, despite its potential illustrated by related works, integrating spatial, inclusion or even photometric relations [START_REF] Bloch | Fuzzy sets for image processing and understanding[END_REF]- [START_REF] Nempont | A constraint propagation approach to structural model based image segmentation and recognition[END_REF], often applied to medical This research was conducted in the framework of the regional programme Atlanstic 2020, Research, Education and Innovation in Pays de la Loire, supported by the French Region Pays de la Loire and the European Regional Development Fund.

imaging [START_REF] Colliot | Integration of fuzzy spatial relations in deformable models -application to brain MRI segmentation[END_REF]- [START_REF] Moreno | Using anatomical knowledge expressed as fuzzy constraints to segment the heart in CT images[END_REF]. This information is commonly represented using graphs, where vertices correspond to regions, and edges carry the structural information. Then, recognition turns into a general graph matching problem [START_REF] Fasquel | An approach for sequential image interpretation using a priori binary perceptual topological and photometric knowledge and k-means based segmentation[END_REF], [START_REF]A graph based image interpretation method using a priori qualitative inclusion and photometric relationships[END_REF], [START_REF] Lezoray | Image Processing and Analysis with Graphs: Theory and Practice[END_REF]. Fig. 1. Semantic segmentation and spatial relationships. Images are extracted from the public FASSEG database [START_REF] Khan | Multi-class semantic segmentation of faces[END_REF], [START_REF] Khan | Head pose estimation through multi-class face segmentation[END_REF]. a) Initial image. b) Semantic segmentation where some distinct regions belong to the same class (e.g. eyes). c) Semantic segmentation where each region belong to a specific class (e.g. left eye, right eye). d) Spatial relationships modeled by a graph where each vertex corresponds to a specific region of c: background, hair, left/right eyebrow, left/right eye, nose, mouth. Edges carry relationships, corresponding to distances between regions in our case. For the sake of clarity, only some of the edges of the complete graph are displayed.

In this context, we propose to combine two approaches: deep neural networks that have proven very efficient but often require large training datasets, and graphs for encoding high level relational representations in the visual scenes. Of note the use of deep neural networks for graph matching is a topic that is currently attracting a lot of interest from the scientific community, for problems other than computer vision such as biology, social sciences, linguistics [START_REF] Gao | Graph U-Nets[END_REF]- [START_REF] Li | GCN-LASE: Towards adequately incorporating link attributes in graph convolutional networks[END_REF].

The originality of the proposed method is that it allows for such a combination, formulated as an inexact graph matching problem applied to the output of deep neural networks. It enables to correct the semantic segmentation resulting from the single use of the probability map produced by the neural networks, by taking into account spatial relationships observed in the annotated database. The use of the global spatial structure For the sake of clarity, although graphs are complete, only some edges are reported. Vertex color correspond to colored region labels. Semantic segmentation: the neural network produces a semantic segmentation, possibly with artefacts (e.g. small bright region within the eyebrow and dark region on the neck). A graph Gr is then built from this segmentation and matched with the model graph Gm. This inexact graph matching is achieved in two steps. First, correctly segmented regions are retrieved (one-to-one matching), artefacts being ignored (two remaining vertices in this example). Secondly, remaining artefact regions are matched (many-to-one matching): both artefacts are correctly relabelled (see surrounded areas).

of the scene also enables to be less sensitive to the diversity (and therefore the size) of the training dataset used to train the neural network. In order to manage spatial relationships in an explicit manner, we consider a graph matching approach formulated as a quadratic assignement problem (QAP) instead of a neural-networks-based approach. The proposed method is detailed in Section II. Preliminary experiments illustrating the potential of this combination are presented in Section III. Section IV concludes the paper with a discussion.

II. COMBINING DEEP LEARNING AND STRUCTURAL

KNOWLEDGE

Figure 2 gives an overview of the proposed method, using, for illustration purposes, images considered in the experiments. Using an annotated training dataset, the deep neural network is trained to perform semantic segmentation. Moreover, using annotated images only, spatial relationships between the different regions are measured (e.g. average of measured distances), leading to the model graph G m : vertices and edges correspond to annotated regions and spatial relationships, respectively.

When processing an unknown image, the neural network provides a segmentation proposal, from which a hypothesis graph G r is constructed (see G r in Figure 2). This hypothesis graph is then matched with the model graph (two steps inexact graph matching, as illustrated in Figure 2). The purpose is to match the vertices (and thus the underlying regions) produced by the neural network with those of the model, involving the relabelling of some regions (many-to-one matching in Figure 2). This produces a final semantic segmentation corresponding to the high-level relations observed in the training dataset.

We detail hereafter the step of construction of the hypothesis graph from the output of the deep neural network (Section II-A) and then its matching with the model graph (Section II-B).

A. Graph construction

The input image is processed by the neural network which produces, as output, a tensor S ∈ R I×J×C with I the width (in pixels) of the image, J the height (in pixels) of the image and C the total number of classes. At pixel location (i, j), the value S(i, j, c) ∈ [0, 1] is the probability of belonging to each class considered in the segmentation, with the constraints :

(∀c = 1, . . . , C, 0 ≤ S(i, j, c) ≤ 1) ∧ C c=1 S(i, j, c) = 1
The segmentation map L * selects the label c of the class with the highest probability:

∀(i, j) ∈ {1, . . . , I} × {1, . . . , J}, L * (i, j) = arg max c∈{1,...,C} S(i, j, c) (1)
From this segmentation map, we define a set R of all resulting connected components (see Figure 3, where

R = {R 1 , • • • , R 4 }). We also define a set R * = {R * 1 , . . . , R * C }, where, for each class c ∈ {1, • • • , C}, R *
c is a set of regions corresponding to the connected components belonging to the class c according to the neural network (see Figure 3, where

R * = {R * 1 , . . . , R * 3 }).
This set R * is used for constraining the graph matching as described in Section II-B1.

From the set R, a structural representation is built and modeled by the graph G r = (V r , E r , A, D), where V r is the set of vertices, E r the set of edges, A a vertex interpreter and D an edge interpreter. Each vertex v ∈ V r is associated to a region R v ∈ R with an attribute, provided by the function A, which is the average membership probability vector over the set of pixels p = (i, j) composing R v , therefore computed on the initial S tensor (see Figure 3):

∀v ∈ V r , c ∈ {1, . . . , C}, A(v)[c] = 1 |R v | (i,j)∈Rv S(i, j, c) (2)
We consider a complete graph where each edge e = (i, j) ∈ E r has an attribute defined by the function D, associated with a relation between the regions R i and R j (see Figure 3). In our experiments, we choose the minimum distance between the two regions:

∀e = (i, j) ∈ E r , D(e) = min p∈Ri,q∈Rj (|p -q|) (3)
The model graph G m = (V m , E m , A, D), composed of C vertices (one vertex per class), is constructed from the training set. The attribute of a vertex is a vector of dimension N with only one non-zero component (with value equal to 1), associated with the index of the corresponding class. The edges are obtained by calculating the average relationships (in the training set) between the regions (according to the D relation considered).

B. Matching with the model graph

In order to identify the regions, the purpose is to associate each of the vertices of G r to a vertex of the model graph G m . According to the realistic assumption of having more regions in the image associated with G r than in the model (i.e. |V r | ≥ |V m |), we face a problem of inexact graph matching, namely many-to-one matching [START_REF] Lezoray | Image Processing and Analysis with Graphs: Theory and Practice[END_REF]. We propose to formulate Step #1 : "one-to-one" matching

Step #2: "many-to-one" (refinement) K matrix combining K and K v e Fig. 4. Graph matching formulated as a quadratic assignment problem (illustration inspired by [START_REF] Zhou | Factorized graph matching[END_REF]). Left: our proposal consists of two steps, where the X matrix models a matching between Gr and Gm graphs. The first step (top-left) focuses on a one-to-one matching (each vertex of Vm is associated with only one vertex of Vr). The second step (bottom-left) aims at matching remaining vertices of Gr, leading to the final matching (many-to-one matching). Right: For finding the optimal one-to-one matching, a K matrix is used, combining both Kv and Ke matrices, which respectively measure dissimilarities between vertices and edges. For the sake of clarity, only two edge attributes are reported.

this problem in the general form of a quadratic assignment problem (QAP), as recently considered in [START_REF] Zhou | Factorized graph matching[END_REF].

In our case, the concept of matching is represented by a matrix X ∈ {0, 1} |Vr|×|Vm| , where X ij = 1 means that vertex i ∈ V r is matched with vertex j ∈ V m . This is illustrated in Figure 4-left in two cases ("one-to-one" and "many-to-one" matchings). The goal is to determine the best matching (X * ), minimizing the following cost:

X * = arg min X vec(X) T K vec(X) (4)
where vec(X) is the column vector representation of X and T denotes the transposition operator. The K matrix, not detailed here for the sake of brevity (see [START_REF] Zhou | Factorized graph matching[END_REF] for details), embeds the dissimilarity measures between the two graphs G r and G m , at vertices (diagonal elements) and edges (non-diagonal elements):

K = α K v + (1 -α) K e max K e (5)
where K v embeds dissimilarities between vertices (Euclidian distance between class membership probability vectors). In the example in Figure 4 The α parameter (α ∈ [0, 1]) allows weighting the relative contribution of vertex and edge dissimilarities (K v terms range between 0 and 1, and K e is normalized in Equation 5).

Due to the combinatorial nature of this optimization problem [START_REF] Zhou | Factorized graph matching[END_REF] (i.e. set of possible X candidates in Equation 4), we propose a two-steps procedure, relying on the initial semantic segmentation provided by the neural network:

1) Search for an initial one-to-one matching (Figure 4-topleft). 2) Refinement by matching remaining vertices, finally leading to a many-to-one matching (Figure 4-bottom-left). 1) Initial matching: one-to-one.: One searches for the optimal solution to Equation 4 by imposing the following three constraints on X, thus reducing the search space for eligible candidates:

1)

|Vm| j=1 X ij ≤ 1: some i vertices of G r may not be matched. 2) |Vr| i=1 X ij = 1: each j vertex of G m must be matched with only one vertex of G r . 3) X ij = 1 ⇒ R i ∈ R * j : vertex i ∈ V r
can be matched with vertex j ∈ V m if the associated R i region was initially considered by the neural network to most likely belong to class j (i.e. R i ∈ R * j ). For instance, in the case of Figure 3, only vertices related to regions R 1 and R 2 would be considered as candidates for class 1 (R * 1 ). The first two constraints ensure to search for a one-to-one matching Thanks to the third constraint, one reduces the search space by relying on the neural network: one assumes that it has correctly, at least to some extent, identified the target regions, even if artifacts may still have been produced as well (to be managed by refining the matching). This step allows us to retrieve the general of the regions (thus verifying the prior structure modeled by G m ).

2) Refinement: many-to-one.: We assign each remaining vertex k (i.e. k ∈ V r | |Vm| j=1 X kj 0) to the vertex i * ∈ V r by considering the following cost function between two vertices i and j of G r :

cost(i, j) = α|A(j) -A(i)| + (1 -α) D(j) -D(i) max u∈ER D(u) . (6)
For each remaining vertex k, the best matching vertex i * , among already matched vertices with V m , minimizes the cost function:

i * = argmin i∈Vr| |Vm| j=1 Xij =1 cost(i, k). (7)
According to this formulation, it appears that remaining vertices are matched to vertices of G m by indirectly searching for correspondances with already matched vertices of G r . Therefore, one focuses on similarities within the current image and not with the model. In Figure 4-bottom-left, this corresponds to finding the matching between vertex 1 and vertex b by indirectly studying the relevance of the matching of vertex 1 with 3 (vertex 3 being already matched with vertex b).

The formulation related to Equation 7 and concerning one vertex only is similar to the matrix formulation related to Equations 4 and 5 (and concerning simultaneously several vertices). The only difference is that we consider vertex and edge dissimilarities within G r graph instead of considering dissimilarities between G r and G m .

III. EXPERIMENTS

We present, hereafter, the dataset used in the experiments, then the evaluation protocol, and finally the results.

A. Data

We consider the FASSEG1 public data set. This dataset focuses on the multi-class semantic segmentation of the face [START_REF] Khan | Multi-class semantic segmentation of faces[END_REF] (see Figure 1) as well as the estimation of its pose [START_REF] Khan | Head pose estimation through multi-class face segmentation[END_REF]. For this preliminary study, we consider a subset of this dataset corresponding to a specific pose (the front view).

This subset contains 70 images. FASSEG is a dataset of images with semantic segmentation rich in structural information that does not however distinguish certain regions (i.e. left eye and right eye, left eyebrow and right eyebrow). We have therefore refined the annotations in order to give a unique label to these regions. Note that we have also refined some semantic label maps for the creation of our model graph (see Figure 1).

B. Evaluation protocol

For these experiments, we consider the U-Net neural network [START_REF] Ronneberger | U-net: Convolutional networks for biomedical image segmentation[END_REF] that copes well with a training set with a small number of samples. For these experiments, we split our dataset as follows: 20 images are used for training (reference training set), 10 for the validation (reference validation set) and 40 for the test (reference test set). 100 epochs are used for training the network.

The model graph is constructed by calculating the average distances (Equation 3) between the different annotated regions of the training set. The α parameter is empirically selected, based on observations on some images, and set to 0.4 for experiments, yet without any optimization.

As this was experimentally investigated, a simple distance measurement between the centers of gravity of the regions appeared inappropriate, due to the variability of the shape of some regions (e.g. hair).

We evaluate the difference between the quality of semantic segmentation at the output of neural networks and that after matching, i.e. with integration of structural information. We consider the Intersection over Union (IoU or Jaccard index) to evaluate the quality of our results against our manual annotation used as ground truth. This evaluation measure is used to compare regions at the pixel level and also at the bounding box level. This comparison of bounding boxes allows us to quantify segmentation errors corresponding to a correct main region but with errors related to one or more minor sub-regions far from the main region (and not very significant in terms of number of pixels). These measurements are performed for each class, the overall mean value being also computed.

We also study the impact of the size of the training and validation sets on the quality of our semantic segmentation. Experiments are performed for various sizes of the reference training and validation datasets expressed in percentage i.e. 100% (20 images, so the full reference training set and 10 images for validation, so the full reference validation dataset), 75% (15 training images from reference training set, 7 validation images from the reference validation set), 50% and 25% (5 images for training, and 2 for validation). Experiments are run 20 times with random image selection for reference training and validation datasets for 75%-25% set sizes, and average performance results are reported for these cases.

C. Quantitative Segmentation Results

Overall, the semantic segmentation is improved thanks to graph matching as can be seen on average over all classes (training sets size between 50% to 100%) in Table I providing results in terms of pixelwise IoU index and IoU index of bounding boxes.

The degree of improvement is more significant when using IoU on region bounding boxes in comparison to pixel based IoU. This is due to the fact that, for a given class, misclassified pixels represent a small proportion of the ground truth area (benefits appear higher for small areas such as eyes and eyebrow). With IoU based on bounding boxes, one favors the analysis of the spatial distribution of misclassified areas. In this case, improvement appears particularly significant: 24.4% of improvement for the class right eyebrow (R-Br) with 50% training set size (Table I). The use of spatial relationships allows avoiding misclassified regions that are spatially incoherent (e.g. piece of hair between the nose and the mouth).

Table I also reports the influence of reducing the training data set. It appears that the smaller the dataset is, the higher are the benefits of using graph matching. Our modeling capturing spatial relationships combined with graph matching allows then improving segmentation results, compensating for the lack of representativity of a small training dataset.

Note that with only 25% of the training dataset, depending on randomly selected images, some trained networks appeared unable to propose region candidates for some classes on some test images. This is due to the poor representativity in the training dataset. In such cases, graph matching fails because of missing class candidates, this case being not yet managed by our approach. For this reason, results reported in Table I for 25% ignore such cases, and reported performances are only averaged on segmented test images providing at least one region per class (48.9% of images are not considered).

D. Qualitative Segmentation Results

Figure 5 gives some examples of semantic segmentations with both U-Net and our approach. As visually observed, improvements are significant in many cases (e.g. part of the eye on the middle image, neck), in particular when reducing the size of the training dataset (i.e. partial compensation of an under-trained model). The proposed approach appears efficient to improve the semantic segmentation achieved by the U-Net deep neural network. The improvement appears particularly significant for some classes (up to 24.4% for the Right-eyebrow label, compared to the average improvement of 8.5% considering the bouding box IoU with a training percentage of 50%). This is due to the nature of the considered information, which appears highly complementary to the low-level (pixel-level) information considered by the U-Net that intrinsically ignores high-level spatial relationships. As observed, an additional strength of our approach is its ability to compensate for segmentation errors resulting from the reduction of the training database. This is an important aspect as the size of the learning database in deep learning is a strong constraint, as underlined in the introduction.

While promising, our approach suffers from a few limitations. First, our approach is invariant in translation and rotation, but not yet in scale. This could be managed by introducing a scale factor as part of the matching process (in particular regarding the computation of the K matrix reported in Equation 4). Moreover, several parameters (e.g. α) need to be set empirically.

Secondly, partial occlusions in images that could affect the computation of graph G r are cases not yet considered in our current formulation. Indeed, each vertex (region) is assumed to be necessarily matched to a region of the model. Handling occlusions could be managed by relaxing hypotheses pertaining to matching the vertex of the model.

Thirdly, our modeling may not to be robust to face pose changes that alter spatial relationships. This issue can be addressed by choosing a more generic and representative model. Note that other domains of applications for our technique such as 3D medical images segmentation may not present the same challenges in practice as face segmentation presented here as illustration for our technique. To support high variation in spatial relationships, an alternative could be to consider a set of representative graph models instead of only one, with the underlying difficulty of selecting the appropriate one when segmenting an image. Our future work will also investigate how deep learning on graphs [START_REF] Zhang | Deep learning on graphs: A survey[END_REF] can help in improving our approach.

Finally, small computation time may be crucial in some applications. In particular, the high complexity of the first step (formulated as a quadratic assignment problem) may involve a dramatic increase of the computation time, if the number of classes and regions grows. Nevertheless, raw estimates provided in the paper are encouraging and further improvements could be sought using dedicated hardware (e.g. GPU).

V. CONCLUSION

In this paper, we have proposed an original method combining spatial relationships with deep learning for semantic segmentation. Preliminary results show how this approach improves significantly segmentation, with the additional benefit of using a limited number of training samples. The best obtained improvement over the U-Net neural network used alone for segmentation is of 8.5% IoU on average, trained with only 50% of the available training dataset. Future work will focus on the scale invariance, a finer evaluation of the computational complexity and on experiments in other application domains. Another aspect to be studied is the ability to manage graph matching using neural networks (i.e. neural networks on graphs and not only on images [START_REF] Gao | Graph U-Nets[END_REF], [START_REF] Li | GCN-LASE: Towards adequately incorporating link attributes in graph convolutional networks[END_REF]).

Fig. 2 .

 2 Fig. 2. Method overview. Training: the annotated training dataset is used to train the neural network and build a model graph (similar to Figure1-d). For the sake of clarity, although graphs are complete, only some edges are reported. Vertex color correspond to colored region labels. Semantic segmentation: the neural network produces a semantic segmentation, possibly with artefacts (e.g. small bright region within the eyebrow and dark region on the neck). A graph Gr is then built from this segmentation and matched with the model graph Gm. This inexact graph matching is achieved in two steps. First, correctly segmented regions are retrieved (one-to-one matching), artefacts being ignored (two remaining vertices in this example). Secondly, remaining artefact regions are matched (many-to-one matching): both artefacts are correctly relabelled (see surrounded areas).

Fig. 3 .

 3 Fig. 3. Graph construction from the tensor S (output of the U-Net) and the resulting L * entity. Each point of the left image is associated to the probability vector represented by a blurring effect. R *1 is the set of regions (regions R 1 and R 2 ) that belong to class 1 (according to probabilities). Edge attributes of the graph are computed from spatial relationships between regions R i . Vertex attributes are mean probability vectors, computed over related regions R i .

  -right, K v [1, 1] = 0.4 (row and column named 2a) represents the dissimilarity, in terms of probability vectors, between vertices 2 of G r and a of G m , if one would match these two vertices. The matrix K e is related to dissimilarities between edges. For instance, in Figure4right, K e [6, 1] = 9 (row and column respectively named 3b and 2a) corresponds to the dissimilarity between the edges (2, 3) ∈ E r (scalar attribute whose value is 16) and (a, b) ∈ E m (scalar attribute whose value is 7), if we would simultaneously match vertex 2 with vertex a and vertex 3 with vertex b. In such a case, K e[START_REF]A graph based image interpretation method using a priori qualitative inclusion and photometric relationships[END_REF][START_REF] Goodfellow | Deep Learning. Cambridge[END_REF] is computed using edge attributes: K e [6, 1] = 16 -7 = 9. K e terms are related to distances between regions (normalized in the final K matrix).

Fig. 5 .

 5 Fig. 5. Examples of segmentation results (initial image, result with U-Net, result with the proposed approach), for different sizes of the training dataset (only 75%, 50% and 25%). Bounding boxes highlight regions with significant improvements

TABLE I COMPARISON

 I OF SEGMENTATIONS PROVIDED BY THE U-Net AND OUR APPROACH ADDING GRAPH MATCHING (U-Net+GM) CONSIDERING THE IOU INDEX RELATED TO REGION BOUNDING BOXES AND PIXELWISE PRECISION COMPARED TO THE MANUAL SEGMENTATION.RESULTS ARE PROVIDED AS AVERAGE AND FOR EACH CLASS: BG (BACKGROUND), HR (HAIR), FC (FACE), L-BR (LEFT EYEBROW), R-BR (RIGHT EYEBROW), L-EYE (LEFT EYE), R-EYE (RIGHT EYE), NOSE AND MOUTH. RESULTS ARE ALSO PROVIDED FOR DIFFERENT SIZES OF THE TRAINING/VALIDATION SETS.

	Training dataset (%) 100 75 50 25	Approach U-Net U-Net + GM U-Net U-Net + GM U-Net U-Net + GM U-Net U-Net + GM	Pixelwise Classes Fc L-br R-br L-eye R-eye Nose Mouth Average Bg 88.0 88.2 91.9 61.8 60.5 76.9 Average Bg Hr 75.3 72.8 67.3 77.2 76.0 84.0 82.5 96.1 66.3 63.6 74.0 Bounding box Classes Hr Fc L-br R-br L-eye R-eye Nose Mouth 75.1 69.0 78.2 75.4 88.0 88.9 91.8 62.1 60.7 77.0 72.8 67.3 77.2 78.4 84.0 92.4 96.1 66.5 68.3 79.4 75.2 70.4 78.9 74.0 88.5 85.9 91.0 60.8 56.8 75.5 72.8 64.7 77.9 74.7 84.0 74.5 95.8 66.7 59.5 78.0 75.1 64.9 79.8 74.3 88.3 86.8 91.1 61.7 57.5 75.6 72.8 64.7 78.0 77.5 84.0 84.9 96.1 66.8 69.2 78.8 75.0 67.5 79.9 72.0 86.0 84.9 90.9 54.6 54.2 73.5 72.4 65.9 75.4 68.0 85.4 75.7 94.5 52.4 41.8 77.7 66.6 63.8 70.6 73.7 86.7 86.9 91.0 59.9 57.9 74.5 72.5 66.1 75.6 76.5 85.4 86.0 94.4 65.3 66.2 78.5 74.9 68.5 74.7 38.2 84.5 83.6 56.8 2.4 28.1 61.7 25.6 57.7 70.0 36.3 81.2 80.1 90.5 1.7 33.7 68.4 12.8 55.1 66.9 39.8 83.8 86.4 61.0 2.7 30.6 65.7 26.5 55.1 71.4 42.3 78.0 90.1 82.8 2.9 34.8 65.9 27.4 59.3 70.1

The FASSEG annotated public dataset can be downloaded at the following address: https://github.com/massimomauro/FASSEG-dataset.


Badr Bahloul

Fathi Safta

Mohamed Ali Lassoued

Hélène Dhotel

Johanne Seguin

Nathalie Mignet

Souad Sfar

Use of mouse model in pharmacokinetics study of poorly water soluble drugs: Application to fenofibrate

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT

2

ABSTRACT

Fenofibrate has recently been used as drug model in several studies with the objective of optimizing the development of some drug delivery systems to overcome the problem of poor aqueous solubility of the newly discovered API. The adequacy of the drug delivery systems to improve the oral bioavailability of encapsulated drug is generally evaluated by a pharmacokinetic study. The use of mouse as animal model for pharmacokinetic studies has become more important in the last decade because of many similarities with the human model in terms of the mechanisms of absorption, metabolism and elimination. Nevertheless, the mouse is often hampered by the very small volumes of blood that could be obtained during sampling. The aim of this work was to overcome the problem of lower volumes of plasma withdrawn by developing an appropriate protocol for sample preparation and a suitable HPLC method for drug quantification in mouse plasma. Linear calibration curve was obtained over the concentration range from 0,16µg/mL to 32µg/mL (r²=0,9999) with LLOQ of 0,16µg/mL The RSD in both intra-run and inter-run precision study was less than 11% and the extraction recoveries were above 91.9%. The reproducible method was successfully applied to the pharmacokinetic study of fénofibrate in mouse.

KEY WORDS: Fenofibric acid, HPLC-UV, Pharmacokinetic, mouse, fenofibrate, poorly water soluble drugs

INTRODUCTION

Fenofibrate, is a well-studied lipid regulating agent used for the treatment of hypercholesterolemia and hyper-triglyceridemia. After absorption, the drug is completely hydrolyzed by esterases in its active metabolite, the fenofibric acid, [1] which is eliminated with a half-life of 20 hours in human after one daily dosage. Its maximal plasma concentration is obtained around 3 to 4 hours post dosage.

Fenofibrate is practically insoluble in water and exhibits an extremely poor oral bioavailability [START_REF] Bahloul | A novel approach for the development and optimization of self emulsifying drug delivery system using HLB and response surface methodology: application to fenofibrate encapsulation[END_REF]. Therefore, fenofibrate has recently been used as drug model in several studies with the objective of optimizing the development of some drug delivery systems to overcome the problem of poor aqueous solubility characterizing more than 50% of the newly discovered API. Among these systems : solid lipid nanoparticles The rat appears to be the primary species most commonly used in preclinical pharmacokinetic studies. However, it was shown that the oral bioavailability values and the metabolic enzymatic levels in intestine are distinct between humans and rats [START_REF] Li | Oral bioavailability: basic principles, advanced concepts, and applications[END_REF].

Moreover, the use of the mouse as an animal model for pharmacokinetic studies has become more important in predicting oral bioavailability due to several similarities in the absorption, metabolism and elimination processes between humans and mice [START_REF] Mannhold | Drug bioavailability: estimation of solubility, permeability, absorption and bioavailability[END_REF]. Besides, the presence of gall bladder in mice led to the occurrence of an absorption mechanism, entitled collisional transfer, by which highly lipophilic drugs are diffused through the glycocalyx when they are intercalated in the bile acids (fasting state) or in the mixed micelles of bile acids (fasted state) [START_REF] Gao | Development of supersaturatable self-emulsifying drug delivery system formulations for improving the oral absorption of poorly soluble drugs[END_REF]. As rats do not possess a gall bladder, we chose to perform our study in mice. Nevertheless, the mouse exhibits a particular challenge because of the very small volumes of blood that could be obtained during sampling and hence impediments for pharmacokinetics [START_REF] Mannhold | Drug bioavailability: estimation of solubility, permeability, absorption and bioavailability[END_REF][START_REF] Hem | Saphenous vein puncture for blood sampling of the mouse, rat, hamster, gerbil, guineapig, ferret and mink[END_REF].

The aim of this work was at first to overcome the problem of lower volumes of plasma withdrawn by developing a one-step processing method for samples preparation that would be rapid in applying and ensuring at least 90% of drug extraction efficiency. Secondly, a fast and This quantification method was subsequently validated according to the recommendations of the SFSTP [START_REF] Hubert | Validation des procédures analytiques quantitatives Harmonisation des démarches[END_REF] and the FDA guideline for validation of bioanalytical methods [START_REF] Food | FDA guidance for industry: bioanalytical method validation[END_REF]. The feasibility of the animal model and the extraction protocol were evaluated by carrying out a pharmacokinetic application to fenofibrate in its commercialized form LIPANTHYL® with an oral dose corresponding to 5 mg / kg.

MATERIALS AND METHODS

Chemicals and reagents

Fenofibric acid (purity ≥98.0%) was provided from Sigma-Aldrich and the internal standard (I.S.), carbamazepine (≥99.0% purity) was purchased from Sigma-Aldrich (China).

Methanol and deionized water of HPLC grade were bought from Carlo Erba (France). Ammonium acetate was of analytical grade was purchased from Sigma-Aldrich (Netherlands) Hydrochloric acid and diethyl ether were acquired from Carlo Erba (France).. Normal mouse plasma was brought from Sigma-Aldrich (USA) and stored at -20°C.

Chromatographic conditions

Analyses were performed on a Shimadzu HPLC Class VP series (Shimadzu, Kyoto, Japan) which was composed of a LC-10ADvp quadratic pump, an auto-sampler (model SIL-10Avp Shimadzu), a variable UV-Visible wavelength detector (model SPD 10Avp; Shimadzu) and the data were analyzed by Class-VP series software, version 5.03 (Shimadzu, Japan). The analytical column used was LiChrospher 100 RP8 (125mm×4.6mm I.D, 5µm particle size). The detection wavelength was fixed at 287 nm and the column oven temperature was set at 35•C.Elution was obtained by applying the gradient steps summarized in table 1 and corresponding to solvents A (Methanol) and B ((5 mM ammonium acetate buffer (pH 3.3)). In all cases the flow rate was 1.5 ml/mn and the UV detection was achieved at a wavelength of 287 nm.

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT

Preparation of standard and stock solutions:

The Stock solution of fenofibric acid was prepared in methanol at a concentration of 1600µg/ml.

In the other hand, the preparation of the internal standard stock solution was done at a concentration of 4.7µg/ml by dissolving 2.35 mg carbamazepine in 500 ml HCl (1M).

Then, the standard solutions of fenofibric acid with concentrations of 1600, 800, 500, 160, 80, 16and 8µg/ml were prepared after serial volumetric dilution of fenofibric acid stock solution in methanol.

These solutions were prepared and stored in appropriate conditions (at -20°C) until analysis.

Extraction methods and sample preparation procedure:

25 µL aliquot of mouse plasma was mixed with 12.5 µL of the internal standard solution in 1.5ml Eppendorff tube.

Precipitation of the plasma proteins has been carried out by adding 250 µL of methanol then by vortexing the mixture for 30 minutes using a vortex mixer (Eppendorf, Hamburg, Germany).. Afterward the samples were placed in an ice bath for 60 min, followed by centrifugation for 10 minutes at 12.000 rpm ( 4 ° C). The top layer was injected (20 µL) into the HPLC system for analysis.

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT 6

Calibration curves and quality control plasma samples preparation

Calibration curves of fenofibric acid were prepared with concentration levels of 32, 16, 3.2, 1.6, 0.32and 0.16 µg/ml by spiking 0.5µL of the standard solution in 24.5 µL of mouse plasma. This ratio of 2:100 was chosen in order to avoid plasma alteration with the standard solution [START_REF] Nicolas | Stratégie de validation de méthodes de dosage en bioanalyse en vue d'études pharmacocinétiques et toxicologiques[END_REF].

Three control samples (QC) were prepared in blank mouse plasma at different concentrations of fenofibric acid (0.64, 16 and 25.6) µg/ml. All samples were conserved at -80 °C until analysis.

A single set of quality controls and standards were analyzed each day of the overall work using the same procedure described above for plasma samples.

Method validation:

A complete validation of the used method for the determination of fenofibric acid in mouse plasma was done according to the FDA guideline for validation of bioanalytical methods [START_REF] Food | FDA guidance for industry: bioanalytical method validation[END_REF]and to the SFSTP recommendations [START_REF] Hubert | Validation des procédures analytiques quantitatives Harmonisation des démarches[END_REF].

The method has been validated for selectivity, linearity, lower limit of quantification (LLOQ), precision and accuracy, stability and recovery.

Selectivity

Chromatograms of drug-free mouse plasma were compared (n=6). All samples were processed using the protein precipitation procedure in order to ensure the absence of endogenous peaks co-eluted with fenofibric acid or the internal standard.

Precision and accuracy

To assess the intra-and inter-day accuracy and precision , repetitive measurements of the three QC sample concentrations (0.64, 16 and 25.6 µg/ml) were realized. Intra-day accuracy was assessed through the QC samples analyzed in replicates of five per day. The inter-day precision was determined by performing the same procedure once a day during three consecutive days. Precision and accuracy were expressed as relative standard deviation (RSD)

and relative error (RE) respectively. The precision and accuracy acceptance criterion for each QC sample concentration was within ±15%.

Linearity and limit of quantification

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT 7
To assess the linearity, six-point standard curves had been run within the concentration range 0.16 -32µg/ml on three different days with replicates of four for each concentration per day (n=4).

Standard curves were fitted by plotting peak area ratio of fenofibric acid to internal standard versus the corresponding concentration. Calibration curves were obtained by least-squares linear regression method using weight scheme as x (x= concentration)

The acceptance criterion of the coefficient of correlation for the calibration curves was 0.999 or greater and every back-determined standard concentration had to be within 15% deviation with the exception of the lower limit of quantification (LLOQ) that was within 20%.

The LLOQ is corresponding to the lowest concentration on the standard curve which could be evaluated with adequate precision and accuracy.

Recovery

Extraction recovery was calculated by analyzing the three QC samples (three samples each) in order to evaluate the extraction performance of the used method.

The recoveries had been evaluated by comparing the peak areas of the extracted samples with those in which the analytes were subsequently added.

Stability study

Stability of fenofibric acid in mouse plasma during processing and storage was checked using the QC samples at three distinct concentrations that were analyzed under different conditions.

QC samples were analyzed after short-term storing at room temperature for 24 h, after longterm storage : one month at -20°C, and after three freeze-thaw stability cycles. The postprocessing stability was also evaluated by analyzing reconstituted QC samples that were kept in auto-sampler for 24 h.

Every stability study was carried out in replicates of four for each QC sample..

Application to pharmacokinetic study

The in vivo study was carried out according to the EU Directive 2010/63/EU related to animal experiments. 8-week-old male BALB/cJRj mice (Janvier, St. Genest de Lisle,

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT 8
France), weighing approximately 20-22 g were housed in a controlled environment and were fasted overnight for 12 hours, nevertheless they were allowed to have free access to water.

The animals were dosed via oral gavage with 5 mg/kg of fenofibrate. The compound was orally administrated as suspensions of LIPANTHYL micronized.

Blood samples were collected at 0 h (pre-dose) 0.5, 1, 2, 3, 4, 6, 8, and 24 h post-dose from the tail vein using a sparse sampling schedule consisting of two blood samples for each mouse and 6 mice per time point. The samples were transferred to heparinized tube and centrifuged at 10,000 rpm for 10min. The resulting plasma was separated and kept frozen at -80°C prior analysis.

RESULTS AND DISCUSSION

Method development

The separation of fenofibric acid and carbamazepine (internal standard) from the biological endogenous components in mouse plasma was found to be efficient and remarkably within satisfactory range. Various combinations of mobile phase, isocratic and gradient systems were investigated to achieve the optimum separation condition. We found that the best wavelength for detection was 287nm.. The used mobile phase was Methanol and ammonium acetate buffer (5 mM, (pH 3.3). And optimal elution was attained by applying the gradients steps (summarized in table 1) of solvent A (Methanol) and solvent B (ammonium acetate buffer (5 mM, (pH 3.3)) at a flow rate of 1.5ml/min. The retention times of I.S and fenofibric acid were 3 min and 9 min respectively.

Clear supernatants, which were immediately injected into HPLC, were obtained after centrifugation of plasma samples that had been simply treated with Methanol and then cooled to 0 °C by immersion in an ice bath for 30 minutes. The sample processing is consequently convenient, inexpensive, and rapid. Furthermore, no evaporation step or a reconstitution phase was needed. Otherwise during the method development, protein precipitation step was also evaluated using diethyl ether but recovery, extraction rate and selectivity were found to be unsatisfactory specially compared to the method using methanol which allowed a high recovery and selectivity.

Selectivity study

Six different sources of blank plasma were screened in order to investigate any eventual interference from endogenous substances. Fig. 1 shows chromatograms obtained from the analysis of the extracted drug-free plasma, plasma sample spiked with fenofibric acid and the internal standard, and a plasma sample obtained from a mouse post dosing at 3h. The retention time for carbamazepine (internal standard) and fenofibric were, respectively, 3min and 9 min. With no endogenous compounds in the plasma eluted at the retention time of fenofibric acid and internal standard.

Linearity and lower limit of quantification

The peak areas ratio (Y) of fenofibric acid versus the internal standard (I.S) were proportional to the concentration of fenofibric acid (x) in plasma within the tested range (0.16 -32 µg/ml).

Blank mouse plasma spiked with the corresponding standard solutions giving concentrations of 0.16, 0.32, 1.6, 3.2, 16 and 32µg/ml were analyzed. Excellent linear correlation was obtained for all analytes over the entirestudied concentration range. The standard equation was y = 0,4316x -0,0227 and the correlation coefficient was r²=0.9999. The analyte response at the lower limit of quantification (LLOQ) was found to be 5 times the response of the blank response (S/N ratio greater than 5) and was equal to 0.16µg/ml for fenofibric acid. The accuracy and precision at the LLOQ were acceptable, with 8.5% RE and 2.06% RSD (n=5).

Precision and accuracy

The intra-and inter-day precision and accuracy results are summarized in Table 2. The RSD and RE were less than 11% for the intra-and inter-run precision and accuracy assays. These described results inferred that the developed method was precise and accurate.

Recovery:

The extraction recovery was calculated for the three quality control concentration levels of fenofibric acid with three replicates for each level. The recovery of fenofibric acid at concentrations of 0.64, 16 and 25.6 µg/ml were 93.39% ± 1.06, 91.98% ± 6.83, 92.49% ± 8.25, respectively. The recovery of carbamazepine at the working concentration was 98,10 ± 3,348%.

Stability:

The results of stability study, summarized in Table 3, showed that no significant degradation occurred to fenofibric acid during storage and extraction processes in mouse plasma samples.

CONCLUSION

A HPLC method for the determination of fenofibric acid in mouse plasma has been developed

and completely validated for the first time. The preparation procedure of plasma samples was suitable without any need to a reconstitution process or an extraction-evaporation phase. The method was reproducible and the extraction recoveries were above 91.9% . Linear calibration curve was obtained over the concentration range from 0,16µg/mL to 32µg/mL (r²=0,9999) with LLOQ of 0,16µg/mL The RSD in both intra-run and inter-run precision study was less than 11%. Besides, this study has shown that it was possible to overcome the problem of lower volumes of plasma withdrawn by the developed sample preparation process and the drug quantification method. Hence, the feasibility of mouse model to be used in pharmacokinetics study of fenofibrate was confirmed as well as the extraction protocol with reference to the results of analytical validation and successful application of the pharmacokinetic study. Finally, this method may allow scientists to perform rapid and robust oral bioavailability studies of

[ 3 ]

 3 nanocrystals [4], nanosuspension [5], selfmicroemulsufying systems [6] that were developed using fenofibrate as a drug model . The adequacy of these drug delivery systems to improve the oral bioavailability of encapsulated drug is generally evaluated by a pharmacokinetic study which requires a suitable animal species that should closely resembles to human in terms of the mechanisms of absorption and elimination of the active ingredient.

  was developed for the dosage of the active metabolite of fenofibrate (fenofibric acid) by HPLC-UV.

9 Fig. 1 .

 91 Fig. 1. HPLC chromatogram of (a) blank mouse plasma. (b) plasma sample spiked with standards of carbamazepine (Rt-3 min) and fenofibric acid at a concentration of 0.64µg/ml (Rt-9 min). (c) plasma sample from a mouse, post dosing at 3h.

Fig 2 .

 2 Fig 2.Mean plasma concentration-time profiles of fenofibric acid in plasma following a single oral dose of 5mg/kg fenofibrate to 24 male BALB/cJRj mice.

5 Table 1 .

 51 Gradient elution system for separation of fenofibric acid and internal standard

			Gradient
	Cumulative time of
	acquisition (min)	% Solvent A Methanol	% Solvent B 5 mM ammonium acetate buffer (pH 3.3)
	5	56	44
	10.5	66	34
	12.5	76	24
	13	66	34

Table 2 .

 2 Intra-and inter-day precision and accuracy for the determination of fenofibric acid in mouse plasma.

	Spiked	Intra-day precision and accuracy (n=5)		Inter-day precision and accuracy (n=15)
	(µg/ml)			RE			RE
		measured (mean ± SD) µg/ml	RSD (%)	(%)	measured (mean ± SD) µg/ml	RSD (%)	(%)
	0.64	0.57 ± 0.03	4.82	10.00	0.58 ± 0.04	6.23	8.23
	16	15.02 ± 1.02	6.82	5.70	16.13 ± 1.31	8.09	1.22
	25.6	24.6 ± 2.69	10.92	3.51	25.75 ± 1.73	6.71	-0.99

Table 3 .

 3 Stability data of fenofibric acid in mouse plasma under different storage conditions

	243
	244	(n=4)
	storage conditions

.7. Application to pharmacokinetic study:

	246
	247	concentration in mouse plasma after oral administration of 5mg/kg fenofibrate to 24 male
	248	BALB/cJRj mice. Fig 2. Illustrates the mean plasma concentration/time curve of fenofibric
	249	acid. The plasma level of fenofibric acid reached a maximum concentration level (Cmax)
	250	equal to 0.831 ± 0.119 µg/ml, in (Tmax) equal to 2h and the AUC 0_24h was 4.240 ±
		0.571µg.h/ml. The obtained pharmacokinetic profiles were in close agreement with the PK

245

After validation, the proposed method was applied for the determination of the fenofibric acid 251 profiles of fenofibric acid which were reported recently in literature [14-16].

fenofibrate in mice when it is used as drug model in some new drug delivery systems and innovative formulations strategies.

M A N U S C R I P T A C C E P T E D ACCEPTED MANUSCRIPT