M Kas

Y El Merabet
email: y.el-merabet@univ-ibntofail.ac.ma

Y Ruichek

R Messoussi
email: messoussi@uit.ac.ma

New Framework for Person-independent Facial Expression Recognition Combining textural and shape analysis through new feature extraction approach

Keywords: Facial expressions recognition, Textural and shape analysis, Landmarks, Local Binary Patterns, Support vector machines, Deep features

published or not. The documents may come

Introduction

Believing that computing devices can autonomously perform complicated tasks by being trained rather than programmed, machine learning relies on endowing machines with the cognitive skills naturally acquired by the human brain. Machine learning and artificial intelligence (AI) are dominant topics in terms of how advanced analytics will appear in the future as well as their expected outcomes and benefits. More precisely, ongoing studies are oriented toward the development of machinelearning-based algorithms capable of fulfilling brain functions by allowing them to iteratively learn from data to improve, describe data, and predict outcomes. The perception of the human brain is the key to understanding and interacting with the environment based on hearing, touching, and in particular, natural vision sensors. Similarly, through state-of-the-art computer vision, researchers are attempting to implement humans-like visual analysis capabilities. One of the most complicated human visual analyses is facial expression recognition, which relies on sensing the emotion of a given person within the environment based on the individual's facial expression. This task serves diverse applications that are of interest to many different markets. For example, it can be useful to help children diagnosed with autism better understand their social environment. Moreover, such technology will allow an accurate real-time evaluation of E-learning content and public services to be more easily achieved. Furthermore, the robot industry will be able to develop human-support robots qualified to adapt their interactions according to the emotional atmosphere. In real-world scenarios, the desired system is expected to recognize the emotion of unseen individuals in real time, which makes this task among the most difficult in computer vision. In the literature, the recognition of facial expression has four different levels, as can be seen in Figure 1, which demonstrates the level of difficulty regarding the ways in which emotions are expressed (spontaneous versus posed) and the person expressing it (the same person used in the training or a different individual). Spontaneous emotions are difficult to classify because each individual expresses a given emotion differently compared to another person. Furthermore, this fact often leads to interclass sample interference, meaning that two emotion classes are represented over two images with the same overall appearance. Moreover, the recording of spontaneous emotions must be applied while the subjects are un-aware of it, which is extremely difficult to establish because the subjects should deliver authorization for the recording and use of their images/videos. Therefore, there are only a few studies that have been interested in spontaneous emotion recognition and have focused solely on verifying the assigned labels and whether they match the corresponding observation, as reported by [START_REF] Kayyal | Language and emotion: certain english-arabic translations are not equivalent[END_REF]. The majority of available databases for spontaneous facial expressions have been collected from the web, based on saving images from search engines (mainly Google and Flicker) by specifying the emotion-related keywords. The well-known and widely used databases of this kind are FER and AFEW. By contrast, posed expressions are obtained by requesting the subjects to perform the facial expressions in a uniform way in order to avoid intra-class similarities that would confuse the classification task. The subjects are usually skilled persons (actors), and thus their expressions can be computationally classified. The second challenge of facial expression recognition relies on correctly decoding the observation of individuals not taking part in the training session. Here, the objective is to implement a person-independent application and standalone framework that can be deployed on various platforms.

This paper deals with person-independent posed facial expression recognition (FER) and proposes an automatic FER framework based on the shape and appearance descriptions. Two popular approaches have been proposed in the literature for decoding facial expressions. The first is geometric-based feature extraction. This approach relies on encoding geometric information such as the position, distance, and angle on the facial landmark points that should be first identified by a landmark detector, and then extracts the feature vectors. The second approach is the appearance-based technique, which characterizes the appearance textural information resulting from the facial movements related to each of the emotion classes. Therefore, a set of features is extracted and is expected to contain relevant discriminative information to classify the different classes. The appearance-based approach utilizes many techniques for feature extraction, including those based on different transforms such as wavelet sub-bands, Gabor filters, an optimal matrix factorization and a steerable pyramid transform, an independent component analysis (ICA), a Zernike moments method, a global Gabor-Zernike feature descriptor, a principal compo- nents analysis (PCA), and a linear discriminant analysis (LDA) based Fisherface method. Introduced by Ojala's study, local binary patterns (LBPs) constitute a new philosophy in feature extraction. The motivations behind this philosophy rely on overcoming the limitations of global features by allowing the extraction of local relevant features based on pixel neighborhood thresholding and then combining the obtained vectors to construct the final image descriptor. The LBP operator was originally proposed for texture classification, although considering its discriminative power and low computational cost, it has also been adopted in many other computer vision applications, mainly face-related. Since Ojala's study, many state-of-the-art LBP variants have been proposed to enhance the original LBP capabilities. Indeed, researchers are still searching for robust local descriptors with a high discriminative power, and numerous powerful LBP variants continue to be developed in the literature. Notable recent methods include a local optimal oriented pattern (LOOP) [START_REF] Chakraborti | Loop descriptor: Local optimal-oriented pattern[END_REF], local neighborhood difference pattern (LNDP) [START_REF] Verma | Local neighborhood difference pattern: A new feature descriptor for natural and texture image retrieval[END_REF], and local directional ternary pattern (LDTP) [START_REF] Khadiri | Local directional ternary pattern: A new texture descriptor for texture classification[END_REF]. This paper introduces a new automatic FER framework based on a hybrid approach that combines geometric and appearance concepts by extracting the textural and shape features from facial landmarks. The proposed combination is expected to promote an enhanced performance for person-independent FER because we consider geometric and appearance information that carries sufficient relevant features to describe the emotional classes. The geometric representation is obtained by interpolating the positions of 49 keypoints (landmarks) detected in the input image generating a binary patch, which is exploited to compute the shape features using the HOG method. By contrast, the appearance description is also extracted based on the detected landmarks instead of the whole face image, which makes our proposed FER framework able to fulfill the personindependent constraint. The appearance features are extracted from 32 pixel ×32 pixel sub-images centered on each landmark using a brand new handcrafted descriptor, which is referred to as orthogonal and parallel-based directions-generic query map binary patterns (OPD-GQMBP), which is also proposed in this paper. The OPD-GQMBP handcrafted descriptor is based on orthogonality and parallelism geometries for selecting the most prominent neighbors. It adopts an n × n neighborhood region to extract four feature maps based on four defined thresholding structures for each central pixel. The four feature maps are then decoded into one histogram. Afterwards, the 49 feature vectors are concatenated to form the final appearance feature. During the classification step, we use the SVC library preprocessed by the PCA technique intended to reduce the dimensionality of the feature vectors. The framework is evaluated on five of the most widely used state-of-the-art benchmarks: KDEF, CK+, RaFD, JAFFE, and OuluCasia. To ensure the person-independent evaluation, the leave-one-subject-out (LOSO) protocol is adopted on all five datasets. To prove the superiority of the proposed OPD-GQMBP descriptor for the FER, we conducted a comprehensive experimental comparison against 12 recent and powerful LBP variants and 10 state-of-the-art deep features (e.g., VGG, ResNet, and Inception) using our FER framework on the five datasets. We also compared our FER framework perfor-mance to those reported in the literature (from articles in journals with a high impact factor and from highly indexed conferences). The major contributions of our study can be summarized as follows:

• An FER system based on the coupling shape and appearance information is proposed. This system applies the Dlib package to detect 49 landmarks on the facial image and then extracts the shape and textural features before proceeding to the SVM-based classification stage.

• New handcrafted LBP variants our applied for a texture analysis based on the orthogonality and parallelism geometry concepts, which are referred to as OPD-GQMBP. This is a generic descriptor that takes the neighborhood size as a parameter. This parameter offers the opportunity to balance the computational speed with the discrimination performance.

• We considered five widely used benchmarks from the literature for evaluation: CK+, KDEF, JAFFE, Oulu Casia, and RaFD. We adopted an experimental person-independent evaluation on each dataset, using the LOSO protocol.

• The proposed FER framework has outperformed many recent studies published in journals with a high impact factor, including deep-learning and handcrafted frameworks, following the LOSO protocol.

• The performance of the OPD-GQMBP descriptor is fairly evaluated based on its comparison to 12 recent handcrafted LBP variants and 10 deep features. The results showed the superiority of OPD-GQMBP against all of the tested handcrafted and deep features.

To provide readers and field-interested researchers with a better reading experience, this paper is organized as follows. Section 2 presents some state-of-the-art approaches devoted to the FER problem, covering handcrafted and deep-based features. Section 3 introduces the proposed OPD-GQMBP descriptor and highlights its process to compute the textural feature vector. Moreover, this section shows the overall proposed FER framework and explains how the proposed OPD-GQMBP and HOG operators are applied and combined to obtain the textural and shape features from the detected landmarks of a given input face image. Section 4 provides comprehensive experiments on five widely used databases and shows a comparative evaluation of state-of-the-art descriptors (handcrafted-based and deep-based) and FER systems. The last section provide some concluding remarks regarding our study and areas of future research.

Related works

The computer vision community has conducted many studies devoted to facial expression recognition (FER) by applying machine learning techniques. In this section, we briefly present some state-of-the-art FER frameworks to highlight some of the proposed architectures that rely on either handcrafted descriptors or deep-learning methods. Shan et al. [START_REF] Shan | Facial expression recognition based on local binary patterns: A comprehensive study[END_REF] proposed an approach, referred to as Boosted-LBP, based on combining a basic LBP descriptor with the Adaboost algorithm to enhance the classification performance. They conducted experiments on CK+, MMI, and JAFFE databases, and found that the Boosted-LBP outperforms the basic LBP combined with a multi-class SVM classifier. Moreover, they reported that local methods (LBP) perform better than global methods (Gabor filters). Zhang et al. [START_REF] Zhang | Facial expression recognition using lbp and lpq based on gabor wavelet transform[END_REF] proposed a novel facial expression recognition method using a local binary pattern (LBP) and local phase quantization (LPQ) based on a Gabor face image. First, Gabor wavelets are applied to capture the prominent visual attributes, which are separable and robust to illumination changes, by extracting multi-scale and multi-direction spatial frequency features from the face image. Then, the LBP and LPQ features based on the Gabor wavelet transform are fused for face representation. Considering that the dimensions of a fused feature are too large, the PCA-LDA algorithm is used to extract compressed features. Finally, the method is tested and verified using multiclass SVM classifiers. Lekdioui et al. [START_REF] Lekdioui | Facial decomposition for expression recognition using texture/shape descriptors and svm classifier[END_REF] proposed an automatic FER framework based on a local appearance approach, extracting the features from seven regions of interest (ROIs) covering the left eyebrow, right eyebrow, left eye, right eye, eyebrows, nose, and mouth. They evaluated the LBP, LTP, and CLBP texture descriptors and their combination with the HOG operator cascading with a linear SVM classifier. They found that the concatenation of LTP and HOG leads to the best FER performance on three datasets (CK, FEED, and KDEF). Their framework strengths rely on extracting the appearance features from seven sub-images defined from landmarks carrying information about the facial expression class, in addition to combining the LBP-Like descriptor with the HOG operator. However, this architecture presents certain drawbacks that we can point out. The seven extracted sub-images have different sizes and orientations, but their computed features have the same length. We found that the nose region of interest is vertically oriented compared to the eyebrow regions, which are horizontal, and the eye regions, which are almost square. Therefore, different amounts of information on different locations are represented over feature vectors of the same length. Furthermore, this study did not cover an important number of handcrafted methods, and no deep-learning method was evaluated. The method proposed by Makhmudkhujaev et al. [START_REF] Makhmudkhujaev | Facial expression recognition with local prominent directional pattern[END_REF] uses a new handcrafted LBP descriptor referred to as local prominent directional pattern (LPDP) for FER application. It is also an appearance-based approach exploring the benefits of extracting features from three patches: edge, curved edge, and cornerlike texture maps. Their study focuses only on the handcrafted descriptor LPDP and its scheme to extract textural features. The authors also used a thresholding parameter to discriminate significant features from insignificant patterns in featureless/smooth regions of a face. Afterwards, a feature selection method is applied to reduce the dimensionality of the final feature vector because they use the spatial division on the input image. However, this system takes as input the entire face image, which makes it inconvenient for person-independent FER applications. In addition, no shape descriptor has been adopted in the overall framework, relying only on LPDP extracted features. Minchul et al. [START_REF] Shin | Baseline cnn structure analysis for facial expression recognition[END_REF] used a convolutional neural network model to achieve facial expression recognition. They adopted and aligned cropped faces from FER-2013, SFEW2.0, CK+, KDEF, and Jaffe with respect to the landmark position of the eyes. The training data were augmented 10 times by flipping them. Five types of data input (raw, histogram equalization, isotropic smoothing, diffusion-based normalization, and difference of Gaussian) were tested. They then selected the one that showed the highest accuracy as a target structure for fineparameter tuning. Finally, the CNN network with histogram equalization images was chosen as the baseline CNN model for further research. Yu et al. [START_REF] Yu | Image based static facial expression recognition with multiple deep network learning[END_REF] proposed a method that contains a face detection module based on an ensemble of three stateof-the-art face detectors, JDA, DCNN, and MoT. Subsequently, a classification module composed of an ensemble of deep convolutional neural networks (CNNs) was adopted based on averaging the output responses. Each CNN model is initialized randomly and pretrained on the Facial Expression Recognition (FER) Challenge 2013 database. The pretrained models were then fine-tuned on the training set of SFEW 2.0. To combine multiple CNN models, they presented two schemes for learning the ensemble weights of the network responses: minimizing the log-likelihood loss and minimizing the hinge losses. According to the results reported in their study, the hinge loss performs slightly better than the log-like and single CNN models on the validation and test sets of the FER2013 and SEFW databases. Therefore, their framework is computationally heavy, and the outcomes are not very promising. Jung et al. [START_REF] Jung | Joint fine-tuning in deep neural networks for facial expression recognition[END_REF] proposed a new CNN framework based on combining the temporal appearance and temporal geometry extracted from two CNN models. The faces in the input image sequences are detected, cropped, and rescaled to a pixel resolution of 64 × 64, and 49 landmark points are then extracted using the IntraFace algorithm. Finally, these two models are combined using an element-wise sum of the outputs of the last fully connected layers from the two temporal CNN models. Through several experiments conducted on the CK+, MMI, and Oulu-CASIA databases as well as numerous data from various data augmentation techniques, the framework built showed that the two models cooperate with each other. However, the joint model did not improve the recognition of all of the facial expressions and achieved the same performance as the temporal appearance and temporal geometry models of the Disgusted, Fear, Happy, and Surprised classes. In addition, the temporal appearance CNN model outperformed the geometry model on all tested databases. Most of the previous methods have considered the entire facial region as the input information, and have paid less attention to the sub-regions of human faces, which may lead to a large difference between the extracted and expected representations. Indeed, when the extracted information obtained from the entire face image is irrelevant, the final recognition result will be affected. Because the judgment of the facial expression is usually based on the information of several sensitive components in some areas of the face, such as the eyes, nose, and mouth, in this paper, a new method is proposed that concentrates the feature extraction on these sub-regions, which not only allows for the extraction of more relevant features, but will also further improve the overall recognition rate.

Proposed framework

In this paper, we propose an enhanced framework for facial expression recognition (FER). The system is based on the SVM classifier to predict the class of a given input image. It considers 49 landmark points and extracts and combines the shape and appearance features to be fed to the classifier. To clearly highlight the contributions of our study and describe the workflow of our system in detail, we first describe the new textural handcrafted descriptor referred to as OPD-GQMBP. We then present how it is combined with the HOG shape descriptor to build the overall workflow.

OPD-GQMBP: New handcarfted descriptor for FER

As discussed in the introduction, the LBP operator is extremely flexible and many of its aspects can be employed to develop enhanced descriptors for specific tasks. In our case, we propose a new LBP variant, referred to as OPD-GQMBP, which is based on new neighborhood topologies leading to four discriminant feature maps, and adopts the LBP original kernel function that outputs low computational codes. The motivation behind the OPD-GQMBP descriptor relies on selecting orthogonal and parallel neighboring pixels that are believed to present the most information within a sub-block. In mathematics, orthogonality is defined as the generalization of the perpendicularity notion, which was adopted by [START_REF] Barkan | Fast high dimensional vector multiplication face recognition[END_REF], who proposed a reduced LBP version referred to as OC-LBP, which considers two sets of four pixels located on the orthogonal lines. Thus, it produces only a feature histogram with a 2 × 2 3 feature histogram. . The OPD-GQMBP descriptor is generic and adjustable depending on the needs of the considered application. It adopts a n × n sub-block neighborhood, where n is an odd integer (3,5,7,9,...), to maintain symmetric neighborhoods. The concept behind this is the selection of prominent pixels within this neighborhood. Given a central pixel I c , as can be seen in Figure 2, we define four pixel groups, each of which contains n × 2 pixels forming two lines. Two sampling groups are based on orthogonality {S G 1

Ort , S G 2 Ort }, whereas the two others are based on the concept of parallelism, i.e.,

{S G 1 Par , S G 2 Par }. Therefore, each sampling group S G is de- fined on two lines S G k t (I c ) = {L t k,1 (I c ), L t k,2 (I c
)} where t stands for the type (Ort/Par) of the sampling group and k the group number (1/2). Figure 3 shows a Cartezian coordinate system centered on the central pixel I c to encode the position of each pixel considered in each sampling group within a 7 × 7 neighborhood. The sampling groups are defined as follows:

S G 1 Ort (I c ) = L Ort 1,1 (I c ) = {(x, 0)/x T } L Ort 1,2 (I c ) = {(0, y)/y T, } (1)
S G 2 Ort (I c ) = L Ort 2,1 (I c ) = {(x, x)/x T } L Ort 2,2 (I c ) = {(x, -x)/x T } (2) 3 4 5 1 13
Geometry Concept S G 1

Parallelism Orthogonality

Par (I c ) = L Par 1,1 (I c ) = {(x, 1)/x T } L Par 1,2 (I c ) = {(x, -1)/x T, } (3)
S G 2 Par (I c ) = L Par 2,1 (I c ) = {(1, y)/y T } L Par 2,2 (I c ) = {(-1, y)/y T } (4)
where

T = -n-1 2 , n-1 2 (5)
Here, T is the interval of values defining the coordinates (x, y) of the pixels constructing the two lines {L t k,1 , L t k,2 } of each sampling group S G k t (I c ). Because all pixels within the n × n neighborhood are identified, we can proceed to the thresholding process. In this step, we generate for each sampling group S G k t its feature map k t , and obtain four feature maps:

(I c ) =                1 Ort (I c ) = Γ(S G 1 Ort (I c )) 2 Ort (I c ) = Γ(S G 2 Ort (I c )) 1 Par (I c ) = Γ(S G 1 Par (I c )) 2 Par (I c ) = Γ(S G 2 Par (I c ))                (6) with Γ(S G k t )(I c ) = ∆(L t k,1 (I c ), L t k,2 (I c )) ( 7
)
where ∆ is the Heaviside function, which was originally used in the LBP operator defined in Eq 8, and applied the two lines of the same group to the threshold element by element. Thus, the length of the generated binary code for each feature map is the size (n) of the neighborhood, and the number of possible produced patterns is 2 n . Thus, by concatenating the patterns produced by all feature maps, we generate 4 × 2 n possible patterns. After encoding each pixel in the input image and obtaining the four feature maps, we transform them into a histogram vector as the final descriptor for the image, as defined in

∆(x, y) =        1 , x ≥ y 0 , x < y (8) H(F) = H 1 Ort , H 2 Ort , H 1 Par , H 2 Par (9)
where

H k t (p) = χ∈F δ( k t (χ), p) (10)
In Eq 10, p ∈ [0, 2 n -1] is a pattern used to compare to the patterns k t (χ), χ is the gray-scale value of the computed feature image F, and the delta function δ(•), which is defined as follows (see. Eq. 11):

δ(a, b) = 1, if a = b; 0, otherwise (11)
To include more spatial information into the OPD-GQMBP descriptor, the feature image is spatially divided into w× w small non-overlapping blocks B i . Then, all corresponding histograms H(B i ) extracted from all blocks are concatenated to form the final holistic image representation through Eq. 12.

H = w 2 i=1 H(B i ) ( 12
)
where H is the final descriptor, is the concatenation operation, and H(B i ) is the histogram of the OPD-GQMBP descriptor computed on the i th block. Note that each elementary histogram H(B i ) has a length of 4 × 2 n , whereas the dimension-

ality of H is 4 × 2 n × w 2 .

FER system

After defining the neighborhood topology and thresholding kernel of the OPD-GQMBP descriptor, we now present the overall view of our proposed system for the FER task. The idea behind this framework is to combine the shape and appearance information to provide a more accurate FER, whereas most of the state-of-the-art proposed FER systems rely only on one piece of information, either geometric or appearance based. To do so, we computed the textural and shape features based on the location of 49 detected keypoints (landmarks) on the input face image. The shape representation is obtained by interpolating the 49 landmarks to form curves to be further analyzed using the HOG operator, whereas the appearance representation is based on texture analysis by applying the proposed OPD-GQMBP operator on specific sub-images of the input image. To make the FER system more able to fulfill the personindependent constraint, the appearance features are extracted from sub-images with a pixel resolution of 32 × 32 and centered on each landmark carrying sufficient and relevant information about the expressed emotion and less irrelevant information of the person's face. Figure 4 illustrates the overall pipeline of the proposed FER system. First, the input image is fed to the dlib landmarks extractor to locate the 49 points (green color). These locations are then interpolated to generate a binary patch of the expressed emotion, upon which the HOG operator is applied to compute the shape feature vector. Meanwhile, the OPD-GQMBP descriptor (or state-of-the-art descriptors for comparison) was used to extract the textural features from each 32 × 32 sized sub-image centered on one landmark leading to a set of 49 histograms (49 landmarks) that are further concatenated together to construct the appearance feature vector. Note that spatial division was adopted to compute the OPD-GQMBP feature vector by dividing each sub-image into non-overlapping blocks of size w × w, as illustrated in Figure 5. The number of spatial blocks that divide the sub-image depends on the considered dataset and is related to the camera resolution and image blur. Indeed, blurred images require fewer blocks than clear images, which present more details to be detected. At the end of the feature extraction stage, the HOG and OPD-GQMBP computed histogram vectors are concatenated to compose the final image descriptor that is further fed to a dimensionality reduction using the PCA method before proceeding to the classification phase based on an SVM. We used the LIBLINEAR 2.30 library as a multiclass kernel-based vector machine implementation for MATLAB/Python environments. This library provides many classification and regression solvers. We chose the support vector classification based on the Cramer and Singer solver (Kernel = 4) as a simplified multi-class SVM. Furthermore, this kernel allows optimized training and takes less time compared to the LibSVM library implementation.

Experimental Analysis and Discussions

The previous sections introduced a new framework for FER based on combining the shape and appearance features computed using the HOG and our proposed OPD-GQMBP descriptor. Our FER framework relies on extracting the features from To highlight the results of each of the contributions, we first evaluated four possible configurations of the OPD-GQMBP descriptor, and then compared its performance against 12 recent handcrafted methods and 10 state-of-the-art deep features (see Table 1) within our FER framework, keeping the same evaluation protocol and conditions. Afterwards, the performance of the proposed FER framework is compared to those presented in previous state-of-the-art purchase, published in highly indexed and well-known journals.

Image textural based feature vector

Neutral

Happy Sad Surprise

Deep-Features for Facial Expression Recognition

In this study, the proposed descriptor is compared to the 10 deep features described in the literature. Deep methods are inspired and rely mainly on convolutional neural networks (CNNs). Owing to the technological progress made in the GPU computational field, a CNN has proven to be one of the most widely used techniques for computer vision applications, such as image classification, object detection, and face recognition. A CNN typically consists of convolutional layers, pooling layers, and fully connected layers. Convolutional layers are the core building blocks of a CNN.

We considered 10 deep networks, which are briefly introduced in the following, based on a recent survey [START_REF] Lateef | Survey on semantic segmentation using deep learning techniques[END_REF]:

• AlexNet: Referring to its author Alex Krizhevsky, AlexNet was proposed in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC 2012). AlexNet is a deeper configuration of a LeNet5 network. Therefore, the high performance at this competition comes at the cost of a high computation that was possible using only graphic card units. It consists of five convolutional layers, two fully connected hidden layers, and one fully connected output layer.

• VGG: VGG network architectures were introduced by Simonyan and Zisserman in 2014. VGG stands for the Visual Geometry Group of Oxford University. Compared to LeNet and AlexNet, VGG networks are conceptually simple employing only stacked 3 × 3 convolutional layers combined with a max pooling layer to reduce the volume size, leading to two fully connected layers of 4096 nodes each, followed by a softmax classifier. VGG19 has three more convolutional layers than VGG16. To employ deep learning architectures for deep feature extraction in solving the FER problem, we follow the basic procedure shown in Figure 11. Initially, the model was trained endto-end on a big dataset, mainly the Facial Expression Recognition 2013 (FER2013) database. Afterwards, the model is expected to achieve a good training performance using the validation set. We then proceed to the transfer learning technique to extract the features of the subject database that belongs to the same application as the database used for the initial training (same classes). Once the features are obtained, we train the SVM classifier and evaluate the performance of each deep feature. To further improve this deep-based FER architecture, the calculated features may be concatenated with other descriptors such as HOG, Gabor filters, and LBP descriptors before proceeding to the classification step.

Evaluation of OPD-GQMBP neighborhood size configuration

The proposed OPD-GQMBP descriptor is a generic method defined by the neighborhood size n, which can be seen as a userspecified parameter depending on the needs of the considered application. To find the best value for FER, we conducted an experiment evaluating the performance of four configurations: n = 3, 5, 7, and 9. For each, we evaluated the FER framework on the five datasets using the LOSO protocol. The smaller neighborhood sizes provide less computational cost, but with weak discriminative power, and larger sizes enhance the discriminative power, but require more resources to store and classify the extracted features. For example, a neighborhood size of 3 (OPD-GQMBP 3 ) generates only 4×2 3 = 32 possible patterns, whereas neighborhood sizes of 5 (OPD-GQMBP 5 ), 7 (OPD-GQMBP 7 ), and 9 (OPD -GQMBP 9 ) produce 128, 512, and 2048 patterns, respectively. Table 2 shows the recognition rates obtained from this experiment. It can be concluded that with a higher neighborhood size, we obtain more discriminative feature extraction. The most effective configuration is n=7, which managed to reach the top performance on 4 databases. Thus, the 512 generated patterns proved to be sufficient for characterizing the seven emotional classes. Here, OPD -GQMBP 9 achieved a top accuracy of 97.53% on the CK+ database outperforming OPD -GQMBP 7 , but suffered a performance drop on the other datasets. Indeed, in some cases, methods that generate a high number of patterns may cause a performance drop owing to pattern redundancy. It is clear that with OPD -GQMBP 3 the configuration cannot outperform the other configurations; however, the recorded accuracies remained prominent regarding the low computation (32 patterns only). The performances of OPD -GQMBP 3 and OPD -GQMBP 5 are extremely similar with small variations. We acknowledge that we could not evaluate neighborhood sizes of greater than 9 owing to the required computational resources (out of memory). Therefore, we adopt OPD -GQMBP 7 in the rest of this paper because it corresponds to the best among the tested configurations.

Comparative analysis against state-of-the-art handcrafted and deep feature methods

The goal of this comprehensive analysis is to compare the performance of the proposed OPD-GQMBP descriptor to those recorded in the literature on feature extraction methods, including handcrafted and deep-based approaches. We considered 12 recent and powerful LBP variants published in high-impact journals and proposed for various applications, mainly texture classification and face recognition. In addition, we evaluated the top-10 state-of-the-art deep learning models proposed thus far. These models were initially trained on the FER2013 database, each of them reached a validation accuracy of above 60% on 25,000 images, which can be considered extremely significant. We then use transfer learning to extract the features of the five databases adopted in this study. We respected the same evaluation protocol (LOSO) to provide a fair and systematic analysis. Table 3 lists the performance achieved by each method or model. We provide two metrics. The first metric is the average accuracies recorded for all runs of each database depending on the number of individuals, of which JAFFE has 10 runs, CK+ has 106, KDEF has 70, OuluCasia has 80, and the RaFD database has 67. The second metric is the maximum accuracy reached for all runs per database. We highlight the top 3 average values with green color.

It can be seen from the average accuracies that the proposed OPD-GQMBP descriptor managed to score the top performance on all tested datasets. For the CK+ database, the OPD-GQMBP descriptor achieved a score of 96.48% on 106 runs with a maximum recognition accuracy of 100%, maintaining more than a 2% gap over the next best method, i.e., the handcrafted MNTCDP descriptor, which managed to secure an average accuracy of 94.36%. All of the evaluated methods were capable of reaching a maximum accuracy of 100% on the CK+ dataset. Moreover, the majority of the handcrafted methods performed at above 87.81% when applying the DCP descriptor, whereas the deep features reached a maximum average of 88.76% with VGG19 only and 54.18% (AlexNet) as the minimum average for CK+. As indicated in Table2, we found that n=9 for the configuration of the OPD-GQMBP method scored 97.53% on CK+, which was a 1% improvement compared to OPD-GQMBP with n=7 adopted in this evaluation. In addition, OPD-GQMBP reached 77.62% on JAFFE with a lead of 3% over the rest of the methods, with the second highest average accuracy being 74.76% for the DCP descriptor followed by ELGS as the third-best performing method at 73.81%. According to the maximum accuracy on the JAFFE database, only the OPD-GQMBP and DCP methods managed to score 100%. The Japanese females with high facial similarities make the JAFFE an extremely tough benchmark for FER systems. The results demonstrate that the JAFFE database is quite challenging, particularly for deep features where the top average accuracy was limited to 56.67%, as obtained by VGG16, whereas the handcrafted features granted an average accuracy of above 64.76%, which was reached by the LOOP descriptor. For the KDEF database, the first remark to be concluded is that the OPD-GQMBP descriptor is the only method that breaks the 90% average performance ceiling, and the best state-of-the-art method reached 88.37% (LDTP). The performance of the handcrafted methods and deep methods varied from 78.57% to 88.37% and from 65.31% to 76.73%, respectively. The lead gap on this database was again approximately 2% in favor of the OPD-GQMBP descriptor, which proves its discriminative power for FER applications. Despite the quality of the recorded images in terms of resolution, lighting conditions, and a uniform background, the KDEF database is also challenging in face of CK+. It presents fewer individuals (fewer runs) expressing the seven emotions in different manners, resulting in more intraclass similarities and approaching spontaneous facial expressions. Oulu-Casia can be considered as the toughest among the adopted benchmarks owing to the blur images and the South Asian individuals composing this database. The OPD-GQMBP descriptor scored 77.32% as the overall best performing method, followed by ARCS-LBP (75.5%) and DC (74.64%) descriptors. The lowest accuracy was recorded by the AlexNet deep network, reaching only 29.64%. In addition, all methods achieved a 100% maximum accuracy except for MNTCDP, AlexNet, GoogLeNet, and Inceptionv3. Moreover, only the ELGS, DC, ARCS-LBP, and OPD-GQMBP methods exceeded a 70% average accuracy, whereas the DCP descriptor, which had the second highest rank on JAFFE, found its performance limited to 45.89% as the lowest average accuracy over all handcrafted LBP-like methods. [START_REF] Ding | Multi-directional multi-level dual-cross patterns for robust face recognition[END_REF] Multi-Directional Multi-Level Dual-OPD-GQMBPs Patterns (Face recognition) 2015 3 DRLBP [START_REF] Mehta | Dominant rotated local binary patterns (drlbp) for texture classification[END_REF] Dominant Rotated Local Binary Patterns (Texture classification) 2016 4 QBP [START_REF] Zeng | Quad binary pattern and its application in mean-shift tracking[END_REF] Quad Binary Pattern (Target tracking) 2016 5 ELGS [START_REF] Elmerabet | Attractive-and-repulsive center-symmetric local binary patterns for texture classification[END_REF] Extended Local Graph Structure (Texture classification) 2016 6 LNDP [START_REF] Ouslimani | Rotation-invariant features based on directional coding for texture classification[END_REF] Local neighborhood difference pattern (Texture classification) 2017 7 DC [START_REF] Verma | Local neighborhood difference pattern: A new feature descriptor for natural and texture image retrieval[END_REF] Rotation-invariant features based on directional coding (Texture classification) 2018 8 LCCMSP [START_REF] El Merabet | Local concave-and-convex microstructure patterns for texture classification[END_REF] Local Concave-and-Convex Micro-Structure Patterns (Texture classification) 2018 9 LDTP [START_REF] Khadiri | Local directional ternary pattern: A new texture descriptor for texture classification[END_REF] Local directional ternary pattern (Texture classification) 2018 10 LOOP [START_REF] Chakraborti | Loop descriptor: Local optimal-oriented pattern[END_REF] Local Optimal Oriented Pattern (Spieces recognition) 2018 11 MNTCDP [START_REF] Kas | Mixed neighborhood topology cross decoded patterns for image-based face recognition[END_REF] Mixed Neighborhood Topology OPD-GQMBPs Decoded Patterns (Face recognition) 2018 12 ARCS-LBP [START_REF] Elmerabet | Attractive-and-repulsive center-symmetric local binary patterns for texture classification[END_REF] Attractive-and The RaFD database is collected by a set of well-trained individuals that clearly express eight emotions (basic emotions + contempt). Indeed, the peak average accuracy exceeded 97% by the proposed OPD-GQMBP descriptor with a minor lead (0.19%) against the LDTP and LCCMSP methods (97.01%). Moreover, all of the handcrafted descriptors reached above 90%, except for DCP (87.31%). By contrast, ResNet50 was the best among the deep feature methods, with an average accuracy of 84.51%. In terms of stability, the OPD-GQMBP descriptor performed well on the five datasets, always reaching the top average accuracies and 100% at all times. The ELGS method also presented a stable performance across all databases. By contrast, DCP suffered a performance decrease on the KDEF and OuluCasia datasets. For deep feature methods, VGG16 can be considered as the best performing deep feature method. The deep learning networks did not perform well on the five benchmarks, despite reaching a validation accuracy of above 60% on 25,000 images of the FER2013 database. The problem here is that the deep learning methods should be fine-tuned on each dataset before extracting the features to obtain satisfactory results. The applied application in this paper is personindependent, and to ensure that the probe images of a given person are unseen by the framework, we should perform fine tuning on each run and for each deep method. Hence, because we have a set of 333 persons on the five datasets and we consider 10 deep learning models, we need 3330 fine tunes to expect satisfying results from these models, which are time-and resource-consuming. Moreover, such frameworks are intended for real implementations and deployments, and fine tuning is not always a possible option. In addition, the probe images will have generally different characteristics than the trained images.

Comparison against state-of-the-art FER systems

In this subsection, we compare the results obtained by our proposed FER framework to those achieved by previous studies in the field of facial expression recognition. Tables 4,5, 6, 7 and 8 list the highest accuracies on the five datasets reported in wellindexed journals and conferences of the literature. We tried to collect the maximum number of studies that followed the same adopted evaluation protocol (person-independent).

As can be seen in Table 4, the proposed FER framework outperforms all listed systems, including both handcrafted and deep-based features in the CK+ database. We reached an accuracy of 96.48% (97.53% with the OPD-GQMBP at a neighborhood size of n=9), whereas the best accuracy of the other state-of-the art methods was 94.96% achieved using LPQ with an SVM classifier. Moreover, the majority of the published approaches reached between 90% and 94%. In the JAFFE database, the accuracies of the other state-of-the-art methods are low compared to that of CK+, where the maximum reported accuracy is 76.46% scored by the CFER-based framework, which was outperformed by our FER framework (77.62%). The proposed framework managed to surpass with a significant margin all other approaches on the KDEF dataset, except for the WCFN (89.55%) and AlexNet (89.33%) based systems, where the margin is small (0.65% and 0.87% of our framework performance, [START_REF] Sun | Automatically query active features based on pixel-level for facial expression recognition[END_REF] Deep 69.4 C-classLDA-NN [START_REF] Kyperountas | Salient feature and reliable classifier selection for facial expression classification[END_REF] Deep 74.73 LBP based LDA [START_REF] Shan | Facial expression recognition based on local binary patterns: A comprehensive study[END_REF] Handcrafted 73.4 CFER [START_REF] Sun | Cognitive facial expression recognition with constrained dimensionality reduction[END_REF] Handcrafted 76.46 Features fusion [START_REF] Poursaberi | Gauss-laguerre wavelet textural feature fusion with geometrical information for facial expression identification[END_REF] Handcrafted 70

Proposed

Handcrafted 77.62 [START_REF] St | Emotion recognition: The influence of texture's descriptors on classification accuracy[END_REF] Handcrafted 78 VGG-Face Deep [START_REF] Zavarez | Crossdatabase facial expression recognition based on fine-tuned deep convolutional network[END_REF] Deep 72.55 SCAE [START_REF] Ruiz-Garcia | Stacked deep convolutional auto-encoders for emotion recognition from facial expressions[END_REF] Deep 86.73 DFD [START_REF] Sun | Discriminative feature learning-based pixel difference representation for facial expression recognition[END_REF] Handcrafted 82.24 WCFN [START_REF] Ye | Facial expression recognition via region-based convolutional fusion network[END_REF] Deep 89.55 MobileNet [START_REF] Guo | Multi-view laplacian least squares for human emotion recognition[END_REF] Deep 73.74 EDR-PCANet [START_REF] Sun | Automatically query active features based on pixel-level for facial expression recognition[END_REF] Deep 80.61 Proposed Handcrafted 90.2 respectively). On the OuluCasia dataset, the proposed framework (77.32%) outperformed all other state-of-the-art methods, where the top accuracy was limited to 75.52%, which was achieved by Atlases. On the RaFD database, our proposed framework obtained 97.2%. Many studies described in the literature achieved an accuracy of nearly 97%. However, the majority applied only seven emotion classes. Overall, we conclude that the proposed facial expression recognition framework managed to outperform all of the state-of-the-art methods tested.

Confusion matrix-based analysis for the FER

The confusion matrix allows the performance of the recognition according to each label (each emotion in our case). Through this chart, we are capable of analyzing the recognition rate of each emotion, as well as which are the easiest and most difficult emotions to recognize. In addition, this analysis allows us to identify which emotions affect the others. Figures 12,[START_REF] Jung | Joint fine-tuning in deep neural networks for facial expression recognition[END_REF] In the CK+ database, the Happy and Surprise classes were perfectly recognized, whereas Fear and Sad experienced the highest misclassification rate (16.0% and 14.3%, respectively). Fear was confused three times with Happy and once with Sad. A Neutral emotion was the most affective emotion (with a 12.5% false-negative rate) and was predicted four times in the case of Sad and twice for Angry. For the JAFFE database, Happy and Neutral were the easiest to identify with an accuracy of 86.7%, and the hardest was Fear (60%), which was confused with all emotions, i.e., three times with Disgust, Neutral, and Surprise, and once with the remaining emotions. The Sad class presented the highest false-negative rate (34.4%). The females composing this database express Angry and Sad emotions in similar ways because five Angry samples were identified as Sad. For the KDEF dataset, the errors were less than those of JAFFE. Happy and Neutral once again were highly recognized emotions (98.6%), but Angry was a challenging class, with a rate of only 78%. Our FER framework confused Disgust seven times with Fear and confused Fear six times with Sad. Happy and Disgust did not affect any other emotion except once each for Angry. As expected from OuluCasia, the misclassification errors were extremely high. The Happiness emotion is the only one to be highly recognized with an accuracy of 96.3%, followed by Surprise (91.3%). The remaining accuracies were between 66.3% and 80%. In addition, Angry and Neutral dramatically influenced the other emotions by 34.5% and 36.6%, respectively. In the RaFD database, all rates were high with perfect recognition rates for three classes (i.e., Contempt, Fear, and Sad) in addition to Angry and Disgust, which were misclassified only once. However, there is a mutual confusion between Happy and Surprise because three Happy samples were identified as Surprise and Surprise samples were identified as Happy four times. For all databases, we determined that Happy and Neutral are the most recognized emotions, and that Neutral is an extremely perturbing framework, presenting high falsenegative rates on many benchmarks.

Implementation and execution time

The FER experiments presented were conducted on an Alienware Aurora R8 with a 4.6 GHz Core i7-8th Processor Boost, 12 threads, and 48 GB of RAM, running with the Ubuntu 18.04.2 LTS (Bionic Beaver) operating system and equipped with two GTX1080Ti GPUs. The developed framework is coded using Python 3.7 and MATLAB 2019b environments. The authors are willing to share all codes used by the community once this paper is published. The computational cost is one of the key performance indicators considered in machine learning applications. Therefore, we ran an experiment that calculates the elapsed time required to predict the label of a given input image with a pixel resolution of 762 × 562, highlighting the execution time of each step of the proposed framework:

• Dlib landmark detection.

• Shape feature extraction based on HOG descriptor.

• Appearance feature extraction using the proposed OPD-GQMBP as well as the state-of-the-art handcrafted methods denoted as "getFeatures".

• PCA dimensionality reduction.

• Label prediction using the SVM library.

We excluded the deep-learning methods from this evaluation because they require GPUs to perform the feature extraction; thus, it will be unfair to compare CPU-based methods with GPU-based approaches regarding the computation power of the GPUs. The CPU can be used to calculate the feature vector using a deep model, but it takes approximately 4 s to compute it for an input image with a size of only 224 × 224 (three-times smaller than the size of the original image), which is extremely high compared to the handcrafted features. The obtained computational times are included in Table 9 and illustrated graphically in Figure 17 (for greater readability). As can be seen, the elapsed times for the landmark detection, HOG shape feature extraction, and SVM prediction did not change across the evaluated methods, where they recorded times of 15.9, 19.58, and 2.306 ms, respectively. The process of extracting the appearance features was demonstrated to be the most time-consuming within our framework (requiring more than 50% of the total time). The fastest handcrafted method is DRLBP, which extracted the features from the 49 landmarks in 31.5 ms, whereas LDTP took 311.6 ms to extract these features and is thus judged as the heaviest approach. However, the DRLBP did not perform well in terms of classification accuracy, whereas the proposed OPD-GQMBP descriptor managed to offer an execution time of only 90.34 ms, which is quite beneficial regarding its high performance as demonstrated earlier. Moreover, all of the bestperforming descriptors took more than 100 ms to compute the appearance features. Although the PCA stage is common to all descriptors for a dimension reduction, its execution time was variable and affected by the number of generated patterns of each handcrafted method as a high number of patterns leads to more computations. Nevertheless, it can be remarked that the PCA computation times are similar for the methods sharing the same sized patterns generated. The methods producing 256 patterns such as LOOP, QBP, DC, ARCS-LBP, and DRLBP recorded a PCA computation time of approximately 11 ms. The PCA process of the proposed OPD-GQMBP descriptor as well as those producing 512 patterns took approximately 20 ms, whereas LDTP and LNDP methods generating 1024 patterns took approximately 36 ms. By contrast, the PCAbased dimensionality reduction process related to the LCCMSP (2048 patterns) descriptor was completed within 83.5 ms. Overall, we can disclose that the proposed framework along with the OPD-GQMBP method managed to predict the label of an image with a pixel resolution of 762 × 562 in less than 150 ms, allowing a processing of 7 frames per second, which is considered as a real-time feedback according to the specifications of person-independent FER systems [START_REF] Duncan | Facial emotion recognition in real time[END_REF][START_REF] Michel | Real time facial expression recognition in video using support vector machines[END_REF].

Conclusion and Future works

In this paper, we proposed a new handcrafted descriptor, OPD-GQMBP, along with a framework for facial expression recognition. We considered using individual people as the most challenging task acquiring the intention of many computer vision researchers, as can be seen from the number of studies published in this context. The OPD-GQMBP method applies orthogonality and parallelism geometrics to cover the most prominent pixels within a neighborhood. OPD-GQMBP is a generic descriptor with the neighborhood size as a parameter, which can be adjusted to obtain the most discriminative feature extraction regarding the intended application. For FER, we found that the neighborhood size of n = 7 is the best configuration of the proposed OPD-GQMBP descriptor. With regard to the developed FER framework, we combined the shape and texture feature extraction techniques applied considering 49 landmarks detected by the Dlib package on each input image. This strategy allowed a powerful FER system to outperform many existing methods in the literature on five widely used benchmarks. To fairly judge the performance of the OPD-GQMBP descriptor, a comparative analysis was conducted on five benchmarks. We assessed the performance of 12 handcrafted and 10 state-of-the-art deep feature methods applied within the developed framework and respecting the same evaluation protocol. Based on the analysis of the experimental results, it was inferred that the OPD-GQMBP descriptor managed to outperform all other methods, including deep features, and reached the top recognition rates. The accuracies recorded by the developed FER system combined with the proposed OPD-GQMBP descriptor were the highest in comparison to the those reported by the existing state-of-the-art FER systems that adopted the same protocol (person-independent LOSO). Although it is true that our system managed to outperform many of the state-ofthe-art systems, it needs improvement on databases containing Asian individuals (e.g., JAFFE and Oulu CASIA) because they tend to present similar facial features, which confuse the classifier in the case of the LOSO protocol. We believe that the ultimate solution is to reduce the number of extracted appearance features and focus on the binary patch calculated based on the detected landmarks. This patch should incorporate more information, and not only the landmark location, and therefore handcraft a shape descriptor to obtain the most prominent information. Moreover, this proposal will help in the development of generic person-independent FER systems because the input images will be coded into a common patch. Even though our framework, along with the proposed descriptor, performs efficiently in terms of computational time, we think that there is room for improvement using other dimension reduction strategies and/or landmark selections. We are also investigating an enhancement of the performance of our framework by utilizing other sophisticated classifiers than an SVM and combining learnable features with the proposed OPD-GQMBP descriptor. In addition, we intend to extend the set of the studied emotion classes with the compound classes, reaching 20 different classes. We also considered creating a mixed database from existing databases to gather all challenges in a unique benchmark, which will offer more challenging testing and evaluation.

15.9 15.9 15.9 15.9 15.9 15.9 15.9 15.9 15.9 15.9 15.9 15.9 15.9

Figure 1 :

 1 Figure 1: Facial Expression Recognition difficulty levels.

Figure 2 :

 2 Figure 2: OPD-GQMBP neighborhood topologies.

F a c i a l E x p r e s s i o n s R e c o g n i t i o n 4 5 13Ic

 5

Figure 3 :

 3 Figure 3: Cartesian system used to identify the pixel coordinates for each line and group.

Figure 4 :

 4 Figure 4: Overall view of the proposed FER framework.

Figure 5 :

 5 Figure 5: Texture feature extraction workflow based on the proposed OPD-GQMBP operator.

4. 1 .

 1 Experimental datasets • The Japanese Female Facial Expression (JAFFE) dataset has 213 facial expression images, representing the 7 basic emotions: Anger (30 images), Disgust (29 images), Fear (32 images), Happiness (31 images), Sadness (31 images), Surprise (30 images), and Neutral (30 images).Figure6illustrates some examples of facial expressions. This database is extremely challenging regarding the particularity of Japanese females that have similar face features, generating more inter-class visual features• The Karolinska Directed Emotional Faces (KDEF) is a widely used dataset for evaluating FER methods. It includes 70 individuals (50% men, 50% women) that uniformly express basic emotions over two sessions, leading to a total of 980 images. In our experiments, we considered only one session to have only one observation per emotion for each person (490 images), which resulted in

Figure 6 :

 6 Figure 6: Samples of two subjects from JAFFE database

Figure 7 :Figure 8 :

 78 Figure 7: Samples of a subject from KDEF database

Figure 9 :

 9 Figure 9: Samples of two subjects from the OuluCasia database

Figure 10 Figure 10 :

 1010 Figure 10 displays shows the 8 facial expressions of a person from the RaFD database. Sad Neutral Happy

Figure 11 :

 11 Figure 11: Deep-feature based FER approach.

  , 14, 15 and 16illustrate the confusion charts generated from the results of our proposed FER framework for CK+, JAFFE, KDEF, OuluCasia, and RaFD, respectively.

Figure 12 :Figure 13 :

 1213 Figure 12: Confusion matrix of the seven emotions of CK+

Figure 14 :Figure 15 :

 1415 Figure 14: Confusion matrix of the seven emotions of KDEF

Figure 16 :

 16 Figure 16: Confusion matrix of the eight emotions of RaFD

Figure 17 :

 17 Figure 17: Elapsed time in milliseconds required to compute the features and predict the label of an input image

Table 1 :

 1 Summary of the Handcrafted and Deep feature methods of the literature tested and evaluated in this paper

	N •	Abbreviation	Complete name (Application)

Table 2 :

 2 Average FER rate of each OPD-GQMBP configuration (neighborhood size) for all databases

				-Repulsive Center-Symmetric Local Binary Patterns (Texture classification)	2019
	13 VGG16		Visual Geometry Group -University of Oxford -19 Layers	2014
	14 VGG19		Visual Geometry Group -University of Oxford -16 Layers	2014
	15 ResNet18		Residual Neural Network -Microsoft Research -18 Layers	2015
	16 ResNet50		Residual Neural Network -Microsoft Research -50 Layers	2015
	17 ResNet101		Residual Neural Network -Microsoft Research -101 Layers	2015
	18 AlexNet		Convolutional Neural Network by Alex Krizhevsky -8 Layers	2012
	19 DenseNet		Densly Connected CNN -Cornell Tsinghua Univs & Facebook -201 Layers	2017
	20 GoogLeNet		Inception version1 based on LeNet -Google -22 Layers	2015
	21 Inception v3		Inception version3 -Google -159 Layers	2015
	22 InceptionResNet v2 Inception-ResNet-version2 CNN (Xception) -Google -126 Layers	2017
	N S ize Config	CK+	JAFFE	KDEF	OuluCasia	RaFD
	OPD -GQMBP 3	95.74	73.33	88.57	73.93	96.08
	OPD -GQMBP 5	96.01	73.33	87.96	74.82	95.9
	OPD -GQMBP 7	96.48	78.57	90.2	77.32	97.39
	OPD -GQMBP 9	97.53	71.9	87.96	75.95	96.08

Table 3 :

 3 Average and maximum accuracies recorded on the five datasets by each method.

			JAFFE	KDEF	CK+	OuluCasia	RaFD
		Method	Avg	Max	Avg	Max	Avg	Max	Avg	Max	Avg	Max
		VGG16	56.67 76.19 76.53	100	86.34	100	56.96	100	82.84	100
		VGG19	53.81 76.19 76.73	100	88.76	100	58.57	100	81.53	100
		ResNet18	35.24 57.14 72.24	100	72.67	100	52.32	100	83.02	100
		ResNet50	44.76	61.9	75.71	100	77.32	100	56.96	100	84.51	100
	Deep Features	ResNet101 AlexNet DenseNet	52.86 71.43 70.82 43.81 66.67 66.94 48.1 80.95 70.2	100 100 100	76.79 54.18 76.15	100 100 100	53.57 29.64 71.43 67.54 100 78.17 50.36 100 77.8	100 100 100
		GoogLeNet	47.14 66.67 71.63	100	69.98	100	44.64 85.71 79.66	100
		Inceptionv3	39.52 57.14 65.31	100	71.22	100	44.46 85.71 70.52	100
		InceptionResNetv2	47.62 66.67 76.33	100	80.17	100	55	100	81.72	100
		ELGS	73.81 95.24 85.71	100	92.59	100	70	100	95.34	100
		DSLGS	60.95 85.71 78.57	100	91.2	100	64.64	100	91.23	100
		MNTCDP	69.05 85.71 85.51	100	94.36	100	55.36 85.71 95.15	100
	Handcrafted LBP Variants	QBP DRLBP LNDP DCP DC LCCMSP LOOP	62.38 85.71 79.39 70 85.71 84.49 69.52 95.24 86.94 74.76 100 69.18 66.67 95.24 84.69 71.43 95.24 86.12 64.76 85.71 85.92	100 100 100 100 100 100 100	91.99 90.35 92.86 87.81 89.87 92.28 91.7	100 100 100 100 100 100 100	66.43 65.36 68.75 45.89 74.64 69.29 69.64	100 100 100 100 100 100 100	91.79 96.08 96.08 87.31 95.52 97.01 96.27	100 100 100 100 100 100 100
		LDTP	70	90.48 88.37	100	93.3	100	67.5	100	97.01	100
		ARCS-LBP	65.24 95.24 85.51	100	91.45	100	75.5	100	96.64	100
	Proposed OPD-GQMBP descriptor 77.62	100	90.2	100	96.48	100	77.32	100	97.2	100

Table 4 :

 4 State-of-the-art person-independent FER accuracies on CK+ database

	Methods	Type	Avg accuracy
	LPDP [22]	Handcrafted	94.5
	DCNN [27]	Deep	94.44
	DNN [26]	Deep	93.52
	CNN+AFM [43]	Deep	89.84
	AlexNet+SVM [43]	Deep	86.83
	GoogLeNet [43]	Deep	85.71
	STM-ExpLet [21]	Deep	94.13
	LTeP+SVM [2]	Handcrafted	94.93
	LPQ+SLPM+NN [41] Handcrafted	94.61
	WPLBP [6]	Handcrafted	91.72
	Proposed	Handcrafted	97.53

Table 5 :

 5 State-of-the-art person-independent FER accuracies on JAFFE database

	Methods	Type	Avg accuracy
	LTeP+SVM [2]	Handcrafted	67.14
	LPQ+SLPM+NN [41] Handcrafted	67.61
	EDR-PCANet

Table 6 :

 6 State-of-the-art person-independent FER accuracies on KDEF database

	Methods	Type	Avg accuracy
	AlexNet+FC6+LDA [10] Deep	89.33
	HOG+SRC

Table 7 :

 7 State-of-the-art person-independent FER accuracies on OuluCasia database

	Methods	Type	Avg accuracy
	STM-ExpLet [21]	Deep	74.59
	LBP+Gabor+SVM [50] Handcrafted	74.37
	HOG 3D [17]	Handcrafted	70.63
	AdaLBP [49]	Handcrafted	73.54
	Atlases [11]	Deep	75.52
	Proposed	Handcrafted	77.32

Table 8 :

 8 State-of-the-art person-independent FER accuracies on RaFD database

	Methods	Type	Avg accuracy
	Visual Attention CNN [37]	Deep	95.2
	DS+FE+GEM+SVM [25]	Handcrafted	90.8
	LPQ+FE+GEM+SVM [25] Handcrafted	94.4
	LBP+FE+GEM+SVM [25] Handcrafted	94.5
	Metric Learning [12]	Deep	95.95
	BAE-BNN-3 [35]	Deep	96.93
	W-CR-AFM [43]	Deep	96.27
	Net1-Net2 [36]	Deep	93.41
	Proposed	Handcrafted	97.2

Table 9 :

 9 Elapsed time in milliseconds to compute the features and predict the label of an input image

	Method	Dlib	Hog	getFeatures	PCA	SVM	Total
	ARCS-LBP	15.9	19.58	61.4	11.6	2.235	110.715
	DC	15.9	19.58	167.3	10.1	2.235	215.115
	DCP	15.9	19.58	62.37	20.4	2.235	120.485
	DRLBP	15.9	19.58	31.6	11.7	2.235	81.015
	DSLGS	15.9	19.58	30.8	20.6	2.235	89.115
	ELGS	15.9	19.58	65.22	20.6	2.235	123.535
	LCCMSP	15.9	19.58	145.72	83.5	2.235	266.935
	LDTP	15.9	19.58	311.6	36.21	2.235	385.525
	LNDP	15.9	19.58	34.22	35.5	2.235	107.435
	LOOP	15.9	19.58	293.2	11.7	2.235	342.615
	MNTCDP	15.9	19.58	101.8	20.5	2.235	160.015
	OPD-GQMBP	15.9	19.58	90.34	20	2.235	148.055
	QBP	15.9	19.58	35.5	12.4	2.235	85.615

19.58 19.58 19.58 19.58 19.58 19.58 19.58 19.58 19.58 19.58 19.58 19.58 19.58 61.4 167.3 62.37 31.6 30.8 65.22 145.72 311.6 34.22 293.2 101.8 90.34

1

landmark points detected using the Dlib algorithm on each input image, which are believed to carry relevant and sufficient information to recognize the emotion expressed in the input image. To show its effectiveness for the person-independent FER, our system is extensively evaluated on five well-known and widely used benchmarks in the literature: KDEF, CK+, RaFD, JAFFE, and OuluCasia. To ensure person independence in our

Acknowledgments

The authors gratefully acknowledge the funding received from CNSRT-Maroc (Centre National de la Recherche Scientifique et Technique) and the French government (Eiffel scholarship).


Helene Lazareth

Neila Talbi

Nassim Kamar

Charlène Levi

Bruno Moulin

Sophie Caillard

Luc Frimat

Jonathan Chemouny

Valérie Chatelet

Clément Vachey

Renaud Snanoudj

Thibaud Lefebvre

Alexandre Karras

Laurent Gouya

Caroline Schmitt

Hervé Puy

Nicolas Pallet

Kidney transplantation improves the clinical outcomes of Acute Intermittent Porphyria

Keywords: Acute intermittent porphyria, kidney transplantation, chronic kidney disease, end-stage renal disease, aminolevulinic acid, porphobilinogen, porphyrin precursors, porphyria cutanea tarda

Background. Acute Intermittent Porphyria (AIP) is a rare inherited autosomal dominant disorder of heme biosynthesis. Porphyria-associated kidney disease occurs in more than 50% of the patients with AIP, and end stage renal disease (ESRD) can be a devastating complication for AIP patients. The outcomes of AIP patients after kidney transplantation are poorly known.

Methods. We examined the outcomes of 11 individuals with AIP, identified as kidney transplant recipients in the French Porphyria Center Registry.

Results

. AIP had been diagnosed on average 19 years before the diagnosis of ESRD except for one patient in whom the diagnosis of AIP had been made 5 years after the initiation of dialysis. Median follow-up after transplantation was 9 years. A patient died 2 months after transplantation from a cardiac arrest and a patient who received a donation after cardiac death experienced a primary non-function. No rejection episode and no noticeable adverse event occurred after transplantation. Serum creatinine was on average 117 µmol/l, and proteinuria <0.5 g/l in all patients at last follow up. All usually prescribed drugs after transplantation are authorized except for trimethoprim/sulfamethoxazole. Critically, acute porphyria attacks almost disappeared after kidney transplantation, and skin lesions resolved in all patients. Conclusion. Kidney transplantation is the treatment of choice for AIP patients with ESRD and dramatically reduces the disease activity.

Introduction

Porphyria encompasses a group of eight metabolic disorders of the heme biosynthesis pathway [1,2]. Acute Intermittent Porphyria (AIP) is an inherited autosomal dominant disorder due to a defect in hydroxymethylbilane synthase (HMBS) [1,[START_REF] Siegesmund | The acute hepatic porphyrias: current status and future challenges Best[END_REF], the third enzyme of the biosynthetic pathway that catalyzes porphobilinogen (PBG) polymerization into hydroxymethylbilane (Figure 1). The estimated prevalence of a symptomatic disease in the Caucasian population is ~1/180 000, and the estimated frequency of clinical pathogenic variants is ~1/1750 [START_REF] Chen | Acute Intermittent Porphyria: Predicted Pathogenicity of HMBS Variants Indicates Extremely Low Penetrance of the Autosomal Dominant Disease[END_REF][START_REF] Nordmann | Acute intermittent porphyria: prevalence of mutations in the porphobilinogen deaminase gene in blood donors in France[END_REF]. As AIP is a monogenic disorder, this extremely low penetrance (~1%) suggests a critical role for modifying factors (genetic and/or environmental) in predisposing to symptoms, but also a possible underreporting of the cases due to misdiagnosis. Indeed, AIP is a relatively unknown disease for many physicians. AIP is characterized by the occurrence of acute neurovisceral attacks (abdominal pain, nausea, vomiting, constipation, neurologic or psychiatric symptoms) accompanied by the excretion of large amounts of the porphyrin precursors delta aminolevulinic acid (ALA) and PBG in urine.

Whereas the vast majority of AIP patients will experience a limited number of attacks over a lifetime, 3-5% of patients with AIP will have recurrent acute attacks, which can lead to a debilitating medical condition [START_REF] Elder | The incidence of inherited porphyrias in Europe[END_REF].

ALA is a highly hydrophilic molecule that can cross blood barriers, and is freely excreted in the urine. As a potent vasoconstrictor, ALA promotes ischemia and injury in target organs, and, as such, is considered to be directly responsible for the symptoms of AIP [START_REF] Laafi | Pro-oxidant effect of ALA is implicated in mitochondrial dysfunction of HepG2 cells[END_REF]. In the liver and kidneys, ALA is produced by ALA synthase 1 (ALAS1), an isoform under the negative control of heme [START_REF] Manceau | Acute hepatic and erythropoietic porphyrias: from ALA synthases 1 and 2 to new molecular bases and treatments[END_REF] (Figure 1). The capacity of a drug or a clinical situation to precipitate acute porphyric attacks, i.e. the "porphyrogenicity", will depend on its ability to inhibit ALAS1 physiological feedback-control via reduction of the regulatory heme pool within the hepatocyte, for example in inducing P450 cytochromes expression or degradation. Thus, ALAS1 overexpression is a feature of AIP attacks, leading to an increased synthesis of ALA and PBG, and to their systemic accumulation since deficient HMBS cannot fully metabolize PBG into porphyrins.

Porphyrin precursors have a deleterious effect on the kidney. Porphyria-associated kidney disease (PAKD) occurs in more than 50% of the patients with symptomatic AIP. Supporting this, the specific role of ALA and PBG in the chronic renal structural deterioration that accompanies AIP has recently been demonstrated [START_REF] Pallet | High prevalence of and potential mechanisms for chronic kidney disease in patients with acute intermittent porphyria[END_REF]. Two types of renal lesions characterize PAKD: tubular atrophy and interstitial fibrosis associated with non-specific arteriosclerosis, and less frequently, chronic fibrous intimal hyperplasia associated with focal cortical atrophy is observed in the most severe cases [START_REF] Pallet | High prevalence of and potential mechanisms for chronic kidney disease in patients with acute intermittent porphyria[END_REF]. PAKD is not a severe disease in most of the cases, and only a small proportion of patients apparently reach end stage renal disease (ESRD). However, ESRD is a detrimental condition for AIP patients [START_REF] Sardh | Porphyrin precursors and porphyrins in three patients with acute intermittent porphyria and end-stage renal disease under different therapy regimes[END_REF], and kidney transplantation (KTx) can be considered as a therapeutic option. Indeed, case reports suggest that KTx in AIP patients may have a favorable outcome [START_REF] Nunez | Renal transplantation for chronic renal failure in acute porphyria[END_REF][START_REF] Barone | The tolerability of newer immunosuppressive medications in a patient with acute intermittent porphyria[END_REF][START_REF] El-Haggan | Sirolimus tolerability in a kidney transplant recipient with acute intermittent porphyria[END_REF]. However, critical issues remain unresolved, including the role of the KTx in the activity of the heme biosynthetic pathway, the changes in disease activity after KTx, and the tolerability of several of the medications prescribed to kidney transplant recipients (KTR). To address these issues, we undertook this study to examine the long-term clinical outcomes of a cohort of KTR with AIP in France.

Patients and methods

Study population

The French Porphyria Center (http://www.porphyrie.net/) centralizes clinical, genetic and biological information from all AIP patients in France in relationship with caregivers and self reports. For the purpose of this study, a detailed analysis was completed on the subset of patients who developed ESRD and underwent KTx. The data collection period for the number of acute attacks was isometric, that is, centered on the date of the KTx, with same time periods before and after KTx, and dependent on the length of follow-up after KTx.

The institutional review board of Louis Mourier Hospital approved the study, written informed consent was obtained from all patients referenced in the database, and the database has been deposited to the Commission Nationale Informatique et Libertés (CNIL agreement number 1187326).

AIP diagnosis

The criteria for AIP diagnosis followed the European Porphyria Network guidelines [1,[START_REF] Tollanes | Establishing a network of specialist Porphyria centres -effects on diagnostic activities and services Orphanet[END_REF].

AIP diagnosis was performed in the French Porphyria Center. Two "AIP status" have been considered. i. The "AIP patient" status is defined by the association of at least one acute porphyric attack (occurrence of abdominal pain, nausea, vomiting, constipation, neurologic or psychiatric symptoms) with a typical porphyrin precursors excretion profile urines. The diagnosis is confirmed by a 50% decrease in HMBS activity in erythrocytes and/or by the identification of a causative mutation in HMBS gene by direct sequencing as previously described [START_REF] Puy | Molecular epidemiology and diagnosis of PBG deaminase gene defects in acute intermittent porphyria[END_REF]; ii. The "asymptomatic carrier" status is defined after a family screening to identify those with asymptomatic (or latent) disease and is based on a deficient HMBS enzymatic activity complemented with a DNA analysis by direct sequencing to identify the causative mutation in the HMBS gene (which requires prior identification of the mutation in a related affected family member).

Clinical chemistry

Urinary ALA and PBG were measured using the Bio-Rad ALA/PBG column kit, according to the manufacturer's protocol and the values were reported to creatinine concentration in urine. Plasma ALA and PBG were measured by Ultra High Pressure chromatography (Inversed phase chromatography) coupled with a Tandem Mass spectrometer, after protein precipitation and derivatization.

Plasma porphyrins were measured, as already described [START_REF] Elder | Laboratory investigation of the porphyrias[END_REF]. Briefly, the plasma in phosphate buffer was compared with a uroporphyrin standard by scanning the fluorescence emission signal from 570 to 750 nm (excitation at 405 nm). Plasma porphyrins were separated by normal-phase high-performance liquid chromatography (HPLC) with fluorimetric detection.

Results

AIP and kidney disease histories

As of March 2020, 1889 individuals (alive or dead, 626 AIP patients and 1263 asymptomatic carriers) were listed in the French Porphyria Center registry. Eleven individuals (10 AIP patients and 1 asymptomatic carrier) were identified as KTR. Baseline clinical characteristics of the KTRs are shown in the Table 1 and the list of HMBS allelic variants is detailed in the Table S1. These patients except the asymptomatic carrier experienced a median value of 4 attacks in their life before KTx (range 2-28), and 3 of them required heme-arginate maintenance therapy.

The etiology of kidney disease attributed at the time of ESRD diagnosis was: "unknown cause" in 3 cases, "AIP" in 3 cases, "chronic tubulointerstitial kidney disease" in 2 cases, and "nephroangiosclerosis" in 2 cases (Table 1). The asymptomatic carrier had a malformative uropathy (therefore a kidney disease unrelated to AIP). Familial cases of chronic kidney disease or ESRD have been identified in 3 KTRs. All patients were hypertensive. Six dialyzed patients (out of 8 under dialysis, 75%), developed visible light-exposed site bullae and skin fragility similar to as to those that occur during acquired porphyria cutanea tarda (PCT).

Kidney transplantation outcomes

These patients were transplanted between 2001 and 2017 and were followed in diverse transplant centers throughout France. Median age at KTx was 57 years (range 38-71 years), and all but 2 patients were female (Table 2). Three patients received a kidney transplant preemptively, and the rest of the group underwent dialysis therapy for a median period of time of 2 years before transplantation (range 0-7), including 2 patients undergoing peritoneal dialysis. One patient received a living donor KTx, and one patient received a donation after cardiac death (DCD). All patients were followed for at least 5 years (median 9 years, range 0-15 years) after KTx except for the DCD patient who was transplanted in 2017, and a patient who died 2 months after KTx from cardiac arrest. No rejection episode was reported. No graft loss was noticed. At the last follow-up, median serum creatinine 117 113 µmol/l (range 73-196 µmol/l), and proteinuria was <0.5 g/l in all patients. Histology was unavailable for the great majority of kidney allografts.

In terms of infectious complications, 1 patient had recurrent urinary tract infections (treated with either cephalosporins or fluoroquinolones), 1 patient contracted hepatitis E virus (which was cured by ribavirin), another a CMV viremia, which resolved with valganciclovir. One patient had a nephrectomy for kidney cancer on a native kidney. One patient completed pregnancy without complication.

Most drugs usually are considered safe in patients with AIP (http://www.drugsporphyria.org/; https://porphyriafoundation.org/) with the exception of trimethoprim/sulfamethoxazole, which is porphyrinogenic, and therefore not recommended (Table 3). Atovaquone or aerosolized pentamidine were was prescribed in all patients except 2, who received trimethoprim/sulfamethoxazole to treat a pneumonia caused by Pneumocystis jiroveci and for the primary prevention of pneumocystosis, without noticeable side effect.

AIP clinical activity after KTx

Skin lesions suggestive of PCT that had occurred in 6 AIP dialyzed patients (out of 8)

disappeared after KTx (Table 1), with melanodermia as a sequel. Since the accumulation of porphyrins may cause skin lesion in ESRD patients with AIP, we explored the plasma porphyrin profile of these patients.

First, plasma total porphyrins could be measured before and after transplantation in the patient who received a kidney from controlled DCD. Primary non-function occurred after the KTx with the DCD kidney, and the patient received a second kidney transplant 18 months after the first KTx. Before the first KTx, the patient was on dialysis, and total plasma porphyrins levels remained steadily 20 times above the normal values (Figure 2A). After the first KTx, and despite the absence of renal function, total plasma porphyrins levels decreased and stabilized at a level less than 10 times the normal value. After the second KTx, renal function normalized, and plasma porphyrins reached near normal values (Figure 2A).

Before KTx, the porphyrins profile of this patient was mostly made of uroporphyrin (> 90%), whereas in PCT, both uroporphyrin and heptacarboxylporphyrin predominate due to a reduction of uroporphyrin decarboxylase activity (Table 4 and Figure 1). In AIP patients with ESRD, uroporphyrin accumulation (mostly the isomer I, Table S2) derives from the non-enzymatic polymerization and cyclisation of four molecules of PBG (Figure 1).

Uroporphyrins promote lesions in regions of the skin exposed to the sun.

Consistent with the observation we made of the DCD KTR that plasma porphyrins concentrations decrease after KTx, the plasma porphyrin levels dramatically dropped after transplantation in 6 other KTRs in whom it was possible to perform the measurement (Figure 2B).

Acute attacks also almost disappeared after KTx. The occurrence of reported acute porphyria attacks before KTx ranged between 0.21/year and 5/year (median 0.29/year). After KTx, only 4 patients experienced 1 acute attack, and the rest of the cohort, none (Table 2 and Figure 3). 2B).

We also measured urinary ALA and PBG in a systematic way before and after transplantation in the patient who received a kidney from controlled DCD. However, the ALA and PBG urinary excretion profile is more complicated to interpret because the DCD KTx was oliguric for years before the second KTx, and measurement of urine ALA and PBG may be unreliable, and the urinary concentrations of ALA and PBG may not be a good indicator of disease activity. That being said, the excretion of urinary PBG and ALA remained above normal values before and after the successful KTx: mean urinary PBG 26.2±5.9 µmol/mmol creatinine (N<1 µmol/mmol creatinine) before the second KTx, and 14.3±3.9 µmol/mmol creatinine after the second KTx; mean urinary ALA 5.2±1.9 µmol/mmol creatinine (N<3 µmol/mmol creatinine) before the second KTx, and 7.1±4.8 µmol/mmol creatinine after the second KTx, indicating that precursors are still produced by the liver after KTx, and that transplantation of a heme-producing organ does not compensate the enzymatic deficit activity of the liver. However, when renal function returned to normal after the second KTx, the PBG/ALA ratio has significantly decreased (Figure 4): the mean PBG/ALA ratio (normal value<0.3) was 5.1±2 before the second KTx and 1.6±0.2 after the second KTx. These results are consistent with previous findings [START_REF] Sardh | Porphyrin precursors and porphyrins in three patients with acute intermittent porphyria and end-stage renal disease under different therapy regimes[END_REF], and our own experience (Figure S1),

indicating that an increase of the urinary PBG/ALA ratio is associated with the decline of renal function in AIP patients.

Discussion

We provide evidence that KTx is the treatment of choice of AIP patients with ESRD. Even if the number of KTx patients in this cohort is small (AIP is a rare disease, and AIP patients rarely reach ESRD), we report the largest experience to date, and our data indicate that the post-transplantation history is simple without noticeable complication.

A major finding of our study is that KTx considerably decreases AIP activity. The most likely mechanistic hypothesis is that porphyrin precursors (ALA and PBG), which are responsible for the symptoms of acute porphyria, accumulate in parallel to the progression of renal failure. In turn, renal function improvement after KTx fosters precursors clearance and alleviates AIP symptoms. Supporting this notion, plasma measurements of ALA and PBG before dialysis sessions usually show a dramatic increase of precursors concentrations, especially PBG [START_REF] Sardh | Porphyrin precursors and porphyrins in three patients with acute intermittent porphyria and end-stage renal disease under different therapy regimes[END_REF], and PBG and ALA are readily filtrated by dialysis membranes and concentrations decrease in plasma after dialysis session to about 70% of the value before dialysis; dialysis has even been proposed for the treatment of attacks when heme-arginate is not available [START_REF] Sardh | Porphyrin precursors and porphyrins in three patients with acute intermittent porphyria and end-stage renal disease under different therapy regimes[END_REF][START_REF] Miyagi | The serum porphobilinogen and hepatic porphobilinogen deaminase in normal and porphyric individuals[END_REF][START_REF] Prabahar | Hemodialysis: a therapeutic option for severe attacks of acute intermittent porphyria in developing countries[END_REF]. Thus, the increased clearance of precursors after KTx is certainly responsible for the improvement of the symptoms of AIP. In addition, ALA, which is freely filtered by the kidneys, is reabsorbed in the proximal tubule by the human peptide transporter SLC15A2 [START_REF] Tchernitchko | A Variant of Peptide Transporter 2 Predicts the Severity of Porphyria-Associated Kidney Disease[END_REF]. If ALA is reabsorbed by a kidney allograft expressing functional HMBS, it will be metabolized into PBG, which, in turn, will be transformed into porphyrins.

As a result, porphyrin precursors could accumulate in the organism in smaller quantities.

We also provide evidence that blistering lesions on sun-exposed skin that often occur in AIP dialyzed patients disappear after KTx. By definition, skin lesions and photosensitivity are not characteristic of AIP. Consistent with what was suggested for a long time [START_REF] Sardh | Porphyrin precursors and porphyrins in three patients with acute intermittent porphyria and end-stage renal disease under different therapy regimes[END_REF], our results

indicate that the very high concentration of porphyrins found in plasma of AIP patients undergoing dialysis is made of uroporphyrin I that derives from the non-enzymatic polymerization of the linear tetrapyrrole hydroxymethylbilane, and which spontaneously forms the cyclic tetrapyrrole uroporphyrin I (a compound unsuitable as an intermediate in the synthesis of protoporphyrin IX) (Figure 1) [START_REF] Bogorad | Conference on Hemoglobin: 2-3[END_REF]. Uroporphyrin causes skin lesions in sun-exposed areas, resembling to those seen in PCT [START_REF] Sardh | Porphyrin precursors and porphyrins in three patients with acute intermittent porphyria and end-stage renal disease under different therapy regimes[END_REF][START_REF] Wahlin | Combined liver and kidney transplantation in acute intermittent porphyria[END_REF][START_REF] Smith | The prophyrins of normal human urine, with a comparison of the excretion pattern in porphyria cutanea tarda[END_REF]. However, the plasma concentrations of porphyrins are not affected by dialysis therapy because these molecules are bound to plasma albumin. As a consequence, 75% of the dialyzed patients in our cohort suffered from photosensitivity and skin fragility resulting in vesicles and bullae of sunexposed skin. Note that a similar disorder associated with bullous lesions in dialyzed patients called "pseudoporphyria" occurs in patients with normal porphyrins levels but clinical and histological features similar to those of PCT [23]. The disease appears to develop due to the association of photosensitizing medications and perhaps with an excessive ultraviolet exposure and the presence of uremic toxins [23,[START_REF] Brivet | Porphyria cutanea tardalike syndrome in hemodialyzed patients[END_REF].

Our data also confirm previous observations made on the evolution of the PBG/ALA ratio during the progression of kidney disease: the PBG/ALA increases in parallel to the impairment of renal function, and a normalization the glomerular filtration rate is accompanied by a reduction of the ratio around values observed in AIP patients with normal renal function, that is approximately 2 [START_REF] Floderus | Variations in porphobilinogen and 5-aminolevulinic acid concentrations in plasma and urine from asymptomatic carriers of the acute intermittent porphyria gene with increased porphyrin precursor excretion[END_REF]. The mechanisms supporting a selective accumulation of PBG over ALA during renal failure are not fully understood [START_REF] Sardh | Porphyrin precursors and porphyrins in three patients with acute intermittent porphyria and end-stage renal disease under different therapy regimes[END_REF][START_REF] Miyagi | The serum porphobilinogen and hepatic porphobilinogen deaminase in normal and porphyric individuals[END_REF][START_REF] Unzu | Renal failure affects the enzymatic activities of the three first steps in hepatic heme biosynthesis in the acute intermittent porphyria mouse[END_REF]. One can speculate that a reduction in porphyrin precursors clearance upon renal failure will primarily direct PBG since ALA will continue to be metabolized into PBG by ALA dehydratase, unlike PBG that will accumulate due to the HMBS enzymatic defect (Figure 1).

The drug management after transplantation is unremarkable in AIP patients, and at the exception of trimethoprim-sulfamethoxazole, which is not recommended (but has been prescribed in 2 patients without side effect), all the drugs usually prescribed after KTx are authorized in AIP patients. The porphyrinogenicity of a drug is related to its capacity for induction of the rate-limiting enzyme in heme biosynthesis, ALAS1, an depends on chemical properties of the drug, structure and metabolism pointing to affinity to P450 cytochromes, hepatic load in therapeutic use, its capacity for P450 cytochrome induction or irreversible inhibition, and capacity to activate or modulate the transduction mechanisms of nuclear receptors affecting ALAS1 gene transcription [START_REF] Thunell | Guide to drug porphyrogenicity prediction and drug prescription in the acute porphyrias[END_REF]. In addition, a porphyrinogenic drug may not systematically trigger acute porphyric attacks, because the porphyric susceptibility of the patient depends on the gene mutation predisposing for acute porphyria, age and sex, previous and current disease activity, and recent or current exposure to other ALAS1inducing agents [START_REF] Thunell | Guide to drug porphyrogenicity prediction and drug prescription in the acute porphyrias[END_REF]. It is thus proposed that in the absence of a safer alternative, an urgently needed drug should not be withheld on the grounds of its potential porphyrinogenicity.

Another important lesson of this study in that AIP should always be considered as one of the possible causes of chronic tubulointerstitial nephropathy, possibly associated with nonspecific arteriolar lesions, of unknown cause, even if the patient is dialyzed. The identification of the disease is critical because it would lead to the diagnosis of AIP in the propositus family, and effective therapies targeting ALAS1 in the most severe forms of AIP are currently developed [28].

A limitation of this study is the lack of systematic pathological examination of kidney allograft biopsy specimen that would have provided information on the possible recurrence of PAKD in the kidney allograft. However, in the context of care, kidney biopsies were not indicated in almost all of these patients and the risk of a systematic screen for histological lesions would be superior to the expected benefits in terms of treatment. The risk of recurrence if PAKD is theoretically not excluded, but PAKD is a slowly progressive disease affecting AIP patients with a severe disease. Since AIP activity seems to turn off after KTx, and some KTx have a normally functioning kidney allograft for more than 20 years, the risk of recurrence is likely to be low.

In conclusion, KTx is the treatment of choice of AIP patients with ESRD because the posttransplantation history is simple, and the drug management is easy. Critically, KTx reduces the disease activity in AIP patients and improve their quality of life. Nicolas Pallet: participated in study design, clinical data interpretation, manuscript writing and final approval.

Acknowledgements

Funding: none

Disclosures: all authors have approved the final article.

  AIP was diagnosed between 1976 and 2015 (median age at diagnosis, 35 years, range 21-53 years) and on average 19 years (range 0-31 years) before the diagnosis of ESRD except for one patient in whom the diagnosis of AIP was made 5 years after the initiation of dialysis. This patient reached ESRD in 2011 (the kidney disease was of undetermined cause at that time), and developed progressive peripheral neuropathy 2 years later, resulting in diagnostic uncertainty until AIP was proven with the identification of mutation in the HMBS gene in 2016.

Figure 1 .

 1 Figure 1. Porphyrin metabolism and heme synthesisIn red, hydroxymethylbilane synthase, which is defective in Acute Intermittent Porphyria. In green, uroporphyrinogen III decarboxylase, which is defective in Porphyria Cutanea Tarda. Uroporphyrin I derives from the non-enzymatic polymerization of the linear tetrapyrrole hydroxymethylbilane, and spontaneously forms the cyclic tetrapyrrole uroporphyrin I (a compound unsuitable as an intermediate in the synthesis of protoporphyrin IX). As a negative feedback, heme inhibits ALA synthase 1 activity.

Figure 2 .

 2 Figure 2. Plasma porphyrin profiles in KTRA. Plasma concentration of total porphyrins in the DCD KTR before a first KTx, after KTx with a non-functioning kidney allograft, and after a second KTx with a normal renal function. Dash line represents normal value of total porphyrins in plasma (<20 nmol/l). B. Plasma concentration of total porphyrins in 6 other KTR before and after KTx.

Figure 3 .

 3 Figure3. Occurrence of AIP attacks before and after KTx. The number of acute porphyria attacks before and after KTx has been collected in the health records over an isometric period of time centered on the date of KTx, and which depends on the length of follow-up after KTx, and as such, vary according to patients. Note that Patient 11 is the asymptomatic carrier.

Figure 4 .

 4 Figure 4. Porphyrin precursors profile in a KTRUrinary PBG/ALA ratio in the DCD KTR before a first KTx, after KTx with a non-functioning kidney allograft, and after a second KTx with a normal renal function. Dash line represents normal value of PBG/ALA (=2) in urine in AIP patients with normal renal function.

  The impact of KTx on porphyrin precursors accumulation (which cause AIP symptoms) is exemplified in the DCD KTR by the dramatic reduction in plasma ALA and PBG concentrations after KTx: ALA 5.4 µmol/L before, 1.6 µmol/L after; PBG: 39.3 µmol/L

before, 3.4 µmol/L after, PBG/ALA ratio: 7.2 before and 2.1 after. Since the accumulation of uroporphyrins in AIP patients with ESRD depends on the production and ALA and PBG, a drop in uroporphyrins concentrations after KTx is also consistent with the reduction of AIP activity (Figure

: none Conflict of interest:

  This research was unfunded and the authors have no conflicts of interest to declare.Hélène Lazareth: involved in all aspects of study designs, data collection, data analysis, manuscript writing and final approval; Neila Talbi: involved in all aspects of study designs, data collection, data analysis, manuscript writing and final approval; Nassim Kamar: participated in clinical data collection & data analysis and manuscript writing/final approval; Charlène Levi: participated in clinical data collection & data analysis and manuscript writing/final approval; Bruno Moulin: participated clinical data collection, manuscript review/revising and final approval; Sophie Caillard: participated in clinical data collection,

	Authors' contributions:

manuscript review/revising and final approval; Luc Frimat: participated in clinical data collection, manuscript review/revising and final approval; Jonathan Chemouny: participated in clinical data collection, manuscript review/revising and final approval; Valérie Chatelet: participated in clinical data collection, manuscript review/revising and final approval; Clément Vachey: participated in clinical data collection, manuscript review/revising and final approval; Renaud Snanoudj: participated in clinical data collection, manuscript review/revising and final approval; Thibaud Lefbvre: Valérie Chatelet: participated in clinical data collection, manuscript review/revising and final approval; Alexandre Karras: Valérie Chatelet: participated in clinical data collection, manuscript review/revising and final approval;Laurent Gouya: Valérie Chatelet: participated in clinical data collection, manuscript review/revising and final approval;; Caroline Schmitt: Valérie Chatelet: participated in clinical data collection, manuscript review/revising and final approval; Hervé Puy: participated in study design, clinical data interpretation, manuscript writing and final approval.


Raphaël Coscas
email: rcoscas@gmail.com

Marc Coggiaa

COUP D'OEIL Acute Aortic Thrombosis Revealing a COVID-19 Infection

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.