Aldo Maalouf
email: aldo.maalouf@geenov.com

Mohamed-Chaker Larabi
email: larabi@sic.univ-poitiers.fr

CYCLOP: A Stereo Color Image Quality Assessment Metric

Keywords: Stereo quality assessment, cyclopean image, 2D multi-valued wavelet, disparity field

In this work, a reduced reference (RR) perceptual quality metric for color stereoscopic images is presented. Given a reference stereo pair of images and their "distorted" version, we first compute the disparity map of both the reference and the distorted stereoscopic images. To this end, we define a method for color image disparity estimation based on the structure tensors properties and eigenvalues/eigenvectors analysis. Then, we compute the cyclopean images of both the reference and the distorted pairs. Thereafter, we apply a multispectral wavelet decomposition to the two cyclopean color images in order to describe the different channels in the human visual system (HVS). Then, contrast sensitivity function (CSF) filtering is performed to obtain the same visual sensitivity information within the original and the distorted cyclopean images. Thereafter, based on the properties of the human visual system (HVS), rational sensitivity thresholding is performed to obtain the sensitivity coefficients of the cyclopean images. Finally, RR stereo color image quality assessment (SCIQA) is performed by comparing the sensitivity coefficients of the cyclopean images and studying the coherence between the disparity maps of the reference and the distorted pairs. Experiments performed on color stereoscopic images indicate that the objective scores obtained by the proposed metric agree well with the subjective assessment scores.

I. INTRODUCTION

With the increasing advances in 3D TV and 3D cinema and the increasing interests in enhancing the viewing displays by incorporating multiview/stereo imaging, more and more digital stereo images are being compressed, transmitted, and used in computer systems. Therefore, stereo color image quality assessment (SCIQA) methods are needed to evaluate the quality of stereo images. For SCIQA, objective or subjective methods are used. The subjective measurement Mean Opinion Score (MOS) is a widely used method for the assessment of image or video quality, but it has several obvious disadvantages. It is very tedious, expensive and impossible to be executed automatically. Instead, an objective image or video quality metric can provide a quality value for a given image or video automatically in a relatively short time. This is very important factor for real time applications. Generally, quality metrics can be classified into three categories, including full-reference (FR), reduced reference (RR) and no-reference (NR) according to the availability of the original signal [START_REF]Vqeg: The video quality experts group[END_REF] [START_REF] Wang | Objective video quality assessment[END_REF]. On the other hand, FR metrics [START_REF] Sheikh | An information fidelity criterion for image quality assessment using natural scenes tatistics[END_REF] need full information of the original images and demand ideal images as references which can be hardly achieved in practice for some applications (such as broadcasting...). The traditional methods of FR (such as peak signal-to-noise-ratio PSNR) are based on pixel-wise error and have not always been in agreement with perceived quality measurement. On the other hand, NR metrics aim to evaluate distorted images without any cue from their original ones. However, most of the proposed NR quality metrics are designed for one or a set of predefined specific distortion types and are unlikely to be generalized for evaluating images degraded with other types of distortions. While RR metrics are between FR and NR, they make use of a part of the information from the original images in order to evaluate the visual perception quality of the distorted ones. As the transformed and stored data are reduced, RR metrics have great potential in image quality assessment (IQA). Several IQA metrics have been proposed in the literature integrating the properties of the HVS. However, few efforts have been devoted to the quality assessment of stereoscopic images and no efforts, to our knowledge, to the SCIQA. In [START_REF]Subjective assessment of stereoscopic television pictures[END_REF], recommendations on subjective quality assessment of stereoscopic television pictures are given. In [START_REF] Marc | Visual discomfort in stereoscopic displays: a review[END_REF], the factors affecting the quality of stereo images are described. These factors include the perception depth, stereoscopic impairments and the visual discomfort. In [START_REF] Campisi | Stereoscopic images quality assessment[END_REF], the authors performed a series of subjective tests to find whether 2D evaluation metrics can be applied for the evaluation of stereoscopic images. They have also proposed a methodology for subjective evaluation of stereoscopic IQA and they concluded after a series of experiments that 2D IQA methods are not suitable for stereoscopic IQA (SIQA). In [START_REF] Benoit | Using disparity for quality assessment of stereoscopic images[END_REF] and [START_REF] Benoit | Quality assessment of stereoscopic images[END_REF], Benoit et al. proposed to combine depth information and 2D quality metrics to develop an objective SIQA metric. For that, they compared the disparity maps, to measure the disparity distortion, and the corresponding stereo images to obtain an average measure of the image distortion. Finally, they combined both measures into one objective SIQA. In [START_REF] Paul | Stereoscopic image quality metrics and compression[END_REF], Gorley et al. proposed a SIQA metric based on the point matches regions of high spatial frequency between the left and right views of the stereo pair. Their metric accounts for HVS sensitivity to contrast and luminance changes in regions of high spatial frequency. Recently, in [START_REF] Sazzad | Stereoscopic image quality prediction, First International Workshop on Quality of Multimedia Experience QOMEX[END_REF], Sazzad et al. proposed an objective SIQA based on the fact that perceived distortion and depth of any stereoscopic images are strongly dependent of local features such as textures and edges. Their method is intended for the evaluation of JPEG compressed images and takes into account the blockiness of the images. All the above mentioned approaches do not take into consideration the properties of color images (i.e. multispectral images, multispectral disparity...). To cope this problem, we propose a method for the estimation of the disparity of color images using structure tensors. Then, the cyclopean images of the reference and distorted pairs are obtained and the multispectral wavelet transform proposed in [START_REF] Scheunders | An orthogonal wavelet representation of multivalued images[END_REF] is used to describe the different channels in the HVS. In order to achieve a uniform visual sensitivity to different scales and bands for human perception, each wavelet band is masked with the CSF. For that, we have used the CSF model proposed by Daly [START_REF] Daly | The visible difference predictor: an algorithm for the assessment of image fidelity[END_REF]. Thereafter, contrast sensitivity and rational thresholding are performed to obtain the sensitivity coefficients. Finally, error pooling is performed by studying the coherence between the disparity maps and comparing the sensitivity coefficients of the cyclopean images. The rest of the paper is organized as follows: in section 2, the procedure for obtaining color disparity maps is described. In section 3, the procedure for computing the cyclopean images and obtaining the sensitivity coefficients is described. In section 4 we describe our SCIQA metric. Section 5 is devoted to the experimental evaluation and section 6 concludes.

II. COLOR DISPARITY MAP

Color image geometry can be described by using the structure tensor proposed by Di Zenzo in [START_REF] Dizenzo | A note on the gradient of multi images[END_REF]. In fact, Di Zenzo pointed out that the correct method to combine the first order derivative structure is by using a local tensor. Analysis of the shape of the tensor leads to an orientation and a gradient norm estimate. For a multichannel image I = I 1 , I 2 , ....., I m T (m = 3 for color images) the structure tensor is given by

M = I T x I x I T x I y I T y I x I T y I y (1)
The multi-channel structure tensor describes the 2D first order differential structure at a certain point in the image.

In order to describe the disparity between two color images, we first extend the tensor defined in (1) to account for the difference between the two stereo images I L (the left image) and I R (the right image). The extended tensor is defined by:

S =       
I y I z m i=1 I 2 z        (2) (a) (b) (c) (d)
θ + =   ∂I ∂x ∂I ∂y ∂I ∂z   , θ -=   ∂I ∂y -∂I ∂x 0   , θ z =    ∂I ∂x ∂I ∂z ∂I ∂y ∂I ∂z -∂I 2 ∂ 2 x + ∂I 2 ∂ 2 y   
(3) θ + is the gradient vector, and the subspace generated by θ - and θ z is orthogonal to the gradient ∇I. In fact, θ -and θ z are chosen so that (θ + , θ -, θ z ) form an orthogonal basis. The disparity tensor can therefore be defined by:

F = θ + θ + T + θ -θ -T + θ z θ z T (4)
Clearly, F can be approximated using the largest eigenvector, therefore we let:

F = θ z θ z T (5)
The disparity map is finally defined by the largest eigenvector ν of F which is computed at each pixel. Figure 1 shows an example of a residual image and the disparity map computed using the disparity tensor F . Once we have obtained the disparity map, we proceed to compute the cyclopean image. This step is described in the next section.

III. CYCLOPEAN IMAGE AND PERCEPTUAL PROCESSING

The ability of the brain to perceive a single image by combining the images from left and right eye, so-called cyclopean image, is a result of combination of locally matched stereo regions in a single global image. To obtain the cyclopean image, we proceed as in [START_REF] Henkel | Fast stereovision by coherence detection[END_REF]. That is, for each pixel, we take a local window from the left view and averaging it with the disparity-matched window from the right view. Figure 2(a) shows an example of a cyclopean image for the teddy pair shown in Fig. 1. Once we obtained the cyclopean images for the reference stereo pair and the distorted one, we proceed to perceptually compare them in order to assess the quality.

To do that, we have first to perform a wavelet decomposition on the two cyclopean images to describe the different channels in the HVS. For that, we have used the color wavelet transform proposed in [START_REF] Scheunders | An orthogonal wavelet representation of multivalued images[END_REF] . This wavelet transform makes use of structure tensors to accumulate color edge information. Therefore, all detail information of the different bands is accumulated into one single representation. The representation allows for a multiscale edge description of color images. Figure 2(b) shows the color wavelet transform of the cyclopean image shown in figure 2(a). After computing the color wavelet transform, we have to find the sensitivity coefficients of the two cyclopean images, i.e., the most attractive wavelet coefficients as seen by an observer, so that we can measure the perceptual similarity by using few coefficients. We begin by performing contrast sensitivity function (CSF) filtering. The CSF describes the variations in visual sensitivity as a function of spatial frequency and orientation. In fact, it has been reported that human visual system has different sensitivity to signals with different frequencies. So, to make full use of the wavelet coefficients and orientations in different wavelet scales, we used the CSF to render the sensitivity of human visual system the same to different frequencies. Particularly, we used the model proposed by Daly in [START_REF] Daly | The visible difference predictor: an algorithm for the assessment of image fidelity[END_REF]. The CSF is applied for all wavelet bands at a given scale j as follows:

Ŵj = W j • N CSF ( 6
)
where W is the wavelet coefficient and N CSF is the value of the 2D CSF defined by Daly [START_REF] Daly | The visible difference predictor: an algorithm for the assessment of image fidelity[END_REF].

After the normalization of the frequencies at different bands and scales, we apply sensitivity thresholding to find the sensitivity coefficients. For that purpose we used the thresholds proposed by Daly [START_REF] Daly | The visible difference predictor: an algorithm for the assessment of image fidelity[END_REF]. In this model, the visibility threshold elevation T j (x, y) at site (x, y) and scale j is given by:

T j (x, y) = 1 + k 1 • k 2 • Ŵj (x, y) f b 1 b (7)
where k 1 and k 2 determine the pivot point of the curve, and the parameter b determines how closely the curve follows the asymptote in the transition region. In the initial work of Daly, a value for the learning slope is chosen depending on the cortex subband. Ideally, this value should depend on the uncertainty of the signal masking. f is the slope of the curve of the masking contrast.

The value of f is between 0.65 and 1.0, k 1 = 6 -7/3 , k 2 = 6 10/3 and b = 4 [START_REF] Daly | The visible difference predictor: an algorithm for the assessment of image fidelity[END_REF].

After performing the thresholding, SCIQA is performed by comparing the sensitivity coefficients of both cyclopean images and by studying the coherence of the disparity maps of the reference and distorted pair of images. This step is described in the following section.

IV. CYCLOP: THE PROPOSED METRIC

Our SCIQA metric is based on the fact that any distortion in the received image pair leads to a distortion in disparity map and a perceptual difference between the sensitivity coefficient of the reference cyclopean image and the distorted one. We first define a coherence metric to measure the discrepancy in the disparity maps of the reference and the distorted pair of images, then, we define a perceptual measure to quantify the difference between the two corresponding cyclopean images. Our SCIQA metric will therefore be the sum of two measures. The coherence measure is defined as follows: Let ξ r and ξ t be the largest eigenvalues of equation ( 5) for the reference and the transmitted image pairs respectively. Any change in the image content between the reference and the transmitted pairs will lead to a change in the disparity direction characterized by the largest eigenvectors ν and consequently, to a change in the values of the eigenvalues. Based on that, we define the coherence metric that measures whether the disparity of the reference pair is coherent (or the same) with the transmitted. The coherence measure is defined by: coh = 1 -

H,P ξr-ξt ξr+ξt 2 H × P ( 8
)
where H and P are the height and the width of the image respectively. It is to be noted that, if there is no difference between the two disparities (i.e. the same eigenvalues and eigenvectors) the coherence measure ( 8) is maximum and consequently, coh tends to 0. For the perceptual difference measure between the reference and the distorted cyclopean images, we have adopted the normalized absolute difference

d perceptual = 1 R R i=1 Ŵj,r -Ŵj,t 2 (9)
where R is the total number of the remaining coefficients after applying the sensitivity thresholding, Ŵj,r and Ŵj,t are the sensitivity coefficients of the reference cyclopean image and the distorted one. Cyclop the SCIQA metric is finally defined by:

D = coh + d perceptual 2 (10)

V. EXPERIMENTAL RESULTS

We evaluate our metric named Cyclop on the stereo image database used in [START_REF] Sazzad | Stereoscopic image quality prediction, First International Workshop on Quality of Multimedia Experience QOMEX[END_REF]. The evaluation is performed by comparing the objective quality scores obtained by our SCIQA metric with the subjective Mean Opinion Scores (MOS) provided by We evaluate our SCIQA metric by computing the three measurements recommended by VQEG [START_REF]Vqeg: The video quality experts group[END_REF] to test the consistency of our Cyclop and the subjective perception, namely, the correlation coefficients (CC), the rank order correlation coefficient (ROCC) and the outlier ratio (OR). We also compared our results with others obtained by the NR metric proposed in [START_REF] Sazzad | Stereoscopic image quality prediction, First International Workshop on Quality of Multimedia Experience QOMEX[END_REF]. The results are shown in Table 1 for both metrics. From these results, we can see that our metric has a high prediction accuracy (larger CC), stricter monotonicity (larger ROCC) and better stability (smaller OR).

VI. CONCLUSION

In this paper, we presented a new stereoscopic color image quality assessment metric. The proposed SCIQA metric makes use of a color disparity tensor to estimate the disparity map of color stereo images. The properties of the HVS are integrated in the proposed metric by performing CSF filtering and rational thresholding. The SCIQA is performed by computing the coherence of the disparities and the perceptual difference between the reference image pair and the distorted one. The experimental results showed that the proposed SCIQA metric performs well and has good consistency with subjective quality assessment. For future work, we are extending Cyclop to measure the quality of stereoscopic video sequences.

Fig. 1 .

 1 Fig. 1. (a) and (b) "teddy" stereo pair, (c) the residual image and (d) the disparity map

Fig. 2 .

 2 Fig. 2. (a) Cyclopean image for the "teddy" pair and (b) its color wavelet transform

Fig. 3 .

 3 Fig. 3. Stereoscopic Images

This work was supported by the French National Research Agency (ANR) under the QuIAVU project


S A Amirshahi
email: seyedali.amirshahi@unijena.dechaker.larabi@sic.univpoitiers.fr

M C Larabi

Spatialtemporal video quality metric based on an estimation of QoE

In this work a new Reduced Reference (RR) Video Quality Metric (VQM) is proposed. The method takes advantage of the Human Visual System (HVS) sensitivity to sharp changes in the video. The proposed method has a spatialtemporal approach and because of that it is named as STAQ (Spatial Temporal Assessment of Quality). In the first step of STAQ we take a temporal approach and find the matching regions in consecutive frames. In the next step, a spatial approach is taken in the way of calculating the quality of the matching regions in the temporal approach. In the last step, the quality of the video is calculated based on the parameters gathered in the spatial and temporal domain and using the motion activity density of the video as a controlling factor. An important improvement lies in taking into account the Quality of Experience (QoE) represented as the motion activity density of the reference video. The results show a great improvement in the case of H.264 and MPEG2 compressed and IP distorted videos even when compared to state of the art Full Reference (FR) metrics.

INTRODUCTION

With the huge amount of video in our daily life, there has been a significant increase in researches on Video Quality Assessment (VQA) [START_REF] Wang | The handbook of video databases: design and application[END_REF]. Running subjective tests for video quality evaluation is the main way of evaluating Video Quality (VQ) but it is rather an expensive, time consuming and tedious procedure. All this makes applying objective Video Quality Metrics (VQM) the best option for evaluating VQ for real time applications. VQM like Image Quality Metrics (IQM) have three main categories. Full Reference (FR) metrics were we have access to the reference video. Reduce Reference (RR) metrics which some information regarding the features of the original video is available. No Reference (NR) metrics that there is no information regarding the reference video. Most VQM that have been proposed so far are FR. These methods range from simple methods such as the MSE and PSNR methods to rather complicated metrics such as metrics proposed in [210]. VQM could also be categorized based on the approach they take to evaluate the VQ. For instance, most of them have a spatial approach and often evaluate the VQ by applying IQM on different frames and then by pooling the results achieved from each frame. Few of the metrics have a temporal approach and others have a SpatialTemporal approach. In the later case most metrics combine results from a spatial approach and a temporal approach at the last step of the pooling procedure [START_REF] Seshadrinathan | Motion tuned spatiotemporal quality assessment of natural videos[END_REF]. In the proposed metric we would calculate the VQ based on a SpatialTemporal. The difference between the proposed metric and other Spatial Temporal metrics is the fact that instead of combining the results from each domain in the polling system our method uses the data from one domain in the other domain. This way the two domains have a close relationship with each other in every step of the work. There has been an agreement on the importance of the integration of Quality of Experience (QoE) in the quality evaluation community. This has been shown through numerous workshops, conferences and special sessions dealing with this issue. Metrics base on QoE try to bring the observer in the loop and model his/her emotions towards the content of the video. Video content plays an important role when trying to base a VQM on QoE. With respect to what was mentioned, one of the aspects which could be useful when trying to model the observer's opinion is the amount of motion in the video. For example, if we apply the same type and level of noise to two different videos with different motion density the observer will give two different quality ratings to the video. Take the example of a football game and a news anchor. In the first case, the observer has a higher sensitivity and the amount of noise in the video should be low. In the second case, the observer will give a higher quality rating to the video even if we apply a higher amount of noise to the video. This aspect has been dicussed and experimented in [START_REF] Larabi | Relation between bitrate, motion, and framerate for scoring of image sequences[END_REF] allowing to find a formulation of the quality function of the motion of a sequence. Keeping this in mind the amount of motion density in videos is what the proposed method is based on and tries to use to take QoE into account. With respect to the literature, in this paper a new reduced reference metric is proposed. The metric has a spatial temporal approach and so is named as STAQ (Spatial Temporal Assessment of Quality). The reminder of this paper is arranged as followed: we will first introduce STAQ in section 2, present the results in section 3 and finally in section 4 a conclusion of the work is given.

PROPOSED APPROACH

An overview of STAQ is shown in Figure 1. As it can be seen the proposed metric has three main blocks, temporal block, spatial block and the pooling block. STAQ is based on the fact that the Human Visual System (HVS) is sensitive to sharp changes in the video. This gives the advantage of finding common regions in consecutive frames. In STAQ, we will find these regions in the temporal approach by calculating the appropriate Motion Vectors (MV) for the sub-blocks in the current frame and use their coordinates in the spatial approach to estimate the quality of each frame of the video for different color channels. For the pooling system we use the quality values we have from the spatial block along with other factors that we have gathered in different steps prior to the pooling system. For the developed metric, QoE is introduced as a controlling factor based on the motion activity density of the reference video. A description of each section of the proposed metric is given in the following.

Temporal A pproach

As shown in Figure 1, in the first step of the temporal approach the MV of the test video is calculated resulting in finding matching subblocks in consecutive frames. The method used for calculating the MV is the Adaptive Rood Pattern Search (ARPS) [START_REF] Nie | Adaptive rood pattern search for fast blockmatching motion estimation[END_REF]. An important change made when calculating the MV compared to previous methods is the use of Complex Wavelet Structural Similarity index (CWSSIM) [START_REF] Sampat | complex wavelet structural similarity: a new image similarity index[END_REF] instead of using the Mean Absolute Difference (MAD). The reason behind this change is that by using the CWSSIM method the precision of the results increases. Although by using CWSSIM instead of MAD we will increase the complexity of the calculation but having results with a higher precision will increase our chances of finding the matching regions with better accuracy. Another important reason for using the CWSSIM method is the fact that we would use the calculated results in the later stages and so decrease the calculation time for the proposed VQM. Table 1. shows the MSE values between the original and the motion compensated frame using the CWSSIM and the MAD method. After calculating the MV for the test video we match the results with the MV of the reference video. Using the percentage of the matching MV which is named as the Number of Matching Motion Vectors (NMMV), the value of NMMV is sent to the pooling system. Table 2 shows an example of the MV for a frame in the test video and the corresponding frames in the reference video. The last block of the temporal approach is dedicated to check if the subblock in the test video matches the assigned sub-block in the reference video. Figure 2 gives an example of the calculated MV for a frame; arrows in the figure correspond to the MV for each sub-block in the frame. As it can be seen in Figure 2, MV in a frame tend to have approximately the same size and direction. To separate the correct MV with the incorrect ones we will take the following procedure. Assume that 𝑀𝑉 = {𝑀𝑉 & |𝑖 = 1, ⋯ , 𝑁} is the MV in a specific frame, the total number o MV's been N. Eq (1) calculates the mean value of the MV. If we assign a scalar value as the threshold to the mean value calculated in Eq (1) named T, any MV which does not belong to the interval given by Eq (2), is not a correct MV and so the matching sub-block is not used in the spatial approach. Although in the procedure we do not take the angle of the MV into account but experiments have shown that we will still reach promising results. In the case of the MV of Figure 2 we will not accept the dashed MV as the correct ones. In other words the sub-blocks which are shaded grey and correspond to these MV are not used in the spatial domain since they do not match the corresponding sub-blocks in the next frame. the neighboring subblocks. In [START_REF] Nie | Adaptive rood pattern search for fast blockmatching motion estimation[END_REF] we will have a predicted MV for each subblock with the same size and direction of the subblock on the left of the current subblock. In the first step, the predicted subblock is checked along with the rood pattern distributed points. Then, the best matching subblock is selected and a Small Diamond Search Pattern (SDSP) is used until the MV of the subblock is found. As mentioned before STAQ uses the strategy introduced by the ARPS method for finding the MV with a small change. That is, instead of using the MAD method in STAQ uses the CW SSIM method.

𝑚 = ∑ 𝑀𝑉 & 0 &12 𝑁 ( 1
)
|𝑀𝑉 & | < 𝑇𝑚 (2)
Figure 2 example of calculated MV for a frame. Arrows show the MV.

Complex Wavelet Structural Similarity index (CW-SSIM) [12]

CWSSIM IQM is based on the SSIM [START_REF] Wang | A universal image quality index[END_REF] IQM. As it is mentioned by the authors in [START_REF] Sampat | complex wavelet structural similarity: a new image similarity index[END_REF] "the CWSSIM index is an extension of the SSIM method to the complex wavelet domain". This will make the CWSSIM method not sensitive to nonstructured geometrical image distortions which we will face in the video environment such as rotation, scaling and etc. CWSSIM is calculated using Eq (3). In the equation 𝑐 > = ?𝑐 >,& |𝑖 = 1, ⋯ , 𝑁@ and 𝑐 A = ?𝑐 A,& |𝑖 = 1, ⋯ , 𝑁@ are two sets of coefficients extracted at the same spatial location in the same wavelet subbands of the two images being compressed, N representing the number of coefficients. is the complex conjugate of and is a small positive constant mainly used to improve the results when Signal to Noise Ration (SNR) is low. Like SSIM, the maximum value of CWSSIM occurs when the two images compared are identical. In this case the value of CWSSIM is 1.

𝑆C𝑐 > , 𝑐 A D = 2F∑ 𝑐 >,& 𝑐 A,& * 0 &12 F + 𝐾 ∑ F𝑐 >,& F J 0 &12 + ∑ F𝑐 A,& F J 0 &12 + 𝐾 (3)

Spatial A pproach

In the spatial approach the first block calculates the Visual Attention Map (VAM) for each frame using the Achanta et al [START_REF] Achanta | Frequencytuned Salient Region Detection[END_REF] method. The VAM will later be used to weigh each sub block in the luminance channel according to the importance it has in the frame.

In the second block, we calculate the quality of the channels in the spatial approach. The chrominance channels are evaluated using the MSE and the luminance channel by using the CWSSIM method. The reason behind the two approaches is the higher amount of information in the luminance channel compared to the chrominance channels.

For this reason we will need a metric with a higher accuracy for the luminance channel. The procedure is done by calculating the quality score, 𝑄𝑆𝐶(𝑖, 𝑗) of each subblock in the frames using Eq (4). 𝑄𝑆𝐶(𝑖, 𝑗) = 𝐼𝑄𝑀(𝑠𝑏𝐶(𝑖, 𝑗), 𝑠𝑏𝐶(𝑖 + 1, 𝑗)) (4) In Eq (4) 𝑠𝑏𝐶(𝑖, 𝑗) is the subblock j in frame i. depending on the channel we are assessing 𝐼𝑄𝑀 is either MSE or CW SSIM. Each quality score (𝑊𝑄𝑆𝐶(𝑖, 𝑗)) of each subblock is weighted using Eq [START_REF] Seshadrinathan | Motion tuned spatiotemporal quality assessment of natural videos[END_REF].

𝑊𝑄𝑆𝐶(𝑖, 𝑗) = 𝑄𝑆𝐶(𝑖, 𝑗) 𝑊(𝑖, 𝑗)

𝑊(𝑖, 𝑗) is the weigh associated to subblock j in frame i. As mentioned before we would weigh the luminance channel based on the VAM we have calculated. Due to the low amount of information in the chrominance channel we will give a weigh of 1 to all subblocks in the frame. In the next step the mean value (𝐹𝑆𝐶(𝑖)) of all subblocks (N) for each frame is calculated as shown in equation ( 6)

𝐹𝑆𝐶(𝑖) = ∑ 𝑊𝑄𝑆𝐶(𝑖, 𝑗) 0 U12 𝑁 (6)
Finally, we calculate the quality score of each channel named C S , over the M frames shown in Eq [START_REF] Yang | A novel objective NoReference metric for digital video quality assessment[END_REF].

𝐶𝑆 = ∑ 𝐹𝑆𝐶(𝑖) 0 &12 𝑀 (7)
For a video in the YUV colorspace we will have YS, US and VS representing the quality of the channels.

Visual Attention Map

At the speed the frames are being played the observer does not have enough time to pay attention to all the details in the video. This is why we would give a higher weight to regions which according to the VAM will take the focus of the observers. Achanta et al [START_REF] Achanta | Frequencytuned Salient Region Detection[END_REF] introduced a VAM for coloured images. The method could be summarized in Eq [START_REF] Opreal | Perceptual video quality assessment based on salient region detection[END_REF]. 𝑠(𝑥, 𝑦) is the saliency score for each pixel (𝑥, 𝑦), 𝐼 X is the mean image feature vector and 𝐼 YℎZ (𝑥, 𝑦) is the corresponding image pixel vector value in the Gaussian blurred version of the reference image, here we use a 5 × 5 separable binomial kernel. Since the saliency map is calculated in the Lab colorspace ‖ ‖ is the Euclidean distance. 𝑆(𝑥, 𝑦) = ]𝐼 X -𝐼 YℎZ (𝑥, 𝑦)] (8) The last step in the calculation of the VAM is the normalization procedure. In this step we will divide the pixel values in the VAM by the maximum value they have in the frame. We would then calculate the mean value for each sub block. This way the subblock with the highest importance will have the maximum value. As the importance decrease the value for each pixel will decrease as well. Figure 3 shows an example of a frame in the video along with a normalized VAM for the frame which we will use in the weighting procedure in Eq (5).

Motion A ctivity Density G roup

As mentioned before, an important improvement in our metric compared to the previous VQM lies in the use of QoE.

In [START_REF] Larabi | Relation between bitrate, motion, and framerate for scoring of image sequences[END_REF], we demonstrated the relation existing between quality and the motion. So QoE is introduced as a function depending on the motion activity density group of the video i.e. Very Low, Low, Medium, High and Very High motion activity. 𝑄 = 𝑓(𝑚), 𝑓: 𝑀 → 𝑀𝑂𝑆 where M is the set of possible motion activity classes and MOS= [1 : 5]. A value of 1 means that the quality will be rated as poor by all users while a value of 5 indicates an excellent quality. Most motion activity density algorithms which are based on statistical values calculated from the MV. The motion activity group is determined based on the median, max2 and the variance value of the MV magnitude as introduced by Peker et al. [START_REF] Peker | Framework for measurement of intensity of motion activity of video segments[END_REF]. Max2 is the maximum MV magnitude after removing the top 10% values.

Pooling Procedure

As mentioned in the previous sections, for each video we have YS, US, VS and NMMV along with the motion activity group in our pooling system. The quality score of the reference video is also used in the pooling system. This constitutes a part of the reduced reference. For pooling the data, a weight is assigned to the values sent to the pooling system and the values are added up shown in Eq [START_REF] Maalouf | A NoReference color video quality metric based on a 3d multispectral wavelet transform[END_REF]. In the last step, the difference between the MeanS value calculated in Eq [START_REF] Maalouf | A NoReference color video quality metric based on a 3d multispectral wavelet transform[END_REF] for the test video is compared to the corresponding value for the reference video (referred to as the quality score of the reference video) as shown in Eq [START_REF] Wang | Video quality assessment using a statistical model of human visual speed perception[END_REF]. 𝑚𝑒𝑎𝑛𝑆 kl = 𝑀𝑒𝑎𝑛𝑆 mnonmnpZn -𝑀𝑒𝑎𝑛𝑆 qnkl [START_REF] Wang | Video quality assessment using a statistical model of human visual speed perception[END_REF] For pooling the data we use a neuralfuzzy system having 𝑀𝑒𝑎𝑛𝑆 kl as its input and objective score as output. The system has five different pooling functions each representing one of the motion activity groups described previously. For this we use Matlab's fuzzy toolbox and its ANFIS (Adaptive NeuroFuzzy Inference System) structure. For training the algorithm we will use a hybrid optimization method with three two sided Gaussian membership functions (MF's) as its input and a linear output.

RESULTS

The proposed method was tested on the H.264 and MPEG2 compressed and IP distorted videos from the "LIVE Video Quality Database" [1517]. The subblocks used were in the size of 8 × 8 pixels. Also based on different experiments we used a value of 𝑤 2 = 𝑤 J = 𝑤 i = 𝑤 j = 0.25 for the current database. Figure 4 shows the scatter plots between the objective and subjective results for the mentioned categories of videos. Table 3 shows the Spearman and table 4 presents the Pearson correlation between the objective and subjective results for ten state of the art FR metrics and also the STAQ metric which is a RR metric. As it can be seen in the case of H.264 compressed videos the results of STAQ are ranked first compared to the others. Especially in the case of the Spearman correlation, we see a large increase in the accuracy of the results. For MPEG2 compressed videos, STAQ is ranked second and third compared to the other metrics. Note that the results are so close and STAQ is a RR metric and not a FR metric which is an advantage for the proposed metric. In the case of IP distorted videos STAQ is ranked first in the case of Spearman correlation but in the case of Pearson correlation it is ranked sixth. This is mainly because of the small difference between the results.

CONCLUSION

In this paper a new Reduced Reference VQM named as STAQ (SpatialTemporal Assessment of Quality) which is based on QoE is proposed. One of the most important aspects of STAQ is the fact that we take Motion Activity Density into account and so try to present a metric with a QoE approach. STAQ has been compared to some of the state of the art FR metrics and it shows great improvement in the correlation between the subjective and objective scores. In the case of H.264 videos the results have a higher correlation compared to 10 different state of the art FR metrics. In the case of MPEG2 and IP distorted videos the results are highly competitive with FR metrics. Keeping in mind that STAQ is a RR metrics this shows the advantage STAQ has compared to other metrics. [START_REF] Wang | Video quality assessment based on structural distortion measurement[END_REF] 0.6514 0.5545 0.4550 MSSSIM [START_REF] Wang | Multiscale structural similarity for image quality assessment[END_REF] 0.7051 0.6617 0.6534 SpeedSSIM [START_REF] Wang | Video quality assessment using a statistical model of human visual speed perception[END_REF] 0.7086 0.6185 0.4727 VSNR [START_REF] Chandler | VSNR: A wavelet based visual signaltonoise ratio for natural images[END_REF] 0.6460 0.5915 0.6894 VQM [START_REF] Pinson | A new standardized method for objectively measuring video quality[END_REF] 0.6520 0.7810 0.6383 VVIF [START_REF] Seshadrinathan | Motionbased perceptual quality assessment of video[END_REF] 0 [START_REF] Wang | Video quality assessment based on structural distortion measurement[END_REF] 0.6656 0.5491 0.5119 MSSSIM [START_REF] Wang | Multiscale structural similarity for image quality assessment[END_REF] 0.6919 0.6604 0.7219 SpeedSSIM [START_REF] Wang | Video quality assessment using a statistical model of human visual speed perception[END_REF] 0.7206 0.6270 0.5587 VSNR [START_REF] Chandler | VSNR: A wavelet based visual signaltonoise ratio for natural images[END_REF] 0.6216 0.5980 0.7341 VQM [START_REF] Pinson | A new standardized method for objectively measuring video quality[END_REF] 0.6459 0.7860 0.6480 VVIF [START_REF] Seshadrinathan | Motionbased perceptual quality assessment of video[END_REF] 0.6911 0.6145 0.5102 Spatial MOVIE [START_REF] Seshadrinathan | Motion tuned spatiotemporal quality assessment of natural videos[END_REF] 0.7252 0.6587 0.7378 Temporal MOVIE [START_REF] Seshadrinathan | Motion tuned spatiotemporal quality assessment of natural videos[END_REF]

Figure 1 -

 1 Figure 1 -Overview of STAQ

Figure 3 -

 3 Figure 3 -(a) original frame. (b) VAM normalized and divided into blocks.

Figure 4 -

 4 Figure 4 -Objective vs. subjective results for (a) H.264 and (b) MPEG2 compressed videos (c) IP distorted videos.

Table 1 -

 1 MSE values between the original and the motion compensated result using the CWSSIM and MAD method.

	Method used	MSE
	CWSSIM	29.3877
	MAD	37.22332

Table 2 -

 2 Example of calculating the value for NMMV.

	Frame	MV of a frame in	MV of a frame in	Matching
	number	the test video	the reference video	MV
	1	2,3	2,3	Yes
	2	1,2	0,2	No
	3	2,4	2,4	Yes
	4	2,3	2,2	No
	5	2,3	2,3	Yes
	6	0,0	1,2	No
	Result	3 𝑀𝑉 𝑚𝑎𝑡𝑐ℎ ⟹ 𝑁𝑀𝑀𝑉 = 50%

2.1.1 Adaptive Rood Pattern Search (ARPS) Method

[START_REF] Nie | Adaptive rood pattern search for fast blockmatching motion estimation[END_REF]

ARPS method is based on the fact that we expect the MV of a subblock to have the same direction and size as the MV of 2011 Third International Workshop on Quality of Multimedia Experience

Table 3 -

 3 Spearman correlation between the subjective results and 10 state of the art FR metrics results along with the results for the proposed metric for H.264 and MPEG2 compressed and IP distorted videos.

	Prediction Model	H264	MPEG2	IP
	PSNR	0.4296	0.3588	0.3206
	SSIM

Table 4 -

 4 Pearson correlation between the subjective results and 10 state of the art FR metrics results along with the results for the proposed metric for IP distorted videos.

		.6807	0.6116	0.4736
	Spatial MOVIE [5]	0.7066	0.6911	0.7046
	Temporal MOVIE [5]	0.7797	0.8170	0.7192
	MOVIE [5]	0.7664	0.7733	0.7157
	STAQ	0.9132 (1)	0.76515 (4)	0.7202 (1)

Third International Workshop on Quality of Multimedia Experience 978-1-4577-1334-7/11/$26.00 ©2011 IEEE

Third International Workshop on Quality of Multimedia Experience


Samia Bouzefrane

Khaled Garri

Pascal Thoniel

A user-centric PKI based-protocol to manage FC² digital identities

Keywords: Identity management system, public key infrastructure, federated identity, circle of trust, digital identity, security

published or not. The documents may come

A user-centric PKI based-protocol to manage FC 2 digital identities

Introduction

With a boom in online services generally accessed through a lot of login-password couples, Internet users are having an ever increasing number of digital identities. Indeed, Internet was not originally designed with the digital identity idea and, some solutions have been proposed to deploy digital-identity management architectures using existing standards and protocols such as InfoCard standard that is a user-centric approach or Liberty Alliance standard that is based on the notion of identity federation.

NTX Research 111 avenue Victor Hugo, 75116 Paris -France

A typical identity-management architecture requires basic components like an identity provider (IDP) that authenticates the user in a secure manner allowing him to access to a service provider (SP) and an attribute provider (AP) to supply the user attributes to any authorized agent while not compromising privacy. FC² (Federation of Circles of Trust) [START_REF] Consortium | Fédération de Cercles de Confiance et usages securisés de l'identité[END_REF] is a French project initiated by several companies jointly with government and academic actors. It takes into account the following points: the user must have control on his personal data, a great number of certificates must be provided at low cost, multiple services may belong to distinct groups and accessible via various material supports like usb key, smart card or a mobile equipment. FC² tries to bring a solution to these requirements by implementing a comprehensive platform that allows new secure electronic services based on a transparent and interoperable federated identity management. In this context, a new PKI 1 Our contribution, in this paper, concerns the "user" PKI that integrates an entity called "electronic notary" used instead of a certification authority, allowing the registration of new users (citizen/consumer/professional) within a registration authority that may be a proximity agency (telecom agency, banking agency) viewed as a trust third party. The proposed crypto-system is based on the same principle whatever asymmetric algorithm is used. The local registration authority delivers a "public key certificate" to the user along with a private key using his usb key, his smart card or his cell phone. The local -based protocol, called "2.0", has been proposed to guarantee secure access to electronic services at low cost. Based on three levels, FC² project integrates: -An international PKI that delivers and manages server certificates for identity providers, service providers and attribute providers.

-An internal PKI deployed by each registration authority (associated to each circle of trust) for all its agencies. -A "user" PKI that addresses final users. registration authority uploads user's "public key ownership certificate" to its central electronic notary server through a secure channel. Thus, anyone, any IDP, any application and any process can request this electronic notary server to authenticate the digital identity of the user. This trusted user is now able to access, at any time services belonging to distinct circles of trust, federated by FC² system in a transparent manner. The topic of trust and PKI management has been addressed during the last few years like in [START_REF] Linn | Trust Models and Management in Public-Key Infrastructures[END_REF], [START_REF] Blaze | Decentralized Trust Management[END_REF], [START_REF] Maurer | Modeling a public-key infrastructure[END_REF], [START_REF] Perlman | An Overview of PKI Trust Models[END_REF]. For example, John Linn in [START_REF] Linn | Trust Models and Management in Public-Key Infrastructures[END_REF] presents and compares several trust models and applied for use with public-key certificate infrastructures based on the X.509 specification, including subordinated hierarchies, cross-certified CA, hybrid CA, bridge CAs, and trust lists. More recent works deal with identity management systems and focus on the use of PKI within a federated architecture like in Liberty Alliance [START_REF]Liberty Alliance Trust Models[END_REF]. Another work on Liberty Alliance targeted a pan-European multi Circle of Trust environment [START_REF] Van Tran | Identity Federation in a Multi Circle-of-Trust Constellation[END_REF]. On the other hand, Windows CardSpace delivered with recent versions of .NET Framework manages identities according to a user-centric approach [START_REF]Windows CardSpace[END_REF]. A more sophisticated work introduces a formal semantics based calculus of trust that explicitly represents trust and quantifies the risk associated with trust in PKI and identity management [START_REF] Huang | A calculus of trust and its application to PKI and identity management[END_REF]. However, all these research works targeted a particular identity management system. Since CardSpace and Liberty Alliance are not interoperable, Jorstad et al. in [START_REF] Jorstad | Bridging CardSpace and Liberty Alliance with SIM authentication[END_REF] tried to integrate the current SIM authentication used in GSM with both Liberty Alliance and CardSpace such that it can be used for Internet services. Unfortunately this work is limited to mobile equipments and is not used with other physical supports. The PKI-based approach proposed in this paper allows managing different circles of trust defined in FC² project while each circle may adopt any identity management system independently from the system used by other circles of trust. Moreover, with our solution the user may access to FC² services after registration and authentication phase thanks to any physical support (smart card, USB key, cell phone, etc.) he has. In the rest of this paper, Section 2 recalls the objective of FC² platform. Section 3 describes the user-centric PKIbased approach that is proposed to access electronic services of different circles of trust while guaranteeing security at low cost. Section 4 details the way keys and certificates are generated and used. Section 5 explains the role of each involved actor through distinct use cases. Section 6 outlines the implemented software architecture, before concluding in Section 7.

The FC² project

The FC² project (see Figure 1) is a French R&D crosssector initiative including companies, government and academic actors. The project started on July 2007 and ended on June 2010. It aimed to:

• Define and implement interoperable identity federation architecture schemes, fully agnostic versus underlying technologies (Liberty Alliance [START_REF]Liberty Alliance Project[END_REF], Microsoft/Cardspace [3], Higgins [4], Open-ID [START_REF] Le | Identity management systems and interoperability in a heterogeneous environment[END_REF], etc.),

• Implement a dedicated infrastructure for service providers enrolment and support,

• Provide strong authentication and privacy management services,

• Provide a high level of protection against digital identity attacks,

• Provide a simple and convenient user experience, targeting user empowerment and trust as well as universality of use across a large variety of end-user devices,

• Create innovative business models, acceptable and/or adoptable by all players in the value chain,

• Provide the needed technologies and processes to master identity management technologies on a large scale basis, from national/government level to corporate/enterprise level.

The R&D developments within the project targeted especially the problems of Federated Identities in a crosssector environment, encompassing the e-government, local administrations, telecommunications and financial domains.

Regarding the FC² issues, the interoperability of various identity-management technologies and the user privacy have been addressed through research works undertaken by FC² partners, like in [START_REF] Le | Identity management systems and interoperability in a heterogeneous environment[END_REF] and [START_REF] Davoux | Federation of Circles of Trust and Secure Usage of Digital Identity[END_REF].

In this paper, we focus on the access to on-line services within a federated identity platform while allowing secure, low-cost, and user-centric properties.

The FC² context

As for any PKI, the main objective is naturally to ensure data exchange confidentiality, integrity and authenticity along with strong authentication of the actors. The nonfunctional objectives are the fact of allowing the materialization of trust within FC² framework, between distinct circles of trust and the user. It was decided in FC² project, not to impose higher certification authority nor hierarchy between the circles of trust, but to provide a security solution that may be collaborative between the actors, and that may be evolutionary, flexible for the integration of new partners (circles). The PKI-based solution that is deployed uses three levels:

• A bridge PKI: to which is assigned a certificate that is auto-signed or delivered by a known CA which signs for a given circle of trust the certificate of the internal CA.

• An international PKI: to deliver the public certificates of identity providers and attribute providers when communicating with users or electronic notaries.

• An internal PKI: with a root certificate auto-signed and a certificate signed by the bridge CA certificate; this PKI signs the certificates of the identity and attribute providers for their inter-circle communication, and those of each registration authority and each electronic notary when communicating each other.

PKI 2.0 principle

Our contribution concerns a protocol called PKI 2.0 to register new users within the FC² platform. It consists of two parts. The enrolment phase involves the registration of new users and the generation of keys and certificates. The verification phase that assumes the publication of new public-key ownership certificates on an electronic server acting as a notary that checks the validity of the user certificates. The first step concerns the Registration Authority (RA) that allows registering new users. The RA must have multiple proximity agencies distributed over the territory. In the real world, the RA and the proximity agencies are the following according to the circle of trust considered: In the governmental circle, the RA corresponds to "les Notaires de France" and the proximity agencies are notaries. In the banking circle, the RA is the bank and the proximity agencies are the banking agencies. In the telecom circle, the RA is the telecom operator and the proximity agencies are the local telecom agencies. RA has the role of checking the identity of the user, and of executing a dedicated procedure to deliver finally an autosigned public-key certificate and a "public-key ownership certificate". In such model, the cost is low since that certification authorities are not necessary. In our solution, the user holds his key pair and his certificates (certificate of public-key and certificate of public-key ownership) generated by the proximity agency called the Local Registration Agency (LRA). The certificate of public-key ownership published by a LRA on the Electronic Notary (EN) server is used to check the validity of the auto-signed certificate of public key. To distinguish them from the certificates generated by a server, they are called "customer" certificates. The user can use his certificates for encryption, authentication or signature. This protocol avoids the use of a certification authority to the benefit of the EN server. No certificate from registration authority is necessary since the user certificate is auto-signed. However, the LRA checks the certificate generation and insures the secured publication of the corresponding ownership certificate on the EN server. The auto-signature of a certificate does not bring any guarantee on its validity, it only insures to be in compliance with the standard X509v3 so as to be used by existing applications. In fact, the certificate validity is obtained on-line by requesting the EN server.

PKI 2.0 keys and certificates

In this section, we explain how keys and certificates are generated, and how the enrolment is performed.

Key-pair generation

To have a PKI 2.0 certificate, each user must have a pair of keys. The key generation can be done according to different ways:

-personal way: the user runs locally a software tool provided by FC². This software corresponds to an identity selector installed on his machine or his cell phone.

-decentralized way: the user goes physically to a Local Registration Agency that generates keys for him. -centralized way (not recommended): a RA -one by circle of trust -generates the keys for each person. At this stage, the user has two keys: a public key K pub and a private key K priv .

Certificates Generation

The PKI 2.0 recommends for each user two pairs of keys, one pair is dedicated for self authentication and electronic signature and the other pair for encryption. These pairs of keys have to be stored in secure way, especially for the private keys. The public-key certificates X509v3 are auto-signed and stored in plain text. The first one is an authentication/signature certificate whose legal value rises from European directive 1999/93. The second one is dedicated to encryption. The generation of certificates can be also carried out according to a personal, centralized or decentralized way. However, the decentralized procedure is the optimal way since it allows the generation of a public-key certificate and a public-key ownership certificate that guarantees the authenticity of the latter. These certificates are added to the information system for the first time during the enrolment phase.

Enrolment operation

The user enrolment process is done directly within a Local Registration Agency (LRA) according to three steps: -First step: checking user identity The user presents one or more identity documents to the registration agent that physically authenticates user. This face-to-face step is easy to realize within FC² project thanks to the LRA that are distributed over the territory.

-Second step: Generating public-key certificate

Once the user identity is checked, the registration agent launches the public-key certificate generation after having generated the corresponding key pair. This certificate contains: third party nationality (FR), third party type (registration agency), third party circle of trust, timestamping (validity period of the certificate), user identity, public key, auto-signature (with the user private key). Once the certificate is generated, the registration agent must register it on the user physical device (USB key, smart card or cell phone).

-Third step: Generating certificate of public-key ownership PKI 2.0 principle is that the consumer/citizen is responsible for certifying the ownership of his public key without involving any certification authority. For this purpose, PKI 2.0 adds a new certificate, called "public-key ownership certificate". This certificate does not contain the user public key. Instead, it contains the hash value of the public key (by using a hash function like MD5, SHA-1 or RIPE-MD). Hence, a certificate of public-key ownership contains: nationality of the third party (FR), type of the third party (registration agent), third party circle of trust, time-stamping, user identity, public key hash value. Once generated, this certificate is encrypted with the user private key.

The following section describes the different entities involved in the infrastructure.

Principal actors and use cases

PKI 2.0 gathers the following actors: LRA, EN servers, service providers, and finally users. The local agencies and servers of each circle of trust have an internal PKI which delivers the necessary certificates to establish a SSL communication. The electronic-notary servers have also certificates issued by a known certification authority for communication between them and users or service providers. Figure 2 illustrates the general architecture of the PKI 2.0 including the main actors and the interactions between them.

Main actors

User entity

The user, as consumer or citizen, is linked to a circle of trust. First of all, he has to register himself within a LRA to access the system, it's the enrolment phase. Afterwards, he will be able to communicate with the system to manage his own account (update, revoke or renew). Of course, he is also able to check any public-key certificates on any EN servers delivered by any RA of any circles of trust.

Local Registration Agency

This entity manages clients' enrolment by generating certificates and publishing and deleting of public-key ownership certificates onto the EN server.

Electronic Notary server

The EN server contains a register of ownership certificates. It is requested by other actors to authenticate the publickey certificates. Indeed, it stores the public-key ownership certificates published by LRA. The communication by a EN and a LRA is secured thanks to SSL certificates delivered by an internal PKI. EN server is requested to verify the users public-key certificates delivered by its own LRAs.

Service Providers

The service providers correspond to the web sites where users are identified using their public-key certificate. These providers ask EN to check the public-key certificates used by users to access the provided services.

Identity providers

The identity provider, one per circle of trust, stores publickey certificates of PKI 2.0 generated by LRA. The publication of these certificates is done automatically thanks to a software module integrated within the user identity selector.

Use Cases

New users enrolment

As stated before, this step is the first one allowing the user to access to FC² platform. It is performed by the user within a LRA. First of all, the user is authenticated by presenting an identity document. Once authenticated, several steps may be carried out in order to issue the public-key certificate, that is, to: -register the user in a database which contains the list of users. The information stored in the data base are: Common Name, gender, date of birth, postal address and e-mail. These data items are inserted automatically into the certificates generated as in the following steps.

-generate the public-key certificate, that is auto-signed with the private key of the user.

-register the public-key certificate (in . P12 and .cer formats) on the physical device of the user.

-generate the public-key ownership certificate under .P12 format, and finally encrypt this certificate using the private key of the user. This operation allows user, identity provider, or service provider to check a public-key certificate in real time, in order to access an Internet service or to exchange data with another actor. The verification process is done as in the following: -request a EN server whose the address is in the certificate using a SSL communication and authentication with the concerned EN -the public-key certificate is sent to the already authenticated EN -then EN extracts the serial number that is also the serial number of the public-key ownership certificate, -EN looks at the public-key ownership certificate in its data base, -if the public-key ownership certificate is found in the data base, EN tries to decrypt the public-key ownership certificate with the public-key extracted from the given/received public-key certificate, -if the public-key ownership certificate has been successfully opened, EN extracts the hash value from the public-key ownership certificate, and compares it with the one computed with the public key contained in the received certificate. If they are equal, the verification is successful. As described in the following section, an implementation of PKI 2.0 has been performed.

Software architecture

We implemented the PKI 2.0 by developing three modules: the first module proposes a tool for the LRA. It allows to manage user accounts and to control the publication of ownership certificates on the EN server. As an example, Figure 4 illustrates the case of adding a new user into a circle of trust. The second module is devoted to the EN, it is composed of two programs: one to answer the publishing requests from his Local Registration Agencies, and the other to answer the check requests from any service providers and any users. The third module is devoted to external customers (users, IDP and service providers) that want to check a public-key certificate of PKI 2.0 (see Figure 3).

Conclusion

Our PKI-based solution seems to be an answer to the FC² needs in terms of security. This proposal is a user-centric system that fulfills the requirements of a large number of users at low costs. Moreover, the PKI 2.0 solves the problem of managing multiple digital identities by allowing the use of only one or few identities within several circles of trust, by replacing Certification Authorities by Registration Authorities that have proximity agencies easily accessible to citizens/consumers. As a perspective to our work, we aim to develop the module of checking as a plugin to Web browsers.

Fig.3. Checking a public-key certificate

  Fig.1. FC² architecture

Fig. 2 .

 2 Fig.2. PKI 2.0 Architecture overview

Acknowledgments

This research is part of the project called FC2 (Federation of Circles of Trust -www.fc2consortium.org). Khaled Garri has been partly financially supported by the DGCIS (Direction Générale de la Compétitivité de l'Industrie et des Services). We are thankful to students Nesrine Jlidi, and Mohamed Mammeri, who contribute in the implementation of the software tools.

Publication of public-key ownership certificate

The public-key ownership certificate allows checking the authenticity of the public key through the hash value of this latter that is inserted into the certificate. This certificate must be published onto the EN server. This is done thanks to SSL communication with the NE server. The SSL authentication is done mutually by using SSL certificates generated by the internal PKI. Once the mutual authentication is achieved successfully, a message is sent to EN with the following information:

: is the identity of the LRA i, M : the message contains the ownership certificate encrypted with the private key of the user, and other information like : serial number, version, signature algorithm. H(M) : the hash value of M. All these information are encrypted with the public key of EN. The use of the signature enables us to guarantee the message integrity. The encryption with the public key of EN guarantees confidentiality since that only the appropriate EN will decrypt the message. Upon receiving the message, the EN begins decrypting the message using its private key, and then checks the signature. For this purpose, the EN reads the identity of the sending LRA because it has a register of all the public-key certificates of his local agencies, indexed with their serial number. Then, EN begins extracting the certificate that contains the public key of the concerned LRA, in order to verify the signature. EN computes the hash value of M and compares it to the one received. If they are equal, EN stores the certificate of ownership in its database, otherwise an error message is notified to the LRA.

User account management

This task concerns the update, delete and read operations carried out on the user account. The update may concern the modification of some personal information or even the generation of a public-key certificate with the new information. The account deletion implies the deletion of the public-key ownership certificate within EN. In this case, the LRA sends the following message to inform EN about the revocation:

Where M={serial number of public-key certificate, removal} The renewal of the public-key certificate is necessary after removal.

Public-key certificate verification