Mohamed-Chaker Larabi
email: larabi@sic.univ-poitiers.fr

Sabine Süsstrunk

Evaluation of a Hyperspectral Image Database for Demosaicking purposes

Keywords: color filter array, CFA, digital still camera, spectral sensitivities, demosaicking, hyperspectral images

We present a study on the the applicability of hyperspectral images to evaluate color filter array (CFA) design and the performance of demosaicking algorithms. The aim is to simulate a typical digital still camera processing pipe-line and to compare two different scenarios: evaluate the performance of demosaicking algorithms applied to raw camera RGB values before color rendering to sRGB, and evaluate the performance of demosaicking algorithms applied on the final sRGB color rendered image. The second scenario is the most frequently used one in literature because CFA design and algorithms are usually tested on a set of existing images that are already rendered, such as the Kodak Photo CD set containing the well-known lighthouse image. We simulate the camera processing pipe-line with measured spectral sensitivity functions of a real camera. Modeling a Bayer CFA, we select three linear demosaicking techniques in order to perform the tests. The evaluation is done using CMSE, CPSNR, s-CIELAB and MSSIM metrics to compare demosaicking results. We find that the performance, and especially the difference between demosaicking algorithms, is indeed significant depending if the mosaicking/demosaicking is applied to camera raw values as opposed to already rendered sRGB images. We argue that evaluating the former gives a better indication how a CFA/demosaicking combination will work in practice, and that it is in the interest of the community to create a hyperspectral image dataset dedicated to that effect.

INTRODUCTION

Digital photography is a part of our daily life because many devices, such as digital still and video cameras, smartphones, webcams, etc., are available at affordable prices. The consumer electronic device industry has mostly adopted single-sensor imaging, which captures the three spectral wave-bands red, green, and blue on a single sensor. These cameras are more cost-effective and usually more compact than tri-sensor cameras, which use three sensors for full-resolution red, green, and blue scene information. Spectral selectivity on a single sensor, be it a charge-coupled device (CCD) [START_REF] Dillon | Color imaging system using a single CCD area array[END_REF] or a complementary metal oxide semiconductor (CMOS) 2 is achieved by adding a color filter array (CFA) [START_REF] Lukac | Color filter arrays: Design and performance analysis[END_REF][START_REF] Li | Color filter arrays: representation and analysis[END_REF][START_REF] Lu | Designing color filter arrays for the joint capture of visible and near-infrared images[END_REF] in front of the sensor, such as the most common Bayer filter. [START_REF] Bayer | Color imaging array[END_REF] The resulting image from CFA acquisition is a gray-scale image with one single color at each pixel. To recover tri-component or full-color at each pixel, the image needs to be processed with a color reconstruction algorithm called demosaicking. Figure 1 gives an example of a Bayer CFA image (a) and its demosaicked result (b).

The design of color filter arrays (CFA) and associated demosaicking algorithms is still an active research topic, as the perfect spatial arrangement of the filters and their spectral characteristics have a large influence on image quality. Image demosaicking could be regarded as an interpolation problem that creates full-resolution color images from CFA-based single-sensor images. In interpolation, the aim is to estimate the missing half amount of green pixels (quincunx interpolation) and reconstruct the missing three quarters of the red and blue pixels (rectangular interpolation) as performed by bilinear and bicubic interpolations. However, this approach is a simplistic view of the real demosaicking process because it does not take into account intra-and inter-channel dependencies.

Demosaicking has been studied by many researchers, we have a rich literature with various methodologies in the spatial or frequency domain, using image geometry, applying refinement and postprocessing techniques, etc. Many comprehensive comparisons and/or surveys have been published to date. [START_REF] Ramanath | Demosaicking methods for bayer color arrays[END_REF][START_REF] Alleysson | Linear demosaicing inspired by the human visual system[END_REF][START_REF] Gunturk | Demosaicking: color filter array interpolation[END_REF][START_REF] Li | Image demosaicing: a systematic survey[END_REF] Although all these efforts have resulted in very efficient demosaicking algorithms, we claim that there is no appropriate image database dedicated to these types of evaluations. Many authors used and continue to use a test set composed of KODAK PhotoCD images to prove the efficiency of their demosaicking algorithm and the quality of image reconstruction. The frequency spectra of these images are very interesting, especially on the widely used lighthouse image, as they allow to easily visualize reconstruction errors. However, the original PhotoCD images are slightly compressed due to the Gaussian pyramid used in the encoding that allows extracting different resolutions from the same PCD file. Additionally, they have been rendered to a limited color gamut, similar to the current sRGB color encoding standard. [START_REF]Multimedia systems and equipment -colour measurment and management -Part 2-1: colour management-default RGB colour space -sRGB[END_REF] The experiments are thus conducted by recreating the CFA structure from the final rendered image without taking into account that the images have been preprocessed and that the color component information is no longer original. Li et al. [START_REF] Li | Image demosaicing: a systematic survey[END_REF] have shown that most existing demosaicking algorithms achieve good performance on the Kodak data set but their performance degrades significantly on another set they used (IMAX high-quality images with varying-hue and high-saturation edges). Their study demonstrates that for testing CFA design and demosaicking algorithms, there is a real need for new content that is adapted to the task.

In this paper, we study the applicability of the hyperspectral image database proposed by Foster et al. [START_REF] Foster | Information limits on neural identification of coloured surfaces in natural scenes[END_REF] for demosaicking evaluations. This database is composed of eight hyperspectral images (see Figure 4). The idea behind our experiments is to simulate as close as possible the in-camera processing steps of a digital camera. Thus, we also use real measured spectral sensitivities of a camera in order to reproduce raw sensor response values. The obtained results are compared with those obtained by mosaicking/demosaicking the rendered images, and additionally with ground truth images obtained by using the 1931 CIE color matching functions (CMF) to sample the hyperspectral data.

The remainder of this paper is organized as follows. Section 2 describes our experimental procedure in detail. Section 3 summarizes how the hyperspectral database was obtained. The selected demosaicking techniques are briefly discussed in Section 4. Metrics are an important part of evaluating demosaicking results and they are presented in Section 5. Section 6 discusses the experimental results, and Section 7 ends the article with conclusions and future work.

EXPERIMENTAL PROCEDURE

This study's experimental procedure is summarized in the synopsis of figure 2. Our purpose is to provide an answer to whether or not evaluating mosaicking/demosaicking algorithms directly on rendered images is correct from an image quality point of view. As discussed above, most of the published research study demosaicking algorithms on already rendered images, while in most digital cameras, demosaicking occurs in raw camera RGB before color rendering. We thus simulate the relevant in-camera processing steps of a digital camera, from acquisition to rendering, and compare results obtained by applying demosaicking before and after sRGB rendering. Our simulation is similar to the one of Alleysson et al., [START_REF] Alleysson | Influence of Spectral Sensitivity Functions on color demosaicing[END_REF] who optimized camera spectral sensitivities based on one given demosaicking algorithm. [START_REF] Alleysson | Linear demosaicing inspired by the human visual system[END_REF] The different steps of the synopsis of Figure 2 are described below. Instead of acquiring real scene information from a (real) sensor, we use an already existing hyperspectral image data set that is described in Section 3. The images' raw sensor values are obtained by applying the well known image formation model. We thus first multiply the reflectance spectra at each pixel with the spectrum of the illuminant (i.e. D65 in our case), and then multiply the resulting color signal with the spectral sensitivities. We chose D65 as the illuminant to avoid having to white-balance the image, as it is the standard illuminant used for sRGB encoding. 11 However, as the spectral sensitivities were derived by measuring the quantum efficiency of a real digital camera (see Figure 3-a), we do apply a gain control to each channel to compensate for the different quantum efficiencies of the red, green, and blue channels. At this stage, we have obtained an image in "camera RGB" used in the following steps.

Image formation is an analog process, and we thus calculate in floating point up to now. In order to simulate the analog to digital conversion, we perform a quantization to 12-bits per channel, which corresponds to the common coding length nowadays. The quantized image is mosaicked according to the Bayer 6 CFA and then demosaicked with the algorithms described in Section 4.

In order to render the mosaicked/demosaicked images to sRGB, we first need to find the linear transform that maps the pixel values from camera RGB to XYZ tristimulus values. We use a simple least squares fitting, but there are much more sophisticated methods to obtain the matrix. [START_REF] Finlayson | White-point preserving color correction[END_REF] Note that real camera sensitivities do not fulfill the Luther condition, in other words, they are not within a linear combination of the CIE color matching functions. As such, any linear transform will not correctly map all camera RGB values to the corresponding XYZs, and a residual error is obtained that will be reflected in the results of our simulations to the ground truth images.

After camera RGB to XYZ conversion, the images are mapped to 8-bit sRGB using the method described in the standard. [START_REF]Multimedia systems and equipment -colour measurment and management -Part 2-1: colour management-default RGB colour space -sRGB[END_REF] We do not consider any additional rendering operations for preferred reproduction, [START_REF]Photography and graphic technology extended colour encodings for digital image storage, manipulation and interchange part 1: Architecture and requirements[END_REF] as is the case in more sophisticated digital cameras that apply image specific rendering. Such additional color rendering operations will of course also influence the results, and could be considered in our simulation framework. However, as such operations are highly image and preference dependent, we omitted them in this preliminary study The steps described above represent scenario 1 of the synopsis given in Figure 2. Omitting the mosaicking/demosaicking step results in the "original" image, called O2, that we use for evaluation. For scenario 2, we additionally mosaick/demosaick O2, analogue to the procedure followed in the demosaicking literature, i.e. mosaicking/demosaicking an already rendered images (e.g., Kodak PhotoCD images). Scenario 3 illustrates how we obtain the ground truth image, called O3, by simply applying the CIE color matching functions on the hyperspectral data and then rendering to sRGB. We use the image O3 to compare the demosaicking results of scenario 1 and 2.

Note that our simulation does not include all in-camera processing, as the above mentioned white-balancing and image specific color rendering, as well as linearization, flare subtraction, noise removal and filtering, sharpening, and compression. While these are important steps that will also influence the demosaicking result, they are highly camera and/or image dependent, which makes their inclusion into a simulation very challenging.

The evaluations of the our scenarios are performed using different well-known metrics such as CMSE, CPSNR, s-CIELAB and MSSIM for structural informations. These measures are described in Section 5.

HYPERSPECTRAL DATASET

For our simulations, we use the hyperspectral image database created by Foster et al. [START_REF] Foster | Information limits on neural identification of coloured surfaces in natural scenes[END_REF] in 2004. For the capture of these images, the authors used a high-spatial-resolution hyperspectral imaging system to acquire data from rural and urban scenes in Portugal, namely a low-noise Peltier-cooled digital camera providing a spatial resolution of 1344 × 1024 pixels (Hamamatsu, model C4742-95-12ER, Hamamatsu Photonics K.K., Japan) with a fast tunable liquid-crystal filter (VariSpec, model VS-VIS2-10-HC-35-SQ, Cambridge Research & Instrumentation, Inc., MA) mounted in front of the lens, together with an infrared blocking filter. Focal length was typically set to 75 mm and aperture to f16 or f22 to achieve a large depth of focus. The line-spread function of the system was close to Gaussian with standard deviation approx. 1.3 pixels at 550 nm. The intensity response at each pixel, recorded with 12-bit precision, was linear over the entire dynamic range. The peak-transmission wavelength was varied in 10-nm steps over 400-720 nm. This set is composed of 8 hyperspectral images, as shown rendered to sRGB in Figure 4.

(S1) (S2) (S3) (S4) (S5) (S6) (S7) (S8)

DEMOSAICKING ALGORITHMS

As mentioned previously, there are many algorithms for single-sensor image demosaicking. Some of them are sequential, i.e. color components are interpolated separately and the others exploit inter-channel correlation. In this section, we only briefly describe the methods we use, the reader is referred to the original papers for more details. As this paper does not intend to compare demosaicking algorithms but only to evaluate the effect of using hypespectral images with a camera simulation that includes a demosaicking step, the selected algorithms are not intended to cover the whole state-of-the-art.

Bilinear Model

Bilinear interpolation is one of the simplest and most used algorithm performing a high-quality linear interpolation. It interpolates a missing channel by taking the averages of the closest neighbors of the same channel. For example, the green channel at a red or blue pixel can be estimated as shown by the following equation:

Ĝ(i, j) = 1 4 [G(i -1, j -1) + G(i -1, j + 1) + G(i + 1, j -1) + G(i + 1, j + 1)]. ( 1
)
Bilinear interpolation is perhaps the most trivial demosaicking algorithm. It completely ignores inter-channel color correlation because each channel is estimated separately. This approach offers fast demosaicking but with questionable quality, such as noticeable false colors and blur along edges.

Alleysson et al.

Alleysson et al. [START_REF] Alleysson | Linear demosaicing inspired by the human visual system[END_REF] showed that the spatial multiplexing of the red, green, and blue signal in a Bayer CFA is equivalent to multiplexing the frequency of an achromatic "luma" component and two modulated chroma components. In addition, the luminance and chrominance components are sufficiently isolated in the frequency domain to consider the construction of demosaicking algorithms based on frequency analysis. The algorithm separately extracts estimates of luminance and modulated chrominance by filtering the Bayer CFA mosaick using two-dimensional filters with appropriate bandwidths, and then converts the estimated luma and the two demodulated chrominance values at each spatial location into RGB values.

Dubois et al.

Dubois [START_REF] Dubois | Frequency-domain methods for demosaicking of bayer-sampled color image[END_REF] defined a locally-adaptive luma-chroma demultiplexing algorithm that exploits the redundancy of one chrominance in the Bayer CFA mosaick by selecting, locally, the best estimate using the more decorrelated component to the luma signal. The work in [START_REF] Dubois | Filter design for adaptive frequency-domain bayer demosaicking[END_REF] introduced a least-squares approach for optimal filter design that replaced the window filter method used in the previous version. This new filter design method produces lower order filters that achieved virtually identical demosaicking quality as the higher order filters. We use the second method in this paper.

EVALUATION METRICS

To perform analytical assessment of the defined scenarios, we need one or several quality measure to assess different types of artifacts. We selected the commonly used CMSE and CPSNR, s-CIELAB, which characterizes the regions in the test image that are visually different from the original image, and MSSIM that measures differences in structural content. Recall that O 2 and O 3 are the "original" images without demosaicking, obtained by either applying camera sensitivities or color matching functions, respectively. Thus, the metrics are applied to evaluate the demosaicking results of: 1) Scenario 1 in comparison to O 2 ; 2) Scenario 2 in comparison to O 2 ; 3) Scenario 1 in comparison to O 3 ; ad 4) Scenario 2 in comparison to O 3 . These metrics are briefly described below.

CMSE & CPSNR

In the demosaicking literature, it is very common to use the composite peak-signal-to-noise ratio (CPSNR) to compare the reconstructed images to full color RGB images. We use equation 2 and 3 to calculate the CPSNR of a demosaicked image compared to the original one. Here, I(i, j, k) is the pixel intensity at location (i, j) of the k-th color component of the original image and I'(i, j, k) of the reconstructed image. M and N are the height and the width of the frame.

CP SN R = 10log 10 255 2 CM SE , (2)
where

CM SE = 1 3 × M N 3 k=1 M i=1 N j=1 [I(i, jk) -I ′ (i, j, k)] 2 .
(3)

s-CIELAB

s-CIELAB was proposed by Zhang and Wandell 19 as a spatial extension to CIELAB to account for how spatial pattern influences color appearance and color discrimination. The spatial extension is accomplished by performing a pre-processing on the CIELAB channels before applying the color difference formula. In our application, the input image is first converted to an opponent encoding (one luminance and two chrominance color components). Each component is spatially filtered to mimick the spatial sensitivity of the human eye. The final filtered images are then transformed into XYZ so that the standard CIELAB color difference formula can be applied.

MSSIM

The Multiscale Structural Similarity Index (MS-SSIM) [START_REF] Wang | Multi-scale structural similarity for image quality assessment[END_REF] attempts to model the physical properties of the HVS. The MS-SSIM follows a top-down paradigm that first decomposes images into several scales and then measures contrast and structure in each scale. In addition, the luminance of the lowest scale is also measured. Finally, all the data is pooled into a single score. MS-SSIM has the advantage that it is computationally tractable while still providing reasonable correlations to subjective measurements.

RESULTS AND DISCUSSION

Here, we evaluate the results of the scenarios described in Section 2 and illustrated in Figure 2. Recall that scenario 1 is the simulation of a camera pipe-line (demosaicking before sRGB rendering). In scenario 2, we perform demosaicking after color rendering (as done in the literature). The mosaicking/demosaicking step corresponds to creating mosaicks according to the Bayer CFA 6 before applying the three different demosaicking algorithms. For scenario 3, we generate a ground truth image obtained by applying the CIE color matching functions to the hyperspectral data and rendering directly to sRGB. For each scene S*, except S5, we executed the three scenarios, thus obtaining the originals of scenario 2 and 3 (O S * 2 and O S * 3 ) and the results of the selected demosaicking algorithms for scenario 1 and 2 (BI1 S * , AL1 S * , DU 1 S * , BI2 S * , AL2 S * and DU 2 S * ). Scene S5 was rejected from the experiments because it doesn't contain the same number of spectral bands than the 7 others. Figures 5 and7 show the results obtained with all scenarios for scene S4 and S7. We can notice that the rendered images are close to those given by Foster et al. (see Figure 4) except for a color difference because they manually edited the pictures. It is difficult to visually evaluate the difference between the results except for bilinear interpolation. The latter gives very different results when applied in scenario 1 or 2.

Camera rendered image O 2 BI1 AL1 DU1 CMFs rendered image O 3 BI2 AL2 DU2
In order to better perceive the artifacts generated, we zoom into parts of scenes S4 (Fig. 6) and S7 (Fig. 8). We note some demosaicking artifacts around the pistil of the flower in S4 and around the window in S7. Also, there seems to be more artifacts in the images from scenario 2 than in those from scenario 1, especially around the windows. However, that result is not corroborated by the objective metric s-CIELAB in Table 1 and 2, which is supposed to predict visual differences. For the quantitative evaluation, we used the metrics described in section 5, i.e. CMSE, CPSNR, s-CIELAB and MSSIM. These metrics have been calculated between demosaicked images of scenario 1 and 2 (BI1 S * , AL1 S * , DU 1 S * , BI2 S * , AL2 S * and DU 2 S * ) and the camera rendered image O S * 2 (see Table 1) and the CMF rendered image O S * 3 (see Table 2), respectively. By applying all these different metrics, we evaluate different types of artifacts. The CMSE and CPSNR focus on color signal differences, s-CIELAB aims at detecting perceived errors, and MSSIM evaluates the structural content of the image.

The first remark that concerns both tables is that the results are highly dependent on image content, arguing that a large image data set should be available as an analysis based on average performance might not be meaningful. Additionally, these results also argue for a common image data set to evaluate different algorithms, such as is available with the Kodak images. However, both tables clearly show that there is a difference between scenario 1 and scenario 2 with regards to performance. In general, the more realistic camera processing, as simulated with scenario 1, results in worse performance then the usually applied scenario 2. This is similar to what was found by Li et al. [START_REF] Li | Image demosaicing: a systematic survey[END_REF] when applying mosaicking/demosaicking to IMAX images. This argues for a more realistic simulation to evaluate such algorithms.

Among the selected techniques, bilinear interpolation is the worst for scenario 1 and with the highest difference (up to a CPSNR of 14 dB for S4, for instance). Alleysson et al.'s technique is performing better than Dubois for all the images, but with a higher difference in scenario 1. Thus, the difference in performance of algorithms is better evaluated with a simulation that is closer to a real camera pipe-line as opposed to what is currently done (i.e. scenario 2).

As expected, all errors are much higher when comparing the performance to the CMF rendered image O S * 3 (see Table 2). When only evaluating the influence of mosaicking/demosaicking, thus assuming the other processing parameters remain the same, it is thus probably more appropriate to use O S * 2 as the ground truth to compare with. Table 3 evaluates the difference between O S * 2 and O S * 3 , using the same metrics, for all 7 scenes. Note that the difference can be very high as for the case of S2 and S4. Additionally, it is more difficult to discriminate the demosaicking techniques between scenario 1 and scenario 2, the difference is smaller. Thus, comparing with the ground truth images tends to compress the difference between the demosaicking algorithms, which in the case of this study is coherent with the visual results of figure 5 and7. It may be suitable to compare the output of a simulated camera pipe-line like scenario 1 with the original image of scenario 3 to get a better visual judgment. This can be confirmed with a psychophysical experiment. Table 4 shows the average correlation between the four metrics used in our evaluation, for the seven scenes and the measures listed in Table 1 andTable 2. We can thus evaluate whether the metrics give consistent results. From the correlation of Table 1, the high values between CMSE and CPSNR are not surprising because the computation of the second depends on the first. However, the high correlation between s-CIELAB and CPSNR (0.96) and thus CMSE (0.98) was not expected, especially because these measures do not focus on the same artifacts as stated before. Finally, the results of MSSIM are not highly correlated with the others because it focuses mainly on the structure of the image. That being said, the correlation ratio is high enough to say that all the metrics indicate similar performance. These observations do not hold for the results in Table 2. The correlation between s-CIELAB and CPSNR (CMSE) decreases drastically with losses around 30%. It is also lower for MSSIM but the decrease is less, around 7%. This again argues against using scenario 3 for comparison.

CONCLUSION

We studied the use of hyperspectral images for the purpose of single-sensor image demosaicking evaluation. We designed an in-camera processing pipe-line to render the hyperspectral data to sRGB images. We could thus evaluate the performance of demosaicking algorithms applied to raw camera RGB values (scenario 1), which is closer to real camera design, and to compare with current evaluation practices that evaluate demosaicking on already rendered images (scenario 2).

We demonstrated the usefulness of using scenario 1 by comparing three different demosaicking algorithms and evaluating the reconstruction results with four different metrics (CMSE, CPSNR, s-CIELAB, and MSSIM). We found that in general, the demosaicking algorithms perform worse in scenario 1 than scenario 2. Additionally, the differences between the algorithms are more evident in scenario 1. We thus conclude that scenario 1, which is closer to real in-camera processing, provides a more accurate evaluation of demosaicking than current practices, which is to evaluate on already color rendered images. However, to implement scenario 1, we need a hyperspectral image data set.

Even though Foster et al. did our community a service by creating the hyperspectral image database and making it freely available, these images are not adapted for demosaicking evaluation purposes. This is partly due to the low-pass behavior of the optics of the real camera used to capture the images, which filters the high frequencies that often create problems for demosaicking algorithms, but that facilitates visual judgement. Additionally, there is not enough variation in scene content and chromaticity to be a representative sample of the world.

In this contribution, we focused only on the evaluation of demosaicking algorithms. The same framework can of course also be applied to study joint color filter array design/demosaicking. Thus, it is of benefit to the community to build a new hyperspectral database specific to this purpose by selecting scenes like the famous "lighthouse" from Kodak PhotoCD, which has image characteristics that facilitate the visual interpretation of the algorithms' performance.

Figure 1 .

 1 Figure 1. Bayer CFA-based single-sensor imaging: (a) grayscale CFA image and (b) full-color image.

Figure 2 .

 2 Figure 2. Synopsis of the simulations used in the paper.

Figure 3 .

 3 Figure 3. (a) Spectral sensitivity functions of the our digital camera and (b) the 1931 CIE Color matching functions.14

Figure 4 .

 4 Figure 4. Rendered versions of the hyperspectral images used in our experiments.

Camera rendered image O 2 BI1Figure 5 .

 25 Figure 5. Results for the scene S4 from the hyperspectral database. O2 and O3 are the "original" images rendered in scenario 2 and scenario 3, respectively. BI1, AL1, and DU1 are Bilinear, Alleysson, and Dubois results for scenario 1, respectively. BI2, AL2, and DU2 are Bilinear, Alleysson, and Dubois results for scenario 2, respectively.

Figure 6 .Figure 7 .Figure 8 .

 678 Figure 6. A zoom on part of scene S4. O2 and O3 are the "original" images rendered in scenario 2 and scenario 3, respectively. BI1, AL1, and DU1 are Bilinear, Alleysson, and Dubois results for scenario 1, respectively. BI2, AL2, and DU2 are Bilinear, Alleysson, and Dubois results for scenario 2, respectively.

Table 1 .

 1 Evaluation of scenario 1 (BI1, AL1, DU1) and scenario 2 (BI2, AL2, DU2) demosaicking results in comparison to the original image obtained in scenario 2 (O2) using CMSE, CPSNR, s-CIELAB and MSSIM. BI1, AL1, and DU1 are Bilinear, Alleysson, and Dubois results for scenario 1, respectively. BI2, AL2, and DU2 are Bilinear, Alleysson, and Dubois results for scenario 2, respectively.

	Scene	Measure	BI1	AL1	DU1	BI2	AL2	DU2
	S1	CMSE	121,22	31,07	37,47	21,68	23,79	27,03
		CPSNR	27,30	33,21	32,39	34,77	34,37	33,81
		s-CIELAB	6,18	2,84	2,69	0,83	1,45	1,43
		MSSIM	0,9827	0,9880	0,9853	0,9934	0,9890	0,9862
	S2	CMSE	233,98	54,06	73,76	28,17	46,76	56,61
		CPSNR	24,44	30,80	29,45	33,63	31,43	30,60
		s-CIELAB	13,74	4,36	5,20	1,47	2,42	2,48
		MSSIM	0,9787	0,9851	0,9799	0,9935	0,9871	0,9837
	S3	CMSE	63,45	47,28	53,54	13,68	20,76	25,13
		CPSNR	30,11	31,38	30,84	36,77	34,96	34,13
		s-CIELAB	4,15	2,28	2,77	0,84	1,33	1,36
		MSSIM	0,9864	0,9700	0,9686	0,9930	0,9877	0,9853
	S4	CMSE	265,60	19,68	26,38	8,34	11,06	12,54
		CPSNR	23,89	35,19	33,92	38,92	37,69	37,15
		s-CIELAB	18,97	1,83	2,01	0,52	0,70	0,77
		MSSIM	0,9787	0,9809	0,9741	0,9921	0,9872	0,9846
	S6	CMSE	187,91	17,31	17,70	13,15	8,12	10,19
		CPSNR	25,39	35,75	35,65	36,94	39,04	38,05
		s-CIELAB	12,25	3,55	3,70	0,77	0,79	0,91
		MSSIM	0,9904	0,9960	0,9951	0,9961	0,9962	0,9947
	S7	CMSE	28,53	12,31	13,64	15,05	9,39	10,95
		CPSNR	33,58	37,23	36,78	36,36	38,40	37,74
		s-CIELAB	3,31	1,91	1,76	0,78	0,82	0,93
		MSSIM	0,9927	0,9957	0,9940	0,9953	0,9961	0,9949
	S8	CMSE	16,09	3,56	3,94	10,68	4,09	3,70
		CPSNR	36,07	42,62	42,18	37,84	42,01	42,45
		s-CIELAB	1,64	0,53	0,63	0,53	0,44	0,45
		MSSIM	0,9940	0,9974	0,9969	0,9962	0,9975	0,9969
	Average	CMSE	130,97	26,47	32,35	15,82	17,71	20,88
		CPSNR	28,68	35,17	34,46	36,46	36,84	36,28
		s-CIELAB	8,61	2,47	2,68	0,82	1,14	1,19
		MSSIM	0,9862	0,9876	0,9848	0,9942	0,9915	0,9895

Table 2 .

 2 Evaluation of scenario1 (BI1, AL1, DU1) and scenario2 (BI2, AL2, DU2) demosaicking results in comparison to the original image obtained in scenario 3 (O3) using CMSE, CPSNR, s-CIELAB and MSSIM. BI1, AL1, and DU1 are Bilinear, Alleysson, and Dubois results for scenario 1, respectively. BI2, AL2, and DU2 are Bilinear, Alleysson, and Dubois results for scenario 2, respectively.

	Scene	Measure	BI1	AL1	DU1	BI2	AL2	DU2
	S1	CMSE

Table 3 .

 3 Evaluation of the difference between O2 (original image obtained from scenario 2) and O3 (original image obtained form scenario 3).

	Metrics	S1	S2	S3	S4	S6	S7	S8	Average
	CMSE	31,34	143,44	30,93	126,10	13,30	22,66	5,14	53,27
	CPSNR	33,17	26,56	33,23	27,12	36,89	34,58	41,02	33,23
	s-CIELAB	4,89	10,41	2,26	6,01	3,00	2,77	1,85	4,46
	MSSIM	0,9931	0,9885	0,9945	0,9557	0,9964	0,9948	0,9983	0,9888

Table 4 .

 4 Correlation between evaluation metrics.

95,66 54,57 65,82 50,37 50,74 55,50 CPSNR 28,32 30,76 29,95 31,11 31,08 30,69 s-CIELAB 4,49 4,54 5,10 4,96 5,25 5,19 MSSIM 0,9809 0,9838 0,9795 0,9873 0,9837 0,9795 S2 CMSE 192,27 147,69 234,82 170,85 187,84 199,40 CPSNR 25,29 26,44 24,42 25,80 25,39 25,13 s-CIELAB 14,64 8,00 10,04 10,18 10,43 10,50 MSSIM 0,9711 0,9738 0,9658 0,9820 0,9756 0,9701 S3 CMSE 44,27 70,95 66,52 45,04 49,25 55,45 CPSNR 31,67 29,62 29,90 31,59 31,21 30,69 s-CIELAB 4,76 3,51 3,83 2,48 2,74 2,78 MSSIM 0,9830 0,9708 0,9680 0,9872 0,9848 0,9815 S4 CMSE 231,89 152,93 174,89 132,15 136,93 137,82 CPSNR 24,48 26,29 25,70 26,92 26,77 26,74 s-CIELAB 13,98 5,24 7,71 6,10 6,08 6,13 MSSIM 0,9407 0,9341 0,9294 0,9508 0,9443 0,9420 S6 CMSE 183,47 25,15 30,62 25,80 21,56 24,16 CPSNR 25,50 34,13 33,27 34,01 34,79 34,30 s-CIELAB 10,84 3,23 5,26 3,25 3,13 3,17 MSSIM 0,9891 0,9934 0,9918 0,9929 0,9927 0,9907 S7 CMSE 56,23 40,46 39,63 37,27 32,23 33,38 CPSNR 30,63 32,06 32,15 32,42 33,05 32,90 s-CIELAB 4,76 3,64 3,38 2,91 2,91 2,94 MSSIM 0,9874 0,9902 0,9884 0,9901 0,9906 0,9890 S8 CMSE 19,04 7,78 9,92 16,13 9,17 9,12 CPSNR 35,33 39,22 38,17 36,05 38,51 38,53 s-CIELAB 2,70 1,78 2,17 1,87 1,88 1,90 MSSIM 0,9923 0,9960 0,9954 0,9943 0,9960 0,9952 Average CMSE 117,55 71,36 88,89 68,23 69,67 73,55 CPSNR 28,75 31,22 30,51 31,13 31,54 31,28 s-CIELAB 8,03 4,28 5,36 4,54 4,63 4,66 MSSIM 0,9778 0,9774 0,9740 0,9835 0,9811 0,9783

ACKNOWLEDGMENTS

A special thanks to Foster et al. for making their hyperspectral image database 12 freely available.


Omar Benahmed Daho

Mohamed-Chaker Larabi
email: larabi@sic.univ-poitiers.fr

Jayanta Mukhopadhyay

A JPEG-Like Algorithm for Compression of Single-Sensor Camera Image

Keywords: Color Filter Array, compression, JPEG, demosaicking, quality

This paper presents a JPEG-like coder for image compression of single-sensor camera images using a Bayer Color Filter Array (CFA). The originality of the method is a joint scheme of compression.demosaicking in the DCT domain. In this method, the captured CFA raw data is first separated in four distinct components and then converted to YCbCr. A JPEG compression scheme is then applied. At the decoding level, the bitstream is decompressed until reaching the DCT coefficients. These latter are used for the interpolation stage. The obtained results are better than those obtained by the conventional JPEG in terms of CPSNR, ∆E2000 and SSIM. The obtained JPEG-like scheme is also less complex.

INTRODUCTION

Digital photography is a part of the daily life of millions of users because of the large availability of devices at affordable prices but also because it represents a new way of expression through social networks, blogs, emails and so on. It has allowed users to avoid thinking before shooting a scene since it is repeatable and does not cost anything in a digital format. Due to its popularity, the associated term digital will progressively disappear because that is the way almost all photography will be done. Even though the technology is mature enough and the market adoption is very high, it is rare to find the tri-sensor cameras. This is due to the needed storage space and the bandwidth in case of a transmission. Single-sensor imaging is the most optimal solution for this problem and is widely used for digital cameras, smartphones, webcams, etc. The used sensors can be the well-known charge-coupled device (CCD) [START_REF] Dillon | Color imaging system using a single CCD area array[END_REF] or the more recent complementary metal oxide semiconductor (CMOS) 2 sensor. The tri-component aspect is solved by using a color filter array (CFA) [START_REF] Lukac | Color filter arrays: Design and performance analysis[END_REF][START_REF] Li | Color filter arrays: representation and analysis[END_REF][START_REF] Lu | Designing color filter arrays for the joint capture of visible and near-infrared images[END_REF] on top of the sensor like the most common Bayer filter 6 represented by figure 1. Obviously, each type of sensor cell has its own spectrally selective filter and the resulting image from the CFA acquisition is a gray-scale mosaic-like. The obtained image requires thus a specific process, to recover the tricomponent or full-color aspect, called demosaicking. Figure 2 In a digital still camera, the step following the demosaicking process is the compression which is very important to allow to store several images on the same memory. This way of dealing with camera sensors images is call a demosaicking-first solution as represented by figure 4-a.

It can be easily seen that using such a process, the amount of information at the input of the encoder is tripled and then compressed using the commonly JPEG standard. [START_REF] Pennebaker | JPEG Still Image Data Compression Standard[END_REF] This raises the following question: Why increase the amount of data to reduce it thereafter? For this reason, several interested parties worked to achieve the goal of ultimately increase performance while decreasing complexity. Great attention has been paid to this demand especially by the AIC (Advanced Image Coding) group which has called for technologies and data sets [START_REF] Aic | Call for technologies of advanced image coding for camera sensor data[END_REF] for the compress-first scheme (figure 4 The main idea is to reverse the order of the two processes of compression and demosaicking. The sensor data is first compressed rather than interpolated. Thus, several methods have been proposed and studies have shown that this approach gives better results in terms of quality and compression ratio while reducing the resources required for processing; very important for handheld devices. However, most of these methods deal only with the compression stage, ignoring completely the demosaicking stage in spite of its importance in the whole process. Efficient compression algorithms are accordingly used whereas totally independent demosaicking algorithms are chosen for their efficiency. Unfortunately, efficiency is not the only criterion to take into account. Complexity, speed and even energy consumption in a mobile application have to be considered.

In this work, we propose a JPEG-like coder based on a compression-first scheme. It aims at adapting the compression scheme to the demosaicking process, where the data is directly used in the compressed domain to reconstruct the full resolution color image, contrary to the most of the existing methods which perform demosaicking in the decoded-RGB space.

The remainder of this paper is organized as follows: section 2 makes a brief state-of-the-art of the field of joint compression-demosaicking. In section 3, we focus on the proposed work followed a complete set of experiments in section 4. This paper end by some conclusions and future works.

STATE OF THE ART

The interpolation-first scheme is implemented in the majority of digital still cameras (DSC), where demosaicking is performed first and then the resulting full-color image is compressed using a JPEG standard compression. It has been reported that the demosaicking process decreases the efficiency of compression as it introduces redundancy that will need to be reduced by the used coding technology. It was even found that compressing the image before applying the CFA demosaicking yields better results in both bitrate and output image quality 9 . 10 As mentioned previously, this scheme is called the compression-first scheme. In 1991, Tsai 9 pointed out this problem from a theory point of view and introduced a signal processing method incorporating a DCT-based compression to avoid the drawbacks of the conventional scheme.

Compared to the traditional scheme, the compression-first has several advantages. Visual quality is better since it is possible to transfer the (stored) CFA image to a PC and apply complex demosaicking methods and even some post-processings that are impossible to implement on a hardware architecture. Besides, the required time between two shoots is reduced which is very helpful for professionals while the autonomy benefits from this. This is seen also as a "green" development because it helps in increase the lifetime of batteries and reduce the consumption. Moreover, compression starts with the third of the data processed in a conventional scheme thus efficiency is also increased.

The research in this area is very active. Several methods have been proposed to date for lossless, Near-lossless (visually lossless) and lossy compression. Note that for visually-lossless compression, one must define perception thresholds or run psychophysical testing in a normalized environment. [START_REF]Methodology for the subjective assessment of the quality of television pictures[END_REF] Xie et al. [START_REF] Xiang | Low complexity and high efficient method for image compression with bayer cfa[END_REF] proposed an approach where low-pass filters are used followed by a down-sampling of the green component to process the quincunx green array. Lee et al. [START_REF] Lee | A novel approach of image compression in digital cameras with a bayer color filter array[END_REF] and Cuce et al. [START_REF] Cuce | Compression of images in cfa format[END_REF] convert the RGB component to YCC color representation and used a reversible image rotation of the Y component. This step was followed by a JPEG lossy compression. The method proposed in 15 is based on estimating color differences for the captured red and blue samples and also uses a JPEG lossy compression for the encoding. Other methods are based on a vector quantization approach such as. [START_REF] Battiato | A fast vector quantization engine for cfa data compression[END_REF][START_REF] Andriani | Lossless compression of bayer mask images using an optimal vector prediction technique[END_REF][START_REF] Battiato | Coding techniques for cfa data images[END_REF][START_REF] Bruna | Predictive differential modulation for cfa compression[END_REF] Through the state of the art, we can notice that the existent methods of CFA image compression deal only with the compression process, ignoring the demosaicking process. They all aim at delivering the compressed CFA image in the RGB space after decoding and in no way try to improve the whole chain i.e. compression + demosaicking. This can be considered as a suboptimal solution for the problem of CFA images compression since the two processes are strongly linked.

PROPOSED APPROACH

Our JPEG-like method adapts the two principal processes i.e. Compression and demosaicking in a digital image acquisition pipeline. The decoded CFA image is given in the DCT domain since we stop decoding before the IDCT. The full-color image is reconstructed using the method presented in, [START_REF] Mukherjee | Demosaicing of images obtained from single-chip imaging sensors in yuv color space[END_REF] which perform demosaicking directly in the DCT domain of the YCbCr space. We choose this method because it uses the DCT and the YCbCr color space, exactly as the JPEG compression standard. This structure helped in adapting the demosaicking and compression in the compressed domain. The use of other more powerful demosaicking method and/or compression method is not in accordance with the goal of the present work where joint approach is developed.

Our method has been developed specifically for the red Bayer pattern. In order to extend to other CFA types some minor modifications are necessary. Figure 4 gives the flowchart of the developed JPEG-like coder. The proposed method consists of several steps described in the next sections.

Encoding

This stage is the one embedded on the handheld device and in our case we used an encoding scheme similar to the JPEG standard in order to be able to make actual comparisons. The CFA image with combined Red, Green and Blue pixels is separated in three distinct components. At this stage, the size of the green component is the double of the red and the blue because of the Bayer CFA. This step is followed by a color conversion from RGB to YCbCr giving thus four components Y ul , Y lr , Cb and Cr as shown by figure 5. To do so, we used the formula of equation 1 which is fully reversible and, lossless. All the coefficients are the same as the standard RGB to YCbCr conversion except a 31 , a 32 , a 41 and a 42 are half. the color components are divided in 8 × 8 blocks. Each block is transformed into DCT and the coefficients of the two Y components (Y ul , Y lr ) are merged to form only one component.

For this work, the merging consists on a average of both luminances. This is of course not optimal but further work on this part will consist in finding a combination rule by a learning process allowing to maximize the PSNR. After the combination step, each block is quantized using the typical quantization matrix as specified in the original JPEG Standard. The coefficient are rearranged in a zig-zag pattern and are then encoded with RLE (Run-Length encoding). Finally, the resulting data is encoded using a Huffman coding.

    Y ul Y lr Cb Cr     =     a 11 0
        G ul G lr B R     +     0 0 128 128     (1)

Decoding

For decoding the CFA compressed image, the reverse operations are used. Initially, a Huffman decoding and a RLE decoding are preformed. From the resulting data, every 64 coefficients are rearranged in a zig-zag pattern to reconstruct the Y, Cb and Cr components in the DCT domain. For that, the resolution of the original CFA image must be stored during compression. Finally, Each element from each resulting 8 × 8 block is multiplied by the corresponding value in the quantization table as specified in the original JPEG standard. Decoding ends at this stage, without computing the IDCT or YCbCr to RGB conversion as in the JPEG Standard. So, instead of decoding, we proceed to the demosaicking step in the compressed domain as described in the next section.

Demosaicking

This step describes how the DCT coefficients of the image are used to reconstruct the color image in the RGB color space. So, at this stage, the image in the DCT domain is ready to be demosaicked using the method presented in. [START_REF] Mukherjee | Demosaicing of images obtained from single-chip imaging sensors in yuv color space[END_REF] First, the 8 × 8 blocks are padded with zeros to form 16 × 16 blocks in order to operate the interpolation. Then, IDCT coefficients are computed and the resulting blocks are rearranged to form the YCbCr image.

In other to reduce false colors present in the final image, a median filtering is performed on the chromatic components only because filtering the Y component will create blur in the image. Finally, the standard YCbCr to RGB color space transform is used to obtain the full resolution image (represented in the RGB domain with 3 components per pixel). The demosaicking and thus the whole process is completed.

Our method allows eliminating four processing steps: Inverse Discrete Cosine Transform and YCbCr to RGB color space transformation during decoding; Discrete Cosine Transform and RGB to YCbCr color space transformation during demosaicking. Hence, complexity was reduced and speed was enhanced which is a significant improvement since these operations are quite expensive (requires real by real multiplications and divisions).

EXPERIMENTS

This section is dedicated to the experimentation of the proposed approach. The goal is to demonstrate the efficiency of the proposed solution for a JPEG-like scheme which is the one adopted by the digital photography market.

Image database

The proposed method has been applied on an image dataset composed of 25 images coming mainly from the Kodak PhotoCD set. Thumbnails of the 24 kodak image are shown in figure 6. The 25th image is the well-known "bike" image. The complete set of image are used to generate CFA images by subsampling the RGB component with respect to the bayer pattern (figure 1).

Evaluation metrics

To perform analytical assessment of the defined framework, we need one or several quality measure to tackle the artifacts. In this paper, we selected the commonly is used CPSNR, the ∆E2000 which focus on the perceptual color difference and the MSSIM that give an information about the difference of the structural content. These tools are briefly described below.

CPSNR

In the demosaicking and the compression literature, it is very common to use the composite peak-signal-to-noise ratio (CPSNR) to compare the original Bayer-pattern images and the reconstructed ones to full color RGB images or original RGB images to compressed/decompressed images. So, we use equation 2 and 3 to calculate the CPSNR of a result image. Here, I(i, j, k) is the pixel intensity at location (i, j) of the k-th color component of the original image and I'(i, j, k) for the reconstructed image. M and N are the height and the width of the image.

CP SN R = 10log 10 255 2 CM SE , (2)
where

CM SE = 1 3 × M N 3 k=1 M i=1 N j=1 [I(i, jk) -I ′ (i, j, k)] 2 .
(3)

∆E2000

∆E2000 is one of the most known color difference formula. It has been proposed by the CIE in 2000 and is based on an equation for color difference computation in the CIE Lab color space. issued from the work of Luo et al. [START_REF] Luo | Derivation of a rotation fuction for the new cie colour difference formula[END_REF] The reader can refer to the literature for long list of equations.

SSIM

The Structural Similarity Index (SSIM) [START_REF] Wang | Image quality assessment: From error visibility to structural similarity[END_REF] tries to model the physical properties of the HVS. The SSIM follows a top-down modeling paradigm that first decomposes images into several scales and then measures contrast and structure in each scale. In addition, the luminance of the lowest scale is also measured. Finally, all the data is pooled into a single score. SSIM has the advantage that it is computationally tractable while still providing reasonable correlations to subjective measurements. Its formula is given by equation 4 .

SSIM (x, y) = (2µ x µ y + C 1 )(2σ xy + C 2 ) (µ 2 x + µ 2 y + C 1 )(σ 2 x + σ 2 y + C 2 ) . ( 4
)

Preliminary experiments

Before developing and evaluating the proposed, we started by performing a set of tests using a compression-first scheme based on the well-known JPEG compression standard family i.e. JPEG, [START_REF] Pennebaker | JPEG Still Image Data Compression Standard[END_REF] JPEG 2000 [START_REF]Information technology -the jpeg2000 image coding system: Part 1[END_REF] and JPEG XR. [START_REF] Iso | Reference software implementation of jpeg xr iso/iec jtc1[END_REF] Both, lossless and lossy (at different bitrates) compression were used on a set of 25 images.

The testing procedure used consists of the following steps, starting with a full color RGB image: 1) Subsample the RGB image with the Bayer pattern to form the CFA image. 2) Compress the three color channels. 3) Save the compression ratio. 4) Decompress and perform demosaicking. 5) Evaluate the distortion with reference to the full color RGB image by giving the defined metrics. We used the demosaicking method presented in 24 because it gives better results compared to other methods. [START_REF] Mukherjee | Demosaicing of images obtained from single-chip imaging sensors in yuv color space[END_REF][START_REF] Driesen | Wavelet-based color filter array demosaicing[END_REF][START_REF] Glotzbach | A method of color filter array interpolation with alias cancellation properties[END_REF][START_REF] Hirakawa | Adaptive homogeneity-directed demosaicing algorithm[END_REF] Here, the aim was not to do a comparative study of the demosaicing methods, we only used the best available method. For JPEG 2000, evaluation of distortion is given although lossless compression was used. These distortions are due to the sub-sampling performed to obtain a CFA image and also to demosaicking. The values shown in table 1 are respectively: bitrate, CPSNR, SSIM and DeltaE2000 (top-to-bottom).

The results of table 1 have shown that even a low lossy compression (visually lossless) gives an acceptable quality for the reconstructed image (especially using JPEG 2000). Besides, subjective tests show that such processing does not produce highly visible artifacts (small artifacts can be seen in some images). As a consequence, we adopted the lossy compression for our method.

Results and discussion of the proposed approach

This section deals with the evaluation of the proposed approach. For this we use the image database and the metrics described previously. Since it is a JPEG-like coder, we restricted our comparison to the JPEG standard to be able to have comparable results. Of course the JPEG results are obtained after a demosaicking stage with the approach defined in. [START_REF] Menon | Demosaicing based on wavelet analysis of the luminance component[END_REF] Table 2 gathers all the results for the proposed method. The values shown of the table are respectively: bitrate, CPSNR, SSIM and ∆E2000 (top-to-bottom).

From table 2, we can notice that the proposed approach gives good results in comparison to the standard JPEG. The metrics scores show in average an improvement of 1.5 dB with the CPSNR, 3.5% in SSIM and approx. 1 in ∆E2000. Let remind that for ∆E2000, lower is the difference better are the results. In addition a the achieved low bitrate (around 1.5 bpp). However some artifacts can be observed on images of figure 7. They are mainly due to the demosaicking stage which need to be improved. Besides, our approach is less complex than any other JPEG-based method because of the adaptation between the compression and the demosaicking processes. At the same bitrate, our JPEG-like method outperforms the joint standard JPEG and demosaicking as described in, [START_REF] Menon | Demosaicing based on wavelet analysis of the luminance component[END_REF] although our JPEG implementation is not too accurate and the used demosaicking method is more effective than the one we used. The benefit of this adaptation can be easily estimated.

CONCLUSION

In this paper, a JPEG-like coder for color filter array images has been proposed following the compression-first scheme. It is a joint compression/demosaicking approach where the interpolation is performed in the compressed domain. The lossy compression is not the only reason for the artifacts present in the resulting image; the used demosaicking method is also a major factor. We have obtained good results with a relatively low bitrate in comparison with JPEG baseline. Moreover, the outstanding particularity of our method is the elimination of four principal processing steps of the standard scheme, which yield to a gain in time and complexity. As future works, we should improve the visual quality of the reconstructed images by using a more accurate JPEG implementation. A similar strategy could be applied for JPEG 2000 compression algorithm by adapting it with a wavelet-based demosaicking method.

Figure 1 .

 1 Figure 1. Bayer color filter array.

  gives an example of a Bayer CFA image (a) and the demosaicked image (b).

Figure 2 .

 2 Figure 2. Bayer CFA-based single-sensor imaging: (a) full-color image and (b) grayscale CFA image.

Figure 3 .

 3 Figure 3. Single-sensor image compression: (a) a demosaicking-first solution and (b) a compression-first solution.

Figure 4 .

 4 Figure 4. Flowchart of the developed JPEG-like coder.

Figure 5 .

 5 Figure 5. Color conversion from RGB to YCbCr.

Figure 6 .

 6 Figure 6. Used image Database

Figure 7 .

 7 Figure 7. Some compression results: (a) and (c) conventional JPEG compression, (b) and (d) proposed JPEG-like coder.

  a 13 a 14 0 a 22 a 23 a 24 a 31 a 32 a 33 a 34 a 41 a 42 a 43 a 44

Table 1 .

 1 Preliminary results with different standards. Values represent respectively bitrate, CPSNR, SSIM and ∆E2000.

	Image	JPEG 2000 JPEG XR	JPEG	Image	JPEG 2000 JPEG XR	JPEG
		(Lossless)	(Q=20)	(Q=89)		(Lossless)	(Q=20)	(Q=89)
	1	6.163	3.462	3.387	14	5.907	3.187	3.096
	768x512	38.770	36.825	34.294	768x512	36.862	35.464	33.639
		99.42	99.01	98.63		99.32	98.82	98.24
		1.23	1.47	1.63		1.29	1.62	1.84
	2	4.802	2.209	2.007	15	4.536	2.247	2.008
	768x512	39.961	37.614	35.803	768x512	39.173	37.584	35.865
		98.79	97.65	96.98		98.99	98.33	97.72
		0.91	1.28	1.45		1.03	1.39	1.59
	3	4.231	1.783	1.739	16	4.826	2.160	2.143
	768x512	42.697	39.605	37.972	768x512	43.947	39.712	37.165
		99.57	98.77	98.52		99.57	98.77	98.33
		0.62	0.90	0.95		0.78	1.09	1.19
	4	4.965	2.444	2.217	17	4.999	2.402	2.335
	480x720	39.345	37.395	35.547	480x720	40.938	38.271	36.326
		99.29	98.89	98.76		99.73	99.41	99.39
		0.94	1.23	1.40		1.29	1.69	1.79
	5	6.404	3.747	3.672	18	5.947	3.338	3.073
	768x512	37.988	36.341	34.009	480x720	36.181	35.053	33.361
		99.582	99.28	98.94		99.26	98.98	98.85
		1.59	1.98	2.29		1.86	2.16	2.44
	6	5.329	2.836	2.671	19	5.215	2.792	2.469
	768x512	40.199	37.995	35.447	480x720	40.102	37.696	35.808
		99.48	98.99	98.37		99.45	99.15	99.04
		0.82	1.02	1.18		1.04	1.26	1.37
	7	4.610	2.110	2.157	20	3.595	1.822	1.728
	768x512	42.038	39.250	37.338	768x512	40.699	38.561	37.075
		99.67	99.15	98.99		99.17	98.71	98.39
		0.78	1.07	1.14		0.82	1.05	1.11
	8	6.395	3.89	3.711	21	5.167	2.673	2.422
	768x512	36.543	35.347	33.366	768x512	39.324	37.466	35.587
		99.463	99.22	98.77		99.33	98.81	98.35
		1.49	1.70	1.91		1.09	1.29	1.41
	9	4.602	2.132	1.945	22	5.338	2.711	2.506
	480x720	42.089	38.889	37.306	768x512	38.232	36.512	34.763
		99.63	99.26	99.24		98.94	98.22	97.66
		0.78	0.99	1.02		1.14	1.36	1.48
	10	4.821	2.342	2.141	23	4.136	1.681	1.603
	480x720	41.583	38.644	36.793	768x512	43.020	39.732	38.581
		99.67	99.31	99.20		99.46	98.74	98.55
		0.82	1.05	1.11		0.64	0.87	0.90
	11	5.257	2.762	2.535	24	5.670	3.202	2.970
	768x512	40.080	37.812	35.488	768x512	34.545	33.725	32.557
		99.45	98.89	98.19		99.31	98.86	98.35
		1.09	1.42	1.59		1.27	1.53	1.69
	12	4.545	2.090	1.896	Bike	5.755	3.429	3.240
	768x512	43.394	39.697	37.616	640x480	33.284	32.725	31.806
		99.53	98.65	98.06		99.36	99.23	99.15
		0.53	0.73	0.77		1.51	1.66	1.74
	13	6.704	4.057	3.866	Average	5.197	2.700	2.541
	768x512	35.51	34.195	32.429		39.442	37.284	35.438
		99.02	98.75	98.22		99.38	98.87	98.51
		1.74	1.91	2.15		1.08	1.35	1.49

Table 2 .

 2 Comparison between our method and JPEG

	Image	JPEG+[14]	Ours	Image	JPEG+[14]	Ours
	1	1.91	2.08	14	1.83	1.71
	768x512	23.668	24.802	768x512	25.299	27.147
		89.22	96.66		87.15	94.93
		7.05	6.12		6.54	5.27
	2	1.08	1.31	15	1.16	1.21
	768x512	28.545	29.818	768x512	27.953	29.159
		85.35	90.92		90.08	92.21
		4.49	3.82		4.54	3.97
	3	1.08	1.17	16	1.16	1.26
	768x512	30.217	31.424	768x512	28.882	29.700
		91.08	91.39		90.17	94.60
		3.13	2.83		4.35	3.98
	4	1.23	1.43	17	1.42	1.46
	480x720	28.797	31.180	480x720	28.168	30.084
		92.37	96.25		96.13	97.74
		4.05	3.02		4.49	3.64
	5	2.33	1.95	18	1.80	1.75
	768x512	23.329	25.283	480x720	24.872	26.177
		89.39	96.55		92.69	96.07
		9.18	7.08		7.40	5.96
	6	1.50	1.51	19	1.42	1.45
	768x512	25.254	26.307	480x720	25.425	26.293
		89.23	95.33		93.54	95.22
		5.40	4.72		4.92	4.34
	7	1.41	1.37	20	1.00	1.23
	768x512	27.857	31.005	768x512	27.462	28.962
		93.21	97.32		92.83	90.55
		4.47	3.00		3.38	2.99
	8	2.25	1.90	21	1.41	1.40
	768x512	21.567	22.377	768x512	25.873	27.212
		92.03	96.05		92.11	95.30
		8.12	7.418		5.07	4.31
	9	1.13	1.23	22	1.41	1.46
	480x720	28.3869	30.345	768x512	27.3098	28.769
		5.88	97.03		7.03	93.97
		3.32	2.71		4.53	3.74
	10	1.23	1.43	23	1.00	1.08
	480x720	28.2229	29.577	768x512	30.4419	32.597
		5.39	94.79		2.80	95.81
		3.58	3.31		2.95	2.320
	11	1.50	1.57	24	1.75	1.68
	768x512	26.538	27.829	768x512	24.330	25.320
		89.22	95.32		89.76	94.83
		5.54	4.68		5.63	4.80
	12	1.08	1.17	Bike	2.11	1.76
	768x512	28.656	30.651	640x480	21.209	21.939
		90.47	93.65		94.06	96.16
		2.82	2.33		6.52	5.74
	13	2.25	2.04	Average	1.50	1.51
	768x512	21.993	22.917		26.410	27.984
		86.91	95.84		91.12	94.98
		8.65	7.55		5.20	4.39

ACKNOWLEDGMENTS

This work has been partially supported by the project CAIMAN funded by the French Research Agency.


Hrarti M 12

Saadane H 13

M-C Larabi

Improving the quality of H.264/AVC by using a new Rate-Quantization model

Keywords: H.264/AVC Standard, Complexity measure, Rate-Quantization model, Inter/ Intra Frame, Rate control algorithm

Rate control plays a key role in video coding standards. Its goal is to achieve a good quality at a given target bit-rate. In H.264/AVC, rate control algorithm for both Intra and Inter-frames suffers from some defects. In the Intra-frame rate control, the initial quantization parameter (QP) is mainly adjusted according to a global target bit-rate and length of GOP. This determination is inappropriate and generates errors in the whole of video sequence. For Inter coding unit (Frame or Macroblock), the use of MAD (Mean Average Differences) as a complexity measure, remains inefficient, resulting in improper QP values because the MAD handles locally images characteristics. QP miscalculations may also result from the linear prediction model which assumes similar complexity from coding unit to another. To overcome these defects, we propose in this paper, a new Rate-Quantization (R-Q) model resulting from extensive experiments. This latter is divided into two models. The first one is an Intra R-Q model used to determine an optimal initial quantization parameter for Intraframes. The second one is an Inter R-Q model that aims at determining the QP of Inter coding unit according to the statistics of the previous coded ones. It does not use any complexity measure and substitutes both linear and quadratic models used in H.264/AVC rate controller. Objective and subjective simulations have been carried out using JM15.0 reference software. Compared to this latter, the global R-Q model (Intra and Inter models combined) improves the coding efficiency in terms of PSNR, objectively (up to +2.01dB), subjectively (by psychophysical experiments) and in terms of computational complexity.

INTRODUCTION

H.264/AVC (Advanced Video Coding) is the newest international video coding standard that provides approximately a 50% bit-rate savings for equivalent perceptual quality relative to the performance of prior standards. This is because it has many outstanding features, such as various Intra/Inter prediction modes, 4x4 integer transform, rate-distortion optimization, and improved entropy coding [START_REF] Wiegand | Overview of the H.264/AVC video coding standard[END_REF] . These new techniques do not consider the issue of maintaining a consistant quality and a Constant Bit-Rate (CBR) through the network channel. Hence, it is necessary to adjust dynamically the quantization parameter (QP) by implementing a rate control algorithm in the video encoder. Figure 1 illustrates the two main steps in H.264/AVC rate control process: Bit allocation scheme (at frame or basic unit layers) that uses MPEG2 TM5 4 as a benchmark, and the bit allocation achievement, i.e., quantization parameter determination using quadratic Rate-Quantization (R-Q) model.

It should be noted that in Figure 1, we do not consider B-frame due to the fact that this latter takes up only a little part in total encoded bit-stream, which can be ignored in low bit-rate. Therefore, we only consider I-and Further author information: (Send correspondence to Hrarti M.) Hrarti M.: E-mail: hrarti@sic.sp2mi.univ-poitiers.fr !"#$%&'(#)*+,&#-.&/,0+%"&1%2342"#-5%&6!!!7&%.*+%.&8,&/(0#-&19&:#2-#-.7&:2#-0&;#,<%"#7&12459&43&/1!=>!/?@&=)%5+24-*5

!"#$*-$7&/1!=&64)9&ABCA7&ABCADE&F&G&HDII&/1!=>!/?@&F&JJJ&54.%K&DHAA>ABCLMIIMNIB&F&.4*K&ID9IIIAMIH9BAHOHC /1!=>!/?@M&64)9&ABCA&&ABCADE>I are simpler than that of H.264/AVC, because this latter applies the Lagrangian method (or RDO: Rate-Distortion Optimisation) which requires the QP to be known before motion estimation and mode decision, thus leading to a chicken and egg dilemma because until the end of motion estimation and mode decision, the rate control algorithm cannot access the statistics such as MAD (Mean Average Difference), which are essential to calculate QP. It can be also seen from Figure 1, that the rate control process is different for both Intra-frame (I-frame) and Inter coding unit.

In the Intra-rate control, an initial QP is determined for the I-frame and the first P-frame in a given GOP. This selection is given in the following flowchart: bpp is the number of bits per pixel, GlobalBitrate is the global target bit-rate for the whole sequence and Fr is the frame rate. L 1 , L 2 , and L 3 are defined in [START_REF] Lim | Text description of joint model reference encoding methods and decoding concealment methods[END_REF] . As mentioned above, this initial QP is adjusted mainly according to global target bit-rate (GlobalBitrate) and length of GOP. This approximative choice generates errors that are propagated in the whole of video sequence by mean of motion estimation and motion compensation frameworks used in H.264/AVC. To overcome this problem, several Intra-rate control studies have been conducted 1 8 12 . The proposed solutions however, present two difficulties, to be applied directly in H.264/AVC. Firstly, they need information after actually encoding the current frame to determine the appropriate QP which does not comply with H.264/AVC RDO procedure. Secondly, they need a lot of additional computations for a complexity measure including I-frame feature 11 18 .

In the H.264/AVC Inter-rate control algorithm, the chicken and egg dilemma is always present. To get around this problem, Li et al [START_REF] Li | Adaptive Basic Unit Layer Rate Control for JVT[END_REF] propose a linear model. This latter assumes that the complexity calculated by MAD (Mean Absolute Difference), varies gradually between the current and the previous coding unit (Frame or Macroblock).

M AD cur = α 1 * M AD prev + α 2 (2)
where M AD cur and M AD prev represent respectively the current and the previous MAD of a given coding unit. α 1 and α 2 are the linear model coefficients. Once the predicted complexity M AD cur and a target bit-rate for a given P-frame (or Macroblock) (TBitsP ) are determined, they are then included into a quadratic model used to compute the quantization parameter step Q step :

T BitsP = M AD cur × x 1 Q step + x 2 Q 2 step ( 3
)
where x 1 and x 2 are the first and second order coefficients. The Inter-rate control in H.264/AVC has also been the subject of several investigations. Efforts have been focused on the improvement of the MAD and its prediction through a linear model. The use of MAD, as complexity measure, handles image characteristics locally but in some cases remains inefficient, resulting in improper quantization parameter (QP) values. QP miscalculations may also result from the linear prediction model which assumes similar complexity from frame to frame. Like for the Intra-rate control, the proposed solutions are either applicable in a given context 3 15 16 or require more additional computations [START_REF] Kamaci | Frame bit allocation for the H.264/AVC video coder via Cauchy-Density-Based rate and distortion models[END_REF] . In this paper, we propose a new R-Q model resulting from extensive experiments. This latter is divided into two models: An Intra R-Q model and an Inter one. The Intra R-Q model solves the initial QP problem by determining an optimal initial QP for Intra-frame while the Inter one overcomes problems related to MAD estimation and linear model. This latter substitutes both linear and quadratic models (Eq. 2 and Eq. 3) used in H.264/AVC rate control algorithm by a new logarithmic R-Q model. The global R-Q model (Inter and Intra models combined) does not need any complexity measure estimation. Objective and subjective simulations have been carried out using JM15.0 reference software [START_REF][END_REF] . Compared to this latter, the global R-Q model improves the coding efficiency in terms of quality measured objectively (PSNR up to +2.01dB), subjectively (by psycho visual experiments) and in terms of computational complexity. The rest of this paper is organized as follows. Section 2 describes in detail the proposed R-Q model. The experimental results are given in Section 3. Finally, we draw a conclusion in Section 4.

PROPOSED R-Q MODEL

As it was mentioned previously, our proposed R-Q model is divided into two models Intra and Inter R-Q models:

/1!=>!/?@M&64)9&ABCA&&ABCADE>P

Intra R-Q model

The improvement of the Intra-frame quality plays a key role in the enhancement of coding efficiency for all subsequent Inter-frames, because these latter are motion compensated directly or indirectly from the Intraframe. In H.264/AVC Intra-rate control, it is well known that the initial QP determination using the bits per pixel number (bpp) (Eq. 1) presents, at least, two defects: the range between QPs is large and the QPs values do not consider any frame feature (As the Flowchart in Figure 2 substantiates). To overcome these defects, a new R-Q model for Intra-frame is proposed. The first step of this algorithm, consists in determining a relationship between the target bit-rate of I-frame (TBitsI) and the fixed global target bit for the whole sequence (GlobalBitrate). To establish this relationship, three test sequences have been encoded (Bridge, Carphone and Foreman) at several low bit-rates ranging from 10Kbps to 80Kbps. The actual bit-rates and two complexity measures (average gradient per pixel and Euclidean distance) of I-frame are computed. Then, the ratio between actual bit-rate and complexity measure of I-frame is plotted vs. global target bit. Whatever the complexity measure, the model which best fit the experimental data is the non linear model given by:

T BitsI δ = a * GlobalBitrate Fr b (4)
δ represents an I-frame complexity, Fr is the frame rate which equals to 30fps. The model parameters a and b are determined by the least squares method. Eight other sequences have served to confirm the accuracy of the above relationship (Akiyo, Claire, Grandma, Miss-America, Mother-Daughter, News, Salesman and Silent).

Once the ratio RI/δ is determined by the first model, we use a second model R-Q allowing the determination of the quantization parameter (QP) of the Intra-frame without having to calculate its complexity δ. To generate a relationship between the ratio RI/δ and QP, we have used the same test conditions described above with an initial QP varying from 10 to 40 with a constant step of 2. Fitting the experimental data RI/δ vs. QP leads to the non linear model of Eq. 5. This model allows an optimal determination of Intra-frame initial QP without having to calculate its complexity unlike methods described in 1 12 14 :

T BitsI δ = 1 (p 1 * QP 2 + p 2 * QP + p 3 ) (5)
where p 1 , p 2 and p 3 are the Intra R-Q model coefficients.

Inter R-Q Model

The goal of a model-based QP is to develop a Rate-Quantization solution that intends to achieve more consistent video quality across the entire sequence for a large variety of sequences with different characteristics. In this section, we built a new Rate-Quantization model for Inter coding unit by analyzing the actual data generated by coding, using constant QPs across frames of the whole sequence. To achieve this goal, we performed a lot of experiments on various video sequences with various contents (Bridge, Carphone, Coastguard, Foreman, Highway and Suzie). These sequences are used to generate the new Inter R-Q model, and eight others (Akiyo, Claire, Grandma, Miss-America, Mother-Daughter, News, Salesman and Silent) are used to confirm its accuracy.

Each frame of the above test sequences is coded with constant QP ranging from 10 to 40 with a constant step of 2. The first frame is encoded as I and subsequent frames as P. In Figure 3, we plot the curves QPs versus bits per P-frames for three test sequences "Bridge", "Carphone" and "Foreman".

It can be seen from Figure 3 that there is a logarithmic relationship between QP and a target bit-rate (TBitsP) used to encode each P-frames in the sequence. This can be expressed as follows:

QP = α * log (T argetBitsP )+β (6)
where α and β are the model parameters which are defined by using a linear regression method that uses the coding statistics of the previously coded frames. As it can be shown, the proposed Inter R-Q model does not use any complexity measure unlike the H.264/AVC rate control algorithm which uses a linear model (Eq. 2) to predict the complexity measure and a quadratic model (Eq. 3) to compute the QP function of this complexity. /1!=>!/?@M&64)9&ABCA&&ABCADE>Q

EXPERIMENTAL RESULTS

The quality assessment has been performed at the macroblock level and the JVT reference software JM10.2 baseline profile [START_REF][END_REF] has served for implementation. The proposed model R-Q has been implemented using the same software, so that all parts of the encoder except for rate control are the same. The computing system used for the experiments is composed by 2.16 GHZ Pentium 4 CPU and 2.00 GB RAM running Windows Vista. Fifteen common test sequences have been used (Akiyo, Bridge, Claire, Carphone, Container, Grandma, Highway, News. . . ) in QCIF format (4:2:0) and 30 frames per second. Several global target bit-rates (RT) have been considered ranging from 15 to 64Kbps. We have encoded the first frame as I and the subsequent frames as P (1 GOP of 90 frames). The JM15.0 7 reference software is used for comparison. This latter is one of the most recent versions of H.264/AVC encoder. It is used with the rate control mode set to "2" (rate control is applied with an intelligent QP selection for I and B slices). We evaluate the experimental results by both subjective and objective assessment.

Subjective quality evaluation

For the subjective quality evaluation, we used the Double-Stimulus Continuous Quality-Scale (DSCQS) methodology described in ITU recommendations [START_REF] Itu-R | Methodology for the Subjective Assessment of the Quality of Television Pictures[END_REF] . The experiments were run in a dedicated room with neutral gray walls, a controlled D65 lighting, a calibrated display. . . 15 observers with normal acuity and color vision have made the test. The scoring is performed on discrete scale composed of 5 quality levels: excellent, good, fair, poor and bad that correspond respectively to scores ranging from 100 to 0. For this experiment, we preferred the usage of these values rather than 5 to 1 to simplify their interpretation. Since the proposed Intra R-Q model determines an optimal initial QP for both Intra and the first P-frames, we subjectively evaluate their quality. Table 1 lists the evaluation results obtained by using DSCQS method when the Intra R-Q model is applied in comparaison with JM15.0. From Table 1, we can notice that the proposed R-Q model is visually efficient because it providers higher DSCQS scores in comparison to the JM15.0 and it implies at least a score gain of 12.8. It can be said that larger the score is, better is the obtained perceptual quality.

Objective quality assessment

In order to validate the effectiveness of the proposed R-Q model, we test this latter on image sequences, different from those used to generate it. We use average PSNR difference and average bit-rate difference as performance measures. [START_REF] Bjontegaard | Calculation of Average PSNR Differences between RD-Curves[END_REF] These measures are often used to compare R-D performance between two different methods. We also compare the initial QP values and the computational time with the JM15.0 reference software. The results are shown in Table 2.

Three remarks can be made from Table 2. Firstly, the initial QP values determined by the proposed R-Q model are smaller than those calculated by JM15.0 (In this case, only one value of QP is selected whatever the considered target bit-rates). Secondly, the proposed approach outperforms JM15.0 by improving the coding efficiency in terms of PSNR (up to +2.01dB). Finally, the computational time is reduced (up to +18s) because the proposed R-Q model incorporates an Inter R-Q one that substitutes both the linear and quadratic models used in JM15.0 and alleviates the memory occupation. Figure 4 and Figure 5 give the Rate-Distortion and computational time versus target bit-rates for "Akiyo" and "Container" sequences. One can see that the proposed R-Q model achieves a higher PSNR and lower computational time in comparison with the JM15.0 rate control, for all selected bit-rates ranging from 15Kbps to 64 Kbps. The improvement of the R-D performance when our proposed approach is applied, enhances obviously the visual quality.

Finally, Figure 6 and Figure 7 depict the variation of the PSNR and the number of bits per P-frame for "Grandma" and "Salesman" sequences coded respectively at 20Kbps and 30Kbps. Figure 6 shows that for the proposed R-Q model, the PSNR variation in latter frames is lower than in earlier ones. However, the PSNR improvement of earlier frames is enough significant and leads to a more consistent quality with a lower variation of average PSNR/sequence (Average PSNR gain of +1.5dB in "Grandma" case and +1.24dB in "Salesman" /1!=>!/?@M&64)9&ABCA&&ABCADE>A case).

/1!=>!/?@M&64)9&ABCA&&ABCADE>C
Figure 7 shows the actual bits ditribution among P-Frames for the same sequences. It can be seen that the bits/frame variation is lower for the proposed approach. Moreover, the average bits/frame of our algorithm remains lower than the average target bits/frame.

CONCLUSION

In this paper, we have proposed a new Rate-Quantization model. This latter aims at overcoming some defects related to H.264/AVC rate controller for both Inter and Intra-frames. Our proposed approach resolves firstly, the issue of inappropriate calculation of the initial QP for I-frame by determining efficiently, an optimal initial QP according to both target bit-rate and complexity of the I-frame without having to determine this latter. Secondly, it overcomes the problems related to the use of MAD measure and the linear model by proposing a new logarithmic R-Q model for Inter coding unit (Frame or Macroblock). The logarithmic model substitutes both the linear and quadratic models used in H.264/AVC rate controller, and does not need any complexity measure. Objective and subjective experiments show that the proposed approach enhances significantly the R-D performance in lower bit-rate by reducing computational time and improving the sequence quality (in term of consistence and average PSNR). In order to include further improvement in the visual quality, we aim to improve /1!=>!/?@M&64)9&ABCA&&ABCADE>B /1!=>!/?@M&64)9&ABCA&&ABCADE>R

the bit allocation process at frame and macroblock layers by including some complexity measures able to allocate bits according to frame/macroblock characteristics.

Figure 1 .

 1 Figure 1. H.264/AVC Rate control schema for both I-frame and P-Basic Unit

Figure 2 .

 2 Figure 2. Flowchart of the Initial QP selection process in H.264/AVC Intra-rate control.

Figure 3 .

 3 Rate-Quantization curves of some P-frames of (a):"Bridge", (b):"Carphone" and (c):"Foreman" sequences.

Figure 4 .

 4 Figure 4. Rate-Distortion results for two sequences encoded at various bit-rates when our R-Q Model is applied and compared to the JM15.0 reference software for -a-:'Akiyo' and -b-:'Container'.

Figure 5 .

 5 Figure 5. Time results for two sequences encoded at various bit-rates when our R-Q Model is applied and compared to the JM15.0 reference software for -a-:'Akiyo' and -b-:'Container'.

Figure 6 .

 6 Figure 6. PSNR variations per P-Frames for 2 sequences when our R-Q Model is applied and compared with the JM15.0. 'Grandma' encoded at 20Kbps':(-a-) and 'Salesman' encoded at 30Kbps':(-b-).

Figure 7 .

 7 Figure 7. Bit-rate variations per P-Frames for 2 sequences when our R-Q Model is applied and compared with the JM15.0. 'Grandma' encoded at 20Kbps':(-a-) and 'Salesman' encoded at 30Kbps':(-b-).

  [START_REF]MPEG-4 Video Verification Model V18.0, Coding of Moving Pictures and Audio N3908[END_REF]

Table 1 .

 1 Subjective scores using DSCQS method for both Intra and the first P-frames when R-Q is applied in comparison with JM15.0.

	Sequences Target	Frames DSCQS DSCQS Score
		Bit		Score	Score	Gain
		Rate		JM15.0	R-Q
		(Kbps)			Model
		15	I-Frame	41	61,6	20,6
			P-Frame	42,2	66,4	24,2
		20	I-Frame	40,2	65,7	25,5
	Akiyo	30	P-Frame I-Frame	40,2 41,3	64,6 66,4	24,4 25,1
			P-Frame	35,5	63,6	28,1
		45	I-Frame	40,1	71,8	31,7
			P-Frame	41,8	70,4	28,6
		15	I-Frame	31,6	49	17,4
			P-Frame	36,2	49	12,8
		20	I-Frame	31,6	50,7	19,1
	Container	30	P-Frame I-Frame	36,7 31,2	59,4 54,8	22,7 23,6
			P-Frame	31	56,1	25,1
		45	I-Frame	27,9	54	26,1
			P-Frame	33,7	52,3	18,6
		15	I-Frame	33,9	54,2	20,3
			P-Frame	31,5	51,1	19,6
		20	I-Frame	29,1	58,3	29,2
	News	30	P-Frame I-Frame	30,7 31,6	53 55,2	22,3 23,6
			P-Frame	25,2	55,3	30,1
		45	I-Frame	32,9	57,8	24,9
			P-Frame	33,8	60,8	27
		15	I-Frame	29,8	47,9	18,1
			P-Frame	28,9	50,5	21,6
		20	I-Frame	29,8	62,7	32,9
	Salesman	30	P-Frame I-Frame	35,6 40,6	57,7 64,6	22,1 24
			P-Frame	34,7	61,2	26,5
		45	I-Frame	34,5	57,8	23,3
			P-Frame	27,7	56,2	28,5

Table 2 .

 2 Comparison of initial QP values, Average PSNR, Average Bit-Rates and Computational Time Differences between the R-Q Model and JM15.0 reference software.

	Sequences	Target	Methods	Initial QP	Average	Average	Computational
		Bit-Rate	and	Value	PSNR	Bit-Rate	Time
		(Kbps)	Gain		(dB)	(Kbps)	(s)
		15	JM15.0	35	34.14	15.14
			R-Q Model	29	35.49	15.14
			Gain	-	+1.35	0	+8.17
		20	JM15.0	35	35.12	20.17
			R-Q Model	28	36.79	20.18
	Akiyo	30	Gain JM15.0	-35	+1.67 36.56	-0.01 30.26	+13.337
			R-Q Model	27	38.40	30.22
			Gain	-	+1.84	+0.04	+10.011
		64	JM15.0	35	40.08	64.38
			R-Q Model	24	41.60	64.31
			Gain	-	+1.52	+0.07	+9.705
		15	JM15.0	35	32.10	15.13
			R-Q Model	29	33.23	15.14
			Gain	-	+1.13	-0.01	+12.028
		20	JM15.0	35	32.97	20.13
			R-Q Model	28	34.67	20.15
	Container	30	Gain JM15.0	-35	+1.7 34.62	-0.02 30.12	+9.777
			R-Q Model	27	36.09	30.15
			Gain	-	+1.47	-0.03	+6.878
		45	JM15.0	35	36.09	45.19
			R-Q Model	25	37.52	45.19
			Gain	-	+1.43	0	+7.546
		15	JM15.0	35	29.61	15.89
			R-Q Model	29	29.61	15.33
			Gain	-	0	+0.56	+12.91
		20	JM15.0	35	30.46	20.26
			R-Q Model	28	30.96	20.22
	News	25	Gain JM15.0	-35	+0.5 31.20	+0.04 25.23	+12.366
			R-Q Model	27	32.27	25.18
			Gain	-	+1.07	+0.05	+13.476
		45	JM15.0	35	33.27	45.35
			R-Q Model	25	35.28	45.37
			Gain	-	+2.01	+0.02	+12.404
		20	JM15.0	35	29.62	20.17
			R-Q Model	28	30.16	20.15
			Gain	-	+0.54	+0.02	+17.447
		25	JM15.0	35	30.43	25.15
			R-Q Model	27	31.35	25.16
	Salesman	30	Gain JM15.0	-35	+0.92 30.98	-0.01 30.19	+18.002
			R-Q Model	27	32.22	30.19
			Gain	-	+1.24	0	+17.064
		35	JM15.0	35	31.63	35.20
			R-Q Model	26	33.04	35.18
			Gain	-	+1.41	+0.02	+16.498

/1!=>!/?@M&64)9&ABCA&&ABCADE>II